A vision of Responsible Research and Innovation

This paper outlines a vision of responsible research and innovation and develops an implementation strategy for it.

I will provide a responsible research and innovation matrix consisting of lead-questions to be addressed by stakeholders in order to satisfy responsible research and innovation requirements

Introduction: Technical inventions, innovation and responsibility
In order to be able to specify the 'responsibility' in Responsible Research and Innovation (RRI), I will contrast the process of modern innovations with mere technical inventions.In order to be able to understand the responsibility concept solely with regard to technical inventions, I will give an historical example of pre-modern times, so that we "isolate" the role of responsibility and rule out any connotations of 'responsibility' with a view on a modern innovation context with which we are so familiar.
At the very beginning of the 18 th century the Portuguese priest Bartolomeu Lourenço de Gusmão claimed to have developed a "machine for sailing through the air".His invention was called Passarola (meaning "ugly bird" in Portuguesebecause of its resemblance to a bird) and the "ship" was filled with numerous tubes through which the wind was supposed to flow and would fill out bulges (see design of the prototype, fig 1 .).
Bartolomea informed the Portuguese king John V about his inventions in a letter 2 : "Father Bartolomeo Lourenço dares to inform Your Majesty that he has discovered an implement by which he can travel through air […].With it, he will manage 200 leagues a day: thus, he will be able to bring armies and far away countries news and orders […].The furthermost regions will be discovered, at the world's poles, and the Portuguese nation will benefit from such discovery […]. 2 The quotes come from the original letter Bartolomeo wrote to King John V. It is displayed at the exhibition "Lux in Arcana.The Vatican secret archives reveals itself "(Capitoline Museum, Rome, March 2012-September 2012).The Museum display gives the following further information: Gusmão presented a demonstration of his inventions, but we do not know for sure if the passarola itself was used, or simply a hot-air balloon.Neither do we know how big the prototype was: it seemed to be triggered by a strange combination of sails, wings and electromagnetism.Bartolomeo pointed not only to the potential benefit of his invention but also to its negative sideeffects: "Many crimes will be committed, as it allows to easily flee from one country to the other: its use will have to be limited." On April 17, 1707, John V decreed the exclusive right for Bartolomeo to perfect […] his machine […]", and assigned him a lifelong appointment at Coimbra University.In the same decree John V made clear that anyone trying to copy his work would receive the death penalty.This historical example shows that "responsible" use and control of technical inventions should be limited to those that are supposedly deemed able to act responsibly: in this case the king (Grinbaum and Groves, this volume).Technical inventions up to modern times are still considered with a view on "who is in control" and "who can make use of it''.Negative consequences of the technology are notably associated with who can use/misuse the technology, rather than with the properties of the technology itself.The priest was apparently not very much occupied with the risks of the use of his "ship' and the safety of the prospective travelers.The politics of non-proliferation of nuclear weapons still echoes this tradition: only a few 'responsible' governments are supposed to control the production of these weapons.All others should keep moral constraint and trust the "responsible" governors of this technology.
Whereas technological inventions where, historically, controlled by a central agent to avoid abuse, modern innovations are distributed through market mechanisms whereby property rights allow, in principle, the further improvements of the innovations by other market operators over time.Economic exploitation of innovations implies a loss of a sole control agent; yet the state demands from industrial operators that they address the so called three market hurdles of efficacy, quality and safety before they can legally market their products or processes.
Responsible marketing is thus ensured by conditions required by state regulations and (product) law specifies the legal requirements prior to marketing.In the area of pharmaceuticals, even a fourth hurdle of clinical effectiveness and cost-efficiency became operational under modern legislation to some extent, not in the least in order to be more responsible for the outcomes of innovation processes.For modern innovations, responsibility for the consequences of implementation is primarily related to the properties and characteristics of the products or the technology and less to the privileged owners and creators of the technology.On the contrary, all informed citizens should be able to make (safe and responsible) use of it and the "benefits" of new technologies are determined by their success on the market (rather than the glory of a nation or of the king-national space and defense programmes, however, still echo the pre-modern pride associated with the mere ability to do things others cannot do).
Modern technological innovation therefore receives its specific form by technology which has been democratized in its use and privatized in its production.Competition on the market should ensure product improvement for the benefit of all, rather than a demonstration of the capabilities of a single actor (the king or the state) and its establishment of superiority.Technology from now on can be discussed in terms of benefits and risks for all citizens.Competition on the market is fostered by an openness and access to knowledge 3 .Innovation becomes a goal itself, with improvements of existing products and services through innovation being achieved via the free market.
However, this 'evaluation" scheme of benefits and risks of technology is now put in question by the call for responsible research and innovation.I will elaborate this claim in the section below.

Responsible research and innovation and the quest for the right impacts of research
In modern societies we do not have a specific forum or policy for evaluating particular technologies within the legislative context.We only have at our disposal formal safety, quality and efficacy assessment procedures evaluating the properties of products in the course of passing the three market hurdles.Different technologies are often combined in single products.Thus, eventually the benefits of technologies are "demonstrated" by market success and the potential negative consequences are evaluated under formal risk assessment schemes.This gives a peculiar division of responsibilities among the stakeholders concerned.The state is responsible for defining the risks of technologies under product authorization procedures and product liability law and ensuring market operators compliance, whereas we lack a particular responsibility for what could count for as a positive impact of the technology.The assumption here is that these "benefits" cannot be universalized and that through a pluriformity of the market, consumers are offered a variety of choices and thus the diverging preferences of consumers can be satisfied.Competitors can improve their products through innovation, driven by market demand.Thus, the normative dimension of what counts as an "improvement" is decided by market mechanisms.On top of that, technological innovations are unpredictable and 3 In the time of Bartolomea, it was important to keep your knowledge for yourselves rather than sharing the knowledge with a view on scientific progress or innovation.The curators of the museum explain: Nowadays the passarola seems to have been conceived by Gusmão to trick the many snoopers who wanted to know the results of his experiments.It seems that the scientist also contributed to the spread of false news on the press about one of his flights from Portugal to Vienna aboard the Passarola.positive impacts of innovations under public research and innovation policy schemes are solely justified in purely economic terms.For instance, one assessment is that achieving the EU target of 3% of EU GDP through research and development could create 3, 7 million jobs and increase annual GDP by close to 800 billion Euros by 2025 4 .This assessment is completely 'neutral' to which particular technologies (with the accompanying benefits and risks) will eventually hit the market and which technologies are specifically associated with the increase of jobs 5 .The positive impacts of research and innovation are generally couched in terms of fostering the prosperity and wealth of nations and the availability of finance for research and innovation in general is seen as a condition to achieve this prosperity.
The Flagship Innovation Union is a central part of the EU 2020 strategy and within this innovation is seen as means for "smart growth", defined as "developing an economy based on knowledge and innovation" (EC, 2010, page 3).The Innovation Union aims "to improve framework conditions and access to finance for research and innovation so as to ensure that innovative ideas can be turned into products and services that create growth and jobs".
Discussing particular technologies in terms of benefits or risks within this frame are informal: there is no formal weighing under public policies of the benefits of particular technologies versus their risks.While there is a clearly defined responsibility for operators and the state to address the risks in formal procedures, there is no equivalent for a formal evaluation of the benefits.The responsibility for the positive outcomes of the use of technologies evaporates once they are marketed (whereas responsibility for the negatives outcomes remains).More importantly, there seems to be no normative baseline to which we could judge the positive impacts and benefits of technologies.The responsibility for the positive impacts is left to market operators who look for 4 P. Zagamé (2010) The Cost of a non-innovative Europe.(quoted in the Europe 2020 Flagship Initiative Innovation Union (2011), p5. 5 Although the positive impacts of research and innovation cannot be specified for particular technologies, as their development is unpredicitable, this unpredictability is sharply contrasted with rather precise economic figures in terms of GDP increase and job production (Would we require less investments in RTD if the figure of 3,6 million jobs turns out to be far less, or call for more investments, if the figure is much higher?).economically exploitable products.Public investment in research and innovation policy and thus the positive outcomes of science and technology are primarily justified in macro-economic terms.
This implies that a discussion on the benefits and risks of a particular technology is not only necessarily informal, but it is also artificial A formalization of an evaluation of the positive outcomes (other than in macro-economic terms) is not possible: the success of innovation on the market is unpredictable and reflects a continuous shift of needs and preferences of consumers on the market.Innovation is not fully in the hands of the producers of technology, but users of the technology can dramatically shift the context of use of the technology and thereby trigger off new innovations.For example, the Kinect interactive games made for home computers by Microsoft have been recently used by surgeons to carry out delicate keyhole surgery.This shift of context of use by the users/consumers of this technology was completely unforeseen by Microsoft, yet experts now believe that this technology, further adapted to the surgery context, will be the norm over the next 10 to 15 years (Adam Brimelow, health correspondent, BBC news, 31 May, 2012).German researchers have transformed the Kinect technology into an interactive, augmented reality X-ray machine.These new applications are enabled by the availability of open source framework software.It is likely that many more contexts of use will trigger off innovations which course cannot be foreseen.This brings us to the apparently impossible question to answer: can we justify our public investments in research and innovation beyond uncertain and unpredictable macro-economic benefits?
Eric Cantor, Majority leader in the US senate thinks we can and should, and he launched a website "You cut", allowing citizens to vote on cutting particular research funding programmes.
Cantor, at the launch of his website three years ago, complained that federal funds had been used, among other things for supporting researchers to improve video gaming technology.Cantor wants to change the "culture of spending" and invites citizens to vote on cutting wasteful federal programmes.Currently, Eric Cantors website, among others, allows citizens to vote on the termination of the Christopher Columbus Fellowship foundation, which was designed to produce new discoveries in all fields of endeavor for the benefit of mankind but allegedly does not "demonstrate clear outcomes"(majorityleader.gov/You Cut, 112 th Congress, week 27).
The website is specialized towards negative voting (e.g."you cut") rather than what people do wish to support.The following question may arise: would people have complained about (or voted to cut) support to develop video gaming technology, if they could have known the potential of other contexts of use, such as surgery?How are citizens to evaluate whether the Christopher Columbus Fellowship program does not deliver?It seems tricky enough to "negatively" vote on programmes such as public support for video-gaming technology.Is it then not virtually impossible to decide upon positive outcomes, which we all wish to achieve?Or even more complicated: is it not possible to direct innovation and its funding mechanisms to the 'right impacts'?
Responsible research innovation would then need to be related to two issues: Can we define the right outcomes and impacts of research and innovation?Can we subsequently be successful in directing innovation towards these outcomes if we would agree upon them?I will deal with those questions in the subsequent paragraphs.

Defining the right impacts and outcomes of research
Some philosophers of technology have recently argued that science should move beyond a contractual relationship with society and join in the quest for the common good.In their view, the "good in science, just as in medicine, is integral to and finds its proper place in that overarching common good about which both scientists and citizens deliberate" (Mitcham and Frodeman, 2000).This view may sound attractive, but it fails to show how various communities with competing concepts of the "good life", within modern societies, could arrive at a consensus and how this could drive public (research) policy.Moreover, an Aristotelian concept of the good life is difficult to marry with a modern rights' approach, whereby, for instance in the case of the European Union, the European Charter of Fundamental Rights provides a legitimate and actual basis for European Public Policy.Nonetheless, their point of departure remains challenging: "We philosophers believe that publicly funded scientists have a moral and political obligation to consider the broader effects of their research; to paraphrase Socrates, unexamined research is not worth funding" (Frodeman and Holbrook, 2007) European policy however is also increasingly legitimized in terms of public values driving public policies towards positive impacts.The following citations of prominent European policy makers illustrate the case: • "The defence of human rights and a justice system based on the full respect of human dignity is a key part of our shared European values" Jerzy Buzek, European Parliament President (10 October, 2009) • "Europe is a community of Values".Van Rompuy,First European Council President,19 November 2009 • "My political guidelines for the Commission's next mandate stress the idea that Europe's actions must be based on its values".President Barroso, European values in the new global governance, 14 October 2009 Indeed, European public policies are arguably driven towards positive impacts, underlined by common European values.European Environmental policies for example, highlight the European value of maintaining a high level of protection for the environment.Research and Innovation policy seem to have been an exception to the rule and, although we articulate research and innovation policy since recently more and more in terms of public values, research and innovation programme assessments are typically limited to economic terms that "imperfectly take into account these values" (Fisher et al, 2010).
The US National Science Foundation assesses their proposals in terms of "broader impacts" in the framework of considering research proposals worth funding.Under the European Framework Programmes for Research, there is a long tradition of awarding research grants on the basis of anticipated impacts.Indeed, even at the stage of evaluation of research proposals particular impacts are sought.Currently, expected impacts of research topics which are subject to public calls for proposals are listed in the work programmes of the 7th Framework Programme.But are there legitimate, normative assumptions which support these expected impacts that allow an articulation of the 'right impacts' that allow us to steer public research agendas?We can't make an appeal to concepts of the good life, but we can make an appeal to the normative targets which we can find in the Treaty on the European Union.These normative targets have been  To promote (..) harmonious, balanced and sustainable development of economic activities, a high level of employment and of social protection, equality between men and women, sustainable and non-inflationary growth, a high degree of competitiveness and convergence of economic performance, a high level of protection and improvement of the quality of the environment, the raising of the standard of living and quality of life, and economic and social cohesion and solidarity among Member States.
Rather than pre-empting views and concepts of the "good life", the European Treaty on the European Union provides us then with normative anchor points.These normative anchor points and their mutual relationship thus provide a legitimate basis for defining the type of impacts, or the "right" impacts of research and innovation should pursue.(See figure 2 below).These are of course normative anchor points which have impacts beyond the EU.The EU's commitment to promote Human Rights and demonstrate solidarity with the poorest on earth is reflected in its international policies.If applied to international Research and Innovation policies, this could invite us to address issue such as "technology divides", ethics free zones and broad benefit sharing from scientific and technological advance (see Ozolina et al, 2012).Research and Innovation policy can also be a form of development policy.Arguably, the "grand challenges" of our time reflect a number of normative anchor points of the Treaty and thus can be seen as legitimate.The Lund declaration states that in order to be responsive the European Research Area must develop processes for the identification of Grand Challenges, which gain political support and gradually move away from the current thematic approaches, towards a structure where research priorities are based on these grand challenges".It hopes to give direction to research and innovation in the form of "broad areas of issue-oriented research in relevant fields".It calls for (amongst other things), broad stakeholder involvement and the establishment of public-private partnerships.
The macro-economic justification of investment in research and innovation emphasizes that innovation is the "only answer" to tackle societal challenges: "returning to growth and higher levels of employment, combating climate change and moving towards a low carbon society" (EC, From Challenges to Opportunities: Towards a Common Strategic Framework for EU Research and Innovation Funding.Green paper, com (2011)48, p.3.).This approach implicitly assumes that access to and availability of finance for research and innovation will automatically lead to the creations of jobs and economic growth, thereby taking on the societal challenges along the way.The more innovation, the better.The faster it becomes available, the better.In this macroeconomic model, innovation is assumed to be steerless but inherently good as it produces prosperity and jobs and meets societal challenges, addressed through market-demand.
The Lund declaration gives however an alternative justification for investing in research and innovation, primarily framing this in terms of responding to societal Grand Challenges and further stating that "meeting the grand challenges will be a prerequisite for continued economic growth and for improved changes to tackle key issues".Here the assumption is that sustainable economic growth is only possible when particular societal objectives are met, in the form of responding to Grand Challenges.Innovation is neither seen as steerless nor as inherently good.
Economic prosperity and the anticipation that h innovation yields positive anticipated impacts (such as the creation of jobs and growth) become dependent upon the social context.The Lund Declaration points out those measures are "needed to maximize the economic and societal impact of knowledge" (italics by the author).The idea is clear; to steer the innovation process towards societal beneficial objectives.Additional measures that go beyond removing barriers for research and innovation, availability of and access to finance of research and innovation become then necessary.The Lund declaration defines a type of justification of investment in research and innovation towards particular positive outcomes.The Lund declaration underlines a justification of research and innovation beyond economic terms and with a view on particular outcomes.
Recently, European Commissioner for Research, Innovation and Science, Geoghegan-Quinn stated at a conference on 'Science in Dialogue' that 'research and innovation must responsible to the needs and ambitions of society, reflect its values, and be responsible'' 6 .

Responsible Research and Innovation: Organizing collective responsibility
The impacts of technological innovations are difficult to predict.Social scientists have given up the idea of ever being able to foresee technological innovations and the field of science and technology studies has abandoned ideas of "technological forecasting" since the 1970s.Recent forms of technology assessment (among other "real time technology assessment", Guston and Sarewitz ( 2001) generally focus their attention on monitoring of research and innovations processes or make them more dynamic and inclusive (Schot and Rip, 1997).Responsible research and innovation has to reflect these circumstances.Pre-modern technical inventions can be still be judged by the moral intentions of their designers or privileged users.Modern innovations hardly ever have a single "author" who can be held responsible for the use (by others).Moreover, the negative consequences are often neither foreseeable nor intentional.The fear of a mad scientist creating a Frankenstein is not appropriate in the context of modern innovation-where knowledge is co-produced by many "authors".Modern "Frankenstein's" are not intentionally created by a single actor, but, if they arise, are more likely the unforeseen side effects of collective action.Indeed, techno-scientific applications can remain ethically problematic even in cases where scientists and engineers have the best possible intentions and users have no conscious intention to misuse or abuse.This situation constitutes the major ethical challenge we face today.An ethics focused on the intentions and/or consequence of actions of individuals is not appropriate for the innovation 7 (Grinbaum and Groves, this volume).There is a collective responsibility both for the right impacts and negative consequences, whether these impacts are intentional or not.This is why I have argued for the necessity of knowledge assessment procedures (Von Schomberg, 2007).
In order to specify a scheme, which effectively organizes collective responsibility, we should first get a better picture of what counts as irresponsible innovation in the modern context.

Examples of irresponsible innovation
As many actors are involved in innovation processes, "irresponsible" outcomes are seldom the GM maize in 1998.This second authorisation was remarkable as virtually all previous procedures were inconclusive, and this particular "authorisation" success seemed to have triggered a further collapse of the system, mobilizing the EU Member States unwilling to grant approval for cultivation. 9 What is the irresponsible dimension?
This example shows how substantial dissent among major stakeholders frustrates responsible development.NGOs felt that they had little influence on the direction this technology would lead us in.Regulations were exclusively focused on the safety aspects and the broader environmental, social and agricultural context was not brought into the equation.The need for European harmonization of market introduction sharply contrasted with a variety of national cultures and led to a defacto moratorium in 1998.The outcome should be seen as irresponsible because one company took the lead with a technology push, some NGOs then entered the discussion with a radical view at the moment the technology hit the market stage: the result was that the rest of the industrial sector had to pay a price.A cumbersome slow political process on adopting ever new measures under the already comprehensive framework sealed the sectors fate.
8 The Eurobarometer survey of October 2010 mentions (among other) that 61 percent of Europeans feel uneasy about GM food, and 53 percent believe that GM does harm the environment.In not a single Member State, there is a majority believing that GM food is good for the national economy.The example shows the requirement for stakeholders to share co-responsibility for innovation trajectories.Technology push is a self-defeating strategy.Unlike Monsanto, BASF operated clearly more in line with the requirements of responsible research and innovation.It had to take a painful decision but has gained from this by promoting, a good company image, whereas Monsanto is still often perceived as the "bad guy".Interesting to note is that in the Netherlands discussions among stakeholders (including NGOs) on the course agricultural biotechnology should take (i.e.prior to the marketing of any product) were reopened after the actions of Greenpeace and delays concerning implementation of proper labeling schemes at a national level.There was consensus at the national level among some operators and public administration (Von Schomberg 1999) despite disagreements at the EU level.This shows that EU legislation which frustrates stakeholder agreement can in fact make things worse.

Neglectance of fundamental ethical principles
The Dutch government had to abandon a project on constituting an electronic patient record system (EPRS) in 2011, after the senate voted down the project with a view on unresolved privacy issues.The decision is, in economic terms, disastrous: 300 million euro had been invested over the previous 15 years.EPRS projects elsewhere in the EU face similar problems.
The reason for the failure is that privacy issues were only dealt with at a very late stage of the project, which was initially fully technology driven.Issues such as "who owns the data", and "who is responsible for mistakes" (and their consequences!)became difficult to deal with once the project was technically matured.In addition, the technology evolved over a long period (as ICT technology became significantly more powerful), and lack of proper technology foresight precluded proper implementation.Economic loss for a project which in itself carries a legitimate public purpose should be seen as irresponsible.The costs of acting "irresponsibly" are always substantial.A top-manager of Nokia states: "Typically, the costs of corrective actions are a thousand times more costly when a service is in the operational phase compared to the design phase" 10 .
Earlier stakeholder involvement, earlier and better public engagement, notably taking into the account the implications of a right to privacy (with the associated "right to be forgotten" as proposed by Commissioner Reding for Justice, Fundamental rights and Citizenship.)would have made the project more successful.A similar problem has occurred with the introduction of smart-meters to be installed in private homes to allow for actual monitoring of energy use.As it became clear, that third parties (e.g.possible thieves) would be able to identify the presence of people in their homes, the authorities had to give up the idea of a mandatory introduction.

Policy Pull
A strong policy pull has catalysed the introduction of security technologies, such as the use of biometrics for passports, asylum applications and whole body image technology ("body scanners") at airports.Politicians and policy makers have been eager to accept and promote the implementation of these technologies, sometimes beyond their technical feasibility.
The introduction of the body scanner was discussed fully and consensually within technical advisory committees and within European policy.There seemed to have been no doubt about the reasonableness of its introduction until the German Supreme Court of Justice ruled the introduction as being disproportional to its ends.The European Parliament, which had already ordered the body scanner for use on its premises, cancelled its implementation 11 .The introduction of the body scanner seemed to be incident-driven, after a terrorist managed to board a flight to Chicago from Amsterdam airport late 2009.More recently, after the wide introduction The general assessment problem in this case was how to judge the proportionality of the introduction of this type of technology.The European Commission must deliver general impact assessments on all its major legislative proposals within the framework for better regulation (European Communities, 2006).
The Commission impact assessment follows an integrated approach which was introduced in 2002 (European Communities, 2006).These impact assessments include among others, social, environmental and economic impacts.Thus directives and regulations related to the introduction of security technologies such as biometrics have been subject to such an assessment.Such an analysis should identify whether particular measures, or potential infringement of privacy and data protection, are proportional, and constitute legitimate objection against implementing security technologies.However, the determination of proportionality cannot be fully left to legal experts.One would need to assume normative baselines for acceptable risks or acceptable infringements of privacy rights.These baselines are essentially political and not defined in legislation.Therefore, the baseline should be subject to public debate.As a consequence we find in the EU diverging practices concerning security technologies: biometrics for example is allowed in the Netherlands, for people to enter into public swimming pools, whereas in France it is prohibited.In the Netherlands, biometric passports have been introduced whereby the data are stored in a central data base (this is found to be disproportional in most other Member States of the EU).The risk of failure of biometric estimated by technicians (1 in many millions) has translated in practice to 1 out of 5 passports not being read correctly.

Lack of precautionary measures and technology foresight
The report "Late lessons from early warning" (European Environmental Agency, 2002) gives an impressive account of 12 cases, such as benzene, PCBs, hormones as growth promoters and asbestos, the latter for which the European death toll alone is estimated to become 400 000.I will not here elaborate in too much detail on the negative (anticipated or not) consequences of innovation as they are well described by many others, including those in that particular report.
Nonetheless, a framework for RRI needs to address these consequences, as well as build on the work of these authors.The lessons learned from these 12 cases mainly relate to decision making under scientific uncertainty and scientific ignorance.However, it also relates to the benefits of innovation by making the appeal to" promote more robust, diverse and adaptable technologies so as to minimise the costs of surprises and maximise the benefits of innovation" (European Environmental Agency, 2002).The authors of the report make the case for channelling innovation into alternative routes, for which for example, the cases of asbestos and halocarbons provide forceful illustrations.Rather than a constraint, the precautionary principle can thus provide an incentive to open up alternative research and development trajectories.

A framework for Responsible Research and Innovation
The following definition for Responsible Research and Innovation is proposed:

Definition: Responsible Research and Innovation is a transparent, interactive process by which societal actors and innovators become mutually responsive to each other with a view to the (ethical) acceptability, sustainability and societal desirability of the innovation process and its marketable products( in order to allow a proper embedding of scientific and technological advances in our society).
There is a significant time lag (this can be several decades) between the occurrence of technical The normative anchor points should be reflected in the product dimension.They should be: (Ethically) acceptable: in an EU context this refers to a mandatory compliance with the fundamental values of the EU charter on fundamental rights [right for privacy etc.] and the safety protection level set by the EU.This may sound obvious, but the practice of implementing ICT technologies has already demonstrated in various cases that the fundamental right for privacy and data protection can and has been neglected.It also refers to the "safety" of products in terms of acceptable risks.It goes without saying that ongoing risk assessments are part of the procedure towards acceptable products when safety issues are concerned.However, the issue of safety should be taken in a broader perspective.The United Kingdom's largest public funder of basic innovation research, the Engineering and Physical Science and Research Council asked applicants to report the wider implications and potential risk (environmental, health, societal and ethical) associated with their proposed research in the area of nanosciences (Owen and Goldberg 2010).This highlighted the fact that, often, the risks related to new technologies, can neither be quantified nor a normative baseline of acceptability assumed by scientists (acknowledging that any, particular baseline cannot be assumed to represent the baseline of societal acceptance).
-Sustainable: contributing to the EU's objective of sustainable development.The EU follows the 1997 UN "definition" of sustainable development, consisting of economic, social and environmental dimensions in mutual dependency.This overarching anchor point can become further materialized under the following one: -Socially desirable: "socially desirable" captures the relevant, and more specific normative anchor points of the Treaty on the European Union, such as "Quality of life", "Equality among men and women" etc.(see above).It has to be noted that a systematic inclusion of these anchor points in product development and evaluation would clearly go beyond simple market profitability, although the latter could be a precondition for the products' viability in market competitive economies.However, it would be consistent with the EU treaty to promote such product development through the financing of research and development actions.In other words, at this point, Responsible Research and Innovation would not need any new policy guidelines, but simply would require a consistent application of the EU's fundamentals to the research and innovation process reflected in the Treaty on the European Union.Perhaps it has been wrongly assumed that these values could not be considered in the context of research and innovation.
Since the Lund Declaration, a process to take into account societal objectives in the form of addressing Grand Challenges has been set in motion.

Responsible Research and Innovation features both a product and process dimension:
Product dimension: Products be evaluated and designed with a view to their normative anchor points: high level of protection to the environment and human health, sustainability, and societal desirability.

Process dimension
The challenge here is to arrive at a more responsive, adaptive and integrated management of the innovation process.A multidisciplinary approach with the involvement of stakeholders and other interested parties should lead to an inclusive innovation process whereby technical innovators become responsive to societal needs and societal actors become co-responsible for the innovation process by a constructive input in terms of defining societal desirable products.The product and process dimension are naturally interrelated.Implementation is enabled by five mechanisms: technology assessment and foresight, application of the precautionary principle, normative / ethical principles to design technology, innovation governance and stakeholder involvement and public engagement.
Table 1(see after references, end of the text), provides a matrix which describes examples of lead questions to be answered by the stakeholder either from a product or process perspective in order to fully implement an RRI scheme (the lead questions with the same colour, represent the alternative emphasis on either the product or process dimension).
1. Use of Technology Assessment and Technology Foresight.This is done in order to anticipate positive and negative impacts or, whenever possible, define desirable impacts of research and innovation both in terms of impact on consumers and communities.Setting of research priorities and their anticipated impacts needs to be subject to a societal review.This implies broadening the review of research proposals beyond scientific excellence and including societal impacts 12 .
Specific Technology Assessment methods also help to identify societal desirable products by addressing the normative anchor points throughout their development.Methodologies to further "script" the future expected impacts of research should be developed (Den Boer, Rip and Speller, 2009).A good example exists in the field of synthetic biology by Marc Bedau et al. (2009).They have identified six key checkpoints in protocell development (e.g. cells produced from nonliving components by means of synthetic biology) in which particular attention should be given to specific ethical, social and regulatory issues, and made ten recommendations for responsible protocell science that are tied to the achievement of these checkpoints.
Technology Assessment and Technology Foresight can reduce the human cost of trial and error and take advantage of a societal learning process of stakeholders and technical innovators.It creates a possibility for anticipatory governance.This should ultimately lead to products which are (more) societal robust.

Application of Precautionary Principle
The precautionary principle is embedded in EU law and applies especially within EU product authorization procedures (e.g.REACH, GMO directives etc.).The precautionary principle works as an incentive to make safe and sustainable products and allows governmental bodies to 12 The Netherlands Organisation for Scientific Research (NWO) has developed a research funding programme on Responsible Innovation under which research proposals are subject to a review in terms of societal relevance.See: http://www.nwo.nl/nwohome.nsf/pages/NWOA_7E2EZG_Engintervene with risk management decisions (such as temporary licensing, case by case decision making etc.) whenever necessary, in order to avoid negative impacts.
The responsible development of new technologies must be viewed in its historical context.Some governance principles have been inherited from previous cases: this is particularly notable for the application of the precautionary principle to new fields such as that of nanosciences and nanotechnologies.The precautionary principle is firmly embedded in European policy, and is enshrined in the 1992 Maastricht Treaty as one of the three principles upon which all environmental policy is based.It has been progressively applied to other fields of policy, including food safety, trade and research.
The principle runs through legislation for example in the 'No data, no market' principle of the REACH directive for chemical substances, or the pre-market reviews required by the Novel Foods regulation as well as the directive on the deliberate release of GMOs into the environment.
More generally, within the context of the general principles and requirements of European food law it is acknowledged that "scientific risk assessment alone cannot provide the full basis for risk management decisions" (European Commission, 2002) leaving open the possibility of risk management decision making partly based on ethical principles or particular consumer interests.
In the European Commission's Recommendation on a Code of Conduct for Nanosciences and Nanotechnologies Research, the principle appears in the call for risk assessment before any public funding of research (a strategy currently applied in the 7th Framework Programme for research).Rather than stifling research and innovation, the precautionary principle acts within the Code of Conduct as a focus for action, in that it calls for funding for the development of risk methodologies, the execution of risk research, and the active identification of knowledge gaps.

Innovation governance a. Multistakeholder involvement
Multistakeholder involment in RRI-projects should bring together actors from industry, civil society and research to jointly define an implementation plan for the responsible development of a particular product to be developed within a specific research/innovation field, such as information and communication technology or nanotechnology.Responsible innovation should be materialised in terms of the research and innovation process as well as in terms of (product) outcomes.The advantage is that actors cannot exclusively focus on particular aspects (for instance, civil society organizations addressing only the risk aspects) but have to take a position on all aspects of innovation process as such.Thus allowing a process to go beyond risk Policy development treads a fine line: governments should not make the mistake of responding too early to a technology, and failing to adequately address its nature, or of acting too late, and thereby missing the opportunity to intervene.A good governance approach, then, might be one which allows flexibility in responding to new developments (Owen et al, this volume) In the absence of a clear consensus on definitions, the preparation of new nano-specific measures will be difficult and although there continues to be significant scientific uncertainty on the nature of the risks involved, good governance will have to go beyond policy making that focuses only on legislative action.The power of governments is arguably limited by their dependence on the insights and cooperation of societal actors when it comes to the governance of new technologies: the development of a code of conduct, then, is one of their few options for intervening in a timely and responsible manner.The area of data-and privacy protection in the context of the use of ICT and security technologies should also be impacted by forms of self-regulation and standard setting.Data controllers based at operators need to provide accountability, which can be termed as a form of verifiable responsibility (Guagnin, Hempel and Ilten, 2011).The involvement of third parties which can implement, minimally, a transparent verification practice will be crucial.In other fields, the whole certification can be carried out by a third party.For example, in 1996, the 14 The European Project NANOCODE makes this point concerning nanosciences and nanotechnologies, see: http://www.nanocode.eu/World Wildlife Fund (WWF) and Unilever joined forces and collectively constructed a longterm programme for sustainable fisheries.They founded an independent non-profit organisation to foster worldwide fisheries.They also apply "standards of Sustainable Fishing", which is also monitored by independent certifying agencies to control those standards.
Standards will also need to reflect particular ethical considerations and go well beyond mere technical safety issues.Currently, the development of new ISO standards for Nanofood might involve the inclusion of ethical standards (Forsberg, 2010).Recently "Midstream Modulation" (Fisher et al., 2006;Fisher, 2007, Fisher and Rip, this volume) has emerged as a promising approach to increase social-ethical reflexivity within research practices.In the form of laboratory engagement practices, social scientists and ethicists are

Outlook
Responsible Research and Innovation need to be addressed by various actors and institutions.
Institutionally some progress in under way at the level of programmes of Research Councils.As an positive counterexample to the "You cut" initiative of the American senator a noteworthy experiment with which, through a process of public deliberation, reflected on the purposes of research and used this reflection to frame a research funding call in the area of nanotechnology for medicine and healthcare (Jones, 2008).The public dialogues provided a clear steer about the relative priorities of six potential application areas of nanotechnology for healthcare, informing and shaping the nature of the funding call itself, such that it could respond better to social values (for more detail see Sykes and Macnaghten, this volume).One can imagine further initiatives to have citizens shape calls for research proposals.
The most crucial advancement of RRI will be dependent on the willingness of stakeholders to Furthermore, further institutionalizations of technology foresight and technology assessments are necessary within the legislative process.At the European level, now impact assessments have been made mandatory, an opportunity arises to make better and systematic use of assessments.I have argued that we have to go beyond assessing research and innovation beyond their economic impacts.Bozeman and Sarewitz (2011) have proposed a framework for a new approach to assessing the capacity of research programs to achieve social goals.The further development of such frameworks are badly needed as the promises of scientist to address social objectives (regularly leading to a "hype" and corresponding increased levels of research funding) while developing their research is often sharply contrasted with the actual outcomes.
Internationally, a global perspective needs to be developed.Diverging ethical standards at the international level and "ethics-free" zones pose challenges to the introduction of RRI at the global level.Ozolina et al (2012) have recently addressed the challenges RRI faces at the global level and advocate to advance an international framework for RRI by means of multilateral dialogue.
All these initiatives may well help us to address socio-economic concerns around research and innovation processes, without formally introducing a fourth hurdle.Instead of a "hurdle", RRI should become a research and innovation 'design' strategy which drives innovation and gives some "steer" towards achieving societal desirable goals.

Figure 1 :
Figure 1: Prototype of the Passarola (adopted from Wikipedia.org) democratically agreed and provide the legitimate basis for having a public framework programme for research at the European Level.From article 3 of the Treaty on the European Union (European Union, 2010) we can derive the following:  "The Union shall (…) work for the sustainable development of Europe based on balanced economic growth and price stability, a highly competitive social market economy, aiming at full employment and social progress, and a high level of protection and improvement of the quality of the environment.It shall promote scientific and technological advance.It shall combat social exclusion and discrimination, and shall promote social justice and protection, equality between women and men, solidarity between generations and protection of the rights of the child.

Figure 2 .
Figure 2. Normative anchor points derived from the Treaty on the European Union result of one single irresponsible actor.More typically, irresponsible innovation is reflected in practices where stakeholders were unaware of the importance of societal context, or where stakeholder interactions were unproductive in the resolutions of conflicts.I categorise here four types of irresponsible innovation: Technology push, Neglectance of fundamental ethical principles, Policy Pull, and Lack of precautionary measures and technology foresight.One seldom finds examples which reflect only one of these four dimensions of 'irresponsible' innovation, as more often a mix of these are at play.Nonetheless, in particular examples, one particular dimension may play a more dominant role than in another.Technology push has occurred in the European Union when Monsanto tried to accomplish a fait accompli with the market introduction of genetically modified soya in the mid-1990s.Environmental groups, (notably Greenpeace who did not react to GMOs as an environmental concern prior to their introduction on the market) responded with an outright rejection when the first shipment of genetically modified soya entered the port of Rotterdam in 1996.The further process of innovation (or lack of it) was framed by an often bitter fight among a very few industrial actors and a growing group of NGOs opposing the introduction of the "technology.This occurred amidst indecisive and reluctant European national governments devising contradictory measures, some of which had to be challenged at the European Court of Justice by the European Commission.During the subsequent years after the first marketed GMO, NGOs and European governments were overbidding each other with calls for stricter regulations of GMOs, eventually resulting in a revised set of GMO regulations and directives (notably directive 2001/18, EC, 2001), which up to date were never applied consistently and currently the European Council considers a response from the European Parliament to a Commission proposal to give Member States somewhat more flexibility in banning cultivation of GMOs on their own territory.The general public learned to become deeply sceptic about future applications and the general public perception was constituted that this type of innovation does not deliver a sufficient benefit 8 .A major European operator, BASF, made the announcement in January 2012 not to market a genetically modified potato after lengthy consultations with NGOs, and withdraw from the European market even though it managed to receive an authorisation to cultivate the potato in 2010.This was only the second granted authorisation, the first being for

9
See the instructive article of Vesco Paskalev (2012)."Can Science tame politics: The collapse of the new GMO Regime in the EU" in: European Journal of Risk Regulation, 2/2012, p. 190ff.
inventions (or planned promising research) and the eventual marketing of products resulting from RTD and innovation processes.The societal impacts of scientific and technological advances are difficult to predict.Even major technological advances such as the use of the internet and the partial failure of the introduction of GMOs in Europe have not been anticipated by governing bodies.Early societal intervention in the Research and Innovation process can help avoid technologies failing to embed in society and / or help that their positive and negative impacts are better governed and exploited at a much earlier stage.Two interrelated dimensions can be identified: the product dimension, capturing products in terms of overarching and specific normative anchor points (see discussion above) and a process dimension reflecting a deliberative democracy.
governance and move to innovation governance.The company BASF, for example, has established a dialogue forum with civil society organizations and also developed a code of conduct for the development of new products 13 b.Use of codes of conduct Codes of Conduct, in contrast to regulatory interventions, allow a constructive steering of the innovation process.They enable the establishment of a proactive scientific community which identifies and reports to public authorities on risks and benefits at an early stage.Codes of Conduct are particular useful when risks are uncertain and when there is uncertain ground for legislative action (nanotechnology for example).Codes of Conduct also help to identify knowledge gaps and direct research funds towards societal objectives.
4. Ethics as a "Design" factor of Technology and increasing social-ethical reflexivity in research practicesEthics should not be seen as being only a constraint of technological advances.Incorporating ethical principles in the design process of technology can lead to well accepted technological advances.As discussed above, in Europe, the employment of Body Imaging Technology at Airports has for example raised constitutional concerns in Germany.It has been questioned whether the introduction is proportional to the objectives being pursued.The introduction of a "smart meter" at the homes of people in the Netherlands to allow for detection of and optimisation of energy use, was rejected on privacy grounds, as it might have allowed third parties to monitor whether people are actually in their homes.These concerns could have been avoided if societal actors had been involved in the design of technology early on."Privacy by design" has become a good counter example in the field of ICT, by which technology is designed with a view to taking privacy into account as a design principle of the technology itself.Yet, practicing it is still rare.The European project ETICA 15 has recommended the introduction of specific governance structures for emerging (ICT) technologies in this regard.
work together toward social desirable products.Up till now, the examples of industry-NGO cooperation has been primarily limited to addressing the risks, e.g. the negative aspects of products.Under the European 7 th Framework Programme for Research and Innovation, the 2013 Science in Society Workprogramme provides an opportunity for a "demonstration project" incentivizing actors from industry, civil society and research institutions to "jointly define an implementation plan for the responsible development of a particular product to be developed within a specific research and innovation field".Responsible Research and Innovation should be shown in terms of the product development process (such as stakeholder involvement, etc.) and the quality of the final product (complying with, among other standards, those relating to sustainability and ethics).
the body scanner at airports in the US, the device has come under attack, as it apparently does not deliver the substantial security gain, or worse can introduce new security threats.The introduction of body scanner at airports in Europe was eventually approved by the European Parliament in 2011, however with substantial requirements: there is no mandatory use for passengers, the body scanner does not make pictures, but rather representations of the body and the representations are not stored.Such requirements could have been anticipated much earlier, if the technology had been guided by proper technology assessments and public scrutiny. of . After a regulatory review in 2008, the European Commission came to the conclusion that there is no immediate need for new legislation on nanotechnology, and that adequate responses can be developedespecially with regard to risk assessmentby adapting existing legislation.
Bush (2010)n Commission states in the second implementation report on the action plan for Nanotechnologies that "its effective implementation requires an efficient structure and coordination, and regular consultation with the Member States and all stakeholders" (Commission of the European Communities, 2009).Similarly, legislators are dependent on scientists' proactive involvement in communicating possible risks of nanomaterials, and must steer clear of any legislative actions which might restrict scientific communication and reporting on risk.The ideal is a situation in which all the actors involved communicate and collaborate.The philosophy behind the European Commission's code of conduct, then, is precisely to support and promote active and inclusive governance and Code at European level on a biannual basis.It should be considered that such Codes of Conduct would in the future extend their scope beyond research and also address the innovation process.14Adoption of standards, certification and self-regulationThe adoption of standards and even "definitions" are fundamental requirements to allow for responsible development.The outstanding adoption of a definition for nanoparticles, for example makes legislation and adequate labelling practices difficult, if not impossible.Bush (2010)notes that the use of standards, certifications and accreditations constitute a new form of governance which progressively has replaced and transmuted positive law, as a product of the state, with its market equivalent.Although this form of governance is in need of improvement, we unavoidably have to make productive use of it, as the flood of products and processes coming on to the market will not be manageable through governmental bodies and agencies alone.Yet, the perception and working practice of these standards is significant.In 2005, it was claimed that the EU had forced local authorities to remove see-saws from children's playgrounds.No such EU measures were taken.Some standards were set by the European Committee for Standardisation (CEN), a voluntary organisation made of national standards bodies.CEN sought to limit the height from which children could fall, by specifying the maximum height for seats and stands, and by setting standards for hand supports and footrests.Manufacturers could choose to follow these standards, which carried the advantage of being able to export across Europe, instead of having to apply for certification in each country(European Communities, 2006).
(Van Est, 2010)ms of natural scientists.The embedded social scientist engages natural scientists in the wider impact of their work, while doing research in the laboratories.Reports from these practices could feed into schemes on responsible research and innovation.Continuous feedback from information generated in Technology Assessment, Technology Foresight and demonstration projects to policy makers could allow for a productive innovation cycle.Knowledge assessment procedures should be developed in order to allow assessment of the quality of information within the policy process, especially in areas in which scientific assessments contradict each other or in the case of serious knowledge gaps.(TheECpractises this partly with its impact assessments for legislative actions).Knowledge assessment could integrate distinct approaches of cost-benefit analysis and environmental and sustainability impact assessments.In short: models of responsible governance should be devised which allocate roles of responsibility to all actors involved in the innovation process.Ideally, this should lead to a situation in which actors can resolve conflicts and go beyond their traditional roles: companies addressing the benefits and Non-Governmental Organisations the risks.Co-responsibility implies here that actors have to become mutually responsive, thus companies adopting a perspective going beyond immediate market competiveness and NGOs reflecting on the constructive role of new technologies for sustainable product development.In this context, Technology Assessment, as practised, for example, by the Dutch Rathenau Institute, can take up the function of "seducing actors to get involved and act"(Van Est, 2010)."On-goingpublic debate and monitoring of public opinion is needed for the legitimacy of research funding and particular scientific and technological advances.Continuous public platforms should replace one-off public engagement activities (Sykes and Macnaghten, this volume) with a particular technology and, ideally, a link with the policy process should be established.The function of public debate in viable democracies includes enabling policy makers to exercise agenda and priority setting.Public debate, ideally, should have a moderating impact on "Technology Push" and "Policy Pull" of new technologies which sometime unavoidably may occur.