Legal provisions of COM(2016)5 - Response to the Report of the High Level Expert Group on the Ex Post Evaluation of the Seventh Framework Programme

Please note

This page contains a limited version of this dossier in the EU Monitor.

dossier COM(2016)5 - Response to the Report of the High Level Expert Group on the Ex Post Evaluation of the Seventh Framework Programme.
document COM(2016)5 EN
date January 19, 2016
EUROPEAN COMMISSION

Brussels, 19.1.2016

COM(2016) 5 final

COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS

On the Response to the Report of the High Level Expert Group on the Ex Post Evaluation of the Seventh Framework Programme

{SWD(2016) 1 final}
{SWD(2016) 2 final}


COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS


On the Response to the Report of the High Level Expert Group on the Ex Post Evaluation of the Seventh Framework Programme


1. Introduction


According to the Decision on the Seventh Framework Programme of the European Community for research, technological development and demonstration activities 2007-2013 (FP7) 1 , independent experts carry out an external evaluation of the rationale, implementation and achievements of the Programme by the end of 2015. To this end, an external High Level Expert Group (HLEG) was set up in September 2014. It submitted its report to the Commission on 19 November 2015. 2 This Communication sets out the findings and recommendations of the HLEG and the Commission's response. It is accompanied by a Staff Working Document (SWD) in which the Commission services have evaluated FP7 as regards its effectiveness, efficiency, relevance, coherence and EU added-value, in line with Better Regulation requirements.


With a voted budget of EUR 55 billion, FP7 was one of the largest transnational competitive RTD programmes in the world. The main focus of FP7 was on science, especially the promotion of collaborative research and excellence. In the last years of FP7, increased emphasis was put on innovation and the participation of Small and Medium Enterprises (SMEs). In order to facilitate the award and management of grants, the Commission also implemented various measures to reduce administrative burden.


2. Key Findings on FP7 Achievements


FP7 promoted excellence by involving the best EU and non-EU researchers in more than 25,000 high-level inter-disciplinary, collaborative projects generating world-class research results. However, to date, 50% of these projects are still on-going. Therefore, the ex-post evaluation of FP7 cannot present the complete picture on FP7 results and impacts. So far, FP7 projects have generated 170,000 publications. The share of publications that is in highly ranked journals lies above EU and US averages. More than 1,700 patents and 7,400 commercial exploitations have so far resulted from FP7 projects. 3 FP7 promoted ground-breaking research through the European Research Council (ERC). The number of publications in top-rated scientific journals that acknowledge ERC funding, as well as the number of Nobel Prizes and Fields Medals received by ERC grantees, all demonstrate that ERC grants have become a hallmark of scientific excellence.


Involving participants from 170 countries, FP7 was open to the world. It widened EU participation and contributed to the achievement of ERA. FP7 created durable cross-border, inter-disciplinary, cross-sectoral networks: on average, 11 organisations from six different countries and nine different regions collaborate in each FP7-funded project. FP7 encouraged the alignment of national research and innovation systems and policies. In most EU Member States, FP7 contributed to scientific excellence, addressing societal challenges, and set standards for research funding mechanisms and selection processes.


FP7 strengthened the training and long-term mobility of researchers, enhanced the quality of doctoral training and helped improve working conditions for researchers in the EU. FP7 Marie Curie Actions supported 50,000 researchers, including 10,000 PhD candidates from 140 countries. The programme stimulated the mobility of researchers across Europe. It also contributed to sustainable employment for researchers in Europe and helped increase the participation of women researchers and of international researchers in beneficiary research teams.


FP7 accounts for about 7% of total public expenditure on R&I in the EU. However, both directly and by leveraging private and Member States' funding, its economic impact on, for instance, GDP and employment is substantial. It has been estimated that FP7 will increase GDP by about EUR 20 billion per year over the next 25 years through its indirect economic effects and that it will create over 130,000 research jobs per year and 160,000 additional jobs 4 . There is also evidence of positive micro-economic effects: participating enterprises report the development of innovative products, processes and services and increased turnover, productivity and competitiveness. Results of econometric analyses 5 show that SMEs participating in the FP7 scored 38% higher than the control group with regard to employment growth and operating revenue. However, as the HLEG points out, it is premature to make a final assessment of the market impact of FP7 projects.


FP7 engaged industry and SMEs strategically. 6 Large corporations as well as SMEs have been involved extensively through Public-Private Partnerships – including Joint Technology Initiatives (JTIs) and contractual Public-Private Partnerships – and through SME-specific instruments. 7 This contributed to Europe’s innovation-based competitiveness.


FP7 addressed European societal challenges in fields like health, energy, transport and security and contributed to tackling global challenges such as sustainable development. With respect to gender equality, the focus shifted from promoting individual female scientists to facilitating structural change in institutions.


In FP7, the overall proportion of female evaluators was slightly higher than the 40% target. In addition, 38% of the FP7 reported workforce was female.


3. HLEG Recommendations and Commission Response


This section provides the Commission response to the HLEG recommendations, which are based on the results of their evaluation of FP7. 8  


Recommendation (a): Ensure focus on critical challenges and opportunities in the global context


The Commission agrees with the overall intent of the HLEG recommendation.


Horizon 2020 is focused on the resolution of grand societal challenges. It is underpinned by a challenge-based approach that promotes the actual deployment of the new knowledge generated. In order to ensure the actual deployment of research results, Horizon 2020 takes an all-encompassing approach to R&I, supporting the entire innovation chain from frontier research to close-to-market activities e.g. through pilot and demonstration activities. At the same time, Horizon 2020 steps up investment in frontier research through bottom-up actions. Horizon 2020 has been designed with sufficient built-in flexibility to tackle new and unexpected challenges.


Horizon 2020 fosters a permanent, structured dialogue with the private sector through established platforms including the European Technology Platforms, industrial roundtables in the context of the digital industrial strategy, and formal advisory groups on all programme areas. Horizon 2020 involves the private sector strategically in the formulation and implementation of comprehensive research agendas in key industrial sectors, and in the mobilisation of the knowledge and financial resources required for that purpose, through Joint Technology Initiatives (JTIs) and contractual Public-Private Partnerships. JTIs under Horizon 2020 benefit from a number of implementation features that cut red tape for business and make them better fit for purpose. The openness, transparency and effectiveness of the JTIs will be assessed in the context of their interim evaluations to be completed by June 2017.


Horizon 2020 significantly enhances the opportunities for innovative SMEs, by allocating 20% of the combined budgets for the societal challenges and Leadership in enabling and industrial technologies for SMEs, and through the SME Instrument, the Fast Track to Innovation instrument, and the Eurostars Public-Public Partnership. The over-subscription rate shows that services and networking on European level are considered by SMEs to provide distinct added-value.


The Horizon 2020 Work Programme for 2016-2017 aims at ensuring the increased availability of funding for innovative companies by leveraging opportunities in the context of the European Fund for Strategic Investments (EFSI). In order to streamline access to funding for innovation purposes, the Commission will explore the feasibility of launching a European Innovation Council.

In line with Recommendation (a), Horizon 2020 is strongly focused on the resolution of grand societal challenges, Horizon 2020 aims to foster a permanent, structured dialogue with the private sector and involve it strategically, and to maximise the involvement of innovative SMEs.

Furthermore, the Commission will:

• Implement a new strategic focus for Horizon 2020 in order to maximise its contribution to open innovation, open science' and open to the world;

• Maximise the synergies between R&I in thematic priorities of societal challenge areas and new and emerging digital and key enabling technologies and infrastructures;

• Explore the need for, and the feasibility of, a European Innovation Council as a means to boost innovation and streamline existing instruments;

• Evaluate the JTIs in terms of inter alia their openness, transparency and effectiveness by June 2017;

• Facilitate the elaboration of important projects of common European interest, which can foster vast deployment of research into mature technologies. 9


Recommendation (b): Align research and innovation instruments and agendas in Europe


The Commission agrees with the need to better align research and innovation instruments and agendas in Europe.


Framework Programmes can play an important role in aligning national research strategies and programmes. Different EU instruments should also be aligned to address the societal challenges Europe is faced with and to ensure continued investment in those sectors where Europe has a leading position in the world.


With regard to the alignment of national research strategies and programmes, Horizon 2020 further strengthens instruments already developed under FP7 – for instance, ERA-NET Co-Fund and Article 185 initiatives – to pool resources across Member States, define common strategic research agendas, avoid duplication, implement joint calls, etc. Furthermore, the Commission has launched a Policy Support Facility under Horizon 2020 to assist Member States to implement effective reforms, in line with ERA priorities.


In order to achieve alignment with EU Structural Funds programmes, the Commission will continue to work with national and regional smart specialisation strategies that enable the pooling of resources and the achievement of critical mass around shared priorities. The bottom-up entrepreneurial discovery process for developing and evolving the smart specialisation strategies in the coming years will bring together the key R&I actors. There is a need for clear communication about how the different EU funding programmes work. To this end, the Commission has developed a guide for policy-makers and implementing bodies to support synergies and the combination of different funds 10 and launched a new Participant Portal with detailed guidance for applicants on how to apply for and manage a project under Horizon 2020. In addition, a number of JTIs have signed Memoranda of Understanding with European Regional Development Funds managing authorities setting up a framework for structured cooperation.


Recently, the Commission launched the 'Seal of Excellence' scheme, which will allow Member States and their regions to recognise the quality label awarded to promising project proposals submitted under Horizon 2020, and promote their access to different funding sources like ESIF and other national, regional or private investment programmes. 11  


Horizon 2020 will continue to support a broad approach to innovation, including social innovation.


The Commission agrees with the HLEG that other policies and regulations should be conducive to innovation. In this context, the specific R&I tool within the impact assessment guidelines of the Commission’s Better Regulation package will enable the Commission to ensure that its new legislative proposals will be ‘fit for innovation’. In addition, the Commission is exploring the set-up of 'Innovation Deals’ at EU level to address existing regulatory uncertainties which hinder innovation in partnership with stakeholders. 12

In line with Recommendation (b), the Commission aims, through the implementation of Horizon 2020, to ensure the alignment of national research strategies and programmes and to facilitate synergies between Horizon 2020 projects and relevant smart specialisation strategies.

Furthermore, the Commission will:

• Use the Policy Support Facility and Cohesion Policy capacity building support to assist Member States to implement effective reforms of their research and innovation systems;

• Continue to foster synergies between Horizon 2020, the Structural Funds and LIFE, and report on this issue in the context of the Horizon 2020 interim evaluation; potential synergies with the European Fund for Strategic Investments (EFSI) will also be promoted;

• Ensure that new Commission proposals are fit for innovation by applying the Better Regulation Guidelines, and in particular the Research and Innovation Tool of the impact assessment guidelines;

• Screen the EU regulatory landscape to identify relevant existing regulatory uncertainties, which hinder innovation and, consequently, impede the potential to create economic value in Europe;

• Improve the framework conditions for better innovation ecosystems in the EU;

• Explore the feasibility of setting up 'Innovation Deals' as a new way of addressing regulatory uncertainties and to support the development and deployment of innovations, which could help build investors' confidence.


Recommendation (c): Integrate the key components of the Framework Programmes more effectively


The Commission agrees with the HLEG that fragmentation and the emergence of silos in Framework Programmes would undermine its efficiency and coherence. The Horizon 2020 structure, around three priorities is expected to improve the efficiency and effectiveness of the programme. Cross-cutting issues have been integrated into the three priorities, thereby avoiding silos and fragmentation. For example, Blue Growth and the Internet of Things are cross-cutting initiatives requiring coordination between Societal Challenges and Leadership in Industrial and Emerging Technologies (LEIT). Horizon 2020 fosters linkages between specific programmes and financial instruments and allows budget transfers between different sub-programmes.


In order to maximise client satisfaction and efficiency, Horizon 2020 grant management has been delegated to four Executive Agencies. The division of labour between the Commission and the Executive Agencies is clearly defined and documented in Delegation Acts. The governance structures of the Executive Agencies are designed to ensure proper supervision by the Commission and transparency. Special attention is paid to ensuring the effectiveness and efficiency of the feedback loop feeding project results from the Executive Agencies back to the Commission for policy purposes.


The Commission is committed to the consistent application of a single set of rules for participation and dissemination in Horizon 2020 across all actors implementing the programme. To help coordinate and deliver the programme, a Common Support Centre (CSC) has been set up in the Commission. It provides services in legal support, ex-post audit, IT systems and operations, business processes, programme information and data to all research DGs, Executive Agencies and Joint Undertakings implementing Horizon 2020. The effective coordination between all actors implementing Horizon 2020 is a permanent challenge that figures high on the Commission's agenda and that the Commission strives to meet on a daily basis.


Several measures were implemented in FP7 to simplify the management of proposals and grants and to reduce administrative burden. It is estimated 13 that the changes in the cost calculation regime in FP7 led to savings of EUR 551 million in FP7 compared to FP6. However, the European Court of Auditors, in its 2014 annual report, concluded that the persistently high error rate reflects risks inherent in the design and implementation of FP7. This was important input for the design of Horizon 2020 that resulted in large-scale simplification of the funding rules, processes and IT.

In line with Recommendation (c), the Commission ensures coherence between the different Horizon 2020 priorities inter alia through cross-cutting issues and has established effective mechanisms to coordinate with the Executive Agencies and to apply consistently a single set of rules.

Furthermore, the Commission will:

• Continue to identify and implement simplification measures;

• Complete, as legally required, the evaluation of the operation of the Executive Agencies REA and ERCEA by the first quarter of 2016.


Recommendation (d): Bring science closer to the citizens


The Commission agrees with the HLEG that the engagement of the general public with the Programme and more generally the active participation of citizens in science is important. In its implementation, Horizon 2020 engages citizens and other relevant stakeholders through the development of responsible research and innovation agendas and by providing greater support to public outreach activities.


The Commission has implemented a number of measures to support the communication of impact and relevance of EU grant support for R&I. In Horizon 2020, beneficiaries of EU R&I funding are obliged to reach out pro-actively to non-specialised audiences to highlight the societal impact of their work. 14 For example, the European Researchers' Night events 15 aim to increase citizen's awareness of research and innovation.


The Commission will also strengthen efforts to communicate about research results and impact to the public and by developing a more visible and active presence in social media. A repository of Framework Programme success stories is available online 16 and is regularly updated to demonstrate the impact that science has on citizens' lives.


Furthermore, the Commission has intensified open access to scientific peer reviewed publications in FP7. 17 In Horizon 2020, the Commission has made open access to scientific publications mandatory, paving the way for full transparency with respect to innovations and breakthroughs produced by EU funding. In addition, the Commission is running a pilot exercise on open access to research data in Horizon 2020 involving open access as the default mode (but including an opt-out option) and obligatory data management plans.


Transparency and involving citizens in EU policy-making are also key principles in the Commission’s Better Regulation Agenda. Stakeholders were consulted widely when Horizon 2020 was being designed and the Commission will continue to involve stakeholders in the evaluation and design of Framework Programmes.

In line with Recommendation (d), the Commission has brought science closer to the citizen by involving them in the design of Horizon 2020, through the implementation of dedicated and cross-cutting activities in Horizon 2020 focusing on greater citizens involvement, and by better communicating the impact that science can have on their lives.

Furthermore, the Commission will:

• Further strengthen open access to research publications and data;

• In line with the Better Regulation agenda, involve citizens in defining user-driven research and innovation agendas, in particular when preparing new Framework Programmes and specific Work Programmes.


Recommendation (e): Establish strategic programme monitoring and evaluation


The Better Regulation Package introduces a systematic and coherent approach for all evaluations carried out by Commission services and will ensure robust quality control of Commission evaluations and the uptake of evaluation results in future policy-making. The Better Regulation Guidelines on evaluation guarantee the quality control of contracted evaluations through Steering Committees and standardised quality assessment forms. The Commission will make further efforts to improve data quality and coherence across different exercises, including monitoring and evaluation, in order to provide transparent and accurate data stocks in support of evidence-based decision-making.


Horizon 2020 has defined concrete key performance indicators, indicators for cross-cutting issues - including on gender and international cooperation – and specific targets, including for sustainability, climate change and SMEs, which allow to compare the achievements of Horizon 2020 with the baseline at the start of the programme in a more systematic and recurring way.


However, it takes years before new knowledge generated within the scope of a single project or a portfolio of projects is valorised in the form of new products, processes and services and before they generate economic, social and environmental impacts. Currently, more than 50% of FP7 projects are still on-going. Therefore, the Commission will continue to report on FP7 results in the Horizon 2020 Annual Monitoring Reports. Moreover, in line with Article 32 of the Horizon 2020 Regulation, the Commission will report on the longer-term impact of previous Framework Programmes in the Interim Evaluation of Horizon 2020.


Key data for monitoring and evaluation are provided by Framework Programme beneficiaries through standard project reporting templates or surveys; they are also generated by project and scientific officers in Commission services and Agencies. The Commission is exploring the use of new technologies to mine and link existing data, to track individual researchers, company turnover and employment. Furthermore, the Commission is exploring the use of new evaluation methodologies to report on the wider socio-economic impacts of the Framework Programmes. New information technology methodologies should support monitoring and evaluation as well as the feedback of research results into policies, and externally for outside users. Data linking of available information with external (commercial and open access) databases should be introduced so as to minimise the reporting burden for participants, automatize processes and improve data quality and the reliability of the analyses. At the same time, to evaluate the long-term socio-economic impact, beneficiaries should continue to report on results and impact after the finalisation of the projects.


Finally, in order to support evaluations at national level, the Commission has also initiated a dialogue with Member States to foster the evaluation of the impact of EU Framework Programmes at national level.

In line with Recommendation (e), the Commission has established a monitoring and evaluation system under Horizon 2020 based on key performance indicators.

Furthermore, the Commission will:

• Ensure data quality and coherence to strengthen monitoring and evaluation systems, in line with the Better Regulation requirements;

• Explore the use of new text and data mining tools, evaluation methodologies and reporting obligations beyond the life time of the projects to improve and develop data sets that will allow monitoring and evaluating the longer-term socio-economic impact of EU Framework Programmes;

• Establish data links with external databases to complete and improve the quality of data sets;

• Support Member States in the national evaluation of the impact of EU Framework Programmes;

• Improve the support Horizon 2020 provides to the knowledge base for policy making on local, national and European level, including SME's competitiveness.


3. Outlook

FP7 has paved the way towards contributing to the achievement of the priorities of the Juncker Commission, by developing a common knowledge and technology base and innovative solutions in areas addressing pan-European challenges, for instance in the areas of environment, transport energy, digital single market, health, food safety and security. FP7 was a global programme with participants from 170 countries and funded projects in all areas covered by the Sustainable Development Goals. FP7 fostered growth and jobs and, in particular during the financial and economic crisis, helped maintain research and innovation activities at national level.


The Commission welcomes the HLEG recommendations. For the current and future Framework Programmes, the Commission is therefore committed to continuing to take action to ensure a focus on critical challenges and opportunities in research and innovation for Europe; to align research and innovation instruments and agendas in the EU; to foster the internal coherence and synergies of Framework Programmes; to involve citizens and stakeholders in an open and transparent manner; and to monitor and evaluate the Programmes' results and impact as a basis for future policy-making.


The Commission will report on the progress with the implementation of the HLEG recommendations in the Horizon 2020 Interim Evaluation, due by the end of 2017.


(1)

     Decision No 1982/2006/EC of the European Parliament and of the Council of 18 December 2006 concerning the Seventh Framework Programme of the European Community for research, technological development and demonstration activities (2007-2013).

(2)

     See http://ec.europa.eu/research/evaluations/index_en.cfm.

(3)

     CORDA data of 1 December 2015.

(4)

     HLEG Report p. 59-60.

(5)

     Panteia, May 2014.

(6)

     HLEG Report p. 6

(7)

     See Annex 15 of the annexed Staff Working Document.

(8)

Further information, including on FP7's shortcomings, can be found in the annexed Staff Working Document. Further details about the HLEG recommendations see http://ec.europa.eu/research/evaluations/index_en.cfm?pg=home . The recommendations on the FP7 Euratom programmes and the Commission's response are annexed to this Communication.

(9)

See Communication 2014/C 188/02.

(10)

  http://ec.europa.eu/regional_policy/sources/docgener/guides/synergy/synergies_en.pdf .

(11)

http://europa.eu/rapid/press-release_IP-15-5801_en.htm.

(12)

Commission Staff Working Document 'Better regulations for innovation-driven investment at EU level', SWD(2015)298final.

(13)

Study on 'Budgetary impact of the changes in the cost calculation regime in FP7 (EC and Euratom) as compared to FP6 (EC and Euratom) and its effects on the administrative burden for participants', 2015.

(14)

     See Article 38 of the grant agreement for projects funded under Horizon 2020.

(15)

http://ec.europa.eu/research/mariecurieactions/about-msca/actions/researcher-night/index_en.htm.

(16)

  http://ec.europa.eu/programmes/horizon2020/en/newsroom/551 . 

(17)

To date, 54% of all FP7 scientific publications are open access.