Self-assessment as a Tool for Research and Innovation Improvement

Alliance4Life has completed a big task of self-assessment within the Culture for Excellence work package of the Alliance4Life_ACTIONS project. The purpose of the self-assessment was to understand how external and internal factors shape the research performance of its member institutions located in WIDENING countries of Central and Eastern Europe (CEE) and to find out how good management together with research of high integrity, diversity and inclusion can improve their progress. The recently competed Self-assessment report consists of several parts including SWOT analysis of both internal and external conditions in the CEE region, survey of research culture based on the methodology of Wellcome survey published in 2019 and 2020, assessment of managerial practices showing the progress in implementation of recommendations and best practices developed within the Alliance4Life, and benchmarking study based on quantitative R&I indicators providing comparison between the baseline status (data from 2015-2017) reported during the first project of the Alliance4Life and the new benchmark results of the current A4L_ACTIONS project (data for 2018-2020).

This is the first comprehensive self-assessment bottom-up initiative of the research institutions of the CEE countries, which combines benchmarking with the elements of research culture and governance. The report brought a number of interesting observations that could be of a great benefit not only to the Allince4Life members, but also to all other institutions in the CEE countries, to the policy makers in the involved countries, as well as to the European Commission representatives. The participating institutes can use the finding for identification of strategic instruments and clear formulation of specific actions towards further institutional development. Other institutes can regard it as a source of inspiration and collection of good practice examples. Whereas national governments can unveil objective correlations between science policy and level of research excellence achievable in particular national environments. European Commission representatives can get an objective insight into realistically available opportunities as well as barriers for closing the gap between CEE and other EU countries. The findings could be even translated into strategic decisions positively affecting the future of the European Research Area.

The first step towards the complex assessment of the institutions participating in the A4L_ACTIONS project was their self-evaluation using SWOT analysis. The most valued strengths included the unique expertise, international collaboration, good management, formulated strategy, and academic freedom. The main internal weaknesses were suboptimal internationalization, low competitiveness for prestigious grants, weak cooperation with industrial sector, weak support services. Commonly perceived opportunities involved new research funding schemes, strategic partnerships, collaboration with industry, national recovery plans, and open labour market for talents of new generation. All A4L_ACTIONS partners identified unpredictable and insufficient funding of research, high brain drain, enormous bureaucracy in implementation of projects, as well as complicated and lengthy public procurement processes as major obstacles in closing the R&I gap in Europe.

The institutional managements of A4L_ACTIONS partners than responded to an online survey on managerial practices that covered topics related to science evaluation and benchmarking, research integrity and ethics, internationalization of human resources and mobility, grant preparation and implementation, research infrastructure management (core facilities), technology transfer and IP management, and science communication. The survey has identified several member institutes who showed very high-level managerial rules and processes that can serve as best practice examples to the other partners that are on the way to build and/or improve their managerial practices.

Anonymous survey of perception of internal research culture in A4L_ACTIONS partner institutions were accomplished with the aim to learn about the opinions of research and administrative staff on working conditions and quality of research environment. The questions were created on the basis of existing surveys, in particular the Wellcome survey on research culture from 2019. The A4L_ACTIONS survey revealed several key factors that need improvement: internal communication, performance evaluation, cooperation of leaders / superiors with students and team members, career plans, trainings of management and skills, administrative support for researchers, public procurement processes. Several findings are in line with the Welcome survey: pride in belonging to the scientific community, confidence in one's own abilities, importance of communication, presence of creativity in the work environment, dissatisfaction with performance evaluation, negative impact of metrics, importance of wellbeing in the workplace, insufficient funding as a research barrier. There are also clear differences in the perception of A4L_ACTIONS respondents in comparison with the Wellcome survey participants, in higher job security, higher confidence in management decisions, higher importance of obtaining projects, lower presence of unhealthy competition and less fear to inform about cases of violation of standards and research ethics.

The benchmarking exercise was used to evaluate quantitative parameters of the outputs/achievements in five domains: research excellence, knowledge transfer, funding and grants, human resources, and core facilities and/or special infrastructures. The results clearly show that the best performing partners in terms of research outputs work in the environment of relatively well functioning national systems, well implemented practices of institutional management and well-developed internal research culture. Despite objective barriers and threats, all A4L_ACTIONS partners can demonstrate very good research achievements that reflect their endeavours and deep interest for improvement. This is particularly visible when comparing the outputs of the period of 2018-2020 with the outputs of the preceding period of 2015-2017. However, it has to be considered that the year 2020 was affected by the COVID-19 pandemics, which interfered also with research activities.

The A4L_ACTIONS partner institutions will use this Self-assessment report as a background information for the peer review of performance-based evaluation by the International Scientific Advisory Board (ISAB). Based on the peer review, all partners will design research management strategies with action plans detailing how the strategic feedback will be implemented in to the institutional practice. During the entire process, the representatives of the A4L_ACTIONS institutions will have opportunities to consult potential challenges with the partners, learn from their experiences, and gain new proficiencies through participation in A4L trainings dedicated to particular managerial skills.

Thus, the A4L_ACTIONS project offers a unique opportunity for advancing the institutional research governance and thereby contributing to sustainable development of research excellence in CEE countries with the aim to close the gap Health R&I gap in Europe.