Draft:Periodic Evaluation of Research, Development, Artistic and Other Creative Activities

• Comment: Please add more sources. Vanderwaalforces (talk) 15:04, 3 November 2023 (UTC)

---

The Periodic Evaluation of Research, Development, Artistic and Other Creative Activities (abbreviated as Periodic Evaluation), or Verification of Excellence in Research 2022 (abbreviated as VER 2022), was an evaluation of the publishing and other creative activities of Slovak scientific and research institutions (public universities and public research institutions) for the period 2014-2019. The subjects of the evaluation were the creative outputs produced by Slovak public universities and public research institutions. The Periodic Evaluation was administrated by the Ministry of Education, Research, Development and Youth of the Slovak Republic, through its Department of Methodology and Evaluation of Creative Activities.

The evaluation process

At the beginning of the evaluation process, each public university and public research institution, or their parts, submitted applications for evaluation in one or more of the 28 research areas (falling under 7 main groups of research areas). Each application contained a maximum of 25 outputs (scientific articles, monographs, conference papers, artistic outputs, etc.) in a specific research area. A maximum of 5 outputs were submitted per each scientist. The final selection of outputs per each application for Periodic Evaluation was set on the basis of an attempt to respect the principles of representativeness and evenness.

Thanks to the Recovery and Resilience Plan of the Slovak Republic, evaluation committees, composed primarily of foreign experts from 19 countries and 66 universities (e.g. the University of Cambridge, Princeton University, McGill University, University of Melbourne, Technische Universität München, ETH Zurich, Université Paris-Saclay, Sapienza Università di Roma, etc.), were set up in 2022 to evaluate the level of Slovak science from the perspective of the international scientific community.

For each of the research areas, an international assessment committee (the so-called Research Area Committee) was selected by the Council for the Periodic Evaluation of Science, Research and Higher Education Institutions to evaluate the applications in the given scientific area. At the same time, the members of the panels for each of the groups of research areas (called Research Area Panels) were also selected by the Council for the Periodic Evaluation. These panels oversaw the work of the research area sub-panels, the evaluation process itself, and the consistent application of qualitative criteria in the evaluation of applications submitted for each research area.

In the period between September and December 2022, each of the project evaluation committees not only assessed the quality of the outputs themselves, but also produced a methodology and specific criteria for assessing the quality of research, taking into account the specificities of scientific research in the particular research area.

Area of evaluation

The evaluation area is one of the 28 research areas for which applications for Periodic Evaluation could be submitted.

Main groups of research areas	Research areas
Natural Sciences	Physical Sciences Chemical Sciences Mathematics Biological Sciences Earth Sciences Environmental and Ecological Sciences
Technical Sciences	Civil Engineering and Architecture Metallurgy and Materials Science Mechanical Engineering Electrical Engineering Computer and Information Sciences
Medicine	Basic Medicine Clinical Medicine Health Sciences
Agricultural, Forestry and Veterinary Sciences	Agricultural, Forestry and Veterinary Sciences
Social Sciences	Political Science Media and Communications Sociology and Social Anthropology Psychology and Cognitive Sciences Education Social Work Economics and Business Law
Humanities	Philosophy and Theology Languages and Literature History and Archaeology
Arts and Theory of Art	Performing Arts and Related Art Fields Visual Arts and Related Art Fields

Assessment results

The results of the Periodic Evaluation, or the work of the evaluation committees, are the so-called institutional quality profiles.  The quality profile provides information on the level of research, development, artistic or other creative activity within each specific application submitted for the Periodic Evaluation by a public university or public research institution, or a part thereof. The quality profile is an assessment of the research level of the institution as a whole, containing for each application a percentage of the number of representations of outputs within each of the five quality levels.

The five quality levels into which the assessment panels placed outputs were:

1.      World leading quality

2.      Internationally excellent quality

3.      Internationally recognized quality

4.      Nationally recognised quality

5.      Below the standard of nationally recognized work

The importance of evaluation

The VER 2022 was the first evaluation in the history of independent Slovakia that was based on the assessment of the real quality of creative outputs, without taking into account quantitative measurable parameters of science and research.

The results of the Periodic Evaluation will be reflected in the funding of science and research in the Slovak Republic, starting in 2024.

Inspiration

The inspiration for the Periodic Evaluation of Research, Development, Artistic and Other Creative Activities was drawn from proven foreign systems of science evaluation, in particular the British Research Excellence Framework (REF for short).

The future of science assessment

In the framework of the VER 2022 evaluation, only the quality of scientific outputs was assessed. However, in the future (in the planned evaluation of the VER 2027), the Periodic Evaluation will also include an assessment of the impact of research, development, artistic and other creative activities outside of academia, as well as an assessment of the environment of research, development, artistic and other creative activities. In the context of the planned assessment of the impact of creative activities in the VER 2027 evaluation, the Ministry of Education, Research, Development and Youth has prepared a guide for applicants and a selection of case studies (taken from the UK REF evaluation) illustrating how to declare the impact of creative activities.

Periodic evaluation in the media

According to the State Secretary of the Ministry of Education, Michal Fedák, the primary objective of the Periodic Evaluation is to increase the quality of scientific production and to promote the competitiveness of scientific institutions. As for the results of the evaluation itself, they can also be informative, for example, for future students or researchers when choosing a university or research institute.

As the published results of the quality profiles will be taken into account in the allocation of the subsidy of public universities from 2024 onwards, the results of the Periodic Evaluation were presented at the VER2022 conference with the participation of the then Minister of Education, Ján Horecký, at which a debate on the subject was held with representatives of the applicants (institutions).

The Periodic Evaluation was mostly positively perceived by the evaluated institutions (public universities or public research institutions). Representatives of Comenius University in Bratislava informed that this university produces high-quality research in the social and natural sciences, while the Vice-Rector of Comenius University in Bratislava, Jozef Masarik, stated that, despite several reservations, the Periodic Evaluation is one of the most objective evaluations that have been carried out in Slovakia so far. Ján Híveš, Vice-Rector of the Slovak University of Technology in Bratislava, also reported on the competitiveness of the Slovak University of Technology in Bratislava in the context of its scientific performance and appreciated the fact that the evaluation was carried out independently by international evaluation panels.

Jozef Masarik (Vice-Rector of Comenius University in Bratislava) views the results of the Periodic Evaluation positively, especially in the context of the level of science funding in Slovakia. According to him, the evaluation showed that 4 percent of scientific institutions achieve world leading quality, 18 percent achieve internationally excellent quality, 33 percent achieve internationally recognized quality and 29 percent achieve nationally recognized quality. Despite the overall low level of state funding, Masarik was surprised that we have a sufficient number of world-class outputs. According to Masarik, the percentage of outputs of world leading and internationally excellent quality is lower compared to foreign countries (such as Great Britain, the United States of America, or Germany), but if these percentages were recalculated with respect to the level of science funding, the Slovak scientific ecosystem would be able to compete with foreign countries.

According to Jozef Masarik, the problem of the development of Slovak science and research is the level of resources that the state puts into their funding, as well as the instability of this funding. He suggests that the problem is also the outdated scientific infrastructure, as well as the quality of human potential in some research fields. Development is also hampered by the fact that it is difficult to attract many Slovak experts who have studied abroad back to the Slovak research ecosystem. Masarik argues that the fact that, especially financially, the prospects of young graduates in some fields of science (e.g. medicine or computer science) are not sufficiently attractive compared to their prospects in applied practice, can also be considered a limitation for the development of science. According to Masarik, the conviction of political leaders that Slovak universities, and Slovak science as such, are not of sufficient quality may also be problematic for the development of science in Slovakia. It is the Periodic Evaluation by independent, foreign experts that, however, proves that this is not the case and that quality science is produced in the environment of Slovak scientific institutions.

The lesson from the Periodic Evaluation, as Jozef Masarik states, is that although the Slovak research ecosystem has world-class results, often these results do not have an impact on the global scientific community because of the way Slovak scientists publish - especially due to the trend of publishing in domestic scientific journals. In this respect, the unpreparedness (or inflexibility) of Slovak grant schemes in providing support for scientists to publish under Open Access may also be a limitation.

According to Jozef Masarik, individuals and teams who have cooperation with foreign partners or who have foreign grants that allow them to improve the financial situation at their home workplaces have succeeded in the Periodic Evaluation. Similarly, he states, teams dedicated to local studies that are of global significance for knowledge in certain areas of research (e.g., in history and archaeology) have also succeeded.