Open science and research assessment reform – A happy marriage?
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Title of Series | ||
Number of Parts | 41 | |
Author | ||
License | CC Attribution 3.0 Germany: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
Identifiers | 10.5446/60361 (DOI) | |
Publisher | ||
Release Date | ||
Language | ||
Producer | ||
Production Year | 2022 | |
Production Place | Kyiv, Ukraine |
Content Metadata
Subject Area | |
Genre |
39
40
41
00:00
Computer animation
04:05
Computer animation
07:47
Computer animation
10:38
Computer animation
Transcript: English(auto-generated)
00:02
Good afternoon, everyone. So I just would like to do a quick talk on open science and research assessment reform and also a question that I would like to address. Is this a happy marriage? So many of you are probably familiar with some of the ongoing and I believe quite important
00:23
developments around research assessment and reform and research assessment in Europe, in particular in Europe, but also elsewhere. So this is an example of a recent development, the launch of the Coalition on Advancing Research Assessment with an agreement that is being signed
00:41
by an increasingly large number of institutions, especially in Europe. And this agreement contains all kinds of commitments, commitments to responsible research assessment practices. I think this is really a key development. I saw there are also signatories from Ukraine, which of course is great. I would like to say something about how I think these developments
01:02
relate to developments in the area of open science and how the two can actually strengthen each other if we organize things properly. So it's quite interesting from my point of view to also see what the UNESCO recommendation on open science says about this. So the UNESCO
01:20
recommendation is a very interesting, I believe, document with lots of very relevant perspectives on open science, but in connection to research assessment. There are two quotes from the document that I would like to share with you. So on the one hand, the UNESCO open science recommendation says that reviewing research assessment and career evaluation systems
01:47
is needed in order to align them with principles of open science. So here the idea is that practicing open science is something that takes time and resources, effort, all of that, and researchers expect to be recognized for doing these things, for practicing open science,
02:07
given the time it takes. And it's important that research assessment systems indeed do give recognition to these types of activities. So this is a perspective that we see a lot in policy discussions about open science. So often people point out the need to give recognition
02:26
to open science practices and the need for assessment systems to take it into account. At the bottom of this slide you see a different perspective, which I think is equally important. So this is another quote from the open science recommendation. So it says that
02:43
encouraging responsible research and research evaluation and assessment practices requires or is needed to incentivize quality science and to recognize the diversity of research outputs, activities, and missions. So what we see a lot in statements about
03:05
promoting responsible assessment practices, we see a strong emphasis on the need to recognize not just articles in scientific journals, preferably in high impact journals. That's the traditional way of thinking. Instead, what is being emphasized is the
03:21
need to recognize a broad range, a diversity of research outputs, and not only outputs, but also all kinds of other activities, things that researchers do, things that matter, things that have value, and that researchers should be recognized for. So this is also mentioned in UNESCO open science recommendation.
03:42
So we see actually there are two ways in which research assessment is connected to open science. On the one hand, it is about the need to make sure that open science practices are being recognized. At the same time, it's also about the need in general in research assessment to take a broader perspective and to also take into account all kinds of other outputs and activities
04:03
that researchers perform. And this is where I think we see potentially a happy marriage between the two, between research assessment, reforming research assessment on the one hand, and open science on the other hand. But I do want to emphasize that a happy marriage requires a two-way street. And so if it just goes in one direction,
04:24
then it doesn't work. And sometimes I'm a little bit afraid that we think too much about this from a one-directional point of view. So what I see a lot is this idea. I see a lot the idea that research assessment needs to incentivize and to reward open science practices. And we have discussions about indicators of open science that could indeed
04:45
enable us to give recognition and reward to researchers that practice open science. And I'm kind of in agreement with this. This is indeed important and needed. But I also think that this is just one part of the full picture.
05:01
So this is just one direction. And we also need to think about the other direction from open science to research assessment. And this is at least as important, but sometimes it is not getting enough attention. So if we are serious about reforming research assessment, and if we, for instance, in line
05:20
with this European agreement, if we feel that research assessment should not just be about publications in high impact journals, but research assessment should take into account a much broader range of outputs and activities, then we need to also fundamentally rethink the types of information that should be made available to evaluators.
05:46
So the types of information that are taken into account in research assessment in order to give recognition to a diversity of activities and outputs. And this, I think, can only be done if we provide information about research outputs and
06:02
research activities in a way that is itself aligned with open science, with an open science philosophy. So the information that we provide to perform research assessment should itself be open, but the information should also be organized in such a way that broad range of activities and outputs can be taken into account,
06:27
and the information should be possible to tailor the information, to customize the information in such a way that it aligns with the particular missions of research students or researchers that are being assessed. So one size fits all solutions don't really
06:45
work, don't really make sense. And this requires an open approach, an open approach in which the information itself is made openly available, is transparent, but also an open approach in the sense of enabling open participation in who is able to have a say in assessment processes,
07:01
in making information available that informs research assessment. And that is what I call open indicators of science. So that's not the perspective that you see at the top of this slide, indicators of open science. It's kind of the complementary perspective of open indicators of science, open in the sense of transparent, but also in the sense of who is able to
07:25
contribute and participate in developing, designing these types of indicators. So we need that two-way street to indeed realize a happy marriage between research assessment and open science. And if you have just a one-directional setting, then I'm not sure if we are going to be successful in realizing all the promises around reform of research
07:45
assessment and around open science. So in the UNESCO recommendation on open science, you also see these types of ideas. It's a little bit hidden in the recommendation, but they also make a plea, for instance, for open infrastructures. They mention open bibliometrics, open scientometrics, and that's very much aligned with the things I just
08:03
mentioned. And many of these developments, which of course are also being discussed today in this conference, they are kind of, from my point of view, they're part of this thing that we need to realize. We need to realize open research information, research information that is produced and collected according to open science principles,
08:23
and that can be used to inform new responsible approaches to research assessment. So this, I think, is a key thing, and this is what we need to accomplish in order to have that happy marriage between the two, between open science and responsible research assessment. There's one other thing I want to emphasize. I learned quite a lot about your
08:45
country in recent years, Ukraine, and also all the expertise that's available in your country. So I just want to emphasize that the reading that I have done about Ukraine, and also by researchers from your country, I found it really very informative and helpful for me to learn
09:08
more about kind of the current state of the debate in your country. Just want to mention that I feel there's really a lot of expertise available in your country, and I think it's important to make use of that in addition to what international experts are able to offer.
09:25
So to conclude, it's essential that open science and reforming research assessment go together, so that's really the key thing. If you are trying to do one without the other, then it's going to fail, most likely, but this happy marriage is a two-way street. It is not only about
09:43
incentivizing open science, so redesigning assessment is such a way that researchers get rewarded for open science practices. Yes, that's important, but it's just one part of the full picture. At the same time, we need to make sure that research assessment is taken into account
10:03
a broader perspective on all the activities and outputs that matter, and this requires open research information, so research information itself is compiled in an open science way, with the information being openly, transparently available, but also with open participation in
10:21
the process of compiling that information. So the need for open research information is really crucial, and as I mentioned, don't forget the experts that you have available in your