Preprint motivations and the effect on scholarly impact: an analysis of bioRxiv authors
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Title of Series | ||
Number of Parts | 3 | |
Author | ||
Contributors | ||
License | CC Attribution 3.0 Germany: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
Identifiers | 10.5446/50506 (DOI) | |
Publisher | ||
Release Date | ||
Language | ||
Producer | ||
Production Year | 2020 |
Content Metadata
Subject Area | ||
Genre | ||
Keywords |
00:00
Meeting/InterviewComputer animation
00:15
Computer animation
Transcript: English(auto-generated)
00:00
To introduce Niklas Fraser, he's a postdoc at ZBW, and of course, core worker in the Waze project. He got a PhD in Geosciences also at University of Kiel. And yeah, he was two years at Frontiers working in the industry, also has very
00:23
internal knowledge of the open access publishing practices. Yeah, and we are very happy that we got you for the Waze project, and you are currently working on a couple of things. I think one preprint I have to mention because it's a very brand new one and a very interesting
00:42
one. Nik has published a preprint on preprinting a pandemic, so the role of preprints in COVID-19 literature. So this is a brand new paper placed, I think, on bioRxiv. I'm not 100% sure, but I think that's it there.
01:03
I recommend it highly. I'm very happy to have you here, Nik, and yeah, the floor is yours. We are hearing a presentation from a recent bioRxiv author study, which you have undertaken.
01:21
Yeah, please. Yeah. Thank you, Philip, for the nice introduction, and thank you, Stephen, for the very nice talk just now. I have the difficult task of following that up, but I'm going to give a talk today about some work myself and my collaborators have been doing, which is looking at preprint
01:43
motivations and the effect on scholarly impact, and this is based on an analysis of bioRxiv authors. So, to give a little bit of background here, last year, we, in our project, we published a paper in which we looked at differences in citation rates and in indicators of online
02:07
sharing of journal articles that have been deposited as preprints or not deposited as preprints, and for this study, we focused on the preprint server bioRxiv, and that is what is shown in this upper plot here, so we compared the citation rates of preprints
02:23
that were deposited as, sorry, journal articles that were deposited as preprints versus those in a control group published in the same journals at the same time that were not deposited as preprints, so on the y-axis is the CPP, it's a log-transformed measure of citations, but essentially in the positive direction is more citations, and we tracked
02:43
citations for 36 months after publication, and we found that those articles that had been published or deposited on a preprint server actually, from the very point of publication, began to accrue citations more quickly, and after about 36 months, this equated to about 50% more citations for those articles that had been deposited to
03:04
bioRxiv versus those that had been not, and this effect was also visible in indicators of online sharing, so for example, we found that those articles were tweeted 40% more than those articles that were not deposited as preprints. The problem, or maybe not the
03:20
problem, but one of the questions that we couldn't answer in this study was, is this really a causal effect? Is it that you put your article on a preprint server, more people can access it, or they can access it earlier, and that makes it more cited? There may also be things like an author self-selection effect, so it might be that authors who are, I don't know, aware that by preprints in the paper they can get more people to read
03:46
their paper, that they do that preferentially for their most exciting or highest quality on most novel papers, so we wanted to look at these questions, specifically that last question really about this author self-selection effect, and we did this through a mixed method survey
04:02
of bioRxiv authors, so a quantitative and qualitative survey. This was our survey design, and I know that's way too small for you to read, but it was just to show that we had a survey that consisted of three parts. I will enlarge each part so you can see exactly what we did, so in the first part we collected demographic information related to these
04:23
survey participants, so we asked them about their countries of residence, their scientific disciplines, gender, career status, and institution types. In the next section we asked them specifically about articles that they had published in journals in the position of a corresponding author, and for those articles that they had
04:42
published in journals we asked them what proportion of those articles they had also deposited as preprints, and depending on those answers we gave them conditional questions about what were motivations for depositing articles as preprints, what were the differences between articles they posted and didn't post as preprints for the authors that said that they only
05:02
published a proportion of their articles as preprints, and what were the reasons that they did not want to post certain articles as preprints. Then in the third section of our survey we did exactly the same again but when we focused on co-authorships, so with the idea that a co-author on a publication probably takes a more passive role in the publication strategy
05:25
of an article than the corresponding author. So we contacted 25,000 approximately bio-archive authors, we had a response rate of around six percent so we had around 1,444 respondents to
05:43
our survey which was great, and on this slide you can see the demographic information that we collected from our survey respondents, so we looked at the disciplines and these broadly correlate to the submission categories on bio-archive, we asked about the countries that
06:02
our participants come from, so you can see that we had an over-representation of authors from the US, also large amounts from the UK, Germany, France, and these are probably expected as these make up, or these are commonly the authors that make up a large proportion of bio-archive
06:22
submissions. One interesting thing is that we had relatively few respondents to our survey from China, which was surprising because China actually makes up also a large proportion of bio-archive authors. In terms of gender we had an over-representation of male participants,
06:41
that's not surprising again because in our previous study we looked at the gender of authorships of bio-archive preprints and found that they were over-represented by male authors, most of our participants were from universities and non-university research institutions, and we had a range of career statistics from professors, post-doctoral researchers,
07:00
associate and assistant professors, PhD students, etc. Okay so one of the first parts or blocks of our survey was then to ask about motivations, so why did authors deposit certain articles as preprints, and we asked a lot of questions in this section so I'm not going to show
07:22
everything that we asked but just some of the important results that came out of that, and I also want to mention that for each of these survey blocks we also collected qualitative results, so we asked authors to add context to their answers in free text form, and I won't present that today because we're still working on that coding and content analysis,
07:41
but we have some additional data to supplement to this as well. So we asked, we gave our survey participant statements such as preprints were deposited to increase awareness of my or our research and asked them to respond to that on a five-point scale, and the two strongest motivators that we found for
08:02
depositing preprints were that researchers did it to increase awareness of their research or to deposit or share their findings more quickly. We also with this, with our data, we ran a regression analysis and I again didn't want to put too much, show too many details
08:22
of that, but just to highlight a few results we divided our participants into groups on whether they were either early or late career researchers, if they were male or female researchers, or if they were U.S. or non-U.S. based researchers, and the rationale for that is that we saw that U.S. authors are over-represented amongst our sample and BioArchive itself is a U.S. based
08:43
platform, and what we found out of that is that early career researchers were more likely to deposit preprints to increase awareness of their research, but less likely to deposit preprints to stake a priority claim or to receive more feedback on their work.
09:00
U.S. researchers were more likely to deposit preprints to increase awareness of their work or to benefit the scientific enterprise or to share their findings more quickly. So we also asked reasons why authors did not deposit their preprints. Following the same structure we gave statements and asked them to reply on a five-point scale, and you can see, I won't cover each of
09:26
these points, but you can see that generally the options we gave them, most of those options were disagreed with. The exception that perhaps tended slightly towards agreement was this top option here, and that was that authors were simply unaware of the options to deposit
09:41
preprints at that time. So that seems to be a big factor, is actually having awareness of preprints in whether an author decides to submit a preprint or not, of course. From our regression analysis we found that female researchers and early career researchers and non-U.S. researchers were even more likely to be unaware of this option to deposit preprints
10:03
at that time. Okay and then we came to the additional block of our survey which was to ask authors about differences between articles they deposited and didn't deposit as preprints, and this is important trying to understand, as I mentioned at the start, this author selection
10:22
effect where authors maybe select to their best preprints, best articles to deposit as preprints or their most novel articles, and so we gave them these statements such as the articles that were deposited as preprints were of a higher scientific quality than those that were not deposited as preprints, and also that the articles contain more novel results or had a
10:47
greater societal value or significance or were published in journals with higher impact factors, and for those statements we saw a mixed response but tending towards disagreement. So most authors
11:01
were reporting that they didn't see any big differences between articles that they did and they didn't deposit as preprints. What was interesting is that when we asked them as well about their expectations for their articles, even though they didn't report that there were very many differences between articles, most authors reported that they still expected the articles
11:21
deposited as preprints to get more citations than those that were not deposited, and they expected the articles to be disseminated more widely online on various social media platforms than those that were not deposited. From our regression analysis we also found that male researchers in particular were more likely to report that articles deposited as preprints had
11:42
a greater societal value or significance than those that were not. Okay so I'm running out of time so I'll just summarize very quickly. We found that authors who deposit preprints do it most often to increase the awareness of their work and the speed of dissemination, and the most common reason for not depositing is simply a lack of awareness. Authors generally still
12:05
expect that depositing preprints will increase their citation or online dissemination metrics, but we find weak support that authors specifically choose articles to deposit based on their quality or novelty or societal value. So the next steps in this study, as I mentioned earlier, we
12:22
have qualitative analysis still to do of around 1,500 pre-text responses, and we're working on that at the moment, and that's going to add some more context to these results, and then hopefully in the not too distant future we will be publishing this, and of course we will be publishing that first in a preprint. Okay thank you. Thank you Nick. Now we got just one
12:46
question, a technical one. How did you manage to write so many researchers, or to reach so many researchers? Did you do it via script or email or manually?
13:01
We did this via a, we had a script which we used to pass the email addresses from bioRxiv public web pages, so every preprint has a web page, and in that they have the details of the authors, including the corresponding author email address, and we had a script to
13:23
grab those email addresses, and then we had an additional automated method of writing to those researchers. Okay, perhaps one quick question from my side. Did you see some peer pressure
13:41
in the answers or in the open questions, so that there is the postdoc or the the senior of the group doing or providing pressure to the people to preprint things, or perhaps also the funder? Could you see something like that in the answers? Well we have, I mean, I showed in
14:06
I think one of the earlier slides that, I'm not sure where it is now, but funding mandates and open access mandates didn't seem to play a big role in that. People responded generally that they took the decision to pre-print a manuscript themselves, and actually
14:24
I do have that in an additional slide, maybe I can go back a little bit, because we also asked this question, so we didn't just ask about these motivations, I didn't have enough, I said I didn't have enough time to mention everything, but we also asked about who takes the decisions to deposit a manuscript, so we asked, for example, if it was necessary to deposit
14:46
articles to comply with an institutional open access or preprint policy, most people disagreed with that, and the same with the funding agencies open access or pre-print policy. Usually people took the decision themselves, and most of the corresponding authors
15:04
said that they took that decision themselves, and for co-authors they were perhaps pressured by co-authors into that. Okay, thank you.
Recommendations
Series of 10 media