Preregistration - How to bring transparency into animal research

Video in TIB AV-Portal: Preregistration - How to bring transparency into animal research

Formal Metadata

Preregistration - How to bring transparency into animal research
Title of Series
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
The Open Science Conference 2021 is the 8th international conference of the Leibniz Research Alliance Open Science. The annual conference is dedicated to the Open Science movement and provides a unique forum for researchers, librarians, practitioners, infrastructure providers, policy makers, and other important stakeholders to discuss the latest and future developments in Open Science. #osc2021
and uh i'd like to now introduce our second distinguished
speaker we have dr celine heinel with the federal institute for risk assessment that is the bfr in germany and berlin hello celine can you hear me there hello david yes i can hear you fine i just wanted to give a brief introduction the bfr for those that don't know has the task of providing scientific advice to the german government on issues relating to food safety product safety chemical safety contaminants in the food chain animal protection and consumer health protection so it really is helping us keep healthy and making sure that what we eat and consume is safe yeah today celine will be talking about the idea of pre-registration how to bring transparency into animal research when it comes to animal experiments several studies indicate that a large part of animal experiments are never published this non-publication not only leads to a distorted view of the state of research but also strongly contradicts ethical principles this is the reason they propose a pre-registration system to improve transparency and animal welfare at the same time so good morgan celine if i'm not mistaken you are in the beautiful city of berlin our capital correct yes correct i'm in berlin now cooper i ask all of our speakers maybe just i know berlin has probably dozens and dozens of highlights one highlight for you for the city of berlin multiple i wouldn't say a special area but perhaps just the atmosphere in a lot of different areas i really like and i enjoy i have to laugh because in washington dc we call that a very diplomatic answer very good okay excellent then wonderful celine at the digital stage is yours and again around 20 minutes and afterwards we'll have a few minutes for q a please go ahead thank you david for the introduction and
uh thank you also for the organizers for inviting me and uh presenting our platform the animal study registry to an open science community which we uh yeah are not used to usually we present in front of a laboratory animal scientists and so i'm very happy to show you today why we think we need pre-registration and animal research and how this can on one side improve research quality but also improve animal welfare at the same time because david you said in the introduction i'm working at the bfr but the um so we are working on risk assessment but here's also the place for the center of a german center for the protection of laboratory animals and that is actually our goal and that's why we created this this pre-registry and so i would like to really start at the very beginning so the ethical challenge and animal experiments are still a very controversial topic in our society and already children small children really refuse the instinctive legal concept of animal testing however at the same time we still need animal experiments to advance medicine and also to guarantee safety for consumer and for consumers and um so there is a legal framework allowing um allowing animal experiments under certain condition conditions and just under the assumption that this will bring knowledge gain to our society however now here comes the problem that as was said in the introduction a large part of animal research is probably never published and now what are we talking about what is the quantity what is the number of animal experiments that are not published this is actually quite difficult to find out but there were several attempts which i would like to show you to kind of find out what is the part of non-publication and there was one study trying in 2012 to assess the publication rate by a survey for among animal researchers in the netherlands and they ask what do they believe that overall is the part of animal experiments that are published and the median response was 50 which i find actually quite shocking and when they were asked what was the percentage they believe uh their experiments uh are published they said 80 i think that's quite typical the others are always doing worse but in general both are actually quite high numbers so 20 percent of non-publication is already i find a significant part and in the last year there were two more studies which the with the both used a similar approach to find a bit more objectively what is the quantity of non-publication of animal research and they followed up animal study protocols this means that every scientist who wants to perform animal experiments need to write an animal study protocol with a little bit of background and how many animals they want to use and why do they need to perform these experiments and then the competent authority can permit this study protocol and this they try to follow up and look at the any at the study protocols how many of these protocols led to a publication in the end and what they found out is on two german university medical centers that only 67 of these protocols led to a publication and if you would not take into account doctoral thesis this number would even drop to 58 percent and another study oh sorry use the wrong remote another study from the netherlands did in a similar way now in utrecht they also followed up study protocols and looked at more than six years after application how many led to a publication and they found a similar number that only 60 of these animal study protocols actually led to at least one publication and they even counted abstract abstracts and they also looked at not only just the publication for one whole protocol but they really looked at individual animal numbers and there they found out that only 26 of animals which were written in the study protocol were reported in the end in the publication i think it's quite hard still to really put a number on the problem but i think all agreed even in the survey everybody said there is a large part which is not published so we're really dealing with a problem and now of course come the important question what is the reason for this and i think it's probably a reason you all know also from different fields of research the main reason which was also said in the survey is the publication bias so there's a strong incentive uh for for positive finding it's easier to publish and that negative findings are simply often not recorded uh reported and the second reason were all reasons um around technical issues in the experiments and i think both reasons actually uh things which should be reported to the broad scientific community to really make medical progress and to learn out of these experiments because the non-publication will of course have all these an ethical concern i mean all these experiments are just permitted under the assumption that they will bring gain of knowledge and this is simply not happening if they are not reported to a broad scientific community and this can also lead then again to unnecessary duplication of experiments in different labs and all of this leads of course also to waste of public money because most research is still publicly funded and um to an incisive and could also explain an insufficient translation of biomedical research into clinical research later there's a lot of failures when it comes to effectiveness of drugs this might also be due of course to this publication bias and leaving experiments out so now uh enough to the problem but really we need to find a solution and that is why it was proposed in beginning of 2010 that similar to clinical research it could might help to tackle or one one possibility to tackle this problem would be animal study registries so similar to what is already done in clinical research to ask scientists to before they start any experiment fix their study plan and register it in the public registry and then afterwards it allows the comparison between what was actually in the original plan and what is published later or whether there is any publication later so this would help of course to encourage the publication of all gained without and then then not only diminish the size of the publication bias but also just tackle all the ethical issues i was talking in the beginning and it could help fight the p hacking so this is still quite common practice in biomedical research to try to keep the p-value low to a significant level by using the flexibility of analysis so but now in a pre-registration if you would fix and advance the the sample size or and exclusion criteria the variables and the statistical tests it becomes nearly impossible to perform p hacking then it would help
to clearly distinguish which parts of the study are hypothesis generation and which ones are hypothesis testing so there wouldn't be any harking so hypothesizing after results are known and um would help to later really assess what what is the validity of the outcome and it promotes at an early time point now the planning of the study and supports the scientists in planning their study and take into account all the important points of a well-designed study and in the end it would promote research transparency and also help to regain trust in research would i think which i think is especially nowadays very important as we can see and um why do we believe it's true it would work is because we know from clinical trial registries that exist already since uh 20 years in 2000 early 2000 the fda and the nih launched the clinical trial registry which is still one of the biggest registries um for clinical trials and um [Music] it became now also mandatory for most of clinical trials to register their study in the public registry and there's already some studies showing that this led to an increase in the reporting of null results so this might also happen then to animal studies so here is a study from 2015 where they looked at all it was they looked at studies evaluating treatment treatments for a cardiovascular diseases and they put some different categories whether the result was in the end harmful null or beneficial so the treatment they tested and you can easily see that before 2000 you have the majority of studies reported a beneficial effect for the treatment they tested and after the year 2000 for these studies to receive some funding it became mandatory to register in the clinical trial registry and you can easily see that there's a clear drop in beneficial effects and you have a lot of null results so i guess this might also happen if registration pre-registration would happen in animal registries and all of this brought us to the uh idea that we should have an animal study registry and we launched it in 2019 so here the bfr it is for all studies around the world which involve any kind of animals and if you're curious just have a look you can browse through the page and through our registries without and without creating any account this is just if you want to register a study and so if you register a study we asked for some details about your study especially for the study design also your method and the statistical analysis and also some details about the animals used and the housing conditions and if you are using any refinement measures and i would like to quickly run you through our registration process so if you developed any idea on what you want to investigate you can enter your study in our form and then the study is submitted and after submission you still have two weeks to retract your study or to add any changes it's only after two weeks that the study gets automatically registered and when it's registered it receives a doi so digital object identifier and you cannot change anything anymore but it becomes also citable from this time point on and this doesn't mean that it's immediately public you have the option or for up to five years to to choose an embargo which means that your study will only be visible with a title um the institution where it's performed and the short abstract and optionally the name optionally the name of the author but we of course know that you fix a steady plan but a lot of changes can happen and for this we give the opportunity to opportunity to add comments where you can account for any protocol changes or even if you need to stop the study this is also an information which would be important in which you could add in the comments and in the end these comments will also help you to link any data sets or publication resulting from this registration and usually i said in the very beginning that we present a lot to animal research scientists and we get a lot of positive feedback but still also a lot of concerns and one of the main concerns i would say is that most animal science is still exp is still exploratory basic signs there's where most animals are used and that scientists say it's to it's not flexible and that they cannot plan their study to the same extent and we would say you can really you don't need to to register a whole study like it would be later in a publication but you can really also register small study parts and change it and and just build up on it and register small parts which you can just you can always just copy your study and change the parts which need to be changed um and sorry for the delay also of course there's always a question of scientific freedom but i think by registering small parts and by really using also our command function where you can explain any any changes i think it's still very uh you're still very free in your research and of course all of this also takes time so this is uh i think just a practical reason why some scientists are concerned but i think if you really pre-register in the beginning of your study you will save time in the end because now also most journals already enrolled some reporting guidelines the arrive guideline and most of the things which you need to fill out later to to write a manuscript already asked now in our pre-registry and so you will save time later and of course there are always concerns about intellectual property but i think with our five-year embargo we're actually on the good side and also um i think it's more you you can even help with the doi to really show that you're that your um your study is it was uh the idea came already very early you have a time stamp for for the generation of your idea so of course we i see there were there there are a lot of concerns still from the animal scientist community so we really need to work on creating incentives so we discuss with funding agencies with universities and with journalists to really try to to advance pre-registration animal research that's our main work at the moment and um so now just to come to an end um i think it would be if you're interested just have a look at our our registry we also wrote some publications on it so if you want to know more about details and um so we are federal institute is perhaps also important so we are independent of sponsoring which makes author our registry quite sustainable and with this i'm really happy to take your questions and if you have further questions also of course in the meet the speaker session and thank you for your attention
and thank you for your wonderful presentation celine i just wanted to make a quick announcement we had several questions regarding your slides there's a lot of interest in the slides so i just wanted to remind everyone all the slides they're available on xenado that's again all slides are available on xenodone so thank you for the questions on that yeah we have a few minutes time for some q a so i'll jump right into it um celine the first question we have is how do you want to motivate the researchers to pre-register their work is there any way of positive motivation or must be done mainly by funder rules i think i mean we try to just convince them by talking really going to the scientists and giving presentations and really it's quite a new topic in most labs so um sometimes it helps i think especially among young scientists to just try some convincing um work but of course we also want try to create really incentive so via the funding agencies as was said then also by talking to journals which sometimes also put it already in guidelines to really also support pre-registration and also by universities i mean there were already universities who just gave up some money to for for pre-registration of studies but i think of course it's nice to count on free will but incentives are of course important yeah absolutely absolutely our next question is the aim that only pre-registered studies can be published how to prevent stealing of ideas or researchers registering a lot of ideas so that others can't work on them yeah i mean i think published to make it mandatory to publish a study we're still it's still quite a long way i mean people first need to know that they can pre-register study so at the beginning especially for journals would just be to really acknowledge that people made the effort to to try this open science uh way and um to why we yeah of course you can always try to to i would say cheat by um by just registering plenty of ideas but after five years people would see that you just registered a lot of ideas and if there's never a publication perhaps the community will ask some questions so i think this would be kind of a self-regulation within the scientific community i would hope but that's a good question we also thought about it very good great our next question uh is from marcus concall and he's asking uh why not going for registered reports peer-reviewed pre-registration this would help to tackle the fear of being scooped and publication bias or do journals in your field not provide this option there's registered report the option of registered report i think it's a very good option and very important also it's still not very common in our field but it starts being there i think both has has advantages the problem is registered fields i think what we have perhaps is more flexibility you can smaller studies really come back and add some changes and um you don't need to to already in the beginning first to fix the journal where you want to uh to to publish and you can uh yeah you have more flexibility by registering several things and deciding later what you want to put in one uh publication and the second thing especially for scooping i think um i mean registered report still as a problem you you're registering uh you you perhaps i think there's still some concerns uh might be that you you send your idea to uh other experts from the field and it's not clear that your your your registered report is already accepted so this might also be something where uh we have uh yeah an advantage because it's uh basically no one will see it uh the first five years or just the things you want to say so in the abstract which you can still keep quite vague so i think but of course uh registered reports are great especially to uh also work on publication bias yes yes yeah i got you uh we have time for two more quick questions i'd ask you if you can keep the answers brief because it's really fantastic here besides the pre-registration of studies should it also be mandatory to publish the data for all animal studies also for negative results without any published paper yeah would be great of course but uh yeah we are far yeah we are fast yeah still it's not mandatory but i think yeah this is what we want because of course making a publication is always writing in publication is always also some work so now i think that also in our pre-registry just ask for links for data sets or something just to leave a trace for the scientific community it doesn't need to always be just a publication yeah okay so and the final question here um how do you communicate these improvements to the public and then we have a hashtag science hashtag science communication that's also an excellent question i think um it's still pre-registration is still a bit complicated but we also try i mean especially for our where center which works also with yeah just the protection of laboratory animals and we do a lot of communication on just animal experiments and to show that it's still a very difficult position to take on and what can be done to to ensure animal welfare pre-registration we also did some press releases or something but it's still quite a complex topic but of course yeah it's always good to talk to the public absolutely absolutely yeah with that our final question we do have our speakers session coming up will you be available for that and so do you have a time already that you know yes i'm looking forward to it it's at 12 15. so in 15 minutes okay 15 minutes central european time