The GreyLit Report: Understanding the Challenges of Finding Grey Literature
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 2 | |
| Author | ||
| License | CC Attribution 3.0 Germany: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/59657 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Computer animation
00:32
Computer animation
Transcript: English(auto-generated)
00:05
Hi, we were invited today to discuss our 2016 research on how users find grey literature. I'm Danielle Aloia and I currently work as the Assistant Director of Information Management at New York Medical College Library, and I'm here with my colleague.
00:22
Hi, I am Robin Naughton. I am the Assistant Professor, Web and Digital Services Librarian at Queens College Library, CUNY. So today, this is a little bit about what we're going to talk about. We'll give a brief introduction to our grey literature report, our research.
00:41
We did a literature review of systematic reviews. We did in-person, well, not in-person, live data collection from people. And then we'll do our results analysis and conclusion and some recommendations. So about the report, the report was produced in 1999 and it captured timely grey literature.
01:07
And it was originally started as a subscription just as an email service to other librarians looking for the same information. We culled, you know, over 300 producers of grey literature from websites from across the country.
01:23
You know, worldwide, we have international coverage and there's just over 20,000 resources in it. Unfortunately, it was discontinued in 2016 for lack of funding. So it's no longer updated, but it's still available and accessible.
01:40
And most of the reports are archived so that they're accessible, even if the site where it originally was from is no longer there. So a little about our purpose. We conducted this research in order to understand how researchers and librarians were searching for grey literature. When we presented this six years ago, there was a real push to include grey literature in the systematic review process.
02:06
So we wanted to understand how researchers were searching for this elusive publications. What did that look like? How were they documenting it? And so our three main research questions were, you know, what are the challenges of finding grey literature?
02:24
You know, how do researchers look for it? And, you know, does the grey literature report help people? You know, since it was such a great product at that time. So for our methods, you know, we we searched PubMed in the traditional way,
02:45
just looking for papers about systematic review if they included grey lit in that search. So we got over 2000 results and we limited that to 1200. Then we just took a random sample of 400. We wanted to get to a manageable size.
03:01
You know, so in the end, we got about 297 papers that talked and analyzed about the types of sources that they use for grey literature searching. And we picked 57 of those articles to analyze some, you know, some of
03:21
the exclusion criteria, which is that they just didn't even include any of those. You know, and some papers only indicated they just searched PubMed and Web of Science to get grey literature. So hand searching was the most popular way that people recorded searching for grey literature, which is going through peer reviewed articles and looking at the citations and seeing where they cited different grey literature sources.
03:46
And of course, Google is the next most well, I mean, websites which include like WHO, you know, think tanks, government institutions and then Google, of course, just doing a broad search on Google. Those are the three main ways that people searched.
04:03
So in those 57 articles, we found that the best one of the best articles about how to include grey literature searching is that you have to, you know, include in your search results online databases, including your search strategies, online databases, web search engines, websites, repositories, your local catalog, experts and hand searching.
04:28
So those are the main ways that those can be included in your research. And time is a factor as well, right? One article really mentioned that it takes about an hour and a half average to
04:41
look for grey literature on one site, if you're on WHO or just even in Google. And searching the title field in Google really helps it produces the best results. If you put keywords in that title, it really brings up it unpublished reports. And the bulk of that, unfortunately, is found on the 35th page.
05:04
So in our recommendations, we think that we believe that the Brisco article really laid out recommendations on how to report your grey literature findings in a systematic review. And that means to include the database name, the URL you got from the dates
05:26
you searched, the search terms you used and, you know, a sampling of the results. If they're too large, you don't want to put too much in there, but a sampling of your results. And then Odin really talks about, you know, developing a plan like lay this all out before you go to do your searching.
05:43
It means have it paper ready with those field names and then record it as you go along. And next we'll talk, Robin, we'll talk about data collection. Hi. So in terms of data collection, we did interviews and there were 12 interviews total with a total of 14 participants.
06:06
Two of the interviews were group interviews because the participants invited colleagues to join the interview in the discussion about grey literature. The interviewees came from a variety of professions, including librarians, information professionals and researchers.
06:22
And they also represented different places across the globe, including the UK, Australia, the US and Canada. And in terms of the data collection, we collected data using the semi-structured interview protocol. It was an hour interview and it was conducted as a remote interview on the phone or via Skype.
06:45
It was audio was recorded and then transcribed. And we used thematic analysis to analyze the interview to get to the themes in the data. So one question we asked participants enough about their search strategies.
07:06
What did they do to search? How did they find grey literature? And the majority of participants found grey literature using Google. That is the number one thing they used. They went out searching Google to find grey literature. But they were also participants who mentioned using the grey literature report,
07:24
as well as government publications, websites and systematic reviews to find grey literature. So there certainly was a varying in terms of their search strategies and tools that they used to find grey literature.
07:40
In doing analysis of the data, the interviews, what we identified were these topical searches of interest. Multiple responses from participants. So a participant can say as many things as they wanted to say regarding how they did their topical searches. In terms of the participants' fields, some of the topics were very specific to that participants' field
08:05
and focused on what they were researching and the ways they researched for that particular field. Some of the big themes we found in the data had to do with the clinical medicine or health care or policy or public health.
08:21
So we got over 75 responses to this question. And a lot of the themes really centered on these areas of searching and getting information in terms of the interest in what participants want to do. We asked participants what is their experience with the grey literature report.
08:43
Half of the interview participants used grey literature report as a current awareness tool. So a lot of participants said that if something is new and out, they'd go to the grey literature report to find out the newest version of what's going on and where they can find the grey literature that they are looking for.
09:04
When asked if they'd recommend it, about 50% of the participants said yes, they would recommend the grey literature report to others searching for grey literature. And overall, 42% of five participants said they had a good experience with the grey literature report and finding information there.
09:28
In other resources index, the idea of other resources in this, we asked participants what else can the grey literature report do and how else can it be of help in their search for grey literature and what they're trying to do.
09:41
And the participants mentioned it would be good for the report to include conference proceedings, data sets, webinars. And these are the three that were mentioned at least by eight participants in the study regarding what additional resources would be valuable in a grey literature report.
10:03
Overall, in terms of this study, what we learned is that there are a variety of methods to find grey literature. Researchers use the tools that are available to them. And in some cases, that tool is Google. But in some cases, that tool is the grey literature report, which is the fifth most cited place that people go to find grey literature.
10:25
In terms of searching, the number one tool that researchers and librarians and information professionals use is Google to find grey literature. And then the second most tool they use is by contacting experts or doing hand searching on papers to find additional places to get grey literature.
10:48
The grey literature report is seen as an alert or current awareness resource, and it is highly recommended by librarians. And the last piece of this is that the thing that came out of this study that was very clear is that we do not publish enough on the search method.
11:10
So a recommendation is definitely for when doing systematic reviews, particularly in terms of finding grey literature, is that we publish the search methods.
11:20
So that can help researchers understand the method that is used and how to use it. It can help with replication, and it can really help to promote the process of finding grey literature and the tools and resources available to someone looking for grey literature in the field.
11:43
Thank you very much. Daniel, do you want to say anything about the current state of affairs? Just, you know, it's still six years later an issue with systematic review reporting. And it's just, I think, part of the massiveness, the massive undertaking of doing a systematic review in general anyway, right?
12:06
And to add this other layer to it, it's just going to take more time for people to get on board. They're not putting it in regular systematic reviews, their search strategies, so that's about it. All right.
12:21
And if you have any questions, you can contact us if you want to learn a little deeper about what our methods and data look like. Thank you. Thank you very much.