And not just for scientists, but also for those who are using gray literature, and not only in scholastic areas. The project started out with four names, the first four, Dominic and Jerry, with regard to the, and that's the responsibility for this project,

the survey, Stefania and Carlo, who's not here, with regard to the acquisition and the RGL, the collection that was in our new collection of multidisciplinary gray literature in the Gray Guide repository.

And then a host of other names, two of whom are here today, Antonella De Robio and Anna Szarek. Where's Anna? There's Anna. I don't want to get ahead, but what happened was we did the survey, got the results,

made a data paper, entered both the data and the data sets in the Don's data archive, and it was time to do the analysis. And I got scared because it was not enough time, I thought. So I said, the 29 of the 56 respondents to the survey,

I said, well, they gave their names and their email addresses, that's their own fault, and I contacted them and I said, would you all, the data's in the data archive, the data, the paper's in the data, would you be willing to do an analysis of the data? And five responded.

And you'll hear three of those analyses later in my presentation. The project goal was twofold, very simple, to learn the opinions, uses, and applications of persistent identifiers within the gray literature community. And the second goal was to explore the use of persistent identifiers,

namely the DOI, in the acquisition of gray literature. The method of approach, simple also, I think. It was a survey or questionnaire. It would be used by online stakeholders, and those stakeholders were within the gray literature community. You can say that the total number of our distribution lists,

combined distribution lists, is about 950 active, double that if you're looking at the non-active. So we used, with the active population of the gray literature community of practice, and there are a number of communities of practice in gray literature.

So GrayNet is one example of a community of practice in gray literature. And the second step was to initiate a campaign among GrayNet's diverse stakeholders by using the DOI as the system-generated citation. It was an incentive, throwing out something, and hopefully that we would see the deposit of documents flowing into the GrayNet repository.

Yammer, too bad. Okay, survey question. The questionnaire, about the questionnaire. The questionnaire was first carried out in the GLP collection.

This is the gray literature proceedings collection, and that was the conference papers, the individual papers. And what I wanted to do is, before we formulated the questionnaire, is to find out, over the years, how persistent identifiers, DOI's,

were referred to in previous conference papers. So we used our own source in order to develop the questions for this survey, this questionnaire, ten of which were drafted, nine of which were standardized, that is, in terms of the possibility of response. One was open-ended, that was the end,

and all of the questions did have a comment field, so that it would provide us more information. The questionnaire was entered in SurveyMonkey, and a link was generated. Now, the survey population, I had mentioned there were 950 active members, but what we wanted to do was we wanted to define the population.

We just didn't want to have an open stakeholder survey. And by doing so, what we did was we said we will limit it in time, and we wanted to be able to make sure that we were sending it to a real person.

So we had, the time was a little bit more than five years, from January 1, 2014, to April, that was actually when the Access Great project began. And only personal names were used, and we needed a surname, as well as either a first name or an initial.

And that was then the selection for the population. I'm not going to go through the profile, how many men and how many women, you'll read that in the conference paper. Now, the survey questionnaire, excuse me, I was going the wrong direction. In terms of the results, I'm not going to go through all of the results of the survey.

What was of me of interest, of my interest, was there were a number of categories, strongly agree, agree, disagree, uncertain. And I said, I want to know for myself the highest number,

or the highest percentage of response per category. So, for example, overall, the survey was very positive, it was upbeat, so that was good. But we wanted more out of the, more results. So 59 percent responded to the question, persistent event identifiers are vital in linking and cross-linking data.

So that was actually the highest percentage for the category, strongly agree. When we went to uncertain, the highest percentage who were uncertain to any of the questions, of the nine questions, were persistent identifiers serve as an incentive in the acquisition of grey literature.

Now, this was interesting because one of the goals was to determine the acquisition, the incentive for the acquisition of grey literature. The one in terms of disagree was 11 percent, and you say that's not a large percent, but actually that's the highest percent within the category disagree,

and that was over a quality indicator that increases the value of grey literature. And when we go to the analyses that I will be giving you from these three co-authors, that this is interesting because it's very important to know

if the DOI in itself is a quality indicator. In terms of yes and no responses, highest was one of, one does have a DOI, one or more DOIs. Now, of course, GrayNet is attributing DOIs to a number of the authors here, so that was okay. And in terms of no, and that was, Plato would be interested in this,

that only 23 percent said that they did not have an ORSID. So that means that we have to work on that, but we're doing okay. If I'm just giving an example in terms of who are the contributors to this access grey project,

I'm looking at the content providers, those are the grey literature, the grey net stakeholders. I'm looking at the DOI service that GrayNet played in this particular project, the role of DOI minting service. And the system provider that is in PISA, ISTE, that they,

these were the major players in this particular project. If I'm looking at the project workflow, I'm not going to go through all of the steps, there was a particular workflow, but with any workflow, it doesn't really follow exactly. Sometimes it jumps around.

But if you can get all of the particular points, then you're scoring well. So this was for us more of a guideline in terms of where we wanted to go with this project. Now, the acquisition groundwork, this was actually the second part of the project,

the goal being to populate a repository. We had a template for RGL, the resources in grey literature,

but of course we wanted to bring it up to speed with regard to our project. So that meant that it had to include a metadata field, there had to be a system generated citation for each of the entries, the existing records, that was not a lot, but they had to first be assigned a DOI, minted,

and then there was a collection, that was the original collection from five years ago, which wasn't getting much traffic or wasn't being populated,

we said we'll move that, because RGL, the concept was being, or is, that is a multidisciplinary collection, and therefore it was easier for us to make that determination. So those records also were assigned a DOI and merged to the RGL collection.

And then, of course, all of the records were merged, all of the records now 56 have been assigned DOIs. The guidelines were simple, as simple as possible, and we wanted to communicate that also to the prospective contributors,

self-archiving, OWL, metadata in English, no problem, other languages were acceptable in terms of a descriptive field, that there would be an accompanying full text, which was required, and then all of the records, one had to know that all of the records

that they did enter in self-archiving were open access compliant via this license, CC-BY-SA license. Now, the strategy, it's an ongoing strategy, and it was using the community of practice, what we had in terms of our context and resources,

in terms, to communicate that and to see how it would be the response. So we were using the distribution list for GrayNet. We were using the WHOIs in gray literature, that is the bio collection in the gray guide. I think that's something like close to 150 records, biographical records,

that there was the geoconference authors and co-authors, so we went back five years to see, okay, you all are authors, you're producing conference papers, but we know you're doing more.

You're doing newsletters. You're doing technical reports. You're doing websites. So these are also types of gray literature, and these are also certainly very welcome into the gray guide, and multidisciplinary and interdisciplinary. We used also GrayNet's social media, namely the Facebook and the LinkedIn,

and GrayNet has also a bit dated international directory of organizations, and so we have a contact person by each of them. We have not yet completed following the strategy line here and doing all of this,

and that was because the startup of the acquisition part phase of the project took longer than we had anticipated. Now, that was the phases and goals, and now I want to look at an analysis of the survey results,

and then I'll look at the analysis of the acquisition results. And now I turn to my co-authors, Anna Tonella and Anna, but I will begin with June Crowe, who is in Tennessee.

June is a GrayNet award recipient years ago. June is also an honorary member. She's retired. She's an honorary member of GrayNet, and she was last year at the conference in New Orleans, and June said this about the data and about the data paper that was in the repository. Persistent identifiers, such as DOIs, are making research more efficient.

Additionally, as the existing protocols become more widely adopted, there will be even more improved access to information. Persistent identifiers are not only useful for identifying data, but can also be used to store relationships and point to where other data may be stored.

They were developed to prevent link rot and to ensure that objects remain available and unchanged. In this way, they improve access to information and increase trust in scholarship and research. It is assumed that the DOI would be only one factor

to be considered as adding quality to research publication. Other factors, such as peer review, citations, impact factor, what other researchers say about the paper, author, date, etc., would also need to be considered in order to indicate the quality of research.

The assignation of DOIs to metadata records would not necessarily attract more content providers. However, the open access policies of the repository would definitely play a role in attracting content providers.

The grey literature community, and here is a pat on the back for all of us, of respondents to the survey appear to be on the forefront of knowledge about the importance of PIDs and DOIs. Research in the field of grey literature and its related data will become increasingly accessible as the research information infrastructure becomes more standardized and widely adapted.

Wow! Thanks, June. Now we go on to Anna. Anna is here because I had an incentive, if those who respond to the survey,

they will go into a lottery and they can win a pass to the conference, and she won. So here is Anna's take on the survey analysis. Researchers often search for references that are listed at the end of relevant articles. Over the years, the URL links to some sources such as grey literature

often are only accessible online, and they may not work anymore. If there is a persistent identifier, the access to grey literature remains stable and allows continuous access over time. The process of selecting resources or sources that are scientific and relevant to the work of researchers

is becoming more important than the ability to find and collect countless sources. In academic circles, when a mentor or professor checks the references listed and sees the DOIs next to the sources, even if they are grey literature, these actually count.

There is a possibility that persistent identifiers such as DOIs, which are recognized worldwide, would encourage researchers to cite their local sources more often. And also she implies language, other than the English language. If this is the case, then in some fields such as education,

where work and training are led and informed by government guidelines and evaluations, all of which are grey literature, then the results of this survey are encouraging. The DOI by itself does not inform us about the quality of the document or data.

It does, however, in cases like the dissertation and thesis, increase the possibility of connecting the research data with the thesis, which can be seen as a quality indicator in itself since the content becomes more scientifically provable when the data are available. The creation of the DOI depends on the policy of the repositories

and is not directly connected with the quality of the individual record. If Slovenian repositories added the DOI... Why did you say this?

If the Slovenian repositories added the DOI identifier to each thesis, then all would have a DOI, not only the best ones. However, this is not yet the case. Where we stand now is that the less experienced researchers

may find it difficult to recognize a trustworthy piece of grey literature. In such cases, an international, well-known identifier such as a DOI could assist them. And last but not least, we hear from Antonella.

DOI is not only a persistent but also actionable because one can plug it into a web browser and be taken to the identified source. In this way, persistent identifiers are strategic to research data outputs

because they can be reused for new research. The persistent characteristic is a guarantee even if the location of the data file may change when an academic changes institution or when data archive systems become replaced. Examples are not uncommon for grey literature.

Concerning the question whether persistent identifiers serve as an incentive to the acquisition of grey literature, a near 27% of the respondents were uncertain. The comment, probably right, by one of the respondents may be interpreted as a selling point to those who were not certain,

given the fact that over 30% of the respondents strongly agreed. When asked if a repository or data archive that assigns DOI's to metadata records is more likely to attract content providers, one comment was eloquently formulated.

In practice, this is the case, but the mere fact of assigning DOI's should not replace the other, more intrinsic reasons for content providers to choose a certain repository. This question is in need of further insights to better understand

and decide future choices for repositories and digital platforms. This holds particularly in the open access environment where grey literature could be a strong pilot light. The word choice is interesting.

When asked in the final survey question to provide contact details along with other comments or recommendations, it is only after being asked to analyze the results of the survey do I come to recommend an online course such as a MOC,

Massive Open Online Course, that would deal with the meaning and functions of persistent identifiers, their structure, environments, uses and different types. As but one of the 50 plus respondents in the survey, we are all assumed to be interested and somewhat experienced.

Imagine all the others, librarians, documentalists who are working with grey literature, training in persistent identifiers such as DOI's and ARCADES, would no doubt prove worthwhile. And I end on this note.

Thank you very much and thank my co-authors.