Using LOD to crowdsource Dutch WW2 underground newspapers on Wikipedia
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 16 | |
| Author | ||
| License | CC Attribution - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/47580 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
6
11
12
00:00
Lecture/Conference
02:23
Computer animationProgram flowchart
02:55
Computer animation
03:23
Program flowchart
04:13
Computer animationMeeting/Interview
04:47
Computer animation
05:14
Computer animation
05:55
Computer animation
06:29
Computer animation
07:06
Computer animation
09:12
Computer animation
09:51
Computer animation
10:18
Computer animation
11:37
Computer animation
12:09
Computer animationProgram flowchart
12:43
Computer animation
13:28
Computer animation
15:20
Lecture/Conference
16:04
Lecture/Conference
17:19
Lecture/Conference
Transcript: English(auto-generated)
00:01
The Library of Latvia, and I will be leading this first session. So our first speaker is Olaf Janssen from the National Library of the Netherlands, with a presentation on using LOD to crowdsource Dutch World War II underground newspapers on Wikipedia.
00:21
Well, thank you for the introduction. I'm not only working with the National Library of the Netherlands, I'm also a volunteer of Dutch Wikipedia, and for the next 20 minutes or so I'm going to talk about a project to make as many underground newspapers from the Second World War from the Netherlands
00:42
as possible, available on Wikipedia, using linked open data techniques and crowdsourcing. And I would like to give you a bit of background perspective on these underground newspapers. During the Second World War, the Dutch Resistance issued quite a number of illegal newspapers,
01:05
and they came in every shape and form. On the one hand, there were these really big newspapers, big titles, well organized titles, nearly professional, often with tens of hundreds or even hundreds of people working
01:21
on them, and these had also local and even regional editions. And on the other hand of the spectrum, there are these really small, amateur-made, pamphlet-like issues. For instance, there was this student who built an illegal radio, and he listened to the BBC in London, and he wrote down what he heard, and he handed that transcript to
01:43
his neighbor, and that neighbor handed it again to their neighbors, and in such a way the news was distributed. So from really big to really small newspapers, everything was collected, and right after the war, the Dutch Institute for War and Holocaust Studies in Amsterdam, the NEOT, started collecting those newspapers for posterity, and over the years they managed to collect
02:05
some 1300 individual titles. I'm talking about titles here, not issues, and in total there are some 200,000 pages of illegal newspapers, and it's about the archive that you see at the back of the slide there, that's the entire archive.
02:23
And of course when you start to collect stuff, you want to make sure that it is properly described in a library catalogue, and this has happened in the Netherlands, there is a common Dutch catalogue where this is available, and all those 1300 titles have been described, so this is obviously bibliographic metadata.
02:41
And throughout my talk, I will focus on this particular title as an example, De Geus Ondo Sudenten, which was a student's newspaper from the area of the Hague and Leiden, so you'll see this come back more often. Six years ago, all these 200,000 pages, they were digitized.
03:02
And now they are available in a website called Delfer, and Delfer, that's our Dutch national aggregator for historic full-text publications, so not only these illegal newspapers, but also other newspapers and books and magazines as well. And all these 1300 titles are available there, for instance like this one, again,
03:25
De Geus Ondo Sudenten, and obviously you can read the newspaper, but you can also search it on the word level, because there is also full-text OCR and also auto files for a word highlighting available.
03:41
But now if you want to ask contextual questions about these newspapers, like for instance, what sort of newspaper was this? Was this a really radical, illegal newspaper, or had it a more moderate tone of voice? What was the history of the newspaper? Who wrote it? Where was it printed?
04:00
How was it financed? How was it distributed? And were there any other relations with other illegal newspapers during the war in the Netherlands? And when you look at the kind of questions there, the what, the who, the where, the how, and the relations, no surprise, that's already, you're asking semantical questions.
04:25
But the bad news is that you can't answer these kind of questions on the level of Delfer, because that's a big drawback of Delfer. There is no contextual information available about these illegal newspapers. And when I was thinking of this project, I thought, well, where would most normal people
04:44
go if they want to read about historical text? For instance, these historical newspapers, where would they go? What would your bets be? If you want, say you're interested in a particular newspaper, where would you most
05:04
likely go when you want to read about it? Google, yeah? And where would you end up then? Yes, that's the correct answer. Yeah, I would have a problem if you would have answered it differently.
05:23
Because that's true, yeah. So for this particular title, the Heus und Studente, there is a decent Wikipedia article. And now we have a problem. Of course, you can't do a project when there are no problems. I was a bit nervous about using this particular slide here in Germany, but as you can imagine,
05:47
I'm not beaten away here. So I guess it's okay. And when you do a project, you have a problem to solve. And the problem is that if you want to discover these newspapers or study them or understand them or place them in a historical cultural context, then it's currently harder
06:05
than should be necessary. And that's because the information on these illegal newspapers is distributed across multiple and even unconnected resources. And to be precise, we have the descriptions, which are in the library catalog, metadata,
06:21
we have the content, this is the full text in Delfer, and we have the context in those Wikipedia articles. You see there at least, because there is not a little problem, because the article I showed you, that's actually a really carefully chosen exception. Because the reality is that there are only very few illegal newspapers that have their
06:43
own Wikipedia articles. In other words, if you look at the list of available articles on illegal Dutch newspapers, at the beginning of the project, some two years ago, this was the full list. These are about 15 to 18 titles, while it could be 1,300.
07:01
So there was still a lot of work to do here. We can tackle both problems, and for that, I started a project, and I started a project on Wikipedia, that's why it's called WikiProject, and the aim of the project is to systematically and uniformly describe and also interlink all these 1,300 illegal newspapers from the
07:26
Second World War. And why did I choose Wikipedia? Of course, I could have chosen to do a non-project environment, but I thought, well, if you want to reach a big audience, then you should do your project directly on Wikipedia, because
07:42
over 80% of the Dutch population uses Wikipedia. And if you would do it in your own project environment, you would have to do a lot of extra effort to get the audience to your own environment. And the second consideration was, because the open nature of Wikipedia, you make sure that all the data you put in the project, you put in Wikipedia, can also be taken
08:04
out again for other reuses, like think about Wikidata, DBpedia, or you can even base data visualizations on it. So the big challenge of the project was to get hold of contextual information.
08:21
We have all the metadata, we have the full-text content, that's already available. The lack was for contextual information. And I was really lucky, because I discovered there was a book printed in the Netherlands in the 80s of the last century, the Underground Press, 1940-1945, and this book has contextual
08:46
entries about nearly all of those 1300 newspapers. And when you look at an individual entry within the book, and again I chose the article for De Geus Ondestudente, every newspaper in the book has a unique ID, in this case
09:05
it's number 199. And also in the red box there, you see it says De Hegg there. This is the place of publication. So you can make relations between newspapers and place names. And obviously, no surprise for this audience, that starts to smell a bit like semantics
09:23
already. Then there is this blob of text, and this is ideal raw material for your Wikipedia article. And when you look at it in a bit more detail, over there you see two persons mentioned. Those were brothers, Mr. J. and H. Drion, and they were law students at the time.
09:46
So you can make a relation between titles and person names. And when you look closely there, in the red box, you see three red numbers, 106, 360, and 748. And those are IDs of other students' newspapers with which the number 199 collaborated.
10:07
So there were collaborations between that newspaper and the other three. And again, you can start to have another building block of your semantic network. So we needed to free up and open up the information that was in this book.
10:25
And the first step we took was we actually scanned the book. We ran it through Acrobat so that we got full text OCR. And then we contacted the copyright holder of the book, which is the Dutch Institute for War and Holocaust Studies in Amsterdam.
10:40
And we asked them, well, could you please release the book under a Creative Commons license? And they very happily collaborated with us. So now this book is available online, just as a flat file PDF. And it's correctly licensed so that we can actually do further work with it.
11:02
And then there were three other problems that we needed to tackle. First of all, we needed to convert this PDF file into a structured database. And we needed to link the titles in the PDF to place names, person names, and other titles. Next, we needed to make links between those titles and the same titles in our library catalog
11:24
and in Delfer, so about the metadata and the full text. And the last step was to create links from the titles to external sources other than the library catalog and Delfer. Well, those three challenges, this is where the co-author of this presentation, Gerhard,
11:42
comes into play. Gerhard couldn't be here today. And that faces me with a problem because Gerhard did all the really hardcore technical work. And yesterday, I only had my first introduction into linked open data. So I'm not going to make an effort to explain on the technical level what he did,
12:02
because I think you're all much more bigger linked open data gurus than I am. So I'm just going to show you what he did. He used these kinds of tools and techniques and services. And then you all think, ah, that's what he did. And if you really want to know, he gave me his slides, which look like this.
12:22
And so you can come to me after the talk, and you can look at the slides, and I hope you can make your own conclusions about the work he did. And those slides will also be, of course, available in the presentation that will be, I hope, available after this conference.
12:41
But the main outcome of his work, that those three red crosses changes into three red checks, of green checks. So we made, that's what it comes down to, we made a linked open database, a virtual triple store, containing information about these underground newspapers.
13:01
And actually, that was the first time that it was done in the Netherlands, which I found quite surprising, because, well, it's more than 70 years after the war, and there was still no proper database about underground newspapers. And now we did two, we could do two things with the database.
13:22
First of all, we could generate Wikipedia articles from the database. That's what we already did. And the second thing, but it is much more future work, is that we could also, for instance, export it to Wikidata, to DBpedia, or we could use it as a basis for data visualization.
13:42
So let me focus a bit further on the first thing. In my project goals, I highlighted the words uniformly. So I wanted to make sure that all the Wikipedia articles that we generated from the database would have the same layout, that they will all have an info box,
14:00
and all the source references would be on the same spot. So all the articles would look more or less the same. And when you talk about uniformity, you always have to think about templating. So we made a template, and using the database, we generated 1,300 Wikipedia articles.
14:21
We didn't yet generate full articles, but we generated article stubs. And just to show you what these stubs look like, this is a full Wikipedia article, again, on the same newspaper, and on the yellow box at the top, you see the URL this page is available under.
14:42
And now I'm going to cross out everything that is not automatically generated. So all the stuff you see there, which is not grayed out, that comes from the database and the template. In other words, if I invert this slide,
15:01
the text in the red box, that was the only bit that was added by humans, to make the stub into a full article. And this is a process we're currently doing. This started about two months ago, and we are involved with Dutch Wikipedia community to expand those stubs into full articles.
15:22
Currently, we have a group of about 8 to 10 volunteers, who with, I must admit, various intensities work on the project. But it's going on, gradually. It's not like 100 articles a month, but bit by bit, the list of available Wikipedia articles is expanded.
15:43
This was the list I showed you. This was before the project. And this is a list about three weeks ago that I last checked it. So you see, gradually, the number of articles that's available is growing. And of course, we do this to make the Dutch population happy.
16:03
And I hope that there will be other volunteers from other Wikipedia chapters that will translate the Dutch articles into English, for instance, or in German, so that even more people can know the history of the Dutch illegal newspapers.
16:20
And this man won't be happy. Thank you. Thank you. And now for the questions. When you're asking questions, please also get a microphone.
16:41
These are working. OK, so.
17:08
No, I would lie if I would say that would be true. The majority of the newspapers is notable.
17:20
But there are quite many, especially those really small newspapers, of which little contextual information is known. And if you would create an article, that wouldn't meet the notability threshold. So what we're doing for those really small article titles is to make an overview page, like the article on small illegal newspapers.
17:43
And it has just an overview of all these smaller individual titles. All right. Next questions. Mark? How did communication with the Wikipedia community work in this project?
18:03
Particularly, I know that sometimes I speak sorrow about putting lots of data which was generated into Wikipedia. How did you communicate with the community to make this possible and accepted by the community?
18:24
We involved the community right from the start. We explained our project approach and our project goals right from the start of the project. And we asked input from the Dutch community. Then when we got our input, we generated a small number of test stops.
18:45
And we asked the community, well, what do you think of them? Do they meet your standards? And we got feedback from that. And once we got an OK from the community, we generated all the Wikipedia stops. But we generated them not in the main namespace of Wikipedia, but in a separate corner in the project namespace.
19:03
And that's where they are at the moment. So every time a stop gets expanded, it gets copied over to the main namespace of Wikipedia. If you wouldn't do that, you would indeed probably get a lot of problems with the community. Yes.
19:21
Next question. The biographical data you attempted to link to, because you extracted it from the publication, did you succeed in linking it to library data?
19:40
Not yet. That's a stage of the project we still have to do. Or have there been Wikipedia articles already about, for instance, these two brothers? Yeah, you see that as a spin-off. Not only articles about those newspapers are written, but also new articles about persons who worked on the newspapers have been created.
20:04
Actually, some volunteers only write articles about the persons working on the newspapers. You see that happening, yeah. All right. And time for one more question.
20:20
I have a question about the reference of the persons behind the newspapers. Some of them paid very dearly for their activities underground. I'm having a hard time understanding. I have a question about the persons, about the names you are linking to. You are linking and referencing the names in your project about the persons that edited those underground newspapers, right?
20:46
And are you also referencing or linking to the information to their ultimate fights? Because actually, in the Dutch resistance, many were executed in concentration camps at Gestapo headquarters.
21:03
Others survived the war. Are you also retrieving external data about the ultimate fights of the persons that you reference? Yeah, we made, we used the links that are available from DBpedia
21:23
to get hold of that kind of information. There are links from the database to DBpedia, and from there we extracted the kind of information you asked for. Does that answer your question?
21:40
Okay, that's okay. You'll have a chance to follow that up after this. Just to conclude, is the data that you created just linked open data? Is it available for us to play with? Yes, of course, yes. It's open data, so it's available. Okay, thank you.