Crowdsourcing the Reconstruction of Lost Heritage


Formal Metadata

Crowdsourcing the Reconstruction of Lost Heritage
Title of Series
Part Number
Number of Parts
Coughenour, Chance
Vincent, Matthew
CC Attribution - ShareAlike 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
The destruction of cultural heritage by human or natural causes isn’t new, but the way we can help is. Although the original artifacts and structures can never be replaced, their cultural memory can be preserved thanks to uniting web development, image-based reconstructions, and crowdsourcing. Building on its first version as Project Mosul, associated with the demolition of archaeological sites and museums in northern Iraq, Rekrei pushes this development and vision internationally to regions in Asia, Africa, Europe, and the Americas.
Hypermedia Computer animation Lecture/Conference Local Group
Area Musical ensemble Addition Direction (geometry) Multiplication sign Forcing (mathematics) Feedback Projective plane Computer program Mereology Computer animation Personal digital assistant Universe (mathematics) Interpreter (computing) Whiteboard Quicksort Subtraction Self-organization
Context awareness Building Code Model theory Scientific modelling Bell and Howell Basis (linear algebra) Web 2.0 Medical imaging Bit rate Core dump Pattern language Pattern recognition Mapping Moment (mathematics) Drop (liquid) Bit Staff (military) Digital signal Instance (computer science) Computer Open set Web application Digital photography Website Self-organization Pattern language Quicksort Resultant Mapping Open source Computer-generated imagery Computer Open set Computing platform World Wide Web Consortium Self-organization Electronic data processing Computer Projective plane Content (media) Cartesian coordinate system Local Group Frame problem Summation Uniform resource locator Computer animation Software Logic
INTEGRAL Model theory Scientific modelling System administrator Multiplication sign 1 (number) Bell and Howell Port scanner Mereology Demoscene Storage area network Web 2.0 Data model Medical imaging Computer configuration Internet forum Negative number Area Metropolitan area network Process (computing) Mapping Content management system Structural load Drop (liquid) Flickr Mereology Port scanner 10 (number) Category of being Digital photography Network topology Natural number Self-organization Website Quicksort Freeware Data type Task (computing) Point (geometry) Slide rule Freeware Computer-generated imagery Open set Revision control Crash (computing) Internetworking Computing platform Self-organization Home page Matching (graph theory) Content (media) Cartesian coordinate system Local Group Uniform resource locator Computer animation Fingerprint Local ring
Frame problem State of matter Multiplication sign Computer-generated imagery Visual system User-generated content Point cloud Interface (computing) Weight Discrete element method Computer Perspective (visual) Image resolution Medical imaging Virtual reality Term (mathematics) Hypermedia Dependent and independent variables Videoconferencing Square number Hydraulic jump Task (computing) Data recovery Machine vision Projective plane State of matter Mass Virtualization Group action 8 (number) Uniform resource locator Computer animation Website Quicksort
Android (robot) Scientific modelling Direction (geometry) 1 (number) Bookmark (World Wide Web) Formal language Web 2.0 Word Medical imaging Hypermedia Data acquisition Square number Logic gate Position operator Personal identification number Boss Corporation Spacetime Interface (computing) Virtualization Digital signal Degree (graph theory) Natural number IRIS-T output Website Right angle Digitizing Resultant Computer programming Open source Computer-generated imagery Average Event horizon Number Revision control Lecture/Conference Computing platform Dot product Focus (optics) Scaling (geometry) Debugger Projective plane Content (media) Line (geometry) Cartesian coordinate system Sign (mathematics) Word Hypermedia Computer network Object (grammar)
Point (geometry) Collaborationism Scaling (geometry) Mapping Length Scientific modelling Structural load Geometry Projective plane Virtualization Line (geometry) Cartesian coordinate system Revision control Web 2.0 Medical imaging Graph coloring Personal digital assistant Database Phase transition Videoconferencing Website Data conversion Units of measurement
Data mining Spacetime Social software Lecture/Conference Hypermedia Projective plane Constructor (object-oriented programming) output Demoscene
Meeting/Interview Electronic mailing list 1 (number) Electronic visual display Physicalism Bit Demoscene Number
Pairwise comparison Meeting/Interview Constructor (object-oriented programming)
Word Graph coloring Lecture/Conference Stress (mechanics) 1 (number) Letterpress printing Set (mathematics) Subtraction 2 (number)
Focus (optics) Spacetime Lecture/Conference Meeting/Interview Projective plane Virtualization Event horizon Position operator
Collaborationism Cellular automaton Projective plane Independence (probability theory) Water vapor Mass Distance Sphere Perspective (visual) Revision control Lecture/Conference Forest Quicksort Position operator
Projective plane Website
Arithmetic mean Strategy game Lecture/Conference Meeting/Interview State of matter Revision control
Point (geometry) Propagator Lecture/Conference Internetworking Backup Local Group
Digital photography Building Video game Meeting/Interview Hypermedia Scientific modelling Multiplication sign Website
Medical imaging Dataflow Digital photography Meeting/Interview Natural number Projective plane Content (media) Website
Goodness of fit Lecture/Conference Analytic continuation Mereology
the theft group them group and this is the users users what
are they and everybody can hear me I've been told talk loud but generally not a problem for me and if you need me actually quiet
down I can do that as well if you need me to speak slower I can do that as well but when I get really excited about something I can really get moving so and we're going to talk about a project that chance and I have been working on for just over a year now and something that we're really excited to be able to presented today usually when chance and I are presenting this at a conference it's a bunch of archaeologists and it's a kind of a different different atmosphere people are thinking about some different questions and different thoughts so we're really excited to be in front of for the 1st time a totally new audience and and get some feedback in some thoughts and maybe even help us think about new directions at a project like this can go in and the 2 of us are the founders of the project
I am a I will summary greed fellows we are researching in different institutions I'm at the University of more the air and uh that's in the south of Spain nearly granted because no 1 ever know from what the user that but it's a great place and chances of the universities do got in the Institute for Photogrammetry but we also have some people that we're really just want to thank publicly because they have been key to making this project I work for us so funny or MMD no you may have heard of him before he worked on the the virtual restorations of the bombing and Buddhas in Afghanistan and over a decade now and women my personal advisers from the University of more the above both that victor and muddy and chances advisor different from the University of Stuttgart and in addition we have had some support from a lot of other great uh centers out there in the Iraqi Ministry of Tourism has been on board with this proposal cultural museum or what is left of the personnel have been supportive of this the newly-formed gilgamesh center for the protection of heritage and of course the united for heritage initiative by UNESCO now the destruction of cultural heritage is unfortunately nothing new these are actually recent examples we can point out that destruction has been happening for thousands and thousands of years and it tends to be that whenever a new occupying force comes into a region they will you raise the path that has been there before for their own ideology whatever that might be i in some cases it's you raising the faces of past kings and putting their own in place and making their story in the history of the place that they come into the recent cases we can look at in the 16th century when the Spanish from the lotus came into a Latin America to burn the mind codices was a way to the race that pass of the mind and replace it with whatever narrative they wish to put in at that time likewise with the bombing and Buddhas the Taliban destroying the bombing and Buddhas was a way to remove the buddhist past about part of Afghanistan and replace it with strictly an Islamic past and then course most recently we have the destruction of the Mosul Museum which was a way to rebuild we remove everything that was pre-Islamic of that area and even in some cases we move specific parts of the Islamic history of the Mosul area to replace it with their own interpretation and most recently of course we have the occupation of Palm IRA and the destruction of certain Paarl parts of palm IRA as well and again the same sort of thing removing anything that that wasn't part of their own interpretation of Islam
so what I'm gonna do is I'm gonna show you guys a little bit how the web platform that we've created works chances that talk about some of the other issues and summary of the results surrounding the project and but hopefully this will give you an idea of how it works and once you know how works you can maybe think about that in context of what Chantal presented and so the web
application is pretty simple we basically have 4 steps that happens on the web and I should emphasize here that this web application is completely open open source from start to finish but if you wanna start your own project you can do that we have all the code upon get and but it's very on project we much rather that you just contribute to the code base that we already have going so that we can all benefit from it in the end and 4 steps so the 1st step is identifying lost heritage so this means that anyone can log in the only thing have to do is create account and that's pretty simple you log in you can drop a 10 anywhere on a map where heritage has been destroyed and of course that is restricted to the Middle East or Nepal where we had that earthquake that happens here in Europe as well often times there are some places that is simply are protected heritage sites you have vandalism and you have looting you have that after that happens here in Europe North America all around the world this is something that is a universal phenomenon that we all face once a location has been unidentified it opens it up for people to simply upload photos that they may have taken so if you are a tourist in Mosul 10 years ago unfortunately not many of us were uh but if you were you could upload your photos from the Mosul Museum and then those go into a photogrammetric reconstructions later on organization is probably 1 of the fundamental things that we need computers still aren't that good at doing the kind of pattern recognition that our brains can do in an instance we can look at a group of pictures and we already know what belongs together so that it's a lot more resources to do on a computer then we simply don't have the money to put into the core sort of crowdsourced or crowd computing applications that would make that happen forest and but by having the humanize dedicated to looking at images finding images that belong together grouping those together in some sort of logical way north the side of the temple of Bell and palm IRA for example uh all the images that someone season and then that allows us to quickly download those logos and some sort of photogrammetric software and add to staff memento and frames whatever it might be and that lets us then stop producing the the reality based photo realistic models that we get from there and then cost that forced people are able to do and that sort of uh reconstructions on the around everything that's happening there is happening by people participating in the community I should emphasize that uh the results from the site so far are primarily from members of the community not for myself and not from chance so the web platform is completely open and the moment someone uploads content to our website is accessible for the rest of public so there's no blocking on that there is no reviewing what that photo might be thankfully we haven't had any issue with people loading dodgy content on their it's all been great so far but but we don't believe in this idea that we are the gatekeepers for the content to go on there we let that be open and let the public decide and you know we can implement sort of a red it ratings on that people can up the about down but the content as they see fit but for let that happen and so completely open source and it's also I should point out the 1st project to do crowdsourcing are for lost heritage uh that's out there right now and so the users are contributing the content the pattern matching the data processing at the narrative building we have people who are constantly contacting us saying hey can we use this model can we do this do that we always sum of course you can it's completely open source and we
encourage everybody who does a reconstruction to release it under some sort of Creative Commons license if it's a fully open 1 even better but basically people are doing everything here we have people creating their own exhibits from it that people creating applications people are doing all kinds of
different users with this on here people even remixing some of the content without some interesting and 3 versions of models come under the site other people have done along the way so this is what you see when you get to the home page you see a map with all the different sites on there and it you know if you go up to the locations area that lets you create a new pair otherwise you can simply just click on any of those areas and it takes you to 1 of the identified locations and shows you know what we actually have in those areas and you get the um this is actually going to other locations were not the palm IRA location you see that there are several and groupings already created that we have the temple of Bell and you know which you could then go to either upload more images directly to that grouping or you can simply dump a bunch of images to that location so if you bend upon my Myra if you have digital photographs you can take that entire load just dump it straight there they show up on the stack of unsorted images which then volunteers can come in and start sorting into relatively rare and 2 related groups uh that that belong to that and so you can see you can just drop those images directly into their once you come to 1 of the reconstructions and which you see right here Quentin whether reconstructions and you have the option to either remove add removing doesn't remove it altogether from the site it simply removes it from that group so that you know using laxatives belongs to another part of Paul Meyer whatever it might be and you can then also download the original images so that you can stop processing those or do any sort of modeling that you want to based on those images and then you also presented with all of the images available at the site at that location so you can choose whether images you might want to add from there at that point in time and yeah try the ago so in summer we're talking about here is that users of our the ones creating and identifying the locations it's a lot better for someone who's is from a particular area to build to indentify those sites of lost heritage to be able to talk about it then myself as a North American living in Europe you know I can talk about Spain I can talk about North America but it's better for people with local knowledge to identify those sites are more people were willing to do research on them but then uploading the photos and this is particularly useful for areas where we have a lot of those photographs that's not everywhere but you know where we can take any photo really even the old and scanned negatives or scan slides can be useful for certain types of photogrammetric reconstruction uses to doing the organization grouping all those photos together into useful image categories they're doing the processing and those users there are also retelling the story and a couple other things on the web platform we also have just is the basic site administration so we have the the content management system that some admins over moderators could have access to so they can kind of healthy and checks on the site as needed but we've never needed to do that so far and there's a couple things that I wanna touch on before it passes over to Chancellor were looking forward to the 1st thing that we are actually have this implemented but it's still too resource intensive intensive crashes server very quickly we have a flicker integration in place now so where ever any of these sites are at it will automatically fact available license free or open license photos on Flickr here so that you can then just simply add those photos to any relevant grouping of photos that you want to this means that you know we can already say that there are thousands or tens of thousands or hundreds of thousands of photos available for a lot of these sites that already exist on the Internet today we don't need to stand out new equipment had to these areas particularly areas where there's a conflict a danger for people and those photographs are available it's just a matter of finding them identifying them and this is 1 way that we can do that by integrating with other existing platforms the other things that we're trying to do now our our interest the automation so getting the sort of a tree matching of images finding images that already share things
together through computer vision and quality control the use images this isn't to say that were filtering out good or bad but if things images that have X is X if data images that have higher resolution and the sort of things are more useful for Photogrammetry that an image that does not so we're trying to come up with the way that we could say that this is a five-star crowdsourced image this is a 1 star crowdsourced image so people can on that automatically say OK well I wanna grouping of just 5 star images that we do there and then finally getting down to cloud-based photogrammetric solution so that we can do this automatically and in the cloud are also working on a new visual experience right now which will Chantal talk about and finally things like gamification where people can you know kind scores of the different things that they're doing on their have their personal dashboard so that they can see that the kind of things that are already working on and user-generated virtual gag gallery so people can then read tell the stories of heritage as they want to be able to do and so in summary we got the locations were working on a task gamification we already do you have a 3 gallery in place and using sketch that users can already create collections and tell stories as they want to do we have an API that's already out there so people can consume the data that's that's on the website it's all open on and we do have a slack channel people wanna a jump in there it's half active some people talk a lot we don't always have a regular community in their chance thank your just ended everyone hear me yes OK we're on OK so thank you very much Matthew I'm going to take a step back and give you some perspective to where this project began
and where it is today so this project did begin as the term I was named Project muzzle of course the reason being in late February of last year many of us were heartbroken when we saw a video that went viral of members of the self-proclaimed Islamic state destroying artifacts in the most museum Matthew and I discussed it uh we were talking about it and trying to find a solution or something that we could do and the idea was sprung to mind 1 we crowdsource reconstructions of these artifacts and put them back into a virtual museum we can tell the story of what happened so in March of 2 that in March 8th of last year we did exactly that so we actually have the website up and running within 10 days and as a short timeline so we have the destruction at the most museum belonged to the project on the cake we received half of probably 30 40 50 60 images at the beginning we had no idea where this project was going to go which is why was his name project most of the beginning but then after the earthquake occurred in April of last year in the hall we realized weights we don't have to do this only this is perfectly applicable to any region in the world for cultural heritage has been destroyed and it doesn't have to only be destroyed by men they can be destroyed by earthquakes so we actually have a full reconstruction of the bar square in Katmandu from before and after the earthquake as a photogrammetric reconstruction you can find that out on a website then you have the destruction of Paul Meyer and later we were awarded the Innovation Award 2015 bytes sigh are but at the same time we began to look into ways to actually put these these are artifacts the virtual reconstructions into a virtual museum in the Economist Media Lab The Economist magazine they reached out to us and they said we think we can build a virtual museum could we take your data and we said of course you can take a data this is what we want
to do we want to collaborate with anyone and everyone that wants to participate so I'm going to show you some examples of that and it will be released within 2 weeks to the public then
I just read just returned of from last Thursday we had a museum events in the ones of even a new version of the front end as an interactive
interface to a website so this is some examples of the 3 D reconstructions that you see that have been done but the ones that you see here it will actually you will see them in the virtual museum that is released for Google cardboard IOS Android it's all going to be free and it's all going to be out there for anyone this is 1 of my particular favorites this is the new murdered this the entrance gate to an image if any of you happen to have been following the news you would know that this entire site has been either destroyed with dynamite or used the used bulldozers to destroy so this is actually 1 of my favorites because it's so large this by 1 us to that you see here to your right were enormous enormous artifacts and you can see in the British Museum they have examples there so as the project began as project mosle and then with all my right but the occupation of Paul Meyer and cultural heritage being destroyed in in a Paul for example we decided to take a global focus and with the global focus you can see now that we have pins that volunteers have dropped since on every continent basically every continent and so we are focusing on a global scale and promoting the positive narrative with digital preservation that is 1 of our main focus so here's an example of the boss square in the Paul from before the earthquake in April of 2015 and if you keep your eye on the model and its switches that is after now of course you're only looking at a still image because if you go to our website you can rotate around that three-dimensional model you zoom you can pan and you can click the before or after button to see that yourself this is all integrated into style with sketch of course so we decided if we're going to expand our project to a global scale why don't we change our names and we did so our project is called rectory which use all on the on the program for today which is to recreate in the language of Esperanto and here you have a camera that's being put back together by multiple dots which is very similar with photogrammetry so some of the results are we have a web based platform for its open-source billing volunteer community public 340 gallery and a network a growing network of private and public partners as well as the media coverage that we have received is directly proportional to how many volunteers contribute to our project and how many images we received for example March and April moles museum last year uh I believe the occupation of Myra happened in june so you also see the number of images that are being uploaded is because it's being reported on in the news which is why we need you to spread the word because someone you may know has been to 1 of these sites and they would like to contribute so some examples of again the results the 1st virtual museum application is due for public release and that is called recover moles all that will be released uh as I spoke as I mentioned before we are supported by the Iraqi Ministry of Tourism as well as you UNESCO like for heritage this and 1 direction actually 1 of the key directions for this project is the idea of the narrative the discussion of what are we how are what are we doing by generating this 3 D content of lost heritage it's a really good question it's something that that hasn't really been too has hasn't been discussed in certain ways like we're doing here so here you have a 360 degree of uh render example of the virtual museum this is the virtual museum that is the model that was reconstructed with I believe 16 images the wrongly taken from 1 side because the line itself was positioned against the wall so it's a partial reconstruction placed into a virtual version of the museum this application the virtual museum you can come and see for yourself we have maker space tomorrow from 12 15 to 115 in the main hall I recommend you come and you can you can check it out yourself with the headphones and listened and have the experience the a very obviously what is also connected with digital reconstructions and if some of you have been following the news recently you may have I seen that there are some people are you know printing out some of these objects actually have an example with me here so the line of most of course is very large but you can also print it out very small
but what what we can begin to do is it is were beginning to go in making conversations about something that haven't been discussed before for example in this case we have 3 D printed a small version to scale of the line but what you see by the length of the line is white in the same way that you would visit a museum in rome for example in the sculpture was only of the hand in the torso of a statue but the museum curator attaches the arm they complete the reconstruction this is the same thing that we're doing because with 16 images the color that you see is reflected in the geometry we have we recreated if we get more images we can complete that model that's a very important point to raise so here's an example the 1st version of the Virtual Museum of being shown with these models in Amsterdam at at thought last year and I just want to show you because this is the the absolute new version of the website other we're doing interval with it in collaboration with Unite for heritage of UNESCO and it is called reclaiming history if you go to reclaim . org you can experience this is the 1st phase of the website itself so the birth of loads up you click it's a web GL application here you have the pens that were dropped originally on the project's website there now being integrated and pulled from the database so that people can explore it and see themselves this does work for mobile devices as well you just have to turn your mobile device horizontally and so a person can move around and it creates a more interactive experience than what we saw before with the Google I Google Maps based application and with this map you can you can move around as I I showing here in this video
so here we have the moles of museum and it loads up and you can scroll through some of the reconstructions so this basically just creates a more interactive experience and eventually this this entire website has been has been made for our project which is a volunteer project is that is received 0 funds since its inception and yet this has been made by a pro bono work from an ad agency in a tech company as well so it's an amazing
example of what we can do together from volunteers to large corporations to academic institutions so I recommend that you follow us on social media come visit our maker space tomorrow from 12 15 to 115 and these are some of the people that make it happen from behind the scenes in our project thank you very much and would like to thank our volunteers as well of course yeah thank you very much
chance and Matthew we've got 5 minutes for questions from the audience at the yeah we always ask people to get into the mine and I except for that that's not museums what users have an idea of the constructions inputs you already know
how many of you it's because I don't know film comes to mind other such
that we haven't had anybody looking directly to Estonia and some people have talked about that in I yet not specifically like people try to recreate a scene or something that we have had people talk about using this in in creating actually their own little physical art displays a retelling somebody story so that's kind of a common 1 by the most recent ones that we've got a bit about another idea that that we discussed recently actually since the beginning is that many of these artifacts may have been destroyed or they may have been looted and sold and there currently being sold on the black market which is number 3 in the list of lists of sale of anything in on so for example maybe a
3 dimensional reconstruction of an artifact that went missing could be used in comparison to someone pointed in a public grants or in a private collection so I think this would be a different story any other
questions and what about i which material is this review construction made from what everyone else is the 1 in the sense that it is a 70 per cent would filament 30
% you lay out printed on a small differences so that would take 7 hours
to go out with the word of the bigger ones using this so those are just some friends of of printers and so the the larger ones in the color powder friends as well so you get the whole thing it's the other set for printers at a venue for that but I would like to stress really quickly just 10 seconds is that 3 print 3 friends for this
specific purpose is not exactly 1 of our main focus is on the project because it draws people in in in a public events space but we're more interested in what we can do with digital reconstructions as we reach a much larger problem virtual museums and yes exactly in terms of French
museums and I was wondering about your position to works managed to such as global conscience Institute for example is it because I am 50 and convinced that this but it needs a huge visibility and
huge audience to to have impacted deserves the presence of water so when the user is there any and that potential
for proliferation and cell if he I mean he coming from the academic skills I to connect the success of the pilot project is somehow I know what that conflicts that have undamaged and there's always this standard no between visibility and mass audience by various forest that's important projects and that keeping a distance and the independence and stuff Collioure whatever position there it it's a project that i'm were were fortunate in the sense that project is not purely academic and we recognize that were sort of halfway between we have our feet in the academic world we keep ourselves in today there we recognize that this is actually public project more than in academic and so we are actively seeking out the sort of collaborations and dual is 1 of the people we are seeking a collaboration with if you have any other ideas please let us know where you are sitting in anything because it is like the back producing publications are you from the academic sphere presenting this at academic conferences very important for more perspective from the public sphere
to some republics of public sites so being able to do that I think needs something a lot more than what an academic the EU funded project would would possibly and
looking forward to the future and you would have to be a tech company that would have to help make this still yeah of the the and do you think is the virtual museum
as that under threat sometimes from this post and Islamic States dude the witnesses and things like that question on whether that'll museums
the downside is under threat to by the so-called Islamic State has multiple so the mean of the data is very well protected and 1 of the strategies that we've already put in place to have several
other companies making backups of everything that we've created so it would be really difficult for them to be race that as well at this point I'm just as we see in the Internet tends to propagate things it's hard to remove something from the Mr. all last question over you and there
and do you have many different uses of the you have uh like a small group all
the heavy users so so interested in uploading photos that you get most of the pictures from this model it's
changed over time at it when it when it when it hit really big deal with with media after the most museum in and Meyer last year we saw we had people from I visited the site here my photos I don't understand how to create an account taken descendants of Dropbox then we have people that archeology architects and they say I will study architectural drawings and we can reconstruct it partially in some buildings and we have archaeologists people's it's really from every walk of life it's really really everyone was and and we can't really quantify that our we kept everything anonymous so far
and Intel we that we can go back and forth on this gamification question because of that but because of the sensitive nature of the project people's contributions have an anonymous so we can't really say how many people contributed how many people done that all we can say is we have uh 350 user accounts bank created but
we have like a hundred thousand visits to the site so you know there's a disparity between people contributing people visiting consuming content but yet and were also willing to work with the with photographers for people who have images that don't want to have them displayed on-line publicly but we can still use them and provide them to the volunteers and they can be used to make the digital reconstructions as long as we we basically wanted to build the flow so that the credit is given the whole way from the from the beginning of the where the images come from 2
people who have not sought to make the reconstructions we like that to be part of the semantic documentation and the continuation of the story of the artifacts really there was still a very long continuation of your own
story and the success and beat the hell out of it and it's loose your available so thank you very much sense in Matthew and good luck they they will be available for further questioning face-to-face minors IV


  673 ms - page object


AV-Portal 3.9.1 (0da88e96ae8dbbf323d1005dc12c7aa41dfc5a31)