Integrating Collaborative Data Collection And Versioning Into Open Source Tools For Disaster Relief.
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 95 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/15546 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Production Place | Nottingham |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSS4G Nottingham 201346 / 95
17
25
29
31
32
34
48
50
56
58
68
69
70
82
89
91
00:00
Open setEnterprise architectureData managementImplementationSuite (music)Integrated development environmentDemo (music)Distribution (mathematics)Component-based software engineeringMobile WebDISMAReplication (computing)DisintegrationDecision support systemGeometryServer (computing)Software architectureSource codeInformationData managementReduction of orderSoftware developerConnectivity (graph theory)SoftwareRepresentational state transferProjective planeDecision support systemImplementationLevel (video gaming)Server (computing)Bridging (networking)ParsingGoodness of fitRule of inferenceMobile appClient (computing)Hazard (2005 film)Electronic mailing listFile viewerTraffic reportingReplication (computing)Latent heatMobile WebDemo (music)Software architectureLine (geometry)Operator (mathematics)Series (mathematics)BitQuicksortYouTubeInternetworking1 (number)GeometryState of matterService (economics)Element (mathematics)Slide rulePresentation of a groupMultiplication signContext awarenessObject (grammar)TouchscreenINTEGRALInterface (computing)Mass
06:14
Computer-generated imageryType theoryView (database)EmulationInformationData typeScale (map)Field (computer science)Level (video gaming)Workstation <Musikinstrument>MIDIMetropolitan area networkUniform resource nameNewton's law of universal gravitationSpecial unitary groupSide channel attackSummierbarkeitStorage area networkFluidPhysical lawInfinityState of matterRule of inferenceRaw image formatQuery languageDigital filterFile formatElectronic data processingServer (computing)Demo (music)Field (computer science)Representational state transferView (database)Key (cryptography)Symbol tableWorkstation <Musikinstrument>InformationFile formatWeb pageGeometryData storage deviceService (economics)Line (geometry)Scaling (geometry)Function (mathematics)TouchscreenoutputDescriptive statisticsQuicksortFilm editingInteractive televisionHoaxOnline helpDifferent (Kate Ryan album)CuboidMereologyObject (grammar)Level (video gaming)Vector spaceVolumenvisualisierungClient (computing)MassObservational studyPlastikkarteWordStreaming mediaMedical imagingThresholding (image processing)Electronic mailing listAreaQuery languageMultiplication signFilter <Stochastik>Projective planeBridging (networking)Extension (kinesiology)CircleSquare numberTriangleRevision controlBitIntrusion detection system
12:28
Library (computing)Wide area networkWebsiteGoogle ChromePasswordOverlay-NetzStorage area networkOpen setChi-squared distributionDiscrete element methodGamma functionMenu (computing)RSA (algorithm)ArmLipschitz-StetigkeitImage resolutionTouchscreenServer (computing)Workstation <Musikinstrument>Data managementLevel (video gaming)Instance (computer science)Point (geometry)Independence (probability theory)Dot productCASE <Informatik>Incidence algebraContext awarenessWeightObject (grammar)NumberComputer animation
14:00
Web pageGeometryServer (computing)Coma BerenicesModule (mathematics)File formatInterface (computing)Focus (optics)Function (mathematics)InformationDataflowComputer iconPoint (geometry)Morley's categoricity theoremFloating pointQuery languagePolygonLine (geometry)Link (knot theory)Computer networkDemo (music)Drop (liquid)View (database)Default (computer science)Group actionGoogle EarthComputer fileConfiguration space3 (number)DecimalMaxima and minimaLie groupPhysical lawCurvatureInfinite conjugacy class propertyMaß <Mathematik>Computer-assisted translationDew pointSynchronizationEmulationRight angleComputer fileSampling (statistics)PolygonPoint (geometry)Square numberChief information officerDifferent (Kate Ryan album)Line (geometry)Software testingWebsiteProjective planeView (database)TrailMultiplication signMathematical analysisServer (computing)DatabaseInverse elementAreaMobile appLevel (video gaming)CASE <Informatik>Drag (physics)MereologyNumeral (linguistics)WordOnline helpStack (abstract data type)Term (mathematics)Exception handlingElement (mathematics)GeometryService (economics)File formatSoftware developerGreatest elementInformationOperator (mathematics)Dependent and independent variablesCollisionSoftwarePhysical systemChemical equationProxy serverDemo (music)QuicksortCartesian coordinate systemContext awarenessRevision controlConnected spaceSelf-organizationLink (knot theory)outputDecision support systemSymbol tableMorley's categoricity theoremPlotterComputer iconRow (database)Client (computing)Field (computer science)Data managementRepresentational state transferWeb pageArrow of timeFigurate numberFile viewerHard disk driveUsabilitySynchronizationTimestampClosed setPasswordPlanningFiber bundleINTEGRALOpen setDot productDrop (liquid)Personal identification numberMultiplicationDenial-of-service attackEvent horizonSurjective functionIncidence algebraQueue (abstract data type)
Transcript: English(auto-generated)
00:04
So, my name is David Ascove. I'm with Pacific Disaster Center. And Pacific Disaster Center, we apply information science and technology toward decision making and disaster risk reduction. We have a series of software tools to act as a decision support system for emergency managers.
00:26
And then this is Scott Clark with LMN Solutions. And I can let him, do you want to talk a little bit about your company or should we just dive in? Okay. So, we are collaborating together on a project that is called Rogue.
00:41
And I'm not going to read this whole thing, but you can, I guess, watch it later on YouTube or whatever. And so, basically Rogue is being implemented by OpenGeo, who I guess just recently changed their name to Boundless. Is that what it's called? And so, I didn't change the slide, sorry.
01:01
And LMN Solutions. And then at Pacific Disaster Center, we are what is called the transition manager. So, basically we are sort of the entity that agrees to take on all of these things that are developed under the project. And hopefully, you know, find some practical use for them. So, without further ado.
01:24
Basically, there are four main components to this project that we're working on. And the first two are really driving the majority of the development on this. So, the first one is called GeoGit. And I apologize if the title of this presentation, you know, drew you here.
01:44
But we're going to speak a little bit about that, but not exclusively. And basically, it's data versioning and replication. And then there's Arbiter, which is the mobile data collection. And then we are working on also this thing that's, we're calling it the GeoServices Rest, which is named after the specification that Esri proposed.
02:09
And then ultimately was not accepted. But basically, what it is, is we are working on making GeoServer implement the Esri GeoServices Rest specification.
02:20
So that clients that are built for an ArcGIS server can talk to GeoServer and hopefully think they're getting something useful out of it. And so, we're working on that. We have a demo here if the internet cooperates, we'll do some demos. And then we're also working on a KML uploader. So, you can actually load KML up into GeoNode and then it will come out of GeoNode and GeoServer as a, you know, native GeoNode or GeoServer layer.
02:51
So, those are the things that we're working on. I put in yellow the ones that we'll be talking a little more about today. So, as far as the GeoServices Rest integration, PDC has a decision support tool called Disasterware.
03:07
And it was built, it has some support for WMS. Previously, we supported WFS. We found that was rather difficult to support partially because it's a client side architecture.
03:21
And so, we actually wrote some parsing rules in the client side. I know it somewhat duplicates open layers. But we also had issues with SLD, which was a whole other spec that we had to support. And so, about three or four years ago, we pretty much went to an ArcGIS server implementation. And it still has pretty good support for WMS built into it.
03:42
But we've more or less abandoned the WFS. And so, when we started this project, you know, really the question was how are we going to get the data from GeoServer that we're using for this project into Disasterware. For things that paint a picture, that was easy. We'll just use WMS for features that needed to be delivered into a client side app.
04:03
You know, we said, well, do we go back to WFS and do that again? And at the time, there was the, I guess it was a proposed specification is what it's called. And it was ultimately not accepted. But at the time, it was in the proposal stage. And so, we decided to begin building a tool that would work with that.
04:24
And ultimately, we did not go with the spec. We ultimately just went with an ArcGIS server. 10.0 is mostly what we've been testing it against. But I'm fairly certain it will work against 10.1 or 10.2. And so, you know, again, not really working with the specification, but more just, you know, building a tool.
04:47
So, this is a screenshot of Disasterware. And as you can see, it has, you know, a list of the current hazards that are here. I'm not sure. I think this is some rainfall data that we're looking at.
05:00
There are earthquake data, a tropical storm over here. So, this is our map viewer. And pretty much all of these map layers, except the Google background, are coming from ArcGIS server. And so, this is the challenge that we're facing as far as getting data into that. And then it also has some ability to bring in TV news feeds for, you know, disaster managers.
05:26
You know, we try to get up on the emergency operations center. We try to get one of those screens. And then they say, but we need to watch the news. We said, okay, you can do both. You know, it's pretty interactive. You can hover your mouse over it and get features. And then we also have reports that we bring into it.
05:47
So, as far as, you know, this is kind of what we started with. We have disaster where we use the ArcGIS server REST interface. And then I put WMS here as a dotted line. I mean, it works fine.
06:01
ArcGIS server has decent support of WMS with a few little hiccups. But we really have no reason to use this because we can just speak over this REST interface to ArcGIS server. So, what we were doing was bringing in GeoServer and trying to figure out how to make that bridge. And so, what we decided to do was to use REST for feature data.
06:25
So, any kind of dot, really it's all the vector data. Anything that's not, you know, megabytes and megabytes of vector data can be streamed back to the client and rendered there. And so, we decided to build that as a REST. And then we just continue to use WMS for anything that's image, whether it's raster or maybe something
06:44
like contour lines where it is vectors but you just don't want to stream it to the client. And so, really this REST here, the idea is disaster where, as far as it knows, it's talking to an ArcGIS server. And then the GeoServer, of course, can be based on a PostGIS data store.
07:05
And then one of the other things of the project that I mentioned is this GeoGit. And so, many of you have probably heard of GeoGit. It got some nice billing in the keynote this morning. And one of the great things about GeoGit is, of course, they can be chained together in a distributed and versioned fashion.
07:28
Okay, and so that's how we can get all of this into disasterware. And so, you know, all of this work here is great, but this is when we go to our operational demos, our exercises, showing it to the stakeholders and the funding agencies, that's really what they're going to be seeing.
07:44
So, we need to be able to get the data into there. So, as far as ArcGIS server, there are really three main interactions that we're working with. The first one is to get map service info. So, it just describes the map service. It's really fairly superficial.
08:00
GeoServer doesn't exactly have a concept of a map service. It really mostly just goes straight to layers, so we had to kind of fake that a little. Whereas ArcGIS server, you can break it down into folders and different services. So, this is the HTML view of the map service info. And then ArcGIS server, as part of its rest interface, you know, the rest can produce HTML or the rest can produce JSON.
08:27
And so, this is the JSON output. They have what they call pretty JSON, which formats it nicely and puts indentations and everything, or you can just get it all in one line. And then, here we have a screenshot of our server here, producing rest output.
08:49
So, this is describing the map service. And this is coming from GeoServer. As you can see, there are a few things left to be done. You know, there's a description field here that has some information, and I think it might be missing from this one.
09:07
So, you know, there are a few things left to do. But we do have the IDs and the names of all the layers that are present. The next thing that you do is you go and you get the layer info. And again, you're still working by map service. This is sort of the way that the GeoService's rest spec works.
09:22
But now you go in, and even though you're still working by map service, you're asking a lot more detailed information about the layer. So, you want to know the field names, you want to know the extent of it, scale thresholds. And then, one of the biggest challenges that we faced was the symbology. So, it actually returns the symbology to you.
09:40
And then, if you request it in JSON format, oops, I pressed the wrong way. If you do it in JSON format, you can see here are many of the things that, you know, were just in the HTML view. Except now, it encodes that icon. So, the whatever layer this is here, I guess it doesn't really matter. But it encodes it into ASCII text, okay?
10:01
And so, that was one of the biggest challenges that we faced. For many months, we were just dealing with very simple, you know, squares and circles and triangle marker symbols. And so, getting that was something we did just in the last couple of weeks. It was a big victory because every time we demoed this, they said, that's great, but can we not have circles and squares, you know?
10:21
So, here it is coming from GeoServer. And so, you can see the really smart folks over at OpenGL and LMN figured out the secret sauce to encoding those images into the REST output. And so, really when it connects, it has to connect to this.
10:40
It gets the list of layers. The layers are all referred to by a numeric ID. And then, it has to get the symbology, scale thresholds, et cetera. And so, this is really the key, is before you can draw anything, you have to get this information. And then, the next step is to query the layer. And that's where you pass in a bounding box and you say, well, what police stations are within this area?
11:01
And you can put in time queries, where clauses, spatial filters, that's your bounding box, et cetera. You know, again, we are not supporting all of this. The Esri REST spec is over 200 pages. We're not going to get there. But essentially, with some of your help, maybe we will someday.
11:21
So, we'll talk about that in a little bit. But basically, when you make a request, here is what it would return in the HTML format. And so, you can see there are only three fields. These are hospitals. And then, there's a geometry. Now, again, no symbology. You've already gotten the symbology from the previous step.
11:41
Here's what it looks like in the JSON view of the REST output, same information. And when I said there's three fields, there's actually, I guess, five. I wasn't counting the, you know, like object ID field or whatever it is. And here is a layer, I'm not sure what layer we're looking at.
12:03
I guess it's fire stations. And this is for Tegucigalpa and Honduras, which is where we did a demo recently. And so, this is coming out of GeoServer and it's returning features that look exactly like a REST output from ArcGIS server.
12:21
So, I'll do a demo if the Wi-Fi cooperates here. I've already pulled it up, actually, so hopefully it won't take long to show. This is an instance. Yeah, unfortunately, our screen's gotten kind of compressed here due to the resolution. This is an instance of DisasterAware that we call EMOPs, and this is for registered users only, for emergency managers.
12:47
And this is exercise data that we put in for Tegucigalpa, Honduras. And so, you can see all of the, you know, it's a Google Maps background, but all of the dots on the map here are coming from GeoServer.
13:02
And as viewer, as far as it knows, it thinks it's talking to an Esri ArcGIS server. So, it's, you know, completely neutral to the vendor, you know. So, what do we have here? We've got temporary first aid stations.
13:20
We were simulating, what was it, a stadium collapse during the parade on the, what was it, National Independence Day or something like that. So, you know, they're setting up all kinds of temporary first aid stations. These are the incidents that they entered at the command center. Here are control posts.
13:40
These are actual existing medical clinics that are permanent. So, you know, but these, I mean, it kind of doesn't matter what the layers are as far as the technology is. You know, the point simply is it's talking to a GeoServer, but it really thinks it's talking to ArcGIS server. And I had the screenshot of that just in case the Wi-Fi wasn't working.
14:10
So, as far as the plan scope goes, you know, we've implemented close to as much as we're going to do. There's still more to go, but we're almost there.
14:21
So, basically, the three things that I just walked you through, you can get a point line or polygon out of it, and we've gotten to the point of doing icon marker symbols. So, what is coming up, we still have about another year left in our project. So, we have multiple point icons for per layers. We're talking about like categorical or numerical classification on a field, say.
14:45
We're hoping to release this as a GeoServer community module, hopefully get other people interested in this. I don't know if any of you work with clients who have built, you know, tools based on the Esri stack, and you might like to integrate with them. So, you know, maybe hopefully, you know, get this tool out there, hopefully get some help developing it a little further.
15:04
And then we really only support the JSON format right now. So, hopefully trying to support the HTML format that makes it more user friendly. It's really hard for non-developers to wade through that. And it also gives you hyperlinks that you can click on to follow through the different links.
15:20
But as I mentioned, you know, the spec is 221 pages. It basically is everything that's in ArcGIS Server. We're not going to support all of that, you know. Maybe someday it'll get there. I don't know. And just as a for example, there's no way right now and not even planned to actually get a map out of the system. It's really, we're only planning to do feature data.
15:43
So, you know, that might be something somebody else could extend and work on with the base there. The next thing that I wanted to discuss was the KML uploader. And this is something that has really plagued us for a long time. KML is a great way to pass data around from one person to another, to visualize it and view it.
16:04
It has a lot of challenges in terms of trying to get it interoperable into a system. What if I need to read that KML and figure out if there's some hazardous situation within my area? You know, what if I need to find out all the critical infrastructure that's within the, you know, shake map or whatever it is that's in that KML.
16:22
And it's just something that at Pacific Disaster Center we have just really struggled with because it's a great format. But it's just very hard for us to get into our system and work with. So, we are working as part of this project. I guess LMN started it and then OpenGeo is going to, or Boundless is the other way around?
16:44
Okay, Boundless started it and then LMN is going to pick up the ball with it. Uploading KML into GeoNode and GeoServer. And basically when you upload it, it serves it out as a brand new layer. And so the current support, and this is really, we're on the bleeding edge here.
17:03
So, you know, I don't want you guys to get too excited and run home and, you know, think there's a solution to this. But basically right now it's point features only. There are no styles built into it yet and no network links are supported. So, this is really just in the very, very early stages of this.
17:23
And so as far as a demo goes, I think we're running a little short on time. So, I'm going to, I have some screenshots that I prepared in case the Wi-Fi is bad, which I think will give you the same idea with much less time. So, what I did was I just Googled for sample KML files. And somebody had made one of the Google Earth Campus here.
17:43
So, they put a sample place mark. It's got a yellow push pin and some attribute text in it. So, this is what has been built. So, it's part of GeoNode and you come in and it has an upload layers and you drop files here
18:01
and you can drag a KML file right onto that. I don't believe it supports KMZ right now. And so literally, you know, just drag it on there. There is also a file chooser too if you want to go grab something off of your hard drive or the network or whatever.
18:22
Once you have it, it reads the file and it says, okay, I have this to upload. I believe you can do multiples. You can kind of queue them up and then you say upload files. It's pretty straightforward. And then when it is over, hopefully it will say your layer was successfully uploaded.
18:43
It doesn't always say that. But if you've done everything, you know, according to that, you know, what we support, it will. And then in this example here, it's very hard to see but right there is an orange dot which is our yellow place marks.
19:01
I told you, no style. So, you know, the yellow push pin became an orange dot. And the text there, the name, simple, I think if we expanded that. But what it said was simple place mark. And so when we come in here, it has, you know, many of the information that came with the original KML.
19:26
So it's pretty simple. They actually, the guy who developed it told me it doesn't support polygons. I actually loaded it in and this sample from Google had 3D polygons and all kinds of extruded features and just, you know, it was like a, there was like 30 samples of all the kind of crazy KML you could do.
19:43
And it actually took some of them. So it took some of the 3D polygons and it flattened them to 2D but it did work. So there are other things that we've gotten to work. And then once you load it in, I showed you here in GeoNode how it's now, you know, available as a layer for you to work with.
20:05
And then it's also available as a layer in GeoServer. So here, so as soon as you load it in, you go back to your GeoServer, hit refresh, you've got a new layer there ready to work. And then it supports all of the things that a GeoServer natively supports. And then I loaded it back into Google Earth and like GeoNode showed it as an orange square.
20:25
This shows it as a kind of turquoise dot. But it actually, you know, came through fairly well for an initial test. And again, the sample that I was using was sort of like what kind of crazy features can we throw at it. You know, if we're dealing mostly with like points, lines, and polygons, I'd expect that to be, you know, in well-defined rows and columns of data.
20:43
I'd expect that to be a little easier. So as far as the next step, lines, polygons, we need to do symbology. Error handling is very important. You know, you load it in and it doesn't work and, you know, why didn't it work? What do I do to fix this? So that's what we're working on. And then the last thing to mention is really, we're not really sure how to share this back to the community.
21:07
I mean, that is the goal. But it spans across, you know, GeoTools, GeoNode, GeoServer, you know, all these different things. So we're, you know, it's not like a neat little bundle that we can release. But we're working on something.
21:22
And Scott, do you want to talk about this? Oh, right. So this kind of brings it all back into, brings it all home. So basically, we had recently done operational demonstration for Rogue.
21:41
This is back into Rogue. And we use GeoNode at both the Southcom location, GTF Bravo, and then at CoPECO, which is their disaster response organization. The mobile application for collecting data and pushing it up. And then everywhere you see a green arrow is actually a GeoGit syncing between the different nodes.
22:05
So we are able to keep all of the different nodes in sync so they can actually see off their own servers. And if they lost connection, then they still at least had a server local to them so they didn't lose the data. And we keep the provenance of the data through GeoGit so that if somebody actually
22:22
deletes something, you know it got deleted and you can actually see that in history. And the other part is both organizations can actually edit the data. And then both organizations also had the disaster aware viewer open. So the difference being the GeoNode, we had the editing capability. So that allowed for users to input information, to edit, and to put things in.
22:43
And the disaster aware is a really great situational awareness tool for when you're talking about what is the emergency management center looking at, or the decision makers when they want to come in, they just want to see the big picture. And this is actually, this link down at the bottom is actually CAPECO's post on their website about it.
23:02
That's the minister in the middle and two of the CAPECO responders using Arbiter and uploading incidents into the system. So then again we use the GeoServices REST to visualize it in PDC.
23:22
We didn't actually use the KML much for that operational demo, but it's extremely popular in one and so it will happen definitely in the next one. So that kind of all ties it together. If you have any questions, then I think that's that time.
23:44
And one plug is the GeoServices REST, the lion's share of the work was done by David Winslow from Boundless. And he's at the desk out there, so if you want to ping him about technical details if you're a developer, he's the guy to run down.
24:01
Any questions for the guys before I ask a really complicated one about our server?
24:26
So the question is if the data services from DisasterAware are all public. Actually we have an application that is sort of a DisasterAware public version which is called Atlas. That's at atlas.pdc.org. It's about 80% of the layers I would say, but for the EMOPS website that I showed you,
24:50
basically the difference between what's in the Atlas and what's in EMOPS, those generally require restricted access. And it's fairly loose, I mean anyone really working in disaster management can request an account, but it's really not intended for the public.
25:13
Absolutely they could, they are password protected, but if you authenticated absolutely they're just ArcGIS server services,
25:24
or as far as the GeoServer goes, it's GeoServer services that mimic ArcGIS server if you will. Are you asking why we chose?
25:46
It's just a problem that we have over and over and over again. When we try to integrate disaster data from other agencies, there's all kinds of data out there, but often KML is the method of interchange.
26:01
And I think a lot of times disaster agencies are really looking at how do we reach end users. So there's a great feed out there from Japan Meteorological Agency. They have tropical storm tracks throughout all of the western Pacific. It's great, it's published as KML. It's great for a user to be able to just pull into free tools.
26:22
It's not so great for us to try to pull into a database and do analysis on it and try to say okay what's in this area. And it's just a challenge, and that's just one example, I mean just countless examples. So if all we want to do is display it on the map, that's one thing, but if we're actually trying to pull it in and do any kind of analysis on it,
26:41
then we really have to be able to get it into some kind of native format. Invariably somebody will bring you a KML file. I mean when I was down in Quebeco. Well I've got this flood layer in KML. The problem is the time stamp. You've actually got a history of events in the format. So it's not a single observation, but probably 10 or 20 observations.
27:03
Yeah, yeah. Right now we're not doing anything like that with our KML. I mean it's very simplistic, you know, so just really development is just kind of barely underway. Any other questions?
27:21
You said you have one Ken? Oh, you're just kidding. Thank you.