GeoScript - A Geospatial Swiss Army Knife
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Anzahl der Teile | 188 | |
| Autor | ||
| Lizenz | CC-Namensnennung 3.0 Deutschland: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
| Identifikatoren | 10.5446/31625 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache | ||
| Produzent | ||
| Produktionsjahr | 2014 | |
| Produktionsort | Portland, Oregon, United States of America |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
| |
| Schlagwörter |
FOSS4G 2014 Portland144 / 188
7
10
14
15
16
23
24
25
28
29
33
37
39
40
43
45
46
48
50
56
64
65
69
72
74
82
89
91
98
102
107
111
114
118
128
131
132
135
138
141
143
147
149
150
157
158
161
164
165
166
173
174
175
179
185
00:00
Attributierte GrammatikBildgebendes VerfahrenCodeDatenbankDatenstrukturImplementierungInformationProgrammierspracheTopologieProgrammbibliothekTypentheorieGebäude <Mathematik>IterationMAPVektorraumTaskElementargeometrieBitEinfach zusammenhängender RaumGeradeGruppenoperationLastLeistung <Physik>MereologieProjektive EbeneResultanteSpeicherabzugStellenringStichprobenumfangStützpunkt <Mathematik>TabelleVirtuelle MaschineVisualisierungZahlenbereichZentrische StreckungQuick-SortSoftwarewartungVersionsverwaltungSystemprogrammServerBasis <Mathematik>CASE <Informatik>Faktor <Algebra>VerzeichnisdienstAbstraktionsebeneDistributionenraumPunktProgrammierparadigmaInformationsspeicherungKartesische KoordinatenBitmap-GraphikDruckspannungBimodulAppletMailing-ListeSkriptspracheDateiformatSchnelltasteElektronische PublikationAbgeschlossene MengeEndliche ModelltheorieSchlüsselverwaltungMini-DiscSelbstrepräsentationObjekt <Kategorie>Kontextbezogenes SystemMultiplikationsoperatorSuite <Programmpaket>VolumenvisualisierungDefaultOrdnung <Mathematik>StabDigitalisierungSoftwaretestArithmetisches MittelGreen-FunktionHistogrammTermÄhnlichkeitsgeometrieProzess <Informatik>NummernsystemVollständigkeitWort <Informatik>Shape <Informatik>QuellcodeEntwurfsmusterDifferenteFreier LadungsträgerMapping <Computergraphik>Rechter WinkelApp <Programm>
09:42
AnalysisBildgebendes VerfahrenCodeInformationProgrammierspracheProgrammbibliothekTypentheorieInzidenzalgebraMAPExplorative DatenanalyseGenerator <Informatik>Funktionale ProgrammierspracheVektorraumBenutzeroberflächeTaskGrenzschichtablösungElementargeometrieDichte <Stochastik>AggregatzustandGeradeHochdruckInverser LimesLambda-KalkülMereologieProjektive EbeneVirtuelle MaschineZählenQuick-SortFlächeninhaltVersionsverwaltungReelle ZahlPrototypingMatchingÜberlagerung <Mathematik>ComputervirusCASE <Informatik>Prozess <Informatik>Perfekte GruppeNabel <Mathematik>PunktQuaderKreisflächeKartesische KoordinatenBitmap-GraphikBimodulAppletMailing-ListeSkriptspracheGraphfärbungElektronische PublikationSchlüsselverwaltungTouchscreenObjekt <Kategorie>sinc-FunktionMultiplikationsoperatorSchreiben <Datenverarbeitung>Mapping <Computergraphik>Rechter WinkelDienst <Informatik>Demo <Programm>BenutzerbeteiligungREST <Informatik>SoftwareentwicklerDatenbankNatürliche ZahlTopologieBitGruppenoperationLoopMaßerweiterungTermZellularer AutomatMagnetbandlaufwerkServerGebundener ZustandGraphische BenutzeroberflächeShape <Informatik>SummengleichungMinimumEuler-WinkelPuffer <Netzplantechnik>URLApp <Programm>
19:24
GeradeMomentenproblemBrowserDienst <Informatik>Computeranimation
19:43
MAPPolygonServerProzess <Informatik>BitrateOverlay-NetzRechter Winkel
20:09
MathematikIterationProgrammierumgebungMaßerweiterungMereologiePolygonResultanteQuick-SortFlächeninhaltFunktionale ProgrammierungServerProzess <Informatik>Ein-AusgabeProtokoll <Datenverarbeitungssystem>MetadatenAppletSkriptspracheRechenbuch
21:15
ImplementierungProgrammierspracheMAPBitGruppenoperationMereologieRechenschieberVersionsverwaltungNummernsystemFeuchteleitungSkriptspracheGerade ZahlDifferenteMultiplikationsoperatorURLSoftwareQuick-SortGüte der AnpassungApplet
Transkript: Englisch(automatisch erzeugt)
00:00
So just some quick introductions. My name is Justin the Olivera. I work for boundless. I'm a contributor on the geo script project and some of the other Java projects and the ecosystem like geo tools and geo server. And I'm Jared Erickson I work for Pierce County, which is a local government between Portland and Seattle. We're actually
00:23
just south of Seattle and I'm a member of kudos, which is the Cascadia geospatial users group that meets in Seattle. Yeah, sorry, we'll try to be good about talking into the microphone here.
00:41
Okay, so first question, what what is geo script. Well, geo script is a library that does spatial stuff. So it provides you spatial utilities that you would use in building an application or something like that. So sort of along the same vein as a lot of the other libraries are probably used to working with OGR Google
01:05
Some of the Python libraries Fiona rasterio shapely, you know, along that sort of same idea, but geo script is targeted at the JVM. So it's only supported by languages that run on the Java virtual machine.
01:22
And those languages include the following list here and this list is actually sort of in the order of completeness, but there's implementations for groovy Python. And JavaScript. And again, these are the Java implementation. So when I say Python, I don't mean see
01:42
Python, I mean, Jython. And same for JavaScript, not talking about node or VA talking about the Java Rhino JavaScript engine. There's also implementations for scale and Ruby, but they're somewhat less complete at this point.
02:01
So geo script is sort of spatial library for this language. For these different languages and across the different languages. We, the idea is to have a relatively consistent API, but at the same time recognize that different languages often have very different syntax and paradigm. So
02:25
While we try to maintain things like package names and overall structure and utilities, you know, syntax across the languages can vary. And so geo script is builds on top of the geo tools library. So
02:42
geo tools Java library that provides all sorts of spatial geo utilities Java based. It's been around forever. Key parts and a lot of projects such as geo server you dig and the list goes on. So really geo script can be looked
03:04
at as sort of script bindings for for geo tools, sort of the same way that a lot of the sea based libraries provide Python bindings. So what sorts of stuff can I do in geo script. So the the API sort of broken up into modules and a lot of
03:26
these modules, you know, are are just things that you'd expect from sort of any sort of spatial library. Sorry if that's hard to see but Sort of at the core of the library. There's utilities for interacting with geometries and projections and then
03:42
sort of building on that. There's the idea of format abstraction. So all your spatial formats both vector raster And then we get into visualization capabilities styling and rendering. I'll talk a little bit about that coming through. And then, you know, sort of upper level, we get into sort of more specialized stuff.
04:00
Doing some statistical stuff with plotting support for. Excuse me, geo processing, stuff like that. And we'll be showing some some examples. So the idea is to provide a really convenient lightweight API pass what you get with libraries like JTS and geo tools and Jared's going to show you a few examples.
04:24
All right, so as Justin said this is geo script is really a scripting API on top of JTS and geo tools. So we're going to show some examples of how we make things easier. And this isn't to knock JTS or geo tools because those are both really powerful Java libraries. It's just they're
04:43
they're very powerful and with some of the scripting bindings, we're going to make it easier. So the first case is, you know, this is how you buffer a point with the Java topology suite use a geometry factory use that to create a point but first have to create a coordinate first and then you can buffer it
05:01
In groovy and Python, you know, we get rid of the geometry factory you just create geometry objects directly. So, you know, what was three lines turns into one line of code. And So let's take a more complex example. How do you render a map using geo tools. This is actually the same rendering technology that geo server uses
05:21
So we have to read an SLD from files, we have to create a style factory in an SLD parser, we have to access your data. So we're using a shapefile data store. We have to wrap those up in a map layer feed that to the map context we create a buffered image, you know, there's a lot of ceremony here. We and then we finally use the streaming renderer to actually paint our GIS data to the image and then we use image IO to actually write it to disk.
05:47
It's actually pretty terse but you the scripting bindings, we make it a lot easier. So here's the example in groovy where we're reading a shapefile as a layer we're attaching a style from an SLD on disk and you will use a map renderer
06:02
Which takes an array of layers width and height and then renders it out. So we're just trying to make things easier. So let's load some features from a post just table in geo tools here. There's a connection map. So we were putting all of our connection information in that map.
06:21
We get a data store from that. Then we get the feature source, which is the actual, you know, post just table. We get a feature collection. Then we use the feature iterator to go through it. We can access the attributes and the geometry here. It's not too bad. We have to do a lot of try finalize to make sure that we're closing connections to databases and things like that in Python, you know that 10 lines of code turns into like three.
06:46
We get rid of the whole, you know, map data store abstraction and we just use a post just layer we feed it. The database and then some sensible default. So, you know, localhost by default. Yeah, I screwed up on this code sample. The first line is actually the groovy version. That's why there's no.
07:05
Oh, so it's it's just it's just groovy Python. Yeah. Yeah. So, okay. And then the our last example is, you know, how do you create how do you create a shapefile in geo tools you use a simple feature type builder.
07:22
You're probably noticing that in geo tools. There's lots of design patterns, which is wonderful for building a library, not so much for actually using it. We're building up the schema. You create a feature simple feature type, which is like the schema from that. We're creating a shapefile. We actually create the shapefile on that line, get the feature store and then we, you know, use a simple feature builder.
07:44
It gets pretty long. I mean, it's not bad, but the JavaScript version is much more easier to make sense of. We're doing the same thing. We're creating a shapefile. We're feeding it the schema using simple JavaScript objects and then we add some data to it.
08:06
OK, so those are sort of, you know, just a few short little code samples, again, trying to stress that, you know, the idea is really just to provide as convenient an API as possible. And to make the simple things easy geo tools and JTS, like Jared said, do
08:23
a ton of very cool advanced stuff, but it's not always easy to do simple stuff. So that's sort of the problem we're trying to address. And we can just go through sort of some more samples here of things that you might do on sort of a day to day basis. So, you know, translating between spatial formats is a pretty common task that people do.
08:43
And so, you know, here's an example from Groovy that, you know, shows just taking a shapefile or directory files and converting them to or importing them into PostGIS. Again, sort of sticking with the whole idea of spatial formats.
09:02
Number of exchange formats like GeoJSON, GML, and it's really easy to sort of convert between different representations. Analyzing data. So, you know, oftentimes you download data, we have no idea what it looks like, what attributes it might make available, what its distribution might look like if it's, you know, an elevation model or something.
09:25
So we want to explore. So, you know, here's an idea of actually taking a DEM, getting a histogram, and actually plotting, you know, the results. So, you know, get a nice little bar chart of my elevation values and sort of how they're distributed in my data.
09:43
So this is cool because, you know, then I can take that and maybe, you know, create a color map for a style or something like that. So, you know, sort of exploratory type task. Again, just sort of getting into general processing. This is taking, again, a
10:01
shapefile and just reprojecting it on the fly and dumping it into PostGIS. No, this is just reprojecting on the fly. What else have we got here? Generating styles. So this is something that's actually really painful if you just do it with straight geo tools. So this is something we really focused on to make quite a bit easier.
10:24
And so any GeoServer user has seen this style. It's very beautiful. We spent a lot of time on it, as you can tell. And, you know, the SLD for that in GeoServer is, I don't know, something like 70 lines or something like that of XML. And, you know, we're really collapsing it down to, you know, four lines here.
10:44
This one I was inspired by this morning by the talk from Mike and, you know, the idea of chloroplasts maps. So, you know, here's, you know, five or six lines of Python that, you know, can generate a style that looks like that.
11:02
And I'm trying to build up my Python functional programming cred here. As you can see, there's lots of lambdas and map, reduce, and zip. So I think I'm covered. This is a cool example that Jared came up with, taking a raster and cropping it based on not just a bounding box but an actual geometry.
11:25
So in this case using a circle to create a new raster that has the, you know, the cropped image in it. We're still good, I think. Okay.
11:41
Okay, so we have all these great modules in Geoscript for dealing with vector data, geometries, you know, projecting. So how can you use it? So we're going to give you some ideas. The first way is really the reason why we did this so you can use it on the command line. You can start a REPL, redevelop, print loop, and, you know, explore your geospatial data.
12:01
So this is an example of a session using the Geoscript groovy shell. You can connect your post database, list out the layers, grab a layer, you know, find the bounds, find the count, and then iterate through the features. So you can do that interactively and then you can also write this as a script in, you know, Geoscript Pi, Geoscript JS, and then run it.
12:23
The second way is this is actually there is a community module to plug Geoscript into GeoServer. And so you can do things like you can write your own WPS scripts using these scripting languages. You can write filter functions that you can embed in your SLDs.
12:41
And you can also write just like mini applications. So it's like a rest endpoint in Python, JavaScript, and groovy, which is pretty neat. And new in GeoServer 2.6, we did a user interface for this before you'd actually have to get on the server and write your scripts. Now we have like one of GeoServer's real strong points is it has a really nice UI.
13:05
So we provided a UI for this. And so this example is showing you the scripts. You can create a new one. This is how you create a process to buffer a geometry. And later, Justin's going to show a really nice example of a much more complicated web processing. Service that he wrote using Python.
13:23
And then there's a Geoscript also comes embedded in UDIG in the spatial toolbox. And the neatest thing about this is we didn't have to do it. The UDIG developers did it. Andrea Antonelli, whose name I hope I didn't mispronounce, he did this.
13:41
And so Geoscript comes embedded so you can run GeoScripts from UDIG. So this is an example of using the style API that Justin was showing and actually creating a style and then displaying it on the screen. The one limitation with scripting support in UDIG right now is you can't access the UDIG object.
14:00
So if you've done any Python scripting like in QGIS where you can create a layer and then add it to your map, you can't do that yet in UDIG. But still it's pretty neat that it comes embedded in UDIG. And then also you can use it as a library. So you can embed Geoscript in your Java applications. It's as easy as adding a dependency in Maven.
14:22
So you can use this to write your own RESTful web services, to write your own web apps, command line apps, desktop apps, whatever you're going to write. And here's the same sort of dependency information for Gradle. Just quickly mention, I don't know if you guys are familiar with LocationTech.
14:46
But it's sort of this newer initiative to bring together a lot of spatial projects along the same lines as what OSGEO does. And Geoscript is actually in incubation for LocationTech right now.
15:02
We're still very early stages with it but we're definitely looking forward to that. So we've got about five minutes here. So we're going to sort of talk and show you some sort of real world examples. Jared sort of hinted at this before. Do you want to do?
15:23
Okay, sure. So we have two real world examples. The first one is sort of like a typical GIS analyst spatial analysis type of an example. And my goal was to take the West Nile virus data from the CDC and actually create an animated map. So this exercises several things in Geoscript.
15:41
You're being able to read and write spatial data and then being able to render it all in one library. So I went to the CDC to actually find this information. The information is in a PDF which is great for printing. But it's not so good for scraping. So one of the really key selling points of Geoscript is it is on, it targets the Java virtual machine.
16:03
So you can access in a Geoscript script all the Java libraries. So any problem you have there's probably five libraries that will solve it for you. You just have to pick one. It's an embarrassment of riches. So we're using iText to actually extract the text from that. And then there's some really boring code to actually turn that extracted text because it's not pretty into a CSV file.
16:25
But this is what the CSV file looks at. And then we're downloading data from Natural Earth. So this isn't using any Geoscript. This is just using Java features and groovy features to download the shapefile.
16:42
And then we actually want to join all of that data on the counts of West Nile virus incident per state per year. So we read in the state's file. We create a new layer. We write all the features and then we actually down here at the bottom, we
17:00
actually iterate through the features, find the data in the CSV file and then attach it. So this is updating shapefiles in place. And then finally, the interesting part, actually drawing those maps. For each year, we're going to create an image, creating a gradient style which is one line of code using equal interval.
17:21
And we're actually using ColorBrewer, which that's what the reds is. That's a ColorBrewer style that Mike Bostock talked about earlier. We create the image with our map. And then this part I thought was neat. I wanted to draw the year on top of that image. So we provide really nice APIs to wrap geo tools, but you can always drop down to Java and use nitty-gritty Java 2D stuff.
17:47
And then Justin wrote a really nice animated GIF. I mean, everybody loves animated GIFs, right? So he wrote really nice support for that in our library. And so this is what you end up with.
18:02
And I put the stamen watercolor base map underneath it because that just makes demos nicer. There, it ends. So real quickly here, sort of a second example. And this is actually something that was actually done by some folks at the USGS.
18:23
They were experimenting with some of the geoscript stuff to do some processing. So this is kind of a simplified version of it. But basically, the idea here was to take some protected lands data and basically do a classification over a specific area. And this is sort of what the application looks like, but I can actually show you a demo.
18:45
So what this is doing here is it's using the scripting support in GeoServer. So GeoServer has a processing extension, WPS. And so not only can you serve up maps and serve up data from your server, but you can actually run your own processes on it.
19:07
And so it's kind of a perfect use case for geoscript because often processes are very sort of ad hoc. You want to be able to prototype them quickly. So it's kind of a nice match. So I'll show you some of the code here.
19:22
Actually, I'll cut to it now, but this is sort of what a process looks like in GeoServer. This is in Python. I'm going to gloss over a bunch of the details here, but it's about 38 lines and it's showing off some intersection, fancy stuff.
19:42
So I'll cut back to the browser here. So the idea is I add my protected lands overlay and then just draw a polygon on the map. And then I get a nice classification right there. And this is all being done on the server by that process I just showed you.
20:04
Pretty snappy. So again, the chart is using D3. So again, this is sort of what a process looks like. You define a run function. You have to define some metadata about inputs and outputs.
20:23
And this is really sort of part of the WPS protocol. And then very simple, I can access my data. So I have this protected area data set. And then I basically just take that polygon that the user draws on the map, iterate over some features,
20:42
do an intersection, simple area calculation, and sort of aggregate the results. Sorry, I'm missing the very bottom part there. So yeah, pretty easily able to sort of extend GeoServer through scripts, which is really nice because you can do the same thing in Java, but you have to recompile everything.
21:01
You have to restart the server anytime something changes. Scripts, the way the script engines work, it's all loaded on the fly dynamically, as you would expect from a scripting environment. Is that it?
21:21
And yeah, so that's it. Thanks everybody for listening. I think we have a few minutes for questions. And if you ask a question, please wait for the gentleman with the mic to come around to you. So just raise your hand if you have a question.
21:40
Up here. Oh, one there. How much of a performance hit am I taking for going with scripting? Good question. It's really dependent on the language. So with the Python one, you actually take quite a bit of a hit. So, you know, compared to C Python versus Jython, there's quite a noticeable difference.
22:04
You know, other languages, not so much. Like the Java version of Ruby is actually claims to be faster than the C version of Ruby, so it's really dependent on the language. But sort of crossing the barrier from Java to the scripting engine is fairly minimal.
22:25
What's the level of feature parity between all the different versions? Because you mentioned Python, Ruby, Groovy, and JavaScript, and I have to believe that there's at least some level of difference. Yeah, and there is. So going back to that first slide, the Groovy implementation,
22:40
which Jared is responsible for, is the most complete. Python is fairly close, and then JavaScript, there's a few things it's missing, and then it drops off pretty fast. So one of the things we actually want to do, and this is part of the location tech incubation, is really standardize on that and come up with a version scheme
23:02
that indicates, hey, this is how complete this implementation is, and really just kind of get on people to add the features to get them up to parity.
23:22
So yeah, you can just go to find out more or download software to play around. Just go to geoscript.org. And yeah, thanks.
Empfehlungen
Serie mit 2 Medien