An OGC Api to get geospatial data
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Alternative Title |
| |
| Title of Series | ||
| Number of Parts | 295 | |
| Author | ||
| Contributors | ||
| License | CC Attribution 3.0 Germany: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/43348 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
FOSS4G Bucharest 2019235 / 295
15
20
28
32
37
38
39
40
41
42
43
44
46
48
52
54
57
69
72
75
83
85
87
88
101
103
105
106
108
111
114
119
122
123
126
129
130
131
132
137
139
140
141
142
143
144
147
148
149
155
157
159
163
166
170
171
179
189
191
192
193
194
195
196
197
202
207
212
213
214
215
216
231
235
251
252
263
287
00:00
Coefficient of determinationStandard deviationJSONXML
00:18
2 (number)Presentation of a groupSheaf (mathematics)Content (media)Data storage deviceMultiplication signLink (knot theory)Lecture/Conference
00:47
Content (media)Table (information)Time evolutionProjective planeSlide ruleSheaf (mathematics)Link (knot theory)QuicksortContent (media)Software developerPresentation of a groupBitBit rateComputer animation
01:35
Content (media)Table (information)Time evolutionFile formatWorld Wide Web ConsortiumModul <Datentyp>Extension (kinesiology)Core dumpProcess (computing)Standard deviationCollaborationismVideo trackingProbability density functionComputer virusProjective planeSoftware developerKey (cryptography)Server (computing)Process (computing)Uniform resource locatorArithmetic progressionClient (computing)Web applicationStandard deviationExtension (kinesiology)Group actionRow (database)Domain nameWeb 2.0Software design patternPlug-in (computing)World Wide Web ConsortiumLatent heatParallel portLattice (order)Similarity (geometry)Core dumpLevel (video gaming)Analytic continuationDemosceneElectric generatorBlog1 (number)Point (geometry)Network topologyMathematical singularityComputer animation
06:27
Computer virusCore dumpSoftware frameworkPlug-in (computing)File formatDefault (computer science)AbstractionConfiguration spaceMetadataMereologyWeb 2.0Term (mathematics)Line (geometry)Field (computer science)Core dumpData typeStandard deviationFront and back endsDefault (computer science)Open setElectric generatorMultiplication signRow (database)QuicksortPlug-in (computing)File formatTime zoneDebuggerConfiguration spaceDifferent (Kate Ryan album)Commitment schemeHecke operatorFunction (mathematics)Server (computing)Web serviceLattice (order)Software testingWebsiteSynchronizationSoftware frameworkAbstractionExtension (kinesiology)Connected spacePhysical lawOffice suiteProjective planePoint (geometry)1 (number)Pi
10:04
MetadataConfiguration spacePlug-in (computing)Software frameworkInstallation artAbstractionCore dumpDefault (computer science)File formatKeyboard shortcutArchitectureElasticity (physics)Core dumpSoftware design patternExtension (kinesiology)Archaeological field surveyComputer animation
10:37
Keyboard shortcutShape (magazine)File formatVector spaceAttribute grammarAxonometric projectionQuery languageComputer-generated imageryContent (media)Server (computing)Level (video gaming)Pointer (computer programming)Point cloudComputing platformWeb serviceGeometryPredictionMetric systemImplementationMessage sequence chartInformationTestbedDistribution (mathematics)Dependent and independent variablesDigital filterDifferent (Kate Ryan album)Instance (computer science)File formatQuicksortInternet service providerProxy serverKeyboard shortcutScaling (geometry)Workstation <Musikinstrument>GeometrySet (mathematics)Entire functionDependent and independent variablesLatent heatFood energyBitWater vaporProduct (business)Web-DesignerContent (media)Web serviceSearch engine (computing)InformationProjective planeComputing platformImplementationFile archiverMereologyTheory of relativityComputer programmingFunction (mathematics)Codierung <Programmierung>Interactive televisionVapor barrierLinked dataServer (computing)Core dumpReal-time operating systemTime seriesNumbering schemeGoogolResultantMetric systemSoftware testingInformation systemsGroup actionVideo gameStandard deviationWeb 2.0Computer animation
15:38
Digital filterDependent and independent variablesContent (media)Level (video gaming)Online chatEmailLink (knot theory)Demo (music)PermianMappingLatent heatType theoryTesselationDifferent (Kate Ryan album)NumberStructural loadEmailMechanism designInstance (computer science)Projective planeWeb 2.0Software developerElectronic mailing listSearch engine (computing)Tablet computerLink (knot theory)QuicksortWaveGoodness of fitWeb-DesignerInteractive televisionComputer animationLecture/Conference
17:11
Library catalogLine (geometry)Predicate (grammar)View (database)Multiplication signCombinational logicQuicksortRevision controlProxy serverGoodness of fitHookingExtension (kinesiology)Query languageInteractive televisionWaveFormal languageCartesian coordinate systemElectronic mailing listFront and back endsType theoryWrapper (data mining)TesselationPlug-in (computing)MappingRootGrass (card game)PiArtistic renderingDifferent (Kate Ryan album)Web pageLecture/Conference
Transcript: English(auto-generated)
00:10
Are you first? Welcome, everyone, to this second talk of this session on Python tools and OGC standards.
00:21
So for the second talk, Tom Kradidis and Paul van Hunuchen, if I didn't destroy your name too much, are going to talk about the PyGeo API, which implements the new OGC API. So I guess I'll just let you go. OK, thank you, Daniel.
00:44
Thanks, everyone, for attending. So we'll try to get through this as fast as we can. There's a bunch of content in this slide set. We're going to skip a couple of sections in the interest of time, but the presentation is online. If you go to pygeoapi.io, there's a link called Presentation, so you can sort of look
01:02
at this on a Friday night and look at it in full. You need to get out more, Tom. I know, eh? So at any rate, we'll walk through a few different sections. Paul will start off with some of the OGC activity, and then I'll talk a little bit about the actual project and features and so on.
01:22
So Paul and I are presenting. There's other developers in the room, so Just Vandenbroek is mainly contributing on a lot of the OGR sort of back ends. And then Angelos, who most of you know, works on a lot of the packaging and deployment side of the project.
01:41
So we're happy to have both of them here. So if you have any questions with regards to that stuff, they're your people, and they're core contributors to the project. So I'll move it over to Paul to start us off.
02:11
So let me start with a question. Who read this document? Who of you has read this document? That's too little, huh?
02:20
This is a very disturbing document for our community. We're not used to the terminology in this document. So this document was created in 2017 in a joint OGC working group of OGC and W3C.
02:40
And so a lot of people from the web world, search engines, linked data, came together with people from OGC and the spatial domain to say, hey, we're disconnected, but how can we reorganize this so we will be connected? So this document is based on data best practices
03:02
on the web, which is also a very recommended read. So based on that document, some new movements started within the OGC. And all working groups were, of course, reading the document. And that led to kind of new design patterns
03:25
within that world. So being webby was mentioned in all OGC meetings. Being developer friendly, having a lightweight specifications for development, removing HTTP tunnel, and moving into APIs, URLs
03:43
like this instead of the long key value pairs. Idea was to start with a small standards and then develop plugins and extensions on top of that.
04:01
So this was kind of the recent years that we had. So it's very hackathon oriented, as you see. It started with this document, spatial data on the web. And then there was a white paper. And then there was a hackathon in 2018 where some of this what we are presenting was born.
04:23
But probably it was born in your head before. No comment. No comment, yeah. And this year we had a continuation of that hackathon with a lot of progress. And in between, of course, there was a lot of discussion on Gitter and on the various channels.
04:40
And so we'll be presenting some of the outcomes. There's more? Ah, yeah, there's more. So here we're talking mostly about OGC features, previously known as WFS3. So we started out with WFS3.
05:02
But at some point, this led to an OGC had a similar parallel discussion that led to this OGC API common specification. And that one required that all those new APIs would be called OGC API something. So then we had to rename WFS3 to OGC API features.
05:25
Coverage is coming, maps, styles, processing, records. So that will, in the beginning, will start to live next to the other ones. Who knows? At some point, we'll replace that.
05:44
Very OGC, very GitHub oriented, very open, Gitter, Webby. So you can create issues and pull requests on GitHub, even without being an OGC member for most of the standards written in ASCII doc
06:00
and released as HTML documents. So even the standardization process itself is very Webby. Thanks, Paul. So everything you saw, you can believe what you see.
06:21
This is really happening. Things are really changing in OGC. And it's getting a lot easier to either make servers or clients or web applications using this new generation of OGC standards. So we'll give you an overview of the project itself.
06:40
I look back on the commits. And the first commit was on Valentine's Day last year. And yeah, my wife wasn't happy about that. But I have to buy her something on the way home. So we like to think of it as a geospatial data API framework, which you can put any kind of web front end
07:03
or RESTful front end API front end on top of it. It's already OGC compliant. So as part of working in these hackathons that Paul mentioned before, we are working with OGC and also the site team, neck and neck, if you will. So the site test for WFS3, which
07:24
is the first standard that PyGOO API officially implements, have been really helpful for us to cross the line in terms of being fully compliant. We're already an OSGO community project, which we find valuable to be part of the OSGO ecosystem. And there's a bunch of different people
07:41
from all sorts of time zones. So the time and date meeting planner is a very valuable tool in our team. So there's a lot of contributors. And you can see we stand on the shoulders of a heck of a lot of tools and people that have contributed. The technical overview is it's a core abstract API.
08:02
So you can actually use PyGOO API on the command line to interact with your data using the web front end as just an extra layer on top. So that was by design. That didn't happen by accident. There's a server configuration where you can define server service metadata, layer connections, and so on.
08:20
That's in YAML. With regards to supporting open API, we generate the open API document before we deploy a server. So if you have the configuration and you have five or six different layers, if you will, or data sets, the open API document generator will go through each of those data sets,
08:42
pick up all the fields, what their data types are, and make them part of the open API document. So you can know exactly what the fields are in every record and how you can query them and so on. We have a really, really robust and powerful plug-in framework. So we have a framework which allows
09:01
you to create a plug-in, any kind of plug-in, and basically extend as you wish. So we have plug-ins for data connections. So as I mentioned, Just worked on an OGR plug-in, which basically opens the door to a whole multitude of formats. I wrote the Elasticsearch plug-in for the back end. And George, I think, who's not here,
09:24
but he's part of this project. He implemented, I think, the spatial light and a few other ones as well. We also have plug-ins for formats. So the default output is GeoJSON and HTML. That's sort of core. We also have a plug-in for CSV output.
09:41
So if you want the data as CSV, so you can sync up any format you wish and work with it as you wish. It's easy to deploy. I'm not sure how long the install takes. We'll need to time that as well. But you can do it through PIP or Docker. It's on the Ubuntu GIS unstable line now,
10:04
thanks to Angelos. And it has minimal dependencies. So it can get really simple, or if you need to install more complex dependencies, you can do that as well. But the idea is following the design pattern of what's going on at OGC, which is core and extension. So it shouldn't take you very long to stand up
10:22
a basic PyGeo API to serve out your features in accordance to the core spec. There's a graphical overview of what we just described. There's a different data providers, as I was mentioning.
10:43
So as we mentioned, I just worked on the OGR Python binding so that opens the door to all sorts of different formats. So that makes it really flexible. There's also the capability to use it as a proxy for your existing WFS one and two instances.
11:01
So don't feel so bad if you have them. If you do need to expose them as WFS three or OGC API features, you can just plug them in this way. It's also easy to deploy. So as I mentioned, there's Ubuntu GIS. We're working on including it in the OSGO live package, which some of you may have heard about.
11:23
Obviously, we have Docker support. And it's very easy to download the images and run them. There's some examples there. So there's, after running the Docker, you can see test.
11:40
Those are WFS three commands, basically. So as you can see, very straightforward to interact with a feature server now compared to what it used to be. And that's by virtue of the spec. So here we can see an example of some HTML encoding.
12:02
So as I mentioned, the spec has HTML as a core output. And thanks to Paul and others, schema.org has been implemented to facilitate search engines. So now you can go on Google Data Set Search and scrape through an OGC API feature.
12:20
So again, Webby, lower barrier, web developer friendly, all lowering the barrier to extend the reach of the data. More Docker. Which allows you to sort of scale things out as you wish, as you can imagine that you can do with Docker.
12:43
So we already have some production instances. I'll talk a little bit about the one that I work on. So as part of the Meteorological Service of Canada, we have a platform called GeoMet, which is all of our geospatial web services. So we include data on real time weather,
13:03
climate, and water data. So we implemented, we deployed PyG API as part of one of our climate services projects. So it exposes all the Canadian historical hydrometric and climate archives right to 150 years back kind of thing.
13:25
And you can hit that URL and interact with the WFS3 and get a time series for an entire station of hydrometric data, water levels, all the way back throughout its lifetime. So there's also something called the Global Soil
13:41
Information System, which George and Louis worked on, are working on. That's part of ISRIC. So they're using PyGeo API as well for their, for disseminating their data through features. And they've done some really neat work, if you go to that URL there, with JSON link data.
14:01
So you're able to open up a feature and see its relations and so on and so forth. So really innovative and interesting way of implementing the OGC API spec through PyGeo API. So where do we go from here?
14:21
So this project is a year and a half old or so. It had a lot of, it had and has a lot of energy. I'm happy to see all the people in the room. From concept, from the initial thinking to having something on the web, it took us maybe three days,
14:40
which is really thanks to the simplicity and the straightforwardness of the new OGC specifications. So again, I can't say enough about the new OGC API efforts so I encourage you all to take a look at those and follow those and that's basically gonna be the future of standards-based geospatial information exchange.
15:01
As a result, it was very easy for us to initially do an implementation. So obviously, we have plans to add more data providers, so more backends, if you will. Content negotiations, so I mentioned that you can get features in GeoJSON or HTML. We're looking at adding responses to provide a geo package
15:24
or other formats. In the OGC, we're working on an advanced filter specification as part of the new program. So the PyGeo API project is following that to see what the filtering capability is going to be. It's gonna get a lot simpler than the OGC filter XML
15:41
that some of us have been working with for better or for worse. Somebody's laughed, that's you, right? Yeah. So some of us are showing the pain of working with that but at least we can laugh about it now. And more APIs, so like we said, the OGC API effort is spanning all the different types
16:00
of specifications, maps, tiles, processes, coverages, and so on. So our goal is to try to implement as much of them as possible and to have somebody deploy a PyGeo API instance and load it with coverages, features, and spit them out in a really easy search engine, web developer-friendly way.
16:21
Support. So there's a number of different support mechanisms. Obviously we have folks like Geocat who are helpful and have been participating on the project which is more than appreciated. There's also Joost who we referred to before.
16:40
Angelos, so there's a number of different support mechanisms if you need development or deployment support. Here's some links, so basically everything's there. Take a look at what you will. If you have any questions, there's a mailing list as well as a Gitter channel. There's an online demonstration.
17:05
That's pretty much it, so I'd like to thank all of you for coming. I'd like to thank everybody that's contributed to the project. This is, again, as I mentioned this morning, the new wave of OGC and the interaction with OSGeo is a good sort of synergy
17:21
and we're happy to be involved and exciting times are ahead. Thank you. Thank you, Tom and Paul. We have a few minutes for questions, so we'll start with you, Mike.
17:44
Do you know what the first few planned extensions are going to be? They are talking about the query extension, so they need to define a query predicate language. Is it the first one? That's one that's being actively worked on and there's a OGC Hackathon in November,
18:04
so that's one of the extensions that they're talking about, that I know of anyway. Will you be able to support the old CSW and the new one against the same data provider?
18:25
That's a good question. So for something like WFS, we do have a wrapper which allows... Or the catalog only. Or the catalog only. I think the catalog only, this will have a... Well, initially we're going with catalog four,
18:41
but we can implement something that basically proxies to a previous version of the catalog too, so that's possible as well. Other questions? Up. And I'll go there next.
19:04
At the moment, will it be easy to import the PyGeo API into existing Django application and publish the API as a Django view? Good question. So the way it works now is there's a, there's sort of a Python API,
19:22
and then on top we have Flask on top of that. The Flask implements basically the endpoints, which are really small roots, and sends it over to the PyGeo API API. So you can just build a Django view on top of that, which is very minimal. All you have to do is define the roots,
19:41
which are very simple and very minimal, so in around 100 lines you can put a Django front end, and then you can hook it in that way. Yeah. Yeah, please contribute, yeah. Thank you. About WMS protocol,
20:02
do you plan to let different rendering engines to develop plugins to act as a rendering machine, or do you intend to implement your own rendering tool, because there are QGIS, GeoServer, and so on?
20:23
That's a good question. We're still, a lot of good questions, we're still sort of thinking through that, like on the Python side, like you say, there's a lot of different rendering engines out there that we can use, so how that actually looks is to be determined, but I mean, we're not gonna reinvent the wheel,
20:43
if there's something that we can just build off of, then we're just gonna use that, so. But we're also waiting on the OGC maps and tiles to evolve and see what that looks like, so. Anyone else?
21:06
Yeah, you said you have an Elasticsearch backend, or is there any plans to implement a PostGIS backend? Sorry, I neglected to mention that there is a PostGIS backend. Okay, I hadn't seen it on the list. Yeah, yeah, yeah, George is gonna kill me.
21:21
He implemented it, yeah, so there's a PostGIS backend. Actually, there's a PostGIS, and then there's a PostGIS plus PostGIS combination. Anyone else? Well, thank you, everyone, for coming, for joining,
21:42
for having, there's definitely interest for that type of stuff, as you can tell from the room. So the next talk, thanks. Thank you.