OGC API standards for geospatial data cubes
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 266 | |
| Author | ||
| License | CC Attribution 3.0 Germany: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/66495 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSS4G Prizren Kosovo 2023107 / 266
10
17
23
44
45
46
47
48
49
50
52
53
80
84
85
91
110
116
129
148
164
167
169
173
174
181
182
183
186
187
199
202
204
206
209
215
241
248
265
00:00
Flow separationWeb-DesignerProgramming languageUser-generated contentReflection (mathematics)Server (computing)Cross-platformCategory of beingVideo cardInterior (topology)Integrated development environmentSoftware development kitModule (mathematics)SoftwareSubsetObject-oriented programmingLevel (video gaming)Formal languageGraphical user interfaceComputer animation
00:52
MereologyPerimeterModul <Datentyp>outputAreaLevel (video gaming)BuildingBlock (periodic table)Core dumpWeb 2.0Connected spaceFunction (mathematics)ConsistencyLatent heatCategory of beingField (computer science)TesselationDifferent (Kate Ryan album)Set (mathematics)Vector spaceProduct (business)Dependent and independent variablesSocial classParameter (computer programming)Computer configurationImplementationDialectClient (computing)CuboidSingle-precision floating-point formatMoment (mathematics)BitServer (computing)GeodesicMappingProcess (computing)Computer animation
04:22
Range (statistics)Field (computer science)Time zoneLanding pageMereologyTime domainChainingNumbering schemeoutputFunction (mathematics)Extension (kinesiology)Time zoneFunction (mathematics)Product (business)CubeConformal mapRange (statistics)InformationMatrix (mathematics)Field (computer science)Partition (number theory)Greatest elementAttribute grammarInformation retrievalLibrary catalogAuthorizationImplementationServer (computing)TesselationDeclarative programmingDomain nameProcess (computing)Physical systemDiscrete groupFreewareQueue (abstract data type)Volume (thermodynamics)Category of beingHierarchyDescriptive statisticsDialect1 (number)ExpressionMereologyFocus (optics)Wave packetRaw image formatBlock (periodic table)Client (computing)Mathematical analysisChainMappingLanding pageMultiplication signThree-dimensional spaceSlide ruleQuery languageStapeldateiAuthoring systemRight angleDemosceneSystem callLogicGraph (mathematics)File formatOpen setDimensional analysisInformation securityLevel (video gaming)PressureConnected spaceHexagonPentagonContent (media)Buffer overflowWeightRepresentation (politics)Type theoryOrder (biology)AreaArithmetic progressionImage resolutionSubsetAddress spaceMechanism designNumbering schemeCache (computing)Computer fileAdditionMetadataSoftware frameworkVector spaceArtistic renderingRow (database)GeometryoutputSocial classElectronic mailing listNetwork topologyBit rateUniform resource locatorComputer animation
13:04
Relational databaseSynchronizationSteady state (chemistry)MetadataSource codeDemosceneServer (computing)Thomas BayesUniform resource locatorQuery languageEndliche ModelltheorieImplementationCubeLevel (video gaming)Repository (publishing)MetadataStack (abstract data type)MathematicsDifferent (Kate Ryan album)Limit (category theory)Computer animation
14:14
Open setView (database)Element (mathematics)
14:31
Hazard (2005 film)File formatoutputProcess (computing)MereologyMenu (computing)Natural numberSingle-precision floating-point formatView (database)System identificationGraph coloringNeuroinformatikStapeldateiFeedbackProcess (computing)Set (mathematics)MereologyLevel (video gaming)PredictabilityType theoryVirtual machineHazard (2005 film)Wave packetImplementationoutputMathematical analysisFunction (mathematics)Multiplication signVisualization (computer graphics)Slide ruleResultantSynchronizationRemote procedure callEndliche ModelltheorieComputer animationPanel paintingProgram flowchart
16:40
PredictabilityForestGraph (mathematics)Process (computing)Computer animation
16:59
FreewareJava appletINTEGRALTesselationRoutingComputer animation
17:15
Core dumpSocial classMereologyNetwork topologyImplementationFlow separationData exchangeProcess (computing)Device driverParameter (computer programming)SubsetQuery languageFunction (mathematics)File formatSampling (statistics)Content (media)Image resolutionMultiplication signConformal mapLevel (video gaming)Extension (kinesiology)Real-time operating systemConnected spaceMappingComputer animation
18:06
PressureLevel (video gaming)Web 2.0Source codeComputer animation
18:21
Filter <Informatik>outputDuality (mathematics)Filter <Stochastik>String (computer science)Level (video gaming)Mobile appDerivation (linguistics)Category of beingNeuroinformatikMereology
18:44
UsabilityMetadataExtension (kinesiology)Slide ruleTesselationMultiplication signDimensional analysisComputer animationEngineering drawing
19:00
Core dumpOvalRight angleGreatest elementTime zoneSquare numberRemote Access ServiceSet (mathematics)Matrix (mathematics)Time zoneIntrusion detection systemComputer animation
Transcript: English(auto-generated)
00:08
Hi everyone, I'm Jerome Sairoui from HRA. I'm also a co-editor of several of the OPC-API standards and I'm the lead developer of the Gnosis geospatial software. So I've implemented a map server, a 3D viewer, and a software development kit
00:23
that implements most of the OPC-API standards and other OPC standards. And also I'm the lead developer and founder of the cross-platform HRA SDK which is a 2D and 3D graphics engine, a GUI toolkit, an IDE and also a programming language called C, which is a superset of the C language
00:41
with reflection, modules, properties, object-oriented features, and it's all native and compiled. I've been developing all of this since 1996, which means a lot. So what I want to talk to you about today is how basically all the OPC APIs are building blocks.
01:01
You've probably heard that before, but they're building blocks in many different ways. So those are the four ways that I see how they are building blocks. So first, the OPC-PIs reuse existing and defined web API building blocks which are reused between the different specifications to improve consistency.
01:22
So examples of OPC-API building blocks are perimeters, like for example the bBox perimeter to select an area of interest. Another example are the schemas and responses. So if you return the same thing, you try to do it the same way in a different of the CPIs. So one example of that is how JSON schema is used to describe the properties
01:44
or the fields of a particular collection of data. JSON schema of course is also used in a meta way to describe the schemas that are returned as well. And GeoJSON is another example. So the GeoJSON schema is to find a response for an OPC-API features response, for example.
02:04
But it can also be used in other APIs. And finally, another example of a web API building blocks are resource paths. So for example, the slash collections are used in all the OPC-PIs for data access. Slash map can be used at different places.
02:21
For example, for a map of the whole data set or a map of a single collection. Tiles can be used in different places for vector tiles, coverage tiles, map tiles, tiles of a collection, tiles of a data set as a whole. So these are the examples of web API building blocks. And this improves consistency.
02:41
Another example of how OPC-PIs are building blocks are requirements classes. So the OPC-PIs follow the OPC modular specification. Which basically says that requirements are organized in two modular classes. And you only need to implement the core requirement classes to conform to the standard.
03:01
And then you can also implement optional requirements classes. So this allows you to progressively implement more and more functionality in your products. A third way is how you can take all these different OPC-PIs standards, like OPC-PIs features, OPC-PIs maps, tiles, and you can actually implement them all in the same web API at the same endpoint.
03:23
And also, these APIs are built on top of each other. So several of the OPC-PIs have a dependency, whether it's mandatory or optional, on the OPC-PIs common standards. So at the moment, part one is approved. And part two, which in my opinion is a little more interesting one,
03:42
is what goes with slash collections and slash collection slash collection ID. And all of the OPC-PIs that access data follow most of what is defined in the standard that's still work in progress. And finally, a fourth way of how the OPC-PIs are building blocks is you can actually connect one API to the other.
04:02
So an OPC-PI collection can be an input to an OPC-PI process, for example. And you can connect them either locally on the same server, or you can also talk to remote APIs. And again, using the APIs themselves as the connectivity between these different APIs.
04:23
So all of this allows us to basically give us a framework for doing geospatial data cubes. And this is my attempt at presenting all of how these different standards work together to give you the functionality you need in a geospatial data cube. So first, starting from the top left,
04:42
for the API description, we have OPC-PI common part one. So it gives you the landing page, API definition that's often defined using OpenAPI, the declaration of conformance. Then from there, from the landing page, you can go to the list of collections. So that's the collection of geospatial data.
05:00
That's what's defined by the draft common part two. And basically what you do there is you can describe basic information about the collection, like attribution, a title, an ID. And you can also go and describe, for example, if you implement also the records for a stack, you can also integrate functionality to catalog products
05:23
and scenes, add additional metadata, extra fields. And also connecting to this is an upcoming extension to OCPI features that probably we would like to make common, which describe the properties or the fields of your collection of data.
05:40
So that's the OCPI features part five, I believe, for schemas. And we've been discussing recently to also use this in coverages and all of the other data access APIs. So you have a consistent way to describe your fields or your properties. So that's describing your data queue.
06:01
So one thing I forgot to mention here is as part of the collection description, you can also describe the domain using the extend property. And recently we also, as part of the published OCPI tiles and the OpenAPI schemas, you'll see there's a way to add a grid property that
06:21
allows you to describe a multidimensional grid of data, whether the grid is regular or irregular. So this allows you to describe the domain. So you have the domain, the range, or the fields, the basic info. All of that is handled by that button left block. So yes, all of this connects to the data queue. And then one of the most important functionality for the data
06:43
queue is to be able to retrieve. And you usually want to retrieve the raw data. So you can do analysis on the client or you can use it as part of a processing chain, for example. So there's three APIs that allows you to retrieve the data in an arbitrary rate. So there's OGC API features that's published and OGC API coverages that's still work in progress.
07:03
That basically allows you to define an arbitrary subset of what you'd like to retrieve. So you select your area, your time of interest. You can have extra dimensions like the pressure levels. And you can make your request, and you get your data back. And you can use HTTP content negotiation to negotiate net CDF, coverage JSON, geoJSON,
07:24
whichever format are supported. So that's where the raw data. And then OGC API maps. The purpose of that is to send you a visual representation of the data. So basically that's if you want to do server-side rendering or if you have, let's say, a single static map that you
07:42
can retrieve. And again, there's a whole bunch of requirements classes in OGC API maps to adapt the functionality how you want. So if you rather than for more efficient requests, if you use a predetermined partitioning scheme,
08:01
it will be better for the caching, everywhere between where the data resides and the client. So this is where the OGC API tiles come in, which is a successor to WMTS. And basically OGC API tiles work with rectangular zones that follows the OGC 2D tile matrix set and tile subnet data standard.
08:24
But with OGC API tiles, you are not limited to map tiles. You can also get coverage tiles and feature tiles or vector tiles. So that allows you to do client-side rendering attached to raw data. And this can also be used as part of processing chains, for example.
08:40
Then if you prefer to use more complicated access mechanisms that do not follow rectangular zones, like the discrete global grid systems, some of the discrete global grid systems actually are axis-aligned. If they are axis-aligned, normally it's possible to define
09:02
2D tile matrix set for them, and you can actually retrieve the data using OGC API tiles. But if your GGS is, for example, ICF3H, which uses hexagons and pentagons, then you need the GGS data retrieval requirements classes, which basically allows you to retrieve data for a zone that is not rectangular.
09:25
And another functionality in the GGS is to be able to say, where is my data? So for a collection or for a particular query, you want to get a list of zone ID to know where the data is. So that's the GGS zone query. And finally, if you have 3D data,
09:40
and there's 3D geo-volumes, which allows you to retrieve things as a bounding volume hierarchy, for example, using 3D tiles, or also if you want to use a fixed bounding system, where you first determine how the 3D space is styled, and then you'll retrieve whatever data is in that particular zone, then you have an extension of the 2D tile matrix set,
10:02
which makes it very similar to OGC API tiles, but you can retrieve data, for example, in a GLTF to retrieve data. So that's for the predetermined partitioning, the maps. So if we go up in the middle center, so for advanced processing, there's OGC API processes part one,
10:22
which basically tells you how to execute a process that's already defined on the server. With part two, you can upload your own process. So this way, if you have authorization to upload something to the server, that can be executed, then after that you can make part one call to execute that data. If your processing need is not that advanced,
10:43
if you can express what you'd like to process using a simple expression with arithmetic and logic, for example, then there's a way to use CQL2, the OGC Common Query Language, to basically express what it is you want to compute. So an example of that is an NDVI, for example,
11:01
that you'd like to compute, so you can express your NDVI expression and you can derive new fields from existing ones. And another use of CQL2 is also to filter the data. That's actually where it's being introduced in OGC API features part three, where you say, I want to review the data, but only the data that matches this particular filter expression.
11:23
So that's for the expression processing. And finally, the workflows and chaining on the top right basically is kind of a glue between the data cube, the raw data access or the collection of your spatial data and the processing part one. So when you execute a process in the execution request,
11:43
you can say the input to this process is not just like a URL or a base64 encoded data that I send you, but it's actually an OGC API collection. So then the data can be retrieved on demand from the coverages or features or tiles of the GGS API, so that basically you don't need to back-process everything at once.
12:04
And whenever you do a request at the end of the chain, so it's client-driven, that will trigger a whole chain of process, focus on an area, time and resolution of interest. So that's what process for tree collection, input-output, allows you to do. And also it's ad hoc workflow execution
12:22
in that when you define this workflow, you don't necessarily need to deploy the workflow first. You can just describe it and execute it right away. So this might be easier for implementations to provide that if they don't have an advanced authorization system, for example, because the process has less security issue
12:41
by only using processes that are already installed and approved, and then it's just connections between them. And one alternative way, instead of the part one processes, part one execution request, to be able to describe the workflow, is the OpenEO process graphs. So that's the big picture overview. I'm probably almost out of time,
13:01
and I have about 100 more slides, but we'll try to put them quickly. So this is one data cube that we have is the Sentinel-2 level 2 bay from ESA that we host on our server. So there's a URL here where you can go to and play with most of these APIs that I've described.
13:21
So this is source from stacked metadata, and probably we want to add improvements to be able to list the scenes that are inside of the collection and maybe also providing our own stack implementation with SQL queries. And something we're very interested in and hope to discuss this week
13:41
is a way to efficiently synchronize basically different repository of stacked metadata because I found that retrieving millions of scenes is very slow. So if we could synchronize that, only the changes, and also use a relational model so we avoid sending the same thing all over again,
14:00
it would really improve things and it would help us keep our data cube up to date. Right now I think it probably goes to 2022 sometime when we actually fetch that data, but now there's a whole year or more of new data that is not there yet. So this is how we basically, this is the whole Sentinel-2 coverage that we have, and so we support down-sampling
14:20
all the way up to a global view, and this is from the AWS open data that's managed by LMN84, hosted on the AWS. And this is a view in basically natural color, near-infrared, to the single identification map, the NDVI computation.
14:41
We're also doing work in the OGC pilots on wildfire hazard protection, and basically this is one map from the Copernicus Emergency Management Service, and what we're trying to do is machine learning, so training, and then prediction for vegetation fuel types, based on the Sentinel-2 data.
15:01
So this is OGC type processes, part one in a nutshell, so I won't have time to really go into the details, but the slides are there for those who want to go through it. So basically this is synchronous execution, OGC API processes, part one also supports asynchronous execution, part two is deploy,
15:22
replace, update, and it workflows and it's chaining. The goal really is to instantly integrate two spatial data and processing capabilities available from anywhere for visualization and analysis, so bring the processing and the data together no matter where they are, so all the OGC API implementations basically acts as one giant worldwide API that all self-connects
15:43
between all the parts. So geoprocessing, workflows, hurdles, you have very long processing time and it's difficult to bring the sets together from different origins, so this is really what we're trying to solve, and some of the drawbacks of lengthy batch processing is by the time the results are computed,
16:02
they may be out of date, it's difficult to prioritize, there's a long feedback loop, and it's not an efficient use of the resources. So it's part three, it extends the part one execution request so you're able to chain one process as the input to another, you can have local and remote OGC API collection, you can modify the input and output
16:22
with the CQL2 for example, you could parameterize it to deploy it as workflow, you can have a collection output so when you request data from the output collection it triggers the processing, and you could do ad hoc execution with OpenEO and see the model. So this is the details of all of this,
16:41
I'll just skip through. This is some example of a crop classification when we use a scikit-learn at Random Forest to do crop classification prediction, this is what the workflow looks like, and this is like we're just requesting one request from that prediction, and that triggered the classification process. This was with the 5200 Java PS integration
17:03
for the routing server, and then we were visualizing this into Q-SUS and GDAL, so you could actually see the routing calculated, and it's actually using tiles to request the route, I believe. These are some of the implementations of Part 3, and Tom, I'd like to talk to you,
17:20
and Stefan about the Part 2 API status of this, I don't know if it's there yet. In GDAL and QGIS there's an OGC API driver that supports Part 3. So basically we're plan-chaining, easily connect data from anywhere, trigger on the processing, minimize the data exchange by requesting subsets and downs of sample data,
17:40
so a real-time resolution of interest always, as OGC API maps are very simple, get map resource, get the map back, there's several other conformance classes for more functionality, many query parameters, format, so all of the OGC API support with HTTP content negotiation, so there's some conformance classes
18:01
that define some formats, but you can always add your own. So there's some extensions that are planned, this is trying to show 4D data, so basically we have the pressure level, temperature at 500 hectopascal, and very quickly to the end, so relative humidity, so we have all of these data sources on our web, on our API,
18:22
this is showing what you can do with filter maps, so using filters SQL2 with OGC API maps, you can only get one country, or you can say elevation greater than 100, coverages quickly, this is the interesting part here where you can do the filter
18:41
and the properties to compute the OGC API properties, very quick overview of OGC API tiles, please read my slides when we have time, skipping through tiles very quickly because I had all the details, this is extending tiles to three dimensions, there's three different ways, you can read that through the geometric set standard,
19:02
the GGS is a way to just browse through the data using zone IDs, and this is saying where is the data and only getting zone ID back here, and thank you very much.