Open Standards for Big Geo Data

Video in TIB AV-Portal: Open Standards for Big Geo Data

Formal Metadata

Open Standards for Big Geo Data
Alternative Title
Geospatial - Open standards Big geo data
Title of Series
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date
Production Year

Content Metadata

Subject Area
Satellite Pixel Multiplication sign Range (statistics) Bit rate Solid geometry Mereology Data model Medical imaging Envelope (mathematics) Square number Endliche Modelltheorie Abstraction Area Covering space Service (economics) Point (geometry) Gradient Internet service provider Bit Measurement Open set Degree (graph theory) Type theory Data model Message passing Sample (statistics) Angle Conformal map Spacetime Ramification Implementation Service (economics) Image resolution Virtual machine Average Regular graph Graph coloring Field (computer science) Goodness of fit Term (mathematics) Energy level Integer Software testing Implementation Maß <Mathematik> Domain name Suite (music) Tape drive Consistency Volume (thermodynamics) Basis <Mathematik> Cartesian coordinate system Inclusion map Word Basis <Mathematik> Point cloud Object (grammar) Computer worm
State observer Trajectory Pixel Multiplication sign Range (statistics) File format Solid geometry Web 2.0 Data model Medical imaging Virtual reality Type theory Different (Kate Ryan album) Extension (kinesiology) Social class Physical system Covering space Service (economics) Mapping File format Temporal logic Open set Connected space Type theory Data model Process (computing) Internet service provider Right angle Data structure Server (computing) Service (economics) Overhead (computing) Geometric modeling Computer file Link (knot theory) Codierung <Programmierung> Computer-generated imagery Canonical ensemble Discrete element method Binary file Metadata Number Codierung <Programmierung> Data structure Focus (optics) Information Surface Physical law Point cloud Computer worm
Three-dimensional space Implementation Functional (mathematics) Pixel Service (economics) Identifiability Multiplication sign Canonical ensemble Average Mereology Discrete element method Semantics (computer science) Metadata Dimensional analysis Subset Power (physics) Time domain Data model Latent heat Military operation Core dump Energy level Cuboid Computer multitasking Diagram Endliche Modelltheorie Extension (kinesiology) Installable File System Curve Service (economics) File format Electronic mailing list Coordinate system Core dump Two-dimensional space Open set Subset Arithmetic mean Exterior algebra Function (mathematics) Statement (computer science) Fiber bundle Object (grammar) Identical particles Extension (kinesiology)
Service (economics) File format Multiplication sign File format Time series Core dump Average Open set Subset Time domain Function (mathematics) Computer multitasking Object (grammar) Extension (kinesiology)
Slide rule Three-dimensional space Functional (mathematics) Implementation Transformation (genetics) Multiplication sign Time series Computer reservations system Average Subset Time domain Medical imaging Profil (magazine) Operator (mathematics) Core dump Energy level Computer multitasking Extension (kinesiology) Partition (number theory) Metropolitan area network Domain name Service (economics) Scaling (geometry) Core dump Cartesian coordinate system Open set Subset Particle system Function (mathematics) Cube Self-organization Fiber bundle Pressure Extension (kinesiology)
Server (computing) Implementation Functional (mathematics) Interpolation Service (economics) Identifiability Transformation (genetics) Multiplication sign 1 (number) Time series Average Mereology 2 (number) Medical imaging Latent heat Profil (magazine) Core dump Computer multitasking Integrated development environment Selectivity (electronic) Codierung <Programmierung> Extension (kinesiology) Physical system Social class File format Computer file Moment (mathematics) Projective plane Analytic set Core dump Bit Cartesian coordinate system Flow separation Open set User profile Subset Type theory Revision control Right angle Musical ensemble Bayesian network Communications protocol Extension (kinesiology)
Satellite Group action Presentation of a group Multiplication sign Range (statistics) Combinational logic Database Parameter (computer programming) Repeating decimal Mereology Demoscene Semantics (computer science) Dimensional analysis Variable (mathematics) Web 2.0 Medical imaging Array data structure Visualization (computer graphics) Core dump Process (computing) Endliche Modelltheorie Extension (kinesiology) Multiplication Physical system Point cloud Service (economics) Theory of relativity Mapping File format Temporal logic Range (statistics) Attribute grammar Database transaction Bit Mereology Open set Array data structure Process (computing) Raster graphics Green's function Energy level Resultant Geometry Slide rule Implementation Server (computing) Service (economics) Identifiability Computer file Codierung <Programmierung> Disintegration Mathematical analysis Parallel computing Web browser Canonical ensemble Average Graph coloring Metadata Power (physics) Number Frequency Goodness of fit Latent heat Musical ensemble Implementation Installation art Operations research Server (computing) Weight Database Subject indexing Visualization (computer graphics) Query language Predicate (grammar) Revision control Point cloud Musical ensemble Table (information) Integer Window Extension (kinesiology)
the Stanford I would like to tell you a little bit about standardization in the big geodata area myself I'm Peter Boland and from the consumers resident Germany I'm using OGC ISO and other areas in that field so I how I wanted to bring some technical backgrounds on that and hopefully soon with questions and discussions afterwards to exchange and get out the message and also learn a and get new requirements may be so coverage is the key word uh that is defined in the use standards and that is what I want to bring to you the coverage data model of OGC and after that the service model which is to a distinct separate things and this has ramifications beyond OGC meantime walls in so inspire and others and I want to put a brief glance on that 1 and also a little bit to get into the implementation details good so we all know that future the you graphic object and we know that coverage is a special kind of a future 1 could lose a classifier that as some space time-varying multidimensional phenomenon and in practice that means we talk about regular grids like images about irregular grids about point clouds and about measures so that's actually what typically contributes the Big Data in terms of volume at least and no so that is the kind of things that I wanna talk about so colleges the scuttlebutt technical solve on you as level it looks like that we have a specialization of feature that is the coverage and but coverage is defined by the domain set that is where do the values whether the the range set that is the pixel payload and what we have added over the tumor definition is the range type that is the pixel type so that you know what you're talking about so that RGB is not just a bit integer but it's uh gradients in some spectral ranges and has some other values and all of the things that are contained in here also there is an annotator hope so that you can plug in any other stuff you want to transport I would like to contrast this to the definition of as a 91 2 3 some of you may know that 1 it's the mother or the coverage definitions in ISO but is an abstract model so if somebody says to you I'm complied with ISO 2 1 2 3 and therefore an interoperable role this is so abstract that you can have many implementations and there are many implementations out there there are definitely not and over this 1 the coverage definition actually of what you see is an and operable 1 and we have conformance test that can check a coverage down to pixel level whether it's consistent by the way you will often find this named geometric cover that is a convenient shorthand defined because 0 she's he had the idea to name this she and and 3 . 2 . 1 application schema for colors which is the thing that anybody wants to pronounce OK so geometric cover the common so that can look like that just to throw a few angles few X in that time that you you have a great coverage of their um the envelope and gives you an idea of where this is in a WGS 84 we know that it's two-dimensional with flat long waits is measured in degrees and then we have the bonding books then all the gory details come and from this and love you can already see her roughly where is I'm range type definition as until told you has a little bit more than just a bit integer it has for example but it is a panchromatic channel in measured in watts per square centimeter this is fixed syntax so that can be evaluated by machines OK that's what it can look like and I mentioned geometric America but please don't assume that this only can transport x and a GMM data XML data but that would not make sense you don't want to do that with satellite images you don't want to do that is with weather forecasts this is just a model which is formalized enough because we have finally validators so that we have a solid basis but of course we can encode coverages in any other suitable for much and that's the next part so
we could do the whole thing in Gmail of course and sometimes you like to do that when the times years for example perfectly fine sometimes however we want to have you want to have as some other former decoded special formant magnets CDF like if they can contain more or less of the metadata so you may lose something when retrieving that format but hey you wanted it like that you know what you're doing and sometimes you want to have both you want to have canonical metadata in XML class the payload encoded efficiently in a binary format this is where we use of multipart mime the method like in your e-mail attachments where we have a whole thing in XML is defined except for the range set which consists of an X link it to the data file coming later so actually we are quite flexible in transporting coverages in different formats and we need to because actually have a coverage types that we have are quite diverse typically we think about the right hand sides so we have pixels and we have a quadratic pixels so something like lawful images that of 1st of all this can be spatial-temporal we might have irregular grids that we might have a very strange it's like they're like that to have in climate modeling and the other will doesn't stop we have point clouds called . coverages trajectories surfaces and solid bonds and all of that this is where we close actually the gap to geometric modeling like city tumors and things like that so this is not full reinventing the wheel but forgetting that and a connection there's a into these OK so that was a very brief glance at the data model other so that the services that we have on this again let me start over if I UML picture because that actually describes nicely what the server has in mind the map coverage service is that service which is focus most on the cover structure you can serve coverages by anything as as well by web features service by the processing service by sense observation service whatever but this has the most functionality as we will see in a minute so what does this overhead in mind we have a college offering that has some so metadata per what coordinate systems do I support what data formats do I support and stuff like that then you have a bunch of coverages which are number 1 the coverage is as we have seen them but also again we foreseen slots to hold any other descriptive data which may be related so service provider or some WCS extension they want to store additional information here good once again this can be any encoding here and so we have actually something like 1 virtual document and now we can get our request types and
fire them against this conceptual model so the get capabilities request which is the standard canonical 1 assess what Service extensions what formats covered uh coordinate systems to be support and a list of all colleges we can easily spot that in our diagram we get the top level box from the coverage we just give the headline the identifier the for this crap coverage request he withdrew down into a particular college and get the metadata so you get to the college part but without the pixels and you get the service metadata and finally get coverage there were of course is the 1 where you get the coverage or a subset of it that means no Madrid down and get exactly this 1 so with that conceptual model we can very clearly say what we want to uh what service means and we get a clear semantics definition yeah OK so what does it do in the end on before we go into the functionality I must mention that 0 seen at some time has decided to establish a core extension model is is called or uh modular specification of a model that means we have a clear way to distinguish implementation alternatives that was really a pain in the neck for everybody for us this cation riders and for the implementers and for the users In the old WCS wondered x specification I counted and incidentally are found 63 if statements their normative statements so tool to the 63rd power is the implementation you have good luck what we do now is we have no ifs in 1 specification but we have a bundle of specifications that you can pluck together much better to overseas so we study the course but is the 1 that actually does nothing but you see the coverage or a subset of that's
already quite useful so if should think of this as being a prime serious x y and t I here you'd could get everything about this place a long time this is called trimming it keeps the dimension so three-dimensional subset from a three-dimensional object two-dimensional subset from a to the object slicing on the other hand reduces dimensionality so you may want to know a temperature curve over to Brussels for 1
year gives you a one-dimensional time series
or you may want to have a particular time slice what is the weather like in year and this particular day you can combine this arbitrarily trimming and slicing and 1 request and say you can retrieve any subset that you want and you can do that in any format that matches um OK a 3 D object you cannot transport and PNG for example so we have some restrictions but that's practically motivated and we know it anyway so this is the
only thing that the core does give me a coverage or a subset of it in a particular formant the extensions null and in front of functionality and add further facets this is what I will show in the next slide but 1st let me mention that we have a 3rd level that is application profiles implication profiles are a bundle let us say for particle application domains so the question is for an implemented man I want to do something for remote sensing data or what extensions should I supported reasonably so we have an you know WCS that says OK you should support their scaling you should soaps support CRS transformation and by the way we have some extra uh um some of extra functionality which allows you to search in remote sensing imagery OK same thing is under ROC for mid-ocean and is uh and work has started now for stanzas so that is just some packaging some bundling I small incentive there is other standards doing that as well and uh 1 in particular I want to mention is what mn 2 . 0 that also uh going into time serious how ever it will not be efficient what mn is perfect for 1 dimensional time series so you have a serious with temperature with a pressure with whatever if you plug in images here you can do that and you get a three-dimensional data cube but what happens if you do slicing perfectly fine if you want to get a time series can out uh then you're pretty much lost because you have to touch upon every 5 and have to extract that's extremely inefficient where is the coverage concept is more abstract we don't say anything about the internal data organization and therefore an implementation can do that in a clever way like for example doing some partitioning in there which is suitable for the kind of operations we want to do we can be flexible and adaptive which is what we are actually
doing in our implementation for example as to the whole thing if there is a little bit simpler to model so time series is not always equal to time series knowledge me come back to the big picture that is actually the big picture of colleges in the coverage service so it may seem complicated but actually it is let's walk through it so we have the data part where we have the geometric comforting I introduced before and several format encodings that describe how to map let us say coverage to the whole thing and no I won't step in the nice right the um depends on some other specifications and the coverages are used by the poor so we are entering the service part and that we have several columns again some of them are open and waiting for ideas of what we could additionally specify the most important thing is functionality here so we have an extension that allows to update them using the lab data coverage we have processing extension for analytics range subsetting for band selection scaling serious transformation interpolation these are the function class has been on implemented can decide whether it wants to plug in this or not the service in the get capabilities will announce which extensions its support to know exactly what you're can't be a service you contact when it can offer to you the then we have encodings of the usual suspects here get KBP pose the old ones uh soap arrest and uh possibly change the future so that is just the transport protocol nothing special and then yeah floating down here you find the application profile OK so looking into the request for a moment just an vendor that would so that's what I get cover this looks like a simple as that request it was good coverage then coverage idea equals you get it you get it in the native format as stored on the server you get in the native projection system are stored on the server it's as simple as that if you want to do subsetting then you say so for every exists so longitude and latitude and maybe time the notice that be allowed to use calendars here some people say we should use seconds since they poke which is not exactly what I find convenient if you want to have another formant you specify that by way of the mind type identifier so that is sent as well and that's the get auditory crest as prove core by the way a core mission over a WCS is not to deliver images we can do that too but the main mission is to deliver data that are unchanged so if I
get bathymetry I want to know what that is and therefore i in the specifications mandated that values are delivered unchanged except of course if you ask for something like J peg the OK then you wanted but if now we have an
extension is simply adds new parameter combinations like you for exam for the range setting you can request the red band or you want to do a false color image near infrared to red green want to transpose colors or you want to have intervals or all of that combined so that is the syntax weights by extracting from climate variables or from hyperspectral and the same thing it works actually with all of those extensions so the transaction for example I want to insert a new coverage period is to take that next CDF file generated a new identifier for me and that's it so so much for showing the schema of how the extensions like themselves into the core 1 extension is very dear to me because that is the analytic part gets to processing actually we have a specification for that called Web Coverage Processing Service which is nothing but a clearer language for across the data with Q semantics their lives OK the looks like that's all it may remind you of exterior and actually recovered with X community so they have metadata data search you can filter some predicates akin to processing and recur recur in the whole of thing good that fits nicely into the big picture that we have so as for upstream stanza capturing prove anything that you want to transform it into the standard canonical is the data formats and then serve adults downstream wire that w something services and that allow for all those things that users want to do OK now of 1 implementation powers which happens to be core reference implementation is the Rastaman system which is like a database system but it doesn't need a database underneath it it kinda of advisers well add spices of SQL with the race so something like that it's a little bit as the syntax so before but it's no real SQL but that has been implemented that works operational installations up to 130 terabytes datacubes with the and the queries have been distributed over 1 thousand cloud nodes let me skip that 1 and so we can generate visualizations here like redhead terranes breaking with queries that we put in relation the offer channel and the result is something that actually the a GPU can understand directly that is where g l and you can have a 3 D in a presentation on your that in your web browser window parallelisation I mentioned already that he's made clear is I'm sorry I have to be fast enough as always have too many slides and but I promise a little bit about so so the 1 thing that's going on now that has started a few months ago is that OTC colors standards are transposed into ISO standards that's a common thing happened with web map services geometric and with others and that's the time has discovered the 19 1 2 3 needs a revamping so actually the whole thing is getting shaped up now and so there is an additional activity loosely related eyes in 19 1 6 3 0 4 a satellite imagery where we tried to convince people that it makes sense to streamline that with uh the coverage model actually they agreed the 1 thing that I would like to mention particular however because this is so dear to me is that to be as good working group after 2 and a half years of discussion has agreed that we add a support to the SQL query language and so this is what it will look like you can define that in a table your large arrays whatever size whatever number of dimensions and then you can use that in queries like getting vegetation index and stuff like that so how many ministers have left minus 2 yeah OK so then let's skip this 1 and just have a summary slide arising of what I have said already so in not plug-in stop here and the sorry for this speeds talk and so looking forward to your questions