Serving earth observation data with GeoServer: addressing real world requirements

by there are many amazing words

indicating the user and the we're going to talk a little bit about how to say it's not going to be on the approximation it's a special component and so it's going to call it a little bit also Meteorology Consumer graphic they must have that as the additionally mentioned by finding vision had we vote were

produced solutions it's semi that in Europe so we can use the word as it needs a small company OK we do work mostly in the EU savages server community but also we do work on other things like where seeing what not so that they about lots metadata especially in Europe in you the world is like you know about so it's have this other bookie June of people be in there last 1 year and not of BNP in working with the constant like it is for the opening up that so we have to work with the schools inside the schools outside the schools except except in the presence OK

this is the usual use cases that without the work we we use this terminology I would like like this that is like a use of the 1st that like that is going on in that environment that that is where these things come from that is recognizable environmental feature common operational picture goes 1 day the mediator is always there is a lot of the and they usually diffuse for because that's why you need help with the logic interacting models expediency observation as well as the remote sensing because you want to compare work-in-progress and what is actually going OK you can get some some

basic about the image was like this was like his our basic tool to

provide In this was liking and multidimensional of of 1 the and and the data sources so Panama's was like is just a collection of granules images that are put together by an index the index contains the location of the files that the bounding box and then there several attributes that you can from from typically dimensions so time innovation but you could have and that's actually quite handy

in you can query whatever average there

are a few assumptions working animals will it so do you be of the requirement that to have all the granules holding it is in the same color model so all RGB or agree or collected but that that has been removed in situated now

you can mix different color models to some reasonable extent like you cannot mix of digital additional In a satellite images of the same time they have 2 different in images of 0 to 10 which is going to be losing October 2016 we're also going to remove their their limitations and that all according to the resistance of same thanks so whatever here that the work and so we can have the encouraging news was I can different production good can overlap as they please it's not a problem you can control the sticking out there so there is a way to to score title by data or by a resolution or by whatever other to be the think that can drive the importance of of a granulosum should In the 2nd and can be in different file formats the index the index is what makes them more like take it is normally just a table containing all the information about the granules typically it's a implemented by using Digital's vector sources so could be a shape prior and posterior historical or Hsu each 1 has its own advantages of it and these advantages chip ideas is really really using is what we used by default if you just throw them in this was like after the are 3 unintelligible well OK give me and was like of whatever it is inside is gonna was I could using the shape of but you can also set up some control configuration files and making it the most like starting a database which is handing that you can index the other views maker for fast searches and it also means that you can drive the most like by the databases the so you can in just about the new grounds in it if you want to the dimensions that I

mentioned my too often I attributes in the index so

typically have dates times and numbers but it could be pretty much everything so we can have also customer mentions this medium besides the basic time innovation and as I will show later they can be advertising negative abilities documents of the OGC services so that clients can discover them and they can discover their domain so as to make queries on on the

multidimensional data typically

when we played with the data sources which are not easily multidimensional so we're not talking about an entity if you're about a bunch of due to files each 1 associated with a particular innovation in time we have to get out of using information from someone like the time duration and so on typically in these environments those information coming from the finding itself they could be embedded in some violated by that's that's less common so we have a bit of considerable machinery that you can set up to extract by means of is out of the defining using regular expressions

but once you have those set opinion was I can you can go in the user

interface User again that enable the publishing of time innovation and customer mentions so here we have a snake that out of the the villagers document showing at the time and then we have a long list of times which of the available ones and then on innovation and the list of reservations and then we also have some an updated on a 5 customer mentions that we added and it that you can also inspect and use in the get mapped requests the fun stuff

about originals like was the

fun stuff from well the mall's liking to get the images in space and on In in time or mission is fine it's interesting but we can do more in the masking for example so we have supported both masks in vector and has the form that it can be digital style of binary mask embedded in the file or they can be sigh profiles in the world Yorkshire from and if you're not happy with that you can plug in your own API as a must provide to get in the last for or whatever why do you want to use must well on will make you a few examples 1 is compression may be your compression is ruining the date of the borders between like areas and validate areas and wanted to retain only the good part and you can't do it anymore by just saying black is going to be transparent because of the border of compression artifacts of monkeys and other cases your Sutherland company of sorts or you're getting aerial images of sorts and the you got flares and clouds in your name your images you don't want to show that you use masks to cut them out of the images on the right on the flight yes of course coverage of

user it's an art things that we do with I extend the

machinery in general so sometimes you have data sources which are a bit complicated like an insidious angry that and they can contain multiple phenomenon registered in the same file and sometimes you get this In this case the difficult case of wind and files that they contain amplitude rosters to separate trusted whichever you and the component of the wind and you if you follow strictly they would use more than you would have to publish them as 2 separate layers and that's not very useful because they are related and it is the relation but luckily carries useful information so we have a tool called covers view in this sort of parallel to the sequel views that we have in the way back toward in which we can put together should balance the 2 sort of occurrence G as students of 1 culture and up and up point that we are publishing with just

1 later which events that we can then use to create wind barbers and the like so this both was elucidation and

and as I said that the index can be queried fooling so if you just want to use the as the so they would you see standard extension for multidimensional you can and it's fine but you're limited to say I want at this time they want this elevation what if you have a more active use or you wanted to express more complicated queries well known in just over we have an extension that allows you to just put us equal fulfilling the given up request right whatever where you want to try in this case I'm making up a case in which I say OK give me anything that's in the index where the sensor is a star and the satellite is this particular suddenly and I will get it and liking only the ground was the satisfy that requirement so you get a lot of power you can literally build a very complex more with several data sources and then fuse the dynamic and what you want to see depending on the user there access rights and not this was like is

also quite interesting that it's a very portable also if you are the development and that it's not satisfying your

requirements out of the box there are a number of extension points that you can plug in to write your own little jar you're only the library that adds extra functionality to it such as an extra ways to collect contributes out of the granules figuring out what coverage each granulysin to the process and the granules deciding whether or not a particular renewal is part of them was like a part of a particular college and was like are there other recent

work about managing together image using different coding of differences is done by a particular so southernmost I producers so that's the part where you can put together the edges and the cost you can build your own and I don't for example we don't we don't do image merging like we don't the other registered pixels we just what's top stays on top and what sort of you know it it becomes even but we could build a for example something that marriages together being medicines by some sort of alpha blending and that would be in use of was like we have a couple of which represents the index it's a way to access it and and that's also avoidable and we have a few implementations there and someone is going show some examples where we are actually existing legacy catalog which is implemented as a service and it's not to be usually non-CS

possible to small OK if you're still not sleeping history because nothing together will the actually assigned the wrong probability we should talk about the formats before and then the mosaic afterwards physical mosaic is the structure on top of which you can actually produce separate asserted that other people complex form so that the body of the wondering why it's the sophisticated before the of the world everybody sophisticated the point is that we try to reduce the time it takes to to go go from there the serving the interests of the means should be read from the repository of all of most of the various formats of there actually pretty complex again use the word is that if you ever worked with that severe really year although not accessible you can find everything people claim to the Convention's compliant meant at time you find new things that actually more or less compliant but nobody about using them that way so it's always up to something new that's why the mosaic sits there configurable it's that sophisticated because it allows you to in the we is originally in a preprocessing you but I found that it was so it is especially important when you work with

the example and suggest that that uh as you might know there is some formats like the fact the

standards that they believe is like for example let's the the notion a although its use also put some atmospheric modern Greek for uh metrological there he was each year for 5 formants sensing except except that can the is is that the number of format that people are used they the really want to reprocess them and they're really most of the time because actually all they're not man for observing that at the most demand for shaping the because they contain a lot of the not XML files from the accident information they can be used reports so if you know that it's

a DFE is think about what is entered the logical model generic model think about the forecast for temperature it is something I have said they focused on multiple iterations next year for people to actually contain those in put all these there in a single file you can get a huge files into their bikes that contain forecast for like 7 7 days 6 this is usually not more than the 6 3 hours 12 hours so multiple innovation it's many most of the time it's a huge number of people smoking degrees so the point is not about preprocessing that a lot because agrees that you're going to use values of small but it's about being can find the inside this so that's what is the only thing that would be naturally that is the most of the time it's trying to to make sense that's a flat and the structure of this file and we can find in the world we worked on this support for that CBS News was funded by the alarm and actually due solutions a couple of years ago In the past always going from angry benefits tools stack of duty files but as I said was taken the website so we actually be the bullet then decided to to go there support nets if in order to get the supporting that means like speed in all the languages in the world so we I ask you to old a your work properly if you produce them you can your models was if you want to and please support the courts mesh if you don't know what conventions that I'm not going to explain it's actually it doesn't you should start to unify that can understand what's in the file without having to call you the time which is the baseline for for us year's convention which is the most widely used its climate for a kind of what the mosaic is and what was it does it fit a member of the talked about the going that's it goes from a structure of a these multiple attitudes to something like that so when you ask is diamond is that they meditation to another mentioned for them to read redouble division uh other thing which

we did was actually it's sort of posts their 74

breaking the that was you so that you could have only a single college from a single bite or if you want a single cell because as you as this certain most of the time in this files being

logic convince you have multiple colleges if you can have hundreds of them and then use colleges to create new ones by inmates in them and turning different you physical parameters in the back and that's what you want to do when your current fields we guidance that see things that you needed to balance the data you want to do my friend like we infer a these in the past was actually requiring preprocessing was separately create file with the the lens so that except that was taken I want talk about this does is a full model of the and the show you this

simple example the mosaic but if you want to support for there is a very nice XML configuration file everybody lots of and so when you can actually the edges that exactly which files which you physical parameters you want to be served out of the original file most of the time of the file you

will warning want use it for you and your physical parameters not all of them usually out of that is controlled by parameters so you really don't want to serve them but instead of asking people to

preprocess them or reprocessing didn't get rid of them you just use everything not the

so powerful coordinator this is that it's not that I talked about convention this 0 you specify or don't exist in city of file using this year from and so if you follow the convention we stored all the definitions so if can create system on the fly without having the

mutations that we have the

conventional sense and there's not much more but I the behavior of the

basic support for Mr. that allows us to actually turn this complex next to the F. regional and talking about it's mostly single although it's the formats that is a set thing overlap of the 2 models so this the same criteria variants of them the image mosaic is actually use the ones who have made sense of being done structure book of law you to create a long time series of this that usually models sort remote sensing you actually acquire seeded with over time it's a constant from that so once you continue this so you can keep up the use is usually politically forecasts

most of the time you the 1 that which is older than 3 7 days than a long-term archive it's usually not useful but serving with remote-sensing that's the thing because intractable online so that they belong to and so I'm not saying that people turn away methodological scientific models but they usually not that useful after a while actually used from all multiple times a day so you get multiple forecasting you know what you want to use the pressures were for a single it but

it will take a bit of time to explain in detail how you can use the XML

definition but actually that induced over all to extract information from a prominent suggest it's again another concept model that that's a complex it's already vis-à-vis together you can actually define the beach dimensions you are going to support time innovation you can additional dimension in this case we actually for because of the truck me keep track of when the fire was 1st generated when it was up to because it's on the practice of those so read on the model right away but they provide

and they are actually the edges several OK obviously information from the files for them in activities of these and expose only these variables using the

domain 74 so what you end up having as on the social media is actually Europe there the

plaster the metadata in and out of these rules that describe each single slice out of your data and that's what you users will serve as the process knowledge and then you it aggressively get

marked time addition exhibits we use the index to restart the machinery to get a better from the original form so if you know when talking about you don't from all 2 governments in the In all that it takes the more than 40 mixing we created the if you arrest operations because as a set is usually a constant flow of data so once to set this up you just want what the government data and result is that the I 2 is a partition of the data according the index updating the data and deleting data so once this flow

so that you don't have to go as I

said support for agreement by partially to because we use the same libraries so as the size of the library and then you get better than

support that yeah so once you have set up your answer your data you're contiguity you passive playing with it so a few

examples of how about what kind of request you can make and so here is again map with the time innovation and to customer mentions of striking a particular slice of lived out of the multidimensional moles like we

can when when we display when when we have this kind of rest data sometimes we we I mean it's it's a good to to display based on a false color Oracle map but sometimes you want to extract the features so we can use a rendering transformations which are an extension of DL we added induce server and that can that has been

particularly well optimized for performance and then you can use it to to extract from the fly from your estimator contour lines of code fields and when mobs and more complete this system is completed pluggable so if there is a on the fly transformation that you are missing you can implement just drop conducive over on the fly and is the is the actually got bigger Europe mosaic right we don't preprocessing is that this and we about some of the of the of the development of the computer the center of us and the same goes for computers in the past you actually don't remember the produ is computed all these things you needed so speaking more time with people is in forgetting the max in Inland parameter that was especially with mythological models 1 and when operational sometime around among them but usually they're gonna accepted as socially it can be the simplest the right models sample along time so you cannot be 5 minutes when you on right with this way we don't skip the preprocessing analytic it's right away published and you can also customize on-the-fly some model the parameters of transformations so we also

supported the limits your for a complex set of observation problems and their derived products so it's a it's a way to advertisers the structure

of the product and also the advertiser must like cloud mask or water masters and the like which could be a roster vector

the protocol is rather complicated so would be the URI to help the building there the problem 3 which is only a range server but but in a particular 1 particular type

and once you have a set it up you can just review it as normal but I by default you will get to that the browsing most of it and then you travel a client about the use of your comfortable they will be able to extract the most of your products we can also download

the data of course we implemented policy to going on and we have a full implementation of the protocol the protocol is really really pluggable so that the basic implementation can just do

crops over the data without any anything else but that user implements also all the other extensions of can risk a lot project and also control how you are including the outputs the so this is

described covers output which

is a let's a similar to the GEP abilities in the limit so you get that standard summary of the top which is a 4 the bounding box where the space elevation and time described in terms of limits and then we add them that the protocol allows for custom sections in the output and we leverage started to enumerate the time innovation and customer mission values so that you can fully discovered the of course this is user over all of specific extensions you will need at some clients for understanding

our OK this is a an insidious extraction so you can also if you have and data you can have an end the output in this case a making for the bounding box aquarium I'm getting out of a hypercube of data and will say that was

yes your which adds that

some extra metadata against again similar to that of a messy on going to get into the that's about that

someone is not going to talk about some real world use cases we will the way we use the

initials A. T. it's usually quite sophisticated and we tend to interpret these with the existing infrastructure goes 1 thing is the preprocessing the other thing that is most of the time people would have their own catalog information unthinkable genetic makeup of life it's where they actually store information about whether they were these information about the dynamics of except for that's why there's is all those extension point there was a was can actually applied customer behavior that relates to to your infrastructure is a simple use cases we integrated there uh it's actually a graphical application that we did for the for clients where they sell off Sicily imagery and processing individual probabilities updated so cheap and the quality action all the point

is that all that information and the information about where the rest of the day is coming from the legacy of

and the image mosaic and refer 6 of examples in the real world but relying on the information is coming from this external system to feel the images and we created the customers and such as manager so security is customized data block of the rules which they so people

like integration is actually a much

deeper integration there was there for combining itself remote sensing imagery like a lot of images of although we cannot use the name and was out secondly there right but the customization it's very deep because they the permission scattered in the legacy system and its information to the user it so restructuring something on a lot of structure anything was not the possible but we were able to produce the image mosaic on top

of it using the extension on talked about before so the kind of information aware that it plays a minute that accounts directly on the storage information is filtered according to the user and if it's only rules it's coming from the we need a customized to the collapse so the user model as a

standard user we broke the customs story that

actually it's used also for the fact that certain where using of the 1st information about you and this is

something I would not describe what is actually a fully fledged everywhere right and user a client x except it's actually just about its validity and it's it's actually quite sophisticated use case there is a collision used uses as a library as opposed to you as as an application and then you were talking about the availability and use of advanced research within the prohibition phase from the capitals and from the just and is room like

this and hi I was using the image mosaic previously to publish a large amount of satellite data with large coverage and takes problem richer experiences once you have the high resolution data museum altered of of course that's like just a whole lot of data and by that time it didn't support as far as a member of having permits within an image mosaic is that something you support now there will be a key with the user was the trade off is between all many fights you need to answer a single request and of course you are even if they're completely optimized the

fear of 2005 that was

the in that case you probably would

elect recommend having multiple layers of the belt that's what like 3 different layers of 4 difference in order cook and OK other questions then they can take it kitchen I would