Raster Data In GeoServer And GeoTools: Achievements, Issues And Future Developments

Video thumbnail (Frame 0) Video thumbnail (Frame 590) Video thumbnail (Frame 1300) Video thumbnail (Frame 2427) Video thumbnail (Frame 3247) Video thumbnail (Frame 3791) Video thumbnail (Frame 4594) Video thumbnail (Frame 5346) Video thumbnail (Frame 6171) Video thumbnail (Frame 6761) Video thumbnail (Frame 7447) Video thumbnail (Frame 8913) Video thumbnail (Frame 9943) Video thumbnail (Frame 10785) Video thumbnail (Frame 11482) Video thumbnail (Frame 12360) Video thumbnail (Frame 13261) Video thumbnail (Frame 14159) Video thumbnail (Frame 15325) Video thumbnail (Frame 16323) Video thumbnail (Frame 17644) Video thumbnail (Frame 18572) Video thumbnail (Frame 19412) Video thumbnail (Frame 20186) Video thumbnail (Frame 20790) Video thumbnail (Frame 21802) Video thumbnail (Frame 23395) Video thumbnail (Frame 23945) Video thumbnail (Frame 26064) Video thumbnail (Frame 27031) Video thumbnail (Frame 28118) Video thumbnail (Frame 29475) Video thumbnail (Frame 30175) Video thumbnail (Frame 30816) Video thumbnail (Frame 31368) Video thumbnail (Frame 33651) Video thumbnail (Frame 34704) Video thumbnail (Frame 35326) Video thumbnail (Frame 38753)
Video in TIB AV-Portal: Raster Data In GeoServer And GeoTools: Achievements, Issues And Future Developments

Formal Metadata

Raster Data In GeoServer And GeoTools: Achievements, Issues And Future Developments
Title of Series
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date
Open Source Geospatial Foundation (OSGeo)
Production Year
Production Place
Portland, Oregon, United States of America

Content Metadata

Subject Area
The purpose of this presentation is, on a side, to dissect the developments performed during last year as far as raster data support in GeoTools and GeoServer is concerned, while on the other side to introduce and discuss the future development directions.Advancements and improvements for the management of raster mosaic and pyramids will be introduced and analyzed, as well as the latest developments for the exploitation of GDAL raster sources.Extensive details will be provided on the latest updates for the management of multidimensional raster data used in the Remote Sensing and MetOc fields.The presentation will also introduce and provide updates on the JAITools and ImageIO-Ext projects. JAITools provides a number of new raster data analysis operators, including powerful and fast raster algebra support. ImageIO-Ext bridges the gap across the Java world and native raster data access libraries providing high performance access to GDAL, Kakadu and other libraries.The presentation will wrap up providing an overview of unresolved issues and challenges that still need to be addressed, suggesting tips and workarounds allowing to leverage the full potential of the systems.
Keywords GeoTools GeoServer Raster WMS WCS ColorMap GDAL RasterAlgebra
Server (computing) Goodness of fit Computer animation Software
Computer animation Open source Block diagram Line (geometry) Library (computing)
Dialect Implementation Java applet Structural load Projective plane Image processing 1 (number) Set (mathematics) Basis <Mathematik> Drop (liquid) Number Power (physics) Operations support system Medical imaging Process (computing) Computer animation Semiconductor memory output Library (computing) Computer architecture
Operations support system Process (computing) Just-in-Time-Compiler Computer animation Vector space Set (mathematics) Bit Scalability Active contour model Library (computing) Element (mathematics)
Medical imaging Computer animation File format Variety (linguistics) View (database) Source code output Summierbarkeit Ext functor Computer architecture Library (computing)
Server (computing) Data management Service (economics) Computer animation State of matter Forest Energy level Set (mathematics) Mass
Medical imaging Functional (mathematics) Process (computing) Computer animation Vector space Mapping Term (mathematics) Multiplication sign Projective plane Bit Function (mathematics) Graph coloring
Mapping Projective plane Bit Function (mathematics) Line (geometry) Revision control Pi Computer animation Vector space Term (mathematics) Program slicing Reduction of order Self-organization Nichtlineares Gleichungssystem
Medical imaging Goodness of fit Computer animation Vector space Personal digital assistant Auditory masking Image resolution Normal (geometry) Form (programming)
Area State observer Server (computing) Scaling (geometry) Inheritance (object-oriented programming) Code Number Product (business) Revision control Process (computing) Computer animation Doubling the cube Profil (magazine) Order (biology) Website Extension (kinesiology)
State observer Beat (acoustics) Computer file State of matter Multiplication sign View (database) Bound state Dimensional analysis Hypercube Neuroinformatik Medical imaging Computer animation Profil (magazine) Musical ensemble
Complex (psychology) Curvature Computer animation File format Multiplication sign Iteration Endliche Modelltheorie Mereology Measurement Flow separation Hypercube
Server (computing) Computer animation Computer file Different (Kate Ryan album) Concentric Multiplication sign Weight Set (mathematics) Raw image format Sequence Form (programming) Hypercube
User interface Computer program Multiplication File format Multiplication sign Interface (computing) Weight Source code Characteristic polynomial Two-dimensional space Computer animation Single-precision floating-point format Order (biology) Asynchronous Transfer Mode
Computer program Server (computing) Computer file State of matter Multiplication sign Interface (computing) Content (media) Electronic mailing list Set (mathematics) Operations support system Computer animation Program slicing Data structure
File format Multiplication sign Range (statistics) Set (mathematics) Function (mathematics) System call Metadata Hypercube Wave packet Subset Frequency Latent heat Computer animation output Cuboid
Area Satellite State observer Complex (psychology) Beat (acoustics) Group action Concentric View (database) 1 (number) Water vapor Mass Mereology Medical imaging Process (computing) Computer animation Profil (magazine) Point cloud Video game Musical ensemble Quicksort
Area Server (computing) Computer animation Characteristic polynomial Maxima and minima
Scaling (geometry) Computer animation
Bytecode Scripting language Default (computer science) Machine code Computer animation Java applet Code Cellular automaton Text editor Library (computing) Physical system
Scripting language Revision control Computer animation Mixed reality Multiplication sign Moment (mathematics) Flow separation Event horizon Pole (complex analysis)
Computer animation File format
User interface Server (computing) Beta function Computer file File format View (database) Source code Gradient Food energy Graph coloring Prime ideal Medical imaging Computer animation Touch typing Self-organization Configuration space Quicksort Alpha (investment)
Satellite Medical imaging Pixel Computer animation Average Multiplication sign Design by contract Special unitary group Neuroinformatik
Operations support system Medical imaging Photographic mosaic Computer animation Computer configuration Multiplication sign Mereology
Computer file Multiplication sign Image resolution Set (mathematics) Raw image format Mereology Graph coloring Metadata Dimensional analysis Medical imaging Population density Different (Kate Ryan album) Computer configuration Endliche Modelltheorie Data structure Extension (kinesiology) Algebra Mathematical optimization Partial derivative Multiplication Dependent and independent variables Demo (music) File format Projective plane Database Bit Flow separation Vector potential Particle system Subject indexing Computer animation Vector space Website output MiniDisc Right angle Procedural programming
hello good morning everybody my name something I'm and I'm allergic was just over a co-developer work and produce solutions an Italian company that providing and the support for you to just servers network c can have your there is an obvious was just special tools to their here to talk about the recent developments arrested the supporting due to induce a so that's a look at the technology
stack that your server and due to user to deal with the rest of the this
is more or less the block diagram and that we have so there main libraries that we use to build the so albeit just residencies we looked up was you tools which is in turn build uh on top of a few open source libraries dealing with prosody in particular we use j lines of
advanced imaging Jarvis of imaging is a very nice library very powerful very extensible flexible uh that provides a set of abilities to do many kinds of image processing such as the the common ones as clipping rescaling warping images and so on in the Srinivasan in that style basis so that we can handle very large imagery without actually having to load and falling memory but process them and it might be instead dust handle both like very large input from very large up with the 1 thing which is not nice about the library is that it was the book my son and that because it is no more under active development however it is also extensible that uh you can basically
replaced every single we can eat without your implementation just retaining the overall architecture and that's what we did in the JI next project we basically created a set of a drop renal replacement for the JI operations that we we need the power higher-performance pure Java and that support no data data that support the regions of interest that have a number of fixes compared to the base year i in the support masking properly and the well
this is the set of operations we implemented so far so as a said we implemented the more or less what we need you to some just over to the processing and uh we pushed up the scalability of the library quite a bit by implementing some praying implementing some key elements using higher scalability tools from uh
if what's JIT uh is not sufficient to you is what the provides is not sufficient we also use generic tools and you for which are high-performance reciprocity in libraries that implement a new JI operations such as a vectorization contour extraction and also us out of a very efficient us out about that allows us to do some computational Russell later in an efficient and flexible way the uh of course before processing the data we have
to read it somewhere so we have a material which is again a sum library that provides support for reading PNG packs and what not but also very important from duties uh any thousand active acceleration for some of them unfortunately again uh there is no source code the for the native the coders and development that's more or less stopped but fortunately again the architecture is pretty much like a JI 1 so we are
free to replace and improve upon it and that's what we need to demerge 0 ext that provides high-performance trust that your compared to check to image the old provides indicative support which wasn't already supported provides a fast Sergei tag encoding and decoding by to which a pagan by ejido we can support the next revival variety of input and output formats such as just the 4 thousand CW cedar in and so on and so on themselves you tools views on
top of all this uh providing a higher level tools for dealing with the past data to a present the coverage is to represent their georeferencing to project them and so on and by putting together all the tools that we know not I just cited in a sensible whole your
server builds on top of tools to provide the OGC services that you're used to use so the mass that was yes the book assessed on the previous we then go below the 10 Google Maps support so this whole stack provides a very comfortable a set of forest management abilities and that are necessary to the OGC services to do the publishing of the state now let's see
what we recently achieved at uh in terms of new functionality and tools and you server to uh my Council of us
that the process so this is a little pet project to find has last Christmas never was a little or they have some time to go to work and then implemented a new Piaget color which is easily 55 per cent faster than anything that we had before so uh it's even if it's pure job of it that actually beats hands down there uh native the engine called was provided by a material and this gave us quite a bit of speed up any time you have to encode a PNG image output which is common when you are dealing with the vector of the of course publishing mentally but it's also important when you are dealing with the rest of the time because of some of your maps can be digital elevation model and the like and in those you probably want to use this BGO for but we
had there are quite a bit of a push in terms of improving the quality of reproduction of roster some difficult projections here we have an example on the posturographic reduction before when trying to reproject hold war the image so that this is a learning precipitation molded covering the entire toward the uh Gordon to coordinate and before when we were trying to reprojected to tool to posturographic and you can see a a slice of the pie was missing uh there was problems in thinking that I that a solution for the output and this is a new version of the of the output which is constantly equator or has that I solution and everything fits into the we
also added support for cross the date with the rest of it as well so a few the marks about sitting across a dateline commonly New-Zealand but common also in every organization that deals with the worldwide mapping and now we go and we support exactly what we have supported for years on vector related entities replicating the data that you have on the other side of great line so that you haven't seen this map and you can see uh your your map as a whole even if you're looking at the hand of acidic they we had a
support for uh masking any much was i which is very handy in case you have imagery major is partially overlapping with each other and has a bunch of no developments uh you can uh associated to each of image your was I can together a vector mask that tells you just over where the good guys and the user with good accounting for you and it then was like a for you of course this is a useful form was liking but in any
case you want you can also use it to to go pick out you're measuring to ascertain shaping in this case we have a high very high resolution data showing up in they're both on a province in Italy on top of all of Member marble normal data so they actually have the problem was like to have a larger coverage but they only wanted to display the high-resolution data in the region and and their footprints support that we implement allows them to cut to the measuring exactly what they want he just server we uh
implement in during the last year there were covered so this 2 . 0 which I II there say is the 1st this 1st saying a version of Web Coverage Service ever that it's easy to use and can be used by you and being the the has 1 0 and that was just 1 1 were plagued by a very complex so uh syntax to do their request studying the process 2 when 0 we have a very nice simple syntax do most of their normal things like extracting among invoke Service skating name at projecting it without increasing the process
but we implement the number of the scale of the extension so that basically everything that was yes is an extension to our production is an of scaling is is an extension to the coding is an extension and so on so I Bayes that Lucius server actually only implements extracting ascertain area everything else is an extension which us a reminder site or not to implement user basically implements them all in order to be compliant with an order profile which is the parent observation profile which basically demands everything plus something double CSA
all the observation profile uh we implemented this for D a lot the German specialize in C along with the BCS in along with a good support for then and that was yes you always interesting because if you have multidimensional data that is rosters that is not just a flat demand the but it's a hypercube of data in time and elevation dimensions and maybe other hand time of your forecast and so on so it might be 5 dimensions are more uh that Lucius allows you to the advertise the x-ray emission and describe them so that people can build a request the extracts exactly the beat of a hypercube they want to images
several we actually implemented also on your feature which is coverage views which allows you to on take of complex trusted like maybe 7 then last Landsat the image and just decided that you want to 3 bands out to the user or and or taken several colleges that so the that is several files and merge them together as if they were a single give college taking bets from the several 5 so this is the current state of the coverage of use we're going to push apart from Ireland computation of new bounds on the fly uh I'm talk going to talk about it later during the last year along with a double
CS uh you'll effort we made a lot of work to support more coded more complex coverage readers In particular we support so that a city of 100 for us which are you common formats for supporting a hypercube of measuring that is a measure which is not just a part of flat snapshot but which contains light all the innovation at at several times that that sorry not innovation temperature at 7 times now and in the future and that several iterations so that is 3 4 5 dimensional model questions later you and um we implemented so that we can expose acquired
your server uh and we can MOS I keep because of several times you have net CFI files and you have several of them in the form a seamless sequence in time so you actually wanted to put them all together to form of the of all of the hypercube but that is made I actually of little chunks this is an example
of a data set that that we are serving with if it's the Polish famous dataset which contains the concentration of 3 gasses at different elevations in different times and it's 1 of the datasets that we allowances just we to
what where their general characteristics of net CDF angry back and made a general programming interface that you can implement Angelakos you server to server whatever order that the format has inside of a single source multiple coverage is multiple times multiple innovations and so on so we stop at implementing that CF angry but we opened the door for implementing whatever all the former whatever all or mode dimensional format you why wants to support in the future
um we in an expanded user interface of just over to handle these so if you're usages history you know that you publish duty-free tells you all just 1 layer inside this G if now when you are going to publish and that's CDF for two-dimensional malls IT it will list all the colleges that inside the cost as a set before then polyphemus example contained 3 yeses himself Korea call a phenomenon uh in the same spot and and uh we pushed it up a
bit Fatah to describe each and every time slice that you can find that in a minute In the high in enough usable way and that again we rolled out interfaces programmable interfaces that people can use to implement their own introspection of our multidimensional notice and we use them to
expose the structure of the data sets to both thing that was here so as I said before that is of ascribed college called that you can call to a few out what's inside your dataset but also by a list duration of operations so that you can actually uh revenue tolerated revivalist API so make and upload have Jews are harvested fewer out what's inside and then just several will expose the contents of dyslexia file to you again of either state PI so that you can basically tell Witiges server and then inspected its contents once it's published the to end
in once it's a are harvested you can also decided to throw away certain be that you don't want to to show to the user like throwing away way a certain period of time throwing away away ascertain range of elevations in or update some of the metadata so it's pretty fair flexible you can do a lot of stuff with it but so far I talked about supporting inputs that from that city if we can also support the output in city which makes lots lots of sense if you think about it we can read more two-dimensional hypercube so it makes sense but also be able to output them so we implemented and it's easiest output format for the yes uh which allows you to as a set of and you specify only like a hypercube often that I Large i-vector of data that you have been extracted enough format but still supports it in in its multidimensional specifications this is just an example of the
call this training in that was here so I specify the bonding box quite viable longer laughter subset here and then I'm specifying a set of elevation and a set of times so I'm I'm being basically building a four-dimensional hypercube and and asking just over to a article me that is another in that city a former now and then
uh what we generated is our climate and for Custer of convention compliant and that CF which can be opened with the most of them uh and that's to was around so I told you I applying and so on uh in the mass we also
implemented the Earth observation profile which is great you sure have the uh complexes satellite images that you wanted to publish the to the public to know the profiling and assumes that you wanted to remove provided us sort of eyebrows of an overview of your data and then you are you publish different views of the 2 as a sub layers in 3 groups so you can publish in the footprints of your and of your because of a total area you publish and this several bands of making up your acquisition and allow people to filter on those on those lands the gets only the ones they want you can publish their uh that i've data so not not the older what you apply some processing and I don't know you decide what's their concentration of chlorophyllin in the water and you can also publish it that uh as part of their observation 3 and finally solved flexed such as where where the clouds were aware of the flares bodies every single beats that might compromise the quality of the data in your life
I already described most of this
we implemented there for this 1 the scientific data support also a dynamic a quorum up supporting which basically just server can be told the tool of platinum up between asserting thinks of given values but basically apply decorum of stretching out from between the minimum and maximum of value all the data that I miss playing out of that particular layer that can explain to better visualize the dynamic characteristics of my data set in a certain area what's
cooking whether we bought on so we're
working on it even all the support of most scientific data has a on notion of 90 node data is of particular value in your dataset of the say is I don't know what's here and that right now JI does not support it uh so you basically takes that value and interpolate the diverse scales it along with the rest uh creating artifacts with J I x to implemented full loaded the support and we are going to merge it into which it was in the next few months so my hope is that we are going to have a full at supporting due to just server by March of 2015 when we are going to Italy is just over 2 7 and digital 15 we're
working on a high-performance restaurant around uh by the previous using their default library which is a high-performance pure Java library for
doing us out of with between in values are sub layers and is very nice because it basically allows you to specify your your own a little script that knowing the competition cell-by-cell with all systems are not and that it then it takes it and turns it into byte code into the JVM bytecodes code and then the the the JVM will say OK I'm using this by code over and over and over about what was doing the competition of other cells and he will turn it off right away into native code so within a split-second you will actually have a urine script that you wrote in a text editor running as not if coding ages so top speed
so as I said we're going to mix of this concept with coverage of user so that you can not only selected events you want to use from 1 dataset of several datasets but also say OK I wanted to add a few extra bonds which are going to be computed from the older dynamically using interval script and again TopSpeed because it's gonna be termed the into called it the 1st time you use
and this is it this is a more less the overview of what we implemented this and and so you if have questions In in which versions are are all those features at the moment are and aside from the poles
last winter to 7 all the other 2 6
exactly yes and it well sector I'm using support of bread format is a 1 hour to to him yeah
forgot to write about I'm sure we support agreed
to because does the format that was
given us by you at sort which is European uh organization that deals a lot with great files I have seen no sure where we support also grade 1 or not nothing we just tested and you may have mentioned some minus Mr. but can the the rest view configuration that be done programmatically interested in of course yes the and their coverage the user touches on X prime beef of configuration in into the the coverage of the source and as usual with CPI whether they know I suggest people to do is try go use a user interface to implemented and the beta that you you'll that you would like to talk to the ultimate without then the reader there there's was there is a magic due sort exam energies resource out of the API and looking at how it's done but yet if there the it and yes I was in trouble and making true color images and your server to make you know colormaps fine but because of the node data value uh issue um I think every maybe there's a way to do it but I've never been able to figure out of tried alpha where I am and adjust it it's not transparent is that
be due to this GeoTools issue it's really a JI she'll
J I was made by Sun early on it on contract with Mesa and is needed it for satellite images of they believe didn't have any no data so they didn't actually care about the problem so that the promise that when every time you interpolate so or Oriskany magically have to take a bunch of pixels and compute new 1 N J I visit a steeper uh there the data 1 it just blend them in as a they were the data so you you get some all the average is being done done uh in those processes and that's what we
uh have fixing J I extolled operations are now noted however so they're going to skip the bending of the time just care for the good parts of the image that said the urgency expects there's going to be added in a marriage into 2 tools but as a replacement for JI uh by March 2015 is a said the the other option that you can have no for
image mosaics there are 2
ways and I mean I am as well as I can be made only of 1 image if you want to so you can actually you whether the humans was like provide for any image right so 1 thing is to create the cost of the sorry vector footprints saying where they would be days and then was like would cut them another option is um that that when you're using them all as I you can specify their input transparent color and albatross particle and you can actually uh teller and teller touting because political reasons minus 1999 which in my data and it will get you some way towards what 1 it's not a fix everything otherwise you would have implemented text but yeah so I say this issue with the with the no data and the JI tools of that density of the preventive work in multiple resolutions and not really all we um so multiple resolutions you meaning the same like no we we cannot fully support the corrosion is more ICA having data at different resolutions what we don't support get that is having image more as I can made of data in different the projections or different color models we actually have a deep before of your after 1 of of our customer sites that handles both of cases so that you actually MOS ICA and gray in RGB and start data and a hole in the same image and that maybe there in different projections but is not the mayor's into the official just over a year and the 1 thing with opening up to that suggest support is bring huge depends potential for huge data on dealing with large 100 year time steps and 50 year daily Times century 55 thousand time steps that almost half terabyte for a standard procedure next I file worker optimizations have been done for spirit Kantian or time slicing whether it's time your
mind right OK so in terms of next if we worked quite a bit on the on the topic you alone and just you're just spoken about generally speaking in an hour when dealing with the and that's it if data jackals partial extent and after this the solution of the of the grid is not the greatest as you said you have a million times uh several elevations and so on so they multiply and that is grows to are a very you then the set and then the problem is getting quickly to the time elevation you want among the did giant dataset you have 4 . we create index is on disk we create 1 4 1 index 5 pertinent CFI to quickly locate given are ascertained time innovation the start of the candidate find a multidimensional thing that's 1 thing that we do and then we have a global index for demos kink is you're putting together several metadata files uh which is both spa tially and temporally and whatever other dimensions index it so we keep hitting the operational database normally so that you can index all the columns that that the final the dimensions in the winter request comes in we quickly locate of either the index in a city of find that satisfies the response and which part of it in a CDF file UCLA contains the data we want and then go to these can read exactly from that offset in part the multidimensional right we need so I pretty fast or check is the same question and then the same optimization for a on format it yes of course exactly the same structure is prepared for a great performance of Hey quote no more questions I guess we'll do for the last with