Raster Data In GeoServer And GeoTools: Achievements, Issues And Future Developments

Video in TIB AV-Portal: Raster Data In GeoServer And GeoTools: Achievements, Issues And Future Developments


Purchase DVD

Formal Metadata

Raster Data In GeoServer And GeoTools: Achievements, Issues And Future Developments
Title of Series
CC Attribution - NonCommercial - ShareAlike 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date
Production Year
Production Place
Seoul, South Korea

Content Metadata

Subject Area
The purpose of this presentation is, on a side, to dissect the developments performed during last year as far as raster data support in GeoTools and GeoServer is concerned, while on the other side to introduce and discuss the future development directions. Advancements and improvements for the management of multidimensional raster data (NetCDF, GRIB, HDF) and mosaic thereof will be introduced, as well as the available ways to manage sliding windows of data via the REST API and importer. Extensive details will be provided on the latest updates for the management of multidimensional raster data used in the Remote Sensing and MetOc fields, including support for WCS EO and WMS EO, and some considerations on the WCS MetOc extensions. The presentation will also introduce and provide updates on jai-ext, imageio-ext, and JAITools. jai-ext provides extended JAI operators that correctly handle NODATA and regione of interests (masks), JAITools provides a number of new raster data analysis operators, including powerful and fast raster algebra support, while ImageIO-Ext bridges the gap across the Java world and native raster data access libraries providing high performance access to GDAL, Kakadu and other libraries. The presentation will wrap up providing an overview of unresolved issues and challenges that still need to be addressed, suggesting tips and workarounds allowing to leverage the full potential of the systems.
Service (economics) Computer animation Personal digital assistant Projective plane
Server (computing) Service (economics) Computer animation Java applet Software developer Image processing Website Energy level Bit Library (computing) Physical system
Bytecode Read-only memory Musical ensemble Implementation Pixel Machine code Just-in-Time-Compiler Java applet Code Multiplication sign Geometry 1 (number) Sheaf (mathematics) Food energy Number Formal language Chaining Medical imaging Operator (mathematics) Energy level Extension (kinesiology) Physical system Default (computer science) Process (computing) Software developer Element (mathematics) Image processing Moment (mathematics) Projective plane Interface (computing) Bit Variable (mathematics) Particle system Cache (computing) Process (computing) Computer animation Auditory masking Order (biology) Active contour model Library (computing)
Read-only memory Implementation Process (computing) Scaling (geometry) INTEGRAL File format Software developer Multiplication sign Source code Basis (linear algebra) Interface (computing) Bit Set (mathematics) Flow separation Medical imaging Process (computing) Computer animation Linker (computing) Utility software Representation (politics) Library (computing) Form (programming)
Service (economics) Computer animation Internetworking Projective plane Image processing Bit Mereology Automatic differentiation
Interpolation INTEGRAL Multiplication sign Virtual machine Ultraviolet photoelectron spectroscopy Raw image format Number Chaining Medical imaging Causality Operator (mathematics) Single-precision floating-point format Database Subtraction Traffic reporting Position operator Tunis Physical system Default (computer science) Process (computing) Scaling (geometry) Key (cryptography) Forcing (mathematics) Projective plane Moment (mathematics) Model theory Bit Set (mathematics) Affine space Arithmetic mean Computer animation Graph coloring Iteration Quicksort Data type
Area Pixel Computer file Multiplication sign Water vapor Shape (magazine) Representational state transfer Computer font Number Chaining Medical imaging Category of being Latent heat Computer animation Vector space Data compression Integer
Musical ensemble Process (computing) Service (economics) State of matter Projective plane Bit Client (computing) Mereology Code Workstation Degree (graph theory) Medical imaging Category of being Video game Population density Computer animation Database Authorization Representation (politics)
Email Mapping Divisor Multiplication sign Projective plane Inverse element Function (mathematics) Number Computer animation Data compression Cuboid Four-dimensional space Data conversion Units of measurement Form (programming)
Medical imaging Algorithm Computer animation Mapping Presentation of a group Linker (computing) Projective plane Representation (politics) Bit Parameter (computer programming) Library (computing) Vector potential
Computer animation Civil engineering Moment (mathematics) Point cloud Contrast (vision) Subtraction Usability
Chaining Algorithm Computer animation Block (periodic table) Multiplication sign Model theory Moment (mathematics) Parameter (computer programming) Event horizon Freezing Position operator Relief
Revision control Implementation Arithmetic mean Computer animation Operator (mathematics) Electronic mailing list Interface (computing) Social class
Point (geometry) Computer animation Vector space Personal digital assistant Concentric Device driver Mathematical analysis Sound effect Pressure Condition number
Spacetime Process (computing) Code Java applet Bit Weight Revision control Process (computing) Linker (computing) Stress (mechanics) Information security Physical system Library (computing)
ieee work for you solutions and
Italian based company that is providing support services producers you tools and well assistant who would you have just for and so on and we are core continue to suffer all of these projects today I'm going to talk about
percentage tools and your server what so how we process that isolated and the recent achievements in issues and future developments so on the 1st a bit
of a Introduction to the technology stack so this is
more or less Auspex looks like at the very bottom we have j i and j i on which are the things that arranged in general to the city was image processing on top of it we have a Mazzeo acts the JI extant AI tools that we build tools on top of that you know to have the special referencing and so because of the lower level libraries web site essentially this as people are unaware of the Java from there just image processing libraries and then went on to lose we have to server providing the services where you still all the which systems the so the the
baby's head element is just advanced imaging gender-balanced imaging is a very nice I bring to a performance of pure Java the image processing it's time ways which is very important to process very large images cause you don't ever have needed to build the full image memory that you can and processes these babies it has in the rectified caching it's easily extensible so that you can have your own variables and actually in a moment of we extensively leverage this extensibility but there is the ability to plug in a library that provides um native code acceleration for the common operations it supports multi-threading in competition of the times it has 1 big issue it was developed by Sun now are called on the health of NASA which is a very good start but then the development and the the so what do we do what we do about it well we built a j i x so j i is a fully pluggable system with interfaces for basically everything and it allows you to the plug in your implementation in place of the default ones so we do the JI next on top of the chair I to replace the most common JI operations that we use in order to uh support the concept of noted that which was not available in JI cost was meant just for image processing and only by more about representing physical phenomenon and a lot of them uh we have full support for processing ROI region of interest this masks um which was present at the was a notion of J of ROI JI but it was not to properly implemented in all operations we made of several fixes we have added support for the energy of band masks into it and so on so that's our current active the development of chair particles then we have j tools this is a separate project them which is by the way promised heaven to join which I I x this 1 provides a high-performance roster processing extending a JI but we basically built on top was instead of replacing it says JI at us so we have a number of fine level operations such as the contour extraction the victimization uh ROI based on GPS geometries and so on and we have a nice package called you full not that
implements has the about us all they got their own your language to do all of that pixel manipulation of and that it nicely integrates with that is a i processing chain and we have uh on some projects exposing that appears process for everybody to a high performance on a library 1 nice a little bit about this 1 is not a legally compliance all the way down to the difficult if if the transcript executed long enough goes this library passes the text that you have and generates byte code of the JI job title and then the just-in-time compiler if it sees there is a whole section not cold it will compile it down code so if a really nice way to have a pleasant blazing fast right just
then we have today I JIT misery all different problems a I 2 of connect with their bit of sources so there you most of it in that it allows for the fact that the links on not just loading everything was shot by loading time of 5 from this allowing large scale processing without and in large-scale memory it is extensible it has some accelerated calling song again that his library but it's development stop so we made a mistake text which builds on top of the same interfaces as he Mazzeo all other places most of the implementation so we have our own duty reader that can do because if we have an integration we to reject that we've got to which you know for several data sets would be the fast the pure job the engine a lot faster than the native PNG of available in the material uh in the uh let's say every time we had the ability to even your former we put in image then on top of all
this would be a utils which adds the geography concept on top of the measuring so the representation of your reference read the breeding and writing during fierce format so for example in the next week read these and attacks but we don't know what that means and you and you lose we cover the fact adjective and you're refusing that we do all the rendering the reproduction of the processing and so on in the of these forms the basis of our for use on
just gestural ads on top of all that uh rendering image processing and so on although this is services so everything that we do too dangerous over land unless the bloodiest of CS and PS is an utterly goes through the whole stack don't use which in turn calls all the aches the projects which in out there in the internet uh at the top of the Bayes part off on
so rich in recent achievements what's new j there's been work done quite a bit
so we have our big quite a bit of performance tuning and might but fixin meaning we have managed to make it on MOS affinity is the thing which is even if the images are a different color models which was a common requirement from our user base to have a MOS like of images that our some great song called some and so on and put them together in a single laws like without having to accommodate them into a single mother and it's new it's just it's just in meaning rated Julja comes but uh we have sort of positive reports that we have company key from uh so somewhere instituting in the United States had a has been using the JI replace yeah i in the has been noticing uh so very significant speedups in the processing of arrived there and processing chain which flows of 30 year 2nd basically the database they've seen speed of between 3 times and 68 times faster depending on the type of interpolation of the a so we're very pleased with this and this is not an next integration has been integrated for the 1st time introduced for the images servitude made at the moment it's not enabled by default cause we have still to ionize out a few issues so if we enable the lady now we will reap some benefits but we would have 2 or 3 iterations that scale of tool I allowed so it's something that you can enable by passing a system valuable to the different machines so it's there for everything as there to to play with and once we fix those are a few issues we hope to enable at MIT default either in just over 2 9 later just over 2 and when feeds writing so it is 1 of the benefits of JI exenteration injured was adjusted well and 2 and another supports which is very important for scientific doesn't uh must support full force of our must supporting all operations it in the agenda is more like support and on a number of features of a JI as a set this year I I hope project is basically that of women and replace the operations and thus fix some but and also as shown by the common comments from company clicking we have very interesting speed ups and serving operations not all of them don't expect 60 everywhere it happens in some operations all the recent past achievement
arrested that I'm asking uh
since we have that we have no a food chain supporting otherwise we that supporting the vector must of forcibly them so that you can tell us using shape file or other OK the final of PE files wherein they image that would data is and separated from from the from the Nordic now you we could do it with within little support tool but depending on the compression that's not always the best option like if you take the conference your their courage in 2000 Conference the fate of the border between that would be there and then let let's say then that the that data you have an area in which some black getting to the good they got some good data gets something the black and well if you try to explain to you will find out blank each outline our during injury which is due to the lossy compression would even if you give us a proper outlines instead that we are able to come out of exact where we want to show the of the guidelines are nice their competence on that have a little problem they tend to rescaling but the pixels go down by the integers instead you can have a half full of water of the peaks of it's just the pixel so when you are scaling down you you start with the I don't know them pixel and stand on 3 times you have to decide whether to use and with the release of more for big so you can have 3 . 3 pixels and that the vector outlines that scares only have so how we mentioned to the vector 1 uh it's complicated it's nice to have instead of support also for across the must restaurants are but in light from the rest of the which is embedded in your a firestorm and you know that's not for example and they have the property of scaling down just as that there the data they are masking because they are also made of pixels they can compress very very well so they don't increase the size of your data a lot in the problem again and it varies for the same purposes just they scaled-down them exactly as they believe that they are supposed what and when you have all reviews and the 5 you can have a use of the mosques as and the
so we have a number of methodological and mythology and of sonography specific improvements such as support
for authors in there the longitude ranges between 0 and 360 degrees normally debatable being minus 1 and minus 1 the last making but many meterological institutions like to have the they of 0 to 360 instead and that poses a bit of a problem when trying to process of that data on this is not just about the seizure of some mad scientist sometimes you have the sense of that really collecting data on both ends of the make life so that if a seamlessly going through it and we need to be able to process it so that uh we added support uh for for it uh this image of the you have these are the same so very close to the great big clients and somewhere in the Pacific and the physically gathering data that this crossing the so we need to be able to display then representing property which is even more it despite its just by so it
I much of scientific data is in the density of formant band many scientists was once to have their data of enough as precise as possible representation so they normally choose a lot of rejections and when any longer as I don't mean to choose the projection of the typical protection of the state of the art in East send their on their very very uh weather station where the data comes from so it's completely custom it's not part of the official you gives you that there is some we have support for reading that and for presenting it with custom codes in a seamless way so you bring you bring you prepare a database of discussed elections then the slope of quite nicely up from the data into user is things in our service so
in this in shows reading the headers from a CDF mapping it into that i projection shown but
along with that from this you have we have a number of improvements for CS output formant which is a publicy at some form the before that allows you to download the multidimensional legions of yes you can select the pupils both data longitude latitude about a bounding box there and elevation arranged and then time ranges so you have a lot of four-dimensional q if you want to represent in the world you can do it with that CDs of formant the in we have support for this and that means in the climate and forecast the conventional custom make up thinking that is this notion that you don't want to have the true 1 them numbers of their your physical phenomenon but you provide an offset and now and now I'm and I'm multiplying factor and then you just keep the inverse of number seats for compression purposes we added support for unit of measure conversion of maybe you're the dying in using the but you want tools publishing meters and for uh this 1 the
the I saw the bits that we need we extended the advance of potential and link to us that I made a presentation probably badly named on this topic was a mapping beyond 38 57 work in the 57 is what capable on of us projection and link is about dealing with that of libraries over the various protection when you have a global dataset and want to display the then new tool in each in any production possible available in the this improves the representation of of prosody the when you have project them some unfriendly projections that that that is a global remain set that was present in the in the posturographic before the you and after that you it
when improved the conference and spent quite a bit so if you are convinced that in unseen images now you can control in more detail the algorithm used in the parameters for a so here I have an
image of civil and I made it to different contrast enhancements 1 with the stretched min-max tool emphasize a moralist there and even usable features and other kidding which basically shows me only the clouds instead so just a quick note
about what we're working on at the moment
we have a very nice feature that just means that the feature freeze of just over 2 8 by like 1 week there was nothing we could do about it and use of events that there is more than this time blocks so when it's time to cut we got and so we don't look so know we never had the ability to apply you shading or shaded relief to digital additional model or whatever so phase you want we are about to add it uh so you have the ability to control that if after and in another bid and here is
an example of global digital additional over with you shading apply to it as you can see the mountains are very much uh more visible on the moment we are using a fixed at some position by the algorithm inside the chain of Italy is able to take the position of the size of the world and elevation into account so we're to expose others vendor parameters cost actually doesn't have such but it would be still nice to you know put the sun and I position when the when we use initiated and finally we have for help of
funding to build a full JI replacement as a told you we already replaced most of the means of most of the implementation of the operators of JI but we are still relying on the interfaces in some basic classes from JI we would like to become completely independent from from that in our own a fully maintainable version of the i and possibly lysed licensed in a different way the and so we have looking for that and and funding to get there because it's so I use amount work so if everybody here wants to to be in list of the and with this I
encoding methods few stood in the good and 1 here 1 the but how can I just to clarify the Giants fixed yet the work you're doing uh increases that
from 2nd significantly in some cases
for working with higher Rasta data is it also on the effects just normal to the analysis by rendering for affected in not for victory that no have because chastisement involved told in and uh the 1st request that only involve the vector B it speeds up it may speed up where it left the tested there are cases in which you are publishing us history that was used yelling for new cropping and scaling down of things in so those are the cases like an improvement and the according to turn the use case of war was improved the bike is much was was seen floating point of a socalled agrees with you know pressure of concentration of some pollutant or whatever you know stuff like the conditionals that yeah the thank the it for working with those from that's where you have built of Judah did a driver's and native JI
next du recommend using 1 or the other for a performance for
flexibility but generally speaking a link into a native library from of this calling trouble so if there is a pure Java version of the of the code that and I suggest to you that for example a while we could be treated you are using the dialog we just not to and there's a performance reasons you have to cross the JI variant going from their job our way of managing the security status as on the sea weight when you go back and sort things of that systems which comes at the price and this the 2nd issue is that many of the native libraries were not deal to from the beginning to support multithreading they got there it might be at our but under under stress they can still crashed and if none of the library attached to it a bit of a process goes down takes down the entire process so this is not the is just over goes down the space that you it's not going to get started by much so there's a text and


  612 ms - page object


AV-Portal 3.16.0 (9cfa3864b8acb689056f9c67aa39bc8ec4c75d58)