Add to Watchlist

Crunching Data In GeoServer: Mastering Rendering Transformations, WPS Processes And SQL Views


Citation of segment
Embed Code
Purchasing a DVD Cite video

Automated Media Analysis

Recognized Entities
Speech transcript
then so on and there's gonna presented some of us work on on WP has which actually started that and that comes from yes you working at a given sequence we have 20 minutes survive in this presentation OK so processing data just with double above use as a original meanings and I know I work for just solutions without company
based in Italy we provide support for the use of reduced rules network and so on and not just support for user we are actually co-developers in each and every of these projects meaning we can push changes here today and think of them and improve their and we do so regularly so today's
presentation is about to while the BBS web nothing so many years ago doing web mapping mapping basically meant to push back on and that and that the use of the style and there may be some entertainment is something breathing and that was all it since a few years ago it's almost impossible for solutions to make about web application without having some processing and with most of the new obligations we make are not just me visualization and data extraction tools but today are actually providing some capability in terms of processing so this presentation is about showing up and how we use it and use of a PS 2 power in several types of
applications so 1st so much weaker minus about verbal PS the regressor stands for Web Processing Service so the OGC way to expose for a change not data but processes the Web and the Web can then provided the data the Web is the competition and provides make results a simple
example here applying a buffer to align with the line is labeled I invoke a buffer asking for this sense of to whatever to means traders may be an eigenvector uh a buffer this is sort of the the simplest example I can think that sociologists partial but a also known geospatial like I could write about war in the process if I want to that wouldn't be very meaningful the
movie is 1 of the few few the proper protocols that supports asynchronous requests which is very important for a number of cases and we're going to see some of them so In most of these 2 protocols to make a synchronous request you send an HTTP request to the server and you wait for the requester come back when he comes back you get the answer period which is nice it's simple it doesn't work if you have to wait for a minute and look at the request that because there has to be connection is gonna rot in the meantime problem yet said in asynchronous you send a request you get back right away a response that says OK i took care of that request and I'm going to process it and here is a and that you can bowl to verify how far longer and and then you pull it it's cute it's skewed started 10 % 50 % in and % all look here is that result the the commendable
BSS about well the basic 1 assumes that the BS has no data a data and is gone affected from other reduces services or from a random http server recalls so it's going to fetch the data processing and then return it back to to the client which is a sort of limiting if you have to play with a very large amounts of data In induced server where the number of PS in is integrated in line in a consistent and user has its own local layers has its own local services and so on and so the can planning in this ecosystem so images of PSU can fetch data from local local layer just by using their names instead of passing around the data and there is a nice integration with the blessed with international transformations which have way to put in your style of active that will call to local peace process before and during the day time so you can transform the on the fly 1 example contour extraction I we haven't traditionally you server UI so we have a bit more pages to try out to with this request and so on so we got a bit more connections in order a standard normal this is the
Democrats bill the money spent time it it's the interesting than the
estimation is a sad 1 example of a set of applying a transformation on the fly while uh so we kind of stuff so that we don't have to keep 10 and 15 hundred versions of the same data are processed in a different way we just always do it on the fly during the transformation of particularly optimized in such a way that the only process the area that's visible in the current get map request and only after that that their solutions requested so we never work have their native solution if you're looking out of the giant digital additional them over the entire world of 10 meters can you look at it you're looking at the top title the altocumulus pixel and ask for their solution we use to extract the contour line for about some level and that makes for fast extraction it gives us a better less of what to look for OK
so before we invite into the let's take 1 step back to the processing you don't always needed the last
partially BMS is that you can use special nearest such as posteriors can now do competition on the flight SQL it's pretty efficient because all the data locally the type of complications so why not leverage it and the answer is well if it defeats the purpose of the week induce
we have the notion of parametric sequence you so then almost equally you would be that instead of point you just sort of table you give an SQL statement to as the hazardous source and without you can already do some processing but makes it more interesting is that in the test well I can embed the parameters that will come from the client from the Web but to control or what how the competition is that of course you have to protect itself against against SQL injection that's so we have a sort of validation going on there but generally speaking if you're being phase only in special database and you can express your processing as a query maybe with some parameters to control it by all means do your processing is it's probably been 1 of the most efficient ways to do it it's not always the best and I'll show you some cases in which the different approaches are
required this is an example of query that
builds using the make line and lines out of set of points on the love of telling me OK so
let's have a look at some examples for some applications that would be the new solutions that are
using either the media sources used audible so no 1st example of the soil monitoring it's a pure water
liberal PS so the Solomonic or the 1st lady take a step back in PS processes are provided we just server we have like a the simple process is already available but you can write your own and your own and this is what we normally do when we have a specific a customer project we believe that our own oblivious processes in this case a we had the 2 computer if any level of
so this is the some processing zone of the sort feeling and change metrics in terms of farm land use so this is a heavy as the competition for to accelerated is when we use the Jake would have to integrate the edges of the previous with the graphic arts and the speed up their their the competition doing it is possible doing it is not and ones is often not reveal because you basically have to take our entire large image for each of the last the story of the graphic card and make sure that you do all the steps of the competition within the graphic before coming back if you try to started going back and forth and doing steps biggest occurrences equivalent to be slow but if you do it right it's fast so we
had the to complete a number of sources steering algorithms although and calculated in which included in and this is
an example of a change so I have to ask them up so well with the land use in 2 different years and then shows me how much of their water bodies become became agricultural land and so on and of course I got a bunch of zeros but if I look at the agricultural areas that I can see that some of them and became artificial surfaces over time and the fight choose a particular
type of soil I can also build that a shot off where the land of how the land transformed over time in other types so all of this is that I have a BS as a set of mostly had a competition is the language could we do also some bits and John to perform the final final aggregation and dieselization
all these processes are running asynchronously because you even we Jake where they take time so in the client we also built a a visualization of the process is about running when they started their how far along they are itself so you can track on each computation another
example completely different this 1 and this 1 is pure sequel views of some number that would be a simple place the between plus from
FAO we in this case we have a larger that the baby is in each cell represents a good sorry contains the catchment of Tunis classified by species and that by the technique used to fuse them and that we have on historical perspective on this the the so we have a time they mention the quarter by quarter for a number of years and we use a user interface you can build the and and not choosing filtering the type of feature type of the fishing technique and so on and you get aim in this edition up as a result what powers
this method is very simple query that we are running an Oracle in this case and as you can see there are a few parameters in it they are the ones between percentage assigns 1 of them is the application of functions not all the percentage of which allows us to to do average uh some uh on the flight of course that we add to find out about expressions to make sure that we cannot be extracted so for example for the old the default some individual expressions specified AVG yourself and it works fine it's product processing on the fly without the
BS In this project we also created the enemy tool which allows us to specifies a certain number of years in quarters and get mad and animated gift of these maps so he shows evolution over time and so this is only
1 example of the you calling animator and we are specifying you promise yeah PA and that its values and each value will generate a strain some power
this is another interesting example this is all the pure WPS example in which Kazan independent yeah in
this case we're doing just downloading so you might say wait wait a 2nd the process and the was yes are meant for all data extraction so wireless part you not using those protocols while for a very simple reason these extractions can take time because we are talking about extracting hundreds of megabytes of data is no more so so we can just wait for the process of will that many job and get get us back have there is Ijspeert West would expire so instead we have created a process and we leverage the PS asynchronous calls to the largest fraction and that get than that of the client you know more manageable way and of course we can ensure that the expected life this works for vector layers had words for us that layers you can find it in the community section of your service so it's not part of that is but if you check that I can be used to find it you can do clipping so
this is our user interface on top of the process of diffusing the the buffet processes to depict a buffering area if if you define 1 in the I like you can say OK I'm going abroad polygonal line and then I want to Boston and then give me then the click inside the buffer in there it's possible to tackle the Donald status again why a senior PS and this
is more less than the structure of the system so we have a much sort the the front it fetches their list of layers from that of a mascot abilities and then you can call buffer to the buffering because a process cuckolded all this semester we because we have limits on how much of you can extract and and end with my gets status you get snow how finally
or another is an interesting example this 1 of it's pure PS but it's against promote processes sort of processes that are not run locally but when other machine difficult cases scientific data processing you have someone of rotating it Whitethorn Altay matlab whatever procedure to do any processing on the data it's meant to be around 1 of the machine of the the real PS 1 you might have a cluster in between and so on so we put together and uh and other community wanted the confines which is called
the remote by which he if a impediments by phone or command line tool can be invoked it uses X. as the communication level between the remote process and end-user XMP is up protocol thought to about shots but it was very well for this case as well and have subsequent that sequence diagrams but basically the idea is that you can stand up on new processing know that it will adjust to the chapter room just several with recognizes with fetch the processes that today and then you know that can run and then expose them as oblivious processes on on the web but when they are called it will pass all the parameters down to attack the execution and eventually get back to the results by some
shared file system so the idea is that the value of the use of a better directory that is shared on the file system so and well this is 1 example of a change detection in coregistration algorithms to have having also machine sometimes it's
best to do both so sometimes you have to run the ball that would be a sensible
views this is a a project in which we had to cut calculated the vulnerability of people and the environment against accidents involving the transport of dangerous goods like I also liquid oxygen patrol gasses and so on so you can imagine
there is a large number of latest theories of the large number of tiny segments from because for each segment every 500 meters the segment we know there the how likely it is to have a car accident in there and the then we have our some buffer says is depending on the kind of of goods transport there are different area is is computed and we actually have a 51 before the senses for each and every segment and we have millions and millions of segments on them the risk of formalize really
really complicated and itself so obligated introductory some parts of it have a meaning of their own so you can compute at all or you can compute part of it and because like I don't know 20 50 parameters that you can set
up in decline usually the result looks like a rendering transformation so depending on the zoom level we display the result either as a possibility or if we are close enough to ground we display the single segments showing their probability sort availability of the target in that segment so again as you can see some but so that vulnerable how do we come to this
very efficiently given that a large number of data and the large number of the dynamic parameters we can compute all on we cannot could compute old with sequel use because there are too many combinations we can compute everything's out because there's too much data so what do we do we make some matched so the parameter
queries are actually stored in a database and we compose them up a like a legal kits and there's a of this process that builds on the slide that ITER problematic view and runs it
and that we have a preprocessed a number of so part of the competition can actually be preprocessed not only all of it but we preprocessed as much as we could and the result is that we computation ficiently as possible in the database on the fly that even by the this process but most of them come up to a calculation is done by the but in the database will so the cross-layer filtering so let's find me whatever intersects these features in another layer and that's why I and yeah we also
have to work to be done computing 51 different buffer distances on top of so many layers but
this 1 will escape but
they're not really adding much to the presentation and so this is the end of the year and
then some questions from rule is problem is any configurable time hold on to which the the results of right so if you if you can just 289 and so the current stable and maintenance you will find a maximum execution time which is the total of queuing and running but if you look in just 7 to 10 which is going to be using October you well could find separate control for maximum queuing time and maximum execution so we need the previous leveling it's for pretty much everything so not just computation but you can also enable invisible processes and then you can go in the process and say OK this geometry can be more than 10 megabytes and this body which is the number of contours cannot go beyond the 100 and so on and so so you can control of its how much are of its and you processing are used for visualization learn for computation we are using it for competition it's this 1 could the your yeah yeah this 1 is
actually competent change mattresses and the soil
sealing so it's going from data to date method of this ization actually the last part of the visualization we do we do with this library called JetForm but allows us to apply some restaurant around the flight and give us a nice view of which to explain else committee was
looking at and I don't have is what they think you know this so I'll wait another 5 minutes for the next thing I knew what you were here
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation


Formal Metadata

Title Crunching Data In GeoServer: Mastering Rendering Transformations, WPS Processes And SQL Views
Title of Series FOSS4G Bonn 2016
Part Number 169
Number of Parts 193
Author Aime, Andrea
Contributors Aime, Andrea (GeoSolutions)
License CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
DOI 10.5446/20305
Publisher FOSS4G, Open Source Geospatial Foundation (OSGeo)
Release Date 2016
Language English

Content Metadata

Subject Area Information technology
Abstract This presentation will provide the attendee with an introduction to data processing in GeoServer by means of WPS, rendering transformations and SQL views, describing real applications and how these facilities were used in them. We'll start with the basic WPS capabilities, showing how to build processing request based on existing processes and how to build new processes leveraging scripting languages, and introducing unique GeoServer integration features, showing how processing can seamlessly integrate directly in the GeoServer data sources and complement existing services. We'll also discuss how to integrate on the fly processing in WMS requests, achieving high performance data displays without having to pre-process the data in advance, and allowing the caller to interactively choose processing parameters. While the above shows how to make GeoServer perform the work, the processing abilities of spatial databases should not be forgotten, so we’ll show how certain classes of processing can be achieved directly in the database. At the end the attendee will be able to easily issue WPS requests both for Vectors and Rasters to GeoServer through the WPS Demo Builder, enrich SLDs with on-the-fly rendering transformations and play with SQL views in order to create dynamic layers. Andrea Aime (GeoSolutions)
Keywords GeoSolutions

Related Material


AV-Portal 3.5.0 (cb7a58240982536f976b3fae0db2d7d34ae7e46b)


  528 ms - page object