Delivering high resolution deformation maps with high performance and extensive proc
Video in TIB AV-Portal:
Delivering high resolution deformation maps with high performance and extensive proc
Formal Metadata
Title |
Delivering high resolution deformation maps with high performance and extensive proc
|
Title of Series | |
Part Number |
106
|
Number of Parts |
193
|
Author |
|
Contributors |
|
License |
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. |
Identifiers |
|
Publisher |
|
Release Date |
2016
|
Language |
English
|
Content Metadata
Subject Area | |
Abstract |
SkyGeo uses Interferometric Synthetic Aperture Radar (InSAR) by satellites for mapping ground and infrastructure deformation. This leads to maps with millions of virtual sensors, each measuring deformation by time series containing hundreds of measurements. Examples of monitoring infrastructure and managing water injection in oil fields are shown. The deformation maps and maps with derived information are delivered via a customer portal. The portal tries to provide the rather complex data derived from InSAR together with extensive features to investigate, analyze and further process the data in a user friendly way. As customers are free to use any GIS package as well as the portals own viewer, all functionality is delivered by fully leveraging the (hidden) potential of the open standards WMS, WFS and WPS. Building the portal proved challenging because of the sheer amount of data combined by the need of live rendering to allow for styling by users and dynamic filtering using WMS dimensions. On top of that, the portal must be alive 24/7, be very secure and required new functionality must be in production within 2-4 weeks. The portal should allow a growth of 10 times per year. How these requirements can be met using Docker, Nginx, Mapserver, Heron-MC, PostGIS and PyWPS and some custom components will be discussed. Special attention is given to the rich feature set while retaining standards compliance and the encapsulation of mapserver for on the fly mapfile building and easy management of a very large amount of layers.
|
Keywords | SkyGeo |

00:00
Satellite
Freeware
Texture mapping
Information
Image resolution
State of matter
Image resolution
Field (computer science)
Number
Product (business)
Process (computing)
Computer animation
Quilt
Diagram
5 (number)
Extension (kinesiology)
Traffic reporting
01:30
Point (geometry)
Satellite
Surface
Group action
Service (economics)
Matter wave
Multiplication sign
Direction (geometry)
Time series
Mereology
Computer programming
Measurement
Geometry
Different (Kate Ryan album)
Phase transition
Spacetime
Pixel
NP-hard
Theory of relativity
Information
Reflection (mathematics)
Mathematical analysis
Bit
Maxima and minima
Measurement
Electronic signature
Orbit
CAN bus
Category of being
Computer animation
5 (number)
03:56
Point (geometry)
Injektivität
Greatest element
Injektivität
Stress (mechanics)
Time series
Mereology
Measurement
Field (computer science)
Geometry
Data management
Computer animation
Personal digital assistant
Green's function
Energy level
Traffic reporting
05:22
Geometry
Multiplication
Computer animation
Mapping
Planning
Energy level
Bit
Resultant
Software maintenance
Row (database)
Planning
06:38
Bargaining problem
Computer animation
Meeting/Interview
Special unitary group
Wave packet
07:09
Area
Geometry
Computer animation
Cylinder (geometry)
Personal area network
Software maintenance
Planning
07:30
Point (geometry)
Service (economics)
Geometry
Process (computing)
Computer animation
Computer file
Meeting/Interview
Texture mapping
Process (computing)
Table (information)
08:22
Point (geometry)
Category of being
Geometry
Computer animation
Videoconferencing
Form (programming)
08:51
Metropolitan area network
Mapping
Fibonacci number
Multiplication sign
Interior (topology)
Dimensional analysis
Client (computing)
Dimensional analysis
Variance
Inclusion map
Frequency
Software development kit
Geometry
Latent heat
Computer animation
09:43
Point (geometry)
Service (economics)
Computer file
Mapping
Run time (program lifecycle phase)
Point (geometry)
State of matter
Parameter (computer programming)
Bit
Maxima and minima
Parameter (computer programming)
Differenz <Mathematik>
System call
Geometry
Latent heat
Software development kit
Tablet computer
Computer animation
Data structure
10:28
Point (geometry)
Slide rule
Geometry
Presentation of a group
Graphical user interface
Computer animation
File viewer
Bit
10:52
Point (geometry)
Geometry
Wechselseitige Information
Pointer (computer programming)
Computer animation
Personal digital assistant
Query language
Query language
1 (number)
Maxima and minima
Attribute grammar
Planar graph
11:31
Service (economics)
Mapping
Service (economics)
Server (computing)
Sound effect
Database
Entire function
Geometry
Computer animation
Query language
Physical law
Queue (abstract data type)
Process (computing)
12:12
Geometry
Building
Electronic data interchange
Computer animation
File format
Interface (computing)
Information retrieval
1 (number)
Process (computing)
Mereology
Measurement
Resultant
12:48
Standard deviation
Functional (mathematics)
Dependent and independent variables
Multiplication sign
Grass (card game)
Client (computing)
Generic programming
Mereology
Computer programming
Variance
Product (business)
Time domain
Geometry
Insertion loss
Process (computing)
Implementation
Task (computing)
Metropolitan area network
Service (economics)
Standard deviation
Dependent and independent variables
Proper map
Information
Mapping
Computer program
Hecke operator
Binary file
Scalability
Sphere
Entire function
Software development kit
Process (computing)
Computer animation
Integrated development environment
Volumenvisualisierung
Video game
Normal (geometry)
File viewer
Task (computing)
14:59
Satellite
Metropolitan area network
Server (computing)
Presentation of a group
Service (economics)
Mapping
Content (media)
Ultraviolet photoelectron spectroscopy
Mereology
Likelihood function
Web 2.0
Architecture
Proof theory
Geometry
Process (computing)
Computer animation
Diagram
Drum memory
Computer architecture
Proof theory
16:02
Context awareness
Metropolitan area network
Rounding
State observer
Service (economics)
Demo (music)
Computer file
Information
Computer file
Data storage device
Sheaf (mathematics)
Formal language
Architecture
Geometry
Computer animation
Personal digital assistant
File viewer
Summierbarkeit
Exception handling
Proof theory
16:54
State observer
Group action
Server (computing)
Mapping
Scaling (geometry)
Multiplication sign
Menu (computing)
Mereology
Revision control
Geometry
Hash function
Identity management
World Wide Web Consortium
Standard deviation
Dependent and independent variables
Server (computing)
Software developer
Plastikkarte
Menu (computing)
Mereology
Cartesian coordinate system
Limit (category theory)
System call
Measurement
Entire function
Cache (computing)
Software development kit
Data management
Computer animation
Software
Artistic rendering
Video game
Quicksort
20:22
Point (geometry)
Satellite
Presentation of a group
Service (economics)
Computer file
Multiplication sign
Connectivity (graph theory)
Maxima and minima
Similarity (geometry)
Streaming media
Web browser
Client (computing)
Parameter (computer programming)
Distance
Theory
Front and back ends
Product (business)
Cumulant
Meeting/Interview
Different (Kate Ryan album)
Videoconferencing
Software testing
Software framework
Social class
Standard deviation
Matching (graph theory)
Information
Surface
Interactive television
Subject indexing
Word
Process (computing)
Computer animation
Vector space
Ring (mathematics)
Software
Volumenvisualisierung
Object (grammar)
Freeware
Resultant
00:08
it was definitely the domain of
00:12
money in state current you analytic of EU post intensity pretending 1st talk today and then in a market like from its value because of lack of the high resolution deformation maps with high performance and excessive extensive process and tried the mistake or OK well good afternoon thank you for being here at the end of this longer today I like to show you a few of my experience with those serving number that higher-resolution information so proud of that tragedy of minus market diagram and to guide young responsible for delivering actually the maps in the usual portal to the end of the basic ideas company who makes a living out of monitoring infrastructure with that said let's actually satellite is organized as a sort of start up so reluctant to foster an ultimate restoration at explaining nice products we will make then and how it looks when we deliver it and then also field or trip report so just shared experiences with you and then do notes the
01:31
1st the principle of the instructor technique is good as it is cooled in stock out to simple concept actually uh this that light comes over every for 11 days or something and then it that its acquisitions and interesting part is that it can follow a point in time by it's a spectral signature shown somewhere reflected around the house and by the spectral signature of 2 to reflect on you can follow it in time we don't know exactly where the point is that interesting that you can follow it time we do full time OK the what we
02:14
used to to see if the point moves towards over or away from satellite is actually not the amount of wavelengths from satellite to the point that we look at the face different and that makes it possible to measure the very high accuracy that is also very difficult because we cannot clearly see if something has moved a bit too what's the satellite orbiting and the wavelength forbidden to wavelength of emitted trio things we don't really know that without introducing additional information that additional information can come from nearby points preferably known to be stable or by careful analysis of the time series we need to introduce some action of the resulting data properties are I think very interesting in against measure movements and its minimum which is so it's very precise and quality of intervals for to diversify days depending on satellite to you choose an interesting part is that the data available from 1992 onwards so today you can decide to start monitoring something from 1990 to 1 or no it's always interesting and we always start related monitoring program so finally there is a technique that today you can decide to start monitoring and OK and as you know we do need pardon reflectance services through to get something and I we measure in the direction of the satellites that we can do some smart decompose into that we can actually measure horizontal movement on the eastern resurrection inferred from well the
03:58
typical never delivered to a customer's look like this this is a part of the report of what about it's about to trigger 4 kilometers this crucial so you see of green and red points and green points are more or less stable at that point citing and if you then click 1 of these points using the time series measurements and at that time series shows that it goes down to we have a lot of attention
04:26
already we applied this kind of data do our customers do it is 1 of the more interesting part is actually injection manager if you deployed in our field getting on all of this will subside or even for those very fast and it might give you a little the trouble with the neighbors for example something else and they can inject will show that the so more that keeps the same level you and inject too much what you actually who's also robust to surface you don't want that if you do too slowly you get a little few stress in the bottom and you can get something we goal of locating the well casing is very expensive and develop a single and of course about a million to exchange since our customers using our data they didn't have evolved well casing damage and more so than that that really is a very good this is a very interesting
05:24
uh is also interesting is is used for a gas pipeline monitoring and in Netherlands houses are to begin to bleep built on foundations that the guest by going to the house is not of the soils exciting and against pipe can have only done much strain before it breaks off they evidently there's a bit of a problem so it's difficult to find gasoline before it explodes and always nice to actually go being at the right place not be delivered this kind of map so that the guys who has to go digging in finding gets we actually know where to start dating and has the highest chance of finding a truly saves a lot of money
06:03
and then again and then example from the Netherlands and maintains planning for them in performance of the the uh the roads tend to sink in the Netherlands and then every now and again you have to move put up something because we don't want to have 2 rows multiple levels at least 30 centimeters in most places and as a result with this kind of next they can actually produce predict when the road will subside to be loaded to centimeters above ground level and then they have to do something to the they do better planning on on the basis of of this kind of thing that's
06:41
interesting would like to think of the
07:08
but the river at the church next
07:12
light and cylinder Wilson area nothing left but you
07:40
something the figuring this
07:45
data to the eligibility to deliver Mapping Services WMS we deliver data WFS and deliver process in the sense that we'll be at he also delivered data there always ask for data table doesn't work because innovative much point so we deliver the CSV files and typical customers then discovers that you cannot render that kind of files and difficulty and so we also provide our customers fewest with data and then meaningful tools the uh 1 of these
08:25
meaningful tools is sliders to actually flight uh to dial-in properties so that you can actually only see the the most interesting
08:33
point your and that he did happening here in this little video form is actually quite low this because I was doing this very slow Wi-Fi campsite OK
08:51
this is done using a hidden gem you WMS specification that WMS dimensions well known from elevation offer time dimensions which again intergeneric principles so you can use for just and kind of filtering actually or so uh it's actually have also advertised in the capabilities document so you can uh have filter for example if you had this very rarely if you get my request you need to understand what real uh that range very useful for clients built on the bill so supply for some customers at the time slider so that today actually consuming only a certain that period in time as something happened something interesting happens a lot you
09:41
see it happening in the background but
09:45
actually very easy to do with the map service we use it a bit to entice institution for the ones like that the support in the and the WMS that is not that great enough to do that in the the specific parameters and it also is important for our customers is that they can apply for files so
10:06
they get maximum insight into structure of the data and all they have to select a point sizes and they have to stress that the call is over values and so on and in WMS that tends to be out get quite cumbersome because you you all the files to to supply for the system so that they can actually see today how well it's not very satisfactory so we find a different solution
10:29
and actually now have all of viewers to slide is just all set for the point size and
10:35
slide stretch the cost overdevelop just as we would like it but was not very easy to implement actually use
10:42
of quite strong and I hope to In the end of presentation i can tell a little bit more on how we did this that and the principle is to just slide has is
10:54
also developed and suddenly you see very clearly on these cases the less clear but have
11:03
a K and also important of course is prairie uh the data for the customer and here you see Korea happening
11:12
in spatial query just selecting a few points and that gives you a points that they can walk through and all the and things of course you can also query for now should so that you can actually select all pointers and foster than national human unity year something of that so just simple planar WFS
11:32
note that that interesting but we did find that fuel
11:36
use WFS is a download service to actually know reading and if you do that it tends to be quite slow in the and even after 7 which is much better at doing effect uh flows so it's handy to have something I ready for for users who want to download the entire it has been notice how many possible OK uh and be also provides for customers is the the ability to actually process the data but you see happening here is that the the user is going to
12:12
combine all those school adults that those formation measurements with the building so that you actually can see which buildings are going down which buildings are more stable and the retrieval just right from the interface didn't need any GIS and it doesn't take long I didn't have a fast and this
12:31
movement this movie something just normal speed and you see that these 2 buildings are definitely not stable and that most of the the other ones that are stable there in the part 1 which is orange result also lots of 1st and better OK
12:50
processing data we use the WPS from that standard
12:54
for that and that the standard defines a way to send the quest for processing and also a way to send a response back to the client and the use the Allied WPS due to implement it and United WPS allows you to write just about anything Python for processing usually you not bound to but you should the WMS WFS or whatever you can just and just about anything that you like programming that is the killing and killing of tasks that you have to write yourself that's also matches and it's very easy to extend the heck so for people like and that's going OK well this is about the the more interesting parts we have to put in the viewer so that left us when we want to build it and a would requirements for portal and of course the normal mapping of functionality and the more usual once showed just now and the had to be and the dual to have fast because we need to render everything in life you can do that the usual piling catching thing most people do when you need to fast madness musical shall we have to do that life so uh these to be reliable has to be there for their customers an entire also cannot take things down and we tend to think there lot of customers have a lot of requests and most of the time we want to react to that within a week or so new features we bring into production within a week and that means tinkering in their production environment and you don't want to troll things over for other persons them as well so that's interesting and requirements of course has to be secure you don't want to for example the oil drilling the pre-drilling sense you don't want to be there to the public sphere in much a sense that information seems to be scalable of course because we have developed a lot of customers and then those flexibility and talked about and and the last thing of course it needs to be standards
14:59
compliance so that every customer can actually use it in the 70 s of this
15:05
architecture we use that to become that's something I kissed proof keep it simple stupid and not the simple architecture it wouldn't let down uh so we went back to as much as proven technology as possible so that means that there are actually you know looking at the diagram we wrote uh just wrong at 1st day I started their designing resources this entire portal and it's still the likelihood of a bid for this presentation but that's about alternate and it's still there as in now in the lower part here see that the processing satellite data that's not what I'm talking about here it started both yes there's all the data and and then we provide services so it's a matter of a little map service and that the light of a PS and on top of that the use the engine x as the web server for an all out duplication is happening there uh engine X also static content and some sort of such pretend
16:04
interesting bodies that labeled ever made almost everything file-based except for others in both yeah so
16:10
here you see the uh what the store for to gross domestic customer demo and customer turn and the customer demo in this case has it has fewer cold water than and if you were as download section uh service section and if you sections all file-based agreement to take a cosmo of languages deleted the full and you call this uh well if you look at services so there you have the little all of the files observer a the files defining an idea after fewer and we can't tinker we get for every customer separately if you want to everything is in and so on so that the from to do something for us all customers at once just said Texaco's information itself OK so now
17:00
I call that the interactive part of how much time do I have still left treatment identity and what did I miss them I hear about it alligator and I have very little time that's actually so from the entire menu I'm afraid have to skip most of you almost all them I think and pop up on I think I and the Bird to conclusions and specimens any questions and I might grab a slight of the union last part so fast forward to the
17:36
conclusions and this give only yes it's that's still the thing is always running into a brick wall and and what they found during journey is actually that our OGC standards the good old the WMS WFS WPS come a very long way in creating a rich that application for custom tools and things and so on this again have really like specialized software for customers we do of customers who say we want GIS like a and the life rendering of the goal to assist in the development of smart caching by engineers who really becomes a way in serving high performance matched by given high-performance experience wildlife rendering for everybody thinking you always need dining caching if you act like really work carefully and have performance in mind you can get from that's without dieting that another nice conclusion that I call to backup itself still commands which can of back up with that we decided to equalize used by the groups due to an observer into WSC I applications and improved proved to be very flexible call true 100 and very flexible then you have 1 place where responses from the request comments and responses go out so that gives you the ability for example to adapt the capabilities to comment on the fly when is going out so that can much more customized the way your application is behaving so when you run until the limitations on that server then you can fix it in your own by constructing is really really flexible and actually much easier to deployment serve you you put to make it to the WSC I application and stunned that CGI applications and called for those of you who never regular with doctor doctor is really a sort of the assignment I mean I fall deployment of problems if you using docker you really isolate applications into the of container manageable units and that's really helps not tool to try out new versions of software tools and deploy a special fairly different server we have a lot of hassle and is really enables us to move fast and break not too much like thank you emerge we need
20:25
questions hunting for your presentation and regarding the interactive Europe have you thought about to using vector data do stream to the client and them do do this cumulative chlorine of plants and the edges actually this class on that testing along well below we don't think solar surface would really do it on the client side of course if you do the find it's often very snappy more possibilities for interacting both 2 things opposable this that we lose a lot of standards compliance so we the to the customer then really has to use our if and cannot rely on services that much as 1 thing the other thing it's is there's no way that you real render about a million objects in a browser the browser doesn't care about rendering of political militant million for distance that's no problem with rendering a million separate points is really a problem that doesn't perform can tell you have to discount it but fortunately that this question I will ask the question of about something different and should than edges so that's that's what kind of tools are using for the the information and how these videos have got that actually a and B wrote a lot of proprietary software to do that and for the for the Sentinel 1 satellite the little amount of Copernicus we partly used as long as sentinel on walks and arrest this all Allerton ourselves and it's unfortunately not of interest as saying parameters similar question of what kind of components for using for the use of interactive few words and I'll be using a product called to here only I think it just from looking at something about that and the fact that that's actually a XGA S G O X OpenLayers due fewer and it's it's a nice framework you get a lot of for free that do have right a lot of addictions itself but today theory were that to have question you're you're storing your results as a CSV file that that you has a ring directly from massive did you compare the performances of like that was seizing posters at the back end now the the this is the follows just for download from customers originally known true matches while is also at the actually his face falls we import into the database as except saying to the voluntary provide for them of customer just as a convenient time to the result of the processing is is the father than you at the end of the day the OWL they're actually the result of processing something else with me then expertise free-fall for important and that would be the the crux of course you need a spatial index and without something like POS here's a ship or something you don't have special index and then you really lost 50 In other questions no well thank you very much much of the public
