Why is popularity the biggest enemy of WMS?
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Title of Series | ||
Number of Parts | 266 | |
Author | ||
License | CC Attribution 3.0 Germany: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
Identifiers | 10.5446/66599 (DOI) | |
Publisher | ||
Release Date | ||
Language |
Content Metadata
Subject Area | ||
Genre | ||
Abstract |
|
00:00
Computer animation
00:09
Category of being
00:23
MiniDiscDependent and independent variablesVisualization (computer graphics)Service (economics)Front and back endsRepository (publishing)Metric systemDivisorScalabilityThermal expansionSource codeMereologyElectronic visual displayComputer fileMappingLevel (video gaming)Structural loadChemical equationVirtualizationServer (computing)Computer configurationData storage deviceMetreMultiplication signHorizonImage resolutionSoftware repositoryMiniDiscWeb 2.0MetadataRaster graphicsLastteilungPolarization (waves)Set (mathematics)Subject indexingNumberMoment (mathematics)Computing platformSource codeMatrix (mathematics)Configuration spaceSocial classObject (grammar)CASE <Informatik>Variety (linguistics)Sound effectKey (cryptography)Zoom lensScaling (geometry)Front and back endsComputer animation
04:29
Simultaneous localization and mappingArtistic renderingResponse time (technology)AverageStructural loadMusical ensembleSoftware testingMetreMultiplication signComputer animation
05:09
Computer animation
Transcript: English(auto-generated)
00:08
My name is Martin Nemeski, I'm a junior data scientist at GloFero and today I would like to talk to you about why the popularity is at least a big problem when it comes to serving data through WMS.
00:22
So, let's start. So, maybe let's try to define problem before solving it. When I was working on our Eo4Ua initiative, which aims to serve the spatial data related to the web.js platform,
00:42
I realized that the OpenStreetMap base map, WMS, which I was using, works better at specific hours, and more precisely, it works better when everyone is asleep at the middle of the night. It helped me to define two problems, that WMS is not capable of managing a huge load of requests,
01:11
and it struggles when there is big traffic on reading the source data.
01:21
So, I tried to take this use case and recreate it using the possibilities created by my company. So, I took the satellite imagery, I took the EoData repository, which is over 35 petabytes of data, on S3 storage with server backend, it's over 10,000 of this, and it solves the problem of reading the data,
01:51
at least when it comes to the satellite imagery stored on the efficient storage with server backend. The second problem, the huge load of requests is solved by deploying map server on Kubernetes cluster,
02:08
with defined auto-scaling. So, how does it look on the Kubernetes side? The happy user sends a request, which hits the Kubernetes cluster,
02:25
and more precisely, it hits load balancer. The load balancer request is redirected to one of the nodes. And what's important, the number of nodes depends on the number of GET requests hitting the cluster.
02:45
We define it using the Kida, Scaled Object Definition, and the dramatization matrix. So, and from there, from the node, there's only one node per node to maximize the effectiveness of map server,
03:01
and from there, the request goes to the set matrix and hits the source data, which is CLG, or JPG 2000, which is not so cloud optimized, but about this in a moment. And how does it look on the map server side?
03:21
So, map server is using the super tool called Time Index, which is the spatial index for raster layers. It's a footprint vector, storing the path to file, and its metadata, like polarization for Sentinel-1,
03:42
or cloud cover for Sentinel-2, also time is possible to use. And we are using the virtual storage infrastructure by Jida to hit the S3, and from the S3, we are using the COG files, or JPG 2000, which is, like I said, not so optimized,
04:04
but offers the variety of options when it comes to spatial resolution, and within the map file, which is a configuration file for a map server, we are able to create a pyramid from it. So the data for 10 meters resolution is only requested for really high zoom levels.
04:30
And yeah, I ran some tests, I did run some tests using the JMeter load tool, and as you can see, the throughput increases with time,
04:42
the cluster is scaling, and even though the throughput is so big, the average response time remains stable and even decreases. And here, unfortunately, this is a little bit blurry, but you have to believe me that this is the screenshot of a horizontal port-out-of-scale definition from Kubernetes.
05:09
And thank you.