Cloud-Native Geospatial with JavaScript
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 351 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/69228 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Production Year | 2022 |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
00:00
Slide rulePoint cloudSoftware engineeringWeb browserGoodness of fitUML
00:13
Slide rulePoint cloudWeb browserData storage deviceServer (computing)Computer fileOperating systemLibrary (computing)Stack (abstract data type)GeometryGoodness of fitWeb browser1 (number)Link (knot theory)Field (computer science)Visualization (computer graphics)Connectivity (graph theory)RoutingServer (computing)Slide ruleBefehlsprozessorPoint cloudTesselationWeb 2.0QuicksortSemiconductor memoryObject (grammar)MereologyProjective planeDifferent (Kate Ryan album)Process (computing)CurvatureMultiplication signNeuroinformatikMathematical analysisCASE <Informatik>Point (geometry)Data storage deviceDisk read-and-write headService (economics)Social classSoftware maintenanceLatent heatVector spaceMedical imagingComputer configurationView (database)Functional (mathematics)Library catalogType theoryLevel (video gaming)Partial derivativeReading (process)Category of beingSet (mathematics)Utility softwareMetadataInformationoutputOnline helpImplementationRevision controlElectronic mailing listRight angleMathematical optimizationPlug-in (computing)Computer animation
07:04
CodeFile formatPoint cloudBitWeb 2.0Design by contractCodeLibrary (computing)File viewerSoftware testingComputer animation
07:41
Array data structureFile formatReading (process)Library (computing)Mathematical analysisSlide ruleBootingSoftwareAxonometric projectionPlug-in (computing)Point cloudShape (magazine)Device driverTelecommunicationClient (computing)Component-based software engineeringVisualization (computer graphics)Interactive televisionText editorStapeldateiFrequencyMonster groupGroup actionWeb browserStack (abstract data type)WordLibrary (computing)Web 2.0Different (Kate Ryan album)File formatReading (process)Point cloudCASE <Informatik>Visualization (computer graphics)Single-precision floating-point formatCanadian Light SourceMappingComputer fileQuicksortExterior algebraException handlingLatent heatClient (computing)MereologyProjective planePlanningConnectivity (graph theory)ImplementationSlide ruleWeb browserComputing platformCurvatureVector spaceOpen sourceTowerInstance (computer science)Stack (abstract data type)TesselationArray data structureData storage deviceMIDIDimensional analysisFitness functionGoodness of fitServer (computing)Object (grammar)Level (video gaming)Shape (magazine)Revision controlGeometryProcess (computing)Operating systemComputer animation
13:44
Computer animation
Transcript: English(auto-generated)
00:00
Thank you. Good morning. My name is Mathias. That's Daniel. We're both two more or less freelance software engineers working a lot with cloud native and JavaScript and basically in the browser. So It would probably be good if Matt was talking now and we were after because now we need to go
00:21
Into the like high level information about what cloud native is. Otherwise that would be his part. So cloud native geospatial basically means that you have very large data sets And you want to store them usually with like setup server or web map service or something like that But in this case you would store them in a
00:42
like different way Specific way that makes it cloud native and you can then place it easily on object storage for example And then only download the parts that you actually need that is in the end faster and cheaper both in costs and in time And as such you usually don't need a server, which is called so-called serverless design
01:04
So you probably for example, if you have a geotiff on online that is cloud optimized You wouldn't need a tile server like tie-tied or so And what we're going to do today is to try to convince you that you may go this new route
01:21
Making stuff cloud native and that you can work actually with all the data available Usually in the browser as well You don't need a server that pre-processes the data for you and makes it available for the browser Like it was in the past We are focusing on JavaScript and typescript and if there's nothing available in that natively then we also look into a web
01:41
assembly We ignore node.js for now because that's again server side So quick overview of what we were we are tackling here So that's first the part of discovery which we heard before from Pete is stack Then we're having two categories types of data, which is rest or array based data
02:04
Which is cogs cloud of my view tips PM tiles cloud of my star and czar and then for vector data we have geo parquet flat geo buff and thought of my point clouds and For cloud native processing you have open yo
02:20
A short disclaimer we're working on a couple of projects that we will mention in this Slide and all the ones that have a star there that is maintained by us So just as a disclaimer that we might be biased in some of them So we heard about stack And stack in the browser very much fields native because it's just chasing right Jason is coming from the JavaScript ecosystem, basically
02:46
So reading it is like very easy you don't usually need it really a library for it So there is no general-purpose libraries like pi stack and you can use Stack with something else then Python of course Pete was mostly focusing on Python. I think
03:03
so That's good. And you have also like the stack items of you Jason so you can all use every tooling that is around for geo Jason processing there's a couple of Tools that help you to actually Work with the data in more advanced ways. For example If you have like old stack files and want to migrate into the latest version
03:24
There's tech migrate which just takes your stack item and collections and migrates into the latest version There's stack fields which converts tech metadata to HTML and all that is then used again in more high level Implementations like for example stack browser open your view components and stack layer
03:42
Which all are for visualizations tech browser you've seen before in the stack talk which is for basically a UI for for stack Where you can browse through all the catalogs and API search in them visualize them, etc There is a small utility class in stack browser as well that you might want to spin out at some point
04:04
Just with half of a head of functions To get it asset specifically or link or whatever And then stick layer is a leaflet plug-in basically to visualize tech items and collections which also includes coke visualization if there is cloud optimized geotiff in the
04:21
Item basically in the assets and view components is basically a web component Library for also for stack items and and collections originating in openio So you can load that basically and then you have HTML text available without like really needing to write JavaScript where you can input your Data and show it just in the browser without any like coding involved except for HTML of course
04:49
So this slide is on cogs cog stands for cloud optimized geotiff and cloud optimized geotiff is a
05:02
TIFF image with geospatial metadata, and it supports partial read requests Which mean you can grab just part of that image You don't have to read like the whole thing So there's a lot of different options when it comes to reading visualizing and analyzing
05:21
We're not going to have time to go over every one But I have a question for the audience so raise your hand if you work with cloud optimized geotiff's oh Wow okay, that's no no no keep them up keep them up Okay now for anyone who didn't raise your hand look around and if you have any questions
05:43
You find one of those people and you ask them now or at the icebreaker So yeah lots of options there's low-level options like geotiff JS, which is an amazing library Fabian Schindler is the the lead maintainer there
06:04
And eoxis supported it, thank you in a lot of the the reading libraries depend on that there's also Libraries that will try to get gdal into the browser those come with a lot of features But you also have to pay for the the large file size
06:22
But you know as computers get faster. It's not it's not unreasonable to check those out Visualizing there's a lot of different options there from sort of more memory optimized CPU based visualization to real high-powered GPU enabled visualization And then there's also analysis
06:42
options too But I forgot to say is that there are a lot of names in here and listed a lot of them some have links You can go to foster G to lieutenant at the e and then the slides are there you can just click the links Google the names if there are no link Just so that you don't need to write all these like lists down
07:05
Then the next cloud optimized format is clocked up to my point clouds There is a library available That is called copy CJS, which is based on typescript and then has some kind of reader in the background. Which is web assembly Unfortunately for that that is still very limited in the documentation, so there's just one example available no API documentation
07:26
So you need to be prepared to dig into the copy siege code or contract the company that is in actually maintaining it So there's a bit more work to be done, but there's also an online viewer that you can use to actually test it out
07:45
So czar Is a format for chunked or in other words tiled compressed multi-dimensional arrays you might be wondering What's the difference with cogs? Well czar will have more than one dimension The the tiles will be kind of large, so it's a good fit for climate data
08:08
There's it's it's sort of a newer format Geotiffs have existed since the 80s In cloud optimized more recent, so there's still With some of these newer cutting-edge formats. There's more work to be done
08:23
But there's a really awesome Step forward that carbon plan did with visualizing czar data On the web, and there's also a couple different efforts in JavaScript So we're looking forward to tracking this and seeing where it goes
08:45
Then another one that has recently been published as a specification and still is not really finished But it's it's there and it seems to be the step forward for vector is geo parquet There is nothing specifically implemented yet for geo parquet in JavaScript
09:01
But there is like there were two implementations, but geo parquet or parquet itself is pretty Difficult so they have been abandoned in favor of web assembly based implementation Which is parquet bosom And that is also used in Lotus GL
09:20
So if you're like a deck deck GL user or want to go into any like GL based with PL based Yeah, visualization then Lotus GL might be the place to go as it's also available for a couple of other Formats that you're talking about like flat geo buff
09:40
Flat geo buff is an amazing new format by And I apologize, I'm an American I actually live in the south so you can imagine that the accent it Bjorn Hartell apologies on the pronunciation, but Really, it's find that guy here on github and ask him about it. It's been getting a lot of
10:06
Support across the community on a bunch of different projects most importantly gdao Which means you can really use it everywhere and Reading is fully supported in
10:20
JavaScript There's there's a lot of different work going on and visualization and reading but sort of the main takeaway would be you can use it to Replace a shapefile and it's cloud optimized Then we have cloud optimized single file map tile archives
10:43
Which is basically a replacement for your tile server in the background that can be used on object storage There are two of them that one is PM tiles. The other one is kotar, which is cloud up my star Which sounds rather weird, but is the thing Yeah kotar there is a general-purpose reader for kotar and PM tiles as an implementation for both leaflet and map Libra
11:09
and Openio is the last part which is about processing that is a specification to unify all the Processing platforms out there with a specification There's a JavaScript client available. There are realization components and a web
11:24
component for editing in a non coding way, basically So there you go. And now the probably most important or interesting slide is in the background now the conclusion like we have listed now all the Like formats that we have available
11:41
And give some hints about whether Libraries available that the visualization mapping tool is tooling is available and really would recommend it in the browser and How good the documentation and examples are? so you see there is a couple of really JavaScript implementations as libraries available and if there's not there is a
12:03
web assembly Alternative available so you can all use all these cloud of my formats in the browser For most of them there is also visualization available except at least in an open source Way, not for cloud optimized pong cloyds kotar and geo parquet right now
12:22
and for those are depend very much on the use case and What we would recommend in the browser right now without any doubts is a lot of nice geo tips kotar PM ties and stack And for the others, it depends usually on the use case like for example
12:40
Cloud on the spot clouds. I said the documentation is lacking So there's something that you still need to like you would need to dig into the actual Library and understand it deeply. Otherwise, it's hard to implement it and get it running For the jet flat geo buff. We have for example the issue that the files get large You don't have overviews like in Cox, so you
13:00
Can't like get broader more generalized instance of your vector data For Gia parquet, it is implemented but the geo part is missing in the library So there is no no special handling. You still need to add the geo part to the to the parquet version library Openio, it depends on the use case and for czar there is an issue that
13:22
Depending on how it start the chunk size is an issue like for some as our fights chunks are very large So it doesn't really it's handled well in the browser But if you can manage to optimize that to chunk size that are Small enough for the browser then it actually works pretty well
13:42
Yeah, that's it I think right yep