What exactly is "Cloud Native Geospital", anyway?
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Title of Series | ||
Number of Parts | 17 | |
Author | ||
License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
Identifiers | 10.5446/54378 (DOI) | |
Publisher | ||
Release Date | ||
Language |
Content Metadata
Subject Area | ||
Genre | ||
Abstract |
|
2
5
6
8
10
11
17
00:00
Axonometric projectionDisk read-and-write headSoftware developerFinitary relationSoftwareGoodness of fitSelf-organizationEvent horizonBitDisk read-and-write headGeometrySoftware developerGeneral relativitySoftwareTouch typingComputer animation
00:28
DecimalWeb page
00:36
Point cloudSoftwareStandard deviationPoint cloudWeb pageSoftwareLevel (video gaming)Computer animation
00:54
Complete metric spaceLibrary catalogPoint cloudStandard deviationPoint cloudData storage deviceRaster graphics2 (number)Standard deviationCore dumpFile formatLibrary catalogMetadataTemporal logicAdditionGeometryBitForm (programming)EmailSoftwareMiniDiscVector spaceComputer animation
02:20
Point cloudClassical physicsFile formatMedical imagingWeb browserGroup actionRemote procedure callRaster graphicsPoint cloudXML
02:46
MultiplicationData storage deviceOpen setObject (grammar)Configuration spaceComputer fileProduct (business)Point cloudComputing platformGoogolGoogolMedical imagingWebsiteGroup actionConnected spaceMultiplication signPoint cloudStructural loadComputer animation
03:16
Electronic visual displayUniform resource locatorPoint cloudSample (statistics)Instance (computer science)Lambda calculusScale (map)Open sourceFile formatStreaming mediaSoftwareVolumenvisualisierungView (database)Decision tree learningStructural loadUniform resource locatorTesselationWeb 2.0Field (computer science)Regular graphLevel (video gaming)Shared memoryZoom lensComputer filePoint cloudData storage deviceGoogolComputer animation
03:44
SpacetimeSharewareSpacetimePoint cloudUniform resource locatorLink (knot theory)Level (video gaming)Interface (computing)Set (mathematics)Standard deviationWeb 2.0Raster graphicsComputer fileZoom lensGoogolComputer animation
04:26
Temporal logicLibrary catalogPoint cloudDifferent (Kate Ryan album)Program slicingMaxima and minimaCountingInternet service providerVisualization (computer graphics)Medical imagingAreaStack (abstract data type)WebsiteFormal languageSlide ruleDigitizingJSONXMLUMLComputer animation
05:54
Library catalogFormal languageStack (abstract data type)Source codeJSONXMLProgram flowchart
06:08
Convex hullExecution unitLink (knot theory)Compilation albumDigital object identifierCore dumpStack (abstract data type)Computer fileBuildingBlock (periodic table)Source codeXML
06:19
Wage labourInternet service providerPermianCore dumpComputer fileWeb pageSingle-precision floating-point formatUniform resource locatorXML
06:30
Sample (statistics)Axonometric projectionOpticsView (database)GeometryAnglePoint cloudPlane (geometry)Data typeSynchronizationSpacetimeNetwork topologyDemosceneLibrary catalogMIDIDependent and independent variablesPoint cloudInteractive televisionWeb 2.0Medical imagingUniform resource locatorLibrary catalogWeb pageTesselationWeb browserLevel (video gaming)Row (database)Series (mathematics)Computer fileComputer animation
07:15
SimulationCoprocessorDependent and independent variablesPlane (geometry)DemosceneMetadataSystem callSample (statistics)View (database)GeometryAxonometric projectionPoint cloudTouchscreenMathematical analysisWeb pageComputer animation
07:27
Web browser
07:38
CoprocessorPlane (geometry)DemosceneMetadataPoint cloudSample (statistics)View (database)GeometryAxonometric projectionSystem callStandard deviationComputer animation
07:50
Point cloudStandard deviation
08:07
Reduction of orderShift operatorReal numberMathematical analysisMultiplication signTemporal logicMathematical analysisDifferent (Kate Ryan album)Universe (mathematics)Basis <Mathematik>Shared memoryMachine visionPoint cloudCubeOpen setProjective planePotenz <Mathematik>Differential geometryPhysical systemComputer animation
09:31
Standard deviationTouch typingMultiplication signWebsiteLink (knot theory)EmailAddress spaceComputer animation
Transcript: English(auto-generated)
00:00
Good morning, good afternoon, and good evening. Thank you to our lovely organizers for inviting me to contribute to this event today. My name is Sarah. I'm based in Austin, Texas, USA. I'm the head of developer relations at Planet, which means I get to do a little bit of a lot of things. I'm lucky enough today that what I do and what I'm passionate about have a lot in common,
00:20
namely software, geo stuff, and people. If any of that speaks to you, I'd love to hear from you, so please do get in touch. My goal today is to give you a gentle introduction to cloud-native geospatial. What is it exactly, and why does it matter? So let's start by getting on the same page by answering the what. The simple answer is what you might expect from the name.
00:41
Cloud-native geospatial is geospatial that assumes you're running on the cloud. At a broad level, cloud-native geospatial, or CNG, refers to an ecosystem of standards and software that's built from the ground up, or natively, for the cloud. This whole ecosystem starts with a question. What if geospatial tools and software
01:01
were built from the ground up for the cloud? What if you never had to download data to do GIS or to do remote sensing? So if any of you are like me and you have a background working in GIS more than a decade ago, you'll remember when downloading data was the brave new world. Like, downloading was the new shiny
01:20
that not everybody was doing. For a lot of us, we were still stuck ordering our data by mail. Like, I remember physical disks that were shipped to you. It took days, at best. Downloading was fast compared to that. Now, we're asking the question, what if you never had download data? What if it was just immediately available? So at the center of the cloud-native geospatial ecosystem
01:42
there's two standards. The first is cloud-optimized geotiff, or COG, which is emerged as the core format to store geospatial raster data in. Second standard is the Spatial-Temporal Asset Catalog, more commonly known as STAC, which complements COGs with additional JSON metadata.
02:01
STAC also works with a variety of other emerging cloud-native geo formats, like formats for vector data, as well as legacy geo data formats, like non-cloud-optimized geotiffs. I'm gonna dive a little bit more into each of these standards as we go. So first up is cloud-optimized geotiffs. A COG is a special form
02:20
of one of the most classic file formats of remote sensing, the geotiff. If you've ever worked with geospatial raster data, you're familiar with geotiffs, and you've worked with those. But a COG is a geotiff that's specially formatted to work better on the cloud, which lets web clients request exactly the piece of the image they want. Instead of having to access the entire image before you can do anything with it, you can just replace the piece of the image
02:41
that you're interested in. Okay, so to explain this better, let's actually look at a COG in action. Here's one that's stored on Google Cloud. You'll see in the screenshot that it's 3.7 gigabytes, which is pretty large. It'll take 10 minutes to download if you're lucky enough to have a fast connection, which I do not have this week, so it takes even longer.
03:02
So that means before you can even begin to view the image for the first time, you have to wait and wait and wait and wait and wait. If it were a COG, then you can view it right away. You don't have to wait on the whole 3.7 gigabyte image to download before you can do anything. So this GIF shows a site that lets you load in any COG via its URL.
03:21
And then it turns it basically instantly into web tiles so you can share it, zoom around, look at it. So here I've taken that public URL for the file that was in my Google Cloud Storage. And then I put that URL in the URL field on this map and ta-da, I can instantly see it, I can zoom around, I can scroll about,
03:40
just like a regular slippy map with map tiles. Okay, so you don't have to believe me, you can actually try this for yourself. The first link here, it points at that same 3.7 gig giant raster file that I just showed you. That doves in space URL is just a short link to the longer Google Cloud link to save you some typing, but they're the same thing.
04:00
And then the second link is the web map interface itself. So try it for yourself. You can actually verify the COG at that URL is not a tiled data set. It is indeed over 3.7 gigabytes in size. If you try to download it, it'll take you 10 minutes, maybe more. But if you put it to URL in the COG tiler map, you can just instantly surf around, zoom in and out,
04:22
just like it were as if it were tiled, which is pretty cool. The other core standard in the cloud native geospatial ecosystem is stack, or the spatial temporal asset catalog. The stack originally came about when a bunch of folks got together and were trying to solve a really annoying problem in earth imagery.
04:40
So say you're interested in searching for all the available imagery in a given area. So you have to go to lots of different sites that catalog data from lots of different providers, just to be sure that you're searching everything. There's gonna be data from PlanSat or Sentinel or Planet or Digital Globe. All of these different providers have different ways of accessing their catalogs.
05:01
And even if they've got APIs, the situation's just as bad. Every API is gonna serve it slightly differently. So on this slide, every one of these visualizations is powered by a slightly different API, just to show you the vast differences. So each catalog and each imagery API is gonna have a different way
05:21
of organizing their slice of the global data. So one API might have a min and max cloud cover or where another API might just have a cloud percent value. Or one API might count cloud percentage in an imagery from zero to one, normalized between zero and one, where another one has zero to 100.
05:41
So because every image provider, every image catalog has a slightly different way of representing their data and making it accessible and searchable, you're gonna have to search each one individually, which is kind of a pain. So what STACK does is it aims for a common language for searching imagery. Instead of having a different way of searching imagery or a different way of categorizing imagery,
06:03
STACK wants to make a common way of doing this, a common catalog approach. STACK focused on the smallest, easiest building blocks so that other people could innovate on top of it. So the core of STACK is just a JSON file. JSON file can be shared statically, you can email it to somebody, or you could also serve it dynamically through an API.
06:22
So with this core JSON file, every single record of imagery that's online can be represented as a webpage. So this lets people refer to a single location online for any geospatial asset. So any imagery, any image data that exists, you have a single canonical record that you can point to.
06:40
So in this example, there's a JavaScript tool that can take a stack of JSON files into a series of interactive viewable pages in your browser. And here we've got actually a cloud-optimized geotiff in this catalog. So the image is even browsable and you can preview it, you can slide around on the map on it, just like that web tile example previously.
07:01
And it's not downloading the image or copying it to another catalog, it's just referencing it from its canonical location, but here in this unified catalog. So really stack and cog work together the complimentary pieces of one ecosystem. This page, for example, gives you what you need. Everything that you used to use your desktop GIS for,
07:22
you can see here. So like you can go full screen, you can share what you're looking at with other people. You can start doing analysis work on the cog in your browser. You no longer have to move anything to your big clunky desktop GIS. So this is radically simplifying the idea of GIS,
07:40
making data much more accessible, much more available by doing just one small thing, which is creating a common standard of accessing and categorizing data. All right, so I've told you the what. What is Cloud Native Geospatial? It's an ecosystem built on two standards that are evolving and working together to make data more accessible.
08:02
But cool, what's so great about this? Let's talk about the why. The Cloud Native Geospatial ecosystem shares a lot of the same goals and a lot of the same vision that drives the Open Data Cube project. Cloud Native Geospatial concept, the whole purpose is really unlock geospatial data so it can have a wider global impact.
08:21
We wanna make global analysis of data easier, unlock data that's stuck in silos in different systems, in different APIs, which in turn is gonna make it easier for us to handle the explosion of data. If you're involved in the EO universe at all, you know that data has exponentially grown. The availability of data, the amount of data that's being produced on a daily basis
08:42
is just exploding compared to even a year ago, much less five or 10 years ago. So we wanna make it easier to avoid data duplication because when you've got this exponential explosion of data, you really don't wanna be duplicating it to get anything done. So we wanna enable streaming of the data, that's where the cog comes in, which lets us do a lot more real-time analysis.
09:02
It reduces the time to analysis for any of the data, which is great when you're talking about high temporal cadence EO data as we are in 2021 with a lot of our data. And finally, the real purpose, the real driving goal behind all of this is just to make geospatial data more accessible. When you make the ecosystem more accessible,
09:23
then spatial is no longer special. It is just one more piece of the data science that everyone around the globe is doing. So that said, if any of this is interesting to you, if you'd like to learn more or get involved, there's a couple of websites here, one for each of the standards, the stackspec.org and cog or cogo.org.
09:42
And each of these communities has a chat room. One is on Gitter, one is on Slack. These are short links to join. My name is Sarah, like I said, would love to hear from you. My email address is Sarah at planet.com. Please do get in touch anytime. And thank you so much for your time.