Fast rendering from vector tiles in deck.gl
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 351 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/69060 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Production Year | 2022 |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
FOSS4G Firenze 2022184 / 351
1
7
13
22
25
31
33
36
39
41
43
44
46
52
53
55
58
59
60
76
80
93
98
104
108
127
128
133
135
141
142
143
150
151
168
173
176
178
190
196
200
201
202
204
211
219
225
226
236
242
251
258
263
270
284
285
292
00:00
Vector spaceArtistic renderingPrincipal idealPoint (geometry)2 (number)Binary codeLibrary (computing)MappingBuildingVisualization (computer graphics)VolumenvisualisierungScatteringClient (computing)Computer architectureLevel (video gaming)Open sourcePlotterCartesian coordinate systemSoftware developerPhysical systemSystem callTesselationVector spaceArc (geometry)BitCASE <Informatik>Set (mathematics)DiagramWeb browserConnectivity (graph theory)TessellationCore dumpGraphics processing unitFocus (optics)Context awarenessVariety (linguistics)File formatReliefServer (computing)Multiplication signSoftware frameworkRow (database)Perspective (visual)Computer animation
03:29
Revision controlThomas BayesNeuroinformatikPosition operatorVisualization (computer graphics)Type theoryElectronic mailing listObject (grammar)RadiusGraph coloringDrag (physics)Functional (mathematics)Computer fileLibrary (computing)Revision controlZoom lensCASE <Informatik>Personal area networkLine (geometry)Demo (music)PixelFile formatMappingCore dumpView (database)CodeMereologyWeb 2.0Level (video gaming)State of matterGame controllerMaxima and minimaJSON
05:29
Revision controlFunction (mathematics)ResultantPoint (geometry)Level (video gaming)Vector spaceTesselationWeb browserLimit (category theory)Computer animation
05:47
Dean numberWeb browserLimit (category theory)Row (database)Electronic visual displayStreaming mediaComputer animation
06:10
VotingVolumenvisualisierungSet (mathematics)Web browserTesselationVector spaceInformationComputer animation
06:33
System of linear equationsRandom matrixIntrusion detection systemFile formatVector spaceMean value theoremBootingPoint (geometry)PolygonPrice indexGeneric programmingTesselationFile formatComputer architectureVector spaceComputer animation
07:11
Vector spaceDifferent (Kate Ryan album)TesselationCategory of beingDegree (graph theory)Pattern languagePower (physics)Template (C++)Formal languageMean value theoremTessellationFile formatTouchscreenVector spaceJSONXML
08:05
Zoom lensDemo (music)PixelGraph coloringTouchscreenOpen sourceSoftware developerLevel (video gaming)Visualization (computer graphics)Library (computing)Drop (liquid)Cartesian coordinate systemConfiguration spaceComputer animation
08:54
Vector spaceExtension (kinesiology)Power (physics)MereologyTesselationCartesian coordinate systemVector spaceFilter <Stochastik>Run time (program lifecycle phase)Multiplication signCategory of beingMetadataExtension (kinesiology)TessellationRaster graphicsRange (statistics)Computer animationJSONXML
09:55
Read-only memoryClient (computing)Arrow of timeProcess (computing)Software developerArrow of timeGraphics processing unitProcess (computing)File formatAreaCircleSpeicherbereinigungStandard deviationMean value theoremLibrary (computing)Auditory maskingWeb pageArithmetic meanTouchscreenGraph coloringShape (magazine)Real-time operating systemUser interfaceCodierung <Programmierung>Object (grammar)Extension (kinesiology)VolumenvisualisierungPA-RISC 2.0GeometrySquare numberBefehlsprozessorPolygonMereologyPosition operatorRange (statistics)BitPower (physics)BootingDefault (computer science)TesselationSemiconductor memoryAsynchronous Transfer ModeMultiplication signLevel (video gaming)CodeCore dumpFrame problemCartesian coordinate systemPixel2 (number)ParsingBinary codeVector spaceLeast squaresComputer animationXML
15:41
Auditory maskingComputer fileElectronic mailing listCodeVisualization (computer graphics)Operator (mathematics)JSON
16:26
Time zoneTime zoneAuditory maskingMultiplication signSet (mathematics)
16:48
Default (computer science)Instance (computer science)Point (geometry)Line (geometry)PolygonInstance (computer science)Default (computer science)PolygonFile formatPoint (geometry)Line (geometry)Auditory maskingCircleBitTime zoneCategory of beingComputer animation
17:14
Wide area networkGraphics processing unitLaserCategory of beingState of matterBoundary value problemArc (geometry)Line (geometry)Instance (computer science)Computer animation
17:33
Instance (computer science)Graphics processing unitPolygonField (computer science)Object (grammar)Line (geometry)State of matterContrast (vision)Visualization (computer graphics)Boundary value problemComputer animation
17:51
Arc (geometry)CirclePoint (geometry)Different (Kate Ryan album)Time zoneMultiplication signSet (mathematics)State of matterInstance (computer science)Auditory maskingBuffer solutionComputer animation
18:36
GUI widgetVotingState of matterTessellationUser interfaceRow (database)Vector spaceSet (mathematics)Computer animation
18:52
Digital filterTotal S.A.GUI widgetParameter (computer programming)Right angleGUI widgetLeast squaresSystem callAreaProjective planeProcess (computing)2 (number)Computer animationEngineering drawing
Transcript: English(auto-generated)
00:00
Okay, hi everyone, thanks for the intro. Pretty accurate, so I, as you said, I'm working at Carto, which is a geospatial company, but my main focus is on Deck.GL, an open source rendering framework, and integrating that with Carto systems. So this talk is gonna be focusing
00:21
on rendering large data sets using vector tiles in Deck.GL. But first of all, I'm gonna give a little overview of Deck.GL, because I think a lot of people have heard of it, but maybe not familiar with how it works in detail. Then I'm gonna talk a bit about vector tiles, how to render them, style them, and filter them.
00:42
And finally, I'm gonna go onto two developments on more of the technical side, which we've done the last year to make vector tiles work really nicely. One is binary data format, and the second is GPU masking. Okay, so a quick recap of Deck.GL.
01:02
It's a JavaScript library for big data visualization, and the core component in Deck.GL is something called a layer. So this is something which lets you define in a declarative way how you'd like to map some data that you have, and this can be in a variety of data formats,
01:20
to a visualization. And even though this is running in the browser, then it runs pretty quickly, thanks to using the GPU, which is the graphics card in the browser, to accelerate the rendering using WebGL. I would just make a call out to my colleague, Bora, who yesterday had a talk, which was more kind of an introductory overview of Deck.GL.
01:43
So I would encourage you to take a look at that at a recording, if you find this interesting. So first off, I would like to kind of clear up something which I find is something of a misconception with Deck.GL. Deck.GL is not a base map library.
02:01
It's a pure visualization library that sits on top of a base map library. So you would typically use Deck.GL on top of another rendering library, a map rendering library, like MapLibre, or Leaflet, or Google Maps. And then Deck.GL will draw its layers on top in a separate context.
02:21
So that is what this diagram here is showing. We have a collection of Deck.GL layers, one, the blue arc layers, and a second one, a scatter plot layer. So this is just a bunch of points. Deck.GL is gonna draw these together onto one canvas. And then separately, a mapping library here, the example is Mapbox,
02:42
is gonna draw the streets, the labels, the buildings. And these two are gonna get linked together by the browser in a way that it appears to be a seamless visualization to the user. So from the user's perspective, they can't tell that it's basically two libraries at the same time.
03:01
And one great use case that this architecture gives us is the ability to swap out the base maps. So you can build your application using Deck.GL and have Google Maps. But if later on the client says, actually, I want to switch to my own tile server, and you want to use Map Libre instead,
03:21
then you can just swap out that library and everything will work as before. You're not being tied to the API of the base map library. Okay, I just have a short demo here of how we can do the most kind of basic mapping of data to a layer. And if some of you have your computers, then I've put a live version here on JS fiddle
03:41
that you can play around with and see that really doing a simple visualization in Deck.GL is like maybe 20 lines of code. So here I have some example data in JSON formats. It's just a list of what are airports with each object having some coordinates and each airport having either a major or a minor type.
04:05
To render this in Deck.GL, all we have to do is initialize a Deck.GL object and define an initial view state, which is basically saying, where is our map gonna start? So what latitude, what longitude and what zoom.
04:20
Tell Deck.GL that we want this to be interactive by adding the controller. So when we just like click and drag and pan and zoom and all of these things that we're used to from web maps, the map is gonna update. And then the core part is this collection of layers. So here we just have one. And you can probably understand
04:40
what some of those things are doing. So I'm just pulling the data from this JSON file. I'm setting the radius to be a minimum of three pixels. But the core part to understand, and this is quite different from other visualization libraries, are these two functions in the middle, which are called accessors. So this is the way that Deck.GL is informed
05:00
how you'd like to map your data to the visualization. So this function, get position, is gonna be called automatically by Deck.GL on every object in your data array. And it's gonna pull out the position and say, okay, this is where I want to have it on the map. And similarly with the fill color, it's gonna look at the type.
05:21
And if it's a major airport, then it's gonna color it with one color. So here in this case, it's red. And if it's not, it's just gonna use black. So this is the result that you'd get from this map with some points on it, nothing surprising. Okay, so what are vector tiles?
05:41
The example that I was just showing you now, is just some static data. And this has got some pretty big limitations in the browser. If you are rendering a large data set, so if we're talking like a billion rows or something like that, there's just no way that you can download this much data into the browser in a fast way.
06:02
If you're trying to solve this problem, another way that you might sort of display all this data is to stream it in piece by piece, but still you're gonna be hitting browser limits and it's not gonna work. So the correct solution to make such a big data set renderable in the browser is to split it and aggregate it into tiles.
06:21
So when you have the whole world, then you have an overview. And then as you zoom in, the browser is downloading more and more detailed information to draw the tiles that you're actually focusing on. And vector tiles is basically quite a generic concept. Like it's essentially a way of tiling geographic information, which can be points,
06:41
polygons, blinds, or even spatial indices in a tiled manner. So in Deck.GL, the first format that we supported is the most common one, Mapbox vector tiles, but really because of the way the architecture of Deck.GL works, you can plug in pretty much any format.
07:00
You could have your tiles contain JSON or GeoJSON or CSV or well-known binary, pretty much anything that you'd want. It'll still work nicely. So an example of how we'd go about styling vector tiles, it's pretty much the same as with the static data set. The only difference is that rather than downloading
07:22
the static data, we point Deck.GL at an NBT endpoint using this template here. So as the user is gonna move the map around and pan and zoom, then Deck.GL is gonna automatically download the individual vector tiles. And it's then gonna use the same styling language
07:42
as we had before, except it's gonna apply it to each tile individually. And here, this pattern of accesses comes in and shows kind of how it's nice and powerful because you don't actually have to worry about the data format you're working with. You just have to worry about how do I map the individual properties of my features
08:02
to something visible on screen. Yep, I think I've covered this. I'll just show a quick demo here to show that in practice, when you're trying to create a visualization, people would use some kind of UI where rather than going along and typing colors
08:23
and manually setting pixel sizes, you can use a UI. And here, this example is an open source library called Kepler GL, which is built on top of Deck.GL. And here, you can be using a bunch of predefined color styles, drop boxes, sliders,
08:41
configure your map in a way that looks good for you. So typically, you'd have some kind of cartographer or designer who would create this and then let the developer know, okay, this is the style that I want, and you would put this into your application. Okay, so now onto what I think is the part that really shows some of the power
09:02
of Deck.GL and vector tiles, which is filtering. So because in a vector tile, all of our features are stored with their metadata, we can instruct Deck.GL to filter them at runtime in a way that we couldn't do with raster tiles.
09:22
And the way this works with Deck.GL is that you instantiate an extension, which enables this data filtering, and you add a extra accessor here called get filter value. What this is gonna do is it's gonna instruct Deck.GL to pick out a specific property to use for filtering.
09:44
So here, this property is gonna be time, and separately, I've specified in the filter range the filter that I want the GPU to do the filtering based on. What is very powerful about this approach is that this filtering is happening
10:02
directly on the graphics card. So all of the data is uploaded there with this defined filter of time, and then on every frame as the user is changing the user interface or creating an animation, then the only thing that's being updated between the JavaScript and the graphics card
10:21
is this filter range. So the animation is running super smooth, like no processing is being done in JavaScript at all, and it's as if you weren't doing anything in the web page so it's gonna run very, very quickly. Okay, a couple of improvements that we've done
10:42
in this last year that actually make these things possible. I'd first wanna talk a bit about how we work with the data, the binary data in the NVT layer, and the next even more powerful way of doing filtering on the GPU using the new mask extension.
11:04
So historically, the way the NVT layer worked was that we had some code which could decode NVT tiles into GeoJSON, and then DecGL could render GeoJSON. And this is quite a common way that libraries work because GeoJSON is more or less the de facto standard
11:22
because it's easy to understand for humans, but that doesn't mean that it's fast to render. And the core problem is quite clear to see from this kind of like stage of rendering that we have in DecGL, where when we're looking at the actual vector tiles, then they are in a binary column-based format.
11:43
So all of the positions in memory are right next to each other, and then after that we might have all of the colors, and then after that we might have the sizes. This is completely different to JSON where we're grouping things by object. So converting from one format to another one is quite expensive. When it comes to actually drawing using the GPU,
12:05
then we have to go back to a column-based format. That's just something the GPU requires, and there's no getting around that. So essentially we're going from one format only to go back to the other one, which first of all is a lot of CPU power, but secondly is creating lots and lots of garbage,
12:22
which means the JavaScript engine has to do garbage collection, which means your webpage kind of slows down and stutters and leads to bad user experience. Unsurprisingly, the way that we fixed this was by getting rid of this middle step and rewriting the NVT layer in a way that it could deal with the binary data natively.
12:45
So this was a great first step to getting better performance. Basically the parsing speed has doubled, but one thing that is quite exciting about this is that it makes it much easier to introduce other column-based formats.
13:00
So some of the other talks at this conference have talked about things like GeoParquet and GeoArrow, and these are super, super fast formats, like some of the performance charts that I've seen here, then the parsing and filtering speed is super quick. But an added bonus of these formats is that they're almost the same as the format that we want to upload
13:20
to the graphics card. So not only are you getting better filtering with these new formats, but when it comes to actually rendering them to screen, it's gonna be much, much quicker. So this is something that we're looking at in Deck.GL where we can take GeoArrow specifically and have a loader for it that can natively draw out onto the graphics card.
13:48
So this binary mode that we have in the NVT layer, because we're using access to style the data, is essentially completely transparent to the user in most cases, if you're not doing something quite complex.
14:01
So you basically can just switch it on and it is on by default and things would just run quicker. You don't have to care about it. You just use your accessors through the styling and your applications will work. Okay, so final part I'd like to talk about is mask extension,
14:21
which is kind of similar to like a Lasso tool that you might have in Photoshop where you can define one layer as a mask for another layer and this masking is being done in real time on the GPU. So this example we have here
14:41
is from a UI tool that we have at Carto where the user can just draw a shape on a map and move it around and it will automatically filter the polygons that we have inside of that tool. So the way this works is that we have at least two layers
15:01
where one of them is designated as a mask and you can draw anything into this layer. In this example, I'm just drawing a square and then you have another layer which references this first layer and its data is then masked. So this scatterplot layer that's second from the left
15:21
is not being masked. When we apply the mask then only those circles which are within this masked area remain on screen and we can also choose to switch to a different mode where we're not masking by object but masking by kind of pixel and I'll show you some examples of this in a second.
15:42
So here is an example of how we do this in code. Again, because we're defining this in a declarative way it actually becomes pretty easy to create a quite complex visualization. So here I'm loading one GeoJSON layer in from a GeoJSON file, so the whole of Italy
16:00
and instructing DEC-GL to rather than draw it to screen draw it with this mask operation. So we won't actually see it in our visualization and then the second layer that's below it is more or less the same as the one we had before where it's just a list of airports but we're saying, okay, take a mask with mask ID Italy mask
16:21
and apply this to this layer. So we're gonna end up with airports that are only in Italy. So here I have an example of something similar where one data set is all the cities in the world. Another one is a set of time zones and as the user clicks a time zone
16:40
then that becomes the currently active mask and it's gonna mask out only the cities within that time zone. So I'd like to dig in a bit deeper on this masking by instance. So depending on what data format you have, if it's points then those by default are gonna be masked by instance
17:01
because it makes sense that you keep your entire circles, you don't wanna clip your markers but things like lines and polygons generally will be clipped but this can be overridden by setting a property. So this example here shows all the flights between cities
17:21
and in the US and as the user clicks on a certain states then it's masking those arcs by instance so that these lines are not getting cut off by the boundary of the states but it's keeping the whole object. By contrast, the lines and polygons within the state,
17:40
so the roads and fields are getting clipped at the edge. So you get this nice visualization of the states being clipped at the boundary but the lines going everywhere. You can even define more than one mask at the same time up to four. So here I have one mask doing the same thing as before where the arcs are going to different states
18:03
but then the second mask is masking a different set of layers which are showing just the points for the airports and then also buffer zones of I think 100 kilometers around each airport. And here I've done the opposite thing where I've taken these circles
18:22
which normally would be masked by instance and said, no, I want these to be masked like clipped. So the buffers don't extend beyond the edge of the state. Okay, one more minute and I have just one more example.
18:41
Here this is taking all of the things that I've been talking about. So this is a vector tile data set of a billion rows of population data in the US and the user interface here is both filtering it down based on some parameter using this widget on the right hand side and then further filtering it down using this lasso tool.
19:04
Okay, that's pretty much all I have. I'll just use the last 30 seconds to say that we're looking for contributors to join our project. So if this seems interesting to you then please check out our GitHub or our Slack channel and I'll also just do a quick call out for my company Carto.
19:24
If anyone is looking for a job in this area then come talk to me or check out our website. And with that, I say thank you and any questions.