Vector tiles for fast custom maps

Video in TIB AV-Portal: Vector tiles for fast custom maps

Formal Metadata

Vector tiles for fast custom maps
Title of Series
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date
Open Source Geospatial Foundation (OSGeo)
Production Year
Production Place
Portland, Oregon, United States of America

Content Metadata

Subject Area
Vector tiles are becoming a common solution for fast clientside rendering of spatial data in both browsers and mobile devices. With the recent release of TileMill 2 Mapbox has made it easier to design and render vector tiles. This talk will cover the open source technology under the hood in TileMill 2 as well as other available tools. Also discussed will be the status of an emerging specification for vector tiles and recent advances in the format.
Keywords vector tiles tilemill2 tm2 mapbox rendering maps
Point (geometry) Computer animation Mapping Vector space Tesselation Cuboid Staff (military) Office suite Demoscene
Implementation Texture mapping Computer file Tesselation Multiplication sign Set (mathematics) Numbering scheme Web browser Tessellation Medical imaging Computer animation Vector space Term (mathematics) Endliche Modelltheorie Library (computing)
Complex (psychology) Medical imaging Computer animation Vector space Information File format Representation (politics) Probability density function
Medical imaging Open source Vector space Tesselation Connectivity (graph theory) Website Quicksort Attribute grammar Geometry
Scripting language Server (computing) File format Tesselation Java applet Client (computing) Mereology Thumbnail Social class Formal language
Scaling (geometry) Computer animation Vector space Different (Kate Ryan album) File format Artistic rendering Geometry
Type theory Texture mapping Computer animation Vector space File format Tesselation
Axiom of choice Computer animation Vector space Product (business)
Beat (acoustics) Group action Computer animation Vector space Tesselation Oval Network topology Right angle Data structure
Point (geometry) Array data structure Computer animation Vector space Key (cryptography) Data storage device Formal grammar Integer Geometry
Point (geometry) Computer animation Different (Kate Ryan album) Personal digital assistant Single-precision floating-point format String (computer science) Polygon 1 (number) Line (geometry) Codierung <Programmierung> Number
Trail Tesselation Multiplication sign View (database) Zoom lens Set (mathematics) Bit Code Tessellation Sign (mathematics) Computer animation Vector space Buffer solution Negative number Electronic visual display Right angle Endliche Modelltheorie Extension (kinesiology) Spacetime Geometry
Point (geometry) File format Tesselation Image resolution Zoom lens 1 (number) Volume (thermodynamics) Maxima and minima Mereology Symbol table Demoscene Computer animation Vector space Different (Kate Ryan album) Single-precision floating-point format Buffer solution Square number Energy level Self-organization Right angle Communications protocol Resultant
Texture mapping Open source Computer file File format Tesselation State of matter Feedback Uniform resource locator Latent heat Computer animation Vector space Personal digital assistant Ontology Cuboid Software testing Address space
Wiki Implementation Computer animation Different (Kate Ryan album) File format Website Data structure Number Condition number Formal language
Computer animation Variety (linguistics) Quicksort Computer-assisted translation Cartesian coordinate system Library (computing)
Focus (optics) Texture mapping Service (economics) Mapping Tesselation Software developer Multiplication sign Interface (computing) Parallel port Mereology Flow separation Particle system Computer animation Vector space Blog Cuboid God
Tesselation Interface (computing) Cellular automaton Projective plane Polygon Sound effect Maxima and minima Water vapor Metadata Machine vision Attribute grammar Computer animation Vector space Different (Kate Ryan album) Term (mathematics) Personal digital assistant Buffer solution
Point (geometry) Dialect Texture mapping Computer file Block (periodic table) Tesselation Image resolution Multiplication sign Interface (computing) Zoom lens Sound effect Set (mathematics) Bit Database Special unitary group Shape (magazine) Coma Berenices Medical imaging Computer animation Vector space Personal digital assistant Integer Endliche Modelltheorie Mathematical optimization
Scripting language Texture mapping Open source Tesselation Multiplication sign Expert system Bit Cartesian coordinate system Metadata Tessellation Computer animation Vector space Cuboid Endliche Modelltheorie Freeware
Default (computer science) Vector space Variety (linguistics) Tesselation Artistic rendering Coma Berenices Client (computing) Graph coloring Vector potential Geometry
Computer animation Open source Variety (linguistics) Real number Projective plane Sampling (statistics)
Satellite Texture mapping Vector space Network topology Zoom lens Interactive television Saddle point Line (geometry) Hill differential equation Theory
Filter <Stochastik> Server (computing) Texture mapping Touchscreen Matching (graph theory) Mapping Transformation (genetics) File format Tesselation Expression Client (computing) Medical imaging Message passing Vector space Personal digital assistant Operator (mathematics) Buffer solution Cuboid Communications protocol
Overlay-Netz Operator (mathematics) Multiplication sign Sound effect Hill differential equation
Slide rule Shift operator Electric generator Vector space Tesselation Term (mathematics) Artistic rendering Sound effect Machine vision
Slide rule Texture mapping Computer animation Cuboid Cartesian coordinate system Metropolitan area network
Computer animation Vector space Open source Demo (music) Query language Tesselation Profil (magazine) Set (mathematics) Line (geometry) System call Resultant Tessellation
Area Touchscreen Computer animation Query language Network topology Image resolution Range (statistics) Zoom lens Energy level Figurate number Computer font
Computer animation File format Personal digital assistant Software developer Direction (geometry) Polygon Bit Open set Cartesian coordinate system Flow separation Amenable group Geometry
thanks everyone for coming the so I am II Dean promire and I were from that box and on the talk instead of vector tiles from best custom maps on and summer box is based in
watching the scene and now it's seems cisco officers were about just over 50 staff at this point on so where we we're headed
today will basically I wanna dive deep into 1 vector tiles are my hope is that in a few years tiles will be so adopted that we will really need to talk about the details of male model does work but since they're is still a new idea or concept a lot of folks on I wanna dive deep and how they work and before before I start on the talking about about what I'm familiar the terms of the tools and the goals of vector tells for me but you know if if you've ever worked a vector data and in the browser you've really work affected tiles mean dating back to the early WFS implementations it's really that files and so there's a lot of present here and people of people been in wrong time what I'm attack that is a map OX effort to work on a spec a set of libraries and tools are very high performance the that so what is a vector top
well simply speaking it is like an image tile which were all familiar and it's on easy cash and serve rapidly us so that's divided data it's the same addressing scheme scheme as themselves as well on but as the x y . png
means the same thing as as the x y i . vector pdf which is the format that were working as so you can address it is 0 0 0 or at higher
involves the same same we request a tile on and can of vector tells
can represent many complex data layers just like an image tell can you can bake a lot information to and non where we're going with this is we want
a full stack to work affected tiles it's completely open source are so there's no proprietary components there's no but no PC you can't switch off another piece of works better for you so they should be just as common and easy and open source to work with as image style on but of course
lecture tiles should be better but they can contain all sorts of Rob vector data that you can do interesting things with and geometries of road names stereotypes building sites that they should contain are highly packed data that sufficient for rendering but they should contain attributes that you can do interesting things with around and yet they're very compact so the
format the were working on easily fits on a
USB thumb drive so it's 20 to 30 gigs and that's all of what the OpenStreetMap planet they about tiles the and should be
very fast to pass on both client side in a in a language like Java script or server-side you're doing during work with them and so are class I Linda distress just of course and we work and suppose phosphorus in the server side so this should be faster parts on an exceeding
vasopressors you should be all apart from a lazily or incrementally but you shouldn't have to read the whole thing just to get all say 1 layer of upper 1 feature or 1 geometry should go see just right into where you need and get would get what you want 24 you get which is a slight difference from our in textual formats litigious undertook which is what which work fabulously as vector tell formats on but we've chosen a binary prod about this format largely so that we can do lazy or incremental costs on because that scales scales better in our experience so
overall vectors I feel like offer a very bright future for customized radically customize and efficient rendering of data is large OpenStreetMap or larger including arrested so then of
Excel's actually work so and what you can
put any kind of Actor arrested in these are the formats that the map OX tools are prioritizing support for on but it gets
really interested you can put any types of data and you're not in school you have a fixed schema and so I like to think of vector tiles is
radially break inputting inside but the really chefs choice revealing the cartographer the designs and gets to choose which ships with the product on so you you can choose by
whatever flavor you want right so imagine up
near future were many people are publishing vector styles and they contain their own specific ingredients right so she was
published blueberry other i-vector thousand fuel published beats you can if you wanna throw eggs
in a vector tiles that's up to you right all its outcome is the structure of the data right
so the way it works is we have layers that of 1 or many each layer has a feature of features can have ideas on each feature has on an action set up to be securely pairs and trees pretty simple the on all layers are ordered and
named is how you reference and so you might have a vector tau with parks roads in places on and you decide on the names what features
are dictionary coated so where that basically means is we don't store a key more than once in the whole trial so if you have battery name name we turn into miniature integer and then stored at once in the trial after efficiency and then geometries a single flat array we found that if we compress the same geometries a molten points in a single array it stores much more efficient this makes it a little trickier than to unpack back into a nested arrays of duties on formal diet geometries for example but we feel like it's worth that flatter array stores very well OK so yes had a jumped usually
work always for all juncture taste points lines and polygons mold a or Single on we store accord introduced and because that's more efficient and then those introduce a delta encoded in zigzag encoded so where that mean while delta encoding basically storing
differences rather than the actual value so a simplified case would be if we have of aligned string of accordance that looks like this we would store the Rigel corner for the 1st X Y and then from then on how we would sort the differences from that 1st quarter so this is a a dream case right where all of the rest accordance would be ones which also compresses really really well off you have repeated values but in the real world is still compresses better encoding because the numbers are small that takes up less space and
then zigzagged encoding on top of that also helps a space the idea behind zig-zag coding is we do wanna have negative values anorectic tiles because you might have data this buffer outside of the visible extent of that vector tiles is important often for labeling or and render time if you want blurred geometries you need to have a little bit of a buffer for this we need negative corners tiles but we don't pay a price for that and the trick i is zigzag encoding which is basically up but a method of turning signed integers all into unsigned you only store positive number and then when you decode it it's easy to to go get go back into the ritual negative number the OK not so what you do
styles and you can over soon then this is the whole idea right you should build a ship a tile set on and not burn it all again this a zoom 17 years in 20 we should build a ship tau set may be losing 14 and still allow the visible display of those tiles too much deeper z models right so this works from a problem of track precast tiles for the whole world you should be able to render zinc 14 for fairly dataset very quickly and later on view at a
deeper singles shield also composite them right because once you've burned vector tells us for some some layer will combined with other ones so that's easy to do with this format now you might be wondering why how you combine vector tells a different symbols will it's very easy osseous the scenes of all level with Protocol Buffers the for reason you can just concatenate them was really quick but different zoom levels you need to put some out and we render but before doing so so you should build something like
this say you have 3 different vector tile sets from 3 different organizations of 3 different cartographers parts roads in points they make sense to to pre bake it got to a different maximal right points of interest of what you might want to have highly resolved worst parts that are just a square volumes and abuse results so the idea is you should have different ah tells us that exist at different Maxine's we should still be able to render them all together and benefit from the highest resolution of any of those tiles OK so there's us
also this concept of sources and on that were using describe when you have a bunch of vector tiles together they represent 1 layer so that could be uh a bunch of places in MB tiles format or it could be a URL to vector tiles that you can address busy X Y or ontologies on file that describes some also to look online OK so for more technical
details this is where you could head and get home for test map box for suspected tiles back on this is where the emerging specifications and it's on deathly still at the emerging states were looking for feedback in use cases and what we've done is come up with up US solution and inform the work trust publicizing it and then and then hoping for feedback and to learn how many other use cases it works for were not trying to trace back for all use cases and were collecting on the
wiki at that site the various implementations in different languages so there's quite a number of conditions in joules there some pure javascript some no JS ascendancy post plus and some emerging implementations in Python and I'm sure more common in the future so you have to get the spec were still
at a 1 . 0 and that we haven't incremented recently on because there's been lots of work that optimize the tools around that details the the format and structure hasn't been a bottleneck yet were still working on the tools around them so you might be
asking a cat this sound interesting how can I create them OK so there's really to keep
workflows on there is a variety of command line tools of the data available and we maintain 1 of them Koltai alive on which isn't no Jess toolkit that can be used on the command line or you can use to build larger applications and there's Tessa which is sort of modelling by Seth Fitzsimmons from Spain and design of which I he is talking about right now in another room unfortunately what we actually thought maybe we should try to get everyone in the same room with just have a crew but when I saw the size knew that wasn't feasible but there's also a library called other from our Mapquest which was released a few weeks ago which is and I suppose most based in its not for creation of actor ties of single the lousy a variety of tools now to also render them and work them but but I'm focusing on the creation stage and then but there is a
particle map at studio which is what I've been working on for almost 2 years now and it's been in development and actually was launched officially today on the map OX blogs so I encourage you to check out God map OX outcomes blog it is formally known as tunnel to so the reason we rate renamed it is it's just purely based of a vector tiles which introduces new concepts and new performance abilities on 3 felt like you they would be reasonable to think of the tool as something separate from town we do you plan to do 1 of final release of so in the future for those of you that er depending on it and and need some time to until you can upgrade to a map Austria so the 2 will be supported in parallel for appeared time the I'm about CEO has been known for all officially released today so that it has a visual you
UI to creating vector tiles but it also has a styling interface to make maps from vector tiles which you'll probably hear about on and the National Park Service does give a talk about earlier this so you may have seen so what I wanna focus on is the slightly more hidden part of map you which is how you create them on allows you to create them locally with local data and create an export locally without any map box of Ducarme services so is that look like well
it's a pretty simple interface on because we strive strove to make it but without as many options as possible so this would be an example of the interface the idea is you can load in multiple layers hi and this would be a water polygon from OpenStreetMap you can set the projection and the minimums in the maxims in the year rented out and then the buffer size like I mentioned before for cases like labeling on and blurring effects buffers are critical on you can rename some of the metadata in terms of the that attributes are included in this and that cells because it's imagine that you would really curate the data and then prepared for different audience of cartographers atropine render the vision here is those that know the most about a dataset and its metadata would be the producers of the vector tiles they would really prepared for others but it then you can export it
out from studio and just as
example and this dataset as 530 megabytes as it starts the shape file I did is E 2 0 to 10 export which for the resolution there is more than enough is the models so at the same time you can continue using 2 12 13 14 and the only point where gets a little bit block is were the original data gets blocked of if I only did in export and zoom 0 to 2 then you would not be old is in that deep you start to see the the um the effects of the vector Taylor including the optimizations that zoom 10 for this is that was more than enough to capture the original resolution the data at finished in 8 minutes on my mac book and it's about 1 million tiles on the resulting vector tall set inside an ass the light database is much smaller than the original file so that's really because of the integer encoding delta encoding zigzag because we in this case a referred we're capturing the regional resolution John so what can you do that will you can obviously uploaded to map out . com just like you could in and tiles of image files you can passed on and then you can now see online and
then you can bring it back into town to the sky is the map but studio to the style of and this would be an example the sun interface it's
very very familiar with but free those of you that have used time on the past which should be familiar should be exact it's up what happened under the hood is your data is tiles so it's extremely fast access on but you can also do headless expert i
mentioned tile lied is a tool that we maintain that map out 3 is built upon but you can download it free herbs and open source and you can use it headless without installing that box you QE on the command line and it's a no OJS application in its dependencies are of the new just model for mapping so if you were to run and can install manic and tiles tile Ivan TeleBridge you have all the dependencies in a matter of minutes to run this script and in the way that the script works is you run the talent copy command you pass it on atomic XML which is that data that I smell a bounding box and then it dumps the tiles to tells all the 1 trigger gotcha the you'll see if you try this is that data XML is created and then map at studio and has a little bit of metadata in it so that we know what we know we have we know basically what had to be used in the in the vector thousand fewer things so it's not it's not our automatic the style sheet it a little bit of extra metadata that map OX added but is not and is not not too hard to offer that yourself so you could get set up for this export with outside of member studio um and yet then you've got an expert in the task and so the whole idea here is that
this workflow should scale to you any large datasets that are complex and have a lot of potential layers you could represent potential actually so this is an example of of the X style applied to the vector tiles of OpenStreetMap and the X-ray style basically a DeVault style that you get when you upload in details to map OX . com and also default style that is embedded in a variety of the client tools that can read in the tiles basically just on renders it but in bright colors the various geometries and and labels so you can get can get a sense of what's in there
so that was in New York and this is our Washington DC and then the other thing that is also 1
day of an open source and there's a variety there were added this week as per the launches sample styles so there's an awesome bright eyed that tunnel to that you can use to secure started styling things in very quickly and I just wanna show you map like studio real quick and I'll show you some of the starters styles
so if I were to go to projects here styles style right now in the latest release as of last nite I have 4 starters styles OK the only old
theory of thinking so I have 4 starters styles realize will cut off from a basic goes embrace is actually built in map access doors and then so this satellite of the political saddle afternoon yeah so see if I can figure sizes I wanna show you satellite afternoon it is 1 of the most complex because it combines three-vector tells us a vector tile set of Rasta imagery selling tree of each tile set of the hill shades and Tober lines and then also the Osan data curated interaction tells us that we call map on streets understand that real quickly accretion
zoom in to if I can go full-screen here zoom in the mouth the give you a
sense of of courses and for the full
screen so the whole idea here was not prepackaged maps but the you can change absolutely everything about the style of including the composite operations about how the different layers interact and just a technical note you might be wondering OK so he just said that there's always a vector tiles expression being mined so what's happening is not box con is fielding your request for what later what that tells us what our map on the server side is concatenating the Protocol Buffers and then shipping back to the client in this case my desktop at 1 of the message in a probe format and we can iterate through and see what layers are in there and then match up the styles styles here so is what that means is if I 1 of these styles they gracefully degrades basically this is not a match between the style of name and the landing the so let's see the 1 of big tricks here in the style is this image filters were doing a beta-cell transformation so I can't comment this out and get a look at what the
raw so I'm sure looks like and or I
could go and see the hills shaded being applied with overlay composite operations cut that out and see what the effect it's having great so the idea is is not a styles so for anything anything even mentioned look at some of just about up with time them to go back to representation and
show you a few more slides so what the
vision here is that the vector tiles are going to make possible a new generation of renders that don't have to worry as much about how big the days mean shift to them and they can go crazy with styling effects on and that's because the whole idea's unleash a new generation of carelessness in terms of data size and fears was fierce fierceness about how beautiful and interesting in interactive maps can be can be made so were working on our
that's kind of screwed up the well for a few slides of this is a man box Schiele and that is cut off the the
would may not be able to fix a sometime in summer
working on that application colormap which is on plant written in JavaScript and where they are which can render a 2000 you haven't heard about it go go ahead and then go to go to map was still on map what's a common C C and what is about
on and also that she sells such you should think of them as about more than just rendering and this is a toolkit called vector tile query but also get had that basically allows you to you in a very quick API calls say here's a source of vector tiles and here's 0 line that I'd like to query them along and here's yet treated like to pull out so what you can do with that is things
like this if you have a vector tile set of elevation and futile set of temperature you could add do line profile and get back to back the results so this demo that real quick and before I break
so the cool thing about this is that I which probably put that on the screen figures because
query now any zoom level or any east and and we automatically be backdated to the relevant resolution so you can do a query across the whole US or of course you can zoom in up to the Portland area and say what is attempt to an elevation range from Portland to hood books yeah the display of civil the messed up with the size of the font you get the idea so said different trees you check that out this is very new and experimental but has a lot of promise the but get real quick
Future work on there's a severe lack of documentation for tankers amenity working on that the future so if you developer that's a little bit confused that's an opportunity for me to answer questions and encourage you in the right direction sticking tinker hack and pull apart how this actually works you can build back up applications on top of it set of features that I wanna work on as well as more new use cases beyond rendering enquiring and we have some work to do and geometry robustness the polygons are not I don't don't we don't worry about them being OGC Simple spec so we have some work to do their tools to clean up and validate and then of course there's there's a big tool sets out there that would be interesting Explorer maybe support there is a rumor heard data may be processed opposite considering support for that styles maybe in this format maybe another I think he there would be fantastic to entertain and maybe toolkits like like open and a leaflet is is considering support as well so thank you very much