Logo TIB AV-Portal Logo TIB AV-Portal

Create Vector Tiles from OpenStreetMap

Video in TIB AV-Portal: Create Vector Tiles from OpenStreetMap

Formal Metadata

Create Vector Tiles from OpenStreetMap
Title of Series
Part Number
Number of Parts
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date
Production Place

Content Metadata

Subject Area
The OSM2VectorTiles project offers free downloadable vector tiles ready to use by people interested in hosting custom base maps on their own infrastructure. The whole world fits on a USB stick and can be served from an ordinary web hosting and styled and enriched to make beautiful and fast maps for web and mobile applications. The advantages of vector tiles over traditional raster tiles are well known. There are already a handful of vector tile provider present, but they may not always serve your use case optimally. After this talk you will know how to create your own custom vector tiles based on OpenStreetMap and will know the tools and processes you need to use. The talk will cover how to import OpenStreetMap data into PostGIS and then shows how to generate vector tiles using Tilelive and Mapnik. We will present the open source workflow we use at OSM2VectorTiles to prerender global vector tiles and instruct you how to adapt the workflow to create custom vector tiles. Thanks to Docker and tools such as Mapnik, PostGIS, Tilelive and Mapbox Studio Classic the process is straightforward and repeatable. Manuel Roth (HSR University of Applied Science Rapperswil Switzerland) Lukas Martinelli (HSR University of Applied Sciences Rapperswil Switzerland) Petr Pridal (Klokan Technologies GmbH)
Keywords HSR University of Applied Science Rapperswil Switzerland Klokan Technologies GmbH
Meta time 3rd bits Arm vector tasks
area man mapping time images projects computation bits VPU tilings words Computer animation raster VPU sets computer scientist
point mapping Free Manufacturing execution system Tiled encoding PROM projects VPU CAP open subsets vector tasks
area bottom Free demo mapping link response directions time demo sheaf 3rd argument lines sun fileserver vector CAN-bus Computer animation VPU case website WPAN
words time demo Right
presentation mapping mapping formating demo sources projects shape vector web specific data management means Software VPU configuration boundaries box
graphical user interface mathematics mapping Computer animation files administrations projects sources box part metadata vector
chess polygons building map time equal Databases water VPU completion impulses splitting geometric rates configuration VPU Query model Render mapping Menu bits entire vector processes preprocessing URN touch Iris configuration hill modes functionality mapping table files Sequel Integrationen student Stream Codes metadata van powerful tasks scale Tiled polygons Databases subset particle words Computer animation case string calculations mod box table gradient descent
meter current point Virtual Home Environment digital filters building bottom mapping time corresponds sources limitations part metadata write geometric VPU naturally single Query square box extent sum descriptions data types area sources Zoom's mapping building projects bits Databases vector degree Types words Computer animation case Query melt Right level box geometric
Actions states directions time Databases part orders of magnitude geometric graphical user interface VPU memory Query file system box clone descriptions mapping building graded computation statistics several vector processes Multi-Agent Right level Results purchasing digital filters mapping Sequel files machine maximal division limitations entire powerful write Gamma sum MaƟstab Zoom's Tiled server projects counting Databases directories graphical user interface Computer animation Query case Data Centre box 5th table
Manufacturing execution system overlay files machine sheaf sets real time VPU part powerful mathematics VPU boundaries Display mazes Render tasks overlay relation projects maximal Databases templates machine tilings processes Computer animation visualization provide pattern geometric
man presentation code time sources 3rd open subsets Emulation Twitter Location processes Densities VPU Em's Rolling website information model tasks spacetime
files Meeting/Interview Tiled VPU Gates
collaboratives Meeting/Interview projects
Computer animation time 3rd
the all right so hello everybody
I would like to welcome you to
our talk about creating meta task from OpenStreetMap it's great to be here we enjoyed that this conference very much from where you for the 1st time and now we I hope that we have something interesting for you this afternoon so a bit about us this is because
Martinelli and I'm monitor what we are both computer science graduated from the University of Applied Sciences their opposition in Switzerland and for the past year we worked on a project called wasn't electric cars sold what you will know after his talk so we'll talk about what makes ties on word with ardent Wantage's over traditional rest times like little bit we heard before and what the the purpose of those and director tile project is and how you can use it in your projects well and in general how you can turn OpenStreetMap data into vector tires and watching you need to look out for and especially how you can create your own that
and just to add to what you were before it's important the biggest advantage of houses that it always called same area and this means we can precomputed can catch it very easily we shouldn't forget that that's the biggest advantage and
way tells very similar and I'm very glad boundless a great explanation how that works and makes a task is even better than frosted we can just create once for purposes like for the purpose of the basement and everyone can use them and apply the constant styling so we hope there is no need for you to create to go through all this work will show you to create your own he was erected tiles and really keep that in mind you only need to create them once and other people can use them and they're really help you see more open challenges in the future so
the goal of those direct has project was to try to build a place where you can download the securitized like the entire planet and have over 200 countries and more than 600 cities 3 give your starting point where you can download these vectors and use them and secondly the target was to build a workflow based on which should be very straightforward to go from raw OpenStreetMap data to record tiles and 3rd like most important when we went to empower everyone to create their customer base that so now it gives a
little demo from our website hope it's some
area that is our website is follows into vector the you can go in it and there's a map section where you can
choose be between 3 predefined styles there is the bright star liberty and the basic and overlapping conceived and it additional cheaper and just copy and you got a full-screen mad but which uses this year and for hosting directed path from us but when when we actually think is that that the way to go always like go to the novel section novel direct ties and serfdom them yourself so that you can be independent send about sexual checks section we provide better time for the whole planet for many counties more than 200 and 1 in 600 cities and I'm going to demo how you can do so because they are involved in researching for Bonn Germany we can just covered link going to come on line and you can see that download centiles find them because 5 is basically just that as light DB we'd all the vector thousand and so we see we downloaded and then we can use a file server in this case they use as the G alive which will be presented after a talk we can just think the argument like this and then we have lot of the times of room for party and here as in from the bottom and you get the next review which
shows you what's inside is that time is so get really basic you can hover always or certain features and on the right hand side you can get you see that attributes so you can really just play around the words in them what you can actually style laughter words also with you want to create your own style that is really the that's the place to get started now back to the
presentations and in the rest
of this presentation we we really want to focus how you can create a vector doesn't when you need to cover so creating static web thousand not the top manager example had do with map of studio classic we were we are always talking about like the child and the many books make detailed specification now and there are other options as is seen before and we're easy way to get started with the traditional shape so for
OpenStreetMap and you can download the shape file format to extract and so we download that can be answered and now we're going to choose that have meaning and boundaries so that means shapefile underground and create a new source project networks due to their
opening opening it create a new source project and we will
have a new layer the bill
reference the change file so the
browsing and reference the previous admin what we have
each year we can Click on the data and we see the metadata and we consume around and now we already started creating around own parts not exporting them as an because formant is easy with the go with their graphical user interface you can just go to export and it
will down the sequel light file for you so it's not scary it is easy to do however if you do it
on a bigger scale that no longer works and we don't recommend that you download shapefiles for OpenStreetMap and started in directed tiles really find that using as a database like POS chess gives you the most power because you're gonna want integrate all the data was there's not not like natural and you want to generalize you want to calculate labels and all the power and post just gives you so we build their workflow around this and if you build maps with metadata you will notice that and it's essentially the same you imported with tool like it wasn't feature SQL composite water data and then there's just and codes that that status equal curious indirect time and that's what you function X so now we're going through each the idea of 1 so we don't cry the entire process so it's very easy for you to use and then you stop composed orchestrated content they don't need to install anything he just the doctor and the doctor on and you can do with this entire thing yourselves you put know is simply defined and in the end you get a vector task and the task so now we we talk a little bit about each step so 1st is important OpenStreetMap data into postures the 2 options basically wasn't to be just going Poseidon 3 I we decided to choose involves and 3 because it's very fast and it has this nice configurations for you can define which where we can find which OpenStreetMap text get imported into each database table something on the right-hand side you can see an example of this configuration there is defined database typical buildings which all the stream of text with the key building gets important we found that it's very important to only imports the OpenStreetMap data which we actually want inside the vector tires and to leave out all the rest so this is where a convenient way to do it is would also was very helpful us important is people preprocessed as much as you can occur through heavy stuff like that generalization and chemical at heavy functions like calculated century labels or splitting large polygons students before and so that you don't have to calculate these things at the time of your generating director applies to this particle it keeps a fast so he uses them all out of words and there is another compose among which starts prosperous database started we downloads a a PBS in this case it's is Switzerland and consisted in Switzerland and in the next step we are going to import these OpenStreetMap extract the around the model was need to read this file imported with impulse and 3 and after the city will do generalisation splitting the polygons calculating descent rates for the label placement so we did all this work for you so now
where things in the data base all the data everyone to have vector ties after words now we're ready to go and create vector parts and the it seems that we but before we can do that we actually need to define yeah how should you say that the airlines the vector for this we need to know a little bit more what what actually is inside a vector time so a single vector type consists of and that means that all of leaders this could be a building later wrote layer what really or whatever you want and each of these layers contain the geometric features and their metadata so a description is used to define all of this layer and where to get this data from in our case is an SQL query which queries PostgreSQL database but this could also be a shapefile the futures so here we can see here in in an example on the right hand side again we have had least this description of the layer for a building layer and corresponding as colloquially so if you want to rank the they times for the entire word to a little 14 is our 300 around 350 million times and for each of the whole each in all of these ties these query runs so we really need to be focused on making this grew very fast this should be that should keep this in mind and what's also interesting is queries and the bottom we can see is termed the box so this limits the data only to the extent of the current vector time after time so we only query this data and degrees also specific to the to the still for the scalar nature defines that in this example 13 only dealing with an area greater than 1 thousand 7 hundreds of square meters should should show up and consume less 14 the 14 all the other buildings and so level 12 thousand below no feelings that appear so this is how you define and what's inside your rectum time to put all these layers there where to get the data from and because there's a nicer way to do this then duties by hand we
use map of oxygen plastic there can there easily create source project and all these layers that you're the rest and query and on the left hand side it graphically shows up with that with that but yeah actually query you can hover over the point geometric features and see what's actually it's hot and it is really when you say that it will ultimately generate
these description so now
you're ready to grade elected
so and we saw how we do it with the graphical user interface we can just use this command under the hood users Tyler or place so if pile of copy you can just say copied this bounding box that its bounding box from soon level 0 in this case found assuming 14 we use our project to create before and you put it into a destination in this case it is an NP tells it could also be asked 3 or for example it could be just a file system and now I will show you how the direction our workflow with the pre-composed and it's very simple we just run down composed export vector tiles and give it the bounding boxes in this case of search and we will shortly see how it starts rendering the specter trial you say it's magnitude does that does the job and on the right is the decide so after seeing the data directory have power was in November tells with the vector tiles because it's just single like database you can look into it we can count the tiles will be 70 and now we have that file we consider count unpack it can throw interest the we know no longer need a complicated so and that's the power of the and we always all the projects GeoServer maps results working on this and there are other phosphogen guys that they're working like on Durex or table or that just the post yes into vector tiles so you might want to look into those if that our and please you now we want to precompute the vector times for the entire planet so that we can create extracts and everyone can put it on a USB state on their mobile phones and to do that we cannot use a single machine so we do our import into the post database at 1st we have a purchase database and then what we do is we just close to physical bytes we just can't do this and we create clones like for example 16 machines in our data centers and each 1 of them has a separate posters database and so we created workers so that everyone who use their own databases and this makes it very false otherwise database would be the bottom like if you fire these billions of queries and now we just divide and our planet integrated divided in 64 thousand jobs each of his a bounding box you need to render and their job will take the bounding box rent all parts within stored is equal like database and send it back and in the end we just take all the sequel like databases combined and and with a single file with the entire planet so is mapreduce workflow we just met the job description to the sequel light and reduces and so that is the if you're running on a single she with 50 kilograms and only once you through this 128 based so you need to scan it horizontally and we can bring it down to 3 days to 16 machines of around 60 machines with a course of action on referred to gigs of memory and so if you have money you can make that the force as possible and that is the entire planet pre-rendered so serene performance will be a problem for you anymore but we already did that for you
so now updates we can provide real-time updates about like weekly or daily updates of still possible and to do that we just detected change features so what geometry change and then we have all these geometries over the planet and just find the minimal set of tasks that and then we just apply this entire process to these files and in the end you're going a database with just the policy change and now we can just leave these parts into our planet from previously and if the updated database and conservative again and this way you once you have friend the planet it's not such a huge investment you can have a single machine and then can can keep OpenStreetMap up-to-date and you also see here that these patterns like the boundaries an natural changes in the boundary all of those files that contain the bound have to be re-rendered so what can you do with
this workflow if you just want the nice for example if a visualization of the crater light basement or probably highlight something else perhaps you can use our task this checking out so you don't need to do this work of rendering it spent that money it's very easy you can just copy out files and serve to yourself you no longer need us independent and if you just need like an overlay you have your 5 features we have opened the for example that display and marine features like always or all those things and then you can just use of cooperative templated we prepared here to create a thing vector tile relate and against combined it with the basement and that section the power affected task that we can every 1 of us can create their own set of vector tasks and can in the combine and you really need something complicated with with them and you don't like what we did then you can just for project that its merger on data and use of this through the process and other people are doing this as well so yeah
this is basically please check out our web sites really would love to see if you go there we have tutorial check everything out there on the top will have a look at our code running through the process of defining issues we're happy a few 5 5 and he shorted those up and Twitter and we hope that is that we to give you something interesting and get you motivated to work at vector tires so we hope that there are getting like more and more open vector times that's out there so we can all the other fundamental task OK thank you so thank you for your presentation model please please just being a microphone because actually supposed to be reported our are the OpenStreetMap from source hosted on as 3 4 are they host to public locations in the solar system clock and that provides us with the density can access that but if you don't and you CDN or please follow them and just for the money has 3 you have needed foster all of this is it's not
it's not that much money itself what's that the space requirement for an entire global coverage
and you don't have a lot of file psychological is 54 gates and I mean that vector tiles when I think jeez 54 about so
it fits on a specific everyone 1st of all I have to say I'm sorry I'm going to ask an inconvenient questions you're probably already distinguish from that OK could you
please comment on the on this current status of the disputed with with many books of all this like IP claims on photography because I love you project to very much and I'm like be a bit worried if like the good guys like you and and MRI all striving its so I feared it would come um please
read the issue if you really want to more to more I want to make up your opinion you make your own opinion and we don't think there are legal issues on our side however we just so many books and was that you've seen before so we want to work together with them that there had been because without them it wouldn't even be possible so we we're not talking with them to make these problems go away so you cannot expect collaboration from us don't see
currently any more questions but please correct me if I'm wrong so we're doing extremely well in time so it's still 10 minutes left and so basically if there are no more questions i suggest that everybody can get out and have a coffee house because time is that's quite a good time and we can see other like once again thank you for your presentation and