Hello and welcome to our second section. I want to present the Foskis Social Club first in the English language.

So the technical difficulties are away, I hope so. Okay, the Foskis Social Club is the German chapter of the OSBO and OSM.

We make this conference together with the FrostCon team here with the OSBO and OSM track. Now we have a talk from Hartmut Holzgräfe.

I saw his talks last year here also at the FrostCon. So he's maybe the oldest presenter here at the GEO talk at FrostCon.

Yeah, but now have a nice talk. Okay, thank you. So my name is Hartmut, we already know that. And I'm living in Bielefeld here in Germany, really.

I studied electric engineering and computer science and doing OpenStreetMap since almost 10 years now. So next weekend it's going to be 10 years for me.

And in my primary job I do not really work with geo data or maps or stuff. I am a database support engineer for MariaDB and formally for MySQL.

So I'm usually doing the boring database stuff in my day job. I try to do other things on the side like what we have now. So I'm going to have two talks today. This one is describing the basics I learned while working on the application

that I will present in the second one in the afternoon about printable OpenStreetMap for which this big printer here is hopefully being operational in the afternoon. So what is MapNec? For this we have the usual computer science diagram.

We have some big box in the middle. We have input coming in and output coming out and some configuration coming from the side and some code that is controlling what is happening. MapNec is actually a library that sits in the middle

and that is processing map data and is converting it into pretty pictures. That's the simple story. So for this MapNec can read several different kinds of input

like shape files which is a specific format for geo data and it can read geo data that comes from SQL databases. It has native drivers for Postgres and for spatial light. I'm losing the microphone.

And it also has support for other database via the special extra plugin library. It can natively read GeoJSON

and it can also read a lot of other formats via two extra plugins. There is one using the OGR library that is supporting both vector and raster images. So this can be used for example to read raw OSM data and GPX files.

There is GDAL that is only there to read raster images and display pre-rendered images that you want to embed in a map. And on the other side we have different output formats that we can support.

That is we can produce PNG files, both full color files or reduced color 8-bit files to make the files a bit smaller. As maps usually don't have that many colors. We can produce JPEG. We can produce scalable vector graphics.

That's especially interesting if you want to create a map and then use that in a graphics program like Inkscape or Adobe Tools to then add more stuff to it. We can produce printable PDF right away

and also still can produce printable PostScript whether it is still needed. Whereas PostScript and PDF are sort of similar. So we have our input on one side that goes in and on the other side we want to have pretty pictures.

So we need the third thing and that is the render styles that tell the MapNIC library how to take the input data and transform it into pretty pictures. For this we can either define render styles directly in the program code

or we can use external XML style files that describe how we want to see our stuff rendered. Both have their disadvantages that we will see later. There are also some other style sheet languages

that can be converted into the MapNIC XML format like Carto CSS for example that is used for the main OpenSuiteMap style that are a bit more readable than the XML format. MapNIC itself is not a standalone program.

It is a library. So we need at least a little bit of extra code around it to make a full rendering application. The library itself is written in C++. Then there is Python bindings so that you do not have to use C++ code directly

but can use it in parsing programs which is usually going to be easier. We also have experimental bindings for PHP which I have found just lately and have not tried yet. I cannot really tell how good that works yet.

This talk is only about the Python bindings. A few things we need to start is that we obviously need Python and it works both with Python 2 and 3 but some features may only be in the latest version

that only works with 3. We also have two different versions of MapNIC version 2 and version 3 that is not related to the Python versioning. It is just also 2 and 3 by coincidence. We had Python bindings in MapNIC 2

included in the main MapNIC source code and now with MapNIC 3 it is two separate projects or two separate repositories. You have the main library in C++ and then you have a second project that does the Python bindings. Usually you do not really have to care about it.

Like on Debian or Ubuntu you just install Python MapNIC and that pulls in all the other things that are needed. There is still a todo item here on the slides because I did not test it in other platforms. I know that on Mac OS all the stuff is in the Homebrew system.

I know that there are Windows packages somewhere but I do not know anything about Windows so I have no idea how to install it there. What MapNIC does on the input side is that all the input front ends we have

deliver just four data types to the actual library and that is points, that is points of interest that have some attributes. That is lines, like for example a street. That is polygons, that is all kinds of areas like land use areas or the shape of a building.

So everything that is a closed line that actually defines an area is a polygon and there is also the possibility to import images that are already pre-rendered and for example this is used in some OpenStreetMap online maps

to include height information by making one side of a mountain a bit darker to have a 3D effect. That is done not by MapNIC itself but by using pre-rendered images that have this height shadow simulation data

pre-rendered as a big TIFF file. This is not necessarily the data format that data is in the actual input files like when we use raw OSM files the input format is different so the input layer has to convert it into these formats

and each of these objects can also have additional information not only the geometry information but also different attributes that can be used to decide what gets rendered and how it gets rendered like in the most basic case almost every object that we are going to render

is going to have a name and this name can be used in style rule to say I want the name displayed here. You can also see that later. So then we have the style definition in MapNIC

that again consists of three different basic kinds of objects. The first is what is called the layer. The layer defines what data we want to use and what style or what multiple styles

we want to use to display the data. The style can filter like it can say I only want to I want to take the input data but I only want to render the stuff the things that have a name or that have a certain size or whatever attributes you have

and then it uses what is called symbolizers that is the MapNIC components that do the actual drawing of things. So we have four basic types of symbolizers that directly map to the four basic kinds of objects.

There is point symbolizer, line symbolizer, polygon symbolizer and raster symbolizer and these are pretty easy like the point symbolizer either just draws a little square or you can give it the name of a symbol file you want to render.

Similarly for the line symbolizer that just gets information how thick you want to draw your line what color you want to draw a line in and you have some basic options

like you can have a dotted line or you can describe I want the dots in this size and the gaps in this size. Polygon symbolizer just takes an area and fills it with solid color and the raster symbolizer just takes the input raster image maybe resizes it but otherwise

just puts it directly into the output without modifying it. And then we have more complex ones this marker symbolizer that can put markers on a line like when you have a one way street you can put the little arrows as marker on the line

to show that you can only use it in that direction. We have what's called the line pattern symbolizer no spelling error it's pattern not patter and that can be used to draw little symbols on the side of a line like when you have an edge

in the landscape where the landscape goes down you have these little triangles and usual map styles at the side of the line these can be drawn using a line pattern symbolizer and there is a very important thing

the text symbolizer that is as the name says used to show text and usually it uses the name attribute or whatever you have in your data that gives the name for an object and then puts that on the symbol

and that can be used for points for lines and for polygons for points it will just put the text next to the point for lines it will put the text along the line and for polygons it will just try to find the middle of the polygon and put it there and depending on how complex the shape is that can be perfectly fine

or perfectly somewhere where you don't expect it like if you have like a half moon the text will not be in that half moon shape but it will in the middle of the circle that would be the full moon and then there is a special kind of a marker symbolizer

that is the shield symbolizer I'm using the microphone again okay so the shield symbolizer is used mostly for highway numbers so that's why it's called shield symbolizers

because on US maps these are usually shown in little shields so what the shield symbolizer does is it has a basic image of the shield and then it knows how to extend it to fit the number of the text you want to display in it so if you have a one digit number it's small two digit, three digit

it automatically gets wider and then the final one we have is polygon pattern symbolizer that is similar to the basic polygon symbolizer but it can fill the polygon not only with a solid color but with a background image

so usually you take a small repeatable image like for a forest you have green background and one or two trees on it that give the polygon pattern symbolizer the instruction use this image just tile it

and put it into the polygon until it's fully filled up there's a very special one the building symbolizer that's a pretty new experimental one that draws buildings in sort of a isometric sort of 3D view

so you have instead of just a rectangle for a building you have a little sort of box that also can take into account the height of the building so now let's get to the

bison part again this is the very basic use of the mapnik library so you have to import it you have to define a map object and give it a size and then in the end for the map you created

give it a file name and a file type and then a file is written so this obviously does nothing but it at least creates an empty file with transparent background not that interesting yet so we add a bit more here so we start by giving

the map a background color it's still blue here then we define a polygon symbolizer it also gets a fill color in light green and we create a rule object we append the polygon symbolizer to the rule we create a style object we append the rule to the style

so the style can have multiple rules rules that get stacked while drawing like when you want to draw a road you may first draw a white line then you draw a slightly smaller line in a different color so you have a road with visible edge

and then as a third rule you may then put text on top of the road to print the road name here we only have one rule so one symbolizer one rule, one style we append the style to the map so now the map object knows there's a rule called countries

with the given style which just creates a lot of polygons here and then we create a layer so that's what combines the data and the style we call that layer world and as a data source here we use a shapefile

it's a predefined file that has the shapes for all countries in the world we append that style we defined to the layer we append the layer to the map we give the map instruction to zoom all it is to zoom far enough

that everything in the input file can be seen and in the end we render our file again so it looks better already we have the blue background the green foreground we didn't give any rule for showing the borders the borders here are just

artifacts due to gathering on the edges of the polygons so by default it has anti-aliasing enabled that's more obviously the edges here but you can turn that off then we would only have the continents in green without borders

so next let's add a second rule so we have a polygon symbolizer again the other polygon symbolizer is the previous code and now we add

a second polygon symbolizer that fills polygons in red and we add a second rule here called Germany and for this rule we define a filter that filter is supposed to only render things that have the name Germany and the name attribute is also

set in the shape file we imported so we append the second rule in the same style and the rest of the code is like the previous one and something else changed here because really the borders are gone

but we see now Germany is not green anymore Germany is now red but we also see that we have to write a lot of code here so there is the alternative to not do this all in Python code

so we replace all the rule and style and layer definitions we had with just a call to load map and give it an XML file to read this is what the XML file looks this is exactly the same style

as before all countries are filled green one country is filled red and we get country information from that country shape file and so the result is obviously also the same

but even as unreadable as XML sometimes is I think it's still more readable than this so the advantage of doing things in Python code directly is that you can change rules dynamically as you go so in an interactive application

where you won't use this to be able to change styles this would take more work but it would be more flexible while this is more suitable when you have predefined styles that you do not want to change anymore so for the rest of these examples

I will use the XML format which is a better fit for the slide size here so I have a few examples of symbolizers here unfortunately I was not able to finish to copy all of them over because of the network failure and didn't finish my slides in time

guilty but you get the idea and then the other styles are also described in the mappnik documentation and for most of the symbolizers there are code examples both for XML and for Python so we only do the very basic things here

like the file name is still word XML here but the contents are different as you can see on the next slide so what we do here is we still zoom into all the data but the new data file only has two points in it

and if you just zoom in on these two points we would have the two points at the edges and then style as symbols for these two points you would only see half of the point because the other half would be outside of the image so we do another zoom call that has a negative zoom factor

so it zooms out and the factor is 1.1 so we zoom out by 10% and that is enough to see all the features we want and that is our style file we have a point symbolizer that just

shows a point dot png that is a png file and now as a data source we do not use the big clunky shape file anymore but a small geoJSON file this is the geoJSON file that we are going to use here it only has two points in it

in different places and so this is the final map we get with just these two points obviously not that useful but as an example how it works it's the most basic thing to do so in real map you would have multiple styles stacked on each other

that would draw different things to make a complete map but here we only have two points and the second example now this is a bit more complex let's maybe see the geoJSON

yeah I told you a few slides are missing so in this example we have actually another geoJSON file that only has a simple line in it and we use two symbolizers one blind symbolizer that draws a simple line and one text symbolizer

that uses a certain font a certain size this black color and a white halo around it we will see what the halo is soon and so that's also fully correct here

so please ignore the background text there this style file is actually rendered it's just a blue line and the text here on top so the text aligns with the line and you can see it is black text on blue

with a slight white frame around it that's the halo so that is when for example you have black text on top of something else that is also black you can still read the text yes so

the other symbolizers I didn't cover in the examples we had the polygon symbolizer in the first code example already I told you about the marker symbolizer that is used for lines with symbols on the side with symbols on top

shield symbolizer for highway numbers and stuff like that line pattern symbolizers for lines that have some small things on the side and polygon pattern symbolizer for filling a like a forest area

with a repeated forest background image and the building symbolizer for buildings in fake 3D so and all of this we could have done with just a stand alone map neck renderer wouldn't really need to write our own code for it

but what makes the combination of python map neck interesting is that you can also use other drawing functions to not draw things around or on top of your map like you can use Cairo graphics and tell

map neck not to render files itself but we create a PDF service so we are going to generate a PDF file to create what is called a context context is what you actually draw into in Cairo

we still create a map of a certain size we load our style sheet we zoom in but now what is different is we do not render to a file right away but we render to the surface we define so we actually render into an image that is controlled

and change later that is what we are going to do here we set the color we want to use to black the line width we want to use to five points and then we

say we want to have a rectangle of a certain size and context stroke is what actually draws the rectangle using the line widths and the color we defined and then in the end this map neck does not render the file anymore we tell the Cairo surface

that we finished and that it is now supposed to write the PDF file so the result is we have our two points again here which are from the map data and we have the rectangle that is not map data at all but that we put on top ourselves

and we can also use that to not only draw stuff directly but we can also pretty easily put SVG images on top so we also need to import our SVG library here we draw our map as we did before

then we read our SVG file here set the coordinates where we want to draw it this image is actually too large for the map we are going to render so we resize it so it's only half of the size

we render it to the Cairo context that our map data is already in and again say finish, write our results out of the file and so now we get this we have our two map points again now we have the SVG image that tells us where the north on the map is

so it's a summary this is what a complete application looks like that uses all these components to draw a map so we have the actual map here we have decorations on the side

on the top and on the bottom so we have a title on top we have some copyright notice and other text on the bottom we have an index on the side and on top of the map we have extra SVG markers for the things in the index

like bus stop Weststrasse is here in map square C4 at this position and we have the red circle which is the you are here marker and all this was created using the components we've seen in this talk

but in a larger application that I will talk about in the afternoon in more detail and that's also what the big printer is here for that is hopefully going to be operational so if all goes well if you want to have a map of your neighborhood we could arrange that in the afternoon during the breaks, not during the talks

so to summarize this talk what have you learned or what have I learned in my first adventure into Python land it is once you've figured out all the ways how not to do it it is actually pretty simple you have not seen much code here

and the devil here is in the styles not in the code and obviously it's always in all the details but most of the work is not on the code side but on the style file side the combination of Mapnik, Python and Cairo

allows for a very flexible map rendering and putting things around the map on top of the map in addition to the actual Mapnik rendering and what I learned personally is the Mapnik documentation

is not as good as I expected it to be so there is mostly a wiki on the Mapnik git repository and it describes all the stuff but sometimes you only have a header and just this does this without any further explanation

and what's more annoying is often enough you have this only works with this version so does it work with the version I have now if it doesn't anymore what is the replacement now you don't really know you often enough have to find out yourself so that's it and I want to give a special thank

to our cat Leila which has helped to produce this talk or has not succeeded in preventing me from doing it so now is the time for questions Can I get some style files

some predefined style files for instance those used on the OSM street map things

there are different layers Can I download them somewhere in a compatible format? I've set up all the public style files I could get hold of like the official OpenStreetMap style and the German OpenStreetMap style and stuff like that but that's actually stuff I cover

in the other talk in the afternoon Yes, it is time consuming to do a style from scratch so you usually either use one of the normal styles that already exist or there is a style called OSM bright

which can be used as a basis to put your stuff on top or what I do in that application I'll show in the afternoon is first use a standard style to render the map and then use a second style to just put on top stuff I'm really interested in like hiking routes or fire fighter facilities

so that way you don't have to do a style from scratch just put on top of existing style what you're interested in so putting several styles into the same Cairo context is not a problem Any more questions?

Can you hear me? Yes Mapnik has its own style file format is there an option to have map CSS for definition of styles? Unfortunately there is no option to have map CSS because I have one style

I really like to offer in the other application that is only coming with the map CSS style and there are converters for converting Mapnik into map CSS but not the other way around Ok, thanks So further questions?

What is the preferred data source you use in your style files? Is it really Postgres?

No, what I'm actually using and what most of the predefined styles use is actually an OSM to PGSQL import of OSM data into a PostGIS database but I wanted to have stand-alone examples here that do not need an extra database that's why you choose gayo-json here to have something small self-contained But I guess that's not really feasible

for Germany or Europe or something Alright Additional questions? No, I don't see Ok, so at first thank you for the great talk