State of OSM in QGIS
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 351 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/69103 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Production Year | 2022 |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
FOSS4G Firenze 2022141 / 351
1
7
13
22
25
31
33
36
39
41
43
44
46
52
53
55
58
59
60
76
80
93
98
104
108
127
128
133
135
141
142
143
150
151
168
173
176
178
190
196
200
201
202
204
211
219
225
226
236
242
251
258
263
270
284
285
292
00:00
Web browserProjective planeSoftware developerServer (computing)Open sourceCore dumpWeb 2.0Plug-in (computing)Computer animation
00:38
VolumenvisualisierungBridging (networking)Vector spaceServer (computing)
00:54
Web browserView (database)Vector spaceDatabasePlug-in (computing)Data typeGame controllerRange (statistics)Mathematical singularityMathematical analysisGame controllerTemporal logicSource codeComputer animation
01:11
Attribute grammarTheory of relativityNoise (electronics)Object (grammar)Different (Kate Ryan album)Attribute grammarPolygonMixed realityField (computer science)Line (geometry)Complex (psychology)Data modelGeometryKey (cryptography)String (computer science)Level (video gaming)MultiplicationOpen setPoint (geometry)Rational numberComputer animation
02:10
Different (Kate Ryan album)SubsetAreaMathematical analysisMetadataKey (cryptography)MereologyDatabaseMathematicsPopulation densitySoftwareExtension (kinesiology)Level (video gaming)Multiplication signElectronic visual displayOpen setJSONComputer animation
03:37
Service (economics)Web pageVector spaceOpen setWeb browserLevel (video gaming)TesselationComputer animation
03:57
File formatWebsiteProjective planeComputer fileTransformation (genetics)Level (video gaming)Open setFile formatWeb portalComputer animation
04:48
Web browserBookmark (World Wide Web)PolygonMultiplicationLine (geometry)Point (geometry)Line (geometry)Open setWebsiteComputer fileSet (mathematics)Different (Kate Ryan album)Source codeJSONXML
05:14
Natural numberConvex hullPoint (geometry)Address spaceKey (cryptography)Latent heatProcess (computing)Row (database)Computer fileSingle-precision floating-point formatWebsiteField (computer science)Open setVideo gameComputer animation
05:56
ExplosionAlgorithmProcess (computing)StapeldateiKey (cryptography)Function (mathematics)Open setoutputDefault (computer science)Electronic mailing listField (computer science)Parameter (computer programming)Point (geometry)Regulärer Ausdruck <Textverarbeitung>String (computer science)Text editorZoom lensField (computer science)ExpressionVector spaceFlow separationDescriptive statisticsKey (cryptography)Process (computing)Function (mathematics)ExplosionLevel (video gaming)Source code
06:48
Plug-in (computing)Revision controlKey (cryptography)VotingExpressionOpen setProcess (computing)CodeInformationHome pageRepository (publishing)SynchronizationLocal ringParsingCurve fittingQuadrilateralObject (grammar)Query languageParameter (computer programming)Default (computer science)BuildingPlug-in (computing)Descriptive statisticsRepository (publishing)Lattice (order)Instance (computer science)Software bugDoubling the cubeLevel (video gaming)Service (economics)Software developerFormal languageObject (grammar)Different (Kate Ryan album)Single-precision floating-point formatEqualiser (mathematics)Query languageBuildingContent (media)TopostheorieRevision controlPoint (geometry)Expert systemAreaKey (cryptography)Default (computer science)Open setSet (mathematics)Extension (kinesiology)User interfaceType theoryQuaternion groupNeuroinformatikTurbo-CodeLimit (category theory)MultiplicationTouchscreenSource codeComputer animationXMLProgram flowchart
11:44
Default (computer science)Water vaporMappingLevel (video gaming)Query languageStructural loadMultiplication signNatural numberWritingXML
12:31
Query languageRaster graphicsVector spaceOpen setFunction (mathematics)AlgorithmProbability density functionMetreRight angleAreaDegree (graph theory)Endliche ModelltheorieEqualiser (mathematics)Buffer solutionComputer fileCase moddingAlgorithmInstance (computer science)Level (video gaming)Extension (kinesiology)Local ringTesselationProcess (computing)Program flowchartXML
13:47
Zoom lensPermanentAttribute grammarTable (information)Open setComputer reservations systemScale (map)CountingDigital filterGreatest elementMathematicsSource codeQuery languageComputer fileDatabaseOpen setXMLUMLProgram flowchart
14:05
Web pageObject (grammar)Text editorDefault (computer science)Zoom lensContent (media)Cellular automatonWeb browserMultiplication signGroup actionField (computer science)Key (cryptography)Level (video gaming)Default (computer science)Open setWebsiteUniform resource locatorXMLUML
14:52
Normed vector spaceAsynchronous Transfer ModeWebsiteUniform resource locatorGroup actionDefault (computer science)Link (knot theory)Translation (relic)Formal languageComputer animation
15:18
Complex (psychology)Theory of relativityFile formatFormal languageBoundary value problemPolygonMultiplicationString (computer science)GeometryEqualiser (mathematics)RootDefault (computer science)Plug-in (computing)Server (computing)Computer animation
16:04
outputStapeldateiAlgorithmFunction (mathematics)GeometryString (computer science)Attribute grammarAddress spaceOpen setProcess (computing)Data typeWeb browserComputer configurationPoint (geometry)Projective planeDatabaseUniform resource locatorNominal numberCuboidComputer filePoint (geometry)WebsiteGeometryLevel (video gaming)Complex (psychology)StapeldateiDifferent (Kate Ryan album)Internet service providerAddress spaceField (computer science)Open setMatrix (mathematics)Line (geometry)CalculationRootAreaGraph (mathematics)Plug-in (computing)WikiService (economics)Operator (mathematics)Default (computer science)Set (mathematics)Artistic renderingTable (information)Greatest elementCore dumpSource codeXMLComputer animation
19:04
Function (mathematics)Type theoryView (database)Table (information)System administratorSet (mathematics)Projective planeMappingDefault (computer science)DatabaseView (database)Computer fileAmenable groupSource codeJSONXML
19:55
Vector spaceOpen setView (database)Level (video gaming)TesselationPresentation of a groupMultitier architectureRevision controlState of matterComputer animation
20:28
Link (knot theory)SubsetLink (knot theory)JSONXML
Transcript: English(auto-generated)
00:00
So, quickly, I'm working for FreeLiz, an open source GIS company based in France. All our work is on GitHub. So, as I said, we are contributors in QGIS, mainly on the server side. We do QGIS plugins, yeah, QGIS core developments for desktop or servers, because, yeah, we
00:22
use these tools on our every day. We do also our own open source project called LizMap Web Client. It's to publish a QGIS project on the web, thanks to QGIS desktop and QGIS server. So quickly, I guess most of you know what is QGIS.
00:43
It's a great tool. It has, like, a very nice rendering engine to make complex rendering like this one. It's showing OSM vector data. You can do a lot of isochrones analysis. There is some advanced features like the temporal controller with WMTS data.
01:08
You can have a 3D viewer inside. So there are quite a lot of features. And on the other side, we have OpenStreetMap. It has its own data model. So in OpenStreetMap, it's node, way and relation.
01:20
And in GIS, it's point, line and polygon. But it's different kind of objects. A node same way can ever be a GIS line or a GIS polygon, according to its OSM tags. And noise emulation, it's quite a complex object in OpenStreetMap. It can be a very mixed geometry, like point, line, multiline string, multi geometry.
01:42
It can be everything. Another difference is the attributes. In OpenStreetMap, sorry, in GIS, we create a layer. Then we define fields and all features within this layer as this kind of fields.
02:01
But in OpenStreetMap, the layer does not exist. Each features is individual and we define its own keys and values for this OSM feature. So therefore, when we switch from OSM to QGIS, there are just a few differences and we might wonder some questions.
02:21
Like, do we need to have up to date OSM data? Because the OpenStreetMap database is updated like every minute you have new contributions. Do you need a static export or do you need to follow all contributions in OpenStreetMap? Then do you work with small or big extent?
02:42
By the way, what is a small and big extent? It depends on the density of the area. Like in Europe, it's quite dense and in some part of the world, it's less dense. So it makes change when you want to ask for a request on the API. Do you need to have OSM metadata like the contributors, the timestamps,
03:04
the change set if you want to make some analysis about contributions in a specific area? Do you need data design for display, just map rendering or analysis or routing?
03:20
Sometimes, you might have just a subset of keys in GIS software. And but you may have a lot of keys in OpenStreetMap, like defining some very precise values. So do we need all the keys and values? So let's check what we can do.
03:41
So just with QGIS only quickly, like in your QGIS browser, you can already have like the OpenStreetMap map next styles. Just go in your browser and then XYZ tiles, then you can have OpenStreetMap. Then you can find some online websites doing the transformation from
04:03
OpenStreetMap to GIS format. The most famous one would be like download.geofabric.de. It's providing worldwide files. For me in France, I use like this one. It's quite convenient because it's coming with the QGIS styles,
04:22
the screenshot on the right. It's coming with a QGIS project. It's not only in France, but there are also other countries. And sometimes I should, yeah, try this style on other OpenStreetMap files because it's just nice to have a style already pre-made. And there are like other schools, data portals.
04:43
And look in your own country for OpenStreetMap data sets. Then with QGIS also we can open straight like PBA files or OpenStreetMap files. It's working, like you can load them. Thanks to OGA and GDA in the background.
05:04
There's different websites to download these files. Again, geofabric.de and there are plenty of other files, websites. But it started to be an issue when you want to work with some specific keys in QGIS. If you open a row OpenStreetMap file in QGIS,
05:23
you will see this field called other tags. It's combining all the keys in a single field. As you can see, like denonation, landmark, genus, leaf, sickle.
05:40
So it's all in a single field. And it's not convenient sometimes to just work on a single key of this field. So this is called an htor field. There is a support in some tools about htor. In QGIS you can use the processing toolbox and look for htor.
06:02
You will find the explode htor field. So you put your vector layer, you put your field in it, and then it will explode all your keys and values in separate fields. Is what I'm using. If you don't want to use the processing toolbox,
06:21
you can use a QGIS expressions. I guess all you know about this dialogue. So look again for htor. You will find htor to map. It will explode all your keys and values. So here I'm using like htor to map other tags, then I want the description.
06:41
And the output is just the value of the key description. Then there are a lot of plugins in the QGIS plugin repository. So let's have a look to a few of them. This one I use it a lot, quick map services.
07:05
It's showing like open topo map in the background. It's based on open street map. So just be sure that you go in your quick map services, then go in settings, more services, and then you can click get contributed pack.
07:21
Cuz by default, quick map services is just providing just a few base layers. And by going in these settings, it will add way more backgrounds like shown on the screen shot. Then quick OSM plugin, so it's to download data on the fly.
07:45
So it's in the background, it's doing some request to an HTTP API called the Overpass API. Maybe you have used the Overpass Turbo, it's a web interface for Overpass API. So Quick OSM is doing exactly the same, but in QGIS desktop.
08:05
So I am the developer of Quick OSM, and I've just released the last version like this weekend during the QGIS contributor meeting. So I really invite you to upgrade because I fixed some annoying bugs, I would say.
08:20
And if QGIS proposes you to upgrade your plugins, just do it. Because I know we are sometimes introducing new bugs, but we are also fixing quite a few bugs in some release. So please upgrade. So Quick OSM is targeting a few kind of users.
08:40
Like it can be non-OSM contributors, and also some OSM Overpass API experts who know how to write queries. And it's doing some magic for you in the background, like removing the htorque tags automatically. But it's using an API, so you have a limited amount of data that you can
09:03
download in the single request. So let's make an overview. The quick query panel is the first one. You can search in your native language. For instance, in English, it's a bakery. In Italian, sorry, I don't speak Italian, but it's Panetalia.
09:24
In French, it's Boulangerie. And in OSM, maybe you don't know, but it's shop equal bakery. Sometimes we forgot it could be, I don't know, another key. So that's convenient because now you can write in your native language. And Quick OSM will just find the correct key and
09:44
value, we added multi-key support, we'll show that. So yeah, I'm French, I like cheese and bread. So I'm looking for Aver Boulangerie. So I could write Boulangerie, so I have shop equal bakery.
10:04
But I also want to find places where I can buy cheese. So I could type shop, then equal cheese. So this in the background will generate for me the query where I can find either bread or cheese.
10:20
Then I can write the city where I want to look for data. Map presets, for now there is only one preset by default in the plugin. The idea is to download many layers, many OSM objects with many like different QG symbology by default in a single click and click button.
10:45
So urban, like as you can see, there is roads and buildings with a style with some labels on the roads. So the idea is that like you just double click on urban and it will download for you, it will generate the queries. And it will apply the QG symbology automatically.
11:05
So we can also, you can also create your own presets on your own computer. So you need to define a preset name, a description. For the preset, then on the left, you can define like your different layers that you want to query.
11:22
So this one, it has buildings and roads. And then for each query, you can define the content of the query. Like this one is building with all values. You can define the extent or the area. And the last step, you can define some QG symbology to apply automatically
11:40
when the download will be done. So as I said, for now there's only a single preset. So it would be very nice if we could have more presets. I'm thinking about like a bicycle map, land use. Some people asked me yesterday, it would be very nice to have a natural layer, a natural water layer, so the loading's automatically like rivers,
12:03
dams, and a few things. So I finally took some time to write the documentation out. You can contribute to Quicko SM. So if you are doing the same kind of presets, and you think that it would be nice to have it by default, yeah, you can contact me and I will add it to have, yeah,
12:24
automatic maps, no knowledge to write queries, and it's coming with nice styles. Maybe also we can run Quicko SM in Processing. It's the toolbox available on the right.
12:42
Right side of QGIS, and there is a QGIS modeller where you can like make a complex algorithm by adding a few algorithm in. So for instance, this one, yeah, it's downloading fire agents in an area, reprojecting in meters because of data from OpenSuitMap coming in degrees.
13:02
Then doing a buffering and applying a QG style. So when I launched this QGIS model, it's only asking me for an extent, then all it done, and then I have my final layer with my buffers and fire agents.
13:23
If you have a local OSM file, you can also open it, download it with a website, or you can open the PDF or XML file, and you can do some filtering, like ShopEqual Bakery again. But it's not using the Overpass API, so it can do more like you can have a more,
13:43
a bigger amount of data. Some quick tips, every layer is made with KUKO-ASEM. Now you have a button like reloads a query in a new file because the OpenSuitMap database is updated quite often, so you can right click and reload.
14:01
It will reload the query and then add a new layer in your QGIS. You can do like, you can search in many places at the same time, like Montpellier and Paris. By, separated by a semicolon. KUKO-ASEM will also parse all your fields in your layer,
14:22
and it will add some default actions. If you don't know what is an action, it's like a trigger that you can, like this one, it will like scan, like oh, it found a website, or it find a URL or Wikipedia key. And then it, either opening like the website
14:41
or opening JOSM. This one, it's like there is a Mapillary picture attached to this OpenStripMap node. And I could open like the Mapillary website.
15:00
So it's the same, like if there is a Wikipedia or URL, or it's just scanning and trying to find some default actions. Translations, I put the link if you want to translate in your own language. But the issue with KUKO-ASEM is like,
15:22
there is some issues with complex OSM relations because as I said, it's not maybe compatible with GIS format. So only relations equal root, multi-line strings, multiple ligands or boundaries are supported. If you have other relations, then you need some post-processing to like,
15:42
to know what you really want about this relation, and to build like the geometry. About geocoding, that's new in QGIS 3.20. Before it was a plugin, but now it's integrated by default,
16:00
thanks with a new server like nominatime.qgis.org. So they included a new batch nominatime geocoder. If you have a lot of address that you want the geometry, it will use in the background, yeah, this server and ask like the point.
16:22
And it's integrated as well in the locator bar at the bottom of QGIS. You can start typing an address, and it will also ask the nominatime API in the background. So before as well, it was a plugin, but now it's integrated in QGIS Core.
16:46
If you do some routing, there's some plugins that you can use. I'm using sometimes RLS tools, it's using also an online API. You can choose between open root service or a graph helper.
17:01
It's the same as on the OpenStreetMap website, you can choose the provider. So you can do different like isochrones or matrix calculations. Then talking about database. So if you go on the Wiki page on OpenStreetMap, you will find a table like similar to this one.
17:21
And then it's showing you all different schema or tools that you can use. And you can see there is some green and red box. So according to what you want to do, if you want your database to be updatable, or like if you want all fields, like H store, we can find it again.
17:40
You might need to choose this tool or the other one. Just the main one which is used is called OSM2PGSQL, mainly used for rendering. And it's very widely used. I'm using in POSM when I need. This one is also updatable.
18:00
I would show that just after. With Docker, it's a nice project that I mean, like to quickly spin up a database using Docker. So there are a few containers inside this Docker project. A POS SQL container, a container to make the update.
18:26
So the idea behind a Docker Compose project is like it's hiding all the complexity behind like this command line tool. So the idea is like you drop a PDF file in the folder.
18:42
You set the area that you want to keep in your POS SQL database. There is just one file for settings provided, but I can let everything by default. And I do a make run. Then all the magic of Docker is like
19:01
everything is already set up behind. So it's providing me a schema with all tables and layers. There is a mapping file by default, like OSM admin, OSM IRAs, amenities. So it's possible to change a mapping.
19:23
There is a lot of settings. So by default in the Docker project, it's set up to automatically update your POS GIS database every two minutes. This is, you can change it. Of course, two minutes it's very, very quick.
19:41
Then there are some advanced things like SQL triggers and views that you can set up when the project is starting and also some styles. Vector tiles, that's new again, like since a few versions in QGIS 3.14.
20:04
There is map tiler and open map tiles. There is a conference, the first 4G, I think it's this afternoon if I'm correct. Is also one like from the state of the map last weekend. But yeah, if you have some,
20:21
I won't be able to present vector tiles. But yeah, I have a full presentation about their tools. So yeah, that was just a subset of tools that I'm using with OSM in QGIS. Maybe I missed some links or some tools that you might use. So yeah, if I forgot some, just tell me.
20:43
I will have a look.