Creating GPX tracks from cycle routes in OpenStreetMap
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 490 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/47017 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 202039 / 490
4
7
9
10
14
15
16
25
26
29
31
33
34
35
37
40
41
42
43
45
46
47
50
51
52
53
54
58
60
64
65
66
67
70
71
72
74
75
76
77
78
82
83
84
86
89
90
93
94
95
96
98
100
101
105
106
109
110
116
118
123
124
130
135
137
141
142
144
146
151
154
157
159
164
166
167
169
172
174
178
182
184
185
186
187
189
190
191
192
193
194
195
200
202
203
204
205
206
207
208
211
212
214
218
222
225
228
230
232
233
235
236
240
242
244
249
250
251
253
254
258
261
262
266
267
268
271
273
274
275
278
280
281
282
283
284
285
286
288
289
290
291
293
295
296
297
298
301
302
303
305
306
307
310
311
315
317
318
319
328
333
350
353
354
356
359
360
361
370
372
373
374
375
379
380
381
383
385
386
387
388
391
393
394
395
397
398
399
401
409
410
411
414
420
421
422
423
424
425
427
429
430
434
438
439
444
449
450
454
457
458
459
460
461
464
465
466
468
469
470
471
472
480
484
486
487
489
490
00:00
Process (computing)Cycle (graph theory)Complete metric spaceAreaPersonal identification numberCycle (graph theory)TrailRoutingBitSampling (statistics)Level (video gaming)Process (computing)Computer animation
00:39
Multiplication signCycle (graph theory)State of matterRoutingWebsiteSpeech synthesisInjektivitätComputer animation
01:06
Cycle (graph theory)Boris (given name)Cycle (graph theory)RoutingEvent horizonSign (mathematics)MetreOnline helpComputer animation
01:42
Cycle (graph theory)WebsiteRoutingMathematicsCycle (graph theory)Open setLevel (video gaming)Computer filePresentation of a groupComputer animation
02:15
Computer fileRoutingSheaf (mathematics)BitNeuroinformatikMobile appComputer fileCycle (graph theory)Open setLevel (video gaming)TrailSpacetimeView (database)Computer animation
02:40
Finitary relationData modelCycle (graph theory)Theory of relativityRoutingData modelLevel (video gaming)Open setMereologyLibrary (computing)Computer fileLine (geometry)Bit ratePersonal identification numberTrailEncryptionComputer animation
03:16
MereologyDifferent (Kate Ryan album)Functional (mathematics)Line (geometry)Computer fileConsistencyRoutingOpen setLevel (video gaming)
03:50
Graph coloringSheaf (mathematics)Open setLine (geometry)Level (video gaming)Near-ringShape (magazine)RoutingComputer animationDiagram
04:13
Cycle (graph theory)RoutingLattice (order)Line (geometry)TrailPoint (geometry)Bit rateRight angleGreen's functionWebsiteWeb 2.0Computer animation
04:53
RoutingPoint (geometry)Line (geometry)BitCycle (graph theory)View (database)Level (video gaming)AdditionComputer animation
05:19
Process (computing)Matrix (mathematics)CoefficientDistanceDirection (geometry)Dijkstra's algorithmAlgorithmPoint (geometry)Cycle (graph theory)Open setRoutingBitLevel (video gaming)ResultantPoint (geometry)Ferry CorstenMatrix (mathematics)Process (computing)Computer fileCASE <Informatik>Direction (geometry)CoefficientQuery languageComputer configurationLine (geometry)Dijkstra's algorithmDifferent (Kate Ryan album)Binary fileMultiplication signNumberSheaf (mathematics)DistanceTheory of relativityHydraulic jumpConnected spaceRoundness (object)Bit rateMachine visionConnectivity (graph theory)CoalitionSet (mathematics)Condition numberRight angleReading (process)Extreme programmingComplete metric spaceExplosionComputer animation
10:04
Link (knot theory)Computer configurationHydraulic jumpRoutingComplete metric spaceAlgorithmScaling (geometry)Process (computing)MappingCycle (graph theory)ConsistencyComputer animation
10:32
RoutingAlgorithmConsistencyCycle (graph theory)NumberSheaf (mathematics)Different (Kate Ryan album)Medical imagingComputer animation
11:06
WindowModemMIDISheaf (mathematics)Resolvent formalismGroup actionRevision controlReading (process)RoutingPhysical lawBit rateComputer animation
11:45
Cycle (graph theory)Computer fileSheaf (mathematics)RoutingView (database)Reading (process)Computer animation
12:26
Physical systemContinuous functionPoint (geometry)NumberMultiplication signPoint (geometry)Moment (mathematics)FeedbackProcess (computing)Graph (mathematics)Sheaf (mathematics)Analytic continuationCycle (graph theory)RoutingComputer filePhysical systemDependent and independent variablesAdventure gameOpen setProfil (magazine)Coordinate systemCASE <Informatik>Limit (category theory)Network topologyFunctional (mathematics)Mobile appShooting methodAdditionReading (process)Digital photographyLevel (video gaming)Bit rateComputer animation
15:23
FrequencyRing (mathematics)Goodness of fitRoutingCycle (graph theory)Latent heatMultiplication signOpen setGraph (mathematics)RootMoment (mathematics)CausalityBoss CorporationInformationWhiteboardGodLevel (video gaming)CASE <Informatik>Computer animation
17:37
Level (video gaming)RoutingSoftwareEndliche ModelltheorieDataflowPerformance appraisalOpen setProjective planeDigitizingLaptopCartesian coordinate systemMappingMultiplication signStandard deviationTrailMereologyConsistencyCycle (graph theory)Different (Kate Ryan album)Coordinate systemCASE <Informatik>BuildingOperator (mathematics)Point (geometry)Focus (optics)Bit rateService (economics)Condition numberQuicksortShared memoryAreaProduct (business)Graph (mathematics)Data storage deviceCodeComputer animation
22:59
WindowLevel (video gaming)Division (mathematics)Computer clusterRow (database)BitLatent heatType theoryDifferent (Kate Ryan album)CoefficientMatrix (mathematics)RoutingSheaf (mathematics)WeightRevision controlComplete metric spaceExtension (kinesiology)Insertion lossElectronic mailing listComputer animation
23:59
Point cloudFacebookOpen source
Transcript: English(auto-generated)
00:05
I'm Henry and I'm going to speak about creating GPX tracks from cycle routes in OpenStreetMap. So I'll give you a brief introduction to me and the problem I'm trying to solve, a little
00:20
bit of an overview on OpenStreetMap data and then I'm going to speak about how we process cycle routes from that data. Then finally I'm going to give some examples of completed cycle routes and then speak about some of the challenges I encountered and some of the next steps. So as an introduction, I'm Henry, if you want to contact me, Twitter, LinkedIn.
00:45
I actually work for a company called Cytira, which we make a device to grow human and animal cells, so it's quite far away from what I'm speaking about today, but something I really, really love is cycling. I use cycling every day as a way to get to work, but then at other times cycling
01:02
it's less of a to get to a destination, it's more about of a journey. And so here is two examples of some of the amazing cycle routes in Europe. This is the Danube cycle path and this is a very long route in Italy. So an issue that comes when trying to follow these cycle routes is often the signage
01:24
is very poor. In the UK you get these stickers stuck on lampposts and then people peel the stickers off. Sometimes the sign is 20 metres down the road that you've meant to turn down. And then once you're off route, it's quite hard to recover. There are some amazing resources out there that help with that.
01:44
Open Cycle Map is an excellent website where it shows all the regional and national and international cycle routes, but it's not always the easiest to follow on your phone. So I'm going to present Open Cycle Export, where given a country and a cycle route reference,
02:05
you can take the data from Open Street Map and then download it as a GPX file. So why a GPX file? GPX files are portable to many different devices. So you could follow the route on your watch, on your cycle computer, or there's many,
02:25
many apps available that you can use to follow GPX tracks. So the next section is to a little bit of an overview of Open Street Map data with relevance to cycle routes. So a lot of this will be familiar to people here.
02:42
But as a quick overview, the Open Street Map data model consists of NODES, which is coordinate in GPS space, Waze, which is ordered collection of NODES, which could represent a road or a footpath, and relations that are collections of NODES or Waze.
03:05
And that's how a cycle route is stored. So a cycle route is stored as a relation where they're just stating all the different ways that make up that cycle route. So all the different roads involved. So I thought, great, it'll be very, very easy to take the data from Open Street Map
03:22
and download it and make a GPX file. So I used a function called line merge, which is part of the Shapley Library in Python, that's meant to take lots of different lines and put them together to one continuous route. However, as you can see, the route that came out is not one you'd want to follow.
03:41
And that's because these large inconsistencies. And to drill down and have a look at where these inconsistencies originate from. Here is an example of one of the issues. So this is a roundabout near Peterborough. And obviously, if you're trying to take a continuous route, the Shapley line merge
04:04
doesn't know which section of the roundabout to follow. So that's an issue that needs solving in Open Street Map data. And also, what we can see from looking at why this occurred is the colours represent
04:21
different ways that make this up. And the green way is the roundabout, and then there's the two joining roads. So actually, there's no point that joins the ways of the joining road and the roundabout. So there's two problems here. One, the line merge doesn't know which track of the roundabout to choose.
04:42
And two, the roundabout way doesn't join at an endpoint to the ways that join the roundabout. There are other issues in some of the cycle route data. For example, here, we again get the same problem where right at the top where the yellow joins the green, this is a joining point of two ways where that doesn't correspond
05:04
to an endpoint of a way. So that's hard to route. In addition, there's just a complete gap. So line merge completely falls down in the scenario where it's trying to resolve a route. So that was a little bit of an overview of what the data is stored in Open Street Map.
05:24
I'll now talk about how we can process that data and try and result in a cycle route that you can then follow on one of your mobile devices. So as a very basic overview to the tool, it uses the Overpass API, which is an excellent way to query Open Street Map data.
05:45
And then you don't have to download any big binary files. So you can download just what's relevant to your use case. So here, we're downloading a particular relation for a certain cycle route. And this comes with all the ways and nodes that make up that cycle route.
06:03
We then process the ways. As I showed before with the roundabout, there's places where the ways join, but that doesn't correspond to an endpoint. So we need to split the ways by the points that they join other ways. And then once we have those way points where they all join,
06:24
we need to do some processing to work out the best way to go between them. And finally, once we've done that processing, we then create a route from that setup of waypoints. And finally, we can download a GPX file from that.
06:41
So processing ways. We find all the places that ways intersect, and we call that waypoints. Then we split the ways at all the waypoints so that we have subsections of ways. Then we create a cost matrix between all these intersections.
07:00
And so cost is something that we've defined as a distance, the distance of the line between two points, times by coefficient. And so a coefficient for a way that is primarily for cycling along, the coefficient is very low. We choose one coefficient.
07:20
For a way where cycles are permitted, a road with the specific segregated component of cycleway, we'll choose a cost such as two, so a little bit more penalising that route. Or for just a generic road that has no cycle provision, a cost of 10. And finally, for a road that it's maybe you going the wrong way down a one-way street,
07:44
so it's an impassable road, but there's still a connection, we'll choose a coefficient of 100. So we're really, really trying to penalise, but the whole purpose is we want to, whatever the data holds, we want to be able to get a final cycle route all the way and then identify the issues at the end.
08:02
And finally, we have a coefficient where there's no connection between two waypoints. And then that's a coefficient of 1000. So it's very penalised, but we still will make that jump if we need to make that jump. So that's how we build up the cost matrix. And to take that back to the example with the roundabout here,
08:21
these are two directions between the different waypoints because sections can only allow you to pass one way. This is a roundabout in the UK, so we have to go round it clockwise. Therefore, the join to the roundabout, it's a two coefficient each way.
08:41
To go round the roundabout clockwise is a two coefficient, but to go round it anti-clockwise is 100 coefficient to penalise that option. And again, the way on the other exit to the roundabout is two both ways because it's permitted, but it's a road, it's not specifically for cycling along.
09:00
Then we have this cost matrix and we want to create a route from it. So we've downloaded a relation. And the first thing we need to do is work out where the route is going from and to. Initially, what we do is we just take the furthest two points. This falls down in a number of different scenarios, but initially it works for trying to get the longest route available.
09:23
And then we use Dijkstra's algorithm to find the shortest path between these two furthest points. And we create a route in both directions. Obviously, one way you'd want to go one way around the roundabout and the other way you want to go the other way around the roundabout. So some examples of the created route. You can't see the blue line when it's on the entrance and exit to the roundabout.
09:44
But if you are coming from this point to over here, you go clockwise around the roundabout. And if you're coming back, you'd obviously go clockwise, but the other way around the roundabout. So this is how, in a very low level example, we choose the correct direction to go along the different options for the road segments.
10:05
For the example where we have a complete lack of connection, we choose what is close to being the, what looks to be almost the closest. To my eyes, actually, this one looks closest. But so this is not actually a way in OpenStreetMaps,
10:23
but the routing algorithm has done the jump to try and find the best available option to try and make up the route. So that was examples of, at a very low scale, what the routing algorithm does to weather the slight inconsistencies in the data.
10:46
Here's some examples of some completed cycle routes. So this is the RV4 in Belgium. And you can't see on this image, but maybe if I zoom in,
11:00
there is a number of different, there will be sections in here that resolve one way sections. So for example, here, this route actually has a version that goes from Brussels
11:27
and a version that comes to Brussels and resolves the one way sections in it. So that's an example of a route in Belgium. There's also some fairly long routes. This is one in France that follows up the Lotte Valley.
11:44
And another example of a route in the United Kingdom. If you'll notice here, what's quite alarming is these completely straight sections. It is the discontinuity of either the route or the data that represents the route.
12:03
So I think the downloaded GPX files have to come with a caveat of declaring what percentage of the route is actually following ways, because that wouldn't be achievable to follow. But we're trying to make a very tolerant routing algorithm. So you can, from a particular national cycle route,
12:22
you can get an overview of the whole journey. So I had a number of challenges that are still present and got some next steps to look through, which I'd really welcome for feedback and help on, because I think there'll be quite a lot of time for questions and suggestions. At the moment, there is no elevation data for the points on the route.
12:45
And if you're trying to plan a route on your bike, in the apps, they have really, really excellent elevation profiles. But this elevation profile would be entirely zero. I've tried to use open elevation API, but haven't been successful in getting the responses accurate.
13:04
Often it just times out. So I'd like a way to get elevation data for these route downloads for the GPX files. In addition, I'd like to use correct spatial reference system. At the moment, it does what is a horrendous thing.
13:21
Really, it uses just the GPS, the latitudes and longitudes in order to find distance. It doesn't use the have assign function, which is a huge limitation. But I wanted to find a nice way to change between coordinate reference systems or spatial reference systems. So I'd welcome thoughts and feedback on that.
13:42
And the continuity of cycle routes or route data, as we saw in the example on Route 2 in the United Kingdom, there was massive sections where there was gaps. So this is either a process of trying to encode that data into OpenStreetMap or to see why the route doesn't actually continue in reality,
14:03
which is also a big concern. And finally, finding the true start and end points. If a route, say, did a horseshoe, this process would not result in the whole route because it would just choose the two furthest apart points. And so it would miss almost half the route in that case.
14:23
So those are some of the current challenges. Next steps, I'm thinking that the best approach to share the generated GPX files is on GitHub releases, because that means that they're always available online. And so because they take a little while to process,
14:40
it then doesn't require someone to have to wait for that processing to download them. I would like to declare which routes are not continuous. So you have a percentage of the route that doesn't actually follow ways. So you know that in advance when you're trying to use these GPX files to go on a long adventure. And additionally, because often people won't necessarily want to follow
15:03
the 200 miles of route as a continuous section to split the routes by the towns they pass through, which is a easily achievable thing with the Overpass API by looking at whatever towns they pass through and then working out little sections that you can then download. So those are some of the next steps available.
15:23
And we've got a good period of time for questions. So I'd love any questions or comments or suggestions on how to solve some of the challenges. I'm sure it will be easy for others. Did you need to incorporate some country specific knowledge? So in order, the question is around
15:44
incorporating country specific knowledge about routes. So I didn't include any country specific knowledge about the routes. But I did use the OpenStreetMap tags
16:03
regarding whether something is a one way route or one way permitted. Well, two way for cycles apart from one way for cars. Yes, temporary, a convenient situation
16:23
because in Brussels, I know this example, very good situation. There are private streets, there are also walks in some streets. So, you know, the little ring in Brussels they are made by bicycles, by bicycle paths in this little ring.
16:41
But after it will be very nice. But now there is some section. It's very not absolutely not convenient for cyclists during the walks. And so are you taking this in account in your in your. So the question is regarding the road works on a route and how that's taken into account
17:02
if the if there isn't data in OpenStreetMap for it at the moment, I won't take into account for it. But if you know of some tags in OpenStreetMap that encodes that information, that would be very useful to put into the tool.
17:44
OK, a lot of not all, but a large quantity of the one way roads allow for bicycles to go near the road. The Google Navigator, I have read on OpenStreetMap how they say that if a company, a country's standard is to allow
18:04
opposing flow for cyclists, it then is as opposed to tagging it when it's allowed. Contra flow for cyclists, they'll tag it if it's not allowed contra flow for cyclists. So that's very good point. I need to incorporate that because it was very UK focused when I was originally building out the tool.
18:34
So this is sorry, I'm going to be a little bit long. OK, I only want questions.
18:40
Very concise questions. So you can answer them concisely. This, I'm terribly sorry, isn't the question. It's a software recommendation for projecting your cartographic coordinates. There's a thing called libproj, libproj, libproj for. Excellent. Great takeaway. You were no, no, no, no. You're helping this whole day.
19:02
This place to be ready. You are. I was going to ask why you don't get back data from the cyclists about derivation, because almost all people are there cycling. So you get from you can use travel routes and get data from there about the elevation of different roads and potentially even compare the GPX track of their route
19:24
to the route to see if there's any roadworks on the way. Yeah, OK. Yeah, but they're using their own digital evaluation model that's not open source.
19:41
OK, you can take this offline. There's a different question. You come across cyclestreets.net. I have done. Right, OK, because they have a lot of these questions. They have spent about a decade dealing with all these problems of cycle routing on the OSM data. So there's probably a lot of good ideas you can steal out of that to get less shitty routes. OK, great takeaway.
20:02
Maybe since they were there was a new project, which is called SQL OSM, which replaces deprecated open cycle map and which. Cycle OSM.
20:20
OK, you learn more from them. Works well for me. That's something that I think is an important part of the project. So when I have the website available to download the GPX files, it's also a case of saying hearing consistencies in the map,
20:41
because if you can't route it from a cycle, if the cycle route is already saying you can get to these two places, then if you can't route over a certain point, then it's an issue with the ways which might affect then car routing on top of open street map. OK, that's been in the back.
21:08
I didn't, but I'd really like to see it. What? What's it called?
21:27
So there are there are lots and lots of things available to say I want a cycle friendly or a fast route between two points, but to download just a cycle route.
21:48
OK, I'm going to cheat. Yuri, could you come forward and sort of plug yourself, your laptop or both in? Who? OK, next question.
22:00
Where was that you got with regards to the elevation model? Did you look into governmental city applications of open data? They often have their elevation models published online as open data and that you could use. And that would be for a city level or for a country level? Depends.
22:20
There's a lot of things. Thank you. Cool. You had a question. Have you been thinking about the incorporating heat maps like to use data in which parts are most used? I haven't thought of that. That's a really nice feature. I use it all the time when I'm mapping.
22:41
Gosh, you have so much to take. There's another question in the back. Sorry, the last one. I haven't seen cycle.travel before. Oh, brilliant.
23:02
OK, gentlemen, in the one bit last row. Good question. So is it possible to download the intermediate data that has like the matrix weights so that you can perhaps add references that can be taken into account in that sort of
23:21
like shortest path, such as, for example, if you prefer specific types of moves or extend it to one like things currently the matrix is not available to download, but you would really want to get in just before that step, before the coefficients are applied.
23:40
So then you could provide your preferences. So perhaps a extensible version where you can choose your own coefficients for different sections of route would be a good idea. OK, that was the last one. Thank you for having a great meeting. Thank you.