Surveying amenities for OSM at scale
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 351 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/69136 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Production Year | 2022 |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
FOSS4G Firenze 2022108 / 351
1
7
13
22
25
31
33
36
39
41
43
44
46
52
53
55
58
59
60
76
80
93
98
104
108
127
128
133
135
141
142
143
150
151
168
173
176
178
190
196
200
201
202
204
211
219
225
226
236
242
251
258
263
270
284
285
292
00:00
Scale (map)Reverse engineeringAmenable groupData storage deviceRow (database)Type theoryBitTable (information)Different (Kate Ryan album)Execution unitMultiplication signBuildingAddress spaceNumberInflection pointOpen setMeasurementObject (grammar)Data modelMultiplicationPoint (geometry)Texture mappingReverse engineeringGeometrySemantics (computer science)Source codeExistenceAttribute grammarCodeState of matterPresentation of a groupPairwise comparisonComputer animation
04:27
Lie groupTexture mappingText editorDigital photography
04:59
Open setPoint (geometry)Texture mappingField (computer science)Mobile appQuicksortAreaSet (mathematics)Computer animation
06:01
NumberServer (computing)Type theoryMobile appForm (programming)Centralizer and normalizerComputer animation
06:19
Data modelDoubling the cubeValidity (statistics)InformationDigital photographyDiagram
06:50
Open setExterior algebraProcess (computing)TheoryTexture mappingDisk read-and-write headAreaSoftware developerComputer animationJSON
07:56
Robot
08:11
Message passingType theoryRobotPoint (geometry)Message passingComputer animation
08:47
Displacement MappingVisual systemDisplacement MappingComputer iconOnline helpInformationTexture mappingPoint (geometry)Electronic mailing listTap (transformer)Boolean algebraLetterpress printingProduct (business)RobotAddress spaceProjective plane2 (number)DiagramComputer animation
10:42
Random numberPlastikkarteLink (knot theory)Digital photographyField (computer science)RobotDigital photographyOpen setData modelTexture mappingUniform resource locatorDatabaseLink (knot theory)Different (Kate Ryan album)Computer animation
11:23
Random numberText editorTexture mappingCase moddingData modelOpen set
11:50
Mobile appOpen setVideo gameInflection pointAreaLogical constantText editorSoftware developerMultiplication signProduct (business)Process (computing)Texture mappingField (computer science)Mobile WebComputer animation
13:08
Open setSet (mathematics)Operator (mathematics)Source codeTexture mappingType theoryLink (knot theory)Plastikkarte
13:53
Operator (mathematics)EmailUniform resource locatorLibrary (computing)Endliche ModelltheorieOpen setTexture mappingCase moddingData modelSet (mathematics)Address spacePoint (geometry)Process (computing)Reverse engineeringComplex (psychology)Type theoryComputer animation
14:55
Closed setText editorExterior algebraOpen setEndliche ModelltheorieTap (transformer)Texture mappingType theorySet (mathematics)Source codeComputer animation
15:27
Logic gateInsertion lossOpen setOpen sourceAmenable groupMobile appTexture mappingOpen setFlow separationArchaeological field surveyState of matterSoftware testing1 (number)Point (geometry)Text editorInflection pointRight angleAreaReverse engineeringRobot10 (number)BuildingMathematicsAxiom of choiceSource codeLevel (video gaming)Server (computing)Computer animationJSONXML
Transcript: English(auto-generated)
00:01
Hello everyone. I'm Ilya Zverev. I know everything about OpenStreetMap. I like to map it. And I really care about shops and amenities in OpenStreetMap, which is a bit unfortunate, because, well, OpenStreetMap has very few of these.
00:26
I don't use OpenStreetMap for searching for nearest H&M store or copy shop. I open Google Maps. And that's the common sentiment. Like, when you need POI, even commercial companies,
00:46
they just turn to commercial data sources, because it's much simpler. When I was working for a big American company to assess the quality of multiple POI datasets,
01:02
well, the state of OpenStreetMap was obvious from the beginning. Like, in the United States, it has only half a million POI. In comparison, all commercial data stores have 40 million, 80 times more. That fact alone was pretty decisive.
01:23
So, yeah, we paid for commercial data and so on. Even if you close your eyes at quantity, using commercial data is basically a simple table.
01:42
Each row has all the data for point of interest you need. Type, name, measure of existence, when it was surveyed, open addresses from house number to country, everything. Very simple to use. With OpenStreetMap, not so much.
02:04
With OpenStreetMap, you got tags. Data model has been evolved over 18 years, with thousands of people contributing it, and you can imagine what happens. Of course, there are some obvious tags, like name.
02:22
Actually, no, but kinda. There are types, which are kinda simple, but no. There are open hours, which are documented, but not easy to parse, but still. And there are ambiguous tags, like for floor in a shopping mall,
02:43
we got three different tags with different semantics, and you're not promised any. For payment types and fill types, you have to collect many tags into one single attribute again. And addresses are hard, which I guess Sara Hoffman can tell you in her presentation.
03:10
There are so many tags, and every country has their own different ideas on addressing. Like, you can't compare how buildings are addressed in the United States, in, I don't know, Czechia, in Japan, in Africa.
03:29
And more than that, in OpenStreetMap, in United States, only half of POI have addresses on objects themselves. Many mappers consider that you should get address for a shop from geometry,
03:48
like from enclosing building, or somehow revert geocoded. And as a person who has written, I think, the best revert geocoder for OpenStreetMap, revert geocoding is awfully hard.
04:02
You shouldn't make anybody do that, especially when they just need to know the address of a shop. So, yeah, commercial data source. But I care about OpenStreetMap. What can I do? Can I somehow fix this?
04:23
Well, I could try fix at least a quantity problem. Like, I can just take my camera and go outside, which I did like ten years ago. And I spent the entire day making photos of every shop, you know, the old way. And then came back and opened OpenStreetMap editor,
04:40
and was typing all the collected data for four days, morning to evening. And, yeah, OpenStreetMap got like 700 shops richer. But, like, everything hurts. Like, my feet, my hands. Never again have I repeated this experience, because it's very hard.
05:04
All these shops are obsolete now. It was ten years ago. Nobody, like, updated them. And I did that. Pretty, like, interesting experience. And I was, like, highly motivated, because I care about OpenStreetMap. People usually need some external motivation, like money.
05:26
Does OpenStreetMap have money? No. But there are commercial companies. Some that also care about points of interest, like Streetcred. They paid money to people.
05:43
Well, not directly paid, they organized a set of competitions. Who can collect the most points of interest in some enclosed area in a big city like New York? They used a simple, like, field collection app.
06:03
Kind of Next.js collector, or Qfield, or ODK, you know, the type. Like, when you prepare the form, spread it to apps, and then give that to people, and they collect it to a central server. Kind of like that. Except more flashy, because it was about the money.
06:22
And you can't trust people you pay money to. So you have to double-check everything and collect as many info as you could, including photos. Validation was very hard. But it worked. They collected, like, ten thousand shops, which is impressive.
06:41
All obsolete now, because it was very long ago. But, I don't know, the data model, there were not too many customers. The companies did not. And there are no alternatives. Sorry. So, what do we do?
07:04
Maybe... I still care. And maybe I can figure this some other way. Maybe it's not about quantity, but about quality.
07:20
Like, maybe, as a developer, I could invent some tools that make this whole process simpler, because we can't rely on paid people, there is no money in OpenSeatMap. We have to rely on the biggest strength OpenSeatMap has. It's millions of volunteer mappers. And for that, to make their lives easier, to not make their feet and head hurt,
07:42
we need to maybe make it simpler. I tested this tool's theory a year and a half ago. Like, privately, not in OpenSeatMap. I collected, like, in my very small area with lots of shopping malls,
08:02
half a thousand shops in, like, two weeks, on and off, using just a telegram bot. It's in Russian, sorry. This is the bot for searching, you know, the type. But the idea is that I didn't use any external tool.
08:22
I just opened my messenger, telegram is a messenger, it doesn't have anything besides, like, buttons and messages. No interactive maps, no nothing. And collecting half a thousand points was a breeze. I really liked the experience. I thought I was onto something.
08:42
How did I do that? Well, two things. First, telegram doesn't have interactive maps. Turns out this was really helpful, because we, as mappers, are easily distracted by the promise of interactive maps.
09:00
They are really easy to make, and they have everything, and they're fun to navigate, except they don't help with, like, work. Interactive maps for editing has very, quite a few drawbacks. Like, for example, they distract you
09:21
by having to scroll and zoom the map, like, instead of editing. You can't see all the shops, because there are too many in one point, and there's label and icon displacement that just doesn't allow you to see them. And there's basically, for each shop, there's just icon and label,
09:40
so you can't print all the information in it. It is hard to use interactiveators, and I did away with that. You know, when you don't have a map, you can have just a list, and lists are great. Like, I just saw shops around me in a list,
10:04
filtered by floor and address, of course, and it was really simple. So I look at a list, I look around and compare, and I immediately see what's missing, what's extra, what has changed. It got pretty fast.
10:21
And in the list, you are not limited by an icon and a label. You can print any info you want, like open hours on the phone, so you don't have to tap anything, you just immediately saw whether they have changed. Try, like, removing interactive maps from your project. It may boost your productivity.
10:41
The second thing, diagram what was created, is custom data model. I use my own database. I chose what fields I want to be searched and to be stored, like, I don't know, photos inside and outside, keywords for searching,
11:03
different links, like, locations, so on. Not, like, serious fields, but really helpful. I really benefited from this being my own database, not OpenStreetMap. So after I presented this Telegram bot,
11:21
enforced them and somewhere else, I got tons of questions. Are you planning to import this to OpenStreetMap? Why not use OpenStreetMap in the first place? Maybe you could make OpenStreetMap editor as a Telegram bot, and just know, why not use OpenStreetMap? It's a custom data model. It's, like, very different.
11:41
I don't want to deal with 18 years of tagging schemas in OpenStreetMap, except I care about OpenStreetMap. So that's got me thinking and presenting my ideas and, like, opening the editor,
12:02
learning mobile developments, because why not make open a mobile app to make people's life easier? And last week, I published the Every Door editor, which is a mobile app with basically the same principles, but for OpenStreetMap, that, like, fixes all the issues we have
12:22
with collecting POI data. And it's been, like, a blast. Since this March, I have been going out and mapping every shop I could find in my area and coming home and finding something to improve in the editor
12:40
to remove slowness, to make it more robust, more quick to collect. Like, this is a product of, like, half a year of constant mapping and making the process more effective. It's no longer, like, field data collection app.
13:01
It's like this juggernaut that will destroy everything. And at one time, I went to the second largest shopping mall in Estonia with 150 shops. I surveyed them in two and a half hours, basically a shop a minute.
13:21
And this might sound a lot, but it's not just names and types, as usual in OpenStreetMap, nor open hours. There's also, I don't know, wheelchair accessibility, whether you can pay by cards, and phones, links, operator names,
13:41
everything you can find in a serious data source. Every one of these 150 shops now has the full set of data collected in only two and a half hours. What enables it is that you don't see OpenStreetMap data model here. No text, nothing complex.
14:02
And also, it subtly changes the perceived priority of the text. Like, the address is actually the second after the name. So you're, like, inclined to just tap an address and set it directly to point of interest.
14:22
No reverse geocoding, finally. And multiple tags are just conflated into one, like Wi-Fi access. That's actually two tags in one button. And payment types, that's actually three tags in one button. Yeah, processing is not simple, but now it's documented.
14:43
Now you can have the data set of the same quality as commercial one from OpenStreetMap. And everything is optimized. Open hours. Open hours model in OpenStreetMap is very powerful.
15:00
Nothing you can find in commercial alternatives. Like, you can specify how the venue works on the first Thursday of the month or on December 24th. It's pretty simple. And the editor doesn't make you type, like, Monday, Friday, one, zero, seven.
15:20
No. Tap, tap, tap. You're set. Next step. Very fast. And finally, the date of serving. In OpenStreetMap now, you don't get when the shop was last seen in OpenStreetMap. All you have is date of last change.
15:42
And that can be like anything. That can be like reversion of malicious edit. This can be some bot, like, adding some useless tag. Now, this editor puts survey date on the forefront in these check marks. You go in the shop, near the shop,
16:01
you see it's still there. Nothing has changed. But it's still there. You just tap the check mark, and everybody in OpenStreetMap knows that you have seen it. It's still there. It works. It can be used. I went back to the shopping mall with 150 shops, and I confirmed every shop several months later
16:24
and added some new ones and removed some obsolete. In just half an hour, everything is made very fast. In the past half a year, when the app was beta testing, 700 mappers made 100,000 changes.
16:43
Roughly half of them were shops. Now, imagine what will happen, like, I published this app last week, and what will happen when tens of thousands of mappers from OpenStreetMap
17:01
get their hands on it, and they will start walking all around the world, adding millions and millions of shops and amenities they see, because it's so fun and easy. I just can't get my hands off the app, and I'm pretty sure other people too.
17:21
The state of points of interest in OpenStreetMap, it will be, like, on a completely different level. This changes how we map OpenStreetMap. This changes how we perceive OpenStreetMap as a data source. The choice between commercial data and OpenStreetMap
17:45
will not be that obvious in, like, three to five years. And obviously, that would put, like, many companies that fare by collecting POI data out of business.
18:03
OpenStreetMap was made to disrupt geospatial companies, and it doesn't stop at roads and buildings. It will continue, like, eat all of the areas that data companies profit from,
18:20
including right now POIs, but more things later. So, to stay in business, you obviously need to know how to work with POI, like we do in our company. We know everything about POI, we can do it. But you will have to learn how to live with OpenStreetMap too.
18:43
So, sorry about that, and thank you.