We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

“Keepin' it fresh (and good)!” Continuous Ingestion of OSM Data at Facebook

Formale Metadaten

Titel
“Keepin' it fresh (and good)!” Continuous Ingestion of OSM Data at Facebook
Serientitel
Anzahl der Teile
70
Autor
Lizenz
CC-Namensnennung 3.0 Unported:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
Identifikatoren
Herausgeber
Erscheinungsjahr
Sprache

Inhaltliche Metadaten

Fachgebiet
Genre
Abstract
Building forward from our work on Mobius Logical Changesets (presented last year at SotM US 2018), we have created an automated ingestion and integrity framework for OSM data that allows us to selectively update parts of the map instead of doing a full snapshot change all at once. Decomposing a large set of changes in this way gives us the flexibility to rapidly ingest our own additions to the map, focus on geographical areas of importance to downstream products, and allows us to quickly apply hotfixes whenever egregious problems do arise. With millions of tiny changes happening every week, we have created a system that is built on per-feature approval and preprocessing, that allows us to ingest changes at scale, while creating rules to automatically process logical changesets and enforce integrity constraints (e.g. anti-vandalism, anti-profanity etc.). Due to the contextual nature of some of the changes in OpenStreetMap, the system combines Human Approval, necessary for highly visible features such as names of large administrative areas, with Automated AI/ML-based approval: for example, using computer vision techniques to reconcile newly created features against satellite imagery ground truth, or applying NLP techniques to determine whether other user-visible string changes are sensible and valid. These components are combined to create a continuous ingest-validate-deploy cycle for OSM map data.