We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Crazy data: Using PostGIS to fix errors and handle difficult datasets

Formal Metadata

Title
Crazy data: Using PostGIS to fix errors and handle difficult datasets
Title of Series
Number of Parts
188
Author
License
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language
Producer
Production Year2014
Production PlacePortland, Oregon, United States of America

Content Metadata

Subject Area
Genre
Abstract
Inteligeo is a system that stores a lot of information used by the Brazilian Federal Police Forensics to fight crime, initially in the environmental arena with a later expansion to other types of crime. During the construction of the database a lot of problems appeared for which PostGIS was the key to the solution.This presentation describes problems encountered by the team while loading 850+ shapefiles into the database, linking with external databases and building 950+ views of the data.Although the content of the recipes is very technical, the general concepts will be explained in an accessible language and correlated to real world cases.Topics:*Definition of crazy data in our context*Quick recipes- Spike removal- Invalid geometry detection and fixing- Filling holes- Raster image footprints- Hammering data into correct topologies- Speeding data visualization with ST_Simplify and PGSQL 9.3's materialized views- Rough georeferencing using an auxiliary table- Creating constraints*How is crazy data generated and our experience in handling each case- Large datasets- Lack of validation- Reprojection- Geometric operations- Topological errors- Imprecise definitions- Legacy databases- Bad georeferencingWe will also discuss why is handling crazy data important for the Brazilian Federal Police, our efforts in cleaning up data at the source and the implications of geographical data in general for fighting crime.
Keywords