We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Developing with MapStore; creating a custom dashboard to map crime data

00:00

Formal Metadata

Title
Developing with MapStore; creating a custom dashboard to map crime data
Title of Series
Number of Parts
351
Author
License
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language
Production Year2022

Content Metadata

Subject Area
Genre
Abstract
MapStore is an open source product developed for creating, saving and sharing in a simple and intuitive way maps, dashboards, charts and geostories directly online in your browser. You can use MapStore as a product to deploy simple geoportals by using the standard functionalities it provides but you can also use MapStore as a framework to develop sophisticated web gis portals by reusing and extending its core building blocks. MapStore is also integrated inside geOrchestra as well as GeoNode open source project. The presentation will focus on the use of MapStore as web gis framework to create a modular, reproducible, simple yet powerful dashboard to visualize crime data (but in reality many different types of location based data) leveraging on GeoServer and PostGIS advanced functionalities. We will describe the main steps for creating such an infrastructure leveraging on the MapStore components and framework then we will cover how the existing dashboard template can be configured to work with your own data sources (eventually touching the needed processing steps for the data itself) including GeoServer and PostGIS advanced functionalities. We will eventually discuss further improvements and new features to evolve the current capabilities to capture new and emerging requirements. The goal of this presentation is twofold, on one side we are addressing developers in order to show them advanced usage of MapStore to develop compelling applications on the other side we will be addressing power users and system administrators willing to deploy the Crime Mapping dashboard to make their own data available without writing code.
Keywords
202
Thumbnail
1:16:05
226
242
Presentation of a groupUniform resource locatorCASE <Informatik>Data storage deviceLevel (video gaming)XML
Source codeSoftwareOperations support systemPerturbation theoryData storage deviceLevel (video gaming)Standard deviationClient (computing)Web 2.0MereologyProduct (business)CASE <Informatik>Front and back endsShared memoryServer (computing)DebuggerMappingComputer animation
SoftwareSource codeWebsiteVisualization (computer graphics)Category of beingLatent heatProcess (computing)CASE <Informatik>Uniform resource locatorCartesian coordinate systemLevel (video gaming)GeometryElectronic visual displayType theorySet (mathematics)AdditionInformationCategory of beingPoint (geometry)Table (information)Traffic reportingResultantComputer animationSource code
View (database)Software testingPointer (computer programming)IntelInterior (topology)Inclusion mapElectronic data interchangeTotal S.A.WebsiteEuclidean vectorSource codeSoftwareOperations support systemSingle-precision floating-point formatWeb pageLevel (video gaming)Service (economics)Data storage devicePlug-in (computing)View (database)Server (computing)Connectivity (graph theory)Revision controlSampling (statistics)MereologyCASE <Informatik>Cartesian coordinate systemProcess (computing)DatabaseSoftware frameworkTable (information)Source codeRight angleOpen setFinite differenceDebuggerType theoryComputer architecturePhysical systemLibrary (computing)Computer animationProgram flowchart
Point (geometry)Physical systemResultantComputer animation
Source codeSoftwarePolygonType theoryInformationPolygonMorley's categoricity theoremPoint (geometry)Event horizonVisualization (computer graphics)Type theoryCategory of beingMultiplication signSampling (statistics)Different (Kate Ryan album)GeometryInformationTable (information)Source codeSource code
SoftwareSource codeOpen setBeta functionComputer-assisted translationData conversionEvent horizonAbelian categoryGeometryTimestampDivision (mathematics)PolygonFloating pointServer (computing)Open setLatent heatTable (information)Event horizonCategory of beingComputer fileCASE <Informatik>Data conversionNormal (geometry)Shape (magazine)Point (geometry)Field (computer science)Connected spaceInformationFile formatData structureScripting languageRange (statistics)Multiplication signSource codeSource codeComputer animation
Data Encryption StandardEvent horizonExtension (kinesiology)Escape characterSource codeSoftwareView (database)Type theoryLocal GroupSuite (music)CountingASCIISyntaxbaumClient (computing)Data storage deviceDifferent (Kate Ryan album)Visualization (computer graphics)Source codeType theorySquare numberPolygonStatement (computer science)QuicksortCountingCASE <Informatik>Server (computing)Event horizonFunctional (mathematics)Point (geometry)Level (video gaming)Green's functionCuboidElement (mathematics)HexagonGeometryView (database)LogicTable (information)Computer animationSource code
Plug-in (computing)Fiber bundleData storage deviceLevel (video gaming)Cartesian coordinate systemEvent horizonVisualization (computer graphics)Different (Kate Ryan album)Connectivity (graph theory)Configuration spaceSoftware frameworkScripting languageProduct (business)Projective planeSet (mathematics)Fiber bundleTable (information)CASE <Informatik>Front and back endsDemo (music)Template (C++)Default (computer science)Computer fileRepository (publishing)File viewerLink (knot theory)Plug-in (computing)Server (computing)DebuggerComputer animationSource code
Component-based software engineeringTotal S.A.Total S.A.Normal (geometry)QuicksortPlug-in (computing)PlastikkarteCore dumpCountingMappingCategory of beingClient (computing)Connectivity (graph theory)CASE <Informatik>Different (Kate Ryan album)ResultantFile viewerIterationType theoryConfiguration spaceContent (media)Range (statistics)Data storage deviceRight angleTable (information)Server (computing)Line (geometry)Source codeVisualization (computer graphics)SynchronizationPresentation of a groupSingle-precision floating-point formatLevel (video gaming)Computer animation
Source codeSoftwareOpen setSampling (statistics)Software repositoryDemo (music)Event horizonWebsiteSource codeComputer animation
Range (statistics)Extension (kinesiology)View (database)IterationClient (computing)Computer animation
Transcript: English(auto-generated)
Thank you. Today's presentation is about developing with Map Store. So I will show you a use case where we were able to create a custom dashboard using data related to crime location. To do that, we use Map Store. And what's Map Store? Map Store is one of our products.
And it's a web.js client that allows the user to create and share maps, dashboards, or just stories. But for this use case, we extract the front-end part of Map Store and use that to interact with a geo-server post-js backend. And using also OGC standards inside this use case.
Let's start from the design process. So how we start and think, and how we work on the specific use case. We added some initial requirements. That was the crime location data that was composed for
the geometry point, the type, so the type of crime that we needed to filter then later with application with, and the report date. So this one was our initial data set that we had. And as a result, as a requirement, we needed to
visualize this information inside a map and display also additional data around with charts and table, and also be able to filter this data based on the property that we have inside the crime location data. These two screenshots show our initial work.
So it was a simple map and a simple chart done in JavaScript. We use a simpler version of Map Store in this case. Behind there is open layers. And we took the sample data that we had and put inside post-js, postgres servers, and started to insert these
through Azure server to play around and understand what kind of OGC service we could use with the front end of Map Store. In this case, we use WFS and SQL view to render the data on the map. And later, we'll show you the SQL view and what they are
to server side. We use the WFS to process the data and create charts, counters, and tables. And finally, we use also WMS to have a background map with data that we had from OpenStreetMap data source.
So this first part shows how we start to think and work with the back end side. So we decide to use a post-js, postgres server database, GeoServer to provide the processed data. And then we start to work with the Map Store framework.
So we took Map Store, extracted the front end with the React part. So we use the base component that provides open layers, leaflet, and cesium types, so a library behind. And the possibility to configure additional plugins to the application.
I don't know if there is sound. OK. The screenshot that you see here is one of the previous experiments that we did with the Map Store COVID map where
we use Map Store to interact with custom API that were provided and showing the data from the COVID. And with that, we were able to use the plugin infrastructure architecture to add the custom plugin such as the counter, the table, and so on.
So one of our steps is to start through sketch or mock-ups. So we started to draw around how the final UI should be composed, so the graphic composition, and also start to base our previous experience to enhance with new plugins the UI in particular.
The filter, now here is a sketch, so something is a mess there, but on the top right, we started to think on the interval date filter, and on the top left, the table, how to filter data. This one was an initial step that we used
to create our UI. Finally, when we added all the three pieces, the database with PostGIS and Postgres, GeoServer, and SpatialPodRider, and Map Store, we put them together inside a Docker container, so we were able to
share these in different operation systems. Next step, so when we finally added an idea on our infrastructure works, it could work, because at that point, we did not know the final result, but we had an idea on how to structure this. We start to prepare the data. As I mentioned before, it's a crime data, so it was
data related to crimes, and to generalize and normalize the table that contains information, we call that event, so that could be applied with a different type of data source. The important value that we need for the event are the
categorization property, a date, so a time, and a point geometry. For the crime, we also add a region, because the crime has usually, for the sample data that we use, administrative region to work with, so an idea where
we can associate this region. So the polygon that represent the region could be used to create the final visualization. So events region was our normalized table. Here, there is an example on how we use the New York Open Data Portal. We found the data related to the complaint, where there
are some fields that fit our use case. We pre-filtered the data to make sure that all the categories that we needed were there, and also to choose a specific time range. And finally, we export as CSV.
The same we did for the region. In this case, we look for precinct. And for the precinct, instead, we download a format that could contain spatial information, so a GeoJSON or a shapefile that could be then imported on a Postgres, PostGIS table.
Once we had downloaded all the data, we needed to add additional step that was the let's call normalization. So I'll ask a script that through the Docker entry point, allowed us to read the data and convert in a normalized structure.
These are the two structures, one for the event table. So the event table for our normalized structure should have these fields. And finally, the region table. These two tables are important because later we
were able to connect to a GeoServer through the data sources that GeoServer allowed. One of these is the PostGIS connection. And we were able to configure the layer to provide the layer to the client. So the data store that we had was event and region, but
wasn't enough for the visualization. Because we had the point and polygon without logic that could show the count inside each of these elements. So what we did, we used CQL view, that's a way that
GeoServer provided to configure a PostGIS data source. And with that, we could apply a statement written in CQL. And with this statement, we combine the two data sources.
So we have the event region, in this case creating a CQL statement, we get a precinct with a count for each crime on the map. Instead, for other type of visualization that we experimented, different CQL, same entry point data sources, we create hexagon with the PostGIS hexagon
green grid function. And then join and found which point and count them inside each of the hexagons. So we obtain sort of with map visualization through
hexagon, same thing with the square. So with the square here, we created a polygon geometry through a bounding box that we had. And basically intersect the point from the event table and obtain the count for each visualization.
So this completes the second step. So we add the data sources, we connect the data sources to GeoServer, we create a layer needed in the final visualization that could be provided as this one through WFS or also WMS. The experiment that we did was mainly focused on the WFS through JSON rendering client side.
Finally, we have the Map Store event tracker application. This application is not the usual Map Store, but we created a project through Map Store and extend it to contain a new plug-in, so custom plug-in, and use only
the framework front-end to be able to connect it to the API that GeoServer provides instead to the default and custom backend. And finally, we published also a custom bundle. This bundle is under the Map Store events tracker
package, and this package provides a script to create a template, a folder template. With all the configuration files, if you want to extend with a new dashboard the repository that we have created, I'll show you later also the link if you want to try the demo.
So these configurations that are provided by the Map Store events tracker allow us to connect to different tables and different layers to the GeoServer. In particular, the assets and the configuration file that
I want to highlight are the local config that is common with the product of Map Store, and it's used to configure the viewer. So decide which plug-in should be visible in the dashboard in this case. We call it viewer because in Map Store we have the map viewer, but in this case we have the dashboard.
So decide which component we should include or exclude in the final visualization. Then we have the map JSON, a configuration to change the map that is used as a background. And finally, the viewer config that is specific for the events tracker for this custom dashboard, that allows you to configure to a different set of
layers on the end point. Let's say that you want to add a new set of events and regions, you can create a new Map Store tracker folder with the client and connect the new configuration. This is the UI, the final UI that we get after this first
iteration. At the center we have the maps as a core component of Map Store that we provide some different visualization that could be selected by the legend slash visualizer selector, let's call it.
So in this sort of stop motion, we have the difference that we can select a different visualization. Then there is the possibility to query the properties on the layer. This is done client side through, we have JSON
client side so we can structure in the tool tip the result as we prefer without additional request to the server. That's the reason we choose for this experimentation the use of GeoJSON instead of WMS. On the right, we added this custom plugin that is a
counter plugin, show all the category from the data that we have from the crime. So we select, in this case, five categories. So we decide to extract all these five from the original crime data sources. And this card shows the total count for each of these
values and allows you to click on them to select and all the viewer, the dashboard, will synchronize based on the filter and select the categories. So as you can see in the charts below, there are now two lines that represent the two selected categories.
And type. At the top, we have a normalizer plugin where you can select if the data. Now we can see count that is the actual count. And we can select the different data.
We have our charts. And the main charts show months to crime. The filter to select the timeline range that we want to filter the data, and it synchronize all the dashboard. Finally, the table where all the region, in this case,
present, are selected. And you can select each of these region and get a detailed highlight of the content. So you can compare the total to the single region present. If you want to try this demo, you can download the
repo event tracker. There are documentation there, and there are some sample data from the New York Open Data Portal and the Denver Open Data Portal website. These are some improvement that we want to propose. One is included to extend probably next iteration
trade reviews and generally improve the performance of the client. And that's all. Thank you.