Developing with MapStore; creating a custom dashboard to map crime data
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 351 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/69166 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Production Year | 2022 |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
FOSS4G Firenze 202278 / 351
1
7
13
22
25
31
33
36
39
41
43
44
46
52
53
55
58
59
60
76
80
93
98
104
108
127
128
133
135
141
142
143
150
151
168
173
176
178
190
196
200
201
202
204
211
219
225
226
236
242
251
258
263
270
284
285
292
00:00
Presentation of a groupUniform resource locatorCASE <Informatik>Data storage deviceLevel (video gaming)XML
00:16
Source codeSoftwareOperations support systemPerturbation theoryData storage deviceLevel (video gaming)Standard deviationClient (computing)Web 2.0MereologyProduct (business)CASE <Informatik>Front and back endsShared memoryServer (computing)DebuggerMappingComputer animation
00:43
SoftwareSource codeWebsiteVisualization (computer graphics)Category of beingLatent heatProcess (computing)CASE <Informatik>Uniform resource locatorCartesian coordinate systemLevel (video gaming)GeometryElectronic visual displayType theorySet (mathematics)AdditionInformationCategory of beingPoint (geometry)Table (information)Traffic reportingResultantComputer animationSource code
01:36
View (database)Software testingPointer (computer programming)IntelInterior (topology)Inclusion mapElectronic data interchangeTotal S.A.WebsiteEuclidean vectorSource codeSoftwareOperations support systemSingle-precision floating-point formatWeb pageLevel (video gaming)Service (economics)Data storage devicePlug-in (computing)View (database)Server (computing)Connectivity (graph theory)Revision controlSampling (statistics)MereologyCASE <Informatik>Cartesian coordinate systemProcess (computing)DatabaseSoftware frameworkTable (information)Source codeRight angleOpen setFinite differenceDebuggerType theoryComputer architecturePhysical systemLibrary (computing)Computer animationProgram flowchart
05:04
Point (geometry)Physical systemResultantComputer animation
05:18
Source codeSoftwarePolygonType theoryInformationPolygonMorley's categoricity theoremPoint (geometry)Event horizonVisualization (computer graphics)Type theoryCategory of beingMultiplication signSampling (statistics)Different (Kate Ryan album)GeometryInformationTable (information)Source codeSource code
06:12
SoftwareSource codeOpen setBeta functionComputer-assisted translationData conversionEvent horizonAbelian categoryGeometryTimestampDivision (mathematics)PolygonFloating pointServer (computing)Open setLatent heatTable (information)Event horizonCategory of beingComputer fileCASE <Informatik>Data conversionNormal (geometry)Shape (magazine)Point (geometry)Field (computer science)Connected spaceInformationFile formatData structureScripting languageRange (statistics)Multiplication signSource codeSource codeComputer animation
07:52
Data Encryption StandardEvent horizonExtension (kinesiology)Escape characterSource codeSoftwareView (database)Type theoryLocal GroupSuite (music)CountingASCIISyntaxbaumClient (computing)Data storage deviceDifferent (Kate Ryan album)Visualization (computer graphics)Source codeType theorySquare numberPolygonStatement (computer science)QuicksortCountingCASE <Informatik>Server (computing)Event horizonFunctional (mathematics)Point (geometry)Level (video gaming)Green's functionCuboidElement (mathematics)HexagonGeometryView (database)LogicTable (information)Computer animationSource code
10:04
Plug-in (computing)Fiber bundleData storage deviceLevel (video gaming)Cartesian coordinate systemEvent horizonVisualization (computer graphics)Different (Kate Ryan album)Connectivity (graph theory)Configuration spaceSoftware frameworkScripting languageProduct (business)Projective planeSet (mathematics)Fiber bundleTable (information)CASE <Informatik>Front and back endsDemo (music)Template (C++)Default (computer science)Computer fileRepository (publishing)File viewerLink (knot theory)Plug-in (computing)Server (computing)DebuggerComputer animationSource code
12:14
Component-based software engineeringTotal S.A.Total S.A.Normal (geometry)QuicksortPlug-in (computing)PlastikkarteCore dumpCountingMappingCategory of beingClient (computing)Connectivity (graph theory)CASE <Informatik>Different (Kate Ryan album)ResultantFile viewerIterationType theoryConfiguration spaceContent (media)Range (statistics)Data storage deviceRight angleTable (information)Server (computing)Line (geometry)Source codeVisualization (computer graphics)SynchronizationPresentation of a groupSingle-precision floating-point formatLevel (video gaming)Computer animation
14:56
Source codeSoftwareOpen setSampling (statistics)Software repositoryDemo (music)Event horizonWebsiteSource codeComputer animation
15:11
Range (statistics)Extension (kinesiology)View (database)IterationClient (computing)Computer animation
Transcript: English(auto-generated)
00:00
Thank you. Today's presentation is about developing with Map Store. So I will show you a use case where we were able to create a custom dashboard using data related to crime location. To do that, we use Map Store. And what's Map Store? Map Store is one of our products.
00:21
And it's a web.js client that allows the user to create and share maps, dashboards, or just stories. But for this use case, we extract the front-end part of Map Store and use that to interact with a geo-server post-js backend. And using also OGC standards inside this use case.
00:44
Let's start from the design process. So how we start and think, and how we work on the specific use case. We added some initial requirements. That was the crime location data that was composed for
01:02
the geometry point, the type, so the type of crime that we needed to filter then later with application with, and the report date. So this one was our initial data set that we had. And as a result, as a requirement, we needed to
01:21
visualize this information inside a map and display also additional data around with charts and table, and also be able to filter this data based on the property that we have inside the crime location data. These two screenshots show our initial work.
01:42
So it was a simple map and a simple chart done in JavaScript. We use a simpler version of Map Store in this case. Behind there is open layers. And we took the sample data that we had and put inside post-js, postgres servers, and started to insert these
02:02
through Azure server to play around and understand what kind of OGC service we could use with the front end of Map Store. In this case, we use WFS and SQL view to render the data on the map. And later, we'll show you the SQL view and what they are
02:22
to server side. We use the WFS to process the data and create charts, counters, and tables. And finally, we use also WMS to have a background map with data that we had from OpenStreetMap data source.
02:45
So this first part shows how we start to think and work with the back end side. So we decide to use a post-js, postgres server database, GeoServer to provide the processed data. And then we start to work with the Map Store framework.
03:01
So we took Map Store, extracted the front end with the React part. So we use the base component that provides open layers, leaflet, and cesium types, so a library behind. And the possibility to configure additional plugins to the application.
03:25
I don't know if there is sound. OK. The screenshot that you see here is one of the previous experiments that we did with the Map Store COVID map where
03:40
we use Map Store to interact with custom API that were provided and showing the data from the COVID. And with that, we were able to use the plugin infrastructure architecture to add the custom plugin such as the counter, the table, and so on.
04:03
So one of our steps is to start through sketch or mock-ups. So we started to draw around how the final UI should be composed, so the graphic composition, and also start to base our previous experience to enhance with new plugins the UI in particular.
04:23
The filter, now here is a sketch, so something is a mess there, but on the top right, we started to think on the interval date filter, and on the top left, the table, how to filter data. This one was an initial step that we used
04:40
to create our UI. Finally, when we added all the three pieces, the database with PostGIS and Postgres, GeoServer, and SpatialPodRider, and Map Store, we put them together inside a Docker container, so we were able to
05:00
share these in different operation systems. Next step, so when we finally added an idea on our infrastructure works, it could work, because at that point, we did not know the final result, but we had an idea on how to structure this. We start to prepare the data. As I mentioned before, it's a crime data, so it was
05:25
data related to crimes, and to generalize and normalize the table that contains information, we call that event, so that could be applied with a different type of data source. The important value that we need for the event are the
05:44
categorization property, a date, so a time, and a point geometry. For the crime, we also add a region, because the crime has usually, for the sample data that we use, administrative region to work with, so an idea where
06:00
we can associate this region. So the polygon that represent the region could be used to create the final visualization. So events region was our normalized table. Here, there is an example on how we use the New York Open Data Portal. We found the data related to the complaint, where there
06:22
are some fields that fit our use case. We pre-filtered the data to make sure that all the categories that we needed were there, and also to choose a specific time range. And finally, we export as CSV.
06:41
The same we did for the region. In this case, we look for precinct. And for the precinct, instead, we download a format that could contain spatial information, so a GeoJSON or a shapefile that could be then imported on a Postgres, PostGIS table.
07:02
Once we had downloaded all the data, we needed to add additional step that was the let's call normalization. So I'll ask a script that through the Docker entry point, allowed us to read the data and convert in a normalized structure.
07:20
These are the two structures, one for the event table. So the event table for our normalized structure should have these fields. And finally, the region table. These two tables are important because later we
07:43
were able to connect to a GeoServer through the data sources that GeoServer allowed. One of these is the PostGIS connection. And we were able to configure the layer to provide the layer to the client. So the data store that we had was event and region, but
08:04
wasn't enough for the visualization. Because we had the point and polygon without logic that could show the count inside each of these elements. So what we did, we used CQL view, that's a way that
08:21
GeoServer provided to configure a PostGIS data source. And with that, we could apply a statement written in CQL. And with this statement, we combine the two data sources.
08:40
So we have the event region, in this case creating a CQL statement, we get a precinct with a count for each crime on the map. Instead, for other type of visualization that we experimented, different CQL, same entry point data sources, we create hexagon with the PostGIS hexagon
09:04
green grid function. And then join and found which point and count them inside each of the hexagons. So we obtain sort of with map visualization through
09:21
hexagon, same thing with the square. So with the square here, we created a polygon geometry through a bounding box that we had. And basically intersect the point from the event table and obtain the count for each visualization.
09:40
So this completes the second step. So we add the data sources, we connect the data sources to GeoServer, we create a layer needed in the final visualization that could be provided as this one through WFS or also WMS. The experiment that we did was mainly focused on the WFS through JSON rendering client side.
10:05
Finally, we have the Map Store event tracker application. This application is not the usual Map Store, but we created a project through Map Store and extend it to contain a new plug-in, so custom plug-in, and use only
10:23
the framework front-end to be able to connect it to the API that GeoServer provides instead to the default and custom backend. And finally, we published also a custom bundle. This bundle is under the Map Store events tracker
10:42
package, and this package provides a script to create a template, a folder template. With all the configuration files, if you want to extend with a new dashboard the repository that we have created, I'll show you later also the link if you want to try the demo.
11:03
So these configurations that are provided by the Map Store events tracker allow us to connect to different tables and different layers to the GeoServer. In particular, the assets and the configuration file that
11:22
I want to highlight are the local config that is common with the product of Map Store, and it's used to configure the viewer. So decide which plug-in should be visible in the dashboard in this case. We call it viewer because in Map Store we have the map viewer, but in this case we have the dashboard.
11:42
So decide which component we should include or exclude in the final visualization. Then we have the map JSON, a configuration to change the map that is used as a background. And finally, the viewer config that is specific for the events tracker for this custom dashboard, that allows you to configure to a different set of
12:02
layers on the end point. Let's say that you want to add a new set of events and regions, you can create a new Map Store tracker folder with the client and connect the new configuration. This is the UI, the final UI that we get after this first
12:21
iteration. At the center we have the maps as a core component of Map Store that we provide some different visualization that could be selected by the legend slash visualizer selector, let's call it.
12:42
So in this sort of stop motion, we have the difference that we can select a different visualization. Then there is the possibility to query the properties on the layer. This is done client side through, we have JSON
13:02
client side so we can structure in the tool tip the result as we prefer without additional request to the server. That's the reason we choose for this experimentation the use of GeoJSON instead of WMS. On the right, we added this custom plugin that is a
13:23
counter plugin, show all the category from the data that we have from the crime. So we select, in this case, five categories. So we decide to extract all these five from the original crime data sources. And this card shows the total count for each of these
13:44
values and allows you to click on them to select and all the viewer, the dashboard, will synchronize based on the filter and select the categories. So as you can see in the charts below, there are now two lines that represent the two selected categories.
14:04
And type. At the top, we have a normalizer plugin where you can select if the data. Now we can see count that is the actual count. And we can select the different data.
14:21
We have our charts. And the main charts show months to crime. The filter to select the timeline range that we want to filter the data, and it synchronize all the dashboard. Finally, the table where all the region, in this case,
14:43
present, are selected. And you can select each of these region and get a detailed highlight of the content. So you can compare the total to the single region present. If you want to try this demo, you can download the
15:00
repo event tracker. There are documentation there, and there are some sample data from the New York Open Data Portal and the Denver Open Data Portal website. These are some improvement that we want to propose. One is included to extend probably next iteration
15:20
trade reviews and generally improve the performance of the client. And that's all. Thank you.