Block grid system usage at the example of Estonian Postal Services
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 295 | |
| Author | ||
| Contributors | ||
| License | CC Attribution 3.0 Germany: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/43510 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
FOSS4G Bucharest 201974 / 295
15
20
28
32
37
38
39
40
41
42
43
44
46
48
52
54
57
69
72
75
83
85
87
88
101
103
105
106
108
111
114
119
122
123
126
129
130
131
132
137
139
140
141
142
143
144
147
148
149
155
157
159
163
166
170
171
179
189
191
192
193
194
195
196
197
202
207
212
213
214
215
216
231
235
251
252
263
287
00:00
Physical systemService (economics)Exception handlingQuicksortLevel (video gaming)Computer animation
00:17
Software developerMappingException handlingNetwork topologyProgram slicingAreaLogistic distributionBitState of matterLecture/Conference
00:44
Image resolutionExpert systemGroup actionRadical (chemistry)Entire functionWater vaporComputer animation
01:16
Client (computing)Virtual machineClient (computing)ChainCASE <Informatik>AreaLogistic distributionRoutingPlanningCartesian coordinate systemError messageTask (computing)LogicMachine visionDisk read-and-write headMoving averageMereologySpacetimeComputer animationLecture/Conference
02:02
TouchscreenLogicPhysical systemClient (computing)Computer clusterDistanceMusical ensembleWhiteboardBlogDressing (medical)Interface (computing)Row (database)Service-oriented architectureCartesian coordinate systemMultiplication signUsabilityWave packetLogicSpacetimeProgram slicingComputerValidity (statistics)Context awarenessBitPolygonFunctional (mathematics)Computer animation
02:58
Software development kitProgram slicingException handlingFunctional (mathematics)CASE <Informatik>Error messageCovering spacePhysical systemNetwork topologySoftware testingPolygonAreaBitContext awarenessLogicCategory of beingProjective planeData storage deviceComputer animation
04:25
Maß <Mathematik>Computer networkShape (magazine)Line (geometry)Execution unitDifferent (Kate Ryan album)Error messageRepresentation (politics)ResultantSoftwarePoint (geometry)Special unitary groupWhiteboardWebsiteState of matterWater vaporSource codeComputer animation
05:05
State of matterCovering spaceSurfaceWater vaporError message
05:32
1 (number)MetreInterface (computing)State of matterSpacetimeFeasibility studyUser interfaceComputer animationEngineering drawing
05:53
Line (geometry)DatabaseShape (magazine)Different (Kate Ryan album)Representation (politics)Basis <Mathematik>Computer animation
06:35
DatabaseGeometryDatabaseResultantBasis <Mathematik>QuicksortTape driveSquare numberFreewareShape (magazine)Uniform resource locatorAreaRevision controlComputer animation
07:18
Library (computing)InformationShape (magazine)Finitary relationAddress spaceCodeTime zoneVirtual machinePersonal digital assistantInheritance (object-oriented programming)DatabaseCartesian coordinate systemShape (magazine)IdentifiabilityTheory of relativityArray data structureUniform resource locatorInformationRelation <Mathematik>Inheritance (object-oriented programming)Process (computing)RoutingCASE <Informatik>Different (Kate Ryan album)Virtual machineError messagePlotterCoordinate systemFile archiverDistribution (mathematics)Partial derivativePhysical systemScaling (geometry)1 (number)Limit (category theory)CuboidExecution unitSpacetimeDemosceneRow (database)Line (geometry)BitComputer animation
10:06
Inheritance (object-oriented programming)Square numberLevel (video gaming)CuboidDatabaseComputer animation
10:28
Inheritance (object-oriented programming)User interfaceCartesian coordinate systemSpacetimeSocial classParameter (computer programming)CASE <Informatik>Array data structureComputer animation
10:47
Parameter (computer programming)Computer animation
11:08
User interfaceStack (abstract data type)Vector spaceOpen setBootingSpring (hydrology)DatabasePreprocessorServer (computing)User interfacePhysical systemBootingChaos theorySpring (hydrology)Relation <Mathematik>Open setCodeVector spaceQuery languageDialectFile formatCodeMessage passingMusical ensembleOcean currentWeb serviceData managementFront and back endsWeb applicationCASE <Informatik>Instance (computer science)Image resolutionComputer architectureArray data structureStatement (computer science)Source codeInsertion lossForm (programming)Process (computing)Order (biology)QuicksortFerry CorstenNormal (geometry)Text editorDatabaseCartesian coordinate systemComputer animation
13:51
Stack (abstract data type)PreprocessorFitness functionOrder (biology)Digital photographyQuicksortSoftwareLink (knot theory)Reverse engineeringMultiplication signPoint (geometry)Set (mathematics)Different (Kate Ryan album)Process (computing)Web serviceSpur <Mathematik>Theory of relativityDuality (mathematics)Line (geometry)Doubling the cubeMereologyMathematicsLevel (video gaming)Linked dataVideo gameSource codeVirtual machineCASE <Informatik>Flow separationData storage deviceVector spaceTesselationClient (computing)Right angleSoftware developerProjective planeAdditionGoodness of fitAreaMultilaterationElectronic data processingPartial derivativeInformation overloadComputer animationLecture/Conference
Transcript: English(auto-generated)
00:07
So hello everybody, my name is Kim Olop and I'm from Estonia and I'm going to show you how our team created map editing solution for Omnia employees to build the geographic areas which are perfectly editable without creating whole slices or topology exceptions.
00:26
So I want to ask you raise your hands who are involved in mapping editing solutions, developers or anything else? Okay Little bit about Omniva. Omniva is an international post and logistics company based in Tallinn, Estonia
00:47
with the entire Baltic States as its domestic market. Omniva group is five companies from the Baltics, has employees around
01:01
2,300 and is the leader of parcel terminal market in Baltic. I myself am not an Omniva employee but an outsourced GIS expert to solve Omniva key-informatic problems and to build solutions. So client, client wants to build geographical areas that they can constantly change.
01:24
In case of our client Omniva, they need to constant changing because they need to daily logistic planning which include managing addresses, adjusting parcel machine errors, when overloading soccer, planning quarter routes and so on. So when we have logistics who have business logic in their head and
01:44
when they have great vision how to solve some geospatial tasks like organizing parcel machine errors or managing quarter routes are rated, then creating application where we can't rely on key-informatic and acknowledge is a must.
02:08
Wait a second. I'm a little bit stuck here. There is also a way to teach them to use already existing geospatial editing tools like cookies, but then it's a training expense and mostly those people are not so excited about it.
02:27
That's mostly because of learning context time and they don't trust themselves when it comes to critical use of data. So we started to build application with business logic where all the validations are made inside the workflow and
02:43
to build user-friendly interface where we gave user functionality that won't confuse them and we all know how it's typically going. Most basic way to let user to create data is to enable polygon drawing tool and then adding all business logic behind it like how does overlapping are
03:02
accepted or what region is allowed to edit at all. And after enabling drawing tool user would also like to have slicing tool to cut existing errors to pieces and that's not all. User would also like to have merge tool to union errors back together.
03:20
And also if there is a regard to have fully covered errors by region use function that validates created shapes and so the user aware of the holes. With this editing solution user quickly runs into cases where slices come up, little holes overlapping so where it's not accepted and diverse topology exceptions.
03:44
Which means that we still need person who have given informatic knowledge to clean up the mess. And this is not the way we are going to solve problems. There are some examples I have made up here. There are a little bit slices there in the purple area and also this
04:05
polygon topology misbehaviors. So we started to show solution how basic user can create errors without going into tests of go informatic systems but can create errors that
04:21
will cover all business logic. Regardless and are easy to modify and user can actually understand how geographic errors are made without creating geographical mess of data. We went to create blocks. Blocks are geographical representations of pre-generated errors which we made out of using different side layers like
04:44
settlement unit borders, road, network railways, river center lines and cadastrial parcels were applicable. Using those layers we put them all together and sliced with each other.
05:00
And the result is approximately 2.5 million shapes that covers Baltic states, land and water errors. In the cities we have usually small surface blocks and on the countryside or on the waters we have bigger surface blocks. But all Baltic states are now covered side by side from border to border and this is the example of
05:23
Baltic states and it's rotated. On the right side is Estonia and on the left side is Lithuania. Smaller blocks are mostly about half square meters and biggest ones are about 1,500 hectares, kilometers and that's mostly in the sea.
05:45
From now on we created a web user interface where we have only those blocks visible. But user can add different classifications to the block which defines how user wants to draw a wire. This means that user never modifies spatial shape itself, but instead selects multiple blocks and sends the database.
06:04
And here you can see purple lines to represent block borders. Red line is drawed by user with drawing tool by freehand using snapping tool to snap with the road layer.
06:21
And the blue now line now represents how aggregate the blocks are graded. There is not much difference. Here you can see example when layer snapping is not used or where not accessible which means that every movement must be made carefully and this is a lot of digitizing on daily basis.
06:41
Also, it's hard to validate the result. Now in the database we collect all block IDs, request all those block shapes and union them all together and because of block shapes are topologically correct, then the union of them is error free and perfectly merged together.
07:01
This is how merged array will look that have been graded out of blocks and this is how blocks in the diverse array look alike. For example, covering 5,000 square kilometers of land array takes about 50,000 blocks, but it depends on location. In the cities it would be much more and countryside of course less.
07:28
Block holds very basic information. It has identifier that will be used to create arrays and it has its own shape that we hold in database in World Geodetic System 84.
07:44
We are also transforming block shapes into Estonian national coordinate system on the application side because of use of the Linberg Geodetic System. When using approach where user is digitizing arrays and slicing and uniting them,
08:02
there would be more problem when transforming coordinates because of coordinate plot error will create topological misbehaviors. Block has relations with classifications that can be made out of anything. This gives us very much information where it belongs to and this information can be requested without making spatial requests.
08:26
Some examples that we are currently using addresses, zip codes, servicing arrays. For example, when there is one block has been selected to multiple courier arrays, each weekday might be serviced by different courier routes.
08:42
If we want to know when this one block is serviced by courier today, we can easily make requests without spatial processing. There is also a price tag for this block and the relation with partial machine errors.
09:03
If there is those inside block, then we can request how many mailboxes are in the block. Relations can be created to unlimited use cases. It is just a matter of defining classifications and this is a great opportunity with block's approach that we can request all the information that one block holds.
09:23
For example, this is useful for providing speed performance for the distribution center where packages are managed to hand out over the courier. Blocks can be sliced. This means that if there is a host built into block
09:42
and this needs to be sliced, we just archive the old block and create new ones and update all the relations. Super blocks. Super blocks are a collection of blocks which have been pre-generated to be shown on upper scales where rendering each block will affect performance.
10:05
Super blocks can be created out of any collection. We did build them using settlements, but there is no reason to just create them out of square kilometers of grid. User can now select super blocks and add some individual blocks.
10:22
And in the database, we aggregate super block shapes with individual block shapes. This will give faster performance when visualizing on map and also when aggregating in database. Let's go to the application we built. There is some animation.
10:43
So, blocks benefits are that we can easily show from application how the arrays are made. Where are the overlaps, which means that one block has more than one same classification parameter, or if there is no classification parameter specified, then this block is unused and this means we don't have full coverage for these classifications.
11:07
Let me give you some awesome examples with this approach. For example, it is easy to restrict blocks that can be used when drawing, and user will only have some specific reason to modify.
11:22
Another good example is that we have defined zip code arrays into blocks. In application, we can let user to use only zip codes to build up new layer from it. When zip code arrays changes, then all the relatives will change along. Another example, when user creates overlapping, then it is easy to show where it happened.
11:44
And now user can decide to remove this overlapping from editing array, or to remove it from the array that were below editing array. This is the red array he is showing the overlapping, and it can be removed from the below or from the editing array,
12:02
and there is no overlapping after all. And there is much more ways to use block approach. It is just a matter of imagination. About the technology we used. To solve this approach, we used a chaos server to serve vector dials in Mapbox vector dial format
12:24
for rendering blocks in open layers. On the open layers, when user creates an array, the vector dial feature is styled on the fly. When user selects or deselects blocks, there is no regards to the backend in these cases.
12:41
But we do use the feature service currently. If user wants to select blocks with the rubber band tool, that's because vector dials in open layers didn't have the proper way to filter them out. Web application itself is built with React and Redox State Manager,
13:01
which provides fast performance on rendering user interface. So on the backend, we use the Spring Boot microservices architecture, along with RapidMQ message exchange system.
13:20
And for the database, we had to use SQL Server because of only architecture requirements. And on the background, we use more technologies that will help us to solve data management or build up services. Usually, all the data preparing is made up with Postgres, SQL, and Postgres instances
13:45
because of simplicity of writing queries that SQL Server can't handle, or it requires too much writing. And actually, I'm finished now. It went much faster than I tried at home.
14:07
So we have more time to ask questions about this.
14:22
You used roads and rivers and everything to cut the blocks. How did you get centerlines for the roads? Or did you use double lines if it was a dual carriageway?
14:42
Mostly, double lines are centered between them. There is Dennis here, who is my colleague, who is making this data processing. I think you can ask him later about this. I myself am more like a front-end developer or this architecture.
15:13
Really nice what you did with all these blocks. And it looks to me that that would also be a great opportunity to put in some linked data.
15:21
Did you consider that, or did you think about that? Linked data between making relations with them, or what? What do we mean by linked data? Well, with linked data you can keep on adding links or relations to separate blocks. In your case, to combine with external data sources, or even your internal ones.
15:47
So it comes to me as a great opportunity to put in linked data as well. Right. I'm not aware of this. Thank you. Thank you for listening. So it looks to me that the user is kind of adding links to the blocks.
16:07
Is that what the users actually do, or what does the user do with the software? With blocks. With the software? With the software. User is creating areas, like currently it's probably a machine area, what it's doing.
16:23
But there is also those corridor roads, what is measured, how corridor is traveling. And also when partial machine errors, if there is overload in partial machine, then they shrink this area to smaller. And the hole, which is left over, it's been used to redirect to other partial machine error.
16:51
Sorry, I think there was a question in the back here. Hi, a question about just the process of creating the blocks. How clean, or how much did you prioritize how clean that dataset was?
17:05
Like, if you had these different datasets and they also didn't overlap precisely, and made little slivers and left artifacts in those blocks, did you care about that? Did you try and clean that up? Or was that not that important?
17:21
We used different datasets, also OpenStreetMap data, and also in Estonia we used Landboard data. And it's quite clean. And it can be more...
17:41
I lost my words. Let's ask later. Let's talk about later. So did you also make some tools for the customers to find where the miss selections,
18:00
like if the client makes the area and there's one parcel or one block that's missed, like it's in wrong zone, did you make any tools for finding that one small block that should not be there? It's quite hard, actually, to detect what's not to be selected into which block is not well configured.
18:29
We can show where are the leftovers, like which block is not selected, and also where are the overlappings. But for defining where this block is not correctly selected, it's quite hard.
18:53
As I understand, you use GeoServer for background services and also generating vector tiles.
19:04
Did you encounter any performance issues with vector tiles generated by GeoServer? We're running about 2.5 million blocks and they're actually running quite fastly.
19:22
There are no performance issues, more issues about rendering on map? Well, yeah, because they are related. Somehow. Yeah, because in my experience it's hard to fine-tune vector tiles generated by GeoServer.
19:45
They became too big to render fastly. But if you didn't have any performance issues, it's good. That's why also we created super blocks to cover those areas where vector tiles blocks are going too small,
20:06
then they are not rendered at all. And if we are styling this on the map, it's really easy to see where the holes are making. But when you're zooming in, then you can see that these blocks are already selected.
20:22
But super blocks will cover it. Any additional questions? I'm just thinking, how many blocks are there? 2.5 million blocks in project space.
20:48
Alright, well thank you very much.