Developing a topographic data production system based on open source
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Title of Series | ||
Number of Parts | 351 | |
Author | ||
License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
Identifiers | 10.5446/69032 (DOI) | |
Publisher | ||
Release Date | ||
Language | ||
Production Year | 2022 |
Content Metadata
Subject Area | ||
Genre | ||
Abstract |
| |
Keywords |
FOSS4G Firenze 2022212 / 351
1
7
13
22
25
31
33
36
39
41
43
44
46
52
53
55
58
59
60
76
80
93
98
104
108
127
128
133
135
141
142
143
150
151
168
173
176
178
190
196
200
201
202
204
211
219
225
226
236
242
251
258
263
270
284
285
292
00:00
Open sourcePhysical systemPresentation of a groupContent (media)Computer architectureVideoconferencingDemo (music)Software developerClient (computing)CollaborationismInternationalization and localizationComputer programDecision theoryDatabaseService (economics)Plug-in (computing)Web serviceAreaData managementRevision controlProduct (business)Operator (mathematics)MetreTerm (mathematics)Link (knot theory)Ferry CorstenComputer programmingTexture mappingSoftware developerOperations support systemAreaVertex (graph theory)InformationProcess (computing)Physical systemDatabasePlug-in (computing)Decision theoryCartesian coordinate systemRevision controlGoodness of fitOcean currentData managementComponent-based software engineeringPresentation of a groupComputer architectureCollaborationismSquare numberBitOpen sourceSet (mathematics)Product (business)VideoconferencingService (economics)Demo (music)Cursor (computers)Web 2.0Confidence intervalCASE <Informatik>Video gameClient (computing)Projective planeContent (media)Cycle (graph theory)Object (grammar)Gene clusterWeb serviceMathematicsComputer animation
05:31
Revision controlComa BerenicesNetwork topologyLogic gateMereologyImage registrationProcess (computing)BitVideoconferencingAttribute grammarMusical ensembleRevision controlVertex (graph theory)AreaPersonal area networkPoint (geometry)Cursor (computers)Zoom lensUniform resource locatorTexture mappingMathematicsMultiplication signBuildingPerfect groupComputer animation
08:07
AreaRandelemente-MethodeSystems engineeringBulletin board systemPeer-to-peerOptical character recognitionBit error rateData managementRevision controlAdditionOperations support systemGeometryDatabaseWindowComputer-assisted translationOpen setMultiplication signObject (grammar)AreaProcess (computing)Different (Kate Ryan album)Resolvent formalismMoment (mathematics)MathematicsRight angleImage registrationBitComputer fileAttribute grammarComputer animation
11:21
Object (grammar)Operator (mathematics)Internationalization and localizationComputer-generated imageryRepresentation (politics)Texture mappingObject (grammar)Operator (mathematics)Demo (music)BitView (database)MereologyMappingSocial classRepresentation (politics)Computer-generated imageryComputer animation
11:53
Texture mappingBinary fileSineOperations support systemConnected spaceComputer programmingField (computer science)Attribute grammarView (database)CASE <Informatik>Table (information)Graph coloringBuildingRight angleObject (grammar)Zoom lens1 (number)Shared memoryComputer animation
13:09
Density of statesRootkitTexture mappingPhysical systemComponent-based software engineeringLatent heatPlug-in (computing)Traffic reportingCore dumpRun time (program lifecycle phase)MathematicsMultiplication signLink (knot theory)Selectivity (electronic)Core dumpSoftware testingSoftware bugConservation of energyAdditionSoftware developerPlug-in (computing)Component-based software engineeringComputer animation
14:31
Texture mappingOperator (mathematics)Internationalization and localizationPhysical systemPrincipal idealOpen sourceCoroutineTopologyDisintegrationSystem programmingProduct (business)Field (computer science)EstimationEstimatorSelf-organizationData managementComputer programmingSoftware developerProjective planeOpen sourceMappingPhysical systemField (computer science)Product (business)Texture mappingDifferent (Kate Ryan album)Graphical user interfacePhase transitionMultiplication signWave packetOperator (mathematics)Computer animation
Transcript: English(auto-generated)
00:00
is yours. Thank you. Can everybody hear me there? Yeah, good. So yeah, maybe now it's better. All right, yeah, my name is Ollie Rantanen and I come from National Land Survey of Finland. So nice to see so many of you. I know there's a big talk in the auditorium,
00:23
so nice to see so many of you here. So I'm here to tell you about our current project, which is to develop topographic data production system based on open source. So my presentation contents, I'll tell you a little bit about the background of the project, some facts
00:44
about the architecture of the system, show you some demo videos of the developed tools, so I hope those videos will work. Let's see. Then something about how have we contributed to the OS community so far, and our challenges faced during our development, and our next steps. So
01:07
for the background of the project, so NLS is the only institution in Finland which collects this comprehensive data set of whole of Finland's topography. And the decision to develop the collecting system based on open source technologies, and especially QGIS was made on the fall of
01:25
2020. The reasons for this was that the user and developer base are strong, which provides us confidence that the product lifecycle is also secure, and the programs would also receive updates from other developers as well. Then also that, of course, open source then allows us
01:43
enhanced collaboration with other governmental agencies and the OS community and whole. So about the architecture of the system, this is a really like general picture of it, but just to ease it for you, so we have like four key components there. We have two database clusters, application or
02:03
web services, and their client side. So that first database cluster there, it holds that primary database, which contains all that topographic data Finland from which all that map products are then made. We also have the version history there, and I'll show you the tool,
02:20
how we access the version history, and for example, if you modify objects or delete objects, you can acquire them later on. And then we have info related to the jobs, which means kind of in a way that we have these work databases, which use that primary database to, if you, as an operator, you select an area from the primary database, work
02:42
in there on your own work database, then push those modifications to the primary database back again. Then the application services, we have the job management there, so basically creation of the jobs and the modifications and some quality management there. Also, we have the VMTS and VFS services to display the features while the operator is
03:03
planning them up. Then also, we have the client, of course, which is operated on good GIS, and we have some pre-made NLS plugins there to ease the operator's job. So let's go for the first video, if this works. Yeah, so we first start the job manager plugin, so
03:24
that when you start it, you get the map of Finland, and you can then zoom in to a selected area you want. Basically, you see the topographic map of Finland there, and those red squares there are the other jobs. It has been a lazy day, so there are not too
03:42
many jobs there, but let's see. So basically, what you see here are the topographic data of Finland. So there's rock areas, cultivated lands, whatever. So over 80 different layers there. And I hope I soon push that I want to create the button. Let's see. Yes, so now
04:08
it's adding the layers. So basically, when you create the new job, then you get access to these layers, and you can start the modification in a bit. And I'm putting the cursor there
04:21
because while displaying the other people's jobs, you could also see those, but in this case, we didn't have those too many. But let's extend the modification area. So on this modification area, it's the current map canvas, which you have on there. And I zoom in a bit, and there you can see the actual data coming out. And I think I'll do just a
04:47
small, small modification here, but we use that button to make all the layers editable. Then I'll just show you some of the features. And using pick layer to check the right
05:00
layer there. Doing some modifications for vertexes. Yay. And just saving all those edits. So it saves it for all the layers. And then going ahead and registering these changes. So now I have the work database there modified, and I update the main version
05:23
based on that. Yay. Successful. And then about the version history. So this time, I'm acquiring an old job. So basically, it happens in a way that you just check the ID
05:41
you want to access off the job. And it's doing the same thing, basically, then creation the old new job too. But you get the layers there. And when we zoom in, we can already put all the layers in edit mode. Then trying to pick two buildings, but failing first.
06:05
Yes. Now I have two selected. And I can delete those. Then picking new group, the rock area, and modifying again the vertexes. And then just changing this point, rock points
06:27
location a bit. Then it's the same workflow as before. So we save these edits and register them. All right. And then we get to use the history tool. So it looks not that
06:47
beautiful yet, but we are making these changes later on then to make it more beautiful. But as you can see here, you can also see the registrations of the areas when you have done those. Basically, I can use the cursor now to pick up the versions. So now I have
07:04
the two versions there. So I have the previous version and the version I modified a bit. And I can also check how those versions differentiate with each other. I could also see if I have changed the attribute values there. Now I didn't change them, so
07:25
not anything to see now. And I can also access the points and kind of upwardly zoom in to this one point. But yeah, they are there. And so then we have the other feature there while I pan the map, which is showing that deleted feature. So it kind of access
07:46
the whole map extend again to show these features. And you can also change the year you're going to change, fetch these deleted features. And yay, they appeared there.
08:02
Perfect. All right. I have two videos left. Bear with me. So this is conflict management. So basically there is a job open on the right-hand side and on the left-hand side the same area is being modified or this cute little cat feature here. So we are picking
08:25
that up and this awful operator wants to cut his ear off. He does this. Oh, man. Saves these edits and registers them. Then we go to the other person's job in a second,
08:50
apparently. And we can see already that the registration has been made here, but let's turn it off because it's confusing now. But already we can, well, this guy can access
09:06
the same file, but he hasn't refreshed the job so that updates haven't been received. So this guy wants to make the year a bit bigger. Yes. And he goes ahead again and saves those edits. And what happens now? Oh, the job needs to be refreshed. So this works in
09:30
a way that when you have made these two modifications, the same every hour, same feature, you get this conflict management. It says that you need to refresh it and then resolve the conflicts.
09:42
So you get the conflict management window then. I'll turn these off. All right. And then we can see that in the conflict management window when you click the object which has the conflicts, it zooms into it and then you can see how the feature difference from
10:03
work primary and sorry, work database, primary database and how these features share the same area. So that's the common base. So the work database is this guy's database at the moment, the primary and the right-hand side guy's database. And in a second,
10:27
I will click it and then I can check the geometry differences. I could also check the attribute value differences here. And then when I have this displayed, I can select the geometry then either accept or reject my own additions, how they differ from the primary
10:42
as you know. And the same thing for the other guy. But I'm the good guy. I don't want the cat to lose the year. So I rejected his modification. All right. And that's how that conflict was basically solved. But we still need to refresh the job once more. But this stupid operator didn't do that. He wants to register it all the time. But yeah,
11:05
let's refresh it and then we get zero conflicts again. All right. So it works in a way that that operator who made the last change has to do that conflict resolvent. All right.
11:24
And then for the last part, a little bit about stereo mapping. So that's the usual way for the counter-preferance of operators of NLS to collect their topographic data from the aerial images. So basically, it does, you know, accurate representation of objects
11:41
in 3D stereo view and allows more precise mapping with topography. You would need stereo distro and stereo glasses to actually see the objects in 3D. But in this demo, well, you won't see it, but you will see something that it looks like. But we have this ESPA city program working with cool GIS so they communicate with each other. Now I'm
12:04
connecting to the ESPA city. So these share the connection right now. So I go ahead and click on the stereo view. It zooms in. You see that it's kind of blurry because of the
12:21
stereo view. And those, you know, objects there are the ones you have selected from cool GIS. So now they are buildings and those are displayed in this purple color. So I'm making a new building there. I'm not the best operator, as you can see. It's a field, not a building, but it doesn't matter. I get the attribute table on the cool GIS
12:45
view. All right. And I can add that. And it was displayed there on the right-hand side. I can also do that in cool GIS. I can choose a feature. In this case, it will be a pool.
13:02
And these are also communicated to the ESPA. So the operator can work on both of these programs at the same time and do these changes. All right. So for our contributions so far, so we have developed the system for about one and a half years now. And a lot of the
13:23
components have been specific for our needs, so to speak. Keeping that in mind, we are aiming to publish those tools we can find useful for others as well. But since they have been really specific, we need to generalize them a lot and it would require some additional time. Of course, we have made some contributions. This could be found on GitHub,
13:50
but now the links are not displayed. But maybe I can tell those later on. But yeah, we have contributed to the PyTest cool GIS, which is actually made by Gispo,
14:01
but we have also contributed that, which is made for easier testing of plugins. Then we have the cool GIS plugin dev tools. That's for plugin development and packaging. It makes managing runtime third-party dependencies more easy. And also we have made contributions for the pick layer plugin, which you saw me use a couple of times there,
14:21
and it's used for the smoother selecting of layers. Also, we have been reporting, even fixing some bugs for the cool GIS core. Yep. Oh no. So some challenges we have faced during our development. So as you know, like the operators of NLS have used a different
14:42
mapping system for decades and learning to use cool GIS and other new mapping technologies requires some extensive training and also making sure that the routine work is done so that the maps are made and stuff like that. And of course, adapting to these new ways of working with different technology needs time. Like for example, the topological editing is
15:03
really important, but it's quite different from cool GIS to the previous program and how it's handled. And of course, from the governmental side, governmental organizations policies to the open source principles or adapting those. For example, in Yannick Ilmah's talk, he told you
15:21
about how our policies have changed now for the opening up or going for the open source. So we have just received a couple of months ago that like the policies on how can we publish our projects code. And then for the next steps for our program, we are trying for the quality
15:46
management. So basically, cool GIS has a lot of these quality management tools, but we have some specific needs which don't comply with maybe with some of those. So we need to do our own
16:01
quality management system there. Then adding the elevation to the features. So that's one thing. Managing the geographic names, then integrating the system to the other systems that produce the geographic products, then getting a field mapping solution. And just like a general
16:22
timeline for our deployment is that our current estimate suggests that we could do it the first phase productional system deployment in about two years time. Yes, thank you.