More correct maps/data with Postgis Topology rather then Simple Feature
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 266 | |
| Author | ||
| License | CC Attribution 3.0 Germany: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/66487 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSS4G Prizren Kosovo 2023114 / 266
10
17
23
44
45
46
47
48
49
50
52
53
80
84
85
91
110
116
129
148
164
167
169
173
174
181
182
183
186
187
199
202
204
206
209
215
241
248
265
00:00
Gamma functionDifferent (Kate Ryan album)Overlay-NetzTable (information)Attribute grammarOperations researchAnalogyVector spacePoint (geometry)Physical systemTexture mappingTopologyDatabaseMusical ensembleApproximationReal numberMappingTable (information)AnalogyException handlingSoftware testingShared memoryCASE <Informatik>Overlay-NetzPower (physics)Computer animation
02:29
Drill commandsInformation systemsConnected spaceWindowComputer animation
03:26
System programmingInformationTopologyIntegerGeometryExistenceSerial portAreaTable (information)TopologySurfaceMereologyDatabasePhysical systemInstance (computer science)BitException handlingEndliche ModelltheorieDifferent (Kate Ryan album)Object (grammar)Data structureTexture mappingRow (database)Game controllerInformation systemsLine (geometry)Gastropod shellInheritance (object-oriented programming)InformationNetwork topologyComputer animationXML
07:24
TopologySerial portIntegerGeometryExistenceView (database)outputTable (information)Structural loadLine (geometry)SurfaceOverlay-NetzData structureAttribute grammarParameter (computer programming)Group actionAreaCovering spaceLine (geometry)Table (information)CASE <Informatik>SurfaceDifferent (Kate Ryan album)DatabaseGeometryPoint (geometry)Functional (mathematics)Speech synthesisTopologyOverlay-NetzInstance (computer science)Resolvent formalismSoftware testingCodeBitSelectivity (electronic)View (database)Topologischer RaumStructural loadException handlingTopostheorieMultiplication signPolygonRight anglePhysical systemGame controlleroutputObject (grammar)Online helpConnected spaceAreaTracing (software)Quicksort5 (number)TupleSet (mathematics)Inheritance (object-oriented programming)Smith chartComputer animationDiagram
15:40
Covering spaceAreaException handlingDifferent (Kate Ryan album)Software bugError messageTopologyMultiplication signAreaComputer animation
16:21
Source codeRevision controlCombinational logicAttribute grammarSurfacePolygonView (database)Computer animation
17:02
outputTopologyIntegerGeometryData bufferBoolean algebraFloating pointAreaBounded variationLatin squareTable (information)CASE <Informatik>Total S.A.PolygonResultantoutputComputer animation
17:29
TopologyPolygonDatabase normalizationDatabaseUniqueness quantificationVertex (graph theory)Product (business)SurfaceFinitary relationTime domainMobile appServer (computing)Client (computing)Client (computing)Attribute grammarGroup actionPolygonPhysical systemConfiguration spaceNumberDomain nameGame controllerComputer fileDatabaseTopologySet (mathematics)Particle systemComputer animation
18:57
Client (computing)Declarative programmingTopologyServer (computing)PrototypeService (economics)TopologyComputer fileProduct (business)PolygonLine (geometry)Well-formed formulaComputer animation
19:29
Local ringTopologySource codeMaizeClient (computing)Software testingError messageComputer fileTopologyComputer animation
Transcript: English(auto-generated)
00:08
Welcome everybody, I will claim that you will get more correct maps by using Postgres topology and using single feature.
00:21
Not everybody share that opinion, but what we see is that when you run overlaps, when you run overlays between many tables, with many features, you find tiny gaps and tiny overlaps. And that's a problem, it may not cause any problems, but we have to remember that map
00:45
data are analogue, they are not digital. If I'm here or here, you have to do some kind of approximation or some rounding to
01:01
find out where you are or what you choose for the real point. So this is not working now, of course, well, I have to click down, yes, oh this is the
01:40
real power, I don't know, but anyway, one of the things we complained about when using
01:45
simple features was that we got topology exceptions. So we complained to the Postgres guys, you have to fix those topology exceptions, you can't use the system. So we made test cases and I fixed it and I fixed it and I fixed it.
02:01
And is that a good idea? I don't know if that's a good idea, because you basically cover up bad data in your database. And then you stay there until some day you actually see the problem. Here's a side step.
02:30
Here is a house, this is our house built 26 years ago. Looks nice, doesn't it? What happened here was this, the house started to fall down in the middle.
02:44
So we asked the builders, what's the problem? They said, oh no, the ground foundation is perfect, the walls are perfect, it is falling down. They had to adjust the windows and doors every second there. So after some years I got kind of pissed off, I took a cobar drill and just knocked
03:03
down the foundation, not the foundation, but the wall to check what's the problem inside. And what was the problem? The wall didn't hit the ground foundation, it was a gap. It's kind of crazy, it was a gap. There was no connection between the wall and the ground foundation.
03:23
So what's this to do with information systems then? Well, I will claim that in an information system, one of the most important parts to build a robust system is your database. And if you use simple feature in your database,
03:43
I will claim that your information systems is not as robust as if you use Postgres Topology. Because in simple feature, nobody knows about, you do not know about your neighbors.
04:02
A simple feature only knows about itself. With simple feature, you know your neighbors. Let's dive a little bit into database explanations here. Here you have a simple feature model and a simple feature father.
04:24
It's a very nice structure, because then when I got my daughter, I could update my record without involving the mother. Easy to update, I have 100% control of my data.
04:41
But what happens then? I spell it this way and my wife spell it that way. And the government likes to keep control of how many shells there are. And there you have different spelling, almost the same, just as you talk about borders.
05:02
Two borders, almost identical. Same problem here. Instead, if you have two poor mothers, two poor fathers and two poor children, you have this system. This is a much better way to do it, because here you have one instance of the name.
05:23
You cannot get in a situation where name is spelled differently and then two children suddenly occur. Here you see a map.
05:42
There's no differences in that map, except that red line, which is impossible to see. It's a duplication. That line exists two places. So if I, as a surface, will change my border,
06:06
and we do not want gaps or overlaps in this layer, I also have to change that border. Or I cannot just don't care and just change my border and leave the problem to the neighbor,
06:23
and therefore adjust it after what I did. But with topology, you have one instance of the border. That means that if I move that border, I have to move only one border.
06:41
And there's another thing there also. These surfaces are appearing because of the edges. No edges, no surfaces. Just as a parent ship, I cannot be a parent with no child.
07:03
Pretty much the same. Not all the same. So it's a huge difference. This means that the edges are the driving part of the realization of the objects.
07:26
Yeah. Here, you have the database schema more like it's in the real world. Here, you have the simple feature, and you see that the geometry is in the feature.
07:45
It only knows about the feature itself, and it does not know anything about the neighbors. In topology, you see a more complex schema. Looks more like the topo child, topo mother, topo father case.
08:05
You have an edge. The edge knows a lot about its surroundings. It knows what edge is following after it, before it. It knows about the left face, right face, and that means what surface are on the left and right.
08:27
Here, you have a much better control. Why? So what are we doing? By the way, I'm not able to pronounce this name. I got some help from Sandra to actually naming it.
08:43
So I just call it Resolve Overlap and Gap Overlay. So there's probably some English speaking here that can pronounce this for me. But let's just call it Resolve Overlap and Gap Overlay now. What we do here, we have a method that loads from a set of layers.
09:05
It loads all lines into Postgres topology. It smiths the line if needed. It collapses tiny gaps and overlaps. What we do get then, we get a Postgres topology structure with all edges and all surfaces.
09:33
This means that this can be quite a bit of edges.
09:43
But it gives you a very nice total view of the system. You may have overlapped, you may have a gap, but you have control over where they are. Here's an example. Here's two layers. You have the yellow layer, that's a layer with big polygons,
10:01
and you have a tiny red layer in there. We're going to merge those together. It's two tables. Then we call the strange named function, Resolve Overlap and Gap Overlay. The main input here are two tables and a schema name and an asterisk.
10:21
What kind of ending asterisk should you use? When you call that, before you call it, you see here there are overlaps and there are gaps. What you see here, there are no nodes here.
10:41
That's the problem with simple features. This has to be computed every time you have two simple features that have a connection or overlaps or whatever. Those lines need a node.
11:01
In Postgres topology, what happens is that those nodes are added. Then, in your database, you get something like this. You get all the edges.
11:20
You see there's basically nothing new here except the nodes. That is stored in the database. What happens when you store that? A new surface comes to the surface here. That surface did not exist in the simple feature world
11:41
because the simple feature objects did not know anything about each other. About that red edge there, the left face is different from the right face.
12:01
What does that mean? That means that there are different surfaces on both sides of that edge. If you look at this case, this is a test case I was working on a couple of weeks ago.
12:24
It's not the best picture, sorry for that. What you see here is that there should be a red edge there. It was missing. You can find traces of it. We pushed everything into Postgres topology. We got no exceptions. It was gone.
12:42
We started to check it out. That line, that simple feature has about 20,000 points. It's a quite big outer edge now. People seem to like all kind of strange polygons with 100,000 points. I don't know why.
13:01
But that's the real world. What we did see, we checked this further on, was that here we started to check. Here, left face are equal to right face. What happened was that from here and on, that edge was gone.
13:25
To fix that, we had to find out where the problem was. You can guess where the problem was. It was a simple feature, of course. It was a line merge. Basically, what we have been doing is that it's a kind of heavy work to push all the lines into Postgres topology.
13:45
We have tried to take the load of Postgres topology by using some simple feature functions. We send the line merge to do what we call heavy work. It gets a nice thing back, but that was thrown away.
14:02
It happened every time. Of course, I should have made a test case for the simple feature guys, but they don't like test cases with polygons of 20,000 points, and I do understand that. It makes sense.
14:24
By removing line merge and putting all the work into Postgres topology, it worked. Here's another case. We had an edge totally disappearing. Here, I also used a simple feature function called snapToGrid.
14:42
That also removed an edge. So, we had to stop using snapToGrid. Non-covered areas. If you have to search for those in simple feature, then you have to use as the difference for instance.
15:02
And what we do in the reserve overlap and gap code is that we generate you and tables. If you look here, it says gaps is true. So, if you want to find all gaps in your tables that is sent into reserve overlap and gap,
15:23
you just do a selection where gaps is true. Yes? And you have the same for this, the other layer. Here, it also was like gap. Just as easy to find.
15:41
And with simple feature, we reported this bug some time ago. Here, I started the difference. I wanted this area, but I got this area out. No errors, simple feature returns, always returns now.
16:02
It's very hard to get those to return topology error. And that's both a good thing and a bad thing. And I can't partly blame myself for complaining about getting topology exceptions. So, but it's, it's, yeah.
16:24
Here, we see where we have overlaps. We can just jump that one. We can jump that one. Here, you see a combination of two layers.
16:42
This surface, or polygon, has attributes from both layers. So then, all the values are filled out here. And we do not, we do not copy the values. We use views, so we relate back to the original data.
17:04
Here is our case. We sent in this as input. We got this out about some million edges, a total of eight million polygons. After working with it, yeah, this just shows the result tables.
17:30
We get this. So we produce a set of simple features. The domain people know what they want, so they do a group by,
17:40
and we find and fill in all the gaps with some sensible values. And we reduce the number of polygons from eight million to about 716. Okay. Why not let the client update simple feature databases?
18:01
It's too much control to the client. The client needs to know everything about every feature. Both also, not only for the spatial part, but also for attributes. And you can do the update in many different ways. So you also need advanced spatial tools on the client,
18:21
and if you use POSKIS to POSKIS, simple feature, that cover up all your bad data, and if you send those bad data to the client, the client may not be that robust. So with the POSKIS topology, it's much easier,
18:40
because what you do is that you can set up a simple system. We have done that together with Sandra Santelli. You have to generate something based on a config file. You get a database, you get a POSKIS API, and you get a client that Mattia is here working on. So then you can do like this. You can draw a polygon, you can draw a line, and you can split it.
19:01
We can remove a line. Everything, this is used in Norway by almost all communities. This is based on POSKIS topology when updating farmland. So we are actually, the commercial products,
19:22
they have commercial products, and we are switching to this because this is much easier to use. Okay, there are some README files on the first two there, so you can actually test it yourself. You can run, you can test POSKIS topology,
19:44
and you can test the client, and it should be quite easy to test. I will not say that POSKIS topology is out of errors. It has errors, yes, and the speed we have to do something with. So we need more work on it,
20:02
but we at least need to open our eyes according to the simple feature problems. I don't know how popular it is to say this, but I think we need to talk more about that.