Laying the foundation for an artificial neural network for photogrammetric riverine bathymetry
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Title of Series | ||
Number of Parts | 351 | |
Author | ||
Contributors | ||
License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
Identifiers | 10.5446/68902 (DOI) | |
Publisher | ||
Release Date | ||
Language | ||
Production Year | 2022 |
Content Metadata
Subject Area | ||
Genre | ||
Abstract |
| |
Keywords |
FOSS4G Firenze 2022340 / 351
1
7
13
22
25
31
33
36
39
41
43
44
46
52
53
55
58
59
60
76
80
93
98
104
108
127
128
133
135
141
142
143
150
151
168
173
176
178
190
196
200
201
202
204
211
219
225
226
236
242
251
258
263
270
284
285
292
00:00
Artificial neural networkIntegrated development environmentArtificial neural networkResultantGreatest elementDistortion (mathematics)Goodness of fitComputer animation
00:31
Variable (mathematics)MeasurementMathematical modelMeasurementProcess (computing)BitWorkstation <Musikinstrument>Archaeological field surveyPoint (geometry)FrequencyDuality (mathematics)InformationGreen's functionRaster graphicsObservational studyFunction (mathematics)Medical imagingGreatest elementDoppler-EffektTotal S.A.Cross section (physics)Reflection (mathematics)Matrix (mathematics)Computer animation
02:02
RefractionRadiometryInterface (computing)OpticsSatelliteMeasurementFehlerschrankeData modelSurfaceAngleWater vaporAngleParameter (computer programming)Flow separationProcedural programmingPosition operatorSurfaceInformationDependent and independent variablesRefractionCausalitySpectrum (functional analysis)Different (Kate Ryan album)Reflection (mathematics)Interface (computing)Point (geometry)PhysicalismDigital photographyData structurePhysical systemArchaeological field surveyThetafunktionCartesian coordinate systemArithmetic meanType theoryGoodness of fitMultiplication signWordOpticsComputer animation
04:19
RefractionMachine learningArtificial intelligenceComplex analysisLinear mapSatelliteObservational studyArtificial neural networkVariable (mathematics)Endliche ModelltheorieArtificial neural networkData structureTerm (mathematics)Wave packetForestMachine learningMeasurementSoftware testingMultiplication signSound effectSlide ruleOrder (biology)Integrated development environmentGreatest elementRadiometryVirtual machineComputer animation
05:59
EstimationRadiometryRefractionData structureStatistical dispersionFocus (optics)FehlerschrankeArtificial neural networkMathematical modelMusical ensemblePhysical systemMultiplicationEndliche ModelltheorieWater vaporDuality (mathematics)Data structureTerm (mathematics)Mathematical analysisFrequencyArtificial neural networkDependent and independent variablesWave packetCASE <Informatik>Software testingObservational studyOrder (biology)Basis (linear algebra)Validity (statistics)Different (Kate Ryan album)Archaeological field surveyMereologyAreaoutputEntire functionWeightGreatest elementInformationProcedural programmingSurfaceSet (mathematics)String (computer science)PermutationStandard deviationMetreStreaming mediaComputer animation
08:37
TrianglePoint (geometry)Square numberSurfaceRWE DeaLine (geometry)Graphics tabletStreaming mediaNumberModal logicPoint (geometry)InformationState observerReal-time operating systemGreatest elementFrequencyDifferent (Kate Ryan album)Duality (mathematics)CuboidComputer animation
09:42
Image resolutionTelecommunicationModule (mathematics)Musical ensemblePhysical systemImage resolutionDirected graphMereologyObservational studyEndliche ModelltheoriePoint (geometry)AreaGreatest elementCASE <Informatik>Order (biology)InformationPhotographic mosaicWater vaporFehlerschrankeArmDigitizingMoment <Mathematik>Data acquisitionData structureComputer animation
11:23
Musical ensembleData modeloutputSheaf (mathematics)SurfaceAreaVariable (mathematics)Dependent and independent variablesAdaptive behaviorEstimationWeightMathematical optimizationFunction (mathematics)Linear mapInsertion lossPerceptronArtificial neural networkPerformance appraisalPredictionAlgorithmPermutationMathematical analysisStandard deviationReal numberSubsetFehlerschrankeAverageScale (map)Discrete groupSoftware bugGreatest elementHausdorff dimensionMathematical morphologyCharacteristic polynomialData structureObservational studyAdditionAnalog-to-digital converterMeasurementVirtual machineMathematical modelWater vaporFehlerschrankeInterface (computing)Linear regressionDecision tree learningWave packetArtificial neural networkDifferent (Kate Ryan album)Software testingVariable (mathematics)FunktionalanalysisCASE <Informatik>MeasurementState observerIntegrated development environmentProcedural programmingGreatest elementData structureAlgorithmBitRankingMusical ensembleScaling (geometry)FingerprintLibrary (computing)Mathematical optimizationPresentation of a groupType theoryMoment <Mathematik>Dependent and independent variablesGoodness of fitL-functionSimilarity (geometry)Set (mathematics)WeightInformationSurfaceResultantPairwise comparisonTerm (mathematics)Standard deviationMultiplication signDimensional analysisSquare numberTwitterNetwork topologyPower (physics)DataflowPoint (geometry)PixelRoundness (object)Line (geometry)Service (economics)Natural numberArithmetic meanArchaeological field surveyPermutationSlide ruleRight angleEndliche ModelltheorieMathematical analysisPole (complex analysis)Maxima and minimaPattern languageImage resolutionTensorMetreNormal (geometry)Direction (geometry)Stability theoryAsynchronous Transfer ModeGraph coloringInsertion lossAnalogyAbsolute valueComputer animation
Transcript: English(auto-generated)
00:00
Thank you, thank you, Paolo, for the nice introduction. Good morning, everyone, all few of us. I think it's because of the yesterday dinner. So I'm presenting the first results that my colleague, Vicente de Pietra, and I obtained for the creation of a simple neural network
00:23
for the correction of the distortion derived from the photogrammetric reconstruction of river bottoms. But let's try to contextualize a little bit this work. So bathymetry through the measure of the depth of waters
00:41
in land and ocean is a variable that is crucial and very important for the hydraulic modeling and for the study of river processing. So it can be measured with several methods. The traditional one, like topographic survey with total station and reflectors, but also dual frequency receiver,
01:02
like the one that we can see in the first image. We can use other methods, like profilometers, that are based on Doppler measurements. They are a little bit more faster than traditional methods. They work in cross-section along rivers,
01:20
exactly like topographic surveys. A little bit more expensive, but faster. Then we can use new technology, like green leaders. I didn't put it in the picture, but we can also have echo sounders that instead of working on cross-section, so on measure realized point by point,
01:42
they acquire information in a matrix way. So the final output is a raster to simplify it. So the real problem in the reconstruction of the bottom
02:02
from a photogrammetric application is connected to the refraction of the light in the atmosphere and in the water, because the light has a different reflection angle in these two means.
02:21
And the passage of the light within the interface between the air, the atmosphere, and the water cause a different, OK, yeah, I have a pointer, good, cause the structure for motion procedure to underestimate usually the point.
02:42
Because as you can see, theta 1 and theta 2 are different and defined by the means, so water and the air. As you can see here, this system can be solved quite easily if you know the two angles and the water surface position.
03:03
And what to solve this system, in photogrammetric world, several methods have been developed. We can call them photogrammetric or geometrical methods. They are well-performing, interesting, but unfortunately, they require a lot of time, a lot of information, starting from the parameters
03:23
of the camera, the position of the cameras, the water surface position, and quotes. So in time, other type of approach have been developed that are usually based on the spectral response of orthomosaic photos.
03:45
So instead of being in a physical system, we're now in, well, always a physical system, but it's based on optical information. So optical remote sensing have been developed different method and techniques
04:04
to correct the, to estimate survey the depth of the bottom covered by water. Such methods are usually developed for big rivers, lakes, and ocean.
04:20
Some of them rely on artificial intelligence and machine learning. Most of them are based on random forest and starting from early 2000, also method based on neural network are available. But most of them, as I said, are
04:42
created for oceans, sea, and lakes that have very, let's say, simple structure in terms of bottom respect to alpine rivers. So when we first time were in the situation of measure
05:06
and estimate the bathymetry of an environment like this one, which I, all the methods that I briefly recap you in the previous slides, and realized that we're not so well-performing. So Michael Licks and I asked ourselves, what can we do?
05:26
We try with traditional topography, but it was very time consuming. And so we thought that we can create an artificial neural network, a very small model that can be trained using data
05:42
that we already collected in all these alpine rivers in order to estimate the bathymetry. So what we did was test the effectiveness of empirical radiometric methods through deep learning
06:04
model to correct the underestimation of the bottom elevation derived from the structure for motion methodology. We, as I said, decided to focus only
06:20
on small water courses with less than one meter depth that are mainly busy in the alpine region of Piedmont in Northwest Italy. OK, here we have very short workflow. First, all the part of data collection
06:41
in which we realize a drone survey using a multi-rotor system, different sensor, different system. This doesn't matter in terms of analysis and quality of data for the model. Then we collect information of the depth
07:02
and also of the bottom of the river using dual frequency GNSS receivers and profilometers. The data set was then generated through a standard structure for motion procedure. And the depth was calculated as a difference between the water
07:28
surface, the Ws that I showed you before, and the real quote that were measured to GNSS and the profilometers. Then we prepare some extra feature
07:41
in order to improve the performances of the network. We selected different features that are basically derived from the radiometric response of our data. All data were scaled and divided into training data set,
08:01
test data set, and a part of it, an entire case study was used as unseen data set in order to check the ability of our net to generalize. We define the regressor network, perform a permutation analysis in order to check the importance of each the input
08:22
feature within the network, and then perform the validation on the test and the validation on the unseen data set. Our data were collected in three different areas. So they have in common of being alpine streams.
08:44
And they have pedals as the bottom with a granitic box. As you can see, we have a different number of observation. That's mainly because the data were collected with two different tools. So GNSS in our etaK modality, real time cinematic,
09:03
and the profilometers also embedded with dual frequency receivers. For the first two, we have 170 observation, while for Palice River, the last one, we have more than 600 observation,
09:21
which are basically points. So each point has x, y coordinates, so longitude and latitude, and innovation information regarding the bottom. Here you have the profilometers, as I said.
09:41
It has a dual frequency antenna embedded on it. And we are able to determine the elevation of the points in the bottom of the river, knowing the lever arm and the sinking of the vessel, too. As I said, we use two different cameras
10:01
for the data collection. Both are embedded on DJI drone system, our RGB cameras. We also have information regarding the infrared spectrum for two of the case studies. But for this specific work, we only focus on the visible part.
10:21
So very shortly, the errors and the accuracy that we got from the structure for motion reconstruction, we are around two centimeters and one centimeters and half of error on the reconstruction in the three areas,
10:40
and resolution of the oromosaic and the digital surfing model of one centimeters to four centimeters. Oh, this is the data collection with the profilometer. As you can see, as I said, it is fast, but not so fast, because not having an angina, and it's
11:02
to be moved from one bank to the other one really, really can wait in order to not move too much the water and create an error in the data acquisition. OK, so how we measure the depth.
11:26
We basically subtract the information of the water surface to the elevation of the points measured on the bottom. The water surface has been calculated in two different way
11:42
according to the tool that we use for the data collection. So for the GNSS with pole acquisition, we measure them manually along the interface between water and the riverbank, while for the profilometer, we used the quota measured by the GNSS antennae,
12:04
to which we subtracted the value of the lever arm, the sink, and the depth measure realized by the vessel itself. We then computed all the band rushes that you can see here on the right of the slide,
12:21
and scaled them using Schickett algorithm, the minimum access color. And all the data set from the very barrier, the one with 120 observation, was used as unseen data set or virgin data set.
12:42
Here there is the structure of the net. It's quite simple. It's a regressor, so we have the dependent variable, our depth, and the independent variable of the remaining including the depth with the BS caused
13:02
by the photogrammetric reconstruction. Everything was written in Python using Keras and TensorFlow as a back end. It is composed by a tree layer using two different activation functions. And the weight optimizer that we decide to use is the atom.
13:24
The loss function was measured on the mean absolute error, and everything was trained in 100 epochs. So as I said before, we computed also
13:42
an importance value for all the variables that we inserted in the net. The importance was calculated with a permutation analogy that basically considered the accuracy improvement or the crease for each of the variable
14:05
by training the model different time every time without a variable. The most important feature were the direction between red and blue and the NDTI that is the normalized stability index.
14:22
And then here I show you some results. The network performed quite well. The loss function converged very quick, as you can see. We have 068 on the test and 074 on the training.
14:45
This is mainly due, I think, to the bad generalization ability of the network for very swallow waters. So here you can see the direction between the real
15:04
depth and the predicted one. All the values are scaled. So in this case, we're not talking about centimeters or anything. As you can see, everything is under 0.5 as much more dispersed, while for deeper water,
15:23
the performance is much better. We can see similar trend also in the unseen data set. Indeed, the observation that you have here, observation 65 and around 90, are much more
15:44
in respect to the real data. So the trend is kind of well-predicted, but still there are some outlier. In this key, on the unseen data set, we have RMSE of 10 centimeters and an absolute error
16:05
of 8 centimeters. So these values are acceptable for the type of work that we are realizing on the one hand, because they are in line with RTK measures that
16:24
have crossing precision around 3 centimeters or less, and also because of the nature of the environment that we surveyed and we measured. As you can see, most of the patterns
16:42
that we have in the river bottom are more or less than centimeters. So obtaining procedure or accuracy, greater service mode in this have small meaning.
17:01
We see that the performance of the network is improved, it's better in high depth, but we believe that this is due to the dimension of the rock
17:20
in the bottom, the type of sediment, also the presence of vegetation. We believe that the performances of the net can be improved if we add more observation, because the demand gap is that we have a 500 observation for a neural network that even if it's a regressor,
17:43
they're still not enough. But this is only the beginning. We actually already add two more survey and two more data set to our net, and from the first results, we already see some improvements.
18:00
So for the future, we would like, of course, to enlarge our data set and try to investigate a little bit more on some aspects. So first of all, understanding why in the ranking of importance of the feature, the photogrammetric depth, so the depth of the water
18:24
estimated through structure for motion is not so important. We expected it to be linearly correlated to the real depth, so the corrected one.
18:40
This wasn't true, maybe because the permutation method is not the best one to check the importance of the feature between a regression, or maybe it's because of the scaling method, which is well-known being very, very influential
19:01
on the result of the model. Some others consideration on tests that we would like to do is the methodology for the measurement of the water surface, which we believe is still very critical, specifically for the GNSS measure relies on the pool,
19:22
because it depends on the user and the accuracy of the measure itself. So these are only the first step. We hope to be able to increase the net very soon and add much more observation that, for now, are
19:44
concentrated on the alpine arch. But who knows, maybe in the future, we can add also other type of environment, in fluvial environment. So thank you very much for the attention. If you have questions, I'm here.
20:00
Thank you. OK, so I see that there are no. So first of all, thank you for your presentation. Sorry, I'm a bit sleepy this morning, I believe, most of us. OK, so I see that there are no questions on venus-less,
20:23
but I don't know if the people here on the floor, they have questions or curiosity about this presentation. Oh, I see that there are. Oh, there is one person. You have to go there, where there is the microphone, please.
20:41
Yeah, I believe that it's working. It's already working. Ah, right now it is. Yes, can I hear you? OK, good morning. I'm also very sleepy. Pardon me. So yeah, Elin, a nice presentation. I really enjoyed it. I'm wondering something.
21:01
You have a relatively short set of observations, and you went for the neural networks. I'm wondering, did you consider regression trees? Because you can do very similar exercise with regression trees. And what are your thoughts about it, and why did you opt for this way? OK, thank you very much for this question.
21:22
Yes, we definitely consider also regressor trees. And actually, all the model that you see, all the data, were used to compare different methodology. One of them were the regressor tree, the standard linear regression with multi-parametric
21:41
regressions, and also artificial neural network. We decided to go for a neural network because of, well, the idea is to add much more data, and we believe that somehow is better able to generalize. Indeed, from the first result that we obtained
22:00
from the comparison, the regressor tree performed very well. We have a good score in terms of errors. But then when we apply it to an unseen data set, as we did with the artificial neural network, we have immediately a little bit more higher error. We're actually writing a paper for a journal about this.
22:26
So you will see the results soon. You're welcome. Thank you. Yes, and I see that there is a question also now, which is, you flow from 40 over 50 to 88 meter
22:44
above surface. What if you flew with a surface acting drawn and could be, for example, five meters above the water rock surface? Would that improve or worsen the precision?
23:03
OK, thank you for the interesting question. Well, we flew very, it's a relatively low flow as they say, 40, 80 meters. And this was because we needed a dimension of the pixel around one or two centimeters.
23:22
That is what was required. Of course, if we had to fly higher, we would have a larger pixel. So the spectral response of each of them would have been not so specific to each parallel or rock. And for sure, something would have changed.
23:40
So it would be interesting, actually, try to run the same test and develop the model also with the same data set, but maybe with a lower resolution in terms of pixel dimension. So that's it. I hope I answered.
24:01
OK, thank you. And is there another question? No, I don't know. So I believe at this point that we can close here. So thank you again for your presentation.