FOSS4G based high frequency and interoperable lake water-quality monitoring system
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 237 | |
| Author | ||
| Contributors | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/57265 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSS4G Argentina 2021144 / 237
12
15
16
23
26
36
44
52
53
54
59
72
90
99
114
121
122
123
124
127
129
130
139
154
155
166
203
204
219
223
224
230
00:00
System programmingPresentation of a groupGeometryLevel (video gaming)Projective planeSpeech synthesisFrequencyUniverse (mathematics)Source codeMeeting/Interview
01:16
IntegerSystem programmingFrequencySimulationCoordinate systemData managementReal numberComputer wormAnalogyDigital signalDigitizingSurface of revolutionIntegrated development environmentTransformation (genetics)Process (computing)InformationTelecommunicationConnected spaceSpreadsheetSoftware developerState of matterUniqueness quantificationSample (statistics)SystementwurfChainVector potentialFile formatUniformer RaumData integrityElectronic mailing listStandard deviationMetadata8 (number)Charge carrierAreaParameter (computer programming)AuthorizationProjective planePoint (geometry)Vector potentialWater vaporState of matterDigitizingQuicksortProcess (computing)System programmingMetadataUniqueness quantificationData managementUniformer RaumINTEGRALComputer configurationMereologyWorkstation <Musikinstrument>DatabaseError messageCASE <Informatik>Mathematical analysisMultiplication signDigital signal processingSpreadsheetData managementTwin primeView (database)Phase transitionDigital signalResultantBeat (acoustics)Physical lawGoodness of fitRight angleBit rateLeakArithmetic meanDebuggerNumbering scheme1 (number)Closed setExecution unitRAIDMaizeMusical ensembleComputer fileLatent heatLink (knot theory)System callData miningMetropolitan area networkFrequencyRoutingThumbnailDistanceReal numberSoftware bugComputer animation
08:32
Uniformer RaumAddress spaceNumerical digitKeilförmige AnordnungStatisticsForcing (mathematics)System programmingData managementSoftwareDigitizingLocal ringWater vaporOpen setComputer animation
09:12
Enterprise architectureSystem programmingState of matterProcess (computing)TransmitterAmsterdam Ordnance DatumMaxima and minimaImplementationMacro (computer science)Stability theoryDrum memorySoftware maintenanceSynchronizationTelecommunicationFormal verificationSoftwareSoftware testingFactory (trading post)Nichtlineares GleichungssystemCalculus of variationsHand fanUniform resource locatorSet (mathematics)Sanitary sewerInteractive televisionSpecial unitary groupComputer architectureSoftwareTelecommunicationInstallation artFile formatState of matterProjective planeField (computer science)Type theoryLevel (video gaming)State observerProduct (business)Endliche ModelltheorieCuboidMultiplication signCASE <Informatik>UsabilityDemosceneDirected graphSystem programmingData managementThomas BayesSource codeTwitterResultantDifferent (Kate Ryan album)Point (geometry)Software testingLocal ringControl flowAssociative propertyForm (programming)Social classExecution unitNumberArithmetic meanGoodness of fitView (database)Metropolitan area networkPhysical lawSpeciesWhiteboardProcess (computing)Maxima and minimaCategory of beingService-oriented architectureRight angleValidity (statistics)MereologyPerformance appraisalInstance (computer science)outputService (economics)Standard deviationComputing platformData qualityData managementTime seriesPairwise comparisonCoprocessorGame controllerCommunications protocolDataflowKey (cryptography)AlgorithmMeasurementConfiguration spaceAuthorizationServer (computing)Profil (magazine)BackupFood energyFlow separationOpen sourceSelectivity (electronic)Maß <Mathematik>Compilation albumQuicksortUniformer RaumSelf-organizationPrice indexParameter (computer programming)Procedural programmingEstimatorOntologyPhase transition2 (number)Mathematical analysisSubject indexingData storage deviceTransmitterDatabaseDecision theorySoftware maintenanceComputer animationProgram flowchart
18:44
Calculus of variationsComplete metric spaceoutputSystem programmingDisintegrationExt functorPerformance appraisalStandard deviationReal-time operating systemDigital signalFrequencyMach's principleForm (programming)Lemma (mathematics)DataflowProduct (business)TwitterData qualityResultantEstimatorDifferent (Kate Ryan album)Time seriesSystem programmingDigital electronicsView (database)Limit (category theory)User interfaceEndliche ModelltheoriePoint (geometry)Computer networkAlgorithmState observerExpert systemReplication (computing)Software testingComputing platformWebsiteFood energyWater vaporoutputSound effectConstructor (object-oriented programming)Multiplication signCartesian coordinate systemMoore's lawSubject indexingFunction (mathematics)SoftwareTable (information)Software maintenanceSpacetimeInformationDigitizingMappingMoment (mathematics)MereologyParameter (computer programming)Core dumpLatent heatNichtlineares GleichungssystemService (economics)Band matrixVoltmeterCommunications protocolSystem administratorDemosceneCuboidMetropolitan area networkGame theoryRevision controlLine (geometry)Web 2.0Sheaf (mathematics)QuicksortMixed realityBit rateOctaveTerm (mathematics)GoogolECosTorusFrequencyPhase transitionEntropie <Informationstheorie>Pay televisionMachine visionStandard deviationVideo gameTime zoneLevel (video gaming)Projective planeCASE <Informatik>Rule of inferenceSystem callRoutingFamilyLogicMathematical analysisComputer animation
28:16
Function (mathematics)Inclusion mapMultitier architectureMeeting/Interview
Transcript: English(auto-generated)
00:14
So, hello Maximiliano. Hi there. So the next presentation is Phospho GIS
00:25
based on a high-frequency and inter-foreable lake water quality monitoring system with Maximiliano Cannata for Italia. Maximiliano is professor, a tutor of geometrics and the Institute of the Earth and Science
00:43
for La Escuela Universitària Profesional della Isvitra Italiana. I think my Italian is very good. So the University of the Applied Science in Art of the South of Switzerland.
01:01
So Maximiliano, I tried a lot of the next Phospho GIS. So Maximiliano, this stage is all yours for your presentation, thank you. Okay, thank you very much. And yeah, I'm going to talk you,
01:21
describe you a project that we are running that is related to the high-frequency monitoring of water quality parameters of a lake. And this is an indirect project. It means that it is a project between Italy and Switzerland.
01:41
In fact, one of the aspects that we want to approach is that there are three different lakes that are really close in the area and that are monitored from different institutions and also the national authorities
02:00
that define the policies are different because one is Switzerland and one is in Italy. And the idea was to try to homogenize the monitoring to have a comparative potential between the state of the lakes from the water quality point of view, since most of them are interconnected
02:22
from hydrological point of view. And then the second thing was try to make some sort of innovation and bring some changing in the management aspects of this water quality. And yeah, this is the topic.
02:43
So it's about the management and the monitoring of the lake ecosystem. So water quality and also of the ecosystems. One of the things that we have seen in this sector related to limnology,
03:02
so water quality and ecosystem of lakes is that they are still in a phase where they're moving in digitization and they're trying to move into the digitalization but this is not really happening so fast. So the real topics there is also
03:22
how we can move into this digital transformation in the lake ecosystem management. So try not just to use digital data but to extend the concept with the sort of digital twins of the lakes and try to use it as a reactive system
03:42
that can provide insight to the managers to adapt and use a more adaptive policy in the management of the water rather than just passive and define it. So we have to deal with two somehow different but really linked topics.
04:02
One is about the data management and the other one is about the monitoring. The issue with the digital transformation has different reasons and what we have noticed like in most of the cases we have disconnected system
04:21
that are managing spreadsheets and people are not used to take advantage of the new technologies but we see that there is a lot of opportunities and we try to push this digital transformation.
04:41
Where we start from? We start from monitoring campaigns that are run in this case where I'm talking about the Lake Lugano which is in Switzerland which is monitored by my institutes, my colleagues and technologists and they go once a month with the boat on the lake,
05:01
they use some sensor that they put in the water then they perform some measurements, manual drive and water sensor perform it. They collect some water samples that goes to laboratories and then we get back some analysis from the labs
05:22
and this is a regular campaigns over the year, once a month and we have data for more than 50 years like this. Of course, you see that in these ways is almost impossible to react to different phenomenon that can happens in the lake
05:42
with the shortened time interval rather than one a single month. There are different options that are this new solution. One of them is the remote sensing but I'm not talking about this. This part has been addressed by the polytechnic of Milan
06:01
and CNR of Milan while I'm talking with the automatic high-frequency monitoring system which is about physically a sort of weather station that monitor the water quality in the lake. What is the state of the art in this high-frequency monitoring system?
06:21
We can bring a couple of the two example. One is on the left side is L'Explora is a floating laboratory on the Lake of Geneva, Switzerland. This is like a unique research opportunities because this is a very big, very rich in sensor
06:40
and things like that and it's very expensive. I don't know exactly but certainly more than hundreds of thousands of US dollars. On the other side, there is also some experiments of a smaller voice system, a lower cost.
07:02
These more oriented to specific near needs and is more a customized solution done by Palanza. And from the data management side, we are dealing with data that are currently managed
07:23
in access database or a multi-DB file, Excel file, text file, and et cetera. And there is really a lack of uniformity in data formats, ontology, interoperability. There is the error prone processes in copy and paste
07:41
and that integrity is not guaranteed and there is also data latency between sampling and then data availability. So there is a lot of potential that we could try to exploit using, integrating with some interoperability,
08:02
standards, metadata schemas, and unfair principle. While we are still there, there are a lot of answer, possible answer. And you see here, probably is a matter of cost also, but also a matter of digital expertise of the personnel
08:23
and also sort of resistance to change in the usual, let's say, resistance that you can have in the digitalization process. Our question is, how can we somehow make the management
08:41
benefit of these advantages of the digitalization? And can we really integrate a fully open software solution that can address such a problems and foster the digitalization of the water sector? And how the system might help in taking the local efforts, for example, of climate change. So it's not just about monitoring,
09:02
but then also enabling the usage of this monitoring to tackling some, for example, climate changes efforts. We proposed an open integrated system with starting from data sources, some pre-processing,
09:21
then some storage in database and then offering as a standard services. And we wanted to make these everything integrated and automate so that also the limnologists can easily access the data and elaborate and take decision. The project applied experimental testing methods.
09:42
So it's quite straightforward. We designated a study, we elaborate the state of the art, we design a solution, we develop the solution and then we make some testing, preliminary testing of the solution and then deploying the fields.
10:00
And then we start to analyze what we can get from the solution and evaluate the results, finally. Now we are in the stage, you can see in the box at the end of the second year and beginning of third year of project. So we have deployed the system and we are in the phase of developing that analysis
10:23
and taking some preliminary conclusions. The methods that we wanted to access as an example for climate change is, how can we estimate the primary production of the lake, which is the production of primary organic.
10:44
And this has been generally estimated with these monthly campaigns using the C14 easel top. And this is a expressive procedures. This is somehow dangerous because it is also radioactive, et cetera.
11:01
And as I said, you have 12 values a year. So we started to look into literature and we find out that there are modern systems and models that can make an estimate, automatic estimate based on sense of measurements.
11:21
We start to design the architecture and we combine several open source technologies. The source for the data management, which is a NOGG software,
11:40
Grafana for the plotting, MQTT broker for accessing data from the sensor and then a key clock for authorizations and access management. And then we implemented also some new configuration services. And this is from the software point of view.
12:00
So the data management, then we have also the monitoring part and we design a system fully open based on Raspberry Pi. And that's using BIOT as a communication protocol and that connect with the standard sensor that you find on the markets. And we designed this system with idea
12:23
to be replicable as much as possible. The approach in the data management that we follow is a two-tier data flow. So some of the data collection is on the edge at the, let's say, boy or lake platform.
12:41
I will show you later some pictures. And then there is on the right side in green, the server side. So data collected locally, but are directly at the edge checked for their validity before being inserted into the source instance.
13:01
So at the edge on the boy, at the sensor side, we have an installation of the software that manage the data so that we can take advantage of the services. We can access data with the standard formats and we can also use their integrated data quality assessment.
13:21
So then data from the raw data, we perform some aggregation and data check before transmitted. So when that arrive at the server, that are already pre-flagged with some quality index that we can decide how to use then this data. When they arrive at the server,
13:41
of course they are still available for using the standard, the sensor observation service standard for validation of the data, for generating reports, analysis, and alerts for custom, et cetera. So somehow we move part of the data quality
14:00
on the edge side. We have implemented a dashboard and in particular, we have developed some data imported for historical data, because one of the things that we wanted to do also is to put together the new data sources
14:22
that comes from the sensor with the traditional middles so that we can combine the data and take out the maximum out of that. So we have implemented some automatic importer for the data that has been used and we have also together with the other partners,
14:40
we have defined what is a common ontology of the parameters and uniform either unit of measures of the different indicators and parameters that we want to access. As you can see, we have different type of data in the data management system,
15:01
which is based on resource. You have in the box, the data from the sensor in the fields. So these are, they are real time data, but you still also have 50 years of data, for example, from 72 to 2024, a different type of sensor and monitoring.
15:25
With the dashboard, taking advantage of Grafana, you can have different plotting of the data of historical selecting and filtering by location, filtering by time with the server property and et cetera.
15:41
You can visualize the profiles with the recently added new feature in the source to manage profile data. And also you have a dashboard, so you can also see time series of a single sensor in a given depth, for example, and see the different properties that has been measured.
16:03
And then you can activate some comparison and evaluation of the data. So now we have the data, we have the sensor in the field, collecting data, historical data, and then we have to implement some processes and so modeling of this data.
16:22
So we have implemented an asynchronous processor. So within e-source, we have created, extended the source to create these asynchronous processes so you can define a sort of new procedure, so a sort of new sensor and new values that is being calculated every time
16:41
the original data source get new data. So it's quite easy. You select the process that you want to simulate, you select the sensor or the data that you use as an input. Every time your input data get some new data, then as you can see, this process are activated
17:02
and this is a reactive approach to make the compilation. This is the small platform that we have deployed on the fields. You can see there is a solar panel that provide the energy with a battery unit for backup
17:25
and then the main unit with the fine center, some solar regulator, voltage regulator, et cetera. And I repeat all of these, he's built up using commonly fundable pieces on the market
17:43
and is fully replicable. So everything is open. Together with the deployment, we started to keep the maintenance log box because one of the things that we want to see is how after running the system in production,
18:02
how does it work? Does it break often? Do we lose data? And how did the system performs? So we had some small issues, but so far we are quite happy with the system. We are collecting a good number of values.
18:22
We did some quality control in post-processing using two tests, a plausible value and a step test to verify if the data makes sense. And then we implemented some algorithms to estimate this lake metabolism,
18:40
this primary production, as I said. And this can be estimated from observation of oxygen, dissolved oxygen in the water at different times. And we can make estimation of the gross primary production because during the night the respiration is not active.
19:03
So you can do some differences between night and day and estimate. I don't go into detail of the algorithms and the equation of the model that we have implemented and integrated in the system.
19:22
These are some primary result of the data quality for the data completeness, for example. We can see that 98% of the data are available. So it means that we have lost some data, but just a few data in this,
19:40
almost this estimation was done by the eighth month of January to August 21. And this is also from some specific parameter for oxygen that we want to use. We go also to 199%. You see these two, QI101 and 103,
20:02
are these two different quality index that relates to the data quality. From the solar panel point of view, you can see that the system is well-dimensioned and we have still space to add the new sensor
20:24
that consume energy. These are in fact, we plan to extend the sensor on the platform adding other sensor to monitor the chlorophyll and to detect algal blooms in the lake, for example.
20:40
These are some time series that are the inputs for the primary production estimation. So we have the dissolved oxygen in a different depth and you can see that we have continuous data and there is an effect of wind speed and temperature and solar irradiation, of course.
21:02
The preliminary result of the estimation of primary production is quite positive. In fact, we can detect a different trend from the seasonal trend that we want to expect from the winter and starting from the summertime
21:23
where primary production increase and also comparing the values in the table from historical data that has been monitored with the other system monitoring the C14 are in line. So we are quite confident that we can somehow change
21:45
the methodology to monitoring the primary production, having less invasive, less expensive methodology and having a high-frequency data. And this is somehow the point where we wanted to reach is how you can use all this open stuff
22:04
and all this new digital technology to improve the way you are doing and have new information to solve the challenges. So in coming to the conclusion and the system is working without any major issues,
22:24
the cost of the system is about 15,000. This really depends. You have to think that almost 5,000, 10,000 is about the mooring of the platform. And so the cost of the sensor is 5,000.
22:42
In fact, we are thinking for the future, try to miniaturize the system and bring it to the boy. So we have new integrated system which collect data and put together all the data of the limnological sector and make them readily available for analysis.
23:04
The data are available using a standard, the sensor observation service and we have new digital application of data. So we are creating new digital value. For the next year, we have to evaluate more in detail
23:22
the cost of the maintenance of the system. We have to deploy more system, more sensor and of course we are continuing to enhance the system in general, like from the software point of view and from the output point of view. And thank you for the attention.
23:42
So thank you, Maximiliano. We have at the moment one question and I think you respond the first part of the question but let us know if,
24:01
let us know, see if you have some considerations about this. In general terms, what is the core cost of the infrastructure from monetary? I think you just answered this part. So there is some limitation for replication
24:24
in another ecosystem like a tropical lagoon or reservoirs. It is some limitations about application. Okay, so from the cost, as you said, I've already answered, more or less is 15,000.
24:42
The cost, but really 10,000 was about the construction of the platform. Why we did select the platform? Because we wanted to deploy more and more sensor for testing, more for a scientific approach and being able to have a site where we can perform more development and testing.
25:03
If I will go in production, I will try to miniaturize the system but in general, there is no limitation for the replication in other ecosystem and tropical lagoon, probably you will have more production of algae, so the maintenance of the sensor
25:21
should be more often with respect to the alpine lake but in general, there is no restriction. The sensor are widely available, software and everything is there. Okay, we have some questions here. Is the public assessed of the data?
25:43
In this case, you present. Okay, all the data, not so far. We are implementing a web interface for the users because such a kind of data are quite, let's say academic, so are available
26:02
for the net work of academic person and experts but the administration didn't want to open up everything to the public because this information may be misunderstood and can generate some alarmism,
26:20
for example, for high production of algae in the lakes or things like that. So we are developing a web interface to show general parameters which are easy to understand by citizens but the data are available on request.
26:41
Okay, the next question is, how you transmit the data from the fields? MBIoT, the answer is. We tested with LoRa, we started with LoRa protocol but then soon we realized that the bandwidth is too low to transmit such amount of data.
27:02
So we migrated to MBIoT now. Okay, in the last questions is, what kind of the signal you get for this in source? Mostly 05 volt digital output, yeah.
27:22
Okay, so let me see if you have another questions. No, oh, okay. Thank you, Maximiliano, for your presentation. We have a lot of, we have some of hello for your friends here.
27:42
So your time is up. I want to make a short pause for drink of water or drink of tea. And we can go back with Willie Gautier in the next session of this morning.
28:04
Thank you, Maximiliano, we will see you soon in the social gathering in phosphate GIS mapping. Okay, thank you so much, bye-bye. Bye.