An open early-warning system prototype to help in management and study algal blooms on Lake Lugano
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Title of Series | ||
Number of Parts | 156 | |
Author | ||
License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
Identifiers | 10.5446/68429 (DOI) | |
Publisher | ||
Release Date | ||
Language |
Content Metadata
Subject Area | ||
Genre | ||
Abstract |
| |
Keywords |
FOSS4G Europe 2024 Tartu12 / 156
6
33
35
53
55
59
61
67
70
87
97
99
102
103
104
105
107
111
121
122
123
124
125
126
127
128
134
144
150
151
155
00:00
Gamma functionIdeal (ethics)Complex analysisSound effectOrder (biology)Projective planeWater vaporEvent horizonGroup actionStructural loadSound effectObservational studyCausalityDivisorSanitary sewerDegree (graph theory)Lecture/ConferenceComputer animation
01:45
AreaTable (information)Total S.A.AreaBridging (networking)Maxima and minimaObservational studySpeech synthesis
02:14
Musical ensembleTime zoneMultiplication signEvent horizonContext awarenessWater vaporComputer animation
03:12
Group actionPower (physics)Module (mathematics)Data transmissionExecution unitMicrocontrollerMetreAnalog-to-digital converterModule (mathematics)Total S.A.Standard deviationClient (computing)Service-oriented architectureServer (computing)Computer-assisted translationOpen sourceLocal ringPasswordBit rateSerial portSharewareDefault (computer science)Integrated development environmentSoftwareStatisticsConfiguration spacePrototypePressure volume diagramData managementDatabaseObject (grammar)Service (economics)Interface (computing)AuthenticationOpen setGraph (mathematics)Data loggerPower (physics)Module (mathematics)MereologyPhysical systemWater vaporService (economics)Module (mathematics)Stagnation pointData transmissionStorage area networkInsertion lossFlagElement (mathematics)Communications protocolAuthenticationRepository (publishing)ImplementationExecution unitClient (computing)Software testingAsynchronous Transfer ModeConnected spaceGoodness of fitDataflowCuboidDynamical systemWorkstation <Musikinstrument>Scripting languageGroup actionSlide ruleGame controllerOpen sourceSoftwareVoltmeterServer (computing)Data managementAreaRadical (chemistry)Feasibility studySound effectSurfaceBus (computing)Integrated development environmentMicrocontrollerData recoveryComputer architectureLoop (music)Reading (process)FluxComputer animationLecture/ConferenceMeeting/Interview
09:23
Pressure volume diagramObject (grammar)Service (economics)Open setStandard deviationGraph (mathematics)AuthenticationData managementDatabaseSuite (music)SoftwareArchitectureRevision controlCategory of beingWorkstation <Musikinstrument>Digital signalIntegrated development environmentCorrespondence (mathematics)Raw image formatDistribution (mathematics)InterpolationLinear regressionData modelMathematical analysisSystem identificationWebsiteThresholding (image processing)Vector potentialLimit (category theory)AverageSeries (mathematics)Process (computing)Rule of inferenceChainSoftwareServer (computing)SpeciesOrder (biology)Service (economics)Electronic mailing listThresholding (image processing)Presentation of a groupProcedural programmingSlide ruleState observerPhysical systemFlagConcentricSet (mathematics)ResultantString (computer science)Projective planeIntegrated development environmentSocial classParameter (computer programming)Rule of inferenceTraffic reportingFrequencyMathematical analysisGraph (mathematics)Multiplication signProcess (computing)Generic programmingAreaPower (physics)Similarity (geometry)Repository (publishing)Computing platformPhase transitionBranch (computer science)Computer architectureComputer fileTime seriesComputer animationLecture/Conference
15:33
8 (number)Social classMedical imagingMultiplication signOffice suiteWater vaporComputer animation
16:07
Event horizonPhysical systemComplex analysisStandard deviationControl flowSoftware maintenanceMathematical analysisSatelliteComputer-generated imageryMedical imagingKey (cryptography)Standard deviationSocial classConcentricSet (mathematics)Structural loadIntegrated development environmentMathematical analysisEvent horizonMultiplication signSatelliteBitGame controllerData qualitySound effectGroup actionOrder (biology)SurfaceInternet der DingeComplex systemMeasurementPoint (geometry)ResultantData managementClient (computing)Limit (category theory)Physical systemProjective planeValidity (statistics)View (database)Software maintenanceOpen sourceData loggerSoftware development kitEmailParameter (computer programming)Computer animationLecture/Conference
20:40
FrequencyServer (computing)Slide ruleSoftwareDampingSoftware developerComputer animationDiagram
21:17
PrototypeLevel (video gaming)Event horizonNumberImplementationSlide ruleMathematical analysisEvent horizonOpen sourceSoftware testingValidity (statistics)Process (computing)Integrated development environmentOpen setComputer animationLecture/ConferenceMeeting/Interview
22:52
FrequencyCalculus of variationsServer (computing)System programmingReal-time operating systemBitService (economics)SoftwareComplex analysisCommunications protocolGroup actionEmailAxiom of choiceSet (mathematics)Point (geometry)Physical systemDifferent (Kate Ryan album)Endliche ModelltheorieLecture/ConferenceComputer animation
25:14
BuildingComputerProjective planeOpen setOpen sourceStrategy gameOrder (biology)Computer hardwareSoftwarePhysical systemTask (computing)Lecture/ConferenceMeeting/Interview
26:48
Message passingOpen sourceServer (computing)Different (Kate Ryan album)Service (economics)State observerComputer architectureInformation systemsLecture/ConferenceMeeting/Interview
27:49
SoftwareSoftware developerLecture/Conference
28:28
GEDCOMComputer-assisted translationLecture/ConferenceComputer animation
Transcript: English(auto-generated)
00:00
Okay, thank you. Good morning. Yes, this is a small project that we run in order to face these issues that is affecting Lake Lugano. First of all, I would like to introduce what algal blooms are. Maybe it's better, because algal blooms are basically an abnormal growth of algae in lakes.
00:23
There are different causes, like an excess of phosphorus and nitrogen, so an excess of nutrients coming from farming, coming from agriculture, using fertilizer and so on. Then there are also industries that can take water from lakes and ingest some chemicals.
00:45
Then there are cities. We have to think about residential sewers. And then there are also heavy rain events, which can increase the loads of nutrients inside the lake. Another factor is the increment of the water temperature. We are in a world affected by climate changes, so also the water temperature is increasing.
01:09
You have to think that last summer, on Lake Lugano, we reached a temperature of around 29, and some channel bacteria, which can produce channel toxins, can grow with a temperature above 24 degrees.
01:29
Which are the effects of these algal blooms? They have effects on human health. They have effects on the ecosystem, of economics. So these are the phenomena that we have to face for these reasons.
01:46
The study area is Lake Lugano. We can speak of Lake Lugano as two lakes, because there is this moranic deposit in Melide. Upon that, it has been built the bridge. The north basin has a maximum depth of around 300 meters.
02:06
The south basin has a maximum depth of 90 meters. And they have quite different behavior. This is the situation during last summer, in some leaders. It was quite critical. It was the first time that we saw huge events.
02:31
Many newspapers speak about this. Either local or national newspapers raise the awareness of citizens.
02:43
Also because the lake is a very important source for economics. We have to think that here there are many companies for navigation, there are many leaders. Also the water of the lake is used for drinkable usage. In fact there are captions at
03:03
30-40 meters depth to catch water from the lake and get them to the residential houses. Lake Lugano of course is affected by these two elements, climate change
03:21
plus human activities, which are causing unknown dynamics, which have to be studied. And so also there is a need of monitoring. The lake is a recovery status because from the 70s there are many mitigation actions that recover its health.
03:40
But now we are reached somehow a stagnation, probably due to the climate change effects. The rise of the temperature are doing negative feedback on this recovery status. So what we have done, basically we have built a simple device composed by a control unit and a sensor.
04:05
We put it in the Arriba San Vitale, which is one of the most affected areas from this phenomenon. And the sensor, we put it in the surface. How is this built, this system? Basically it is composed by these elements.
04:29
First of all a Trilux sensor from Chelsea Technologies to measure algal pigments. It is a 40 meter with a cost of around 2,600 francs.
04:42
Then there is a microcontroller unit, we use Raspberry Pi. Pre-COVID it has a cost of around 40, but now I think it's 60-70, I don't know. Then I put also the cable, because you can see that it is quite expensive, 400 francs for a cable of 5 meter.
05:03
Why we use it? Because it should be water resistant, it should be weather resistant since the system is working in a harsh environment. And so to produce good quality of data we need a good cable also. Then there is a power module just to convert the current from 12V to 5V with a cost of 7 francs.
05:29
Why we use it? Because 12V is usually the voltage that powers up professional sensors like the Trilux. But 5V is usually the voltage used by, for example, the Raspberry Pi from the microcontroller.
05:46
Then there is the power module for data transmission, we use MBIoT. We chose the module for the microarray company. Why? Because it's quite interesting, because this company uses the same bus for many modules.
06:05
So you can, what does it mean? It means it has a cost of 70 francs. What does it mean? It means that if you would like to change the module and use, for example, LoRa or Wi-Fi or other transmission mode,
06:20
you can just change the module but without changing the connections. Then other 350 francs for Michelinus like metal box, cables, terminator for a total budget of 3500 francs. This is quite expensive, it could be, but we have to think that if we would like to buy such a system from a company,
06:43
it can be also much more expensive. This slide is just to understand better how easily the data flows at the device side. Basically, we chose to put Raspberry Pi instead to have more power.
07:02
So we could install inside the ASOS software, which is a Python implementation of the SOS standard, instead to have a data logger that can speak SOS. So we would like to try if this does something feasible and also useful.
07:21
The data flow is like that, so every minute we have data read from the sensor, there is a quality control test, then data are inserted into a source with a flux that can see if the value is good or not.
07:42
Then every 10 minutes, everything at the device side, data are aggregated, other tests are run, and then the data are transmitted to the source at the server side through MBIUT and MQTT protocol.
08:01
Then there is an orchestrator, which is sub-Sky2, a topic, so the orchestrator can ingest the data into the source. Here, just to say that the software that we've developed for the device is available on this repository, I think that maybe the most interesting part, if I can say this, it's inside the folder module,
08:27
where there are some drivers that we have written to speak with some commercial sensor, like loop stations, like Chelsea technology sensor, or for a pencil sensor, and so on.
08:40
Then there are other scripts, and if you are interested, you just have a look inside this repository. Here, there is the server side architecture. Everything is powered up by Docker Compose. There are different services. Basically, when a request comes to our servers,
09:02
there is an orchestrator, which asks to the authentication server if the client is authorized or not. If yes, the request goes, for example, to its source, which is the general data management that we chose. Inside this architecture, we would like also to test Grafana.
09:22
Grafana, I think many of you know something about this software, and we use it for graph and dashboard generator, but also for the alarm notification service. We chose to have a look into Grafana inside this project because it is a small project,
09:41
so we would like to understand if it respects our requirements, so if also in a productive environment, it can be useful for us. Also, this architecture, there is a repository, and you can have a look.
10:01
It's public available. Please have a look into the prod branch, where you can power up the system with Docker Compose after maybe changing some parameters inside the .env files. And now some results.
10:21
We collected data for more than one year. The system is running still now, but in this presentation, I focused on just one year from the 2023 March till March 2024. We expected to collect 52,000 observations, but just 41 have been flagged as good data,
10:46
which constitutes 78% of the expected data set. This percentage seems quite low, but for us it's a quite interesting result because we are not measuring air temperature, which is a quite easy parameter to measure,
11:03
but we are measuring alcohol pigments concentration with a fluorimeter. So there are many biases that can happen, for example, a fish that can swim near the sensor, or the solar reflection, which goes inside the fluorimeter and creates other biases.
11:28
I put here this slide to speak also about the Simile project, where we firstly build something similar, but in a more complex way. In this project we just create a platform in the middle of the Lake Lugano
11:45
to measure many parameters with a chain of sensors. And with this project we just replicate this system, but in a smaller device.
12:01
If you would like to know more, please check our publication, our article, where there are more details. After collecting data, we started the analysis. We just defined some thresholds in order to understand if there is a bloom or not.
12:23
We identified just three classes. The first one is the monitoring, where the pico-chaining concentration should be more than 3.4 micrograms per liter, but also this value should be greater than the chlorophyll one.
12:40
In fact, sorry, but I can bring the previous slides. Here you can see that usually the chlorophyll is higher than the pico-chaining concentration expected during some periods where there are algal blooms.
13:02
Then there is in the monitoring phase, we just have to wait and monitor the situation in order to understand if there is the possibility for a bloom. Then there is the alert threshold, where the pico-chaining should be greater than 6.7 micrograms per liter.
13:22
This means that there is an abundant growth of channel bacteria, and so there is no need of parasite inspection. The last threshold is the BUN, where the pico-chaining concentration should be greater than 13.4. Here we have a critical situation. We have to take water samples
13:43
and understand if inside there are some particular species of channel bacteria, for example, Binkrochistes, which can produce channel toxins. We can do it with sensor. In this situation, we just have to take water samples
14:01
and analyze them in laboratory to understand if there are dangerous species or not. Now some slides just to understand better how the alarmers workflow works.
14:20
Basically, there is a string, as I said, every 10 or 15 minutes, I don't remember, that is sent to the server. The string is like this. There is the time, there is the value of the chlorophyll, a flag which identifies if this value is good or not, another value for the pico-chaining concentration with its flag,
14:44
and then the last one, which is the pico-heritrin, but we are not interested in this parameter. Then this value string goes inside the server into the tree-lux-rive procedure managed by the ESOS, and then there is a listen-notify system using the PostgreSQL,
15:06
which can fire an aggregation process, which will aggregate the 10 minutes time series to one hour time series. Such data are then collected inside a new procedure,
15:21
and this procedure will be used by Grafana in order to create alarms and create a dashboard after setting, of course, rules and notifications. This is the dashboard that we usually use to share the data with our stakeholders,
15:42
which are mainly the continental office for water supply, and here you can see that there are the three main classes, and you can see also in 2023 how many times, the image is not smooth, but how many times the values are in the bank classes.
16:09
Here there is a little bit of a more analysis. We just measure how many times the values are inside, for example, the monitoring classing,
16:21
and it happened 100 times, in the alert class it happened 900 times, in the bank class it happened 600 times. But basically we face just three main events,
16:43
one in June, one in August, which is the huge one, and then the third in September. Here there are also three different images for the three different events. Of course, it's true that we don't fire an alarm,
17:04
we don't send a notification every time that a value passed a class, but of course there is management in the notification in order to not stress the stakeholders with too many mails. And now some conclusions.
17:22
First of all, we tested the replicability of the system that we created in Simile, in the Simile project, and I can say that we have a successful result. We have also promising results, which can bring us in producing more sensors for the lake,
17:43
in maybe making some geospatial analysis, putting more sensors on the leaders. And also it's very useful to have sensors in order to validate satellite images, because we know that some parameters like chlorophyll a concentration
18:01
can be calculated from satellite images. But we need to validate this data. Then there is also the possibility to work with citizens in a science approach, because we see that there is a lot of interest in the people that are managing, for example, leaders and navigation company.
18:25
So they would like to have something like a kit or a system very similar to that one that I showed you, to understand better the status of the lake in order to better manage the client's requests. And then there are some negative points for our point of view.
18:45
First of all, Grafana in our situation, it's a very good tool to provide easily dashboard to create in an easy way notification. But when you have to create some complex system,
19:00
I see that it has some limits. Then, of course, we need a more suitable standard, because we try to put each source on the device to have a data logger which is SOS compliant, but SOS is very verbose,
19:21
so actually the device cannot speak SOS. It's impossible, because we have to reduce energy and reduce the load of the body. And then another thing that I would like to highlight is about the data quality controls and the maintenance activities,
19:42
which are the key to have a successful data set. We can put, with IoT, we can put in the environment many sensors, but this data should be of good quality, otherwise are useless. Why I speak about maintenance?
20:02
Because using fluorimeters in the lake, there are many issues. For example, the fouling effect, where biofilm can grow over the surface of the sensor, affecting the measurements. And also there is another effect called quenching,
20:23
where the solar light can reduce the concentration that is measured by this sensor. So there is the need of monitoring, and there is also the need of a technician that can look at the data and understand better what they really say.
20:42
Some future developments. I speak about notification. We tried Grafana, and we understand that we need something that is more suitable for IR requirements. So I don't want to speak about this in depth in these slides,
21:02
but making notification, we figure out that it's not so easy. So we are creating, from scratch, a software that will manage these notifications. And the last slide, I think, maybe,
21:23
is about our new implementation. I speak about Ist-SOS. I said that otherwise I can't speak really SOS. So we are really near to release our first release of Ist-STA,
21:41
which is applying to implementation of the Sensor Things API. We already passed all the tests from the Open Geospatial Consortium Validator, and we are just reviewing our environment, making it suitable for a new approach, event-based.
22:03
So everything will be an event which will fire services and which will fire notification, which will fire quality analysis on the data, which will fire processes, and so on. Thank you.
22:26
Thanks, Daniele. Any questions? I am Sultan from Saudi Arabia. Would you go to the last three slides,
22:41
the conclusion one? Sorry, but... Go more. Also one, four. Yes. When you sit here, Grafana is not enough to manage complex notification systems. Do you mean you need more sensors? Can you give an example?
23:01
Can you do in this system model, you are providing real-time monitoring, day and night, for the lake? Okay. I will try to understand what you... Because the sound is really bad here. Yeah, no. My precise point is real-time monitoring.
23:22
Okay. We use Grafana to fire... Real-time monitoring. Real-time notification. Real-time monitoring, yes. Yeah. We use Grafana to fire real-time notification, but we figure out that, okay, you can do it, of course. It's a very good software.
23:41
But when you have different kinds of stakeholders, you have many sensors on the network, maybe you will have to create groups of people that are interested in having some alerts on a set of sensors, instead of all the network, no? So we figure out that it's not so easy.
24:04
Also, it's good to have... Here, I put here something. When you fire an alert, then you have to wait instead of stressing the stakeholder with emails.
24:20
It's not an easy thing to elaborate, to create. So I think that Grafana can be used effectively in a team, maybe managing some small services. Okay, it will be a good choice.
24:41
Otherwise, I don't know. We tried the other microphone. Please try to keep it this far away from the mic. Okay, I'll try. You mentioned that there are some professional,
25:03
like off-the-shelf sensors that do similar things. Do you think it's hard to connect? Or do they speak similar protocols? Is there a way to integrate them? Because I imagine citizen science and building them, you might struggle from quality issues.
25:23
Is there a way to implement commercially available sensors in this as well? Yes, I think so. If you speak just about sensors, it's a really difficult task.
25:43
Because a fluorimeter, having a professional fluorimeter, it's not so easy to implement. But if you speak about all the systems, all the devices, we are writing right now a project in order to get some funds and make something that can be sold.
26:06
Of course, maintaining the open source approach. So what does it mean? Open hardware, releasing the hardware with an open hardware license, release the software with an open software license, and then try to get money from this system in other ways.
26:24
And we know there are other strategies. Other questions? I think that microphone works better.
26:46
If you consider using Apache Kafka for streaming data, what's the difference between a Kafka and Apache Kafka for streaming data? Could you repeat, please?
27:02
There's also an open source solution for streaming data called Apache Kafka. Have you considered using that for your streaming data? Yes, in the new architecture we will use Kafka. We are using Kafka to manage messages.
27:23
Anyone else? Quick one. It was nice to see the IST as the source server. So the sensor observation service, will you continue to keep using this?
27:43
Or are you considering something you also like, sensor things? Or is it good to stay with source? Yeah, yeah. If you would like to have interoperability, SOS, it's okay. So we will keep developing SOS, of course, to maintain it.
28:05
But new developments, I think, in the future will be on IST-STA. Because in IST-STA we bring there many features. So it is used for managing our meteorological network of the canton.
28:24
So the development of that software will continue, of course. Thanks a lot. We need to stop here. But of course it's reachable. Please approach me for questions. Thanks.