SensorThings API in practice: the AIR-BREAK project in Ferrara
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 351 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/69016 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Production Year | 2022 |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
FOSS4G Firenze 2022228 / 351
1
7
13
22
25
31
33
36
39
41
43
44
46
52
53
55
58
59
60
76
80
93
98
104
108
127
128
133
135
141
142
143
150
151
168
173
176
178
190
196
200
201
202
204
211
219
225
226
236
242
251
258
263
270
284
285
292
00:00
Control flowPresentation of a groupMereologyPhysical systemProjective planeControl flowGroup actionFocus (optics)Systems integratorXML
00:28
Mixed realityPresentation of a groupInternet der DingePoint (geometry)MereologyPlug-in (computing)Software developerImplementationStandard deviation
01:18
AreaCircleZoom lensComputer animation
01:34
DialectCircleArea
01:50
Reduction of orderFinitary relationPhase transitionComputer networkControl flowMathematical analysisSpeciesIntegrated development environmentSystem administratorEvent horizonProjective planeMobile WebSoftwareNetwork topologyIntegrated development environmentComputer programmingGroup actionDialectWorkstation <Musikinstrument>Reduction of orderOrder (biology)AreaMereologyComputer animation
02:51
Presentation of a groupGoogolComputer animation
03:17
Computer wormPOKEMereologyDialectAuthorizationWeb portalMappingFunctional (mathematics)Integrated development environmentComputer animation
03:54
StatisticsPressure volume diagramWeb pageExecutive information systemDynamic random-access memoryOrder of magnitudeCohen's kappaCommon Language InfrastructureWeb portalSimilarity (geometry)MappingOpen setComputer animation
04:13
Digital filterOperator (mathematics)Self-organizationCone penetration testLogic gatePlastikkarteHypermediaNon-standard analysisSimilarity (geometry)Level (video gaming)DialectState observerMappingWorkstation <Musikinstrument>Open setComputer animation
04:33
Temporal logicContent (media)Cone penetration testWorkstation <Musikinstrument>Data structureVolumeStandard deviationDigital filterCodierung <Programmierung>User profileServer (computing)Library catalogTransformation (genetics)Product (business)InternetworkingSystem programmingMereologyEvent horizonElement (mathematics)Open setSelf-organizationBit rateWeb portalProjective planeMereologyOrder (biology)Standard deviationProfil (magazine)DialectStorage area networkOpen setPoint (geometry)WordLevel (video gaming)Uniform resource locatorComputer fileSocial classObservational studySet (mathematics)Web portalData streamPolygonBulletin board systemData modelLine (geometry)Cross-platformType theoryTime seriesData storage deviceDomain nameView (database)Object (grammar)Presentation of a groupTask (computing)Multiplication signTraffic reportingMappingWorkstation <Musikinstrument>Core dumpSoftwareMultiplicationState observerDiagramTemporal logicData structureInternet der DingeTrigonometric functionsDifferent (Kate Ryan album)2 (number)Computing platformCoordinate systemComputer animationSource codeXML
09:41
Wrapper (data mining)NavigationLocal GroupProcess (computing)Parameter (computer programming)DataflowServer (computing)DialectLevel (video gaming)CASE <Informatik>Military baseComputer animation
10:20
Server (computing)ImplementationOpen sourceSource codeMereologyInternetworkingEmailUsabilityView (database)Differential (mechanical device)File viewerControl flowIntegrated development environmentFunctional (mathematics)Domain nameVideoconferencingLocal ringBitComputer fileWeb 2.0Task (computing)Different (Kate Ryan album)Parameter (computer programming)Electronic mailing listCASE <Informatik>MereologyAssociative propertyUniform resource locatorImplementationServer (computing)Web pageStandard deviationWeb applicationWorkstation <Musikinstrument>Open sourceProjective planeWeb browserInstance (computer science)Arithmetic meanWebsiteExecution unitData streamLibrary catalogSystem administratorState observerMultiplication signSource codeProfil (magazine)DialectMappingStudent's t-testEqualiser (mathematics)Maß <Mathematik>Statistical dispersionMoment (mathematics)Computer animation
14:30
Instance (computer science)Interactive televisionSinc functionSlide ruleServer (computing)GeometryLibrary (computing)Order (biology)Polygon meshMobile appComputer animation
14:49
Plug-in (computing)FAQElement (mathematics)Random matrixInformationRaster graphicsProcess (computing)Attribute grammarTotal S.A.RotationCoordinate systemVideoconferencingPhysical systemStokes' theoremServer (computing)Quality of serviceMIDIBriefträgerproblemDecimalDeterminantDot productMiniDiscStack (abstract data type)MalwareExt functorWorkstation <Musikinstrument>Proper mapTime zoneRemote Access ServiceControl flowStatisticsVideoconferencingCategory of beingSoftware developerServer (computing)Electronic mailing listSystem callCASE <Informatik>Latent heatProjective planeInstance (computer science)Moment (mathematics)Client (computing)Type theoryStatistical dispersionUniform resource locatorFunctional (mathematics)Streaming mediaSet (mathematics)2 (number)MultiplicationDynamical systemWeb 2.0Extension (kinesiology)Multiplication signMathematicsStandard deviationPlug-in (computing)DiagramMereologyPopup-FensterBitPresentation of a groupWorkstation <Musikinstrument>Group actionInterface (computing)Computer animationDiagramProgram flowchart
19:57
Address spaceEmailQR codeComputer animation
Transcript: English(auto-generated)
00:00
Hello everybody, my name is Pier Giorgio Cipriano from DataNext Italy. DataNext is an IT system integrator, a large company, Italian based, part of the data group. And this presentation is about one project called Airbrake, so the focus, the topic is on air quality,
00:22
but not limited to air quality only because the second issue of the presentation is related to the lack of interoperability when talking about and using data coming from IoT devices and sensors.
00:43
That's why I will briefly introduce the OGC Sense of Things API standards. And I will conclude with the last part of the activity that we have undertaken in the recent months,
01:07
that is the drafting implementation, the development of QGIS plug-in for connecting two sensor things, API endpoints. Let's start from here, so we're talking about air quality and the red area is the so-called Povale, the Pianora Padana,
01:29
that is one of the most air polluted areas we have in Europe. Zooming in, I highlighted with this red circle the area of Ferrara.
01:41
Ferrara is a medium sized city located in the Milia Romagna region, but very close to the borders with the Veneto region and also with Lombardia. In Ferrara, 20 months ago, at the beginning of last year, January 21, we started a project co-founded by the European OUYA program
02:04
aiming at improving the network of sensors for measuring air quality in the city, in the urban area, and collecting more data than those that are already available coming from authoritative official stations managed by the regional agency for environmental protection.
02:28
And of course, the project itself is not limited to the soft part, but is also considering specific hard actions like planting new trees or incentivating sustainable mobilities in order to reduce air pollution in the city.
02:52
This question is mainly addressed to Italian people present here, but of course extended also to not Italians.
03:02
I think that there are some Italians over there and my question is, have you ever tried to search for air quality data about Povale? And we did it. We started from a simple and stupid search in Google and we
03:21
landed, of course, on regional portals provided by regional authorities and regional agency for environmental protection. Here we are in Piedmonte. We considered also Piedmonte even though Piedmonte is quite far away from Ferrara, but it is part of the Pienuor Padana, part of the Povale.
03:45
And we discovered these beautiful maps and also capabilities, functionalities for downloading data. Also Lombardia is offering similar features, similar capabilities, so forecasting maps and also data
04:06
that can be downloaded thanks to the open data portals of the original Lombardia. Similar situation in Emilia-Romagna too with maps about observations, not forecast but observations.
04:22
And daily and hourly data about air quality at station level coming from the open data portal of the regional environmental protection agency. Veneto, the last one. In Veneto we only have this type of maps. We have daily bulletins,
04:43
daily reports at provincial level and we don't have any data available for download, at least here. Please keep in mind this posted because we will recall it later on in the presentation. So, summarizing. In Piedmonte we have daily maps for both observations and forecast.
05:05
Data can be downloaded but only station by station and only pollutant by pollutant. There aren't any APIs available or at least they are not searchable, they are not described.
05:20
And data are provided in CSV files with this structure. While in Lombardia, daily maps are only for forecasts, data can be downloaded for all stations and for all pollutants. APIs do exist but they are not standard. They are proprietary APIs since
05:45
the regional open data portal is based on Sohrata solution, that is proprietary software. And CSV encoded files are having this structure quite different from the previous one. In Emilia-Romagna, daily maps and charts as well only for observations, data can be
06:06
downloaded for all stations and for all pollutants, but limited to the last two months. Again, also in Emilia-Romagna there are APIs available, but again, not standards based on a second.
06:24
And the structure of data is similar to Lombardia and different from Piedmonte. So, in two words, no interoperability. And we definitely need IoT interoperability. That's why we propose to project partners, in particular to the municipality of Ferrara that is leading, that is coordinating
06:46
the outbreak project, to think of a solution implementing an OGC standard, that is the OGC Sense of Things API. For those that are not familiar with the OGC Sense of Things API, it was published a few years ago, I think in 2017 or something.
07:10
And it is a platform independent, reality free, and most importantly, at least from my point of view, it is domain agnostic.
07:22
So, it is not related to sensors for a specific domain. It can be applied to all types of temporal data of time series. And the standard is divided in two parts, the sensing profile and the tasking profiles.
07:43
This is the quite simple data model with, I hope that also from the back, this UML diagram can be viewed, can be understood. There are some important concepts, of course, like the thing that can be related to one or more location.
08:10
The location class is the one having the references to the coordinates. Location can be expressed by points or other types of features, like also lines and polygons.
08:31
Things can be also related to historical location. This is interesting also for moving objects. And the core, the main concept is represented by data stream.
08:45
In the standard, two different types of data streams are foreseen. The simple data stream and multi-data stream, and we will see a practical example later on.
09:03
So, what we did, since the lack of interoperability existing at regional level, at least in Italy, we decided to implement a set of data ingestions.
09:20
That ingestion flows, so collecting and ingesting hourly and daily data from regional open data portals, by connecting through Socrata and Secan APIs, in order to store data into a sensor things API compliant solution.
09:40
And for doing this, we decided to use Apache NiFi as a server-side engine to manage and orchestrate flows. Flows that are, in the case of data about our quality coming from regional agencies,
10:01
are running every day, because data are not available at a more frequent level. But with other data, we have more frequent flows, hourly based or sub-hourly based.
10:20
And for the sensor things compliant solution, we chose Frost, implemented by Fraunhofer Institute. Thanks to the brilliant idea of Dr. Hilke van der Schaff. It is the first complete and open source implementation of this standard, in particular of part 1,
10:44
but is evolving also for part 2 on tasking. At the moment, we have data coming from these sources. So we have data coming about a little bit less than 200 regional and official air quality stations,
11:04
plus new municipal air quality stations, plus bike counters and so on. So we are applying this standard not only to air quality, but also to other domain. In the fall, in autumn time, we will also start collecting data from other new 45 low-cost air quality stations
11:26
that will be provided by the municipality, citizens, schools and local associations. So the Frost instance has been deployed on the server farm of the municipality of Ferrara.
11:40
Here we have the base URL from a simple browser. It can be accessed and we can navigate the single concepts of the standard that we saw before. So we can access the list of locations and based on a specific location,
12:08
we may have the things related to that location and then we can go through the data streams, so have the list of all possible data streams. In the case of air quality, data streams are corresponding, for instance, to the variables,
12:25
to the parameters that are measured, like ozone, PM10 and so on. And then, of course, based on the data stream, we get observations. The first implementation that we started and we concluded at the end of last year
12:43
was on top of the Web GIS portal of the municipality of Ferrara. So we extended their own Web GIS solution by adding functionalities for interacting with the Sense of Things API endpoints.
13:00
I will stop now to call it Sense of Things API because it is too long. I will call it just STA. So when clicking on a specific location, you can see data as a simple table or as a chart like this
13:21
and from here you can download data as local CSV file. Of course, we added also functionalities for the municipality to configure other endpoints when available in this data catalog that is an admin user web application
13:44
for configuring maps, credentials, profiles, users and functionalities and so on. Then we focused on the fact that air quality is not so simple domain.
14:03
When talking about air quality data, we have to consider that we have different unit of measures, we have different meanings and what we wanted was to somehow humanize things. Air quality data and we run some workshops with student, teachers and so on.
14:23
This video is showing a web page hosted by the project website that will be publicly available in autumn. I jump these slides. One of the ways of presenting air quality, for instance, is with this mesh app.
14:41
We use the JavaScript libraries interactive with the GeoServer in order to get hourly updated data. And then we started QGIS. Since I've got five minutes less, I start this video and I can talk on top of them.
15:00
So we decided to add a capability that apparently QGIS is not having at the moment that is the ability to access dynamic data about sensors. And, of course, we focused on endpoints that are compliant to sensor things API, to SDA standard.
15:25
So here we are connecting to the municipality of Ferrara endpoint. Here I add a layer showing the location that will appear in some seconds
15:41
about the air quality stations, but then I also add the bike counters. So here it should appear two layers. And just by clicking on one location using the second functionality of the plug-in,
16:08
we can open a pop-up window showing the list of properties that are available for that specific location.
16:20
So in the case of air quality stations, the list of variables that are measured, like NOx or CO or even temperature and so on, then we can select one of them and have data like in the Web GIS client that I showed before.
16:40
We can change the time extent and download locally the data set in CSV, always interacting, so performing some request to the SDA endpoints. I jump a little bit, I skip this part because it is about bikes, bike transit.
17:12
Okay, I stop it a little bit here. You remember before the post-it, I said that in Veneto, Reggione Veneto or ARPA Veneto, that is the regional authority,
17:23
are not publishing data about air quality in their own portal. But when we started to develop the plug-in two months ago, we discovered an interesting endpoint by the Fraunhofer Institute
17:41
that is collecting all air quality data for all stations present in Europe. So here I can select and filter stations based on their own properties. For instance, I can only select stations from a specific country or by type,
18:05
like rural stations or urban stations. And here we have stations also about Reggione Veneto.
18:21
And again, when interacting with the second functionality by clicking on the location, we can explore data, we can navigate data and also for multi-data streams,
18:40
I skip here, we can have this kind of multi-diagrams, of course, if available at server side. Okay, I want to close the presentation, now the video. Okay, so the plug-in is already available on GitHub.
19:04
Okay, I'm finishing. You have also some instructions and more important, the list of publicly available endpoints providing SDA interface. And this is the last one. What we are in mind is to complete the development that we started two months ago,
19:30
also exploiting a new project called usage. So we are asking for call for contributions. We are searching for external developers, so who is interested, please contact me.
19:44
And Geocat already confirmed their interest, also because we are partnering together in usage project. And the idea is to have an open collaborative project and possibly have an official QGIS plug-in next year. I thank you very much.
20:00
Here you can find my email address if you want to contact me or if you want to scan the QR code. Okay.