We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

A standardised approach for serving environmental monitoring data compliant with OGC APIs

00:00

Formal Metadata

Title
A standardised approach for serving environmental monitoring data compliant with OGC APIs
Title of Series
Number of Parts
156
Author
License
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
Environmental monitoring is fundamental for addressing climate change. Environmental data, in particular air quality and meteorological parameters, are widely used for risk assessment, urban planning, and other studies regarding urban and rural environments. Finding open and good quality environmental data is a complex task, even though environmental and meteorological monitoring are considered some of INSPIRE's high value datasets. For this reason, having robust, open, and standardised services that can offer spatial data is of critical importance. A good example of open, high-quality, environmental and meteorological data is one of the Regional Agencies for Environmental Protection, ARPA Lombardia. This agency maintains the air quality and meteorological monitoring station networks of the region and serves a high volume of sensor observations. The Lombardy region is located in northern Italy and is considered its financial and industrial muscle. Due to its topology, during the colder months of the year, the pollution levels of the region increase, in particular the concentrations of particulate matter (PM10 and PM2.5), as portrayed in [1]. For this reason, having a well-established monitoring network is critical. The ARPA Lombardia monitoring network generates huge volumes of data, which is served through its catalogue and a set of services. It is possible to download air quality and meteorological observations, as well as the information of the monitoring stations. These data have been extensively used in research, in particular, in the study of air quality in the region [2][3]. ARPA Lombardia environmental monitoring data is served through the API (Application Programming Interface) of the Lombardy region, Open Data Lombardia. Although this service is highly functional, thoroughly documented and works correctly, we identified some limitations that could pose problems for researchers, especially in the field of geospatial information. This service has geospatial capabilities, such as the possibility to download data in GEOJSON format, however, it is not compliant with other open standards such as WFS, WMS, or OGC APIs, posing a problem of interoperability with other geoportals and catalogues that do follow these standards. Additionally, column names of the meteorological and air quality observations and meteorological stations datasets are not homogenised, making them not fully interoperable. Finally, the ARPA Lombardia services and data fields are only available in Italian, which also poses interoperability concerns. Highlighting the societal, environmental, and economic importance of this kind of information, in this work we present and document the implementation of a web API compliant with OGC API specifications for exposing the air quality and meteorological information from ARPA Lombardia. The data provided by ARPA Lombardia is shared under the licence CC0 1.0 Universal, meaning it is public domain. The developed API serves environmental monitoring data (both air quality and meteorological) in compliance with a set of OGC APIs. This API is capable of exposing data in different standardised formats, filtering by multiple fields and locations, and performing server-side processing of the observations. OGC APIs are modern standards for geospatial information. Although they are still in the adoption phase, many reference implementations are being developed, and governmental institutions are starting to adopt such standards [4][5]. They differ from widespread, older OGC standards such as WMS or WFS as they are based in JSON and OpenAPI, while older standards are based in XML. By implementing new OGC API applications we contribute to the spread of these standards in academic environments and their overall development. In particular, we developed a web service compliant with OGC API - Features for exposing the stations' information and locations as vector data, OGC API - Environmental Data Retrieval (EDR) for serving observations from environmental and meteorological stations, and OGC API - Processes to allow researchers to perform server-side processing to the underlying data, such as data cleansing, interpolations (e.g., conversion to a coverage format, obtaining data at an arbitrary point, etc.), and data aggregation (e.g., by day/month/year, by station). The API also complies with OpenAPI standards, HTTP content negotiation, and homogenised column names in English, to improve the usability of ARPA data by foreign researchers. This work is not intended to replace ARPA Lombardia API, but to provide an alternative for accessing the data and extend even further the possibilities of researchers with additional processing capabilities. Additionally, to further improve the ecosystem of OGC API implementations available and push forward those open standards in academic literature. The full paper will provide a system architectural description and the particular technologies used to develop the application, a comparison with ARPA Lombardia's current API, and a case study portraying the API capabilities for research. This work is of interest to the FOSS4G community and European regional agencies as it is an implementation of a promising open standard for environmental monitoring and sensor networks, as it is the OGC API - EDR, and as an example of the infrastructure and the capabilities that services for environmental monitoring should have.
Keywords
127
Open setRun-time systemInformationStandard deviationContext awarenessWorkstation <Musikinstrument>ImplementationElectronic data processingInformation retrievalProcess (computing)Software developerScripting languageBlock (periodic table)SynchronizationIdeal (ethics)Web pageAerodynamicsConfiguration spaceGraphical user interfaceQuery languageAttribute grammarDigital filterMetadataMeasurementInstance (computer science)Dependent and independent variablesUniform resource locatorLink (knot theory)Virtual machinePlot (narrative)Core dumpOpen sourceScripting languageOpen setResultantProcess (computing)Information retrievalState observerMetadataService (economics)Information privacyDependent and independent variablesRun-time systemSynchronizationMultiplication signQuery languageTable (information)Graphical user interfaceVisualization (computer graphics)Attribute grammarImplementationGoodness of fitHierarchyStudent's t-testConfiguration spaceBitInformationCASE <Informatik>Set (mathematics)Uniform resource locatorWorkstation <Musikinstrument>Observational studySelf-organizationInstance (computer science)Context awarenessDatabaseLink (knot theory)Software developerStandard deviationConnected spaceLevel (video gaming)Different (Kate Ryan album)Key (cryptography)Moment (mathematics)Web 2.0Computer architectureFunctional (mathematics)File formatSoftware testingDynamical systemCartesian coordinate systemData managementVolume (thermodynamics)BuildingPolygonSoftwareData storage deviceResponse time (technology)TimestampLecture/ConferenceComputer animationMeeting/Interview
Computer-assisted translationGeometryComputer animation
Transcript: English(auto-generated)
Okay, good morning everyone. My name is Juan Pablo Duque. I am a PhD student in Politecnico in Milano in Italy. Today I will present you the development that we have been doing in Politecnico about an OGC API for environmental data monitoring. It's a little bit different from what Daniele was doing. So without further ado, because I have very few time, first I will tell you a little bit why we're doing this.
Also some of the key features of this API, the case study that we've used to validate our implementation and what is the next steps of the research. So first of all, maybe you all know what OGC APIs are. They're basically modern standards for the old, say also legacy OGC standards. And they allow and enable fair access to data, so findability, accessibility, interoperability and reusability.
Also, in the context of environmental monitoring, we have that environmental monitoring stations and observations are considered high-value datasets by INSPIRE. So the countries and the municipalities must provide this data publicly.
But the organizations provide this data in non-standardized formats. So this makes this not fully interoperable with each other. For this, we implemented OGC API compliance service, so basically it's a web API, for serving this environmental data monitoring.
It's not for managing the network but for serving the data. And our use case was the implementation of this service for the Lombardi Region air quality monitoring service. Some of the features of Rose API, we have the implementation of the OGC API's common features, environmental data retrieval, EDR and processes.
This is the architecture of the application, if you want to discuss it later, we can discuss it. Some of the features is dynamic collections, so one of the key features of this is that you can create collections which are geospatial or non-geospatial layers on the fly using a graphical user interface.
We have also an automatic open API document, which means that it's automatically generated from the collections and the open API document is a document describing the API. We have also a processing engine, so you can write your own Python scripts to further enhance or create new functionalities to the data stored in Rose API.
And this also comes with asynchronous and synchronous processing capabilities. We have pagination of results to manage big data sets. We have dynamic configurations which you can, let's say, tweak certain things of the application, also from a graphical user interface. We have OGC API features and EDR endpoints for managing the visualization and the retrieval of information.
And this enables additional metadata and the spatial queries. Finally, we have the interconnected collections, which is something like hierarchical connections between collections. So basically you can have collections that point to other collections using an attribute.
For example, you can have sensor information in one collection and you can have the same observations in one collection. And you can, let's say, interconnect this to provide the location information to the observations without overloading the database. A little bit of the case study that we did. We used ARPA data, which is the environmental monitoring for the north of Italy, for Lombardy region.
And we use it a lot in Politecnico. But every time a new researcher comes, they need to relearn everything. Also the tables are sometimes not standardized, not homogenized. The names of the columns, they are only in Italian. And we have many international researchers. So every time we have to do this learning process.
So we replicated the ARPA Lombardy environmental monitoring air quality data sets within an instance of ROS API. A little bit of the data. We have our observations from 2010 to 2024. And around 1000 individual sensors. This is the map where you can see a little bit how spread they are.
We have more than 55 observations. We also wanted to test how good was our implementation with big data, with very big volume of data. And we have 7 pollutants. And we tested 4 aspects of our implementation with respect to the API that ARPA provides to access their data. Basically we have better results with querying response metadata and the processing capabilities.
Which is something ARPA does not have. But the response times, they are not very good at the moment. So there is a lot of improvement for that. Well, basically for the querying capabilities, we provide these special queries directly on the observations.
Which is something ARPA does not provide. We also have timestamps, we have a lot of metadata available, pagination and so on and so forth. We have the metadata, we have link building, that is something very important from the new ODC API standards. And the pagination and more information than the ARPA data, the ARPA API.
We also have processing capabilities. For example, we implemented this aggregation script to aggregate the data in monthly or weekly or daily observations. Because this is one hour data. And downloading one hour data, for example for one month or one year or three years for all the sensors can be very expensive and very heavy.
So we have this script for aggregating data and we have been using this in our research and it's working actually very good. And finally for the response times, sadly we are still working on this because there is a very big bottleneck with the technologies that we are using.
But there is a lot of improvement. And in fact we tested with respect to the ARPA API and it's not that bad. So there is a very good improvement in there. Well, the next steps, the improve of course, this implementation is still in a very early stage. We have to do testing and documentation.
Also we are open to contributions because this is launched as open source C software. We have our GitHub. Also you can tell us your use case to see if actually Rose API can help in what you are doing. And thank you and let me know if you have any comments later.