We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

A QGIS plugin for local weather sensor data

00:00

Formal Metadata

Title
A QGIS plugin for local weather sensor data
Title of Series
Number of Parts
266
Author
License
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
Ground-based weather sensor networks are essential in monitoring local weather patterns and climate. Integration of such data into GIS environments is critical to supporting manifold applications including urban planning, public health studies, and weather forecasting. These networks use scattered geolocalized sensors to measure multiple atmospheric variables (e.g. air temperature, wind speed, precipitations). Often, data is distributed online by network managers which can be either local/national authorities, private companies, or volunteers. Due to the diversity of data providers, both formats and access patterns of meteorological sensor data are heterogeneous and the preprocessing tasks (e.g. temporal aggregations, spatial filtering) are generally time-consuming. Given the above and to increase end-users exploitation of such sensor data, we present the development of an experimental QGIS plugin facilitating access and preprocessing of openly available data from ground-based sensor networks and enabling their direct use in QGIS. The plugin is designed to implement REST APIs connections and HTTP requests to download data. A user interface allows for selecting time intervals and types of observation to be downloaded. Once data is retrieved, the plugin provides options for filtering, outliers removal, time aggregation with summary statistics as well as observation mapping into a standard GIS layer. These functionalities are only partially available in similar existing QGIS plugins. The plugin leverages FOSS Python libraries for data handling including Pandas. The Dask parallel computing library is also exploited to speed up I/O operations on raw data. The current version of the plugin is developed to retrieve and process weather sensor data provided by the Environmental Protection Agency of Lombardy Region (ARPA Lombardia), Northern Italy. The data retrieval is based on the Sodapy Python library, a Python client for the Socrata Open Data API. The plugin's work-in-progress source code is available at (https://github.com/gisgeolab/ARPA_Weather_plugin) released under MIT license. The plugin is being developed within the LCZ-ODC project (agreement n. 2022-30-HH.0) funded by Italian Space Agency (ASI), which aims to identify Local Climate Zones within the Metropolitan City of Milan. Ongoing work includes the extension of the plugin functionalities to incorporate additional data providers, starting from other Italian regional ARPAs. The goal of this project is to provide a reproducible framework to access and handle weather data into QGIS, thus extending the capability of the software to support a wider range of practitioners and applications.
Arrow of timeGoodness of fitPlug-in (computing)Projective planeLocal ringSpacetimeComputer animation
Local ringProjective planeSpacetimeSoftwareState observerIntegrated development environmentLecture/ConferenceMeeting/Interview
Computer networkFile formatForm (programming)Plug-in (computing)Repository (publishing)Information retrievalProcess (computing)Interface (computing)Library (computing)Open setWorkstation <Musikinstrument>Temporal logicDirection (geometry)FrequencyOpen sourceAttribute grammarWater vaporOpticsRepository (publishing)Attribute grammarSeries (mathematics)Interface (computing)Client (computing)Prime idealCountingMaxima and minimaComputer fileLevel (video gaming)CASE <Informatik>Task (computing)Time seriesRange (statistics)Power (physics)Software frameworkFile formatFunction (mathematics)File archiverSoftwareOpen sourceMultiplicationInformationPlug-in (computing)Multiplication signState observerOpen setDialectRight angleBitParameter (computer programming)FacebookNeuroinformatikDot productLageparameterArithmetic meanLibrary (computing)Selectivity (electronic)Strategy gameTable (information)Data structureSet (mathematics)Parallel computingData managementComputer animation
Computer animation
Transcript: English(auto-generated)
So, good afternoon everyone. I'm Emanuele Cappizzi, I'm from Politecnico di Mirano and today I'm going to present an EJAS plugin for local weather sensor data called ARPA Weather Plugin.
This is a project developed between the Italian Space Agency and Politecnico di Mirano. So, this plugin is able to facilitate the assets to ARPA Lombardia, which is an environmental protection agency in Northern Italy, as shown in the picture.
And the goal of this project is to analyze local climate zones in metropolitan city of Milan and correlate these local climate zones with air temperature that comes from both ground sensor data, which is the ARPA Lombardia network, and geospatial data and data observation technologies.
And this project is, as I said, developed together with the Italian Space Agency. The problem with the sensor data is that usually they come from different sources, so for example, there are air quality data, weather data in national networks, regional networks,
so it's like a complicated situation. And there are also heterogeneous data formats, for example, APIs, CSV files, also sometimes PDFs are used. And it's time consuming to data cleaning of these data sets that are sometimes pretty big
and it's a lot of processing tasks. So, our plugin manages API when it's possible and they are available. In our case, the last month data of meteorological sensor are provided with API, while CSV files are provided for archive data, so years in the past are available in CSV files.
And everything is performed with user-friendly interface. And we are using Pandas, which are efficient data frame handling library, Dask, which allows parallel computing and so efficiently manage big CSV files,
and Sotapai, which is a Python client for soaking up open data API, which is used by ARPA Lombardia. The repository is available on GitHub. And this is the structure of the plugin where it is possible to select the data source,
the range time, the type of sensor, the outlier removal strategy, and the province selection. And it is possible to start multi-point JS layer, for example, containing summary statistics such as mean, maximum, and the count of observation.
And it is possible to export the full-time series and sensor information, and also combining them with multiple output with time series and sensor information as well. So, this is the plugin that we developed. It is done to facilitate users to access to this data.
And it is possible to select the data source. So, from the APIs, the files are automatically downloaded and processed. And it is possible to select parameters like the year, the time range, and it is possible to export the CSV file containing time series
and all the necessary information that the user might need. On the right side, is it possible to see the attribute table and an example of a map that is automatically produced by the plugin where you can see the red dots are the temperature sensor already pre-processed.
And on the right button side, is it possible to see the time series that is provided as output of the plugin. So, for me, it's everything, and these are the contacts, and thank you for your attention.