"Fast Big Data?" A High-Performance System for Creating Global Satellite Image Time Series

Zitierlink des Filmsegments
Embed Code

Für dieses Video liegen keine automatischen Analyseergebnisse vor.

Analyseergebnisse werden nur für Videos aus Technik, Architektur, Chemie, Informatik, Mathematik und Physik erstellt, bei denen dies rechtlich zulässig ist.


Formale Metadaten

Titel "Fast Big Data?" A High-Performance System for Creating Global Satellite Image Time Series
Serientitel FOSS4G 2014 Portland
Autor Swanepoel, Derick
Lizenz CC-Namensnennung - keine kommerzielle Nutzung - keine Bearbeitung 3.0 Unported:
Sie dürfen das Werk bzw. den Inhalt in unveränderter Form zu jedem legalen und nicht-kommerziellen Zweck nutzen, vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
DOI 10.5446/31627
Herausgeber FOSS4G
Open Source Geospatial Foundation (OSGeo)
Erscheinungsjahr 2014
Sprache Englisch
Produzent FOSS4G
Open Source Geospatial Foundation (OSGeo)
Produktionsjahr 2014
Produktionsort Portland, Oregon, United States of America

Inhaltliche Metadaten

Fachgebiet Informatik
Abstract Description:We describe a system that transforms sequences of MODIS images covering the entire Earth into time-optimized data cubes to provide rapid access to time series data for various applications.Abstract:Satellite time series data are key to global change monitoring related to climate and land cover change. Various research and operational applications such as crop monitoring and fire history analysis rely on rapid access to extended, hyper-temporal time series data. However, converting large volumes of spatial data into time series and storing it efficiently is a challenging task. In order to solve this Big Data problem, CSIR has developed a system which is capable of automated downloading and processing of several terabytes of MODIS data into time-optimized "data cubes." This time series data is instantly accessible via a variety of applications, including a mobile app that analyzes and displays 14 years of vegetation activity and fire time series data for any location in the world. In this presentation we will describe the implementation of this system on a high-performance Storage Area Network (SAN) using open source software including GDAL and HDF5. We discuss how to optimally store time series data within HDF cubes, the hardware requirements of working with data at this scale as well as several challenges encountered. These include writing high-performance processing code, updating data cubes efficiently and working with HDF data in a multi-threaded environment. We conclude by showing visualizations of our vegetation and burned area time series data in QGIS, web apps, and mobile apps.
Schlagwörter Big data
time series
Storage Area Network
mobile app

Ähnliche Filme