We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

"Fast Big Data?" A High-Performance System for Creating Global Satellite Image Time Series

Formale Metadaten

Titel
"Fast Big Data?" A High-Performance System for Creating Global Satellite Image Time Series
Serientitel
Anzahl der Teile
188
Autor
Lizenz
CC-Namensnennung 3.0 Deutschland:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
Identifikatoren
Herausgeber
Erscheinungsjahr
Sprache
Produzent
Produktionsjahr2014
ProduktionsortPortland, Oregon, United States of America

Inhaltliche Metadaten

Fachgebiet
Genre
Abstract
Description:We describe a system that transforms sequences of MODIS images covering the entire Earth into time-optimized data cubes to provide rapid access to time series data for various applications.Abstract:Satellite time series data are key to global change monitoring related to climate and land cover change. Various research and operational applications such as crop monitoring and fire history analysis rely on rapid access to extended, hyper-temporal time series data. However, converting large volumes of spatial data into time series and storing it efficiently is a challenging task. In order to solve this Big Data problem, CSIR has developed a system which is capable of automated downloading and processing of several terabytes of MODIS data into time-optimized "data cubes." This time series data is instantly accessible via a variety of applications, including a mobile app that analyzes and displays 14 years of vegetation activity and fire time series data for any location in the world. In this presentation we will describe the implementation of this system on a high-performance Storage Area Network (SAN) using open source software including GDAL and HDF5. We discuss how to optimally store time series data within HDF cubes, the hardware requirements of working with data at this scale as well as several challenges encountered. These include writing high-performance processing code, updating data cubes efficiently and working with HDF data in a multi-threaded environment. We conclude by showing visualizations of our vegetation and burned area time series data in QGIS, web apps, and mobile apps.
Schlagwörter