We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Scientific Data in the Cloud - Oct 18

Formal Metadata

Title
Scientific Data in the Cloud - Oct 18
Subtitle
HDF5 in the Cloud
Title of Series
Number of Parts
19
Author
License
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
Processing Structured Scientific Data in Cloud The HDF5 file format has been used extensively in the HPC community for the storage of scientific data (e.g. multi-dimensional arrays). Unfortunately, the traditional HDF5 library doesn't work so well for applications running in the cloud. To address this, we've developed a service based implementation of HDF5, HDF Kita. Kita utilizes object based storage (e.g. AWS S3) and runs as a cluster of Docker Containers. In combination with the service, JupyterHub enables users to easily run notebooks in the cloud that can use an unlimited amount of data and take advantage of the parallelization capabilities of the Kita Server.