Scale your data, not your process: Welcome to the Blaze ecosystem
Formal Metadata
| Title | Scale your data, not your process: Welcome to the Blaze ecosystem |
|
| Title of Series | |
| Part Number | 163 |
| Number of Parts | 173 |
| Author | |
| License | You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license. |
| Identifiers | |
| Publisher | |
| Release Date | |
| Language | |
| Production Place | Bilbao, Euskadi, Spain |
Content Metadata
| Subject Area | |
| Genre | |
| Abstract | Christine Doig - Scale your data, not your process: Welcome to the Blaze ecosystem
NumPy and Pandas have revolutionized data processing and munging in
the Python ecosystem. As data and systems grow more complex, moving
and querying becomes more difficult. Python already has excellent
tools for in-memory datasets, but we inevitably want to scale this
processing and take advantage of additional hardware. This is where
Blaze comes in handy by providing a uniform interface to a variety of
technologies and abstractions for migrating and analyzing data.
Supported backends include databases like Postgres or MongoDB, disk
storage systems like PyTables, BColz, and HDF5, or distributed systems
like Hadoop and Spark.
This talk will introduce the Blaze ecosystem, which includes:
- Blaze (data querying)
- Odo (data migration)
- Dask (task scheduler)
- DyND (dynamic, multidimensional arrays)
- Datashape (data description)
Attendees will get the most out of this talk if they are familiar with
NumPy and Pandas, have intermediate Python programming skills, and/or
experience with large datasets. |
|
| Keywords | |