We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Caching for Jupyter Notebooks

Formal Metadata

Title
Caching for Jupyter Notebooks
Title of Series
Number of Parts
131
Author
Contributors
License
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
Caching data and calculation results in jupyter notebooks is a great way to speed up development by making expensive cells easier to re-run. Data scientists and developers using notebooks on a daily basis, can improve their notebook workflow with low-effort changes in the notebook code, cut the time spent waiting and reduce context switches. This talk targets developers and data scientist of all experience levels and will cover: Why caching in notebooks? Setting up the context in which developers and data scientists use notebooks for exploratory work and how caching is relevant in it. What is caching Quick definition of caching, introducing the different types of persistence (in-memory, on disk, database, object storage …), cache invalidation strategies (parameters, code changes, ttl, …), with some cautionary comments about data security when caching protected data. Caching Techniques Going through readily available options from the python standard library, and how to use them in notebooks. Introducing a few off-the-shelves options like ipython % magics, and cachetools. Showcasing how one would build their own mini-caching framework, that fits for their specific use case, using pandas and spark for the example Explaining when to stop trying to cache, and keeping the caching framework mini, what are the signs that caching went overboard.