We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Aerodynamic Data Models: Flying Fast at Scale with DuckDB

Formal Metadata

Title
Aerodynamic Data Models: Flying Fast at Scale with DuckDB
Title of Series
Number of Parts
798
Author
Contributors
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
At Rill, we rely on DuckDB to power uniquely fast dashboards for exploring time-series metrics. To achieve this interactivity, Rill’s dashboards generate up to 100 parallel queries in response to each user interaction. In this lightning talk, we'll share a series of optimization and data modeling techniques that have been pivotal in achieving remarkably fast, sub-second response times using DuckDB. Our primary tactics include employing parallel connections to facilitate simultaneous query processing and organizing data in a chronological order to enhance the effectiveness of min-max indexes. We also utilize enum types for more efficient handling of string column queries, along with config tunings. These approaches have collectively enabled us to enhance DuckDB's capability to handle larger datasets (100+ GBs) with sub-second query responses. We invite you to join us in this insightful session to discover how these optimizations can significantly improve your data processing and query performance in DuckDB.