We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Building an actor library for Quickwit's indexing pipeline.

Formal Metadata

Title
Building an actor library for Quickwit's indexing pipeline.
Title of Series
Number of Parts
542
Author
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
Quickwit is an opensource distributed search engine. The most challenging component of a search engine is its indexing pipeline. It takes a stream of JSON documents, processes them in batches, and emits index artefacts we call splits. We needed our implementation to be robust, testable, observable, and efficient... But even more importantly, our code had to be easy to work with. I will explain why we chose to develop our own actor framework and discuss the unique features of our implementation.