We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Building a distributed search engine with tantivy

Formale Metadaten

Titel
Building a distributed search engine with tantivy
Untertitel
How lnx is solving the challenges of builing a distributed search engine in Rust
Serientitel
Anzahl der Teile
542
Autor
Lizenz
CC-Namensnennung 2.0 Belgien:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
Identifikatoren
Herausgeber
Erscheinungsjahr
Sprache

Inhaltliche Metadaten

Fachgebiet
Genre
Abstract
Follow me as I walk through my journey of building lnx, a distributed search engine written in Rust akin to Elasticsearch or Algolia that aims to be faster and more efficient using tantivy. I cover the challenges and solutions I encountered while developing lnx over the last year, the tradeoffs made and how you can build your own search engine using the tools the Rust ecosystem provides. In the first part of the talk we'll look at what a search engine like Algolia or Elasticsearch does, how tantivy works, what things we need to do in order to make it suitable for user-facing search and how we implement it into our code to build our own application. In the second part of the talk I talk about the biggest issue you run into when building a search engine or any other system designed to store data... Fault tolerance and replication, together we'll go through the challenges of implementing this for a search engine using tantivy, keeping the performance competitive, reliable and the tradeoffs that are made to make it work. We'll look at some existing implementations like Raft and take a dive into the world of eventual consistency! With any luck, you'll have a better understanding of how we can build services like search engine using libraries like tantivy along with solving technical challenges like consensus and replication for distributed systems in Rust.