We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Evolving Yelp's Streaming at Scale

Formale Metadaten

Titel
Evolving Yelp's Streaming at Scale
Serientitel
Anzahl der Teile
64
Autor
Mitwirkende
Lizenz
CC-Namensnennung 3.0 Unported:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
Identifikatoren
Herausgeber
Erscheinungsjahr
Sprache

Inhaltliche Metadaten

Fachgebiet
Genre
Abstract
Yelp relies on near real-time data to enable critical functionality ranging from in-flight business data moving across different data stores, to operational and user data used to train machine learning models. In this talk, we will detail the evolution of Yelp's streaming data platform. - We will delve into the past to review the data platform we built abstracting storage and compute, and the middleware we implemented to tie everything together. We will also visit observability and maintenance challenges derived from hyper-adoption. - In the present, we've standardized on SQL for data pipelines, except for advanced use cases. - Looking towards the future, we're transitioning from custom to open-source data formats and registry, leveraging Flink. We're also exploring Streamhouse to address current challenges. This talk will provide insights into the challenges faced in building and scaling a streaming data platform, as well as the strategies employed to overcome them, making it a valuable session for anyone interested in data engineering and streaming technologies.