We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Column-level lineage is coming to the rescue

Formale Metadaten

Titel
Column-level lineage is coming to the rescue
Serientitel
Anzahl der Teile
60
Autor
Lizenz
CC-Namensnennung 3.0 Unported:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
Identifikatoren
Herausgeber
Erscheinungsjahr
Sprache

Inhaltliche Metadaten

Fachgebiet
Genre
Abstract
OpenLineage is a standard for metadata and lineage collection that is growing rapidly. Column-level lineage is one of its most anticipated features of the community that has been developed recently. In this talk, we: * show foundations for column lineage within OpenLineage standard, * provide real-life demo on how is it automatically extracted from Spark jobs, * describe and demo column lineage extraction from SQL queries, * show how the lineage can be consumed on Marquez backend. We aim to provide demos to focus on practical aspects of the column-level lineage which are interesting to data practitioners all over the world.