We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Kafka Streams and Protobuf

Formal Metadata

Title
Kafka Streams and Protobuf
Subtitle
stream processing at trivago
Title of Series
Number of Parts
611
Author
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language
Production Year2017

Content Metadata

Subject Area
Genre
Abstract
A lot of components in the Kafka and hadoop ecosystem assume you are workingwith avro messages. In this talk I will highlight some of the advantages andchallenges of stream processing with protobuf encoded messages, and why itmight be worth using them. A powerful feature of protobuf messages is its handling of unknown fields, andthe ability to partially parse messages with different schema versions or noschema at all. This helps in stream processing when you don't actually need toknow all message fields in order to process and forward them.