We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Taming the cost of Kafka workloads in the cloud

Formale Metadaten

Titel
Taming the cost of Kafka workloads in the cloud
Serientitel
Anzahl der Teile
64
Autor
Mitwirkende
Lizenz
CC-Namensnennung 3.0 Unported:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
Identifikatoren
Herausgeber
Erscheinungsjahr
Sprache

Inhaltliche Metadaten

Fachgebiet
Genre
Abstract
Cloud computing has changed how we operate and think about software: At any time, we have instant access to a seemingly unlimited capacity of compute resources and can scale our applications elastically on demand. We no longer have to procure physical machines and pay for them upfront but are billed for only those resources that we have actually used. While the usage-based pricing sounds great at first glance, it makes cost planning a complicated task, often leading to huge surprises when receiving invoices for cloud services, as revealed in numerous blog posts of authors waking up to invoice amounts that are multiple orders of magnitude higher than expected. This talk explores the main cost factors of Kafka workloads in the cloud: compute, network, and storage. We identify their contribution to the overall cost and walk through different techniques for reducing their cost footprint by, for instance, using Kafka's follower fetching to reduce cross-AZ traffic, compression to reduce overall traffic, or scaling streaming applications ""to zero"" in the absence of incoming events to save compute resources. We focus on setups where both Kafka and associated workloads, such as Kafka Streams or Flink applications, are operated on cloud platforms. The goal of this talk is to enable developers to benefit from the advantages of cloud computing in the context of Kafka workloads while taming the associated costs.