We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Realtime Distributed Computing At Scale: Storm And Streamparse

Formal Metadata

Title
Realtime Distributed Computing At Scale: Storm And Streamparse
Title of Series
Number of Parts
160
Author
License
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
Realtime Distributed Computing At Scale (in pure Python!): Storm And Streamparse [EuroPython 2017 - Talk - 2017-07-12 - Arengo] [Rimini, Italy] Realtime distributed computing is tough, especially at scale: managing a large data pipeline is tough, and it’s even tougher to keep latency low and availability high when processing tens of thousands of items per second. Many people turn in despair to Java or Scala when it comes time to scale up, but we can do it in Python: Apache Storm is a distributed realtime computation system that can let you scale up- and no need to reach for a new language! This talk will walk the audience through the basics of Apache Storm and how it’s an elegant, useful solution to realtime distributed computing, as well as how streamparse can let you write your storm components in Python by writing some code and a basic storm topology in Python. We’ll also look at how Parsely uses Storm in production to handle billions of realtime events a month. If we have time, we’ll go a bit into how Storm has several advantages over other common Python computing data streaming solutions, like Spark’s microbatching. Goals: At the end of the talk, ideally you should be able to understand: What Apache Storm is, how it works generally, and what scenarios it’s useful for How streamparse can be used to write your Storm topologies How Storm + streamparse is used in an actual high-availability, low-latency production environmen