We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Advanced asyncio: Solving Real-world Production Problems

Formal Metadata

Title
Advanced asyncio: Solving Real-world Production Problems
Title of Series
Number of Parts
118
Author
License
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
By building a simplified chaos monkey service, we will walk through how to create a good foundation for an asyncio-based service, including graceful shutdowns, proper exception handling, and testing asynchronous code. We’ll get into the hairier topics as well, covering topics like working with synchronous code, debugging and profiling, and working with threaded code. We’ll learn how to approach asynchronous and concurrent programming with Python’s asyncio library, take away some best practices, and learn what pitfalls to avoid. Outline: (40 minutes + 5 min Q&A, if unable to get 45 minutes, then 30 min slot with no time for Q&A) Intro (2m) Speaker/company intro Setting the context/purpose of talk Foundations (9m - trimmed to 6m for 30 min slot) Initial setup of an asyncio service (2m) --- Required boilerplate code --- Inspiration from official asyncio tutorial docs Making a service emactually/em concurrent (5m) --- non-blocking vs concurrent --- when to be concurrent vs serial --- using callbacks vs awaits vs scheduling tasks (create_task) vs asyncio.Events --- Making synchronous code asyncio-friendly (2m) Intermediate (9m - trimmed to 6m for 30 min slot) Graceful shutdowns (3m) --- What a signal handler is, why it’s needed --- What signals to listen to --- Gotchas of cancelling tasks, asyncio.shield + shutdown behavior Exception handling (3m) --- Difference between top-level exception handling and handling within other coroutines --- Avoid mistakenly swallowing/missing raised exceptions --- Making use of loop.setemexception/emhandler Making threaded code asyncio-“friendly” (3m) --- Calling threaded code from coroutines (aka running within a ThreadPoolExecutor) --- Calling coroutines from from threaded code (aka runemcoroutine/emthreadsafe) Advanced (19m - trimmed to 15m for 30 min slot) Testing asyncio code (7m) --- Benefits of debug mode --- How to mock coroutines Debugging an asyncio service (5m) --- Reinforce debug mode --- Using “tricks"" like codeasyncio.all_tasks/code with logging, codeloop.slow_callback_duration/code, adding context/stack trace in default exception handler Profiling (7m) --- Basic profiling (cProfile, strace) - not that different from sync code --- Continuous profiling with 3rd party tools, i.e. github.com/what-studio/profiling --- PyCharm’s asyncio & thread profiler --- How to properly trace a workflow/request (e.g. for the purpose of distributed tracing) (to be cut if not enough time) Wrap up/Review (1m) /ol
Keywords