We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Building Workflows With Celery

Formal Metadata

Title
Building Workflows With Celery
Title of Series
Number of Parts
50
Author
Contributors
License
CC Attribution - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
Task Queues is a topic which most developers will eventually have to dive into, specially in today’s web development world. The idea is really simple: whenever one has any functionality which might take too long to perform, one can spawn a process which will take care of this functionality without having to block the app’s main loop. A task queue will use worker processes to execute these long-running tasks and the user does not have to wait until the task is done. Instead, an acknowledged message is presented to de user while the task is executed in the background. This concept is really important when building web applications. HTTP Requests have timeout and making the user wait a long time for something to finish is not a good user experience practice. Usually, these tasks are used in groups creating a workflow where the work is distributed into smaller tasks. Celery is usually the first project one encounters when searching for task queues and Django. I have been using Celery for over four years. The Celery project is one of the most robust task queues out there. It is certainly not the only task queue. And, it can be difficult planning the correct architecture for a specific workflow. This talk will explain enough of Celery’s basics to understand how to build workflows with Celery. Building workflows with Celery is never straight forward. This is mainly because Celery offers the building blocks to build workflows but it tries to move out of the way. By not being too intrusive, Celery allows building complex workflows. I will explain common patters and tips to successfully use celery to build workflow of different complexities.