Parallelism Shootout: threads vs asyncio vs multiple processes
Formal Metadata
| Title | Parallelism Shootout: threads vs asyncio vs multiple processes |
|
| Title of Series | |
| Part Number | 143 |
| Number of Parts | 173 |
| Author | |
| License | You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license. |
| Identifiers | |
| Publisher | |
| Release Date | |
| Language | |
| Production Place | Bilbao, Euskadi, Spain |
Content Metadata
| Subject Area | |
| Genre | |
| Abstract | Shahriar Tajbakhsh - Parallelism Shootout: threads vs asyncio vs multiple processes
You need to download data from lots and lots of URLs stored in a text
file and then save them on your machine. Sure, you could write a loop
and get each URL in sequence, but imagine that there are so many URLs
that the sun may burn out before that loop is finished; or, you're
just too impatient.
For the sake of making this instructive, pretend you can only use one
box. So, what do you do? Here are some typical solutions: Use a single
process that creates lots of threads. Use many processes. Use a single
process and a library like asyncio, gevent or eventlet to yield
between coroutines when the OS blocks on IO.
The talk will walk through the mechanics of each approach, and then
show benchmarks of the three different approaches. |
|
| Keywords | |