Async Await: Mastering Python's Time-Bending Tricks
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 131 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/69403 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Optical character recognitionBendingLevel (video gaming)Computer animationLecture/Conference
00:16
Principle of maximum entropyState transition systemReading (process)MultiplicationPersonal digital assistantProcess (computing)Multiplication signSystems engineeringLecture/ConferenceComputer animation
01:10
Optical disc drivePseudodifferentialoperatorComputer worm2 (number)Client (computing)Projective planeService (economics)System callScalabilityInstance (computer science)Virtual machineComputer animation
02:46
PseudodifferentialoperatorLoop (music)ResultantTask (computing)Group actionDisk read-and-write headCore dumpWordCoroutineMultiplication signEvent horizonFunctional (mathematics)Computer animation
04:11
Concurrency (computer science)Parallel portStorage area networkMultiplication signCodeVirtual machineMehrprozessorsystemRegular graphComputer programmingQueue (abstract data type)Different (Kate Ryan album)Task (computing)Parallel portConcurrency (computer science)Computer animation
06:47
WindowAsynchronous Transfer ModeInstance (computer science)CodeFile systemLine (geometry)DatabaseMessage passingMultiplication signBound stateComputer fileEndliche ModelltheoriePersonal digital assistantQuery languagePower (physics)InternetworkingWeb applicationSingle-precision floating-point format2 (number)Computer programmingNumberCASE <Informatik>outputFunction (mathematics)BefehlsprozessorSystems engineeringComputer animation
08:40
Instance (computer science)Systems engineeringMultiplication signLine (geometry)CodeMathematicsComputer animation
09:07
Number2 (number)Line (geometry)Multiplication signCodeFunctional (mathematics)Computer animation
09:36
Order of magnitudeWebsiteThermische ZustandsgleichungAdvanced Encryption StandardAsynchronous Transfer ModeFunctional (mathematics)File systemCodePersonal digital assistantWeightMereologyLibrary (computing)Open setOrder (biology)Systems engineeringResultantAiry functionWritingVirtual machineSystem callLecture/ConferenceComputer animation
11:14
BootingMultiplication signLibrary (computing)Personal digital assistantCodeFunctional (mathematics)Computer animation
11:48
CAN busFunctional (mathematics)Library (computing)Multiplication signBefehlsprozessor2 (number)QuicksortRevision controlNumberOperator (mathematics)Block (periodic table)Computer animation
12:41
ProgrammschleifeFunctional (mathematics)System callSystems engineeringCodeLogicMultiplication signComputer animation
13:29
Repeating decimalMessage passingAsynchronous Transfer ModeFunctional (mathematics)2 (number)CodeResultantInheritance (object-oriented programming)Thread (computing)Airy functionParameter (computer programming)Systems engineeringWeightNumberArea
14:57
BenchmarkMedianResultantFunctional (mathematics)CodeMultiplication signComputer animation
15:46
Software testingCASE <Informatik>CodePlug-in (computing)NeuroinformatikFunctional (mathematics)QuicksortScripting languagePerfect groupSoftware developerLibrary (computing)Task (computing)Test-driven developmentBenchmarkComputer animation
18:26
Software testingPersonal digital assistantFunctional (mathematics)Web-DesignerFocus (optics)ResultantCodeCASE <Informatik>Systems engineeringSlide ruleComputer animation
19:43
Uniform resource locatorConnected spaceQR codeMultiplication signComputer animation
20:18
Lecture/ConferenceComputer animation
Transcript: English(auto-generated)
00:06
Thank you a lot. It's a great honor to be sharing this stage with such an amazing amount of speakers and a lovely audience. So from me to you, a huge applause.
00:23
Now, I've been working in Python for the last 14 years. And I have managed to spend 20 years without using asyncio. I was very happy during that time. Python, like myself, is very horrible at doing multiple things at the same time.
00:41
In Python, we have multi-processing, multi-threading, async. They're all horrible. But async is the least horrible of them, so I decided to talk about that. Now, first, I want to introduce my assistant. He's going to be helping me today.
01:00
It's his first day on the job. I usually have a unicorn, but unicorn decided to take a vacation. So be nice to assistant. He made some spelling mistakes, but he's learning. The main reason why I'm talking about asyncio is that I'm a very, very stubborn person. So I don't learn new things unless I have to
01:22
or unless it bites me really, really hard. So I created a project for a client. And like most machine learning projects, it was actually just making a bunch of calls to OpenAI. And those calls take a lot of seconds. So the project worked.
01:41
And since it was done in a hurry, it was monolithic. So I couldn't do some horizontal scaling magic. And it turns out that the client actually had users, and they weren't able to use the service. So I did what any normal person would do. I just got a bigger instance, and everything
02:02
worked very nicely. Now, I was very happy with this. Client was even happier with this, but the person who's happiest with them all is Jeff Bezos, because I was using AWS. And this is mostly the story of how AWS took about 30 kilos of ice cream
02:22
from me at the end of the month when I had to pay bills. And that's like the saddest thing that can happen to me, because I'm a very great lover of ice cream. So if you see me in the hallways, don't hesitate to ask me any questions. I'm currently in the ice cream deficit phase, so offer me ice cream. I will gladly accept it.
02:41
But even without that, I'm free to offer advice. Now, things that I'm going to cover during this lecture. I know it's almost a lunch, so this is going to be a very light talk. I'll be telling you what async is. I'll be telling you how we can use it with minimal effort.
03:02
So I'm aiming here for going to give you two or three functions that you can use. Everybody is going to look at you and say, that person is super cool. They can use async. You want to get amazing performance out of it? What I will not be covering is deep tech dive
03:22
into how async works. Now, in async, there are some scary stuff like quarantines, futures, event loop, tasks, task groups. I will not be talking about any of that. What I'll be focusing on is how you can use async by changing few words, tweaking it,
03:43
and it's going to produce results. So I hope you're ready for that. Now, there's some housekeeping to do. I always mix up terminology in my head. We have parallelism. We have concurrency.
04:02
And for me, it's the same thing. I'm doing multiple things at the same time. And I learned quite recently not to mix those things up, because they are not the same. And the difference is very important, especially in Python. So I'll be using an example of a topic dear to me,
04:21
ice creams. So imagine and suspend your belief, for example, that you have a magical ice cream machine that produces ice cream. And unlike regular ice cream machine, you don't have to look at it. It will just create an ice cream. So when task is not concurrent and not parallel,
04:44
you go to a check register. There's a nice person there. You order an ice cream. They go to the machine, press the button. Ice cream gets created. They wait for the creation of ice cream. They come back and give you an ice cream. That's neither concurrent and not parallel.
05:02
You might think this is inefficient, but this is like the best program, because you can easily debug it and understand it. My brain works that way. Now, concurrent would be you go to the register, ask a nice person for the ice cream. They give you a receipt. They press the button on the ice cream-making machine.
05:23
Then they go to the queue, and they ask the next person what they want to do. They don't wait in front of the machine and watch it work. This is a very good thing to do, because it reduces the waiting. However, important thing to realize here is,
05:43
what this person is doing, it's just avoiding the waiting. It's not doing two things at the same time. This is going to come back later multiple times when I talk about async. Now, parallel would be the same thing as the first example, except here we would have two queues.
06:05
Now, you can complicate this. So in parallel, it's not concurrent. It's blocking, but there are two queues. Now, if both of the task registers,
06:20
the person working there, were nice and they were not waiting for the machine, it would be concurrent and parallel. That is very good, but in reality, you really don't want to maintain that code. As I said, multiprocessing in Python is really horrible. I will not go into details. I hope you can trust me.
06:44
We have gotten an ice cream. Now, I'm going to go over how you can make your code faster and more efficient, and most importantly, how you can give less of your money to AWS by using smaller instances.
07:00
Now, me and my assistants have created the most powerful AI engine that exists. Let's go slowly over each line of the code. It's a web app. As every AI model, it needs to wait some time. In my case, it's half a second. And then it returns a message that it cannot give you an ice cream.
07:23
Important lesson to all of you when you are working with LLMs. If an AI model tells you that it will give you an ice cream, it's lying, it's hallucination. So my model is 100% hallucination-free, better than OpenAI. I want you to pay attention to this single line of code,
07:41
time slip. This is what you will usually see in your code. Now, you might say, I'm not using time slip in my code. I want you to think of the time slip as something that's input-output bound. It can be a query to the internet. It can be a query to the database.
08:00
You might be writing a file to the file system. You might be getting stuff from the file system. Important thing here, I'm underlining it again, you are waiting for something to happen. You're not using your CPU. And that's where you're going to use async. So this here is a very rude program.
08:23
Because it starts, then it blocks everybody until it does its own thing, then it continues. How many requests per second can this program handle? Throw out numbers. Exactly two requests per second.
08:41
Now, I want this to be more efficient, because I don't want to buy higher instances. So I have changed one line of code. Instead of time slip, I called async slip. Actually, I changed more lines of code, but I'll get back to that.
09:01
So async slip, one line of code of change. And do you know how much faster this is? Throw numbers. Don't be afraid. Good guess. It's 106 requests per second.
09:23
And every time I get 50 times improvement in speed by changing just one line of code, I get very happy. So I'm going to give applause to this little function. Now, there are a couple of things
09:43
that I would like to show you here. My assistance, ladies and gentlemen. First, in order to use this amazingly powerful function, you need to do some ugly stuff to your code.
10:00
First thing is you need to write async in front of your function. That's the easy part. Now, a synchronous function does some weird stuff. They do not give you results right away. I'm not going to go what they return, but for our purposes, if you want to get result from async function,
10:22
just put await in front of it. I have spent hours and hours for debugging stuff because I forgot to put await in front of async function. So how to make this super easy? Just put async in front of it and call async method
10:41
at await. Those three stuff you do, it's going to be much faster. Now, you're not going to use async sleep. However, there are async libraries for Postgres. There are async libraries for file system handling. There are async libraries even for OpenAI.
11:02
So automatically, just by doing this stuff, your machine learning code is going to be so much faster. And you're going to be giving so much more money to OpenAI's API. Think of the benefits. Now, you might be thinking, as it happens in my own code
11:22
a lot of time, what if there is no async function? I have something super slow in my code, or I'm using this nice library that doesn't have async support. I'm not really good at async. So most of my code is not going to be async code.
11:42
But I really want to get speedy improvements without doing much of anything. So me and my assistants have created another function that does the time sleep blocking. That's the first one. Same as what we saw in the first example, it was super slow. That one was two requests per second.
12:02
Async version is 100 requests per second. And async library has this beautiful, beautiful function, which I'm going to show you now, to thread. You throw something that's slow and blocking, but, as I mentioned before, does lots of waiting.
12:21
It doesn't do any CPU, GPU, TPU, any sorts of operation. It's waiting. You throw it, just call this function, and then pass it your function. And then this is going to work like a magic. I'm not going to ask you numbers for this, because this gives you 10 requests per second.
12:45
This is five times faster. And you only had to call one function. You didn't even have to change your synchronous code. That's a very, very good thing. I love not having to change many things. Now, usually when you're making calls to external resources,
13:03
you're going to be wondering, OK, what if I want to call multiple functions? You know, you have four loops and all that stuff. Well, in async land, things tend to get very weird. So they don't like to use.
13:21
There are efficient ways of doing things that are encounter to the human logic. Let's go with that. So what I have done is I have created two smaller AIs. One has much less parameters, so it runs only in 0.4 seconds.
13:42
And I want to execute them. And since I'm super lazy, I'm using AsyncIO to thread for both of them. They're blocking functions. And I want to get results from both of them. And I want to do it fast. I don't care which one is executed first.
14:02
But once I run the wait, I want to get the results from both of them. So AsyncIO gather, beautiful function. I do not fully understand, but it does magic for me. Now, you know that when I run the first code that
14:25
is blocking, it does two requests per second. When I run fully asynchronous code, it does 106 requests per second. When I use just AsyncIO to thread, it does 12 requests per second. Now, I'm calling two AsyncIO thread to second functions.
14:44
How fast do you think this is going to be? Throw numbers. 10. Do we have more, less? 15. It's actually 12.9.
15:03
So you might be wondering, if I have called two functions here and one function there, why do I get 2.9 requests more? Answer to that is very simple. I'm horrible person at benchmarking. I run this two times, got the median value, and use it.
15:23
So my lesson to you is when you're developing something with Async or anything else, do your own benchmarking code. Results are going to be different. You're not going to get 50%, 5% improvements on all that. So don't trust me as a speaker. Don't trust anybody.
15:41
Do your own research. It's going to be very nice. Do's and do not with Async. OK. I have given you several functions. They are very useful, and they require very little use of code.
16:01
And you get a bunch of cool stuff. It's very easy to impress friends. Now, what I have left out is, how do I run all this stuff? It turns out you can just run Python, name of your script, and things work. It has to be more complicated in Async.
16:21
You have to call Async run, the name of your function, and then it works. It's just one step. It annoys me a lot, but you got to do it. Also, I'm going to underline this again. If you're doing any sort of computation, do not use Async.
16:41
If you're doing some sort of waiting, and it's not super long, do not use Async. It's perfect for that, but it's just going to complicate your code. Make your code as human readable as possible. So do some benchmarking. Test your code.
17:01
And then, if you see, OK, this is slowish, or in my case, I'm giving too much ice cream to Jeff Bezos, then I'm going to try using Async. And you can do a lot without actually understanding what's happening under the hood. Now, I like using fast API.
17:20
So I'm going to link to fast API documentation, which is great with examples of what's happening there. Official Python documentation is amazing as well. They do like to use terminology, coroutines, future tasks, and all that stuff. You can figure it out. But for most of the cases, what I have shown you
17:42
is going to be super nice and easy. Now, all of you are very test-driven development people. You want to make sure that this is running correctly. And from what I have shown you, you might be wondering, Async IIO is probably super difficult to test.
18:01
It's not the case. Actually, that's the good stuff about this. So there are many testing libraries in pytest, and I do not judge. But pytest is the best one, and you should use it totally. So pytest has a pytest async plugin. You just install it, and that's it.
18:22
You don't have to do any manual work. You create a function. And in my case, the async function is adding two numbers. And if you remember what I was telling you through this whole lecture, you should never create an async function that does computation. But for the purposes of this slide,
18:41
me and my assistants have created that function. So ignore the function. Focus on the purpose of this. You just add this beautiful decorator, pytest.mark async IIO. Your test also needs to be async, but you don't care here because pytest is going to do the magic behind you.
19:01
You await the result, assert the result. Now, you will notice that code is indented here. That's wrong, and I'll tell my assistant not to do it in the future. But imagine that this is not indented. And this is how you test stuff. It almost, if you squeeze your eyes,
19:22
it almost looks like a normal test. Now, further ease. As I said, fast API is what I use for my web development. It's pretty nice. Official documentation is very, very nice.
19:42
And that's it. I want to thank you. Also, I love getting applause. Connect with me on LinkedIn.
20:02
You can use QR codes if you're fancy. You can use URLs now. I try to be super fast so you can get to the launch very quickly and not get hungry. If you have any questions, I'm here. We have time. Thank you very much.