Asyncio in production
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 132 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/44946 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
outputPhysical systemProduct (business)BitCASE <Informatik>TheoryNumberComputer animation
01:28
Exterior algebraLibrary (computing)BlogSynchronizationNumeral (linguistics)Product (business)Computer animation
01:52
Computer programmingConcurrency (computer science)Pairwise comparisonMereologySoftwareWordPhysical systemMultiplication signMereologyProduct (business)Library (computing)Arithmetic meanComputer programMilitary baseBitStability theoryParallel computingPoint (geometry)Cartesian coordinate systemService (economics)Formal languageTouch typingWeb pageSoftwareScripting languageSequenceComputer architectureNumberCodeEvent horizonEndliche ModelltheorieBookmark (World Wide Web)CodeDirectory serviceoutputLine (geometry)MultiplicationSoftware testing
06:29
Instance (computer science)Service (economics)Scale (map)Client (computing)MereologyFormal languageHuman migrationSoftware frameworkDevice driverStructural loadCartesian coordinate systemMultiplicationStandard deviationDemosceneEvent horizonSoftware frameworkSoftware developerInstance (computer science)Primitive (album)CodeParallel computingBound stateStructural loadWeb 2.0Multiplication signProduct (business)Library (computing)CodeRevision controlCapability Maturity ModelPhysical systemRepresentational state transferBitProjective planeDevice driverAuthorizationMereologyProcess (computing)Level (video gaming)Service (economics)SoftwareDatabasePoint (geometry)Classical physicsServer (computing)Variety (linguistics)Formal languageExistenceElectronic visual displayEmailSynchronizationAutomatic differentiationoutputSlide ruleNP-hardMaizeDirected graphComputer animation
11:37
Software frameworkFlash memoryQuarkHuman migrationObject (grammar)Human migrationBitMiddlewareException handlingInheritance (object-oriented programming)Optical disc driveDevice driverDatabaseINTEGRALLibrary (computing)Different (Kate Ryan album)Web 2.0Slide ruleParameter (computer programming)Network socketError messageMultiplication signGoodness of fitSoftware developerPoint (geometry)Data miningTrailConnected spaceCapability Maturity ModelCartesian coordinate systemSoftware frameworkInternetworkingSoftwareInteractive televisionClient (computing)outputService (economics)Directed graphElectronic mailing list
15:27
CodeFunction (mathematics)Revision controlMereologyLoop (music)Service (economics)CodeFunctional (mathematics)BitException handlingLevel (video gaming)DampingSoftware developerImplementationOpen sourceProjective planeEvent horizonPerformance appraisalCore dumpDevice driverMultiplication signWeb 2.0Medical imagingDemoscenePoint (geometry)TheoryTask (computing)DatabaseSoftware frameworkProgrammschleifeSign (mathematics)CurveTransport Layer SecurityRevision controloutputWeightSoftwareCoroutinePerfect groupElectric generatorTransportation theory (mathematics)Communications protocolComputer animation
19:27
Pairwise comparisonProjective planeParallel computingCodeWeb 2.0Process (computing)BuildingLoop (music)BenchmarkDirected graphWordBlock (periodic table)Event horizonThread (computing)Product (business)RoutingLibrary (computing)Service (economics)DatabaseMultiplication signTask (computing)Object-relational mapping2 (number)LastteilungStatistical hypothesis testingVariable (mathematics)Revision controlCASE <Informatik>Direction (geometry)Physical systemConfiguration spaceOverhead (computing)Cartesian coordinate systemPairwise comparisonCommunications protocolClient (computing)Transportation theory (mathematics)Point (geometry)Queue (abstract data type)outputInstance (computer science)Graph (mathematics)Line (geometry)String (computer science)ResultantStandard deviationArithmetic meanTerm (mathematics)Operating systemMedianConnected spaceNumberMusical ensembleSoftware developerSoftware testingServer (computing)Virtual machineSlide ruleLevel (video gaming)Right angleArmPlanningRepeating decimalRecursionQuery languageBoss CorporationSequelComputer animation
Transcript: English(auto-generated)
00:03
Thank you so much And thanks for coming along. It's very nice to see all of you as he said my name is Robin Erickson And I like a lot of people around this conference. It seems. I'm quite interested in asyncio and
00:20
That's what I'm going to be talking about today. You've maybe been to some of the other asyncio talks this morning In my mind they were a little bit kind of theoretical But I'm here to try to represent the more practical side of things So yeah, you know just trying to use this thing and make it work So the first thing that I want to do today is to you know get a feeling of how many people in the audience have
00:44
Have toyed around with asyncio in Python shows of hands We definitely are in a Python conference that's for sure The more interesting question in my mind however is the following how many of you have created actual mission-critical systems in
01:01
Python using asyncio and kind of you know deployed them to production It's still quite a lot but yeah still way less than the number of people that have toyed around with asyncio and This seems to be the case everywhere where I ask this question Most Python istas have played around with the thing
01:22
But it hasn't really made its way into their production systems for one reason or another And this is despite the numerous blog posts and articles and all of this stuff that details how massively Asyncio libraries outperform their kind of synchronous counterparts, but also like their asynchronous
01:44
alternatives So why aren't everyone using asyncio in production when it's so obviously all rainbows and unicorns? And there are definitely a number of reasons for this I'm just quickly gonna go through like some of my personal favorites First of all asynchronous programming is it's very different. It requires a very different mental model of how to do programming and
02:08
Takes some time to adjust to Secondly the asyncio feature in Python is it's relatively new to the language And by that I mean asyncio. I don't mean like tornado and all of that kind of stuff because that dates back to
02:22
2002 But programmers are a little bit kind of reluctant to new shiny things when it comes to their production systems It's kind of interesting because at the same time we are very much drawn to new shiny things, so it's a very thin line So third to convert an already existing Python application to asyncio is
02:43
non-trivial it takes effort Fourth due to the lack of native support in the language in the past to do asynchronous programming the community has Come up with solutions to this They've come up with the libraries like event let tornado twisted all of the stuff
03:00
And many of these libraries are widely used and they they just work and people cannot just you know stick with them so Last probably most importantly asynchronous programming is not always the solution to your problems Now it's so happens that I do have some experience with migrating actual in production Python code bases to use
03:22
asyncio and Even though asyncio is by no means perfect my part like my experience thus far has been quite positive And that's kind of why I'm here today a little bit to spread the word So my talk today is split roughly into three parts the first part in the first part
03:40
I'm gonna be discussing discussing what led me to commit a lot of time and effort you know into migrating Software that was already working in production to use asyncio Then I'm going to talk about some of the roadblocks that I hate during this time And you know some of the caveats that you might want to be aware of if you want to do the same thing
04:02
Last I want to talk about some of the benefits of this work And you know try to back this up with some empirical data I should probably also mention what this talk is not about just so we're on the same page It's actually not a lot of things, but mainly it's not an introduction into asyncio and how it works
04:22
So yeah, you're gonna have to look that up somewhere else, but even though you have absolutely no knowledge of it at all Don't worry. I'm hoping this talk might at least kind of trigger you to explore the subject further Interesting
04:41
So why would you want to bother with asyncio? Sorry, I'm just not gonna touch this thing So your software already works, right? To understand where I'm coming from personally We're gonna need to explore my background a little bit just a teeny tiny little bit so in my past
05:03
I worked for a company that Was obsessed with using node.js for everything Admittedly I was kind of one of the people in charge, so I cannot really free myself from that But despite node.js is and JavaScript's most stable pain points These technologies still taught me how useful asynchronous programming can be and how efficient it can be
05:26
Especially when your applications are just mostly IO bound By the end of my tenure of this company I had gotten used to like writing everything even what would be very simple sequential scripts in Python in an asynchronous fashion
05:41
So when I joined my current employer a company called Smarkets I had to I had kind of to get our news like used to a new reality Smarkets is a company that relies heavily on a micro services micro services based architecture and most of these services are Coincidentally written in Python
06:01
so initially when I started looking into how these micro services were structured and And how they were coded all I could see was just plain old blocking Python codes To be honest I wasn't too impressed Especially coming from my awesome asynchronous backgrounds so at this point in time
06:20
I started thinking like how are they scaling these things? Because some of these micro services were taking on some serious serious traffic So after a bit of research I noticed that most of these services were scaled out in a fairly classical Pythonic way its Service was running in multiple instances application instances and its instance was a gunicorn server that had you know multiple workers
06:45
So scaling out was a matter of adding new application instances or adding new workers or both So because our micro services are very IO bounds It was kind of apparent to me that all of these workers on the end
07:00
We're just going to be stuck waiting on IO the whole time I was just thinking like what a waste And after thinking about this for a while I noticed that a big part of the services depended on a packets that was somewhere deep down doing this thing here Probably some of you know this thing
07:21
David actually touched upon this in the keynote this morning, but Obviously my node.js acclimated minds did not realize immediately what this did but after some research I figured out that this actually patches the standard library So you're previously kind of blocking code will magically be doing asynchronous things behind the scenes
07:43
cool So Before asyncio came along this was apparently one of the kind of standard ways of achieving concurrency in Python In my mind, this is problematic in multiple ways Mostly because this is kind of hidden away from the developer. It's quite magical and it's hard to reason about as well
08:04
Also, like many developers on my team and in the company were not making use of event Let's multiple concurrency primitives because they didn't even know this was happening somewhere in their kind of software so the moral of the story is that asyncio itself and the kind of async slash await keywords make this way more explicit and
08:26
Someone told me that explicit is better than implicit I Should probably know that I have absolutely nothing against eventlet in general quite the opposite I find it fascinating that they were able to create something that works so well that I didn't notice it for you know
08:43
Months that it was kind of happy like doing asynchronous stuff behind the scenes But now that we do have asyncio, I think we should kind of start using that So yeah to summarize a little bit there are multiple ways in which we can achieve concurrency in Python, but in my mind asyncio is the obvious way to go because it's
09:03
Explicit and it's a part of the language It's also supposedly quite performant, but we will get back to that later on So assuming that asyncio is indeed the future and that we want to kind of you know
09:23
Make our new projects use it or migrate our existing projects to use it What issues are we going to run into and how simple is it? I've already mentioned. It's actually not super simple But it's definitely getting better. It's getting way simpler than it was
09:41
Not that long ago when I was taking a look at asyncio in Python for the first time I noticed that people were relying quite heavily on Monkey patching again their dependencies just to make them work with their asyncio codes For anyone that's serious about their code base and like serious about the reliability of their system
10:01
That's probably not a great idea But since then for the kind of past, you know a couple of months maybe up to a year It seems like the asyncio ecosystem in Python has grown and matured rapidly We now have loads of mature web frameworks loads of mature kind of database drivers and just a lot of libraries for a lot of stuff and
10:24
Yeah, it's it's it's getting quite good So I kind of feel now like it's getting to the point where Library authors should no longer be ignoring asyncio and instead kind of defaulting to it Now because I'm a very practical person as I spoke about earlier
10:42
I think the best way to describe how you should go about migrating a Python project to asyncio is simply to describe the process that I went through recently when I Moved one of our kind of in production microservices to use asyncio
11:00
Having done this before I knew that the first thing I'd need to do Was to kind of map out the dependencies of the service and research if there would exist asyncio compatible versions of the dependencies and Also, since this was a microservice that exposed like a RESTful API
11:21
Obviously, I had to figure out which kind of web framework to use always a classical problem and when researching these kind of Asyncio web frameworks that we have now I was pleasantly surprised to see the kind of wide variety of web frameworks that we have access to This slide just shows a few example of those
11:40
And all of these are kind of specifically targeted towards asyncio Some of them aim to be flask-like some of them aim to be fast some of them aim to have good web socket support and some of them Aim to be compatible not only with flasks public API, but also its private API in such a way that you're
12:01
You're basically supposed to be able to do a find and replace of flask to quartz And you just sprinkle a few kind of async await keywords around and it just should work. I Haven't tried it out, but I don't know a colleague of mine Phil actually wrote this framework So yeah, it probably works. He's a good guy
12:25
This can be obviously this can be very useful if you are migrating a large kind of flask application to asyncio And you just want to get up to speed quickly Now I didn't care that much about API compatibility with flask so I decided to go with another web framework Which is called AIO HTTP simply because it's known to be mature
12:44
Sputtle tested, and I don't want to take my chances with new shiny stuff like quartz But even though the AIO HTTP API is a little bit different from flask the flask API we all know and love Migrating to it was super simple and in fact
13:02
I kind of felt a little bit kind of relieved to be rid of the global request object that I've always found a little bit odd That's just me Anyway The same can be said about the migration from psycho PG to which is the old database driver that we used for Postgres to asyncpg Which is the new de facto awesome Postgres database driver for asyncio
13:25
Except for the desperate lack of named parameters if the developers are in the room I was a little bit worried that migrating sentry the error tracking software that we use would be problematic and After searching the internet for a little bit I found some packets on github that was supposed to be you know an integration between sentry and
13:46
AIO HTTP couldn't really make it work So I figured I just it's just gonna be simpler for me to do this myself So I had already worked with middlewares in AIO HTTP And I knew I could easily implement something like this as middleware and literally five minutes later. This is what I have
14:04
a Very typical middleware just runs the request handler race it and if the request handler races on uncut uncut exception my sentry middleware will will capture the exception fire it off to sentry and then just race again
14:20
Super simple so obviously this also makes use of the fact that The Python sentry client library also has support for a AIO HTTP as a transport so so far so good Unfortunately my migration was not completely without issues One of the features of my microservice prior to my migration was that its API was built and defined in Swacker
14:46
And it had some kind of fancy Swacker UI thing on top of it, so you could kind of interact with the API It's really cool Anyway this support came through a library which is called connection Probably some of you know it and after researching connection for a little bit
15:03
I It looked like it would support using AIO HTTP as a back-end But then I kind of tried it out, and it didn't really work and the documentation was kind of non-existent, so What I probably should have done at this point in time was just to fix the connection library and the documentation
15:21
But it's on my to-do list, no worries Yeah, instead I just dropped the Swagger thing So up until this point I've been talking about asyncio like it's the perfect piece of software, and it's not Yeah, that's just it's absolutely not There are quite a few things you might want to be aware of before starting to use asyncio
15:44
And I'm just gonna mention some of the issues that I've personally experienced Yuri spoke about more issues earlier and And Yeah, and I think someone is talking even more about issues tomorrow in another talk so plenty of that
16:01
First while asyncio is conceptually simple in Python. There is a steep learning curve to understand the whole beast We're talking about event loops event loop policies, awaitables, coroutines, generators, futures, tasks, Executors, transports, protocols, etc etc etc So if you're migrating a large important code base to you know use asyncio
16:26
You'd ideally want to understand most of these and kind of how they interplay, so yeah this takes time Another issue that many developers that try out asyncio mention is As soon as you start developing
16:41
Like migrating your Python code base to use asyncio your code base is going to be littered with async and await everywhere To me like yeah, I can experience this, but to me. This is like. This is not a huge deal This is just the cost of explicitness, and I'm gonna welcome that
17:01
An issue that definitely did hit me personally was the fact that sometimes the asyncio stack would just You know make my debugging live a little bit too difficult by kind of swallowing my exceptions And I had like no idea what was going on But this is definitely getting better every day and This is something that Yuri the one of the Python core committers around asyncio spoke about earlier as well
17:25
So I'm very happy to hear that Now accidentally running synchronous code that waits for IO inside of your asynchronous functions yet another source of issues Here what I can tell you is like it's very simple to monitor the event loop
17:42
It's basically just about figuring out which tasks on the image loop take longer than X milliseconds and If some tasks are doing that this might be a sign that it's actually doing some IO or some kind of blocking IO Inside of the async functions, so yeah as mentioned. This is just some of the examples
18:02
So to summarize if you plan on migrating a large Project through asyncio or you know create the project from scratch the most important thing I can tell you is just you know do the research first map out your dependencies see if there exists asyncio compatible versions of them and
18:20
You know if they exist actually make sure that they work, and you know check them out like that's very useful What's out for some of the gotchas and yeah profit? So you've migrated all of your projects to asyncio was it worth it. Are you any better off? Let's see how it went for me
18:43
I'm gonna start out by taking a look at the kind of before after evaluation of a microserve the micro service that I spoke about earlier I've already mentioned the initial version of it had you know flask to declare an API that's Psycho PGT to to communicate with the Postgres database a lot of all the stuff that matters less
19:03
Also utilized event let to make everything asynchronous behind the scenes so in theory it should have been quite efficient We've also gone through how the after version of this microservice looked like a IO HTTP as a web framework async PG as a Postgres database driver and to run the event loop itself we used the Python
19:24
implementation of the uv loop which is apparently very fast and This is kind of our experimental setup quite standard we used the work project or wrk to perform HTTP benchmarking of both versions of the service We ran its configuration of the benchmark for 30 seconds 10 times using a variable number of connections and
19:46
Noted the median 25th percentile and 75th percentile in terms of throughput request per second For all of the 10 runs. We also had some delay between runs just to kind of allow the server and client to settle and
20:01
On this slide you can see this throughput comparison Of the microservice running in quite a basic configuration with only one gunic on worker and just on my machine So just in development We then you use this work thing The bands parking tool to test the throughput of both versions
20:20
On a very simple just ping route that doesn't really do anything interesting. It kind of just returns the string pong And we use this quite a lot in our microservices just to do health checking like are you alive? And as you can see the results here are quite conclusive like the asyncio version of the microservice That is the yellow line is able to handle on the like on average 1400 records per second
20:45
While the event lead version is able to handle somewhere around 650 So in this case the performance increase is somewhere on two-fold Here we can see another benchmark that we did which is on a route that actually does things it does You know some interesting database accesses
21:02
And then kind of returns the results of the database access and some transformations and in this Experiment, it seems like the before the event led version is able to handle somewhere on 200 to 300 records per second While the asyncio version the yellow line again
21:21
It seems to be able to handle somewhere on 800 to 1000 now the most interesting one is probably this one This is an actual Benchmark on the microservice both of them in production both before and after the migration and In this particular experiment the number of gunic on workers were just set to three and we only ran the benchmarks on one
21:42
application instance usually we have more application instances in production, but Testing all of them in conjunction is kind of besides the point here because they are supposed to be able to scale linearly if the Load balancer that you're using is okay So as you can see Here the before version the event led version of the microservice
22:03
Performs okay like it's handling one like one thousand and three hundred records per second quite consistently while the asyncio version confusingly the green line In this graph is able to handle around seven thousand records per second And this is more than a five-fold increase in throughput, so this kind of means that we can just get rid of
22:26
like 80% of the like application instances that we have in production Which is quite nice because we save a lot of money So just to conclude the whole thing is asyncio worth the effort
22:41
We finally have explicit native support in Python to do asynchronous programming That is kind of slowly, but surely becoming the de facto standard in the community and kind of replacing other concurrency libraries and as we just saw the performance the
23:01
Performance for of I see asyncio is quite good It's very positive results and these mirror what I have seen on the web So all in all yes, I think it's definitely worth it go check it out if you have not Thank you
23:46
I Have yes I was doing some like I was using some stuff in our code that was apparently doing some very interesting Requests like using the request libraries somewhere kind of deep down I
24:00
Didn't really understand why he was doing it But yes, I experienced this problem, and it's completely terrible Very hard to figure out in this case Well like this is basically an issue that
24:20
like it didn't take a long time for me to figure out why this was because I saw that some requests were just taking too long and We were basically Everything was being held up because of this one request that was doing like this thing So it was pretty simple for me to do it I basically just found the code, and I just literally threw it out. It wasn't necessary, so I replaced it
24:45
So that was my solution like I know the solution would have been to actually start using the a IOS TP client instead of the request library then everything probably like It would have been able to make this work I
25:05
Might need like just yes or no piece of advice, so what what do you think about like find a sink? Processes and waiting in the queue in the main thread to just pick them up Wait so running thousand processes. It's with like a nice async event loop or
25:31
Waiting in the loop on the queue, and they're just the process of writing inside does that make sense I? I'm not sure I understand but
25:43
Using processes for the using actual like operating system processes. Oh, yeah, I see I think there's probably a lot of overhead in that to be honest like a lot of overhead Compared to what a sink IO gives us which is very lightweight kind of you know just like some kind of task queue
26:05
It's running the queue If you you have to maintain all of these processes and the operating system has to switch between them correctly So I think there would probably be a lot of overhead in there Yeah, we can discuss the point
26:28
I can hear you No, I don't I don't but like I feel like I need to I'd be like like I have never done anything with transports or
26:48
protocols but Given that I'm writing an actual you know in production very important micro service I feel like I need to be on top of what this stuff is doing No, I'm not saying that absolutely not I'm just I'm just saying I should probably know what these kind of basic building blocks that are
27:08
Behind a sink IO. I should know what they do But no, I haven't used them and you really actually talked like talked about this earlier like I'm not supposed to be able No, I'm not supposed to need to know what these things are
27:21
So maybe I should just take his word Maybe but I have not known but I still feel like I need to know it
27:47
Yeah, yeah, so basically the API is no longer defined in swagger and we don't have this swagger UI for this particular microservice Which is I have not
28:00
So that exists. Yes. Oh, that's very good to know. That's very good to know. Thank you learning a lot today So
28:20
How should I put it like the code that I'm working with The company that I'm working for it doesn't use or amps a lot and definitely the projects that I have been using they don't so If you are yes You probably that's like this is one of the research things that you would need to do if you want to migrate a project
28:43
That's using on an ORM. You're gonna need to figure out if there is some support for or amps somewhere else I I don't know the answer like I I we don't really use ORM So we just use you know, like SQL queries directly. So yeah research it
29:07
Yeah, like I've probably run into some of these but like these are not things that I'm dealing with a lot these services that like For example, this service not complex. It's not it doesn't have to deal with a lot of timeouts and stuff like that
29:24
so No, I haven't really run into a lot of these no No worries