Asynchronous I/O and the real-time web

Video in TIB AV-Portal: Asynchronous I/O and the real-time web

Formal Metadata

Asynchronous I/O and the real-time web
Title of Series
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
Asynchronous I/O and the real-time web [EuroPython 2017 - Talk - 2017-07-11 - Arengo] [Rimini, Italy] Building web applications is one of the most common uses of Python. With a plethora of different web frameworks aiming at varying audiences, it has become a rather simple and well-documented process to develop web applications and web services. So popular it has become that the WSGI specification is the industry standard protocol for developing web applications with Python. But the WSGI standard is lacking with two major limitations: It supports only the ‘traditional’ synchronous http request-response cycle thus creating a ‘glass ceiling’ with the amount of requests a single server can handle – often described as the C10K problem It does not provide an easy and efficient way of developing two-way communication between servers and clients. Often referred to as the real-time web this capability has become increasingly popular with modern web applications. Over the past decade multiple networking libraries have been developed to address these limitations. In this talk we will explore: What is an asynchronous web server and how it differs from other, WSGI-based web servers. What is concurrency and how it can help us achieve supreme performance and scalability in our web applications. We will survey noteworthy libraries which will help achieve true concurrency. How to develop real-time communication, using web sockets, with our Javascript application Concerns, limitations and pitfalls when developing asynchronous web applications
Web 2.0 Medical imaging Arithmetic mean Estimator Context awareness Process (computing) Concurrency (computer science) Software developer Mereology Food energy Position operator
Metre Web 2.0 Digital photography Server (computing) Kernel (computing) Touchscreen Web-Designer Gender State of matter Software framework Information technology consulting Number
Point (geometry) Gateway (telecommunications) Computer-generated imagery Server (computing) 1 (number) Interface (computing) Cartesian coordinate system Binary file Product (business) Web 2.0 Latent heat Software framework Abstraction Proxy server
Dataflow Server (computing) Thread (computing) Dependent and independent variables Code Multiplication sign Virtual machine Product (business) Web 2.0 Different (Kate Ryan album) Computer configuration Operating system Monster group Physical system Dependent and independent variables Scaling (geometry) Server (computing) Database Cartesian coordinate system Flow separation Thread (computing) Arithmetic mean Process (computing) Right angle Communications protocol
Point (geometry) Scale (map) Server (computing) Dependent and independent variables Building Thread (computing) Scaling (geometry) Observational study Code State of matter Direction (geometry) Water vapor Client (computing) Event horizon Connected space Optical disc drive Web application Telecommunication Communications protocol Communications protocol
Frequency Word Arithmetic mean Group action Concurrency (computer science) Concurrency (computer science) Multiplication sign Cellular automaton Computer programming Form (programming) Neuroinformatik
Operations research Parsing Server (computing) Concurrency (computer science) Computer file Dependent and independent variables Multiplication sign Database Client (computing) Web 2.0 Web application Arithmetic mean Befehlsprozessor Different (Kate Ryan album) Query language Operator (mathematics) File system Statement (computer science) Diagram Statement (computer science)
Operations research Parsing Game controller Greatest element Digital media Multiplication sign Neuroinformatik Goodness of fit Mixed reality Order (biology) Videoconferencing Statement (computer science) Quicksort
Context awareness Game controller Server (computing) Functional (mathematics) Thread (computing) Code Multiplication sign Coroutine Similarity (geometry) Number Web 2.0 Revision control Term (mathematics) Synchronization Single-precision floating-point format Operator (mathematics) Core dump Weight Database Instance (computer science) Control flow System call Word Process (computing) Befehlsprozessor Query language Personal digital assistant Video game Pattern language Directed graph
Concurrency (computer science) Thread (computing) Code Multiplication sign Angle Mereology Formal language Mechanism design Different (Kate Ryan album) Single-precision floating-point format Computer network File system Arm Data storage device Computer Bit Thread (computing) Befehlsprozessor Process (computing) Telecommunication Right angle Physical system Point (geometry) Game controller Functional (mathematics) Socket-Schnittstelle Duplex (telecommunications) Web browser Regular graph Event horizon Programmschleife Telecommunication Operator (mathematics) Communications protocol Loop (music) Installable File System Operations research Weight Duplex (telecommunications) Cartesian coordinate system System call Single-precision floating-point format Loop (music) Kernel (computing) Event horizon Personal digital assistant Network socket Communications protocol Window
Point (geometry) Server (computing) Concurrency (computer science) Direction (geometry) Multiplication sign Client (computing) Mereology Web 2.0 Natural number Computer configuration Software framework Message passing Form (programming) Concurrency (computer science) Code Supersonic speed Client (computing) Bit Open set Event horizon Network socket Telecommunication Software framework Library (computing) Directed graph
Mobile app Server (computing) Concurrency (computer science) Code Multiplication sign Database Thetafunktion Telecommunication Software testing Right angle Descriptive statistics Physical system Condition number
Game controller Functional (mathematics) Gender Multiplication sign Weight Coroutine Similarity (geometry) Insertion loss Contingency table Event horizon Digital photography Mathematics Coefficient of determination Process (computing) Different (Kate Ryan album) Query language Normal (geometry) Right angle Figurate number Object (grammar) Resolvent formalism Library (computing)
on OK so before I introduce myself so I wanna tell you quick story so recently the idea of interviewed an engineer for a position who had no no to as background and love knowledge is is really great but this is no Jews no conference no I'm OK so we have a lot of energy is the ground and an estimate question I said so what he like about images and he said well it's a single-threaded awesome and I said OK what will what does that mean and he could explain it the so on so he get the job but very important part of the story is that it made me realize that maybe there are developers out there who either using and concurrency in asynch technologies or want to use it and they really don't know what it means in the kind of scared of this whole topic so I think it's interesting that I think this room alone hosted a 3 topics that talks about they sing they just today and which really is a pretty significant of and tells that a lot of people are interested in this so I'm the stock I'm really gonna try to demystify what this is and why is that relevant within the context of Web people so
the higher everybody my name is a meter a meter Navarro I come from Israel this is photo of Tel-Aviv on kernel looks like remaining as follows and it's really hot right now and the home but it's a lot of fun some I work for a of 4 7 5 cumulus which is a consultancy consulting agency on an idea rent and laid out on my idea to the feed on over there the OK so how many people here by doing web development with Python OK that's a good good number so all you guys probably
know at least 1 of the frameworks of roaches now on the screen of probably gender was the most prominent 1 or 2 of them 1 mostly used but there all other kinds like flask and and I think she really long list and and all those frameworks they have and they have 1st of all those our frameworks those and not web servers if anyone has ever made a mistake to confuse that those are libraries and but the important thing is that they all have 1 thing in common on and that the common
thing that share is that they all work or implement whiskey whisky is a standard which was 1st established in 2003 and basically what it is it's it's a glorified CGI bin ones old enough to know what that is on and it basically the whole point of whiskey was to try to and create a specifications where Python web frameworks can easily work under of on production Watson around and and and the way was the works is
that you write on in a framework which supports whiskey and then pretty much any production webserver like but you and UNIX can Hoke up to it and serve your application and you don't really need to know how that works and only need to carry can really just focus on the application on so what what's wrong
with whiskey yeah well actually nothing is wrong so I think you all have a great conference this difference OK I yeah the 2 things that are wrong with whiskey but 1st of all it's synchronous it synchronous in the sense that some it can't handle multiple requests at the same time and I'm gonna put that in in in quotes because it's not entirely true but I will talk about this in a minute and then the 2nd thing that's probably more important that it only supports the HTTP protocol my doesn't support any other protocol so it's dive to
elevate look so if you are writing a whiz GUI-based application this is kind of how you're flow looks like and you have a web server were whiskey server in every request comes in love with the server creates a thread of an OS Thread of an operating system thread the and then I dislike processes your your of requests your code runs in this thread and you handle it and you do stuff you get good database and you do all kinds of gene whatever your application what does and then when the response is ready it sent back to the client in during that time that thread is blocked OK so on if you were to run a whiskey several with a single thread and you want doing that when you're using 1 server Jangles monster it's a single-threaded of web server he can basically only yeah process 1 requested to around if running productions webserver like to Unicorn were you is the then potentially you can create as many threads as you want but if you only have few courses your machine that's really matter common thread you have you very limited but the amount of course of of and the CPU cost so I said earlier it's not exactly synchronous well all your code is running synchronously 1 right that 1 after another and then you really don't get them on a lot of options there status all these how do we how do we get a mean gossip huge systems out there Instagram running on Django Washington Post running on Jaguar hardly doing the holidays processing so many requests will what they do is the scale
yeah OK that's that's what they do they just running water In more services and they optimize their code and they run more and more servers and that's that you know it's pretty good approach can solve the problem ideally thing that is disappointing in this approach is that it's very linear in OK so eventually depending on your code depending on what you're doing let's say you're a single thread to a single score can handle yeah a hundred requests per 2nd OK so if you need to handle a thousand then you need 10 of those and if you need to run more than that then you need to duplicate you're URIs service so that the scaling is very linear and you cannot actually improve all where your pain points are and where you're on bottlenecks OK some suggest that's 1 of the biggest problems of whiskey and
the 2nd problem is that it only supports HTTP some studies so a CDP anyone does remember and is a stateless protocol it means that the client sends a request Chris is processed and then on the response goes back to the client and that's it there's no more connection anymore and if the poor policy stateless then it's very difficult to create stateful communication between clients and servers and stay for communication or directional communication or whatever you wanna call it to pretty hot item that when he was to due today on In an HTTP just as a support so at odds there are all kinds of work-arounds OK you can do long calling you can now do server-side events I don't know what that is I'm really nasty stuff we reached kind of like a glass ceiling OK was these wonderful allows such a huge community of engineers all over the world quickly build web applications with Python but for some perhaps is just not not really relevant anymore
so there is a solution and the solution is concurrency come from here so that's that's a hell of a word in let's let's see what what convincing means OK let's look at
wikipedia so according to Wikipedia concurrent programming is a form of computing in which several computations are executed during overlap in time periods concurrently instead of sequentially but yeah that's what Wikipedia's says and we all know that the 2 is never wrong so we're gonna take that for granted but when it actually means OK what action means of that cell it's if we can
use this diagram here on but before I talk about the diagram wanna make it's for a statement OK and my statement is that most web applications most of what they do is they perform I O operations OK I'm gonna buy Web applications mostly that's what they do they go to the database to fetch data and they go to the cash to fetch data the good a file-system Daisy send HTTP requests to other servers micro-services or what not these are all I O operations on those I O operations on are I O intensive but not necessarily CPU intensive OK and then what happens is what ends up happening is that those web servers they mostly just wait for things to happen OK we have those really powerful CPU's and most of the time just wait for Ohio to leave or come back from some other place so if we were to I'll look at this example that the diagram that I'm showing here if we have a single threaded ways the server and it got for request from for different clients it will process them sequentially OK you 1st do the blue 1 and then the orange 1 then the green 1 you guess the purple 1 day OK and concurrency is all about doing it differently and instead of doing this we want to do
this OK was it mean means that OK I am sending an I O requests to my post this database and while I'm waiting on a reply from my post these databases of waiting I can do something else I can handle another request which maybe once another post press query or maybe once we begin a file from the file system OK so instead of
during all those requests sequentially the sort of mix them
together and we hand over control from 1 request to another OK in order to optimize what the computers to and so here's
a nother way of thinking about it I if you look at the bottom video USIA 8 you all request and a six-year-old request happening sequentially OK 1st what is it teach 1 and then the white 1 and then on the top media you see them happening concurrently right they're both processed at the same time by the way they're very proud to be here today just to select an on so OK OK good good you're saying we can now process 1 request while waiting on the other request OK sounds good but how do you do that well there's only 1 secret ingredients to this nobody don't let anyone tell you differently you have to
explicitly to give up control 9 people don't like that term give up control but I think that within our context that's a very good thing to have and when it and and and the really the the emphasis here on the world on the word explicit while you run a query on your database in year code you will explicitly hand over control to some other requests that comes in to do its thing knowing fully well that someone else will eventually relinquish control and give it back to you once Europe query returned from the database today and that's kind of scary fir 1st timers but let's see if we
can clear case so if you've been doing some jobs that are in your of professional life and then you probably know this about pattern here on the left where I use J. query to get some data from a web server in this is so called by the way home and then as soon as that data comes in and I have a call back in an anonymous function in this case and the in I call I do something with this call that once it comes back and it was done just as before knows that if we look at the code up here on the left we know that do something will execute after do something else OK because there's something else is called immediately when get is done in do something is called as a callback 1 star out of their request finishes in Python we can do something very similar and it depends on which version you're running but in Python 3 . 5 and I'm sure you've heard it before we have those new cool words awaits and sync and essentially this is kind of similar it's not exactly the same but it's kind of similar where I call a function calls fetched data and fetch data is in a regular function is what's called core routine on asynch routine and if you remember earlier I said we relinquish control we say OK I sent that request now I'm gonna wait for it to somebody else here going to use the CPU and once my data is fetched I can do something and then I can do something else in this case do something is executed before do something else because in Python but unlike just 5 we have an easy way to write asynchronous code in a structured way while you just 5 we cannot just 6 copy difficult thing from by themselves now it's all they have awakened yield and what not and but yeah those concepts have been around for a while in Python before 3 . 5 we used yield from rather than a weight and we use of decorator rather than the word a sink OK so how does this work earlier I said that was the server operates under the assumption that every request creates a new thread and at the request is processed by the thread being that if you get a lot of request you get 1 thread pair request the 2 problems here problem number 1 is that thread management is done by the OS not by you which I guess most of time it's a good idea and and the 2nd problem is that no thread creation is expensive and limited creating a thread destroying the thread is inexpensive always operation it takes time not a single instance instances server is limited with the amount threads it can create OK therefore if you're a server now has to handle 50 thousand requests a 2nd and it's gonna run out of threads at some point but on the way
concurrency works is that everything is handled on a single thread arm and I like that interviewee found an accident explained to you I'm so what uh comprising non is using is using a concept which is called an event to event hope is something which is triggered by but the home and the mechanism in the operating system kernel in Munich school people I forget what it's called of windows and essentially what it means is that we can create functions we can call those functions but without we don't actually call them we just declared and stick them into the event loop and then at some point they can be pulled out executed and then shoved back into the event loop for all of our code to receive OK so are here on the right we see all those different kinds of operations which we can do so in an and asynchronously we have file system access we have data store axis and then a request comes in it creates so a coroutine it shoves it into the event will in the village has fewer right at each step you of a lot of our own of handles 2 call which you process then it'll process and 1 at 1 at a time and as soon as 1 call routine explicitly gives up control can with go back to that giving up control thing as soon as you call weights 4 yields from or whatever so hum mechanism in different languages you basically tail the event of OK i'm gonna wait now go ahead run those other I go easy here in your heart you and then come back to me when I'm done the and to me this calid good citizenship you know it's you you say OK I'm I don't waste those shared CPU resources I'm going away on my thing and then knowing well that I'm going to get back to CPU once my data comes back so essentially this is how an event loops worked in a nutshell OK it's still a bit more complicated than that but we don't have time for that today in and and then they're really solves the problem of processing a lot of requests at the same time the 2nd thing about which part is really nifty and cool is this thing called
WebSockets clinical back to Wikipedia here case so WebSocket is a computer communication protocol providing full-duplex communication channels over a single to speak and a case for sold do not mistake WebSockets with regular sockets it's not the same thing in a pay WebSockets is a protocol on top of HTTP which allow was created for single purpose and that is having a bidirectional communication with the browser that's it OK and has that work actually alone
you have a client climates nature of your request to the server and with a request upgrade upgrade my relationship with you from of single direction to be directional communication and then at that point both server and client can send back and forth requests from 1 another and then it from each side of can also terminate the request on if needed and that's how you the shortest the explanation of WebSocket ever in OK
so let's talk a little bit about what's available for us and when it comes to libraries that that that do concurrency we have twisted and tornado which are pretty old but we're tornado is just absolutely fantastic around in a few were forced to use Python to then turn it is probably your option otherwise I would that suggest to use a zinc ILO and it's part of the standard library and its has a growing ecosystem around it and it's very promising and if you need a web framework confuse framework would libraries and then on top of a sink I 0 we have sonic which I was exposed to 1st time here in this conference and I was really impressed she come from flask sonic would be really nice a at the p and is also fantastic option of using it for a while now and I'm very happy and very happy with the way forms in the way of moves forward and then if you are on Django and really doing non whiskey is not an option for you then gender channel seems to be a good compromise then I suggest you look into it
loans OK the advantages or efficiency the and self-evident but by what I just explained and having not not having to wait for something happened in the meantime being able to do something else and it's also the see 10 K problem seeking carried is of you can look up on wikipedia essentially of the description of what happens when you have a dense thousand requests per 2nd on your server on you can spontaneously and you can do it easily without a system like salary and not is amazing but if you just need simple stuff then you can do that without celery and obviously gives you a home bidirectional communication and and that's really you know if you wanna increase the user experience of your apps and bidirectional communication is kind of must
pitfalls very hard to debug even worse when you have to test travel time actually you are you really have to watch for locks and race conditions and once you get your hands dirty than running to those in the and the most important thing is that I tell that everyone if you don't rights yeah concurrent code all the way then you're wasting your time so if you have a concurrent server but you database access isn't concurrent then you've done nothing it's just it's everything is not concurrent so your code has to be concurrent always thank you thank you very much for listening
if it questioned loss and the question of the different things that the I'm so you photos and the process with the keyword a weight which hinders the they're giving up the control what does the other a keyword is doing so the asynchrony async is essentially telling take it it acts collective decorator it takes a normal python function and it says instead of just executing it returns something which is called a future and fortunately we don't have time to delve into that and then put that future into the events open that gets executed on the event time rather than immediately you can take but once you get once you dive into this and you get your hands dirty you figure this out quickly and yeah so you find that the main thing is that you should give up the control explicitly and can you compare or a contingency works about like solution like human or by the implicit a feature so G event talk thanks for bringing that up G event is is an implicit in the way of the handing over control of and by the way gender channels is using G events under relate events not really sure what the differences among and I think the main difference is that you you really have absolute control over when you are handing over control rather than let the library figure it out based on what's going on know I think that's the main difference Boris here a great great dog and so he said as saying arbiters of future Socrates certain futures and the awake keyword kind of gives control or to the bathroom and after all of this is ready we continue with the rest of the cult is that right is there an API if we want to do something more complex like work with the future object make a promise change when evidence resolve called this goal but something something like in joules world right absolutely absolutely that that the example I showed is like the simplest example of you can launch does the hundreds of coroutines finish calling all of them collecting all year your future objects and then wait for them you can say OK that happened 1 after another but if 1 of them fails I want out that you have a lot of flexibility if you coming from the jobs that world anew doing J. query would promises and then in wind and all that kind of similar it's not the same but you you have a lot of flexibility that so 1 final wish so then just this if you know you give it