Real Time Django
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 9 | |
| Author | ||
| License | CC Attribution 4.0 International: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/62305 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
6
9
00:00
Canadian Mathematical SocietyCore dumpCartesian coordinate systemUniform boundedness principleSoftware developerKeyboard shortcutLecture/Conference
00:39
GoogolMoving averageLocal GroupFunctional (mathematics)RoutingStudent's t-testServer (computing)Directed graphProjective planeCommunications protocolAddress spaceCodeRegulärer Ausdruck <Textverarbeitung>Different (Kate Ryan album)Parameter (computer programming)Filter <Stochastik>SynchronizationCartesian coordinate systemLibrary (computing)Moment (mathematics)Revision controlMessage passingCategory of beingGoodness of fitMereologyDebuggerReal-time operating systemWeb-DesignerRadical (chemistry)Process (computing)Silicon Graphics Inc.Sampling (statistics)WritingWeb 2.0ASCIIView (database)Point (geometry)Home pageLine (geometry)Web browserNumberElectronic mailing listComputing platformLatent heatCycle (graph theory)Computer configurationAsynchronous Transfer ModeMultiplication signVideoconferencingAttribute grammarRight angleBit1 (number)Streaming mediaField (computer science)Front and back endsGreen's functionIntegrated development environmentMathematicsSingle-precision floating-point formatWeb pageSlide ruleModule (mathematics)Phase transitionPerspective (visual)TDMATelecommunicationProduct (business)CASE <Informatik>Instance (computer science)Bridging (networking)Data storage deviceSoftwareDefault (computer science)Uniform resource locatorConfiguration spaceGraph coloringString (computer science)Keyboard shortcutDependent and independent variablesImplementationNetwork socketEmailSerial portSocial classWordSoftware frameworkConnected spaceFormal languageCore dumpComplex (psychology)LogicGroup actionClient (computing)Block (periodic table)BuildingConcurrency (computer science)Reading (process)System callState of matterSoftware testingLevel (video gaming)Data dictionaryEvent horizonDemo (music)MeasurementAxiom of choiceGodFreezingFAQFrame problemEndliche ModelltheorieDatabase normalizationScaling (geometry)TheoryoutputFunction (mathematics)RootMassContext awarenessArithmetic meanDiscounts and allowancesComputer fontInternet der DingeCountingExpandierender GraphSocket-SchnittstelleLecture/Conference
Transcript: English(auto-generated)
00:07
Oh, yes. So I'm happy to be here. I'm Jacobos Paleti. I'm the founder and CTO of Nephila. We build Django applications every day.
00:20
And I've been working with Django since 2009. And I'm a developer of quite a few Django applications and Django CMS core developer. I'm also a beer enthusiast, as you can see from my shirt. So if I'm not at the keyboard, you will probably find me in some pub having some beer.
00:43
So real-time web. For some time now, the web is not more the usual request response cycles over and over and over. And it's much more complex and rich. And to achieve this, you have to lose a lot of tools,
01:01
which is really nice. The little nerd inside us is very happy about using new tools. But in the end, you end up hitting a complexity wall where you are using a lot of different languages, different infrastructure, and keeping all these things together. It can be quite complicated.
01:24
So why not doing the Django way, at least this part? There are a lot of tools in Python and also in other languages that solves this problem already. But if you are working with Django, having an integrated solution would be really nice.
01:44
So channels. Channels has been created to go beyond the usual synchronous way of the working of Django, which only deals with HTTP request response. And it's been developed outside of a core.
02:03
And then it's been adopted by the Django project as an official Django application. And channels is a framework to deal with asynchronous protocols, basically, not web specific.
02:21
So WebSockets is obviously a first class citizen inside channels, because the obvious first way to interact with asynchronous protocols from a web development perspective. But it can be used for any kind of protocols, both IoT or whatever you may need.
02:44
But for the rest of the talk, I assume WebSockets, because it's the more natural environment for us. So while I was preparing this talk, I have the nice news of Channels 2,
03:00
which has been released two weeks ago. And it scared me a bit, because Channels 2 changed a lot of things. And so I decided to stick with Channels 1. So all my code and examples will be for Channels 1. I'll talk briefly about the changes in Channels 2. And the reason why I still stick to version 1
03:21
is because I live tested the version 1. And the sample application is actually a reduced version of something we have built for our clients. And Channels 2 still not have 100% of the Channels 1 features. And in the end, I had little time to write all the talk and all the examples for Channels 2.
03:43
But this doesn't really matter at this stage, really, the version you're using. So this is the talk outline. I will briefly introduce the concepts of Channels, which are different than the Django ones. And if you don't know Django, that doesn't really matter.
04:03
I will make a few references to Django. But Channels is not really, it uses Django, but it's not completely tied to Django, at least for code matters. Then I will show this example application.
04:25
And then we will go down to the code and see how it works, to see how you can build your own real-time applications. And then we will have a few highlights of Channels 2. So what's Channels?
04:41
Channels is based on Channel, which is first in, first out, at most once queue. It's important, at most once, its concept is very important, because you are not guaranteed that your messages are delivered. And this is basically for performance reason and also for the use cases that a real-time application
05:03
has. And a channel is a channel to write and read messages. So you have producers, which are the protocol server and the consumers, and are read by the consumers.
05:22
In this context, let me simplify things. Asynchronous means even driving. So you have events that arrives to the protocol server somewhere, are generated and put into the Channels. And something will pick these events as some messages
05:43
according to the events. Asynchronous code is more complex to understand, because it's non-linear, obviously. And the choice that Channels 1 made was to hide the synchronous part. Channels works asynchronously, but you don't see it.
06:03
You don't write asynchronous code. This has been a little changed in Channels 2. The whole Channel concept has been, I'll say, freezed under this SGI protocol, which
06:20
is basically the specification that brings the synchronous world into the WSGI. So it's like a version of a synchronous version of WSGI. The version 1 was built for Channels 1, and the version 2 for Channels 2, quite obviously.
06:41
And version 2 is much simpler to implement. That's one of the reasons for the Channels 2 approach. The protocol server is the thing that speaks to the network. So it implements the ASCII specification. And it's the bridge between the network and the application
07:01
For example, DAFNE, which is the reference implementation, speaks WebSockets and so re-takes the data from the network, from WebSocket network, and put into the channels. And it's its responsibility to create the channel, so every protocol server can define its own channels.
07:25
Then you have routing, because when things come in, you want many different consumers or many different functions to be called on these messages. And so the routing is a way that you can say to the channels how to route the messages
07:46
to the different consumers according to some properties, which are called filters. And so in a way, it's similar to Django URL Conf. If you are familiar with it, you put the URL, you put some parameters in that, et cetera, et cetera.
08:03
And then there are the consumers, which are really the parts you are going to write. It's a code. It's your code, basically. The role of consumers is read messages from channel, do something, and write messages on the channel again.
08:20
They are similar, in some way, to the Django views. But the point is that you put messages in the channel. You don't return anything. You just write something in the channel. And so messages are moved around, both in the input and output. And then you have the front end.
08:41
Obviously, if you use WebSocket in a browser, you will have to write some JavaScripts. And Channels ships a very lightweight library to help with this. But you can use whatever WebSocket client you want to achieve this. And so enough theory.
09:01
Let's see something in action. I created this application, which is very stupid. The application is a simplified version of something we built for our clients. And the logic is not super fancy,
09:22
but allows you to get a bit of how channels work. So I implemented three features, like all these poor folks Google Docs. You can count how many active users you have currently on the platform. You know which document has been opened by which user in which state.
09:41
So you can actually do some concurrency checks. And you get browser notification where something happened. And as I don't want to sacrifice kittens to live demogod, I prepared this video.
10:04
In this video, you see on the left, sorry, yeah. On the right, you have Leela. And on the left, you will have Fry. They're both using the same application. And with this counter, you will
10:23
see the number of active users on the platform. So at the start, you have only Leela working on it. And so you have only one user. Then at some point, Fry will join. This is the notification message that tells you
10:43
if you want to enable it. You have to write some very brief code to ask the browser to allow this. And, oops, come on, let Fry log in again.
11:06
And you can see here the number has been increased to two. The documents has this green label. At the moment, the labels are lower green because no document has been opened. Now Fry opened this document. And this label become yellow.
11:22
And you have the Fry name on top of this. Now Fry is going to edit the document. And the label will become red. Fry will make some changes. But Leela at this point knows that Fry is doing something. Then Fry saves the document.
11:41
And you have this notification here. Leela want to check what Fry has written. And Fry gets this notification that someone has opened the document. And that's all for this sample application.
12:01
OK. So let's see some details. These are the three main things you are going to work on when you create a channels application. And channel layers is basically the configuration. You just specify the backend, which
12:21
is the message of storage. And Redis is basically the default recommended. But you have many others, and you can write your own. And then you have the routing, which as we have seen here, is basically the URL conf of channels. The project routing is very simple.
12:40
You're basically going to include the routing of the different applications you want. And OK. And this is the application routing, the one I included up there. It's pretty dense. But in the end, you map all the consumers,
13:01
which are these classes, to a channel, which is usually included as an attribute, provided as an attribute by the consumer, and this filter. WebSocket has only path as filter. But other protocols may specify others.
13:22
Like, I don't know, if you have an email protocol server, you may have the address and whatever you want. And with the usual regexp, you can parse the path and get more arguments to pass to the consumer. And you have this as root or root class
13:43
to map the consumers to the routing. And now let's go to the consumer, which are actually the interesting part of building a channel application. Consumer for WebSocket, because at this point, you really need to know the protocol you are using,
14:03
are something like this. I'm using this JSON WebSocket consumer because it deals with a lot of low-level stuff. So you can express much more clearly what you want. And then you have these three methods, which are the events that are exposed by channels.
14:26
And you have at least to implement one of these. Otherwise, your consumer won't get any event. But you can implement anything you want, according to your needs. And then you have others, like this one, which will provide you data about the session
14:42
and the user picked from Django. So the first thing we have seen is user counting. So how we do this, the idea is that whenever a user entered the application, it opens a WebSocket
15:01
onto an endpoint. And this triggered the connect event. And when the connect event is triggered on this specific endpoint, I get a message here. And so I can pick the user, increment the user counter in some way, and then report back to the user
15:21
how many users there are. But I'm not just sending the number of the users to the user who just connected, but all the users in this specific group. So all the connected users that are in this group will receive this counter.
15:43
When it disconnect, I obviously decrease the number of users. When a message is emitted, the browsers receive it. And thanks to the channels JavaScript library, I can just create this bridge, connect to the endpoint, and I start receiving messages through this listen function.
16:02
Whenever a message arrives, I know it's the kind of messages with the user number, and then I update the DOM. Now we are going to check how many.
16:22
We want to know not only how many users there are, but also what they are doing on our application. And so we have a bit more complex status because we have the status of a single document, and we also want to inform the users which are on the home page, on the dashboard of our application.
16:42
So this is a bit more complex. But the idea is that when the user connect and message is received, I update the status for my document, taking into account the slug, for example, and the phase, which is really if it's opening in read or edit mode.
17:00
And then once I have registered this, I prepared this message, which are just dictionaries. And I sent to all the users attached to the group named after the slug of the document. So these messages will be sent to all the people, all the users connected to my document,
17:23
whether in read or edit mode. But then I want also to update the users which are on the dashboard because they want to know which documents are opened. And so I sent to this second group, which is just an attribute for convenience.
17:43
And so when a user connect to a document, it updates all the connected users, both the ones on the current document and the document, and the others attach it to the list. And this is the second consumer for the users connected to the list.
18:01
This is basically this previous version here that's expanded. I just cycle through all the documents. This is Django EM. This is basically, I think, the only line of Django code in this slide. And I create this dictionary separating users according to the phases, et cetera,
18:21
and I send to the same group. Thus, in the front end, I listen to the messages on the new path. So on a new endpoint, and according if I have a single document of ORM list, I update the DOM by changing colors and doing some stuff.
18:41
The third one is the browser notifications. So people works with documents updated, and users may want to be notified of any changes going on. For this, I use it at an higher level API provided
19:02
by channels one, which is no longer in channels two. But I think it's nice, so I decided to show it anyway. And there will likely be out of core implementation of this because it's very useful. You have basically the binding, which is a serializer. And that's the reason why I have been removed,
19:21
because there are a lot of serializers in Python world and having another serializer, which does exactly the same thing, is redundant. And you pass it a model of fields you want to serialize. And then you add this stream attribute. Streams are the subchannels that
19:40
are used by the multiplexer. The multiplexer is basically a concept in communication. You can put more signals basically in the same beam or whatever. In this case, in the same channel, you can put different kind of messages.
20:02
And with multiplexer and demultiplexer, you can combine this. This allows to create more general consumers. That's basically the reason for demultiplexer. And in this case, it's very simple. I just map a single consumer to a stream, which is actually a string, and I declare a group.
20:28
And that's all. Data binding uses Django signals to notify the users. So whenever a document is updated, the data binding picks the message,
20:42
create the serialization according to these fields, and it sends to this group by using the multiplexer. The multiplexer are normal consumers, so you just put on the writing like this. I just used the two different syntax,
21:02
but they are equivalent. On the front end, you use a slightly different function, which is demultiplex instead of listen, because you have to take the stream into account. But other than that, it's the same thing. You get the data, and by using HTML notification,
21:21
we can send the notification back to the users. And so I hope it's not been too fast, but you can have a look at the application, which I commented a lot, so hopefully it will be useful anyway. And now, channels two.
21:43
Channels two is basically a completely new project, because it's based on asyncio. This means it is Python 3.5 only, why channels works on Python 2.7. And you can mix synchronous and asynchronous code in the same consumer and with Django code, and it's great.
22:04
I mean, you can define async def and then asynchronous function, and the other method is asynchronous, and this allows to make much more complex things, because one of the channels is very limited in some way, in a good way, but it's limited anyway.
22:23
And the other big thing is that now, the protocol termination is done in the same process as Django runs and where the consumer runs. Because in channels one, you have a protocol server, a process, and you have multiple workers.
22:41
And this brings, obviously, much more infrastructure complexities, and with channels two, this is superseded. And the other big thing is that you have, need to reduce this slightly, you can have persistency of your consumer
23:03
across the whole connection. So you can store stuff on the consumer. And then you have middlewares, which works slightly like Django ones, so it can filter incoming connections before passing to them to the consumer.
23:20
And all the building blocks of channels are now SDI, so you can mix and match together and with a more flexible approach. Being completely right, a lot of function, a lot of features have been removed. Many of them will not come back. Some has not really gone away, like group view
23:43
don't have anymore the group class, but group concept is there, obviously. And some, hopefully, will return soon, like the multiplexer. So I have my channels one code. How hard it would be to write?
24:00
Not too much, because the complex part, you're really writing, the consumers remained almost the same. There are some things that has changed, but not really dramatically, so it's not a big deal. You have to write your test completely, because it works completely different. And routing is a bit more complex, but not really
24:23
that hard. Very brief FAQ. How is performance? Yeah, quite good. It scales. The thing in Django, you care about performances, but it's not the most performant framework in the world.
24:40
There are other things that are valued, and so goes for channels. But if you need more performance, you can scale your workers, because you can use multiple workers anyway. And deployments, you have two different, you can use two different deployments. You can get rid of whiskey completely and put Daphne in front of everything.
25:03
Daphne will run channels and Django. Or you can run Django with your whiskey server like usually, and then map web sockets to a different path and pass it to Daphne. I use the second option, because I like micro whiskey,
25:21
and I use a lot of its features. So wrap it up. With channel, I've never written asynchronous code before channels, and I find very easy to write one. And also because it uses the Django API. You can use the Django API.
25:40
You can use everything from Django and channels, and you don't have to deal with asynchronous really. Thank you. I don't know if there is more time for questions. We have a little bit of time for one or two questions.
26:01
Raise your hand if you want to ask something. Thanks for the talk. Just the pop-up when Fry logged in, is that required for all web sockets? Sorry? The pop-up on the left that you have to,
26:23
there's no way around getting the user to click OK. You can just close it. You're not required to click on it and open a page. You can just close it. There are the usual notifications that Google Calendar, et cetera, uses now. So it's pretty standard.
26:41
OK, so you can still use the page? Just don't click OK? Yeah. This is really convenient thing, but I'm a bit afraid of doing this in production. So it's definitely the only server right now
27:03
available for ASGI. Or is there, for instance, an Apache module for ASGI? Not that I know of. And definitely, I think it's really the best option you have, because it's channels, basically.
27:20
So it's developed by the same people. And it's the default environment they test it and they work on. So it's really the best option at the moment. And I don't know if there are any other protocol server for WebSockets. There are other protocol server
27:41
for ASGI implementing protocols other than WebSockets. And that's one of the cool thing about channels. You are not forced to use WebSockets. You can use any protocol you want. Thank you very much. Let's do one more question. Does it only work with Django?
28:03
Because Django is not in the name. So would it, for example, also work with Flask? It requires Django to, I mean, at the moment, it's tied to Django. You don't need to use Django code in the consumers.
28:21
So if you have a Flask application, you can just set up a very tiny Django project just to run channels. And then in the consumers, just use the Flask libraries you are using. But the idea of the ASGI protocol is that this is not really completely logically tied
28:43
to Django. So you can have Daphne running the consumers and your own Flask or whatever application running on Wysgi that handles all the synchronous part. Thank you. All right, thank you very much.
29:01
Thank you.