Phoenix Channels: A Distributed PubSub and Presence Platform

Video thumbnail (Frame 0) Video thumbnail (Frame 2238) Video thumbnail (Frame 4451) Video thumbnail (Frame 9357) Video thumbnail (Frame 13381) Video thumbnail (Frame 22845) Video thumbnail (Frame 31780) Video thumbnail (Frame 40435) Video thumbnail (Frame 48293) Video thumbnail (Frame 56321) Video thumbnail (Frame 63872) Video thumbnail (Frame 65003)
Video in TIB AV-Portal: Phoenix Channels: A Distributed PubSub and Presence Platform

Formal Metadata

Title
Phoenix Channels: A Distributed PubSub and Presence Platform
Title of Series
Author
License
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Identifiers
Publisher
Release Date
2016
Language
English

Content Metadata

Subject Area
Abstract
Channels are a really exciting and powerful part of the Phoenix Framework. They allow us to easily add soft-realtime features to our applications. Channels are based on a simple idea - sending and receiving messages. In this talk, we'll take a look under the hood and learn how to build incredibly powerful, event driven systems with Phoenix Channels. We'll also look at how Phoenix makes it easier to track presence in a distributed system.
Focus (optics) Link (knot theory) Multiplication sign Core dump Real-time operating system Twitter Web 2.0 Uniform boundedness principle Web service Arithmetic mean Different (Kate Ryan album) Internetworking Computer science Computing platform Computing platform System programming Software framework Library (computing)
Web 2.0 Socket-Schnittstelle Web service Server (computing) Dependent and independent variables Distribution (mathematics) Software framework Right angle Software framework Client (computing) Connected space
NP-hard Functional programming Building Concurrency (computer science) Model theory Multiplication sign Virtual machine Set (mathematics) Semantics (computer science) Rule of inference Computer programming Power (physics) Neuroinformatik System programming Macro (computer science) Physical system Domain name Programming language Focus (optics) Standard deviation Functional programming Concurrency (computer science) Forcing (mathematics) Model theory Cartesian coordinate system Process (computing) Telecommunication Scheduling (computing) Routing Erlang distribution
Socket-Schnittstelle Server (computing) Game controller Presentation of a group Enterprise architecture Service (economics) Dependent and independent variables State of matter Multiplication sign View (database) Drop (liquid) Client (computing) Event horizon Time domain Web 2.0 Moore's law Distributed computing Representation (politics) System programming Message passing Physical system Domain name Enterprise architecture Dependent and independent variables Email Information Model theory State of matter Database Connected space Uniform boundedness principle Message passing Event horizon System programming HTTP cookie Communications protocol Abstraction Reading (process)
Concurrency (computer science) Transportation theory (mathematics) State of matter Code Logarithm Multiplication sign View (database) Client (computing) Parameter (computer programming) Mereology 10 (number) Web 2.0 Sign (mathematics) Pattern matching Different (Kate Ryan album) Semiconductor memory Network socket Functional programming Recursion Error message Physical system Scripting language Programming language View (database) Token ring Lattice (order) Connected space Message passing Process (computing) Right angle Reading (process) Online chat Row (database) Point (geometry) Server (computing) Game controller Socket-Schnittstelle Module (mathematics) Service (economics) Token ring Virtual machine Drop (liquid) Web browser Event horizon Lattice (order) Frequency Chain String (computer science) Operating system System programming Video game console Interrupt <Informatik> Message passing Routing Module (mathematics) Distribution (mathematics) Demo (music) Information Server (computing) Plastikkarte Client (computing) Cartesian coordinate system System call Uniform boundedness principle Error message Personal digital assistant Network socket Object (grammar) Routing Communications protocol
Trail Server (computing) Presentation of a group State of matter Multiplication sign Virtual machine Product (business) Online chat Web service Centralizer and normalizer Different (Kate Ryan album) Shared memory System programming Software framework Physical system Email Information Server (computing) Weight Surface Data storage device Electronic mailing list Database Line (geometry) Cartesian coordinate system Connected space Type theory Process (computing) Personal digital assistant Blog Right angle Figurate number Quicksort Resultant Local ring
Freeware Consistency State of matter Multiplication sign Virtual machine Replication (computing) Event horizon Computer Revision control Mathematics Type theory Spherical cap Synchronization Different (Kate Ryan album) Armstrong, Joe Operator (mathematics) Theorem Message passing Partition (number theory) Physical system Consistency Computer network Line (geometry) Timestamp Dirac delta function Replication (computing) Connected space Partition (number theory) Message passing Software Order (biology) Synchronization Remote procedure call Figurate number Data type Routing Modem
Laptop Metre Ocean current Trail Presentation of a group Module (mathematics) Service (economics) State of matter Multiplication sign Electronic mailing list Differenz <Mathematik> Event horizon Goodness of fit Mathematics Differenz <Mathematik> Mechanism design Network socket Vector space System programming Video game console Functional programming Physical system Scripting language Service (economics) Email Trail Demo (music) File format Server (computing) Model theory State of matter Electronic mailing list Process (computing) Vector space Personal digital assistant Network socket System programming Right angle Object (grammar) Modem
Email Trail Presentation of a group Service (economics) Code Multiplication sign Electronic program guide Insertion loss Electronic mailing list Client (computing) Web browser Neuroinformatik Workload Roundness (object) Different (Kate Ryan album) Queue (abstract data type) System programming Diagram Software framework Functional programming Mobile Web Service (economics) Email Information Demo (music) Structural load Model theory Electronic mailing list Code Principal ideal domain Maxima and minima Bit Type theory Message passing Process (computing) Workload Personal digital assistant Quicksort Pressure Asynchronous Transfer Mode
Scheduling (computing) Run time (program lifecycle phase) Concurrency (computer science) Thread (computing) Video projector Structural load Code State of matter Multiplication sign 40 (number) Insertion loss Client (computing) Run-time system Computer programming Neuroinformatik Mechanism design Semiconductor memory Different (Kate Ryan album) Befehlsprozessor Core dump Cuboid Software framework Process (computing) Functional programming God Physical system Covering space Programming language Spacetime Structural load Moment (mathematics) Electronic mailing list Coordinate system Parallel port Bit Benchmark Message passing Process (computing) Crash (computing) Telecommunication Order (biology) Queue (abstract data type) Erlang distribution Resultant Server (computing) Socket-Schnittstelle Link (knot theory) Virtual machine Rule of inference Number Element (mathematics) 2 (number) Crash (computing) Operator (mathematics) Operating system System programming Speicherbereinigung Energy level Distribution (mathematics) Standard deviation Demo (music) Memory management Uniform resource locator Personal digital assistant Network topology Scheduling (computing) Abstraction Library (computing)
i. i welcome everyone thank you so much for coming and journey today we're going to be talking about phoenix channels distributed pubs and presence platform. and we're going to be talking about phoenix today in phoenix has some other features but we're going to kind of focus on some of the more real time soft real time system that's provided with in phoenix and gets a little insight into some kind of academia a computer science research that was. the implemented inside phoenix to provide some really nice functionality so. my name is sunny squadron. one of the fourteen members on the phoenix framework and you can find me around the internet i go by scruggs and so if you want to follow me on twitter i can without some links to various things the talk about during this time here and so i. and also have a bunch of different looks for libraries that i author and maintain as well that the you might find useful so really my goal today here is to really convince you to take a look sir are laying and phoenix seriously and to really consider using it. in your stack at work. so what is phoenix so phoenix is what i like to call a distributed web services framework so what exactly do i mean by that while the web itself right we have this idea of web frameworks are generally all a.
about this you know request response kind of thing right and we the they do that pretty well and most of them all kind of provide that otherwise would be pointless right so but the web is evolving. over the last few years we've seen that web technologies are are transitioning to a stateless web to more straightforward and so that provides more efficient ways of communicating with clients and servers and or servers and servers to. so it's no longer about just request in response we we have web sockets. if you are you in a modern browser and and and we also have h.t.t.p. to that's kind of on the on the works right now so in the future we're going to be really able to tap into this kind of connected. web services so and that's really what phoenix is trying to do are trying to reinvent the the modern web framework so kind of reinventing what it means to be a web framework.
so phoenix is written elixir which is a wonderful beautiful language a functional programming language and it is basically the syntax itself is very commonly compared to be but that's about all the room. the influence that that it is just some syntactical kind of things on top. but one thing that's really nice about ruby is the community and the tool sets and so some of those things have been really inspired by by what really has. let's also has a really incredible powerful macro system that allows you to do metre programming and so we've actually taken advantage of that with in phoenix to provide a really nice d.s.l. for reading so you can build up your routes in a very nice way to express. things in your particular domain and so. of course the one of the really awesome things about luxor i think that it's probably the most amazing thing about elixirs the fact that it runs on the ruling virtual machine and so we've heard a lot about the or language machine this week during various talks and so i i actually had like probably twenty so. whites was going to show how the v. and works and all these really cool things that that erling provides for us but i figured you know there's already been a lot of talk so i district those out and will focus today on phoenix itself but for those of you who haven't seen those other talks. we can get just a just a brief introduction if you don't know about it. or lang was developed at ericsson about thirty years ago and it powers pretty much most of the world's telecommunications systems today and if you know anything about telcom we we really rely on our phones and want to make sure that those things are are up and running and so are lang provides ways to build fault on systems that don't go down and they. run for many many many years. and so with airline we can take advantage of all of what the t.v. and does and let it do all the hard heavy lifting force you know so early has wonderful failure semantics that allow us to respond to failures really nicely and of course it's also distributed programming language. so anyone see jos talk yesterday. i was awesome i love the had a really good time but it's all about distributed programming and distributed computing and are lying has those semantics built into the language and so and so therefore does a lecture and it really makes building the system's really enjoyable and makes a lot easier as well. so as i said earlier is a functional programming language it's based on immutable data and it's got actor base concurrency model and it also has that pre-emptive schedule or so it allows processes erling process is to be scheduled across all your corps.
so back phoenix phoenix. provides you a way to build applications as i said before and so we have the ability to write you know the kind of stateless things that most of us earlier of familiar with today building jason a piazza or crowd applications kind of standard stuff and of course those are neat though those are needed.
regardless if we have the ability to build kind of more state full stuff we still need to kind of drop back in to provide some kind of synchronous a p as and stuff. but where phoenix really shines is in state full services so persistent connections with web sockets or any kind of the protocols of work clients can connect your server so it's more efficient because as we already know and state. stateless kind of systems you have a lot of overhead every time you want to check to see who's who's actually requesting this thing because we don't have any a way of knowing of other than like using cookies and sessions so of course because it's built on all on or lang and looks or we get the nice ability to build distribute systems. so let's talk about stateless for second just really quick essentially we're probably all familiar with this you get a request and you do response so this is really inefficient every time you can request you have to get some cookie value and find a user or something like that and then. you keep going to the database on every single request parsing headers and all that kind of stuff takes more computing power in time to do those things and. we have better ways of doing that but as i said this is still necessary for some for something so in in phoenix it's an n.b.c. kind of architecture so you have your domain model where you get a request we parse out the the route and and we kind of choose which controller is going to be routed to based. off of some reading information and then that controller might go often talk to the your domain models would not get some make my do some data access and in anything like that get that data back present to the view it will give us jaison regime representation and then we can send response. presenter stuff. so what state full is where we have given event driven or message driven design and and this is really where i think phoenix shines and we have this abstraction that we call channels and this is an abstraction around pubs of so we can.
subscribe to particular topics around the system and when those things happen our pub subsystem will push it out all connected clients and and we have this kind of thing.
so persistent connections between the client server and so as i said we work off this idea of topics and a topic essentially just any kind of identify are usually it's just a string right so and by convention it has two parts so it has the topic here. kind of the main topic and then we also have the sub topic so and the sub topic this kind of way that you can use these works really nicely with the way that message reading works so you can actually have this this room prefix here. works really nicely with how the channels work so we can have this channel we can say that anything prefix with room and colin here will we have this asterix which is a wild card that says route anything with the prefix of room to this room control or so and this allows us to. have build build up handlers for that and so this is the server a.p.i. four channels so we have we work with channels we have a socket right so we plugged in our socket into our web server and we tell it that any time you see a route messages coming in or connections on these. slash socket in point i want you to route through this user socket module so our user socket is the the module that is going to be responsible for maintaining the connection and connection state and things like that. and and it's got a couple things to know in here so this is again the the the reading information of reading messages to channels. and then we also have the transport so the transport that we ship with we have to transport for web sockets and we have a transport for long polling so if you still have to maintain applications for my customers or or users that are using all browsers that don't support web sockets we have long pulling as well. and these transports you can build your own safe you have. x m p p or am q.b. all these different protocols that you want to use you can actually build your own transports and have them connect to your web server. so the couple callbacks here we have to connect and this receives like some parameters for when you're actually connecting so you might have liked a user token that you know is that allows us to understand what user it is that is trying to connect. and then you can just return this ok socket to pull to allow the connection to can complete so. so the channels themselves these modules have some different callbacks that you can implement and the main once you're drawing so you have joint when you're joining a particular channel what you're going to do is tell it the first argument is the actual channel name. and you can see here we actually have two different join functions and in erlangen elixir reuse pattern matching to to route these things and so explicitly on the first one were reading to room colin lobby so that one will be chosen automatically by the m. if we're trying to join the lobby channel and so in this case as a you know just a demo were saying that anyone that wants to join the lobby can go ahead and join it's no big deal because we want people to communicate and they don't have to be authorised or anything like that but the second one down here were saying were matching on the room and. we have some subtropics so and in this case we can maybe say well we knew we want to check to see if that user is actually authorized to join this channel so we can offer some service and say hey is this this user a member of this of this particular room and it will return back you know billion to. false and if it's true we can return ok socket which will allow the connection through otherwise we can return error and a reason for an offer us. so we also have the message. for for handling incoming messages so whenever i sent a message on my of my channel it's going to come to this module and it's going to hit the call back so handle in handel's incoming messages and the the first argument here is the the event itself so we have. a new home message and it will take the message and our socket which is arc our state so in are laying in an elixir we just use recursion to records over our state and that's how process is kind of stay alive and so we always have to receive our state and return it as well so that's why we have these callbacks at the very. and of every one of these function calls are returning it to pull that has the socket in it which is art which is our states so we get this message and then we can say cool i got this mess is now want to broadcast it to everyone who's listening on this particular topic so. now that we see that let's look at the client a.p.i. now warning javascript might be hazardous to your health so i'm in this is a drop of clients kind of example of how you can use it to import the socket this is the yes six jobs per cent tax it actually makes jobs. a slightly more bearable. and and you can actually imported from the phoenix package and then using that socket we can set up new connections so will say we want to connect to our server at slash socket. and we also want to.
connect with some parameters so we can solve a sign like a user token into the parameters that are sent back to the server and then we can connect. we can then take that connection or the socket itself and set up a new channel so we say socket channel you pass in a particular topic that we're interested in and then we get a handle on that channel and then we can actually join it so once we join it this is actually done in. kind of a neat way where we send a message to the server what we set up some callbacks to say that a win win this actually returns then i want to do these things so we can say what i once i get a message back from the server i can either get an indoor message so forgiving ignore message then we. some often authentication errors so we're not allowed to join or something like that. and if we receive in ok message that we know that all is good and now we can do something else that we can also of time out as well so we didn't get something back in in a certain period of time we can fail we can also set up of callbacks for different channel of and so on clothes or on air. there are things like that. and then we can push so this is the push a.b.i. we want to send a message to the server say channel to push and then we specify a particular event name so in this case new message and then we just provide a job script object literal that will be. the serialized into jaison to be sent back to the server and then of course we can listen for for these messages as well so when we send this message to the server than our channels going to broadcast it to all connected users including ourselves so this is the call back that will be hit when we get new messages that coming. that is. all. or. yes so the question was can you use can you build your own jobs for client and yes absolutely you could so clients or can be implemented any language so i'm over that emit so actually right here so looking at this from the outside view we have phoenix as the server and we have all these connected clients. so clients can be implemented in any language so if you did want to read do your own client itself because you don't like arts that's no problem just build yourself so in this case we can have i less clients i think there's actually have a client's built and swift and objective see we also have an android klein. and that's available and for the dot net community it would be really awesome we had clients and c. sharp an f. sharp and stuff like that and actually for this particular talk i was going to be working with a with a tedious and and tomas and trying to get them to build build an f. sharp climb but just in we can make the the timing work so. but how this works we have all these clients connecting to the server and then let's say you get to a point where your exhausting the resources on on that one machine well with erling elixir it's just distributed programming language so no big deal we can just add another note and we can connect them together. so and then we can have other a client's connect to different nose and everything still works so all the messaging will still be delivered between all the nodes and all the clients and it just works seamlessly thanks to our legs distribution so that's all fine and dandy but let's take a look. look on the inside and see what is interesting about this so when you have a client the connex. using this travesty client a p i hear was of socket connect there is a process that is added on the server and when we say process in erlangen elixir we don't mean an operating system process but we mean v.m. process and this is an earlier be a process so it's a very very lightweight process. very low memory and it has its isolated has its own memory access no one can kind of region and touch it and mess with so it's very safe. so then that user socket is going to register itself with the pub subsystem. then the client can say hey channel that join on i want to join the lobby so that message will come through the connection will hit the user socket the user socket will spawn a new process for the channel. so the spawn this channel process and that channel process will be registered sell for subscribe to that topic with the pubs of system and same goes for any new topics that we want to spawn so every topic has its own separate process so if you have a bug in your. code and you end up crashing a process it doesn't affect any of the other running processes and the system and this is really great so again or lang processes are isolated and their concurrent they run concurrently through the system so you don't have to worry about crashing one process and affecting anything else. so the. when you want to broadcast say this green process over here uses are broadcast a.p.i. you can it will send it to the pub subsystem and then the pub subsystem will then send the message out to all of the connect the processes that have have subscribed to that topic and then that will make its way to the. users socket and then down to the client. so that's all great those are you can do that with no g s right however of course you don't have the fall tolerance and and concurrency the way that you do with erling but you can you can do it right in other systems allow you to do the same things i'm sure you can do.
to all this stuff with dot net in and everything else but we have something else that is quite interesting so presence presence sure we're all familiar with what kind of is essentially no chat systems and and things like that usually implement these where i am online.
cool i'm going to tell everyone that your online and then you go offline and i tell everyone that you go offline so we its this way of tracking process is to see who's online who's not sounds pretty trivial right well this definitely seems trivial on the surface but there's a backs. sort of this like why why would we actually implement something so trivial within a web services framework and and really the the story goes that so this is chris accord is the creator of phoenix and then we have just a lame whose the a creative elixir and these guys are both work with. we all work together on on phoenix together and so you know the hello world of have a look of hurling elixir is building a chance or so everyone build a chat server because it's a really good thing to do in erling it's very easy so lot of people come in. i received the mailing lists and ask how how do i implement presence like how do i keep track of whose on line and stuff and and make sure that it's it's actually the right results when when people like go offline and things like that and so there'll be a lot of answers on you know i are see an end on the mailing list and things like. that and so you have all these people trying to trying to figure out you know how to do this and getting different different answers and and and from different people because really in innerleithen alexa we have all these different awesome ways to like store state and and you know be notified when process is come and go and things like that so we have like things like agents. send gen servers and wonderful things like that so it makes the problem seemed pretty trivial. so chris enjoys a decided they were seeing all these people having these this this question this problem how to solve it what what are we going to do like we would be nice if we had like one kind of central thing that says hey this is how you do this so i've got an idea let's let's create a blog post. tells everyone how to implement this because this is really trivial so. decide it let's get together on skype and will have kind of figure this out and hash out more real write this blog post so three hours later. they're still trying to work out all of the kind of edge cases that happen when you have this type of problem and again it sounds trivial when you have one machine but what happens when you have a distributed system. and so you know this this is something that people don't don't normally do in their developing on the machine their the and have won that one no running locally and as they go off and they distort all those presents information in an agent and everything's good saw some working but then in production you wanted. the ploy application and now you have two different agents running on two different machines so we have joe robert unlike connect it's an old one and they have their own agent and chris was a in myself we joined no two. but now when you list the presence is on each one of these machines each note only knows what's local right so this is a problem it's obviously not correct so what a lot of different people do is they go out we need a. central source of truth we need a shared state axis so they reach for the database they go i was put in red it's right that this is awesome right quick as fast you know it's great for that but this is a it work. yeah this solution would work for for time being but it still has the problem that when you know it catches on fire and its offline right so now you have a big problem because you have. orphan data. no to crash in didn't have a time a chance to actually clean out all this the state for from its from its node so all these users are now offline but no one does know that so no no one still think everyone's on. so the other problem is that when you have that splits right so no two is now not able to talk to any of the other machines or maybe it just can't talk to read this. but it might still have access from the outside world and so now each of these nodes are getting more connected users but this no to can't tell anyone can tell these other guys hey there's more where users on my machine so.
we have this this problem with this this and it's you know there's this thing called cap theorem and probably will all have probably heard of this since its consistency availability and partition tolerance so you have to choose to though you can't have all three at the same time. so any other problems you you have to choose partition tolerance because you cannot your network is not perfect the network is going to fill you. and so we have to choose either consistency and partition tolerance or availability and partition tolerance so availability says hey you know we might not have we might not have a connection but we still can can operate independently. no problem and still serve customers and what not but when you have this you have data can sit in in consistencies so your data is going to be out of sync. so this is where conflict free replicated data types come into play so or c r d t for short. so scared he tees are used to replicate data across the cluster so and the cool thing is is that it's all about executing updates without the need for remote synchronization on every operation so this gives you what's called strong the. internal consistency so you are not going to be consistent all the time however eventually as as long as all the nodes eventually get all the messages then all the notes in the cluster will be caught up and the be good to go so scared he tees are something that have been kind of in the academic. science and research departments lately the last few years. and the it's all basically proved on with math so you century have these data types that are conflicts are mathematically impossible and that is really cool so and replication without remote synchronisation you know some some kind of systems require that you you. know everything every time there's an update you have to kind of make sure that all everything gets gets the data before you can commit and say ok good we can move on so that's kind of a problem when you have met splits because you now have machines that are off doing their own thing and not not receive me and so this particular pay. paper is the paper that was used to implement this year the tea that's in phoenix and it's called the delta state conflict for the conflict free replicated data type and so the difference is that some here the tees require that all the state is shipped to every note and. that is inefficient and so these guys were able to come up with a solution to do it that doesn't require that and you can actually just do it with delta's so you just ship their deaths and they can work themselves out in the works really nice so this is kind of an illustration of the problem so we have three nodes in a cluster. we have joe that connects the no one. the robber no two in and mike and three in these these names that i'm using our joe armstrong mike williams and and reverting just a just so you know i'm so we have joe just note is going to tell no to hey that shows online however the message hasn't got there yet because. networks are not perfect sometimes messages arrive out of sync or not in time right so but but no two is able to i'm sorry know no one is able to communicate faster no three and no two zabel to send data over the no one so these guys kind of know what's going on so now know. one has john robert no to just still has robert has or hasn't received its up its it from the other notes and then no three knows about my john robert so but what happens when joe goes off line but that message is still in route to know too that joe is online. and then no one actually sense it's difficult for to no three and it tells it updates no three in a no three is actually going to send a message to no two to catch up on on different state events and so the these messages could come and arrive at any time in out of order so. so how should know too. figure out what to do is a discarded joe or does it add joe so. and and what exactly how can we tracked the stuff can you do it with like distracting time stamps right so no to like tags the the operation with a time status as a at this time this is when this operation happen ominous and over here well time doesn't really exist. in a distributed systems you can you don't know whether there's clock drift happening in all its of so you can't you can't worry about you can't the of rely on time because this is a big source of problems so what instead of working on clocks we have.
sector clocks and so vector clocks are essentially a way of a it's basically a counter. and so each know that's in the cluster can have these counters that they keep track of and so every time no one sees an event it starts at zero season event cool and then bounces counter. sees another event sponsors counter so great this is what kind of looks like in a three close three no cluster so we have a three to pull of of each event for for each nodes so no one has an event and so it's easy now it's sense that. it's state over to know to no two season event as no one is actually sending it some states so it seemed that event so it pumps its counter but between the time of no one sending its state to know no to we have no three is his has also seen an event so it's bump. it's counter and then it sees another event so bumps is counted and then it sees another event in bonuses cars are now no three is at three events and it hasn't been synchronised with the cluster so but no two then or i'm sorry no three now tries to synchronize with no to sew it sends its event. and now we have no two has seen two events no one still has only seen one so the snow to send event up to know one another all caught up right so. and so that's just kind of a brief introduction to how c.r.t. teams work there's a lot of research that has to go into it and i'm still reading through that paper to try to understand there's a lot of crazy math that's going on so maybe we can hold up questions for for the and but so let's go on to the actual a.p.i. of everything right so. we have room channel this is kind of stripped out all the other chat you know chat related stuff that this shows you how to use presence so we have the. so let's just say that after you join you want to grab out and and present to the user the current list of users that are online so you can use the the push. a p as soon as they push on the socket this event called present state and it is going to be that the actual data is going to be a list of users online and then we say presence to track. so present start track is going to take a particular topic so in this case if we're on rooms or room lobby topic then it will track or process on that particular topic we can pass an id you could be any value what you where are you use for your system to identify users could be a you idea could be whatever and then we can add. it's a metre data that we want to track for that particular process so we can say yes that is available for online. and then. we have to climb a.p.i. again some more drama script so with here we have also have added a presence model into our of phoenix package and this allows us to do the kind of same stuff we saw before of setting up the socket connecting starting a channel and then we can. take and have a good job script object that will and of the are thing that we track all the state so starts off as an empty object but then when we get a room or a present state event going to take that state and we're going to use the present stott sixty eight. you guys so takes the current state and the new state and then it will actually return to us the the updated state and same thing for this presents diff event and if you if you'll notice i didn't have to add anything into the actual channel to handle this presents distinct. because it's handled by the system it's itself it's handled for you. and then we also have this presents dot list function that you can call to maybe you. the format your present stated differently depending on on your application so. it was time for a demo but unfortunately my laptop doesn't want to connect to the to the of things on borrowing of my friends laptop we didn't have enough time to get all of the stuff on here so it's kind of a shame. but wait there's more. this presents feature of his kind of like an accidental service discovery that. mechanism so in distributed systems we have all these nodes and they have different things that they can do and you know maybe you have an e-mail service or something like that and and this when you have multiple nodes running you can actually have those e-mail services on each.
node and each node locally can take advantage of all those things types of things but on occasion you want to be able to do some distributed computation or you want to find out all will you know the the e-mail service on my local mode is actually really busy so instead i want to see if there's any others available in the cluster. and i can send the work over there so. similar to presence for users you can also have presence for your different services this is one of things that's really wonderful about erlangen elixir is that i mean every process in the system is basically like a little service its isolated its independent from all the others it's running concur. currently you can do work can just all you have to do is just send a message to tell telltale can you do this thing for me or can i have this bit of data you can you do this computation. and so in this case over here instead of pushing around user presence we can just push down the process of presents for for e-mail services so we can see here in this case every single node. it has. is tracking that there is a to e-mails or services so no one in o two has one and of course in this particular example no three just hasn't shared its things so in the end there would be three e-mail services available so how can we use this so this. just like we would with pres tracking the presence of users we could have a topic called services and the services topic we can we can register our service as and as an e-mail service and then we can give it's a meditative in this matter data will be tracked for that particular process so we can we can say that the process. we are pid is self and this is in innerleithen luxor the the self refers to the current process the my own process and then we might have like max jobs so i can only do one hundred things that at one time or take on like a queue of a hundred things and then you know my current workload so this is when you're starting up your. process you can say will my i'm starting off at the zero workload and then we can have a presence list presence that list which is this all the same a.b.i.. we all the services so then it will tell you here's the all the services available within the cluster so we have cost processes that are registering themselves under the e-mails service and and the cool thing about this is that much like you can with user presence right if i open a bunch of different. rather tabs those are all distinct presence in presence information so for every browser to have that you open or maybe i have a mobile clinton i'm connected on my mobile client i have friends information that's in the cluster available for that same thing for here so these e-mail services on this. on that the diagram i was showing there would be an entry in this e-mail list here for every single note on the cluster and and this of course is tracked and prost tracked in and replicated across the cluster which is really cool so. this is some sort of code of what could be done right we can write a wraparound presence to actually exposed a different way of actually dealing with this kind of stuff so maybe we can say service thought track and we can get some process id that we want to say here want to register this process. under the e-mail. service and then we can get the meditative as we did before and this will then basically use the presence a.p.i. underneath the hood and and loss to do that then we can maybe create a function called all we can say service to all and it will go out and give us all the services in the cluster. and then we can say you know service that list e-mail and this would give us a list of all the services across the cluster that are registered as an e-mail service so the and then of course you could potentially get some to build your own kind of reading things. is where you might want to round robin between services or maybe you want to say oh depending on the amount of max jobs versus the current load i want to choose the one with the smallest amount of load. so it's totally all this stuff is totally possible and i think it's i think it's pretty pretty amazing so i'm all right so.
i'm running towards the end the demos actually going to be a little bit longer have a thing and pretty exciting but unfortunately when able to pull that off so i do have now some time to give you some more details on how they are lying v.m. works if you're a port so but before we do that. we also have phoenix framework dot org has all kinds of awesome guides to show you how to use it and get started with it. from building just your standard a.p.i. as using the n.b.c. model or also of course the channel stuff and and. but yeah so this stuff is is really really exciting and the thing again that really sets phoenix apart from most other things that you have available out there is the fact that we run or lying or lang is really really amazing and special thing.
and essentially how it works is your the early b.m.i. is like an operating system for your code and it allows knows exactly how to run your code in the most efficient way possible and it also is a system that will run on all the course in russia. been so essentially this is how how kind of looks you have your operating system your host operating system and then you have the the run time system the ruling on time system. and you have all these different pieces of are lying that kind of put together so you have the colonel system standard libraries you have this thing called o.t. p o t p is a framework that is kind of and an abstraction around processes it allows you to kind of not have to do. think of processes in this low level way but you can actually do more high level and have these callbacks that you can do and so it allows you to write your code in a very synchronous looking fashion but how everything works under the covers it's all happening asynchronously. so these things processes processes is like really the most awesome thing about are linked because they're all independent actors and each process is totally isolated and they're really lightweight so they're very lightweight threads of computation and they maintain their state. by recruiting over themselves and communication is done via message passing so when you want to communicate from one process soon other you just say hey i want to send you a message so you get this process ideally you send a message and this message in this case is going to be this three element to pull of an operation that you want to do like so pay. want to add two numbers together so or a list of numbers together so we could send this over was sent are our own process id so this process knows where to send the thing back. this one gets it receives it into his mailbox and then says ok a cool i know how to add that number together so although i knew that for you and then i will send you back a message say ok in here's a result. so each process in the system has its own memory space it's got its own stack its own heap and its isolated from all other processes and system so you have immutable data and you also have isolation and this is really really the key to. having concurrent systems and you can't have your memory being you know even that from other things other zombies around the system and each process has its own little nell box so this is how it receives messages in the system also has his idea of links and monitor so if i started. process and i want to know if that thing dies but i need in order for me to actually do my job that thing has to be alive so if i need to ensure that that if that thing guys that i also need to dies well i would link to it and then there's the a and a different way which is called monitor. during which is to say hey i want to keep an eye on this this process but i don't want to die just want to be notified when it when it does not receive a message so i can handle that and the governor's question is pretty awesome interlink there's actually probably like seventeen different garbage collectors in the language itself and in the run time but. one of the one of the coolest things about erling is the fact that garbage collection is done on a per process level so in in most other v m.'s garbage question happens at a particular moment in time where it says oh my god we got a freaking car to collect all the garbage so it stops the whole system and it car collects all the garbage. judge and so you end up with these like kind of spikes and and things in your performance so erling kind of gets around that with this mechanism and so the beam itself is lovely every time you create ers start a new node. it will create one operating system process but then for every single three core that you have a new machine it will spawn a schedule or so if you have a corps you get eight schedulers and and each scheduler has its own run q so as you spawn process is those processes will be distributed across all the corps and rush. been so you get actual real parallelism instead of everything just being kind of us scheduled on one single thread. and joe was touching upon that yesterday so in the end you know are lying is absolutely amazing crashes are completely isolated you get data isolation garbage collection is handled on per process and the run time. load balances all of the processes across to all the corps and machine so you get the ability to have peril computation. and with that i think that is all i have so. looks like i have time for some questions. this is that it was that as much as the old you know what like that world vs. as much or of russia's like europe sees way to persist the messages to make sure that there's the air. but when the polls are wrong or role but the message is this going to change i'm so keen to him. replace the question talking about if a process is sending a message in a dies how do you ensure that your messages aren't last right so there's all kinds of different ways that you can do that there's no so we're so remote but the one of the things that's amazing with with earl. during his is these things called supervision trees so you can have process is that act as a supervisor of other processes so and that's where that monitoring comes in so if i start up the supervisor i can say hey all of the processes that you start or get started underneath you you can monitor them and when one dies you can be no. the fight and then you can start that process again with its with its initial state so you can restart the process it's actually going to be creating a new one in its place really but you could you could wire in some mechanism to where it would go and try and fetch some kind of state from from someplace but a lot of times while you. want to crack why processes will crash is because of some data as some bad state and so much like your computer you want to know something you just have to turn it off have you tried turning it off and on again you know it's that kind of thing so. so yeah i mean you can definitely get around some of those things but obviously there's always the chance they have data loss in the case that you haven't designer system in a way to handle that so. so. yet while you would probably you need to do some coordination between processes to ensure that you know you can store some some state in one process and it can spawn off another process that goes and tries to do the work and has a copy of the data and if it dies and doesn't doesn't get back to you then you know that. you need to spawn another one knows the fact that so you. the course it's just that. well as. the losses. as you go. but. thank you. that doesn't watch out. she asked. thousand jobs and was shocked. and i am so so noisy if there's actually if there's data moving across the wire and so the question is is that you know processes are isolated in their all running concurrently end you could potentially have hundreds of thousands of them millions of them and yes that's true or lying systems can. and handle many millions of of can concurrent of process is running in the system as they are very lightweight and and and we actually with with phoenix we've actually done some benchmarking on how this the channels work and the processes work and we were able to get two. million clients connected enjoying a single channel and were able to actually send a message out to all those. channels are all those processes all two million processes send a big giant wikipedia document and that document took only two seconds to reach reach all of the connected clients and so the the problem with that though of course was actually in trying to benchmark that was was dead at was not that the see it this way. also this was a single server so we had a big machine had you know lots of of memory and and lots of course and were able to handle this this traffic but the problem with that actually was was trying to get the clients to connect because generally when you try to have sent out outgoing sockets you kind of our limited like sixty five thousand. connections in so we had to end up like spinning up forty five different servers to try and actually. actually connect all these clients together to the to the server so was a bit of a trickery but we ended up doing it so. up that answers your question so. just yet. it. or are you. i. what are you know that you or i so the question is around distribution and connecting nodes together and essentially the question is is that does phoenix handle that for you. really the answer is phoenix doesn't handle a free but or langham was that for you and so you can actually tell the nodes to to connect and they will connect through or lang's distribution mechanism which is all built into are lying and then now you automatically can schedule send messages across. the cluster and that's one things as amazing about curling is that. message passing his location transparent i can send a message to a process and it doesn't matter which node it's on the wii m.'s in the whole thing will take care of robbing that message to the correct process and so that's really really powerful. anyone else i can see history. on this ok as a thank you. also if you're interested at one and was at one forty one thirty one forty in room ten i think it is and we will do in the functional of programming lab and if you guys want to see that demo that i was going to show on the projector that wouldn't work. and i'll be happy to meet you there and i can show you exactly how it works so thank you very much of it. it is.
Feedback