Bestand wählen
Merken

WebRTC + Ember

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Beta
Erkannte Entitäten
Sprachtranskript
and and the and
I consider demand I work interactive intelligence i in Raleigh North Carolina we have and their application we working on for a few years started pre 1 0 and is now on number 2 . 4 already as were pretty excited about that in the past year I'm working on a building Weber RTC into are replication to provide it with voice and video capabilities and during the last year and learning both member and Weber to that I learned a lot of the different things that if that we need to you have take into account in order to make an application of maintainable by humans but because what I learned from pretty early on is that you can really quickly create a great amounts and when things embers really great at is helping you avoid that mess if you have a pattern in mind and are able to then set out a plan so what is Web RTC whether Odyssey is that of the most basic level as a collection of standardizing the eyes for establishing voice video and data connections between peers this was a really need but it's basically a a set of browser API eyes were native API as uh which every a platform maybe that provide you the ability to build apps from a high level on top of the entire stack of underlying protocols so using uh using ice to collect uh to collect pierced establish connections across networks creating UDP Streams TCP streams that in order to share uh to share binary data for video data media data and and then having and decryption for that is basically the entire breadth that Weber he gives you an is lots of different applications for this and can kind of going to hell a bit more so this this is a an overview of 1 the basic Weber RTC connections looks like this is really really complex uh especially when you take into account the fact that normally your browser API don't really I have a lot of Mr. complexity to so I'd say that by vertices changing the native API paradigm in the browser you get a highly complex discrete and continuous event-based full objects so that really means is you normally use your browser API is to have you know schedule of function around war uh word you call someone to nurture the promise for your rate basically has access something about even with something complex like audio or video you're really just dealing with a single object and might have a stake in some properties and when you have like a nation of 5 video element you can play and pause at element you can change the source you can change volume properties things like that but with appear connection it's it's very stateful and it's highly complex and so not to go over this entire diagram I just wanted to use it to illustrate the complexity of that Stiefel connection so the 3 main API is that we're looking at their body is the media which allows it actually access hardware media from the browser and things like your webcam microphone the Odyssey peer connection which is basically the big Grand box of establishing a connection between peers in RTC data channel which allows you to exchange arbitrary binary or Jason data across the connection between peers n you know it to boil down into the 3 pieces it seems there you know overly overly simplistic and it really is there's a there's a lot complexity to this in order to get into some of the more advanced techniques and topics and or T. C. and and the new were 1 . 0 version where it is you have been released and which which can address a lot a lot of that but not go into that research today because I wanna talk about dealing with over to see with Amber so this this diagram stateful diagrams what you see all just from that 1 RTC peer connection object in the browser of so quirks obstacles for working with whether to see a number and what were the main ones I encounter early on his rear ended down I'll show an example of how action can be a gotcha parent and and rats are lived when you're creating peer connections exchanging media between browsers you really run into this problem of you know your browser that is not necessarily the best a cleaning things up especially highly memory and CPU intensive objects so how do you get around that and how do you deal with that your us dealing with discrete event based CPIs in embers really great at doing bindings to think what what happens we've got lots of events flowing you don't have a single object you can bind the State to you also then have complex state and state transitions it's not just enough to bind to this to be uh to the discrete state of something you also have to know in what is that state transitioning from what the transitioning to yells at build rather permissions dealing with nuclear Apr on each TBS can run on an HDP you know what it's running and the user declines permission to the video camera but grants permission to the microphone as you're dealing with permission snappers noise function how you would like it to and then obviously dealing with humans humans are really great at detecting errors with media but you know any time we've ever watched a youtube video Netflix video and offers even for a split-second you humans are quite at noticing that and that's really terrible for building apps because it makes it hard so dealing with all these things are are the basic challenges that and then you know the other aspect of of the human challenge is your developers if you're not thinking about the developer experience then you're not really thinking about code so you really need to know hello your developers and be able to uh come into a project that's got whether to see amber and really be of function part of that team and be able to understand a complex base so we'll start out with just over this demo here of the issue of 5 video work and so when the show here is there's 2 boxes the blue box red box and a quick start and that get access to my webcam and when I get access to my webcam and keep going to attach that webcam stream to the DOM in both the blue and red boxes so could start and then great so the rear ended component issue is 1 I and to really early on and as a really basic example imagine some pluggable hangouts where you have different video participants displayed on screen and something happens say 1 of them begins speaking you want to take their video and promoted to be the primary well what happens if you do that in number and you re render the component so must simulate an amber render here is that 0 1 of those streams gets frozen so why did that happen this because additional 5 in every browser that supports video that will actually pause the video element if you move it in the down so that's really unfortunate when embers really great every rendering things all the time but so you can see there's little ways to hack around and it really depends on what the humans notice if you the people using application are I really adept at noticing these parsers as mine are then you have come some ultimate solution to prevent every vendor instead of working around it and 1 that's 1 things we tried really early on was
just having at programmatically whenever that whenever the other component would remember we would just programmatically played additional but the long but that delay still was too much on argues this noticed it because the blue in the audio and it use release too many to many problems that so I will kill us and and the and we don't have a great solution to this at this point I II I like it when I go to a talk in some users his problem here's I solved it but illegal but more when I go to doctor to me says here's a problem please help me solve it will this is me asking you for help problem but right now and basically just using degree to ensure that our my my elements get played if I want to allow the rear ended if I don't allow the rear ended then using CSS tricks to move things around as opposed to actually using Amber for all of its uh all of its strengths neither which are optimal solutions so help me solve this problem but so there's a bunch of layers of concern with here and replication but with Weber to see how it can go through my opinionated approach to structuring your application in a way that helps you maintain this over time and maintained in a way that other than other developers can jump in your base understand what's going on in contribute back to it the the 3 main layers of concern I have separated or the session manager the service in your components session manager is not something that is a patent Amber this is something that has kind of emerged or is emerging as a pattern in Weber RTC libraries and this is something that normalizes brother API is in need of events across browsers right however to see is still pre 1 0 that means that multiple browsers support none of them are the same so you don't want to have to in your application code for those differences especially since the 30 been done you also don't have deal with I'm hooking up all of those need of events and adding your about listeners so there's lots of the session manager libraries out there and 1 that I have used and passes simple web RTC on my true to that 1 and get help at will used the dingle JS is a is a signaling layer and can with some of the extra libraries out there can provide you were a provider sessions basically away everyone uses break provided you don't try all code this from hand in your Amber this is not something that I would recommend because it is highly complex and models of the code is it doesn't light separated concerns the next layer is a service the services you're of Manila member service is allows you to create your session manager and management it also allows you to manage signaling channel and and all that about that in a 2nd and then you also want to tie your session state and state transitions interapplication and also a way that I use my service to do that kind of my opinionated approach acting as an event bus and and lastly a components components are important for rendering in interacting with the application and it's also useful for managing you and state and I haven't mentioned point there because of at usually a red flag if for doing you I state but but I'll I'll show how it can relevant here the so the backup 1 so on the so for the service uh that this item managing the signaling channel connections we however TC application you have to exchange information between peers in order to establish the connection and most commonly this is gonna be a WebSocket so if you already have an active would talk in your service here and wrap hopefully it's already managed by service and if you don't have 1 this would be the place that you create that differed only creating a WebSocket connections in your component where I am if you do it in your session manager of which some such mentor libraries out there have the ability to do you can all lose a I control that is beneficial to you so your services this managing signaling there's religious opening a web socket connection of 2 year server in order to change a signaling information to establish a connection the so Amber invented service as an event books the other thing member entered like I don't see it used as often as I would like and it's really powerful and can show how we can use it to act as an event bus so let's say we have a component and we want to handle some event I say this is a message over a WebSocket we've received the message and we want to be able to process that the other component well if our services managing that what we can do is a service can extend ever invented and then when that event happens we can just trigger some event and pass the about it and then in our component where component is initialize it grabs the service which would be injected and then on some of the find a handler to it so then whenever this event happens at the service layer you can whether you can surface that up to your component what that really allows you to do is have components that are short lived Bob reacting to events of the Service letter i in order to to ensure that you're going to get to the right place but art managed at a place that shortly adding a more complex example 2 that's what I have here is showing that was that say there's a few of them on the service where can do in order to create change this event bus for religious will just a bubbling up events from my service to my component is I can the also use as like a chemical gateway so in my service I can look at my stored find a person record associated with that event and when I find that they can trigger the event and pass the person object that person record as a and as an argument to that event handlers then the component does it have to do any lookups to find uh find a person record associate with that event it automatically can handle bad at the same time as a has access to that person and this way your work it really keeps all of that state with you know be this number data whatever at the same layer as your service the right so phenomena such over 2 innings in an example application i put together and kind of show how old uh show how we can build something real with so this is an example of an wrap running understood basic demo can refer loosely based on what our service platform is but what I have here is for example a website where I want but provide the customer the ability to talk to an agent so they land on this website you know this is this is wrapped we've got a couple of different routs you can see as the rout through we can maintain some of the core application under and but this cylinders so able to switch around so looking at the code for this
change
be the the we have a service for video chat that extends ember invented as in my example and in the init method further and the the init function for the service I create my session manager object in a system using simple Weber so that was at that point where so the service man is responsible for maintaining the session manager it wires up a few events for the session manager to some handlers and then it provides some basic functionality of allows us to join a room to leave a room connect to a room and basic uh operations that you would do for a year and a chaperon session and then our component you can see when the component is initialized we wire out a few event handlers for our video service so we actually grab that service that gets injected using and reject and wires up in local media started video added media removed these basic events that are gonna happen really discreetly on your signaling channel with Robert the connection to your components you can actually render them at that level so an inner has couple actions during we video so if we look at In the end we have is the application render this is the rout render here because his which which through the round is rear ended up in the middle and then at the Hi level we have the video chat component rendered at the application level and that's important for a couple reasons the primary 1 being that what you don't want to happen is we near connection is open if you word users can I switch around between happens which 2 different rout you don't wanna destroy that component and it's really important that stays rendered the entire time otherwise you your and your connection is going fail users can it be booted out of the video chat sessions so offended a high level that we know is an enemy destroyed when switch around between rounds so I could about
a couple things happen 1st makes a request to grab user media it then levels that up to the component and the component is able to render that to up so looking at what happens there we got we have a local media started event which get started from ah are service From there we schedule after render we this is this is that that example I've sort of earlier work were doing something Jake it's not ideal we'd love to build a new this number the after render we grab that video element and we attach the string to it in order to render in the dark so why it was that event done what why we have to look at the service to grab that why can we just do that in the component itself Will the reason that's important is that we are allowing our session manager to provide the API is for doing if we're doing that in the component-level we have to manage all of the differences between browsers we have handle permissions all of that complexity that we really wanna keep level that a separate layer concerned so in component when we ask for the but when we ask for media so we say the enjoying video when we join the room we go a service and that provides us the ability to start local media and then what so the next thing that happens if we open this in a separate have it is say another user joints and they want to talk be upon that conversation as well so they did the same thing but right so what happens when we we
want the 2nd person to join the 2nd person comes in
and does basically the same thing in the what I'm now granted in real life they would probably be in a different type of you I if this is a customer to an agent but for the sake of simplicity there can be in the same UI so because the other case were comes in here does the same thing
and what we see are that now the 2nd video stream is established we can actually see the real Weber to see connection is happening here if we opened crowns Weber RTC internals we can look and see the actual real life Weber see connections that are established in the have got to set to tap the open ones connected to the other
so what happens there In our session manager in the service because we have bound to the created pure events the peer stream and and peer from removed what we can see is that in our service we can say look at our video added event that we bound to and assist trigger that and as so to keep a simple here were triggering the appear directly but we can do you can see is you know what like we said in example earlier was look at the store find a person object associated with that here and then amid the event with that person for so that the the component can you know maybe it displays a name associated with the the peer connection maybe it displays a profile picture what whatever it may be and then you I can read that more from the wet stewardess just can emit the peer event so when that pure event is emitted its is triggered on the components the video added element the video added event the is bound to that video added event on the service so the value added event occurs we've got the person record when ad object for that person record and then after render we attach the stream the dumb as well so that's a basic example of using a session manager to and to emit events that the service can act as a gateway and service bus the leverage here to service those events up to the components and From there you can really get much more complex you can start binding to data channel events you can bind to peer she removed so we can see you know this if this tab decides to hang up the other tab can
bind to that and it would be able to remove that that if we bound to the video removed and all kinds of different events occur and as you get deeper into the complexity of Weber to see there's a lot more that you can do and it's important that you have the separate layers of concern because if you don't you're gonna have a massive component that's doing and browser shims you're gonna have In the event handlers you're going have the media you have all these different things in your component and it's a it's just a big problem for code maintenance and and human readability
so that's the basics of it and
well what to will be 1 of the sure is that you know you're acknowledging that there's all these different batches and and there's all these different things that can kind of go wrong and In the past year I've seen all of these things and more go wrong but with the help of Amber being able to and have the same way to seperate layers of concerns using services and components and then having the ability to find 2 to bind my UI state and then having the ability to um hello my permissions in a way that and that I can you know surface the light changes only once the events are already really makes it useful for building levity applications so an so thank you and I appreciate uh part
Demo <Programm>
Bit
Programmierparadigma
Gemeinsamer Speicher
Browser
Versionsverwaltung
Kartesische Koordinaten
Aggregatzustand
Element <Mathematik>
Komplex <Algebra>
Binärcode
Eins
Videokonferenz
Übergang
Streaming <Kommunikationstechnik>
Code
Datenreplikation
Mustersprache
Programmierparadigma
Volumenvisualisierung
Schnelltaste
Lineares Funktional
App <Programm>
Dicke
Webcam
Hardware
Siedepunkt
Datennetz
Kategorie <Mathematik>
Gebäude <Mathematik>
Quellcode
Bitrate
Ereignishorizont
Softwarewartung
Scheduling
Chiffrierung
Menge
Festspeicher
Projektive Ebene
Ordnung <Mathematik>
Statechart
Fehlermeldung
Aggregatzustand
Objekt <Kategorie>
Subtraktion
Quader
Gruppenoperation
Mathematisierung
Automatische Handlungsplanung
Geräusch
Zahlenbereich
Diskrete Gruppe
Zentraleinheit
Systemplattform
Code
Data Mining
Knotenmenge
Benutzerbeteiligung
Vererbungshierarchie
Zusammenhängender Graph
Spezifisches Volumen
Softwareentwickler
Ereignishorizont
Touchscreen
Einfach zusammenhängender Raum
Protokoll <Datenverarbeitungssystem>
Peer-to-Peer-Netz
Parser
Videokonferenz
Objekt <Kategorie>
Diagramm
Komplex <Algebra>
Mereologie
Hypermedia
Wort <Informatik>
YouTube
Benutzerführung
Demo <Programm>
Punkt
Browser
Gateway
Familie <Mathematik>
Kartesische Koordinaten
Element <Mathematik>
Datensicherung
Service provider
Komponente <Software>
Datenmanagement
Datenreplikation
Mustersprache
Kontrollstruktur
Parametersystem
Volumenvisualisierung
Optimierungsproblem
Ereignishorizont
Dienst <Informatik>
Gruppenkeim
Rechter Winkel
Server
Socket
Information
Ordnung <Mathematik>
Message-Passing
Aggregatzustand
Kreiszylinder
Dualitätstheorie
Subtraktion
Web Site
Gruppenoperation
Mathematisierung
Zahlenbereich
Dienst <Informatik>
Systemplattform
Code
Wurm <Informatik>
Informationsmodellierung
Benutzerbeteiligung
Datensatz
Reelle Zahl
Programmbibliothek
Gateway
Zusammenhängender Graph
Softwareentwickler
Ereignishorizont
Hilfesystem
Streuungsdiagramm
Assoziativgesetz
Einfach zusammenhängender Raum
Routing
Peer-to-Peer-Netz
Videokonferenz
Objekt <Kategorie>
Minimalgrad
Bus <Informatik>
Speicherabzug
Benutzerführung
MUD
Umsetzung <Informatik>
Subtraktion
Punkt
Browser
Gruppenoperation
Zahlenbereich
Unrundheit
Kartesische Koordinaten
Element <Mathematik>
Benutzeroberfläche
Extrempunkt
Komplex <Algebra>
Videokonferenz
Übergang
Datenmanagement
Volumenvisualisierung
Zusammenhängender Graph
Metropolitan area network
Streuungsdiagramm
Inklusion <Mathematik>
Einfach zusammenhängender Raum
Lineares Funktional
Nichtlinearer Operator
Routing
Digitalfilter
Physikalisches System
Ereignishorizont
Quick-Sort
Videokonferenz
Objekt <Kategorie>
Dienst <Informatik>
Gruppenkeim
Wurzel <Mathematik>
Rechter Winkel
Chatten <Kommunikation>
Hypermedia
Dynamisches RAM
Hochvakuum
Wort <Informatik>
Ordnung <Mathematik>
Zeichenkette
Videokonferenz
Streuungsdiagramm
Einfach zusammenhängender Raum
Streaming <Kommunikationstechnik>
Videospiel
Gruppenkeim
Reelle Zahl
Datentyp
Mehrrechnersystem
Eins
Subtraktion
Browser
Element <Mathematik>
Komplex <Algebra>
Code
Videokonferenz
Streaming <Kommunikationstechnik>
Datensatz
Datenmanagement
Volumenvisualisierung
Gateway
Zusammenhängender Graph
Speicher <Informatik>
Streuungsdiagramm
Einfach zusammenhängender Raum
Trennungsaxiom
Schnelltaste
Peer-to-Peer-Netz
Ereignishorizont
Wendepunkt
Videokonferenz
Objekt <Kategorie>
Softwarewartung
Dienst <Informatik>
Gruppenkeim
Chatten <Kommunikation>
Hypermedia
Bus <Informatik>
Software Engineering
Streuungsdiagramm
Schnelltaste
Mathematisierung
Kartesische Koordinaten
Aggregatzustand
Ereignishorizont
Dienst <Informatik>
Gruppenkeim
Komplex <Algebra>
Flächentheorie
Mereologie
Ablöseblase
Zusammenhängender Graph
Ereignishorizont
Hilfesystem
Aggregatzustand

Metadaten

Formale Metadaten

Titel WebRTC + Ember
Untertitel Lessons and an Opinionated Approach
Serientitel Ember Conf 2016
Autor Dumaine, Xander
Lizenz CC-Namensnennung - Weitergabe unter gleichen Bedingungen 3.0 Unported:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen und nicht-kommerziellen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen und das Werk bzw. diesen Inhalt auch in veränderter Form nur unter den Bedingungen dieser Lizenz weitergeben.
DOI 10.5446/34721
Herausgeber Confreaks, LLC
Erscheinungsjahr 2016
Sprache Englisch

Inhaltliche Metadaten

Fachgebiet Informatik

Ähnliche Filme

Loading...
Feedback