HTML5 Video Part Deux
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 199 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/32546 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 2014185 / 199
2
3
10
11
12
13
14
17
18
19
23
24
25
27
29
45
46
47
48
49
50
51
52
55
56
57
58
65
67
68
70
72
74
75
76
77
78
79
81
82
84
86
87
89
90
93
94
100
101
102
103
104
106
107
109
110
111
112
113
114
115
119
122
123
124
126
127
128
129
137
139
141
143
145
147
150
154
155
158
161
163
164
165
166
168
171
174
175
176
179
182
183
185
188
190
191
192
194
195
196
00:00
MereologyComputer virusVideoconferencingSelf-organization
00:39
MereologyEnterprise architectureHypermediaSoftware developerOpen sourceWaveProduct (business)Computing platformVideoportalQuicksortInternet service providerService (economics)Standard deviationMultiplication sign2 (number)XMLUMLMeeting/Interview
01:24
Computing platformReal numberDifferent (Kate Ryan album)Plug-in (computing)HypermediaProgrammer (hardware)VideoconferencingMultimediaAdditionGame controllerFlash memoryMassSlide ruleWindowRight angleBitCollaborationismStandard deviationComputer fileWeb 2.0Level (video gaming)MomentumSpeech synthesisWeb browserSoftwareComputer hardwareMechanism designQuicksortRevision controlBuildingOperator (mathematics)1 (number)Physical systemGoodness of fitWritingXML
03:37
SequelCartesian coordinate systemComputing platformWeb 2.0HypermediaReal-time operating systemVideoconferencingStandard deviationMereologyBlack boxStreaming mediaCodePhysical systemTouchscreenOverlay-NetzImplementationRevision controlPlug-in (computing)Peer-to-peerQuicksortLibrary (computing)WordBitSoftwareBinary codeIntegrated development environmentAndroid (robot)Limit (category theory)Graphical user interfaceMultiplication signTorusCompilation albumProjective planeOpen sourceRight angleCommunications protocolNatural numberVaporArithmetic progressionUniverse (mathematics)Software developerOrder (biology)Connected spaceTelecommunicationBenchmarkWrapper (data mining)Web browserComputer hardwareVirtual machineSource codeJava appletNear-ringDifferent (Kate Ryan album)CodeScripting languageMachine codeClient (computing)Scaling (geometry)Event horizonTopological vector spaceRange (statistics)Electronic mailing listAreaPlanningGame theoryMessage passingCASE <Informatik>Casting (performing arts)Decision theoryConsistencyComputer clusterCodecWaveCompilerAbstract state machinesMultimediaDigital rights managementPlastikkarteImage resolutionWhiteboardFreewareXML
12:06
Sign (mathematics)QuicksortTelecommunicationWeb browserContent (media)Physical systemHypermediaGraph (mathematics)Standard deviationDistribution (mathematics)SpacetimePlug-in (computing)MassDigital rights managementBitData storage deviceMobile appWeb 2.0Game controllerTheoryXMLLecture/Conference
13:29
MereologyOpen sourceEncryptionContent (media)FreewareDescriptive statisticsComputing platformCode division multiple accessPhysical systemStandard deviationSoftwareWeb browserProduct (business)VideoconferencingWeb 2.0Formal languageQuicksortShader <Informatik>CodecSpacetimeLecture/Conference
14:26
VideoconferencingWeb 2.0SpacetimeCASE <Informatik>Distribution (mathematics)Multiplication signQuicksortInternet forumGraphical user interfaceExtension (kinesiology)Level (video gaming)Decision theoryFrequencyFlagSlide ruleSystem callPrime idealState of matterLimit (category theory)Topological vector spaceWeb browserSocial classComputing platformSet (mathematics)Computing platformSequelParameter (computer programming)WindowTerm (mathematics)SubsetConsistencyPhysical systemDifferent (Kate Ryan album)Beta functionProcess (computing)Standard deviationFocus (optics)Shift operatorFlash memoryGoodness of fitProduct (business)InternetworkingComputer-assisted translationHTTP cookieWebsiteDerivation (linguistics)Operating systemContext awarenessControl flowOnline helpFraction (mathematics)BitGame controllerPlastikkarteHacker (term)WaveOperator (mathematics)YouTubeAndroid (robot)SoftwareMobile appData storage deviceWikiLecture/ConferenceMeeting/InterviewXML
19:47
Integrated development environmentContent (media)QuicksortVideoconferencingFunctional (mathematics)MereologyAnalytic setAndroid (robot)Computer configurationLogicComplex (psychology)CodeLimit (category theory)Group actionHypermediaTrailComponent-based software engineeringEvent horizonContext awarenessSoftware frameworkMathematical optimizationCombinational logicAbstractionWeb browserComputer architectureOrder (biology)Flash memoryEinbettung <Mathematik>MetadataMessage passingDomain nameScripting languageComputer wormoutputWeb pageLibrary (computing)Associative propertyTerm (mathematics)Dependent and independent variablesProcess (computing)ThumbnailPlug-in (computing)Electronic mailing listFraction (mathematics)BitSampling (statistics)Direction (geometry)SubsetComputer-assisted translation2 (number)Computing platformSystem callXML
24:52
Message passingContext awarenessInheritance (object-oriented programming)Event horizonTelecommunicationCodeBridging (networking)QuicksortWeightArmXML
25:42
Computer architectureRight angleDiagramDifferent (Kate Ryan album)Game controllerNumberPosition operatorBridging (networking)Android (robot)Natural numberIntegrated development environmentAnalytic setLogicSoftware frameworkHypermediaStandard deviationSource codeMereologySummierbarkeitQuicksortContent (media)SoftwareoutputProjective planeComponent-based software engineeringInternet service providerVideoconferencingMachine codeWireless LANScripting languageAbstractionInstance (computer science)Normal (geometry)Computing platformGeometric quantizationWeb 2.0Group actionCASE <Informatik>TouchscreenMeeting/InterviewSource codeXML
27:52
HypermediaCASE <Informatik>Source codeBuffer solutionObject (grammar)VideoconferencingQuicksortOpen setWeb browserCodierung <Programmierung>BitContent (media)Revision controlSampling (statistics)Demo (music)Point (geometry)Web 2.0Event horizonMeeting/InterviewXML
29:26
Bit rateBEEPSoftwareStreaming mediaQuicksortHydraulic jump2 (number)Demo (music)MereologySummierbarkeitOrder (biology)Library (computing)CASE <Informatik>Key (cryptography)Physical systemConnected spaceTerm (mathematics)Message passingHypermediaSet (mathematics)Band matrixSystem callData managementWeb 2.0EncryptionData streamProjective planeVideoconferencingWhiteboardPeer-to-peerJSONXML
31:55
Event horizonBitProjective planePeer-to-peerType theoryQuicksortSource codePoint (geometry)Table (information)Lecture/ConferenceMeeting/InterviewXML
32:32
Combinational logicServer (computing)BitSource codePeer-to-peerContent (media)HypermediaQuicksort
33:11
Projective planeSpacetimeVideo gameQuicksortScripting languageWeb 2.0Java appletLine (geometry)CollaborationismLecture/ConferenceMeeting/Interview
33:59
Key (cryptography)Hash functionComputer filePoint cloudElectronic mailing listTraffic reportingExtension (kinesiology)Web browserEncryptionLibrary (computing)SpacetimeCASE <Informatik>Computer wormQuicksortConnected spaceMechanism designSlide ruleVideoconferencingProjective planeType theoryPerfect groupOpen setSampling (statistics)System callEmailLink (knot theory)Combinational logicSoftware developerDirection (geometry)WebsiteDistribution (mathematics)Computer cluster
36:07
QuicksortEncryptionSubsetRight angleWeb browserType theoryContent (media)Arithmetic meanWeb 2.0Ocean currentInformation managementTelecommunicationComputing platformVideoconferencingSequelMereologyHypermediaGame controllerPoint (geometry)SoftwareFlash memoryInterface (computing)Level (video gaming)Source codeYouTubeData conversionStandard deviationFocus (optics)Content delivery networkInformationWorkstation <Musikinstrument>Water vaporCommunications protocolLogic gateFrictionTerm (mathematics)XMLLecture/Conference
39:31
Interface (computing)Web 2.0Near-ringGateway (telecommunications)QuicksortInstallation artProxy serverAtomic numberXMLLecture/ConferenceMeeting/Interview
40:09
Goodness of fitLocal area networkHypermediaComputer configurationMultiplication signPresentation of a groupPlug-in (computing)Connected spaceSoftwareInformation privacyView (database)WebsiteTerm (mathematics)Self-organizationBit rateWeb 2.0Right angleType theoryQuicksortSource codeNetwork topologyVector potentialAlgorithmSelectivity (electronic)Natural numberGame controllerProduct (business)Web browserFirewall (computing)InternetworkingCommunications protocolAtomic numberServer (computing)CASE <Informatik>GoogolDecision theoryClient (computing)VideoconferencingDifferent (Kate Ryan album)Revision controlWaveYouTubeLink (knot theory)Software developerModul <Software>Point (geometry)Level (video gaming)XMLLecture/Conference
Transcript: English(auto-generated)
00:00
Welcome to Janssen Jonsson for the talk of Michael Dale about HTML5 video part 2. A few hints if you want to leave during the talk or during the question and answer session, please leave quietly so the other audience has a chance to still listen to the talk or to the question and answer.
00:21
And if you leave through the lower back doors, please make sure that the doors are shut because of the bar outside, it's quite loud if the doors are open. So if you leave through the lower doors, please shut the doors behind you. Thank you. Thanks. So, hello. I'm going to give a talk about HTML5 part 2.
00:46
And basically we're looking at HTML5 as sort of the second wave of features that have come about within the standards bodies and kind of the possibilities that they enable. I'm Michael Dale, I'm a director of product at Kaltura, a long time open source media developer.
01:05
A lot of you probably don't know what Kaltura is. It's an open source online video platform. We do a lot of business to business relationships. It's used in education, enterprise, media entertainment, and service providers. We get a lot of first-hand experience with sort of delivering HTML5 experience and therefore also
01:28
get a lot of first-hand experience with the gap between the promise of HTML5 and what's possible. We're going to quickly look at a little bit of history of why HTML5 on the web.
01:42
We had, if people recall, a few years back, there was Windows media plugin, QuickTime media plugin, real media plugin, a lot of different ways to play video, and it was largely dysfunctional, and Flash really came along and provided a unified standard.
02:01
But that obviously wasn't an open standard. It wasn't something that was implementable in free software. It wasn't something that a free browser could implement natively. So out of that, HTML5 video tag was born. We had Opera and Firefox building the early versions of the video tag, and really sort of provided the initial impetus for the HTML5 video tag.
02:26
Meanwhile, Apple was busy working on shiny little new devices, which really set the stage for the main corporate narrative of HTML5 in the mass usage, which was basically the iPhone not shipping Flash.
02:46
Basically, Apple promoted or bootstrapped the messaging around HTML5, which was a fully open, uncontrolled platform, widely respected standards bodies, and it delivered this promise that we would
03:03
have this collaboration among these different vendors, implementing for the shared good of the web. Write once the dream of all programmers, right? To be able to write once and have it work across the different platform targets that you need to support. And performance that was comparable to native, or at least mechanisms to get directly
03:25
at the graphics hardware for resource-intensive tasks, like the CSS hardware accelerated rendering pipelines. And continuously updated, so we were going to have these systems that would evolve, so it was
03:40
going to be an evolving standard, and we would get the new features of the standard widely distributed. And you know, rainbows, kittens, and unicorns. It didn't quite work out that way, because there was initially the codec conflict, which was Apple was not really on board with royalty-free codecs,
04:02
neither was Nokia, as they proclaimed. There was not really a consistent JavaScript API, we had different mobile implementations adapting the HTML5 standard to their own idea of what would be best for the user.
04:21
So that resulted in pretty substantive limitations within the iPhone environment that to this day still exist, which is, for example, no HTML overlays during full screen. So it's a pretty substantive limitation, because you can't build any application that has any supplemental video experience. It's only the video.
04:48
And then adaptive streaming is also still a problem. And then of course there were parties that never showed up, or showed up late to the party, I guess you could say.
05:01
There was a lot of doubt whether Microsoft was going to fully implement the standard, and Adobe obviously resisted it for obvious reasons initially, and then Android as well sort of shipped a broken version of it for a while. But we did get some resolution within the first wave of the HTML5 video standards. We sort of got H.264 for better or worse.
05:32
Cisco has sort of released a H.264 binary, sort of promoting interoperability within free software with the
05:41
best they could do at this juncture, which is just sort of a binary that can be shipped downstream. And it is looking better towards the future. I mean, we have this quote here from Cisco that they're very excited about the future of Dalla. It's going to be, you know, it's an interesting technological development. They're contributing to the project. And this is Cisco, right?
06:03
They're an MPEG LA licenser, you know, so it's not like some small hippie open source company like Google. It's a broader adoption overall towards people pushing forward with the standard of a royalty free codec future.
06:24
And HTML5 has gotten faster, if nothing else. We can see the projects with Mozilla, the asm.js compiler is still making progress. They just released an update late last year where they're getting down to 1.5x native speeds, which is amazing, right?
06:46
Like within a JavaScript shipped code. And they say they're going to even get faster. So it'll become native. I believe that shipping native code will have very little performance difference in the near future.
07:02
Something along the lines of like a virtual machine versus running on hardware. And we'll see that happening over the next few years. And even Apple is making some serious progress on WebView as long as it's in their Safari browser.
07:21
We've seen increases between iOS6 to iOS7, a full 70% boost and some benchmarks up to four times faster. I mean, this is for sort of a gaming benchmark, OctaneJS. They fixed the SVG rendering, so it got a 200x improvement or basically works now.
07:42
They did a lot of improvements there, but it's not always for all use cases. A lot of the improvements Apple is doing don't directly make their way into the WebView or the embedded WebView, which would sort of be necessary for App Store packaging.
08:04
So why do we need a video standard SQL? Well, there's more stuff that needs to be resolved. Adaptive streaming never was really resolved in the first pass. Real-time audio-video communication is an exciting area of the web. It's something that's possible within desktop that isn't possible on the web today, and it's something that we should make happen.
08:25
And DRM. Many think that's not necessarily part of the — should be part of the SQL, but it's also happening. And then HTML5 needs to be more places. We need to be able to run it on a wider range of phones. We need to be able to run it on more screens, like our TVs.
08:42
How are we going to — for HTML5 to really be this universal application delivery platform, it needs to run on more environments. So now we enter into the SQL, and now with even more devices and even more partial vendor buy-in. So let's look a little bit at what I've characterized as some of the major new features within the HTML5 standards body as I see them, I guess.
09:11
First, WebRTC. It's an exciting, you know, real-time video conferencing tool. You can also do screen sharing. This really enables all sorts of applications that previously required sort of, you know, a lot more infrastructure.
09:28
And then we have the peer-to-peer data channel, which opens up a lot of possibilities, which we'll look into a little bit later. Support for WebRTC. We haven't really got a lot of word on that front from Apple.
09:41
Google is a big supporter of it, but it takes time for it to propagate the fragmented Android universe. Microsoft, sort of supporting something of that nature. They have sort of an alternate signaling protocol, which kind of complicates things. It's debatable why or how they're dragging their heels on it, or if it's just — we'll see how it all plays out over time.
10:08
Mozilla, yes, it supports it, and you can actually today do video conferencing between Chrome and Mozilla on shipping versions of the browser. Opera as well, but they're just a wrapper for Chrome now.
10:23
You can learn more at WebRTC.org. Media Source Extension, that's the system for sort of a pending bytes that enables things like MPEG-DASH and live video streaming on a scalable system. So it's different from WebRTC, which opens sort of a data channel for low latency, live
10:45
video communication, versus a pen bytes API, which works really well with CDNs and HTTP-based connections, and sort of just appends bytes to a media stream in order to support live and VOD over CDNs.
11:03
Quick time check. And then, so we're going over the list of items here. We have Apple not really keen on supporting it. They have their own sort of HLS standard that they more or less are committed to.
11:20
Google is working as a first-class supporter of it. Microsoft as well supports it very well, in IE 11 anyway. And Mozilla also plans to support it, but not quite there yet. Smart TVs mostly are committed to HLS. We haven't seen it too much on smart TVs, though the smart TVs do plan to implement MPEG-DASH.
11:45
It's not quite the same thing, because a media source API gives you a lot more flexibility. You can actually take into account sort of network events on a global scale, as opposed to just sort of being a black box that the client makes decisions against,
12:04
or an opaque thing. And then of course Chromecast is sort of the shining white horse coming in, or well, coming in on a shining white horse. It basically supports the full HTML5 API, and you can just plug it into your TV, and then you've got an HTML5 browser on your TV, essentially.
12:22
Well, a highly video-optimized HTML5 browser. And then we have the encrypted media extension. This enables an API to interact with content protection systems. Basically, it's DRM on the web. In theory, it would enable things that, you know, not
12:44
just content controls, but also things like Snapchat, but with more steps to work around the copy. And then also, of course, massive debates on the ethics of standard bodies. Apple, again, no comment. Probably won't, but we'll see what Apple does there in that space.
13:06
They're not a big promoter of protected content distribution outside of the App Store. Google, big supporter. Microsoft as well. Mozilla, not so likely. I mean, it's even more complicated than the H.264 bit.
13:23
As you see, an encrypted media extension. If you just sort of look at the graph here, it should be fairly straightforward. You really can't have a free software system. I mean, basically this part here is CDM, the content decryption module, that inherently has to be closed, and it ties into your platform.
13:48
So it's something that obviously won't fly with the open source browser, but other than that, it's something that's part of the standard. WebGL, I included it here just because it's kind of a tangentially related video
14:06
technology. I mean, there are some interesting possibilities being opened up in that space. There's been some WebGL-based codecs delivery, so basically being able to decode video, sort of optimizing it for the shader language that you have available to you through OpenGL.
14:24
So some folks have written some codecs optimized for that use case, so there's interesting possibilities in the video space for WebGL. The distribution looks brighter than it did a few months ago. Microsoft has committed to shipping it in
14:42
IE 11, so a lot of folks have sort of claimed that it's now ready for prime time. Mozilla, of course, has been a long-time promoter of it. Google as well. Apple only supports it for their iAd standard. Hacks can enable it. So unfortunately we only have things like this, which is almost as annoying as shipping WebGL but
15:03
putting it behind a flag and then outlawing that flag to enter your web store. Almost as annoying. Everyone wants to make their own mobile platform. That's another big challenge of the standards process within the SQL, as we have the Firefox OS.
15:25
At least in some of these cases, obviously, they map over to existing software platforms, so it's not going to be a whole new browser to be supported, but it does just mean a broader set of platforms to be engaged with.
15:42
We have the Windows Phone, which has differences between their RT and their Windows proper operating system. Tizen, Samsung, has hinted at potentially putting effort behind that. It's sort of a WebKit derivative, so it should be not entirely too painful.
16:03
Blackberry is not completely dead in emerging markets. Ubuntu, we would assume, they use some standard web rendering platform. Apple's HTML5 support is sort of frozen in time as they move forward with their native support.
16:23
And again, Android, I can't mention it enough, Android platform is incredibly fragmented. Lots of challenges there. And then not only does everyone want to make their own phone, everybody wants to make their own OTT distribution as well.
16:41
So we have the Sony PS4 WebKit-based, but not necessarily letting you ship your own HTML WebKit, depends on the app store model. Google had Android TV, but has now shifted focus towards Chromecast, which I think has been a really good shift in focus.
17:01
The Chromecast is a really nice platform to target. Windows 8, Xbox One, Apple TV, Roku, Smart TV platforms, lots of stuff. I think we'll see a little bit of, I think having so many different vendors will sort of help the standard process in the long term,
17:20
because there won't be as much central control over what becomes de facto standard. I mean, we've suffered a lot, or there's been sort of breaks put on the process just because Apple has had such a wide percentage of the distribution, and so sort of what they do has become sort of the least common denominator.
17:43
But if enough other platforms are in this space, I think over time it'll promote the standards. But in the meantime, there's lots of challenges. Yeah, so again, the sequel looks a lot like the original. If you recall the slide of some of the challenges we faced within the first wave of HTML5 features,
18:04
we have similar problems, inconsistent buy-in. The unclear roadmap is also a big problem, at least with Mozilla and to a slightly lesser extent Chrome, you have a very clear roadmap with public forums, public wikis that document where the platform is going to be in a certain period of time,
18:26
and then you can sort of see that and make design decisions going forward, or architectural decisions. With some of the other platforms, not so much, you know, it's very up in the air what Apple will do from one day to next, and it literally just happens the day that they ship their beta for the new operating system, that becomes the day that it's defined.
18:48
And then mini-platform hacks as there's still inconsistency. Inconsistent performance, I mean, so we still, HTML5 has gotten a lot faster in terms of a first-class browser experience, but as you try to embed that experience in other contexts, things get complicated.
19:07
Smart TVs also, not so smart sometimes, limited HTML5 implementation, it's not fast enough, a lot of platform fragmentation, Android's still a huge.
19:21
Where does this leave us today? YouTube itself is only able to lead with HTML5 on Chrome, all of the browsers are still using Flash, royalty-free video sites like Wikipedia can only reach about half their visitors with royalty-free codecs, and it's difficult to use new features where support is limited.
19:43
We have sort of a broken, the cats on the internet are sort of only halfway there. How do we sort of transcend these limitations? So, I mean, we'll look a little bit at sort of embedding a player experience, and then we'll look at sort of what the media API and WebRTC offer in terms of new possibilities,
20:03
and we'll try to sort of put a little rocket pack on the cat there. So, player experience, it's worthwhile to talk about this really quickly, just in the sense that a player experience is not just the video playback,
20:25
there's a bunch of other things going on in any sort of modern video playback environment. There's simple features like thumbnail scrubbing or more complicated pieces for sort of responsive layout of text and associated content within the player.
20:44
There's social sharing, related videos, fulfillment events, lots of complexity can be baked into a player in a commercial environment today. All the advertising and monetization configuration, analytics configuration, custom event
21:01
tracking, engagement in video quizzes, chapters, call to action, playlists. There's a huge swath of functionality that sort of gets packaged around video players today, and that's basically part of the challenge. So, when you have this idea of sort of writing it once, you know, IE8, going back to our sort of limitations,
21:28
we're now faced in delivering, even in the browser environment, we have to still sort of play to the least common denominator. IE8 is still a common issue, still out there a lot, and doesn't support HTML5 at all.
21:45
It doesn't really support modern CSS features, so if you build your fancy skin, not going to be possible. It has slow JavaScript engine, and HLS is broken across Android versions, as we've seen before. iOS has many restrictions in WebView, as we also mentioned, and Firefox does not really support the media API yet.
22:08
So, we sort of have to design an architecture that sort of abstracts out the play component, and enables us to sort of swap in either a Flash applet or a native player component,
22:22
where possible in order to sort of deliver this platform as broadly as possible. So, embedding on the web, you want to sort of put it all into an iframe. This sort of helps you sandbox your HTML and CSS, and JavaScript for that matter,
22:43
so that you don't have to deal with the on-page CSS or JavaScript messing up your player. We use something called a friendly iframe, or a same-domain iframe, which basically enables us to inject the content directly. We'll look at how that works in a second.
23:01
And it avoids some of the asynchronous API issues with postMessage. And basically enables you to sort of encapsulate all your sort of workarounds, all your sort of IE fixes, all your iOS fixes, all your Android fixes into a nice little container that has a simple embedding API.
23:21
Here's just a quick sample of how we sort of embed the iframe payload. You can see we basically get the whole iframe, and that can include any per player content. That's the skin, the entry data, any metadata associated with that entry,
23:40
all the list of plugins that are enabled, all that sort of package can be sort of injected into the page, and we make it all part of that initial script include, so we get really nice performance that way as well, because we don't have to do another request for the iframe. That initial request has both the embed logic as well as your API, as well as the iframe payload,
24:02
all with that first JavaScript include. And really sort of gives you a nice combination of optimization and code isolation. And we take that same sort of logic and embed it into a native environment. But in the native environment, instead of the iframe having a video tag in the bottom,
24:26
it's going to have a native video component, so basically the iOS or the Android media framework exposes their video player in that context, and then we're able to play with that
24:41
instead of the HTML5 API, and that lets us get around a lot of the restrictions. When the JavaScript actually goes to communicate with the native component, it's going to call the native bridge as opposed to the HTML5 API. This is just some example code. If you've worked with Cordova or any of the HTML5 packaging tools,
25:07
it'll look somewhat familiar. Basically you just trigger an event on the parent native context by opening up a new iframe, and then that allows you cross-communication.
25:22
Well, that's how you do the sending messages up, and to get messages down, you can just change the hashtag of the... There's a way you can basically change the... Essentially the hashtag was sort of like a post message that gets passed back into the player itself, so you can have two-way communication.
25:43
And basically, going back to that architecture diagram, we're looking at this piece right here, which is the iOS media framework. I don't know if you can see my mouse, but that basically just means that you're able to abstract things. And what does this enable?
26:00
This really lets you enable the full HTML5 player API, all that business logic, all those different screens, all those different interactions, all those skinning, all the logic around analytics and monetization into native environments without the traditional restrictions. So there's also things like synchronized captions or captions that allow different positioning.
26:24
There's just any number of features that are traditionally limited when you have to use the native controls of the device, and doing it this way really opens up a lot of different possibilities. I mean, obviously it has the restriction that you have to package it into a native application,
26:41
but once you're in that environment, it makes it a lot faster to be able to cross-ship your features across Android and iOS. It basically lets you bridge in native features, offline, content controls, normalize, Android playback. It could be used for other things like a software WebM player.
27:01
If, for instance, you wanted to play back WebM content in iOS, you could use this same sort of framework because, again, the JavaScript is communicating directly with the native component, not with the video tag that the WebView provides. You can check out these native player component projects on GitHub.
27:26
So going back to sort of looking at some of the other features here, it's important to keep in mind the whole is more than the sum of the parts quote. I think it really applies to the HTML5 standard as we sort of look at media source API and WebRTC,
27:45
and sort of see how independently they're providing certain use cases, but then they can be combined just by the nature of them all being within the HTML5 platform, and new interesting use cases become possible. First we'll look really quickly at the media source API.
28:02
This is just your basic example. You basically create a new media source, and then you create a new video tag, and then you just set the source to an object URL, which is the media source object that you've created. Very straightforward.
28:22
You add a source open event, and then you're able to append a buffer. And what's magical about it is that, or magical but neat, is that you can append this data from any source that you grab it. So that can be an XHR request, that can be if you have an encoder in your browser that's generating WebM-encoded video content,
28:44
you can plug in the data that way. If you pull the data over a peer channel, you can also inject it as well. So a lot of flexibility in sort of just sequencing in video content. We can see a quick sample demo.
29:04
This is the simplest version of WebM chunk append. Just appends five chunks, plays it back. Very straightforward. The full API is a little bit more robust and enables things like MPEG-DASH.
29:22
If you want to, we'll take a look at this as well. Basically just, you can see here, I don't know if you guys can see it very well from the back there, but it's sort of low quality and then it'll adapt the stream at 10 seconds or so,
29:41
assuming the network is, there you go. So now it jumps to a higher bit rate. Pause it so you don't hear the beeping. And that demo is just using DashJS project, which is a nice library that's been built out by the Dash Consortium.
30:03
It makes it really easy to integrate MPEG-DASH into your projects, and that was sort of a drop-in into the Kaltura player that we were talking about earlier. WebRTC, it's important to keep in mind, it's sort of a set of APIs,
30:20
not necessarily just one. The media stream, or Git user media API, is the one that sort of enables getting at the media stream itself. That's sort of accessing your camera and microphone. The RTC peer connection facilitates the calling and encryption, and bandwidth management that sort of opens up the connection between two peers.
30:46
And then the RTC data channel, which enables sort of message exchange between peers, both in terms of just sending JSON across if you wanted to collaborate on sort of a whiteboard, or do some basic chat system that's supported,
31:04
but it's also a data channel, so you can actually use data blobs being sent between peers as well. And so some of the questions we try to think about is what can it do beyond just a video chat? I mean, obviously, the key use case that it's been designed for
31:23
and supports well is video conferencing, and that's important, and it's fun to be able to do that on the web, but there's also sort of other things that are opened up, as we discussed, in sort of the sum of the parts being able to be combined in new ways. In this case, we'll look at the data channel.
31:42
Here you can see it's sort of equally simple. In order to open these connections, you basically get your data channel established, you create a channel, my data stream, and then you're able to sort of handle the events as the new channel comes in, when the other side creates one.
32:02
I've simplified it a bit. It links back to the full source code if you want to take a look. But basically, the key point that this example is trying to communicate is that you're able to get sort of these binary types across the wire, across peers.
32:22
So that gives you a lot of flexibility. So here you've received a binary blob, and what could you do with that binary blob? A project that we worked on a little bit with a company called Peer5 was basically to support the combination of that media source API with this data channel. So now you're able to take those bits and combine in a signaling server,
32:48
let people know who's missing what, and pass the data over the wire. And then you're ultimately able to deliver your content through a peer-to-peer network as opposed to directly from the HTTP server.
33:06
Yeah, so it's kind of fun. Not necessarily the most useful for VOD content, but perhaps a re-implementation in Live could be pretty interesting.
33:23
There's a lot of other cool projects in this space that I would recommend checking out. The TogetherJS project is really cool. It's basically a tool for creating collaborative experiences very easily, just sort of through embedding a few lines of JavaScript.
33:42
You can now have sort of a shared collaborative editing on whatever you happen to be working on on the web. There's the Sharefest.me, which is a nice little project as well. It basically enables sending files peer-to-peer just by dragging and dropping them onto your browser.
34:05
And it's completely anonymous, well, relatively anonymous and without a cloud, so your files go directly to the person that has the hash key that's generated once you drop the file onto the browser. And it's anonymous to the extent that, or at least encrypted through the same WebRTC data channel encryption.
34:27
Oh, and then P2P XR, also a nice little drop-in library for doing any XHR request as a peer-to-peer request.
34:47
So it's basically a mechanism to fill in your XHR requests that are for large payloads through a peer-to-peer distribution if you have large payloads being sent.
35:00
And then yeah, I don't know, it's kind of an interesting space and there's a lot of possibilities, and this is just a small sample of the type of projects we're seeing evolve here. And so we really look forward to more interesting things happening, and it's really up to the web and see what direction developers are able to take the combination of these technologies.
35:25
And yeah, we're about two minutes early, but basically any other items? Well, there's a link to the slideshow here. I would also give a shout-out to our Kaltura Connect conference in June.
35:41
Sort of deals with the open video and education use cases, which would be, if interesting to you, please see us after or give us a call. Or email. Yeah, that's my Twitter handle and everything. And I guess we'll open it up to questions.
36:02
We're about 11.40 now. Oh, there's one over here. So it's not a question but more a comment,
36:22
but I heard that Vimeo supports HTML5 across all browser platforms. Vimeo? Yes, yes. Yes, Vimeo is leading with HTML5 before YouTube, but I guess they don't have the same, I mean, the conversations we had with YouTube around the topic
36:43
was that it's very important to have that media source API so that they can have lower level control over the network stack when doing adaptive streaming, because they have extra information, because they also run their own CDN, which, I don't know if Vimeo, does Vimeo run their own CDN? I don't think so.
37:04
No comment. No comment, right? But anyway, I think that's a good point that Vimeo is doing that. They've done a really good innovation. You guys just actually did a whole new player, right? Yeah, I saw a nice post.
37:20
Does it use HTML for the controls or does it? I think it's mostly JavaScript-based. HTML, JavaScript. I remember with IE8 at least it was using Flash recently. I did look into it.
37:41
That makes sense. You have to use Flash in IE8. Yeah. Hopefully not soon. I'm not a front-end person. Any other questions? Comments?
38:11
I'm quite interested in the WebRTC part of this sequel of the video features. I was wondering if encryption, end-to-end encryption, is something that will be considered strongly,
38:29
or if this feature is something that we can expect quite soon, because we are living in a world where data gate happened, and all the current means of communication are not encrypted.
38:45
We would be afraid of what we are talking about on the web, so it would be nice if we knew that this feature has encryption end-to-end in mind. Yeah, I think there are some interesting possibilities.
39:01
Well, it was sort of a comment slash question regarding the possibility for WebRTC for end-to-end encryption for web content delivery. Yeah, my question was what's the focus on just make it work and what's the focus on make it safe?
39:21
Right. Well, the WebRTC standard does include encryption as part of the protocol, and I think there would be the possibility for sort of like a Tor on the web type interface would be perfectly possible in the near future, or today really, just as somebody asked to just build it.
39:44
You could have sort of a gateway that would anonymize you a bit, just through the web without even having to install sort of extra add-ons. Thanks. I have a simple question. Is there any hope of all this working well with proxies,
40:08
because at home it's all open, but at work well? Yeah, good question. So WebRTC, essentially you have to write a relay server,
40:22
so if you can't make a direct peer connection through an alternate port, you can write a relay server that works completely over port 80, essentially, but then obviously you go to your Google or some intermediary that runs a signaling server,
40:41
like a bridge, and then that connects out to your peer, so you don't have that direct connection to your peer. Thank you. The protocol does include a lot of tricks for getting through firewalls, and they estimate about 90% of the users on the web today would be able to support a peer connection.
41:08
At least that's what they were saying with the Google Hangout version. Well, that's the developer view. Well, when you are at work, well, you want the video to work,
41:26
but also you don't want to stay late because someone just saturated the work link with a YouTube video, so there are the privacy aspects, but there are also the network user controls that are needed
41:52
when you depend on the connection to work. Will we see a few years where all the tricks you describe successfully help users to saturate their colleagues' bandwidth,
42:13
or is there a way to allow the video but restrict it a little so it cannot be used to saturate links?
42:29
I'm sorry, I didn't quite… Well, you know, you are at work. You have, in any big organization,
42:43
you will have somewhere interns which are bored, which have no work to do, and that will go on any video website to spend time. So how do you allow video without letting those interns kill the connections of other users?
43:10
The question is about internal delivery? Sort of? Network optimization? How do you control any video delivery?
43:22
I think the media source API opens itself up to those types of innovations or options for delivery, because that data can come through any sort of channel, you could potentially have an intranet plug-in that would enable multicast for a live delivery,
43:44
or you could have the local network topology inform the media source selection algorithm in some way, or an extension to dash.js that would work with local network policy. There's potential for somebody to write something like that and create a product that would do that sort of thing.
44:05
At least it's possible now with that more lower level control, it's not possible with the Apple approach of just an HLS and the client makes all the decisions. So I think that's an interesting use case that maybe they didn't think of. And then with a plug-in you're back to the start of your presentation when you had plug-ins.
44:31
So if you said it opens the possibility of plug-ins for this use case, wasn't the whole point to kill plug-ins in browsers?
44:43
Well it wouldn't have to be a plug-in to the browser per se, I just meant an extension or module of the dash, depending on the internal delivery versus network topology adjustments based on a network request. So there's different options, there's different possibilities and it's really up to the product that's created.
45:04
You can talk to us afterwards, we do some work in internal delivery within Culture as well, if you're really interested in it or something. Okay, do you have any questions? Remarks maybe? Nobody anymore? Any questions or remarks? Just wave your hand.
45:24
Okay, well I'll be here. Thank you so much for the opportunity and thanks for listening.