Harnessing FOSS in an End to End Online Video Platform

Video in TIB AV-Portal: Harnessing FOSS in an End to End Online Video Platform

Formal Metadata

Harnessing FOSS in an End to End Online Video Platform
Learn how the Kaltura platform builds and optimizes use of many open source components to build amazing online video experiences
Alternative Title
Open Media - Kaltura
Title of Series
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date
Production Year

Content Metadata

Subject Area
Complex (psychology) Context awareness Machine code INTEGRAL Multiplication sign Demo (music) Real-time operating system Client (computing) Mereology Computer programming Data management Connected space Different (Kate Ryan album) Single-precision floating-point format Core dump Repository (publishing) Videoconferencing Multimedia Webcam Source code Programming language Enterprise architecture Arm Electric generator Distributive property Open source Data storage device Computer Motion capture Representational state transfer User-generated content Front and back ends Type theory Uniformer Raum Order (biology) Website Quicksort Statistics Server (computing) Webcast Content management system Open source Connectivity (graph theory) Mobile Web Disintegration Similarity (geometry) Error correction model Revision control Architecture Term (mathematics) Digital rights management Computing platform Linear map Computer architecture Scale (map) Distribution (mathematics) Focus (optics) Multiplication Scaling (geometry) Demo (music) Mathematical analysis Code Generic programming Database Basis <Mathematik> Cartesian coordinate system Word Embedded system Personal digital assistant Analog-to-digital converter Universe (mathematics) Synchronization Statement (computer science) Videoconferencing Library (computing)
Webcam Presentation of a group Structural load INTEGRAL Decision theory Source code ACID Set (mathematics) Database Real-time operating system Insertion loss E-text Medical imaging Core dump Videoconferencing Cuboid Multimedia Endliche Modelltheorie Library (computing) Thumbnail Physical system Webcam Source code Programming language Service (economics) Email Arm File format Data storage device Metadata Infinity Mereology Entire function Data management Process (computing) Exterior algebra Internet service provider Linearization Lastteilung Right angle Quicksort Row (database) Stapeldatei Slide rule Asynchronous Transfer Mode Server (computing) Module (mathematics) Computer file Link (knot theory) Open source GUI widget Variety (linguistics) Chemical equation Connectivity (graph theory) Computer-generated imagery Virtual machine Data storage device Transcodierung Scalability Architecture Broadcasting (networking) Internet forum Subject indexing Video game console Codierung <Programmierung> Lie group Traffic reporting Computing platform Computer architecture Stapeldatei Information Server (computing) Projective plane Debugger Analytic set Database Cartesian coordinate system System call Frame problem Transcodierung Subject indexing Word Broadcasting (networking) Integrated development environment Personal digital assistant Internet forum Multimedia Video game FAQ Videoconferencing Holographic data storage Library (computing)
Laptop User interface Connectivity (graph theory) View (database) Multiplication sign Source code 1 (number) Parameter (computer programming) Mereology Event horizon Product (business) Term (mathematics) Forest Core dump Videoconferencing Computing platform Installation art Source code Collaborationism Touchscreen File format Optimization problem Projective plane Menu (computing) Bit Software maintenance Hand fan Tablet computer Process (computing) Order (biology) Whiteboard Quicksort Videoconferencing Library (computing)
Source code Server (computing) Touchscreen Sample (statistics) Planning Right angle Videoconferencing Open set
Source code Freeware Sample (statistics) Dataflow INTEGRAL File format Natural language Videoconferencing Communications protocol Product (business) Thomas Kuhn
Point (geometry) Laptop Source code Server (computing) Group action Service (economics) Weight Multiplication sign Open source ACID Expert system Water vapor Parameter (computer programming) Instance (computer science) System call Transcodierung Source code Self-organization Right angle Multimedia Videoconferencing Computing platform
Source code Server (computing) Profil (magazine) Mass Peg solitaire Videoconferencing Open set
Source code Open source Profil (magazine) Planning Database Natural language Line (geometry) Endliche Modelltheorie Videoconferencing Metropolitan area network Form (programming)
Stapeldatei Asynchronous Transfer Mode Presentation of a group Server (computing) Structural load Multiplication sign Computer-generated imagery Combinational logic Virtual machine Bit rate Data storage device Rule of inference E-text Video game console Library (computing) Social class Computer architecture Webcam Source code Scale (map) Service (economics) Server (computing) Software developer Open source Analytic set Mereology User-generated content Transcodierung Synchronization Multimedia Videoconferencing
Source code Presentation of a group Module (mathematics) Computer-generated imagery Source code Keyboard shortcut Client (computing) Menu (computing) Client (computing) Mereology Formal language Revision control Process (computing) Sample (statistics) Subject indexing Revision control Software testing Software testing Holographic data storage Videoconferencing Hydraulic jump Library (computing) Library (computing) Physical system
Source code Meta element Connectivity (graph theory) Planning Instance (computer science) Cartesian coordinate system Entire function Repository (publishing) Network topology Core dump Software testing Videoconferencing Computing platform
Web page Source code Software developer Keyboard shortcut Planning Client (computing) Computer programming Medical imaging Word Process (computing) Term (mathematics) Forest Source code Software testing Right angle Videoconferencing Sinc function Social class Library (computing)
Inclusion map Source code Installation art Uniformer Raum Server (computing) Googol Electronic mailing list Ripping Videoconferencing Open set
I menus or alone with delta and in this talk is about how culture i is basically a platform that packages out a lot of different open source component of different practices but in in some new components that we introduced ourselves in order to uh simplifies in uniform in order to simplify and unify but a and provide a conducive single-platform to answer video but the requirements for any type of video applications I'm so I will insert will probably will basically do a quick what is culture and an older the architecture the and hopefully the do some demos was in discussions of this so the promise of why we why we read started culture at all is because all in Video and video in general is actually very complex but and there's a lot of different requirements those a lot of different solutions and it's very very hard to scale and so it understood the the very simple use case of just taking enough of an beggin running it through a hundred thousand videos becomes a problem and so how do you do that scale how do it reliably i'm and and how do you handle them all the other stuff that that is required in our distribution and so on so so culture was basically built to provide a platform to answer all those needs and basically uh our approach was there needs to be a single server solution and so of 1 back that is basically the are exposed through the eyes and then based on those API is applications for a different industries in different practices will builds a customized solutions so everything is always exposed to the same REST API and we'll see more about this in a 2nd but I don't know why I chose I had the mailboxes you've had the they're supposed to be there's I think if it that the so I as streaming video workflows right so so again you're going back to the mission statement right but it's all about solving the the scale problem with video if you just have to videos of videos if you wanna do home library there's there of solutions of the and what culture was said to solve is how do I do it at scale in a situation where for example a from a large broadcaster and have a lot of you the assets in a lot of light channels and I to coordinate and manage all of them and and create end-user applications and collect the end user statistics and distribute video to all sorts of websites and so on and do it all and manage and also in the same way but then you basically end up doing a lot of different workflows so how do you manage those workflows and and that's basically what we set out to do of being in terms of uh what the design guidelines of the platform of open source it's all a GPL version 3 i so as as a design guideline the focus was around had we solve problems that are a scale problems so once again not like that 2 3 are whole library but but you know 10 thousand videos or distribution for a university or a distribution for an enterprise with In of thousands of of employees generic versatile or in other words bit basically become the Swiss Knife for video applications so 1 back in with a unified API idolize integrate applications that are cross-industry crossed I use cases without having to worry about the underlying How do I manage of offend they how do I work with the storage of their work with that the distribution at the the once again in the eyes 1st and then industry so ever ever since the beginning we actually focused on trying to solve multiple solutions of the sentence so reactions started off with education and media and we expanded into enterprise in doing a similar things around others and in real time today but the idea was basically but at the core problem premise of of culture was basically the idea that yes videos complex but even though they use cases and practices of how do you know take a video and and experience within the end user context would be different in terms of the experience of the back-end of handling the video is actually the same and so on how we work with enough industries enough uh in user types and converge those practices and use cases into 1 unified platform but in a last name was actually released frequently so I we actually that culture is being released under a bi weekly basis so basically every other Sunday there is a new release the up but in and this is part of a broader new but that as we go more and more into alive applications had we become more real-time and in support death problems like real-time analysis of video and being able to collaborate between multiple cameras for example so use cases like that this OK arms in terms of the platform architecture of the core which are basically the servers so think about things that are for example running offend or the database and things like that those are basically in the course everything is then wrapped 3 in a unified rest based API and we have an automatic client libraries generated it basically runs automatically every time there's a new feature added or something like that and and those generate client libraries in many different programming languages so and if you're writing an application you can use a native application to your programming
environment but but it's always kept up to date than a double that we have all sorts of widgets uh you see in the earlier today that the video player there's uploaders and recorders the but who and then we build applications on top of them and there are many different applications both from the community as well as a what's interesting is that the idea approach is actually the same in the back and as it is in the front end so just like applications are using the API to make calls to the server for example to access database records and things like that in the back applications are doing the same so but back in applications like badges of which are as synchronous of systems are running things like ffmpeg or e-mail are using the same API to actually access the database there's no coupling between any component within the system which basically allows for infinite but still in and very agile and scalability so this the what a platform notes so basically if we go back to the slides of 40 architecture right so what are the other servers that are running in the cluster are we have a load balancer obviously at the front machines are the machines are basically handling the DAPI Laird the and serving the applications so players in things like that of batch is our acid for systems that are basically running things like transcoding or modifications or e-mail sending US banks which is the full text search a those indexing server but it's an open source components DWH which is the analytics environment it's basically use for low processing in database for both the you know the alive when we're using multiple components here but my sick or withstood the new that was replaced and on the right to be armed master and slave or is the Indian affairs to handle the storage and then we also of solutions for as supporting other use cases uh there's a an interesting and Jenks model model that we recently released and in the media server which does more real time in life and so on earth open source projects were using the you've heard a lot about million for today a half and half of them Begum encoder and 3 before box socks and that he variety of also commercial encoders and so we have integrations with all of them and culture is basically has a set of linear top of go insert inside of the of the batch manager is basically the culture decision later and basically managing those transcoders in a way that based on the information retrieved from India info we know which encoder is better to use for what format in X forward but it how do I improve the the output of of each other and was also can of a waterfall for that between them image wanted to manipulate the images that you know nice anecdote about these thumbnails encounter are done in real time so for example and if I need to get done after a specific frame from a video file instead of you know having to preprocess the entire video file and an axis only frames that I have index cultural actually on Veldime will go into your source file and generate a thumbnail for you and you can also enrolled and do all sorts of manipulation like you know training the the the the the the thunder were making resize it and things like that I mean obviously it's all cast on both under CDM as well as the server engines and the engines run model so you've heard a lot about different problems I like a how do I take a video and truncated and serve it is until last or how do I do it here's a hydrogen gas and didn't real-time how do I work with those formats and serve it correctly with every other but for every other system answers so this can become a big headache so we recently the no endings we found is that was very efficient in that but it lacked the solution that actually knows how to do those things have to be packaged and in how to create those things so we're basically created then recently released the jinxed by model but there there's a link at the end of the presentation which you can find a red 5 or any alternatives like was and is therefore lie broadcasts in webcam recording of the culture palabra which we talked about then also to libraries that use culture is mostly PHQ based on the server side and there's a bunch of other programming languages but the because the core and my sequel or MRI Debian Sphinx and wanted to know then for it bud DWH like I mentioned before we use a couple of the solutions up and down Hogan on spots but in some notable references before jumping a demo and if you want so essentially I if you wanna get started and try culture I to install there's a link over here we would provide rpm and that packages that balls and the stable because I using shaft but will serve as recipes for sheriff renders a no alive community and you can get support in the forums on you have under knowledge center tons of documentation ifyou wanna see the uh nightly builds on bi weekly builds a status reports from the CIA there a knife you wanna play with their find library is over here I need to do here and another interesting design island while you set up it is 1 of the constitutive but that another interesting design
guideline is whenever we work under another open-source project like the engines model for example
we actually tried to build it in a way that is usable outside of delta 2 so it will be but not as valuable for the community the other community like with engines to the engines community but also in it because it makes maintainability much easier with a platform that is so big in terms of the different components that it leverages under different solutions that involves so you need to find a way to I was segment things and make them easier to maintain by people who are outside of the platform community so for example the player library I was actually built in collaboration over the years with Mozilla and Google and i it's it's actually the planet runs inside wikipedia and the then so a bunch of all a lot of our different our components if you going to do have such culture I you'll see all sorts of projects there and they all basically formed the platform but it's project now actually views on its own as well right so this is just higher 1 some just think you join us on the so lovely sunny afternoon home and I'll be showing as much as the product as possible while we have time so as as are mentioned before transcoding video is complex and I think this screen shows part of that complexity because if you look at it then this is the source entry that we got and we need to France Cody to various flavors and each of those are different parameters to ffmpeg or whatever transcoder we're using room so it's a pretty complex process you can see here the warnings there were getting pair flavor all and the reason why we need flavors of course is that we have various devices so we optimize per device the playback on iPad won't be the same as in my laptop or tablet or 9 board and so forth so we utilize a fan in the other on free party so for that we use in in order to optimize the source video as best as possible per source to G and device so what I'm showing here is basically what needs to get done for that to happen that's our the portion of our core product room was forest we have 10 minutes at home as as far as them only in so let's talk a little bit about the engine makes model that we've just introduced on what it does is it's able to play to create on the fly manifests for dash you and HTS and various other formats so that they can be played it's not directly tied to work platform so it can be used independently and there is an RPM package it you can install outside outsider platform and of course you can integrate with kulturisk platform which is of course the optimize solution on you know where events and stuff come so let's take a little entry here on this one's become quite famous everyone uses
it to demo still tongue body
movie and it's scared it myself on manage on so this is currently plane
HTS come on you know let's play on full screen of the it's my server yeah right
and OK so now not this what you so
just told to explain the integration with and genetics home basically what we
do is we have our sorority
entries saved or else it saved on CDN probably CDM and where a player does is ask for a manifest manifest manifest is just basically usually in XML but it can be of our formats depending on the protocol you're using come anatomy show it to you on the
backhand where things are simpler at least for me only in groups that you can't sell currently among just making a call using 0 which against the ones from other went on to specific acid that we have on coal Toros API is entirely restful so you can see that the service name were asking for years play manifest which is exactly what the player would do or player would say OK let's go to the server was my plea manifest and then you'll know where to place from oscillate statistical look at this mom right so we can see here that it's an XML is the center and the style of made up on the net and it's got the euro for the media and what you can see here as I said there's 1 entry below flavor ideas right per-device optimized for device so you can see here that this is my entry the like so I should have brought in most and all these are some flavors of it so they can be optimized per device so potentially from watching it on my laptop all get this you might get this on your iPad and someone else will get some water flavor and of course it's very flexible so you can adjust the flavors created as pair your needs so for instance an organization might say well you know I'll use I sell them on my employs a room within my land in neural watching it on their desktops no point for me to waste time producing these other flavors so from this just the what and someone else may say you know I have very very special needs for a very specific device and I have my own transcoding expert I just need you to give me the platform all be the parameters myself find you can create using array the IRA using play you can create your own customized flavor if you know on so that's basically the idea of it we can see that the same entry if I take at this new world it didn't that year the we just put it on the stash player here and I happen to have who here the
not just to think and so it plays from external among players right from a JavaScript dashed so considered it's just it's the same exact world what happens is that this will go to Kultura server and
for that particular profile you say OK just a 2nd what profile reasking are asking for M PEG dashed
that's let's see how we handle about specific
on serving profile international go on long database will say OK for that we need to go to engine next engine X is the 1 that knows how to serve this particular form and then the we really called line genetics would actually be the 1 plane so this is the model that we have this is what we based on engine acts on this of course open source it's made by a very bright man who calls on condyle 1 1 and you were most welcome to look at it I have 5 minutes just so you know I don't know why
everyone to know how much time we have a 1
wounded I also wanted to mention that
the right thing to do that that's quite correct move in that our note that I wanted to mention while I was looking at this my presentation here you stopped the you fact stanchion OK who as far as these nodes I I just want to clarify that our architecture allows you to segment dies your class any way you want it so this is just a suggestion of on how you should segmented but you can have them all on 1 server of course it won't be redundant but for development purposes that's fine all you could decide that your Sphinx runs on the same machine as your analytics or whatever other combination you're comfortable with so this is a mere suggestion it makes sense to do it that way but it's not a mosque and so I wanted to touch on that briefly rule and the last thing I wanna show you mine minute and a half that's left
against he stepped from source mentioned we have client libraries
bindings for a lot of languages only features move toward CI system for a 2nd so or system actually has 2 parts it dies food some local tests on the cluster much like the 1 the presentation the same speckle as the 1 on the presentation body also integrates we Travis mn anyone familiar with very CI hope so it's a little she if you're not check it out and whenever we push and new version from each other builds come to assure whenever there's a pull request he do libraries for all these bindings and we have a few more actually work In the process of integrating the rest of them on we have a 10 or 11 client libraries so when something fails you can standing you'll also be betrayed we not who will see eye
and a repository for thank so for instance if I go here some logging have repository the now that's OK
so you can always see the current build styles from here to from as far as our tests we utilize the API uses or mentioned or entire platform planning utilizes itself when doing stuff because there is the API and the various components making your requests to them to the core so essentially anything that we can do internally as far as our applications can be done by an external persons using died API so tree much older wrong so that's important to note and whenever we create something we run it for all the tests from just to show how lot looks like an from ask if anyone has any questions so think about it
now that anyways it releases a lot of testing processes and we have a lot of these forest binding on images of the page class so you can get home the plan
so all the items I guess and also that it goes back to this and that and this is the only meant to be in the development of the of the the use the up-to-date of this guy then only the 1 that we like them all of 1 of the reasons which is the of the course of a year old jams in terms of helping out you can always say the words of the just what was going on in the in that's the in 1 of the the we have a lot of this in the world you to rapidly all with the right right along so any questions and I was stacked clear was i nothing at all what do you want to do so and this or the studio and cultural from a work program in which program his although not in a way that will yeah not so just 1 node since nobody has questions on we're always looking for Roland years so we're very happy to get anyone on client libraries we actually need someone to correctly we tests so go volunteer come from here 1st of all yes we love for requests will happen so much that I made this ugly but very functional dashboard so that
are guys can see what's going on in his Boolean who's asking you the easier it is came real-time sold via we allow you to question you're issues when you need another 1 of the few