Upipe video pipelines for multimedia transcoders, streamers and players

Video thumbnail (Frame 0) Video thumbnail (Frame 735) Video thumbnail (Frame 1798) Video thumbnail (Frame 4205) Video thumbnail (Frame 9284) Video thumbnail (Frame 12189) Video thumbnail (Frame 17958) Video thumbnail (Frame 19656) Video thumbnail (Frame 21316) Video thumbnail (Frame 22614) Video thumbnail (Frame 23393) Video thumbnail (Frame 24663)
Video in TIB AV-Portal: Upipe video pipelines for multimedia transcoders, streamers and players

Formal Metadata

Upipe video pipelines for multimedia transcoders, streamers and players
Flexible data flow framework
Title of Series
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
Upipe is a brand new flexible dataflow framework. It is primarily designed to be the core of a multimedia player, transcoder or streamer. The framework organizes the processing of incoming data in buffers inside a pipeline of modules. It exposes core types for buffers and modules (called "pipes"), and the API for communication between pipes and between the application and pipes. This presentation will show how developers can take advantage of Upipe to build complex processing pipelines
Dataflow Building Computer animation Projective plane Source code Software framework
Module (mathematics) Computer animation Computer file Multiplication sign Configuration space Software framework Right angle Mereology
Complex (psychology) Musical ensemble Presentation of a group Multiplication sign Decision theory View (database) Source code Function (mathematics) Mereology Web 2.0 Derivation (linguistics) Electronic visual display Software framework Multimedia Error message Physical system Area Software developer Superscalar Instance (computer science) Process (computing) Internet service provider Module (mathematics) Right angle Resultant Point (geometry) Domain name Server (computing) Civil engineering Streaming media Parallel computing Transcodierung Event horizon Twitter Element (mathematics) Programmschleife Natural number Computer programming Energy level Computing platform Tunis Task (computing) Module (mathematics) Addition Electronic data processing Standard deviation Distribution (mathematics) Model theory Projective plane Tableau TDMA Cartesian coordinate system Limit (category theory) System call Particle system Computer animation Personal digital assistant Formal grammar Family
Thread (computing) Copula (linguistics) Run time (program lifecycle phase) Code Euler angles Genetic programming Multiplication sign Execution unit Mereology Semantics (computer science) Component-based software engineering Different (Kate Ryan album) Hypermedia Core dump Videoconferencing Streamlines, streaklines, and pathlines Software framework Filter <Informatik> Social class Wrapper (data mining) Moment (mathematics) Instance (computer science) Type theory Buffer solution Right angle Implementation Computer file Motion capture Event horizon Depiction Twitter Product (business) Attribute grammar Goodness of fit Causality Term (mathematics) Energy level Software testing Data structure Codierung <Programmierung> Standard deviation Weight Model theory Projective plane Cartesian coordinate system Elementary arithmetic Computer animation Personal digital assistant Spectrum (functional analysis) Library (computing)
Module (mathematics) Point (geometry) Software developer Moment (mathematics) Planning Branch (computer science) Cartesian coordinate system System call Subject indexing Computer animation String (computer science) Network topology Video game Energy level Library (computing)
Thread (computing) Computer animation Computer programming Source code Stress (mechanics) Module (mathematics) Software framework Function (mathematics) Quicksort Instance (computer science)
Computer file Block (periodic table) Multiplication sign Bit Streaming media Cartesian coordinate system Number Product (business) Subject indexing Data exchange Computer animation Atomic number Computer programming String (computer science) Right angle Software framework Thumbnail
Computer animation Information Network topology Multiplication sign Touch typing 1 (number) Website Software framework Multimedia
Point (geometry) Arm Thread (computing) Forcing (mathematics) Source code Function (mathematics) Line (geometry) Instance (computer science) Unit testing Theory Digital rights management Vector space Module (mathematics) Software testing Software framework Error message
you may find that kind of stuff uh I'm not the capability to today but I will be talking about you by which is a brand new projects which we call a flexible data flow framework and so what is it it's it's a
framework that allows you to build pipelines pipelines of data processing uh moon from data sources into data things that all all in 1 of several sources to 1 of several things when I'm speaking the data itself
I mean essentially have pictures
and sounds because we're in the Canadian business so that they can be use a suitable for any kind of data that has a notion of frames packets um so the pipeline is composed of modules that call bytes and you buy a framework that defines an API is a little out 1st to configure the parts so a typical configuration commands would be at the center you would to this all the time was so that is the you might set of commands all open this file for reading or writing and the reason they the data into the right
this is back at process it's uh move you but also defines an API to allow models to so you can see what I call the event is known for instance a world of unified is reached what do we do all that the fatal error uh what do we do this we can application that that kind of the than someone who can store and so when they get to get that um we also have the nature of that defined what the the 1st use of it in an efficient manner I come back to that later and it allows also to yeah the to do that attributes to each the frog attracted to each state before and the finally there's also an API that allows to interact with a new some are combined the fate of you particle so distribution of of standards uh lazy pipes and is the least so in at the end of the presentation yeah so what could be used for her work in the making a business and the company I work for them and uh the results you buy a move is selling you want get equipment so that would be typically transcoders multiplexers out their systems have of music systems where you can have several of the new sources inside 1 of the output of and also we worked on embedded platforms to nature and the need for additional demonstration but formal and did not yet there is no different lightweight so it's quite so that it's not good to 2 moles of that yeah um you
question really asking her right now is what you get another element framework now in 2014 and even though it is something that exists so that what is there is a moodle is we'll see that I'm pretty well higher there is used in all schools so another framework by and probably you all the other there is that of of projects uh however on most of these brothers scientists 15 years years old and the and the new world new trends that emerged in the since the time living designed in particular uh um but there is a derivation of superscalar architectures now policy views of civil calls all of institutions in your cell phone as a tool for calls um so the framework has to provide ways for applications to take advantage of so called city uh and also a new trend in the development industry is in what we call event-driven loops all asynchronous programming that something's been introduced recently eventually the the uh food web servers initially is something that and it's and it's a technology that allows you to register events the like I want to be notified when there's something to read on that so get all I want to be notified when that time expires and we get cold back by the framework and so this is what kind of a new way of programming applications that extra in generalized so that in the past 10 years and limited efforts need to take that into account um also the in existing literature minutes being more difficult because of the lack of majority her mother and the complexity of the models for instance the Transport Stream the marks in most similar to just 1 module was so you have just 1 other entry points that you configure inuktitut if you look at the source code whether 10 modules implement that is the last 1 4 area in the table and so on and so you can render some of in the fine tune you just not some large and also methods and more difficult because of the confusion but in some of those between what we call data processing and decision what local decision that is a deciding which an stream I want to go want to and or do I want the battle is not me and which parts which models running some most of the time this is done either by the models themselves all the framework itself and we think it should be the task of the application domain the so we designed the API to allow the application to get events uh from the models and do the decision but it's a also on the on the professional work as well and the problem we have the most multimedia frameworks is that a designed for 1 kind of education do we see is of please but this and the use and display output GLX whether the army and sometimes we do want to access a lower level API from that transcoding with and unique and do it smaller too hot into this framework the so you but the designed for that kind of use cases and
so we started writing you only 2 years ago with new policies so we've been decided to specify all about comets usually you specify the appeared top that is as that this is to dpi want to exposed to the application and then I'll go down and try to find of the simplest data structures would do contrary we ask ourselves what is the simplest structure that could be present moment you part what is the simplicity of it them to buffer and then we built on top of that early years of API and so different level of Opel symbol with new 5 again talk to the lower level API which the country was to use but you can imagine a higher level of thinking of well we try to good models of symbol and 2 most as possible but unique still so the and and that allows us to do very intensity unit tests including our moon uh to commemorate links with vibrant all code goes without and um we'll start faces some documentation probably also what you can do with tutorials uh and not only Doxygen but also of the conference in the on the same and the 1 last thing that is very appreciated her room by professional that we try to follow standards as much as possible but we do not aim at playing every single the encoded file if a file is not used to of UN doesn't follow the spectrum that we made the plates and we're happy with that and so for the licensing of how uh we tried the very aggressive approach the core is under the MIT license so that's Austria was viewed as can be you can use it even in the in the video applications some use on the GPL all and if you have a license so how do we
compare against existing frameworks older frameworks and add for that we are lower levels of them most of them uh because the education can really decide how to design of the pipeline is depiction decided to render and design to put buffering she was and so on and we have a unified each ifold canvas models not only few articles will see it therefore seems Nicholas filters with the same API for everybody and that's so thanks to a component of code you found a that allows to to in fact that as a wrapper around a library collections like the the easy which is the 1 we use currently which could be adapted to g label or any other kind of events of library as well and what they did me something the 1st to hear that but there's nothing new about to storm threats of to handle multithreading but we we manage the super scholar and the use case by having a look all weights as data structures all structures from the class when we went like this lethal for the of the proposed we have a look at the full was and so the framework is to optimize the so this guy from the start of Prof fully make some of the use of the atom each intrinsics uh on that allows us to do at any coreference counting almost all of the terms of reference counted so you have to mn to care about when to really the structure it's automatically when the rest comes goes down to 0 the strips of media located in the initialized yeah the uh that also allows us to implement copula rights but semantics so we can have the same the that is in different stages of the pipeline different branches for of the pipelines without having to compute the data will only be copied if it must be modified so that is that is something that from the beginning as well well so as you were cooking semantics that is you can do that 1st from a single from small pieces of buffer saying this picture is made of 100 is packets and DTI allows to tune the visible from out of 1 of them is that without cooking anything until the end until you need to add it's considered in the continued continues but the so I talked about the attributes of the political attitudes is that the triplet and nano value so the name can be totally arbitrary we extended accident types like dates times don't presentation by from coding times don't durations that you can define your own attributes and and adapt them to to 2 but for the time and it the propagated through the pipeline so that's an interesting feature causal and last we have a very interesting feature of her is that you can that it could be of the pipeline of when you catch events from from models for instance you can capture and even that says that from now on I mean you elementary streamlined subtitles trends for instance and the vacation can so that ends and allocate the part of the pipeline that we deal with these all even modify the another part of the pipeline to to do something else because there is something that happens as long as you're in the same thread of you can modify the running time warm so it's quite dynamic so that was on the other side of the product of the downside of this and this and that is is the young project so that means that we have fewer models from the existing from frameworks by far so that you will support 5 encoding our encoding and also fewer users so few people to tests uh has to be completely honest that
so that the status of the with through the use of what we call you by reason the 2 army we quite happy with the unless no plans to change the artist the module level and the application level API for the moment we may want to the truth of the API by adding new calls often make the application developers life easier but the it's quite a good starting point if you into other that is a subject looking for it and there are already many modules that are available to move from a from external libraries that support for this index actually being that supports like 2 6 4 and we have a few native modules and article TSG Maxentius makes this is all called business and the everything the the tree the fireworks read strings on the GPU which and so on and we have U 2 the modules just Dutch which allows you to do put branch in your by Patrick there would do during half data and she was of course uh
related to and to talk with so few examples so this 1 is an example
of the well the GG explain uh the example uh it has to 3 threats the
main thread that is running the G of X of the module and it as 1 so strange that reads data from the MIT program from the fire and 1 thread decoding thread army of that takes data from uh the source and the decoded from 12 to what its finally so you can see the use of cues here or 1 she was to the source rather than 1 you after the interconnects thread again learning the way you want it move we have no problems with frameworks to require you to do all you outputs in 1 thread of for instance the PPP of formal requires you to of all your putting 1 thread what sort of public cues that we reach all the first 1 thing and output them from the stress and you can really build about rising won't find this is an
example of which is a bit more complicated than that of the real application that forms not products that we call interview recollapse it's a program that to do so before full things at once right you streams and that's on top of it uh and it's 1st ritual lots of the burka is to just refers to the disk so that the data the number on the left at the same time we try to the research sometimes of it's the 1st 2 the index file that the oxygen on the on the right of these as follows you to play the play this this thing exactly and use it for the shifting proposes all debugging purposes or whatever and the big block in the middle of century to the notes and the decoder and little close to do 2 things 1st we have a data exchange
so we can write thumbnails of the stream on the on the fly and also we have hash things which right prints and that's a hours to do was quickly moving on the string so all of these runs in 3 threads atomic symbolize the in the schematic but so move it's something that could have been that it is good to doing many of our own frameworks
so I'm going to summarize the we do with that you to keep in touch with us you have more information the website we also on 24 7 a minus on 3 no uh when they released and I have built to mobile movable home uh actual Koch into meet so if you're interested you can ask a question there I'm not sure we have many time for question that if you have questions you can go about tomorrow at 2 o'clock and will be around when we will be able to answer your questions to tomorrow use you time we know from 1 to which the of various from 1 person to move that they going from the and so this may be a tree u quite vague question then you might want to go mole specific ones
though you mentioned you need this for while the multimedia framework and just would like to mold was approached you take for
this arm will be significantly way to a new module by would like the idea I thought the value at a private rebellion that's associated with that that so we'll send a couple commensurately effective daytime theories alone pigs they found randomly generated data and the would take the output is 1 of the same as what is supposed to be defined having and if future and is it's considered to be an error point there's never admitted that I is also supposed to be an error so that's all pipes now we also have a lot of source files and that implements well above before the management of income and these for you but for instance tested by having a unit test that the force that will transcend and tries to what found that it's 2 different issues that will be ready by different threads and when we checked that actors all vectors in the correct order and that we do not fully and so on so as the interest the interests of the of the framework that every module is very autonomous so you don't have to import of thousands of lines of support calls to run them and if you had a desire and I needed to do to to be able to test and when he was writing did you believe this work