What's cooking in GStreamer

Video thumbnail (Frame 0) Video thumbnail (Frame 8836) Video thumbnail (Frame 9756) Video thumbnail (Frame 16312) Video thumbnail (Frame 27381) Video thumbnail (Frame 38450) Video thumbnail (Frame 41350)
Video in TIB AV-Portal: What's cooking in GStreamer

Formal Metadata

What's cooking in GStreamer
Title of Series
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
This talk will take a look at what's been happening in the GStreamer multimedia framework as of late and what shiny new features you can expect to land in the near future. It is targeted at both application developers and anyone interested in multimedia on the Linux desktop and elsewhere. GStreamer is a highly versatile plugin-based multimedia framework that caters to a whole range of multimedia needs, whether desktop applications, streaming servers or multimedia middleware; embedded systems, desktops, or server farms. It is also cross-platform and works on Linux, *BSD, Solaris, OS/X, Windows, iOS and Android. In late 2012 GStreamer 1.0 was released, the next generation of the rather successful GStreamer 0.10.x API series. 1.2 was released just about year later towards the end of 2013 sporting quite a few new features. With 1.4 already in the making and 1.6 on the horizon, we'll take a bird's eye view at what's new and improved and what upcoming new features you can look forward to. Does hardware-accelerated video playback finally work out of the box on the Linux desktop? Can you get a list of available devices now? Does Blu-ray playback work? How about OpenGL? Will it work on Wayland? Can you use it on the Raspberry Pi? Join us to find out!
Point (geometry) Ocean current Functional (mathematics) Context awareness State of matter Code View (database) Multiplication sign Connectivity (graph theory) Numbering scheme Control flow Dimensional analysis Element (mathematics) Revision control Goodness of fit Mathematics Meeting/Interview Different (Kate Ryan album) Hypermedia Term (mathematics) Energy level Damping Multimedia Software framework Plug-in (computing) Computing platform Physical system Stability theory Graphics tablet User interface Dependent and independent variables Software developer Basis <Mathematik> Cartesian coordinate system Evolute Surface of revolution Type theory Photographic mosaic Digital photography Voting Personal digital assistant Right angle Quicksort Table (information) Window Library (computing)
Hypermedia Semiconductor memory Multiplication sign Keyboard shortcut Interactive television Software framework Object (grammar) Perspective (visual) Software bug
Android (robot) Context awareness Code INTEGRAL Ferry Corsten Multiplication sign Combinational logic Open set Mereology Proper map Computer programming Neuroinformatik Medical imaging Estimator Mathematics Semiconductor memory Different (Kate Ryan album) Synchronization Videoconferencing Encryption Cuboid Electronic visual display Software framework Office suite Physical system Touchscreen Electric generator Software developer Bit Connected space Arithmetic mean Process (computing) Befehlsprozessor Internet service provider output Right angle Whiteboard Quicksort Reading (process) Resultant Geometry Laptop Server (computing) Divisor Online help Streaming media Element (mathematics) 2 (number) Goodness of fit Computer hardware Representation (politics) Codierung <Programmierung> Plug-in (computing) Computing platform Form (programming) User interface Authentication Weight Content (media) Cartesian coordinate system Software Personal digital assistant Physikalisch-Technische Bundesanstalt Video game Codec Object (grammar) Communications protocol
Suite (music) Game controller Computer file State of matter Code INTEGRAL Multiplication sign Real number Source code Heat transfer Function (mathematics) Coma Berenices Software bug Product (business) Usability Number Mathematics Goodness of fit Crash (computing) Spherical cap Lattice (group) Meeting/Interview Computer configuration Hypermedia Different (Kate Ryan album) Computer hardware Videoconferencing Energy level Cuboid Software testing Endliche Modelltheorie Mapping Information Projective plane Electronic mailing list Bit Maxima and minima Line (geometry) Unit testing Cartesian coordinate system Type theory Website Summierbarkeit Figurate number Quicksort Library (computing)
Suite (music) Server (computing) Parsing Multiplication sign 1 (number) Streaming media Software bug Number Product (business) Goodness of fit Crash (computing) Word Meeting/Interview Network topology Summierbarkeit Software testing Quicksort Error message Physical system Condition number
do talk just to give you a rough idea within talking Wilson working on for the last 1 year maybe 18 months of the year if the if following G Sumer a development of a daily basis and then this much news on for you and him but of most people aren't so you know which is lecturer people with working and this great introduction while way of time to move on I'm 1 of 2 just tumor co-developers ardent of reason and the Boston it was basically the same idea as you summaries this and will probably also touched every piece of code until now right and we said we were 1st utricular I'm going to you consulting we and right so just a quick introduction in case of you know was to summarize and is basically a bunch of libraries and to do multimedia processing it's pipeline based on which the like lego you can just take in different types of love elements and you can put them together and then you know you media flows through it and got most of components pads elements so we call them and we provide plug-ins of which you can wrap other libraries surrounded you can provide all kinds of different functionalities and this plug-ins are basically you can Lego pieces that can combine them together when we provide a unified API around all that I they used to to do that and what don't we do that everything we invent everything from scratch but those of you know lots of good development stuff of work has been done on you know Codex being written in the base year of empathetic etc cetera all these libraries we don't read and then we just use and make them available by our abstract API and we've got a very high level if your edits on the table or camera photoshop assumed for a photo booths of application we have a high level API for that and if you have a low level stuff you want to do very dynamic part by and you can do that 1 this some high level API missing value there 1 day but know and we integrate with other frameworks so both in terms of being used by the frameworks and using other frameworks so we know we can we can be used in work in firefox cluster you'll that we integrate wasn't so you can use the windows but you know so our goal is basically to to be adaptable to work everywhere all right what are you going to talk about and just what happens in the last years so the extrema 0 and because you haven't heard before release to stroll 1 0 while back home and just a 1 . 2 is the next stable release version and that's come out last October sort of new features i the binary my platform support for Windows cetera and mosaics and I was and Android and after that in the future we have just a 1 . 4 just 6 on you stable future releases so we kind of adopted AA versioning scheme the financial ex-mod the right so i'm just in case of notes and as mentioned because people are confused by the friendly and so as you tend to see all dimension version not develop in 1 of my intent and don't use it for the stuff that we realize something we have to continue with it but it's dead forget what 1 . 0 is the new stable API and if we have a break a guy again hopefully not for a long time it will be 2 . 0 so everything that starts at 1 point something is backwards compatible it's very much like know what to look at this and we have taken what we've started doing regular we started separating future releases from bugfix releases and so we have you know the G. voting scheme now you 1 . 0 and 1 . 0 1 100 provable fixed releases and then use the big future versions are Cobol put tool for 6 the but so we've got the courage to about 6 releases we've done current state respond to won't went for a hopefully is gonna come out From March April but it took us a very very long time to get from 1 from 0 to 1 . 2 about a year and I think it's much longer and we want to take were ritually we have planned to do in new future of these 3 months so that didn't work out so and hopefully we can they are goal currently used to stabilize on a six-monthly we recycle so we can roughly 2 releases a March April and September October by generalizing all the major applications aware of the importance to 1 . 0 I there is a was border Firefox now which is being worked on and enlightenment has reported 1 . 0 as well the the so just a quick review of what does want 1 0 actually mean what have we changed and you in general system API cleanup of stuff you want to get rid of stuff on a fixed but in general from application point of view the changes are more than evolution than a revolution in a given application most stuff is you will have some minor changes in the but most of the changes affect plug-ins and low-level stuff conceptually it's pretty much the same and so from a
risk perspective of Yemen bugs always there but from risk perspective is well understood how it works and how you have a framework of interaction works and we haven't changed that this 1 no but we did fix some of the big problems and that we had to attend the memory handling etc. and hopefully will be 1 . 0 API will be stable for quite time some bindings of some unemployed and wide you inspection we could do that in the past because of our custom media objects stuff
which were what special so do you are I couldn't handle that but now we've changed that and command of the but binary so we now have 1 releases for we so
X Android and IOS we can have all these platforms will give you money and white stuff using open-source much different frameworks and includes or plug-ins will almost all programs if any part in the missing on you shout and maybe we can just add them it's usually oversight and detection understand port than this but this is of right and we started development for 1 conform hopefully comes out in the job market and we work with and since september the discoveries by some of the features and just going to tell you what all of us have so 1 of the very exchanges so we did for 1 . 0 and to the following reasons as some form of integration and you want to use all kinds of special have especially on embedded devices stuff like other codecs and it wasn't what it was possible to do that and so 10 but it was quite often in technicians you really didn't want to do that so what we now have some interfaces to share all kinds of some what harder context between devices so for example you can share on a display in the between all the elements of automatic he when I've stuff to and all kinds of different memory representations so for example in 0 10 was only possible to some have something like system memory something that you can look at was met something else was just act and what we now can do is so we can wrap stuff like the enable Foygel images of stuff like he's purpose Office of the API then what was added quite recently is somewhere with proper negotiation between elements of to and make it possible that some from not and futures on what the best possible solution of the best possible combination of elements is selected but I have a there on your computer so for example what on most of the latest intellect yeah laptops with an interchip such you will automatically use the DAPI elements from decoding if it's possible even without to the application having to know about that and then was over the last months we've also made lots of further improvements there there were lots of boxes and I think nowadays it can just work out of the box so what does this mean basically here using stuff like what where the 2 are just know that and did you play on your desktop it would just use that accordance that are available without the applications knowing about that if there are any problems I also on embedded systems like well 1 and going to on the Raspberry Pi stuff just works and so because of all these new interfaces where edits now much less work-arounds and you achieve of clean code and you don't have to cry to read the code later it's read was offered before I do so especially in the Raspberry Pi what we is properly integration of open exile what is not possible for examples that you have on this 25 dollar minicomputer you can have ht we do decoding going 0 copy to the display it's just happening extremely fast without using any CPU the answer to that using this now nowadays for work just read you pay back but also complex stuff like having a mighty screen display wall producing some kind of streaming server so for example on the rest I also some camera you can use and well it's
can automatically use their part encoders on the device to produce HD video said 6 4 and then you can send it over the network and all this works in just a minor results I let anything special to know for their applications and then other stuff has happened so just TV news TVE-A API was improved a lot by having 2 guys again as mentioned before it's just working on it that wasn't the case before it was the promise for the last few years that's just working and now we finally that then recently year on what was started is the the decoder elements of its using the defining true could API so many of the article digm ineffectual so that people like companies like well some some more the amnesty unknown providing some you for the next rivals for that Codex and what we now have probably been for that and again it's just working for other companies now the industry is a bit slower you sometimes bought Star Office coming out so this stuff is going to be some what we see on that they're working on stuff and so so I think in the near future we will have proper half integration for all the different chip sets all the induced rumor than some other things that have happened this world now being open GL seconds we have all jails of its special you country use all kinds of different threats and open jail UDF to do everything from a single factor well induced amour if the problem we are completely modified and everything is happening in a different that enters in this course of quite difficult to implement but nowadays it's possible to have all kinds of on GL-based elements in your pipeline that can interact with our problems with the so these alignments that can interact with problems with the other hardware related elements so you can just for example if you're on the filter someone in your life and it starts from some kinds of defects or the deinterlacing or whatever of geo chadors and so on well it's currently runs on all kinds of platforms works on as exit works on Android it works on Linux of course and it works now on the Osprey pions then we can integrate lots of smaller things so for example the blues elements were integrated so we now have some support for and directly using as a Bluetooth headset from the stream then um lots of work has been done on the HTTP adaptive streaming protocol stuff like a as there's Microsoft's streaming so that's basically he what everybody nowadays is to using for stuff like video-on-demand and also some work that has been done on the well unfortunately due and stuff because content providers want that and even that stuff is not working just month the then some thoughts of what has been done on an pictures support anti-TB support so for example in well if you have some some kind of IDB device and want to plug it into your laptop is probably going to work now if that supported by the Linux API then um we integrated reaches 6 5 or see support with now policies for that it's integrated in all kinds of elements for decoding the country using the really great decoder that this inside on the then have support well for the next generation according from Google that benign could which is also I think based the recently started to use it on you tube the but then we also have initial support which is what the next next generation good at that is currently developing developed by accident was it's not stable yet 1 but you can experiment a bit with it won't be there doing much useful stuff but if you want to come help on development there was due to my you have very easy ways to test all kinds of video inputs that as to how the decoders I'm working the and I think it's a good tool books on if you want to work on the recorded because you can have all kinds of input said Eric and the budget stream and then put it into the new encoder many many changes were done on the RTP out is been stuff both client-side and server-side so what we now have a say in the sort of for example I'm authentication support for encryption by as RTP with support for stuff like retransmission so for example RTP usually using UDP because going that lost especially on wireless networks and so on that's all of and support for when requesting packets that were lost lost this can happen completely automatically then we improved over the net clock support so what does that mean with a generic of some object and the some of that allows you to share a clock of a single device over the network of all other devices so what you can have for example is all kinds of displaced shown exactly the same video and audio completely synchronize between devices this worked quite good if already this except for well on wireless networks where sometimes it can happen that's you have lots of very long debates because well for example your country and walking with your mobile device from your living room to the kitchen and then you lose what are the life of a connection for some time and then it can happen that sometimes a packet is going to take might the 2nd center it's arrives on young hunt you down this and makes the estimation of the way the usage of of the synchronization and the master clock and 7 that were quite unreliable but some some improvements were down there to work ignore all these complete all players so that come taking much much longer than the average Beckett's and so I think it's working very reliable now then we also have of course will support the greatest you think that this that people are already starting to use now especially in the genotype that in some the women supporters really great nowadays but it's even in supporting the new subsurface stuff so you can our integrates you on on you the dual by other on the descendant of valent applications some then the on an entity at juicy adjusting services of letters were also on board at 1 . 0 lots of improvements were made and finally we will now have new PTB release it is a video data are based on just
remind what they didn't have the release since I don't know 3 years 4 years or something and I think now when they are making the staring at least you will have a good premediated are based on just a month they can actually be used and it doesn't crash all the time like before really in the past it wasn't in my opinion it wasn't reusable but now it is and then of course with fixed lots of lots of boxes with bugs it under control again in the past and that number so we were in the top 10 products with the most box now we all of them I don't know at which was is we are now because it's not listed there I and so I think in the last year of the last 1 and a half year we fixed about 1 thousand 1 thousand 600 box and excellent so lots of stuff has happened there of course some of them also feature requests at right so for the organized at quickly with the future I think about 10 minutes left and 1 . 4 coming up in March April come after that the 1 . 6 hopefully I'm around tumor and of course lots of new features and 1 thing we we really have no road map uh is documentation and tutorials I think we need of more focused better documentation on just remember com which has nothing to deduce from project um has quite good tutorials the convention that a confused so because of of the 10 as well and so we reach work on lattices something we have planned and the genomes lots of stuff around with the with the stuff that should not 5 etc. I would get etc. They there's also media source audio source integration new kind of special API to signal stuff which we don't have use made this kind of stuff but still others depending on what we need to expose additional information to make all that work in usable in WebKit and you picture-in-picture different audio types and suppliers to genes what New buffering modes and unstable and but then you need to be cities at slower the subtitle stuff of love and I would like to see of course you probably have heard of that's what being that the video is 1 of the biggest things we haven't really done yet what's been done 0 10 of the sum of code project but it was sort of very separate from the normal stuff you can integrate it because of the caps explosion stuff we we had so but anyway in most of the we can't do that we have some very very minimal and API for that but there's more API that needs fleshing out so you know we have the I heads together up and look at all the requirements and figure out a way to signal that properly and this is a bit messy or you just to someone is to do it but it is on the road map other thing that for years and uh more hardware support at this time real and more hardware support but this was to but when we're we're kind of making improvements especially by the open to the Max stuff and the vendor provided was used since then those kind of of narrowing down before beautiful denotes API for providing decoder supporter of the get a fair so blu-ray or something of mentioning only the truth as the video long people have have actually been doing all the hard work so there's a library everything just have to look at it shouldn't take much time of these things no alright promises and already do you have any questions comments yeah at the if yeah not just in the Policies again and that of that the that end of the site of the last person it was not that hard work on that todiscovering yes sorry you and I I I we do have time for that we have to figure that out of the idea for the device discovery and the I haven't actually mention the discovery of 1 but 1 yeah but so it's going over that was not working on we currently but can the idea behind that is it's supposed to be a very simple API some targeted at what we know applications now want basically you have an application says you know give me the camera devices will capture devices will be the use of DVD drives so sound output so something I made its post very very high level very basic it's not supposed to be you know you can discover everything that's been looking it's gonna be simple and they're probably mean that this well possible that state which is a good deal with that at all so I but I mean I I don't have an answer for any other questions so and the I don't know have you been looking for transcoding application and this chance I get on for example I have the try that and it was it is I think very simple it's supposed to work quite well so I don't know if it's if you have a million options to do stuff there's a sourceforge project called to Chen transfer of which is more like it come on line thing was different specialized support for transcoding but if we we invest the sort of good but still things you could do better thing applications exist but we don't have dedicated transcoding a giant something like a trans codon something on the list of the work so those the question is and any of the who where and yes we have unit tests of course I'm in the 4th model the can 1 you make check in the source tree and this kind of test so kind of stuff you can run make changes ground and run slowly at a slow grind of that's what we have there is and they're sort of test suites which you can run over media files of this work going on and to integrate all that into Jenkins the books etc. The so
it's you know it's not perfect yet there's quite early I can imagine lots of stuff parsing and we don't actually do it what about the truth and the other people who do it and then they start fighting bugs and I'm but this
also was their commercial vendors which which is all stuff and the products and the kind of thing that internally and this practice of the new is noted a service of stuff where they have been applied for the 1st time and then someone just spent months working on fixing that sort of hopefully we have progressed much i is something we actually need to but I think in general we know that works well I can run to the discovery of the tree for the 1st time the picture of the crash but the error of war went a on of course also there are lots of companies use stream for the systems that have actually quite good test suites which are public but you can tell because basically you change and then they come back later this word you know you change has cost folding problem if you run this number you know 5 days non-stop and suddenly you will get the little ones and they could recognize so or sometimes they find is that they have stuff that runs the same test 1 over over over all over again like nothing else and they were you know the same about 1 every 2 million tons of the race condition somewhere so and they exist but they're not and then there's some metals going to push sum that stuff public of some new API that people have developed a of In this 1st work we know the sorry we but the but also the cerebrum common to thank
thank you