Video in TIB AV-Portal: Voctomix

Formal Metadata

Title of Series
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
In 2014 the C3VOC decided, that it wanted to substitute DVSwitch as a Live-Video-Mixer Software with a HD-Capable solution. Two years later, the FrOSCon 2016 will be produced with Voctomix, our own Live-Video-Mixer System.
Medical imaging CAN bus Computer animation Software Forcing (mathematics) Quicksort
Personal digital assistant Forcing (mathematics) Mereology Machine vision Row (database)
Medical imaging Presentation of a group Process (computing) Meeting/Interview Lecture/Conference Operator (mathematics) Mixed reality Pattern language Mereology Form (programming)
Slide rule Group action Game controller Presentation of a group Functional (mathematics) Open source Video projector Confidence interval Multiplication sign Workstation <Musikinstrument> Control flow Streaming media Machine vision Power (physics) Neuroinformatik Lecture/Conference Computer configuration Computer hardware Videoconferencing Physical system Information Content (media) System call Demoscene Kernel (computing) Software Mixed reality Universe (mathematics) Moment <Mathematik> Video game output Pattern language Musical ensemble
Functional (mathematics) Component-based software engineering Mixed reality Fitness function output Basis <Mathematik> Software framework Streaming media
Filter <Stochastik> Pixel Touchscreen Computer file Open source File format Multiplication sign Image resolution Streaming media Statute Frame problem Medical imaging Personal digital assistant Charge carrier Statement (computer science) Software testing Pattern language Multimedia Physical system Directed graph
Computer program Thread (computing) Multiplication sign Food energy Mathematics Different (Kate Ryan album) Core dump Videoconferencing Color space Position operator Physical system MP3 Touchscreen File format Block (periodic table) Constructor (object-oriented programming) Sampling (statistics) Bit Category of being Type theory Process (computing) Order (biology) Duality (mathematics) output Website Pattern language Right angle Reading (process) Windows Registry Functional (mathematics) Open source Computer file Tape drive Streaming media Event horizon Number Root String (computer science) Boundary value problem Software testing Codierung <Programmierung> Multiplication Cellular automaton Forcing (mathematics) Frame problem Subject indexing Word Computer animation Personal digital assistant Network topology Speech synthesis Local ring
Area Open source Multiplication sign Physicalism Streaming media Process (computing) Software Synchronization Core dump Linearization Right angle Codierung <Programmierung> Information security
Slide rule Touchscreen Multiplication sign Online help Disk read-and-write head Machine vision Goodness of fit Process (computing) Computer animation Object-oriented programming Mixed reality Buffer solution Videoconferencing Object (grammar) Bounded variation Resultant Asynchronous Transfer Mode
Computer program Presentation of a group Existential quantification Group action State of matter Length Code View (database) Multiplication sign Execution unit Source code Real-time operating system Insertion loss Function (mathematics) Disk read-and-write head Mereology Neuroinformatik Medical imaging Mathematics Different (Kate Ryan album) Computer configuration Core dump Videoconferencing Data compression Physical system Exception handling Scripting language Touchscreen Bit Order (biology) Duality (mathematics) output Website Configuration space Right angle Pattern language Quicksort Freeware Reading (process) Row (database) Directed graph Laptop Slide rule Backup Open source Link (knot theory) Control flow Streaming media Revision control Programmschleife Lecture/Conference String (computer science) Energy level Codierung <Programmierung> Traffic reporting Metropolitan area network Content (media) Plastikkarte Digital video recorder Loop (music) Block diagram Software Mixed reality Musical ensemble Game theory
Laptop Metre Open source Multiplication sign Electronic program guide Public domain Streaming media Raw image format Code Graph coloring Neuroinformatik Number Revision control Computer configuration Videoconferencing Software testing Codierung <Programmierung> Sampling (music) Data structure Metropolitan area network Physical system Standard deviation Matching (graph theory) Information Prisoner's dilemma Flow separation Frame problem Loop (music) Mixed reality Duality (mathematics) Charge carrier Right angle Summierbarkeit Reading (process) Row (database) Directed graph
Game controller Dependent and independent variables Film editing Natural number Mixed reality Multiplication sign Execution unit Software testing Computer-assisted translation Communications protocol Asynchronous Transfer Mode
Scripting language User interface Wrapper (data mining) View (database) Schmelze <Betrieb> Combinational logic Repetition Streaming media 2 (number) Computer animation Bridging (networking) Mixed reality Telecommunication Core dump Software testing Family Communications protocol Task (computing) Asynchronous Transfer Mode
Point (geometry) Slide rule Open source State of matter Code Multiplication sign Electronic program guide Open set Mereology Goodness of fit Term (mathematics) Cloning Data structure Data conversion Form (programming) Stability theory Physical system Scripting language Default (computer science) Touchscreen Planning Sound effect Maxima and minima Demoscene Connected space Duality (mathematics)
Revision control Distribution (mathematics) Googol Computer animation Repository (publishing) String (computer science) Right angle Resultant
Scripting language Point (geometry) Computer program Email Functional (mathematics) Real number Multiplication sign Semiconductor memory Mixed reality String (computer science) output Figurate number Object (grammar) Arithmetic progression Row (database)
Computer animation
the wall had to be common them but in the winter sessions for the force commenced the Fung it's to be John would talk to makes uh I'm told us you that sort of hospital I that certain image only consistent and often on demand masterminds is given off a because of mastermind for the software evidence the immediate can and initially was the she we on foot on this kind condo of English muffin them at the opened end at a magazine income must pursue to force monitors the market committee consisting synteny edematous and often on their incomes this mastermind this the I thank you the thank you very much so the as
is that I am an assignment and part of the C. 3 walk uh we are doing colonies especially on the Congress but also on the news on small conferences and on the force could of course and yes
I would like to talk to you about book to make all what makes this all our lives vision makes our lives we do make mixers recorded and uh it's specially designed around the needs of the 3 well but it but I'm sure it would suit to a lot of use cases of the talk recordings and the out but we decided around the needs of some of our use cases and then as you may have seen he sitting
inside all of the 1 of our setups the and behind there's the camera I we have famix operator there friend and revise your the minds of out users of this year
and a lot of stuff around here and because it so much I would like to talk you through some of the pieces and parts that are part of our set up and that that together form the requirements we have to all of the television makes a and that in the end form for Dr. is and I will just talk about the course to talk to makes and the presentation mixing we have the whole pipeline after that so post processing and encoded encoding and last year there was a pretty nice talk form Peter and my and you can watch that all images is a CD and there you can see it all the the remaining pieces of all patterns which are was awesome it but sub 0 In
the beginning there's a camera and there is the public and we want to transport the video of the camera to the public it's simple the that maybe there's another camera and maybe there's a speaker and maybe there's out yeah of course and maybe there's also or there's a speaker but maybe there's also some uh some play all like you the confidence stria FIC my slides here and of course we want we going to because just streaming to the public is good but having it recorded for later use is even better that they are not always talk sometimes there's nothing in the room and we do not want to stream in an empty room or even worse people standing somewhere here and having having the chance in real streaming that left within an hour that wouldn't be good so we might need some kind of falls of breakthrough that we send out when there is no torque and then if it's just the old people would think I'm my because a broken so we need some kind of music sources and so on yeah there's a there's presented us and there's projectors and we want to maybe Scholl decides there but maybe we want also to show the cameras there universe because half well something on the station is more you might want to see it here and there's also some kind of things where we are not really thinking about what we could also do we if we have all this video information in 1 system where can be stream to can may be delivered to another room of the room that's for or something so in the there's a presentation and it's what's what what combines all these inputs and outputs and puts the right thing the the right things in the right places and to the right times but we are not the 1st 1 to do this very well so there is a professional hardware there is things you can buy it just works maybe yeah by the end you switch them money into paid around 2 thousand euros for such a device but it's just a denies it doesn't have control so you need to control panel which cause you want 5 thousand euros all you need which is not so easy to use because it looks like this and you have to use the mouse and it's not easy and it has a lot of patterns and had has a lot of functions and people who were doing that you mixing on the Congress know that we take most of these patterns because you can accidentally it them and then the vertebrates so we have another idea of what we want to do we want our solution to be open source it should be as open source as possible and it should be suffering when should it be sulfur we could also design hardware well we want to run it on commodity hardware you should be able to if something breaks around here if the not behind their egos we can go to the next content by non union of local maybe borrow use and have a drying within half an hour and we have running who set up we want to have it on commodity half as a we can expand easier we can just say OK you have a fast computer bring it although we will use it this no problem and we wanted to be replicated came the the for 2 reasons the 1st reason something might break as I said if 1 of our main accord of resistance die because the the power break is loot was gone we can go to the next call out just by new power clock power break and put it in the end of the work because this is just a PC it has some special hardware but in the end it's just the PC and it should be replicated so that other groups can replicate what we are doing without buying special hardware that they made just not need in another year so we have for example we have and talk to adapt to the 2 people in Australia to the pike colony in Australia Group or to the Linux kernel people and they that hey we we like you set up we have been the same thing here is on the other side of the wrong people replicated our set up and that's pretty pretty cool and that's what we want to do and that's for because it's 2 reasons we want to use software to do that the so if you're looking into the how did To realize video mixing or life vision mixing in self fail and the notes
there are a few options the this is the follow FFmpeg and of course this she's free now and he's stream scenes 1 the
1st few like a perfect fit it has a lot of the functionalities we need already existing it as much a riotous pluggable we will take it in-depth looking at which is stream 2nd it's modular you can you can take a piece out and put another piece in and it's a completely different thing it runs maybe on the GPU on some specialist hotter but the inputs and outputs are the same you can just replace any component with other components and Anthony we use g stream as the basis of the framework to build up makes it because it is important to
know how work away suspended because it's in the end it's not that complicated if you take a look at it I want you to go with me all right then we would step through how walked mix is constructed in the inner the when with and not so in death but and they look that way and that that should give you an idea how you could modify it if it doesn't work for you so to do that the 2nd look at you stream to stream is a
pipeline-based multimedia forever so they say it with the pipeline like something goes in and it goes to some filters and it goes out again here we see but the dude has source it generates a test you but then there is an image sink in the end to which puts it to to accelerate and in between I have a theater that says OK I will only that passes dual that is this high and this is where this wide so but now happens is that G. stream that tries to negotiate between the source and the sink the it tries to find a format that is that both parties are capable of handling and that satisfies the requirements in between so this utensils was a all I can produce you in any resolution you want and it's a missing grisaille they can take nearly any resolution you want maybe up to 4 4 thousand by for those because so she's stream also carrier can all produce any resolution went up to 4 thousand and 4 thousand pixels and then it tries to apply the filter on it and this OK and I'll have to restrict the possibilities to those given values but they have freedom in other things like for example and RGB data all it is centered in the other around blue red green or blue green red it's up to those both systems to decide which they can produce in which they can accept also I'm not making a statement about the frame rate so which has also was a OK I can produce every frame it you want but I would prefer 25 and she's stream tries to negotiate between the source and the sink and find a way to satisfy all needs and if it can't it with and if it can it will be due pipeline and started and the pattern is basically threat where the sources gives the timing Richard has also produces frames and the speed if things statute and the frames are running through the pipeline 2 other agents the test sources could produce frames as fast as it can but it's image thing says OK on the screen and the screen is running 64 and so I can consume 60 frames and it really take as many friends as it can and just as with unique files the the seller
sink but the aspect ratio the sources and the sources starts to run just as fast as it needs to to satisfy the requirements of the that is a little complicated if we talk about a but In most cases just work you that
disasters at zinc and it just works it the G. stream out of the how things for you all the negotiation in the tree to take a look at all the things I just said and here is the introspection of the beta test cells placard we see all the formats that can produce more to prevent you from upstairs RGB X is like a by to abide greener by the blue and bad to align to 32 bits they have dual size formats which are not RGB but another color space and this is this long and it also can produce the from 1 picks up to some big number and it also can produce a nearly any frame that you want so this is just what I say she's growing need to negotiate between all these capabilities and other properties of a registry the that sector look at a little more complicated patterns and I'll have to sources deciliter tests the and as noted his so so I have tool independently running sources both virtues video and audio I know have 2 patterns running basically In both pipelines there more the constructs that that's talked through the from the bottom up but because it makes it easier let's take a look at amount it looks it takes more to but inputs and arranges them into a single stream In this case it makes an M P 4 5 it takes more to the streams and puts them into a P 4 5 we have to encode as here it's 2 6 4 encoder and the MP3 encoder so each of the sources we put them through an encoder and they take raw you in the in our original input and coded you out on the outside and we take these tools Hobbes and put them into the muck said beginning but we get of rotating in the 4 5 in this case but there more to put him in that I didn't talk about that the skew events and the reason for that is that we actually building threats here each of these sources is an independently running threat at the marks on it is to synchronize between them so what would happen is that's say the mock as is OK need 1 2nd audio FIL strain maybe it is like that so the audit has suffered 1 you sentence and the rigid has also to choose 1 frame of the most as is OK I can do anything I haven't enough out of sample so doubted hassles would try to evade and to the MOX can take this on this but the most as OK I can't do anything please go on and continue I have to wait until they reach a friends there so the they both broke each other because they're waiting for each other because the don't run in the same speed the audio sources runs 48 thousand times a 2nd individual videotape reaches source only 25 times a 2nd so I will need a way to to adapt these to speech the and this is what the cues so for they are essentially threat boundaries so everything before that she was a threat and everything after the q like the MOX as a different thread end the because of the set up all 3 things can run as fast as they want and the accused try to to accomplish the spec present OK please don't run too fast but I have to give the doctor on into the muck so and this is 1 of the complicated things Engeström this is really just the doesn't just work doesn't always just work he plot but your sources you encoders into Marx's together and expected to work it doesn't just blocks it stars and this a clear can write and it did to do the the the times just incrementing and the reason is that you have to had in mind what happens underneath and that it builds these these distress for you and they're blocking each other and then nested there's another thing that you need to think about it's this US unchaperoned that's a it's also knew and what it does is it tells us that when you care what this is running program you can take this combined and run it on your is computed it reproduce you at the nice in the 4th but if you kill it usually what happens is a case of threats but the problem is that the empty for is dead broken it doesn't work because 94 has In the start of the file in index and in order to help to be able to write that it's time to this so what we know and forces that indicated the program it sends an end of string signal through the pipeline and has everything OK you inputs on that please finish your work and it's another thing that you have to know and if you don't know it you might wonder why because might maybe sometimes the fireworks and sometimes it doesn't the this sold to take you at a deeper into the revenue back into the rabbit hole in I have basically no on the screen what is what is locked makes In its inner for it's major sources and it's a composite of all componentwise of is especially lament in in in g streamer that allows you to at the aggregating multiple video streams and help each other right and top on top of each other and beneath each other and like here we we're positioning it 50 50 from the top and scaling it to a special the size and also setting some my function and and composer trouble this for you that emerge thank you like applying although sources on top of each other and this is basically the only lament that local makes is in the core it's just a composite of for the read you and we have to simulate demand for the ordeal and everything around is just so support this 1 pipe it the so um we're using your son beta test sources with different patterns so can do so can see which sources which and they don't of both the same and we have this composite are just talked about which aggregates this video streams and it's exactly that's nearly exactly the the the pipe and this running invoked from its but this 1 problem here but we look at this site if still 1 process OK it has to accuse so the idea of the 3 threads running but it's still 1 run process what happens if 1 of the sources that's terminated it's a beta test source it won't contaminated but if it's a we're sorry what happens then yeah sources unreliable that's all because we have no coming back from the G streaming into the real word we have things like Avis and we have people who aren't like that and their polar plaques and there are people who think all I might charge myself on here who does the the camera and another thing is that in the room like this is the root of the requirements may change maybe someone says OK have had to show and we need a new can right here and then we just talk 1 out in the other room and take it year in his story she and predicated back but after the talk through maybe
move the camera backs so there is naturally changes in the set of we're running you can't rely on the sources to be there sometimes this Netragon between and intricacies is and reliable of course and the 2 3 and especially its energy through a pipeline is static on construct set it up just started and then grants and if 1 thing of this pipeline has a problem then the whole of the the the the whole type and the whole G stream of processed food will be terminated the and there are possibilities for than any changes you can say OK I want to have less you and stop all day time then equating in the end of the pipeline and then put another 1 here and then that that cut out this piece of the pipeline you can really think it like like a water pipe system in then removed this piece and put a new warning and then we fear the cues and then then we open this strain and then the rest of the pattern was stopped but doing this in a real-time situation where we have multiple cameras and someone just on but the camera and talk back it's just not viable it didn't work I tried that but it work it didn't work good enough to catch all these events and I was in all the places and and could valves everywhere you needed so we thank you we need something different if we have a set up like this was lots of work to make and
then maybe recording or something else we use something in between we need a blocking the area there was something that allows the camera to run on a different speed maybe go way maybe come back what has the core go away and they become run as different speed by both do not block each other any there are things in G. stream that allow you to do it that the entitlements the nicely using them right now they're running here they're working such between the sources and the core and they're also working in the same way between the core and the things because things also on Bolivar they're spinning method and spinning at X. right may crash or network may go down or something and we do not want to the that the court to terminate when 1 of the sources of things pass away but the inflaton inside a where yeah
they they give a shit about timestamping so they was so there is assuming that time is linear thing and physics is is perfect and everything every crystalline every devices running the same speed and of the of the that's not true so we have cameras and addressed and addressed with temperature and the act it basically screws of audio reducing sync it's of and it it really does this from the 2nd you stop that but just little just a little on something happens like for example um you forget to stop an encoding process and enjoy secures get overloaded and to more than they should then it all you your drifts 4 minutes sometimes so there are obviously not good we do not want to keep using them but currently they ironwork debates and they are 1 of the biggest challenge we are facing because they are re of for they are the the the main source for the that sometimes a lizard perceived um instability that up hats so sometimes you have to restart the then we started again as an and they are 1 of the problems the so we started the rewrite a year ago
and I so it's a complicated thing because the stroma internally is built with the G object I don't know if someone of you has already touched she objects is basically object-oriented programming in C which is basically not such a good idea but OK it's the former they're using and then we're exporting these objects into Python so it works but we could use a little help there it's really a thing of talking about what we how is the timing
recogni I I have no background in these in these topics is on you to me and and I have to really really turn my head around How timing is is working old 1st in thought 1st all buffers could be working between processes beating the data between cues and I would we have to talk to somebody who has more that background in this topic so if you would like to talk about this please come to me after the talk and we will have a nice discussion of the the we we have here in the city rock so it will be a nice discussion I hope but for those who say all this deep technical thing the good news is you have to go on and reviewing all take a look into the fun stuff like How is welcome makes looking what is it doing and how can you use it in you can find your well well you would like to recall talks so 1st let's take a look at how and people watching from the desired the nice thing about to producer so let's take a look on how we which composition modes implemented and you may say hey I know this pragmatic devices or some other designs another vision makes as you can produce any composition you want and that's right but you can do with work to mix up to mix has and a really small amount of variation it can give you in the result of a video and this is 1 of the things we are shooting for because you we do not want I'm sorry we don't want the mixer and is to be able to screw up already recording by falling asleep on the keyboard and and now the camera is right there and most of the screens the and we have seen that people moving around the dual like Picture-in-Picture all of the slides because I look I can direct as people have been doing that on Congress's so we do not we don't want them to be able to do such things so we have a fixed amount of composition most that we can produce the easiest 1 is for example for the Korean just 7 just a camera maybe you
have the slides maybe have a full screen it's easy another thing we can do is
picture-in-picture have a society in big and maybe a camera down below maybe have it and the other relative to speaker's talking real and that is really boring you might have the speaker in big because he's interesting maybe and have the slides and small then we have use like this it's 1 we he was often I think it's 1 did you in big and 1 small enables all that the bit you can figure you can configure them you can say OK how big do I want this to be public should this be where should it be placed but it's a configuration option to configure it and start the work to make the mix and is not able to change that order videos to produce will look the same if we want a conference like this we don't want 1 of the of the talks have this image here and 1 have it in the middle and then the 1 about the role and that doesn't that doesn't compute for us so this is 1 of the EU's reuse of 1 and then we also have this if you can't decide what which pizza every which piece is more important you can have them both the criticized if this is not enough for you and you say OK I want something more complex at 1 3 major sources in the same OK it's possible it's just you know the composer to implement that do his education it can do 2 or 3 or 5 sources but you have to change the code it's not for Dr. mixes made for it's made to of exactly what we needed to see the rock and that may be a problem for some of you would say OK I want to do the Jane and I do not want this I want something else I would encourage you to take a look into the source code is not complicated it's really this is it is is the amount of source-code Optimus carries a would say that later again as you can read it in a weekend and completely understand the docks from the beginning to the end everything then you can change things in order to give you the possibilities to close understand what walked in this already docent what it it maybe you can do with it the next thing I would like to look 1 with you is how reconfigured our inputs and outputs and how they are combined the the so this is an excerpt of the reading um en ts I would like to talk you through some of the things you the obvious things are audio video sources there obviously in in it's the main sources we have it's a camera and maybe a 2nd camera it's this because as input this is the region sources said that our content and they both contain dual NGO yeah OK the speaker some of the time doesn't have value but sometimes detail you back the idea and we want that to be proud of our pattern to then there is it's like a it's like a block diagram then we have the new report which basically gives you everything you get in out but there is this entitlement in between so if you are sure that the mirror part we continue delivering you even if your source breaks away so if you aren't up the camera you we get like the dual and if you plug it back the camera the image we come back wise support free for example use this To make a backup recording or we could use this to make a bank of recording of just the slides to have a video recording of justice lights maybe this is useful we could use that to have stream with just decide what more to believe you cameras we talked about it on the Congress to have the to have the ability like making big football games to change you a view of the stage in real time in the website you can use these parts to do stream according of all you input separately with all the mixing engine interfering from this with the streams the then we have these things they are they have an asterisk they do not honor on the blood or a sexist the reason for them to exist is because in our set up then if you want to learn more about that please take a look at the presentation that our athletic as last year the link is in the beginning of the presentation and we have 2 computers doing artists we have 1 which is running all the video inputs and of core and all the heavy lifting are the encoding and then we have this notebook and in between those devices this offended bent with think there's a gigabit link and 1 of the 1 HD video stream without compression is like 900 may have it so 1 of them would saturate together think and it didn't obviously doesn't work to have all inputs sent to the notebook via a singer gigabit thing so we have to compressed and all and under a book to call so we can send them over to the next and book and this is basically what these parts are they are compressed down versions of the report which you can take below used to take a look on what's coming in the optical or but a computer from the network so then we have this may sense we take we took a look at the patterns that that run in this region except it's basically a composite it's just that it some entitlements popping into composite to unpadding out again and the outer makes as basically the same it can it can have motor value inputs and you could adjust the levels of all of them but could it just selecting 1 this order level 100 % in every every other genocides evil but you could use this this true simulate like a small right and say OK I want to speaker to be this loud and order from the output to be disallowed into it on of optical the the so next we have the sources here and you might have seen it in the slides before if you combine video there's sometimes a backdrop some place when all the jurors and yeah you could just have the black but it looks some kind of oddly and we said everyone to have a looping jewel in the back from behind all or or small images placed on the campus so this is the place where you couldn't play in this this video and when talking about this is 1 thing that might spring into your mind as OK what the makes actually does not any kind of reading video fires all writing them it's just many except read you from sources and sends them out so of uh even the backround loop that you see behind the rituals is produced by the program running and hiding the with the view that into what release and optimize just doing the the composition there are reasons for this design and I will talk about that in a 2nd because there are some other places where this gets important and yeah on the output of is there's a there's a man out part and there is basically the place that the makes the you the mixed you get snatched together and result in a final stream of all the mixed did you know you and others it the same encoder preview thing we had with the sources then we have these things the stream bank and it's a feature that if you have made and do a you might have noticed you can say length the string I talked about this in the beginning if nothing is happening in the room we don't want to stream and in the room even worse people running around here and having private conversations and because the microphone is lying on the desk to have a lesson of them and this it's not good but on the other hand if the mixed as is asleep and it doesn't he doesn't arise and the speaker just begins talking we do not want the recording to just show uh still image so we have it into work to mix and that's another thing where you can see that with the it to says re-mate for the situation we are in now it's just made for this this kind of recordings we have the output port where our recording system is connected and then we have the stream well streaming systems connected and onto the stream you can send the program you dealt with you all you can send another out of sorts all these resources because for example you want to send a still image or as a small animation that says nothing is happening in this room all you might want to send this is another group that says something's happening but we are not allowed to stream that's and maybe you want them to be different view loops on the other hand you might want to play some of your some music we have uh and then an outer loop you might know that from the con was is the basic state everywhere and and these are the the the streams that are entering the city to another source there but so to president together in some bullet points the optical only does the region mixing it does nothing more I does not encode it doesn't decode it does stream it doesn't have any sense of networking other than how it takes in his video it's it's region includes and everything everything the side of the head needs to be in in other programs so for example sources and things external scripts there is a camera source it's never a command running losses in the unit and it reads from the big metagrammar card and throws the region to work mix there's a slight sources it reads from this is of friend of here and throws the dreadlocked mix there's a recording sink it takes a feature from work to mix and cultured and rights to 2 facts and these are all separate processes
and most of them are not written by us they're just configure their ready programs like if synthetic they're just aspect and because of the set up because of the separation of concerns to have tons of options you can say OK I do not want the recording to be made for that authentic I wanted to be something else I want that it is played out we are edging I and then there's a half a recorded that's absentee possible you can do that all I want to stream it via STI to another room and do the recording that it's absolutely possible you can do it whichever way you want the to makes does not it follows the schema and this is another thing that we actually plan even resigning those we want to be able to change things quickly for example here is that false common room 7 and 8 we have a different friend having set up it's all about test if it works we might voted on tonight in our rooms so we are able to have 2 rooms the different configured and all the others my running the sense of fat also the go is 6 times it can be on the same system which needs of raising more secure the system or it can run on another on another computer which also needs more Sibiu because been you need to encode the vigil but we're working on it but having heard on a notebook gives you more flexibility in placing things in the room for example in the room 8 I think all man encoder is somewhere underneath the speaker desk in the mix of because behind them here those of the mix of a because the thing right beneath the camera so both are just a meter away you can place things in the room more easily if the goalie and the car running on different systems with in between the the numbers we
just on this on this so we can use the excerpt these are actually TCP port and they take raw video frames just a RGB RGB RGB following by by by I cannot accept the RGB we're using the 4 2 0 0 sum I don't know some standard used in television and the revision guide and Raul you liked standards just just in PCM encoding and we we place them into will not Oscar contender and we do this because not foster gives us timing information it had other carriers like how big other friends how many bytes with deeper per frame and in which arrangement there and it gives us timing information when as a friend match up with which audio sample but it's right all of you and rob you end that was going in TCP so it's it's and like so 100 megabits and the user do it via loop so every source and sink is running on the same system and they're talking to each other via TCP back yeah actually we should use Unix domain sockets that OK got and we would change that sometimes and but using TCP and using this common doesn't look that combat it is means that you can I use the C to watch it just streams so if if K or an encoder or any other Unix tool that is capable of reading that was gonna and reading TCP the if it doesn't use TCP you can just use netcat to send the duality although it's not a problem we usually use netcat to take a look at the streams running on color because it's just tracking it works the Saul 10 to give some numbers for the preview of foods we're doing currently doing j coding thank you just compressing every frame and detect and putting it together with the same 16 role of the land you know you'll think in Etruscan container and sending it always the going this is what the shells and if you are the do and you see some prisoners some them uh this is an old indigo you this is from the jet including and that the colonies are usually much better quality soul no we have the score and it has sources and it has things in it has some some understanding of its internal structure and knows what we're doing the streaming and recording the but now we need to it to
the to control it so let's take a look at the controller perfectly which you can speak to look to cause to control it and it's actually pretty simple it's just the late nineties TCP protocol so you send command like said to
be due a Camaro wrong and it response was OK the of status is no camera 1 and camera to this is my are and this is maybe port or you can say OK set the complicit mode so it's really easy to do actually mixing and you can easily to overdue mixing with 10 that it's not a problem be clear it's like most of the time I'm testing I'm using our RAP antenna than just writing the commands but you can also the cut remotely SSH so we have a we have situations where we are in the hotel and it's only and so the 1st talk starts at like 8 o'clock and risk nearly asleep you can of course goal down the lobby and Beiderbecke's testing makes the pockets of problem that works it's natural yeah so you can of course cut the meets the unit cat and we are thinking we have been thinking
about implementing an HDP layer so we can actually have about the rep view of this it is motion Gentex so most brothers are able to show it in just open it indirect browser enhance evocative afterward there that might be an interesting challenge for some of you to implement and of course descriptive so during our tests we had looked makes running with the script which change to be due mixing every 5 seconds so we can test it thoroughly on molten combinations it's easy to to strip that you can have the next task which at 8 o'clock in there was the stream it's not a problem it's just 9 this protocol you can write to command tight-knit kid and be done so let's take a look at the going it comes so late in this talk because
it's actually not important I think it's really not that important because it's just a thin wrapper around other things the core things we just talked about it has buttons for other modes and it has a you've used and that's it end it's actually pretty shitty so and um we are treating at every now and then that but I'm really not a good 1 to not to the user interface guys over if you want like would like to change that the quiz actually even smaller it's maybe 7 5 so something is 7 present 5 so if you want to change that and bridge ongoing it's easy and the protocol is really simple communication stuff is already implemented
and so looking into the future In the current state hopefully we'll get there but as then
just the smallest where we actually useful to mix we have a ton of conferences where we where and useful to make successfully I hope none none of them being where a legacy usage but I think they well and made with what the makes and the is the you adjust minimum of 1 of the biggest set up we have done so far with edge rooms it's it's the biggest thing in the year it's even bigger than in Congress but as a as a total but we have also some connections to Australia and they are running to conferences and also that conifers tending to use what to mix and every kind enough to package so there are no other packages in Debian and 1 for a and B and I will talk about that in a 2nd but let's talk about the future plans so when I talked about it I would really like to to increase the stability in the reliability so that you just can't turn it on and it works every time it's not not always like this sometimes it's X was on the start of it you have to start it again but then the rocks but it's a scene where where we need to work on and dropping of light sources is something people are even in 2016 coming with 4 by 3 slides y I don't understand that we are producing a lot of it is in 16 benign they should know that so cropping the black borrowers would be a nice feature improvements going province of course I talked about that if Fuchs compounds after like if someone forgets to turn off their uh the screen reddening thing the just looked reality but it's actually you could programmatically reversed this effect and we cover the the jaw that would be a fun thing to work on and it's a nice name for a future and we want to increase the performance currently running 1 I 7 intensity was and the highest of them like 4 point 2 gigahertz and they have pretty much um they're out there tried loaded but we have some things in the pipeline like running the you according on the GPU was or doing will open jail in in in terms of some uh the dual form of conversion in which a compositing but yeah that's something we need to work on documentation assisting everybody says that but we have I think we have pretty good documentation there is a step-by-step guides on how to set them up there are go to for Pegasus Donald and just in style and I think it should be it should be not too hard to insert it but it it always can be easier especially putting all sources into place enter term yeah the 1st impression that you have would signing it is clearly not dead good you started and saw that because you have no sources and then you need like friend his troops to put data in and out onto the you have a working set up and we need to make it better so some kind of instead of last paid we want some default set up should I inside example sources for you that would be it would be pretty great and it's also a good way to learn how the system works if you want to do that I would really appreciate that full talking about what you can do where this code and it's Python the and it's there and it's public and you can take a look at it and I said it's really easy to read I think I try to keep it clean and take that try to keep structure and it's the amount of code should be readable in everything I think and the there are 3 big parts as a clone which does all the reader mixing there's the green which is pretty Molisch ordered and they are the example scripts and the example scripts are really examples but they are basically mean 99 per cent at the identical to what we're running here so these work for us that might work for you that might maybe don't need not work for you but and there was a good starting point a thing if you know say hey OK I want to run it might offer the talk what should I do but the 1st thing is you need to
produce a version of G string because they have fixed a lot of parks and In the recent versions I would recommend Debian stretched said all really they both have their 3 have the right to strike pictures right there in the repository so you can just inside them and use them if you do not want to run any of these and the we are not willing to build your own G. Stewart you can then adopt a container that we tried and it should work I don't know what the drive and and foremost now for some distributions that are actually packages the debt comes people were kind enough to package them for and so uh
will to has taken all of them and the I'd unknowns and googling of around 4 the makes gives some results for packaging maybe someone wants to do that for is a little far knows ordinal with this so let's packages and so yeah if you have problems running work makes then just come over to this issue often they had I want to talk about some mind I can run up to mix and and I we go with you and try to make you run the self and get you a to point where you can they with that I can't guarantee that a will get there but I would be happy to have someone who had me who says OK I want to take a look at it so if you are interested just cannot enter basic theme that's it if you
have questions you can ask them now please wait for microphone or we're not answer your question if you have questions later you can join on ordered us on a on IRC I'm usually around there we're doing a lot of collocations through see or if it's not your way of communicating please send an e-mail to their and the I would I would think to come back to you pretty pretty pretty quick so there's the 1st question figure In
this and I have a question in your mentioning that missing here then these teams are in which means that what would happen if in a situation when you have 16 that the current is this is going to work I don't know if it's going to work usually have 1 or 2 so I have not tested it with 16 or 32 cameras I would imagine that we did you will run in some problems with the current not being able to dispatch this amount of memory to the process and at this point I will I will I I think I have to repeat it will not work for everyone and not for every situation it is made for exactly such a setup and if you have 16 cameras than I think it might not be the right tool for you unfortunately the place and it will this cable and then replaced is will change for example and you can preserve Ottoman so the last 3 minutes of these the of the yeah and I think I think this is the this would be something that you would do in the next the script so you would have come now and we I think you will you should be able to have a script that we call your video and and then you can say Catani back the last 3 minutes into this input and then makes it back into real progress string that should work here I think that should be possible to but but it's it would not be a functionality of up mix it would be an excellent program on X script and you can add buttons to the queen and to trigger custom scripts and the 2 have would have to write the script and the recording other things right I think thank other Christians the the the OK then the jury much for for time and objectiveness