Movit: High-speed, high-quality video filters on the GPU

Video in TIB AV-Portal: Movit: High-speed, high-quality video filters on the GPU

Formal Metadata

Title
Movit: High-speed, high-quality video filters on the GPU
Title of Series
Author
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
2014
Language
English

Content Metadata

Subject Area
Abstract
Movit (the "Modern VIdeo Toolkit") is a high-performance, high-quality, open-source library for video filters, running on the GPU. Come see what the future holds when open-source video editing steps into 2014! Movit contains everything you need and expect for basic video filtering requirements (scale, blur, sharpen, mix, overlay, simple color grading, and others). Movit is supported in MLT, has full support in Shotcut and is on track for Kdenlive support. In this talk, I'll talk a bit about: * What does Movit do, and what doesn't it do? * What do we mean by "high-performance"? Why the GPU? * What do we mean by "high-quality"? How hard can a blend be anyway? * What goes on behind the scenes in Movit's filter graph? * How do you embed Movit in your own application? There will also be a demo of Movit in action, and of course Q&A if time permits.
Computer program Complex (psychology) Open source Projective plane Model theory Control flow Bit Mass Mereology Computer animation Figurate number Filter <Informatik> Library (computing)
Point (geometry) Pixel Greatest element Assembly language Code Multiplication sign Virtual machine Bit Streaming media Element (mathematics) Particle system Moore's law Befehlsprozessor Process (computing) Computer animation Vector space Software Password Core dump Freeware
Revision control Computer program Heegaard splitting Befehlsprozessor Computer animation Vector space Lecture/Conference Namespace Robot Virtual machine Computer-assisted translation Formal language
Curve Pixel Computer animation Ring (mathematics) Lecture/Conference Gamma function File format Right angle Quicksort
Standard deviation Multiplication sign Sound effect Planning Graph coloring Number Graph coloring Lecture/Conference Different (Kate Ryan album) Interpreter (computing) Video game Right angle Quicksort
Computer program Pixel Perfect group Multiplication sign Digital divide Virtual machine Sound effect Function (mathematics) Unit testing System call Message passing Mathematics Process (computing) Computer animation Commitment scheme Energy level Video game Right angle Software testing
Inference Pixel Block (periodic table) Moment (mathematics) Workstation <Musikinstrument> Sound effect Set (mathematics) Bit Function (mathematics) Inequality (mathematics) Compiler Revision control Arithmetic mean Computer animation Graph coloring Chain Core dump Order (biology) output Color space Quicksort Monster group Form (programming)
Slide rule Computer animation Lecture/Conference Auditory masking Übertragungsfunktion Sound effect Trigonometric functions Convolution
Standard deviation Computer animation Vector space Key (cryptography) Image resolution Quicksort Graph coloring Exception handling
Point (geometry) Group action Digital electronics Range (statistics) Virtual machine Cyberspace Streaming media Graph coloring Theory Usability Mathematics Bit rate Color space Electronic visual display Software testing Data conversion Alpha (investment) Multiplication Inheritance (object-oriented programming) Chemical equation Memory card Sound effect Lattice (order) Particle system Computer animation Graphics software Video game Right angle Quicksort Reverse engineering
Point (geometry) Computer program Group action Virtual machine 1 (number) Control flow Device driver Streaming media Disk read-and-write head Distance Power (physics) Component-based software engineering Term (mathematics) Semiconductor memory Filter <Informatik> Traffic reporting Scaling (geometry) Demo (music) First-order logic Data storage device Planning Generic programming Line (geometry) Limit (category theory) Kernel (computing) Computer animation Right angle Quicksort Library (computing)
Computer program Pixel Group action Presentation of a group Context awareness Code Multiplication sign Decision theory Shape (magazine) Open set Neuroinformatik Emulator Mathematics Semiconductor memory Series (mathematics) Extension (kinesiology) Social class Physical system Texture mapping Floating point Sampling (statistics) Sound effect Bit Thermal expansion Instance (computer science) Message passing Befehlsprozessor Graph coloring Chain output Right angle Quicksort Directed graph Classical physics Slide rule Game controller Open source Streaming media Machine vision Goodness of fit Operator (mathematics) String (computer science) Authorization Energy level Software testing Codierung <Programmierung> Traffic reporting Plug-in (computing) Compilation album Alpha (investment) User interface Multiplication Weight Chemical equation System call Compiler Software Personal digital assistant Shader <Informatik> Video game Library (computing)
Computer animation
Computer animation
Computer animation God
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation God
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
way back after the 2nd lunch break and is that I will be talking about uh move it was is my used of the project were less and as you can see it has like this song on it but it stands for the modeling of the toolkit and is the part dangers because anything called modern usually isn't and called toolkit is hideous that's because it tries to be a high-performance high-quality open-source library the filters and the national stop at 1st we do I mean by high performance and to do that we also go back 25 years in time and see this is an easy answer a phone is a bit small because more things that this is actually 25 years ago and as he has held up remarkably well but this is
how you do if a if you were programmers 1989 stick that all the complexity this is only grayscale there's no our fathers nothing that take an image they and image we want a figure between them it by 1 by something comes mass animals by 1 or the other constant and then you add them together it's clear simple and something figures that I made is the slow so we rearrange it a bit we move things around me on that 1 will this is also a standard trick and then people come along so it's
still too slow and the reason why it's so slow I was using floating point and it's not only the floating-point sulfur slow floating point has been
very far from of the last of the last 20 years at this low always moving things back and forth from to flows so do in fixed-point were does not
buying things a bit more tricky we need around a bit we shift the bed but it's not like the same thing now you condiments the 1997 around an both I figured out that people want to do the same things to many different values right we have this pixels we have always because over here was to inform time the process of course as it becomes very very complicated I want this I use that these half an hour to get to work at all it's probably not the efficient it's even worse if you want to write like pure assembler and the freedom of all of mechanization and how you compiler in the world will all the vectorized previous things this 1 because we used this is this and in general the vectorization is not something you can really relax relaxing need to write something like this and of course but the bottom is still our old from the scalar then for a back for the last few pixels and then we invent not only elements that is the S. C 2 ABX 8 x 2 I here is not coming any excitement 12 and you need to write code for every single 1 of these and then it'll comes along that they have multiple courses we cannot be Moore's Law anymore we need you to display your work OK so you need a new IP friends wherever fire friends and if 1 is slow you have problems in chemistry that somehow more intelligent I've done and is of course on top of all this is the whatever whatever whatever OK so which something is 10 years ago what we figure out how to do this then I'll show you how people actually do this 2006 from particle trial if you have ever my edit the video on your Linux machines you are most likely fading using this stuff or something very much like it is built as you can see completely scalar there's no way Diaconis not supported the core so what the heck it turns out this model of of doing things efficiently on the CPU is very very complicated so people not do it at least we don't in the free software world maybe you may do so it around 0 that it this is not reality so passwords less
14 the GPU you cannot really by the best the machine anymore with and without a GPU and if you're lucky you get cats actually you can buy a
lot of more without having interviews and made
begin buy a cell phone anymore without a reason the powerful GPU and this is a bot
repulsive machine it has so there are no longer uncommon we should not anymore assume the user does not have the so
how does the problem this deals out this is the most common shading language and you would use the program here you and this is the formalizing need do everything the CPU version full vectorization would full splitting for handling alleged faces ally of and let users yourself if you would write this in late the only a little more around uh this is basically because mood is able to change many of the things together so it just needs a namespacing and things like this it is refreshingly simple I think this is maybe the thing about these interviews for anything like this it's not necessarily that is so much faster when fall was pencil or on your GPU that I'm pretty sure that on a lack of I could probably the for most of the things that I really want it's so simple in high performance refreshing so so let's see what what we can
do about this and and 1 a high quality this is of course now I'll try not this a lot of old people too much the this is on the wrong
things in I'm sure I mean this is the graphics that never right I'm sure everyone knows this sulfate nevertheless the pixel values that we store in our images are not linear amount like they are a gamma compressed format which means that they have some sort of curves and if you knew any sort arithmetic on them in need to convert them into the real value 1st if you don't do this you and that legislation here on the right prior faded about 50 50 to get a an image that is dark and light
and especially observed that sort the wrong things come through like this this bright spot here is the way to visible MIT these edges here that weighted with right last 1 is held supposed to look like this is how you look at how to look at it's a physical ring the film at the top of each other this is 1 wrong thing you can do enough converting the right of course the example Köhler showed this not describe the what is it does the same thing it is wrong base is
now I never trust beamers right as an assignment in the effect and thinking time but we talk about RGB we never really say what the red what mean what blue and it turns out there are a number of different interpretations of this and if you don't use the right 1 you're emission lieutenant life emission right with wrong colors in some way and you so you really need to to care about this the sort of thing but this is not
that visible so as you know this is what happens if
you click your chroma planes at the wrong side there are a lot of different standards for a for standards it the wrong you get these color fringes around here so you need a coach and you need to really get this right and before you ask a lot of current and if it is offered at this wrong the effector cell but you will see that if you know where to look and finally when I
talk about quality I mean more than just image output quality quality is not a future insults the this it really is a process that is a commitment I wanna do things node has unit tests then of the tests that you can actually say if the test run in your machine most likely everything will work and I really really Trident status as when you think about you want to make something works and doesn't give people the wrong idea of 0 happy use you really want you really want convince yourself you're going the wrong things that a lot of these things seem a really simple but don't be mistakes right there in mind namely hot there's this whole easy to get a 1 pixel placement wrong at play and 1 life subtract 1 by divide by 2 level by my education is an intermediate his penis still get these things wrong right but try to have tests at every time I write a test I feel like this is the way too stupid pass this can possibly feel it fails a new IP perfect programs and not so again minus and call coverage so let's see how it
is to use mood users change effects the
and this is the only I usually say that if you Hello world is more and 5 months long you failed the inference of failed the through MIT that we do 1st defined input the state as in some color spaces citizens and cannot be defined located in the orders be gerade sometime by always added the effect and the station will affect yet an output of some form is at the moment of the bits you want any finite not locked now it gets compiled uploaded the you'd you'd you can run at very very fast inequality 1 when this set 0 here's new pixels renders means how you may need to be looks like and doing this will create this chain here in is a debug dump from when we work and that yeah goal of that and you have out and then internal mean the 1st thing that does this right itself to use on the hideous monster this is low it will blow effect that sort of I want kind of highlights a lot of work for example of the block vertically and I want to add that the 1st thing and then this a low-cost way
way too small for me and I think even in this but after like 19 passes on this mean compiler thing it has sort all these chemical versions that you need it's switch things up the face is consume different colors here these are at different phases because sometimes it's advantages not to do everything changed and eventually created programs for and all that will
turn this image here this image here more
effect I don't have a lot of effect in and the reason is I would rather have good effects than many so each 1 on their own like 10 current data will be more but each member fast they worked they don't union surprises they look good with that fair and
simple to just darkened corners the net using cosine squared off this
this shop and this is not an unsharp mask is actually the convolution that tries to convert the transfer function of the lens so it's it's you don't get this halo effect and the other thing C 2 to that 1 slide go portion but it's that's usually the inter-class saturation saturation of serving in
that since we are operating in a linear space which we should at the image as a whole fuckin war the darker or lighter it just becomes more or less the saturated
and finally this is naturally and yet this is land flora preventive key that actually dust like a fool all with whatever you want of course we have come all the candles go crazy and the goal of a star or whatever you want to if you want like hexagonal blurred them find money so don't try and I really
hope that my them actually works in this resolution it
it does this is some sort of coming these wouldn't hear of like something I can together right they're not meant to be you like thing but they're like the standard color correction tools they can find almost anything except and the so
you can resort drive your your midtones around the highlights around and you can make it as ugly as you want to but of course then happens is boring right the the this is
taking a life this a meeting asked me the only useable video editor on X right now and I hope there's no wonder people here to take offense and this is like my
early alpha prehaps diverse and that actually uses movement through particle the multi of course I mean employing about the clip here it place right now that isn't doing a lot tried is only doing doing like color space conversion of things like that and I can pull out the an effect here the sale blurry like and were more or less and you see this happen 60th that the 720 p it's not in it's a small we know it is not cheating right is actually editing in 720 pianist failing down for for display but if I'm bored with the of course I can do some sort of faith which will work properly I can of course also the a see White bounce rate is new to find some great point and click here that's just focus on here like this is great and already actually space great belong people Linux white balance things were great does not become great is no and wrote a test for it and worked I'm not getting into the 4 strains getting change a color temperatures like super super warm right still hasn't become work anyway doesn't click it again because we're working in your space and you can also the play and were left in the reverse like on that's sustaining wrong way and is a UIC so again this example in the FE I don't think I can do 6 get better things theory if the circuit is a four-year-old for core-collapse but this is not something high and in any way the and I think that's mostly because we're decoding the video on the so so to really from going from if I want do this in current suffer world this would be 5 years maybe now it's 60 and this also means of course have a much better machine like say like any new range media card you can probably goal for are acknowledging you know traits just do things move around whatever you want I
think this was this was the demo the yes
Future work with the best of course world there will be more filters that they will be added began slowly because I don't want a lot of filters that can work and there are some limitations in the why the meteor Hamming distance ashes surprisingness some get older scaling correctly all the ways I want better interpolation indirect common how many people here like in their waste and no on the right and and unfortunately of course the interlaced video is reality many are still have to deal with and even though it's sort of the modern don't talk it I think I might actually have a here and supported eventually and then of course we ever find the of move that is not a program like move is used by charge the uh which is like this generic video library thing we can do all sorts of things which is a term used by shot that's an live a probably few others slow way of so so again whatever so findfont continents a 0 I shared might right this is the kind of things that obviously I want to support so uh on you will find this case and I have not been released until now i just upload them like an hour ago this you really raise and happen to have a machine that saw what might it I can promise it's soft hanging the imple drivers kernel it used to be I think in might action not sampled on the yellow line as if you're breaking going going get library there indicate in mind I was there and headed and report on Sebastian so questions the standard for like the I have 2 questions for you because of first-order used to you heads of providing of ligand for this river because it's very useful tool for the pipes of pollutants that it is a plan for just J. store a not immediately I would be very happy if someone wanted to add support to G. Stewart I don't unfortunately no these generally well respect 1 challenge here though is that for best possible move it performance if you really want to be able to change things together right the have 3 success in which filters you want is 1 G. Stewart component right you don't want them as we different ones that they have to go all the way to do you do something go down to memory go down this even up again which really works performed for a lot of work has gone into this is about about keeping it on GPU chaining efficiently but if you want to do uh these jurors for a node u storm must bear in mind and I'll be happy to talk it was a point is as a user of looking for a tool that uses reputed to transcoding of beautiful but are you aware of some for your from 1 of so so they're so you talk mother GPU Transco right in research you wasted is needed he called which is very which is pretty well supported now right the city had and data you have the power of my kid in the pipe after the 1st right now but it it can certainly that can play video right now including the other way is
much much more tricky it's a very involved thing you will be aware that in classical extent quick sink uh which essentially dust like alienation 6 4 code and there are some commercial GPU video encoders I don't think there are any Linux 1 based on this because this is a very very complicated things by I hear there's talk for x 2 6 4 to use open CL to accelerate it already OK someone says that it already all of work OK do this just repeat for further stream there is the messages going for look ahead but it's not really done it not all they given up even over OK so so I think the answer from latter 1 no I don't think there's a lot of good things right now the damping of course is the 1 0 GPU is doing all multiple pushing his CPU is free to handle the other works yes it was an act of the policy training working you I really chain you think it's in a program like or the chain and by rendering the target and texture from this is that they are their main chain ass in the program most it is run within 1 program there are a few cases where you need to split the chain 1 s when you do something like the blurred where you sample like 15 different pixel then you want to sample from memory you don't want every single computation around 15 times right there are some other human cases where you need bounced memory and that is especially when you need him splitting your chain read news news but 2 different things so again I'll just go back to the 1 the earlier slides here that fossil if I have here right if you look at the colors here that using the first one the mean those apart the same shape as the input the GAM expansion and 1 and actions of alpha handling is amount work is well so in general we tried to combined into 1 sheet as much in can we really rely on that shaded compilers to optimize in life things away so I would love for the preparatory me like smarter about things I hope we can be smaller for but I assume there are people in the room here who can FIL that draw questions both in the front and back In thing but not control the microphone is I don't have make this decision yeah of some of the 2 questions the 1st is a useful tenant will be reviewed yes I will add that many elaborate a bit on this earth I move it mostly uses 16 bit floating-point support internally this is not enough for doing 12 video would flatter think it's enough for doing and if we have accuracy tests in most of the crew truthful affects the check that we actually get within right it's level so so yes it is supported and this is series of I want for 12 that we would have to go to 32 bit floating point is not hard itself but I think open source things don't mean portal that much yet but it certainly something unload support and it's very close to the because we're working fulfilling but fine we don't have this this by a that locks and the other question was undefeated if you video decoder on the GPU is it's a 0 copy operations from memory to but is it it is a trip to the city and but you don't know how to do the decoding on on GPU can certainly put that in the texture whatever extension you have available and we can use this in again Katie and I've doesn't currently do this but it would certainly be possible to do like all the way without ever touching there is did you go extension for that you go and you could guess and there's also of course not the port emulator for VAD I think and run the things against Inglehart I've never tested that it the you want something like this it's and then if you look at them on here now what is doing a favor CPU-bound is like my CPU not able to the cold war in like 2 strings 40 something of yes and so there's some comments on the question about use to Mosul I thing this would be really fun to implement and freedom of the that that the it was very fashionable t love then um on until you have with the API the possibility to do the decoding on the interview and then get takes from that and in that case it should also be able to perhaps local people sitting quality of but not my question and what's the input and output of the 2 and a half with get to the the authority of Texas as input and output and and also of system memory as input of i and can tell you I think so move this plugin based incendiary have a class that defined input and output and have effects 1st middle uh there are at certain there is in that class that takes in this is the memory and of course lack the ability to detect certain users from there I don't actually think I have but it is a
triple what is like can lights or it's not a problem at all as surrendering move it renders to wherever I feel we give so you can eat around the world rented to the screen that the the 0 0 0 you can render to not beyond and you read pixels balances you so in the not that there is of for you OK thank you it's not question yeah like 40 minutes of the of life yeah here I had the feeling that was not a question up here and of this so what are your software requirements be work in this book but you through our note that some assessed requirements are in sense and all you need a syllabus compiler maybe I didn't natives that libraries you need open GL and by OpenGeo I mean any Cardiff and do like shaders which means any carve likely like 8 or 9 years ago so there are no weird isn't it certainly more simple lead I and again for instance that it works I know people have working on both Linux Windows so it uh I tried as they with like again we're 2014 should be allowed to use the fast right uh but this is sort of the weirdest requirement sense what about over the years such as control the likely to whom open charges 17 I think years so it means that Sunil what you need to do uh to users and the Union OpenGeo context this you need initialize nobody else I think they wouldn't cough medicine and transparently the multi now handles all the uh upload and download think describing change things again there was this with the that that as with like 1 in the Ivanov you might hear I have better but in general it's easy get start with if you want to play with it can go to a shop that would also have actually support the has for a while invention and and In the questions and another 1 and you said to be open GL which traders for about open jelly yes journalists to or from his good question I don't support the GLS but right now I'm not sure if I actually will it depends a bit on the requirements are right what you wanna use it for because editing editing video on your mobile it's maybe not a use case that I heard that much about to begin with and I tried very hard to do only 1 thing right and then do it well and instead of trying to fragment myself if you have a compelling use case from DOS and like which is frozen no like this the mobile modern herself in jail I might consider it but I don't want making a afford right but I don't really require any magic things that we need and what want to have a shade on the 2nd thing in lots of ways of reports but there have been many issues of by doing some of the video of understanding the funniest shape for the call or whatever and see that we can be used to be using some to the only cigarettes things of long debates on Z being able to look at some shapes of some of the extensive very procedural tutorial I think my answer will have to be known and just because this is again deviating from what I wanna do then there's only 1 I mean by uh if you want to you can most certainly is moving and write your own effects but I'm not really sure what you would gain from it most likely to have that kind of specialized computer vision things strike maybe you don't really want what gives you maybe you just want to have that your own shaded and you don't want to blur they you want specialized thing the Navy main with stuff for you is really about for me at least 3 about open-source graphics I think we are at questions and even the thought and the the so the next presentation is going to be about the of floating of video on weighted yeah yeah
my
yeah
yeah yeah yeah yeah yeah yeah yeah
that yeah yeah yeah yeah
yeah yeah but and it is yeah yeah
yeah hi
yeah yeah
yeah thank god my
yeah
what
the my
yeah yeah yeah hi
yeah I'm I
use yeah yeah yeah yeah yeah that
yeah yeah that yeah
yeah the
yeah yeah but
it yeah but it that
and
but that
but yeah the government
and
and
yeah by
my
yeah yeah yeah yeah
yeah yeah yeah yeah
yeah the
yeah yeah yeah yeah yeah
yeah yeah yeah yeah yeah
yeah yeah it yeah yeah
yeah yeah
yeah yeah yeah yeah yeah
yeah yeah
but that the
yeah yeah yeah yeah
yeah
yeah the
the but and
the
the and
the
my
the and
and but
it yeah yeah
yeah yeah by
the
yeah yeah that
it yeah yeah yeah yeah
yeah but that yeah
the yeah yeah
it would yeah yeah yeah
yeah yeah yeah
yeah yeah thank you yeah yeah
yeah yeah yeah thank you
yeah yeah yeah yeah
the end
of the the and
the yeah yeah yeah yeah yeah yeah yeah yeah
yeah yeah yeah yeah yeah yeah yeah it is
and the yeah
thank you yeah yeah yeah yeah yeah
thank yeah yeah yeah
thank God yeah yeah that
yeah yeah yeah yeah yeah
yeah yeah
yeah yeah my
yeah yeah
yes thank you good thank
yeah yeah
yeah yeah yeah yeah yeah
yeah the i
yeah yeah yeah yeah yeah
yeah yeah yeah yeah
yeah yeah yeah
yeah yeah yeah yeah yeah the
end of the
and that
the yeah
the yeah yeah
yeah yeah yeah yeah
yeah yeah yeah yeah
my yeah yeah yeah
yeah
hi
yeah yeah yeah yeah
yeah yeah yeah that
she was the was the yeah
yeah the
that yeah
yeah yeah yeah yeah yeah
yeah yeah my
yeah yeah
let
yeah
yeah thank you yeah
yeah but that
thank yeah yeah yeah
yeah yeah yeah
with
the gift shop and we think that it is the it the only thing you need to
if
Feedback