Hello, everyone. My name is Victor Tozo. I'm here to talk about Spice. About me a little bit, I was born and raised in Brazil, sunny Brazil,

and I came all the way to Czech Republic to work on this project for Red Hat. It's been almost three years now, and I'm truly happy to be here to talk with Spice with you and thank you for your interest in being here. First of all, how many of you know something about Spice?

Okay, two people, that's fine. About you two, do you use it frequently or no, sometimes? Yeah, okay. The scope of this presentation is not going to go deep in the implementation on how Spice works

or a really low level, it's just like a high overview, so it's fine for everybody, I hope. I don't plan to take all the time that we have here. This presentation plan should be like 20 minutes, 30 minutes, so feel free to raise your hands anytime to ask questions as they come.

First of all, what is Spice? Spice is a way to simultaneously connect to some remote device, some remote machine. It could be a bare metal machine, it could be a virtual machine.

Our main focus usually is virtual machines. It was designed for that, but there are several ways to connect to bare metal as well. So in these two images, quite small, but you have from Windows and from Linux,

you are connecting to two different VMs there. From Windows to a different Windows with Spreadsheet or Linux with Gnome running. Here from Linux connecting to different Windows. The main point of Spice is being able to use this virtual machine like it was your real machine right there.

So you can play video, share files, listen music, redirect your USB. That's our main goal. To talk about Spice, it's important to divide it into four different components,

which are the client, which is the Spice application that connects to this remote machine. The server, which is the library used to expose this machine in a way that the client can connect.

The guest, which is the machine itself. And the protocol, which is how we define all those components to communicate with each other.

So the Spice protocol will define a set of channels, a set of interfaces, that the server will then expose to the client.

So the server usually connects to this virtual machine and takes all this information and basically provides these interfaces to the client. Now we have, for instance, the display channel, which is basically what you see, the drawing.

The input channel could be the keyboard, the mouse, so we can interact with it. Cursor channel, which is basically to send the shape position of the mouse and the position itself. Sound channel, actually there are two channels, the playback channel and the record channel,

so you can do input and output. And USB channels that you can, for instance, redirect flash drives, webcams, and any other device. In this presentation, we are mainly focused on the display channel to understand how it works

and the optimizations that we did in the last years or so. But if you have any other questions in regard to other channels and how it works, I'm happy to take them.

So, Spice display, how it works. What does the server do here is that it takes all the images from the guest and has a way to send it to the client. So the client will just take the image and the server is the one that will send it to the client.

Now, how usually it works, in a bare metal machine, you have your application, you want it to do some drawing and some handling, so usually it asks for the driver, the video driver, to do that, to do the rendering for you.

But when you are talking about remote, there is no real need to do the rendering on the remote machine. So what happens is that the video driver will provide these images to the Spice server,

which then will send it to the client. So basically it's like that, you get any update that you have, it will be sent to the client. Now we start to think about some optimizations there. So if you have just an update, do you want to send it everything?

So in this example, it's like I'm sending that image plus the whole background, which is not really bandwidth resource smart, right? We just need that part, that position that is being updated to be sent. So besides that, we also can do some compression on the image itself.

So that's what Spice usually does in the display channel. You just take what is being handled in the guest, maybe compress that image, and then send it to the client. And then I do have a small demo about it.

So here I have a remote viewer, which is a client, a Spice SDK client, and also a VNC client. I'm connecting to a machine here locally, and I'm passing this argument in the... Oh, you cannot see it. Can you?

Yeah, that's better. So GTK debug updates. So what does it do? Our client is a GTK-based client. So with this argument, we can just say, hey, every time that you update something,

give me some information about it. So you see, it's red. Yeah, you see in the top right, every second, there is an update in the screen. So basically, that's what's happening. The guest updates that, provides this information to the server.

It sends just that part. So it's very good bandwidth-wise. And you can see if I move my mouse here, every time that there is an update, there will be something... Yeah? Just a question. Why is there no red background for the mouse function?

Because it's client-side. Ah, yes. So the question was why there is no drowning for the mouse. So you see that we are updating the guest there, but the mouse is not. So what happens is that Spice tries to... This is a smart way to avoid showing the delay that happens between the client and the guest.

So we just get the position, and through the cursor shape, you also know the shape of the mouse. And we do the drowning on the client side. So this is on top of everything. So that's why. This is like the widget itself, being updated, but the mouse is on top of that. So that's the reason. You see that the mouse does not change here?

Because it's like the client-side of the mouse, right? So, good question. So, that's generally how it works. But now, yeah, I had a video here.

You see a lot of updates here. So my intention was to show you a video being played in the guest. Maybe I don't have internet. Just a second.

So you can foresee what will happen. When a video is being played, what will happen is a lot of updates in that region.

Yeah? So, yeah, and then it becomes a little bit hard to optimize everything because it's a lot of updates being done. It's just raw image at this point. So what can we do about it?

So basically, if we have a lot of updates in the same region on the display, on the guest, we can just convert it to a string, which is made for this, basically. So E-Spice does have M-eJPEG support for several years, but that's not really bandwidth-wise good,

or it's not really something new or useful for different kind of type of clients. So with Jstreamer, we just got support for VP8, VP9 as well, H.264, and it becomes very easy to expand that to new features, to new video codecs.

But Jstreamer, do you guys know what Jstreamer is? There's anyone? Okay, cool. So just a basic overview, Jstreamer is a framework for multimedia applications, and if there is two takeaways from Jstreamer,

it's that it's pipeline-based, which means that it's very flexible. So if you want to build a pipeline to encode the video stream, which is our case, after you get it working, we just need to move something and replace it,

and then we can get a totally different video stream. And also, it's very easy to extend because of that, because if you just need something, a new plugin for instance, you don't need to reinvent the wheel and write everything else, just that plugin that you need. A basic run for Jstreamer is this Jstreamer launch,

so it will provide this, I can show you quickly as well, like this. It's very cool that you can do a lot with it already, like there is this other pattern as well,

and this pattern is called a snow, and it's very nice to know about it, because it's like analog TV back in the old days, and why it's so interesting is because this is a nightmare for video codecs, right? So it means that it's like the worst case for trying to encode a stream. So that's something really nice to always use for testing and so on.

So how it works with us, basically it's about the same thing. We get all the images but in the host, in the spy server, and then we use a pipeline, a Jstreamer pipeline, to convert those images in a video stream.

So basically the three main plugins that we use is AppSource, AppSync, and Video Encoder. That means that we just need to change Video Encoder to change it to different stream types, and then we can send it to the client.

And the client basically is just the opposite. We get the video stream, and then we use the Video Encoder to get the image, or several images, and then we can just draw it on the client.

Yeah, that was how we were doing, but in Jstreamer actually they have, for this kind of applications, they have generic plugins called Play Beam, for instance, that do everything. So you just need to provide them some input, and they will do the decoding. You don't need to deal with Video Encoder, for instance,

it will find it automatically and do it almost by itself. So that's something new in our latest release on the client. And it's very nice also because it can, without effort, support Video Accelerated, so hardware accelerated for decoding.

You just need the plugins and the driver capable of that, and of course the hardware itself. So, streaming with SPICE, who benefits with it? So, yeah, as I mentioned before, we just had Meme.JPEG,

and that's not much useful, because if you think about, we do have web clients, for instance, and there is a lot of huge support for H.264, VP8, VP9, and they usually are working in the backend to do hardware decoding itself as well.

Other things, clients like RespirP or these small, low-end machines also usually do have some sort of hardware decoding. So if we can just send them the right video stream, they will have a better interface with SPICE, right?

It will be more smooth, the usage, and so on. Of course, the CPU usage will go down a bit because of that. So this presentation, before, I did with a friend of mine,

and we had two desktops, and we did a whole desktop streaming, connected to his desktop, and so on. But I was not really in the mood to come from Track Republic here, with two laptops, so I just recorded.

Let's see, yeah, I can do it with... The common desk case usually for SPICE is Tux, right? Tux Race.

This is streaming, H.264, the whole desktop, and yeah, it works quite well, and it's using a few new features from upstream that just got into SPICE protocol. No, Vergil is here for quite some time, but Vergil only works locally.

Yeah, it works. It works quite well. This is actually a guest streaming, using PCI pass-through. So the card goes to the guest, we get that buffer, use it for streaming.

And yeah, that in SPICE protocol just got accepted, and hopefully in the near future we have another component that will do it on the guest, that might do for Intel,

Nvidia, and maybe others. So for the future, we do have a lot of work. We start from below actually, which is a lot of great importance. So SPICE remotely works under TCP, and for real-time video streaming,

it's not really good because of the X on the protocol. So we are really considering going for a new type of channel that will use RTP on top of UDP.

So we hope to gain some reduced latency and so on. We do need some way to... We have those things implemented, but they don't work quite well. A way to do dynamic changes in the stream itself. So for instance, you are trying to stream a 4K display,

which is a lot of data, even for decoding, sometimes even hard for hardware decoding. So if it's not working well for the client, it's not good at all. We need to send some information to the server that is dropping a lot of frames, for instance,

and the server should use it as input to change it to a lower quality or something like that. So we are just in the stage of getting everything working well, and then we are going to work on those things. Better handling for hardware encoding and decoding with Jstreamer.

So as I said, now in the client side we just got this Play Bean, which does almost everything for us. So if you have the API plugins and Intel-based system, it will just work. But if you have a different processor, maybe any video card,

it's something that needs to be developed even in Jstreamer itself, like new plugins or something that needs to get more mainstream. And hardware encoding, it's long run. It will take some time, because it's something that we are not working on right now.

We are working with different perspectives for now, so this will take at least another year to be in some interesting shape. Yeah, and that's it. I think a special thanks for CodeWeaver guys, Francois and Jeremy,

because they were the ones that actually brought this Jstreamer code to SPICE. Check them, check the site of them. They do some interesting stuff. Pavel Grunt, who was my friend, and we did this presentation before. He is my friend, and we did the presentation before,

and part of these slides is thanks to him. And yeah, that's it. Do you have any questions? There are alternatives. There are no machine edits. There are other alternatives for remote test contribution.

Why would I choose SPICE? It depends on your use case. It depends on what you're comparing as well. There are some presentations that compare all of them. All of them have some benefits and some drawbacks. SPICE is really cool. There are several... Well, I'm biased, right?

I work for SPICE, so I say, hey, team SPICE. But there are pros and cons in every aspect. Why you are going to choose this? I think you have to consider the possibilities and what they provide to you. I don't know if you want to compare. For instance, I did not try any proprietary version so far.

I know, for instance, there are a few. They do very well in hardware encoding, hardware decoding. There are huge companies just based for this. They provide better solutions, but are proprietary while we are open-source software.

So yeah, it depends. If you want to do, for instance, 4K rendering on a tablet, SPICE will not be for you now. Yeah? I'm following very well with SPICE.

Yeah, basically, if... Yes, that was not a question. He was saying about how SPICE works well with USB, for instance. So for instance, I fixed some bugs on USB.

For instance, there was this guy who was redirecting his webcam across the ocean. So from, I don't know, I think, the Czech Republic to North America. And yeah, it was a huge leak, a lot of memory being used. But yeah, in general, you can just do it like that.

You just redirect your webcam, connect to Skype if you want, or other things. We have shared folders, we have this audio synchronization with video. I don't know, we have a few features that I think are nice. Yeah?

That's correct. It depends on your distro. In Fedora, for instance, we provide it, it's built with it. So when you download the package, it usually comes together. It also works for Windows as well. So your Windows, you need the USBDK, it's a different thing.

Then you can share your USB devices from Windows client to any guest. Yeah? So the question was if, with Spice and Jstreamer,

if we do have a possibility to not stream the whole desktop. It actually, yes, it depends on configuration. So we have three types of configuration in the host. So there are actually several solutions for streaming.

We can do a guest side streaming. So we have someone in the guest that will provide this from maybe some device in the guest that has this frame buffer and maybe even encoded it so we can just use it. And we have the host side, which is this. And the host side streaming, the protocol, it's based on that.

It can actually, if you have two videos being played, it will be two different streams. It's usually like that. So yeah, that's actually the default. Yeah, if you use different... So for instance, for connecting to bare metal,

we do have this X11 Spice, which works only on XOR. And it does using XOR buffer to stream everything. And that's only the whole desktop. But if you are talking about virtual machines, it's more flexible.

A few slides back, you said the benefits. And then i'm wondering, i mean, there's these HTML files, the Spice and so on. Yeah. Or is there any other time? There are mobile files. Oh, really? Yeah.

It's not supported for us. It's from community. Right. Yeah, but they work quite well. Actually, last year there were a company, I think a company that was bought from Telephonic, I think. And they provided two more clients. A web client that's not our Spice HTML5.

It's a different one. And they have their own demon that runs on the guest, which does a lot of optimizations, so it works quite well. And also, I think Android and iOS, I think iOS as well, client. They support the video stream for Jstreamer.

I did not check, sorry. Yeah, for Jstreamer. I know that their client, I think it's based on SpiceTK, which is the library that uses Jstreamer in the client. But I don't think at the point that they released this, we had Jstreamer on the code. So maybe not. Jstreamer works quite well in Android and iOS and so on.

So it might not be so hard to have it, but yeah. Anyone else? Okay. So thank you very much.