Okay. So yeah, I'm Stéphane. We talk about image manipulation using SIMD, using Modo-SIMD. I'm doing that talk because the subject is very interesting, because processing images using SIMD is quite obvious,

but Modo-SIMD was done something like in 2007, and I figured out that a few months ago, there was no application at all, or no big application at all leveraging on it. I had an issue by the Debian Packager asking me,

what? F-Spot is using Modo-SIMD, that's the only application I know of, and basically I wasn't checking the right version of Modo inside F-Spot, and nobody noticed.

So no one noticed that, in fact, Modo-SIMD was great, but was not used at all. So this talk is about using Modo-SIMD in your application,

for example, for doing image manipulation, but you can do a lot more than this. So, Modo-SIMD was developed in 2007, I think Miguel is not there to correct me, and it was done by Rodrigo Campera, it was 2007?

I think so. Yeah, I think so. It was for PDC in 2007. So, basically, it's a set of classes and meters of those classes that are, or that could be, if the JIT, so the runtime permits it,

accelerated using the SSE engine of your Intel platform. So basically, for the one who doesn't know about it, SSE allows you to process 128 bits of information at a time,

instead of just working on the float and then the byte or whatever you want. Is it available? Yeah, as the technology is quite old, it's probably available right now on the machine you're using, if you're using Modo.

It was packaged with Modo 2.2. I'd say, if you can, you'd better use 2.4, because there's some issues in 2.2 with the SIMD. And is it worth it? We'll see that in a second. I will show you how much we can enhance our application

and our image manipulation using SIMD. So, we'll take the simplest case possible, we'll compose an image from two images,

and basically, we'll just take the two pixels and add them. We won't shift them at all, because I had to simplify the sample, because otherwise it would imply a lot of packing and unpacking. So, we will just add the different components of the pixels one by one,

and if it overflows, we don't care. We don't care about the result, it's just about showing something that works. So, if...

That's the code for it. Everybody can see it. So, basically, we are loading two PIXBUFF, we're creating a destination, and then we are mixing both source into a destination,

and saving it to a JPEG. So, the non-SIMD version of it... Is it okay if I make the font a bit smaller, so we can see a bit more? At the back, like this, okay.

Yeah, the bottom is not important. What's important is the loop. That's the version without any SIMD. So, basically, you take all the components of each pixel, and you add them, and you store them to your destination.

You have to do that for every line, and for every pixel of every line. All the rest is just boilerplate to make the loop work. So, that's the version using managed C Sharp as you know it,

and leveraging Mono as you know it. You might think that doing that just this way in C Sharp or Mono would be 10 times slower than doing the same than C. It's not true.

It's just something like 10 or 15% less effective than doing it in C. So, I'm not even sure if you don't want to use SIMD, but you want to do image processing. It's not even worth it to do it in C for the 10 or 15% again.

So, we will compile that code, gmcs-simple.cs. It leverages on gdk-pigs-buff, so we don't have to create our own memory representation of the image.

If you want to compile something with Mono SIMD, you just reference it from the command line, and we're using some unsafe context. We'll come back to that later, and it should compile.

Execute it. Here it is. It took... Is it the right one? It's really fast. But I tested it on the battery. So, yeah, it looks like a 1. And the original image is that one, that one,

and the output was that one. So, that's one of the origins. That's the result. That's the other origin. So, the result looks like this. It doesn't do anything important.

It's not a known transformation or whatever. It just shows that we can add pixels and go ahead. So, what will it take to move that code to SIMD?

Take that sample. And up. There is not a version of the same method I developed using SIMD. Here it is.

Yeah. So, what changed? Basically, steel boilerplate. What changed? Instead of doing a G++ here, I'm reading, I'm iterating over four pixels at a time. Each pixel is four components.

It's four components because I checked on the very first line of the program that I add an alpha channel on everything. So, it's easier for the sake of this tool to have every image with an alpha channel. So, each pixel is four components. Each component is eight bit.

And so, if I read four of them at a time, I got my register full. My register of 128 bits. So, I'm reading four at a time. How do I read them? Instead of storing my pixel component, I'm loading that in a vector 16b.

So, that's one of the types in monosimd. Monosimd got types for vector 16b, signed byte. So, for integer, for short, for float, for double, for integer, and for long.

The length of the type defines the number of elements you can pack in a vector. Basically, a byte is eight bits, so you can pack 16 of it in a vector of 128 bits.

That's pretty common in image processing, having eight bits per pixel component. We could port that to a 16 bit per pixel component,

and we would just, at the time, use a vector eight unsigned short instead of a 16 byte. So, here we are. Those are structs. As we are in a non-safe context, we'll come back to that later,

but we can just cast it and use pointer like you did when you were young and doing C. You can just cast your byte array, because my pixel is a byte pointer,

to a vector and get the content of it. Those ones, that's the trick I use to load pixels from my pixel array and store them as a vector. I can just add them.

At runtime, as I'm running them on a recent Mono, as SIMD is enabled, as my laptop has an SSC capability, and that operator plus on vectors is accelerated on my machine,

that line is automatically, at runtime, that addition is not done on the struct by adding every element one by one,

but that addition is done directly by the SSC processor. So, that's my out, and I'm using store line to store it back to my array,

still using some casting magic. So, basically, at the time you add vectors, you can just add them or use any operator that is defined on SIMD,

or any method that is defined on SIMD, and if, by chance, that method is accelerated on your machine, the JIT will run the accelerated version of it. So, we recompile it.

GMCS. Okay, run it. Nice, it's slower. Excellent. It was doing fine an hour ago.

You will have to trust me on this. That sucks.

That's the old version. How do you write with dash o equals minus SIMD to make sure that it doesn't use it?

Sorry? I'm running without SIMD optimizations from the command line,

because maybe it's detecting that you don't have it back. If your machine doesn't have SIMD, it will fall back to the software implementation. Yeah, I know. So, what I'm thinking is, why don't you try to see if that's what's happening? Yeah, I would just add the detection at runtime if the addition is available for that.

Do we have a non-verbal amount of write for this? Yes, on the command line. Which is? Dash uppercase O. Oh, yeah. Dash uppercase O equals minus SIMD.

Like this? Yeah, but lowercase. Yeah, so basically what's happening is it's not using hardware. It's not detecting that you have it. It's easy. I'm using 2.4, and here I'm using...

Thank you again. Yeah. So, gmcs monosimd package gtk sharp dot zero symbol cs.

Unsafe. Unsafe. Yeah. Which one is it? So, but it's on par with the other one, and it shouldn't be.

But anyway, the output is the same. That's important.

But I will show you a real result on a more complex case in a sec. That wasn't to show you how fast it was, but just to show you how you could easily do it. So, as Miguel said, we can check for this probability at runtime.

So, meaning that it was proven in this case that the fallback that was implemented in SIMD was less efficient than my simple addition using regular arrays.

So, if you want to avoid this and fallback in your code to one or the other, you can easily check if a method is accelerated at runtime.

So, basically, in the second sample, I'm just checking for acceleration on every... I would love to make this a bit smaller. Basically, that's another example we'll show you in a second.

And for every method I will use a bit later, I'm just printing accelerated or not accelerated. That's interpreted at runtime and not at compile time. So, you could decide at the time you write your code.

If I do have SSA4 instruction, I will do it that way. If I only have SSA3, I will do it that way. Otherwise, I will do it yet another way. Maybe by leveraging some other native library or whatever.

Or you can even fallback to the default implementation of monosimd. But, as we just showed, it's basically quite slow. It does the thing it should, but not much. So, basically, you ask the runtime, which is a static class, if a method is accelerated.

And basically, SIMD is mostly, except for those...

That's the multiply I was adding, but it's on the 4f vector containing 4 floats. That's how I would have checked if the addition was available on that machine. And, as most of the methods of SIMD are implemented as extension methods,

that's the way you check for that. The code is a bit cryptic. I will post it on my blog on Monday or next week. No, it's not cryptic, but on that small number of columns, it's not easy to see.

So, basically, you can ask the runtime if a method is accelerating. If the method is accelerated, the runtime will accelerate. Otherwise, it will fall back. But, as you can check at runtime if it's available, you can programmatically decide to pick a fallback or another.

Yeah. Where are you? So, basically, if you want to do... That doesn't apply to SIMD. But, basically, if you want to make some high manipulation on model or .NET,

you can't just use the image as a byte or whatever it is. Because the boundary check of the URI will slow the thing down.

So, most of the time, you just have to put the part of the code that matters in an unsafe context.

Saying that you're allowed to use that CLI construction using pointers, doing... No, that's in the other case. Doing conversion and cast that are unchecked by the runtime.

So, you have to do that if you want to do some efficient image manipulation on .NET. Following that, you have to either declare all your metadata as unsafe

or just a block of it and pass unsafe to the command line or to the compiler. So, it goes. True. Let's move to a more complex... Five minutes. Okay.

I will just show you how I'm doing... So, some general advice. If you're doing SIMD processing on Mono, try to avoid computing it float.

For image manipulation. Even if there is some float operation using SIMD, we don't have any accelerated way of converting a short to a float and a float back to a short, except using .NET casting.

And that's quite expensive. If we were able, but that's not the case, to use the MMX CVT operation, but that's not possible because there is no MMX acceleration on Mono SIMD,

it would be possible. But it's not impossible. If you just want some more precision, you just unpack your byte to a short or to an integer, you do your computation using an integer computation, and you then repack it to a byte or to a byte vector

before putting it to your pixel or to your image. So, that's quite important. And if you do that, it will be accelerated,

but the casting will just kill the performance. Oh, the performance will be just on par with native implementation, not accelerated. So, that's one thing. I heard that SSC5 would have some operation accelerated

for converting float back to end. I've read something about it, but I didn't see anything. I don't have deadline or whatever. Anyway, for the machine you have right now, it doesn't have SSC5. So, for now, just avoid it.

So, the other example is the example I showed on my blog some weeks ago. And that's an application. That's just a port of the GDK pix-buff saturate and pixelate operation.

Saturate and pixelate is a meter of GDK pix-buff that basically you can either pixelate or desaturate. I only implemented the desaturate case.

And you don't have to desaturate 100% of it. You can just desaturate it a bit. So, basically, the math for this is you have your saturated image. You desaturate it completely.

So, desaturating is taking something like 80% of the green channel and 10 of the two other ones. The number will be shown a bit later. And you take the mean and the weighted mean depending if you want a fully saturated or fully unsaturated one.

So, you add both. It's quite hard to do it using SIMD. I will go fast because I'm almost out of time. Which one is it?

So, that's the one. I'm using a child multiplier. No, it's not 80.

It's 30% of red, 60% of green, and 10% of blue. And that makes a white and black image that looks great. So, basically what I'm doing here

is I'm working on a vector containing 8 unsigned short. And there is no horizontal add on... Basically, if you want to desaturate an image, you just multiply all the components by some magic numbers.

And then you add all the components together and you divide by 3. There is something nice if you are working on float. You can do an horizontal add and it will add all the components together. But there is no horizontal add on vector 8 US. So, you can use shuffle.

And shuffle is a way to mix the component of your vector. And what I'm doing, I'm shuffling twice. I'm doing two shuffled versions. So, I have all the variation of it.

And I can add it at the end. And have a vector containing the same value on the 3 main components. And that's it. That's how I'm doing the desaturation. And then, yeah.

So, I'm shuffling and adding. So, I'm having the intensity. And then, what I'm doing... At the time, I should be...

Okay, I will post the blog a bit later. I can show you how it works. GM, oh, it's already compiled. Okay, desaturate, take z. SIMD. Okay, and that operation...

So, that's the check I told you about a bit earlier. I'm not doing any treatment of the check. I'm just printing it accelerated or not. Oh, yeah. So, basically, I'm checking... Yeah, I am.

I'm checking if the methods are accelerated. I'm not doing any treatment of it. Because I know they are on my machine. So, that's the time it took to make a desaturation of an image. Okay, here it is. It's a partial desaturation. And I can just show you how slow it is using GDK1.

What's my argument? There's no argument. Yeah, it's no argument. Okay.

That's GDK. You're using it. You're using it every day. And it's not accelerated at all. So, that code, desaturate, it provides an extension method to GDK pigs buff. And that's really worth it.

It's seven times the better. And that's very worth it. Yeah, so I'm out of time. I don't know if I will have time for questions. But one more thing. The KDs I showed you are oversimplified. Because I was always assuming I had an alpha channel.

I'm doing that on some special images that are a multiple of 16 on each side. So, I don't have to check for the remaining bits at the end. So, it's just for the matter of simplifying the loop.

So, a real implementation of it would take something like 20 lines more. But that wasn't necessarily needed for that speech. So, here it is.