Okay, so 2018, also known as the year of microarchitecture side channel vulnerabilities, we have about 10 of them so far that have been publicly disclosed and what we're going

to do is we're going to go through all of those and we're going to talk a little bit about how we got here and at the end I'm going to take questions. So let's do this pretty quick. So today we are going to cover what went wrong, then we're going to talk about how modern computers work, all of the details.

Then we're going to go through each of the different exploits, each of the different vulnerabilities and how they work and then how we mitigate or address those in software. And finally we'll end with some comments on related research into the future. So I should point out that the issues are not Intel specific.

Intel gets a lot of attention because they power most of the machines that we use but these, Echo, okay I guess it's alright, but these issues are, these issues affect

everyone right, they affect all the companies that make modern computer systems and in this particular case here you can see over the holidays I've got a emulator here, a mainframe emulator installing and testing mitigations for Z architecture right. So this really did affect everybody, it was not just Intel.

Okay so how did we get here? Well hardware and software people never talk to each other right. I mean let's be honest that's the number one problem right. We have this comedic world in which we have this us and every time I speak it's making a weird sound, okay is that better?

Okay alright if necessary I'll switch over to the other mic. Okay so we have this almost, okay we're going to use this mic because this other one is not working, okay there we go, we'll do that, thank you.

Okay so we've created this almost comedic world in which hardware and software people don't talk to each other and we almost take a perverse pleasure in this right, it's kind of strange, you know we talk about how we're software people or they're hardware

people and they're over there and we're over here, that is a huge problem. So one of the things I want you to take away from today is if you are a software person go make friends with hardware people. If you're a hardware person go make friends with software people, it could start that easily and get better from there.

So let's talk a bit about how computers work, I'm going to let you guys download these slides and go through them later for every line, I'm not going to read them line by line but you know at a high level when we talk about computers we have two different pieces that are relevant to today's conversation.

We've got architecture, we've got micro architecture, so architecture describes the software interface that a programmer is using to talk to the machine. When you build your applications you are targeting the ISA, the instruction set architecture, you compile software that matches the ISA and the ISA describes certain behaviors of

the machine, for example the registers, stack, how you can call functions and this kind of thing. It defines user and privileged or kernel modes of execution and it's the lowest level typically targeted by application programmers but usually that just means a compiler, right?

You don't actually write code in binary, you're usually using a compiler and an assembler and other tools or you're writing your code at a much higher level and you never see any of this but this is the level that you are targeting as a software engineer. So, I think I mentioned most of this, application programs typically target a virtual memory

environment so you compile your code, you run your code and it has this nice concept that it has the whole machine to itself and it can do all kinds of wonderful things and it does this by using an abstraction called virtual memory in which it thinks it

has almost limitless amounts of memory and it requests services from the operating system but it doesn't have to be explicitly aware of the other programs that are running and what their needs are, right? We've built these abstractions over time. Some applications have sub-contexts so maybe a sandbox and a web browser running all those

nasty ads, right? That is kind of a separate context running within a program. Operating systems have additional things that they can do that are defined by the ISA so every chip that claims to support an architecture, for example, X86 from Intel

or AMD or VIA or others will target an ISA. It has to provide all of these things. How it does it is what we'll get to next. But some of the things it also has are these privileged instructions that can be used to manage the software that is running.

And those abstractions include things like providing support for virtual memory. So if I try to use a memory location that is not available right now, I will get a fault in the hardware. My operating system can handle it and the ISA describes how this works. Examples of computer architectures. Of course, Intel.

I have to say the first, right? Nearly everybody with a laptop here is running an X86 laptop. It's a complex instruction set. It's got a lot of history to it. It has certain behaviors. Other examples of instruction sets would be ARM, particularly ARMv8 which powers most of

the cell phones and tablets and other devices. And these are two different architectures. They compete in some ways, but they have many similarities. They both have registers, they both operate on memory, and they both load and store values before they use them. Okay, let's talk about microarchitecture.

So microarchitecture is really concerned with the realization of an architecture in hardware. So if you're a programmer, you typically think in terms of processors, the chip inside your machine, usually when people say that they're talking about cores, and we're of

course aware that the average machine today is multi-core, has many cores inside it. Some of these cores are multi-threaded. They actually partition the resources inside the core to make it look like they have additional cores called threads.

And then there are many of these cores integrated into today's system on chip or really the chips inside your computer. They've got fancy interconnects on them that connect all the pieces together, and typically cores will operate on data contained in caches. We're gonna describe that a bit more in a minute.

So on my chip, I have these cores. I have some memory interfaces that connect to my external memory chips, and whenever I want to use memory in my machine, it has to come in from the outside, from these memory chips.

And this happens through the memory controllers. They understand things like if I'm going to use this value here in memory, it's likely that I'll want to use a value nearby in the future. So they have fancy logic like prefetching an address I'm likely to use soon. And as I use data, it comes in from the memory and gets closer and closer to the

cores that do the computation. This happens automatically through this cache hierarchy that goes from an L3 typically to an L2 to an L1, and each of these is faster. Why do I have caches? I have caches because when I want to use data inside my machine, the closer it is to

my core, the faster I can operate on it. So more graphs. This is a typical server. I have two sockets, two chips here, connected to each other. They have lots of memory chips connected to them, and as I use data, it will flow

between the cores through the caches. So microarchitecture refers to a specific implementation of an architecture. It's compatible with the ISA at the programmer level, but the implementation is kind of up

to the person or team designing a particular chip. So at a high level, a microarchitecture might be in order or it could be out of order. And we'll talk a little bit more about those next. So an in-order microarchitecture is kind of the classical old-school design.

It's the kind of thing that programmers imagine. You know, I have a program, it does this, and then this, and then this, and then this. And these implementations operate in that way. They will fetch one instruction, they will work out what that instruction does, and they will go and do what that instruction says, and then they will do the next instruction. Right?

It's really not a very complex machine, and this is what computers were like in the early days, and certainly a lot of embedded machines are still very much like this. In-order microarchitectures are very simple to implement. That's why they're used in the microcontroller that's in my Fitbit and other devices.

But they're inefficient because if I can't do the instruction, if I can't perform the operation that comes next, because I'm waiting for it to come from memory or something like this, I might stall. I might have to wait. Right? And you can read the slides later to understand what that means. But in-order microarchitectures are simple, and I may have to wait around for

data. Out-of-order, which is what all of your modern laptops and servers and high-end desktops and workstations and your phones are, out-of-order machines have this fundamental realization that my program might say I have to do things in a certain

order, but that doesn't matter because what matters is the dependency between the operations. So if you look at the kind of pseudocode example on the left here, you see I have a series of instructions, but there's no explicit dependency between the first three instructions and the last three instructions.

They use similar sounding, you know, load this into R1 or load this into R2 and so on, but the actual values there, there's no specific dependency between them. So if at a hardware level I can understand that my program is formed from different pieces of code that have dependencies between them, maybe I can

reorder how those execute, as long as the program can't see what I've done. Right, so I'm the guy behind the curtain doing some crazy stuff and as long as you can't see that, everything's good, right? So what I do is I take the program order that the programmer wrote, this is the

programmer says, I do this and then this and then this, and I reorder it. I have this structure inside the machine called the reorder buffer. It has, for example, 224 entries these days, so pretty big. And that's how far ahead, if you like, I could go in the program, I could work out

things that I could do that are later in the code while I'm, for example, waiting for some value to come in from memory. Memory is much slower, these RAM chips, they seem really fast, but compared to the actual speed that my processor is running at, they're really slow. So rather than sit around and wait, I might work out how far ahead I can get.

So I use this structure called a reorder buffer and some fancy machinery around it. So it gets more complicated, but you don't have to understand all of this and there is no class test at the end, OK? So in a modern high performance out of order core, it would look something a bit like this. This is a gross simplification.

But basically what I'm trying to do is I'm trying to hide the latency of loading and storing data to and from memory, right? So my processor is designed to operate on this very small amount of memory called the L1 cache. And what I'm trying to do is I'm trying to hide the fact that that L1 cache is

very tiny by running ahead in the program and reordering these instructions. So I take my program that's in order, I dispatch it to an out of order back end and then what I will do is I will complete individual instructions in order so the programmer can't tell the difference.

This is also known as dynamic execution and it was invented years ago as many good things were for the IBM mainframe. But of course. And as I said, these reorder buffers can actually be pretty big.

Let's go to a diagram here. OK, I'm going to skip that. I have a diagram coming up, I'll come back to that. But examples of microarchitecture. So, you know, Intel, right, x86 is an architecture, but Intel also make processors. So the Intel Core i7 in my laptop, that's why it's a bit old, you know, is the one on the left.

Meanwhile, IBM makes PowerPC processors, sorry, power processors. And an example would be the Power8e, which has particular characteristics. They both have cores, they both have threads inside those cores. They both operate on memory, but they have some differences between them.

Right. That's really the magic. The vendor selling you this chip gets to implement it however they want, as long as it can run your programs. OK, let's keep going. We're going to talk about branch prediction and speculation.

So in a program, I may have a sequence of code in which I'm going to check to see if something is true or false. Right. And branch prediction is really about guessing what the outcome of that test is going to be. So if I'm trying to test, is it raining outside?

I grew up in the UK, so the answer is yes. Right. I live in the US, so the answer is maybe. But in many cases, you can tell ahead of time, you can guess which way a branch is going to go. This is what got us into trouble with Spectre, and we'll come back to that. But we've optimized these processors to guess ahead.

And the reason for that, again, comes down to things like load latency. If I'm going to test the result of something, that something might be in a memory chip far away from me, and it might take some time for me to load that. And meanwhile, I want to keep running instructions that I have available to me. So I'm going to guess which way the branch is going to go.

So applications frequently use these program flow instructions or branches, about one fifth of code is branches, in fact. And the process of working out what a branch is going to do is known as resolving the branch. So rather than wait for the outcome, we're going to guess what's going to happen.

And hopefully we're right, and we'll keep running instructions. And if we're wrong, then we're going to have to figure out how to deal with that. And in some compiled languages, you have things like likely and unlikely. By the way, if you write code and you're kind of, hey, I'm cool, I'm using likely and unlikely,

most chips actually ignore that today. So if you're still writing that stuff, then consider whether it's useful. That's because these branch predictors have gotten so good, actually, that they don't really need the software to tell them. So how we do this is through a mechanism called speculation.

So I have this reorder buffer, but you notice now I've added another column that says spec. Is this instruction really happening or is it speculative? So what I can do is I can say everything after this point is pure speculation. I don't know which way this code is going to go.

And I'm going to tag it in the reorder buffer that way. I'm going to do nothing else to change what's called the architecture state, the thing the programmer sees. I'm just tracking it in this internal thing inside the CPU called the rob or the reorder buffer. And I've got some fancy apparatus there with checkpointing and all kinds of magic where I can basically throw away that state very quickly if I'm wrong.

That's how the magic works. And so speculation is an extension of out of order execution. We tag the instructions, we say, not really sure what's going to happen here. And if we're wrong, then we undo things.

As a result of that, bad things that happen during that speculation may or may not actually have happened. So for example, if I divide by zero in my code, sometimes in some architectures, that's a trap. That's an event. That's an exception. Whatever you want to call it, programmer has to do something.

But if I'm speculating and I hit a divide by zero, maybe the program didn't go that way. So what I'm also going to do is tag in the reorder buffer whether to raise an exception. And I'm going to handle it a bit later. This is what leads to meltdown and we'll come back to that as well.

So at the end, I have to figure out did the branch happen or not and I have to fix it. Different kinds of branches exist. So conditional branches are very simple, this or that. And they're also indirect branches.

So conditional branches typically use a branch predictor, which will keep track of which way that branch went in the past. Because history, if you're not an investor, then history is a good indication of the future. Don't take my advice on stock investments.

Right. But trust me, when you're running code, it's very likely that the future is going to be a bit like the past. So what we do is we say for this for this branch in this code, the last hundred times the result was this. So the next time we're going to guess the same result.

And we have this structure called a branch predictor in which we keep the history. And we should differentiate between different programs. But maybe we don't. And that leads us back to variant two. We'll get to that. Over time, we train branch predictors, right? That's the design feature of them is we train them to guess

the future based on the past. There are many different kinds of predictors, some that are documented, some that are not. Some magic ones that people implement because they're cool. And that tends to lead to problems. But we'll come to that as well. We can also do indirect branch prediction.

So an indirect branch is not kind of a if this or that. It's like a a virtual method or a or a function pointer or some kind of reference to some function I want to run that the programmer has control over. It's a more complex indirect call in my program.

And we also can guess what happens there, right? We've built some very fancy hardware over the years that can guess how indirect branches are going to behave as well. There's been lots of work on optimizing these things because throwing away speculative execution is expensive.

So these things tend to be greater than ninety nine percent accurate. In fact, ninety nine point nines following that. It's crazy what effort has gone into branch predictor optimization. Yeah, so I think I covered speculation at a high level.

This whole thing was done with the idea that speculation was a magic black box. I'm going to guess what's going to happen. I'm going to try it out. If I'm wrong, I'm going to throw it away. And you'll never know. Until you do. And if you do know, then bad things can happen. Right.

And we'll find out about that. Let's talk a bit about virtual memory. So applications have this nice notion that they have infinite amounts of memory available to them in a typical system. We have this separation between each application that thinks it's the only thing running unless it wants to find out about other things.

And then the operating system that does things for it. Applications or tasks, when they're running, they run in something called user space. They make system calls. They ask the Linux kernel or other operating system kernel to do things for them. An example would be, give me the time of day.

Right. And there are services that provide these features. Virtual memory, you can on your Linux machine, for example, you could just run this command right now, cat proc self maps, and it would tell you about all the virtual memory address VMAs that are available to the application, the memory that it's using.

Historically, that included a lot of kernel memory. So we have this notion that we have memory inside my application and memory that represents my operating system. And for performance reasons, we had this optimization

where we mapped or had a lot of the operating system in the same address space. But don't worry, it's all safe because we have protections in place that stop applications from touching that. So we're all good. Virtual memory works through something called page tables.

Each process has these page tables that describe the memory that it's using and how that maps to real physical memory. Because that takes some time, we're going to come back to how that works. We also have something called translation lookaside buffers, TLBs. And TLBs cache, they're not like other caches, they're separate.

They store a small number of recently used translations to make it faster the next time we want to access a memory address. It's just another form of cache, but not for program data. We have lots of these things inside our chip designed to make it go faster.

So how does virtual memory work? Well, my program wants to access a memory address. Different parts of the address it's using, the virtual address, will be used to index into these operating system managed page tables. As you can see there, there's four or five references.

So you can imagine it's very slow if I need to translate an address that I'm using, for example, to contain a variable, right? Something I'm using in my program, even the program code, right? That's all virtually addressed. Every time I do something, I have to perform a translation. This is very slow. This is why we have TLBs.

So after we've done this in the hardware, we store the result. So we don't have to keep doing this because it's very slow to perform what's called a table walk and do these translations. We also have something, by the way, called translation caches that store little bits of these tables to make it even faster to look those up.

These are also vulnerable to a nice attack. I don't have time to describe today, but you can Google it and you'll find something fun there. When you're running a hypervisor or a virtual machine, you may do this twice. So if you are running your machine in the cloud,

sorry, that's in the cloud man, right? I've got to put man on the end or dude or whatever, right? It's not even in the cloud anymore. It's all serverless, right? So we don't even use servers anymore. But let's say we did use servers, right? Then what would happen is the hypervisor would take the nice happy view of memory that my virtual machine has

and say that's great, but actually I'm going to translate what you think is physical memory one more time to what I think is physical memory and I'm right. And if I don't do this, then really bad things can happen. And we're going to come back to that as well. So virtual memory is a nice abstraction that we use because as programmers, we don't have to want to worry

about all the other programs running in the machine, what memory they're using and all of that stuff, right? So we built very complicated hardware that lets you have this nice notion that you're the only thing running on the machine and you have infinite memory. And if you use more memory than you physically have in your machine,

that's why we have this concept called swapping. We'll come back to that in a minute as well. I just hit my step goal for the day. That's great. That's the run, you see. We're going to skip this. OK, so caches. I told you to have coffee before we got here, right?

So caches store small amounts of memory, copies of it, right? I've got these RAM chips in my machine I've paid lots of money for and we all know RAM is getting more expensive by the week or the hour or maybe the second. You should probably run out and buy some more RAM right now. And what we do inside the chip is we store copies of recently accessed memory.

That's what caches are. Caches are organized. That's the term that describes how they're implemented. Typically, they work in terms of lines. So they will store 64 bytes at a time.

And which 64 bytes they store will depend on recently accessed memory. So when I access a virtual address in memory, part of that virtual address will be used to work out which line in my cache that should map to. And then the hardware will test, do I currently have that in my cache or not?

And depending on how fancy I am, maybe I have the ability to cache infinite things. I don't, right? So what I might have to do is periodically, if I try to access something and I work out where in my cache I should store it and I find that I've already got something there,

I might have to evict something or throw it out of the cache. But it's already in memory, so it's still fine. And next time I want to touch the other thing, I'm going to have to pull it back into the cache. Intel uses a common optimization of caches called virtually indexed physically tagged. We'll come back to that in a moment as well.

So virtually indexed means that I think this one, yeah. Virtually indexed means that when I access a memory address, I'm going to cheat because I have to translate this address into a physical address.

So what I can do is, while I'm doing the translation, I can use part of the address to start looking up in my cache and I can do the translation in parallel and I can then work out once I've completed the translation what the other part, what the other half of the cache address should be. And this speeds up how I implement cache lookups, right?

It's called virtually indexed physically tagged. There are other ways to do caches, physically indexed, physically tagged, that's slower. So commonly the level one cache closest to the part of my processor that's doing work is going to operate on virtual addresses and be virtually indexed.

And why do caches exist? Because the principle of locality says things that I've used recently I'm likely to use again, right? When it comes to hardware, we have a choice though between small and fast and big and slow, right? So caches are pretty tiny because they can be really fast,

they can run at the same speed as the core logic inside my processor and then as I get further and further away, I have these bigger and bigger caches that are slower. It's going to be important to understand that caches have different latency, right? We'll come back to that as well.

So I think I covered most of this, you guys can read this later. So side channel attacks with caches. So first it's important to understand that side channels are not new and in fact there are many different types of side channel. Put your hand up here and this might date some of us.

Have you ever heard of Tempest, right? Good, good, right? They're out to get you, right? But the good news is, if you're worried about Tempest, it's all true and there's a great paper coming up at one of the CCS, I think, where they're showing reconstructing data using RF emissions.

So it's all true and it's all great. But we're not here to talk about that today. We're here to talk about other kinds of side channel but I want you to bear in mind that there are many kinds out there and we're just going to talk about some of them. So caches exist fundamentally because they provide faster access to frequently used data.

Which means that there will be a difference in time taken to access something in memory based on whether it is in the cache. That makes sense, right? So I can have a piece of code where I can get the current time.

There's an instruction on Intel machines called RDTSC, read the timestamp counter, and it varies and there are other ways you could do this. But basically I take a look at what time it is right now, access something, work out how long that took. And I can compare the access time to see if something is in the cache or not.

It's that simple. Most instruction sets or ISAs or architectures provide a way to do this. There are other ways to do it without having a counter. And yeah, I can do a measurement. Some processors will provide explicit means to remove things from the cache as well,

which is very convenient. And they'll also provide you with the means to say, hey, I'd really like you to load this into the cache. They're just very nice. They're very helpful when you're trying to attack them. So what does that mean for the different kinds of vulnerability?

Well, how are we doing for time? Good. We've done all the intro material pretty quick. So here's a list of vulnerabilities, and we're going to go through them pretty quick right now. Before I do that, I'll just talk about a typical vendor response strategy. So, you know, our job when we're dealing with these is to work out the mitigations, right?

Work out how to make them go away. We have a finite period of time to do this. That's a good thing because you don't want vulnerable machines. And what we do is we collaborate with many different companies and others out there under these industry embargoes to work out what we need to do.

And we do that. We get up streams fixed and so on, and we get our vendor products fixed as well. And then, you know, we try to provide useful things. Some of you may have seen the three minute videos we do to try to explain these things. And we do all this kind of stuff. And just dealing with Meltdown Inspector took us 10,000 engineering hours, right?

That's not talking about the other issues, just those two. What can we do? Well, modern processors are designed to be able to handle some updates in the field. On Intel machines, you have something called microcode, which is this little blob.

You don't know what's in it. It's a signed encrypted binary, which when loaded can change some process of behavior. It is not magic, cannot be used to patch how we do page table walks or change how the cache works or change how the ad instruction works, but it can be used to fix some things.

And more importantly, it can be used to turn things off that are known to be broken if we have a way to turn them off. And we do have a way to turn them off because we also have in the average x86 processor about 10,000 chicken bits. And they're called chicken bits because you chicken out, right? Hardware design is cool at that. Means I like my new feature, but I might have gotten it wrong.

And so just in case, I'll have a way to turn that off so that it doesn't cost a billion dollars to fix it later, right? So since everybody likes to have that option, there's about 10,000 knobs and dials inside the average chip for things you can turn off. So that's very helpful.

Everything else has to be done in software. We'll do questions at the end, but we'll definitely cover that. Okay, on your Linux machine, you can look in a directory for each nasty, horrible thing that comes up. You will see a new file.

Spoiler alert, we've had more than one. So if any more arrive, then things will appear in here. So let's go through the issues. So Meltdown, I would put this t-shirt on, but there's not time. So I have here a collection of shirts given to me by researchers. I like to go and meet them.

So with the guys who did Meltdown, for example, we did a side channel analysis on them. So for a number of months, I knew it was the guys at Graz and others working on this. And I was watching every domain registration that Graz did for quite a few months. And so we saw them register Meltdown dot help as their test domain. And we saw them register Meltdown attack, Inspector attack.

And I enjoyed telling them afterward, by the way, we did a little side channel attack on you guys. So I think they've changed how they're going to register some of these. Anyway, that's a funny aside. So Meltdown is really about recognising that if we have an out of order processor

using this reorder buffer and we wait, we tag something bad, but we don't deal with it immediately, then bad things could happen. So, for example, I could have a program that reads some data it's not supposed to be able to access.

Right. That should crash immediately. But because I've got this out of order processor that's still running some instructions further on, it may speculatively for a small period of time do something based on that value it wasn't supposed to be able to read while it's checking to see if I should be able to read that. And if the next thing I do after is touch some data that I can measure,

I could infer the value of that secret and I'll show you how that works. So I load a pointer I don't have access to, I do some magic to create some other reference to something I do have access to, I then access the thing that I do have access to legitimately,

that will pull that into the cache. And which element it pulls into the cache, I can measure and I can use that to reconstruct the thing that I don't have access to. So one more time, I have something I shouldn't be able to read, I do some magic

and create a reference to something I do have access to and then based on which entry I read, that one or that one, I can reconstruct the data I should not have access to. And I can do it by, again, doing the timing attack I mentioned before. So here's what's happening inside the processor.

I'm loading a kernel address I really shouldn't be able to do. So the processor says, don't worry, we're all good, I'm going to tag that thing, it's going to be fine. I'll deal with it in a little while, just keep going. Okay, I really should stop here, this would be really bad if I kept going, but maybe I'll just, just a little, oh no, a really bad thing happened there, right?

And what will happen is, before this ever becomes architecturally visible, known as retiring, all these instructions, it will throw everything away. But the things I did to the cache can still be measured. So I'm not reading the value directly, I can never see what happened inside that speculation

black box, but I can do something there speculatively that I can observe outside by touching the shared cache that all the programs are sharing. So the fact that we have shared caches introduces a side channel

through which I can extract the microarchitecture state. So when the right conditions exist, bad things will happen. You can read through this later for even more detail on how that happens. In order for Meltdown to work, I have to have the data in the cache,

the privileged data for example, and I have to have a valid translation for it. So there are two things I have to have, I have to be able to have, be able to try to reference that kernel address to which I don't have access. Again, remember, we used to map everything nicely for convenience, and it has to be in the cache.

And so what we typically do on Linux is something called PTI, or page table isolation, where we just stop ever having those addresses visible when I'm running user applications. Now we used to not do this because there's a performance impact. Every time I update my page tables, when I go in and out of the kernel,

I have to write to some register inside the machine, and that costs various performance. That's why PTI loses you some performance, because you're not using this feature. There are some ways to make it more performant, we can use address space IDs, we can use lots of features that let us keep some translations,

but fundamentally what we're doing is changing how operating systems have mapped their memory so that we can't change how the hardware works, it's going to let you do Meltdown, but we're going to stop that address translation from ever being there, so it won't be able to try reading that in the first place.

Let's keep going, we'll talk about Spectre. Spectre relies on exploiting gadgets. Gadgets are pieces of code in an existing program. I don't write a gadget, something that already exists, but I abuse it. Typically gadgets were used in something called ROP, Returnal Oriented Programming.

Gadgets are not a new thing, but Spectre gadgets are something special. Let's do Spectre variant one. If I have some test in a program, some bounds check, I'm going to access some data structure, for example in my kernel, and when I ask the kernel to do something,

I'm going to give it the location inside that data structure. The kernel is going to be very thoughtful, and it's going to say, well I don't trust what you told me, I'm going to check it. But, because we have branch predictors in our hardware, the branch predictor might say, I know you want to test this, and trust me we'll test it,

but let's just run ahead a little bit. Let's just keep going. So we might inside this, the piece there inside that if statement, that code may run speculatively, and it may do things that I can observe in my cache. I see it 15 minutes.

So again, here's another example. I used this one previously. I've got a check to see whether something is in bounds, and I'm going to briefly continue and access something else that I can measure. And that's how I can extract data using Spectre variant one. Keep going.

Mitigating it, well mitigating it, couple things we can do. We can insert what's called a serializing instruction. Basically force things to go in order briefly. That's expensive, but we can do it. That's what we did initially. Or we can do some fancy stuff. We can say, all right, you can keep speculating,

but I'm going to mask this offset so that if you try to speculate, you'll only ever see a value that's either zero or inside the bounds. You guys can go and look this up in Linux. There's something called array index no spec. It's a bit of interesting code that's used to effectively clamp,

force that offset to be in bounds during speculation. OK, Spectre variant two. So I mentioned before, we have branch predictors in our hardware. They can also be used for indirect branches, where I'm calling some function or something like this.

And we typically use the address of the branch instruction, the actual jump, to index into our branch predictor. And historically, we didn't use the whole address, because that's expensive in terms of hardware. So we used a bit of it, which means you could have one program

running and jumping to branches, training the branch predictor so that when it runs another program, it's going to think its branches go that way. That's called training a predictor. So you can poison it. You can have one program that's malicious training the branch predictor so that another program is going to go a different way

and run some gadget code that it should not run. Very similar to Spectre variant one, but using indirect branches. What can we do? Well, we can turn off indirect predictors. That's incredibly expensive, but that's what we did initially. We had microcode that let us do this.

We could turn them on and off. Or we can use a magic solution from Google. Thank you. Thanks, Paul and friends over there, called Repolines, where we say, well, if this hurts, let's not do it. So we'll turn every indirect branch actually into a return instead,

because they're also branches, but they're different kinds. And they use different hardware inside my CPU. So isn't that great? So I can do something like this, which you can read later, but it's a fun little assembly code sequence that's going to make my CPU say, go here, pack up the return

so you think you're going to return back to where I want you to go. And if you'd like to speculate, look, jump to yourself, jump to capture spec, and just sit there and speculate a jump to self. It's very cute, and it's lovely, and you can read more about it on Google's website. It's very nice. So we can use Repolines. They're not a magic fix.

We also have to do some other stuff. There's a return stack predictor, actually. There's a return predictor in hardware, but of course there is. And you can Google something called return to spec to see other nice, exciting vulnerabilities, which we incidentally fixed. And we now need to make sure that we don't unfix them

when we think all the hardware is working. We have to check it also isn't going to become vulnerable to this one. OK, variations on a theme. Variant 3a, that basically says in addition to Meltdown, I might be able to read some system registers that I'm not supposed to be able to read.

So for example, I might be able to read where my page tables are, and I could use that to de-randomize the layout of memory. There's little tricks you can do. It's not that interesting, but there is a variant called 3a. There's variant 4. Variant 4 is concerned with abusing the load store hardware

in my machine. So when I want to load a value from memory or store it, I go through a load store queue inside my core. And for performance reasons, what I need to do is when I load a value,

I need to see if I did a recent store to that same address to make sure you get the right thing. But for performance reasons, I might decide to guess that there hasn't been a recent update to that memory location. And I will then go and check the store buffer just to make sure. And if it turns out that you did recently update

that location, I might have to throw it away and go and do the load again. This is called memory disambiguation or store buffer speculation. And it sounds great, and it buys you about 10% performance win on your machine. That's great. But I can also abuse it with bad code, particularly jitted or sandbox code

that can abuse this to steer what happens inside speculation. So what we can do to mitigate this is we can turn off store buffer bypassing. It has a performance hit. So by default, we don't do it globally. We do it only for programs that might need it.

For example, on Red Hat operating systems with our OpenJDK, we will use a Linux interface, a PR control interface to turn off store buffer bypassing. Another one, lazy floating point save restore. When I switch from running one process to another,

I might decide that it's unlikely you're gonna use the floating point or vector registers because not every program is doing floating point arithmetic. And so for performance reasons, I might decide not to save all of that state until I need to. And earlier chips, earlier Intel chips in particular, actually had a separate floating point chip,

a separate floating point unit. So I could just turn it off. So what I can do is I can say it's not there. And when a program then tries to use it, I'm gonna get something the operating system can trap and go and fix this up. So it's a common operating system optimization. But unfortunately, in that small speculation window,

when I'm running speculative code, I can read those floating point registers and work out what another application is doing. The fix is to not do lazy floating point save restore and upstream Linux hasn't done that for some time. There's a store variant of Spectre variant one

in which I might do a store instead of a load that I could measure. So we had to account for that. There's an attack called tlbleed. Tlbleed targets the TLBs, these memory translation buffers.

It does it because, remember I said some processors are threaded. They have this concept of dividing a core into separate threads. When you look at proc CPU info in your machine, this is what you see. You see processors there. Some of those are actually sibling threads. They're actually inside one core,

but we've partitioned the resources to make it look like you have more cores than you do. This buys us some performance because, well, for various reasons you can Google, you'll get a performance optimization out of using threading. But because when we do have this threading abstraction,

we share a lot of resources between threads. If I can contrive to run one application on one thread while another is running on the other thread, I can, for example, infer the operation of the TLB and I can see what memory addresses are being used by the other thread.

I can use this for a temporal attack to measure when the other thread is doing things and then using some complicated machine learning, but of course, we have to have machine learning in there. What Ben can do is he can infer what memory the other thread is using and then vulnerable code, for example, some versions of gcrypt can actually be exploited.

NetSpector, we're nearly done. NetSpector is Spectre over the network. This says that I have a leak gadget and the transmit gadget. I have some piece of code that is going to alter the microarchitectural state.

It's going to load something like a Spectre gadget. It's going to do something to the cache and then some other piece of code is going to do something that transmits over the network that I can measure remotely. Using this particular attack, I can leak a few bits an hour and I can perform a Spectre attack over the network.

Last one. L1 terminal fault or foreshadow, which requires this t-shirt here. See, I'm getting a big collection here. The picture on the left there is from a... I was on a plane beginning of this year

at, let's say, some time and I finally got this working quite a few months ago and I was super excited and I think I said the words, oh holy, something like that out loud on the plane and then realized I probably shouldn't say anything else. But this was me running a malicious virtual machine that's dumping the memory of the hypervisor,

which is pretty bad. How does that happen? Well, I told you that when we have virtual memory, we parallelize the page table walk as we look things up in the cache to see if the data is there.

As part of walking through page tables, there's a valid bit that says if that page table entry is valid. And what I do in foreshadow is I, again, because Intel wanted to speed things up, they will speculate around any kind of valid checking,

anything like this. They say this address is gonna be fine, just fine. Send it to the cache now and we'll just speculatively send any data that's there, but don't worry because if it's wrong, we'll just fix it later. Very similar to Meltdown. We'll forward the non-valid entry to dependent instructions and if we have a bad page table,

we can actually use it to read data out. As I mentioned before, when we have a hypervisor, I'm nearly getting done. I know, she's very keen to tell me five minutes. So when we have a hypervisor, it might be translating memory again. Well, good news. Not if we have a not present, not valid entry, it won't.

It will actually just stop and it will send whatever's in that entry straight to the cache. So if I can create a bad set of page tables in a guest VM, I can actually just read, happily read hypervisor memory that way. So there's two parts of this attack that you care about.

There's one about SGX, also known as foreshadow, which most of you don't care about at all. But the two that you do care about are that operating systems, when they use paging and they mark memory as non-present because it's swapped out, what's supposed to happen is whenever that address is touched, whenever that address is touched,

it's supposed to get an exception and fix this up. That's called a page fault. But it will speculatively forward the contents of that entry and I might be able to do bad things. So what I have to do is change not present page tables when I swap things out so that that can happen. And in the case of hypervisors,

what I have to do is I have to flush the cache, the L1 cache, whenever I start running virtual machine code to make sure there's nothing in there that the VM can read. I can't stop it from trying, but I can make sure the data's not there to read. There's lots of other related research.

There's a really good one called Magic that I like that's artificially aging silicon by running instructions. I love that one, that's really cool. So there's lots of fun stuff out there. And where do we go from here? Last two slides. So we have to change hardware in the future, obviously. Some of it's easier than others.

Changes to how we design software are also needed. We need to stop having us and them and more be all together. Open architectures can help. RISC-V is great, all of these good things, but it is not a magic solution. Just because I have an open architecture doesn't mean that my chip is not vulnerable

to all of the same problems. And finally, I want you to ask yourself, when you've got things like microcode and these complex processors and you don't know what's going on inside, can you ever fully trust what you can't see? Thank you very much, and I'll take questions.

Well, yes, that was great. And actually, we have time left for questions. I said I would do it. So, questions. We had one here about ME. Yeah, go ahead. Yep. So some people may know there's an Evil Beast

in every Intel processor, made after the core 2 dual time. It's called ME, it was in the NIC, and then it's merged into the Northbridge. Now it's somewhere in the main processor chip. And it runs the ARC processor on 8.4.86. And so, Intel claims that it configures

quotation marks the processor, but it does something else also. And the question is regarding the chicken bits. If it configures the, I would guess that the ME configures the chicken bits. And if it does so, maybe people could exploit the ME to change the chicken bits to do something weird.

So, by the way, Intel got a lot of bad rap about the ME, and some of it was probably deserved, and some of it wasn't. Every, spoiler alert, every high performance server processor out there today has a little microcontroller in there

that does power management and lots of other things. You've never seen it, you don't even know it's there, but trust me, they all have things like ME. They may not be trying to do remote management over the network, which is why that got attention, but all these chips have this complicated microcontrollers and so on. It's conceivable that it could change chicken bits,

but honestly, I can also just write to a magic undocumented memory address. So if I can run malicious code in the operating system, then I can also configure a chicken bit. So it's bad, but it's no different. And we can follow up with more after.

Let's try to get some more questions out here. Yeah, I think there was another, yeah. Thank you, and can you speculate on why these bugs are discovered now and not 10 years ago, 15 years ago, where they also have existed?

Yeah, can I speculate? Yes, well, that was the problem, wasn't it? So yeah, so I think there's, but it was good, you wanted to get that in there and I needed to get it in there too. So three answers to that. Number one, do I think that scary government agency X, Y, and Z knew about things like Meltdown?

It's possible, but I doubt it given the leaks we've seen. You didn't ask me that question, but this is my opinion. I do think though since January, they've all been very busily looking for these. So the sooner we find them, the better we all are, because they're out there to be found.

And so very smart people are looking, both good people and bad people, depending on your view. And I think, look, for most software people, hardware is not sexy, right? Now I like building servers. I like building firmware.

I like working with microarchitecture designers. I debugged page table walkers and all this kind of thing, but I'm not normal, right? And I think it's not been a focus. I think what's happened in the last few years, so there's a guy, I doubt Anders is here, but on the off chance you're here, put your hand up.

Good, because you should be giving this presentation. So Anders Fogg is a guy fairly local to Bonn, about an hour away. And actually I'm seeing him for dinner tonight, so that will be fun to finally meet him. But he's one of the researchers that over the last few years has started to find these issues. And it's been building one issue after another.

One person finds something, another person finds something else. What a lot of them told me is nobody thought that Meltdown should be possible. Anders wrote a blog, you can look it up. His name's Anders, A-N-D-E-R-S, Fogg, F-O-G-H. Good guy to follow on Twitter as well.

He wrote a blog last July which got very close to figuring out Meltdown. And then people said, nah, that can't possibly be true. There's no way that would be possible, right? So I think a lot of it was people saying there's no way that that would be there and no one would have found it. So no one was looking.

of people with a fresh perspective to start to really dig in and say, just because that seems crazy and unlikely, doesn't mean it's not there. Take another question. Yes, one last question. Okay. So, most of the discussion about these vulnerabilities had to be over in the operating

system to user space barrier, and now we have these user space, let's say, networking frameworks, like DPDK, I'm wondering how they are affected, or if you have any comment to that. Yeah, so we've, good question, so what about user space to user space, or acceleration

APIs and this kind of thing, right? So we have focused on mitigating the worst of it, which is typically the cross privilege boundary, a higher privilege boundary. But that doesn't mean that there's not a lot of other issues out there. There are still a number of times, if you run an upstream Linux kernel right now, you're

using Repolines to protect you from, protect your kernel from being exploited by user space. But you're not fully protected against every application to application attack. That still requires use of one of the microcode interfaces, especially in the presence of

hyper threading. And that's why, for example, I'm not trying to plug Red Hat here, but in our kernel, we have an option, it's very expensive, that's why it's not on by default. We have an option that some customers can use that will still use the microcode interface, and it will say, all right, we're gonna guarantee you 100% safety, but it's gonna

be expensive. So we are looking at all these other things. Some of them are gonna be very expensive to guarantee total mitigation, and so they tend to be opt-in, just because of the performance hit. Very much similar to how with L1 terminal faults or foreshadow, if you want total mitigation on Linux when you're running virtual machines, in addition to the cache flush, you also have

to turn off hyper threading, which is very expensive and we can't do that by default because lots of people get unhappy, so we just have to tell them that you might want to consider this, and they have to do it for themselves. Am I out of time? One more? Maybe a very, very short one. A short 10-minute question, all right.

We got a short 10-minute question right there. And I will be outside, and if there's a bar, I'll probably have a drink or something. Okay, so aside from everybody's gonna die, if I run any workload in a virtualized environment from somebody like Amazon or Google or whatever, how can I, from the inside, figure

out they're not lying to me regarding the protections they're applying? So my corporate answer to that, with this shirt on, is talk to your cloud vendor about what they can do to help you. My personal opinion, with a different shirt on, is that, look, these cloud vendors are

really, really smart. I mean, they're absolutely phenomenal to work with, just being honest, and it's not always popular to say it, I'm gonna give a shout out to Microsoft. In Hyper-V, for example, they realized that there was this problem with secrets and needing to flush the cache and so on.

They have a project, it's commercialized to something called HyperClear. Not only does it flush, it doesn't flush the cache, what it does is it tracks every time it might load something that's a secret and scrubs it. So every time I load something, I will zero it out. And they also separate the address space used inside the hypervisor to basically mean

that there's never a secret loaded, and if there is, they wipe it out. So they've done some very clever things that have required them to completely rewrite Hyper-V over the past year. And the other cloud vendors have done similar things. Some of them have special magic hardware, where they run all the IO through an offload

chip, and as a result, if you're running all your networking and storage and everything else through something else, and you've made the x86, sometimes they call it a dumb peripheral, but a peripheral, maybe there are no secrets, no interrupts happening to pull things into the cache and so on. So some of the cloud guys have done some very exciting things.

It's actually harder for those of us dealing with sort of mainstream, you know, standard industry servers or industry-standard servers to deal with. I think we're done. I think we are. OK, well, I don't want to seem like I'm ignoring you. You can answer it after. At the end, at the end. We'll talk outside. Yeah, exactly, because we are already six minutes over time.

OK, thank you. And don't forget to go online and look at at John Masters on Twitter, and you can see a link to the slides.