Confidential Containers and the Pitfalls of Runtime Attestation
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 542 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/61463 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 2023315 / 542
2
5
10
14
15
16
22
24
27
29
31
36
43
48
56
63
74
78
83
87
89
95
96
99
104
106
107
117
119
121
122
125
126
128
130
132
134
135
136
141
143
146
148
152
155
157
159
161
165
166
168
170
173
176
180
181
185
191
194
196
197
198
199
206
207
209
210
211
212
216
219
220
227
228
229
231
232
233
236
250
252
256
258
260
263
264
267
271
273
275
276
278
282
286
292
293
298
299
300
302
312
316
321
322
324
339
341
342
343
344
351
352
354
355
356
357
359
369
370
372
373
376
378
379
380
382
383
387
390
394
395
401
405
406
410
411
413
415
416
421
426
430
437
438
440
441
443
444
445
446
448
449
450
451
458
464
468
472
475
476
479
481
493
494
498
499
502
509
513
516
517
520
522
524
525
531
534
535
537
538
541
00:00
Run time (program lifecycle phase)Projective planeInformation securityBitNeuroinformatikCurvatureComputer animation
00:41
Formal verificationElectronic signatureMeasurementLattice (order)Computer-generated imageryFactory (trading post)Kernel (computing)Workstation <Musikinstrument>MeasurementTraffic reportingWorkloadComputer hardwareAdaptive behaviorMedical imagingPoint (geometry)Projective planeConfidence intervalConnected spaceRandomizationComputer-assisted translationVirtual machineFactory (trading post)Public-key cryptographyProcess (computing)Generic programmingLevel (video gaming)Electronic signatureGoodness of fitOrder (biology)BitAreaTerm (mathematics)Fitness functionNeuroinformatikService-oriented architectureKey (cryptography)Kernel (computing)Validity (statistics)Standard deviationLatent heatComputer animation
03:33
Factory (trading post)WorkloadCodePhase transitionMeasurementMultilaterationPresentation of a groupKernel (computing)Endliche ModelltheorieValidity (statistics)BootingOrder (biology)Service (economics)Computer-assisted translationCASE <Informatik>Water vaporCAN busRight angleProgram flowchart
04:50
Information securitySpacetimeComputer-generated imageryGeneric programmingCodeKnowledge-based systemsPhase transitionData modelField (computer science)Electronic signatureSign (mathematics)Identity managementInformationConnected spaceTrailRun time (program lifecycle phase)Virtual machineField (computer science)QuicksortWorkloadTraffic reportingPhase transitionCodeDifferent (Kate Ryan album)Electric generatorPublic-key cryptographyLevel (video gaming)NumberContext awarenessComputer hardwareFlagAreaMeasurementValidity (statistics)Key (cryptography)Form (programming)Sensitivity analysisClient (computing)Point cloudSpacetimeMereologyArithmetic meanState of matterGeneric programmingInformation securityKernel (computing)Slide ruleDataflowAuthenticationPoint (geometry)Goodness of fitSimilarity (geometry)InformationAdaptive behaviorWorkstation <Musikinstrument>Projective planeEntire functionCASE <Informatik>Confidence intervalControl flowProgrammer (hardware)Right angleOptimization problemMultiplication signGroup actionEndliche ModelltheorieMedical imagingComputer animation
11:00
Field (computer science)Knowledge-based systemsElectronic signatureComputer-generated imagerySign (mathematics)Identity managementInformationWorkloadEncryptionNP-hardInformation securityLatent heatMeasurementCASE <Informatik>Keyboard shortcutWhiteboardElectronic signatureWorkloadProjective planeMedical imagingBitFormal verificationSpacetimePublic-key cryptographyPresentation of a groupValidity (statistics)Field (computer science)Identity managementInformationConnected spaceQuicksortImplementationDifferent (Kate Ryan album)Ocean currentNumberTraffic reportingSimilarity (geometry)Computer hardwareVideo game consoleMereologyAdaptive behaviorMultiplication signGeneric programmingInstance (computer science)Forcing (mathematics)Category of beingRight angleVideo gameState of matterWorkstation <Musikinstrument>Sound effectComputer animation
17:11
Knowledge-based systemsData modelIdentity managementTerm (mathematics)WorkloadMeasurementQuicksortValidity (statistics)Projective planeTraffic reportingMultiplication signArithmetic meanConfidence intervalCASE <Informatik>SoftwareRight angleElectronic signatureImplementationComputer hardwareMeta elementEndliche ModelltheorieInformation securityGeneric programmingLevel (video gaming)DiagramMereologyAdaptive behaviorInstance (computer science)Product (business)Computer animation
20:29
Element (mathematics)Point cloudProgram flowchart
Transcript: English(auto-generated)
00:05
Hello, my name is Tobin, and today I want to talk to you about an interesting security consideration that has come up in the confidential containers community and that might apply to other confidential computing projects as well. I know that remote talks can be a little bit flat, but if it's any enticement, I think this
00:23
is maybe the biggest security issue that our community has faced so far, and possibly also the most interesting. Before we start, I want to thank my colleague Dov Murk, as well as community members Anand Krishnamurthy and Mingwei Shi for helping to discover and uncover and think about this interesting problem.
00:42
Before we get to the main issue, I want to revisit one design principle of confidential containers that is significant here, and that's the idea of decoupled measurement. With a lot of confidential computing projects, the hardware evidence is used to measure or includes the measurement of the workloads. And in some ways, it's very intuitive to want to measure the workload, the most significant thing with the hardware.
01:02
But the drawback to this is that the hardware measurement is often platform specific and a little clunky. In confidential containers, we prefer to use standard techniques such as container signatures or encrypted containers. We do this with a two-stage process where we use the hardware measurement to measure the stack, essentially, the CAD agent and its associated tools.
01:23
And then once those are trusted via this hardware measurement, we can use those tools to measure the workload itself. This is handy because, again, it allows us to do standard stuff, but it also means that we can use a generic guest image, which is a good fit for CAD containers in particular. It will probably become clear why this generic guest image is significant here,
01:44
but for now I want to talk about these so-called evidence factory attacks. I think that's more or less a made-up term, but it makes sense here. The crux of this attack is that we're going to use an attestation report that we aren't really entitled to to get secrets that don't belong to us, to attack a target key broker.
02:02
How are we going to do this? Well, attestation reports, once they're out in the wild, can't really be tampered with. They can't really be stolen. They can't be tampered with because they're generally signed by a private key that's only known to the hardware. And they can't really be stolen because you should probably put a nonce inside of your attestation report that's going to link this attestation report to a very particular connection to the KBS.
02:25
So if you can't steal one that's floating around in the world, how are you going to attack the KBS? How are you going to impersonate somebody who has a valid report? Well, one way might be to launch your own virtual machine. Launch your own guest and request your own attestation report from that guest.
02:42
Of course, if you do this, the launch measurement will probably be wrong because the launch measurement will show, hey, this is some random person's VM with a bunch of random stuff in it. It's not the VM that the KBS is expecting to provision the secrets to. On the other hand, if you were somehow able to get the launch measurement to be correct, well, then the VM wouldn't be malicious and you wouldn't be able to access it because the launch measurement should guarantee that there's no way in.
03:06
There's no easy way into the guest. So to be clear, what we're really trying to do here, the crux of this attack is figuring out if there's a way for a malicious virtual machine to manufacture a valid attestation report. And when I say valid attestation report, what I mean is an attestation report that has
03:21
that reports the correct firmware, the correct initrd, the correct kernel and the correct command line. If this exists, if there's a way to do this, then we have a bit of a problem. So how is somebody going to break into the guest in order to request a valid attestation report? Well, clearly it's not going to happen by tampering with the boot process because that would change the launch measurement.
03:46
Well, that's not what this presentation is about, at least. Instead, we're looking at a later phase of execution, basically, where the guest is already running, where the guest has already booted and already has used a valid attestation kernel.
04:00
So it's already got the correct launch measurement. Can we get into the guest once that has happened? Now, in the community, we've looked at the trust model of confidential containers a lot. And one thing that stuck out from the beginning was the API between the guest and the host. In this case, this is the API of the Cata agent. One discussion from the beginning was how do we harden the API of the Cata agent
04:21
so that we can't have arbitrary code executed inside of the workload or inside of the guest? But it turns out, when we were thinking about this, that we may have overlooked another pretty significant attack surface, a large API. And that is the API between the workload and the guest, the API of the container and the guest kernel, essentially.
04:43
This is a very large API. What if a workload could break out into the guest? Would that cause big problems? The answer, in short, is yes. If a malicious container can break into guest user space, it can generate a valid attestation report and use that to request secrets from any KBS.
05:02
And there's two things that are really bad about this in particular. First of all, when I say any KBS, I really mean any KBS. Not even a KBS belongs to a particular client. Like I said, we have a generic guest image, meaning that the launch measurement we're looking for is the same across the entire project. Anyone who's using confidential containers is looking for the same launch measurement, essentially.
05:23
And so if I can generate this attestation report from a guest that is booted correctly from that image, I can then request secrets from anyone. That's obviously not good at all. Another thing that's pretty bad about this is that containers are executing arbitrary code by design. So we might say that there's some way to limit that a malicious container would be able to run.
05:44
But in fact, there isn't, because part of confidential containers, one of the features is that we can run containers. We can run any container you want, essentially. Now, some of the good news is that this is not actively exploitable because it relies on there being a way to break out of the container into guest user space.
06:04
But the flip side of this is that you might say that the security of confidential containers, at least in this case, reduces down to the security of non-confidential containers. Just to this API, once you break out of that, you have the ability to steal secrets. That's really not good. So let me walk through this attack step by step just to make it absolutely clear how it goes.
06:25
The first step is that the attacker needs to craft a container that can execute code in user space. And this is not trivial, but it's far from impossible, and things like this have happened in the past. Then the attacker is going to run this container with confidential containers.
06:41
They can do this locally using their own, you know, SEV machine or whatever, or they can do it in the cloud. It doesn't matter. Anywhere where confidential containers is running, they can run this. The container, once it starts up, is going to connect to a KBS and make some sort of request. And as part of that, there's an authentication flow where it will get a nonce from the target KBS, right?
07:05
It's going to keep track of that nonce, and then the container will break out and get this valid attestation report. When it requests the attestation report, it's going to put the nonce that it got in it, and it's also going to put its own public key here. So it's basically just setting up its own secure connection. Once that secure connection is there, it's just going to send this, you know, it's just going to use it to request secrets.
07:23
It'll send off the attestation report. The KBS will say, yep, the measurement checks out. Okay, here's the nonce that I sent it. That's correct. Let me use this public key to create a secure channel to wrap secrets. So really relatively straightforward attack in some ways. So let's talk about solutions.
07:40
While this attack is in some ways very severe, there are actually a number of different things we can do to prevent it. Although it's not yet clear exactly which is the optimal solution. Probably the simplest solution is to revoke access to attestation reports. And what this means is that the guest would have some ability to revoke its own ability, its own capability, to get a future attestation report.
08:03
This kind of hinges on the idea that there's different phases of execution in confidential containers, or in a lot of projects, really. In the early phase of execution, we should still be executing really within the footprint of what is measured, right? Really within the footprint of the kernel, the initrd, kernel parameters, firmware, things like that.
08:20
And as long as we're sort of in this area that has been measured, we could be relatively confident about what is going on. One thing that is a big red flag here is that we are allowing arbitrary code to be executed within the guest in the context of a container. So maybe before we start doing this, we should figure out some way that we could get rid of any future attestation reports.
08:41
Make sure that none are generated. And with scvsnp, there's actually a pretty easy way to do this, which is that we can overwrite the keys that the guest needs to communicate with the PSP. They're called VMPCKs, and you can delete them. And if you do that, you will not be talking to the PSP anymore. This has some limitations. I'm not sure of a good way to do this with TDX, but if you know, let me know.
09:05
I think this is a pretty simple way to address the problem, and a pretty good solution in some ways. That said, there are things that would conflict with. There's been some discussion of something that's referred to as the passport model, where the workload actually would get an attestation report at runtime.
09:22
And obviously that would be disabled if we don't allow any attestation reports to be generated at runtime. There are maybe ways to resolve that, but that's a discussion that will need to happen. Another solution is to use the host data field. I'm going to come back to this because it's a fairly complicated proposal, so I'll talk about that on the next slide.
09:41
But let me go through the other two first. One of them is to use IMA to basically constantly check the state of the guest. This would maybe detect a breakout. Doing this would probably require a virtual TPM inside the guest, and this may be available at some point. It's not yet clear if we're going to utilize that for confidential containers.
10:01
In some ways it might be overkill, but here it might actually be useful. The final thing I wanted to mention is VMPLs. This is virtual machine privilege levels, and it's a feature of SEV S&P. This probably isn't feasible for use in confidential containers exactly, but I bring it up because this is actually the way that the VTPMs that are being developed for SEV S&P sidestep this problem.
10:21
A VMPL is a way to create different privilege levels inside of a guest, and one thing a VMPL can limit access to is generation of a particular attestation report. So if we had some sort of hardware feature like that that could isolate the ability to generate these extremely sensitive attestation reports, that could help, and the VTPMs, they have exactly this in the form of the VMPL.
10:46
So let's talk about the host data based solution. First of all, what is the host data? The host data is a field in the attestation report that is set by the host prior to the launch of the guest, and then included in every attestation report requested from inside the guest.
11:01
This is S&P terminology, but I believe MR info is a similar thing for TDX. Now, even if the guest is malicious, there's no way for it to get a report without the host data in it. We could use the host data for a number of different things, but I'm going to focus on using it to store the public key of the KBS.
11:21
This will essentially bind the hardware evidence to one KBS. Now, interestingly, this does not actually guarantee that we are connecting to the correct KBS. It doesn't guarantee the identity of the KBS, in part because the host sets the host data, and the host is untrusted. Instead, what it guarantees us is that the guest will only be able to connect to one KBS.
11:43
When the attestation agent connects to a KBS, it will check that the public key of the KBS matches what is in the host data. If it doesn't, the connection will fail. Similarly, the KBS will also enforce that the correct public key is in the host data that it has when it receives a request.
12:02
Fundamentally, the only evidence available within this guest will be evidence that has this public key of a particular KBS in it. Now, on its own, this does not really prevent the attack because we could direct all requests coming from the guest to the target KBS.
12:21
But this is where signature validation comes in. When we turn on signature validation, the attestation agent will make a request to the KBS for the signature policy before any containers can be run. This is assuming we're using an online KBS. Since we can only connect to one KBS, as soon as we make this request, as soon
12:42
as we request signatures, and as soon as that request is successful, we've collapsed the attack space dramatically. Now the only KBS that we can attack is the KBS that provides the signature verification policy, right? If we use a malicious KBS to provide a signature verification policy, for instance, one that allows us to run
13:01
the malicious container, then we still have to connect to the malicious KBS for the entire lifetime of the guest. So the only KBS that we can attack would be that malicious KBS. That's not much of an attack. On the other hand, we could try to get the target KBS, right? We could direct everything to the target KBS instead and try to get the target KBS to provide the signature verification policy and then attack it later with secret requests.
13:23
But this isn't going to work because the target KBS shouldn't give us a signature verification policy that would allow the malicious container to run. So with these two properties, first requiring that all connections go to the same KBS using the host data, putting the public key of the host data in it, and then requiring that signatures are there so that we know we've connected at least to one KBS and
13:43
we know it's the KBS that's actually running the container, we can really close off this attack fairly effectively. So there's a few things we should still note here. One of them is that we made an assumption, which is that the target KBS is going to require signatures. And this might be setting a precedent for the project in some ways, saying that
14:02
if you want to sidestep some relatively serious security issues, you need to use image signatures. Is this a reasonable thing to take on board or is it too restrictive? Is it leaving aside certain use cases that are really valuable? I think that's something we still need to think about. Another thing is that I mentioned that this is not the only possible use of the host data.
14:21
So another thing we could do with the host data is that we could put workload specific information in the host data field. For instance, we could put the measurement of the container image or something a little bit more sophisticated that achieves the same thing. In my mind, I think that this is redundant to the image signatures, which have so far been really the main way that we've done measurement of the workload in this project.
14:43
And I think one of the important takeaways from this presentation is that there's ways to resolve this attack without compromising on our ideas about decoupling measurement. And in fact, we can have a generic guest image. We can still use signatures and all this stuff, as long as we take a few precautions.
15:01
But that said, I think it would solve the problem if we put workload specific information into the host data. OK, I want to talk a little bit about SEV-ES. So far I've been talking about S&P, which isn't even implemented yet. So this is another reason why in some ways it's not actively exploitable.
15:20
With TDX, I think the current TDX implementation probably is vulnerable to this, although I'm not 100% sure about that. What we've got currently on the SEV side is SEV-ES. And here the story is sort of better and sort of worse. I'm not going to go into all the details here, but the important thing is that the connection to the KBS is actually made from the host, right?
15:43
Because with SEV and SEV-ES, we use pre-attestation, which is driven by the host. So again, connections made by the host, not inside the guest, which means it's very hard to regulate. This thing I was talking about earlier where we check that the KBS is going to match a certain field or anything like that, it's not going to work because it's all happening in the untrusted realm.
16:03
Unfortunately, we also don't really have the ability to revoke a connection inside the guest. So like the first thing, the first solution that I mentioned where we say no more attestation reports, we can't really do that with SEV in the same way using the online KBS that we currently have.
16:20
So essentially the two main solutions that have been proposed of the host data, that won't really work for SEV, and the revoking the attestation reports, that isn't really going to work for SEV either. The good news is that the VM can only connect to one KBS at a time, just the way that the online KBS is implemented.
16:40
You inject a connection to a very particular KBS. Now you could target someone else's KBS here potentially, so the attack still exists, but it can only be carried out targeting one KBS per guest if that's any consolation. So in some ways, the attack is a little bit less severe, but it's probably harder to mitigate, to be honest.
17:02
So I am still thinking about the best way to address this with SEV and SEV ES. So what can we take away from this? First, I want to mention some general things. For one thing, we realize here that the capability of generating a valid
17:20
attestation report or valid hardware evidence is precious and needs to be protected. When I say valid here, I don't mean generating an attestation report that has a signature that checks out. I mean generating an attestation report that will convince someone's KBS to give you secrets. Being able to do that, like I said, extremely precious.
17:41
So let's think, how are we protecting this? For us, a big part of this is that we realize that our protection against this wasn't a hardware protection, really. Instead, it was a software protection of the API between the workload container and the guest. It's also important to think what happens if these protections should fail, right?
18:04
Especially in our case where, again, it's not a hardware protection, it's software protection. What happens if somebody does breach that and gain the ability to generate so-called valid evidence? Is it the case that they can attack anyone, right? There are some dangers of having a very generic measurement.
18:21
We need to be careful in having a generic guest measurement and make sure that that doesn't mean that it's easy to impersonate some other person using confidential containers and convince some other person's KBS to give you their secrets. There's a more complex underlying question here about the identity of a guest. Confidential guests for a long time haven't really had much of an identity.
18:44
There haven't been many ways to give an identity to a guest. And in some ways, our project leans into that by having a generic guest measurement. But there's dangers in having no identity whatsoever. Now, still, I maintain that the decoupling of the of the workload measurement and the guest measurement is a good idea.
19:05
And that having a fixed identity or an identity that corresponds exactly to the workload may or may not be necessary. But we do have to take precautions to make sure, for instance, that evidence isn't interchangeable between KBSs. Interesting. A sort of meta note about all this is that you need to be careful about relying too much on one trust model.
19:24
I showed the diagram earlier. It's a diagram that tons of people had spent a lot of time staring at and thinking about. But we've mainly been thinking about the trust model in terms of protecting the workload container. It hadn't occurred to us that you also need to think about the protecting the ability to generate evidence from everyone,
19:41
protect it from everyone, including protecting it from the workload. Finally, a few things that are a little more specific to the project. I do want to classify, again, that this is in some ways pretty serious, but also not actively exploitable, pretty fixable. So we will work to get this fixed. We're working on the S&P implementation right now, and it will contain some of these mitigations.
20:01
It has not yet been decided exactly which. If you have any ideas for this, please let me know. This is still a relatively young project. And we get confidence in the security of our project by going through things like this and at an early stage where it's relatively harmless. It's good to discover these things as a community and discuss them.
20:21
So if you have any ideas or any questions, please let me know. And let's keep the discussion moving forward on this topic.