Hello, everyone. My name is Moritz Eckert, and in this talk, I want to highlight a challenge of confidential computing that goes more into this DevOps direction. I want to talk about how we can deploy, orchestrate, and manage confidential

applications in a scalable, more cloud-native way using a tool called Maboron. Before we go into the details, I want to first set the scene, give you an understanding of the use case, of the problem set we're dealing with, and then step by step explore the solution that Maboron offers.

And in the end, we might even have time for a short demo. So, we were doing a project together with Bosch, where we were dealing with cars, vehicles, providing video streams that contained personal identifiable information, PII.

That means license plates, faces, other things that we wanted to feed into a model training, an AI model training pipeline. And we wanted to train that model on the raw, real data. That means no blurring, no cutouts.

And at the same time, we wanted to make use of an external labeling service that we would provide this material and they would label it for us so we can train our model on it. Of course, this should all be done in a privacy-preserving way, not sharing any of this PII.

And therefore, we developed a pipeline that essentially splitted the image based on what is PII and what is not, and then would send the, let's call it safe, non-PII footage to the labeling companies,

get back the labels, put together the image with the PII information again, and then feed it to our model training. And for that, we wanted to use confidential computing. That means from the car fleet to the actual trained model, data would always be encrypted also at runtime, and we can have this whole pipeline verifiable.

And this pipeline should be scalable, should hold up against the challenges of a large car fleet sending a lot of footage, a lot of video data. You can find more about this use case of this project. We presented

that with Bosch at Microsoft Build, Intel Vision. You will find those talks online. We, from the actual site, we built open source projects to realize such types of confidential computing applications. We have a bunch of tools targeting more of these new types of applications, and

then we have a tool called Constellation that's more for the lift and shift side. So for this specific project, we choose the Marveron framework, which we call the control plane for scalable confidential apps. But now that we know the problem statement, let's see in more detail what are the challenges when you want

to deploy such scalable AI pipeline or any other type of microservice architecture in Kubernetes or any other cloud orchestrator. So on the right hand side, we have our cluster that's already confidential computing capable. In that case, it were SGX equipped machines.

We already packaged our application into secure enclaves. In this case, we use the Grameen framework for Intel SGX, which will be presented in the next talk after that one. So this is already done. Now on the left hand side, or top left, we have our DevOps team.

And the DevOps team now wants to deploy all those different microservices that together build up the pipeline. And they want to ensure that all of them have integrity, they have the right identity, they want to do remote attestation, essentially. But how can they do that without doing that for each and every

service individually that, you know, potentially can scale up and down dynamically, right? This is going to be a huge pain if you would do that. And then they want to provide configuration, they want to provide parameters. Similarly, how you would do in a regular Kubernetes deployment, but you

can't trust the Kubernetes environment, you can't trust the control plane or anything. So how can they manage, orchestrate such applications? And then once deployed, how can the individual services, how can they securely communicate with each other, knowing that the other end of the connection is indeed a valid secure enclave, it has the right identity, it's also part of the pipeline.

And how can we do that decoupled from the application layer? So we don't have to do that over and over again, where we build a new type of application and need to do remote attestation on the application logic. No, we want to do that in a service mesh fashion where we decouple this functionality and solve it on a different layer.

And lastly, how can an external party like the car fleet verify an entire deployment that's just not one big monolith but consists of several services? How can they verify from the outside in one single simple step without knowing the inner details of our architecture?

So these are the challenges we are facing when we do real world deployments using confidential computing. So now that we know the challenge, let's see how Maveron approaches this problem. So with Maveron, we deploy a control plane component, we call it the coordinator.

So we deploy the coordinator that itself runs inside a secure enclave. That means our DevOps team can first verify that coordinator. And then they can provide or they provide the coordinator with a configuration, a deployment configuration we call the manifest.

And that is similar to any other deployment configuration for Kubernetes. But we deploy it to a trusted controller that we verified beforehand. So I want to show a quick example of how such manifest could look like.

You can see here it's JSON format. The first thing we define here are packages. Packages are essentially the enclaves. This right here is specific to Intel SGX. So we have the MR signer, here signer ID and so forth.

In the future, this will also support other types of CC hardware, for example AMD SCV and so forth. But in this case, we have two SGX enclaves, backend and frontend. And they are identified with their unique signatures.

And once we have these packages defined, think of them like you defined a container, right? A containerized enclave. We define the next section, which is our marbles. And marbles now consume such package. So think like a Kubernetes pod consuming a container. In this case, it consumes the backend package.

And then we can define several parameters like files that should be available to that marble or the environment variables that you want to have defined for that marble and the arguments. And this is similar to any other Kubernetes deployment. But now because it's the manifest, because you can verify it,

the coordinator can enforce this configuration for your enclaves and you can trust these configurations. And that's the important point. What we also have here is called roles. So this model is associated with two roles, third reader and key reader.

And Marberon implements a type of role-based access control. So if I scroll down in this manifest, I will find a section that is called roles. And here, similar to any other role-based system, every role is associated with a resource and specific instances of these resource.

And it defines what action that role can perform on the resource. And these roles, we can then use to attach them to marbles. So marble can do certain operations on resources. And we also have users and users are authenticated using PKI,

and then they can do whatever their role allows them to do. One of the things, for example, would be to define a secret. And the secret could be, for example, a user-defined secret, for example, an API key. And then we have a user, we have a role that allows them to update the API key.

And this would, for example, allow you to have a multi-tenant scenario where you have the DevOps team that deploys this application and you have another team that is managing another service or access to something and that provides a key or an API key into that application.

And using the role-based access control, they can deploy that or set that specific secret after the manifest has been put in place. So this key is then uploaded, is managed by the coordinator, which is trusted

and is from there distributed to your services, to your marbles. And that means the DevOps team can never access a secret. So you have a split of trust, right? You have the owners of that secret that will always stay in control and you have the DevOps team and they can engage in a multi-tenant scenario.

So we now have seen the manifest, we have seen the packages, the marbles, the role-based access control and resources such as secrets. So we go back to our example, after setting the manifest,

the coordinator will then take care of providing those credentials, those secrets or configurations to your marbles, to your services. So once you deploy them using regular Kubernetes means of deploying the application, they will come up, they will be authenticated by the marbling coordinator and then receive their parameters, their credentials.

Part of these credentials are TLS certificates. So every service has a unique leaf TLS certificate, which goes up to a root certificate that is established by Marberon as a PKI infrastructure.

So for every deployment you have one root certificate. And those can then be used to establish secure connections between the services. For certain runtime environments such as Ego, this can be done automatically in a neutral TLS fashion if you're familiar with service mesh kind of things,

you might be familiar with that term, but essentially it means you will wrap every TCP connection between your services automatically into TLS so they are secured. For other types of runtime environments or if they are specific neat, you can also consume those credentials using the Marberon manifest.

You can place them into files, you can place them into the environment and then your application can take them from there and use them to establish secure connections. That's very similar to how you would operate with CERT Manager or other certificate management systems for Kubernetes. The difference being that those certificates are only available inside that secure confidential environment,

inside your secure confidential deployment, that means the coordinator and then your services. So now what we have is we have our running deployment, everything is connected, they can communicate with each other. So in the last step now an outside party can connect to the coordinator again,

can obtain its attestation statement that contains the Marberon manifest so they can verify that indeed a valid Marberon coordinator, they can verify that indeed the deployment, the manifest, they expect it to be in place there. And they will obtain the root certificate from this deployment.

And after verifying that they can use the root certificate to authenticate to their deployment, in this case the front end of the AI pipeline that consumes the video stream from the cars. And transitively because they verify the coordinator, they verify the manifest, they verified the entire deployment

in one concise simple step. And that's it. Now we have all of these problems I've showed you earlier, we have them solved, we can manage and orchestrate that deployment from the DevOps team side, we can communicate with that deployment from the car fleet and potentially any other third party or legislator

or regulator can verify this deployment as well using the same type of mechanism. So in summary what does Marberon do? It adds an orchestratability, a configurability, a manageability to your confidential deployments, make them scalable, make them updatable,

make them manageable. It can run standalone but it runs best on Kubernetes and the usual cloud environments. It supports several runtimes, so far it's SGX specific, so for example Ego, Graphene, Occlum. In the future we plan to also support

other types of confidential computing runtimes like AMD, SAP, Intel TDX, most likely based on the Confidential Containers project. Okay, now we have a little bit of time left to look at a demo. Let me switch to my console.

I have a Kubernetes cluster running here on Microsoft Azure with two nodes that are both equipped with SGX capabilities. And we will now install Marberon, set the manifest and deploy a simple demo application. So we use the Marberon CLI tool

that you can use to interact with the Marberon coordinator and perform your usual DevOps tasks. So Marberon install will make sure to install the Microsoft coordinator. That just should take a second for everything to be set up.

And then we can port forward the client API to localhost so we can interact with the coordinator and this should be, if you're in a production environment you can of course also use an external load balancer or any other way of exporting that service

to any kind of environment you want to. Just need to wait a second for the coordinator to be up and running and then I can use this client API from here. And the second step would be then to verify

the coordinator is indeed a valid Marberon coordinator with the correct enclave. We can use any type of command that will automatically do this verification and in this case we want to obtain the root certificate of this Marberon deployment here.

This would also be what an external party like a car fleet can do to obtain a root certificate and then connect to the application. Of course you can always do that ahead of time and then just distribute that certificate as the root of trust. But once we trust the coordinator which is just a plain constellation coordinator

we can do Marberon check providing the Marberon status. We can use Marberon status to see what status the Marberon coordinator is in.

In this case it's waiting for a manifest to be set. So now we can set the manifest. I have one prepared here. It's a very simple application just having three packages, microservice, demo application, one frontend, two backend services. So three packages, also three marbles

for these packages and they build up such demo emoji voting service. We have already seen the manifest. I'm not going to go into the detail again but of course also this is in GitHub so you can have a look at it if you want to. And then we can just set this manifest using Marberon manifest set

and it will upload the manifest and now the Marberon coordinator should be in the state of being ready to accept and authenticate marbles. So now we can just go ahead and deploy our application and this is just regular Kubernetes application deployments. So in this case we're using Helm

for this emoji voter demo application. It will install those three services and make sure that they are continuously running. And we can now check Marberon and see those authentication requests where those marbles

for example the web frontend or one of those two backend services will contact the coordinator, will authenticate itself and be provided with the configurational parameters and their credentials. So we see the web frontend was first activated as a new marble. And this will continuously be running.

So now if I go ahead and get all the parts in the emoji voter namespace I will see that all of these services are indeed running and if I would now scale up and down one of them they will be automatically authenticated again by Marberon and be added to this deployment

spanning a confidential overlay, a confidential deployment. Yeah, and that's it from the demo. I think we've seen how you can interact with the Marberon CLI, we can install it, how easy it is to deploy a confidential application and we can now keep using Marberon to orchestrate

and update secrets and update things based on what we defined in the manifest. Yeah, let's go back to the slides. That's it. I think we've seen in summary Marberon makes it quite easy to deploy, orchestrate,

just manage applications during the lifecycle, manage confidential applications during the lifecycle and it augments the usual DevOps stack of cloud native deployments using Kubernetes, using your regular service match and so forth. So if you want to try it out or want more details

please see our GitHub page. It also links to a documentation. You can very quickly create your first confidential deployment. We also have a simulation mode so if you don't have access to any type of confidential computing hardware you can just use the simulation mode to run it locally using Minikube or whatever the tooling you have in place.

And then if you have further questions please get in touch. You can find me on LinkedIn. You can join our Discord. I will also be there tomorrow on day two of the Confidential Computing Dev Room. Please hit me up if you have any questions

or just want to have a chat. And yeah, before we go into the Q&A one last cheeky little self advertisement. We have the open Confidential Computing Conference coming up in March. I think you're the right audience for that. There are a lot of confidential computing open source projects

that are going to be presented in the next couple of weeks. and other insights into confidential computing in general. So yeah, it's free. It's online. Please register if you're interested. It will be very, very cool to see you all there. That's it. Thank you very much. And I think we have a bit of time left for questions.