Firecracker, should it work only with a single runtime?
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 561 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/44268 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Run time (program lifecycle phase)SoftwareEmailProduct (business)Process (computing)Data managementSpacetimeSinguläres IntegralVirtual machineVisualization (computer graphics)GoogolOpen setNetwork socketSource codeData typeDynamic random-access memoryContent (media)DisintegrationComponent-based software engineeringEuclidean vectorInterface (computing)ImplementationSimilarity (geometry)Multiplication signINTEGRALElectric generatorMenu (computing)Content (media)SpacetimeArithmetic meanCodecDirectory serviceOSI modelDiscrete element methodPhysical systemTouchscreenSoftware testingField (computer science)Substitute goodInterface (computing)Web 2.0Singuläres IntegralAdvanced Boolean Expression LanguageOpen setFilm editingWordObservational studyUsabilityOpen sourceInstance (computer science)Position operatorCASE <Informatik>CubeApproximationBinary fileRemote procedure callResonanceError messageRevision controlDuality (mathematics)Operator (mathematics)Formal languageSound effectVirtual machineSocial classSpherical capDesign by contractFitness functionPlanningRun time (program lifecycle phase)Data managementType theoryGroup actionCoefficient of determinationMetropolitan area networkMobile appDefault (computer science)Bit rateRoot2 (number)MultiplicationSlide ruleDemo (music)Inverse elementNetwork socketSoftwareMachine codeUniform resource locatorMedical imagingDemon1 (number)Information securityGoodness of fitChemical equationDistribution (mathematics)Projective planeSystem callWorkloadRepresentational state transferSource codeEmulatorComputer hardwareLebesgue integrationRepository (publishing)Different (Kate Ryan album)Computer animationLecture/Conference
08:02
Run time (program lifecycle phase)Interface (computing)Similarity (geometry)Software development kitWrapper (data mining)DemonRootComputer-generated imagerySinguläres IntegralProcess (computing)Web 2.0Metropolitan area networkCasting (performing arts)Single sign-onMultiplication signOcean currentReal-time operating system2 (number)Software testingEvent horizonInterface (computing)Lattice (order)Content (media)Ring (mathematics)SubsetTotal S.A.CodecSummierbarkeitGame controllerSeries (mathematics)UsabilitySource codePatch (Unix)Uniform resource locatorRun time (program lifecycle phase)Lie groupNumberCASE <Informatik>Game theoryLine (geometry)Open setSpring (hydrology)Cellular automatonFormal languageMereologyPhysical systemFirst-person shooterMobile appServer (computing)Singuläres IntegralFerry CorstenRootBranch (computer science)Level (video gaming)Moving averageLink (knot theory)Internet service providerWritingChief information officerInformationSpherical capNetwork socketFilm editingType theoryRight angleTraffic reportingSoftware developerFront and back endsLatent heatComputer fileNormal (geometry)ImplementationMedical imagingGoodness of fitFunctional (mathematics)File systemDemonTable (information)Kernel (computing)Process (computing)Cartesian coordinate systemView (database)Configuration spaceDiagramSoftware development kitArithmetic progressionSoftware repositoryMultiplicationProof theoryStandard deviationSimilarity (geometry)Installation artComputer animation
15:59
Network socketRootStreaming mediaStatisticsContent (media)Data typeSinguläres IntegralComputer-generated imageryLocal ringBootingQuantumGroup actionOpen setServer (computing)SSHSerial portLoginComputer programPhysical systemSoftwareTerm (mathematics)Distribution (mathematics)Computer fileComputer multitaskingDirectory serviceVideo game consolePrincipal ideal domainSequenceRevision controlAsynchronous Transfer ModeAdvanced Encryption StandardMathematical optimizationPasswordRandom numberLoop (music)Read-only memoryError messageInstance (computer science)Metric systemVirtual machineField (computer science)Game controllerMoistureQuicksortFormal languageCausalityEstimationPerturbation theoryDepth-first searchPiEscape characterSpeciesLatent heatNetwork socketContent delivery networkWebsiteCuboidSinguläres IntegralLevel (video gaming)Demo (music)Fraction (mathematics)Metropolitan area networkGroup actionPhysical lawMultiplication signBranch (computer science)Ring (mathematics)Pairwise comparisonSet (mathematics)Semiconductor memoryMathematicsDependent and independent variablesRootBlock (periodic table)2 (number)Kernel (computing)Medical imagingMereologyComputer clusterInterface (computing)DemonType theoryUniform resource locatorResultantBinary fileParameter (computer programming)Normal (geometry)Computer fileTraffic reportingINTEGRALBinary codeProcess (computing)Configuration spaceRepository (publishing)Source codeComputer animation
22:09
Run time (program lifecycle phase)Local ringTerm (mathematics)MetadataComputer-generated imageryRootCase moddingState of matterSpacetimeClient (computing)Group actionData typeExecution unitNetwork socketLengthCodierung <Programmierung>Directory serviceProcess (computing)Computer networkTelecommunicationLiquid2 (number)Menu (computing)Mobile appWebsiteExpert systemRainforestSource codeState of matterDemo (music)Content (media)WritingCore dumpGroup actionCovering spaceData storage deviceGame controllerCoefficient of determinationGame theoryInterior (topology)Bus (computing)Computer fileInstance (computer science)Server (computing)Execution unitDemonMereologyVariable (mathematics)Integrated development environmentConfiguration spaceMessage passingNormal (geometry)Flow separationMedical imagingDependent and independent variablesSource codeComputer animation
28:07
Network topologySynchronizationRun time (program lifecycle phase)Computer wormCAN bus5 (number)Metropolitan area networkSoftware testing2 (number)HypermediaField (computer science)Key (cryptography)Multiplication signCASE <Informatik>Branch (computer science)Process (computing)FeedbackFrequencyWorkloadVideo gameSoftware maintenanceShape (magazine)Similarity (geometry)Computer animation
30:17
Computer animation
Transcript: English(auto-generated)
00:05
Hi everyone. Today I'm going to talk about FireCure and its integration with Container One-Time Managers. My name is Dongsoo Park. I'm working at Kentwork, which is based in Berlin, Germany.
00:27
So, so far, I have been working on Container One-Times mainly, and I work also on black-colonodes and cubespawn. So all are open-source projects. Last year, my colleague, Albin, and I have
00:43
been working on especially the OCI runtime tools and distribution specs and so on. As well, I've been interested in Container One-Times. So if you are interested in an interesting open-source project at Kentwork, then feel free to contact us.
01:08
So, I'm going to talk about FireCure today. This is a very recent project, which was developed a very time ago, but open-sourced in December last year.
01:26
So, simply speaking, it's a lightweight virtualization machine monitor, so we call it VMM. You can spawn multiple micro VMs in an efficient way, without relying on the existing QMOK VMs or hardware emulation layers in between.
01:52
And it's dedicated to short-lived workloads. So you can say that micro VMs triggered
02:03
by FireCure would last for a couple of seconds or minutes, so not like 24 hours. And yeah, this is a good balance between traditional virtual machines and containers.
02:23
Because so many people believe that the virtual machines are better than containers because of the isolation and security issues. FireCure achieves security and isolation issues by implementing jailer in itself, so you can
02:44
specify a whitelist of system calls using seccomp, which is provided by Linux kernel itself. That's pretty cool. And so obviously, FireCure heavily makes the use of Linux KVM interface, so its interface is just exposed via native SDK.
03:08
And this is implemented all in Rust. So I think the recent discussion about this is that the Amazon and Intel have been working on a shared interface, which is called Rust VMM.
03:27
So yeah, yesterday there was a talk about that. Maybe I'm not expert about the Rust VMM, so you can ask one of the Rust VMM members. This is a simple example of how you can run FireCure and how to launch the micro VMs.
03:54
So first you run FireCure daemon without the API socket option, so you can just skip this option, omit this, and run it.
04:04
Then the temp firecure.socket will be the default Unix socket that FireCure listens on. And after that, you can specify two very important requirements. The first one is about specifying kernel image, and the second one is about the root interface image.
04:30
So you have to prepare the two different images on your local machine. And you have to specify, for the first one, the root source URL.
04:42
For the second one, the drives slash root interface URL. The other ones are very similar to each other. And yeah, that's it. So this is a basic preparation for running FireCure.
05:01
After that, after all preparations were done, you can actually start the micro VM, which is also done by this RESTful API through the Unix socket. This is instance start action, so you have to specify actions in the URL,
05:25
and you do specify action type to instance start. Then the micro VM will boot. I'll show this in the later demo. So this is not actually human-friendly at all, and maybe you would think about an easier way.
05:50
So many people have been already working on it, the integration with container managers. The first one is obviously containerd, which is pretty widely used and widely known.
06:06
Under FireCure VM, the micro VM organization, there is a Git repository, FireCure containerd repo. This repository is about implementing three different components, agent, snapshotter, and runtime.
06:23
And after that, you need to install the containers d-specification into your local directory. And the containerd will enable the containerd to be integrated with FireCure. And this depends on the TTRPC interface of the containerd.
06:44
So, yeah, if you are interested in, you can look into this project. And very recently, there is another attempt to integrate kata container with FireCure.
07:04
And you can visit this project. This is already merged, and you can use it already. And yeah, very clean implementation, and yeah, I can recommend it.
07:20
And on the other hand, this heavily relies on existing kata containers interface, like hypervisors and so on. So it's not like a lightweight approach, I would say. So I was thinking about a very simple way, how to integrate in a simple way.
07:47
So cryo would be the best approach for me. That's what I thought. So what's the cryo? Cryo achieves both container runtime and standard, which are OCI and CRI.
08:07
OCI is, simply speaking, this is a spec for open container initiatives. And its reference implementation is RunC. Maybe if you are running Docker in your local machine, for example, then you are already running RunC
08:25
because 99% of usage is running RunC for the OCI runtime. So cryo already implements this. And on the other hand, Kubernetes itself, Kubernetes 1.8 or newer, already has container runtime interface, which is CRI.
08:47
This is done for supporting both container runtime, Docker and Rocket. And this is a typical standard for container runtime that wants to be integrated with Kubernetes.
09:07
And cryo itself achieves CRI and at the same time is relatively simple compared to the container D or any other solutions.
09:20
So that's what I chose to implement. And cryo itself does not have any command line tools, so maybe the best way to install cryo control, another command line tool.
09:42
Cryo control is a part of cryo tools repo, so you can visit this repo. This provides a very simple and similar command line interface shown to users as RunC or Docker.
10:02
So my goal is to make a runtime add-on in cryo for Firecracker. Currently, cryo has a single runtime, which is v1 and should be renamed to OCI in the future.
10:24
And the second approach, which is written in this pull request, which is for supporting VM-based container runtime. This is still in progress and not merged yet.
10:41
But it improves internal interface of cryo. So what I want to do is to take a similar approach as this VM-based container runtime inside cryo. And at the same time, I'm not doing anything like connecting to the VM-based container runtime.
11:04
I'm just running Firecracker under the theme. That's what I want to do. For doing that, I'm relying on Firecracker's Go SDK.
11:21
This is also a very good and thin layer between the Firecracker itself and container runtime or any other third-party applications. And Firecracker is written in Rust. I like Rust, but honestly, I'm not good at programming Rust.
11:47
So Go SDK is a very good approach and solution for me. It allows me to write a third-party application using Go language.
12:01
And then it's easy to be integrated with existing container runtime because most container runtime are written in Go. And it provides a low-level table and functionalities in an easy way. So you can take a look at this report.
12:24
So I can show you a simple diagram. On the left side, cryo daemon should be running with my own Firecracker runtime add-on.
12:41
And on the right side, Firecracker daemon should be running in the background. So each daemon listens on its own Unix socket. So what I do is actually running cryo control command line tool. From my command line, then it should connect to the cryo's Unix socket and send specific commands to that Unix socket.
13:09
Then cryo interprets this command based on Firecracker Go SDK, then sends the commands to Firecracker backend.
13:21
And Firecracker backend also listens on its own Unix socket. And whenever it receives the command, it spawns a new micro view. So if you do this process multiple times or 10 times or 100 times, then the Firecracker daemon would spawn micro views 10 times or 100 times.
13:46
So arbitrary times, you can do it if you want. Current proof of concept is available as open source.
14:00
Branch is kimp-work slash cryo slash my branch name. And this basically reads config file for setting up kernel and root interface for Firecracker. And this can be also done manually, but it should be human friendly and configurable.
14:20
So I had to add it. Also, whenever starting a container, for example, the Cryo runtime would also tell Firecracker to spawn a new micro view process.
14:42
This branch itself is partly working, but it's still in heavy development. So in the future, maybe it might be changing. So in the future, I'm going to clean up something and make a pull request to the upstream report.
15:05
On the other hand, there is a simple tool for creating kernel and root interface image. So as I've shown a little before, there is an example for setting up kernel image and root interface.
15:22
So you should really have two different images, not a single image. So I had to create these two different images using a simple tool based on the simple talk of how to create vmlings.bin and root interface.x4. This is no special thing.
15:43
This is just a stock kernel. No special patches. This is a normal root interface file image based on the x4 file system. I'm going to show demo.
16:30
First of all, I'm going to show just a plain simple Firecracker demo.
16:40
In that process, cryo is not involved at all. After that, the second step is integrating with cryo and configuring and running cryo control commands and so on. And the first demo is just running a Firecracker. So this is my local repository including just an upstream Firecracker report.
17:08
So this is a binary file that I want to run.
17:22
And these are two different images. One is kernel and the other one is root interface.
17:49
So first of all, I want to clean up the existing Firecracker file. And after that, I want to run.
18:06
S is running. Now check if it's really running. S is running. And also Unix socket is active.
18:24
The first preparation is specifying kernel image. So it will send a specific command, kernel root image path and root.
18:48
So these things can be specified in an arbitrary way. So you can just configure as you want. Also, you need to specify this root source URL with this put method.
19:05
As two or four results is good. So it has worked. You cannot see it right now.
19:20
Now the second preparation is about root interface. Can you see? Yes. This looks similar, but you have to specify a different URL here with the drive slash root interface. And you have to specify your path to existing root interface image.
19:48
Yes, it has worked. Still, I'm seeing nothing. This is normal. So until I really start and send an instant start action, you don't see anything.
20:06
Until the preparation, this is just a normal process. The final step is actually sending instant start.
20:21
It looks similar, but the URL has actions and action type is instant start. Yes, this is normal. And you can see now the microvn has already booted.
20:41
This is just a normal data unstable image. Connor is also that one from Debian Unstable. It's been booted. You can see anything that you want.
21:00
And I specified the total memory to 128 bytes. So you can see the memory usage like that. And if I just type reboot, then the microvn would be gone.
21:20
And you will not see anything. Yeah, it's gone. Yeah, that's all. So it's a very simple example, and actually you are able to specify many other parameters. For example, a second block device or another virtual net interface.
21:45
I will skip this demo here. But maybe you will get the idea. Now... Thanks. Second part of demo is about the integration with Cryo.
22:01
So this is the Cryo repository and my custom branch. And there is a binary file, cryo daemon. I'm going to run it with some configuration file which is predefined like that or like that.
22:31
So for example, if I look into this configuration file, then you will see the normal configuration. So it will pull an image from k.io and run some commands using the environment variables.
22:51
And this is just a normal cryo config. And I'm going to run cryo daemon.
23:02
Also on the other terminal, I'm going to send several cryo control command. First of all, it's running.
23:31
You can see daemon is running. And there are no sandbox part available.
23:49
Also no container is available. The first command is creating and running sandbox part. So before running actual container, you need to specify a sandbox config.
24:05
This is the first step. Now the part is ready. Now the cryo daemon shows several messages.
24:24
And this is mostly about sending instant start messages and so on. So it's communication between cryo and cryo daemon. You can see cryo daemon is running.
24:45
Also cryo daemon will be running. Now the second step is creating a container using that sandbox part.
25:02
And what I did was actually using this existing part ID specified two different configs. So one is container config. And the second one is sandbox config.
25:21
So far, I have just one container running, which is in the created state. So not actually running now. But you can see also that this action was done. And with that created container, I'm going to start the container.
25:44
Actually, it started. I'm going to run just PS. And you can see that it's now in running state. Now cryo daemon is also saying that it has done something about this.
26:03
And it got a tool for status response. And I'm going to stop the container first.
26:24
And I'm going to delete the container. That has succeeded. Now we can see it.
26:45
Yeah, the container is now gone. Yes, no container at all. But you can see still the part is running. And the cryo daemon is only running just once because the second instance was already removed.
27:08
Now I'm going to remove the part itself again. Stop it. So it's stopped and removed.
27:26
The part is gone. Yeah, this is a known issue. So I'm going to figure out in the next few days. Anyway, this should actually kill the existing cryo daemon.
27:43
I'm going to just remove this. Yes, it's terminated. Yeah, that's the end of my demo. Thanks for listening to the talk.
28:02
And I'm going to quickly go over the future work. So I'm going to publish this with branch as an actual pull request and get some feedback from the upstream maintainers. There are also missing features like attach or exit.
28:22
So I'm going to follow up with that. And also that's a good idea to do the similar work for, for example, Rocketlet. This is to be discussed. That's it.
28:42
So we can do one question and you were the first one. I have a question specifically to Firecracker itself. I don't understand what makes it limited to micro VM and short life.
29:02
Why cannot be a general hypervisor? I think in general, Firecracker is designed to be living just for a short period. And for example, serverless workloads.
29:20
That was the purpose of Firecracker. And that's what it does the job in the best shape. So sure, you can also adapt and use Firecracker for other use cases. And I think it's not impossible.
29:40
But on the other hand, Firecracker upstream maintainers would think about this. Firecracker wants to keep it very simple and minimal. So if you want to use it as a container, for example, or another feature, then it might be a little difficult because Firecracker maintainers would not think about that.
30:11
Does that answer your question? Yeah, thank you.