CRI-O: All the Runtime Kubernetes need
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 50 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/43093 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
All Systems Go! 201845 / 50
1
4
11
12
13
14
16
17
19
23
24
25
29
30
32
34
35
39
40
41
43
44
50
00:00
SpacetimeRun time (program lifecycle phase)SoftwareSoftware engineeringProjective planeLibrary (computing)MereologySoftwareRun time (program lifecycle phase)Atomic numberComputer animation
01:11
System programmingSoftware maintenanceQueue (abstract data type)Software maintenanceInteractive televisionMultiplication signExtension (kinesiology)Run time (program lifecycle phase)CubeInterface (computing)QuicksortCodeQubit
02:59
Interface (computing)Run time (program lifecycle phase)Client (computing)Server (computing)Communications protocolBuffer solutionRun time (program lifecycle phase)Interface (computing)Client (computing)Alpha (investment)Computer architectureDefault (computer science)Server (computing)Cube
03:45
Run time (program lifecycle phase)Service (economics)Run time (program lifecycle phase)LoginCycle (graph theory)Interactive televisionService (economics)Video gameGroup actionSource codeJSON
04:28
Computer-generated imageryService (economics)InformationSystem programmingGroup actionOpen setOpen sourceStability theoryComputer architectureRange (statistics)Gamma functionConfiguration spaceComputer filePressureRule of inferenceLibrary (computing)Software bugData storage deviceBuildingInstance (computer science)Transport Layer SecurityArithmetic meanDefault (computer science)Run time (program lifecycle phase)Projective planeCASE <Informatik>Series (mathematics)DemonPoint (geometry)Latent heatMereologyFile systemGroup actionService (economics)Multiplication signLevel (video gaming)Queue (abstract data type)CubeNamespaceOpen sourceSpeicherbereinigungServer (computing)Moment (mathematics)Metropolitan area networkSource codeSelf-organizationSpacetimeClient (computing)Cartesian coordinate systemRevision controlFitness functionInformation securityView (database)Open setPatch (Unix)Roundness (object)Physical systemOverlay-NetzInformationStability theoryStandard deviationControl flowLoginComputer architectureMiniDiscCodeSoftware maintenanceData miningComputer animation
12:06
System programmingData storage deviceDefault (computer science)Computer clusterDevice driverComputer-generated imageryRun time (program lifecycle phase)Configuration spaceLibrary (computing)Stack (abstract data type)Interface (computing)Computer networkFormal languageComputer programmingClient (computing)LoginOrder of magnitudeNamespaceArchitectureEvent horizonSocial classMessage passingTape driveToken ringCubeRun time (program lifecycle phase)Group actionDefault (computer science)Device driverMoment (mathematics)Control flowProcess (computing)Data storage deviceLibrary (computing)Software developerOverlay-NetzRight angleSpacetimeCartesian coordinate systemClient (computing)Multiplication signVirtual machineSemiconductor memoryLaptopInstance (computer science)Traffic reportingSign (mathematics)Direct numerical simulationRoutingInterface (computing)MereologyPhysical systemElectronic signatureRadical (chemistry)Formal verificationNamespaceConfiguration spaceComputer architectureAnalytic continuationSoftwareWordPlug-in (computing)WaveOpen setInheritance (object-oriented programming)Condition numberZoom lensRepository (publishing)Projective planeCASE <Informatik>CuboidWeightWellenwiderstand <Strömungsmechanik>Core dumpRootSoftware maintenanceWritingShift operatorCycle (graph theory)Video gameTouch typingXML
19:44
Social classEvent horizonTape driveBroadcast programmingNormal (geometry)AliasingComputer animationSource codeJSON
20:34
Message passingComputer-generated imagerySocial classDefault (computer science)Tape driveScheduling (computing)Event horizonToken ringDemo (music)AliasingBlogNormal (geometry)Physical systemData typeVirtual machineUniform resource locatorServer (computing)Document Type DefinitionOperations researchLocal ringPrincipal ideal domainDemonRootCodeExecutive information systemData storage deviceOverlay-NetzLine (geometry)Limit (category theory)Computer multitaskingTask (computing)BefehlsprozessorRead-only memoryLocal GroupProxy serverNamespaceCubeRun time (program lifecycle phase)IP addressMereologyWeb pageFunction (mathematics)LoginPhysical systemCoprocessorProcess (computing)Computer animation
23:07
Structural loadVertex (graph theory)Set (mathematics)Data storage deviceDemonPhysical systemNamespaceService (economics)Proxy serverRead-only memoryComputer multitaskingTask (computing)Limit (category theory)Metric systemVacuumRevision controlRun time (program lifecycle phase)Operations researchOnline helpMilitary operationData typeQuantileStatisticsDemo (music)Form (programming)Lemma (mathematics)Menu (computing)Metric systemComputer animation
23:48
Mixed realityIntelInterface (computing)Multiplication signGroup actionGene clusterSoftware maintenanceGoodness of fitWorkloadShift operatorCubeMixed realitySoftware testingRun time (program lifecycle phase)MereologyGraph coloringSocial classRadical (chemistry)Open setReading (process)Virtual machineFilm editingComputer animation
25:04
System programmingSoftware testingDisintegrationSoftware testingConformal mapMultiplication signCubeLinear regressionMereologySubsetINTEGRALOpen setShift operatorComputer animationMeeting/Interview
25:50
Local ringSystem programmingCore dumpComputer-generated imageryPerfect groupDemonDisintegrationCommon Language InfrastructureDefault (computer science)Run time (program lifecycle phase)Revision controlVideo trackingWebsiteBlogInterface (computing)Variable (mathematics)Binary codeCASE <Informatik>LaptopCubeTrailLibrary (computing)Data storage deviceDefault (computer science)Self-organizationRevision controlMultiplication signDemonRoutingRun time (program lifecycle phase)Row (database)BuildingDebuggerSynchronizationWindows RegistryPlug-in (computing)Sinc functionComputer fileOpen setDigital electronicsMobile appProjective planeContent (media)Latent heatFreewareShift operatorLink (knot theory)Physical systemLine (geometry)Integrated development environmentRootQuicksortGastropod shellRepository (publishing)Slide ruleComputer animation
31:00
System programming
Transcript: English(auto-generated)
00:06
Welcome everyone. My name is Antonio. I'm a senior software engineer at Red Hat where I work for the containers team where I also maintain the cryo project probably you've heard about it and I'm also maintaining other container
00:23
related tools within Red Hat. Today we're going to talk about cryo which is a container runtime for Kubernetes. We'll go through why Kubernetes introduced a new API called the CRI. What were the issues it was facing, how it
00:45
solved it with the CRI and how we've built cryo from little tools and libraries. We've been building over the past few years I guess. So I'll briefly talk about some of the container tools we've built as part of
01:06
the whole cryo project especially the project atomic project but this will be at the end. So roughly four years ago the only supported runtime in kube was just Docker and that means that at the time the interaction between the
01:24
kubelet and the container runtime was in was built into the kubelet itself like in the code base of the kubelet. There was a sort of maintenance burden in doing that because every new feature being added to kube required the maintainers to actually change also the interaction
01:43
between the kubelet and Docker itself. Then rocket came and again the interaction was wired into the kubelet itself. So the Kubernetes maintainer thought that if another runtime would come then that requires another built-in
02:04
interaction in the kubelet. That wasn't really scalable and on top of that Docker the runtime was actually breaking kube at every release and there was also another issue which was extensibility. So kube was was was
02:27
extensible and it's meant to be extensible but there was no way to actually plug a new runtime at will and so it was pretty much restricted to Docker and rocket. So the community gathered together and they came up with
02:45
this idea of removing the interaction between the kubelet and a give a run time outside of kube itself and that was the time when the CRI or the container runtime interface has been created. The CRI interface is a
03:03
plug-and-play API and it means that as long as your container runtime implements the server side of the CRI everything is going to work just fine kubelet is going to be able to create pods containers and do all the work needs to do. It's a gRPC API and it's available from kube 1.5 as alpha
03:26
features nowadays the default and even the docker runtime is running through the CRI itself. It's a client server architecture and again the container runtime implements the server side of this API and the kubelet implement the
03:43
client side. The CRI itself consists of two main pieces one is the runtime service which is the one in charge of the pods and the container life cycles which includes actions like running a pod stopping it create a container in a given
04:03
pod getting the least of all running containers in kubernetes as well as being in charge of the interaction between the containers for things like grabbing the logs for a container exacting into a running container
04:21
attaching and attaching for the runtime service is the first piece and it's in charge of all of this and then there is the image service which you can imagine it's in charge of the image lifecycle and again it includes actions like pulling an image getting the status of an image to gather
04:43
information like the image file size and it's also responsible for reporting file system information to the kubelet for stuff like the image garbage collection like the kubelet can request the runtime how much space
05:01
there is left so that you can actually evict some pods if there is disk pressure this is a nice overview of the CRI in action you can see on the left side we have the kubelet which is implementing just the gRPC client and it talks to in this case there is a CRI shim this is the default when
05:24
running with the docker runtime the CRI shim is a daemon implementing the gRPC server and then on the right side we do have the container runtime and all the containers so at that point we we've been working with the kubernetes
05:44
community and maintainers and we thought that the status of the container runtime within kube wasn't that stable at the time so we came up with this idea of implementing a new container runtime which was meant just
06:00
for kube and which was specifically in open governance project and nowadays cryo it's it's under the kubernetes CX organization so we we adhere to the cube rules and again that that's open governance for us cryo is also of
06:22
course an open source project we're on github it's really tiny as far as the code base goes so it's really lean it's really stable because it's it's it's small it's also it's also secure because since it's small we can
06:42
also audit the full source code and fix bugs instantly and probably the most important thing is that cryo it's boring because we don't do anything else that just what kubernetes needs in cryo we don't have any such features that
07:03
kubernetes don't need for instance we don't build images kubernetes doesn't need to be images we don't build images and that's really important because when you have a huge project like a container runtime if you implement features that can actually mine stability on other parts of the
07:21
runtime that's best so we didn't want to do anything like that so cryo just implements what it needs for kube and that's it and again that's boring that's also boring because we choose to use the the open container
07:40
initiative specification specifically we are using the runtime specification in the image specification this make sure that from release to release there won't be any breakage in the way we talk to the runtimes and that's pretty much what kube wants as well just wants for every release to run the
08:02
containers and it was doing in the previous release so by adhering to such standards we make sure that hopefully we don't break from from time to time and again cryo is made for kube and just for kube there is no other intended user other than kubernetes so there is no cryo ps or cryo logs or
08:26
whatever it's just meant for kube so to actually narrow down the scope the scope of cryo itself it's tight the CRI so from release to release if the CRI needs a new action to be implemented we'll do that and only that we don't we
08:45
won't do anything other than that cryo itself is shaped around kube and that means that we know what kube needs as far as the container runtime is so we we know that there are certain actions like getting the image status or the
09:05
container status which are recurring actions and so we are able to speed up the code path in cryo itself to make it faster because kube needs that again the only supported user is kube and we will never add any features that
09:22
combine stability and performance let's talk about that in a moment as well but we make sure that we never break kube in any way and we make sure for every release to not regress in performance versioning is also tied to upstream kube so right now 112 is going out the door and guess what cryo is
09:47
also going with the cryo 112 so that it's it's a no-brainer to understand which version goes which with the version of kubernetes supported support is also tied to kube and that means a cryo is backporting patches as long as
10:04
kube needs that so right now the all the supported release is probably one seven and with the still support one seven for security fixes this is again an eye lever view of the architecture of cryo with within kube you can see
10:24
the kubelet on the left side calling through the gRPC API into cryo and you can see it's made of the image service and the runtime service I've talked before for the image service we're using a library called containers image which is the one in charge of pulling images listing images on a on
10:46
a any given node and stuff like that and the runtime service is the one which is actually running the containers through the OCI container runtime we make use of library slice the OCI generate which is a library to
11:02
create configurations for OCI compatible runtime so we make sure that we can generate one configuration for every OCI compatible runtime so you can come up with your own container runtime and it should just work for networking we're using the CNI probably you're already familiar with this if you use
11:22
kube and at the lowest level we use a library called container storage which is a one in charge of for instance exploding the image into layers on a copy-on-write fault system like overlay FS and then run your container
11:41
above you can see those are two examples of two pods running with cryo so there is an infra container because either cryo nor docker for that matter is a pod concept so we did the same as docker did and we implemented the pod
12:00
concept is a an infra container holding nay spaces and C group and then we spawn the actual application containers inside the nay spaces and C groups for the infra container so that we achieve the pod concept in that way and for every container running we also have a little tool called common which stands
12:23
for container monitoring I will dive into that in a moment so the the heart of cryo and kube itself it's running containers so we do make sure that we can run almost every container out there if it's OCI compatible right now
12:44
our default is run C which is a project and a runtime inside the open container initiative repository but we also support new runtimes like kata containers which was previously known as clear containers and for every
13:01
runtime we also make sure that it won't break cube in any way and for instance the kata containers maintainers reports to us for every request if something goes wrong so that we won't merge any features that can actually break the other container runtime container storage is the library
13:23
I've talked to you before and is the one in charge of managing layers on copy on write fault systems cryo by default use overlay FS which is pretty stable and it's our default in container storage is also you can view it as the the storage drivers for for docker like device mapper overlay then
13:48
we do have again the containers image library which is a library again for pulling images this library is probably where everything started because we extracted it from a tool I'm going to talk about later and we embedded it
14:06
into our downstream docker as well so it's really battle-tested you can pull images without any privilege as well like today if you want to with other tools if you want to pull an image you actually need to be root that
14:22
wasn't what we wanted so as part of developing containers image we make sure that we can pull images as unprivileged user as it should be in containers image is also for us has been the playground for for adding and experimenting with new features cryo itself today can run with what we call
14:47
simple signing which is signature verification for containers image so you can enable that in cryo if we want it and that has been possible because we we actually implemented all of this in containers image and we're using it in
15:03
cryo the other library that we use it's a library extracted from a tool which is the OCI runtime tool and as I've told you before is just a library that generates OCI compatible configuration so that every runtime can read it and
15:22
create containers in pretty much the same way as other OCI compatible runtime the CNI is the container network interface and you can come up with your own network stop if you want as long as it implements the this interface cryo
15:40
is able to run with most of the plugins out there supported in cube like funnel wave you name them and also the other shift SDN in them we do have this little binary which is called common which is the container monitor for every container and as the word suggests it's in charge of monitoring the
16:03
whole life cycle of a container it's basically in charge of starting the container itself so when a request from the cubelet comes we do spawn a cry a common process that in turns spawn a run C for our default runtime a run C process and take care of it for the whole life cycle of the
16:23
container itself it's responsible for logging for handling the TTY for the container serving a touch client detecting the reporting out of memory condition and probably most importantly it's also common is the parent process for the container itself that means that cryo can go down
16:44
entirely but your containers will still keep running and serving requests that hasn't been the case before with other runtime as well whether if you updated at the runtime in question or cryo it will go it will bring down all your containers so since we do have this little binary in between cryo and the
17:05
container itself we are able to upgrade cryo without any downtime for the containers running this is a zoom on on the pod architecture we're running
17:20
with run C you can see this all box is the pod itself which shares the IPC net impedance spaces and C group and these are the application containers that joins the namespaces of the infrastructure containers all holding in the namespaces and for each containers we do have a common process
17:45
this is the architecture which is a bit different with core containers and cada containers since this is a virtual machine the cada containers developer implemented a shim in between the common process and the
18:03
application containers and the pod itself so this is it's pretty different than this one but yeah this is how it is in cada containers and now I hope everything is going to go well I'll show you how transparent
18:23
was to actually implement well use cryo with with cube since you you'll have the same experience as with other container runtime you're already used to so I do have a one node cube cluster running on my laptop and
18:42
draining all my battery power you can see here we have the node it's a 1.11 node so it's the lightest I do have the cube DNS which is in charge of the networking for for the whole cluster and the dashboard running and
19:03
we're going to run an HTTP container with the image HTTP Alpine and we should be seeing it running you can see it's running I'll show you in a moment that we're actually running with cryo and nothing else I don't know
19:28
I'm using this terminal for the first time because I thought it was better can you read it not at all are you able to see it now I'm gonna run
20:36
this you know okay so we're gonna run this again the notes all the pods
20:50
running we can see that the HTTP D pod is running and I'm starting a new one here in the true now so the last one you can see it has been started
21:03
successfully we can test it by curling the IP address for that pod we can see it's working we can curl a not found page for the sake of grabbing the logs and see that it's working over there you can see the slash as g.com 2018 we
21:26
can also kubectl exec as you would normally do with other container runtime and you can see a CD BD is running you can also run interactively and it's working and now just check that we're running with cryo itself you
21:43
can see that the output of kubectl describe is showing that we're running with the cryo slash slash which means that we're running indeed with cryo and there is no docker running at all rather all our containers for every pod that you've seen in the output of kubectl get pods is actually running
22:03
through run C and this is our container the HTTP D container running we can check that that is the one by looking at the PS ox output you can see it's still HTTP D and for each container we're running if we are also
22:26
running a system the scope for it so for the sake of blogging and monitoring it so if you prepend cryo dash to the container ready the scope you can actually you can see that it's it's integrated with system D itself so we
22:44
can see the of the processes running for it the logs streaming to to the system this scope as well and we do also run a scope for common as well and see it here as you would normally do with other container runtime of
23:06
course everything is going to work normally so here's the dashboard which is working just fine you can see we do have the two pods running and since we do care about performance in cryo itself with implemented promedius I
23:25
know this is really small with implemented promedius metrics so that if something goes wrong or if something isn't looking that good someone can go
23:42
here and just try to understand what's going on so back here so the status as of today is that the CRI interface is implemented at any time and that means for every release we make sure all the CRI is implemented so if any
24:02
new actions are is required by the CRI itself we implement it before releasing it we do have maintainers from and contributors more than 90 maintainers more than 90 contributors from many companies like read that Intel IBM you name it you can run cryo with cube with cube ADM or mini cube
24:24
there's support for mix of workloads nowadays these are called runtime classes in one in Kubernetes 1.12 that means that you can have certain pods running with run C which is a runtime just not virtualize it whether you can
24:41
have other pods that can run with run times like cut containers so that you have virtual machine for a pot cryo itself is deployed to the to our open shift align cluster so we do have a pretty good user base that it's testing on it and the package for cryo is available in fedora you boom to
25:01
and it's really easy to get started we do make sure that we don't as it told you before we make sure that we don't actually break cube at any time we do run more than 1000 and when test it for every poor request and that that
25:24
allow us not to break Kubernetes in any way we do also run the open shift test not as part of every poor request but we do run them which is a superset of the cube entrant test we do run a conformance testing which is the CRI test we do have our own integration test and before every
25:44
release we also run a performance run to make sure that we know we don't regress if you want to get started with cryo and cube now you can use minikube with this command line I'll upload this slide so that everyone can actually take this or if you're running from a local setup you can
26:06
pass some environment variables to local up clusters and get going as I'm doing here on my laptop to debug what's going on with with cryo and especially with the whole CRI interface the community the upstream
26:22
community in cube came up with a new tool called CRI CTL which is a small binary that just talks to the CRI endpoint of any container runtime so that in case something goes wrong on a node you usually SSH into it and
26:41
docker ps or docker images docker exec you can pretty much do the same with CRI CTL so this is a this has been tested a is a debug tool and we're going to move this project into the kubernetes organization itself since it's crucial for the cube project so everything in cryo has been made
27:07
possible by some libraries like containers image container storage and over time even before starting cryo itself we created some tools that are
27:21
working with containers in general so that we created tools like scope yo which is a tool to run to play with containers image like you can pull images as I told you before by using containers image which is the one scope you're using so you can you can actually pull an image without being
27:43
root as it's desirable there is no demo running with pulling an image so it just straight command and you'll have the images on your roast or you can sync up images between registries another tool that we've built by
28:00
leveraging those little libraries that we're using in cryo itself is builda which is a a tool to build containers images and again we don't have any demo running builda uses a shell like syntax to build images so there is support for docker files but um of course we know that people are still
28:23
using our files so it's still supported and then we created a tool called podman which is just running containers as as docker does like today you can podman run something and all of these three tools maybe except scope you are going to be done with and by Dan Walsh later today so
28:45
make sure to to come to session so overall the Kubernetes community came up with this idea of the CRI interface and will average that by creating a probably I'd say the best runtime since it's the only runtime
29:04
made for cube specifically for cube any other runtime as other supported user whether cry is just for cube and so by also leveraging all the libraries we'd use we use it into cryo we also built many tools for the whole
29:22
containers ecosystem building up from libraries like containers image storage all the open containers initiative libraries so the roadmap very briefly is to well not the first one but we aim to to be the default in cube
29:40
upstream since we we think that we are specifically for cube we will keep pace with upstream cube and that means tracking and supporting kubernetes all the kubernetes version out there we moved out the kubernetes incubator organization to something called the kubernetes 6 which is a repository for
30:03
six specific tools so cry is in there now it's not the default runtime for open shift but that's on our roadmap but it's already the default container runtime in opens use a micro s so if you install and use micro s right now you get the cubelet and the cryo runtime as default and of
30:24
course we are aiming at having more adoption overall if you want to get involved make sure to to go to these links and we do have a free node channel cryo and we usually hang around in the signal channel and cryo channel in the
30:41
kubernetes slack with a website if you want to to see it as well that thank you if you want stickers Vincent a sticker sort of there I ran out of time so I guess I'll take questions out outside