We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

From Go to Kubernetes CRDs and Back

00:00

Formal Metadata

Title
From Go to Kubernetes CRDs and Back
Subtitle
Workflow for building strongly typed APIs
Title of Series
Number of Parts
490
Author
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
Kubernetes is built using Golang. CustomResourceDefinitions are the primary extension points for bringing custom data into a Kubernetes cluster. This hands-on talk is about the workflow of API definitions in Golang, generation of OpenAPI schemas as part of the CRD, client and informer generation and how to use these to process data in real-time using logic implemented in Golang controllers.
33
35
Thumbnail
23:38
52
Thumbnail
30:38
53
Thumbnail
16:18
65
71
Thumbnail
14:24
72
Thumbnail
18:02
75
Thumbnail
19:35
101
Thumbnail
12:59
106
123
Thumbnail
25:58
146
Thumbnail
47:36
157
Thumbnail
51:32
166
172
Thumbnail
22:49
182
Thumbnail
25:44
186
Thumbnail
40:18
190
195
225
Thumbnail
23:41
273
281
284
Thumbnail
09:08
285
289
Thumbnail
26:03
290
297
Thumbnail
19:29
328
Thumbnail
24:11
379
Thumbnail
20:10
385
Thumbnail
28:37
393
Thumbnail
09:10
430
438
BuildingCodeComputer programmingMeta elementStrutData typeNamespaceServer (computing)DatabaseObject (grammar)SpacetimePay televisionEvent horizonMechanism designLocal GroupElectronic mailing listRevision controlMathematical singularityData storage deviceDependent and independent variablesDatabasePrototypeObject (grammar)NamespaceMoment (mathematics)Type theoryGroup actionMetreServer (computing)Revision controlData storage deviceInfinityTwitterGreatest elementRadical (chemistry)2 (number)Client (computing)Field (computer science)Representational state transferInterface (computing)Streaming mediaElectronic mailing listStandard deviationPatch (Unix)Event horizonRepresentation (politics)Library (computing)Instance (computer science)Normal (geometry)LogicCuboidCodierung <Programmierung>Key (cryptography)Service (economics)Validity (statistics)CodeComputer animation
Physical systemPlane (geometry)Control flowScheduling (computing)Local ringData storage deviceReading (process)Error messageServer (computing)Staff (military)Total S.A.Extension (kinesiology)SpacetimeGoodness of fitProjective planeRevision controlNamespaceComputer-aided designSlide ruleSource codeComputer animation
SpacetimeProgrammable read-only memorySample (statistics)WindowNamespaceDefault (computer science)Electronic mailing listLecture/ConferenceSource code
Electronic mailing listMetadataWindowNamespaceConfiguration spaceR-ParitätUser profileSource codeJSON
Electronic mailing listMetadataNamespaceConfiguration spaceWindowHill differential equationView (database)BuildingInstallation artFormal verificationComputer wormString (computer science)Software testingProjective planeType theorySource codeJSONProgram flowchart
Chemical polarityBuildingWindowRaw image formatString (computer science)StrutMeta elementSoftware development kitIntrusion detection systemControl flowCycle (graph theory)Computer iconFocus (optics)Computer-generated imageryMachine visionDuality (mathematics)Contrast (vision)2 (number)Contrast (vision)Source codeJSON
WindowFingerprintPlane (geometry)Integrated development environmentFocus (optics)Rule of inferenceComputer configurationIntrusion detection systemTouchscreenDefault (computer science)Atomic nucleusString (computer science)BEEPComputer filePredictabilityStrutElectronic mailing listMeta elementBuildingDialectCodeType colorGraph coloringType theoryAreaExpert systemSource codeJSONComputer animation
PermianWindowStrutStandard deviationGraph coloringTerm (mathematics)Field (computer science)Computer animation
StrutComputer fileWindowRankingLattice (order)CodeGame controller2 (number)Boilerplate (text)Source code
Configuration spaceLattice (order)WindowView (database)Computer fileRevision controlElectronic mailing listFunctional (mathematics)Directory serviceClient (computing)Utility softwareSource codeComputer animation
Electronic mailing listMetadataWindowNamespaceHacker (term)Configuration spaceProgrammable read-only memoryCodeClient (computing)Representational state transferSource codeJSON
Electronic mailing listMetadataClient (computing)WindowNamespaceHacker (term)Maxima and minimaScripting languageConfiguration spaceInterface (computing)StrutNumbering schemeComputer fileRevision controlLattice (order)Set (mathematics)Client (computing)Electronic mailing listFehlererkennungRight angleConfiguration spaceSource codeJSONComputer animation
Server (computing)Client (computing)Computer fileWindowString (computer science)Reflection (mathematics)Computer configurationElectronic mailing listType theoryLattice (order)NamespaceElectronic mailing listComputer configurationFehlererkennungObject (grammar)Computer animation
Electronic mailing listClient (computing)Lattice (order)Reflection (mathematics)WindowNormed vector spaceRange (statistics)SpacetimeLine (geometry)2 (number)Complete metric spaceProgram slicingComputer animation
Electronic mailing listMetadataClient (computing)Total S.A.Physical systemWindowView (database)NamespaceHacker (term)Hill differential equationScripting languageComputer fileRange (statistics)SpacetimeType theoryGraph coloringRadical (chemistry)Source codeJSONComputer animation
Electronic mailing listMetadataClient (computing)Total S.A.Physical systemHill differential equationWindowView (database)Hacker (term)Programmable read-only memoryConvex hullCodeScripting languageSource codeJSON
Menu (computing)Total S.A.Physical systemMetadataWindowNamespaceConfiguration spaceR-ParitätLocal GroupRevision controlData storage deviceField (computer science)Hacker (term)Type theoryObject (grammar)Open setSource codeJSON
WindowExtension (kinesiology)MetadataLocal GroupRevision controlData storage deviceField (computer science)Game controllerHacker (term)Patch (Unix)Repository (publishing)Slide ruleLink (knot theory)CubeSource codeJSON
WindowScripting languageArchitectureSource codeRepresentation (politics)Client (computing)InformationServer (computing)InferenceCASE <Informatik>String (computer science)Revision controlView (database)Patch (Unix)Hacker (term)Function (mathematics)Gastropod shellHill differential equationGame controllerType theoryPrice indexStrutRange (statistics)Function (mathematics)Graph coloringComputer-aided designSource codeComputer animationJSON
Line (geometry)Scripting languageView (database)WindowRevision controlServer (computing)String (computer science)InferenceCASE <Informatik>MetadataCubeGraph coloringField (computer science)Object (grammar)Source code
Revision controlString (computer science)Server (computing)InferenceCASE <Informatik>MetadataExtension (kinesiology)View (database)WindowScripting languageNamespaceConfiguration spaceGraph coloringDeclarative programmingObject (grammar)CubeSource codeJSON
StrutWindowRadon transformInclusion mapQuadrilateralType theoryPoint (geometry)ImplementationComputer animation
Extension (kinesiology)MetadataString (computer science)NamespaceConfiguration spaceWindowScripting languageGame controllerHacker (term)Function (mathematics)Type theoryPrice indexRun time (program lifecycle phase)Object (grammar)StrutHill differential equationLine (geometry)Pattern languageIntegerServer (computing)Revision controlConvex hullInferenceCASE <Informatik>Numbering schemeView (database)Source codeJSON
Computer fileWindowStrutMetadataRevision controlString (computer science)Pattern languageIntegerServer (computing)InferenceCASE <Informatik>View (database)Game controllerHacker (term)Serial portNumbering schemeLine (geometry)Web pageScripting languagePoint (geometry)Type theoryPattern languageComputer animationSource code
Web pageMetadataRevision controlString (computer science)Pattern languageServer (computing)InferenceCASE <Informatik>WindowScripting languageRootIntegerExtension (kinesiology)Hill differential equationR-ParitätNamespaceConfiguration spaceComputer fileStrutInclusion mapTangible user interfaceCodeBuildingInfinityWorld Wide Web ConsortiumObject (grammar)Type theorySource codeComputer animationJSON
View (database)WindowInformationGNU EmacsPhysical systemScripting languageHill differential equationError messageSource codeJSON
MetadataError messagePhysical systemTotal S.A.View (database)WindowNamespaceConfiguration spaceStrutIndian Remote SensingComputer fileExecution unitNumbering schemeDivisorLinear partial informationInstallation artHill differential equationGNU EmacsInformationFundamental theorem of algebraMessage passingLine (geometry)Game controllerSource codeJSONComputer animation
Duality (mathematics)Convex hullComputer fileWindowBuildingClient (computing)Interface (computing)SpacetimeSlide ruleMereologyEvent horizonComputer animation
Lattice (order)Client (computing)WindowComputer fileInterface (computing)SpacetimeRadio-frequency identificationBEEPHacker (term)Control flowBit rateFunktorLoop (music)BitObject (grammar)Event horizonLine (geometry)Streaming mediaCodeLogicFunctional (mathematics)Point (geometry)Revision controlCoroutineComputer animationSource code
MetadataError messageTotal S.A.Physical systemWindowScripting languageInsertion lossNamespaceConfiguration spaceDisk read-and-write headHill differential equationGastropod shellView (database)Game controllerSystem callEvent horizonBitComputer fontSource codeJSON
Error messageTotal S.A.Physical systemInsertion lossDisk read-and-write headEvent horizonLogicSlide ruleMereologySource codeJSON
MaizeHand fanAnnulus (mathematics)GoogolInternetworkingGame controllerBitComputer animation
Point cloudOpen source
Transcript: English(auto-generated)
I want to talk about CLDs. CLDs, it's an important topic in cube, in Kubernetes. And we won't see much of Kubernetes today, but we want to define APIs which are independent from Kubernetes, our own APIs.
All right. So before I start, I just want to ask, so who is familiar with Kubernetes? Uses it or? So most of you, great. So everybody, not everybody, but most people know kubectl, the CLI tool I will use here. Who has built anything with the Golang libraries for Kubernetes?
Client Go and friends, maybe 20%, 30%. That's great, because the audience or the goal of this talk is to teach you about the Golang interface and see how you can define your APIs. All right, so I said in the, that's my about me slide. So I'm everywhere in Kubernetes around API servers.
You will find my code there, custom resources. And of course, follow me on Twitter here at the bottom. There's my handle. It's about APIs. All right, so I said in the agenda, I don't expect any knowledge about Golang, knowledge around client Go, or even about Kubernetes.
So what we want to do here, so I picked one example. Belgium, Brussels, it's about beer. There are many breweries. And so my example of the talk is beer. So we want to build a beer database today. I have some slides. I will turn to the terminal in a few seconds.
Most of the things we will see here are in the terminal in Golang, so really hands-on critical. I assume everybody understands what a struct is in Go. So we have a beer struct here. This is a prototype of our API. So this is the main type we want to define. It doesn't have much at the moment. It just has some type meter. Everybody who knows Kube manifests it
as a kind in API version. That's just this type meter. Every object in Kube has a name, a namespace, labels, annotations, all those things. Those are object matter. We don't care much about them today. We know they are there because they have to be there. We are more interested into everything
which comes here in the middle. So we want to have more fields here and define our own API to represent a beer. On the wire and if you use kubectl or even if you look into etcd, etcd is our key value store. So it's our database in Kubernetes. You will see not, of course, a Go struct.
You will see JSON. It's about custom resources. They are encoded in JSON, not protobuf. We don't have that. So you will see something like that. If you go into your etcd instance, you will see this piece of JSON. It's not about beer yet, so there's no fields yet, but we will add that in a second. Just to give you an idea what Kubernetes APIs are,
don't think about pots and containers and services and everything. Kubernetes basically offers an object database. And you have objects. I have four on this slide, so there are four kinds of beers. Yeah, and they are their namespace.
So we have a namespace for Lefebere and one for Duvelbier, and two of them for each brand. And those objects live there, and you can create them, you can delete them, you can look for changes, you can list them, and so on. So everything you expect from a normal REST API you can do here.
This is pretty standard REST APIs. Everybody knows that. Namespaces are maybe special. The API, of course, consists out of HTTP paths. So here's an HTTP interface, and every API we publish from the API server looks like that. So in the front, we have APIs.
Then we have something we call an API group. We have the beer DB, because we want to build a beer database here, Fostem Org, just some arbitrary name. We just create the V1 version. We could even version that. We don't do that here today. It's namespace, so Lefebere is one namespace. Duvelbier is another one. And we have the beers inside, so that's what's called a resource.
We define this beer resource. A resource is basically what you have as a type in Go. The Go struct, that's our resource in the API later on. And you have a name of one instance, and we have the blonde Lefebere here, so this is one instance. And if you call get on the API server, just a simple curl or whatever you want to use,
or some Golang code, you get this JSON back. Very simple. What is very special about Kubernetes APIs, they are uniform. Every object has the same kind of interface. So there are seven verbs, logical verbs, if you want. They are mapped onto HTTP verbs internally, but it's what you expect from a REST API.
Again, you can get objects, which we have here in the example. You can list them. If you just leave out the name here, then it gets a list of all objects of the Lefebvre brand. You can create them, update them, patch them, delete them, and there's one special verb that's called watch, and watch gives you an event stream
of all changes of objects. And that's very important in Kubernetes. You don't pull your objects all the time, but you use this watch interface. So you create a watch, which is an infinite HTTP connection, and you get events. Here, Lefebvre brand was changed. Here's old one, here's a new one, do whatever you like.
And you wait, and you wait. On this HTTP connection, you don't pull. You get another event, there's a new beer now, because Lefebvre offered another beer. Watches are everywhere in Kubernetes. To tell a Kubernetes cluster about your new type, so we want to define an API group,
beer DB, FOSTA, MORG, and inside of it, there should be beers. So the main type is a beer. It's namespaced. Yeah, we want namespaces for the brands. That's a choice. You can also have cluster-wide resources if you like. But here, it makes sense to have a namespace. Version, there's just one version, v1. It's a storage version, so it's a version which you will also see in etcd.
It's served, so you can also switch off APIs here if you like, if you have multiple of them. And there's a schema. So I put this nice emoji here because I talked about type APIs. We want to have types later on. So this one is not typed. It's an object. It preserves everything we throw on it. So if you take any kind of JSON
and create an object in Kubernetes, by this definition, Kubernetes will just accept it and store everything we throw on it. It's not really what you want here, but we will fix that in a minute. And yeah, just a deeper look on what we do. So when we create this object here, this description, a custom resource definition,
the Kubernetes API server will create such a box. So there are many boxes here. There are three of them, and you get a new one. So there's an HTTP handler, there's a big box with a lot of logic inside, and this serves APIs, BADB, FOSDEM, ORG, V1, everything below. So, and there's a BS handler as well below that.
Every request which goes in here, yeah, it's decoded. So you get JSON, some bytes of JSON is decoded into a JSON representation of Golang. So Golang has this JSON unmodified function, and that's what's called here in decoding. Yeah, there's some logic. We will see some of it as admission.
So that your request is allowed, basically. It's validated. And then depending on the request you do, get create, yeah, something is done in etcd. So if you create the object, etcd is a key value store. So there are keys, that's what we see here in yellow. So there is one key for our blonde Lefebure,
and this key is just set to the JSON representation. So here's the stream, there's the bytes, which we saw before for this Lefebure, and it's just stored in etcd. That's what in create does, and it tells us, well, this worked fine, so 200 HTTP, we have a new object. Cool. All right, enough slides.
So let's jump into Golang, and the terminal. So I have a cluster here, and a project which I created before, and you can look at it. There are not many pods. We don't start pods today, so it's, is the size okay in the back?
It's okay, or should I change anything? I hope it's okay. All right, so there are some pods. We don't care. Pods are boring today, so we want to create beers. We say kubectl get beers.
kubectl doesn't know about beers. Not surprising. In kube, you don't have beers usually. We have the project here, which has a couple of folders. One has to manifest for our customer resource definition, CAD. It's what you saw on the slide before, so just a definition. We want beers, say, a namespace version one,
and let's do that, kubectl, of course. Manifest CADs, and that worked fine, and if we say now get beers, it knows about beers, but there is no beer yet. So we are near, so not really finished,
but we have something which looks good. I have some examples here. The leffe beers, so just a manifest. It looks like usual manifest in kube, so the namespace for leffe, then the triple one and the blonde one. Let's just create that. This should work because the cluster knows about beers,
so I create all of them, and if we say get beers, they are none because I'm in the default namespace, so let's list all namespaces minus big A, and there they are. So we can also show the YAML, not the short version, and there we have basically what we created before.
Here is one of them, and there are many fields, of course, which we don't care about today, so those are set by Kubernetes automatically. Here is one which is set by the apply command, for example, creation timestamp, resource version, many things.
But basically, this is what we had before in this manifest. All right, so the cluster knows about beer. That's cool. Let's see our test project. So I created a test project here. I have APIs, beer DB, and then I have types, and here you see the beer type.
It's still empty. We will fill that in a second. How? Maybe here, here, ah, here, yeah.
Light is good, high contrast maybe? No, is it better? Try the light one. The dialogs are great, but not the code. So we have the beer here.
Yeah, pretty boring. There's nothing inside. I prepared the color already, so I have a type color string, just areas of the type. I predefined three of them here, gold, yellow, blonde. I'm not a beer expert so much, so I guess there are standard terms for colors for beers. Maybe those are the white ones, maybe not.
We can add, of course, a color field here. I'm lazy. I just copy the JSON annotation. I have the color. It's still pretty boring. Nothing happens. Our CID is still the same. We will fix that in a second. Before we do that, we jump into our beer control command.
So I want to build a beer control command, not like kubectl, not that advanced, but just a command which lists all beers in the cluster. So let's do that. I wrote some code here, boilerplate code down, so it takes the home directory. We read the kubeconfig, where the credentials of the server, of the cluster,
are in. We get the config back. That's all pretty standard. It's just a utility function here. But we want to use beers, so address the beers, so call for a list of beers. All right, let's do that. So I create a client, at least I try to,
but it doesn't know about it yet, because we have a struct, we have a type, but we don't have a client. In Kubernetes, there is a code generator repository, and we use that to create a client. It just reads the APIs we have, so just the package for the APIs,
and it creates a client, a REST API client, basically. I said before on the slide, all APIs look the same, so that's why we can create a client, especially for beers, because it's basically the same, it just uses a beer type, as a return. All right, so it creates a client set. That's what we want. Takes a second, and there we go.
If we look, it called a client generator, and the client generator created a client here. Client set actually looks like that. It has our beer DB V1 API, and if you follow that, we also see a beer getter, and the beer getter can get us beers, the list of beers.
So this looks all good. Let's use that, and let's hope that Golan is quick enough to pick it up. It does, so it knows there is a beer, or there's a new for config from the beer client in our repository, and the first one is luckily the right one. You saw there are many, so we are lucky that it gives us the right one immediately.
Client error, and just copy here the error code. All right, and from that, we can just use our client. It's a rest client, so we choose the beer DB V1 API. We want a namespace. I just use the empty namespace.
This gives us all objects in the namespace, and we list all of the objects, and we have to produce some list option as well. We get back a list of beers, some error code again. I'm lazy. I copy that, and this object has beers inside.
Hopefully, the completion will tell us in a second. Beers, there's an items. This looks good. It's a slice of beers. Let's use that, and let's just print them one by line. Very simple, by namespace, by name,
and this should compile, I hope. Golden is happy. Let's try to run that. Takes a second, and it knows about the beers. And of course, if you look into the beer, which
you get back, this B variable, the B is of the right type. So we can access also our color, which we added. So we added the color. It's also there. Very nice. So let's go back to the terminal. I added the blonde beer, and I added something.
Yeah, I already did something. I added foo and bar here. Foo bar doesn't make so much sense, and as I said in the beginning, if we look at this beer, left, blonde, everything is persisted. So everything is stored. We have a foo bar there. Foo bar doesn't make sense, and our go type doesn't have it. So we don't want that, actually.
So the reason why this happens, if we look at the CAD manifest, it says here it's an object and preserved everything. Very, very bad. So luckily, we have a schema here. So there is the standard open API v3 schema. Who knows this open API or JSON schema?
A few of you know, not many. Let's create such a thing, and we finally have a tool which creates that. For a long time, we didn't have that. There's a controller gen. It comes from the cube builder repository. I have links on the last slide later on, if you want to see that. It just, we call it.
We give it a command schema patch. It should read our manifest. There's just the CAD here. That's what it reads. And we tell it about go long pass. So there's an API package everything below. It should read all of that, and it should output again
into manifests. And if we do that, and look what we have updated here now, there's a color change, obviously. And this tool has added a schema to our CAD. So if we apply the schema to the cluster, cube knows about that there's a color, finally. So here, there's a color.
And it also knows what shouldn't be there. There is no foo bar in the schema. So cube knows that I don't persist foo bar if there is such an object, or such a field in an object. So let's apply that quickly. So now, cube knows about our updates. And if we now get the Lefebure again, the blonde Lefebure,
it knows there is no foo bar. There is a color. There is no alcohol value. We also had an alcohol value inside. It doesn't know about it yet. But it knows about the color, and it shows the color. And everything else is just dropped. We call it pooling. So everything is pooled from the resources. It still persists in etcd, so we created something
etcd before. So those objects have foo bar, and they have an alcohol level in etcd. But cube is so clever to drop everything which we haven't declared. And we can go on. We can go into the types again, and add alcohol level.
And there is a type for floating point values in a JSON safe way. You know that JSON can drop depending on the JSON implementation floating point precision. So we don't really want to store JSON,
but we store strings instead. Same thing again. If we say git diff, it knows about alcohol here. And the schema gets, here we go. Why don't it, why?
Because I misspelled it here. So now it should. All right. Here's alcohol. It's a string, or it's an integer, so there is some polymorphic type we have. It puts a pattern there so it knows exactly which
kind of floating point values you can put inside. And if we apply that again, and we get our object, get beers, it has alcohol back. So that's our complete object. Cool, so this is type safe now. If we create something, we edit the beer,
and we put something like that. This should not work. And kubectl edit comes back, and here in the comments, there's somewhere the error message. This doesn't comply with the pattern, this regular expression, so this is not a valid value.
Let's fix it, and if we fix it, it comes back, and it's updated. All right, so I promise that we do real-time watches, so we want to react on those. So I prepared a revert here which gives us back the controller.
So this is the real-time part. So we have a client, and we have something called informas. You saw the watch in the beginning, on one of the first slides. Watch is an event stream, and informas is a concept in Kubernetes to use watches, to listen for events.
And yeah, here we create a client again. It's the same thing which we did before. Before, we had the for loop here. Now we are a bit more clever. We don't just list, but we want to watch events, and we can create an informa. It's also created a generated code, so here's an informa package which has all of that.
We create that for our beer API. We give a client, we give some recent period, which gives us every 30 minutes a whole set of objects, again, to reconcile if we missed some events. We take the informa for beers, version v1. There's an event handler, so it has three functions.
For edit objects, you just get the object, which you know is a beer for the beer API. So we cast two beer objects, and we can do whatever we want. We have the full object now, and we can react. We can do any logic here. I just print, I'm pretty boring and lazy here. I just print as a new beer. We have an update where you have the old object
and the new object. You can compare, you can see, you can see, what really changed, and do whatever you like. And there's the delete as well, and you can show that the beer was deleted. And you have to start the informa. It's a Go routine inside which starts this event stream. So you start it, and from this point on, all your event handlers are just called.
Very, very simple. You see it's 20 lines of code. You have a real-time interface, event-based. It's not heavy on the API server, so in Kube, we have hundreds of those informas. So just use it. Don't call. And if we are lucky, and we run that here, it starts.
It gives us all beers. We don't know whether they are new or pre-existing. Those who are pre-existing, we still see them in our event handler. And if we jump here, and make the font a bit smaller, and we do something, we get an update event live in the same millisecond, but in the same,
I don't know, 50 milliseconds or something like that. And we can react on that, and do whatever logic we like. All right, so this was the practical part. Let's go back to the slides. Yeah, all of this. Where's my summary slide?
Here it is. It's one small example. Of course, there are bigger ones. We use the KubeBuilder for the controller gen tool below. KubeBuilder is the SDK, if you want, for Kubernetes, which is a bit more convenient. So it does a lot of things. We did here manually. It does it automatically. You can create go-structs, for example, automatically,
and do all the boilerplate automatically. Just take a look on that. Of course, there's an official documentation. Yeah, read through that. Everything is documented more or less well. And there are basically hundreds of controllers and CRTs in the internet. Just go to GitHub and look for custom resource definitions there everywhere nowadays.
So many, many examples. All right, thank you.