Mario’s adventures in Tekton land
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 490 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/47484 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 2020319 / 490
4
7
9
10
14
15
16
25
26
29
31
33
34
35
37
40
41
42
43
45
46
47
50
51
52
53
54
58
60
64
65
66
67
70
71
72
74
75
76
77
78
82
83
84
86
89
90
93
94
95
96
98
100
101
105
106
109
110
116
118
123
124
130
135
137
141
142
144
146
151
154
157
159
164
166
167
169
172
174
178
182
184
185
186
187
189
190
191
192
193
194
195
200
202
203
204
205
206
207
208
211
212
214
218
222
225
228
230
232
233
235
236
240
242
244
249
250
251
253
254
258
261
262
266
267
268
271
273
274
275
278
280
281
282
283
284
285
286
288
289
290
291
293
295
296
297
298
301
302
303
305
306
307
310
311
315
317
318
319
328
333
350
353
354
356
359
360
361
370
372
373
374
375
379
380
381
383
385
386
387
388
391
393
394
395
397
398
399
401
409
410
411
414
420
421
422
423
424
425
427
429
430
434
438
439
444
449
450
454
457
458
459
460
461
464
465
466
468
469
470
471
472
480
484
486
487
489
490
00:00
Software testingMoment (mathematics)Adventure gameObservational studyGoodness of fitPresentation of a groupRight anglePerfect groupSound effectBitGroup actionDifferent (Kate Ryan album)Projective planeComputer animation
00:53
BuildingStandard deviationComponent-based software engineeringShared memorySystem programmingGoogolPoint cloudContinuous functionProjective planePhysical systemConnectivity (graph theory)Open sourceGroup actionSet (mathematics)Arithmetic meanTotal S.A.GoogolComputer animation
01:28
Standard deviationDeclarative programmingScale (map)Computer-generated imageryAxiom of choiceBuildingComputing platformServer (computing)Virtual realityInteractive televisionService (economics)Projective planeTask (computing)Line (geometry)Medical imagingCross-platformRule of inferenceArithmetic meanVirtual machineProcess (computing)RootkitStandard deviationComputer animation
02:58
Task (computing)outputGraph (mathematics)VolumeComputer-generated imageryLine (geometry)Session Initiation ProtocolWeightCore dumpEvent horizonExtension (kinesiology)BitInformation securityTask (computing)Line (geometry)Event horizonProjective planeVolume (thermodynamics)Social classIntegrated development environmentOrder (biology)Connected spaceCombinational logicSeries (mathematics)Source codeSequenceMultiplication signRoundness (object)CuboidMedical imagingDependent and independent variablesMultiplicationUsabilityNumberParticle systemBooting40 (number)Video gameConnectivity (graph theory)Web 2.0Operator (mathematics)Correspondence (mathematics)outputLibrary catalogHookingType theoryFunction (mathematics)Unit testingCodeComputer animation
06:52
BuildingSoftware testingConfiguration spacePhysical systemContinuous integrationScripting languageData managementGoogolPoint cloudComputer-generated imageryService (economics)Type theoryEvent horizonComponent-based software engineeringCore dumpServer (computing)Electric currentInformationGame controllerPhysical systemFitness functionProjective planeScripting languageHookingCombinational logicElectronic mailing listConnectivity (graph theory)SpeicherbereinigungProcess (computing)MereologyStapeldateiPatch (Unix)FrequencyEvent horizonSoftware testingValue-added networkOpen setPoint cloudArchaeological field surveyGoogolObject (grammar)Row (database)SoftwareShift operatorData managementWeb 2.0PlanningGoodness of fitService (economics)PurchasingCuboidArithmetic meanGame theoryComputer animation
10:42
WhiteboardError messageBlogConnectivity (graph theory)Physical systemCombinational logicLoginCartesian coordinate systemPoint cloudBitInsertion lossComputer animation
11:13
Independence (probability theory)Scripting languageComputer-generated imageryTask (computing)CodeProjective planeNatural numberRepository (publishing)CodePhysical systemMedical imagingRevision controlMusical ensembleSoftware testingCoefficient of determinationIndependence (probability theory)Arithmetic meanTask (computing)Library catalog2 (number)MultilaterationComputer animation
13:54
SpeicherbereinigungPower (physics)Coefficient of determinationMereologyProcess (computing)FrequencyTable (information)Pairwise comparisonConnectivity (graph theory)BitValidity (statistics)Medical imagingProjective planeVideo gameSound effectWeightInstance (computer science)Row (database)Computer animation
15:45
Game controllerBlogElectric currentRevision controlSoftware testingMoment of inertiaKolmogorov complexityRevision controlCombinational logicPhysical systemConnectivity (graph theory)Instance (computer science)Projective planeSocial classProcess (computing)ECosMathematicsMultiplication signLoginTerm (mathematics)CodeComputer animation
17:39
Revision controlService (economics)Integrated development environmentSoftware testingGreen's functionParallel portOrder (biology)Projective planeTerm (mathematics)Control flowGoodness of fitService (economics)Computer animation
18:42
Port scannerTask (computing)BlogStreaming mediaInclusion mapType theoryCodeSoftware testingDialectComputer-generated imageryConfiguration spaceEvent horizonSpacetimeRollback (data management)Group actionService (economics)LoginCloud computingMedical imagingMobile appTerm (mathematics)Configuration spaceBranch (computer science)Event horizonConnectivity (graph theory)Process (computing)Multiplication signControl flowPoint (geometry)Task (computing)MereologyRight angleMathematicsService (economics)Confidence intervalTorusRow (database)PlanningBoom (sailing)Figurate numberPhysical systemSocial classUniform boundedness principleMachine visionNetwork topologyNeuroinformatikBlogField (computer science)Stability theoryGoogolInternet service providerCore dumpComputer animation
24:09
Software testingReading (process)DisintegrationService (economics)Online chatData managementTask (computing)Configuration spaceZugriffskontrolleVideo trackingRevision controlFunction (mathematics)outputEvent horizonPoint cloudPhase transitionData storage deviceEmbedded systemImpulse responseComputer configurationINTEGRALMoment (mathematics)Instance (computer science)GoogolScaling (geometry)Software testingProjective planePoint cloudAnnihilator (ring theory)Process (computing)MetreFunctional (mathematics)Task (computing)Right angleNetwork topologyShift operatorMilitary baseBuildingBeer steinRevision controlService (economics)ChatterbotBitPhysical systemCoefficient of determinationClosed setSlide ruleRobotoutputFunction (mathematics)Parameter (computer programming)Computer animation
28:32
Streaming mediaError messageAddress spaceScripting languageGastropod shellView (database)Computer networkDirected setDuality (mathematics)Open setNormed vector spaceService (economics)Demo (music)SoftwareComputer animationSource code
29:08
StapeldateiPulse (signal processing)Programmable read-only memoryAddress spaceError messageStreaming mediaComputer-generated imagery19 (number)Total S.A.Medical imagingData storage deviceSource code
29:38
Computer-generated imageryTotal S.A.Demo (music)Discrete element methodStapeldateiSoftware maintenanceGoogolRevision controlComputer fileSoftwareBasis <Mathematik>Condition numberLatent heatProgramming languageGroup actionWindowSource code
30:29
Computer-generated imageryStapeldateiPunched cardDew pointMessage passingLine (geometry)Computer fileWebsitePhysicalismSource code
31:04
Computer-generated imageryAsynchronous Transfer ModeDemo (music)Object (grammar)Image resolutionThread (computing)Data compressionInsertion lossView (database)Bookmark (World Wide Web)Computer fileOpen setCodeGroup actionRepository (publishing)Sign (mathematics)Message passingNominal numberLaceMaizeMedical imagingRoundness (object)Source codeComputer animation
31:34
Demo (music)ChecklistMessage passingSign (mathematics)Computer fileBookmark (World Wide Web)View (database)Discrete element methodWindowTask (computing)Patch (Unix)Uniform resource locatorProgrammable read-only memoryRobotWordComputer animation
32:06
Task (computing)View (database)Computer fileBookmark (World Wide Web)SpacetimeDefault (computer science)Normed vector spaceNetwork operating systemSoftwareView (database)Video gameComputer animation
32:40
Asynchronous Transfer ModeCone penetration testInsertion lossComputer-generated imageryComputer clusterObject (grammar)Total S.A.Thread (computing)Data compressionImage resolutionCompilation albumGezeitenkraftDemo (music)2 (number)Medical imagingTask (computing)Source code
33:10
Computer-generated imageryDefault (computer science)SpacetimeView (database)FreewareBookmark (World Wide Web)Task (computing)Menu (computing)RobotError messageConfiguration spaceSoftware repositorySign (mathematics)RotationBranch (computer science)Computer fileGroup actionConvex hullPoint cloudWindowLink (knot theory)Demo (music)Element (mathematics)GoogolOpen setDirac delta functionImage resolutionMathematicsLocal ringElectronic mailing listSoftware maintenanceRevision controlSoftwareBasis <Mathematik>Condition numberLatent heatProgramming languageCartesian coordinate systemMedical imagingSeries (mathematics)LoginComputer animationSource code
33:45
Computer-generated imageryMessage passingLine (geometry)Computer fileAsynchronous Transfer ModeDemo (music)Object (grammar)ForceData compressionThread (computing)Host Identity ProtocolWindows RegistryRevision controlLetterpress printingFlagCache (computing)Local ringDefault (computer science)Directory serviceString (computer science)MultiplicationBuildingBookmark (World Wide Web)View (database)Error messageLink (knot theory)Module (mathematics)Context awarenessOnline helpRepository (publishing)InformationTimestampFormal verificationMathematicsLevel (video gaming)Coma BerenicesSoftware repositoryLoginArc (geometry)Insertion lossLoginComputer fileSource codeComputer animation
34:23
Demo (music)Computer-generated imageryPoint cloudBlogDiscrete element methodSign (mathematics)Programmable read-only memoryComputer fileView (database)Bookmark (World Wide Web)SummierbarkeitTerm (mathematics)Latent heatLine (geometry)Task (computing)Online helpSpacetimeDefault (computer science)Right angleError messageComputer animation
35:06
Computer fileView (database)Bookmark (World Wide Web)Task (computing)Default (computer science)Computer-generated imageryObject (grammar)Thread (computing)Data compressionImage resolutionAsynchronous Transfer ModeInsertion lossLimit (category theory)Dirac delta functionSpacetimeWindowSign (mathematics)Demo (music)Programmable read-only memoryElement (mathematics)BlogOpen setLink (knot theory)Traffic reportingSoftware bugDataflowMessage passingComputer-assisted translationLink (knot theory)Medical imagingWeb browserSource codeComputer animation
35:39
Address spaceDemo (music)WindowComputer fileBookmark (World Wide Web)View (database)Computer-generated imageryGoogolComputing platformPoint cloudWindows RegistryVirtual realityHypermediaObject (grammar)Image resolutionDirac delta functionRight angleMedical imagingTrailComputer animationSource code
36:10
Computer-generated imageryDemo (music)Electronic mailing listComplex (psychology)Independence (probability theory)Revision controlMatching (graph theory)Medical imagingSocial classGene clusterSoftware bugProjective planeProcess (computing)Service (economics)Software testingSource codeComputer animation
37:22
Lattice (order)Projective planeGoodness of fitService (economics)Open setComputer animation
38:10
Point cloudOpen sourceFacebook
Transcript: English(auto-generated)
00:05
All right. Good morning, everyone, and welcome to this presentation, Mario's adventure in Tekton 1. I'm Vincent de Mister from Riot. I'm Andrea Fristoli from IBM. And we are at FOSDEM. If people from the back do not hear us, just wave.
00:24
OK, perfect. So let's get right into the agenda. We'll first talk about what is Tekton. And first, one question. Who knows Kubernetes here? OK, that was a non-question.
00:40
Who knows Tekton? Ah, a few, all right. So this will be a bit boring for people who know Tekton. And then we'll talk about plumbing and more plumbing in the future work. So first, what is Tekton? Tekton is an open source project or more a group of projects
01:01
that is aiming to provide a set of shared components to build your own CI CD system on top of Kubernetes, initially, later on, on top of really anything. It's governed by the CD foundation and contribution from there are coming from Google, from Red Hat, from CloudBees, from IBM,
01:22
from Pivotal, from Apple now, from lots of companies. What is Tekton in a nutshell? It's like standard QA style pipelines. What does this mean is we're just extending the Kubernetes API to add CI CD concept in the API.
01:46
It runs everything in containers. It's built for containers. It's run in containers. You can build your images using any Kubernetes tools you want or any tools that is capable of building images
02:02
and running inside a container. So almost all of them. And some example are S2i, Builda, Kanico, Jib. There's like a billion of it. It can deploy on multiple platforms. So of course, we can deploy Kubernetes deployment services
02:21
using Tekton, but you can deploy really anywhere. You just need to have some way to run something like a command line on your container and do your thing. So for example, we have a task, like an experiment task that creates some MacStadium machine, run something on the Mac,
02:40
and then do its own thing. And there's a few projects. One of the project is called Tikan. It's a powerful, not powerful, it's a command line tool that aims to be powerful to be able to manage your pipeline without really having to touch kubectl or whatever. So a bit more getting into the details of Tekton.
03:04
Like Vincent said, we extend the Kubernetes API by introducing a few new concepts, which are actually CRDs, so extensions of the Kubernetes API custom resources. The first concept is a step, which is not a CRD.
03:23
Which allows you to run a command in a container using volumes and environment variables, et cetera. Then we have a concept of task, which is mapped to, corresponds to a sequential series of steps that are run in a pod on Kubernetes.
03:43
A pipeline is a combination of task, not necessarily a sequence, but more generally a graph of them. And then we have pipeline resources, task runs and pipeline run that are runtime resources that are used to trigger, so execute the definition of the task or the static definition of the pipeline.
04:03
Thank you. So this is just to give you an overview of how everything fits together. So we have sequences of steps within a task. Then the yellow orange G box is the pipeline and in the pipeline you have multiple tasks.
04:21
Task takes pipeline resources as input or they output them and this creates the connection and the dependency between the tasks. So the pipeline will take care of executing the tasks in the right order depending on the dependencies they have between them. So for instance, if your first task produces an output container image
04:42
and the second task needs that image as an input, that will automatically be serialized by the pipeline. So the purple below, the pipeline resources that are defined for the executions. So everything which is on the top part, so pipeline, task and steps, they are static definitions
05:01
that you create in your Kubernetes cluster, but to actually execute them, you need to have either a task run for a single task or a pipeline run to execute an entire pipeline. So Tekton is a combination of multiple projects. So pipeline is what contains the definition, the CRDs
05:20
for the components that we just talked about, but we have more projects. So the catalog project includes a number of shareable tasks or reusable tasks. So if you want to do something like build your go code or run unit testing Python, there is a good chance there is a task already defined in the catalog that will do that for you
05:41
and is optimized for doing that. And if not, everyone is welcome really to share their tasks in there. So the CLI is the TKN command is used to interact with Tekton from command line. Then we have a project called triggers that allows you to trigger pipelines in responses of events.
06:01
This is very useful for doing things like reacting to a web hook invocation and trigger your pipeline as a consequence, but it can be used really to response to any type of event. Operator is used to manage your Tekton deployment, install, upgrade, and so forth.
06:21
And then we have a dashboard, so which is a web UI that you can use to, oh, thanks. Yeah, okay. Yeah, okay, thanks. Yeah, the dashboard is,
06:41
we'll show it later, is a web UI that you can use to control your resources in Tekton in a graphical way. Right, so there was Mario in the title of the talk, and as you might know, Mario is a plumber.
07:01
So we're gonna talk about plumbing. So why do we choose the name plumbing? So what's the definition of plumbing? Plumbing is usually a system of pipes, tanks, fittings, and other approaches needed to water supply. For us, by plumbing, we mean anything we need
07:20
to run, to test our software, to run automation on our software, anything that we need ourself as the Tekton project to run automation on the Tekton project. So this means the CI setup, the release setup, the test infrastructure, the GitHub issues
07:40
and pull request management, so like being able to react to commands, the bot, maybe even the Slack bot, and any script that are required for really anything. So initially, Tekton comes from the Knative project. It was first Knative build,
08:00
then it became an experiment as Knative build pipeline. Then it got spun out and became Tekton pipelines. So the first plumbing of Tekton was just the Knative one. So it means all our infrastructure run on Google Cloud.
08:21
We were having an heavy usage of Pro. So who knows Pro here? All right, Pro is the CI system used by Kubernetes and other projects that build on top of Kubernetes.
08:41
And it's made by Kubernetes folks for Kubernetes. And we were sharing a lot of scripts and containers from the Knative project. So, and when we decided to create our own projects to build Tekton using Tekton, we didn't want to go with the common project name,
09:04
which is called testing, frying Kubernetes archive, because it's boring. So it shows plumbing. Okay, so yeah, like Vincent said, Pro is the CI system used by Kubernetes itself
09:21
and by a lot of the projects in the Kubernetes ecosystem. So it's optimized for Kubernetes and OpenShift, and it's made of a combination of microservices. So quickly, just going through them, the main one, Hook, is the microservice that reacts to the web hooks. So to all the events that happen in GitHub,
09:43
like a comment is left or a pull request is pushed and so forth. And then Hook distributes or yeah, gives a job to other components to do. Plank is a microservice that manages the lifecycle of jobs that are executed as part of your CI system.
10:01
Deck provides a dashboard where you can have an overview of what is going on in your CI with a list of jobs. And so Tide is responsible for merging patches, which is a interesting job, because if you want to have a, for a large project,
10:21
you have a lot of pull requests. If you want to have a decent throughput, you need to be able to do this not serially. So you need to make some assumptions on what is going to happen with your patches and merge them in batches. Horologium is the component that takes care of periodic jobs and Syncr is basically
10:40
doing garbage collections. This is what the dashboard looks like. So the deck component. And there is also an application that is using combination with Prow that currently is Goobernator. I think there is a new one being worked on that allows browsing the logs from your CI system.
11:03
So the logs are uploaded into buckets. And then this application that runs in the clouds lets you browse them in a smarter way, so to say. That's me. All right, so our goal for the project was to build Tecton Music Tecton,
11:22
because it kind of made sense to be using our own, like to dog food, because otherwise if we are not using our own project, why people should use it? So the first steps toward independence from Knative
11:42
was to create our own project, to document how we worked, to own our scripts, so to change whatever was only made for Knative, and to own our CI image, meaning build our CI image ourselves, not depending on Knative.
12:03
And the first step into this, like the second but first step in this project was to be able to release Tecton with Tecton. That was the initial project was, are we gonna be able to do a full release of Tecton using only Tecton?
12:21
And we were, I think, starting on 0.2, so the second version of Tecton, we released it with Tecton itself. It was really funky to do, so we managed to find some bugs, we managed to find some problems where it was harder to do than we initially thought.
12:45
And so what we did is, because Tecton is a Go project, we did create some Go tasks for building, linting, testing our Go code, and we put them into the shared repository in catalog.
13:02
We are publishing the images using Co, so for people who don't know Co, and I don't think lots of people know Co, it's a project from Google who, it's like you're doing a Go build, but instead it's deploying something on the cube, so it's making a lot of assumptions
13:21
on how your code is doing and how it's gonna generate the actual deployment. And we generate the release YAML and we publish it. And yeah, it's supposed to be executable anywhere. It's not, because the bucket is Google,
13:42
the infrastructure we were running on is still Google, so we weren't able to run the first release on top of OpenShift, for example. But this is something we had to work on later. Thank you. Okay, yeah, so in this part of the project,
14:02
we also did some analysis and comparison. Actually, this table looked a bit with much more yellow and red in the beginning. This is updated to what we have now, but we compare the features that we have from Pro with what we could do with Tekton. So for instance, to handle GitHub Webhook,
14:21
Pro has a component called Hook, and Tekton, we didn't have anything initially, so we built the triggers project, and now we have custom interceptors that you can use to validate requests from GitHub and GitLab and more coming. For the job execution, the lifecycle of the jobs, of course, we have the pipeline component,
14:41
so which will run your job in a pipeline. For dashboard, we didn't have anything, again, so we built a dashboard, which is marked yellow because it's still not publicly accessible for what we run
15:02
in our dog fooding cluster, but we are working towards it, so if we expose it as it is now, I think it would give too much power to everyone in the world. So for merging, we don't have a solution on Tekton's side. For periodic jobs with triggers, we do have now a solution,
15:20
so for instance, we are running now nightly builds for our projects, and every night also we build our images and we have daily deploy jobs in our cluster. For garbage collections, we don't have a solution yet, and actually, you can see that if you look
15:40
at the resources in our cluster because there are too many of them, so we need a solution soon. So well, initially, we thought, well, we could try to use a combination of Prowl and Tekton because instead of having to rewrite a lot of things, so we can then take the benefit of the dashboard from Prowl
16:01
and just have Prowl running Tekton as an agent, so Prowl is a concept of agent, and it was already able to run builds from the Knative project, for instance, or it can run Kubernetes jobs and so forth, so there was a new component introduced, which basically listens for jobs with the agent Tekton
16:26
and then triggers the pipeline for you, so we tried that out, we deployed it in our cluster, but we had a few issues with that because the logs were not integrated, for instance, so we couldn't use the solution with Kubernetes,
16:41
with Kubernetes that we had before, and also we had some problems in terms of dependencies, so the version of Tekton that Prowl relies on, it's an old version, so it used to be 0.31, now it moved up to 0.80,
17:00
but it's very hard and painful to keep Prowl up-to-date to the latest version of Tekton because of how the dependencies move, and because Prowl is used by so many projects and by Kubernetes itself, when you make a change on Prowl's side, it has possibly a significant impact to the ecosystem, so it takes time, and we want to be able to use
17:23
the latest version of Tekton every day, so we are aiming towards doing nightly deploy so that we actually verify our code changes every day, so we decided this was not a viable solution for us.
17:41
So, naturally, the next step was to create a separate cluster, so let me explain. On the project, initially, we were using Prowl, some of, like most of the projects still are using Prowl for the CI, so in order to not break everybody in one day, we decided to create a new cluster,
18:02
a doc footing cluster, where we would do our experiments and slowly but surely move projects away from the Prowl, cluster, and move it to the doc footing one, so this allowed us to have somewhere where we can break everything and not slow down
18:22
the rest of the projects. We could run anything on this cluster, it's risk-free in terms of service, at least at first, because the more we migrate projects from Tekton CD inside this one, the more we will need to be careful about it, and it's, yeah, it was a good thing.
18:44
I'm gonna do this one, too. So, the thing we didn't have, even with Prowl and Tekton, was the logs for everyone, and there's two reasons. First, the pods themselves are ephemeral and private,
19:01
but they might be killed or removed by the cloud provider. For example, GKE removes the old pods even though the task is still there, so at some point, you're not able to see your logs anymore, and we wanted a way for a user
19:22
to be able to access the logs publicly, and as of today, there's no solution, like easy solution to export the logs directly from the task, from the pod, to somewhere else, so we wrote, I mean, Scott from Google wrote a small app
19:40
that gets the log, put them in a bucket, in Google, I think, and expose them publicly, so now we can report the logs and say, hey, this is your build from this PR, and look at it there, and yeah, and we also, for the 90 release,
20:01
we are making sure we export the logs of the nightly release somewhere so that we can share it and say, okay, this, maybe this nightly failed, why, and we can see why. So yeah, the Tecton in Tecton release, well, yeah, we never, we didn't use Pro for the release,
20:26
so now, I don't think, I mean. Yeah, well, initially, we triggered the releases. Yeah, oh yeah, right. We did the release with Tecton, but it was triggered by Pro, then we switched to doing entirely,
20:40
when we built the triggers, we switched to do it entirely using Tecton, so now it's triggered by Tecton and runs in Tecton. And so for the nightly, we're using a cron job that sends an event to the triggers, and it takes care of it, and for the actual full release, right now, it's still manual, like, we need to just do the correct command
21:02
to start the pipeline from one of our systems, like saying, I want to release this version, and this is the branch of the commits which you have to tag. One day, it's gonna be fully automated, like, we're gonna create the branch from one commit, and it's gonna trigger from there.
21:22
We're quite close to that, actually. Yeah, yeah, we're quite close, and so once the build, like the release is done, usually it's gonna send an event, okay, the release is done, went well, now we can collect the logs and do some, like create the draft release on GitHub.
21:41
So this part sometimes works, sometimes doesn't, because we are trying to update all the components each time there's a release, or each time we feel like we need some feature, and well, sometimes something breaks, especially with triggers, because it's like 0.2.0, so it's really not stable, still.
22:02
So for example, for the last release, we had to do the rest of the usual thing, like collecting the logs and doing the actual release on GitHub manually, because triggers was breaking, and we didn't have really much time to figure why.
22:21
Okay, so, but the final aim is to basically do everything automated in terms of CD using Tekton, so things that we have done already, we continuously build our images that we use in our infrastructure, so every night there is a Tekton job that runs and publish the images.
22:41
Also, we're still using Pro on the CI side, and we maintain the Pro configuration in our plumbing repository, and we deploy automatically using Tekton, so that's hourly, so if there is any change in the configuration, we redeploy it and we make sure that everything is up to date. Also in Git, we keep all the resources
23:02
in terms of Tekton tasks and pipelines and triggers that we define, so we are working on deploying those automatically into the cluster, same way that we do for the Pro configuration, and also we are deploying the services themselves to the Tekton, to the dog-fooding cluster,
23:22
like Tekton pipelines and Tekton triggers and so forth, using Tekton, so this is done using a Tekton pipeline, which is triggered manually today, but as soon as we reach a level of confidence enough that we can deploy that maybe nightly, then we can use our nightly releases
23:42
and do that automatically. The thing is that we are getting to a point now where we built this dog-fooding cluster for experimenting, but now we're starting to rely more and more on it, so if we start doing nightly deployments and we're breaking it, so we probably need to have a new,
24:02
we need a cat-fooding, yeah, a second cluster where we can get our experiments back. All right, so there's still some missing pipes in our system. First, we don't have too much projects
24:22
relying on Tekton on the CI side, like on all the jobs on the PR. We are really this close to do it on some of the projects, but as of today, it's not.
24:42
We need more testing, so the dashboard is almost ready. When we did the slides initially, it was really far from it. Now it's almost ready. We can deploy it, but we can't really expose it right now because you're gonna see some ways
25:02
to schedule yourself as anonymous, a build on our cluster, although it's not gonna work because the deployment won't have the right to do it. You'll see all the buttons to do it, so we don't want to expose it as of now.
25:21
We want chatbot integration, so we do have some Slackbots and some, we even have a GitHub comment bot. I don't know if it's a chatbot or not, but we want more chatbots integration so that people can easily reschedule a failure, build a failure job on their PR.
25:42
We want something like Tide. So Tide is really awesome in the way that it checks for some requirements, like for us, it's a label in GitHub called LGTM and a label in GitHub called Approve, and from there, it has a merge queue. It rebases all the thing,
26:00
and it automatically merge it if it can, and we don't have this at all on Tekton or on our dog fooding, and we need something. Maybe we can just reuse Tide, but this is something to need. On the testing, we need to make tests more portable.
26:22
Main reason is, so the upstream Tekton is all hosted on Google Cloud right now, and some of the tests make huge assumption on this, and for us, like for example, for Red Hat on OpenShift,
26:41
sometimes we have to completely disable some tests because they just work on Google Cloud and some not elsewhere, so we need to have more vendor-neutral tasks, more vendor-neutral tests, and we need to see our own plumbing services because right now, whenever we have a new version
27:02
of our MarioBot or some of our bots, we deploy them manually because we are a little bit afraid of things. Right, so in the process of building this infrastructure, we also find, we found features
27:21
that were missing in our project, so which is the whole purpose of the exercise also to make Tekton better by using Tekton, so for instance, we noticed we definitely need something like a finally close for tasks or for pipelines, and so we are working on it. It's been designed, and we can do, we have some workarounds to have that functionality,
27:42
but it's not ideal. We are also working on building features like output parameters. Again, we have an interim solution for that, but we want to make it more easy to use. We build things like optional inputs that are very useful to improve the reusability of tasks,
28:01
so you can have tasks and say, well, I depend on this resource, or I can take this resource, but not necessarily, and that allows you to create tasks that you can reuse in more different pipelines. Also, we are working on things like notification or being able to reference a task by embedding it or using a URL,
28:22
and things like task hooks and having the ability to run a task against an array input, for instance. Yeah, oh, yeah, so we just wanted to show a short demo of something that we can do.
28:45
Hopefully, the network is still up. Let's see, no, I connected to the wrong one. Okay, almost there.
29:05
So one of the things that we do in our, we have in our plumbing repository, as I said earlier, is the definition of our container images, and they are built nightly.
29:21
So what I'm going to do is to define a new image. So these are like images we have today, so I'll just be lazy and copy one in the window,
29:47
so we don't get, okay, so bear with me a second while I
30:00
modify this. So yeah, we can just use Alpine. This, we can do it, and then, oops, I made a mistake.
30:36
The mistake is by design. Oh, that's a typo.
30:43
I'll leave it like that, it doesn't matter. So I make a commit, I can type.
31:03
I really miss my physical desk there, and then let's push this to GitHub.
31:22
Nice. Okay, so we have a new PR, which introduces a new image. Now, if I'm going to review that image, as a reviewer, I might want to actually see if that image runs. So we built a bot that you can use,
31:45
and ask him, like, can you please build that image, and publish to Fosteema with tag mario.
32:02
So when I put the comment in there, and now I switch to the text on dashboard, which hopefully is connected. If it's slow, it's the network, not the dashboard.
32:27
Yeah, okay. You can show the CLI, using the CLI here. Yeah, maybe, oh, there it is, but it's not, okay. I showed it using the CLI, because it's, okay.
32:55
So this is just 16 seconds ago, so we built an image, and we run two tasks here, two task runs.
33:03
So the last one is what actually answers back to GitHub, and the first one is what built the image. So it actually failed, and in fact, if we go onto GitHub, oh, there's one. Oh, yeah, so something went wrong. So we could not build a requested image,
33:21
so we get the link, and we can browse the logs, using the application that's got built for us. So it might be slow, too, for the same reason. Hopefully, it will show the logs. While it's showing the log, I know already what the issue is, because, yeah, it wasn't on purpose.
33:42
So let's fix the issue.
34:03
Never force a push, please. And then, yeah, this was the logs, and the logs was telling us that the Docker file was invalid. The Docker file was invalid, hopefully.
34:20
No, that the Docker file was not there, so maybe I wrote the command wrong. Oh, yeah, the folder is FOSTA, not FOSTEMO, that's why. So there was an error on top of the error. No way this would work. Okay, so let's try again,
34:41
and then use the right folder name, and then dashboard again. Wanted to select the Mario namespace, but it's, sorry.
35:06
We need to do a bug report to the dashboard team. It's excruciatingly slow. Also, the cluster is in the US, so it can be used.
35:21
So the build ran through already, and then now it was working on the commit message. Maybe if I go here, yeah, now it worked. So the cat said, okay, here is the image requested. We got the link. We can either open it in the browser,
35:40
and it points to where the image was published with the right tag, Mario, or we can run it.
36:12
So it's downloading the image, which is very small, so hopefully it's fast enough. Yeah, there you go.
36:30
Yeah, okay, so was this exercise of doing bug coding useful? Yes, I think we made a few mistakes, so we practically did it wrong. Everything was possibly to do wrong,
36:41
so we built wrong images, we broke out release versions, and we deployed things in the wrong clusters or using the wrong buckets. So this has taught us a lot about the things that we need to implement in our CI CD to make it resilient. So yeah, so we also discovered that CI CD is hard, okay?
37:04
And what kind of things that we need to improve in Tecton. So we built new services, and we have a clear backlog of things that we are going to build to solve the problems that we have. So we started new projects in the process, triggers and plumbing, we have logging application,
37:21
and we keep improving in that. So hopefully, well, we consider our community open, and we're coming, so please come and join us if you want to contribute in any way. If you try out the project, you want to contribute issues, you want to contribute to the code, documentation. Yeah, so we also have, as luck, where you can find us.
37:44
So thank you. Thank you. And thanks. Two minutes for questions. One question, and then question outside if you can catch us.
38:02
And it's okay if you don't have question. It's good then. Thank you very much. Thank you.