Running trusted payloads with Nomad and Waypoint
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Anzahl der Teile | 287 | |
| Autor | ||
| Mitwirkende | ||
| Lizenz | CC-Namensnennung 2.0 Belgien: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
| Identifikatoren | 10.5446/56832 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
|
FOSDEM 202224 / 287
2
4
6
8
12
17
21
23
31
35
37
41
44
45
46
47
50
62
65
66
67
68
71
73
81
84
85
86
90
92
94
100
102
105
111
114
115
116
117
118
121
122
124
127
131
133
135
137
139
140
141
142
145
149
150
156
164
165
167
169
170
171
172
174
176
178
180
183
184
189
190
192
194
198
205
206
207
208
210
218
220
224
225
229
230
232
235
236
238
239
240
242
243
244
245
246
249
250
253
260
262
264
267
273
274
277
282
283
287
00:00
BitNichtlinearer OperatorDiagrammTechnische Zeichnung
00:43
Faktor <Algebra>Produkt <Mathematik>DatenbankDatenverwaltungBefehl <Informatik>RechenschieberMereologieGrundraumTouchscreenNichtlinearer OperatorEinfach zusammenhängender RaumPunktKonfigurationsraumSoftwareentwicklerWurm <Informatik>Faktor <Algebra>PunktwolkeServerEDV-BeratungDatenbankProdukt <Mathematik>BildschirmmaskeCASE <Informatik>QuellcodeGenerator <Informatik>AdditionInternetworkingVersionsverwaltungMultiplikationsoperatorOpen SourceComputeranimation
02:10
DatenbankProdukt <Mathematik>TouchscreenGroße VereinheitlichungNichtlinearer OperatorEinfach zusammenhängender RaumBesprechung/Interview
02:50
CASE <Informatik>Physikalischer EffektProdukt <Mathematik>MereologieGüte der AnpassungCodeComputeranimation
03:58
CodeMetropolitan area networkCASE <Informatik>eCosPhysikalischer EffektSoundverarbeitungKonditionszahlUmsetzung <Informatik>Mandelbrot-MengeTypentheorieValiditätFunktionalBildgebendes Verfahren
04:57
UnordnungPhysikalisches SystemDesintegration <Mathematik>BereichsschätzungWorkstation <Musikinstrument>ComputersicherheitRechenwerkSoftwaretestPhysikalisches SystemProzess <Informatik>CodeDifferentePhasenumwandlungProdukt <Mathematik>SoftwareentwicklerUnordnungMultiplikationsoperatorIntegralDesign by ContractFächer <Mathematik>BitKonfiguration <Informatik>Rechter WinkelSystemintegrationKlasse <Mathematik>Elektronische PublikationSondierungResonatorObjekt <Kategorie>Physikalischer EffektEinfach zusammenhängender RaumDienst <Informatik>Computeranimation
08:07
Güte der AnpassungPhysikalisches SystemBitrateInstantiierungNatürliche ZahlSoftwareentwicklerProzess <Informatik>PunktProdukt <Mathematik>Metropolitan area networkCodeComputeranimation
09:31
SystemplattformModemPlug inOpen SourceDatenverwaltungSchlüsselverwaltungChiffrierungToken-RingPasswortKontrollstrukturCLIDienst <Informatik>QuellcodeCodeProxy ServerPhysikalisches SystemInternetworkingVersionsverwaltungRechenschieberProzess <Informatik>ProgrammfehlerRechter WinkelEinsPhysikalisches SystemInternetworkingCodeMultiplikationsoperatorDatenverwaltungPasswortServerGruppenoperationOpen SourceOffice-PaketKonfigurationsverwaltungProblemorientierte ProgrammierspracheSchreiben <Datenverarbeitung>Produkt <Mathematik>Nichtlinearer OperatorKonfigurationsraumFormale SpracheSchnittmengeGrenzschichtablösungSoftwareentwicklerFramework <Informatik>SoftwarewartungProjektive EbeneTouchscreenElektronische PublikationDigitales ZertifikatValiditätPlastikkarteSchlüsselverwaltungDienst <Informatik>Token-RingStreaming <Kommunikationstechnik>SpieltheorieAbgeschlossene MengePhysikalischer EffektPunktSystemaufrufDifferenteFigurierte ZahlFormation <Mathematik>TermInteraktives FernsehenDomain <Netzwerk>InformationsspeicherungHorizontaleFokalpunktObjekt <Kategorie>NormalvektorQuelle <Physik>Bildgebendes VerfahrenService providerBenutzerschnittstellenverwaltungssystemAbstimmung <Frequenz>Kette <Mathematik>Message-PassingFreewareGüte der AnpassungBildschirmmaskeAdressraumComputeranimation
16:27
InternetworkingStetige FunktionSoftwareDokumentenserverSpezialrechnerCodeData MiningBitNeuroinformatikCodeGamecontrollerKette <Mathematik>DatenflussProzess <Informatik>Formale SpracheZentralisatorProgrammierungCachingService providerDokumentenserverBildgebendes VerfahrenOpen SourceAnalytische FortsetzungMAPÄquivalenzklasseMultiplikationsoperatorSoftwareProdukt <Mathematik>SoftwareentwicklerNetzbetriebssystemZusammenhängender GraphMailing-ListeEinsPhasenumwandlungStellenringVerzweigendes ProgrammInstallation <Informatik>TeilmengeRepository <Informatik>PunktNichtlinearer OperatorValiditätHook <Programmierung>PortscannerGraphfärbungPRINCE2ResultanteEinfügungsdämpfungNP-hartes ProblemGeradeBildschirmmaskeFormation <Mathematik>E-MailMehrrechnersystemGesetz <Physik>Konfiguration <Informatik>VererbungshierarchieDifferenteInformationsspeicherungVerkehrsinformationQuick-SortVersionsverwaltungPerspektiveArithmetisches MittelGruppenoperationGradientWeg <Topologie>Computeranimation
23:24
BeanspruchungSchedulingGasströmungOpen SourceStapeldateiFront-End <Software>RechnernetzTaskServerTreiber <Programm>SpezialrechnerDatenstrukturKonfigurationsdatenbankSoftwareentwicklerKontrollstrukturPhysikalisches SystemCodeProxy ServerQuellcodeKonstanteAppletSpielkonsoleFokalpunktKartesische KoordinatenTypentheorieMereologieKonfigurationsraumFormale SpracheKonfigurationsdatenbankDifferenteBildgebendes VerfahrenPhysikalisches SystemInhalt <Mathematik>SchlüsselverwaltungKontextbezogenes SystemVorzeichen <Mathematik>VersionsverwaltungApp <Programm>ErwartungswertOffene MengeBrowserHook <Programmierung>Hash-AlgorithmusSoftwareentwicklerEinfacher RingBitProzess <Informatik>StapeldateiTwitter <Softwareplattform>Dynamisches SystemCoxeter-GruppeTaskYouTubeDienst <Informatik>E-MailRechenschieberPunktFreewareQuellcodeGebäude <Mathematik>KonditionszahlMultiplikationsoperatorPhysikalische TheorieMomentenproblemGefangenendilemmaGruppenoperationDatenflussDatenverwaltungSystem FFehlermeldungSpannweite <Stochastik>EinflussgrößeInformationsspeicherungPartikelsystemComputeranimation
30:20
Quick-SortMatrizenrechnungMathematikKreisflächeComputeranimationBesprechung/Interview
31:21
Mathematische LogikMultiplikationsoperatorStandardabweichungBildschirmmaskeGüte der AnpassungAdditionDistributionenraumComputerspielBesprechung/Interview
32:03
DatenverwaltungTaskDreiecksfreier GraphDistributionenraumStabilitätstheorie <Logik>VersionsverwaltungKartesische KoordinatenDifferenteBesprechung/Interview
33:39
Äußere Algebra eines ModulsBitBefehl <Informatik>MomentenproblemDatensatzElektronische PublikationService providerLesezeichen <Internet>PunktSchlussregelSpezifisches VolumenZählenBinärcodeGruppenoperationKonfigurationsraumDienst <Informatik>Cloud ComputingBesprechung/Interview
35:17
PunktAggregatzustandDifferenteSchlussregelStandardabweichungValiditätBildschirmmaskeGüte der AnpassungPhysikalisches SystemKonfigurationsraumDateiformatSchnittmengeBesprechung/Interview
36:36
VerkehrsinformationSoftwareentwicklerPunktProzess <Informatik>SchnittmengeProdukt <Mathematik>Besprechung/Interview
37:25
ProgrammierumgebungFunktionalQuick-SortBitSpeicherabzugKlon <Mathematik>MultiplikationsoperatorProdukt <Mathematik>Metropolitan area networkProzess <Informatik>IntegralEigentliche AbbildungBesprechung/Interview
38:41
VersionsverwaltungBildschirmmaskeRandwertKonfigurationsraumDateiformatBesprechung/Interview
39:31
Produkt <Mathematik>DatenflussBitSoftwareentwicklerPunktBesprechung/Interview
40:09
PunktSichtenkonzeptHilfesystemWürfelBeanspruchungRechter WinkelPunktwolkeBesprechung/Interview
40:43
Metropolitan area networkDatenverwaltungGüte der AnpassungTypentheorieMatrizenrechnungKrümmungsmaßRPCBesprechung/Interview
41:32
Besprechung/InterviewComputeranimation
Transkript: Englisch(automatisch erzeugt)
00:44
Good day, good afternoon, good morning, wherever you are. Welcome to my talk about running trusted payloads with Nomad and Waypoint. Let's first introduce myself a little bit. As I said, I'm Frank Vogler and I actually went to university to become a molecular biologist. That seems far away from operations and engineering,
01:03
but it was a quite gradual slide down to the dark part as my colleagues used to say. For a biologist that went to buy an implementation or developer, getting three to 500,000 data points a week that we were generating.
01:21
In an academic world, the developer is also the server guy, so that's how I picked up config management and operation skills. Now, for the last decade or so, I've been in operations and I'm currently a DevOps cloud engineer at a Dutch consultancy company called The Factory. Okay, so to start off with my talk,
01:44
I would like you to ponder the statement that's currently on the screen. So imagine the situation, it's Monday morning, it's stand-up time, and the intern just proudly announced that he solved the problem we've been having
02:02
with our main product, our moneymaker, by fixing the database connection. And I'm probably right when your reaction right now is similar to the person that's on the screen. It was also my reaction on the screen. So as a background, this is pretty much a story
02:22
of my youthful history as an operations guy. So this actually happened, wasn't the database connection we were discussing, but yeah, a fairly junior person quite proudly announced that he fixed
02:40
something in our main product that to most of the people in the room didn't seem related to the problem we were facing. So first of all, everybody's gut feeling was, why did you deploy on your own? Why did you deploy on Fridays? We don't do that here.
03:03
But instead of scolding with my young colleague, I, me and the other people in the room actually started thinking, why don't we deploy on Fridays? Like, why was there nothing in place that would have, so the product was still fine, we were still making money.
03:20
So what would we need to have in place to make sure that the code that our intern pushed out of the goodness of his heart? Like, what do we need in place to make sure that that doesn't cause any problems? Or in a worst case scenario,
03:40
even a bad actor inside or outside our company, if you were to be able to push code to us, how do we prevent bad code or mistakenly bad code, let's say that way into production? And of course, when we looked at this code,
04:02
the case for him didn't make it any better. I'm not showing the actual code, this is actually code. This is valid code, it's functional code. It's code that creates Mandelbrot. A Mandelbrot is a type of image
04:21
and this code is shaped like a Mandelbrot. So it's a Mandelbrot shaped code creating Mandelbrot. It's actually a winner of a competition to create the most pure code and they seem to be winning, at least in my case, but the code the intern produced wasn't as obscure as this, of course,
04:41
but it was hard to read, it was hard to follow. We weren't exactly sure that it actually had any effect or if it had a good effect. So that didn't help, right? So we had to have a conversation about, so first thing we did is lock everything down
05:02
and that's not necessarily a bad thing, but of course you made a mistake of shutting it down too much. So it became the impediment and people started to try getting around it. So closing down on yourself is not always the right option it's making sure that people can do their job
05:23
in the most secure way possible. So because right, we're in the business of, well, selling stuff or servicing people, but in the end we need to get our product in front of people. And to do that, we need to have confidence
05:43
or trust in production. So we need to make sure that everything that we do or put in between the developer station and production or a customer is to the benefit of giving us the confidence that it will work.
06:02
So I borrowed here a definition of chaos engineer, which is an advanced feature of this all, of testing, so getting confidence in production. So we need to open it up, but we need to have a process in place that will provide us confidence so that we can allow people to push code
06:23
or push, well, we don't push code for the sake of pushing code, now we push code for the sake of providing new features. So we should be able to release new features in that price. So first step to gain trust, of course, is adding tests, both unit tests,
06:40
so small tests that will make sure that the code will run, well, operates in a way that we think it does. Then we'll add a layer of integration test, basically exposing it to the wild world for the first time and making sure that assumptions it makes about how the world looks like actually are in place.
07:01
And then, of course, system integration test and end-to-end is actually a surpassing phase of the same concept. We'll open up our system to more outside information, outside places that have different contracts or different promises and how our code acts with those contracts
07:23
or promises from, if you're a fan of promise theory, right, because a system can say that it's supposed to do something, like I'll be cleaning my room, but our code will need to validate that the code, the room, is actually clean.
07:40
If you ask my 16-year-old self, did you clean your room? The answer, of course, is yes, but the truth might be a bit far, far away from my answer. And so your code and your test node only need to test the happy path and it needs to also make sure that it tests the failure path
08:01
or the dark side or whatever you wanna call it. So adding testing, just to make sure that our developer, while our intern doesn't produce unlegible code again, we introduced code reviews or a two-man, four-eyes principle.
08:22
All right, so we always made sure that before code can go to production, at some point during the process, someone else has looked at it, not only for code quality purposes, but also nefarious reasons.
08:41
Not that we suspect anybody from being an actor, bad actor, but there's a possibility, especially when you're in more advanced or high-cap capital places like banks, for instance.
09:01
So put it in place, but make sure that it's helpful, that helpful in a way that it creates trust, it creates better developers, but make sure that the process is not blocking in any way. So make sure that you'll have enough people
09:22
to review your code, that it happens quickly enough. Most of the time, your CI CD system can also help you out. And talking about CI CD and making it simpler, I'm used to a lot of CI CD tools, Jenkins, GitLab Runners,
09:41
Travis CI, GitLab Actions. Most of the time, you'll be allowed to write freeform pipelines. That's good because I can do basically anything I want, but also going from project to project, it makes it difficult. So everybody writes their pipelines
10:01
in a slightly different way, naming things is hard. So steps will inevitably be called different in every pipeline, although they do the same thing. And this is where HashiCorp's Waypoint comes in and it's a fairly new product they released. And it's a, to me, so it's an excellent tool that could help us formalize
10:26
way we deploy things. And that's because it has a built-in step naming convention. Waypoint, the pipelines are written in the HashiCorp configuration language. People that are used to the HashiCorp products suite
10:43
are, should be fairly familiar with it. Packer uses it, Terraform uses it. So not only, now you have the shared DSL, domain-specific language across all your products. So not only the configuration management people
11:02
or your ops, traditional ops people, all write using the same code, but in Terraform, but also the developers that are deploying, creating their own pipelines will now use HDL. And it's easily extendable by having a plugin framework. So if anything is missing, you can quite easily add it yourself
11:23
or hopefully someone in the community will have had the same problem you have currently and will have written a plugin already. So what does it look like? What did I mention already? It has a set nomenclature for pipeline setups. So here was an example product for a project called Lorem Ipsum,
11:44
and it will have three standard steps. It will have a build step, it will have a deploy step, and it will have a separate release step, which is nice, which is you don't see in too many CI-CD tools immediately, so you have to shoehorn it in, but in Waypoint, it's already built in.
12:01
So you can deploy something, but you don't expose it to your user set, which is a very powerful tool you have in your toolbox to be able to make sure that everything is ready and then you deploy it to customers instead of having to show them a, sorry, we're down for maintenance screen for a while.
12:22
So of course, your CI-CD, there's a lot of places that you'll have need for secrets or passwords. And a lot of tools, the first initial way to do it is hard-coding it into code, basically means there is now a plain text password
12:42
somewhere in a config file that basically the entire company has access to. So you can, of course, as you did initially, you can restrict it, but there's another tool that HashiCorp provides, which is Volt, which is their secrets management tool, and it's open source. It can help you with your tokens.
13:04
It can save passwords. You can retrieve passwords. It can help you do certificate management and it even can help you. And that's why I'm focusing on right now is what I call password as a service. So Volt has the ability to create short-term
13:23
SSH passwords or MySQL passwords, and there's several others, but that way you no longer need to hard-code a password somewhere. Now you have just a step in your process that says, I need to deploy something to a new server for which I need SSH. Please generate me a SSH key
13:42
that's valid for the next 10 minutes. It auto-expires, so there's no interactions. There's no lingering passwords in the system that happen to be on everybody's post-its on the screen, because that's your get out of jail card now. Volt now creates the passwords.
14:01
You use it for 10 minutes and it automatically gets removed, which is brilliant. This is where we are in the process of internal. So what I call internal, this is the internal process where one writes code where we build it, we check it,
14:22
we deploy it to customers, but there's other ways of people, well, bad actors, so it can involve the system. So from now on, what I'll be discussing is how can we prevent bad actors
14:40
from interacting with our system or manipulating us in doing something or deploying something that we didn't want to. So before was intentional, unintentional mistakes. From here on in, we'll be discussing intentional bad actors.
15:03
So first of all is an advice I got from one of the CTOs I worked for, do not trust the internet. His version includes profanity, but for the sake of this presentation, I won't. So I'll stick with do not trust the internet. And I think that's annoying
15:24
because we want to stand on the shoulder of giants, which has pros and cons. And we'll be discussing a few of those in the next slide. When we write code in either our node, our PHP, our Ruby, great people have gone before us
15:41
and they've written very good packages. So we wanna use npm install, we wanna use composer install so that we spend our time writing the code that we need or that we make money with and everything else will pull in as a dependency for the boring stuff
16:01
that nobody's interested in writing yet another objects, mapper for any databases, right? Use the ones that are used, use the ones that people are frequently using so the bugs will be out, more people have looked at the code. But that brings us in trouble
16:22
because as we discussed in the previous slides, well, actually let me introduce the concept right now. So the concept of supply chain attacks. So people attacking the bits outside your own infrastructure
16:41
and you have wish you have no control over, right? The npm.org is not under your control. So if people manage to remove packages from npm, you're in trouble, right? You have continuity problems because you happened before a developer got mad
17:02
for some reason and he pulled all these packages, which happened, a lot of other packages happened to depend on and which other packages happened to depend on. So basically half the team tech couldn't deploy on a Friday. Didn't want, not only didn't want to, but literally couldn't.
17:20
So you have a continuity problem. And that's actually in this scenario is the best scenario to have because you can't deploy it. But what if now, if someone managed to hack into someone's npm account or RubyGems account or their Maven account and managed to inject that code.
17:42
Crypto miners come to mind if they're not actually interested in ruining your product, but just your reputation by using everybody's computer to mine code on. You can say that's far-fetched, but it isn't.
18:01
If you Google supply chain attack, then your favorite programming language you'll find that even in the last year, pretty much all of the languages that have a central package repository have been either attacked
18:20
or successfully attacked. So you should have a way of not trusting the internet, which basically means pull everything inside. So have your own operating system repository. So run a pulp for your RPM stuff, for your Debian stuff. Pull in every base image you're interested in,
18:42
run it in your own image repositories, pull in, as I said, make a own cache of npm packages, Ruby packages, use tools like artifactory to pull everything in. So you no longer have a continuity problem. So because if the upstream provider pulls it out,
19:03
you still have the local version, but it also gives you the opportunity to create what's called a software bill of materials. It's something that came out of corporate America, but it's very, very convenient to also do it. It's built, it will take a little bit of time,
19:24
but it will teach you several things. First of all, it will actually teach you what is included in the first place. You think from a high level perspective, you know, because you'll have a Ruby gems file,
19:41
or whatever the equivalent is in all the other languages, but those packages have dependencies, which have dependencies, which have dependency. So especially if we look at the node community, lots of packages are split up in very small components.
20:00
So you'll end up pulling in easily a couple of hundred packages, which is harder to keep track of, unless you spend time creating a software bill of materials. So I have a list of what is included, what they include, and so on and so forth. And that also gives you the opportunity
20:21
to scan for unwanted licenses. For instance, if you have a commercial product that you ship, then certain open source licenses might not be what you want to use. So it makes it easy to scan for licenses. It also makes it easy to scan the packages that you have in your local repository,
20:41
or the images you have in your local repository for problems, vulnerabilities, and other unwanted things. If we look further in how we could do this, the two ones I wanna highlight today is Clara, which is an open source tool to scan images, and SNCC, which also does images,
21:02
but can also scan packages. So if we look at the Waypoint pipeline, we can easily extend the build phase by a before and an after hook. And in this particular example,
21:20
I'm making sure that before anything happens, I'm scanning and doing a SNCC scan on my local repo and my image so that whatever comes in is clean, is good, is usable. Because otherwise when we put garbage in on the one end, it's never gonna get better, can only degrade.
21:42
So from the start, we start scanning, we start doing validation. Taking this further, so when we scan it in the beginning, we're not necessarily sure that whatever ends up on the other end is the same thing. So we're not gonna promote code,
22:02
but we're gonna promote artifacts. And what do I mean with this is do not use, and it's easy, some CI CD tools even promote it, but don't do a merge from acceptance into a branch called production and deploy that.
22:20
That brings several problems, especially when you do the NPM install on production and this angry developer has moved this package, then your continuity problem actually becomes downtime because at that point you can no longer deploy. So if you deploys take longer,
22:40
so you're not completely in control at that point. So make sure that you have a build phase, create an artifact, either an operating system, a native package like RPR, which gives you lots of opportunities to do LCM, or as more modern, you need to stick it in a Docker image
23:04
and promote the Docker image through your Docker repos. So make sure that there's a process or flow in place that doesn't expose your container directly into production,
23:21
but make sure that we can first use it in test and have a process in place that either tags it different or moves it completely into different Docker registry. For that, I wanna introduce Nomad, so we'll have them. For my process, I decided to use Docker.
23:40
Friends don't let friends use Kubernetes because it's great, it's cool when it works, it's terrible when it breaks down and it's basically reinstalled to fix it. On the other hand, Nomads is a very cool tool to do dynamic workloads, sketching. It can do batch jobs, it can do containerized jobs, it actually, which we'll focus on, but it can also do a non-containerized application.
24:02
So if you have a Java jar for you, again, an artifact, we can easily run it. It has console and vault integrations, so we have basically service discovery for free, we have secret management for free, but it also has a built-in token-based access setup. So we can grant people the permission
24:22
to deploy into certain places and again, as I mentioned before, it uses, well, not specifically about Nomad, but it also uses the HDL configuration language, so we have a shared language between that, all our tools that we've been using. What does that look like?
24:41
If we want to deploy our lower MFCM job, we'll have something called a job, which has a group, which exposes my static port on 8080. It has a task of type Docker, and it uses my alarm MFCM image that we've created before and it exposes it on the port that we defined.
25:06
Most of us will trust our upstream providers, say Docker, say AWS registry or the Azure registry, but it has the same problems as I mentioned before.
25:21
We do not trust the internet, so things can even be compromised at the big players. I'm not saying it happens, I'm not saying it will happen, but it's a risk, especially when you run it yourself. So we need to start looking at signing our packages
25:46
for RPMs, we've been doing it for 20 years or more, so there's an established workflow, but in the next couple of slides, I wanna look at how we can add trust to our images
26:03
by signing our images. Here, I'm using a built-in, well, not a built-in, I'm using a image registry which is open source, which is part of the CNCF, which has Docker trust built in. Unfortunately, actually Amazon does not.
26:23
So if you're looking for something open source, have a look at our browser. Trust, content trust, basically in the context of Docker means we wanna sign our images. You probably want to not sign with my key, so first of all, you need to create a key.
26:42
We loaded into Docker saying, I trust my key, and then I start building my images, start signing my images. By signing my images and that, I can actually push them to my registry.
27:00
So my, I'm creating the images, it's now fully signed. Of course, you should make a separate key for your CI CD systems and keep rotating them just like you would an SSH key.
27:21
Because if this starts leaking somewhere, then you're in trouble. So you can also stick them in input. That's an easy way of making sure it doesn't leak. And then on the other hand, on the running side, I can actually, I can expect images saying,
27:44
let's have a look, there's a new release of version one of my demo app. And if something has happened, for instance, of a setting where we are compromised, it's easily, you can easily revoke an image
28:01
and then it's no longer, it will no longer be presented to any of your agents. If you wanna run it, make sure on your Nomad machine, you have a system variable called Docker underscore content underscore trust, and have that set to one. And then you can start pulling in images.
28:20
Of course, you have a problem when your image, by, you know, let me go back a slide. So when you pull in an image, you, and it's not, and you shouldn't be trusted,
28:41
you're basically too late. That's Nomad's already running. You might already have exposed it to your customers. So make sure that in your CI CD, and I was hoping to show a way of doing it in Waypoint already, but I haven't managed is, or directly in Nomad, which does work,
29:00
but it's less convenient because now I need to make sure all my development teams have a certain pre deploy hook in place that basically does this Docker trust inspect. So it needs to make sure that my images is correct, trusted, and it is actually has the hash
29:20
that we're thinking it's gonna deploy. So make sure that you have a little bit CI process surrounding your Waypoint deploy that makes sure that it does validate your input. And that brings us to the end. So I hope that I'm not showing you that we no longer, not only can correctly cuddle our developers
29:45
into helping them create good code, but we also created a ring fence or processes to protect us from bad actors coming in. That's really the end. So thank you for listening. If you wanna talk more, I'm available on YouTube
30:04
or wherever the comments are at the moment, or email me if you really wanna run events, tag me on Twitter at attack.mg. And for people who wanna review this presentation again, by this point I should have uploaded my slides to SlideShare.
30:41
Yes, Bram, thanks for your talk. Hey, so there are some issues with the matrix setup. So I don't think we have all the questions yet. Some people are still frantically typing, but I think one of the questions that came through was basically like, I'm getting myself on the other tab
31:07
because otherwise I hear myself talking, which doesn't work. One of the things that came up is like, what does Waypoint really bring? How does it compare to a traditional CI setup like Jenkins or Circle or whatever?
31:21
So to me, it's their companions. So I would run Waypoint in Jenkins. Waypoint to me really brings standardizations of these steps. So we have standard steps that have standard names that do roughly the things you expect them to do. Whereas in Jenkins, GitLab runners,
31:43
it's basically a free form that allows you to do anything but also screw up anything. Well, now to me this brings standardization, codification of steps and logic to what you're doing.
32:03
Okay, you mentioned a couple of times that you were talking about dependency management packages and things, and the question that popped in is like, isn't this really the task for a distribution? So to me, no. Partly it's a distribution because that's the bottom layer of the things you're running
32:23
but they're in a different release cycle than the packages that land on top of the distro. So some of the stuff you can reuse, you say you're slow releasing Apache's, but some of the stuff, especially if you start looking into,
32:42
the applications are running, that's in a much different cycle than what you're expecting. Well, because you know, an Ubuntu stable comes out every two years. Anything you wanna land on top of that is two years old,
33:00
this is gonna be too old, right? Even if you start looking at Jenkins, for instance, they have an LTS version that comes out at a much quicker release cycle than the two-year cycle. So I don't want to rely fully on OS or distribution packages only.
33:21
You know, look at who's building them. Do we trust them? Yes, if you trust them, we have our own, we have a mirror that we then promote into something that we can actually deploy to. If we, when it gets validated. Mm-hmm, cool.
33:42
So I think you said friends don't let friends use Kubernetes and drag it on the chat that he's also looking into Nomad as an alternative for Kubernetes. Can you tell me a bit more about your experiences with it and your approach? Yes, so I fully agree with my own statement.
34:04
It's much easier to do. It does the, it's the 80-20 rule. It does all the things that are needed to do. It's quickly gaining all the 20 other percent, right? It can do volumes, although I have the same reason, like don't do them in Kubernetes, but it has,
34:23
well, if you buy into the HashiCorp ecosystem, that's all you need, right? You have your service discovery, your service mesh, you have your secrets, that's all beautifully interlinked and all the thing you need is three binaries and three config files and you're done. Whereas Kubernetes, I have never managed
34:42
to build the same stack twice, even with the same config files or there's 10 million tools to do it. You can rely on your most favorite cloud provider to do it but then you probably can't fix it and the only way to fix it is a reinstall.
35:02
So I've never managed to get a comfortable feeling fixing the Kubernetes cluster, whereas I can quite happily fix a Nomad cluster at 3 a.m. in the morning, having just woken up. 3 a.m. in the morning.
35:20
I can already guess what woke you up at that point. One of the earliest question in the chat was like, using AGL to solve something. Is that something that really fits everybody's needs or like the question was, he definitely prefers YAML more.
35:43
Yes, so I'm personally in a state of hating any form of configuration format. So they're now similar to ticketing systems. Everything has something good,
36:01
everything doesn't work for me. But standardizing across my tool set is something that works for me. It's derived of something, a desired state framework, so that also works for me.
36:22
Whereas YAML, having to work out of the alignment is right, you still need some form of validation tool. Did you see the queue talk? No, I did not, unfortunately. Okay, so there's more questions popping up.
36:41
And another question from Draget is, do you give developers SSH access to the machines or do you completely force everyone to go to Waypoint with your own images, secret access to vault, et cetera? The answer to that is yes. So you give developers access? I would give developers access, but I should also teach them
37:01
that it should feel slightly dirty when they have to do it. The tool set should be in place to help us. And if and when things go wrong, that they should be able to act quickly and get some temporary fix in place and then do it properly via the processes we have set up.
37:24
Do you give them funny hats while they hack production? No, but that is an interesting idea. Yeah, I remember a talk some years ago who had actually a terminology to do so, but I totally forgot about it.
37:40
Another question from Victor is, what functionality would you like to see in Waypoint? What is lacking at the moment? To me, Waypoint environment needs to be fixed. So I can, there still is a feeling that you go to production in one go and you need to fudge it a little bit
38:02
to get it into your OTA properly. So I would like to have them spend more time with that. Now they're looking into Nomad Pack, which is sort of a Helm clone to deploy Nomad jobs.
38:25
They used to have Levant, well, it used to be community product that came back into HashiCorp. They ignored it a little bit and now they have Nomad Pack. So I would like to see proper integration between those two.
38:41
I'm trying to scroll through the other questions and definitely people agree that the config format topic is something that needs to have emotional discussions over, well, they claim coffee, but whatever.
39:01
There's also suggestions about giving people access to SSH setup using boundary, but we don't want to turn this into a HashiCorp trade show. So any other final thoughts? What would you tell people
39:22
if they want to start with Waypoint to really look into or not do? What were your mistakes? So it's still basically an emerging thing, Waypoint,
39:41
although developments seem to have slowed down a little bit it's trying really work out what your preferred flow is to get content securely to production and see if this flow works for you instead of going in a whole hog
40:00
and seeing that the rough edges don't particularly work for you. So it's typically still an early adopter point of view? Definitely, it's definitely in my mind an early adopter tool. Would Waypoint help you to actually deploy Cloud Agnostic?
40:23
Like you have a similar workload that you want to deploy to a cube setup on the left and to a Nomad setup on the right. Is that something Waypoint could actually help you with? I believe that's the general goal of they have, but I personally haven't tried it.
40:42
All right. Good. As we run out of questions, I would like to thank you for this talk. And in about three and a half minutes this room will actually go open. So if you stick around,
41:00
there might be people who show up and actually come and ask you questions they either couldn't type in matrix or they didn't dare to ask in rhythm. So, and then there's other talks in the Dev rooms happening. We'll have more talks about Ansible,
41:20
about automation, about immutable infrastructure with flat score, with remote access management and that kind of topics further. So thank you very much, Brown. Thank you.