Early boot provisioning systems
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Anzahl der Teile | 50 | |
| Autor | ||
| Lizenz | CC-Namensnennung 3.0 Unported: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
| Identifikatoren | 10.5446/43095 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
|
All Systems Go! 201843 / 50
1
4
11
12
13
14
16
17
19
23
24
25
29
30
32
34
35
39
40
41
43
44
50
00:00
Minkowski-MetrikSystemstartSystemprogrammierungPunktwolkeBootenBitMereologieInstallation <Informatik>Domain <Netzwerk>SystemstartDienst <Informatik>UnternehmensarchitekturCASE <Informatik>VersionsverwaltungGrenzschichtablösungVektorpotenzialSystemverwaltungSchnitt <Mathematik>SpeicherabzugVerschlingungDistribution <Funktionalanalysis>Besprechung/Interview
01:19
FokalpunktVersionsverwaltungOnline-KatalogSystemprogrammierungMini-DiscSystemstartGasströmungDatenverwaltungPunktwolkeGefangenendilemmaMini-DiscTypentheorieDifferenteSystemverwaltungBitKategorie <Mathematik>Minkowski-MetrikRechter WinkelGamecontrollerDynamisches SystemComputeranimation
01:47
GasströmungDatenverwaltungFokalpunktSystemprogrammierungMetadatenKonfigurationsraumPunktwolkeDistributionstheorieBootenKomponente <Software>Dämon <Informatik>Physikalisches SystemInhalt <Mathematik>RechenwerkSystemstartPunktwolkeMechanismus-Design-TheorieSoftwareKonfigurationsraumZentrische StreckungSchreiben <Datenverarbeitung>Gewicht <Ausgleichsrechnung>BootenCASE <Informatik>TypentheorieMailing-ListeCluster <Rechnernetz>Mixed RealityMetadatenOrdnung <Mathematik>VererbungshierarchiePhysikalisches SystemDienst <Informatik>RechenwerkKategorie <Mathematik>KoordinatenMinkowski-MetrikAlgorithmische ProgrammierungGarbentheorieMAPElektronische PublikationSchlüsselverwaltungVirtuelle MaschineRepository <Informatik>BitMereologieAggregatzustandQuick-SortQuellcodeBimodulDatenverwaltungZusammengesetzte VerteilungGrenzschichtablösungSystemstartDeklarative ProgrammierspracheIterationCloud ComputingKartesische KoordinatenInstallation <Informatik>SchnittmengeXML
04:19
PunktwolkeStellenringModul <Datentyp>Physikalisches SystemRechenwerkEntartung <Mathematik>Dienst <Informatik>Elektronische PublikationCloud ComputingSoftwareKernel <Informatik>BootenGewicht <Ausgleichsrechnung>Generator <Informatik>Microsoft dot netKonfigurationsraumPunktBimodulPunktwolkeStellenringSkriptspracheDefault
05:14
PunktwolkePunktSystemprogrammierungSummengleichungInstallation <Informatik>IndexberechnungInklusion <Mathematik>SkriptspracheSampler <Musikinstrument>Selbst organisierendes SystemImplementierungSystemprogrammBootenSystemstartMini-DiscRechenwerkPartitionsfunktionMathematikNumerische IntegrationGewicht <Ausgleichsrechnung>PunktwolkeMultiplikationsoperatorSoftwareKonfigurationsraumProzess <Informatik>Quick-SortMusterspracheInformationsverarbeitungTaskBootenZweiPunktFolge <Mathematik>DifferentePhysikalisches SystemProgrammfehlerInzidenzalgebraMinkowski-MetrikSkriptspracheRechter WinkelDienst <Informatik>Zahlenbereichp-BlockMechanismus-Design-TheorieInternetworkingSystemstartElektronische PublikationHalbleiterspeicherImplementierungMini-DiscDatenverwaltungDateiverwaltungTypentheorieFahne <Mathematik>AdditionBitrateAnalytische FortsetzungSpeicherabzugDisk-ArrayCASE <Informatik>VererbungshierarchieVirtuelle MaschineServerKontextbezogenes SystemBildschirmmaskeDateiformatHidden-Markov-ModellCluster <Rechnernetz>MereologieKonfigurationsverwaltungWurzel <Mathematik>NetzbetriebssystemReelle ZahlVerschlingungSoftwareentwicklerKernel <Informatik>MAPKonfiguration <Informatik>Nabel <Mathematik>TeilmengeGamecontrollerGarbentheorieExogene VariableAutorisierungRoutingDeklarative ProgrammierspracheAttributierte GrammatikAssemblerSchreiben <Datenverarbeitung>TouchscreenMotion CapturingNichtlinearer OperatorZentrische StreckungKomplex <Algebra>
11:44
Installation <Informatik>Mini-DiscVersionsverwaltungBootenDefaultVisualisierungUnternehmensarchitekturZufallszahlenVirtuelle MaschineATMPhysikalisches SystemSocketKlasse <Mathematik>RechnernetzRechenwerkGeradeStellenringSpeicherabzugEindringerkennungElektronische UnterschriftSpezialrechnerEuler-DiagrammKonfigurationsraumVideokonferenzTreiber <Programm>LastSpielkonsoleMenütechnikWurzel <Mathematik>DateiverwaltungKernel <Informatik>RandomisierungDiskrete-Elemente-MethodeUmwandlungsenthalpieSystemplattformSystemprogrammierungMusterspracheDiskettenlaufwerkSichtenkonzeptPlastikkarteInformationsspeicherungPartitionsfunktionMini-DiscBildgebendes VerfahrenSchlüsselverwaltungBootenInstallation <Informatik>Elektronische PublikationKonfigurationsraumPublic-Key-KryptosystemVirtuelle MaschinePhysikalisches SystemSkriptspracheInstantiierungProdukt <Mathematik>BitKernel <Informatik>ParametersystemMereologieSpeicherabzugPunktMechanismus-Design-TheorieProgrammierumgebungProzess <Informatik>Grundsätze ordnungsmäßiger DatenverarbeitungMotion CapturingMehrrechnersystemCluster <Rechnernetz>Demo <Programm>Zentrische StreckungProgramm/QuellcodeComputeranimationVorlesung/Konferenz
14:27
Mini-DiscDatensichtgerätBootenCD-ROMSpezialrechnerUDP <Protokoll>Prozessfähigkeit <Qualitätsmanagement>InformationsmanagementMIDI <Musikelektronik>RechenwerkSchlussregelGewicht <Ausgleichsrechnung>Hash-AlgorithmusTabelleZehnLokales MinimumTotal <Mathematik>DateisystemBenutzerdefinierte FunktionVideokonferenzDualitätstheoriePolygonnetzKernel <Informatik>DateiverwaltungSystemprogrammierungPasswortElektronische PublikationLoopHackerFunktion <Mathematik>Mailing-ListeSpeicherabzugVorzeichen <Mathematik>Jensen-MaßOverlay-NetzGruppenkeimEinfügungsdämpfungGeradePhysikalisches SystemBootenKernel <Informatik>Physikalisches SystemQuellcodeMinkowski-MetrikDateiverwaltungVerschlingungCodeBimodulExogene VariablePackprogrammEinsSkriptspracheDienst <Informatik>PartitionsfunktionSpeicherschutzp-BlockProgrammierumgebungRepository <Informatik>Case-ModdingRFIDReelle ZahlWurzel <Mathematik>PlastikkarteSchnittmengeBildgebendes VerfahrenSpeicherabzugDisk-ArrayRichtungElektronische PublikationBinärcodeSystemaufrufProgramm/QuellcodeXML
16:25
SkriptspracheBootenMini-DiscBinärcodeSchnittmengeSystemaufrufPhysikalisches SystemBootenMini-Disc
16:48
DatensichtgerätMini-DiscHill-DifferentialgleichungRestklasseVideokonferenzFontDefaultSystemplattformRechnernetzAuflösung <Mathematik>GruppenkeimVolumenvisualisierungVersionsverwaltungTreiber <Programm>Nichtlineares ZuordnungsproblemSCSIPhysikalisches SystemVerschlingungSimplexInterrupt <Informatik>Wurzel <Mathematik>EmulationNP-hartes ProblemStellenringSystemprogrammierungMathematikDienst <Informatik>RechenwerkReelle ZahlKontrollstrukturElektronische PublikationProzess <Informatik>Lastp-BlockVirtuelle RealitätQuantenzustandPasswortSpielkonsolePASS <Programm>Nabel <Mathematik>Kernel <Informatik>SpezialrechnerGleitkommaprozessorStandardabweichungRFIDElektronischer FingerabdruckWeb logGenerator <Informatik>BenutzerprofilKryptologieMaß <Mathematik>BootenInstantiierungRegulärer GraphSystemstartRandomisierungHilfesystemDiskrete-Elemente-MethodeNormalvektorBootenMini-DiscInstantiierungKernel <Informatik>Physikalisches SystemDateiverwaltungMenütechnikPartitionsfunktionKontrollstrukturMultiplikationsoperatorProgrammbibliothekSoftwareDienst <Informatik>Rechter WinkelGenerator <Informatik>Elektronische PublikationPunktReelle ZahlInstallation <Informatik>Wurzel <Mathematik>DefaultParametersystemPoisson-KlammerTopologieComputeranimationVorlesung/KonferenzProgramm/QuellcodeJSON
18:29
NormalvektorDefaultVerzeichnisdienstDiskrete-Elemente-MethodeEindringerkennungKonfigurationsraumDateiverwaltungATMSchnittmengeMini-DiscMAPSoftwareElektronische PublikationSkriptspracheKonfigurationsraum
19:13
Dienst <Informatik>Mini-DiscFahne <Mathematik>MAPZeichenketteWurzel <Mathematik>Zeiger <Informatik>FehlermeldungKonfigurationsraumPhysikalisches SystemKeller <Informatik>Offene MengeEmulationPhysikalisches SystemKonfigurationsraumFirmwareWurzel <Mathematik>SoftwareAusnahmebehandlungPunktwolkeCloud ComputingLastQuellcodeKernel <Informatik>SoundverarbeitungMinkowski-MetrikDateiverwaltungMAPBootenURLDienst <Informatik>MathematikFahne <Mathematik>PartitionsfunktionInstantiierungComputeranimation
20:45
ComputervirusDienst <Informatik>VersionsverwaltungProgrammverifikationMaß <Mathematik>Inhalt <Mathematik>RechenwerkInstallation <Informatik>Physikalisches SystemTypentheorieGeradeElektronische PublikationUmsetzung <Informatik>ValiditätKonfigurationsraumFehlermeldungGarbentheorieComputeranimation
21:06
InformationsspeicherungElektronische PublikationATMInhalt <Mathematik>DateiverwaltungWurzel <Mathematik>MIDI <Musikelektronik>RechnernetzSystemprogrammierungOrdinalzahlDistribution <Funktionalanalysis>Mini-DiscDisk-ArrayPartitionsfunktionVolumenWärmeausdehnungGruppenkeimKonfigurationsraumTypentheorieQuick-SortSchreib-Lese-KopfGarbentheorieURLMinkowski-MetrikPunktBootenElektronische PublikationDisk-ArrayWurzel <Mathematik>InformationsspeicherungVirtuelle MaschineGeradeFormale SemantikDiagrammRechter WinkelCompilerKonfiguration <Informatik>Physikalisches SystemSystemstartSoftwareSchnittmengeCASE <Informatik>Web logMini-DiscÄhnlichkeitsgeometrieCase-ModdingDateiformatPartitionsfunktionElement <Gruppentheorie>SkriptspracheDistributionstheorieMereologieCluster <Rechnernetz>KorrelationsfunktionLesen <Datenverarbeitung>Spezifisches VolumenMusterspracheDateiverwaltungVersionsverwaltungDistribution <Funktionalanalysis>Service providerHardwareGrenzschichtablösungPasswortSchreiben <Datenverarbeitung>TopologieComputeranimation
25:13
Systemprogrammierung
Transkript: Englisch(automatisch erzeugt)
00:06
Hey, everyone. Thanks for showing up. Let's get started. This is the early boot provisioning talk. So today, I want to talk to you guys about provisioners that run during the boot process. So to do that, I'm going to first lay out the provisioning problem we're trying to solve.
00:21
I'll talk about how Cloud Innit initially aimed to solve this problem. I'll then introduce Ignition, talk about how it's different from Cloud Innit, and I'll walk through a typical boot process on container Linux to talk about it. I'll go through a few examples of how you can use it, and I'll wrap up by covering the future potential for early boot provisioning as well as Ignition.
00:41
A bit about me. I'm Dalton Hubbell. I used to work at CoreOS from 2015 till a few months ago. While there, I built various services and installers for doing cluster provisioning. I worked on our tectonic enterprise offering of Kubernetes. These days, I maintain a separate Kubernetes distribution, and that's an informed part of this talk
01:01
because it all uses Ignition and early boot provisioning. Provisioning itself is a super broad domain. It generally covers how to boot many provision, many Linux hosts for some particular use case and for some definition of many. That might involve anything from how to manage and version configurations to how administrators
01:21
are authenticated, do access controls and auditing, stuff like that, cataloging inventory, but I'm scoping this down to just the actual provisioner tool, the thing that gets the right bits in the right places on disk when they're needed in user space. Among provisioners, there's this whole slew of different categories as well.
01:40
There's the sysadmin type tools that just run arbitrary commands on hosts in parallel. We're gonna skip over those. Dynamic configuration management, we're gonna skip over that too. That runs arbitrary commands either in a push or pull-like fashion. Maybe there's a DSL involved, but generally they still do iterative mutations of machines. The hero of our talk is going to be
02:01
the distro-supported mechanisms, CloudNet and Ignition. They're a system where you provide a configuration document to a machine when it's booting, and that's enacted sometime during the boot process to actually produce a fully provisioned Linux host. To define our use case,
02:20
it's gonna be large-scale clusters. For infrastructure cases with clusters, it's increasingly this last category that we're interested in, and it was also this last category that CoreOS was interested in, having sort of a minimal OS that was focused on containers, being package-less, and every host was sort of a dominoed in part of a higher-order cluster.
02:41
Concerns like simplicity and speed and scale were part of why this last category was important, whereas concerns like dynamically reconfiguring applications was not as important because Kubernetes and clusters handle those types of things these days. So let's start with CloudNet. CloudNet's pretty ubiquitous. It's an initializer.
03:01
It's installed on most major Linux distros. It lets users write a CloudConfig YAML document. It describes what they want the system to look like in this mix of declarative and imperative sections. People can set users and SSH keys and install packages and repos and do a little bit of network configuration. CloudNet works by fetching a CloudConfig
03:21
or user data from some metadata service, like a network endpoint or a config drive or some other pluggable data source. It's implemented as a pretty large collection of Python modules, and it runs on nearly every boot in user space. Here's an example CloudConfig document showing that mixture of a declarative write file section
03:42
that says, hey Python module, write this systemd unit to a particular path, and an imperative section, a run command list of arbitrary commands that are run. The order of these sections don't matter. You generally go and read the CloudNet docs to figure out which sections run first. In this example, super important
04:00
that write files happens before run command, so you just read the docs. But to initialize a host to a desired state, CloudNet runs in user space. It runs before networking and also after networking, so specifically it runs in four separate stages and four separate systemd units. Uses a systemd generator. For those who don't know,
04:21
systemd generator is like a small executable. It generates systemd units before any units are ever loaded on the system. It's usually a bash script. The CloudNet system degenerator checks for a CloudNet disabled kernel arg or a CloudNet disabled file. If it finds any of those, it doesn't run, but by default, it does. It enables the CloudNet target,
04:41
which wants these four different services. There's CloudNet local, which looks for a local CloudConfig. It's usually some minimal thing that just sets up networking. There's a CloudNet service. At that point, you have networking. It goes and fetches another CloudConfig, probably from your cloud provider. That, if it contains a network configuration, has to reinitialize networking.
05:02
There's CloudConfig, which runs other modules, and there's CloudFinal, which installs packages and runs arbitrary commands, or at least the run commands. There's also two flavors of, there's boot command and run command. Container links, Linux shipped with CoreOS CloudNet. It was a Go implementation of CloudNet.
05:21
It supported a subset of the CloudConfig document that we just looked at. It was implemented in Go because there was really no interest in shipping Python on the US because it would blow up the image, and Go was pretty popular. It only had to implement a subset because a lot of concepts in CloudNet just weren't relevant. Managing packages was not relevant on a packageless operating system,
05:41
or user management wasn't as big when pretty much everything in a server context uses a user core. This was at a time when configuration management systems were like in their heyday, and CoreOS was pushing this idea that you should just use a CloudNet document, and more than that, we were pushing never changing that CloudNet document.
06:00
It was an early form of immutability of just don't change it. Fast-forwarding into July 2015, CoreOS was vaguely popular. People were writing CloudNet or CloudConfigs to describe their clusters. Infrastructure patterns were evolving. People were starting to use initially Fleet and then Kubernetes, and the scope of what somebody wanted
06:21
to provision a single host to do was shrinking because it was just a member of this bigger cluster. So I would say CloudNet was successful in getting people to provide a configuration document as part of the boot process. It enabled that idea, but there were a number of pain points with CloudNet, so it ran on pretty much every boot.
06:41
They've made some improvements in this recently. I'll give them some credit for that, but at the time, it slowed down boot times a lot, having that run every time. It disincentivized people from using it because the more you wrote, the slower your boots were. It also ran pretty late for various types of provisioning tasks. I mentioned networking earlier. Network configuration is tricky because CloudNet starts.
07:03
It doesn't have networking. It has to load an initial CloudConfig and then it has to set up networking. Your CloudConfig might specify some fancier networking, so you have to restart networking. Simple tasks, like I showed the example of just starting a systemd service. That was way more cumbersome than it had to be because you had to go write the unit
07:20
and then run a sequence of commands. You were mixing declarative and imperative just to start a service, the most basic thing. Sorry. You also have to be really familiar with the different stages of CloudNet. You had to go read the docs for every section to know what ran before what, so it's complex.
07:41
For a lot of users, a ton of users, CloudNet became this mechanism for running arbitrary commands. It's the CloudConfig that looks like this. Basically, it accepts actual bash scripts, which a scary amount of people end up actually doing, or glorified bash, which is what I call this one here. This is really, really problematic
08:02
because it turns out poorly written scripts will prevent your system from actually coming up for very unimportant reasons. Just the other day, there was a case where someone had makedir instead of makedir-p and that prevented a, this caused some outage related to the nodes not being able to actually join a cluster. So simple.
08:21
Also, that example I showed earlier, starting the systemd unit, I don't know if you noticed that dash dash no block flag. That turns out to be super important because if your service is waiting on CloudFinal, like Docker, if you don't have that, then again, your system will hang. So authors have to also be really aware. They have to be cognizant of the fact
08:40
that their CloudConfig is going to be run again on every boot, so makedir versus makedir-p is a great example of that. This was also really problematic because your system would seem to run fine. It would reboot after seven days or something, maybe some auto update, and that's when you would discover an issue. There was this disconnect between a bug in your CloudConfig and the incident
09:01
that it ends up causing. Developers start writing defensively because they start realizing this is a problem, so you end up with partially configured machines. People make things optional. Within orgs, these scripts tend to grow out of control. There are multiple ways of doing things, declarative versus imperative. It also just increases the learning burden on a team
09:21
because suddenly people can call anything. We've reverted to a world where it bashes configuring systems. So enter Ignition in 2015. So Ignition was assigned to Chorus as this replacement for the Chorus Cloud init. It was added into the OS. It was implemented in Go, and it runs from the init RAMFS. It's in early user space.
09:41
So that allows Ignition to fully configure the assist route before it becomes the real route, and before PID1 and user networking have actually started. Like CloudNet, it accepts a configuration document, but very differently. It only allows you to do things declaratively. So it can partition disks, format file systems, assemble rate arrays, write files, write systemd units, mount units,
10:02
but these are all writes, you notice. There's no mechanism that allows you to run arbitrary commands from the init RAMFS. You can write a systemd unit that runs a script in user space, but that's a separate problem of a thing that happens in user space. Ignition happens in the init RAMFS to write files to a disk.
10:21
Any failure to fulfill an Ignition configuration intentionally drops the host into an emergency shell in the init RAMFS. This provides sort of a, in large scale clusters, this is useful because it provides a guarantee that if a system gets into user space, Ignition successfully fulfilled everything in that document. Otherwise, you'd never get there. Prevents bad hosts from ever joining a cluster.
10:41
The way CoreOS integrated Ignition into our distro was so that it would only run on first boot from disk, that forced an immutable style, which is something we were just encouraging before. Let's walk through a container Linux boot process to sort of show how this works. So Ignition runs in early user space.
11:00
What is that? Okay. We're gonna skip that in the interest of time. On container Linux, grub acts as the boot loader. It's responsible for going and picking our kernel, loading it in memory, and actually starting it with the right kernel args, but grub does two super important things on container Linux related to this. First, it has to pick between two user partitions,
11:21
A and B, partitions three and four. That's based on GPT priority attributes. Ignition is totally fine with this system that the user partition can be swapped all the time. This is how auto-updates work. The second is that grub is the integration point that supports Ignition's mechanism to run only on the first boot from disk.
11:41
Let's hop into an example of that. So I actually did a screen capture beforehand, which I hope works. It does work. Thank God. Okay. I'll boot a container Linux instance
12:02
as if it were a bare metal host. I'm doing it in kubectl VM because it's easier to screen capture here. So we're creating a blank kubectl to disk image. I'm gonna start the VM from that ISO. I've sped this up a little bit. I figured you didn't want to actually wait.
12:21
Once it comes up, I'm gonna download a simple Ignition config. That config is just going to say to add my SSH public key to the core users authorized keys. We can look at it.
12:40
So yeah, just add that public key to the core users authorized keys. I'm then going to run the chorus install script. It's just a script that does the actual install to disk. It writes the production chorus image bit for bit to the target disk. And as part of that install, I specify the Ignition config. This is how you do it by hand if you're walking up to a bare metal machine
13:00
with a little USB stick. But in the real world, probably for large scale clusters, in basically the pixie world, you're going to pixie boot all of your bare metal machines in this exact same way into a live container Linux environment. And you would just script out this same install process. Let's take a look at the two partitions that are interesting, partition one and partition six.
13:21
So partition one is the OEM system partition. Inside of it, you'll find a chorus first boot file. That first boot file is actually read by Grub. Yeah, we see the files present after a disk install. If we inspect the Grub config on target here, Grub is actually looking for that first boot file
13:41
and using it to set a kernel argument. Of course, first boot equals detected. So this is the mechanism that ensures that a kernel arg is always going to tell the subsequent systems that this is the first boot from disk. The other partition that we were going to look at was partition six.
14:01
It's the OEM partition. If you're wondering where that ignition config ends up, it's partition six. This is the exact file that we just added. And before I reboot the system, I'm going to modify the Grub config to set an initramfs breakpoint, which will be useful for the next demo. We're setting a breakpoint so that we can halt in the initramfs.
14:24
Yeah, you get the point. All right, let's talk about the initramfs. So the Linux kernel introduced the initramfs in 2.5.46. It's a CPIO archive that gets paired
14:42
with the kernel image, and it contains initialization code to initialize all your devices and mount the real root file system, and this all happens before starting PID-1. Originally, most of this code lived in the kernel itself, but it started to grow and had too many responsibilities, things like it has to go find all the block devices,
15:01
whether those are local or remote, or you start getting into NFS root file systems, getting DHCP leases, setting up networking, knowing about RAID arrays, LVM, decrypting partitions, how do smart cards work. It's a lot of stuff to put in the kernel, so it was moved out. And it runs in this early user space environment.
15:20
It has memory protection and syscalls in Caleb C, and it runs before PID-1. It doesn't have to be POSIX compliant, it's a weird early space world. On container Linux, we used Drackit to prepare the init RFS image in the Chorus SDK. We can quickly, we have to do this again. Maybe this wasn't the best idea.
15:42
We can quickly cork enter into the Chorus SDK. It's just a chroot. We see a set of scripts that help build the distro, and there's a source repo called boot engine. There's a source repo called boot engine that keeps all the Drackit modules and configurations. Drackit's a pretty common tool. Many of you have probably used it before.
16:02
It's a tool that takes a set of modules that specify how to copy files into the final CPIO archive. If we look at the ignition module, this 30 ignition one, we see we call the inst modules directive. It says to copy in the ignition binary, and a couple other useful ones like user add, user mod, make FS.
16:22
Further down, we see it copies in a few system D services and an ignition generator. That's really all it is. It's like a couple binaries. It's not a large set of tools. I think it's actually 10 binaries if you count them that it's calling.
16:43
Let's go ahead and boot that system that I showed earlier. So this is going to be the first boot from disk of our bare metal instance that was set up by doing a disk install somehow, whether pixie or by hand. Grub's going to select the default menu entry. It's going to find the right user partition
17:01
based on GPT attributes. We're going to drop into the Drackit init run FS right before all the interesting stuff happens because I put that break point in last time. We can show the OS release. We see it's Drackit. In this world, we have a early system D available to us. We have journal D,
17:21
and we'll soon have system D network D. It's not turned on yet. We can see at this point, the real root file system is mounted at sysroot. We can check out the system D generator for ignition. It's called ignition generator, user lib system D system generators. If we look through it,
17:41
we can see it's actually looking for that kernel argument, the first boot kernel argument, and it's going to add ignition disks and ignition file services, as well as an ignition setup service that's needed for Pixy and ISO. At the very bottom,
18:01
it's going to add, it writes the OEM to a file called run ignition. The OEM is easy to, or wow, I've lost my place.
18:24
We can take a look at that ignition setup. It's going to look for a device that's labeled for the OEM. You can look at the script. It's going to actually mount the OEM disk and copy the ignition config off. This is how the ignition config gets in place,
18:41
and shortly after this, I'm going to actually manually invoke the ignition setup, which is going to trigger ignition setup to run. It'll bring up networking, and then it'll run the two stages of ignition, ignition files and disks. I wish this were a little more speedy.
19:04
So it runs those, sets up networking, runs the two stages. The two stages will actually go and configure the sysroot. So let's review what we've seen. There's a system D generator. It looks for the first boot kernel arg to decide whether the ignition services are wanted,
19:22
copies the ignition config off the OEM partition, and it runs the two ignition stages. Those can happily have networking because we're not modifying the actual running system. They won't have an effect until user space starts. That covers bare metal. For cloud instances, how do we fetch the ignition config? Well, ignition is passed in the root location,
19:41
the stage to run, and the OEM name. It declares an engine, and the engine is run. This fetcher is actually specific to the OEM. It loads a base configuration, which is like an empty struct, loads a system configuration, which is usually just enough to set up a root file system
20:02
and then it acquires an ignition config based on that fetcher, so it's specific to the OEM, and it merges them all together. The fetcher, there's a different one for each cloud provider. They fetch from the canonical source. You can imagine all the cloud providers fetch from a network endpoint, except for Azure, which uses a config drive, I believe.
20:20
The interesting one is QMU KVM. You can use the firmware config flag, which allows you to pass a blob of data into the guest VM, and there's a canonical location that is used for ignition. There's a fetcher that will go and read from that. So that covers bare metal cloud and hypervisors. Let's change gears a little bit and look at the configuration document.
20:41
You can see the ignition itself is a JSON configuration document. It is not very human-friendly at all. That's by design. The idea was that people would use tools to actually evaluate this to get validation errors and things out. It also simplified a couple of type conversion issues. In practice, people end up using a YAML document
21:00
called a container Linux config. That'll have to change names with the OS. It has a couple of top-level sections, and some niceties like allowing you to put files in line and check these into source control. There is a section like systemd that writes a systemd file. There's nothing really advanced here. It's just writing this file into the sysroot in the correct location.
21:20
Storage does similar things to set up file systems, write files to a particular location. It can partition disks as well. Password d just invokes user add, user mod with like dash dash root set to the sysroot. So again, it's a file modification. Here's a network configuration. Again, this just writes a network d config file to the right location.
21:41
The theme of all these is that we're not actually contending with a live running system. We're just writing files that will be read once PID1 starts. Over the years, there were a bunch of tools that were integrated to work with Coros Ignition. There was CT, it was a command line tool that converts from that YAML format to the JSON format. Terraform provider CT, the Terraform version of that.
22:02
Matchbox, the thing that serves files to pixie-booting bare-metal machines. Typhoon is my own Kubernetes distribution that has container Linux and flat-car support for Ignition. Ignition enables a pretty unique feature there, actually. Earlier I mentioned that Ignition documents are merged together, there's an additive merge.
22:22
That turns out to be a really useful concept for customizability for a Kubernetes distro. You can define a container Linux config that set up a Kubernetes cluster, but then you can also accept user-defined configurations and then merge them all together. So in this example, there's a large cluster, but one particular node might have some separate network set up. Maybe it's a piece of hardware
22:41
that has like a separate set of NICs. What's in store for Ignition and early boot provisioning? To wrap up, Ignition has this concept of file system reuse, and there's a lot you can read about the semantics around it. There's ongoing interest in how do we get that same type of reuse available for like RAID arrays,
23:00
for partition reuse and disk reuse. This is useful if you wanna like fully re-provision a cluster, but you maybe wanna preserve some sort of RAID array that's on some set of disks. Often the case in Kubernetes clusters. There's work to get SELinux-enabled hosts working. No correlation to Red Hat, I'm sure. I think that's actually already in there.
23:22
There's work to get RPM OS tree. Of course, Red Hat's influence again. I think that's related to installing RPM packages in the Sysroot. You can imagine how that would be very similar. It's a similar pattern. There's work to get LVM and encrypted volumes in there. Since Chorus was acquired by Red Hat, that means Ignition's actually coming into more distros that was confirmed in blog posts.
23:42
I drew this amazing diagram here to show. So Chorus container Linux is continuing on for two years. This is my understanding of it. I no longer work for Chorus or Red Hat, just clarification. This is my understanding. Continues on as container Linux. Flatcar forked it. They also have Ignition support and Fedora and Red Hat Chorus will also have Ignition support
24:00
because that was announced. If you wanna compile Ignition yourself and put it in your own init or fs, it only uses these 10 tools. There are compiler options where you can, if you have a different path in your personal init or fs, you could just change these and shove Ignition in there and see how it works. To recap, CloudNet pushed people towards this idea
24:22
of having a configuration document and doing the provisioning as part of the boot process. It had a couple of pain points. Ignition tried to solve those pain points and there were a few key insights, like three of them. Running from the init or fs in early user space is really advantageous. We integrated with Grub so that Ignition would only run on first boot
24:40
to enforce this immutable style. And we permitted only declarative elements in the configuration document, so no arbitrary scripting. And finally, Ignition and the whole concept of early boot provisioning are going to become available on more distributions. I think that's pretty exciting. And maybe you can try it out yourself.
25:02
With that, thank you all for attending. And I think I'm in the last talk for the whole thing. Yay.
Empfehlungen
Serie mit 13 Medien
Serie mit 14 Medien