Building initrds in a new way
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 542 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/61544 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 2023212 / 542
2
5
10
14
15
16
22
24
27
29
31
36
43
48
56
63
74
78
83
87
89
95
96
99
104
106
107
117
119
121
122
125
126
128
130
132
134
135
136
141
143
146
148
152
155
157
159
161
165
166
168
170
173
176
180
181
185
191
194
196
197
198
199
206
207
209
210
211
212
216
219
220
227
228
229
231
232
233
236
250
252
256
258
260
263
264
267
271
273
275
276
278
282
286
292
293
298
299
300
302
312
316
321
322
324
339
341
342
343
344
351
352
354
355
356
357
359
369
370
372
373
376
378
379
380
382
383
387
390
394
395
401
405
406
410
411
413
415
416
421
426
430
437
438
440
441
443
444
445
446
448
449
450
451
458
464
468
472
475
476
479
481
493
494
498
499
502
509
513
516
517
520
522
524
525
531
534
535
537
538
541
00:00
BuildingComputer-generated imageryBootingMeasurementCodePhase transitionSystem programmingMechanism designIdentity managementPhysical systemExtension (kinesiology)RootkitSimilarity (geometry)InformationPasswordPublic key certificateElectric currentComputer fileKernel (computing)BefehlsprozessorLogicModul <Datentyp>Scripting languageIntegrated development environmentLocal ringComplex (psychology)Proper mapDistribution (mathematics)Projective planeMultiplication signRootkitDeclarative programmingKernel (computing)Data managementCodeBootingMeasurementFile systemMetadataOperating systemVirtual machineLibrary (computing)Data compressionPoint (geometry)MereologyScaling (geometry)LaptopLocal ringFunctional (mathematics)Field extensionSet (mathematics)Compact spaceConfiguration spaceMechanism designStructural loadPartition (number theory)Computer fileMiniDiscElectronic signatureHash functionGraphics tabletMaxima and minimaIntegrated development environmentSpacetimeBuildingPhase transitionState of matterPhysical systemNeuroinformatikFigurate numberDirectory serviceMedical imaging2 (number)Latent heatDifferent (Kate Ryan album)Computer configurationDressing (medical)Group actionKey (cryptography)DiagramComputer animation
09:06
Computer-generated imageryCodePhysical systemMechanism designBuildingConfiguration spacePartition (number theory)Source codeComputer fileSeries (mathematics)Kernel (computing)Installation artImage resolutionIntegrated development environmentScripting languageLink (knot theory)LogicDemonMathematicsBootingSuite (music)Type theoryData storage deviceAreaCASE <Informatik>Patch (Unix)Software developerBuildingScripting languageStructural loadLocal ringLink (knot theory)WeightCompact spaceDirection (geometry)Physical systemFormal verificationElectronic signatureField extensionKernel (computing)CodeExtension (kinesiology)Set (mathematics)Distribution (mathematics)Medical imagingFile formatConfiguration spacePlanningElectronic mailing listMathematicsComputer filePlug-in (computing)Module (mathematics)Different (Kate Ryan album)Multiplication signDefault (computer science)Projective planeTerm (mathematics)File archiverMechanism designResultantPartition (number theory)Table (information)Data managementDemonVideo gameError messageAuthenticationRootkitLibrary (computing)Normal (geometry)MiniDiscEncryptionFunctional (mathematics)Overhead (computing)Stack (abstract data type)LaptopImage resolutionSpacetimeLatent heatComputer animation
18:05
Link (knot theory)Physical systemSoftwareCryptographySystem programmingKernel (computing)BuildingKernel (computing)Structural loadRoot systemModule (mathematics)Default (computer science)Normal distributionInstance (computer science)SoftwareLatent heatFiber (mathematics)Computer configurationInformationComputer fileSoftware bugConfiguration spaceCASE <Informatik>Local ringVideo gameStandard deviationEndliche ModelltheorieField extensionPhysical systemExtension (kinesiology)BootingThermodynamisches SystemPatch (Unix)DataflowDistribution (mathematics)Computer hardwarePoint (geometry)Right angleView (database)Traffic reportingMereologyDifferent (Kate Ryan album)Electronic signatureBitComputer animation
22:13
Core dumpProjective planeState of matterLine (geometry)CodeModule (mathematics)RootkitComputer animation
23:33
Program flowchart
Transcript: English(auto-generated)
00:06
Please sit down if you can. So the next speaker is speaking of yen dress keys smack and He will talk about MK OSI in it Adi a new project to build in it Adi's
00:24
from system packages And yeah, yeah, so My talk builds on previous talks, so My name is Bishak. I work in Red Hat in the plumbers group And I work on system D and fedora
00:44
Rasto rpm rpm auto spec Build tools and stuff like that and So we here today about a new approach to delivering the kernel and the
01:04
user space the root file system to Computers and I mean all the stuff that was that was mentioned today so secure boot to something to Trust your code PCR measurements boot phases
01:23
Locking secrets to to the PCR state This creates a situation where we should think How we put how we building are these? And I think it's a good opportunity to kind of throw away a lot of
01:43
All the approaches and use a new approach and Doing the things the way that I that I am talking about today Would have been very hard a few years ago because we didn't have
02:00
those mechanisms that we have like right now like Credentials and system extensions, so so right look I talked about system extensions, so the Compact disk images that carry a
02:21
File system and a one partition the inverted data in another partition and a signature for the root hash of the the inverted data In a third partition And this is squished into a an image with like minimal padding so it's actually it's it sounds kind of awful, but it's
02:47
basically just a file system image with some metadata and it can be that can be trusted and This those those those tools allow us to do things in a
03:01
Completely different fashion than we used to do them in the past. So what do we do now? I mean this varies between distributions, but the the the approach is generally that on a general-purpose distribution like Fedora The user wants to have an interd, so they Scrape some files of the file system. Whatever is installed there sometimes with some local modifications sometimes not
03:26
you use LDD to figure out what libraries should be loaded and Whenever there are extra files that need to be put in the interd or somebody has to remember to do that so essentially, we duplicate the packaging layer
03:43
and We do it on every user machine. So this this this costs time during upgrades. It's actually quite noticeable because of compression and So this was before we boot it and after we have put it into the interd
04:01
Generally For example, the Fedora interd is they have systemd, but they also have lots of extra functionality added that came Before systemd was there and over time dysfunctionally has been moving into systemd And now we are at the point where it's completely useless. I mean the part that is
04:24
Apart from systemd is not not useful because we could just get rid of it and because it is implemented by systemd in a better way In general and Well people here, okay now we should do something some kind of access
04:44
File system in the interd. Oh, let's write some bash to do it and why I mean We should just do the same thing. We do in the host file system and use proper tools and if those tools are Not good enough then we should fix them so that it's nice to use them in the host file system and
05:02
They also nicer to use in the interd and It's like a legacy that this interd environment Used to be much different But we use systemd and systemd does the setup where it sets up slash proc and slash dev and mounts every thing that needs to be mounted and the
05:25
Interd environment In the interd it doesn't have to be the fact that it's different from from a real system. It's just I mean something that we should get rid of and So we have this Duplication where we have the systemd way to do things and the non systemd way to do things in parallel
05:43
it just adds complexity and doesn't it's not beneficial and And Everybody does the interd in a very very different way every distribution and even some distributions have multiple ways
06:02
I Know that one of the goals of trackwood was to unify the approach in between distributions didn't really work So this is another approach. We'll see in ten years So, okay, so we sign stuff If we sign the kernel but not the interd then we are just
06:22
Pretending right? It's a waste of time We need to sign both but in general users want to have the thing and the users don't want to play with local key management It should be signed by the distro if it's signed by the distro then the interd must be also build by the distro
06:41
So all this functionality that we have to inject things into the ID based on local configuration Well, we cannot use it So Essentially the idea is that okay if we are going to be to move the whole build of the interd Into distro infrastructure and build it like a package then let's do it in a clean way
07:04
and For me this means taking Declarative set of distribution package names Letting the distribution package manager figure out all the dependencies and then using the distribution package manager to put the files in a
07:21
In a directory and then just zipping it up into any interd So Before I talk about the like the specifics of how how to do this Let's talk about some some problems that immediately appear, right?
07:44
If we try to build one interd for let's say whole fedora then we will end up with this gigantic blob that will take 60 seconds to load it Whenever the kernel boots, this is not nice. So we need
08:03
Different interds for different people and One option is to just build multiple variants and we definitely plan to do this For example like Simplified interd that works for VMs that only has some some basic stuff that you need in the VM and no other drivers
08:23
and no other tools And like, you know, I know one for laptops and so on but this only scales so far you we can have you know, maybe five variants, but anything more than that would be annoying and The other approach is to use systemd extensions, so the idea would be that
08:45
You have the basic interd and let's say you you want I know SSH D in your interd and you Install this additional extension and I should mention what happens with trust here. So
09:02
The the bootloader Verifies the the kernel and the interd before loading it and then After the interd has been loaded and we want to use we want to the interd code loads the extension so it
09:22
checks the signature of the extension before loading it so systemd extensions are a mechanism to add code in a way that it is trusted before I mean the trust is checked before it is used And if we use Yuki-fies if you use UKI's
09:43
We need to some way to inject configuration into the well, I Mean we build an image that's supposed to work everywhere So it cannot have local configuration and we need to deal with this issue somehow And one one approach is to use the just out of discovery of partitions and not have any local configuration
10:03
Which is nice if we can make it work but More general approaches to use credentials and to inject a configuration that is that has been signed and and bound somehow to the local system
10:21
using systemd credentials and I Would say that this is an area of active research because I don't think that anybody really knows how this is supposed to work in details and I Wanted to mention that if we build interds from from packages we have build reproducibility
10:44
Because we have an exact list of packages that was used and we can download them again and unpack them again in the same I mean the exact same way and we should have bit-for-bit identical
11:00
Result which is good for for checking that the ID was Put together correctly, but it's also very useful if you want to build some system extension afterwards because you build the extension by Adding additional stuff and then taking the difference between the old image and the new image
11:20
And the the tool that we are using for this Is well, the project is called in case I interd because it takes M qsi which is a very simple Tool that takes a List of packages and calls the the package manager to
11:44
Download the packages and put them into an image and It does all the things that we happen to need so it supports GPT tables and the invariably and signatures so we can do some extensions
12:01
And it can also do archives and this is what you need for any term FS format and Mqsi itself is just a set of configuration scripts for our configuration files for mqsi right now only for fedora, but
12:22
the concept carries Very cleanly into other distributions. So I think that if it works in fedora, maybe other people will pick it up, too So just a list of packages and some some local some some tweaks to turn the install packages into an interd
12:41
and The same for system extensions and Well in general the plan is to do it on the distro side But right now this is this hasn't been hasn't happened yet So so we have a kernel install plug-in and you update the kernel and you very slowly build the interd from packages on the local System, it's not very efficient, but it works
13:02
surprisingly stably And for further a39 we want to propose Changes to the to the build system to allow interds to be built in this way in the In the build system and deliver these packages for people who want to try it out. I think it's like years from being
13:23
the default if ever and So this works, I mean it works without too much issue The interds are bigger, but they're not, you know, infinitely bigger. They're maybe just twice as big
13:42
and The biggest difference surprisingly comes from kernel modules Because what trackwood and other interd builders do they they select a specific list of Modules kernel modules that are needed for the local system I wanted to avoid this I wanted to take all the modules from a kernel package and just put them in without
14:08
Knowledge about specific modules. I think that this is not feasible. We'll have to do some some kind of filtering to But the code itself
14:22
There's very little difference in the size of executables and libraries Installed into the interd and this is because the code that we use in the interd is the same Code from the host file system, so
14:41
It has the same library dependencies And you need to put the same set of libraries and actually most of the space is taken by libraries And because the code the functionality has been moving more and more into shared libraries because we build more complicated stacks So this means that like the in principle the the size overhead is not too big and can be reduced
15:04
And I mean did this works in some cases, but like for simple cases for laptops and for some types of storage, but Certainly not for everything And well, what do we get we have less things?
15:20
We use the package dependency resolution mechanism, so we don't duplicate packaging We don't need to care about installation because we have rpm to do it for us or whatever We have purposeful builds because we don't take files from the host and Everybody has the same image, which is important for people trying to debug errors reported by users and
15:44
And well if we build things centrally you can sign them And we use systemd and just get rid of the extra stuff, so our life is simpler And you know there's like a common set of things That people complain about I mean like raise when this is discussed, so I wanted to
16:03
Underline that systemd is already used in the entire D So it we're just removing things not adding new things and systemd sets up the environment, so we if things are already Like they should be So like all the extra work that people have put into having scripts that work without slash prop mounted
16:23
It's not useful and We remove stuff, and we would be moving from scripts to daemons anyway, right? and shared libraries because if somebody tells you to You know provide FIDO to
16:41
authentication decryption in For the root disk you're not going to script it. You're going to use some compiled code to do it anyway And like I don't know net link timeouts retries localization D-bus all this stuff is just Semi incompatible with scripting, and we would be moving in the direction of normal compact code anyway
17:08
and So in case I entered RDS itself is kind of It was it's implemented and it mostly works that the stuff that is happening is like in the surrounding areas so
17:26
In particular this development in systemd Related to credentials is very important because we want to to make use of this and Support for unified kernel images is growing there
17:42
Patches There's a link here patches to for grab to to to load unified kernel images and I mentioned that we want to build M case I entered the emergency in federal. So this will require changes in in the kaji build system
18:02
and That's what I have and I have to have the time for questions One minute three minutes, okay When I was thinking about Systems that boot from network and for instance where the root system is on a scuzzy or in the e over fiber
18:25
You need some information that is really device specific. How can we separate that from the in it ID? So you want a single in it ID for all the systems, yes, so
18:40
One option is to put it on the kernel command line if this is an option and the second option is to Provide the credential that is unpacked and Becomes part of the configuration in the interview. So essentially yes, you you just inject this configuration
19:03
But it wouldn't be part of the either D itself it will be delivered in a different way Same questions. What would you do if you want to have files from the local file system? For example, you need a custom mount command that does more than a feature mount
19:20
Well, I mean I would say ask why do you need that? But if you need that then just do the build locally and The difference is as I think it was Kind of the same question was asked before about the unified kernel images. You can do the build locally You just don't have the distro signatures
19:41
Thank you, it might be a bit similar to the previous question but Thinking from a distribution standpoint. There is a lot of hardware out there which is incompatible with default Configurations or default in it urges and you need to add patches to kernels and you need to add special kernel modules
20:07
Which are not enabled by default? What is going to be the the flow to support this hardware to use it on distribution by default? so I Think that this is much less common than people think right because I mean how many people build their kernels nowadays
20:28
Small minority and Like from the distro point of view this is already outside of scope, right if you if you come Report a bug that your custom compiled kernel does not work. Then nobody is going to help you
20:43
Because people have too much bugs reported for the for the standard distribution the Existing ways of building things locally, they will always be there right and they're not going away so basically the answer is well, I mean if you
21:01
do are doing something specific then you keep doing this specific thing and This and this is the way to make the life for the distro easier, but it's not going to cover All cases maybe like 90% Any more questions?
21:26
Yeah, you mentioned camera module making the entire G bigger Could this be shipped in a standard extension somehow? Can you repeat? Could you ship the kernel models which are in very nearly in system?
21:42
so so the the kernel in the initial interd The kernel must have enough stuff built into to understand the interd and the interd must have enough modules to To be able to load system extensions But once you do that like you can
22:02
Have an extension with kernel modules and whatever. Yes. So last question Is there a path to Getting your init ramfs core
22:22
from somewhere and running a different distro like essentially to Your project or some project providing the core init ramfs with the systemd init inside it and everything inside of it Get the modules from elsewhere and then when you pivot, is that a hard line that you can live? I think that's
22:42
technically is doable because systemd is I'm kind of used everywhere and it there's really no reason why it wouldn't work I assume always assume that distro would want their own code in the initrd but
23:00
Technically, it's not required Okay, thank you, I mean if there's this general requirement that Because there's a switch root where state is passed from the interd To the to the host and you don't want to pass the state from a newer systemd to an older systemd So so they like they did the interd would have to be just older old enough
23:30
Thank you very much