Bridging the stepping stones: using pieces of NixOS without full commitment
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 19 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/50709 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
Nixcon 20209 / 19
2
3
7
8
12
14
15
17
18
19
00:00
Annihilator (ring theory)Commitment schemeSurgeryMultiplication signDistribution (mathematics)Hacker (term)Hecke operatorGroup actionDampingTwitterComputer scienceHexagonRandomizationFlow separationProgram slicingKernel (computing)Core dumpComputer animation
01:29
SpacetimeSoftwareCloningChi-squared distributionPhysical systemBootingConsistencyDeclarative programmingSystem programmingKernel (computing)Axiom of choiceModulare ProgrammierungConfiguration spaceCore dumpMereologyModul <Datentyp>FlagCodeComputer fileContent (media)Scripting languageExecution unitStrategy gameFunction (mathematics)Electric currentService (economics)Computer configurationDatabaseElectric generatorMagnetic-core memoryFeasibility studyFile systemCASE <Informatik>Home pageDistribution (mathematics)Integrated development environmentStrategy gameGenerating functionFunctional (mathematics)MereologyWeb serviceInteractive televisionBitKernel (computing)Configuration spaceExpressionComputer fileBasis <Mathematik>Physical systemPoint (geometry)Content (media)NamespaceModulare ProgrammierungScripting languageEndliche ModelltheorieCloningSpacetimeComplex (psychology)DatabaseData managementBootingProcess (computing)DemonPhysical lawElectric generatorLatent heatExecution unitComputer configurationCodeLimit (category theory)Service (economics)Computer programmingOperating systemFlagMultiplication signMultiplicationEscape characterRevision controlGraph coloringWorkstation <Musikinstrument>Direction (geometry)Instance (computer science)Right angleRule of inferenceRAIDUsabilityFilm editingResultantLine (geometry)Disk read-and-write headPerturbation theoryCore dumpINTEGRALState of matterArithmetic progressionDiscrete element methodHeegaard splittingStaff (military)Goodness of fitComputer animation
08:12
Core dumpPhysical systemMagnetic-core memoryConfiguration spaceBootingService (economics)Feasibility studyScripting languageModulare ProgrammierungParameter (computer programming)Web serviceAxiom of choiceDefault (computer science)File formatComputer configurationIndependence (probability theory)Instance (computer science)DisintegrationWrapper (data mining)Gastropod shellRevision controlWeb browserMaizeUser profileSingle-precision floating-point formatElectric generatorGroup actionSemiconductor memoryPhysical systemLink (knot theory)FlagInstance (computer science)MultiplicationBitData storage deviceBootingFile formatComputer configurationProfil (magazine)Service (economics)CodeRaster graphicsEndliche ModelltheorieSoftwarePhysicalismVirtuelles TerminalScripting languageWritingGastropod shellElectronic mailing listSet (mathematics)Web browserRevision controlMathematicsParameter (computer programming)Line (geometry)1 (number)Duality (mathematics)Wrapper (data mining)Web serviceDemonFunction (mathematics)Power (physics)DatabaseIndependence (probability theory)Configuration spaceComputer fontMessage passingSpacetimeFlow separationArithmetic meanDependent and independent variablesUsabilityAxiom of choiceForcing (mathematics)Level (video gaming)Default (computer science)Goodness of fitFamilyComa BerenicesNP-hardBus (computing)Type theoryData conversionNatural numberTerm (mathematics)Software testingMoving averageGodRight angleImage resolutionPhysical lawMonster groupWorkloadTouchscreenCommitment schemeSurfaceCuboidSpeech synthesisArchaeological field surveyComputer animation
14:55
Relational databaseMetropolitan area networkCommitment schemeBitComputer animation
16:06
Proper mapElectric generatorWeb serviceSet (mathematics)Broadcasting (networking)Data storage deviceComputer animation
17:01
Proper mapService (economics)Web serviceAxiom of choiceDatabaseDefault (computer science)Physical systemComputer configurationSubsetSet (mathematics)GodSpeciesGroup actionMeeting/Interview
18:10
Physical systemNamespaceLevel (video gaming)Computer animationMeeting/Interview
19:05
Modulare ProgrammierungExtension (kinesiology)Mechanism designUsabilityPhysical systemData storage deviceLatent heatUtility softwarePairwise comparisonCASE <Informatik>Overlay-NetzExpected valueRevision controlAxiom of choiceDebuggerEndliche ModelltheorieMultiplicationMultiplication signMeeting/Interview
20:39
Functional programmingConfiguration spaceEndliche ModelltheorieModulare ProgrammierungSlide ruleComputer configurationBitRow (database)Functional (mathematics)Graphics tabletNamespaceMilitary baseGraph coloringLevel (video gaming)Data structureProgramming languageVector potentialRule of inferenceStokes' theoremSpacetimeRight angleMeeting/Interview
22:43
Physical systemCrash (computing)Fluid staticsSoftwareReal numberConnected spaceDemonData miningComputer animationMeeting/Interview
24:54
Latent heatPoint (geometry)Water vaporStructural loadCartesian coordinate systemFormal languageSingle-precision floating-point formatData storage deviceIntegrated development environmentCodeClosed setUniform resource locatorInternet service providerFlagDirectory serviceRow (database)Meeting/Interview
26:34
Extension (kinesiology)Closed setComputer animationMeeting/Interview
Transcript: English(auto-generated)
00:00
Concludes our introduction and we'll be moving into our first talk, which is bridging the stepping stones using pieces of NixOS without full commitment. Our speaker for this talk is going to be Michael Raskin. An interesting thing about Michael is, you may have a hard time finding his GitHub account because it is
00:22
just completely just like a random hex dump of just random hex bytes. So I think I believe it's like 7C6F434C. And yeah, I always have a really hard time trying to figure out, OK, what is his like GitHub again, because it's like just random.
00:41
OK, a little background about Michael is that he's one of the few people who stopped using mainline NixOS, but actually still uses Nix and Nix packages kernels like for more than 10 years, I believe. And he moved to NixOS from Linux from scratch, and he was actually using a separate UnionFS slice
01:00
for each package in that distribution. And he's a postdoc in computer science. I believe it's theoretical to apply computer science. And he started using NixOS in 2007. OK, I believe Michael can now take it away. Hello, I'm Michael Raskin, and I will tell you about bridging the stepping stones or how to use pieces of NixOS
01:24
without full commitment to the full NixOS and without leaps of faith. So what pieces of the ecosystem you are expected to use? Well, first Nix package manager. Yes, you should use it. And it lives well side by side with anything.
01:41
So the only cost is maybe doubling the space you need for installing software. Next, Nix package collection. You most likely want to use it. And what's the cost? Maybe a few gigabytes of clones lying around and standard environments built, which you would build in a slightly more space efficient way, I guess.
02:01
And then there is NixOS operating system distribution. Well, maybe you also want to use that. After all, what's the cost? Only changing all of your habits about how to manage an operating system installation. Of course, NixOS has a lot of nice features, and many of them are listed on the home page. But basically, it boils down to three bullet points.
02:23
Your system is Nix package and you happen to have Nix around too. If you boot up to stage two, you are very likely to boot successfully and into a consistent state. And you have a declarative config, which is just a single expression
02:41
which gets instantiated from zero to complete and can be versioned and whatever. These are really cool features. So now you might be asking yourself, can you now finally, safely experiment with all the cool stuff like engine systems, operating system kernels and services which you can override to your liking?
03:02
And also, can you finally escape all these complicated interactions between the things that get installed imperatively? Unfortunately, the situation is a bit more complicated. Well, first of all, NixOS hardcodes quite a few things. First of all, it is written around a Linux kernel
03:22
and systemd as the init system. And then the configuration of NixOS, this declarative thing, is done by a model system, which is nice for simple things and propagates your preferences across the configuration correctly. But when you do complicated things, you notice that there are a lot of moving parts
03:41
and there is a global namespace and they sometimes touch in the same place in the namespace. Of course, this gets resolved automatically, but the interaction complexity is back here. And also, when you want to play with overriding services, it turns out that models are less overrideable than NixPQS packages.
04:03
And of course, NixOS hardcodes how you describe the core of your system, like you have to use this NixOS specific DSL to describe which file systems to mount on boot. On the one hand, it looks like not a lot of hardcoding because what distribution doesn't hardcode such things. On the other hand, it turns out that inside of all these
04:23
are trapped some less opinionated things like configuration generators for multiple daemons and also start flag knowledge for these daemons. And as an illustration that indeed there is less opinionated code that is kind of trapped inside,
04:41
this knowledge is duplicated in Nix Darwin, even though for a different service management system and Home Manager, which sometimes has a duplication of functionality with NixOS with slightly different approaches in a confusing way. And of course, Nix process management, which you will hear about later today.
05:01
Of course, one can also ask, is this code really trapped? I invoke the law of headlines ending in the question mark to hear the answer. Well, it is complicated. There is a strategy how to use all this code outside the mainline NixOS installation.
05:23
You just evaluate NixOS as a function, as a Nix expression with configuration, which is minimal beyond minimal. It is not enough to build a bootable NixOS or maybe even a container. It only talks about the service itself.
05:41
And then you grab the parts you care about, like most likely the configuration files in the ATC. And also a nice part of NixOS system de-unit generation functionality is the possibility to export to the contents of exact stuff or to a runnable script. Thanks for that. It's really useful in many cases.
06:03
By the way, I use both things I have described here on my own system to grab CUPS running and to get XOR configuration. So currently, all of these functionality I use is online, which allows you to grab service script by service name
06:21
and NixOS configuration and also ETC file or a bunch of files in ETC. And all of this is online and has been online for some time. So what are the implications and the limitations of these approaches? I claim that the main value of NixOS,
06:41
which would take the most time to replicate, even based on Nix, is a large database of configuration generators for many, many daemons and programs. And many services are actually already reusable, which is nice if you know how to do it and you have a use case.
07:01
Of course, there are some catches you need to pay attention, because, for example, many services are configured, not just under their namespace, but all over the place. And also, some services are too complicated for a working runner script to be generated automatically by the generic code. You might be able to grab the parts of the unit
07:24
and slap together the correct runner script on a case by case basis. And then, of course, some services have configs that do not go to slash TC and also do not get an option name for the contents. They just get reference inside the service starter unit
07:44
and they are annoying to grab. So you see that I said that I can replace the NixOS boot script so not to run the full NixOS, but still use the services. But I said something about the module system and what would I use instead?
08:03
I would use something mimicking NixP itself, like list of services, which might be empty by default.
08:20
And in OVA, you overwrite whatever you want to reconfigure. And of course, you want to put as much as possible into this ether set and not into let instances, because you want everything to be inspectable and maybe even modifiable. On the other hand, I can say that what if not the model system is the wrong question.
08:43
The other question is, can we make it not matter? The question is that services could be like packages. They could be packages with parameters and they could provide rich pass-through and they could request and inspect other service instances. And then it's user's choice how to ensure that everything gets passed
09:04
and that every service gets as a parameter correctly configured instance of dependencies. And of course, model system would be the default, would be the only thing supported by mainline NixOS, but it would be easier to extract it and replace it.
09:23
And it would be a well-defined top level layer with clean separation of responsibilities. And another thing that NixOS kind of hardcodes is the bootloader handling. Because of course, NixOS assumes that it's the only thing that could want to configure bootloader based on system generations managed by Nix.
09:44
And naturally, it assumes that all the layout is the NixOS layout, which leads to some kind of unfortunate things, some unmodularity, like that the single Perl script generating the group entries must know how NixOS handles
10:02
Zen and stuff like that. And all of this means that if you have a different system which creates system generations, it won't probably be compatible with NixOS unless I take everything that NixOS does exactly. And that's why I'm actually too lazy to dual boot with NixOS because I
10:23
would need to integrate the two lines of bootloader narration. So if I dream what I would dream about, I would love to see NixOS services as NixOS-like package collection with services configured by argument passing and argument overrides and maybe overlays.
10:43
I would love to see model system then on top of that as one of the ways to connect things, even if the main one, and then I would love to see a multiple independence options for core, like for boot scripts and stuff arise while still sharing the service database in an efficient
11:05
way. And then to make dual booting such options more convenient between each other, maybe NixOS bootloader generator would collect bootloader configuration snippets from each system generation. Of course, that means that you need to provide snippets at once
11:23
for all the bootloaders you theoretically support and probably you would want to fail unless a special force flag is passed. If booted system does not support the loader you are trying to configure for the new system, but still it could be a step forward.
11:42
And if I'm dreaming anyway, I would like NixOS to gain support for atomic slash etc switch hank. It's not completely infeasible. I do use it on my system, which is kind of similar to NixOS and it's a same link into store and so one same link change and that's it. I promised to talk a bit about why did I even bother replacing the
12:04
NixOS boot script? Well, first of all, I like that my virtual terminals are not owned by systemd, which makes it easier to do my custom tricks around launching Nix-org and also it makes it easier to write stuff like power of command
12:21
is privileged, not by password, by a check of physical presence. Sorry, no screencast, because it's very annoying to use xorg screencasting software with a lot of virtual terminal switches, you know, and then my system is managed by custom return, common list daemon mainly.
12:42
And for example, it integrates automatic NSJ ropers. So everything I want can be run in jail because I don't know nobody runs browsers outside containers in 2020, right? And of course only handpicked things have sound access, which means that
13:02
my user doesn't actually have sound access, only the things that are run in jails, which are specifically granted sound. And then my on-boot mounts are a mess, which is for me easier to manage as a straightforward shell script than inside Nix DSL and maybe contrary to its
13:24
assumptions. And then just for fun and ease of debugging, I have full versions of everything I have in Nintra MFS included there. So LVM2 means full LVM2. Yes, with full glibc dependency. Yes, it takes space, but it only takes space in memory until I actually
13:44
switched to the main system. So who cares? And I like to have my services started via explicitly specified, nice traceable scripts. So if something goes wrong, it's easier to debug. And what you can take out of this, if you want commitment, neither to Nix OS
14:04
approach, nor to my Nix approach, well, you can take wrappers for piece extraction out of Nix OS, as I mentioned, and then a wrapper to create an isolated debug session inside a container for something that wants debug session, but
14:20
doesn't deserve access to your main one. If you even have the main debug session, because I don't, maybe you might be interested in Firefox empty profile builder based on launching Firefox ones in Xdami. And then there is some code for converting true type fonts to bitmap
14:40
format for Linux console, because monospace fonts often come with very good hinting and font-forge is pretty good at converting fonts between formats. Thanks for your attention. Other questions?
15:09
Okay. Hello everyone. That was Michael Raskin bridging stepping stones using pieces of Nix OS without full commitment. And I did get news from the channel that we did start a bit earlier than
15:21
expected. So I think it was supposed to start at 7 15. So I am sorry about that. That may make it such that we don't have a lot of questions or the Q&A portion. Let me look into the pad and see if we have any questions, Michael.
15:42
Yeah, I do not actually see any questions. I will also manually check the channel. Nixcon Q&A, if anyone has a question you can ask there. Yeah, I do not see a question. I am sorry about that, Michael. Yeah, it happens, whatever.
16:02
On the other hand, maybe it means that the talk was clear enough. Yeah, that is also a possibility. So I would say you could, I will direct you to like the breakout room, which is bridging steps, bridging dash steps.
16:20
And I am sure that if people watch your talk later or like go back into the broadcast, they can see that and they can ask you questions from there. Oh, I think it is Puck says that two questions appear just now. I will just read them from the chat, Michael.
16:42
So we have one from SRHB. And she said, is abstracting the service generation worth it and Nixos proper? Maybe. Michael, I believe you are muted. I am sorry.
17:00
Let me, worth it in Nixos proper maybe. Well, I mean, I do not know exactly how much you want to say that Nixos is the default set of options. I believe that you want to, well, I believe it is valuable to have a service
17:21
database that we can reuse and that we can work on collectively, regardless of our specific choices. I don't know maybe what subset of the options should be called Nixos proper and what subsets of options should be called using the services from the Nixos
17:42
database on a different system. I think that can be discussed once we have that call, once we have things to compare, I'm not sure like, well, it's okay if Nixos is an opinionated set of choices in that regard. It's just a question of what we want to allow next to Nixos and then we can
18:05
decide what to reintegrate. Okay. I, that's, that was pretty clear to me, actually. So you have a question from Nixor86, and they say it was not obvious I'm going to talk or I missed it, but do you still use system D?
18:24
Well, it turns out that I ended up not using system D at all. And I'm not against using system D in some container or something, but just never needed it enough. I definitely don't want to have system D on anything on my top level
18:45
namespace. And yeah, well, as I don't run it in smaller namespaces and I don't want the top level, I ended up without using system D at all. Well, of course I have everything linked against system D libraries, but
19:01
that's another story. Okay. We have another question from Hyperfect. Oh, I know you. Hey, how does the usability of the module system and your extensibility mechanism compare in practice? Well, in a sense, it's hard to say because for my needs, these
19:22
extensibility via overlays has better transparency and easier debuggability for my specific use case. But that's another story in a sense. I never tried to use module system to have multiple people.
19:44
Multiple people build vastly different configurations, and I cannot exclude for many people. It is a very good choice to use module system because it propagates your
20:01
preferences across the system well. Well, at least as long as you are inside the expectations. So it's, I don't know, I didn't try to see. I believe that there are situations where for some people and for some settings,
20:21
different approaches are more usable, more debuggable, more inspectable than the model system. And as I said, I am not sure. It might be that it's still the best choice for the main line, most popular version of NixOS. Okay. We have a question from Ryan TM.
20:42
He says, it sounds like you're suggesting to move modules to packages is maybe the opposite of ILCOS Repulses yesterday. What do you think? Well, I think the following, first of all, as of me not addressing at all any of ILCOS points, kind of sorry, but World of Peace can confirm that my recording
21:04
has been finished before I had any chance to see ILCOS talk yesterday. And on the other hand, yeah, I think the following, I do believe that, well, when I looked yesterday, for example, at the slides of ILCOS talk, I was a bit worried.
21:24
Then you say, okay, and you can override the configs of the module you extend. And then I saw the config overwrite, and it was just global overwrite. And so, you know, it's complicated.
21:40
So I thought that, okay, so what are the Scotland rules? I think that for putting modules inside Nix, things like scoping rules, not to stop being, you know, purely functional programming language, which gives you at least an option to have full referential transparency,
22:02
not only formally, but also really, and to avoid global namespaces, if you want to avoid the global namespaces and so on. I think Nix being a purely functional programming language was very valuable from the very beginning and is still valuable now. And so I believe, yeah, as I said, I believe that model system is a good thing at some levels,
22:26
but I believe there are layers that are better done as pure functions and fully expectable plain data structures.
22:42
Okay, I'm looking at the pad again for more questions. I do not see any. I think what Virek asked, what's my init? Oh, well, as for my init system,
23:00
well, it turns out that if you have a small enough desktop system and you actually want to see and control what is there and how it runs, it turns out that if you have just a few things and you immediately observe whether they're working or not,
23:21
and you restart them for unrelated reasons anyway, like I restart my local Vint when I change the network I'm connected to, or just so that it reconfigures itself a bit, it turns out that in this situation, I don't even need a real init system in a sense.
23:44
So I do have PID1, obviously, and PID1 is intended as an init system, and it is static init as init from SACLAS. I don't like everything SACLAS does. Well, in the sense, I don't find useful for me all the tools SACLAS produce.
24:05
But as init, I looked at it and it does exactly what I wanted it to do. And so, yeah, my init is as init. And then I have, as I said, I have this jumper daemon, which manages my system.
24:20
In particular, it launches the few daemons I want to have running on my system. So and that's it, more or less. And somehow it turns out that on a simple enough system and they happen to have enough RAM, you know, CUPS doesn't crash. Dint doesn't crash on its own.
24:42
Xorg doesn't crash on its own. And then what is even there for a true system supervisor to monitor? Okay, great. Oh, another question. Is there something else apart from NSJail that I could check to learn more about your jail setup?
25:01
It sounds like you're going to prison. Well, I mean, I assume, I hope at some point, yeah, I will at some point publish the slides, I guess. And I hope that the recording will be there. There is, well, most of my setup, because, of course, there are some small things which are very local,
25:22
like some of the specific host names and stuff, which might not be fully included. But generally, you know, I have most of my setup online in my GitHub account as LangOS.
25:41
And yeah, well, you can look at this. But basically, it's a lot. Well, basically, it's just using NSJail. And then there is a lot of things that you need to tell to NSJail to comfortably rob some application. I need to tell NSJail what directories to provide to the application,
26:03
what of them to provide read-only or read-write. I also run NSJail under a specific user ID, which is unique for every application during a single session and stuff like that. And I set the environment variables, of course, because, of course, you need to set up stuff.
26:22
But basically, well, it's a piece of code to generate a ton of flags to NSJail, I would say. Okay, great answer, actually. I don't see any more questions in the Q&A portion,
26:40
so I believe that will lead us into the closing. Thank you so much for being here, Michael. It seems that you get to have a very, basically an extended Q&A portion because we started early. Yeah, thank you. Yep, nice having you.