Good Enough
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 28 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/61011 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
NixCon 202226 / 28
10
12
20
21
22
24
25
26
27
00:00
BuildingSystem programmingInheritance (object-oriented programming)Virtual machineService (economics)Group actionSoftwareProjective planeBuildingSoftware testingMereologyResultantFreewareData managementOverlay-NetzPatch (Unix)Rule of inferenceCodePersonal identification numberRevision controlGoodness of fitBitIntegrated development environmentDifferent (Kate Ryan album)Game controllerQuicksortMultiplication signYouTubeSystem callIterationFile systemVideoconferencingPhysical systemCompilation albumData conversionLine (geometry)Reading (process)InternetworkingExpressionSoftware developerInstallation artTime zoneDiagramSoftware frameworkComputer animation
07:10
Service (economics)System programmingBootingComputer-generated imageryMiniDiscPhysical systemAlgebraic closureServer (computing)Remote procedure call2 (number)Multiplication signCategory of beingBitCollaborationismIntegrated development environmentPrimitive (album)Point (geometry)UsabilityFormal languageFigurate numberMereologyGoodness of fitSoftware testingCartesian coordinate systemInflection pointVulnerability (computing)Parameter (computer programming)Medical imagingHecke operatorBuildingCodeComputer fileBootingNumberQuicksortProcess (computing)SoftwareAlgebraic closureProjective planeData storage deviceExistenceSlide ruleComputer animation
14:21
System programmingInheritance (object-oriented programming)Virtual machineService (economics)MiniDiscComputer-generated imageryPhysical systemAlgebraic closureBoundary value problemDirection (geometry)Virtual machineAlgebraic closureSoftwareRegular graphDerivation (linguistics)Goodness of fitDemonOperating systemGame theoryWindowQuicksortProduct (business)Integrated development environmentArithmetic progressionMultiplication signOrder (biology)Spectrum (functional analysis)Open setProcess (computing)Iteration10 (number)PlastikkarteDeterminantStructural loadMachine visionPosition operatorPoint (geometry)Software developerFocus (optics)NumberComputer animation
21:31
Computer animation
Transcript: English(auto-generated)
00:02
Thanks for coming, everybody. Really wonderful to see everybody again. It's been so long. It's so good to see everybody in face and have these conversations in person that we just can't quite get the same thing online over video calls or the Discord. It's so special. But I wanna talk a little bit
00:21
about something a little bit different today, which is the idea that maybe things are good enough, and maybe sometimes we're actually going a little bit too far. And I think it's really important to focus on why people really want to use Nix and what they're trying to get out of Nix. To start with, I haven't had any coffee day,
00:40
which is a little weird. Haven't been in front of 200 people in a long time, so that's a little bit weird. It's really nice to be here today. All right, let's talk about why people use Nix. Who uses Nix because of the sandbox? Is it the sandbox that you use Nix for?
01:00
What? Sure, all right, how's that? Louder? All right, how's this?
01:20
Even louder? Okay, all right, good. Okay, so the sandbox, some people say they use it for the sandbox. How about version pinning? Being able to pin Nix packages or their other dependencies, I use it for that too. I think that's a pretty sweet feature. How about reproducibility? That's kinda nice, right? You can run it on your computer.
01:42
You can run it in CI. You can send it over to Luke and have Luke run it, and maybe it'll work there too. Though he's on a Mac, so I'm not totally sure. How about software freedom? Anyone use Nix because of software freedom? This one's a little bit tricky. What is it about Nix that gives you software freedoms?
02:00
I believe it enhances software freedoms because it lets you patch anything in your entire system without any more effort than a simple overlay. Almost no package manager can do that, and Docker doesn't get close. I think that's something we really need to appreciate as a free software community. But I think there also might be one other issue here,
02:20
one other reason that a lot of us use Nix that we maybe aren't talking about yet, which is that it feels great, right? You get some piece of software and you're trying to make it work, and you iterate and you iterate, and it fails and fails and fails and fails, and you try and figure it out. Finally, it builds, and maybe you even have a Nix OS test, and you get that clean results
02:42
in Lincoln again, and oh my gosh, it just tickles all the right parts of the brain, right? And in so many ways, Nix is such a superpower, right? It lets us do things that almost nobody else can do without Herculean effort.
03:00
But sometimes, it takes that Herculean effort to do it with Nix. And in those cases, it's not actually a superpower. In many cases, it's actually getting in the way. If you're doing JavaScript development, a lot of chuckles.
03:21
Maybe you're using some test framework that downloads Chromium from the internet. Yeah, those aren't so fun, is it? And what do you do? Do you fix it? Do you suffer? Do you keep hammering on this build to get it to work? You might. But one of the things about this is that, as a community, we have it in our minds
03:42
that the ultimate goal is to get it to build with Nix. Distribute, get to that closure, build it with Nix, get that closure, get it out on NixOS. You get so many cool superpowers, you can do that. But the truth is, this is a bit of a diagram I call the zones of control. The truth is, is that if you stay up on Nix run,
04:04
how much of the value of Nix do you get out of it? A lot. You can install Nix on a Mac, on a Linux, you can get JQ or Lolcat or a bunch of Pythons and all these different environments, right? And you can get all of that for basically zero work.
04:21
And I think that's incredible. And you don't have to give anything up for that. Nix doesn't get in the way, it doesn't mess you up by default, everything is easy and good. And so maybe you take the next step, so you've had such a good experience, but easy so far and you go and run, you make a flake.nix and set up all your development tools
04:42
and make those all available in a dev shell. And that's really cool, right? That takes you from maybe 80% to like 90% of the value here. Because you can take that flake.nix and use it in CI. You can share it with your colleagues, you can shorten the onboarding time for a new employee from like a death march of two weeks down to like,
05:01
oh, just install Nix and run Nix develop and you've got everything you want. That's super cool. But, you know, the next step here is building with Nix. And actually to do the flake.nix, you had to learn something, right? You had to learn the expression language, you had to learn how to find packages,
05:21
you had to figure out how to make shell, you had to figure out all these little pieces. And that's okay, it's good to expect people to learn things, no problem. But once you get to the Nix build, you start to get to rules. Rules that I quite like. Rules like, you can't talk to the internet unless you know what you're getting.
05:42
And you can't write to just random places on the file system or read from various places. And you can't do a caching build, right? If you do a build and change one line of code and do a build again, you're paying that whole compilation cost again. And that's a big cost. I think we need to remember that's a big cost.
06:01
You know, some tools, some projects I've seen, there is no support for building it outside of Nix build. And so the iteration time is very slow. But if it takes 30 seconds, I'm out. I'm gonna go read Twitter and watch some YouTube videos. I mean, just to be honest, that's just who I am. I'm gonna do that. And so actually, what we have here is
06:20
I sort of show a funnel of the zone of control, how much control Nix has over the overall environment. But if we think about new users coming in, the funnel is the other way, right? It's very easy to get started with Nix run. Anybody can go do that and feel successful. Almost anybody in that group can go write a flake.nix
06:41
and copy it from another Go project and make it work. But it gets mighty sharp, mighty thin at that next build step. A lot of software works there, but some doesn't. And how, like, if some of your software does and some of your software doesn't, why are you gonna bother moving on?
07:01
And I don't mean to, you know, chastise us for having these rules. Like, these rules are critical. We can't give them up, especially within Nix packages. We can't give them up. And that's because so often as a community, we're building software that is at the core, right? And so by being at the core, it's really important that we maintain these principles,
07:22
these principles of purity and usability in the sandbox. But I think we're at an inflection point as a project. I think we're at a point where we could be finding whole new communities of people that are happy with Nix, if we're willing to welcome them to the project.
07:41
And some of them, I wager, they don't need all of these rules. If you're making a JavaScript website and you're pushing out your JavaScript website, you have no use for the closure. You're pushing it up to a CDN or something. You're gonna be substituting from Fastly or Cloudflare or something. The primitives that Nix offers are less effective,
08:01
less important. And I think if we find a way to be more approachable as a project, as a build tool for these tools that are very difficult to build with Nix, I think we could get some really nice engagement, nice interaction, nice collaboration between these communities.
08:21
Imagine if instead of telling JavaScript projects, sorry, they're really hard to build with Nix because they just don't play well with Nix. Imagine if we found a way for us to work together somehow, and then they decided on their own that they wanted the properties of Nix, right? I think that's a really interesting idea.
08:40
And I'd like to invite those communities in. I'd like to sort of be big tent here. Like how do we get these communities who are solving problems for their users to embrace these ideas of the reproducibility and the purity and collaborate with us on a way to do this? And I think it's good. I think if we can do that, that's good. Because if you get down to a closure,
09:01
you can do pretty neat things. You can export to a Docker or container. Antoine's talk was fantastic. I actually deleted a bunch of slides because he just did what I was gonna say I wanted to happen. Amazing, thank you. And then if you can get to Nix OS, I don't know, I think these are super cool, really powerful, and I don't want to give them up.
09:21
But at the same time, as Nix developers, we tend to have this sort of Ice 9 mentality, that you get a little bit of Nix somewhere and then you want to write Nix to do this and a little bit more Nix to do that and a little bit more Nix to do that, and it just spreads and spreads and spreads. And then all of a sudden you're writing Terraform code in Nix, and your colleagues are looking at you like, what the heck is going on? Why are you writing Terraform in Nix?
09:41
I don't want anything to do with that. And I don't know, there's actually an argument in favor of that. The Terraform language is a little bit weak as a language on purpose, and Nix is a powerful language. But I think being like having tools that look friendly, that look familiar, that people can share
10:02
and understand from where they're coming from, I think that's important. And I think, I guess what I'm trying to say here is once we reach the edges, once we get out of Bootstrap, once we get out of Nix packages, once we're actually talking about software we're deploying ourselves somewhere, I think it's okay to get creative.
10:21
And I think it's okay to get creative in ways that maybe doesn't belong in Nix packages, but that's okay, like it doesn't need to be in Nix packages. And so probably the spiciest one here is, you know what? I think some people should turn off the sandbox.
10:40
And I think it's okay. I think if your JavaScript application is downloading Chromium for a test phase, I don't care. It doesn't bother me, probably doesn't bother you. And you want your tools to work well. I think that's cool. I think it's good. I think if you can build it with Nix, even if there's a little bit of impurity there, I think that's good.
11:02
I'm not the only person to say this, Joe and I swear to you, maybe he's hiding. I don't know. I might be hiding after this. And I think there's other opportunities here. We do some funny stuff in Nix packages. We have all these really great tools for building Docker images.
11:22
So this is the part I deleted. Wouldn't it be great if we could, you know, not rebuild all these Docker layers on every single build and just figure out who we need to build and upload at build time? But it requires doing something out of the sandbox. And as Nix developers, it's very tempting to say, oh, we have this great execution environment. It cleans up after itself. It manages all its dependencies.
11:40
It's so nice. We can, you know, hack it out in some dirty bash in a few minutes and it's okay. But by spending a little bit of time on this, Anton was able to make a tool that instead of taking 30 seconds per build takes on the order of seconds. Man, how cool is that? 30 seconds is my limit. 30 seconds, I'm out. I'm back on Twitter, like I said.
12:01
And that's not a joke. Genuinely not a joke. So another example here, something we've experimented with and actually use on the foundations infrastructure. Nix Netboot serve. If you've ever built a Netboot image from the tools in Nix packages, 10 minutes, 15 minutes or more,
12:22
you can take a closure, you put in a squash FS, you put the squash FS in a CPIO, you copy all this data around. If you're remote building, it's even worse because you have to upload it and download it. Oh man, it takes forever. But the thing is that the very next thing you're gonna do with this Netboot image is you're gonna throw it up on a pixie server
12:41
and you're gonna boot it. That's not Nix. Who cares if you're using Nix to actually make that image if you have a good tool to do it? You could instead take, and that's what this tool does is it takes every store path, turns it into a CPIO and smooshes them all together. The rebuild times are like one second. And it's cool. I think we should be really embracing this concept.
13:03
Nix to container. Again, I just can't say enough how much I appreciate the existence of Nix container. Something that we've been experimenting with internally is this idea of ephemera, of what if we could build Nix OS images that if you rebooted everything that was erased,
13:21
like really take the purity all the way, all the way, and we've been able to build Nix OS images in seconds, sometimes less than a second. It just takes a little bit of creativity. Like, what are the principles of these immutable, these non-interfering store paths
13:40
that are isolated and individual and you can process them one path at a time and reuse that processing later on? Basically, all of that secondary is IO because it's just like adding a bunch of files together. And so where else can we get creative? I mean, I think there are a number of places
14:04
we could get creative here. And I think we need to be creative in ways that are good enough. For example, the Dream to Nix tools that are looking at lock files and very creatively trying to find all the dependencies that are needed to build it. But maybe they don't build in the strict sandbox,
14:20
but they probably should, even though they are impure. I think that's okay. I think that's good enough. Things like flakes. There's a lot of, you know, are flakes good enough? Well, I think, I mean, I'm open to discussion here. I'm not saying anything here, but I think they are good enough. And what I found is that people can learn flakes
14:41
in a way that they can't engage with the regular Nix stuff. Like, it gives them a thing that they can copy and paste and they know what they have and it's a concept they can learn. And other ways that we can be taking the closures that we have and be pushing them to somewhere else where they're actually doing something, right?
15:01
What's the point of Nix and NixOS if it's all inert? If it's all pure, it's not really the point. So how do we get from this position where we are regularly making tools that inject, you know, sometimes many tens of minutes into our iteration process and how do we cut all of that out
15:21
and make it a fast, good experience, which people want to get to, right? How do we say, okay, yeah, 90, 80% of the value is that Nix run, 85, whatever, is that Nix developed, but if you can get all the way down to a deployed closure, you could be testing like you're in production every second,
15:41
that'd be a thing. That'd be really exciting. And I guess overall, I just wanna say, like, you know, Nix has been here 20 years, just about. I guess 20 years next February. Why are 200 people here? It's because for 20 years, people have been seeing
16:02
this idea and saying, all right, well, that's amazing, but what if, what if it did this extra little bit? What if we took, what if we made an operating system out of it? That's nuts. I think if you asked anybody at the time, that was nuts. Even Debian's coming around to some of these ideas. So let's keep going, let's keep pushing this boundaries,
16:22
but also consider if taking a single step back from where we are could do something especially cool and especially powerful. Thanks.
16:43
Thank you, Graham. Are there any questions? So personally, I'll keep my pure builds, but if we do want to do the kind of, I feel like it's a wine approach of you make people bring Windows games to Linux
17:03
and then they do the native stuff. Like if we bring JavaScript people in with impure and no sandbox, then hopefully it will get better. But do you see that as possible while you need to restart the daemon to make it not be sandboxed?
17:24
Cool. Yeah, so is it possible to do this when you have to restart the daemon to support on sandbox? So I'm on board with that. I like keeping my pure store, but also if you relax the sandbox, it doesn't automatically turn it off.
17:40
You have to opt into it, and in general, most of my software is built on Hydra, right? So I'm not actually responsible for that remaining, well, it's complicated, but my machine, personal machine is not responsible for that remaining pure, and so I guess I don't feel like you would need to restart the daemon. I think it would be okay to relax the sandbox
18:01
and then have those derivations opt in. And another thing on that point though is I think by bringing these communities in, JavaScript developers are incredibly smart. There's loads of incredibly smart JavaScript developers, and I think if we can get them started and get them on the path, I think they might want
18:21
to come back and meet in the middle here if we give them the chance. Hey, thank you for the talk. So I'm curious, do you see Nix evolving in a direction where you have a more progressive story for people to start with less restriction, maybe?
18:49
And as they go further with Nix, they can start having more MCT, more sandboxing, more stuff like that, which kind of reverses then the current story, because the current story
19:00
is just, oh, you're trying Nix? Oh, you can't do that, you can't do that, you can't do that, you can't do that. Yeah, okay, so the question is, moving the policy to be more relaxed and moving stricter, is that what I'm thinking? Not necessarily, but I think it's good to stay relatively strict, but perhaps we could imagine
19:25
an environment that says, oh, you tried to talk to the internet, you could try to fix it, but also you might relax the sandbox, right? It's sort of this whole spectrum of how pure are you, ultimately pure, and ultimately nasty and dirty, and I think many, many places on that spectrum are okay.
19:43
But I think we should, especially in the core, we should stay strict and then make it possible to get out and not shun people for that. Thanks a lot, I think that was a great talk,
20:01
and it's also a great vision of how we can get more people using Nix. I suppose as a leader of determinant systems, you're also wearing two hats. So I know for a fact you're like the Nix enthusiasts, but as determinant systems, what do you see as your, like what are you up to?
20:20
What is your, what do you see your place to be in the ecosystem? Because you're upstreaming all of the stuff that you're doing, which is great, which helps anyone, everyone gets their stuff done. You're showing this project that you're developing on the open. What are you up to? Yeah, great question. Well, so fundamentally, I think,
20:42
so I think a few things. So one is if we try to go make a big business today, we have 200 people in a conference which is amazing and lovely, and I'm really, really proud of that growth, but I don't think it's big enough. And so what I think in order for it to really make sense
21:02
that the number of people using Nix needs to be much larger. And so really our focus right now is how do we, how do we get more people using Nix free, open, the overall ecosystem. Later on, I mean, I think, I think the, I mean, I think it's sort of, in the air that there's a lot of challenges
21:22
around collaboration, and that's something we're looking at working on. Let's hear it one more time for Graham, please.