devenv.sh - Fast, Declarative, Reproducible, and Composable Developer Environments
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Anzahl der Teile | 542 | |
| Autor | ||
| Lizenz | CC-Namensnennung 2.0 Belgien: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
| Identifikatoren | 10.5446/61606 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
|
FOSDEM 2023150 / 542
2
5
10
14
15
16
22
24
27
29
31
36
43
48
56
63
74
78
83
87
89
95
96
99
104
106
107
117
119
121
122
125
126
128
130
132
134
135
136
141
143
146
148
152
155
157
159
161
165
166
168
170
173
176
180
181
185
191
194
196
197
198
199
206
207
209
210
211
212
216
219
220
227
228
229
231
232
233
236
250
252
256
258
260
263
264
267
271
273
275
276
278
282
286
292
293
298
299
300
302
312
316
321
322
324
339
341
342
343
344
351
352
354
355
356
357
359
369
370
372
373
376
378
379
380
382
383
387
390
394
395
401
405
406
410
411
413
415
416
421
426
430
437
438
440
441
443
444
445
446
448
449
450
451
458
464
468
472
475
476
479
481
493
494
498
499
502
509
513
516
517
520
522
524
525
531
534
535
537
538
541
00:00
SoftwareentwicklerProgrammierumgebungDeklarative ProgrammierspracheBinärdatenCachingWhiteboardDefaultSpeicherbereinigungEin-AusgabeVersionsverwaltungNabel <Mathematik>Deterministischer ProzessGebäude <Mathematik>Elektronische PublikationComputersicherheitCodeBitSoftwareProjektive EbeneFormale SpracheSpeicherbereinigungSoftwareentwicklerGefrierenStrömungsrichtungAggregatzustandVirtualisierungCachingDifferenteRechter WinkelEntscheidungstheorieKonfigurationsraumNetzbetriebssystemProgrammierumgebungCASE <Informatik>AnalogieschlussIntegralAbstraktionsebeneElektronische PublikationFunktionalParametersystemStabilitätstheorie <Logik>Web logOpen SourceHook <Programmierung>Installation <Informatik>DefaultEin-AusgabeProblemorientierte ProgrammierspracheDokumentenserverDatenloggerBimodulBinärcodeDeterminanteNabel <Mathematik>Physikalisches SystemGebäude <Mathematik>MultiplikationsoperatorComputersicherheitComputeranimation
08:02
VersionsverwaltungMaßerweiterungProzess <Informatik>Physikalisches SystemComputervirusDienst <Informatik>SkriptspracheSkriptspracheProtokoll <Datenverarbeitungssystem>Formale SpracheDifferenz <Mathematik>Interface <Schaltung>Projektive EbeneKonfigurationsraumSoftwareProgrammierumgebungAbstraktionsebenePatch <Software>VerschlingungIntegralMultiplikationsoperatorHackerCASE <Informatik>PunktBimodulSoftwareentwicklerTypentheorieSoftwaretestBitCodeRechter WinkelDefaultZeichenketteInterpolationDatenverwaltungProzess <Informatik>VersionsverwaltungVirtuelle MaschineImplementierungPrototypingMaßerweiterungDigitales ZertifikatSystem FInverser LimesMAPDigitale VideotechnikInteraktives FernsehenComputeranimation
15:58
Flussdiagramm
Transkript: Englisch(automatisch erzeugt)
00:07
All right. Thank you-all for coming. Yeah, I had to fit a 40-minute talk into 20 minutes, so let's see how it goes. Sorry. No problem. Yeah, I want to talk about developer environments, and I don't mean this room,
00:21
but I mean how to get software that we need in a project together and do it this obviously with Nix and in a simple way. First, just briefly about me, I started with Gentoo in high school. I did Google Summer of Code, switched in 2012 to Nix OS.
00:42
Thanks Florian, freeze it off for this. In 2014, I wrote a pretty viral blog post like how we can do better than what is the current state of software in the world. Four years later, I created CacheX, which allows you to host binary caches for open source projects.
01:03
I started Nix.dev, which is about 15 tutorials right now, how to get started with Nix in different ways. Last year, we restructured the foundation, I became a bone member, and also I started Dev Env in November very quickly.
01:21
So how many of you are using Nix just to get an idea to who I'm talking with? All right. About one-third. That's good. So developer environments, right? Design decisions are important things. We want to support all the major operating systems so that when you have
01:41
a team of diverse developers that everyone can go and install it. We want to have hermicity or determinism so that if you come to a project, let's say 10 years later, it should still work, right? It should still build, it should still do exactly what I did 10 years ago.
02:03
Yeah, we can talk about that in detail later, but we want to configure garbage collection. So Nix, we just download things. We don't really remove things by default, and when we do remove it, we call this garbage collection, which is an analog to how we handle memory.
02:20
So we want to have our developer environment to take care of that for us, because by default in Nix, it's like an optional thing. We want to integrate Dev integration, which is a nice little tool where you just enter a directory, and it activates something for you, and you exit, and it deactivates that. So it's like you don't have to run any commands.
02:42
We want a high-level abstraction that we can use to pull things together, and glue them, and abstract, and have a nice little language which we will use here at Nix. So how does it look in practice? You create a, I don't know if you can read that from that far, but you create a devenv.nix file where you have at the top,
03:02
a little function that passes in a few parameters. Most common one that you would want to use is packages, and then you can define environment variables, packages, and a shell code that executes once you enter the environment. Bear in mind, there's no containers or anything. This is just like virtual and on steroids or whatever you want to call it.
03:23
At the bottom, you see like, okay, let's say there's no Git installed. You say devenv shelf. You see hello determinism, and there's Git available in this environment. There's devenv search, and you can, for example, search for packages. This will search the whole Nix packages repository that we have,
03:44
and over 80,000 packages, and so on. This is those that are using Nix OS or Nix Darwin, or Home Manager, very familiar with this. So once you familiarize with how the modules work,
04:02
you can switch between different components and be happily using the same idea for the operating system. On the other hand, we want to specify how these environments are created, and this is devenv.yml file where we can specify the inputs. As I said before,
04:21
then the environment is fixed according to these inputs, pinned, and by default, it will use the Nix packages and stable. But you can customize that, add new inputs, and so on. Inputs can be tarble, GitHub repository, whatever you can fetch from somewhere. Then we want to have these environments composable.
04:43
So what's happening here is that we have still some inputs, but we can, for example, fetch the devenv repository and we can reference a devenv.nix file from this GitHub repository, but also locally. So if you have a typical setup where front-end,
05:01
back-end, and so on, or you just have some general stuff you want to reuse, you can pull it in with the inputs and then say, import that developer environment, and you can compose these together. That's more of a already advanced use.
05:20
So what does it mean fast? So with Nix, we have the binary caches ideas. So anything you build, you can upload to binary cache, and when you say dev and shelf, that will be downloaded. So as long as everything is downloaded, this will take usually a second or two to activate the whole thing. If not, then possibly a bit longer. But there's also a CI command which you should run on your CI,
05:43
which will check that if you say there's a way to declare, for example, Git pre-commit hooks, and build packages and all these things, it will make sure that everything is good. So you just run this on CI and upload things to binary cache, and the next time developer invokes dev and shelf,
06:01
things should be there and things should work. So just briefly, the different files, there is devenv.nix, as I said, there's devenv.local.nix, which is something that is not committed to Git repository, where you can override something. There's the YAML file, and then there's the log file.
06:21
This is actually, devenv uses flakes, nix flakes underneath, and this will uniquely pin down your environment that you should commit to a Git repository so that if it's generated every time somebody uses devenv, then you will get a different kind of pinned inputs. And then there is that nfrc,
06:41
which is the devenv glue code that allows you to hop into different folders and activate and deactivate these environments. As I said, there is garbage collection built in. Yeah, there's many details to it. There's a bit of a, it currently works pretty slowly,
07:01
but I think Teofan helped to improve that, and let's see, we still have to work on improving these bits. We have right now 36 languages that you can kind of just get a basic tooling for. So you say, languages Python enable, languages Python virtual enable, and it will create this for you.
07:22
TypeScript, and I think most of the top 20 used languages are supported. All right, so to give you a bit more advanced example, for example, Rust, you can say enable Rust, you can pick a version, your difficulty level,
07:40
nightly, latest, stable. We can define some pre-commit hooks. So for example, if you have formatters, linters, and so on, you can just flip a toggle and it will kind of configure this for you. And for example, you can specify packages. In this case, it's a bit of an advanced case where you can say that enable security SDK from Apple
08:03
in case we are running this on Mac OS. And if we're not, then this package will not be added on Linux, for example. There's quite a few people who use it for PHP nowadays. So in PHP, one interesting thing is like if you enable debug, PHP gets really slow.
08:21
So what you want to do is, you want to have a PHP with debug and without debug, and sometimes you want to use the debug version and sometimes not. And here you can see, you can specify the INI file, which version of PHP to pull in, which is kind of useful when you have different projects that require different versions, and which extensions to enable and so on.
08:46
But that's kind of like just providing the basic tooling. What's usually useful is when we have an environment that we want to start some processes, development processes or whatever. So of course, that's also included as a declarative interface. So you can set its processes
09:01
and it's kind of like bash code that you execute. And the implementation for processes is pluggable. By default, it uses Honcha, which is like a four-man implementation and written in Rust or Python? Rust, I think. Python, thank you. But you can plug that in. There's like Overmind and Havemind,
09:20
which will spawn tmux processes that you can then enter and exit and so on. So you can pick your difficulty level there as well. And then we can go one abstraction further and we can say, okay, now we have these processes. What if we just create recipes that it will just automatically configure the process for us?
09:42
So if I say, for example, Redis enable, this will just start a Redis process with some default configuration. And then you can kind of like configure the Redis port or PostgreSQL port and so on. So any software can be kind of abstracted on top of those processes. And we support a bunch of them.
10:00
I haven't even listed all of them here. How much time do I have left? Yeah, so, and then there's like a bunch of ideas behind like, for example, we have like a scripts kind of entry point where you can kind of define declaratively a script.
10:21
In this case, I defined my script. And once you enter your environment, you can type my script and it will execute this code. And in there, you can reference the packages that you specified before, or you can even hard code them with the Nix, like, how do we call it now? String interpolation feature, yes.
10:42
Just to be called anti-quotation. Here you can, there's like a nice, I think it's called hostctl command line that you can kind of manage hosts on Mac OS or Linux, like kind of as an abstract interface. So you can configure hosts.
11:01
Again, this does not use containers behind it, so it will actually manipulate your machine's hosts. And I think we have kind of a prototype to provision certificates in this way as well. But there is no limit to it. You can take any software, for example, I don't know if you know difftastic,
11:21
which is like a synthetic, kind of like a diffing protocol that is, understands the language and it tries to do a diff in a semantic way. You can just flip a toggle for your project and it's available for anyone that uses this. There is dev container, for example, also there that you can use and so on.
11:41
So any software can be turned into a toggle where you can just flip that thing and your environment has this now available. All right, that's it. I thought, I went briefly through all the features of devenv and you can go to that URL.
12:02
There's a ton of documentation. If there is anything unclear, please let me know. There is also a link to, I think we have a Discord where you can complain about the lack of documentation or if something is unclear. It's really important to me, so please let me know if you get stuck or if it's unclear.
12:21
And yeah, I hope that you'll find it useful. Okay, we have five minutes. Hi, Dominik. Thank you very much. You haven't mentioned the Flakes integration.
12:42
How much serious you are with supporting moving with Flakes? I've submitted all the issues that doesn't work. Yeah, so Flakes integration for me. Repeat the question, please. Oh, to repeat the question. So the question is, there's also Flakes integration.
13:01
You don't have to use the common live of devenv. You can use the experimental Flakes. With the devenv kind of like modules. And the question was how serious I am. I'm quite serious, I don't use it myself, so I'm serious about let's get it in, let's get the tests there, and let's fix all the issues.
13:22
I think we fixed quite a few in the last month. Devenv was created in October, and in November, people started using it widely, and we did a lot of work in the last, what, month? In two months. So the Flakes still need a bit of polishing, but I'm quite serious about it,
13:40
because I understand that some people want to use more than just the developer environments, and that's great. But I think to getting the tests in, that's the crucial bit.
14:03
Like how close is the relationship between the next build, and what the edit and dependency from that source? Right. So the question is how does Nix interact with, if I understand, like kind of patching
14:21
some of the things that you're pulling in, right? Yeah, because you can. So the answer is kind of two-phase. One is that Nix is really good at customizing things
14:40
and overwriting things and patching. And you can take any package and apply a patch to it, for example, and it will just, instead of getting it as a binary, it will just compile. But I don't think there is a really good and simple interface for this. I know that like in JavaScript, there is like patch something package, I think, which is really nice interface.
15:00
So I've been looking into how to integrate that so that it's really easy to just take software from somewhere in the world and kind of hack something on top and override it and do that. So my answer is it doesn't really exist in an elegant way, but it's possible to do it right now. Yeah. You mentioned your end, is it already integrated
15:22
or is it a missing feature currently? Your end is integrated as long as you, yeah, sorry. So is your end already integrated? Yes, it is. It's as long as you install it, then it will kind of work out. When you say, for example, dev init
15:42
and it creates a template, it will already use it and so on as long as you have it installed. So is it like use flake or something like that? No, it will just generate a template for you. So that's it. You don't have to do anything. All right.