NixOps for Proxmox
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Anzahl der Teile | 19 | |
| Autor | ||
| Lizenz | CC-Namensnennung 3.0 Unported: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
| Identifikatoren | 10.5446/50060 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
|
Nixcon 202018 / 19
2
3
7
8
12
14
15
17
18
19
00:00
Virtuelle MaschineInformationsspeicherungVirtuelle RealitätDistributionenraumEmulationZahlenbereichVersionsverwaltungWrapper <Programmierung>KonfigurationsraumRechnernetzMini-DiscInterface <Schaltung>DatentypInstantiierungBeweistheorieJensen-MaßMaßerweiterungPlug inPartitionsfunktionSkriptsprachePhasenumwandlungInstallation <Informatik>Interface <Schaltung>MathematikVirtuelle MaschineArithmetischer AusdruckApproximationRechenzentrumCase-ModdingUltraviolett-PhotoelektronenspektroskopieGeradeMultiplikationsoperatorARM <Computerarchitektur>InformationsspeicherungMixed RealityBitDistributionenraumSoftwareentwicklerVirtualisierungt-TestSkriptspracheTransformation <Mathematik>Reelle ZahlWhiteboardVererbungshierarchiePerfekte GruppeWeb ServicesTelekommunikationRoutingInstantiierungTypentheorieProdukt <Mathematik>Kartesische KoordinatenSoftwareRechter WinkelPartitionsfunktionSystemaufrufKategorie <Mathematik>MereologieAusnahmebehandlungMailing-ListeNetzadresseWarteschlangeVerschlingungMetropolitan area networkZentrische StreckungDienst <Informatik>QuaderInformatikFaserbündelProzess <Informatik>Snake <Bildverarbeitung>Mini-DiscInverser LimesInformationRationale ZahlIRIS-TPunktAlgebraisch abgeschlossener KörperFront-End <Software>Automatische HandlungsplanungIterationStellenringArithmetisches MittelServerPlug inKonfigurationsraumImplementierungCD-ROMCASE <Informatik>BenutzerbeteiligungModul <Datentyp>DatenverwaltungPhysikalische TheorieMechanismus-Design-TheorieComputeranimation
07:59
Virtuelles privates NetzwerkVirtuelle MaschineSkriptspracheDeklarative ProgrammierspracheWeb-SeitePartitionsfunktionBenutzerfreundlichkeitJensen-MaßATMSpezialrechnerSicherungskopieEmulationMehrwertnetzRouterKnotenmengeRegulärer Ausdruck <Textverarbeitung>ChiffrierungMini-DiscDatenverwaltungPunktSoftwarePunktwolkeDemo <Programm>ComputerspielAlgorithmusDateiverwaltungProgrammbibliothekBootenPunktVererbungshierarchieAusnahmebehandlungMathematikVirtuelle MaschinePartitionsfunktionSoftwareBildgebendes VerfahrenFront-End <Software>Objekt <Kategorie>FehlermeldungMultiplikationsoperatorEndliche ModelltheorieAggregatzustandPlug inSkriptspracheChiffrierungTabelleImplementierungMessage-PassingArithmetischer AusdruckDeklarative ProgrammierspracheDatenverwaltungCoxeter-GruppeJensen-MaßProgrammfehlerDistributionenraumDefaultDienst <Informatik>VersionsverwaltungStochastische AbhängigkeitPunktwolkeMini-DiscZustandsmaschineRoutingWikiKoeffizientRechter WinkelDesign by ContractCASE <Informatik>Physikalisches SystemGemeinsamer SpeicherWeb SiteAutomatische HandlungsplanungProzessautomationInstallation <Informatik>SymboltabelleMereologieARM <Computerarchitektur>WarteschlangeUnrundheitProdukt <Mathematik>Alexander-PolynomWeb logPhysikalismusKoroutineWeb-SeiteWort <Informatik>Ultraviolett-PhotoelektronenspektroskopieLesen <Datenverarbeitung>QuellcodeWurzel <Mathematik>NebenbedingungReelle ZahlInklusion <Mathematik>Mobiles InternetMehrschichten-PerzeptronFestplatteJSONComputeranimation
15:59
SoundverarbeitungMechanismus-Design-TheorieResultanteARM <Computerarchitektur>TermStapeldateiEinflussgrößeTabelleMultiplikationsoperatorEinfach zusammenhängender RaumGruppenoperationBitVerschlingungAlgorithmusTrennschärfe <Statistik>ComputeranimationBesprechung/Interview
17:23
DatensatzBesprechung/Interview
17:43
Chatten <Kommunikation>Besprechung/Interview
Transkript: Englisch(automatisch erzeugt)
00:00
We have NixOps for Proxmox by Ryan Lafa. In this talk, we're going to explore how to build and manage dozens of Nixos virtual machines in a Proxmox cluster using NixOps in declarative fashion. A little bit about Ryan, if you're not familiar, he has been a false developer for more than five years,
00:20
and he is a student in mathematics and computer science. He enjoys playing around with infrastructure and he runs his own micro data center using NixOS. Hello, everyone. I'll introduce you to NixOps for Proxmox, which is a plugin for NixOps using Proxmox.
00:43
First, we will see what is Proxmox, what is NixOps, and what are backends in NixOps. The actual Proxmox backend, the challenge I encountered reading this backend, and what can be built on the top of this, and what can we learn from this kind of experimentation.
01:00
Finally, there will be a demonstration showing how to deploy SIR.ht on your own infrastructure using the Proxmox and NixOps. Proxmox is a distribution made to deploy virtual machine in containers. It's using KVM and QEMU,
01:21
and it comes with a nice web UI and API command line. It handles for you a lot of details that you don't want to deal with, such as storage backend, multi-clusters, and it's being actually used in production on real-world deployments.
01:41
As for NixOps, it's a Python application that enables you to transform Nix expression into real VTC. That means that, for example, if you read down what is supposed to look like your infrastructure and you have the right backend,
02:01
so a plugin that implements the right API call to transform Nix expression into reality, then NixOps will ensure that the reality will match what your Nix expression describe. That's the theory. In fact,
02:21
it works pretty much well, not really good at large scale, but it's super interesting. One of the thing I like about it is super modular, meaning that you can read down backends for whatever you want, as long as you have an API or something like that.
02:41
For example, there are Amazon Web Services, Google Cloud, Azure, Eddsworld, Liberty, VirtualBox, and support trivial NixOS machine with a necessary server. A backend is basically a plugin in NixOps that implement a certain part of the NixOps interface and bring the definition,
03:04
so think of Nix expressions, think an expression of a resource like disk into reality. If you had a disk in your definition, the backend will be responsible to bring this disk into your infrastructure by creating it,
03:21
then attaching it to whatever resources you have that mentioned this disk and doing a lot of things on it, depending on your actual implementation. For example, here, I'm taking simple Amazon Web Services NixOps expression, which deploy a machine called machine,
03:43
and it's deployed in a certain region and for a certain instance type. It imports also information about my Amazon Web Services account, that's the EC2-info.nix. What is interesting is that if you build on the top on this Nix expression,
04:04
and then someday you say, oh, I need a beefier machine, you could just change the instance type, and everything would work more or less fine. So that's an interesting thing. Well, other software implements it also in the Nix ecosystem.
04:21
But actually, I'm going just to use NixOps and Proxmox. So I start from the fact that there are no actual real backend for private infrastructure. Liveverde is not enough on its own because it doesn't implement as much things as Proxmox does.
04:44
So after some time of installing NixOS manually using the live, I decided to write down an NixOps plugin, which made sense, and I wanted to validate if it was the right idea, and if NixOps was flexible enough for my use case.
05:02
And actually, it feels a bit like Amazon Web Services, with a lot less feature, of course. But it has this nice property of being a bit declarative, and being able to map over machines and deploy them automatically extra. So I think, in my opinion,
05:23
the experiment is a success because I really want to extend this plugin and go further. And that's actually possible thanks to modularity of NixOps. I will just outline for the talk how the plugin works. So it's not super perfect, of course.
05:43
But basically, it just creates what it needs to create first, so a disk, then virtual machines, then it will attach a CD-ROM at the same time. It will wait for QEMU agent to be up. So QEMU agent is an out-of-bound communication mechanism
06:00
that you can use even if you don't have networking yet at this step. So that's super useful because I'm not sure that I can have DHCP giving me IP, or I can even route some packets to the virtual machine. So using this agent, I can provision SS-HKEY, pre-power the machine for receiving NixOS on the disk,
06:23
doing the partitioning script, and generating an initial configuration.nix before the actual installation. Then I will check if after the install, everything is fine. And if everything is fine, I will mark it as installed and give the job back to NixOps to do its magic.
06:42
So that means that NixOps will now assume that the target is a NixOS machine and that it can be provisioned like a NixOS machine. So sending some closure and switching of configuration. And also secret sometimes, there are more details, not that important.
07:03
One of the challenges I encountered was that NixOps has no real support for IPv6. So at the same time, as it was super modular, I could just bake the IPv6 support in my own plane. And this is what I've done.
07:21
So it looks like this. I just get some interfaces with our IP addresses from the QEMU agent. Then I just do iteration over it, filter for the private IP, the public one, the v6, the v4. I exclude the link local for IPv6 because I don't want to use them,
07:42
but it could be used. And then I take the first reachable or I don't. That's interesting because it brings me to the next challenge I encountered, which is how to select the right endpoint. If you have that many addresses, how do you know beyond the reachability criterion?
08:02
How do you know which one you choose? So I listed two scenarios that I actually encountered in real life. And personally, I decided to fix this by saying I always have IPv6 on my machine. But that's not an acceptable fix.
08:21
But at the same time, it was interesting. I think that research has some algorithm on that. In fact, you want to optimize your endpoint based on multiple factors. For example, you want an endpoint that is fast, an endpoint that is reliable, and an endpoint that is, of course, reachable.
08:43
So that's interesting. I didn't implement the whole solution, and I'm still looking for an implementation on that. And my nemesis currently is partitioning. So partitioning is not declarative in general. In the installer, you have to do your partitioning yourself.
09:04
There are actually some software that can help to do this. Hezner is using Xpart, for example, the Hezner backend. But it's not that flexible for my use case, and I would like to use every file system I want. So ZFS is out of the question using Xpart, unfortunately.
09:24
So I fell back on classical bash. But it's, of course, super error prone. It's not that bad. But I spend too much time reading the manual page and doing stupid mistakes. So I would like to get out of this model to do something better,
09:41
and I will talk about it later. So the current state of the plugin is it's, of course, buggy. It's not absolutely very poor work, because I use it. It works. But it has to be a lot improved so that it could be used in production, for example.
10:01
There are at least some bugs that I'm sure exist. Some of them are trivial to fix, the state machine one, for example. But the most annoying one to me is I cannot use default NixOS images. And I'm forced to use the QStom one, which is provided in the repository,
10:23
because I need to enable QEMU agent. But if I find a solution for that, I could get rid of the default of the QStom NixOS image. Other than that, it's really alpha quality, so you don't have backup, tag support, you don't have route and robin distribution on your Proxmox cluster,
10:42
so you want to have maybe equal repartition of all of your virtual machine over all your nodes. You could do it yourself in the backend, to be fair. So it's not a real problem. What I will look forward to would be to extend this kind of work
11:03
to other kind of software that you can see in private infrastructure. Personally, I use YOS. And it would be interesting to manipulate your routing in NixOS and acting on the top of it using API extra.
11:22
So that's an experiment I would like to conduct. And bring the routing table to the next expression would be really interesting, so that you could build some kind of DSL on the top of your expression, and then you can express a lot of things in a concise way. And changing one thing doesn't have to force you to find which things I need to change, etc.
11:45
Because you can just propagate the changes automatically to everything that it needs. So that would be really interesting. Also, of course, I showed that I don't have declarative partitioning, but I want declarative partitioning. So of course, it's not a solved problem currently, as far as I know.
12:04
There are some efforts to export. There is an interesting draft of the partition mapper in the wiki of Nixport, which is how to transform if you have a file system into ANOVA, and how to emit the commands to migrate the things.
12:21
So that's super interesting, because it means that you can do live migration, live changes to your file system. But of course, it has to be in a secure way for your data. So I'm not sure on this point, but it's based on the library used by Red Hat people. They're using it in their Fedora installer, and it looks like pretty much solid.
12:45
So I think it's interesting to pursue such a thing. And also, I require it because I would like to have automatic LUX decryption and automatic LUX encryption, which is the next point.
13:03
If you don't have declarative partitioning, it's super hard to automate LUX encryption, except by doing some scripts and stuff like that. But it's error-prone, and it's bash, and passphrase management is not super obvious, et cetera, et cetera.
13:20
So if we would have like secret management a la Ansible, a la Vault using by AshCorp, and full disk encryption with declarative partitioning, it would be really trivial to add fully encrypted guests really easily with automatic decryption using Mandos.
13:43
Mandos has an interesting threat model, which is reasonable, and enables you to decrypt machines at boot time using other machines. So it works by maintaining a table of PGP encrypted passphrase, and you send them the encrypted message by your trusted peers,
14:03
and then it's kind of fully meshed. And it works also with a manual passphrase entry or whatever you want also. So it would enable full disk encryption into data centers, which is interesting. So to conclude, one of my personal objectives is to see more offerings of private flexible
14:25
infrastructure like Amazon services does without having to use their infrastructure. So the thing is that you want to hold your own physical service because you have some constraint, and you want to run software in a nice way. So I think that Nix is an important key towards this goal.
14:45
And contrary to Nix apps, which is well known to be this old piece of software that doesn't work that well with everything, the rest. But I think it's salvageable. The new version looks really promising, and it's not that bad.
15:03
But of course, I can understand the criticisms. Finally, if the future work that I presented could be implemented, it would enable a trivial, easily IPv6-only, NixOS-only, fully declarative, micro data center, which to me would be a definitely exciting first step
15:25
towards independency from the public cloud. So that's something I'm looking forward to. And thank you for watching. And I will be available for a question over ERC2. And don't hesitate to watch the demonstration if you find this interesting.
15:42
Thanks. OK, this will now begin the Q&A portion. So I believe we actually only have one question. And I think our presenter actually saw it as well. So I think we'll just rehash the discussion here. Do you think you could do that, Ryan? Yeah, do you hear me?
16:02
Do you hear me? Yeah, I can hear you. OK, all right. So the question asked by Niku was, why not implementing happy haibles for the IPv6 versus IPv4 connection selection mechanism? And the thing is that I'm not sure I exactly talked about it in my talk.
16:25
But you have multiple criteria that you want to optimize, like, for example, reliability. If your link is working well on the first packet you're sending, is it going to work well afterwards? Same goes for speed and reachability.
16:43
So you have to take in account that you're sometimes roaming. So you could retry doing the happy haibles each time. But I think it's a bit not effect efficient.
17:01
I think there are algorithms in research that shows better results in terms of those criteria. So I think we have to implement and measure actually how it works, really. So that was kind of my answer. And some people re-explained the same things.
17:22
Right, a little extra commentary. Yeah. So I think with that, I don't see any more questions. Yeah, I do not see any more questions. So I believe this will elapse the Q&A portion. Thank you for being here. Thank you for your recording.
17:40
It was great having you. Thank you. Yep, and everyone go offer Ryan in the chat. Come on.