Nix at Home - Configuration management for your House!
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 27 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/39599 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
NixCon201817 / 27
3
5
6
7
10
13
16
19
20
22
24
25
00:00
Data managementLevel (video gaming)Enterprise architectureMultiplication signSoftwareVirtual machineLaptopConfiguration managementEnterprise architectureLatent heatScaling (geometry)Level (video gaming)Control flowMathematicsSoftware testingDirectory serviceSoftware repositoryType theoryServer (computing)System programmingoutputGodPhysicalismComputer configurationOnline helpState of matterRepository (publishing)FreewareShared memoryLecture/ConferenceMeeting/Interview
04:00
Software developerData managementEnterprise architectureLevel (video gaming)Computer animation
04:19
Software developerComputer hardwareModul <Datentyp>Content (media)Power (physics)File formatLatent heatIntegrated development environmentRouter (computing)FamilyKey (cryptography)Computer networkLaptopElectronic data interchangeCodeSystem programmingElectronic visual displayScripting languageMachine codeMultiplicationConfiguration managementMaß <Mathematik>Service (economics)Web serviceDemonRight angleKey (cryptography)Server (computing)Computer fileVirtual machineDifferent (Kate Ryan album)Repository (publishing)Type theoryConfiguration managementRevision controlLaptopSoftware repositoryComputer hardwareModule (mathematics)Scripting languageMixed realityLatent heatPlug-in (computing)Default (computer science)Tracing (software)BuildingResultantSystem programmingBootingCodeDemonExecution unitOperator (mathematics)Remote procedure callWeb serviceSoftwareSlide ruleBinary fileRule of inferenceoutputService (economics)MehrplatzsystemData miningMultiplication signMathematicsNormal (geometry)Installation artIP addressTurtle graphicsCloningAttribute grammarPasswordGoodness of fitDirectory serviceCASE <Informatik>Shared memoryRootkitProfil (magazine)Router (computing)Electronic mailing listProcess (computing)Public key certificateInternetworkingStructural loadContent (media)String (computer science)File formatMappingMachine codeLink (knot theory)SimulationPhysicalismPoint (geometry)
13:13
System programmingComputer programmingElectric generatorScalabilityProduct (business)Service (economics)System programmingMultiplicationProfil (magazine)
13:48
Modul <Datentyp>System programmingRadio-frequency identificationCodeImplementationComputer configurationUser profileSoftware design patternConfiguration managementConfiguration managementComputer configurationImplementationMaxima and minimaProfil (magazine)Software repositoryAbstractionTouch typingHacker (term)System programmingCodeMultiplication signProcess (computing)CodecSoftware design patternDirectory serviceSource code
15:19
System programmingConfiguration managementServer (computing)Computer networkVirtuelles privates NetzwerkWind tunnelRight angleSoftware testingState transition systemOpen setLoginRouter (computing)SpacetimeSoftwareControl flowServer (computing)PlastikkarteSoftware repositoryPoint (geometry)Stability theorySemiconductor memoryPlanningMiniDiscBootingComputer animation
17:56
BefehlsprozessorPower (physics)Router (computing)Hard disk driveInterface (computing)Default (computer science)Solid geometryDrop (liquid)BootingVirtual LANWhiteboardState of matterSoftwareSystem programmingVirtuelles privates NetzwerkMusical ensembleRoutingLaptopFerry CorstenFirewall (computing)Server (computing)Public-key cryptographyModule (mathematics)Configuration managementBitSource code
Transcript: English(auto-generated)
00:00
All righty, so it's time for the last talk of the day and so we've already heard about like Nixle set scale and deploying to large physics things and God knows what But similar is actually going to talk about Nix at home configuring Nix for your laptop and
00:21
Yeah, let's start Hello everyone I'm Samuel leathers. I work at IOH K I'm a senior DevOps engineer there And I will be talking about Nix at home Configuration management for your house or I like to subtitle it how I use enterprise level tools to have sanity at home
00:45
Credit where credit is due Graham Christensen and Clever helped me out a lot with getting this in the state it is now So thank you very much for all your help over the last couple years getting this repo where it is
01:02
So why Nix is great for home devices most of us don't have dedicated test networks to test changes out breakage is scary with anything else with Nix just roll back and It's free and easy CI to notify you when things break before you deploy so Types of configuration management with Nix this is how many of you actually use Nix at home
01:28
Almost everyone. Okay, so you probably all know all this stuff So I'll run through it pretty quickly, but on a laptop you have a few options You can do Nix OS rebuild switch
01:41
From a git repo clone Datsy Nix OS and then just manage all your stuff using git. That's nice But what if you have two laptops and you want to share stuff between those? You can do Nix ops deploy from git repository Cloned anywhere so you can clone it in your home directory and do Nix ops deploy to your localhost or to other servers
02:02
in your house But you still run into the state issue that was mentioned earlier With Nix ops where you can only do it from one machine at home. That's usually not too big of a problem though And then what if you have OS X machines and you want to remote deploy those because who wants to
02:22
Actually log into an OS X machine and run Nix Darwin rebuild not me So and then what if you have other Linux systems that are not Nix OS, but you really want to run something Consistently using Nix OS on it. So like if you want to run Prometheus node exporter on centos
02:43
But you don't really want to have to deal with the hassle of updating young packages and everything So all those things we're going to be talking about here today So for laptops managed locally we want to share some configuration we want some stuff specific per laptop and we want to be able to rebuild a laptop quickly or spin up a new one and
03:06
We usually have bleeding-edge software here. Some of you might not I usually run Nix OS unstable on my laptop We have some custom commits cherry-picked That are not on my not in yet sometimes from pull requests or something
03:22
And we want to easily test different channels At the same time. So here's how I kind of do it Etsy Nix OS that's where all your configurations go. That's just a is it not going. Oh
03:42
No
04:21
There now we're going Okay, so how I do it Etsy Nix OS is a clone of my network repo on my github repository These slides will be shared later. If you want to look at that There's plenty of good stuff in there and I'm open to any questions later about it For configuration dot Nix. I actually do a sim link to machine slash host name dot Nix so I can have the same
04:46
Git repository clone to multiple hosts that have different configurations and hardware configurations. I do the same thing It's assembling to hardware dash configuration slash host name dot Nix If I want to share stuff between modules, I have a modules directory that contains profiles roles and custom services
05:07
I'll talk more about that a little later I include it with custom modules equals import modules module list dot Nix and then custom modules Is added to the imports attribute in the configuration dot Nix file?
05:20
After defining a profile I can just do profiles dot vim dot enable equals true And now I have my special vim configuration that I want on all my systems For public shared content, I just create a Nix file called shared dot Nix that just has Content in it. So it's just a huge attribute set
05:41
So I put things like my home CA certificate there and my public SSH keys something that I don't care being in the internet Where everyone can see it, but it's still something I want to split across multiple things But what about the secret stuff like the passwords you don't want everyone to know so I create
06:07
It I can get ignore I create a secrets dot Nix file In the same format as shared dot Nix and then I add a layer of misdirection for CI using load dash secrets dot Nix, which just does a
06:22
Built-ins path exists secrets dot Nix then import it. Otherwise use this other Attribute set that basically usually defines empty strings for everything. So Hydra continues to work For Home deployments. So like I have a server and my main router at home both run Nix OS
06:41
So that's easy. I just use Nix ops deploy for that I have an infrastructure dot Nix file that lists all my infrastructure files and you'll see that right here and Then I can use the same Secrets type stuff there as well as the shared stuff and the custom modules that I use on my laptop So now all that stuff is shared across all my systems
07:02
So if I change it in one place It can affect everything Which is great when you add a new vim plugin and you don't want to have to SSH into all your servers and install it again So this is just a general Nix ops config here with the network and the name of the server and the deployment
07:20
This is the none type. So there's no automatically creating it because it's a physical thing at home And then I can add deployment keys in for secrets that I don't want to be living on the server after a reboot So it has to be deployed again To create the deployment we use the Nix ops create command and you can basically give the deployment a name and then you
07:47
Can specify I pinned the Nix pkgs version? to a specific version that one I believe it was the tip of 1809 When I created the slide deck here
08:01
But basically you can specify dash I Nix pkg is equals and then the path to the tar ball on GitHub and then that will pin it so you have to run Nix ops modify then Which is shown just below that to update it to a new version later And then you just do Nix ops deploy home. That's simple enough deploy dash d home
08:24
and Then for remote OS X deployments, I don't use OS X very often even though this is a Mac It's running Nix OS like ninety nine point nine nine nine percent of the time. I do have another laptop at home That's a MacBook Air that's just thrown in the closet somewhere that I don't want to climb over everything to get to
08:41
Or I don't really want to be and see into it either or SSH and manually run stuff So Nix Darwin is great for Giving you the same configuration management on Linux on Mac OS so Buddy of mine at IOHK Rodney. He's not here today, but he created a really cool tool
09:04
for us there that basically Prepares a system using Haskell installing Nix installing Nix Darwin and everything and then runs deployments Remotely using turtle SSH and I did some improvements to that I won't go into the code here as this talk isn't about Haskell, but Nix
09:26
So remote OS X deployments the premise is to build two tools one called prepare and one called deploy These two tools remotely SSH into the host and handle the Nix Darwin stuff prepare installs Nix and multi-user and deploys Deploy deploys new changes a caveat here is Nix Darwin and Nix OS modules don't always have the same design
09:47
So some breakage may occur if you share configuration Code is in Nix Darwin tools in my repo The prepare script is slightly outdated if you want to do this I highly recommend looking at the IOHK version of this stuff, even though it has a few IOHK specific things in there
10:05
It actually installs with multi-user now with prepare I haven't updated that in my repo because I ran repair since the default change to using single-user on OS X again And it requires an OS X build slave to build the tool
10:20
Running prepare will nuke all traces of Nix from the remote system so don't do this on a system that you've like very carefully crafted your pet on and For deployments you just do Nix build dash a tools and then you can run result bin prepare Mac And then the IP address of it and then result bin deploy rule
10:42
Role and then the role is just a Nix file. That's like a configuration dot Nix file that you'd have on your Darwin system and Yeah, and then you can use the same modules you're using for Nix OS as long as you write them in a way that They work on both
11:00
Darwin and Linux For Linux deployments. This is kind of cool, but I kind of abandoned it because everything runs Nix OS now at home So I don't have much use for it anymore But I wanted to share this anyways, because I'm sure there's a lot of people here that might benefit from it the initial premise was to set up Prometheus node exporter running on non Nix OS Linux systems like CentOS 7 and
11:26
To do this I basically it looks like a normal Nix OS configuration that you'd run Nix OS rebuild on But you do an import of Nix pkgs Nix OS and then all it only supports services
11:42
So it only does system D services. It doesn't do users This is probably good as it would probably Break other things on the system if you started trying to mess with Etsy password and whatnot on a system That's being managed some other way and then you just specify your services out and then there's this recursive attribute set that does
12:05
some build ENV stuff and maps the paths to it and it literally dumps out a so you can then Nix build that on the system and if you do find dot slash results, you'll see everything in there
12:23
All you're gonna see are service files and then Nix ENV dash P specify Where your user root is for it? So in this case I have user root centos monitor and then I specify the file I want to
12:42
Install there and I specify that I want to grab the all units attribute here And then I just sim link that to Etsy system D system centos monitor do system CTL demon reload system CTL start Prometheus node exporter and I now have Prometheus node being managed with Nix code that could be automated further using
13:03
Haskell to like write a tool or Python fabric or any other automation tool that uses SSH to Make this a simple process very similar to how the next Darwin stuff above was done This works I tested it before I wiped my last non Nix OS system
13:22
New runs of Nix ENV create a new generation so you can roll back with this too, which is pretty cool You really just need the Prometheus node exporter But I included other services so you can see that it's scalable and run multiple services this way To use in production. Like I said, I'd recommend using some other
13:41
Deployment tool rather than just manually SSHing in and running Nix ENV. That's never a good idea So the custom modules profiles and roles I said I was gonna touch back on later I first used the profiles and roles design pattern with puppet in a previous job If anyone's used puppet, they've probably heard about the profiles and roles design pattern
14:04
When used properly each system should only have one role and a role should only define profiles My repo does not follow this at all. The initial code came from a repo by offline hacker But it's pretty different from his original implementation in essence both profiles and roles are just Nix OS modules with a different prefix
14:24
I don't use roles a lot at all and my repo I have a base role that doesn't have an enable option for things I want on any system But that's pretty much all I use roles for Profiles are glue around Nix OS configuration So for example something like profiles vim enable equals true abstracts away lots of vim configuration setup
14:43
And most profiles have an enable option if you don't have one It's gonna be on all your systems, which is probably not what you want and My goal here is to clean up all the stuff and my legacy and Nix configs directory Which is kind of a mess and have just like the bare minimum and Nix configs and everything else is just profiles
15:05
But I just haven't had time to play with it yet and These can be shared across all Linux OS X systems So you might have to do some conditional things based on how Nix Darwin does something in Nix OS does something and then most
15:21
Importantly we need CI to test everything so Hydra to test everything I define a Nix OS funk stable Nix OS funk unstable and a Nix Darwin funk Unstable that point to these that are defined in Hydra if you want to see how they're defined in Hydra you can look at my
15:43
Hydra configs repo under the same username on github And then Nix Darwin tools is what I told you about with the tools to actually deploy and So those are all tested as well, so when they break from some
16:01
LTS upgrade and latest Nix I know about it and this will actually test with every single commit that hits Nix OS unstable for serov and The Mac one for all the unstable commits and
16:21
For my servers optina and portal it will hit 1809 any commits that hit it it will rerun and make sure everything's going so I know as soon as Something upstream breaks something that I did that I need to fix And some other cool features about my network repo that I not gonna get into a whole lot of detail
16:43
Here, but I have full I pixie network booting. Thanks to clever. He really helped me with that one I have full IPv6 compatibility on my router my routers fully running Nix OS and I have wired guard VPN tunnels open VPN tunnels Prometheus monitoring and I had an elk stack for central logging by disabled it
17:02
Because of lack of resources on my server and just like running out of memory and disk space And it was like I don't need to keep all my logs anymore That is pretty much my talk here I can jump in I see we're at about 16 Well, no, we're not at 16 minutes because we refreshed this so I don't remember how far and we are
17:25
Okay, so we can jump in here and I can walk you through some of the stuff I've done in my repo that requires Me to mirror this though. No, I had a different plan. That's right
17:41
We are going to seven not attached to the same tmux session. That's not good
18:29
Let's just exit this one and try a tmux attach There we go
18:48
Typing one-handed and holding a microphone is very difficult Let's look at the router config a bit here because this might be interesting to some people that want to
19:15
There we go That might be interesting to some people that want to run a router at home so we can look at that
19:25
and The default dot nix. I basically define some internal interfaces these are V lands on This interface here and then I have some wire guard stuff and those are my VPNs one of the cool things here is
19:42
The I pixie stuff I mentioned and this basically Creates a TFTP route that I pixie can boot And builds a nyx OS module nyx OS System that basically can be booted over the network
20:02
And then I define interfaces here for the firewall a lot of this stuff came from Graham I have some cool extra commands here for like drop port no log Except port on interface ford port to host And then I can basically just map across ports and then yeah wire guard
20:26
Interfaces those are public keys, so don't worry about it Does anyone have any specific questions about this?
20:41
My router does not boot from the network My router is running a TFTP server so I can boot any laptop over the network So It's very useful for installing nyx on things. I've been playing with trying to get raspberry pi
21:01
to network boot as well, but that's a little more difficult because you have to Actually Disable some GPI opens Michael can tell you more about it later if you're interested. He was showing me some of it But Yeah, my router does not network boot. It's it has a solid-state hard drive in it and it
21:27
Has monitoring on it and all that good stuff Any other questions? Well, I guess we can all go drink beer then