Hi, everyone. So I'm gonna be talking about Unix systems programming in Rust, and why I think it's really awesome. So we all, I assume, agree that Rust is awesome, right? We're all here. We're at RustBelt Rust.

This is RustBelt Rust. My analysis shows that this is the twice as rusty as any other Rust conference. Which makes it the rustiest Rust conference ever. So we're all agreed, right? Rust is awesome. And we are in the last session, and I'm gonna direct attention away from me to the organizers

for five seconds, and just say thanks so much for organizing this, this has been really great. Okay, now back to me. So the Rust here is rough conference ever. We agreed that Rust is awesome, but I wanna convince you that systems programming in Rust is awesome. In fact, I think systems programming is awesome even without Rust, but Rust makes it that

much better. But to get started, we have to start with, what do I mean by Unix systems programming? And I've got a tongue in cheek definition, which is where you spend more time looking at man pages than at Stack Overflow. This is extra useful, because it also means that you don't need an internet connection

to kind of make a good deal of progress. But more seriously, systems programming, especially Unix systems programming, I think is a lot about making system calls, and dealing with the interface that we have to the kernel of the OS that we're running on.

So what are system calls? I like to think of system calls as writing a letter to the kernel. It's like, dear kernel, please open this GIF for me. Thanks. And then the kernel gets back to us with this GIF that we saw earlier today. I'm not gonna leave it up for too long, because it's dangerous.

But the key thing here is that there are a lot of operations that is not safe or secure for our programs to do, and we have to ask the kernel to do them for us, opening files, make sure that we have the right permissions, and so on. So we make a system call, dear kernel, and then it does all of the checking and makes it that

we have that file open. There it is again for a couple seconds. So here are a couple of example system calls. They're not that alien. If you haven't done system programming, they should still be familiar. When we're doing file operations, we have an open system call, which we just saw. It doesn't actually give you a GIF.

It would give you a file descriptor, and you'd have to go and do a bunch of other stuff to get the GIF. But I think it illustrates well enough. But open, read, write. These are all things we expect to do with files. Processes. If we have a shell, it's going to fork. It's going to exec the new process. It's going to... You're probably going to have to kill it at some point because it's broken.

In networking, sockets, and you create a socket with socket, you connect, you listen, you accept a connection. These are all system calls that should sound familiar if you've done programming that is involving networking or files. But you may not recognize them as being something the kernel knows specially about, but they are.

So to give a few examples before going further into why I think Rust is great, here are a couple of examples of things I've done or worked on in Rust in the Unix systems-y kind of world I've just been describing. So one is this thing called ContainyThing, because I'm great at names.

I'm just going to very quickly do a demo. But basically the idea is containers like Docker require root, and I don't like root. So target debug ContainyThing. This is running. So the rootfs directory there is an extracted Debian Docker image, and I'm now root inside a little...

Well, if I can type. I'm now root inside a little container, but I didn't need root to get into the container, which is great. I have ideas for where I'm going to use this to allow using Docker and Travis CI sometime soon, hopefully, without it being slow.

But this is a cool little thing. You end up learning a lot when you do something like this about how all of these weird systems are put together. So that's my little project ContainyThing, which I've just come back to after months of hiatus.

Ruby Stacktrace. If anyone was at the RustConf, this came up briefly in Julia Evans's closing keynote. I wasn't able to be there, but I did feature in one slide. That's me. I know Rust. Julia wrote her prototype in C, and C is hard.

And she had reached a point where things were becoming more and more hairy. And I keep talking about Rust, I guess, so she asked me if I could help. And so we translated it into Rust, and then it became much easier to work with. And I can show you what that looks like too.

So here I've got some silly... I don't know Ruby, so this is the best I can do. It's just some recursive calls and they sleep and stuff, and what the Ruby Stacktrace program does is it goes and roots around inside the memory of the Ruby process, and

reads out the Ruby stack, and so it's able to see where time is being spent in that Ruby process. It also is able to do flame graphs, but that's a bit more involved, so I'm not gonna demo here. But this is kind of sampling, I don't know, every 100 milliseconds of what is that Ruby process doing? Whoop.

That is... Go away. Okay. So silly things are okay too. So those are kind of serious projects that could be useful for build systems or in production or something. But at BangBangCon, which is this really awesome conference in New York, which is lots of 10 minute talks of people sharing something exciting about computers.

I on stage broke my computer with pipes with a Rust program by invoking the out of memory killer by exhausting the amount of memory my computer had. I'm not going to demo that one because that actually breaks my computer. I'm using it.

I did break my computer on stage because it was the end of the talk here, we're in the middle of the talk, it's dangerous. But silly things are also an excellent way to learn a lot more about how your computers work. And the reason I think Rust is really great for systems programming comes down to a few things. We're familiar with these slogans a little bit.

Like Rust lets you be fearless, and a good analogy, or not analogy, example I think of this is when we were working on the Ruby stack trace program, and Julia had reached this point where C was getting hard. Also I think Rust is really convenient. I'll go into a few ways, a few examples of what I mean by that in a bit, but if you've

ever written enough of a C program where you'd like a hash table, that's the opposite of convenient. Then also Rust, because of the new runtime and having direct access to what's going

on in the C world, we also get transparent access to the underlying OS, we can kind of bypass the standard library. So to go into a few examples of what I mean by Rust letting us be fearless, so systems programming is traditionally done in C.

There's millions and millions and millions of lines of C code, and there are a bunch of things that are really scary about C if you don't actually know C like me. So there was a person on the first day who said they spent 20 years writing hairy C programs. That's not me, and that's not lots of people, and getting simple things like error...

Well, simple. But error checking is hard, but error checking is much easier in Rust because of the result type than in C where it depends on the library and the program that you're using, exactly how that's handled. There's no standard. Memory management. This is kind of the main reason we translated the Ruby stack trace program to Rust was

because it has a lot of strings, which are the names of functions and the names of files, and it's easy to allocate memory in C, malloc, but safely deallocating that memory is another story. And so if you want your program to kind of not explode your run out of memory, not having

to worry about that as much is great. So Rust lets us allocate in free memory and also not have these dangling pointers because calling free at the wrong time and then referencing a pointer, and then lucky, if you're lucky, you get a segfault, if you're unlucky, I don't even... Just don't.

So in addition to allowing us to be fearless, Rust is really convenient. And here's a really great example as compared to C, Rust makes working with strings not painful. You don't have to remember if you're supposed to stir copy or stern copy or stir... Or whatever all those other stir things are, stir talk to split by spaces, and some of

them are safe and some of them are thread safe and some of them are just really, really don't do that because you're going to have buffer overflows and it's just hard. In Rust land, we have strings and stirs, we can split, we can up case, low case,

all of these wonderful things that you want to do with strings. They're easy. Hash tables. I already mentioned this, but really, hash tables. Like, just having hash tables or maps available to you is... There's a point in...

When I've written little bits of C, there's a point where I'm kind of inventing my own Asoc kind of list of key value pairs and going through it, and then it's annoying, and I wish I had hash tables, but I don't know how to put hash tables in C because I don't know C. Rust just obviates all of it. And then, of course, this is where everyone in the room can come in, is crates.io.

You want some functionality that's not in the standard library, there's probably a crate that will handle it in a best practices kind of way. There's a map crates and signal handling crates and crates for all kinds of things.

And this, I think, that the ability to use somebody else's library by adding one line to a config file, that's also really, really convenient. So moving on to the next thing I think is great, which is that Rust allows transparent access to the underlying OS, and in the standard library, we do have stdos-unix.

This is a kind of grab bag of ways to add Unix extensions to various types. This lets us get file descriptors from files and sockets, it lets us do some extra things

with processes to set up the environment a bit more specifically before we run a command. Oh yeah, strings in Unix are byte strings, and so there's interoperability between oster and str that isn't available, or between OS string and bytes that's not available

on Windows because of the WTF thing that I still don't understand. So that's great. But we can actually go... We can get around the standard library and get even closer access to what's going on below, because we have this fantastic compatibility, we can have C functions that are called transparently

from our stuff, we can have structs that mirror the C struct exactly, and just call directly into libc or into whatever other C libraries that are helping us do what we need to do. So this lets us drop down below the lib standard. And I'd be completely remiss here if I didn't bring up misordering my slides.

Sorry. So examples of where the dropping down below lib standard come in is where we want to go more specific than just Unix versus not Unix, but into specific things about which

flavor of Unix. So the examples that I gave with containy thing is using these kind of container-oriented system calls like unshare, which does very weird things. I highly recommend reading the... I think there's a series on the Linux weekly news about what all this namespace stuff is.

It's kind of mind blowing. There's mount, which is how I was able to change the view of the file system inside the container. In the Ruby stack trace, we're using this really... I don't know how I feel about this system call. It lets you just read another process's memory.

It's just like, hey, I want to read 10 kilobytes out of that part of that person's memory, and then you can just read it, and then you can do whatever you want. So that's what that stack program is doing. It's like... Let's just copy that Ruby memory over here, and then now we can do whatever we want.

We can figure out what was going on. But this is where I was about to go, which is I'd be remiss if I didn't mention Nix, which is a library I help maintain, among others, one of whom is in this room and missing. But anyway. This is a library that rustifies libc.

If you've written C systems programs, there's some slightly annoying parts where error handling is not as nice as you would like, especially from the Rust side. I have a question for the room, actually. What is the right word for this? Turning something more rusty-like?

Rustifies? Oxidize? Anyway. Making libc nicer to use from Rust, right? So the example I mentioned there, result instead of returning minus one and looking at Erno, and maybe you forget to do that and things go wrong, in some sense, we kind

of read the man pages, so you don't have to, because there are some cases where the exact return value that indicates an error is not minus one, maybe. Maybe it's like, you know, plus one? I don't know. But they exist, and it's hairy. We translate it to a result, having read the man pages.

You probably should read the man pages, though, really. If you don't want to break your stuff. And we use Rust types where they make sense. So just a couple of examples of this is tuples instead of int2 for pipe, and see if you want to create a pipe where you get two file descriptors, you have to create an array

on the stack and check that it's minus one and check if it failed and then return minus one. All of that in Rust is just like, try pipe. Or in Rust soon, it's a pipe with an upwards intonation at the end. And this is like, you know, you just get your read-write pointers.

You don't have to remember that the zero... Well, you have to remember which one is the read and the write. I hope I got those right. Yeah, I did. But, you know, referring to pipe FD0 and pipe FD1 kind of gets old. So I typically end up renaming them as read and write anyway. It's great to do that in one line instead of six.

This is one of my favorites, is using slices instead of void star and size pairs. So this is like in system calls like read and write, where you're passing a bunch of memory over and something's supposed to happen to it. Or in the process VM read V1 I gave as an example, where you're saying please get

me that much memory from that other process. It's all void star size pairs. And actually, I'm just gonna go on a little aside here, because I love slices. I think that slices are one of the best things in Rust. They're a standard way to pass pieces of buffers that aren't necessarily yours to

someone else, and everything just works out. Like, standard means it's ecosystem-wide. This is so, so, so important. I've done some C++ programming, and there's no standard. There are just, you know, in Google code, there's string piece in...

I don't know what's in Mozilla code, but I did a little bit of cabin proto code, and there's like an array pointer. They're all start length or start end pairs, but they're not standard. You're gonna have to write all these interconversions if you ever want to reach across into a different part of the ecosystem. And that's just really annoying.

C++17 will, hopefully, have a string view, which will be kind of like a stir slice. That's 2017. It still won't protect you from lifetime mishaps. And the fact that it's 2017, and C++ is gonna be by then 34 years old or something... You won't actually be able to use it if you're interoperating with any other code.

Rust had slices from the beginning. This means that this idea of slices, of transparently being able to pass memory across library boundaries, across into some crate you just got out of crates.io, being able to use a part of a memory mapped file and just pass it to some crate that knows

nothing about memory mapped files. That just works, because we had slices from the very start. And this is a bit extreme, but I think if all ownership, like the whole borrow check stuff got us, was a safe way to pass bytes to other functions without needing to allocate and copy, and without needing to worry that we're

gonna shoot ourself in the foot by having some kind of dangling pointer thing. I think it would have been worth it, all of that borrow checking pain. Of course, it gets us so much more, so that's pretty great. End of aside. So Nix is great, and I think you should check it out. It's on crates.

And now that I've hopefully convinced you a little bit that you would like to explore the systems programming world, I wanna give you a couple of ideas of how you might get started. So this is not a Rust thing, but I wrote a workshop for some conference,

Strange Loop maybe, a couple of years ago, yes, which is building a shell from scratch. And so you go from zero to having the ability to do like ls pipe to wc-l or other such things like that. The materials are in C, but if you ignore all the C stuff and just use Rust

and Nix, it'll be easier because, yeah, I had to supply a bunch of code to work with strings because I didn't want people in my two-hour workshop to be like, I can't parse my commands. But with Rust, you'd be able to split and all kinds of nice stuff.

Going beyond that, just try some other systems programming in Rust with Nix. I think Nix does make it quite a bit nicer, so that's why I'm gonna keep pitching it. And you can learn a lot by being silly. The pipes thing I mentioned where I broke my computer on stage by exhausting the memory of the system via pipes, I learned an incredible amount of weird

but stuff that's actually coming useful about process memory limits and how memory is accounted to different processes and how you can ask for pipes that are bigger than the normal pipe.

This is part of how you can go in this direction of breaking things. You can say, give me bigger pipes, please, Colonel. And I think that systems programming is often viewed as this really kind of serious, you know, I trimmed my beard because I was going through US customs and they hate me.

Sorry, that's... Never mind. Middle Eastern names are difficult. So, but, you know, it's viewed as like, you know, serious, like neck beard, kind of like gray beard. Really, we're gonna do some like hard programming here. You can just be silly. You can do all this really, really silly stuff with your systems programming.

You'll learn a lot. It doesn't matter if you make something that's gonna be like super useful and like Docker or it's gonna be, you know, if you're building some of the awesome things that people in this room I'm sure have built, you can just go off and do something silly and you'll still learn a lot. And how you can help with this kind of thing, I'm just gonna kind of have

a little country to next wink. But more importantly, I think is do cool things and tell other people about them. So this could be writing blog posts. This could be giving talks. And I think that that's like super, super useful to this community because

we're, you know, Rust is starting to get a lot of exposure across the interwebs and the fact that we're able to have this really cool, transparent access to all of these low level parts of systems programming, I think is really great and deserves to be out there more because Unix systems programming in Rust really is awesome.

Thank you. Thank you very much, Kamal. I'm sorry, I interrupted you. I was just gonna say I ran well under time, I hope. So there's time for questions. If you have questions, I will bring this microphone to you.

Not really a question, but I was looking at your examples with Nix OS and having painful flashbacks to my CS 400 operating systems class.

And everything you said that this is awesome, that this is beautiful, this is amazing, it's all true. Thanks. I never did an operating systems class, so I kind of learned all of this stuff via Rust, so I got to maybe avoid all of these painful things you're having flashbacks of.

So a question maybe for you or maybe just for the rest of the room. I'm wondering if there are other good examples of silly systems programs.

So I've only done silly things with pipes so far, but I've done many silly things with pipes. So an extension of the thing that I talked about where I kind of blew up my computer, I think I found a denial of service attack in Linux where you can exhaust memory for a program and invoke

the oom killer without any privileges whatsoever. I consider that silly because it's not really a useful attack. Somebody would have to run your program on their system, and then their system will stop working, but that's not super useful.

Non pipe silly things, since fork bombing. Yeah, I guess you could find other ways to exhaust system resources. Let me think, I don't. So yeah, I'd love to hear it from other people in the room. I've only tried to exhaust system resources and mostly with pipes.

What was that? So port port knocking, which is the idea of having a service instead of having it just listening on a port all the time, you send a sequence of connections

to different port numbers all over the space of the U 16 and if you send them in the right order, like a combination code, you knock on the right doors, it'll open up a port somewhere else that you can connect to the service. Just like some kind of super secret, back doory kind of thing.

Yeah, kind of silly, but somewhat useful too. Nice, cool. Yeah, I've not heard that one. Yeah, that's my, I don't know, I'm out of ideas for silly things, but that's mostly because I'm up here kind of on the spot. I bet that I could come up with others and

I bet that somebody else in this room could too. All right, if there are no more questions, thanks again.