A practical solution for GNU/Hurd's lack of drivers: NetBSD's rumpkernel framework
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Anzahl der Teile | 287 | |
| Autor | ||
| Lizenz | CC-Namensnennung 2.0 Belgien: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
| Identifikatoren | 10.5446/56910 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
|
FOSDEM 2022103 / 287
2
4
6
8
12
17
21
23
31
35
37
41
44
45
46
47
50
62
65
66
67
68
71
73
81
84
85
86
90
92
94
100
102
105
111
114
115
116
117
118
121
122
124
127
131
133
135
137
139
140
141
142
145
149
150
156
164
165
167
169
170
171
172
174
176
178
180
183
184
189
190
192
194
198
205
206
207
208
210
218
220
224
225
229
230
232
235
236
238
239
240
242
243
244
245
246
249
250
253
260
262
264
267
273
274
277
282
283
287
00:00
TermEndliche ModelltheorieKernel <Informatik>SharewareOperations ResearchOpen SourcePhysikalisches SystemInternetworkingDatenmodellProzess <Informatik>SpeicherverwaltungHardwareMessage-PassingSystemaufrufTelekommunikationFunktion <Mathematik>ImplementierungCompilerModul <Datentyp>Mini-DiscZahlenbereichMinkowski-MetrikFramework <Informatik>DatensatzProzess <Informatik>BitEndliche ModelltheorieFlächeninhaltSpeicherverwaltungPhysikalisches SystemQuick-SortImplementierungMAPMathematikCASE <Informatik>BetriebssystemDifferenzenrechnungInternetworkingTelekommunikationSystemaufrufBinärcodeMessage-PassingKonfiguration <Informatik>Rechter WinkelZweiProgrammbibliothekMittelwertVideokonferenzMetropolitan area networkServerFormation <Mathematik>UnrundheitAggregatzustandSampler <Musikinstrument>ProgrammTermKartesische KoordinatenComputerspielEreignishorizontOrdnung <Mathematik>Kernel <Informatik>Reelle ZahlHalbleiterspeicherMikrokernelProzessfähigkeit <Qualitätsmanagement>CodeVersionsverwaltungElektronische PublikationSoftwareHardwaresinc-FunktionInterrupt <Informatik>Mini-DiscProjektive EbeneDateiverwaltungTreiber <Programm>FunktionalRPCGrenzschichtablösungKomponente <Software>SharewareDistributionenraumBimodulBus <Informatik>DiagrammTechnische ZeichnungComputeranimation
08:31
Kernel <Informatik>HardwareImplementierungMini-DiscTopologieCompilerSchreiben <Datenverarbeitung>p-BlockServerStichprobeKonfigurationsraumInterrupt <Informatik>Physikalisches SystemROM <Informatik>Gerichtete MengeLesen <Datenverarbeitung>Interface <Schaltung>Prozess <Informatik>SynchronisierungVersionsverwaltungServerBootenPhysikalisches SystemMini-DiscEndliche ModelltheorieKernel <Informatik>Prozessfähigkeit <Qualitätsmanagement>HalbleiterspeicherBitSkriptspracheLesen <Datenverarbeitung>Minkowski-MetrikZahlenbereichVerschlingungFramework <Informatik>SoftwareentwicklerMultiplikationsoperatorMikrokernelTextbausteinBetriebssystemCodeStrömungsrichtungKomponente <Software>PunktBimodulInterrupt <Informatik>ProgrammbibliothekProgrammTouchscreenHardwarep-BlockMathematikInterface <Schaltung>RechenschieberProzess <Informatik>Nichtlinearer OperatorSystemstartSystemaufrufPlastikkarteInteraktives FernsehenService providerOrdnung <Mathematik>RPCOffene MengeMaximum-Entropie-MethodeEinfach zusammenhängender RaumKartesische KoordinatenElektronische PublikationBefehlsprozessorDateiverwaltungKonfigurationsraumBitrateTurnier <Mathematik>SpeicherverwaltungTermSoundverarbeitungComputervirusFokalpunktProgrammverifikationCAN-BusBildschirmsymbolZweiMetropolitan area networkHochdruckGebäude <Mathematik>RoutingKontrollstrukturStatistische HypotheseComputeranimation
16:52
Interface <Schaltung>Prozess <Informatik>SharewareKeilförmige AnordnungSchlüsselverwaltungTaskLokales MinimumSystemstartVollständigkeitFunktion <Mathematik>Interrupt <Informatik>WärmeübergangAdressraumPhysikalisches SystemDistributionenraumElektronische PublikationSpielkonsoleCachingGammafunktionHash-AlgorithmusTermVerzeichnisdienstW3C-StandardSchwebungPasswortIntelKontrollstrukturBridge <Kommunikationstechnik>RechnernetzArithmetischer AusdruckMachsches PrinzipServerTotal <Mathematik>KonfigurationsraumHydrostatikDateisystemSoftwareSocketMini-DiscPhysikalisches SystemSkriptspracheMereologieEmulatorStandardabweichungRadikal <Mathematik>ProgrammierumgebungServerQuaderKonfiguration <Informatik>Prozess <Informatik>SharewareSoftwareBootenBetriebssystemKernel <Informatik>HardwareSystemstartMinkowski-MetrikDateiverwaltungZahlenbereichKonfigurationsraumProzessfähigkeit <Qualitätsmanagement>RootkitInformationBus <Informatik>Translation <Mathematik>SichtenkonzeptParametersystemLoginFreewareNabel <Mathematik>SpielkonsoleElektronische PublikationPlastikkarteDifferenteProgrammfehlerVerzeichnisdienstHalbleiterspeicherTopologieSelbst organisierendes SystemSoftwarewartungGrenzschichtablösungBinärcodeRechenschieberTypentheorieWort <Informatik>Leistung <Physik>MomentenproblemMultiplikationsoperatorMultiplikationPunktVersionsverwaltungPhysikalischer EffektStrömungsrichtungAggregatzustandRichtungMathematikHinterlegungsverfahren <Kryptologie>KontrollstrukturSchreib-Lese-KopfSchlüsselverwaltungNichtlinearer OperatorZeitreiseCachingRechter WinkelProgramm/Quellcode
25:13
Computeranimation
Transkript: Englisch(automatisch erzeugt)
00:23
Hello everybody, my name is Damien Zamet. Today I'm going to be presenting to you a practical solution for GnuHerd's Lack of Drivers, NetBSD's Rump Kernel Framework. So, first I'm going to go through some key terms,
00:43
then I'm going to talk about the microkernel versus monolithic kernel models. Then we'll proceed with a bit of what is Rump, and how does GnuHerd use Rump. So, these are all valid questions, and at the end I'm going to give you an interesting live demo.
01:00
So we'll see how we go. So, GnuHerd is the original free software operating system. It was started in the 1980s. It uses a microkernel model. It's copyleft licensed, and a lot of people say it's finished, but I don't agree with that. I think we still need some work to do.
01:22
We still have some work to do. We need to finish the drivers, and make it work with some more modern hardware. So I'd say it's not quite finished. That's just my opinion. So NetBSD, that's another operating system. It's the original internet operating system. It's based on Berkeley's software distribution.
01:44
It is permissively licensed, and uses a monolithic kernel model. So it's completely different, and it runs on pretty much anything these days. GnuHerd and NetBSD, they're both operating systems, but they're very different.
02:03
Okay, so what is a monolithic kernel model? This is where we have a kernel, for example the Linux kernel, where everything that talks to the hardware, and provides file system, and that sort of level access,
02:22
it's all in one process, one single privileged process called the kernel. So that might contain things like memory management, ACPI, PCI drivers, disk drivers, file system, that sort of thing. And in this case,
02:41
we have, it's sort of sandwiched between the application and the hardware. So it's sort of in the middle layer there, or the low layer there, near the hardware. So what is a microkernel model? This is a different model. This is where all the drivers are broken into separate user processes.
03:02
And we only have minimal components remaining in the kernel. So for example, we have the GnuMac kernel, which is the Herds kernel, the Herd operating systems kernel. It contains things like memory management, and other things. And, but it's much smaller.
03:22
It's only like a few hundred kilobytes in size. And it has, and all the file system drivers, disk drivers, PCI drivers, they're all sitting in user space above the kernel. So it's a different model.
03:42
But what's interesting is, how do we have these servers that are in user space? They've got to be able to talk to each other. They've got to be able to communicate so that they can send messages, or some sort of inter-process communication, IPC, in order to function. Because in the kernel,
04:01
in a monolithic kernel model, they can all just directly do function calls to each other. And they don't need any inter-process communication, because they're all in the same process. So they can just call function calls. But in this case, in the microkernel case, we need to have some sort of communication by remote procedure calls.
04:23
And GNU MAC provides a message bus to facilitate this. So GNU MAC, the kernel for Herd, provides a message bus for all the servers that are running in user space to communicate via RPCs, or remote procedure calls.
04:42
And they are done this way so that they almost appear like function calls in the code, but they're actually inter-processed. So they're communicating via IPC, via RPC, over inter-process,
05:03
in fashion. So what is Rump? Rump's an interesting concept because Rump is a user space implementation of kernel drivers. So... And it's particularly relevant to NetBSD's kernel drivers
05:22
because Rump is a framework that allows you to run NetBSD's kernel drivers as unikernels. So a unikernel is like a... a separate process that only has one purpose. And it runs in user space
05:41
on any operating system and it's compiled as a library. So you link your program, your binary, you link it with this unikernel library and it becomes a separate static binary that will run as a kernel. I think it actually supports dynamic linking as well,
06:03
but we use it in Herd as a static linked binary. So then there's another thing called AnyKernel which is a concept invented by Rump which is like...
06:22
AnyKernel is where you have two modes, a unikernel running in user space as one option for compiling the module or you have the opposite case where it runs as a monolithic kernel as a module. So AnyKernel is kind of like the best of both worlds
06:43
because you can either run it as a unikernel in user space or you can run the same code compiled as a module running in your monolithic kernel. So you don't have to rewrite the driver, you've got the same code running twice. And this only runs on NetBSD,
07:02
this AnyKernel thing. But in GNU-Herd we use the unikernel version where we run it on a different operating system called GNU-Herd and we compile it as a library and just run it in user space. So a bit of history.
07:20
Rump was implemented as a PhD project of anti-canti. Sorry, I don't know if that's the correct pronunciation. In 2014, it was possible then to run a Rump kernel with real PCI hardware thanks to the PCI user space implementation of the Rump framework.
07:42
So that means we could actually run real hardware through Rump as user space drivers. And in 2015, Robert Millen packaged Rump for Debian-Herd and he implemented the video player called MPlayer
08:01
to work with a Rump audio driver in the Herd operating system. And more recently, I've made a small number of improvements to the PCI user space framework, typically just to improve the interrupt handling so it's compatible with Herd now.
08:22
Or it was compatible with Herd before, but we've changed Herd since then, since 2015, and so we had to upgrade PCI user space to make it compatible. So I made the small changes there and I also introduced the AHCI SATA driver from NetBSD
08:41
to make it a Rump kernel, which involved just a little bit of boilerplate code to be added to Rump to make it compile as a Rump driver. And then I added it to the AHCI, to the Herd operating system to provide disk support.
09:02
So what's the current Rump status? Well, NetBSD used to have... See, the Rump kernel on GitHub is the upstream version of Rump, but it's actually out of date. I think it's running on a version like 7.99 or something like that
09:21
of NetBSD. So we want to use a new version of NetBSD and use its Rump for GNU-Herd. So we did that. We... I mean, NetBSD developers have been
09:41
maintaining Rump all this time since 7.99 or even earlier, but the upstream kernel, the upstream Rump kernel on GitHub didn't get updated at the same time, so they're a bit out of sync at the moment, which is a bit unfortunate. But we're working on the upstream version in NetBSD
10:01
with GNU-Herd, and there's a little bit of work remaining to make the build scripts all work for Rump on other operating systems. So how do you integrate Rump with another operating system? So basically, you have to port the PCI user space
10:20
framework to your desired operating system target, which just is a matter of implementing a number of hypercalls, which let you link with the Rump system. Then you compile Rump libraries for your target, and then you can use a simple program
10:40
like the one on the screen here in the slides to do something with your device. So for example, it's very simple with Rump. Once you have it embedded in your... Once you have it ported to your operating system, you basically just do an include of Rump. You initialize Rump with Rump in it, then
11:01
you open a device that you want to talk to, do something with it, then you close the device and turn off Rump. So it's that simple. So how does GNU-Herd use Rump? Well, we have a number of servers running in user space.
11:21
So, Herd is actually a collection of servers running in user space. already. So Rump is the perfect candidate for adding a user space driver, because everything is already in user space, except for the GNU Mac kernel, which is how
11:40
it's supposed to be in this microkernel model. So, some examples of Herd servers, you've got PCI-Arbida, ACPI, and Rump disk. I won't go through them all, but there's a number of them that provide disk support, and there's a number of the components that had to remain in the kernel, because
12:02
things like interrupts, system timer, and memory management, you can't easily run them in user space. So they're stuck in the kernel for now. This is an interesting slide, but I just wanted to make the point that
12:21
we used to access the disk device via the kernel, so that is not really in a microkernel model kind of way. We used to have the Linux 2.6 disk driver embedded in GNU Mac, but now that we have Rump,
12:41
we can get rid of that Linux 2.6 disk driver and break it out into a separate module that runs in user space. And they're not really a module, it's a static binary, but it runs as a user space program. And in its place, we have, in the
13:00
place of the disk driver, we have some better interrupt handling. But now I want to talk about interaction between servers at startup. So, how do we actually boot up this system in Herd? In Herd, we have GNU Mac as the kernel. It starts
13:21
first, and it gets loaded into memory, and then we have all these other servers that need to start up to provide disk access so that the rest of the system can boot up. So it's a very tricky dance that it has to be performed in order to have disk access in user
13:41
space, but also at boot up, when there is no disk. So, how does that happen? Well, we need to have PCI Arbiter, which is the PCI layer actually talking to memory via the device open mem
14:01
system, RPC. So it makes a remote procedure call to the kernel, and requests the memory device, and then it makes a call to the kernel again to get permissions to use the IOPorts so it can talk to the PCI cards. And then RumpDisk
14:20
actually makes a call to PCI Arbiter via another library called libPCI Access in order to read the configuration of these disk devices via PCI Read. But it also has to make a connection to the kernel to register an interrupt
14:42
handler, so that later on when you get interrupts, the kernel will notify RumpDisk of these interrupts that are happening on the disk. But then when we actually come to use the hardware in the operating system once it's booted up,
15:00
we have something like the application at the top here performing a read system call, and then glibc actually turns that into a remote procedure call in RPC called ioread and requests that from the X2FS, the file system.
15:21
Now the file system knows all about how to open these ext2 file systems, so it works out which block to read and sends the request as an RPC to RumpDisk the block device handler and performs a device read.
15:41
Then RumpDisk directly accesses the hardware and reads the block or asks to read the block, and then the hardware makes the hardware interrupts the CPU Pardon me the hardware interrupts the CPU
16:02
and the handler is called then the interrupt notification goes back to RumpDisk via the kernel and RumpDisk is able to acknowledge that it received the interrupt so that it can continue.
16:23
So what are the advantages of using Rump in GNU Herd? Well, we have more drivers with a stable interface we don't have to reinvent the wheel for new drivers we can debug any process including the actual driver with GDB for example
16:41
and finally, licensing concerns are separated. So what does that mean? That means you can have a static binary running the driver for the disk and it doesn't have to have the same license as the operating system because it's a separate binary running in userspace
17:01
so that means you could have BSD free permissive permissively licensed drivers running in userspace in a GPLv3 operating system OK, so now I'm going to do an
17:21
interesting part which is a live demo I'm going to show you GNU Herd booting up let me just switch the slides OK so you should be able to see my terminals I've got one terminal on the left here that's going to boot up Herd
17:41
and on the right hand side I've got another terminal that's going to shell into the box that's going to access the Herd system so as you can see this is a Linux system not a Herd system and this is a Linux system not a Herd system
18:00
now if I execute Herd as a... I'll just show you the script I'm using to run a Herd it's actually running QEMU the emulator for running Herd at the moment we use that one a lot
18:22
interestingly I've got a network device connected to it and port forwarded quadruple 8 is the port number that I'll be shelling in on OK, so if I run this it boots up grub now I need to select this one
18:44
because I've got a custom environment now interestingly I'm just going to change this to the standard debugging kernel and interestingly I've used the no IDE option with GNU Mac so
19:01
that means it will not use the onboard disk driver that's part of GNU Mac it will actually try and it'll ignore it and so that will allow Rump the Rump disk the user space disk driver to actually detect the disk is there and then use it
19:20
rather than the kernel trying to use it and then they're getting into conflicting arguments so let's boot it up so it's probing for PCI hardware it found the device
19:40
WD0 the disk device and Herd bootstrapped itself and it's now booting up Herd so now we have a login screw shell
20:00
so I'm going to shell into um the demo box and it's it's working there we go so now this actually is a Herd system it's not a Linux system
20:22
as you can see so let's have a look at some of the processes running so here we can see the startup service, the
20:41
kernel the arbiter for the PCI the disk driver user space disk driver the file system driver and a number of other servers that run the system so this is a completely
21:00
self-contained Herd system I can show you how um how can I show you? LSPCI so if I execute that there are no devices because you have to be root to access that I think
21:24
so now I'll just become root temporarily and now you can see it's provided me some information about the PCI devices but interestingly um if I do show trans servers
21:41
bus PCI that is actually showing me that the servers bus PCI node disk file system node is running a translator called PCI arbiter so if I look
22:00
inside this directory I actually get a view of the file system which is not part of a disk it's actually reading it out of memory um and presenting the PCI device tree as
22:20
a tree of file systems as a file system tree called a netFS which is something that Hurd offers so let me just become root for a minute um so ok so I'm now root
22:41
if I cd into 00 I can go into this directory and then again into 0 so this is like the 0.0.0 uh PCI card and if I ls this card this config file
23:01
I can see it's a 256 byte file which actually corresponds to the configuration space for this card so if I dump this card configuration this is actually the
23:20
configuration space presented as a file in Hurd but it's reading it out of memory so that's pretty cool so we can actually do that for all the different um cards so if I go to the zero host I can see all the other cards find dot
23:41
oh yeah you can't find dot in the system ok anyway that's a bug so we'll fix that um yeah so that's pretty much it um I'll just I'll show you shutting down the system so if I log into the actual terminal
24:00
the console I mean um now if I type the word power off it should just shut down the system killed my shell and notified everything that was shutting down
24:23
and then halted the system so that's it so here are my references thanks everybody and
24:40
um I'll be sticking around for the question time hopefully some questions will be asked and I'll be around to answer them ok thanks very much and thanks to all the Hurd FOSTEM organizers and the Hurd maintainer
25:01
who suggested I give this talk thank you very much goodbye