We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

A roadmap for the Hurd?

00:00

Formale Metadaten

Titel
A roadmap for the Hurd?
Serientitel
Anzahl der Teile
561
Autor
Lizenz
CC-Namensnennung 2.0 Belgien:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
Identifikatoren
Herausgeber
Erscheinungsjahr
Sprache

Inhaltliche Metadaten

Fachgebiet
Genre
Abstract
Most people don't realize it, but the Hurd system is actually well established. About 75% of Debian official packages do build fine, it has mainstream gcc/glibc/llvm support, go and rust ports are ongoing, it can be installed with the Debian installer and GuixSD and Arch ports are ongoing... Yet not so much has been happening within the Hurd itself in the past couple of years. We have notably added a PCI arbiter, which allows for both flexible and safe PCI access for end users, and some basic ACPI support is ongoing. But many exciting features could be achieved with a bit of work. This talk will discuss some of these promising features, to give a sort of ideas roadmap for contributions. Some have implementation sketches which just need to be polished to be more production-ready, such as httpfs, mboxfs, or writing translators in more high-level languages than C. Other features are at early stage, such as adding sound support through rump, getting complete rid of disk drivers from the kernel by moving them to userland, or also getting valgrind support. I will also discuss some promising ideas, such as using rump to get support for more filesystems.
10
58
80
111
137
Vorschaubild
15:21
159
Vorschaubild
18:51
168
Vorschaubild
26:18
213
221
Vorschaubild
15:22
234
Vorschaubild
49:51
248
Vorschaubild
23:06
256
268
283
Vorschaubild
28:38
313
Vorschaubild
1:00:10
318
Vorschaubild
21:35
343
345
Vorschaubild
36:13
353
Vorschaubild
18:44
369
370
373
Vorschaubild
44:37
396
Vorschaubild
28:21
413
Vorschaubild
16:24
439
455
Vorschaubild
25:10
529
Vorschaubild
15:36
535
Vorschaubild
28:04
552
SystemverwaltungProgrammProzessfähigkeit <Qualitätsmanagement>Prozess <Informatik>Funktion <Mathematik>PlastikkarteKernel <Informatik>ServerInterprozesskommunikationROM <Informatik>TaskVirtualisierungFehlermeldungSpielkonsoleFontGasströmungATMElektronische UnterschriftSpieltheorieElektronische PublikationBildgebendes VerfahrenBitPlastikkartePartitionsfunktionATMAutomatische HandlungsplanungProzess <Informatik>FlächeninhaltRootkitRechter WinkelKernel <Informatik>ProgrammierungInterprozesskommunikationSpielkonsoleGüte der AnpassungServerProzessfähigkeit <Qualitätsmanagement>Physikalisches SystemSoftwareMini-DiscAutorisierungKartesische KoordinatenNabel <Mathematik>Translation <Mathematik>IdentitätsverwaltungRoutingImplementierungFiletransferprotokollDateiverwaltungVirtualisierungGrenzschichtablösungVerzeichnisdienstDatenstrukturNormalvektorSystemverwaltungTaskSystemzusammenbruchGewicht <Ausgleichsrechnung>MikrokernelKeller <Informatik>HalbleiterspeicherInstantiierungSchnittmengeURLDienst <Informatik>Virtuelle MaschineComputeranimation
Elektronische UnterschriftInhalt <Mathematik>Translation <Mathematik>Elektronische UnterschriftMultiplikationsoperatorElektronische PublikationOffene MengeComputeranimation
RootkitHash-AlgorithmusSpeicherabzugStrom <Mathematik>TopologieDistributionenraumGebäude <Mathematik>PASS <Programm>DifferenteProzess <Informatik>Dienst <Informatik>Translation <Mathematik>ServerAnalytische FortsetzungNotebook-ComputerInstantiierungQuaderVirtuelle MaschineComputeranimation
Strom <Mathematik>TopologieDistributionenraumGebäude <Mathematik>SteuerwerkMinkowski-MetrikMini-DiscSchnelltasteProgrammierspracheAppletProzessfähigkeit <Qualitätsmanagement>Übersetzer <Informatik>Kernel <Informatik>DatenparallelitätOperations ResearchHardwareDateiverwaltungNP-hartes ProblemPhysikalisches SystemProgrammierungMultiplikationsoperatorDifferenteEindringerkennungKreisbogenMinkowski-MetrikNormalvektorElektronische PublikationHöhere ProgrammierspracheBitVirtuelle MaschineWeg <Topologie>DatenfeldAppletGarbentheorieTranslation <Mathematik>HardwarePatch <Software>ValiditätServerInstantiierungVerzeichnisdienstPhysikalisches SystemBildgebendes VerfahrenNabel <Mathematik>HalbleiterspeicherProzess <Informatik>FiletransferprotokollKonfigurationsraumPlastikkarteVirtualisierungPOKEPartitionsfunktionDateiverwaltungSoftwareNamensraumTurm <Mathematik>Mini-DiscWeb-SeiteProzessfähigkeit <Qualitätsmanagement>BetriebssystemProjektive EbeneQuaderDistributionenraumNichtlinearer OperatorCodeGebäude <Mathematik>GeradeDatenparallelitätPunktE-MailComputeranimation
Kernel <Informatik>ATMMinkowski-MetrikRechnernetzMini-DiscInterprozesskommunikationROM <Informatik>TaskNP-hartes ProblemBootenBimodulMikrokernelHackerSystemprogrammierungOperations ResearchProzessfähigkeit <Qualitätsmanagement>Übersetzer <Informatik>DatenparallelitätHardwarePhysikalisches SystemWeb SiteBenutzerfreundlichkeitPasswortInformationExogene VariablePunktCoxeter-GruppeEinsMultiplikationsoperatorInterface <Schaltung>SoftwarewartungMereologieBitProgrammbibliothekPhysikalisches SystemKernel <Informatik>LastProgrammfehlerTabelleMomentenproblemBildschirmmaskeZweiunddreißig BitElektronische PublikationParallele SchnittstelleBimodulMini-DiscPatch <Software>MAPCodeRechter WinkelLokales MinimumProgrammierungSkriptspracheDistributionenraumTranslation <Mathematik>Prozessfähigkeit <Qualitätsmanagement>RootkitPlastikkarteMikrokernelProzess <Informatik>HalbleiterspeicherBootenDateiverwaltungQuellcodeDruckertreiberTaskATMInterprozesskommunikationSoftwareMinkowski-MetrikServerNP-hartes ProblemDisjunktion <Logik>Computeranimation
Turm von HanoiRootkitSpeicherabzugHash-AlgorithmusProzessfähigkeit <Qualitätsmanagement>Übersetzer <Informatik>Kernel <Informatik>DatenparallelitätMinkowski-MetrikOperations ResearchSystemprogrammierungHardwareNP-hartes ProblemPhysikalisches SystemSoftwarePhysikalisches SystemDisjunktion <Logik>Computeranimation
PunktwolkeComputeranimation
Transkript: Englisch(automatisch erzeugt)
So I guess there would be no microphone in the bathroom without some news from the herd. And I welcome Samuel, and we're really looking forward to what's new in the herd. So good morning, everybody, and thanks for coming.
So I won't talk about a specific bit of the herd, but general news and general plans for the future. Just the usual introduction to the herd, it's actually all about freedom zero, being able to run in whichever situation for any purpose.
And notably, that's the freedom for actual users on the system. They shouldn't have to ask the system administrator to do some crazy things like partition a disk or something like this. As long as they have the right to access a disk or some area or whatever, they should
be able to just run F disk and things like this. And also, network access, you should be able to run a VPN as a user and things like this. There's no reason you shouldn't be able to. And also, the freedom to innovate. If you want to store your data in a different way with an experimental file system,
you should be able to do this. Just try new things without the fear of crashing the machine, just because the system will actually prevent you from doing bad things like this. You would like to give a PCI card, an experimental PCI card, to a program which will drive it in a safe way and all kinds of things like this.
And also, the freedom from the programs themselves. If they crash, then OK, that's fine. If a driver crashes, OK, that's fine. It shouldn't harm the rest of the system. So to give an idea, this is how it works. You have the kernel here, a microkernel, which only handles the tasks, memory, and IPC
between user-run programs. And then you have several routes started, services, like pfi.net for the TCP IP stack, proc to know what is a process, who has a PID, and which identity has each process, auth,
which knows who is who, and x2fs, the file system. And then when you have a shell which is starting with TCP, actually, they are talking with x2fs to access the files. And it goes through auth so that x2fs knows who is that program which is asking me to read a file or whatever.
And it checks with auth who he is according to what is stored on the file system, for the permissions, I mean. So the thing is, if a server crashes, then that's not really a problem. If the pfi.net stack crashes, then that's fine. We start another one. And then things will continue.
It's nice to debug because you can run GTP on a file system and things like this. The HERT console is actually a user-learned application. So we could implement crazy things like Chinese in text mode that is using glyphs on the fly to actually show them with a normal VGA card.
And the kernel itself only handles the task, memory, and IPC, and nothing more than this normally. I will talk about it more later. This also provides virtualization at a really fine grain because you can choose for each program which TCP IP stack it will use, which root file system it will use,
which notion of process it will use, which notion of user it will use, which is exactly the same as on Linux with name services and all kinds of separation, except that here it's deep inside the structure of the HERT, that it's that way. So it's not like on Linux, oh, we
forgot to separate sound support, for instance, network support, et cetera, we forgot about this. On the HERT, you cannot forget about compartmentalizing things just because that's the way it is done anyway. You do not have another way than having separations between processes.
So you can do crazy things like have an FTPFS running on top of the TCP IP stack, and then run ISOFS, opening an ISO on the FTP server, and then run a shell within the ISO image. It's like this, so you set a translator on FTP command,
and then you can open a URL, and then start an ISO file system, and then you can look at it. And the nice thing is that since LS just looks at the root of the ISO image, then the ISO translator only has to ask FTPFS to download a bit of the ISO image.
So you don't have to download everything, just to see what's in there. So that's really a nice thing. And you can permanently store this kind of thing. So this set-trans, for instance, you can store it in your home. I have a little example, like here. I have a signature file, which each time I open it,
I get another content just because it's a translator behind, which just starts Fortune for each open of the file. So you have nice things like this which are possible thanks to this kind of flexibility. And just to show you, so I have a lot of processes, of course.
So there's X2FS here, the old server, and all kinds of translators for different services. And if I kill one of them, then that's fine. The rest continues. So the herd is a real thing nowadays, actually. It's really stable. I don't remember when I reinstalled
the boxes I'm using. This one, for instance, I've been copying over and over in virtual machines between my different laptops. And I don't remember when I installed it. The Debian builders are like this. So they keep building Debian packages all day long. And yes, from times to times, there
is maybe a program which takes all resources, because it does make minus j or things like this. And then the system locks. But then, yeah, that's the normal kind of things when a process tries to eat all resources of a system. But yeah, usually it just works. We have like 3 quarters of Debian building,
which is really a thing, because you don't have things without patches like Firefox or LibreOffice. But all the dependencies basically are there, except a few things like cargo and things like this, which we are working on.
But basically, we have XFC, Gnome, KDE. These kind of things do work. And it's supporting upstream. So this is really an operating system which does exist in upstream, like GCC, GDC, LLVM. And we are working on Go and Rust. So that's progressing.
So we have the Debian distribution, which is really well supported. We have the installation working just like a normal Debian port on Linux or on K3BSD. And there are GeeksSD and Arc ongoing. There is a lot of work on GeeksSD, Arc. Maybe it's on post, but there are people working on it from time to time.
So GeeksSD would probably provide one of the most pure GNU operating system with GNU kernel, et cetera. So it's on its track. So now, what's the future? The thing is, there are many, many, many existing bits
in different places which just need polishing. So I will give a few examples. That's the idea of in 10% time, you get 90% of the thing done. And then for the 10 remaining percent, you have to take 90% of the time. And a lot of people don't take that time
to make it just work completely. They had something funny, and then they stopped there. That's a bit sad, because then there are a lot of things we show which are not completely working, but almost. Then I show the ongoing project IDs and a couple of crazy IDs that we could experiment with.
So for instance, there is HTTPFS and FTPFS, which work quite fine most of the time. So you can do, like, you CD to a directory, and then you install all the files with that kind of name. And that's really efficient, because you don't have to do downloading or whatever.
It's just normal shell operations. I think this almost works fine, because the HTTP server here provides an HTML file which HTTPFS can parse correctly. But there are other kinds of HTML pages
that HTTPFS has troubles with. With FTPFS, it works quite fine as well. So you can just find some files, because you don't remember where it is, and then it does it for you. Of course, you have FTP clients which can do this kind of thing, but just use your shell
with normal tools, and it will work fine. We have namespace-based translators, so that's a funny thing. So you can look at the software.tar.gz, and you append comma, comma here, and then you can CD into it. So the idea is that you have a translator
behind the actual file system or whatever, and then it sees that you have put these commas, and then it will start a tarfs translator, so you can actually enter into it and then look what's in there. And again, tar is indexed, so you do not have to read all of the tar file.
You could chain them. You have a disk image, and then you open a partition within it, and then you open the X2FS file system within it. So I'm not sure this one is working, but it's the kind of thing that should be possible, and possibly it's not so many lines of code to get it working.
So yeah, please somebody have a look and have fun with implementing this. MboxFS, if you have an mbox file, it's all mails into just one file. You could start mboxFS and then manipulate things inside it, just put some random IDs.
That's kind of IDs you could have. XMLFS, you open an XML file, and then you can browse into it with directories. So you can find, for instance, the H1 sections, and then just get the text of each H1 section in a nice way. There are some experiments with being
able to write all these kinds of translators in high-level languages. So in Perl, in Lisp, in Java, in Python, these are mostly experimental, but some of them are working. And it's just a bit of polishing the existing thing and to have something which would work and allow
a lot of different possibilities. I'm not detailing those, but there are more JFS to open a JFS file system. It is read-only, but at least it works. But write availability would be fine.
Notice, to notice the modifications of the file, run, I've shown it with the Fortune program. UnionFS, to union some file systems. So these are just to polish and to get them working really fine.
Then there is ongoing work. A lot of on the hardware support field, because we have to have this to continue being able to start the herd on machines. So recently, we've introduced a PCI arbiter which allows to have safe, concurrent access to PCI config
space. Up to now, we would just let programs poke at IOPorts concurrently. And of course, it would be a nightmare if two programs do the same at the same time. So I talked about it last year. The really cool thing would be to be able to use an IOMMU
to make it safe. That is, the PCI arbiter would literally give a PCI card to a process, just a process. It's actually fine-grained virtualization. And then the process would be able to drive the card without anybody else interfering with it and without giving that process access to the whole memory
and things like this because of DMA, thanks to IOMMU. So that would be really cool, just to make drivers well separated. We have an ACPI translator pending commit
to provide access to ACPI impressions, just like being able to shut down the machine. It's really complex to shut down the machine, actually. You have to do some ACPI. But yes, we should have it at some point. Using RAMP, I talked about it some years ago
because RAMP is really supported by a lot of people and also lead guest FS to access file systems without having to re-implement. The thing is, we don't want to maintain file systems and device drivers because that's a lot of work. And instead, we can just put that into a process.
And then we just provide the proper interface to make it interact with the rest of the system. We have a bit of a 64-bit kernel support. There are a few bits missing to let a 32-bit user run on top of a 64-bit kernel.
So at least we could manage a lot of memory easily and then try to bootstrap a 64-bit user. And it looks like an easy thing, but bootstrapping a user and really bootstrapping, that means putting the information of a 60-bit herd
exists in AutoConf, in GCC, in glibc, et cetera, et cetera. It's all these kind of programs which need to be made aware of this. So that's kind of work. But yeah, that would be really great. And also, SMP support. The thing is, the herd itself is already parallel because we have several translators running in parallel.
And we have threads, which are working fine. It's just at the kernel side that we don't have SMP support. But actually, Mach itself does have some support. It's just that it's an old Mach that the herd system is using, where SMP support wasn't updated to passing a CPI tables and all kinds of things like this.
So it's that bit which should be done. And then, of course, fixing the bugs that have remained since it wasn't tested for a long time. But at least, just because I show that we remove as much drivers from the kernel
as possible, it should be easier because we have less source code to make sure it works in SMP mode. So no drivers in the kernel. So for the herd system, the idea is that the kernel provides tasks, memory, and IPC.
That's the basics that was chosen. But at the moment, we have drivers in there. And we used to have network drivers, but we have gotten rid of them into user space. So I talked about it some years ago. But we still have disk drivers.
Just because nobody took the time to move them out. So how could we move them to userland? Just do it. Well, there's one issue. It's how do you actually boot the system. And the solution will be to just use another CRAP-loaded module.
So how does it boot right now? Right now, CRAP loads the kernel and two modules into the file system and the exact server which knows how to execute a program. And the kernel has the disk driver. So the first thing that happens is that X2FS starts the minimal herdish translators.
So to know what is a process, who is who, and startup scripts. So the idea is that X2FS knows it has to start them. It tells exec you have to execute them. And open files which contain the code for this.
And to get the data, we use the disk drivers. And then startup can start init. And then init can start a TCP IP stack which uses user-level network drivers. And so here we have a system which just works.
With the disk as an additional module, the whole thing will be the same. Except that yes, we have to have CRAP load the disk module so that X2FS can load data from the disk right from here. Maybe we will have to add the PCI arbiter here
so that the disk driver can access safely to the PCI card while later on XOR will access to the PCI as well. But then it's just all the same. And then we really have everything running in userland as root while the kernel only handles this kind of thing.
OK. So to conclude, there are a lot of nice things that we have in GNU Herd and a lot of things that we could achieve if just you polished it. It's fun to have on something, to have something starting to work. It's better to have it finished so
that people can really use it. It's something that I've often seen. I get patches and then they are not completely working. I say this, this, that is not all right. And then I get no response. And that's really a concern because I cannot do this myself. So sometimes people have other things to do.
OK, that's fine. But yes, if people can help with just finishing the existing things, that would be great. When I see the microkernel room presentations, I see that there are a lot of microkernel things. And that's great. To have something which is a real OS,
I mean that you have a whole distribution with usual programs. And people just log into it and have their usual comments. And they are not lost. It's really hard. It was mentioned in the Debian on RISC-V talk that just getting Perl to compile
is really a lot of work. And we do have it on the Herd because somebody did it in the past. And we do have it for GCC, for GDB, and all kinds of things. So we have already all of this which is already there. We just need a few things to have it even more
to the point of being a usable system. So thanks for listening. And thanks for all the people who have been working on it. You have the website for more information. Thanks. The question.
One question, then I have time for more. OK. Yeah? So you said a lot of things about what could happen now. What I missed is what improved in the past three years. So I showed what we could do and what happened in the past three years. Basically, what I mentioned here,
so the PCI Arbiter and ACPI Translator, the RAMP as well, I think, so that was 2016. So basically, these points are the ones which are hot. But there are not many people working on it. So it's at really slow pace. There is nobody full-time or things like this. It's only home time, basically.
More concrete, what was finished in the past three years? What was finished? So that people can now test it on a running HERT. So the PCI Arbiter is finished. I mean, here, my HERT system here is actually showing it, I think.
Yes, so the XOR actually uses it nowadays to access. Because we have NetDD, which accesses the network board, and XOR as well. So yes, we had to have something finished for this. Thank you. Okay. Thank you. We can miss.