NetBSD Native APIs - A Userland Perspective (Audio, Input)
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Untertitel |
| |
| Serientitel | ||
| Anzahl der Teile | 490 | |
| Autor | ||
| Lizenz | CC-Namensnennung 2.0 Belgien: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
| Identifikatoren | 10.5446/46939 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
|
00:00
SoftwareEin-AusgabeSpeicherabzugGruppenkeimSystemplattformCodeWeb-SeiteDateiformatKernel <Informatik>StrebeStichprobeVolumenNotebook-ComputerFunktion <Mathematik>KontrollstrukturEmulatorFreewarePatch <Software>StellenringSoftwaretestImplementierungKonfiguration <Informatik>Partielle DifferentiationWürfelZufallszahlenp-BlockTopologieDatensichtgerätServerFunktion <Mathematik>Kernel <Informatik>Schreiben <Datenverarbeitung>Rechter WinkelComputerspielPhysikalisches SystemZufallsgeneratorProgrammbibliothekSoftwareentwicklerStrömungsrichtungEmulatorTaskGamecontrollerEuler-WinkelDifferenteLastStandardabweichungSoftwareNotebook-ComputerCodeMAPSpezifisches VolumenRandomisierungDateiformatElektronische PublikationKlasse <Mathematik>Open SourceSoftwarewartungPunktFront-End <Software>FunktionalEindeutigkeitTreiber <Programm>KugelkappeHypermediaVirtuelle MaschineTrennschärfe <Statistik>Parametersystemsinc-FunktionKonfiguration <Informatik>Spezielle unitäre GruppeVollständiger VerbandProgrammfehlerStreaming <Kommunikationstechnik>ComputersicherheitHilfesystemMultiplikationsoperatorBildschirmmaskeBitGenerator <Informatik>Deskriptive StatistikAbstimmung <Frequenz>ZahlenbereichUnrundheitResultanteInstallation <Informatik>Ordnung <Mathematik>Nichtlinearer OperatorPrinzip der gleichmäßigen BeschränktheitMixed RealityRuhmasseVorlesung/Konferenz
07:04
p-BlockKernel <Informatik>Streaming <Kommunikationstechnik>SoftwareCodeFehlermeldungZeiger <Informatik>HydrostatikSoftwarewartungServerFreewareEin-AusgabeWorkstation <Musikinstrument>SpielkonsoleDatensichtgerätProgrammbibliothekSchnelltasteComputersicherheitProzess <Informatik>IntelGraphikprozessorEin-AusgabePatch <Software>ProgrammbibliothekSoftwareentwicklerNichtlinearer OperatorSchnelltasteFreewareSoftwareProgrammfehlerBenutzerfreundlichkeitSystemaufrufZufallsgeneratorBrowserBildschirmfensterKernel <Informatik>SchnittmengeNotebook-ComputerElektronische PublikationServerRadikal <Mathematik>CodierungDatensichtgerätCodeVollständigkeitQuick-SortCASE <Informatik>ComputerspielMultiplikationsoperatorGamecontrollerComputersicherheitProzess <Informatik>Disjunktion <Logik>Äußere Algebra eines ModulsGanze FunktionRechter WinkelElementargeometrieGenerator <Informatik>SpielkonsoleBasis <Mathematik>ZahlenbereichSystemzusammenbruchOpen SourceBestimmtheitsmaßTermPhysikalisches SystemVerschlingungInverser LimesStandardabweichungAdditionMAPUmwandlungsenthalpieSpieltheorieVideokonferenzComputeranimation
14:32
SpieltheorieIntelBefehlsprozessorMehrrechnersystemMigration <Informatik>Treiber <Programm>Schnitt <Mathematik>IntelKernel <Informatik>ZweiCodeComputeranimation
15:34
GraphikprozessorIntelBrowserSpider <Programm>TouchscreenBimodulSoftwareE-MailMereologieDesign by ContractGewicht <Ausgleichsrechnung>TermSchnittmengePhysikalisches SystemDienst <Informatik>Kartesische KoordinatenBildschirmschonerBrowserUmwandlungsenthalpieUnrundheitTorusProgrammMultiplikationsoperatorTouchscreenRechter WinkelTreiber <Programm>EntscheidungstheorieDatenverarbeitungssystemFunktionalEndliche ModelltheorieURLComputerspielWurm <Informatik>DatensichtgerätRahmenproblemProtokoll <Datenverarbeitungssystem>Office-PaketStellenringNotebook-ComputerSkriptspracheServerHardwareDigital Rights ManagementModul <Datentyp>Spider <Programm>FreewareBildschirmfensterComputeranimation
20:08
SoftwareE-MailBildschirmmaskeFontBildschirmsymbolCursorInterface <Schaltung>Vorzeichen <Mathematik>ProgrammierumgebungProtokoll <Datenverarbeitungssystem>VariableZentrische StreckungDatensichtgerätURLOpen SourceServerTouchscreenKugelkappeDokumentenserverSchnittmengeKartesische Koordinaten
20:58
Element <Gruppentheorie>BildschirmsymbolInklusion <Mathematik>Hill-DifferentialgleichungRandomisierungCodeProtokoll <Datenverarbeitungssystem>HochdruckGemeinsamer SpeicherMereologieSoftwarePRINCE2Halbleiterspeicher
21:22
Gesetz <Physik>MaßerweiterungKartesische KoordinatenCodeSchreib-Lese-KopfEigentliche AbbildungEin-AusgabeProgrammfehlerBildschirmschonerNotebook-ComputerComputerspielRuhmassePhysikalisches SystemRechter WinkelVerzweigendes ProgrammComputersicherheitMatchingSystemplattformCASE <Informatik>SoftwarewartungFunktionalGrenzschichtablösungSpieltheorieOrakel <Informatik>MereologieRichtungEntscheidungstheorieSoundverarbeitungGamecontrollerProtokoll <Datenverarbeitungssystem>MathematikWeb SiteMessage-PassingMultiplikationsoperatorArithmetisches MittelSoftwaretestBildschirmfensterOpen SourceTrennschärfe <Statistik>Offene MengeVollständigkeitModul <Datentyp>TouchscreenGebäude <Mathematik>Digital Rights ManagementLastVorlesung/Konferenz
27:31
ProgrammbibliothekPhysikalisches SystemFunktion <Mathematik>OSS <Rechnernetz>EinflussgrößeBildschirmfensterDigital Rights ManagementMultiplikationsoperatorLokales MinimumWeb SiteEin-AusgabeBitBenutzerfreundlichkeitTouchscreenVorlesung/Konferenz
29:58
PunktwolkeOpen Source
Transkript: Englisch(automatisch erzeugt)
00:05
Ladies and gentlemen, next talk, please. Hi, I'm Nia. I'm giving this talk from an experimental display server running in Wayland on NetBSD. So at least having video output makes it a success to me.
00:25
So first of all, I have to start out being a bit sad because we have a load of cool, original, fairly unique APIs in NetBSD that have been developed over the years, but not a whole lot of software uses them. And yeah, it's kind of understandable
00:41
because working with people is absolutely the worst part of open source. I've been told, been given a lot of reasons for why pull requests haven't been accepted and things like that. And there's an increasing attitude that,
01:03
that even supporting things like BSD is just too much of a burden and it's holding open source development back. I really don't agree with that for obvious reasons. It's just something that's been coming from a particular camp of people though. And yeah, and we don't want to write everything ourselves.
01:21
We don't want to completely reinvent WebKit and Nginx and everything else. Maybe we can make some progress though if we talk to the right people and work on the right things and yeah. Hopefully this talk helps. So NetBSD has its own audio API
01:41
that was originally developed at Sun Microsystems like a lot of technology that people are excited about these days. They seem to just, a lot of things that the tech community is excited about now seems to be things that were invented at Sun in the late 90s and early 2000s.
02:01
It's got quite a lot of support in older software, but it's not, it's adoption in your software since Sun died has not been too great. It's a really intuitive, simple API. I had no idea how to read audio code. I read a man page and now I know how to write audio code. Just man for audio if you want to have a look at that.
02:24
And yeah, you just, really you need to use either standard file operations because like a lot of the APIs I'll be talking about today, everything is a file. And really you just use one iOCTL to set the parameters of the audio formats in a struct
02:41
and pass it to the kernel and then the kernel interprets the audio data or gives you back the audio data in the correct format. So yeah, I wrote a mixer after learning to write audio code. I can probably show it to you if I turn off caps lock.
03:09
So yeah, this is AOINXO. You can install that from package source today. It's freely available to the binary package as well now I think.
03:20
And it has quite a lot of functionality actually. If you look at the upper classes and everything and there's various different controls because you don't just have standard volume control anymore. Your laptop has a headphone port, it has speakers. If you have a desktop, it has lots of different ports and you need to be able to control them independently
03:41
if you want to be doing anything but basic tasks. So that allows you to do that. And there was nothing really that existed that met those needs that I had at the time. And this work then led me to Firefox because audio in Firefox was a complete mess. Let's put it that way. It was a complete mess on all of the BSDs really
04:03
until Sound.io came along. There's an unofficial back end that was using NetBSD's emulation of OSS which is the old sound system that Linux used to use like in the early 2000s before ALSA was invented then Pulse Audio was invented
04:21
and I'm sure someone at Red Hat is working on reinventing Pulse Audio currently. And yeah, it's been deleted from FreeBSD already because it was full of bugs. We've deleted it now too. Nobody likes writing or using emulation layers or shims. I really do not want to spend my life
04:41
making sure we are bug for bug compatible with Linux. I cannot stress that enough. Yeah, and Pulse Audio was the option promoted by Mozilla to meet everyone's needs. It's kind of controversial. A lot of our users don't really like it for various reasons.
05:02
It works, but you don't really need it. We do audio mixing in the kernel. Yeah. And Newer Tech just necessitated that we do this because I wanted to be able to attend work meetings on my NetBSD machine. And to do that, I needed WebRTC
05:21
and we had no way to record from microphones using the OSS back end. So now we have full device selection and a working OSS, sorry, a working Sun Audio driver for Firefox. Yeah, and it's a lot more advanced than the OSS back end and supports various of these features like allowing for higher quality audio,
05:44
allowing for surround sound, allowing you to use USB microphones and things like that because, well, I had a USB microphone. I wanted to use it. And yeah, the Mozilla's media team was really nice and helpful. They were less nice and, well, the team at Mozilla that was less nice and helpful
06:01
was their legal team who threatened to sue me for wanting to call our builds Firefox. And yeah, all of this was done in C because the Mozilla audio library is maintained separately from Firefox. It's maintained on GitHub. The maintainers were really nice to me and yeah, I don't know Rust. I don't see the point of writing this in Rust
06:21
because it's just talking to a C kernel API, really. And that brought me, another thing I have been kind of obsessing over recently is randomness and security and random number generation. And all of the BSDs have this API
06:42
for many, many years called Kern ARAND that you also access through CTL. It has all the advantages that the newer APIs have. You don't need to open a file descriptor like dev random so it can be done at low level. It's just an endless stream of highly secure
07:00
random data that never blocks. And there's only a couple of disadvantages just because of historical reasons. We just can't read more than 256 bytes per system call. I think that's fine. That's enough to seed a random number generator. And it's not fashionable anymore
07:21
because the Linux people decided to invent much fancier random number syscalls after there was some drama. But I was just amazed that this API has been around for so long but so little software used it. All of the BSDs had this.
07:41
They've had it since the early 2000s but it only really, OpenSSL was using this. Nothing else was really. And GNU-TLS and libuv use it now and they accepted my patches and they were very nice and it was very easy to get that done so thank you very much. It just results in much simpler, less error-prone code, I think.
08:02
I think this syscall has kind of gone out of fashion because people think it's a bit complicated. This is the entire code in libuv for getting random number generation on NetBSD. I think it's simple enough. I don't think it's too complicated at all. It fits in a slide. Then on IRC there was some drama
08:23
because Red Hat had just announced that they're deprecating XOR in Red Hat Enterprise Linux and that kind of had a lot of people in our IRC channel panicking. We kind of depend on X11. There's not really any alternative on NetBSD or at least there wasn't.
08:43
That brought me to how can I fix this because most Wayland code is written exclusively for Linux. It's non-modular. It's non-portable. It's just a big blob of Linux kernel API usage and FreeBSD's solution to this has basically been
09:05
we're going to implement the Linux kernel APIs. We're going to have a shim around EPOL. We're going to link this against everything that uses vaguely Linux code. I think I've already stated as much that I do not want to spend my life
09:20
chasing bug-for-bug compatibility with Linux. It's really not something we should be spending our time on because there's so many more important things to be doing. It kind of feels like it's holding us back just having to constantly chase whatever they're doing, particularly if they're just deciding that, okay, UDEV is deprecated.
09:41
You have to use UDEV now. Yeah, it's not a fun situation to be in, basically. It's more of an opportunity than a setback because it allows us to truly differentiate ourselves on all sorts of use cases
10:02
because if you have a X11 setup with KDE or GNOME, like the guy in the previous talk mentioned, you might as well run Linux because it's just exactly the same. I wanted to develop a complete BSD-specific display server. I haven't really got too far with that yet, but early stages, we have this.
10:33
So because we didn't have support for Linux's input API, I just decided to have it. Let's make this use our input API.
10:41
So again, this is a very old API that's been in the NetBSD kernel since the late 90s, I think, and it also works around the everything-as-a-file concept. Your keyboard and mouse are just files that the software opens and it reads and interprets the scan codes from the keyboard.
11:01
And all of the input devices are multiplexed. So if you have multiple keyboards plugged in, they'll all be available through one standard device. But there are additional devices. So if you want your laptop's keyboard to not be usable anymore because it annoys you or something, you have an external one plugged in,
11:21
you can just symlink the external keyboard to the main keyboard device. And I think that's quite cool, to be honest. And most Wayland stuff and even Xorg now is using this library called libinputs, which is developed by the free desktop people.
11:41
It doesn't understand our input API at all. It seems kind of focused on providing support for the Linux input API. And we were kind of in a tight situation because this locks us out of being able to use a lot of software.
12:02
I have to shout out for Michael Forney here because he's been very helpful. He developed a library called Simple Wayland Compositor, or SWC, and I took a knife to it. I ripped out all of the input code. I made and added support for NetBSD's input API.
12:22
It works. It's fairly well integrated with the system, so it just gets the same keyboard layouts and mouse settings as the text console. That previously was not a thing with X. With X, you had to set the keyboard layout separately, and that was just infinitely annoying.
12:42
And it fits the security promises that Wayland promotes you. The access to the input devices is provided through a privilege process. That privilege process is the only thing that has permission to access those devices. Once the privilege process is created,
13:01
it spawns the actual display server, which then spawns the window manager, and all of those are running unprivileged. I'm quite happy with this. It's cute. This has taken so much debugging. I've just run into bugs and bugs and bugs.
13:22
There are still bugs. If I right-click right now while giving this talk, the display server will crash. I've been kind of worried about that. That's been my main worry. But it works well enough. I have working OpenGL. I have working web browser.
13:40
This is kind of usable. It's kind of like DWM, if any of you have used DWM. And yeah, you can go ahead and install this and use it on netPST right now if you're running 9, which I don't think has been formally released yet, but a lot of us are running it like it has been formally released yet because it's basically completely stable.
14:04
Yeah, it's still quite buggy and unstable, but it meets basic needs, which for most people who are developers and running weird operating systems, you probably want a few terminals. You probably want a web browser. You want something to play videos with. That's all available.
14:21
And there's like, say, if you want to play some games, this works.
14:51
I'm very nervous about closing that because in my experience, when I close things, sometimes it decides to crash. Sometimes it doesn't. And you just need a supported Intel GPU, really,
15:02
and that's up to KP Lake in 9, which isn't that great. We could be moving things forward from there. I've experimented with running this on my Rodeo desktop. It worked for a few seconds. That drive is kind of deprecated
15:21
and we need to be moving forward with AMD GPU, but again, we're relying on porting code from the Linux kernel, and this is just hell. Nobody wants to be doing this. You have to take months off work to do this, but this laptop's fine. So we have basically a full set of applications working,
15:41
WebKit GTK, so if you like web browsers like Midori, they run fine. A lot of NetBSD people are weird and things like NetSurf and Vimy and those all work too. And most of XFCE works, but obviously not the full thing
16:02
because XFCE relies on X11 for a lot of its window management functions and so on. And QT5, I haven't tried much of KDE because we don't have much of KDE. I've been working on getting us up to scratch
16:21
with some GNOME free stuff, but it's not really what I want to spend my life doing. I don't care about GNOME. And SDL2, which is the most important thing for me because it means you can play dungeon crawl stone soup. I mean, what else do you need a computer for, right?
16:42
And what doesn't work is a lot of things. Any X11 application is kind of stuck right now because we need the Glamour Xorg driver system. We kind of don't. I've kind of been hoping that someone else will do that so I don't have to.
17:01
I'm looking at someone specific in the audience. And screen locking and screen brightness. Through Xorg, you generally handle this with a program like SLock or anything else that you might like to screen save a program.
17:20
But with Wayland, it's not really a standardized part of the protocol, I believe, so it'll have to be integrated into the display server. I do not agree with a lot of these design decisions that involve a lot of things being integrated into the display server. I think it means that we're gonna have to end up redoing lots of work. Maybe it'll be worth it in the end, I don't know.
17:40
But this is the general model that people have decided to go down with kind of moving away from modularity and integrating things into the server more. And screen brightness. This is actually a problem with my ThinkPad that I've been working on. We need a centralized way to manage screen brightness
18:02
because usually you rely on X-RANDA for that. And if you're running Wayland, obviously you don't have X-RANDA. And again, I guess the people are weird and they like to do things like run a web browser in the frame buffer or run ML term in the frame buffer.
18:21
So if you wanna run frame buffer applications, we should be able to do that with them too. And Firefox is a whole can of worms because it's using a lot of weird APIs and functionality. And I can try to run it and give an example of that since apparently I have tons of time.
18:49
Okay, so that's one problem, I believe. Okay, let's look at my own documentation. I'm sure I've written about how to do this.
19:02
Yeah, that sounds about right. Yeah, pretty much, pretty much.
19:23
I'm not using ZFS. I don't think hardware acceleration even works in Firefox. I don't think I have to worry about it being turned on.
19:44
Yeah, pretty much, I wasn't expecting to finish this quickly. Okay, does anyone have any questions? Anyone wants a DPI? High DPI support. Technically, this laptop is kind of,
20:04
everything's a bit small, but I have some documentation that I can bring up. So I have a repository on GitHub called package source
20:21
Wayland, if you want to have a look at that. And it describes a lot of settings that you can tweak. Yeah, see, I've documented how to run Firefox here and promptly forgotten how to do it. If you have a high DPI screen, you can just use these environment variables and that will scale individual QT and GTK applications.
20:43
And that's currently not how we're supposed to be doing it. I think there's some kind of protocol by which you negotiate a high DPI display, but it has to be handled by the display server. And I have honestly no idea how that works. It's something I need to investigate.
21:01
I'm just kind of sitting here. Debugging Wayland protocol code is hell. I would not want anyone to be put through this. I spent months just inserting print Fs into random parts of the networking code that's doing this awful shared memory stuff,
21:20
crashing constantly and just generally not working. But I found that what the best way to debug Wayland code is just to fucking rewrite it. Any other questions? Yeah. Yeah, I'm part of the illness people over in the building case, so.
21:41
I'm not part of the illness people. I just think they're really cool. Yeah, okay. Yeah. Yeah.
22:08
Yeah. Have you checked with the open Indiana people? Maybe you can add some stuff to them. I talked to some illness people at package source con about a year ago,
22:21
and we were just discussing the Solaris audio API since it came up. And we have a lot of extensions to the Solaris API, and we were discussing getting them merged in to Illumos just so that more functionality is available, like proper device selection with easily printable names.
22:43
That would be really nice to have across everything that is still sharing this API, I think. I haven't discussed with them much other stuff because whenever I talk to them about desktop support, they're like, yeah, I might know a guy who has a laptop, but he needs three other laptops because half the hardware's unsupported.
23:01
Well, you should try and find them. Yeah.
23:24
I mean, I care about these things for entirely selfish reasons. I have a laptop. I want it to run NPSD. I want it to run NPSD well. So, but I've been putting in effort to make sure,
23:41
if I write code that is using an API that is shared with Solaris, I make an effort to read the Illumos documentation and just kind of hope that it works. I can't test it myself. I just hope that it works. NetBSD-specific API, is there any WSCons?
24:03
Can you repeat the question? Sorry, yeah, yeah. No, I mean, Nia, can you repeat the question for the people on the screen? Oh, yeah, I'm sorry. So you said, is there any NetBSD-specific? Yeah, do you use any NetBSes? I don't know, honestly. As far as I know, WSCons originated at NetBSD,
24:22
and that's the only documentation I've been consulting. If people want to port things that I've been working on, I would advise them to do it the way I've done and just create a new file, write your code, try to integrate it with the existing system,
24:41
try to make it modular. Just, there's good and bad ways to do portability. And I think part of the problem why people are so put off supporting BSD is they just think, oh, I'm gonna have to add a load of ifdef crap to my code base. And that shouldn't be the case. You should be able to have modular support for things like this and keep it nice and clean
25:01
and keep it nice and separate. Anything else?
25:22
So screensavers, if you actually want to deny inputs to the rest of the system, an individual Wayland application cannot do that. It cannot get complete sole control over whether inputs or input is passed to other applications
25:40
because that was a conscious security decision they made. But they also haven't developed a real protocol for handling this, I don't believe. So you can have a screensaver, but all it will do is stop your CRT from burning in. It won't provide any kind of security from people taking your laptop and posting silly messages on IRC, which I'm sure never happened to me.
26:05
Like, that's my main use for screen locking, personally. Okay. So basically the idea is that
26:22
everything is becoming increasingly platformized. You might've got that hint from the previous talk. GOM and KDE have gone in a direction where everything is becoming increasingly integrated and part of the whole system. That's not really something that interests me as someone who likes lightweight window managers
26:43
and things like that. So, and, yeah, I don't want to deal with that mess. Hey. Very much related to the talk in itself, but this is the first thing I heard
27:01
about deprecation of Xorg within Red Hat Enterprise or Red Hat in general. Yeah. I also like DWM or smaller ones, particularly Xmona, but there are a lot of these which aren't going to be able to be ported to Wayland. Isn't that also a problem they'll have on the Linux side?
27:21
I mean, yeah, it's something that people are mad about. It shouldn't affect you unless you use Red Hat, really. Most of the Xorg maintenance is actually done these days by Alan Coopersmith, who works at Oracle on Solaris. I imagine he'll keep doing that for the time being, at least until Oracle discovers
27:40
where he's hiding in the basement. I mean, please don't forget to repeat the question for the people in the streaming, and then they have no idea. Yeah, thanks, sorry. That's just about the fate of lightweight minimum and minimum measures if Xorg is being deprecated.
28:04
They're saying they're all going to have a lot of, for example, Xmona that I know about, DWM is probably smaller to just be rewritten, but. Well, no, because you have to do a lot more
28:20
to write a Wayland comparison, you have to implement a lot, Wayland compositors do a lot more than just manage windows. I'm kind of going into the next person's talk, who they're going to talk about that for a bit, and I don't really want to tread on. So it's not so simple as you can just
28:40
rewrite a simple lightweight window manager to use Wayland. You're going to have to figure out how it's going to manage the screens, manage all the accelerated windows, and manage blitzing and things like that, and input, output. It's going to need to support multiple OSs. You're going to need to figure out
29:01
how you're going to do that, and the solution that is increasingly emerging is there's libraries like SWC that allow you to do less work, but we're still going to have to improve those if we want to keep this kind of system usable moving forward.
29:24
Anything else?
29:47
Oh yeah, thanks to all the shock people and the girls who showed up to listen to my talk for emotional support. I appreciate it very much.