A Vulkan driver for the RPI4
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 490 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/47390 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 2020214 / 490
4
7
9
10
14
15
16
25
26
29
31
33
34
35
37
40
41
42
43
45
46
47
50
51
52
53
54
58
60
64
65
66
67
70
71
72
74
75
76
77
78
82
83
84
86
89
90
93
94
95
96
98
100
101
105
106
109
110
116
118
123
124
130
135
137
141
142
144
146
151
154
157
159
164
166
167
169
172
174
178
182
184
185
186
187
189
190
191
192
193
194
195
200
202
203
204
205
206
207
208
211
212
214
218
222
225
228
230
232
233
235
236
240
242
244
249
250
251
253
254
258
261
262
266
267
268
271
273
274
275
278
280
281
282
283
284
285
286
288
289
290
291
293
295
296
297
298
301
302
303
305
306
307
310
311
315
317
318
319
328
333
350
353
354
356
359
360
361
370
372
373
374
375
379
380
381
383
385
386
387
388
391
393
394
395
397
398
399
401
409
410
411
414
420
421
422
423
424
425
427
429
430
434
438
439
444
449
450
454
457
458
459
460
461
464
465
466
468
469
470
471
472
480
484
486
487
489
490
00:00
Software developerDevice driverComputer virusVideo game consoleCausalityDevice driverDescriptive statisticsSlide ruleComputer animation
00:33
TwitterSoftware developerDevice driverGame theoryDisk read-and-write headFrequencyComputing platformHorizonForestAndroid (robot)BitDifferent (Kate Ryan album)TwitterComputer animation
02:18
Interface (computing)ImplementationDigital rights managementComputer hardwareCore dumpVideoconferencingCodeWhiteboardCommutative propertyWeb browserNetwork topologyBefehlsprozessorCore dumpHost Identity ProtocolKernel (computing)DampingBitWritingMultiplication signTheoryComputer hardwareDevice driverVideoconferencingVolumenvisualisierungTesselationComputer clusterCodeFunction (mathematics)CommutatorWeb 2.0WhiteboardExpert systemSource codeOverhead (computing)BefehlsprozessorPower (physics)Interior (topology)SoftwareMetropolitan area networkPoint (geometry)View (database)19 (number)Context awarenessTerm (mathematics)Patch (Unix)Open setFrequencyComputer animation
06:17
Device driverFinite element methodTerm (mathematics)MultilaterationVideoconferencingPlanningCore dumpSource codeDevice driverSoftware testingDirected graphArithmetic progressionPiComputer animation
08:11
View (database)Device driverVideoconferencingNP-hardSymbol tableEmailFile formatSystem programmingLibrary (computing)Compilation albumMaß <Mathematik>Macro (computer science)Electronic mailing listOvalFlagSound effectParallel computingImplementationIntelArchitectureRead-only memoryPoint (geometry)CodeMathematicsComputing platformSoftware testingDevice driverControl flowMereologyPoint cloudPhysical systemIterationBitComputer hardwareCodeSound effectImplementationDifferent (Kate Ryan album)Electric generatorSoftware developerAbstractionMacro (computer science)Standard deviationCodeDevice driverMathematicsBuildingParallel portMultiplication signCompilerSymbol tableEmailDevice driverDegree (graph theory)Software bugCompilation albumSource codeComputing platformExecution unitFile formatMathematical optimizationData structureFlagCASE <Informatik>DataflowObservational studyProcess (computing)Film editingCombinational logicNeuroinformatikLibrary (computing)Hydraulic jumpLatent heatVarianceMoment (mathematics)Table (information)Direction (geometry)RepetitionMathematical analysisSoftware testingSelf-organizationComputer animation
16:26
Computing platformImplementationSoftware testingCodeDevice driverGoogolControl flowBinary fileCoding theoryPoint cloudMappingArtistic renderingRevision controlDevice driverSoftware testingTriangleDevice driverCommutatorCodeMereologyCASE <Informatik>System callShape (magazine)InformationShader <Informatik>BuildingMeasurementSineCodeData recoveryOpen setMathematical optimizationArithmetic progressionPoint (geometry)CausalityComputer hardwareTemplate (C++)Multiplication signNeuroinformatikComputer animation
24:42
Coding theorySoftware testingComputing platformImplementationCodeDevice driverControl flowSlide ruleMultiplication signFrequencyFirewall (computing)Computer animation
25:07
Point cloudFacebookOpen sourceComputer animation
Transcript: English(auto-generated)
00:05
So, hello. Really nice for you to show up, because it's not going to be a technical talk, so it's not going to be a flashy talk or anything, but I have animations, so if that's any consolation. The Vulkan driver forcing, yeah, it's true,
00:21
but on the other hand, that's more important, so it's really a lesson in fertility. These slides are linked, I think, in the description. They only work on Firefox, so have fun. Okay, let's start with that. I will tell you who I am, what roughly Raspberry Pi 4 is, if anybody hasn't heard that.
00:42
A few things about, at least, what I think I have learned about Metal 3D. I think a few things I have hopefully learned about a Galium V3D variant, then this strange driver that I'm currently writing, and about possible features. So, who am I?
01:00
I'm the name is Andreas Bergma, I have a GitHub, I have a Twitter. I work at Auto, that's an e-commerce company in Germany. And back in 2017, I've worked in games. So, we did this game and this game. You see down here, the platforms were Android and Horizon,
01:24
which is basically Nintendo Switch. And the interesting thing is, for both these platforms, in 2017, you could write your games in, well, you can render in Vulkan. So, that's what actually these games did. So, you had here a big Vulkan layer,
01:43
where you would render out. And for Horizon, it's a little bit different. The Vulkan layer is a little bit smaller, and you have this NVM, which is specialized, and I think Nvidia variant. And yeah, and we shipped that, and you can play this game now on Android
02:02
and on the Switch, if you want. And that's when I personally got into Vulkan, or got introduced to Vulkan. We were, I think, one of the first Vulkan games that were out there. And yeah, so what I'm doing now,
02:21
I mean, there's the Raspberry Pi 4. So, what I'm doing on the user space, what I need to do, basically, I need to make a C-ABI, which will help off Mesa, which implements all the things that Vulkan driver would assume.
02:41
So, to do that, I don't wanna write everything from scratch. So, I would like to utilize what the kernel, or what in the kernel already is. That's the V3D driver, which basically can handle the hardware of the Raspberry Pi 4, which is here, this Videocodec 6.
03:04
And yeah, that code is already there. So, without that, it would even be more hard. Yeah, if you wanna look at the source code on my patches, you can go here and see what a moron I am, or anything. I mean, I'm really starting new at this.
03:20
So, you don't have to have any big expectations, because it's just random stuff. Why am I doing that? I started earlier with that. Basically, we have, for example, Raspberry Pis at work. We use them as dashboards, RAM boards. We even try to use Microsoft Teams with that.
03:43
And what you really notice with Raspberry Pis, or the smaller PCs, if you will, they don't have too much of power. So, every bit of power that you can have, be it in the GPU or CPU, you want to have.
04:00
So, personally, I would say, it would be perhaps really beneficial to have Vulkan to get all the overhead from OpenGL out. But that's just a theory. I don't know whether this is actually good. Personally, I've just never been really fond of OpenGL. I've dealt with it a little bit, but Vulkan is rather more my alley.
04:22
It's more explicit. From my experience, I've most experienced in C++ Go and C. So, basically, that makes writing for Mesa, I think, doable. And I have currently a little bit more
04:41
than one hour of commute every day. So, I have free time with my hands. So, hey, why not? In the future, of course, the hip talk, we will hear, I think tomorrow, there's some web GPU and people try to do some abstractions, if I heard this correctly, on top of Vulkan, for example.
05:01
So, maybe it will be a web GPU on Raspberry. Who knows? Yeah, Raspberry 4, I mean, was released 2019, has these two HDMI outputs now. It's a little bit more beefy. Everybody has got every complaint about it.
05:21
So, I will not repeat that here. Basically, it should be able to support Vulkan 1.1. The CPU can actually do 64 bits. The video core cannot, as far as I know. So, currently, I even have 46 bits of the driver.
05:43
I have not tested them yet, but I would assume that there might be some problem somewhere because people just assume that an int is 32 bits or whatever. Who knows? Or vice versa. And basically, the video core is a tile renderer.
06:03
So, we have the spinner thing, which goes to the renderer and you render a tile out. I'm not the expert, not a single expert. Experts are sitting here probably who can explain it way better than me. And I thought, what did I have to learn?
06:23
And that's basically summed up here. So, I started looking at that, at the new hardware, and was saying, okay, what is already there in terms of software, infrastructure, whatever? So, you have here the VS3D. Eric is maintaining that.
06:41
You have the VC4 for the video core 4, which is the Raspberry Pi 3. I think Eric is maintaining that, but I have no idea, quite frankly. And then it gets interesting because if you look at the performance testing at Kronos, there is actually a Broadcom driver in there.
07:04
And Broadcom, I have this from good sources that they internally really have a Vulkan driver, but apparently they have no plans in releasing that to the public. So, I don't know. Okay, that one, I mean, that's what I am doing.
07:22
I named it v3dvk, and I'm basing that on another 3D. There is the other one, which Eric started before he left I think Broadcom, and he named that v3dv, I think.
07:40
It's also based on Mezza, but didn't really compile as far as I remember. And for the older hardware, there is rpivk. I don't know what the name of the guy is, but at least he continuously, every few months, does something on that, and yeah,
08:00
makes some progress, I guess. So, that's currently what the landscape we have, and more to that later. Yeah, so what did I learn from Mezza 3D? I think, and I can be wrong, the biggest part of Mezza seems to be Galleon,
08:21
which is basically the OpenGL VR driver or variants. I don't know how to describe that. There is a bit of infrastructure for Vulkan. It's not nearly as much as for Galleon, might be intentionally, I don't know.
08:40
The big problem is you cannot really differentiate currently between common, between Galleon, and between Vulkan, because the code feels like it grew hysterically. So, basically means that a lot of common uses Galleon.
09:00
So, it's really hard to write your Vulkan code without putting Galleon in it. So, I think there is some cleanup underway currently, but currently it's not really clean. So, and that reflects in the built system. Yeah, Galleon v3d supports vdco6.
09:22
So, Eric did, as far as I know, all the necessary changes to have basic support. So, which is really good for me because I can look at the code and try to figure out what it does and port it into Vulkan. And I already said, without Galleon, it's really hard to get a Galleon out, let's say.
09:47
Vulkan infrastructure. What I noticed is Vulkan drivers have a monolithic, your driver private edge. I don't know what the reason for that is, but it's not nice for people like me
10:00
who have a small Haswell CPU and have to recompile a whole driver each time. So, I did not follow that approach. I have smaller headers because I cannot really, I mean, I have limited time every day. Having big headers is, I don't know, might have some reason, but it's not the reason I understand.
10:23
And finding symbols in this gigantic header is not really fun. The last infrastructure thing I saw was that the formats now moving out of Galleon into the common.
10:40
I think not everything is outside of Galleon yet, but it's slowly moving. So, it's getting better. That's the good thing. And I complained already today. I will complain again. The problem I at least had and have to a certain degree with Mesa is it feels like a cult. So, if you're in that, that's great.
11:00
But if you're new to that, you have a problem because there is all the things you need to do or you would like to do, it's not described, or you just need to be in it. And it's really hard to get in it. But also, I think people are getting aware of that a little bit more.
11:20
Then there's this build system. I'm wearing the Basel shirt not because I think Basel is so bad, because I think Basel is way better than Meson. But your Meson is probably okay, but it's just not great. Isolation and all those things. You really see that it's not as clean as it could be.
11:40
On the other hand, it works. So, yeah, we have to deal with that. So, aside from the Gallium V3D part, there's also a part where it's a general Broadcom library, which is not in Gallium. And that's basically you put XML in
12:03
with a magic Python that we already heard of. Then you get compilation units for the variants out. Currently, I'm just interested in the latest. So, it's not that much of a problem for me. But for the V3D in Gallium,
12:21
they really do a lot of work to support all the different generations of hardware they have. Problems. We have problems documented in the codes. If you're really interested in how the hardware behaves, it's here and there and everywhere, basically. So, basically, what you will have to do
12:42
to really understand it, and try, at least, is to read through all the source code and try to really pick up all the nitty-gritty that some developer documented. And I'm not sure, I couldn't really find it. There's different documentation
13:00
how ready compute already is or not. And I still have to figure that out for V3D. For Gallium, so that's the next layer, basically, which uses the method library. This is, I think, a pretty standard Gallium variant. I don't understand everything from Gallium,
13:22
so bear with me. The only thing that I found kind of notable is that they have some macro abstraction for setting the, in this case, is the binner command, where you're not setting the single bytes,
13:42
but you'd rather go ahead and set the structures, and then it's a macro magic that converts that to the actual bytes. Under the hood, it also handles the variants, so hardware-specific things. And to me, it makes the code more readable.
14:05
It kind of trusts that the compiler does enough optimizations for that to go away. I'm not sure, I don't trust compilers that much. But it's okay, I think. The other really beneficial thing is, to me at least,
14:21
who gets new into the whole topic, is that I can read the code more easily, I think. And that's really beneficial if you compare it with Fridrino, and they do, as far as I know, manual bytes, and it's a little bit clumsy to read for them. The cons are, to me, yeah, the macros,
14:43
I had a bug once in that code, or around the code, and debugging with that or tracing is not so fun, but it's doable, so yeah. There's maybe some performance on the table, but I don't know, I haven't measured.
15:01
And OpenGL is this dirty flag, implicit effect handling, missing parallelism at times, for v3d at least. That's just, I think, OpenGL implementations. So it's not really that bad, but it could be improved. And Vulkan is, if not, anything to improve on OpenGL.
15:25
How did I start? Well, basically, at least I decided where to steal code. I steal code, I implement what I need to change, and I jump back to two, so I steal.
15:40
So basically, if you look at the drivers, everybody is stealing from everybody. I think Envy was the first one, the first Vulkan driver, and we're slowly copying code and trying to make it a little bit better at each iteration. And that works not that bad. Let's put it that way. It's not original, but hey.
16:02
Myself, so at times, I got sick of writing code, and then I was like, oh, yeah, let's do a SCM platform or the CI. So I have, in GitHub, I release snapshots, have SSH to test, I have the CTS,
16:22
which I run periodically, and I have a CI on Cloud Build because I don't really know how to interact with the whole GitLab CI. Perhaps I should look into that, but yeah. Works pretty great in Cloud Build. So the timeline is roughly, so I started in, I don't know,
16:43
probably somewhere, summer. Then in the, around here, I had the Vulkan info running, so the basic infrastructure for reporting the features and things. I had a first CTS passing in October.
17:03
Then somewhere in November, I set up my CI. A few days after that, I released just snapshots, and that's just no really sense or not that sensical. Let's put it that way. And somewhere in January, I had at least 64 CTS passing,
17:22
so that's not great, but it's enough for me. Let's put it that way. Mihai also had to prepare a talk here. If you remember, I began working with the NV as a template, I switched to Turnip instead
17:45
because NV has already a lot of optimizations, and they are hard to translate to my hardware, so rather need, or rather use Turnip now as my base. And currently, I'm trying to figure out how compute shaders work,
18:01
whether the infrastructure is already there or not. And that will keep me busy for some time, I guess. The roadmap, one or the other will have read about Igalia, so they are working since, I think, two months or more
18:21
on their own variant of the Vulkan driver. We already talked together, and we will try to figure out how other people might work on their drivers. So it might be the case that I'm throwing all the code away
18:40
that I wrote, that's possible. If we are really lucky, then we might be interested or able to implement or move some of the code to them. But nevertheless, I hope the road forward is still more coding, getting the driver into better shape.
19:05
And yeah, for me, it's just test suites, getting them to run, and at some point, hopefully have a driver. On the CTS, I would personally be really interested about having a mapping between the tests and the API calls.
19:22
So you could tell, okay, all API calls from, I don't know, creating pipelines or anything are done. So that part of the API is just, should work as intended. So yeah, and with that, I try to breeze through
19:41
all of that, and now questions, please. Sorry? It is really quiet. Yeah?
20:07
No, no, what I could do is I could show you, if all goes well, how the CTS runs, but that's not really interesting, I think.
20:20
So on that explicitly, I think that I have a different approach. So the Igalia guys, if you have seen the news, they have a triangle rendering. So I don't care about rendering. I only care about testing. So for me, it was like, I don't care if I need five years for that.
20:40
I don't need 10 years. I just wanna learn about these things. So I'm more about, okay, I will use the tests and try to comply with the tests and try to figure out what I have to do to appease the tests or appease the APIs. Yeah.
21:20
I think it should be there. Yeah, so the question, sorry, was whether the CTS, or my question was whether the compute shader was already in place. And the comment was that it has to be in place because OpenGL ES31 needs them anyway.
21:45
And I think that's correct. The other way around is that the documentation is not correct. So probably one of the documentation problems. Yeah, sure.
22:08
So the question was how many, many, many days I have put in a completed driver. So then the question is rather,
22:21
how long will we probably take for a completed driver? And the news from the Raspberry Foundation was don't hold your breath. And I think that's realistic. So there's pretty good chance that this might take a year. And I think a year might be realistic
22:42
since the Galia guys work full-time with two people on that. So if we already get a few other people into that part-time or whatever, I think a year might be realistic. Not full-fledged feature, but I think workable.
23:02
Perhaps, I don't know, open arena. I'd say open arena should be doable in a year. Yeah. Is there any way that this work might be of some benefit to the older Raspberry Pi versions? Oh, so the question was whether that work
23:22
is any good for any older versions. And the answer is maybe. So there's currently a lot of handling in the Galium v3d. And at least in my code, I threw all that out. So if you would integrate similar handling in the Vulkan,
23:44
it might be actually. The other thing is I think there's hardware-wise one or two things that are not sufficient for the Raspberry 3. So it might not be totally there, but perhaps enough fish, I don't know.
24:03
On the other hand, there's this RPI VK driver, which is not based on MESA. And I think it may be doable if you get that person, for example, interested in doing that,
24:20
then there may be some progress, yeah. Anyone else? Yes, with the Raspberry Pi, yeah? So if I understand correctly, whether I develop with the Raspberry Pi or? Whether you commute with the Raspberry Pi.
24:40
Oh, no, oh, yeah, sorry. Okay, the question really was whether I commute with the Raspberry Pi, and I have thought about that. But no, that was in a slide here. I have SSH access through my firewall, and I execute that remotely on my Raspberry. So yeah, it works.
25:01
Only time's up, so thanks.