P4 in Nix
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 542 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/61610 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 2023146 / 542
2
5
10
14
15
16
22
24
27
29
31
36
43
48
56
63
74
78
83
87
89
95
96
99
104
106
107
117
119
121
122
125
126
128
130
132
134
135
136
141
143
146
148
152
155
157
159
161
165
166
168
170
173
176
180
181
185
191
194
196
197
198
199
206
207
209
210
211
212
216
219
220
227
228
229
231
232
233
236
250
252
256
258
260
263
264
267
271
273
275
276
278
282
286
292
293
298
299
300
302
312
316
321
322
324
339
341
342
343
344
351
352
354
355
356
357
359
369
370
372
373
376
378
379
380
382
383
387
390
394
395
401
405
406
410
411
413
415
416
421
426
430
437
438
440
441
443
444
445
446
448
449
450
451
458
464
468
472
475
476
479
481
493
494
498
499
502
509
513
516
517
520
522
524
525
531
534
535
537
538
541
00:00
Hill differential equationArchitectureBitGroup actionParsingDefault (computer science)Functional (mathematics)CodeFlagAddress spaceLevel (video gaming)Game controllerEmailComputer architectureCountingResultantComputer programmingCASE <Informatik>Maxima and minimaProblemorientierte ProgrammierspracheKernel (computing)Computing platformInternetworkingStandard deviationType theoryMultiplication signEndliche ModelltheorieElement (mathematics)Communications protocolCompilerSoftwareIndependence (probability theory)Software testingElectric generatorPlanningAbstractionPoint (geometry)Interface (computing)CoprocessorProcess (computing)Source codeComputer hardwareProgramming languageState of matterStructural loadBit rateComputer fileMereologyLogicDiagram
08:34
BootingDigital photographyStructural loadStreaming mediaRouter (computing)Information securitySoftwareLine (geometry)Interface (computing)Programming languageAbstractionCASE <Informatik>Computer configurationSource codeCodeMultiplication signOpen sourceServer (computing)NeuroinformatikField programmable gate arrayHexagon2 (number)Demo (music)Latent heatFunctional (mathematics)Game controllerMathematicsComputer hardwarePoint (geometry)Internet service providerStack (abstract data type)BitComputer programmingPlanningMechanism designDirection (geometry)Different (Kate Ryan album)MereologyArithmetic progression1 (number)CoprocessorClosed setBefehlsprozessorComputer animation
17:03
VacuumGamma functionBinary fileInclusion mapElectric currentProgrammable read-only memoryINTEGRALMach's principleInformation securityBootingWeb 2.0
17:17
Gamma functionWorld Wide Web ConsortiumComputer-generated imageryLattice (order)File formatExtension (kinesiology)Cache (computing)Software testingMoment of inertiaProgrammable read-only memoryError messageDevice driverVirtual machineLine (geometry)SineInternet forumCubeComputer fileMach's principleBootingLevel (video gaming)Computer animationSource code
17:37
Mach's principleRootAreaProcess capability indexVirtual machineGamma functionBootingPhysical systemElectric currentGEDCOMKeilförmige AnordnungRepetition
17:57
Computer animationProgram flowchart
Transcript: English(auto-generated)
00:05
Please welcome the next speaker, Govan. Hi. So my name is Govan Todioni of the H2O Center, which is a really long name. So you can just call me Govan.
00:20
And I'm going to talk about this great thing called Google. So before that, I was sponsored by the Internet Foundation with a grant that is financed through the European Commission called MGI and Sure and Generation Internet. So if you have ideas and just want money, just go back to it. It's actually a really cool thing. So now, what is before?
00:42
So I guess the academic definition would be the programming protocol independent packet processor is a domain-specific language for network devices specifying how data plane devices, switches, NIC, routers, filters, process packets. OK, great, but what does this actually mean? OK, so it's a language basically for hardware-optimized network processing.
01:06
Think SIMD for network. So you can just have those FPGAs, DPUs, whatever, and you can make them process network packets for you at an accelerated rate compared to software. So it roughly looks like C in a way.
01:23
As you can see, it has the same kind of syntax, like arguments, whatnot. The thing is, there's a few things, there's a few hardities. You can see it has this state thing, it doesn't have a return. You have transition select. What is this? It looks a bit weird, so let's explain this a bit.
01:44
Functions in P4 are replaced mostly by things called parser control packets. So, what is parser? A parser is a function that is going to pass an incoming packet according to, well, the same thing as in C, which means struct, type def, etc.
02:00
Say I'm going to define a struct, I'm going to say, if this element of a struct is this, then I'm going to call this function. Basically, that's what the parser function does. You have then the control functions. Control functions modify your past packet. Say, for example, you have this really long packet which says, I have op x, y, z, and then you want to remove the op x because you're on op x.
02:22
You're going to do this in a control function. And then you have pack art, which defines basically the binding logic between the hardware. Like, oh, am I going, say I implement a 5-1. Oh, am I going to add new rules? Or am I going to load new definitions and whatnot?
02:40
So, that's basically it. There's other interesting keywords, there's other things that I'm not going to explain all of this because P4 is kind of, like, this would basically be out of scope for the talk and P4 is already complex enough as is. So, how do you want to use P4 in Nix?
03:02
There's a few ideas. My take on it is let's make a transpiler. Why? Because a transpiler allows you to reuse P4 concepts within Nix. Say, because P4 is a bit verbose. Because, say, you want to pass this IP TCP packet something. You need each time to redefine the IP header thingies.
03:22
So, you want to redefine this struct, you want then to have it passed in a way, and then sometimes you want this thing, sometimes you have this other IP struct because you don't want to pass some optional data in the header that would slow down the process as well. So, it's a bit verbose. Let's just use Nix to basically implement parts of it and have it in a nice packet.
03:42
So, what is a transpiler? In our case, it's a Nix to P4 translator, which means you can define things in Nix, structs, enums, whatnot, and it generates P4 code. Then you have the P4 compiler, which actually processes the code that's generated. Followed by the target compiler,
04:02
which in this case allows you to deploy your thing to, say, an FPGA, a DPU, etc. Basically the thing that allows you to run your thing, your program. So, what does it look like in action? Basically, this is a Nix file
04:22
which allows you to define some P4 concepts. In this case, I defined a header flag which contains a few counts like max-ops, standards-ops, etc. And then a few headers. In this case, I'm just defining standardity, which has two flags, source and destination,
04:41
both of type bit-8. That's basically just a way to redefine the annoying thing of P4. Then you can also have include. All this is processed then by this function called run transpiler, whatever I called it now. You just run this, and it generates code for you.
05:01
And that's great. So, we can simplify this, obviously, because that's a bit annoying to each type, like this header, this header, etc. And the whole point of making this whole thing is to make it less verbose and to reuse code so you can just have your own info, mostly in Nix and a few bits and packages in P4
05:21
so you can basically put everything in the same place. In this case, you can just innovate things that I define in helpers, and you can do the same thing, and you get your P4 source. So, that's a lot simpler, and we don't have to define Ethernet each time, we don't have to define MAC addresses each time, which is a thing in P4, because if you use standard P4,
05:42
it's going to make you have to redefine all this at each program you make. Which is thanks to helpers that I wrote down in my packages. So, what does the hand result look like? Because I'm saying, oh, this is the Nix code, this is what it looks like, etc. Yeah, it's great.
06:00
So, this is basically the transpiler. It's kind of dirty in a way, like you can see it's a lot of messy functions, but basically the idea is that I define a module, a NixOS module, which verifies the types of what I give it, in this case the header,
06:20
I give it a default type, then you can have the unions, the content, the whatnot, etc., which are then passed by the transpiler, which are these huge nested functions, which basically just map Nix address into P4 materials,
06:40
and then just write it. It just starts to search transpiler. It's nothing fancy, but it's a thing. And so, what does the hand result look like after the transpiler? It looks like P4 code, but pretty clean. You have the includes, the define, the ups, max ups, standard metadata, then you have your own structure, etc.,
07:01
and then you can include your own code. And then, this whole P4 code is then processed by the target compiler, which then processes it to its own platform. Say, it can actually generate, say, eBPF code,
07:21
so it runs on Linux. eBPF is basically like this kernel thingy that you can run to have more privileges. I'll explain it later if I have the time. And this is bmv2, and bmv2 is just a simplified model architecture. I'll also talk about it later. So basically, it starts to search transpiler
07:40
that has multiple status and allows you to specify P4 functions and reuse concepts. It's nothing fancy, but it's useful. So, now that I talked about bmv2, I think I have the time to do it. So, what is it? Glad you asked. So, the simple switch architecture is the defactor architecture
08:00
that is used to basically test P4C, which is the main compiler. So, it's basically like this kind of standard test setup that you can use to just see if your code works, but it's not fast. You can't really target it anywhere. It's basically like this low, user-length thing that you can use to just test your program that is not fast,
08:20
but does mostly what you want. It's basically just like this interface for targeting the switch. It's an abstract interface, and it's also used to just verify how P4C works. So, now to continue a bit more on P4C. To use P4C, you need a target,
08:40
and the main targets that are currently implemented in P4C are user-length, which is, in this case, eBPF. Some people are going to criticize this because eBPF is a click-click kernel, but what I mean by user-length is I mean it runs on a computer. DPDK, which has, and this is why I'm saying it's user-length, its own user-length code that can run,
09:02
but the DPDK also is like this, how do you call that? An API, I guess. It's an API that can target a bunch of devices. So it's not only user-length, but it has a user-length point. Then hardware, because you can use DPDK to also target hardware. Say, FPGAs, which are like,
09:21
for people that don't know, things that you can use to program CPUs or whatnot. You can basically just reconfigure the electrical level, electrical gates, et cetera. And then custom ASICs, which are like custom processors and whatnot, basically. And then just emulated. I'm going to call it emulated because of how slow it is. It does basically, let's test the VMV2.
09:43
I told you about it already. So obviously all of this needs some kind of interface to work on. And usually you use the abstract switch interface with a few petabytes of changes and like some control functions usually have a definition specific for eBPF and the like.
10:02
So there's a few changes, but overall you can target basically all of these with mostly the same code, which is great. But this also needs changes to be transpire. So you need to add more options so you can target every different one. So you can say, have the whole mess auto-generated by transpire.
10:20
So say you can just in Nix define, hey, I want target hex and it's going to do it automatically for you without having to change the P4 code, which is mess. So now introducing FPGAs on Nix. And I'm actually not kidding. The thing is I forgot to actually take the picture before going to for them. So imagine it's really hard. Basically an FPGA connected to a computer
10:42
through say USB or whatnot and then connected to the network through Ethernet. The idea is that we are not running Nix on FPGAs. We are using Nix to define what the FPGA wants, in this case the network stack. So we need to define, like say,
11:00
the devices we have in Nix directly. So say I'm going to have like hardware.alastor.type is my FPGA and then I'm going to explain how I can define it. Then you need some kind of auto-reload deploy mechanism. Usually most FPGA providers just provide you a way through USB.
11:23
You can also have just an auto-reload of fools through the control plane in P4 directly if you want to go this way, which means basically no downtime network stack, which is nice. And yeah, you can have a data plane mechanism for feeding data to the host later. All of this is a work in progress for now.
11:42
So the transpiler is done. Basically most of the source-to-source transpiler, including the targets are done. The targets are packaged. You can use BMP2 nowadays. You can use DPDK and whatnot. But the hardware definitions are currently a work in progress. I haven't had the time to work on them yet, but it's close enough.
12:01
Close enough. And software works. So yeah, that's basically it. Any questions? Yeah. How can you get a switch that runs P4? So the question is, how can I get a switch that runs P4? P4...
12:22
So you don't technically need a switch that runs P4. You can just use any, say, FPGA or DPU that you can reprogram and that is targeted by DPDK. You also have some routers that have ASICs that are reprogrammable. As long as you can target it with DPDK, which is basically the API you can target,
12:41
you can just use it and run P4 on it, basically. Maybe I missed it, but what is it that makes NICs particularly handy for this compared to other languages like this? Okay, so the question is,
13:02
what makes NICs particularly nice in this case for P4? So what makes NICs in this case useful is that it's at the border of basically like this Boeing code to generate per target and you also need to actually target an infrastructure and basically NICs is this great thing where you can just define an interface,
13:23
an infrastructure I mean, and you can then write automated code. So NICs in this case has two key roles. First, allowing you to write code with less, not having to rewrite every time thing, making it just basically this handy source tool. And the second is to define your network stack in the same place
13:43
as you would define your standard infrastructure, which means, say, I want to define this server, okay, cool, but now my server has like 100 million requests, I don't know, and I need to actually like half load this for some degree. So I'm just going to say, hey, hardware-definition.FPGA is my FPGA and now I have my speed magically.
14:01
Well, you need to pay for the FPGA and program the thing, but yeah, magically. Any other questions? Yeah? This is useful for things that are not networked quite straight, so you could do things that have nothing to do with networking or you're targeting an FPGA, is there a scope for this
14:23
or is it basically just the networking stuff? So the question is, is this going to be useful for other things than networking? The answer is yes and no. So the FPGA deployment mechanism, yes, is going to be useful in this case because you can just reuse this thing and use it outside of the scope of P4.
14:42
The reason now is that everything else is basically P4 specific. But yes, the last part is probably going to be reusable for other things than P4, first of all. And second of all, there's going to be a few different FPGA targets, probably. Like, there's always going to be the proprietary ones with DPDK
15:00
that have their own, say, deployment binaries, that are clusters, that are a pain. But you can always see, like, try to implement, say, here this thing is and whatnot to have an open source FPGA setup. So, yeah. You can use it outside of P4, basically. Yes? Yeah, sure.
15:24
The P4 language is heavily expensive, you know. Like, less than digital, cash-coherent, in the bloody bank. I mean, if you have any kind of stream that you can process, if you can express it in a way that it can be processed,
15:43
in real time, you can process it in real time. Yeah. That's a really good point, just for the live people. Basically, what was said is that you can use P4 to process any kind of stream and modify it, and it's used by other companies
16:03
to process this in an accelerated fashion, which is very true and very handy. It's not the most common use of P4, but P4 is already not very common. We have too many steps. Yeah. Did you want to do P4? Okay, let's do that. So, thank you, that was me going by.
16:25
Here's my website, here's my email, and yeah, we have one last thing to show. So, hm? He wants to take a photo of your own. Ah, sure. It should be recorded, so anyway, so yeah. Yeah, I'll take on one more line.
16:40
Okay. You don't expect people to actually care about me, honestly. So. Because the next talk is about secure boot? Yes. So, it's a demo about secure boot? Yeah, there's one thing I forgot to mention. I'm working on P4 on this, but I'm also working on the secure boot thing a bit, at least. So, what I wanted to show you is that.
17:03
Let me just do that, do that, so I can just do this. I started this small PR the other day, which is like web support for secure boot. So, yeah, I think now we can actually boot secure boot on XOS using grub, and to add on to that, let me just show it there,
17:24
because I have too many windows. There we go. Let me just. Yeah, there we go. Welcome to grub, Lanza Booty. And I'm going to, I'm just going to zoom for the live, okay? Yeah, there we go.
17:42
And basically just this whole thing is working on a secure boot, and as you can see, it's panicking, but it's close enough. It's close enough. So now, I'm going to let the next speaker explain what Lanza Booty is.