Networking-VPP
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 644 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/41506 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | |
| Genre |
FOSDEM 2018545 / 644
2
3
5
6
7
8
10
12
16
18
27
29
30
31
32
34
39
46
47
48
55
57
58
61
64
67
76
77
80
85
88
92
93
98
101
105
110
114
115
116
118
121
123
128
131
132
133
134
140
141
142
143
146
147
149
162
164
171
173
177
178
179
181
182
184
185
187
188
189
190
191
192
200
201
202
204
205
206
207
211
213
220
222
224
229
230
231
233
237
241
242
243
250
252
261
265
267
270
276
279
280
284
286
287
288
291
296
298
299
301
302
303
304
305
309
310
311
312
313
316
318
319
322
325
326
327
329
332
334
335
336
337
340
344
348
349
350
354
355
356
359
361
362
364
365
368
369
370
372
373
374
376
378
379
380
382
386
388
389
390
393
394
396
400
401
404
405
406
407
409
410
411
415
418
421
422
423
424
426
427
429
435
436
439
441
447
449
450
451
452
453
454
457
459
460
461
462
464
465
470
472
475
477
478
479
482
483
486
489
490
491
492
493
494
496
497
498
499
500
501
503
506
507
508
510
511
512
513
514
515
517
518
519
522
523
524
525
527
528
534
535
536
538
539
540
541
543
544
545
546
547
548
550
551
553
554
555
559
560
561
564
565
568
570
572
573
574
576
578
579
580
586
587
588
590
593
594
596
597
598
601
603
604
606
607
608
610
613
614
615
616
618
619
621
623
624
626
629
632
633
634
635
636
639
641
644
00:00
Slide ruleBitDemo (music)Presentation of a groupSound effectComputer animationLecture/Conference
00:58
Plane (geometry)SoftwareCodeVector spaceProcess (computing)Computing platformComputer networkMicroprocessorMaxima and minimaConvex hullArchitectureCurvatureScale (map)Sound effectDevice driverSoftwareUsabilitySlide ruleDemo (music)Vector spaceProcess (computing)Level (video gaming)System callPressureSource codeOpen sourcePlastikkarteLine (geometry)1 (number)Graph (mathematics)User interfaceInternetworkingCodeCASE <Informatik>Computer hardwareOrder (biology)Cache (computing)Presentation of a groupReading (process)Kernel (computing)Projective planeoutputWeightEndliche ModelltheorieProgrammschleifeSemiconductor memoryDomain nameDuality (mathematics)Image resolutionRevision controlCentralizer and normalizerRegular graphBefehlsprozessorCommitment schemeRight angleShared memoryBitArmQuadrilateralNumberGraph (mathematics)2 (number)Plug-in (computing)Run time (program lifecycle phase)Limit (category theory)Instance (computer science)Multiplication signMessage passingCore dumpIntelSingle-precision floating-point formatInjektivitätPolygonFilm editingKey (cryptography)Lecture/ConferenceComputer animation
10:21
Scale (map)Scaling (geometry)BefehlsprozessorCore dumpHill differential equationProcess (computing)Demo (music)Sound effectRight angleSoftware testingLine (geometry)System callNumberCore dumpMeasurementCache (computing)Slide rule40 (number)Operator (mathematics)LinearizationCausalityMathematics2 (number)Point cloudCycle (graph theory)Endliche ModelltheorieBitNetwork topologyDecision theory
13:06
BefehlsprozessorScale (map)Limit (category theory)outputFunction (mathematics)OctaveConnected spaceLocal ringIn-System-ProgrammierungMaß <Mathematik>Right angleLinearizationGodDemo (music)Server (computing)Client (computing)Address spaceComputer animationSource code
13:47
Term (mathematics)Server (computing)Default (computer science)IntelFourier seriesBand matrixHeat transferMaß <Mathematik>Grand Unified TheorySystems engineeringSystem on a chipState diagramoutputFunction (mathematics)Tape driveInterrupt <Informatik>OctaveInterface (computing)Execution unitClient (computing)Process (computing)State of matterVector spaceArchitecturePlug-in (computing)AverageVector spaceGraph (mathematics)Regular graph2 (number)Server (computing)Presentation of a groupConnected spaceRight angleSound effectProcess (computing)Service (economics)Source code
15:31
Maxima and minimaDevice driver2 (number)Kernel (computing)Regular graphCellular automatonPressurePresentation of a groupOrder (biology)AverageProcess (computing)Hand fanBefehlsprozessorAsynchronous Transfer ModeLoop (music)Core dumpRight angleSystem callStack (abstract data type)Source code
17:19
Computer networkDevice driverSoftware bugArchitectureMechanism designSynchronizationInformationMaxima and minimaInstance (computer science)Client (computing)Endliche ModelltheorieProcess (computing)CloningProxy serverTransport Layer SecurityKey (cryptography)Information securityElectronic signatureMessage passingToken ringPublic key certificateRootVertex (graph theory)Data managementRevision controlOperations researchSoftware testingVirtual LANContext awarenessConsistencyPresentation of a groupRouter (computing)Level (video gaming)EmailRow (database)SoftwareFeedbackProduct (business)BitReading (process)Video gameCrash (computing)Connected spaceOpen setStack (abstract data type)Computer configurationState of matterMobile appSlide ruleComputer architectureDatabase normalizationPoint (geometry)Information securityDevice driverMultiplication signPhysical systemMereologySoftware testingDatabaseRight angleNumberRevision controlContext awarenessInstance (computer science)Differenz <Mathematik>Virtual LANWeb 2.0Bridging (networking)SpacetimeGame controllerMetropolitan area networkDemo (music)Object (grammar)Java appletUser interfaceWeightConfidence intervalPublic key certificateCentralizer and normalizerFood energyElectronic mailing listTheoryWater vaporGroup actionInterface (computing)PlastikkarteRegular graphComputer animation
26:42
Service (economics)Program flowchart
Transcript: English(auto-generated)
00:14
Yeah, that's okay. Thank you. So we talked a bit about VPP this morning, but but I realize many people here
00:22
may not know about About VPP. So let me calibrate my presentation. Who in the room knows about VPP? Okay, so some of you know, but not everybody so I can what I can do is I can I can share few slides about what VPP is and then while
00:43
While gals was talking I did set up a quick container demo using using VPP So perhaps I will be here and it's very risky because I just did it now. So we'll see and Maybe we'll have one of those a demo effect and then I will share with you slides
01:03
Explaining how VPP is integrated into open stack with this networking VPP ml2 driver Okay, so let me Share a few slides first quickly and then we'll go into the demo
01:22
Okay, so VPP stands for vector packet processing. It's a project It's an open source project under Linux Foundation and it actually it's it is actually under another umbrella project Which is FD dot IO which stands for fast data that I owe VPP is a software that does packet processing what that means is that
01:42
It's packet and forward them and modify them and follow up them again So this kind of thing which is usually done in Linux kernel or in other packet forwarding or in hardware, right? So it's a software implementation. Usually it works on DPDK, but then you have also net packet IOS at on top of which it can sit and
02:05
This project sits here in the in the instant stack so this morning Charles was talking about open daylight, which is here. I guess later in the after in the day There will be a presentation about OP and heavy So a VPP is really low layer right the thing reading packet forwarding packet and the thing which needs to be really really fast
02:29
so this Fido project is actually a multi-party project with many people coming from many companies Contributing so we see people from Intel ARM
02:42
Ericsson so many many companies are actually contributors And it's a very active project, right? So here it's I I don't I won't go into the details But basically I took three project in the same domain So yes, DPDK and VPP and try to to show the number of commit. So it's it's it's a very active project
03:05
You have many components but on that I will go quickly So that's the thing I like Well that I will go quickly Why is VPP fast and what is what is that pretty? Okay
03:21
So what VPP does is it treats packets? And forward them so that does layer 2 layer 3 and now we also have layer 4 what is extremely important to understand is The secret source of VPP and don't tell it because it's a secret source It's it works with vectors of packets in order to
03:42
Make the best use of instruction in instruction cache as well as data cache, right? So what we do is when we have to process a packet We go through several nodes So for instance, we can read packet from DPDK and then we'll get a bunch of packets from DPDK There will be processed at the internet level and then we perhaps will do
04:03
IPv6 or IPv4 we do a lookup and then we'll forward it, right? So we have several steps, right? If you look at the usual code data path The usual code for data paths, it's actually pretty long, right? So you have a lot of instructions to do because forwarding a packet is not that simple
04:21
You may have to do a calls You may have to do a NAT or NOT. You may have to do many many things, right? So it's a lot of lines of code And what we are trying to do is to make sure that when we execute an instruction to process a packet This instruction is actually in the cache of the CPU So, how can we do that?
04:41
Either we increase the instruction cache size and I guess our friends from ARM, from Intel They are all doing that But what we can do as well is try to be smarter in writing our software and instead of executing the full data processing, the full code for every packet and coming back again
05:00
What we can do is have a small portions of code that are applied to a bunch of packets So this bunch of packet is actually called a vector and what we will do is we will execute a limited number of lines of code on this bunch of packet. So with that we make sure that the code is always or 99% of the case the code is in the cache and
05:26
then all the packets will take benefit of that because the penalty to fetch the cache The code from central memory is actually huge when you are processing millions of packets per second So that's really what VPP is about. So that's secret source number one. Secret source number two is okay
05:44
Let's assume we are good with our cache for the instruction We need we now need to also to make sure that when we have we are fetching data to process the packets, the data here as well. So in order to do that we We will process packets with quad loops or dual loops and I will show you how
06:06
so this is a very slow animated slide, so I hate that So you have your vector of packet which arrives from your NIC card, right? Or any other thing could be through DPDK if you have a VM that can come from
06:23
a V-Earth user interface that can come from regular Linux kernel and this vector of packet will arrive here and basically we'll process this vector of packet and Because we always have the we always process a vector. So let's assume we are in this node called
06:42
IP v63, right? When packet zero is being processed Probably the code won't be in the cache because before that CPU was executing IPv6 lookup and before that he was executing IPv6 input so probably when packet zero will be processed
07:00
There is a very little probability to have this code of IPv6 rewrite in the instruction cache But now the beauty of this model is that for packet one, two and three They will all have the benefit of the cache warm-up which was done for packet zero So that is really
07:22
one thing which is fundamental in VPP So if processing this graph is slow What that means is when we will then come back to the AF packet input or to the DPDK input We'll probably have a lot of packet waiting to be processed. So what we do in VPP is we tend to measure
07:42
What is the vector size so if we have small vectors that means that VPP is doing nothing because Because it's processing very small vectors and when VPP is very very active or when there is a lot of packet to be processed Then we'll have we'll have long vectors. So so the size of vectors is typically between one to
08:04
256 and if you have 256 vector size that means that your CPU is under high pressure If it's if you have small packets, it means that your CPU is no is doing nothing So that is that is what we do for
08:22
For to process packets, of course You may have vectors that do not go through the same through the same path in this graph like here You have a packet which is an R packet, of course, this packet will won't go here So we are not saying that all packets have to go in the same path in this in this graph
08:44
Okay, one of the beauty of this model as well is this nodes Actually .so file, right? So it's really easy to extend if one wants to add nodes to do some special processing I don't know that maybe for for a curls that can be for whatever you want
09:02
It's really easy to extend you just have to write a node here. You do not modify the rest of the code So you can localize of your specific plug-in that can be open source or even closed source You do whatever you want with that you you have you have your plugins and you can extend this graph at runtime Without having to recompile VPP. So that is something extremely powerful and
09:30
You can think about hardware acceleration like we see people having knick cards, which are able or accelerators Which are able to do crypto for instance in that case or that can do the first, you know
09:43
levels of processing in this graph So that so of course vs chooser and AF packet input these are software nodes But nothing forbids you having cards which will do some level of processing and inject packet later In this in this processing graph in VPP
10:02
Okay That I will skip numbers When I when I say fast, what what do I mean so just few graphs and and I'm sorry This is a bit of the version and we can't read anything resolution is not good enough. Can you read something? No, it's
10:25
It's it's actually very hard. So trust me. That doesn't work. I Can we zoom in And I will show a live demo because slides are good, but they more are better
10:42
So after that, I prepare kind of a demo risky demo, but we'll see What is important here is I did a test with ipv4 and test with ipv6 By the way, we have with VPP. We have also a testing infrastructure, which is all public You can it's called see seat. It's in the cloud. You can connect to it. It's under the Linux Foundation and you can
11:05
Run the test read the measurements all that is public. So We is what is interesting here is So we have ipv4 and we have ipv6 and what we did is we increase number of calls, right?
11:21
To do routing what routing means it. It means three operations. Basically one is receiving packet Two is taking your routing decision and three is forwarding the packet, right? So with two calls we do like twelve forty forty Twenty twenty four million packet per second. Whatever is packet size, right?
11:43
So that is really good and and and one of the beauty of this model is that when you are adding calls Yeah, you have a linear increase of the performance with a number of calls That's extremely powerful because we did this test from two to twenty to thirty six calls and we see this line your increase
12:00
Thanks to this cache effect that we are taking so so that is really good This is with one million ipv4 entries and we have exactly same effect with ipv6 Where we have we are increasing number of calls and we have the same linear increase, right Here, it's 24 million packet per second on two calls, which means 12 million packets per second per calls
12:24
You seem a bit sleep. I Mean wake up 12 millions packet per second, right? Thank you 12 million packet per second per call while receiving Routing and and then forwarding if you do the math, that's not a lot of cycles per packet, right?
12:46
So that's what we do in this test and then we did the same for It sounds like some people are still doing switching in this world. So you have layer 2 processing here and
13:01
with with layer 2 processing this is a 20 million packet per second per call with the same linear increase right so that is pretty cool and Now it's enough talking and forgive me if that doesn't work because I just did this demo before Here it is
13:21
so What I did here is I have I have two containers with an ipv4 address each ipv4, sorry about that and And what I did is I put vpp to connect those two guys and I will run an ipf client and an ipf server
13:41
Between these guys, right? So and then please help me praying the users the gods of demo We'll see whether that will work So where where I'm lost Okay, so ipf to regular Linux containers running ipf
14:01
It's it's this this is not a bare metal servers. It's actually this is a VM, right? So I have two containers in a VM Okay, running in VMware if I remember correctly and and these two containers can do with with a regular TCP connection with vpp in between like
14:22
30 37 gigabit per 36 37 gigabit per second So that's cool, but I told you before what is actually important is what is the vector size for that, right? So what is let's have a look let's have a look at vector size show so we have a magic command for that
14:45
So vector size So these are the nodes in the graph so you you remember the graph in the graph we have the we have all these nodes and vector size
15:01
is thank you is six dot 93rd right for these tabs your output and and And for the TX as well What that means is what that means is the way you have to understand that is if we come back to this Presentation
15:21
Which was here When the vector when the vector of packet arrives we typically process six packets here, right? That's the average we do in order to process 40 gigabit per second in a regular Linux kernel, which is fun is that the bottleneck so that and so so
15:46
When when we are only processing six packets, that means vpp is not the bottleneck here What that mean is let's have a look with a bottleneck then so let me run again this hyper and
16:05
So, I don't know if you are familiar with h-top but the right is kernel processing The green is vpp vpp is in polling mode. So it takes 100 percent CPU when it has to work, right? So in red, this is a TCP stack. So the TCP stack is actually the bottleneck here and
16:23
If I stop my traffic, of course now everything is at zero vpp Does have this call so called adaptive mode when there is traffic pressure it starts pulling the packets, right? So it's an active loop
16:43
but when it when it When there is no longer a lot of traffic pressures and it stops doing that to avoid burning one core So that is what is vpp but this presentation was supposed to be About networking vpp and I only have ten minutes So I will what I will do now is I will switch to the other presentation
17:03
Explaining you how that that can be integrated into open stack Because all that is useless if that is not integrated somewhere Okay
17:23
Okay, so networking vpp is an ml2 driver for open stack It was primarily designed to support Nfv right at the beginning. What are the open stack features? We support with vpp vlan vxlan VM connectivity is only with vios user interface. Otherwise, that's that's too slow
17:43
We support many security features including regular open stack stuff such as security groups, but very advanced stuff such as And we do layer 2 and layer 3 with a chair and all those stuffs so
18:05
Networking vpp has this architecture It's etc. Decentric what that means is when you want when when neutron have to create a port instead of communicating directly with With the compute nodes. What it does is it will put that in etc. D and then
18:24
The The compute nodes will be weaker. We will wake up and we'll take into account that stuff That is really cool because a problem with open stack in such a distributed system is you don't know when a failure will arrive But the failure will arrive at some point and what we are doing here is we are making sure that when
18:46
Let's assume neutron wants to create a port on this VM Okay on this on this compute node if this compute node is dead for some reason Sorry, booting or whatever you want when it will restart. It will actually fetch the states from etc. D instead of readings at them from
19:03
Neutron, so it's kind of intent based networking if you will because neutron is writing its intent the desired states in in etc. D and then the compute nodes are actually reading these those desired states and If the agent, so let me show you what that means
19:24
So typically creating a port will go through these several steps So first there is neutron asking to create a port then this is stored in etc. D later That will be understood by the agent and then the agent will send the feedback and will notify neutron so that the VM can be
19:42
created If for some reasons the agent crash what it will do when it when it will restart it It will do what we call state reconciliation What that means is every object within VPP comes with a unique ID and when we restart this agent We'll actually fetch the state from from etc. D state and fetch the states from
20:03
from VPP and will Reconciliate the D Will do a diff between those guys and and and remove states which are useless now and include new states which are required So that is something interesting other interesting thing is it's very much at CD centric, which means that
20:21
If for some reasons etc. D fails There is another one which can end over there are many many cool features with with networking VPP One of them is now we have this redundancy between layer 3 routers
20:42
Based on keep a life D so that so if one dies then the other one can end over so that's also extremely useful in production Security Security we have a whole based access control So when only few compute nodes the compute nodes comes with rights and there you cannot write and read anything in etcd
21:06
So basically if there is a malicious compute node for some reasons Then you are sure that the database cannot be overwritten by someone who doesn't have the rights So that is another cool feature We have these JSON web tokens that can make sure that
21:24
The states which are written in the database have been written by the right compute node. So we have all that right? So I will just go to the roadmap slide We are now have a lot of features the next thing we need to work on is this villain aware VMs that we don't have and
21:43
Have a better app ending for for v6. So these are really the remaining features We have now and you are more than welcome to contribute or to test it if you want. I Realize I may no longer have time. So do you do we have time for questions? So Okay
22:05
five minutes To until the start of the next presentation before questions I just like to for those who aren't on the mailing list. We're having a gathering. There's some space in the man can piece cafe later this evening from 730, so you're all well
22:24
Some of you are welcome to join us come along if there's room there's room and if there isn't room Then then it's it's also the center of the center of the center of part of town So and so it's a big enough place. We should be able to fit a good number of us in With that
22:40
VPP used to connect containers or for open stack. I know it was a lot of things being showed in this Presentation there is there is another VPP presentation later, which will go into more detail I think in the in the higher level technical details of VPP Anybody have any questions you shout it out and John can you?
23:19
The question just for the recording question was does the agent of VPP replace replace the
23:24
agent of neutron on the compute nodes So there is an I Mean there is not an agent for for a neutron in general people
23:41
Are you people may you use with the agent which which is on the compute node? Will depend on the virtual switch you will use right? So if you are of yes, you will get a neutron agents if you are using You know Linux bridge you will not use another Agent so we have written an agent which is running on the compute node and that will drive VPP
24:03
So yes the agent On the compute node replaces this agent because this agent knows how to talk to VPP and knows how to talk to etcd. I Agent it is absolutely honey comb is is a net confident agent and here it's an etcd thing, right?
24:24
So it's this one is written in Python You said VPP one must be on top of DPDK. I know there's an option to run on top of ODP Please can you spend the benefit? sure, so in the past in the past VPP was tightly coupled with DPDK and during the last version what we did is trying to have
24:46
DPDK being a plug-in an important plug-in, but but just a plug-in beside others So there are many reasons why not having not using DPDK So for instance in the demo I was showing before I was connecting regular containers, which doesn't work with DPDK. So
25:03
So on ODP right now I don't know to be honest with you because I have not done a lot of tests a lot of tests with it But I know people who are using with it working with it So I guess there may be some better performance with some new cards, but I'm not too sure about that So I don't want to answer so last question
25:23
Anybody else have a question? I'll go beyond the usual suspects. Yeah, so the question was can use a VPP agent along with the Open stack open stack enables this option to have hierarchical driver
25:46
So I I don't see good reasons why that wouldn't work But I don't really see the benefit of doing such a thing, right? So in theory, yes But why would you do that? I mean if you use one for water, it's probably for good reasons
26:01
So I don't see why you would you would have those two guys Was that alongside just two for again for the recording that the VPP agent alongside the OBSM L2 Okay or another custom agent I mean perhaps it looks to me a bit like a Frankenstein solution, but
26:24
But but why not? Why not? Okay, that's that's a Word of time. So thank you very much John. Thank you so My next speakers I will be meeting them for the first time on the stage. So
26:40
Java