Merken

Design and Implementation of the MoonGen Packet Generator

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Beta
Erkannte Entitäten
Sprachtranskript
it is away a program that next also don't you all every is the more than all you have to be design of hi I'm I'm going to talk about my package an item which is written mostly in you will and it does someone quite amazing things thanks to ruin if you want to learn more about the package and models of a 2nd talk tomorrow and EST and f from this talk focuses on kind of unique challenges you had when using you off for our we high performance of missing and also scums assumption and which is threatening optimizing stuff integrating the just to name a few of them if you want to a lot more about how to use it or why you would want to on then there's gender from 2 tomorrow a new this the 2nd talk so what a packet generator well there's is a packet
generator is a large box you can buy them from some part an us usually really really expensive and this boxes tens of thousand of yours up to and above 100 thousand euros well what it does test networks you put in your mind he's in a cable and then it sends out lots of packets and to buy lot of packets I mean you can put their in to pretend you give it 100 Gigabit came is in the same and it completely phased in this small packets which is quite challenging to do and so obviously and I can't animosity you do have some findings but not enough to spend 100 thousand euros in a packet generator so we're looking into how can we do the same thing in software and have been using quite some quite a lot of packet generators uh and just and gave an ESC slow or unusable or both slow unusable so I started working can the due respect our elements yeah we can and some are designed Newington which this a packet generator
written almost entirely new world wide you are well because it's fast and by writing everything in you I get that really really huge flexibility if you use that to because a packet generator and softer there's side of a complicated it's configuration that it makes no sense or is it hard to adjust to your specific use case or hard to extend to Newport and yeah so I really didn't want that so what I wanted was the main goal of most flexibility and speed so I opted to would just generate all the packets in you are called that means if you want to send a specific type of packet uh this Mondrian he white coat that generates the packet and send it out in real time it it's obviously cheaper than a hot box from the previous slide um you can get a software system with some tangible parts for ourselves in your or something attended habitat the accounts and or days in the range of a few hundred years so tried affordable compact to heart the production of this so that it's obviously more flexible than not the box because you can easily extended to new particles you can just implement a particle in new are and then on it and it does whatever you wanted to do so how I mean the whole area achieving their flexibility and the speed and both obviously obviously wanted then the firm that did decay you decay is a framework by into which as well as the simplified version that's an optimized set of 2 wireless and because you can't just use the normal operating system interface if you want to send test traffic at rates of millions of packets per 2nd it just doesn't work if you need to do to manage the scores so the decay from market and really nice then we do explicitly call support in all of our API everything you write this you call them and other come to that in a 2nd because doing when deciding and was not something that you usually do and flexibility as already mentioned all packets that are sent out a crafted in real time in a user-defined was good so you write a script instead of a configuration file which works really well and then there another key feature is precise time stamping on the data by misusing some hot future sometimes sums of a commodity Nixon and we want descended drive that set some magic registers of today like magic values and then you get precise latency measurements which is something that on analysis to pick a generative state to do and yeah so how do we do which is setting so you probably know what some of you probably know some memories fluid etcetera came to the what is telling some of you and the Gruber them once and the idea has some other but they well not that moment maintained most of them mandates most of them also do not really iid mapped to our problem domain that we have and so what we did the simple bortfall solution we just starting which appear independently which legislature machines so basically in which a blue states and each 1 and it's been 2 hours approach that and a separate CPU core and then you have modified and of course the obvious disadvantage that's you know have no shared state between the newer forms but it turns out that we for our use case don't actually need shared state on because we are generating test traffic and that typically different types of flows let's just say you're testing some boxes some quality of service feature then you send 1 that traffic that is to be uh a high-priority traffic when low-priority traffic and that's something that's completely independent of each of them can be tested independently so you have to set some and the basic due to the same thing just this different and traffic types and that's actually what then 2nd thing as that and and the and that's a cut if you buy an actor can't they actually natively in which he called where that means and you can use them as multiple independent cats and what the letter cuts to of course traffic you sent to an advert it just makes it all together and receiving traffic you can configure the task and tightening at every that matches 3rd does goes to this and q uh but it's got all you can now this through the traffic based on a hash function at the cart and then you can use each of these cues and at the Carter independent from each other we just map 1 Q to 1 which at the and to 1 CPU core which works really well the of course the sound that communication between the sites but that's only for priority data basically like such report their statistics this throughput they're seeing their latencies emotion at the moment and but can be slow so are and so we wrote some nonentity functions that just basically do a simple message passing API used uh and excellent ivory serpent folders serialization and so we can pass around to arbitrary uh you objects of course it's it's kind of so when I say so I mean and order of a hundred thousand messages per 2nd depending on the complexity of the object the socialization is we read but in anarchy and but that's OK for statistics it wouldn't be OK for packets we are aiming for millions of packets per 2nd what does it all look like well we
have to lay out and about and we have to hunt them which is with the fitting a Alabbas with accused then we have to duplicate of by we which Anderson low abstraction interface and what it does the 2 of us maybe for configuration of material and API is for well pushing packets or receiving packets from the FBI then its cause on Genesis simplified just a wrapper for the PDK where it's not just a wrapper because the of a lot of utility functions and all MIT setting and so on and on the very top this what user-script and the special part about this is that the user actually likes to antiallergic of data packet generator is of course comes example scripts to do get on to whatever and but the idea is to take 1 of these examples scripts which matches or kind of matches the test you're trying to do and then you modified to your needs or whatever or copy and paste from different example scripts to get out of reuse might use and of course all use of the modules are implemented in a in some utility libraries like timestamping of protocols and so on so what does it look like
an for example if the want to generate some some UDP packets then you'd like code like this which I think is very nice and then a configuration because it's easily extendable and all other package in R and as I've used at some time I had to touch the source code just to get them to do what I wanted him to do and we have 1 of the reasons why I just said OK and hashes source code of honor patronizes used anyways so why not let like the source code and OK what this code is doing it's quite simple as trusting as we create their memory poor memory pool is just an area of about fast that uh can be efficiently passed to the carts we hadn't initialize and callback function all about fossils some default value in this case as some default UDP various um you can pass an the argument of the function as you see later and Joe but here for space reasons just some different values and then be sent them out in the may below the menu does just the 9 7 at an exam us and this size this 60 year is not actually a number of but this size of each packet reascending sixties the minimum size that's the only thing that the network and then what you do me a looping aboard buffers to just look at that and then we can just say OK and UDP packet they get UDP packet function this is just a fancy way to do a task to some the structure which Denison UK packets and then you can compare uh access the packet likable some people get and tailored like you want to randomized sozzled adverse and you also want to randomize the source bold and then we just on the low-level retraining checksum offloading which just set some flex and then an attic Archivist on attack sums and then we can know and send out about Q and as you can see this function you can now as 1 of the in the previous slides 1 of this last could say script
in the top flight and you could start this function would appear times in multiple independent of the
ends and pass it different types of parameters in the simplest case different cues and gender just generated traffic natively from which appeared course at the same time but you could also pass on a par to it's uh like they range to randomize of exports to use or anything else you can check out or I get help repository model for example scripts of various use cases so how this perspective data stuff looking 1 important challenge for a picket generative this that so they are written for testing staff and from testing you often test new protocols albeit protocols only ach cases so that's there a lot of configurations actually fatal are not implemented latest uh put codes so we have almost endless number of different protocols and then in literally and this number of uh combinations of just put course because you can stick time into town and what about um yeah and what we need to have in a packet generator especially as via the building all packets in real-time STS and out we need efficient access to all these protocol had us uh forms script called and the 1 piece and these excesses of the underlying packet we wanted to have a new light data structure so we don't want to copy the packet at any time you want to build it at the bit to hold buff in new and and then just pass it to the diamonds and belt so what we did is a kind of dynamic approach and we have described magic function which actually isn't in space but here and so what it does it takes a few put across the particles are defined in modular so if you implement and you put a call you at your module here and there but the guy basically represents the Halstead protocol and correct pack it just takes multiple particles and this text and again and this case we have in the Senate animals they are then we have an IP and above that and if UDP then we have the exam the exam is a tunneling protocol so we have an ID Senate 10 and and NIPS in there and that's and that's was the data the it's just because you need to rename them as you can see in there and next line we can access the packet after we can use the uncertainty a cast which tend to buy into our cast bust was of a packet and then we can do all you try magic here the yeah did already set of functions are defined by the protocols and they had stuff like that engine shopping because usually in by donors not you host by on and so on and it by their and class to discredit dynamically this is it's true in the path and the those thanks to a larger and f of i and its really have benchmark that's even faster than this year package and out of the the and basically the main reason for IP faster is that these packet generators need I you switch statement and in the domain you basically the sizes this config at node and we have to do this and etc. was this logic approach just you are a really small menu approach does just actions you you really need and yeah these these scores to settle for them up you obviously would use this string chasing here there uh the function also accepts as a pre past number and you can do well past IP address and then you get in a park object that you can pass to stuff like this and yeah that's really fast user service then the memory layout is defined by the data see that uh such so we can just send out the packets immediately after the crater that's an old edition of the copies that necessary that just something else to look at on our stuff Bernhard and assignment and like the 8 API that stuff like this and a more complicated this the OK to give a quick summary and what is going on use user-scripts instead of summing complex configuration then no object fast really fast kind of I mean fast and saying what and 50 million packets per 2nd it appears to be require have successfully tested this was up to 100 Gigabit Ethernet and at the marks on his high speeds of course you need and 10 CPU cost to do that but yeah and this is something on a package and it has failed to do and done why despite this high-speed we the script called for each single packet we send out just a simple calculation during 15 million packets per 2nd year it was minimum-size packets and you're secure let's just say has to be ousted mean-shift 200 cycles per packet and on this includes the as you are cause to work and modify half years and cost to random or whatever we can send other completely random packet you know what we can touch on header fields so and send out a packet and use less than 200 clock icassp packet I think that's quite impressive and thanks to larger and yet we make heavy use of the new edge testify and and sea level reflected the decay for a lot of his stuff the um for example in there day the older drivers use either duplicated lemmas and deer and just digit um Afify oppose if you want to see but 1 could to lie the time and you are and I don't know if famous use graph to snap such projects they're doing sector that they are like India drive us into of adjusting this kind of crazy and I was thinking about using and gives you a driver and them but then I decided for the decay because it's slightly faster and has better medical support so that's basically at the 1 of the 4 questions and you can check out the
missing on get up thank you you have do start so what is it
yes this is the the this is the hello and that's the idea of what used to be and took a couple of more than what pretty well Crytek isn't actually misnamed and it should actually be called create particles stick rep or something and but yes you can and apply it on received packets and receiving packets also on the for the future but not something we usually do and in the end this is just a few data stocked and this is just a wraparound done as a vital task so yes and they all always some and as also function which looks at an incoming packet and their determines protocol at this and then automatically best bits for you on gives you and whatever you need and I think at some example codes and that that's receiving packets and sink and quality of service testing and grew up and that's what you want to look at it to suspected some checks the UDP port to work and compare different flows yes you see for a virus to learn the right you look so I'll be doing all of the time or note and the current is too slow for high speed internet working and the decay he really believe have just maps the whole BCI and relevant the PCI devices and it's and dandy may space into a userspace process so and what also doing is there some parts of the code which directly access PCI registers from local the role of art the form yes yeah almost all attend you the counter supported the debit cards and little a bit more tricky because there are often a and annoying and i is sometimes have to cut some people don't want to test this year I often have to tell please by the standard of because it's better even for a gigabit speeds and other than that adds up to the DTK library with their support when you are on a and it's not here is it's not usually something you want to do these drivers are what I mean you share a lot of code to stand and users pay and with the common and space so I'm companions but I think that India but I was under the age of 20 thousand lines of code in the UK yeah think of all of the time in that after 4 thank you thank you
Informationsmodellierung
Geschlecht <Mathematik>
Eindeutigkeit
Schätzung
Optimierung
Computeranimation
Implementierung
Gewichtete Summe
Momentenproblem
Versionsverwaltung
Oval
Element <Mathematik>
Komplex <Algebra>
Metropolitan area network
Skript <Programm>
Schnitt <Graphentheorie>
Dienstgüte
Einflussgröße
Schnittstelle
Umwandlungsenthalpie
Softwaretest
Lineares Funktional
Statistik
Datennetz
Benutzerfreundlichkeit
Biprodukt
Bitrate
Rechenschieber
Software
Menge
Zellularer Automat
Festspeicher
Ordnung <Mathematik>
Computerunterstützte Übersetzung
Message-Passing
Aggregatzustand
Telekommunikation
Subtraktion
Web Site
Multiplikation
Quader
Fluid
Entscheidungsmodell
Subnormaler Operator
Zentraleinheit
Task
Virtuelle Maschine
Bildschirmmaske
Domain-Name
Spannweite <Stochastik>
Software
Datentyp
Hash-Algorithmus
Konfigurationsraum
Gammafunktion
Implementierung
Analysis
Trennungsaxiom
Zehn
Stochastische Abhängigkeit
Physikalisches System
Elektronische Publikation
Datenfluss
Objekt <Kategorie>
Echtzeitsystem
Flächeninhalt
Mereologie
Modem
Serielle Schnittstelle
Speicherabzug
Partikelsystem
Verkehrsinformation
Subtraktion
Gewichtete Summe
Extrempunkt
Entscheidungsmodell
Code
Raum-Zeit
Task
Puffer <Netzplantechnik>
Last
Wrapper <Programmierung>
Programmbibliothek
Skript <Programm>
Datenstruktur
Default
Konfigurationsraum
Implementierung
Schnittstelle
Softwaretest
Parametersystem
Lineares Funktional
Datennetz
Protokoll <Datenverarbeitungssystem>
Physikalischer Effekt
Abstraktionsebene
Softwarewerkzeug
Quellcode
Modul
Rechenschieber
Flächeninhalt
Menge
Festspeicher
Mereologie
Fitnessfunktion
Bit
VHDSL
Raum-Zeit
Last
Lemma <Logik>
Skript <Programm>
E-Mail
Gerade
Inklusion <Mathematik>
Softwaretest
Parametersystem
Lineares Funktional
Dokumentenserver
Physikalischer Effekt
Transitionssystem
Computersicherheit
Gebäude <Mathematik>
Systemaufruf
Rechnen
Auswahlverfahren
Rhombus <Mathematik>
Dienst <Informatik>
Datenfeld
Festspeicher
Digitalisierer
Projektive Ebene
Zeichenkette
Subtraktion
Stab
Klasse <Mathematik>
Gruppenoperation
Schaltnetz
Zahlenbereich
E-Mail
Zentraleinheit
Mathematische Logik
Netzadresse
Informationsmodellierung
Knotenmenge
Domain-Name
Bildschirmmaske
Perspektive
Datentyp
Datenstruktur
Konfigurationsraum
Hilfesystem
Implementierung
Protokoll <Datenverarbeitungssystem>
Graph
Diskretes System
Stochastische Abhängigkeit
Modul
Objekt <Kategorie>
Druckertreiber
Echtzeitsystem
Dreiecksfreier Graph
Codierung
Partikelsystem
Bit
Computervirus
Prozess <Physik>
VHDSL
E-Mail
ROM <Informatik>
Code
Raum-Zeit
Gerichteter Graph
Internetworking
Task
Bildschirmmaske
Programmbibliothek
Dienstgüte
Gerade
Implementierung
Softwaretest
Lineares Funktional
Protokoll <Datenverarbeitungssystem>
Wiederkehrender Zustand
Datenfluss
Chipkarte
Mapping <Computergraphik>
Druckertreiber
Rechter Winkel
Mereologie
Datenerfassung
Prozessfähigkeit <Qualitätsmanagement>
Codierung
Partikelsystem
Standardabweichung
Computeranimation

Metadaten

Formale Metadaten

Titel Design and Implementation of the MoonGen Packet Generator
Serientitel FOSDEM 2016
Teil 92
Anzahl der Teile 110
Autor Emmerich, Paul
Lizenz CC-Namensnennung 2.0 Belgien:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
DOI 10.5446/31007
Herausgeber FOSDEM VZW
Erscheinungsjahr 2016
Sprache Englisch

Inhaltliche Metadaten

Fachgebiet Informatik

Ähnliche Filme

Loading...