Safe and Secure Drivers in High-Level Languages

Video thumbnail (Frame 0) Video thumbnail (Frame 1430) Video thumbnail (Frame 2711) Video thumbnail (Frame 4608) Video thumbnail (Frame 6466) Video thumbnail (Frame 7794) Video thumbnail (Frame 9555) Video thumbnail (Frame 10774) Video thumbnail (Frame 12247) Video thumbnail (Frame 14676) Video thumbnail (Frame 19406) Video thumbnail (Frame 21745) Video thumbnail (Frame 23115) Video thumbnail (Frame 26170) Video thumbnail (Frame 27389) Video thumbnail (Frame 29570) Video thumbnail (Frame 32207) Video thumbnail (Frame 34142) Video thumbnail (Frame 36305) Video thumbnail (Frame 43268) Video thumbnail (Frame 48490) Video thumbnail (Frame 52191) Video thumbnail (Frame 55817) Video thumbnail (Frame 59666) Video thumbnail (Frame 62337) Video thumbnail (Frame 67021) Video thumbnail (Frame 73549) Video thumbnail (Frame 75418) Video thumbnail (Frame 80539) Video thumbnail (Frame 82797) Video thumbnail (Frame 84439) Video thumbnail (Frame 86458) Video thumbnail (Frame 90427) Video thumbnail (Frame 91970)
Video in TIB AV-Portal: Safe and Secure Drivers in High-Level Languages

Formal Metadata

Safe and Secure Drivers in High-Level Languages
How to write PCIe drivers in Rust, go, C#, Swift, Haskell, and OCaml
Title of Series
CC Attribution 4.0 International:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
Drivers are usually written in C for historical reasons, this can be bad if you want your driver to be safe and secure. We show that it is possible to write low-level drivers for PCIe devices in modern high-level languages. We are working on super-fast user space network drivers for the Intel 82599ES (ixgbe) 10 Gbit/s NICs in different high-level languages. We've got fully working implementations in Rust, C#, go, OCaml, Haskell, and Swift. All of them are written from scratch and require no kernel code. Check out our GitHub page with links to all implementations, performance measurements, and publications for further reading.
Keywords Resilience

Related Material

The following resource is accompanying material for the video
Video is cited by the following resource
Roundness (object) Exterior algebra Software developer Device driver High-level programming language Energy level Device driver Musical ensemble Writing
Slide rule Device driver Service (economics) Computer network Neuroinformatik Right angle Streaming media YouTube Row (database)
Intel Observational study Code Line (geometry) Multiplication sign Device driver Student's t-test Formal language Measurement Device driver Differenz <Mathematik> Virtual reality Personal digital assistant Spacetime James Waddell Alexander II Physical system Area Boss Corporation Service (economics) Slide rule Projective plane Code Computer network Neuroinformatik Student's t-test Line (geometry) Device driver Hypothesis Process (computing) Software Personal digital assistant Device driver System programming Spacetime
Code Code Device driver Bit Student's t-test Device driver Formal language Fraction (mathematics) Device driver Expected value Fraction (mathematics) Personal digital assistant Device driver System programming Macro (computer science) Physical system
Inheritance (object-oriented programming) Hoax Open source Inheritance (object-oriented programming) Code Device driver Bit Device driver Device driver Macro (computer science) Kernel (computing) Different (Kate Ryan album) Kernel (computing) Blog Device driver Abstraction Abstraction
Point (geometry) Programming language Proxy server Code Code Formal language Software bug Formal language Category of being Device driver Kernel (computing) Term (mathematics) Different (Kate Ryan album) Kernel (computing) Figurate number Information security Information security Operating system Buffer overflow
Programming language Table (information) Observational study Code Device driver Stack (abstract data type) Bound state Device driver Mereology Formal language Formal language Software bug Device driver Kernel (computing) Semiconductor memory Logic Personal digital assistant Kernel (computing) Device driver Logic Cuboid Information security
Axiom of choice Just-in-Time-Compiler Observational study Code Device driver Compiler Online help Streaming media Student's t-test Formal language Device driver Goodness of fit Speicherbereinigung Spacetime Module (mathematics) Axiom of choice Just-in-Time-Compiler High-level programming language Device driver Formal language Kernel (computing) Integrated development environment Personal digital assistant Device driver Website Resultant Spacetime
Dependent and independent variables Scripting language Java applet Java applet 1 (number) Device driver Computer network Total S.A. Student's t-test Open set Formal language Device driver Different (Kate Ryan album) Universe (mathematics) Energy level Integrated development environment Resultant
Computer program Intel Structural load Direction (geometry) Mereology Information privacy Area Very-high-bit-rate digital subscriber line Semiconductor memory Kernel (computing) Memory management Logic Software framework Physical system Area PCI Express Infinity Bit Flow separation Category of being Message passing Process (computing) Befehlsprozessor Interrupt <Informatik> Speicheradresse Writing Spacetime Firmware Implementation Service (economics) Computer file Open source Device driver PCI Express Student's t-test Black box 2 (number) Device driver Goodness of fit Latent heat Read-only memory Whiteboard Computer hardware Energy level Directed set Spacetime Interrupt <Informatik> Message passing Address space Default (computer science) Server (computing) Plastikkarte Computer network Device driver System call Kernel (computing) Event horizon Software Computer hardware Device driver Family Address space
Game controller Structural load Code Strut PCI Express High-level programming language Electronic mailing list PCI Express Bit Line (geometry) Statistics Open set Device driver Personal digital assistant Kernel (computing) Device driver Spacetime Address space Address space
Table (information) Open source Code Similarity (geometry) Semantics (computer science) Value-added network Formal language Device driver Pointer (computer programming) Read-only memory Semiconductor memory Operator (mathematics) Ring (mathematics) Memory management Queue (abstract data type) Flag Data structure Mathematical optimization Address space Tunis Area Operations research Multiplication Validity (statistics) Interface (computing) Aliasing Web page PCI Express High-level programming language Plastikkarte Bit Control flow Device driver Semantics (computer science) System call Formal language Compiler Similarity (geometry) Pointer (computer programming) Process (computing) Software Ring (mathematics) Device driver Interface (computing) Block (periodic table) Extension (kinesiology)
Implementation Code Multiplication sign Code Student's t-test Formal language Formal language Similarity (geometry) Category of being Device driver Semiconductor memory Device driver Implementation Data structure
Point (geometry) Asynchronous Transfer Mode Implementation Wrapper (data mining) Code Multiplication sign Student's t-test Streaming media Formal language Device driver Pointer (computer programming) Read-only memory Semiconductor memory Flag Implementation Data buffer Just-in-Time-Compiler Code Formal language Compiler Similarity (geometry) Device driver Data structure Window Asynchronous Transfer Mode
Asynchronous Transfer Mode Wrapper (data mining) Code Device driver Student's t-test Bound state XML Formal language Pointer (computer programming) Device driver Goodness of fit Benchmark Read-only memory Semiconductor memory Buffer solution Memory management Speicherbereinigung Arc (geometry) Social class Operations research Forcing (mathematics) Interior (topology) Memory management Vapor Bit Operator overloading Cartesian coordinate system Category of being Pointer (computer programming) Touch typing Device driver Buffer solution output Force Asynchronous Transfer Mode Data buffer
Functional programming Implementation Functional (mathematics) Code Memory management Code System call Declarative programming Formal language Device driver Read-only memory Semiconductor memory Device driver Flag Speicherbereinigung Loop (music) Library (computing)
Reading (process) Functional programming Slide rule Implementation Functional (mathematics) Code Execution unit File format Device driver Student's t-test Mereology Formal language Device driver Type theory Read-only memory Semiconductor memory QR code Speicherbereinigung Physical system Mapping File format Memory management Code Bit Category of being Type theory Buffer solution POKE Physical system Form (programming) Speicheradresse Flag
Computer program Run time (program lifecycle phase) Code Multiplication sign Compiler Bound state Area Formal language Pointer (computer programming) Semiconductor memory Kernel (computing) Memory management Distributed computing Library (computing) Physical system Area Presentation of a group Structural load Physicalism Virtualization Formal language Type theory Category of being System programming Physical system Point (geometry) Functional (mathematics) Vapor barrier Calculation Device driver Approximation Rule of inference Device driver Fluid Read-only memory Googol Program slicing Speicherbereinigung Computer multitasking Integer Data buffer Address space Metropolitan area network Data type Rule of inference Run time (program lifecycle phase) Plastikkarte Device driver Computer programming Compiler Pointer (computer programming) Kernel (computing) Software Personal digital assistant Function (mathematics) Calculation Device driver Atomic number Vapor barrier Operating system Library (computing) Speicherbereinigung Address space
Standard deviation Slide rule Computer program Asynchronous Transfer Mode Functional (mathematics) Implementation Greatest element Run time (program lifecycle phase) Overhead (computing) Link (knot theory) Code Multiplication sign Device driver Set (mathematics) Bound state Rule of inference Variable (mathematics) Formal language Number Device driver Read-only memory Semiconductor memory Memory management Core dump Uniqueness quantification Speicherbereinigung Library (computing) Address space Physical system Rule of inference Mapping Concurrency (computer science) Interface (computing) Uniqueness quantification Content (media) Code Formal language Pointer (computer programming) Software Doubling the cube Function (mathematics) Device driver System programming Website Moving average Freeware Physical system Data buffer
Stapeldatei Server (computing) Overhead (computing) Plotter Device driver Water vapor Data transmission 2 (number) Formal language Device driver Goodness of fit Befehlsprozessor Software testing Stapeldatei Scaling (geometry) Graph (mathematics) PCI Express Cartesian coordinate system Measurement Connected space Testbed Cache (computing) Befehlsprozessor Device driver Resultant
Trail Functional (mathematics) Mobile app System call Graph (mathematics) Multiplication sign Stack (abstract data type) Semantics (computer science) 2 (number) Formal language Device driver Profil (magazine) Semiconductor memory Memory management Computer hardware Speicherbereinigung Pairwise comparison Cumulative distribution function Pairwise comparison Just-in-Time-Compiler Graph (mathematics) Normal distribution Memory management Counting Bit Cartesian coordinate system System call Data management Software Function (mathematics) Device driver Compilation album Object (grammar) Resultant Speicherbereinigung
Point (geometry) Empennage Histogram Just-in-Time-Compiler Cumulative distribution function Multiplication sign Mereology Formal language Device driver Average Speicherbereinigung Information security Descriptive statistics Cumulative distribution function Just-in-Time-Compiler Graph (mathematics) Graph (mathematics) Median Bit Line (geometry) Cartesian coordinate system Measurement Performance appraisal Process (computing) Personal digital assistant Device driver Compilation album Library (computing) Speicherbereinigung
Computer program Implementation Functional (mathematics) Game controller Server (computing) Code Information overload View (database) Virtual machine Device driver Device driver Goodness of fit Component-based software engineering Root Read-only memory Semiconductor memory Operator (mathematics) Computer hardware File system Energy level Spacetime Information security Address space Physical system Area Game controller Default (computer science) Web page PCI Express Memory management Physicalism Virtualization Device driver Cartesian coordinate system Measurement Message passing Root Kernel (computing) Process (computing) Befehlsprozessor Personal digital assistant Buffer solution Device driver System programming Routing Spacetime
Meta element Multiplication sign Stack (abstract data type) Perspective (visual) Diffuser (automotive) Formal language Measurement Timestamp Mathematics Optics Semiconductor memory Kernel (computing) Memory management Repository (publishing) Resource allocation Position operator Physical system Decision tree learning Link (knot theory) Software developer Keyboard shortcut Measurement Root Configuration space Right angle Cycle (graph theory) Metric system Fiber (mathematics) Physical system Resultant Spacetime Ocean current Game controller Computer file Open source Device driver Limit (category theory) Student's t-test 2 (number) Device driver Read-only memory Computer hardware Energy level Spacetime Software testing Data buffer Address space Diffuser (automotive) Code Plastikkarte Computer network Device driver Timestamp Kernel (computing) Software Personal digital assistant Computer hardware Device driver POKE Routing
Meta element Implementation Meta element Link (knot theory) Link (knot theory) Java applet Code Code Device driver Formal language Device driver Software Repository (publishing) Different (Kate Ryan album) Kernel (computing) Device driver Repository (publishing) QR code Energy level Spacetime Exception handling
Point (geometry) Trail Computer program Context awareness Randomization Code Multiplication sign Device driver Hypothesis Formal language Number Revision control Sic Internetworking Different (Kate Ryan album) Semiconductor memory Data structure Series (mathematics) Address space God Physical system Theory of relativity PCI Express Bit Line (geometry) Sign (mathematics) Process (computing) Befehlsprozessor Device driver Interrupt <Informatik> Metric system Resultant Row (database) Spacetime
Pairwise comparison Programming language Multiplication sign Memory management Student's t-test Cartesian coordinate system Flow separation Number Compiler Formal language Revision control Logic Device driver Speicherbereinigung Operating system
Performance appraisal Stapeldatei Process (computing) Software Shader <Informatik> Plastikkarte Sound effect Proper map Formal language Number
Implementation Vapor barrier Plastikkarte Primitive (album) Semantics (computer science) Number Formal language Uniform resource locator Befehlsprozessor Semiconductor memory Order (biology) Device driver Computer hardware Right angle Computing platform
Complex (psychology) Implementation Multiplication sign Direction (geometry) 1 (number) Device driver Semantics (computer science) Number Latent heat Goodness of fit Sic Internetworking Semiconductor memory Single-precision floating-point format Macro (computer science) Stapeldatei Weight PCI Express Sound effect Measurement Performance appraisal Kernel (computing) Software Ring (mathematics) Telecommunication Buffer solution Spacetime
Turing test Programming language Implementation Code Multiplication sign Modal logic Software developer Mereology Formal language Number Pointer (computer programming) Atomic number Semiconductor memory Buffer solution Quicksort Reading (process)
Interactive television Cartesian closed category Musical ensemble Semiconductor memory
[Music] driver development has been dominated by C for many many years and it sadly still is C is inherently flawed and there are better alternatives but those are always fighting against the stigma of being slow and cumbersome to use so today here our speakers are going to tell you that this is in fact not true you can write high level drivers with high level languages and they perform very well here to talk about this are Paul Simon and Sebastian warm round of applause please
thank you yeah and Paul with me uh salmon and Sebastian
and just a few quick notes before we get started I speak quite fast usually so sorry to the translators and sorry to everyone listening if you are watching us on a stream recording on meteor TV CD Ilya is a button to your right where you can reduce the playback speed if you are watching on YouTube you can also reduce the playback speed somewhere also please like and subscribe on YouTube you might have already seen that they have quite a lot of names up on these slides today and these are all the people who somehow contributed to this talk and these are all my students with
me today Shimon and Sebastian who boasted Jesus was me I'm a PhD student at Technical University of Munich and I'm researching performance of software packet processing system and today we are going
specifically to talk about network drivers well why look at Network drivers as a case study well it's obviously our research area so it's the next best
thing to do and also user space network drivers are all directional and user space drivers as fare you can use all the fancy languages I have already talked about user space network drivers here last year so a quick recap of what
I presented here last year is the boss the exe project and the exceed project is a I think I started where I wanted to show off how you can write a user space
driver that is actually readable and understandable and fast at the same time so the goal is is to be used for educational purposes it's around a thousand lines of C code as full of references to datasheet specs and so on and well if you want to know more about that for my talk last year just a quick diff since then we've added support for our bionics and fees now I've a go and set up so you no longer need it we'll have to play around with that you can check it out on github yeah then I float it in see back then well
why would you write a drive and see seems like a kind of obvious question why wouldn't you because most drivers are written in C and if you're going for educational use of the driver then might as well use the language that all the other drivers are in it's also the lowest common denominator of all the systems programming languages meaning everyone should be able to eat C and I also think the C code can be quite beautiful in some cases I can we can we get a quick show of
hands who here thinks they can reach C code so this is way more than expected
it was like basically everyone raising their hands so let's have a look at some C code this is something this is some
actual code from our driver the first edit by a student and when I initially got the pull request for this from my students I was like no we can't at this macro is like the goal was to have readable code you can't act this and then we discussed it a little bit in the end we ended up adding it to the code base because it's really necessary there's no better way to do this than this macro and see who here knows we can immediately recognize the narc or knows what it does I see this I see one hand
to have a few hands okay not bad not bad
does it help if I show you the actual name anyone who who can who recognize this Michael some more hands yeah okay but there's kind of little bit fake
inheritance or a little bit too abstract driver's it's commonly used in drivers and to abstract the different drivers I've Marcos copied from the Linux kernel and I've searched through the Linux kernel sources and found 15,000 users of this marker so it's not at all unusual to have C code like this in your driver so I didn't agree to add it to the code even though probably almost no one can read at the actual code also has a comment to a blog post that explains what is Marco does but it it also chose the problem that maybe a lot of people
think they can read C but then they encounter something like that and suddenly yeah point is it can be ugly and it can not only be ugly in terms of how it looks or feels by programming it can also be ugly when it comes to security properties or security issues
there's a screenshot from CVE details that come no I don't expect you to be able to read all these figures but basically it shows there are security bugs in the Linux kernel is specifically all the Linux kernel bugs found in the last 20 years or so and now you could say well why's the language to blame you can I bet code in any language and what we now would have to do is basically we would have to go through all these bucks and check by the day could be prevented
by using a memory safe or better language and it seems like a lot of work but luckily someone else already did that last year there was a paper by cutler at all who developed a operating system and go and they looked at all code execution backs in thank 2017 in the linux kernel and looked through all of them manually and try to figure out whether they could have been prevented by using a different programming language for 17% of them they were
unsure and because this was kind of ripped back and they couldn't tell whether that could have been prevented for 22 percent of a clear logic parks respect really you would have made that back probably in any language and the language wouldn't have helped but for over 50 percent of the box we are related to memory both use of the free out of bound accesses and so on and these can be prevented by using a better programming language now these are 40
preventable bugs in this case study and we looked at them to figure out how many of these backs were and drive us versus parks in other parts of the kernel because we are specifically looking at drivers well 39 of them were and drivers the other one was in the Bluetooth stack and of the drivers take five come by five driver had certain box yes I know I was shocked box and o'clock on Wi-Fi
driver could have sought now based on these results should you be writing new code in C and 2019 well probably not if you have a choice but you just often don't have a choice if you're adding some kernel code for some reason sure you can write a kernel module and worst but good luck getting it up streams and other languages like good luck writing a kernel module in JavaScript that's probably not going to work and if it's going to work is probably a bad idea now this is why we are looking at user space drivers because they can be written in virtually any language so we are not constrained by any environment here and the question would now be are all the language that equally good choice can I do it in any language should I do it in any language which language should I choose use is a JIT compiler or garbage collector in the driver now I initially wanted to like write one driver in one high-level language to have a case study and then extensively evaluated but then I thought
maybe it would be a better idea to write drivers in all the languages but it turns out I don't speak all the languages but luckily I could recruit the help of a few students and there's a screenshot from my website at a
university where i do like announcement for jesus' and I just added announcements for waiting I talked to I was immersed in go in Java and c-sharp in Haskell and so on and then at first
my my colleagues at the University looked at me in red where's Malik are you serious you realize this the same announcement all over again some of them still can't tell if I'm serious about this but yes I am and I got a lot of response from students I think I talked to a total of 30 students or so who wanted to do one of these teasers and then these two were one of the first ones to talk to me and I try to scare them all away initially I told them all it's going to be really hard you can get in easier teasers probably not a good idea you need to know a lot of low level stuff and so I scared away 20 of the 30 I talked to in the end we did 10 pieces and yes quite nice result so far if you are still ongoing but I hope we will have 10 different languages for drivers soonish I think finished depending on how you count finished 6 or 7 and yeah also turns out giving a talk here is a really nice way to recruit students because a lot of my students mentioned that they saw my talk and contacted me afterwards for teasers now what did I tell my students how to how to go I had
to to write a driver well I explained to them basically the very basics of how to write a userspace driver how to talk to a modern PC I'd first devised and what you need to do basically there are three different ways to talk to modern PCI Express devices we are ignoring a few legacy things here if you like we had old io instructions not on here sorry first way to talk to a device directly simplest way is memory mapped i/o memory mapped i/o just a magic memory area that is mapped to does met into your process and directly goes through to the device and if you read white that memory the device gets the request and can reply to it that's usually used to expose device for just us on Linux you can just end up a
magic file via the UI or framework and then you have access to that from your user space program second way is kind of how the device talks to you or how the device talks to the rest of the system is direct memory access that is just aware how the device can read and write arbitrary memory locations and for user space drivers you just have to figure out where our memory address space is mapped physically then we can tell the device to write something there and then it will just show up in our process without us having to do anything in there in the kernel and the short for interrupt we will not be using interrupts here because we don't need them for a high-speed network driver but let's ayat it on here because sometimes people say well you can't use interrupts from a user space driver that is incorrect you can use the V if I or subsystem that has full support for interrupt but we won't be using them here now what did I tell my students about how they should go to write their drivers well basically they should just remove the current kernel driver do the magic MF call on the under light metric file then figure out the physical addresses and then just watch the driver it's it's really easy so then we have a lot of hardware at the University and I gave them all access to service there's ten gigabit network cards of the infinite likes to be family this is a really common network card that you will find on a lot of sellers like default go to ten gigabit network card on any service you will often find it onboard or embedded in some of the CPUs even the nice part about this has a very very nice data sheet publicly available that documents basically everything fun fact we found it easier to program against this Hardware black box was a good public data sheet and we found that to implemented that I expect which is the open specification has several open source implementations but yeah um then the this network cards a little bit all at ten years old also and it has a nice property that it still low level compared to newer network cards if you implement this on your network at you are usually just exchanging messages let some fuel and that's just boring because the firm they implement everything sure the older network cards also have some film there but you get a lot of more low-level access to the card and you you don't feel like you are just talking to a VM where you feel like you are implementing and actually drive by yourself so now these are the basic things I told my students I will now give over to Sebastian and who will show a little bit
of C code about well how to write a code how to write a driver and C and what
other could be done in a high-level language so just showed you what we have to do in general I'm going to take a bit more detailed look at this so first of all of course we have to figure out our
PCI addresses you can do this via LS PCI and then you get a list of all the PCI devices and you just look for something like into the cooperation 8:00 to 5:00 9:00 9:00 something that looks like this and then there at the front of the line there is an address it looks kind of similar to the MAC address and well that's the address we need so now we got the address from this address we can just go there and unload our driver that's basically it you just right there next up we have two M up our PCI register from the address space and for that we yeah we open our magic file and execute an air map the challenge here is mainly that for every high-level language you want to do this and you have to have some kind of way to actually use an M up either via a library or in the worst case you just have to use some C code for that next thing this is an extract of the
datasheet in the datasheet yeah it's basically like this you have the register names register offsets and then you can just go through the document and you find all the registers you need to read and set things for the driver and yeah a quick example network cards have LEDs often and we can make them blink turn on and off and how do we do this well we have to get the base address of our registers at the offset that we found in the datasheet and then just switch the bit and this turns the light off and on and yeah right the back of course the register also this is one of the very few valid uses of volatile and C because we have to really prevent compiler optimization here as we have memory access from multiple sources so
yeah next up how do we handle packets via DMA packets are transferred via cue interfaces receive queues transmit queues and these are often called rings because well they are organized in a ring like structure and these rings are configured via mm IO and when this is done we can access the device via DMA these rings then usually contain pointers to packets and these packets are then also accessed by a DMA the details vary a bit between cards and devices but that is not unique tune makes this process this is pretty much the same thing for all PCIe devices they all kind of work similar so what are the challenges for a high-level language shows we have seen some for example the MF call we somehow have to get access to this with the proper flags another thing we have to handle externally allocated memory and the layout this memory comes in also we've seen we need volatile to prevent compile optimization so we have to have some kind of semantics so we can force this in other languages and of course often especially for low-level stuff you often have to use some kind of unsafe code for high level languages because well many operations are inherently unsafe when you operate here but we try to contain this unsafe code to as few places as possible to contain this to a small area ok now some basic challenges I'm going to hand back over to Paul who says something about the goals for our
implementations ok um basically this is what I've told my students about what
I'm expecting for these implementations I wanted to have the same feature set as my C driver which was kind of used as the reference implementation and it also was supposed to have the similar structure but at the same time we wanted to have code that looks like it was meant to be within that language it's always a difficult trade-off to well use safety features wherever possible but if they cost too much performance do we really need them in all the places where can be used to safety features where do they make sense and then we wanted to quantify that the idea is to really have on the end like 10 driver implementations that we can quantitatively evaluate and look at the performance of all of them look at the safety properties of all of them look at properties like safety for memory
accesses is guaranteed for a packet buffers yes no for other stuff yes no and so on and I'm now going to look at a few of these languages basically I'm only picking a pezzi I'm only talking
one or two minutes about each of the languages for students who are not here but have already finished that teasers and then we are going to have a deeper look at go and rust but Alan I'll start with the the other languages where it's only a very very short high-level overview well first one is C sharp which seems a little bit unusual but we found a student for that so why not and no we didn't develop a driver for Windows Microsoft coatl are available on Linux on works really well for those who don't know c-sharp c-sharp is a just in time compiler garbage collected memory safe language and it has a relatively obscure rarely used unsafe mode and unsafe mode features full support for point us so you can basically write code that looks like C you just have to compile gift compile a special flag to tell it hey I'm going to use unsafe stuff then how can we access external memory or fun memory there's for example a few Knights rappers in and c-sharp is clean walk or the unmanaged memory stream and so on but that turned out to be slow too slow for our implementation so we used that unsafe mode which basically looks like this you see the unsafe keyword here and the other stuff it just looked like fee and it also feels like fee when you are
writing it so it's a really nice language to add drivers and again the unsafe code is contained to a few very known places that can then be audited compared to a CD driver where the unsafe coders all over your code base and you don't know where the buckets here we know if there's a buckets probably in there that you have while not checking the buff size properly or something and okay that's all it for c-sharp another
unusual language for drivers is Swift also front of students mention what kind of do it in Swift nose like or didn't
even think of Swift sounds like a good idea yeah so no we didn't develop a Mac OS or iOS driver sisters also available
on the notes Swift is a compiled language that's compiled by a LLVM memory management is done by a reference counting there's no garbage collector and it's mostly memory safe well again have to use some kind of pointers there's unsafe path appointed as also unsafe or pointer and more classes and we are using these things to make packets that are stored in DD maybethis available through the application using the driver and for example here's a property that wraps some memory in a unsafe buffer pointer weapon and the unsafe buffer pointer vapor forces you to specify how big the buffer is and then it does the bounce check for you when running in debug mode then pointers are kind of a little bit more verbose compared to C sharp but basically you can use them like like pointers and operator overloading here can help for some of the other managed buffer pointers row buffer pointers there is already in overloaded operator so it looks like an array access but yeah totally possible to do that and surfed even if you are writing a driver instead of a UI application now for the fans of
functional programming we have a fully working implementation in Oh camel o
camel is a compiled language that has garbage collection for memory management it's also memory safe and it's the first functional language we are going to you look at and one nice quick feature of a camel is the C struct library the C stock library allows you to specify memory layout like this kind of looks like a well C struct and then it generates from that code for accesses and this is really nice compared to the Swift example where you would have to hard code to offset somewhere and it's much nicer this kind of code generator here which does two I think for you and also can do automatic engine of swapping and so on and then decode and oh camel now looks quite different compared to what you might expect for a driver code and this is for example just their function that counts how many packets have been received and they were suffering by checking the Flex and we can see here that we are using the get at xwb status which is the data function from the previous struct declaration we are checking a flag in that and then we are just counting the Pakistan and we know okay we have received ten packets or whatever since the last call to that function and we can now pass them on to the user of our driver I'm more
functional programming we also have an implementation in Haskell Haskell again
compiled language memory management by a garbage collection memory safe and also functional unites property of few nice features of Haskell that you might not know about is the system POSIX memory package has a lot of lot of really helpful functions this is compared to - Oh kennel that we had to add some C code to get a map and M lock was the right flex working and mirror was everything available and the foreign package has nice functions like pig bite poke bite and so on where you can just do a memory access and another thing we are using and has quite a lot rd some types because a lot of things and the drivers are where you have inferior Union as basically this part where you write some data and add in a buffer in some the device reads this data and like for a transmit packet descriptor we have to transmit read format which looks one way once the device has transmitted a packet it goes back to the same memory location and overwrites a different thing so we then need to read the same data something else and it's basically a fancy C Union and SS little bit nicer to work less than there than the C unions ok these other languages of students who are not here this year and you can check out all the implementations on on github yes there'll be a QR code to scan on the last slide and I'm now going to hand
back to Sebastian who is going to do a deep dive into golf yeah thank you very much so know you've seen few languages next
up let's go what is Go Go as compiled programming
language it's been developed by Google it's generally as a general-purpose language but as it's been developed by Google it's mainly designed for distributed systems because that's what Google does distribution systems a driver is not a distributed system so why should we even use go well go thus offers a few things that are quite nice it has a runtime that has garbage collection and also enforces memory inside safety also it has very large center libraries so we don't need to use any other code except the library code so how do we program drivers and go actually in most cases it's just like see now a few main differences though on one hand we don't have pointer arithmetic we have pointers but we don't have arithmetic and this is what we need for managing our DMA memory and the second point is we don't have a wallet I'll for the memory barriers for register axles mainly so what do we do instead to compensate for that first of all we can manage the DMA memory by a slices that's pretty easy and the second thing is we can use unsafe pointers for pretty much all the rest unsafe pointers are arbitrary pointers so that's good but they do circumvent the runtime so we have to be careful with them and this is what we use for mainly physical address calculation and register access and there is also a rule set for unsafe pointers so they are still valid and of course we can we follow this rule set so to create examples first of all man pools here you can kind of see how we manage the DMA memories first of all we allocate the Z memory initialized in the mempool and then this man fluid path this is actually the whole memory mapped area and we can just stop this into packet buffers so you see that's pretty easy next thing is for physical address calculation this is where we need our unsafe pointers for the first time because we have to translate our virtual addresses into physical addresses that the network card can then use to actually send and receive packets and for this because the runtime checks many things and and you have to explicitly convert types you first have to convert your pointer to an unsafe pointer and then you can convert it to an integer type the UN pointer and yeah it's how you do this and go also no wallet are no problem I said we need a volatile as we share registers with a network card and we need and is some kind of compiler memory barrier to prevent reordering I notice a volatile and go but the atomic functions do prevent reordering among them amongst other things they provide strict guarantees then volatile but yeah it doesn't really cost us any performance here so we're just going to use that and yeah so we can use Tomic store and load for integer types too then get all memory barriers so as a conclusion I thought go was actually quite nice to work with safety properties have been proven Kotori thou has posted before wrote a kernel lingo and yeah so it is safe it gets you some safety guarantees and the other thing it's kind of a personal opinion but I think it looks like C code and beautiful but it also has downsides in the best case it's approximate approximately ten percent slower than C and have you getting more optimal cases it's even worse compared to C but yeah we have to live with that also descriptor access can be a bit ugly but well as long as it works so next I'm going to end over to Simon who did this
in rust Thank You Sebastian so let's
talk about rust what is rust well the rust website says it is a safe concurrent and practical systems language sounds great that's exactly what we need to write or use this base network driver so is there anything else we should know well yes rust has no garbage collector so we have less overhead for memory handling it has a unique ownership system and some rules for moving and borrowing values so with these rules we can clump accomplished rust goal of memory safety and we have unsaved like another link which is presented before what is the ownership system well it is the core feature of rust actually it's it's basically just a set of three simple rules rule number one each value has a variable that is its owner rule number two that can only be one owner at a time and rule number three when this owner goes out of scope the value is freed these three rules combined with rules for borrowing values keep us safe from memory port bucks like double fries and as they are enforced at compile time we don't have any performance penalties at runtime so our programs are to see programs but we have the great advantage of memory safety what does it look like in our implementation well we have packet struct for network packets that on DMA memory and these packets are passed between the users of our driver in our driver and with them a long ownership is passed as well and that's pretty cool because when the packet is passed to the user only the user can modify the packet and the packet content and when it's passed back to the driver only the driver can modify it so we have basically safe packets handling unlike in other languages and at the bottom of this slide you can see how you would use the driver based on our drivers interfaces you can see how to receive modify and send packets and yeah there's no way to screw up so for example you kind of forget two free packets because packets are free to automatically when they go out of scope and I'll return to the memory pool of our driver so yeah this is safe code and and there's nothing you can do wrong here but unfortunately we also have unsafe code in our and our driver what is unsafe code well not everything can be done in safe trust for example calling foreign functions and dereferencing roll pointers is unsafe but this is nothing and usually the idea is to reduce unsafe code to a few places and to some checks to make unsafe code safe what does it look like in our driver well for example we have to set register m??tode' a method to set the registers of our device and we use pointer right volatile to write to some register of our device and before we do that we have some assertion in our code to assert that the address we are going to write to is indeed inside of the map memory region so we have we have some great code but is it fast well to find
that out we set up a test bed to benchmark our drivers we have two servers packet generator and the device under test they are connected by directionally with two ten gigabit per second connections we use the Gungan moonshine packet generator written by Paul because obviously it's the best packet generator yeah and on the device
on a test we have simply bidirectional packet for water that we implement that you implement it on top of our drivers in all languages and yeah so let's look at the results of our measurements so
this is a graph showing the throughput of our for water on the x-axis you can see the CPU speed on the y-axis the packets per second we look at packets per second because the main overhead is per packet not provided and the top of the y-axis is 30 million packets per second because that's about 20 gigabit per second at minimum sized packets you can see the different plots for the different languages it's linear scaling with the CPU speed rust is the fastest swifters yeah well it performs incredibly poor and so the thing is usually you you don't manually change your CPU speed so we asked ourselves is there anything else we can modify yes this you can change the the batch size so how many packets you send at once to the PCI device because that avoids some overhead and kernel drivers usually use a batch size of 1 on transmission and higher batch sizes are one of the main reasons why users base drivers are faster then kernel drivers batch size of 32 to 64 packets is a very good batch size because higher batch sizes are bad because we have more l1 cache misses and well we asked ourselves why does perform that bad and Paul is going to tell you why just like that yeah so if
you have some program that as performing not as expected well what do you do you run some profiling on it and then you get a lot of data and then you need some way to visualize this a common way to visualize profiling results this thing
called flame graph basically the x-axis is time spent in a function and the y axis is the depth of the call stack and now I don't expect you to be able to read all of this these are just the function names that are in here and if you look at the top most functions here these are the leaf functions by the time is then actually used and we can characterize all these functions what it's doing in there and while we found
out it's due to Swift's memory management Swift at calls to the magic internal release retains and functions for each object used in each function
just to keep track of the reference counting and that's well basically no problem if you are just writing a UI app for some buttons or something but if you are I think a driver that has to pass through millions of millions of packets through a lot of functions all the time then turns out that spends well 76 percent of its time in these release retain calls so it could be four times faster if it had other way to manage memory and for comparison in go we spent less than half a percent in the garbage collector because it's quite simple application so now the big advantage of having these semantics verse reference counting in Swift is of course that there are no unpredictable pastimes and the garbage collector and Gore might just stop your driver for all the time or for some time the question is now is it a good idea to have a garbage collector and something like a driver now the good thing is we can measure that because we have this forwarding application and we did that we measure the latency of all the packets that before wallet this latency was measured at around 16 million packets per second and what you see here is the cumulative distribution function of the latency of last and this is basically a almost perfect normal distribution centered at around eight microseconds ver??osa very Society for comparison a hardware search takes around one microsecond to follow out a packet but eight microseconds is for a software forwarding thing is really nice and fast and now to the same graph let's at the other languages and now we see well goes from similar but c-sharp is a little bit slower at the
top but I do realize that this graph might look a little bit confusing so let's really quickly explain it how to how to read that if you are not familiar with these CDF's now we look at a value at the y axis for example point five so 50% then we go over to their language go down to the x-axis and that just means that 50% of the packets take less than eight point nine microseconds to be processed for c-sharp and the other 50% take more so then looking at any latency where a garbage collector or unpredictable spikes are involved then it's always a good idea to not look at the median but at something like the 99th percentile and for c-sharp one percent of the packets take longer than 30 microseconds and one percent of the packets is a lot if you're processing a lot of packets like one in a hundred packets and you are doing millions of them per second so you are going to get these worst case latency it's quite often what we really want to know is not the 99th percentile but in 99.9999% ID or something like that so we will need to zoom into that graph here if you're zooming into a graph you usually change the axis to be long I would make to zoom in well in this case it would zoom in to the wrong part of the graph so we also have to subtract the excess from one yielding the complementary cumulative distribution function on a logarithmic axis now this is inverted and a little bit a confusing graph but this is what you would see in a academic publication talking something about latency or anything like that but I think it's a really confusing graph but you can quickly see here the percentiles and the top no the bottom line would be the latency of one packet in a million packets and what we can do with this graph to make it a little bit more approachable if we can basically just rotate it and change the access description of the x-axis which was the y-axis before and the x-axis is now the percentile and the y-axis is the latency and now it's easier to read we can for example look at ninety nine times nine nine and check out where it's which latency at that percentile is for the different languages and this is a kind of graph that you will see for a lot of latency evaluations of anything that has latency spikes if done properly and you unfortunately often see people doing latency evaluations and then providing like the average or median latency which is from many cases are completely useless value most of them probably just
don't know better but if you want to evaluate latency please have a graph like this in the end and there is if you want something to Google there's a library called HDR histogram which can generate these graphs from latency measurement data and that's just a really nice way to characterize garbage collection or just-in-time compilation latency or anything like that now we got a driver that is nice and fast and has a relatively low latency for for most languages but we have not yet really looked at safety and security beyond what was offered by the language because our driver still wants its route like
all or virtually all user space drivers run as root by default and well why is this the case well I've shown this code before there are a few operations in the initialization that just require you to be rude like mapping the PCI Express resource requires you to be route for implementation details in the Linux kernel we need non-transparent huge paces for the DNA buffer they require you to be rude to allocate them locking memory requires route so these are clearly all functions related to initialization or setup so the obvious idea is well write a small program that does that for us keep that simple and out of that and check that's good and then drop all the privileges can we do that yeah sure we can do that it's relatively easy just drop privileges after setting up memory but that's still not really secure in any way even though you are now running as an unprivileged user and to understand why this does not work as you might want it to work we have to take a high-level look at how memory access works on a modern system and this
is what a modern system looks like at a very high level view we have two CPU here at the top with our application running on it and and button left we have PCI Express devised but unlikely I have some memory and if we now want to do some memory access from our application it goes like this it goes through a thing called the MMU the MMU is the memory management unit and the memory management unit translates your virtual address that you have in your program to a physical address that can be used by the memory controller now the security or the isolation between processes is controlled by the MMU and only the kernel can reprogram to the MMU to guarantee the isolation now if we want to access our device by a PCI Express for example memory mapped i/o it also goes through the MMU D and a mutant knows this is not going to memory but to PCI Express it talks to the device and that's also all fine now you could argue clearly these are the two kinds of memory accesses we do both are checked by a DMM u so what's the problem well the problem is that when we tell the device to do something that's memory I've previously mentioned we tell the device physical addresses to use them and well of course the device does not go through the MMU of because it doesn't know it doesn't have a concept of it's used by a process or it's being used by this kind of person just has full access to all of your memory and so if you somehow own the program that is running as an unprivileged user it's quite a trivial exercise to get any data from the important area that you shouldn't be allowed to access all right any data there by just telling the device to do it for you meaning any application that has direct access to a PCI Express device is effectively route even if you dropped privileges so the obvious solution is to somehow make this access pass goes through the MMU and there is this fancy a component quality iommu which is exactly for this use case this can be found on any modern CPU that has hardware virtualization features because it's mainly being used to do pass-through of PCI Express devices to virtual machines in a safe and secure manner but you can also use it for user space drivers you just need a way to tell the colonel to configure the i/o mmm use the proper restrictions for us and then the access looks like this and we just need to configure it in a way that the iommu has the same permissions for that device like the MMU has for our user space program and then it's perfectly secure if you're a program that owned attacker doesn't have any privileges and beyond what your process has and this is also useful for safety because when I initially wrote a driver I kind of killed a server and I apparently must configure the DMA and overload something that was important for the file system apparently and then it was dead and had to reinstall it so that wouldn't have happened if I had started with the iommu in the first place how do we do that to
reduce the iommu specifically on Linux which has a night nice sub system called yeah we f i/o subsystem and it's what we can use we just need to prepare the system as a route this is a one time step assault we can bind the device to the VF i/o driver instead of two no driver at all we can change the owner of the recycling metric device file and pass it to an unprivileged user then we have to give the unprivileged user permissions to lock memory to allocate DNA memory but you can also stick them can like you are allowed to allocate - err twelve megabytes of locked DNA memory and then all the remaining steps can be done as an unprivileged user the unprivileged user can now call an M upon the new device and then the unprivileged user can communicate with the kernel yio control commands on the magic device and it can also tell the colonel to please allocate team a memory and also actually a better way to allocate DNA a movie for technical reasons and then you can just use the device as you had before so you just need to change set up steps and then you have a user space driver that can run with well no special privileges at all and the IOM you will check all the excesses and the kernel will make sure that you can't configure the iommu in a wrong way you can just tell it to please configure the iommu in such a way that the device can access my current address space but not anything else and we have implemented this in our C driver the my student who implemented this is actually here today but he was afraid to come up on stage but if you have any questions you can talk to him afterwards there now we have this awesome driver which is safe secure and everything and yet some people still argue our user space drivers are useless I already have a driver in the kernel why do I need another driver by what you want to write a driver and obvious answer to the question is why wouldn't you want to write a driver can be fun at a lot of fun when I wrote the first driver so yeah then maybe you just need a quick and dirty driver 4-wheel device that you found somewhere maybe you just need quick development cycles where I want to don't want to shoot down your kernel all the time or reboot all the time if you have real device that you are developing is maybe you're developing a custom device may be of some FPGA bought that you want to talk to real quick result being involved in some kernel stuff or maybe you just need a feature that's found on the hardware but not yet on the device driver there are some features there's like a like a lot of stuff in the past you have infant IP stack of flooring which wasn't in the open source driver and so on but something we have done recently is hardware time stamping the latency measurements are short before required us to take time stems of 15 million packets per second with nanosecond level position and it is quite hard and people usually use special hardware for this and we ourselves have used meta fides in the past folders which wall can be a lot of fun to use these nice devices but at the same time it's prohibitive from a cost perspective or from a user experience perspective for some people who just needed to take some time stance so we want to do it on the cheap of the Shelf commodity network cards and turns out all of these or of the newer carts have a hotter future that can just at the time stem to the incoming behalf of all packets that they receive but sadly none of the existing drivers supported this feature but showing you how you can access registers you can just skip the step there you unload the original driver you can just poke the just end that the thing by the original drivers running poke to right register and like they tell it to do this please and now you've got the timestamp at the end of every packet I'm sure it probably breaks your TCP stack or whatever if there's extra data but like pcap doesn't care in this case you just did it was DPD can go to war packet buffers and it also doesn't care if they divide things there's four extra bytes right extra bytes for this setup that we had here we use the embedded Nakhon Expy on d and you had a fiber optic splitter to central timestamp of all the packets before and after device on a test unless you lead us with this measurement of all these things and yeah so there's just a simple use case why you would want the driver to conclude I can only really say
that I think driver should be written in better languages now I think you
shouldn't be start writing a new user space Java and C nowadays sadly if you look at the world of user space drivers there is mainly d pd k which is a network user space travel is all see there's both all three because mainly copy pasted from current levels and then there's FP DK which has nvme drivers also all c and big exception to this is snaps which has drivers in Lua which is quite nice and well our implementations
which have a lot of different things and we really want to compare these other these languages you can scan this QR code or just Google for XE languages on github or whatever and then there's like this meta repository that has links to all the implementations has we'll have a link to this talk and yeah basically check it out what your own driver no current code needed thanks for your attention thank you very much all southern and
Sebastian we do have time for questions please line up at the microphones and to get it started please a question from our signal in row from the internet so the IRC first of all was wondering why was the best
proposal only offered as a bachelor thesis yeah the best question kind of expected herself like about adding new random language results well why can't I do it in - the the problem is I tried to implement it in version doesn't work the way I wanted to work I would need to write more C code they're the way I tried it it does not work if I wrote a short C program that called the MF thing and then just slept forever and then the idea was to get the to access the address space of that program that just called the MF via the track FS system and then the other success metric thing which has all the things none with DD you can write and read from there but it breaks at some point because you can it goes through when reading but if you write something at some point via Park FS that doesn't go through the PCI Express so didn't work and only wanted as a bachelor thesis because kind of a master series should maybe be a little bit more serious than a job driver yeah maybe a more serious one godness asking how to use the chest to handle interrupt requests or a code with strict timing requirements if I have strict timing requirements I'm not going to use interrupt interrupt if I have strict timing requirements I just pull the device all the time this is how basically all user space travel sake just ask the device is there and you packet 1 million times a second and interrupts are one of the slowest way to communicate between CPU and device just receiving the interrupt requires you to do a context switch on the CPU then you have to do a context switch back because you don't want to do too much in the interrupt handler and then you have to play the device anyways because to interrupt just tells you that something has changed so if you really care about the latency then you just pull the device all the time and for userspace interrupt check out and we have I or the vfi or I'm back you can do like an Apollo on something and then you get notified if there's an interrupt if you really need that ok let's go to the hall questions please keep your questions to one sentence only and only ask questions because there's many of them microphone number two please so when you compare different user space drivers in different languages why was rust slower than C given that it the memory safety is compile time yeah well we have a few more memory of relations because of the safety so we we have to move the the package structures from inside the driver to to outside to the user and you don't have third and C so rust was a bit slower because of that but I think you could optimize it maybe a bit better because yeah well it was just a bachelor's thesis or III didn't have that much time but yeah I guess it could be a bit faster but I think it would still be a bit slower than C also my C driver doesn't do any bounce checks at all just assume so don't do I think
you mentioned Haskell in the beginning but it wasn't in the comparison can you
talk about that Haskell is not yet optimized for performance that would have been unfair to @a unfinished version for it it's currently still quite slow and I I didn't want to add it there microphone number four please or something or something thank you very much have you consider your internal programming languages like Idris or Koch where your compiler can check the logic of your driver um I don't have a student for that to implement it but yeah microphone number one please okay I have seen several languages with garbage collector including including go so my question is how often do the GC stop the war happens and what is the general heap sizes how often I don't know how often and the latency shot you how long it takes which was like up to 40 microseconds which depending on your application might or might not be a problem there's a few other data from when we reference this paper of the guys who implemented the whole operating system and go they mentioned they see up to 200 or 300 microseconds of past times and heap size maybe something about the garbage collector I'm not even sure if how much it actually works because the go profiling just dropped this node as it was so little time compared to the
others so it's basically irrelevant except for probably some latency microphone number three please yeah I missed a couple of languages OpenCL and
CUDA not not because they're particularly yeah languages but it would be interesting to have the GP the network a paper out there that's called packet radar thing yes they do exactly what there's also another paper out there called raising the bar for GP packet processing or something like this they basically argue against that the
main problem is if you are transferring the packets between the network card and GPU this is kind of slow and you need to use gigantic batch sizes there the packet shader guys used to think batch size of 4000 or 8000 which effect latency and they don't have a proper latency evaluation wonder why but if you're interested and GPU packet processing read that packet read up iPods a few years old thank you microphone number one please so how do
you deal with i/o or during the x86 will guarantee you that this the order in which the CPU posts IO accesses is the same as the device receives them but on other platforms this is not the case and if you do it with you i/o the UI or doesn't give you any such guarantee well for memory ordering it's highly specific to the device you are using which memory ordering semantics you are using the the great thing is in the in the device we are using here the the Intel device in the s11 location where I'm so really
sure that I do need a release memory other semantics because there's something which clearly said some flack and then the device with some other memory based on listen and like 99% sure that this should be release memory order there but none of the driver implementations has any release memory order barrier there and so we don't we don't need it here and other than that for going to ask you of course have low-level primitives to to ensure or to enforce hardware memory barriers for other languages if you for example check out the snap driver which is written in Lua they have a little C stop which cards right entrance instruction at the right place Thanks number four please
thank you yeah the abstract of this talk mentioned that the user space implementation was six to ten times faster than the in kernel implementation what is just because of the batch sizes or were there other reasons for that so this is mainly due to the to the back side the current implementation when using HDPE is also fastest or was compared to current without XTP colonel with HDPE I
think we are 30% faster also if you have a kernel that can exceed to http between different necks because most Nix can just tentacles different definite yes next question again from the internet so our is asking have you considered
writing user space drivers for the more inherently insecure due to specification complexity things like the Bluetooth stack or things like this yeah would be would be interesting yeah I guess that's a good topic for further research we just went for a network because it's really common to have user space network stuff all direction all like look at your iPhone or whatever there's a user space TCP stack running on it so yeah number three please no thanks for the talk just a little question does the iommu affect performance you're not yet sure you can ask him afterwards as fast you have evaluated it does not yet affect performance there's also a paper called I was called PCIe bench PCIe bench something come last yeah I think and you can read this paper they have a performance evaluation of the IOM you so yes there is some effects because the Tildy size is smaller on the IOM you but we couldn't measure it in a lot I set up yet microphone number one please when you access ring buffers you usually have to have like a an excess ones macro that you used to access the memory in C because C is actually allowed to change the semantics of the implementation in a way where instead of say reading memory is stored in a variable and then using that variable would translate into something that would read the memory twice and this is this was a problems was excess a 155 I believe and were in net back net front the communication actually broken in that way because you had to talk to buck how would you enforce like a single memory access and all of those programming languages and for us all of us I have but the thing that's most of
these programming languages do need to copy the descriptive and using it so the Turing sort the descriptors the descriptors are basically pointers to some other buffers and the critical part is when reading the descriptor and I think all of the language implementation copy the whole descriptor just sixteen bytes okay so a memory is always copied guaranteed I hope so please do check in the in the girl code we had the the the atomic read thing and I hope that only does one access not about it considers a back in goes my atomic it twice and see I also copy it I think we have time for one last question from microphone number
two please how can we convince other people especially other developers and the business people of the necessity of moving away from C learning a new language and investing the time of developing stuff in it yeah I honestly don't know I'm it's like a mystery to me
why people keep writing stuff in the yeah no idea sorry thank you very much
to Paul Simon interaction [Applause] [Music] [Music]