We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Linux Kernel Library

00:00

Formal Metadata

Title
Linux Kernel Library
Subtitle
A Library Version of Linux Kernel
Title of Series
Number of Parts
490
Author
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code as extensively as possible with minimal effort and reduced maintenance overhead. It allows us to link the library with any programs (which wish to call as a function call) containing Linux kernel code. There are many use cases: reading/writing files without general system calls, putting experimental protocol implementation without neither of host kernel update nor kernel module installation, using customized kernel in container instance, building a unikernel based on existing rumprun framework, or testing/fuzzing kernel implementation in userspace execution, etc.
Revision controlKernel (computing)Library (computing)Computer programCodeSimilarity (geometry)SoftwareMultiplication signFamilyDifferent (Kate Ryan album)Computer programmingImplementationMikrokernelProjective planeKernel (computing)Operating systemBinary fileShape (magazine)NumberEmulatorFormal languageRun time (program lifecycle phase)Library (computing)System programmingOrder (biology)WindowVirtualizationIntegrated development environmentDirection (geometry)Revision controlSpacetimeBinary codeExtension (kinesiology)Computer architectureCartesian coordinate systemSource codeProgramming languageFunctional programmingSystem callComputer animation
Library (computing)ArchitectureNetwork topologyCodeComputer networkSimulationSpacetimeKernel (computing)Independence (probability theory)FingerprintIntegrated development environmentBootingImplementationComputer hardwareSpacetimeCartesian coordinate systemCodeLibrary (computing)Greatest elementKernel (computing)Computer architectureEndliche ModelltheorieSystem programmingMereologyComputer programmingFreewareAndroid (robot)Interactive televisionSource codeSlide ruleWindowComputer animation
Stack (abstract data type)Computer networkFluid staticsKernel (computing)Embedded systemLibrary (computing)Network topologyArchitectureCodeSpacetimeSimulationLibrary (computing)Computer programmingImplementationKernel (computing)SoftwareSpacetimeNetwork socketComputer animation
Fluid staticsKernel (computing)Stack (abstract data type)Computer networkEmbedded systemSpacetimeSoftwareOverhead (computing)Projective planeVery-high-bit-rate digital subscriber lineKernel (computing)MereologyContext awarenessAsynchronous Transfer ModeCartesian coordinate systemLibrary (computing)Interface (computing)Computer architectureInteractive televisionComputer animation
Interface (computing)Component-based software engineeringIndependence (probability theory)BefehlsprozessorFingerprintVirtual machineCodeKernel (computing)CodeComputer architectureKernel (computing)Library (computing)Scheduling (computing)ImplementationStandard deviationRaw image formatMereologyIntegrated development environmentComputer hardwareSource codeInformationCartesian coordinate systemSemiconductor memoryProjective planeComputer animation
Network socketInterface (computing)MiniDiscBlock (periodic table)Computer-generated imageryHypercubeTap (transformer)Computing platformIndependence (probability theory)ArchitectureBefehlsprozessorImplementationFingerprintThread (computing)InformationInterface (computing)Integrated development environmentImplementationDevice driverSystem programmingKernel (computing)File systemBlock (periodic table)Computer architectureConnectivity (graph theory)Different (Kate Ryan album)Order (biology)CodeElectronic mailing listBridging (networking)WindowVirtualizationProjective planeSoftwareCartesian coordinate systemMereologySystem call
Point (geometry)Kernel (computing)System callArchitectureIndependence (probability theory)Interface (computing)Translation (relic)SpywareLibrary (computing)Standard deviationVirtual realityBefehlsprozessorLink (knot theory)Cartesian coordinate systemStandard deviationSymbol tableNetwork socketSpywareProjective planeMereologyError messageCodeImplementationLimit (category theory)Binary codeNumberInterface (computing)Library (computing)SpacetimePoint (geometry)Run time (program lifecycle phase)System programmingJust-in-Time-CompilerCASE <Informatik>Computer animation
Computer-generated imageryMiniDiscRootkitLoop (music)MiniDiscLibrary (computing)RootCASE <Informatik>Computer programmingWindowCartesian coordinate systemImplementationSystem programmingSpacetimeBootingMedical imagingLatent heatSoftwareFile systemKernel (computing)Finite differenceContent (media)RewritingVirtual machineOperating systemComputer animation
Computer networkNetwork topologyCommunications protocolAndroid (robot)Web pageSpacetimeKernel (computing)Integrated development environmentImplementationMultiplication signCASE <Informatik>Source code
ZugriffskontrolleChainingFunction (mathematics)ChainParameter (computer programming)Kernel (computing)SoftwareOrder (biology)Interface (computing)Computer wormComputer programmingImplementationVideo game consoleCone penetration testProcess (computing)Core dumpKey (cryptography)Game controllerPlastikkarteStack (abstract data type)Telecommunication
Kernel (computing)Web browserCompilerCodeCodeKernel (computing)Web browserComputer programmingEmulatorSinc functionAddition
Data conversionKernel (computing)Web browserCodeKernel (computing)Binary codeSystem programmingFinite differenceScripting languageTask (computing)CASE <Informatik>Source codeSource code
DisintegrationLibrary (computing)Kernel (computing)Run time (program lifecycle phase)ImplementationVirtualizationDemo (music)Content (media)Food energyComputer animation
Kernel (computing)Revision controlLocal ringMedical imagingComputer animation
System programmingScheduling (computing)Default (computer science)Logic programmingBlock (periodic table)WeightCommunications protocolRootkitProcess capability indexBridging (networking)SpeicherschutzArchitectureKernel (computing)Process (computing)Intrusion detection systemMenu (computing)Computer programDisk read-and-write headComa BerenicesComputer fileAsynchronous Transfer ModeStructural loadRandom numberComputer virusRobotBootingSystem callImplementationKernel (computing)Computer animation
DisintegrationSoftware testingSystem programmingFocus (optics)NumberFuzzy logicoutputSoftware testingOrder (biology)File systemImplementationComputer animation
Integrated development environmentDirection (geometry)Patch (Unix)Asynchronous Transfer ModeLibrary (computing)Kernel (computing)Computer virusImplementationLink (knot theory)Interactive televisionEndliche ModelltheorieSoftwareGaussian eliminationAsynchronous Transfer ModeApproximationMultiplication signSoftware maintenanceOrder (biology)Musical ensembleSpacetimeCodeCharacteristic polynomialKernel (computing)Library (computing)Different (Kate Ryan album)ResultantFlagComputer animationLecture/Conference
FacebookPoint cloudOpen source
Transcript: English(auto-generated)
I'm going to talk about Not talk about not talk about micro kernel related stuff, but I'm going to introduce the Our project which was inspired, which was started with the inspiration of the micro kernel stuff
So I'm Hajime from Japan and I'm going to talk about, I'm going to start with why we started this project So as you might know or you are familiar with this There are plenty of the project that is trying to emulate or mimic the Linux kernel behavior in the different shape of the software
like Microsoft Invented invented the drawbridge project which was started as a library operating system But this they are now forming as a another software which is called Windows subsystem for Linux
Which try to emulate Linux binary executable on top of the Windows operating system without any virtualization hypervisor technologies and Google also trying to Introduce another project which is called the gVisor This is kind of a sandbox for the container environment which emulates the Linux kernel
feature in the user space in Go languages and Another academic project like graphene also offer the similar facilities. They provide the Linux Emulation Linux compatibility layer in the various operating system as the drawbridge project has been doing
so they are They are they offer the Linux binary compatibility in various operating system not only for the Linux But also Windows, MacOS and the BSD stuff and the BSD family as well as the different execution environment like an Intel SGX extension
And the final project I mentioned here is a NOAA which is another Another compatibility layer project running Linux application on MacOS So those are very similar the internals of the implementation design
Looks like very similar but The compatibility that can that they can achieve is always incomplete some of the system call are missing always and as I and and
The resulting and they result in the Different direction in order to provide the Linux facilities for example the Microsoft The Microsoft of the Windows subsystem for Linux Is now has a version number two Which they try to shift the architecture from the library operating system base one into the hypervisor
base one which I think they give up to emulate this kind of Linux Linux emulation in this user space Implementation but our motivation in this project in our Linux kernel library project is
We don't want to rewrite the Linux kernel twice or three times or forever Because the Linux kernel is written in C and Then your program or our program may also be written in C program Even though you are you are using different program languages you can use the C you can call the C function from your
language runtime So our motivation why not reuse this Linux kernel source code In a different shape as a reusable library So this motivation is quite similar to the guy that the guy from the netpsd kernel
Which they which is which he introduced any kind of architecture in the monastic kernel So this slide shows the brief overview of the what the LK Linux kernel library looks like so our library our
Our code will be generating a library which can be shared or which can be linkable from the program external programs So the and the the implementation of the LKL is Taking a similar approaches as the user model Linux does which takes
Which create the new architecture directly inside the Linux kernel source tree But this architecture is totally hardware independent We are trying to outsource all the hardware dependent code inside the architecture code into the different one Which are described in the this the bottom one bottom
part of the this architecture And by by taking this design the Linux kernel's code can be executable in the various environments So far you have experienced with learning this Linux kernel code on the Linux user space without any kernel interaction
And we can also this library on Windows operating system Free BSD and Android user space application and some of some of them were playing with the this code in the UEFI bootloader and Some of them are also playing with the using this part this portion is the unikernel as a unikernel
so if you look at this this program and this This program as the user space network stack because you can use this library
Which contain the TCP implementation inside the library and the your socket program can call this library without interacting with the kernel So there are plenty of the user space network project, which I listed in part of the all of the project so Most of them are trying to achieve the higher high speed performance in the user space
execution because they can eliminate the context context switch and overhead between this user and the user and the kernel mode interaction But they they usually suffer from the application
Interfaces because they use their own interface for the application Some of them provide the project layer compatibility for this kind of user space network stack, but it's not always complete It's not always complete like some of them lacks the e-poll support
for the library But our goal should be Identical to the what Linux can do so both tools should be the same In our inner motivation so the internal of the architecture is as I mentioned
We are trying to create the new Newly introduced architecture inside the Linux kernel 3 by eliminating hardware or underlying layer dependency inside this architecture and The outsourced portion of this
Architecture like accessing the hardware resources like a clock memory or process scheduling The this kind of detailed information should be contained is inside this host environment which can solve all the underlying layer dependency inside this hosting environment
so this path should be very portable inside the Linux kernel source and Another goal another goal of this project is we don't want to try to modify this these great part Not only the kernel part, but also the application code should be usable as is
so Because we decided to provide the API to the application But this raw format of API is not compatible to a standard library implementation
so the first part of the host back end is located at and the newly introduced architecture Which try to unify the interface between across the different environment so host environment has a different implementation like we currently have a projects interface and
Windows Windows operating system interface as well as the bridge implementation of the ramp hypercode which can be expand the under and the support list of the underlying layer and In order to communicate this library in with this external component
we provided the and the virtual device layer virtual device implementation inside of this host environment and It exposes as a virtual interface so that the Linux kernel code can use the driver driver implementation of the virtual you
So we have implemented block devices Implementations as well as the network interface Implementation and we also have experimentally implemented the bar to IO bar file system implementation which can be exposed as a 9p file system to the
to the driver layer so this can be almost explained in the previous slide, but Linux kernel 3
And the side component is application interface which is located and on top of the the kernel implementation So with LKL it exports our own system called API which is called LKL system calls But this system call is not compatible as I mentioned before so we provided
the various way to access this Interface by from the application So for the first API is the LKL system called low API So if you have an application and if you want to use LKL
you have to use you have to rewrite the system called part of the application by replacing the symbols from the socket to the LKL system prefix one prefix one So this API is kind of it's slightly defined from the typical standard
projects API because this is the entry points of the kernel, Linux kernel, so the error number error number and the return values slightly different from the projects API So another interface that we provided right now is the which called so called the hijack library
Which is basically based on the dynamic translation on the runtime by LD preload So if you have a socket If you have a projects API Application and you don't want to rewrite the application You don't want to rebuild your application. Your binary can be translated with this additional library
But it has a limitation that Some of the standard library implementation makes some of the symbols invisible from the application side So you cannot rewrite such a symbol. So in that case your application may not work well
So another API is our own standard libc implementation right now we use the muscle libc as a standard library implementation and we call it this muscle libc as
To be able to use the LKL from the user space code So I'm gonna show I'm gonna share you some of the use cases that we use is this Linux kernel library because This is library if nobody used this library This software is useless. So I'm gonna present our our known use cases, but if you have I
Wish to have you I wish you to have a more expanded use cases if you have a nice idea So the first typical use case is to mount a disk image without root privileges
So some of the operating system Has a experimental implementation of the file system in user space Like if you have a ext4 file system image for the virtual machine and if you want to modify these Images on the following operating system like on Windows or Windows operating system
You may have to use such an alternative implementation the ext4 which may not complete implement which may not cover the complete specification the ext4 Or you have to use the virtual machine and the boot the Linux OS and the mount the disk image and
rewrite or modify the contents of the disk images But you don't have to do such a complicated stuff You can just use this library as a Windows application and modify it inside the user space program
So you can you can now modify the betaFS contents of the file system images in the different operating system So another use cases is trying to introduce the kernel feature in the very restricted environment you which you don't have a freedom to
Recompose the kernel space implementation So this is an example of the introduction of the master TCP implementation of the on the Android phone By the way, the MPTCP is already upstream in the previous week
And we don't have to do such a stuff right now But this is a snapshot of the two years ago, I guess and at that time We don't have much for TCP support But you can we can do it because we have a user space implementation of the Linux kernel on the Android phone so another toy
Implementation is the Unix pipe as a network interface card So you if you have a three different or two different? programs using LKL and each of the program has a network stack implementation and if you write the pockets generated by this kernel
into the console and Then if the next program receive the cone this pocket by a pipe As a received channel of the network interface card You can do something. I don't know so if you can if you want to
Make access control by this Pipe communication you may use you can use a grip command in order to filter the specific Payload of the packet with the grip command argument And if you want to duplicate
The generated packet into the some external program like a TCP dump You can do you can you do with the key command by mirroring the packet into the different processes That's almost it and Another example is that trying to combat the Linux kernel code into the JavaScript program
since last year, I guess Linux kernel can be built with LLVM and LLVM can generate the JavaScript code with additional trick You can run the Linux kernel code without any emulation like JS Linux does
but you can directly invoke the Linux kernel code in The browser So this is the initialization task of the Linux kernel in the C code. This code will be translated as Like this Which is with the this is automatically generated by the end script on sources
And you can run The Linux kernel code on the browser So another use case is they're running the Linux binary code on the different operating system So this demo is actually learning on the Mac OS
Is the energy implementation with providing this content of the slides? So we have implemented the OCI runtime implementation which coordinate the invocation of the Linux kernel library and try to communicate or interact with the
Container engine so we only we currently only tested with the container the Implementation as well as our own crafted Docker the Implementation which is running on the Darwin without any virtualization so
So this is the This is running Docker demo and this is not connected connected to It's not connected The Linux host but running inside of this MacBook natively and
You can run the Docker command with our own crafted images With anything's command and It is running the Linux boots
Kernel log with main call of the engines implementation We are sending the additional configuration, but it's not How do I show?
So there is also some folks who is also using LKL in order to Test a file system implementation with their own implemented puzzle I Have not involved this project, so you can if you are interested in you can try to look at their paper
Which is listed in the slides? So there is another implementation using an approximation But I'm going I'm running out of time, so I skip this one so We also try to expand the underlying layer facility in order to integrate other implementation like a solo 5
implementation We have also started the upstreaming this code into the Linux mainline So we actually try to upstream several time in the past, but now we have a suggestion from the maintainer Which we should work with
Work together with the user model in next so we are trying to integrate this code into the user mode Linux implementation as a different execution mode Is the library? So That's almost it. So if you have a program and if you can link this library as a
Link flag you may I wish you will have a another benefit that you cannot see in the past Thanks for much. Thanks so much for your attention, and I'm happy to take a question
I
Didn't bring the performance results today But the characteristic of the performance is almost similar to the user space networks Which can which we can eliminate user space kernel interaction So we will have a benefit of the user space execution like what DPDK can guys now can do
Thank you