Linux Kernel Library
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 490 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/47233 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 2020372 / 490
4
7
9
10
14
15
16
25
26
29
31
33
34
35
37
40
41
42
43
45
46
47
50
51
52
53
54
58
60
64
65
66
67
70
71
72
74
75
76
77
78
82
83
84
86
89
90
93
94
95
96
98
100
101
105
106
109
110
116
118
123
124
130
135
137
141
142
144
146
151
154
157
159
164
166
167
169
172
174
178
182
184
185
186
187
189
190
191
192
193
194
195
200
202
203
204
205
206
207
208
211
212
214
218
222
225
228
230
232
233
235
236
240
242
244
249
250
251
253
254
258
261
262
266
267
268
271
273
274
275
278
280
281
282
283
284
285
286
288
289
290
291
293
295
296
297
298
301
302
303
305
306
307
310
311
315
317
318
319
328
333
350
353
354
356
359
360
361
370
372
373
374
375
379
380
381
383
385
386
387
388
391
393
394
395
397
398
399
401
409
410
411
414
420
421
422
423
424
425
427
429
430
434
438
439
444
449
450
454
457
458
459
460
461
464
465
466
468
469
470
471
472
480
484
486
487
489
490
00:00
Revision controlKernel (computing)Library (computing)Computer programCodeSimilarity (geometry)SoftwareMultiplication signFamilyDifferent (Kate Ryan album)Computer programmingImplementationMikrokernelProjective planeKernel (computing)Operating systemBinary fileShape (magazine)NumberEmulatorFormal languageRun time (program lifecycle phase)Library (computing)System programmingOrder (biology)WindowVirtualizationIntegrated development environmentDirection (geometry)Revision controlSpacetimeBinary codeExtension (kinesiology)Computer architectureCartesian coordinate systemSource codeProgramming languageFunctional programmingSystem callComputer animation
04:11
Library (computing)ArchitectureNetwork topologyCodeComputer networkSimulationSpacetimeKernel (computing)Independence (probability theory)FingerprintIntegrated development environmentBootingImplementationComputer hardwareSpacetimeCartesian coordinate systemCodeLibrary (computing)Greatest elementKernel (computing)Computer architectureEndliche ModelltheorieSystem programmingMereologyComputer programmingFreewareAndroid (robot)Interactive televisionSource codeSlide ruleWindowComputer animation
05:49
Stack (abstract data type)Computer networkFluid staticsKernel (computing)Embedded systemLibrary (computing)Network topologyArchitectureCodeSpacetimeSimulationLibrary (computing)Computer programmingImplementationKernel (computing)SoftwareSpacetimeNetwork socketComputer animation
06:09
Fluid staticsKernel (computing)Stack (abstract data type)Computer networkEmbedded systemSpacetimeSoftwareOverhead (computing)Projective planeVery-high-bit-rate digital subscriber lineKernel (computing)MereologyContext awarenessAsynchronous Transfer ModeCartesian coordinate systemLibrary (computing)Interface (computing)Computer architectureInteractive televisionComputer animation
07:14
Interface (computing)Component-based software engineeringIndependence (probability theory)BefehlsprozessorFingerprintVirtual machineCodeKernel (computing)CodeComputer architectureKernel (computing)Library (computing)Scheduling (computing)ImplementationStandard deviationRaw image formatMereologyIntegrated development environmentComputer hardwareSource codeInformationCartesian coordinate systemSemiconductor memoryProjective planeComputer animation
08:39
Network socketInterface (computing)MiniDiscBlock (periodic table)Computer-generated imageryHypercubeTap (transformer)Computing platformIndependence (probability theory)ArchitectureBefehlsprozessorImplementationFingerprintThread (computing)InformationInterface (computing)Integrated development environmentImplementationDevice driverSystem programmingKernel (computing)File systemBlock (periodic table)Computer architectureConnectivity (graph theory)Different (Kate Ryan album)Order (biology)CodeElectronic mailing listBridging (networking)WindowVirtualizationProjective planeSoftwareCartesian coordinate systemMereologySystem call
10:43
Point (geometry)Kernel (computing)System callArchitectureIndependence (probability theory)Interface (computing)Translation (relic)SpywareLibrary (computing)Standard deviationVirtual realityBefehlsprozessorLink (knot theory)Cartesian coordinate systemStandard deviationSymbol tableNetwork socketSpywareProjective planeMereologyError messageCodeImplementationLimit (category theory)Binary codeNumberInterface (computing)Library (computing)SpacetimePoint (geometry)Run time (program lifecycle phase)System programmingJust-in-Time-CompilerCASE <Informatik>Computer animation
12:45
Computer-generated imageryMiniDiscRootkitLoop (music)MiniDiscLibrary (computing)RootCASE <Informatik>Computer programmingWindowCartesian coordinate systemImplementationSystem programmingSpacetimeBootingMedical imagingLatent heatSoftwareFile systemKernel (computing)Finite differenceContent (media)RewritingVirtual machineOperating systemComputer animation
14:31
Computer networkNetwork topologyCommunications protocolAndroid (robot)Web pageSpacetimeKernel (computing)Integrated development environmentImplementationMultiplication signCASE <Informatik>Source code
15:18
ZugriffskontrolleChainingFunction (mathematics)ChainParameter (computer programming)Kernel (computing)SoftwareOrder (biology)Interface (computing)Computer wormComputer programmingImplementationVideo game consoleCone penetration testProcess (computing)Core dumpKey (cryptography)Game controllerPlastikkarteStack (abstract data type)Telecommunication
16:36
Kernel (computing)Web browserCompilerCodeCodeKernel (computing)Web browserComputer programmingEmulatorSinc functionAddition
17:07
Data conversionKernel (computing)Web browserCodeKernel (computing)Binary codeSystem programmingFinite differenceScripting languageTask (computing)CASE <Informatik>Source codeSource code
17:31
DisintegrationLibrary (computing)Kernel (computing)Run time (program lifecycle phase)ImplementationVirtualizationDemo (music)Content (media)Food energyComputer animation
18:25
Kernel (computing)Revision controlLocal ringMedical imagingComputer animation
18:56
System programmingScheduling (computing)Default (computer science)Logic programmingBlock (periodic table)WeightCommunications protocolRootkitProcess capability indexBridging (networking)SpeicherschutzArchitectureKernel (computing)Process (computing)Intrusion detection systemMenu (computing)Computer programDisk read-and-write headComa BerenicesComputer fileAsynchronous Transfer ModeStructural loadRandom numberComputer virusRobotBootingSystem callImplementationKernel (computing)Computer animation
19:17
DisintegrationSoftware testingSystem programmingFocus (optics)NumberFuzzy logicoutputSoftware testingOrder (biology)File systemImplementationComputer animation
19:46
Integrated development environmentDirection (geometry)Patch (Unix)Asynchronous Transfer ModeLibrary (computing)Kernel (computing)Computer virusImplementationLink (knot theory)Interactive televisionEndliche ModelltheorieSoftwareGaussian eliminationAsynchronous Transfer ModeApproximationMultiplication signSoftware maintenanceOrder (biology)Musical ensembleSpacetimeCodeCharacteristic polynomialKernel (computing)Library (computing)Different (Kate Ryan album)ResultantFlagComputer animationLecture/Conference
21:55
FacebookPoint cloudOpen source
Transcript: English(auto-generated)
00:05
I'm going to talk about Not talk about not talk about micro kernel related stuff, but I'm going to introduce the Our project which was inspired, which was started with the inspiration of the micro kernel stuff
00:20
So I'm Hajime from Japan and I'm going to talk about, I'm going to start with why we started this project So as you might know or you are familiar with this There are plenty of the project that is trying to emulate or mimic the Linux kernel behavior in the different shape of the software
00:43
like Microsoft Invented invented the drawbridge project which was started as a library operating system But this they are now forming as a another software which is called Windows subsystem for Linux
01:00
Which try to emulate Linux binary executable on top of the Windows operating system without any virtualization hypervisor technologies and Google also trying to Introduce another project which is called the gVisor This is kind of a sandbox for the container environment which emulates the Linux kernel
01:23
feature in the user space in Go languages and Another academic project like graphene also offer the similar facilities. They provide the Linux Emulation Linux compatibility layer in the various operating system as the drawbridge project has been doing
01:44
so they are They are they offer the Linux binary compatibility in various operating system not only for the Linux But also Windows, MacOS and the BSD stuff and the BSD family as well as the different execution environment like an Intel SGX extension
02:05
And the final project I mentioned here is a NOAA which is another Another compatibility layer project running Linux application on MacOS So those are very similar the internals of the implementation design
02:24
Looks like very similar but The compatibility that can that they can achieve is always incomplete some of the system call are missing always and as I and and
02:42
The resulting and they result in the Different direction in order to provide the Linux facilities for example the Microsoft The Microsoft of the Windows subsystem for Linux Is now has a version number two Which they try to shift the architecture from the library operating system base one into the hypervisor
03:06
base one which I think they give up to emulate this kind of Linux Linux emulation in this user space Implementation but our motivation in this project in our Linux kernel library project is
03:22
We don't want to rewrite the Linux kernel twice or three times or forever Because the Linux kernel is written in C and Then your program or our program may also be written in C program Even though you are you are using different program languages you can use the C you can call the C function from your
03:46
language runtime So our motivation why not reuse this Linux kernel source code In a different shape as a reusable library So this motivation is quite similar to the guy that the guy from the netpsd kernel
04:03
Which they which is which he introduced any kind of architecture in the monastic kernel So this slide shows the brief overview of the what the LK Linux kernel library looks like so our library our
04:23
Our code will be generating a library which can be shared or which can be linkable from the program external programs So the and the the implementation of the LKL is Taking a similar approaches as the user model Linux does which takes
04:43
Which create the new architecture directly inside the Linux kernel source tree But this architecture is totally hardware independent We are trying to outsource all the hardware dependent code inside the architecture code into the different one Which are described in the this the bottom one bottom
05:04
part of the this architecture And by by taking this design the Linux kernel's code can be executable in the various environments So far you have experienced with learning this Linux kernel code on the Linux user space without any kernel interaction
05:22
And we can also this library on Windows operating system Free BSD and Android user space application and some of some of them were playing with the this code in the UEFI bootloader and Some of them are also playing with the using this part this portion is the unikernel as a unikernel
05:46
so if you look at this this program and this This program as the user space network stack because you can use this library
06:00
Which contain the TCP implementation inside the library and the your socket program can call this library without interacting with the kernel So there are plenty of the user space network project, which I listed in part of the all of the project so Most of them are trying to achieve the higher high speed performance in the user space
06:25
execution because they can eliminate the context context switch and overhead between this user and the user and the kernel mode interaction But they they usually suffer from the application
06:43
Interfaces because they use their own interface for the application Some of them provide the project layer compatibility for this kind of user space network stack, but it's not always complete It's not always complete like some of them lacks the e-poll support
07:01
for the library But our goal should be Identical to the what Linux can do so both tools should be the same In our inner motivation so the internal of the architecture is as I mentioned
07:20
We are trying to create the new Newly introduced architecture inside the Linux kernel 3 by eliminating hardware or underlying layer dependency inside this architecture and The outsourced portion of this
07:42
Architecture like accessing the hardware resources like a clock memory or process scheduling The this kind of detailed information should be contained is inside this host environment which can solve all the underlying layer dependency inside this hosting environment
08:02
so this path should be very portable inside the Linux kernel source and Another goal another goal of this project is we don't want to try to modify this these great part Not only the kernel part, but also the application code should be usable as is
08:25
so Because we decided to provide the API to the application But this raw format of API is not compatible to a standard library implementation
08:42
so the first part of the host back end is located at and the newly introduced architecture Which try to unify the interface between across the different environment so host environment has a different implementation like we currently have a projects interface and
09:04
Windows Windows operating system interface as well as the bridge implementation of the ramp hypercode which can be expand the under and the support list of the underlying layer and In order to communicate this library in with this external component
09:25
we provided the and the virtual device layer virtual device implementation inside of this host environment and It exposes as a virtual interface so that the Linux kernel code can use the driver driver implementation of the virtual you
09:43
So we have implemented block devices Implementations as well as the network interface Implementation and we also have experimentally implemented the bar to IO bar file system implementation which can be exposed as a 9p file system to the
10:03
to the driver layer so this can be almost explained in the previous slide, but Linux kernel 3
10:22
And the side component is application interface which is located and on top of the the kernel implementation So with LKL it exports our own system called API which is called LKL system calls But this system call is not compatible as I mentioned before so we provided
10:43
the various way to access this Interface by from the application So for the first API is the LKL system called low API So if you have an application and if you want to use LKL
11:00
you have to use you have to rewrite the system called part of the application by replacing the symbols from the socket to the LKL system prefix one prefix one So this API is kind of it's slightly defined from the typical standard
11:20
projects API because this is the entry points of the kernel, Linux kernel, so the error number error number and the return values slightly different from the projects API So another interface that we provided right now is the which called so called the hijack library
11:42
Which is basically based on the dynamic translation on the runtime by LD preload So if you have a socket If you have a projects API Application and you don't want to rewrite the application You don't want to rebuild your application. Your binary can be translated with this additional library
12:04
But it has a limitation that Some of the standard library implementation makes some of the symbols invisible from the application side So you cannot rewrite such a symbol. So in that case your application may not work well
12:24
So another API is our own standard libc implementation right now we use the muscle libc as a standard library implementation and we call it this muscle libc as
12:41
To be able to use the LKL from the user space code So I'm gonna show I'm gonna share you some of the use cases that we use is this Linux kernel library because This is library if nobody used this library This software is useless. So I'm gonna present our our known use cases, but if you have I
13:07
Wish to have you I wish you to have a more expanded use cases if you have a nice idea So the first typical use case is to mount a disk image without root privileges
13:24
So some of the operating system Has a experimental implementation of the file system in user space Like if you have a ext4 file system image for the virtual machine and if you want to modify these Images on the following operating system like on Windows or Windows operating system
13:45
You may have to use such an alternative implementation the ext4 which may not complete implement which may not cover the complete specification the ext4 Or you have to use the virtual machine and the boot the Linux OS and the mount the disk image and
14:03
rewrite or modify the contents of the disk images But you don't have to do such a complicated stuff You can just use this library as a Windows application and modify it inside the user space program
14:21
So you can you can now modify the betaFS contents of the file system images in the different operating system So another use cases is trying to introduce the kernel feature in the very restricted environment you which you don't have a freedom to
14:43
Recompose the kernel space implementation So this is an example of the introduction of the master TCP implementation of the on the Android phone By the way, the MPTCP is already upstream in the previous week
15:00
And we don't have to do such a stuff right now But this is a snapshot of the two years ago, I guess and at that time We don't have much for TCP support But you can we can do it because we have a user space implementation of the Linux kernel on the Android phone so another toy
15:21
Implementation is the Unix pipe as a network interface card So you if you have a three different or two different? programs using LKL and each of the program has a network stack implementation and if you write the pockets generated by this kernel
15:42
into the console and Then if the next program receive the cone this pocket by a pipe As a received channel of the network interface card You can do something. I don't know so if you can if you want to
16:01
Make access control by this Pipe communication you may use you can use a grip command in order to filter the specific Payload of the packet with the grip command argument And if you want to duplicate
16:20
The generated packet into the some external program like a TCP dump You can do you can you do with the key command by mirroring the packet into the different processes That's almost it and Another example is that trying to combat the Linux kernel code into the JavaScript program
16:44
since last year, I guess Linux kernel can be built with LLVM and LLVM can generate the JavaScript code with additional trick You can run the Linux kernel code without any emulation like JS Linux does
17:01
but you can directly invoke the Linux kernel code in The browser So this is the initialization task of the Linux kernel in the C code. This code will be translated as Like this Which is with the this is automatically generated by the end script on sources
17:25
And you can run The Linux kernel code on the browser So another use case is they're running the Linux binary code on the different operating system So this demo is actually learning on the Mac OS
17:42
Is the energy implementation with providing this content of the slides? So we have implemented the OCI runtime implementation which coordinate the invocation of the Linux kernel library and try to communicate or interact with the
18:01
Container engine so we only we currently only tested with the container the Implementation as well as our own crafted Docker the Implementation which is running on the Darwin without any virtualization so
18:22
So this is the This is running Docker demo and this is not connected connected to It's not connected The Linux host but running inside of this MacBook natively and
18:45
You can run the Docker command with our own crafted images With anything's command and It is running the Linux boots
19:00
Kernel log with main call of the engines implementation We are sending the additional configuration, but it's not How do I show?
19:27
So there is also some folks who is also using LKL in order to Test a file system implementation with their own implemented puzzle I Have not involved this project, so you can if you are interested in you can try to look at their paper
19:44
Which is listed in the slides? So there is another implementation using an approximation But I'm going I'm running out of time, so I skip this one so We also try to expand the underlying layer facility in order to integrate other implementation like a solo 5
20:04
implementation We have also started the upstreaming this code into the Linux mainline So we actually try to upstream several time in the past, but now we have a suggestion from the maintainer Which we should work with
20:20
Work together with the user model in next so we are trying to integrate this code into the user mode Linux implementation as a different execution mode Is the library? So That's almost it. So if you have a program and if you can link this library as a
20:43
Link flag you may I wish you will have a another benefit that you cannot see in the past Thanks for much. Thanks so much for your attention, and I'm happy to take a question
21:15
I
21:25
Didn't bring the performance results today But the characteristic of the performance is almost similar to the user space networks Which can which we can eliminate user space kernel interaction So we will have a benefit of the user space execution like what DPDK can guys now can do
21:50
Thank you