The Linux Kernel: We have to finish this thing one day ;)
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 490 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/46958 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Kernel (computing)NeuroinformatikLetterpress printingKernel (computing)Series (mathematics)LoginMultiplication signSoftware developerPersonal digital assistantEvent horizonData miningRow (database)Roundness (object)Self-organizationOpen sourceComputer animation
02:03
TrailLevel (video gaming)Self-organizationRoundness (object)Endliche ModelltheorieSocial classMereologyLevel (video gaming)Multiplication signComputer animation
02:42
FingerprintGraphical user interfaceLink (knot theory)Position operatorArc (geometry)Limit (category theory)CountingOptical disc driveKernel (computing)Computer animation
03:22
Device driverComputerBefehlsprozessorNumberSystem programmingNormed vector spaceDevice driverPhysical systemNumberNeuroinformatikBefehlsprozessorCore dumpCoprocessorNormal (geometry)SmartphoneSinc functionCausalityComputer animation
03:59
Kernel (computing)CodeCore dumpBefehlsprozessorScale (map)Scaling (geometry)Kernel (computing)CodeMultiplication signForcing (mathematics)Core dumpBefehlsprozessorComputer programmingParallel portSession Initiation ProtocolContent (media)Scaling (geometry)Software developerEmailComputer animation
04:42
Content (media)Programmable read-only memoryKernel (computing)System callKernel (computing)Physical lawSoftwareComputer animation
05:20
MathematicsOvalKernel (computing)FAQTask (computing)Software developerKernel (computing)Multiplication signWebsiteMedical imagingTraffic reportingScalabilityScaling (geometry)OracleForcing (mathematics)Software developerGame theoryOptical disc driveDevice driverComputer animation
06:15
ScalabilityBefehlsprozessorSystem programmingChemical equationScheduling (computing)Core dumpScaling (geometry)ScalabilityMultiplication signMathematical optimizationData managementObservational studyMemory managementKernel (computing)Structural loadCore dumpScheduling (computing)WritingMathematicsBitSoftware developerComputer animation
06:54
Scaling (geometry)Kernel (computing)WritingDifferent (Kate Ryan album)AreaLocal ringInsertion lossKernel (computing)VirtualizationSoftware developerMetropolitan area networkComputer animation
07:34
VirtualizationBefehlsprozessorCurve fittingKernel (computing)VirtualizationProcess (computing)Computer animation
08:18
Kernel (computing)BefehlsprozessorNetwork topologyKernel (computing)BitVirtualizationGoodness of fitBefehlsprozessorComputer animation
08:59
Point cloudGame theoryGame theoryOnline helpDot productRule of inferencePoint cloudDivisorProof theoryComputer animation
09:36
DivisorCodeControl flowSoftware developerDivisorSource codeCodeGame controllerRule of inferenceSoftware developerCartesian coordinate systemSoftwareComputer animation
10:22
Kernel (computing)Computer networkPlane (geometry)Software developerDatenpfadNetwork socketProof theorySoftwareKernel (computing)Software developerSuite (music)Computer programmingMessage passingNetwork socketExpressionGame controllerCASE <Informatik>Right angleComputer animation
11:10
Similarity (geometry)Proxy serverSynchronizationOverhead (computing)Computer programmingDatabaseKernel (computing)Data storage deviceException handlingWindowNormal (geometry)Proxy serverBitSimilarity (geometry)Ring (mathematics)System callOverhead (computing)Computer animation
11:58
FreewareTime zoneMultiplication signKernel (computing)VirtualizationMultilaterationBitComputer animation
12:37
Time zoneKernel (computing)Kernel (computing)Similarity (geometry)Ocean currentBuildingCAN busMultiplication signNamespaceGroup actionSemiconductor memoryGame controllerProcess (computing)BefehlsprozessorStaff (military)VirtualizationDifferent (Kate Ryan album)CASE <Informatik>Computer animation
13:31
Compilation albumPersonal digital assistantForceComputerPatch (Unix)CASE <Informatik>Different (Kate Ryan album)2 (number)MathematicsBitPatch (Unix)Kernel (computing)State of matterDistanceComputer animation
14:32
Canonical ensembleProcess (computing)BuildingBitoutputComputer animation
15:09
Kernel (computing)SurfaceOperating systemKernel (computing)ForceForm (programming)Multiplication signRight angleMathematicsComputer animation
15:53
Digital filterKernel (computing)ComputerJava appletClassical physicsComputer programmingNeuroinformatikVirtual machineProcess (computing)WeightJava appletKernel (computing)Core dumpComputer animation
16:46
Computer programAsynchronous Transfer ModeKernel (computing)Software developerMultiplication signSoftwareComputer animation
17:42
Computer networkKernel (computing)Content (media)InformationCodeChainSystem callMilitary operationSoftwareSoftware developerRoutingExtension (kinesiology)Kernel (computing)CASE <Informatik>Multiplication signComa BerenicesRevision controlQuicksortAreaComputer animation
18:32
Kernel (computing)Revision controlMikrokernelComputerKernel (computing)MikrokernelEndliche ModelltheorieUltraviolet photoelectron spectroscopyNeuroinformatikGame theoryComputer animation
19:11
WhiteboardDigital filterKernel (computing)Linear regressionVideo trackingElectronic mailing listExecution unitHill differential equationDevice driverPointer (computer programming)Duality (mathematics)Source codeWritingKernel (computing)Traffic reportingMultiplication signLinear regressionTrailPairwise comparisonMikrokernelComputer animation
19:49
Asynchronous Transfer ModeCodeConvex hullElectric currentComputer programContent (media)Execution unitMikrokernelError messageSurface of revolutionSoftware developerSystem callCore dumpSurface of revolutionMikrokernelComputer animation
20:37
AreaError messageComputer programmingPhysical systemLatent heatAreaSimilarity (geometry)TorusComputer animation
21:20
WebsiteCompilerTracing (software)Expert systemTwitterWebsiteComputer animation
22:03
Execution unitMetropolitan area networkGamma functionKernel (computing)Time evolutionBlock (periodic table)BuildingLatent heatKernel (computing)EvolutePoint (geometry)Block (periodic table)Computer virusBuildingSoftware developerTracing (software)MereologyComputer animation
22:44
Axiom of choiceNetwork topologyOperating systemBuildingComputer clusterComputer animation
23:27
PredictabilityControl flowComputer programmingMultiplication signLine (geometry)RoboticsEvent horizonCASE <Informatik>System callPatch (Unix)Digital photographyComputer animation
24:05
Software developerMereologyKernel (computing)Real-time operating systemTorvalds, LinusWebsiteSoftwareGoodness of fitSlide rulePatch (Unix)MassBitMultiplication signLink (knot theory)Family
25:18
Software developerScalabilitySystem programmingSet (mathematics)Patch (Unix)Kernel (computing)Physical systemSoftware developerScalabilityScaling (geometry)Real-time operating systemComputer animation
25:56
Patch (Unix)Online helpRoutingSoftware developerProjective planeComputer animation
26:40
Revision controlComputer configurationReal-time operating systemCASE <Informatik>Projective planeMultiplication signLetterpress printingPatch (Unix)Configuration spaceKernel (computing)Message passingElectronic mailing listEmailSoftware testingComputer animation
27:26
Kernel (computing)Software developerSummierbarkeitMetropolitan area networkLattice (order)Computer animationSource code
28:21
Goodness of fitMultiplication signCommitment schemePatch (Unix)Real-time operating systemArithmetic progressionBitComputer animation
28:59
Software developerTime zoneSoftware developerSpecial unitary groupLatent heatConcordance (publishing)Time zoneState of matterComputer animation
29:40
Physical systemSoftware developerKernel (computing)File systemAreaPhysical systemComputer animation
30:22
RAIDCloningHill differential equationAreaKernel (computing)Computer iconMultiplication signSet (mathematics)Computer animation
31:09
Complex (psychology)Multiplication signKernel (computing)AreaSoftware developerComputer animation
31:48
Complex (psychology)FacebookOracleExpected valueOracleFacebookOnline helpEvent horizonExpected valueFile systemComputer animation
32:33
Control flowInclusion mapInclusion mapGame controllerExpected valueMetropolitan area networkMatching (graph theory)MereologySoftware testingSet (mathematics)Direction (geometry)Field (computer science)Computer animation
33:16
Kernel (computing)Source codeEmailElectronic mailing listSoftware crackingPatch (Unix)Software developerPhase transitionNetwork topologyBitSoftware developerKernel (computing)Multiplication signSource codePatch (Unix)Electronic mailing listTrailPhase transitionPhysical lawEmailCentralizer and normalizerStability theoryPredictabilitySeries (mathematics)Device driverDatabaseDevice driverWeb-DesignerComputer animation
34:09
Device driverComputer hardwareDatabaseSoftware developerPhysical systemRevision controlControl flowComputer hardwareDatabaseWeb-DesignerPhysical systemRevision controlControl system1 (number)Software developerRoundness (object)Projective planeKernel (computing)Computer animation
34:51
Kernel (computing)Data modelSoftware developerKernel (computing)Optical disc driveMathematicsProjective planeUniform boundedness principleWebsiteRevision controlCycle (graph theory)PredictabilityComputer animation
35:36
Web browserNumbering schemeKernel (computing)Kernel (computing)Cone penetration testGraphical user interfaceEndliche ModelltheorieRevision controlWeb browserElectronic mailing listTerm (mathematics)EmailLink (knot theory)Stability theoryVotingAuthorizationComputer animation
36:16
Electronic mailing listPersonal digital assistantSoftware developerAutomationInformation securityCodeElectronic mailing listEmailSoftware developerCuboidCASE <Informatik>Office suiteMultiplication signSoftware bugInformation securityKernel (computing)CodeSoftware maintenanceComputer animation
37:00
CodeSoftware developerSoftware developerMultiplication signPerturbation theoryWeightBitNP-hardComputer animation
37:42
Slide rulePatch (Unix)Software developerRevision controlConsistencyTelecommunicationLink (knot theory)MIDIContent (media)FAQKernel (computing)Programmable read-only memoryLocal GroupContrast (vision)Context awarenessLine (geometry)Thread (computing)Software developerGroup actionSqueeze theoremKernel (computing)Patch (Unix)Instance (computer science)Computer animation
38:22
Open setStack (abstract data type)Multiplication signSoftware developerOnline helpBitHierarchyArithmetic meanEndliche ModelltheorieRoundness (object)Kernel (computing)Computer animation
39:00
Revision controlKernel (computing)Insertion lossLine (geometry)HierarchyKernel (computing)Revision controlNumberPoint (geometry)Line (geometry)Patch (Unix)WordHierarchyVolume (thermodynamics)Computer animation
39:42
Perturbation theoryKernel (computing)Patch (Unix)EmailMessage passingMaxima and minimaLatin squarePatch (Unix)Volume (thermodynamics)Kernel (computing)NumberSoftware developerVotingComputer animation
40:34
Software developerBlock (periodic table)BuildingProcess (computing)Image resolutionSoftware developerBuildingBlock (periodic table)Process (computing)BitResultantDivision (mathematics)Computer animation
41:21
WhiteboardMultiplication signComputer animation
42:03
Real-time operating systemAreaSoftware developerNumbering schemeComputer animation
42:41
Physical systemSoftware developerNumbering schemeSlide ruleNumbering schemeFile systemGoodness of fitSoftware developerSet (mathematics)FreewareOpen sourceSoftwareWordCASE <Informatik>Multiplication signBitCausalityComputer clusterPlanningMoment (mathematics)Point (geometry)Quantum computerElectronic mailing listTrailQuantumDecision theoryShift operatorSlide ruleRight angleAreaOperating systemCodeRevision controlGroup actionLine (geometry)MereologyComputer hardwareDisk read-and-write headKernel (computing)Computer animation
47:24
Overhead (computing)BitFeedbackSoftware testingBefehlsprozessorPhysical systemLine (geometry)Server (computing)Multiplication signDifferent (Kate Ryan album)Group actionFacebookHypermediaEvent horizonPoint (geometry)Endliche ModelltheorieService (economics)Limit (category theory)QuicksortLevel (video gaming)Lecture/Conference
49:46
Point cloudFacebookOpen source
Transcript: English(auto-generated)
00:05
on your phone turn off the sounds on your phone you probably bought like 14 other things attached to the Wi-Fi which is why I'm on 3G right now thanks so much you should probably turn those on silent as well this will be video recorded I can't say videotaped anymore even though that's how I
00:22
learned how to do things this beard is gray so we only have this working microphone because it's an all-volunteer event please donate so during the Q&A session you'll have to raise your hand speak clearly without the assistance of modern technology the speaker will repeat that for the
00:41
recording for all the streaming people who just got lost and decided to stop at a Starbucks and connect to the Wi-Fi so question repeats answers learning everybody ready well you're not even awake yet that coffee hasn't kicked in is everybody ready thank you yeah you don't have to be excited about me
01:01
you should be excited about our first speaker whose name got removed because somebody wants me to take an update while I am traveling that's rude thirsty Lee moose is a Linux kernel development follower he's been starting since the start of the century turns out he's known for his kernel log column which has over the years covered the features of more than 70 new Linux features in detail as elaborate and respected article
01:24
series is published by his employer the German CT magazines you're as big as computer and tech magazine both online and in print if you don't know what print is ask somebody who's got a beard like mine he used to contribute to the Linux kernel but has put that aside just so he can cover and
01:41
come to conferences like this and talk about open source please give a round of applause for Thorsten Lee moose yeah welcome everybody I haven't used such a mic for quite a while so if I don't hold it right just let me
02:02
know yeah but let's go here it's a 20s first time already and I think I'm when giving a keynote that's a big biggest chance to really give thanks to the organizers and all the volunteers that make this great conferences happen so give a round of applause please yeah thanks for all this work or warning
02:27
this talk is kind of part of the history talk but don't worry it won't be a history class I promise I make sure of that and everything I mentioned is kind of relevant for today and there will be a moral of the story at the end so let's get started the stage one of the first first time actually
02:45
happened in 2001 shortly before that Linux 2.4 had just been released and basically had all the important features needed back then kind of everything needed to conquer the world and bring us where we are today positive support graphical interface it was portable to various arc arcs and the
03:06
performance was good since then a ton of improvements have been much into the Linux kernel and I can only mention if quite a few of them because otherwise this talk would take all day or even two days maybe and one of the things that happened Linux was growing up 2.4 for example would likely not run very
03:26
well on today's computers obviously because lots of drivers will be missing but one of the things that wouldn't work is what would be problematic would be the number of CPU cores in modern systems because that back then uniprocessors were quite a norm if you had an SMP system with two
03:44
processors that was a normal back then but today we have CPUs with 12 or 16 cores actually and even smartphones have four cores and that might be way too much for Linux it was actually SMP capable since June 1996 already but back
04:01
then that was kind of brute force how it was made possible there was a big kernel lock and that's basically that only one CPU core was allowed to to execute kernel code at a time so I had two programs running in parallel and both wanted to call into the corner to do make it do something and they
04:20
had to wait for the other one to finish. That actually was solved with finer-grade locking that followed in 2.2 and 2.4 so the next two or three years and that made Linux scale a lot better already so in the 2.4 days it wasn't that bad anymore but other Unixes were known to scale a lot
04:41
better. The Linux developers actually continued to work on that and by 2.6 Linux had got a thousands of finer graded locks yeah but the big kernel lock was still around desperate even if some people had assumed it would be gone by 2.6 already. I looked it up actually 2.6.6 had about 500 locks
05:03
big kernel lock calls back then and it as it turned out many many more steps were needed and had to be taken and to get rid of the big kernel lock networks acting up here and if you're interested in all those details which
05:23
had to be done you can look some of them up on on LWN not which is a great site that reports about the happenings in the Linux kernel all the time as you can see during between 2004 and 2011 when the big kernel lock was removed there were lots of articles that explained where all the problems were
05:42
yeah and as I mentioned it Linux got rid of this big kernel lock that was a problem for scalability in 2012 so basically after about 11 years and that was possible thanks to heroic efforts by various developers especially we have to thank Art Bergman for that who actually took care of the last few
06:01
steps that are really really a lot of work without much gain maybe and yeah it was quite a game but it was really hard doing this this last things because they weren't odd drivers and things like that but scalability since then actually remains something being worked on if you look at LWN that again
06:23
there are many many more articles over the past few years that were about scalability and many small improvements happened over time it's kind of a never-ending story like recently there were some memory management optimizations there was a new schedule load balancing core that got merged into the Linux 5.5 kernel which is the latest right now and many other
06:44
things yeah and most people most of you that are not kernel developers don't notice any of this because all those changes happen basically under the radar but thanks to this Linux is and stays one of the best scaling OS kernels these days maybe it's even the best I was a bit
07:01
careful in the writing that so I went for one of the best maybe I was over overly careful there yeah that's one of the things that happened in various small steps during this past two decades and and getting rid of the kernel lock was really one of the big first achievements with that small
07:23
steps that happened and that was possible because and that was something everybody worked together on it's kind of common goal but more often they some competition between the kernel developers and and one of the thing that Linux kernel was actually still missing in the early first and days was
07:40
built in virtualization capabilities and that's not unusual because virtualization on x86 Linux was not very popular or not famous back then that it only happened during the mid-2000s with then some of you might remember getting popular in 2005 and x86 processes actually starting
08:01
getting virtualization capabilities back then and then back then actually looked like the obvious and fitting solution for the Linux world there was no no competition there everybody thought that would be what we what will make that what everybody would be using there was just one problem support for
08:21
it for runnings for using Zen as a host or a guest with Linux was out of tree but a lot of people didn't care that much and the other problem was Zen was actually kind of a kernel underneath the Linux kernel then suddenly out of the blue in October 2006 KVM came much already half a year later
08:41
into Linux kernel because it was quite small it was quite small and being compared with Zen actually didn't offer that good kind of feature set the performance was worse it had less features and actually required a CPU of virtualization capabilities some sometimes it looked a bit like a toy but in the end it turned out it was quickly improved because various people
09:05
and companies found interest in it and help with little steps making KVM better and as some of you might know in the end KVM turned out to be game gamechaser today it's basically used everywhere these days and actually one of the cornerstones for what it made Linux rule the cloud Zen is still
09:23
around and the support for running as host and guest with Linux actually got only matched five years later in the 2.0 days and these days Zen is quite small when compared to KVM so why did KVM succeed yeah one of the reasons
09:41
that might be a factor might be that Zen sources didn't get the code upstream fast enough but I think the real reason is KVM had a better more flexible and future-proof design and actually one that allowed Linux to stay in control because it was built into Linux not underneath it and that's obviously suited Linux more and it's that developers so they become became
10:03
interested in it helped making it better and yeah that's how it ended and that's why KVM rules the world these days and yeah that's most kind of history lessons history lesson but on the other hand that's things like that happen every day every now and then these days also for example recently
10:24
there was a DBDK that's a technique to make network packages go straight into userland applications and bypass the Linux kernel that obviously didn't suit the Linux kernel developers much so they started to fight back and worked on solution the Express data pass and XDP and something built upon
10:45
that's called a XDP socket where userland programs can can immediately get those packages so the kernel stays in control and it seems those two solutions actually can keep up with DBDK these days so it's likely I'd say if I
11:01
would bet I bet on these two solutions and I think they will gain territory back from DBDK because it's better integrated in Linux another example is asynchronous IO so the program can do something else when it reads or writes makes a kernel read or write data from some storage device that's normal in the
11:23
Windows world and but unusual in Linux except for maybe great big databases but these days something a solution finally shows up that's called IO U-ring it was merged into a kernel a bit more than one year ago and it's an answer to a similar bypass technique that's called SBDK and yeah and it's
11:43
actually will help getting the performance out of these new SSDs because they are really quite quite fast and to make make the get the performance out of this such an asynchronous IO solution is needed because the syscall overhead is way too much otherwise and the thing is just as KVM both these solutions got imported into the Linux kernel and started
12:04
quite small and then got improved in small steps by various people that found them interesting and it shows it that things like that had them all the time virtualization was not the only thing the only solution to make Linux a host for something there was something else and that Linux missed during the
12:24
early foster days there was support for containers and that's actually quite interesting because other Unixes back then supported them already free BSD for example did and Solaris learned it a little bit later for Linux it only became famous in 2014 and a lot of people sometimes wonder why did it take
12:44
so long yeah I took so long basically because the kernel lacked features to build something similar with Linux and yeah that's those features had to be built but that happened one step at a time it took years some of these features actually we use today for containers were
13:03
actually built for containers and docker that like the various namespaces some of those features we use for containers actually not directly one partly built for for container used like the C group stuff the control
13:20
group stuff that make sure one process can't take all the CPU or memory and other staff and that in the beginning actually was used a lot of lot with virtualization with KVM and some of the features we used to build containers these days actually were built for totally different use cases like capabilities and second and a few other things and the thing that made
13:41
docker popular in the end was actually docker combined these and a few other features in a new and more attractive way and that made Linux containers suddenly suddenly quite popular and in the end kind of changed the world funny detail there Alex C was designed to become the preferred solution and
14:01
that's a bit older and there were actually two other Linux container solution which which also an open possess that became quite small thanks to all the other things stock abroad and Linux we serve as the other solution that is basically forgotten these days they came early and used out of three pack patches but thanks to the kernel and small steps and
14:21
features you can you combine in interesting ways they docker overran them basically and is now the leading thing Alex C actually actually is still around definitely not as big as talker and the funniest detail that is just
14:41
imagine if it would have been one company that have invested all the money in bringing making the corner capable of running containers and building Alex C as usual and process on top of it and that actually might have been a pretty bad return of investment for them because yeah other companies can use those features you bought as well yeah and that that really
15:05
makes it a little bit risky for companies to invest that much money but on the other hand that's a problem for the kernel Linux the operating system actually got a way better and more flexible solution thanks to this and thanks to all the various small steps that were taken and what are the
15:25
reason why we have so many small features that docker could combine in a new and more attractive way doc already was quite unexpected but welcome surprise that's not the only time Linux kind of stuff and kind of features show up that's nobody really aimed for and one of those things
15:45
actually is changing the kernel these days and Linux basically is on a trip into the unknown right now what I'm talking about is an improved Berkeley package filter that's called BPF for short or these days CBF or classic
16:01
BPF and the improvements began in 2014-15 that's an internal mini VM mini virtual machine but not a like a virtual machine that emulates a different computer more like a Java VM where you can upload in programs to and then the kernel executes executes them that's something for 20 years ago
16:26
TCP dump for example relied on to only get those packages from the car into into user landed in it to see TCP dump the user was actually interested in that's needed because needed to for performance reason because coping
16:41
everything over simply is so much work it will slow everything way too much down this improved CBF actually got called got called eBPF you might have heard of this thing some people even call it just BPF for short I'm kind of angry with the developers there because now we have an old BPF
17:01
that's called BPF and a new BPF and nobody knows what what somebody's talking about yeah that's really sometimes annoying but most of the time if you read BPF these days it's actually the enhanced one that's meant yeah it became a really fast and much more powerful VM to run programs in
17:22
that's actually if you had suggested something like that to learn a store was 20 years ago I guess the idea would immediately have been shut down because it's kind of crazy but this days it worked and one of the reasons why it's work is actually it's got improved a small thing and then improved and
17:41
improved again and that way everybody could be sure that's not dangerous or something the network developers actually built this eBPF to scratch some interest they had and improved it and approved it again this xtp stuff I mentioned earlier is actually built upon this and really relies on it but
18:01
these days other kernel subsystems started to use it as well or will soon and more and more seem seem seem to be interesting actually on lwn net it's became kind of a running gets gag that some doing the articles there's often the case where some some solution is sort of complicated problem
18:22
and then often be BPF is suggested to use it it won't be used all the time but it seems it's getting into the kernel at various various areas and yeah eBPF is still not done I mean it's got developed over the past few year but it's improved more and more and starts to change the kernel
18:43
fundamentally and makes Linux actually gets more aspects of a microkernel this thing what some of you might remember this big debate of microkernels is a better design than the stuff that the model that Linux currently use which is more like a monolithic kernel but as I said Linux
19:04
gains more aspects of a microkernel and that's actually what Europe's biggest computer magazine wrote the German city magazine it looks like that and if some of you now see you see pump people giggle next to you to you that's because yeah that's what because I wrote that yeah maybe I should
19:25
try that on Wikipedia write something somewhere and then change and give myself a source yeah but I'm not only a popular fault for this kernel reporting I do that for 15 years now I'm also did a few things in my spare time for kind of regression tracking and that I was actually invited for the
19:42
current summit and so I know what I'm talking about and I'm not the only one that actually wrote this microkernel comparison I'll be lwn that actually mentioned it as well and there's another another running joke yeah you laugh but this one of the core developers that made a lot of things
20:03
happening you we all use every day sooner it's we laugh these days but maybe it will happen it maybe we stand here in 10 years and say yeah that's how it all began and there's also an another improvement that's coming where
20:20
the microkernel aspect was mentioned that's really fascinating to watch the rains to be seen what comes out of it maybe it's a we're at the beginning already in the middle of a small revolution that makes Linux more error resistant flexible and more powerful it remains to be seen thing is most people actually don't notice because it's happening in a lot of small steps and
20:42
without disrupting old features so that can can can be you don't have to care if you're not interested in that yeah long-standing wishes that's a different topic another area where Linux actually was behind on the in the early for some days was a proper tracing solution to look into the system or
21:04
specific program to look why the system is slow or why the program is slow and the famous solution in this area is actually D trace actually published in 2005 and built into Solaris and people for years wanted something similar with Linux and they got some things recently that's
21:22
called BBF compiler collection BCC for short and BBF trace and those are actually called D trace 2.0 these days by Brent and Greg which is one of the leading experts on one D trace at least according to Wikipedia and no I didn't write that there yeah and actually those two new solutions can the
21:45
2.0 is appropriate because BBF trace and BCC actually can do more than the old BCC more than D trace it's pretty cool if you want to know more about this look at Brent's website I mentioned it here and he also
22:03
published a book recently that's quite cool and gives a lot of details and into what you can do with modern kernels and and and tracing them and yeah it took basically like 10 or 15 years to get everything into the kernel and the funny thing is that actually happened without a design that
22:24
had something like BCC or BBF trace in mind they thanks they much thanks to evolution because the kernel developers actually built various building blocks over the past 10 or 15 years sometimes with smaller or different goals you have the tools like F trace, trace points, perf, k-probes and all those
22:42
things and then that those were one part of the solution the other part is eBPF there it is again and yeah then somebody combined those tool and and made new things possible and then actually BCC and BPF trace came out by
23:00
some people and tata yeah that's how we have D trace 2.0 these days really great how things work out and without a design like you would have normally when you're building an operating system yeah something impossible Linux will soon offer a really great and important new feature
23:21
one almost and one almost nobody would have expect during the early for stem days it will be real-time capable so it will be able to use Linux for your laser cutter or for your robot and your industry line that manufactures cars or whatever because it's kind of Linux can make sure that
23:43
program that's controlling your laser cutter is always called in time to react to certain events and that's really important for this use case yeah back during the olive early for stem days that was actually nothing people talked
24:02
about yet but it was an idea actually in some people's mind up minds already especially in the mind of Thomas Glaxner who's one of the leading or the leading developer for the real time kind of patches and he actually recently last fall on the Linux plumbers conference gave a great talk
24:20
about this the URL is here I also you don't have to write this down I will upload the slides to the website I try to do that before right before the the talk but the network was overloaded so you have to wait for that sorry but it's a great talk where he's looking back and mentions a few nightmares all this development gave him and one part of the talk is
24:43
actually he gave mentioned a few quotes like real-time people are crazy and not going ever to get merged into the Linux kernel and things like that and for I know I spoil the talk a little bit yeah but these quotes actually are all from Linus Torvalds himself that was most of them were
25:05
from 2005 from a great or 2004 great great debate of if we are making real time Linux real-time capable actually is a good idea yeah but developers actually didn't give up and they basically started working on it having
25:23
this external patch set and got small steps into small patches and small and into the mainline kernel and yeah that actually made Linux better for all of us even if we don't use real-time systems or we don't need them and the real-time patches hit quite a lot of problems and scale scalability issues
25:43
first and that in the end they fix them and that in the end made Linux better for all of us and yeah the thing is the RT developers actually had a lot of body blows and one of the worst was basically like five or six years ago they had basically like 90 to 90 percent of the route done or that
26:04
were needed to get make make Linux real-time capable but they needed basically needed money for the last few last mile on the last five years as it turned out yeah and the thing is that was needed because lots of company
26:20
companies that used the RT patches back then actually didn't help much with development and but lucky luckily the people were successful they came to the Linux Foundation and talk to them and they actually founded a project with a few companies in 2015 and yeah thanks to this project the main trip will
26:43
soon be finished because the real-time capabilities are the biggest the most important patch actually is in the Linux kernel already the config option to enable the real-time support it's not exposed yet because a few things missing the biggest thing is actually a rework for the print case stuff
27:01
that's actually which what's doing your logging which you can see with D message and that's but that's in the work and there was some disagreements but they were settled really recently and the patch was the new patch to make everything happen was sent to the Linux kernel mailing list for review recently so it's likely this will get into the the kernel this year's and
27:24
then make Linux real-time capable describing all the steps the real-time developers had to take was what take also a day or two maybe because if you go to LW and not and actually look what what problems were discussed over those years you you have many articles you can read and but it shows
27:46
even crazy goals that look unreachable can be achieved in small steps and yeah the thing is that's actually how most of the Linux features evolve as they often not designed by some some company or some some in a meeting or
28:04
something often it are simply individuals that want to realize an idea or a dream or and make Linux do something and sometimes they even abused companies to realize their ideas that happens they simply look for companies that might be willing to pay them and some of the developers sometimes have
28:23
to find other places to to find money but it shows really with a good idea and commitment and even big and crazy dreams dreams can be realized in real time patches actually are a really good example for that yeah I mentioned a
28:41
times that Linux is working differently a little bit and nevertheless it learned a lot of things but it took them quite quite long to get those features realized but that's just how the Linux world is because you can't
29:00
simply can't just hire 50 developers make them work for two or three years to build a specific feature like Sun for example could do of zones because if you do that there's a really the risk that after two or three years you get to the current developers and they say no we don't want that
29:21
that's way too big and you're doing this and this one wrong and because they want to see these small step incremental steps because that work quite great for the Linux column and that actually means more work for for the companies if you want to realize something yeah it's after develop
29:42
kernel developers really well often lead to the best solution on the market but it has disadvantages too so in now I'm going to check who's awake whose hands up and leave it open leave it up if you're awake yeah yeah so leave it leave it leave it up if you agree with standing here and it's
30:04
zfs actually the most sophisticated file system in the Unix world yeah yeah quite quite a few hands went down but about not really 50% but but close
30:20
with some people think it is but zfs and the Linux file systems actually one of those areas where a lot of people say Linux is still not not the best kernel for that because zfs is better and the funny thing is there were work on zfs for Linux actually actually was started in 2008 it's
30:45
called butterfess I guess most of you have heard about this but as most of you will likely know it's doesn't have hasn't reached that goal and yet and it doesn't look like it will be anytime soon if you go to Wikipedia we actually see a few features that are called unstable unstable on and a few
31:04
features that were announced or that set of s already had that are not even implemented yet so the big question is what took them so long yeah one other thing for sure it was overhyped and just like all the other features butterfess
31:23
was merged into the kernel when it was still quite small and then improved in various little small steps and that take take a lot of time as I showed with the other examples I gave earlier and it also shows how quick things improve and complicated areas mainly depends on how complex the area
31:42
is the problem actually is you will try to solve and how many individuals or companies back to the development and it turned out the problem scope here is really really complex and company a lot of companies didn't care too much some companies actually helped like Oracle's who's a Facebook and few others but some don't care much and didn't help no complain that's how
32:06
it thinks sometimes is are in the world so the big question is will Linux get something to compete with ZFS I'm pretty sure sooner or later it will the examples I gave really show that it might just take 10 more years
32:21
maybe just five but maybe 15 who knows we will see just to mention a few recent events there was this is file system because a lot of people have high expectations there I'd say be wait and see and keep your expectations under control to not create another hype because history shows it's a hard
32:43
problem the history of butterfess actually shows it's a hard problem that takes a lot of effort and because of s right now is basically a one-man show and not even submitted to mainland inclusion yet so it's unlikely to fly soon even if it got much companies would need to have a lot of testing
33:01
and testing in a field before this really becomes stable so that that will take a while if things really develop in this direction maybe in the end it turns out that but oh that's a butterfess gets improved and that becomes a set of s for Linux in the end just as planned that's nobody knows that yeah I talked a lot about features already but let's switch gears a little bit and
33:23
talk about Linux kernel development how the kernel is developed itself because during the early first time first time days the Linux kernel development meant looked really odd to outsiders there was no central development forge like the source forge gitlab or github development was
33:42
actually totally driven by email there were dozens of mailing lists no tracker for patch submissions no central issue tracker neither for developers nor users there were long unstable development phases there were so new features that got built and integrated into an unstable series sit there long and
34:04
reached users only after two or three years that was made a lot of people unhappy there was no predictable release cadence no driver data database where you could look up if your hardware support and how good and actually we had an overworked lead developer and because we didn't one of
34:23
the reasons why he was kind of overworked and we didn't even have a version control system back then for the younger ones in this round yes 20 years ago we had version control systems already most people most projects actually use them for the older ones yes those were CVS and SVN
34:44
and maybe those were the Middle Ages back then I don't know but git really made things a lot better yeah but then back to kernel development there were way more odd aspects for kernel development and the approach
35:02
actually improved quite somewhat since then like we have gets since 2005 and let me change the world really to the better thanks Linus it's the second project that quite made the world change and now I have your strange
35:20
pop-ups here that's blocking me my site yeah and we actually got a predictable release cadence release cycle since 2005 we basically get new releases from the Linux kernel every nine or ten weeks so it doesn't take that long to get a feature out to the users and this approach where every new version brings new features actually was called crazy by a lot of people
35:43
when when the Linux kernel switch to it but it turned out very well and actually browsers actually later picked it up so Firefox and Chrome we all are used to this this model that's basically whether Linux kernel build the has to actually do that we also got stable and long-term kernels that have
36:02
supported for longer like the latest lot of those long-term cones are supported for six years these days but to be honest many of the odd things I mentioned a few minutes ago are still around and some even got worse though so these days we don't have dozens of mailing lists we have hundreds of mailing lists and development is actually still driven by email there's
36:26
not there is a bugzilla but the thing is we can report issues but the thing is there are lots of developers don't look there because in most cases it's not the office official way to report bugs just a hint for those that want to report bugs to the kernel developers the proper place and most of the time
36:43
is a mailing list security also became much more important since then but still we have no automatic code checking in a central place most of the subsystem maintainers use something and the developer some of the developers also but nothing central here a lot of room for improvements here and on some
37:03
of those things actually is someone is working on them already and there's always the idea why not switch to a central forge like github and github for development because you would get a lot of things for free them but no that won't happen any time soon because just like with features the
37:22
developers demand that things are improved in small steps here too because yeah that's how it works well for them and yeah but that's really need someone that's motivated enough to do that without an immediately return of investment and that makes it sometimes a little bit hard
37:41
and that's why some of those things are still kind of archaic with Linux development it becomes more and more of a problem there was also recently on a plumbers conference and fall talk about it with what problems a lot of developers have to deal with these days thanks to that workflow group was
38:02
actually started and already got to work so a few improvements are coming there's now a Garrett instance which developers can use to submit patches to Linux kernel and went to be seen how the other how many of kernel developers will start to use this and how if they liked it but but as you can see it's
38:25
development is improved with small small steps just like with features and yeah that will take some time sometimes people ask and should the why can't the Linux foundation help you more maybe it should help a little bit more but I'm not sure how much because the Linux development model model really
38:46
worked well and I don't think it would be a good idea to basically organize them like OpenStack or Kubernetes are developed with lots of committees and hierarchy and things like that yeah that might not be the best for the Linux kernel nevertheless Linux kernel development meanwhile runs at the usual
39:05
pace every we get new kernel versions every nine or ten weeks for many many years now if you're interested in a few few numbers we are actually each of those versions brings about thirteen thousand five hundred commits sometimes
39:21
it's a thousand or two thousand more or less and all of those bring about three hundred thousand new lines every version so about every year the kind of gross about 1.5 million lines per year and that actually happens about 15 years after Andrew Martin who was back then number two and a
39:41
hierarchy wrote this and famous last words but the actual patch volume has to rub off one day as I said that was 15 years ago it didn't back then we had like I think six or eight thousand pet comets every kernel release as I said these days we often have 13,500 and the Latin later number is
40:03
actually stable these days so the patch volume is quite constant down yeah but that was not the only thing Andrew Martin wrote back then he also said we have to finish this thing one day yeah I don't think that will ever happen I guess Linux might be forgotten forgotten maybe in hundred years and because all
40:26
developers might work on something else or we still use it in hundred years who knows remains to be seen so now I'm coming to the end slowly summing things up now the Linux developers really solve big problems and small
40:43
steps the big unlock showed that the small steps lead really to better and more flexible solution like KVM did sometimes these small steps actually make groundbreaking new technologies possible like like doctor did the small building blocks that are built in the small steps actually sometimes can can
41:03
even help fulfilling all the wishes like the trace to does it for the wish for tracing solution and this process actually can lead to quite unexpected disrupting results like this BBF thing I mentioned really keep an eye on it it's going to be fun what's coming out of that but thing is that's
41:23
what made and makes Linux so great because those were just a big examples during the past two years at two decades the big features that were developed like that it's basically all the other stuff was was quite similar but it shows it's what makes Linux so great and reaching big and
41:43
chose it you can reach big goals with small steps and even if that takes time yeah and if something takes time it obviously also needs money because we all have to eat in the end of the day and yeah so to realize them that often need someone that's really committed Italy is someone that's an
42:05
individual what that wants to realize a dream and has a dream about a new feature and that drives makes everything happen to realize that in the corner because that's how all those features I mentioned to got developed and the real-time stuff I mentioned is really was a really big and crazy dream
42:24
but even that got realized this way nevertheless in some areas we are still not there yet and to improve things you have to basically become an individual that is committed and find money to get this dream realized and
42:41
then maybe Linux will get the file system that's as good or even better than set of s and we might get developer tools and schemes that are even better than what what we have these these days just like it was for example or other things that will have a positive and impact on the world or
43:02
the world of free and open source software like Linux and git had and have which just everything else I mentioned is we're just a dream in somebody's head that somebody realized in this case Linux that's it and that's if you're wondering slide 234 all those like I have so many things to say and so many
43:36
things in my head that's why I'm using to keep on track so are there any
43:41
questions we now have to switch mics if you're planning to leave this makes questions a little bit difficult we have about five minutes left for questions if someone has a question if folks leaving the room could leave the
44:03
room as quietly as possible so we can hear the intelligent discourse from the questions in the audience
44:22
to make use of quantum computers and quantum chips that are coming up on quantum computers yes thank you I have no idea I don't think anybody is working on on making Linux run on quantum computers maybe we need new
44:43
operating systems for them remains to be seen maybe somewhere and somewhere some lab somebody part of Linux there but I have no idea by the way if if you're exiting the room could you please do it from the middle or the back they were still trying to actually answer questions at the front of the room at the moment thank you for the talk I wanted to know do you think
45:07
but Linux has gotten faster or less fast during the time during the 20 years sorry I don't understand it's too loud could you maybe come here oh yeah
45:23
yeah oh yeah thanks that's now again do you think that the Linux channel has gotten faster during the year because you know a software grows it's sometimes slower there was many performance enhancements so do you
45:40
think it has gotten like slower or faster on the same hardware oh you mean if if it should be if it should be how it could be faster or what I like during all the time that passed oh you talked about do you think but the
46:01
kernel has gotten faster I it's too loud here sorry I can't understand it okay we're gonna try this again does anybody have a question that hopefully isn't oh did you ask if the Linux kernel was growing fat or what and the
46:24
question is whether Linux gets faster with these updates if even faster release updates when Linux gets faster with all the improvements yeah
46:40
it Linux basically gets faster basically with every new version always in little little steps also is but this is too slow in some some areas or where do you think it's too slow I think the point was with the increased code
47:04
base it might get slower over time if can you repeat it with the increased code base over time it might get slower because you have too many lines you mean because it's getting slower because it's so big no not necessarily right now but can it be in the future yeah the Linux kernel is quite modular so
47:24
you can build everything into it what you want but sure some of those features have some overhead that make Linux maybe slower and that's actually likely would become this overhead would be a problem if you would use a modern
47:42
Linux on a quite old system but on the other hand you can configure Linux quite quite modular and I mean it's used on a lot of embedded systems so it shows it can be made quite small and at the same time also work on quite big servers with hundreds of CPUs this is our is our last question last question
48:04
okay my name is Tolik sorry I speak a little bit English I would like to ask about if I understand you're right you said that each year we got 1.5 million lines more just your opinion what's happened in ten years so what
48:29
was it tested it who tested it I can't hear it so the speakers are away from here could you explain me how it's possible to test it at all how to
48:40
Intel doesn't care about commit from IBM or Facebook or another company says tested at all no no I guess nobody tests it all because you can build it in lots of different ways I have a lot of issues what this why they have a
49:01
lot of issues yeah yeah a lot of people try to testing the Linux but you can't test it all and that's also something that really needs to be get better which the workflow group also I think has on its roadmap to make sure the contesting gets better yeah okay I think we are ending this
49:26
if you go and want to give any feedback just talk to me even if you didn't like it yeah and if you want to follow me you can follow me here if you want thanks again and have a great first time