We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

The Linux kernel on dragon wings

00:00

Formal Metadata

Title
The Linux kernel on dragon wings
Subtitle
Compiling the Kernel with LLVM/clang
Title of Series
Number of Parts
199
Author
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
Jan-Simon Möller will introduce the audience to the LLVMLinux project which goal is to compile the Linux kernel with the compiler tools provided by the LLVM project (clang). He will talk about the steps needed to compile the Kernel itself, the issues found during this endeavour and the status of upstreaming the patches to the Kernel and the LLVM project
ConvolutionCodeStudent's t-testLinker (computing)Mathematical optimizationIntermediate languageTerm (mathematics)CompilerMultiplication signVector potentialLevel (video gaming)Projective planeCode refactoringVirtual machineSound effectArmPlanningSocial classMathematical analysisComputer animation
Mathematical analysisPosition operatorStudent's t-testSystem callConvolutionLimit (category theory)Power (physics)Term (mathematics)Projective planeSpacetimeSource code
Bit rateProjective planePlanningComputer architecturePhase transitionMereologyInformationConvolutionFocus (optics)LengthCodeCompilerMetreData structurePatch (Unix)Food energySocial classBoundary value problemTerm (mathematics)Mixed realityArithmetic meanStack (abstract data type)EncryptionVolumenvisualisierungBefehlsprozessorPoint (geometry)Device driverSet (mathematics)Element (mathematics)Variable (mathematics)Process (computing)Heegaard splittingAndroid (robot)SpacetimeCryptographyStandard deviationMultiplicationComputer animation
Multiplication signEuler anglesDifferent (Kate Ryan album)WindowCodePatch (Unix)Real numberControl flowPlanningMathematical optimizationPoint (geometry)Variety (linguistics)Assembly languageEmailVulnerability (computing)AliasingModule (mathematics)Right angleElectronic mailing listData structureWeightSheaf (mathematics)WebsiteDevice driverSummierbarkeitInformationConvolutionSocial classGoodness of fitProjective planeConnected spaceLine (geometry)Standard deviationElement (mathematics)View (database)Link (knot theory)Network topologySet (mathematics)Code refactoringAndroid (robot)MereologyBit rateWhiteboardAttribute grammarVariable (mathematics)Just-in-Time-CompilerLengthComputer fileIsing-ModellComputer animation
ConvolutionDevice driverPoint (geometry)Moment (mathematics)Revision controlBitRight angleSoftware maintenanceError messageCryptographyPatch (Unix)Product (business)Data structureFunction (mathematics)Multiplication signVariable (mathematics)LengthCodeGoogolGroup actionArrow of timeSocial classState of matterExergieComputer animation
Transcript: English(auto-generated)
Okay, last year Yann-Simon Murat gave his talk at 4 o'clock, this year he gave his talk at 3 o'clock, and it will give us an update about how Linux compiles with LLVM.
Let's applaud Yann-Simon. So, the Linux kernel on Dragon Wings, so the, what's our topic here, so who has heard about
LLVM? Raise your hands, oh perfect, who has heard about, or who has used Clang, which is the compiler front end, yeah, okay, so our project is about compiling the kernel with Clang,
so, okay, from the starting as a low level virtual machine and we want to use the compiler but also we want to use things like the code analysis, yeah, checker, and we want
to be able in the end as a goal to use code refactoring. So, why? Well, it's quite some time, if we can speed that up, hey, even better, that's about
compiling, yeah, for the linker we still rely on the GNU-LD, yeah, so there are still two sides of the story, compile times are faster.
We have a lot of potential in Clang and there is still a lot of work and a lot of things which can be done in terms of optimizations, for example, one student last year was trying
to get the LLVM IR, the intermediate representation of the kernel and then he wanted to do some research or apply optimizations or whatever on the intermediate language here, so I think
that has a lot of potential. We also had a GSoC project where one student made sure that the checker, which is the
static analyzer, is able to run against the Linux kernel, yeah, there are a few glitches on the checker side that has been solved and we can now get static analysis runs on the kernel. It works, you can see it on our build bot, it's available just on the checker subfolder,
still there is quite some work to do, well, the checker was first done with kind of user space in mind, yeah, so if you do a checker run against the kernel, you still see quite a few false positives or things that don't apply in terms of the kernel,
but it's a really good tool because it shows you the path in where the issue happens and you wouldn't believe how deep those paths are sometimes in the kernel, yeah, like 36
steps, that's nothing you can find out with just a human review or inspecting the code, yeah, if it descends 36 steps. Also, Clang is, or LLVM is used in a lot of other projects and for example,
RenderScript uses it, it's used, it's meanwhile part in the Android NDK, well, there were some rumors they wanted to compile all of Android with Clang, well, let's see what happens, it certainly has the advantage that in terms of Android,
they can kind of focus on just one compiler, well, let's see, it's part of the Galium 3D, LLVM pipe, drivers and others and if you are interested,
there's a talk tomorrow in the LLVM deaf room from Sylvester, he does a full rebuild of DBN with user space with LLVM or LLVM Linux project, so we want to
compile the kernel for multiple architectures with the LLVM, Clang LLVM toolchain and we want to fix issues which we find on the way, that started out with kind of Clang not supporting whatever nitty-gritty feature that was
used in the kernel and on the other side, there are a few things in the kernel which are kind of horrible and GCC doesn't complain, yeah, no offense, it's just kind of the kernel and GCC, they were always kind of developed,
driving each other and they were kind of pushing the boundaries, so now Clang usually is a kind of more, yeah, goes by the book, yeah, goes really by the standard,
so we find a few issues where, well, the code shouldn't actually work, yeah, still GCC eats it and so we had a few things to fix in LLVM, we are kind of a meter
project, yeah, we collect the patches, we kind of split them up and feed them into the upstream project, that worked out really well, right now with LLVM 3.4, the latest stable release, you need no patches to LLVM, still we need a few
patches to the kernel, yeah, because, well, compiler flags need to be set or the compiler needs to be reset, so there are a few issues in the kernel, what else?
So, for LLVM part, all patches which are needed are upstream, yeah, ongoing in 3.5 and David Woodhouse is doing an awesome job here, we also get rid of one nasty piece
which is code 16 support, yeah, that's for booting the x86 CPUs and that was not present in Clang, meanwhile we have it so we can get the Buddha part solved up to now we still used for just these parts GCC, yeah, but we have that now
and one big thing here which is in the kernel, that is we still have to upstream
a lot of kernel patches, some are small, some are just enabling other compilers to work with the kernel, others they deal with a few, one is for example
what we call flays, don't mix it up with VLA, VLA is perfectly fine so variable length rate is perfectly fine, but variable length rate in the structure is kind of evil and Clang doesn't support it, it's actually even not part of the C standard
still we have such pieces of code flying around especially in crypto and in the net filter stack and regarding crypto the problem is that code gets now copied over in every place of the kernel
like Bluetooth, encryption in Wi-Fi now uses such pieces where you end up having variable length rate in a structure or you kind of nest a structure with
a variable length rate into another structure, boom, that happens a lot and that's a big problem right now, we talked to the Clang guys and they said no, it's nothing we want to support and it's not part of the C standard actually it's evil, last year during the Plumbers conference
we had Linus in the room and after looking at it, it's actually bad but it takes quite some time to figure this out now and get the patches upstream
now just if you happen to work on some code, please make sure you don't use these in new code a difference in Clang is weak alias and section attributes
the kernel uses that when we do the module, driver modules we assign the module in it, once we make this connection, we use a weak alias the problem is GCC makes just this side of the alias inherit all the sections on this side
so the upper element gets all the section and all the annotations from the simple willing to Clang doesn't do that, so that's a difference
right now it's no problem we just apply the necessary attributes at this point over here right now we also do not use the integrated assembler of Clang
it would be possible to compile and link with Clang at once and kind of LTO-ish, apply more optimizations for x86, we still have a few things to sort out, code 16 was one
with that kind of solved, we plan to enable the integrated assembler again and give it a new run for ARM, we have a few issues with the assembly style used in the kernel versus what Clang expects
so if you want to try it out, help us A, tomorrow is another talk in the LLVM Dev Room B, you can try it out, we have a website, link follows
and we have a JIT tree which hosts a set of make files which allows you to pull everything build everything, so you get an x86-64 kernel or what else do we have vExpress as a QEMU target, we have also a few boards and a few Android kernels
kind of Nexus 7 is supported, so you could try that out on your PC or on a real target It's kind of always a moving project because we have two upstreams
so sometimes it breaks because the merge window of the kernel opened or the merge window of LLVM opened, so be aware sometimes the patches do not apply but right now with no patches to LLVM and just the kernel that actually works most of the time
Alright, what else do we have? If you want your target, your board supported, feel free to post a patch on the mailing list It's not that hard and What else? Yeah, contact
Here's the website LLVM.Linuxfoundation.org We have also a mailing list, ISE channel, LLVM Linux on OFTC and all the information is also in the picture over there
Questions? Just a minute Hello, on the beginning of the talk you mentioned code refactoring
Louder please, I can't hear you On the beginning of the talk you mentioned code refactoring What are the goals of the code refactoring use LLVM? Yeah, so the idea is kind of Okay, so for example, when a check-in at Google happens
they compile it for production with 3cc but in parallel with Clang for the debug output and they have it instrumented like that for kind of the most obvious stuff or the kind of recurring errors
they can even find out through Clang what's wrong and kind of propose a patch already So that's kind of what's possible For example, we could kind of do a sparse version whatever
or cochine-like tools with Clang to find and sort of fix issues in the kernel but right now that's just ideas
there's no specific work on that right now Regarding the to-do with the variable length thingies since the LLVM maintainer said no they're not going to support it so the to-do is to remove all of them from the Linux kernel and do you have currently patches patching them out? We have patches for a few things
mostly it's not that hard to move things out of the structure it's kind of just moving it out and accessing it just directly so that's not the point the problem is at the moment it's used in a few crypto bits and right now the maintainers are not in favor of changing it
that probably needs some time and some time to convince them the problem is now that code gets copy and pasted that's our main problem right now
we have a few patches and some went upstream like the USB gadget FS used it and we met the maintainer and said okay, let's just do that in principle, yes, we would have to find and remove all the places where we use Flace
it will not be supported so if there's a driver using it because it was comfortable to use it then it won't work and I need to fix it manually? yep okay okay, thanks
thank you for your talk