Porting Fedora to AArch64
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Alternative Title |
| |
| Title of Series | ||
| Number of Parts | 90 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/40313 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 201328 / 90
2
5
8
10
12
13
14
15
17
19
21
24
25
28
29
31
32
34
36
39
40
43
44
46
50
51
52
54
55
57
58
62
65
66
67
78
79
87
88
00:00
Distribution (mathematics)Goodness of fitComputer animation
00:38
Slide ruleRevision controlEnterprise architecture32-bitDifferent (Kate Ryan album)ArmBitFingerprintLecture/Conference
01:22
Computer-generated imageryArmArmSeries (mathematics)Different (Kate Ryan album)Food energyServer (computing)WikiComputerSpacetimeNumberHypercubeInverter (logic gate)Point (geometry)Real numberCuboidScaling (geometry)Energy conversion efficiencyComputer animation
02:19
Computer hardwareArmSoftwareEnterprise architectureRevision controlMultiplicationArmServer (computing)Projective planeArithmetic meanComputer architectureSoftware developerComputer hardwareBitMultiplication signEnterprise architectureComplex (psychology)Level (video gaming)Revision controlFile formatPoint (geometry)AdditionTerm (mathematics)Demo (music)Computer simulationLecture/ConferenceComputer animation
04:09
Kernel (computing)Enterprise architectureArmRevision controlFingerprintArmCore dumpTerm (mathematics)QuicksortRevision controlRational numberAdditionSet (mathematics)Enterprise architectureThumbnailFingerprintLecture/ConferenceComputer animation
05:00
Revision controlCASE <Informatik>Slide ruleConvolutionBitVirtual machineQuicksortLevel (video gaming)Software testingKernel (computing)Logical constantSemiconductor memoryEnterprise architectureArmMachine codeVarianceFingerprintRight angleLecture/Conference
06:51
Control flowPhysical systemExclusive orReduced instruction set computingClique-widthRead-only memoryDirected setEnterprise architectureSpeichermodellCurvatureKernel (computing)SpacetimeCodierung <Programmierung>CASE <Informatik>32-bitArmEnterprise architectureDirection (geometry)Pointer (computer programming)Computer programmingSemiconductor memoryData storage deviceStructural loadReduced instruction set computingOperator (mathematics)Computer animation
07:36
Semiconductor memoryModel theoryCalculationSpacetimeProcedural programmingTable (information)Address spaceSystem callLecture/Conference
08:17
Control flowPhysical systemExclusive orClique-widthReduced instruction set computingRead-only memoryDirected setStructural loadEnterprise architectureSpeichermodellCurvatureKernel (computing)SpacetimeCodierung <Programmierung>Model theorySemiconductor memoryArmPhysical systemLevel (video gaming)32-bitCASE <Informatik>Computer hardwareDifferent (Kate Ryan album)Enterprise architectureGame controllerCodeFunctional (mathematics)Link (knot theory)Procedural programmingSystem callIntegerBranch (computer science)BefehlsprozessorWordAddress spaceNatural numberComputer animationLecture/Conference
09:37
Control flowPhysical systemExclusive orReduced instruction set computingClique-widthSpeichermodellRead-only memoryDirected setStructural loadEnterprise architectureSpacetimeCodierung <Programmierung>Kernel (computing)Level (video gaming)Exception handlingArchitectureACIDRight angleCore dumpKernel (computing)Revision controlMathematical optimizationSoftware developerComputer architectureQuicksortEnterprise architectureArmComputer animation
10:22
Different (Kate Ryan album)CASE <Informatik>BefehlsprozessorProcess (computing)CoprocessorRegular graphExecution unitGroup actionArmCondition numberBitEnterprise architectureAddress spaceCore dumpGame controllerBootingException handlingLevel (video gaming)MereologyMultiplication signPhysical systemComputer programmingCodeSoftwareDirection (geometry)Software developerKernel (computing)SpacetimeCache (computing)Computer hardwareQuicksortState of matter32-bitAsynchronous Transfer ModeThumbnailDemosceneAssembly languageValidity (statistics)SmartphoneMemory managementServer (computing)Lecture/Conference
14:15
Computer hardwareEnterprise architectureEmulatorModel theorySoftware development kitHardware description languageCore dumpProcess (computing)CodeIterationOpen sourceComputer simulationOpen sourceDifferent (Kate Ryan album)CASE <Informatik>ComputerEnterprise architectureMathematicsProcess (computing)SoftwareArmModeling languagePhysical systemModel theoryComputer hardwareCodeBitSoftware developerMereologyComputer architectureOpen setBit rateDifferenz <Mathematik>SimulationIterationSoftware testingLevel (video gaming)EmulatorComputer animation
15:41
Interface (computing)SoftwareAnalogyPoint (geometry)Software testingComputer simulationMathematicsComputer hardwareHardware description languageSystem on a chipDecision theoryModel theoryEnterprise architecturePhysical systemProcedural programmingTerm (mathematics)BefehlsprozessorCore dumpVirtual machineSpeichermodellEquivalence relationSoftware development kitAddress spaceIntegrated development environmentSystem callInclusion mapEmulatorLogicArmNumberComponent-based software engineeringServer (computing)Lecture/Conference
17:56
Computer hardwareModel theoryLogic synthesisField programmable gate arrayTape driveMereologyEvent horizonAlgebraic closureImplementationPoint (geometry)BitModeling languagePhysical systemLogic gateProgrammer (hardware)Field (computer science)Computer hardwareSoftware testingSoftwareCombinational logicTape driveDistribution (mathematics)Field programmable gate arrayView (database)Process (computing)Model theoryComputer animation
19:37
SoftwareStandard deviationCore dumpComputing platformKernel (computing)Device driverModel theoryComputer programSoftware testingSoftwareLevel (video gaming)Software developerSystem callFunctional (mathematics)ResultantModeling languageCodeChainCASE <Informatik>Procedural programmingPhysical systemLibrary (computing)Assembly languageCore dumpComputer hardwareEnterprise architecturePreprocessorPoint (geometry)Case moddingLink (knot theory)Model theoryArmLecture/ConferenceComputer animation
20:54
Function (mathematics)Kernel (computing)Computer programmingBootingPhysical systemCodeSpacetimeSoftwareChainEnterprise architectureDistribution (mathematics)Assembly languageComputer hardwareSoftware testingCompilerVideo game consoleDifferent (Kate Ryan album)Point (geometry)Lecture/Conference
22:07
SoftwareEmulatorModel theoryStandard deviationCore dumpComputer programSoftware testingKernel (computing)Device driverComputing platformAssembly languageArmSoftware development kitEvent horizonComputer-generated imageryPhysical systemIntrusion detection systemCovering spaceStack (abstract data type)Enterprise architectureQuadrilateralSoftwareCore dumpComputing platformComponent-based software engineeringRight angleSpacetimeDistribution (mathematics)Bootstrap aggregatingAdditionStandard deviationLevel (video gaming)Pointer (computer programming)Order (biology)ArmPhysical systemModel theoryOpen setCodeComputer programmingDifferent (Kate Ryan album)Modeling languageEnterprise architectureComputer animation
24:11
Physical systemCodeCovering spacePoint (geometry)Java appletEnterprise architectureArithmetic meanTransport Layer SecurityFormal languageCore dumpArmRight angleLecture/Conference
25:12
Assembly languageComponent-based software engineeringFile formatTransport Layer SecurityKernel (computing)Front and back endsLinker (computing)Principle of relativityOperations researchGeneric programmingCoroutineFloating-point unitComputing platformTable (information)Web pageSpacetimeModel theoryCore dumpLibrary (computing)Different (Kate Ryan album)Run time (program lifecycle phase)Formal languagePoint (geometry)Rule of inferenceMultiplication signIntermediate languageAssembly languageCASE <Informatik>Projective planeArmKernel (computing)Generic programmingLink (knot theory)Address spaceObject (grammar)Computer fileThread (computing)Front and back endsTraffic reportingSoftware developerCodeNetwork topologyChainLibrary (computing)Computer animation
27:04
Goodness of fitPoint (geometry)Ocean currentProcess (computing)TheoryBitCodePhysical systemArmSemiconductor memoryLevel (video gaming)Address spaceState of matterPhysicalismLecture/Conference
27:44
Kernel (computing)Model theoryCore dumpOperations researchCoroutineLibrary (computing)Computing platformGeneric programmingWeb pageSpacetimeTable (information)Floating-point unitDistribution (mathematics)Component-based software engineeringBuildingArchitectureFeedbackCollaborationismArmServer (computing)Phase transitionTerm (mathematics)Computer-generated imageryRead-only memoryStandard deviationArmSeitentabelleEqualiser (mathematics)MathematicsSemiconductor memoryDecision theoryAddress spacePhysicalismBitCASE <Informatik>Web pageBootingSpacetime32-bitWikiPhysical systemLevel (video gaming)System on a chipDifferent (Kate Ryan album)Mathematical optimizationRight angleBinary codeSelf-organizationSlide ruleNumberOpen sourceProcess (computing)Medical imagingSurjective functionTerm (mathematics)PlastikkarteKernel (computing)Server (computing)Computing platformDiscrete element methodArithmetic meanCollaborationismMobile appLaptopProgram slicingPoint (geometry)Run time (program lifecycle phase)Greatest elementComputer animation
31:26
Level (video gaming)Bootstrap aggregatingRevision controlMereologyOcean currentGroup actionStandard deviationTerm (mathematics)DampingEnterprise architectureInternetworkingServer (computing)BitPiComputerArmMathematicsCASE <Informatik>Floating-point unitBuildingNP-hardExecution unitPoint (geometry)Lecture/Conference
33:18
ArmServer (computing)Phase transitionTerm (mathematics)Computer-generated imageryRead-only memoryStandard deviationSource codeOperations researchPhysical systemInternet forumPoint (geometry)PlanningMedical imagingBeta functionArmSlide rulePhysical systemWeb 2.0OvalDistribution (mathematics)Enterprise architectureBuildingMoment (mathematics)Modulare ProgrammierungBootstrap aggregatingBitComputer hardwareSoftwareComputer animation
34:53
Enterprise architecturePhase transitionFingerprintLevel (video gaming)BitSoftwareLevel (video gaming)Bootstrap aggregatingRevision controlRoutingMultiplication signLecture/ConferenceComputer animation
35:29
Phase transitionEnterprise architectureFingerprintLevel (video gaming)WikiInternet service providerElectronic mailing listScripting languageEntire functionIntegrated development environmentLevel (video gaming)Maxima and minimaVirtual machineSoftware bugPhysical systemPatch (Unix)Point (geometry)Standard deviationRepository (publishing)Computer fileRevision controlRootModeling languageFigurate numberFile systemScripting languageEntire functionRegular graphBootstrap aggregatingBitSet (mathematics)ArmProcess (computing)Enterprise architectureWikiSoftware repositoryComputer animation
37:42
Local GroupRootMereologyRevision controlLoginBootstrap aggregatingProxy serverMultiplication signFile systemComputer fileSource code
38:31
Internet service providerWikiElectronic mailing listPhase transitionEnterprise architectureScripting languageEntire functionFile systemLevel (video gaming)CompilerKernel (computing)Function (mathematics)Content (media)Software repositoryIntegrated development environmentInternet forumConfiguration spaceLimit (category theory)Electronic mailing listArmType theoryLink (knot theory)GoogolWikiSlide ruleSoftware repositoryLevel (video gaming)Process (computing)Standard deviationRevision controlAxiom of choiceSet (mathematics)Bootstrap aggregatingDirectory serviceBuildingBefehlsprozessorBootingData structureFunction (mathematics)QuicksortHacker (term)Physical systemFile systemFigurate numberRight angleLatent heat1 (number)FingerprintDampingMedical imagingMoment (mathematics)CuboidPoisson-KlammerDemonEnterprise architectureComputer animationLecture/Conference
41:34
Level (video gaming)Function (mathematics)Configuration spaceVirtual realityScripting languageVirtual machineServer (computing)Kolmogorov complexityDistribution (mathematics)Standard deviationSystem programmingLevel (video gaming)Maxima and minimaWikiFile systemSet (mathematics)Process (computing)Modeling languageHacker (term)Standard deviationMedical imagingComputer simulationModel theoryScripting languagePhysical systemCASE <Informatik>Multiplication signEnterprise architectureComputer hardwareChainRoutingCycle (graph theory)Online helpComputerRootExpert systemBootstrap aggregatingFingerprintLecture/ConferenceComputer animation
43:44
Kernel (computing)Computer hardwareModel theoryArmLevel (video gaming)CompilerScripting languageField programmable gate arrayConfiguration spaceMassHybrid computerBootingEnterprise architectureSlide ruleBitLevel (video gaming)Set (mathematics)Point (geometry)Computer hardwareMultiplication signArithmetic progressionLecture/ConferenceComputer animation
44:37
Context awarenessArithmetic progressionOpen sourceCodeQuicksortRule of inferenceGroup actionHacker (term)Function (mathematics)Physical systemEnterprise architectureArmMultiplication signChainWeb pageGoodness of fitBootingLecture/Conference
Transcript: English(auto-generated)
00:00
All right, thank you, Lucas. So, good afternoon, thank you for attending FOSDEM and thank you very much for coming to my talk. My name is John Masters and today I'm gonna talk about porting Fedora to new architectures. Please raise your hand if you have heard of Fedora.
00:23
Okay, please raise your hand if you have not heard of Fedora. This is good, this is good. So we are a very popular Linux distribution and we enjoy being cutting edge and kind of having features first. Those are two of our four fundamental principles.
00:42
So I'm gonna talk about porting Fedora to new architectures and in particular, two different ARM architectures. I'm gonna explain kind of a little bit about what that means as well. I'm gonna talk about porting to the newer 32-bit ARM version seven architecture by way of leading into the work we are doing
01:02
porting Fedora to the 64-bit version eight architecture also known as AR64 or ARM64 and in fact, I have a whole slide that will tell you all the different names that you could use for exactly the same thing.
01:21
That's a picture of me. I have to have one of these. This is me at last year's Red Hat Summit in Boston pedal powering an ARM server. So that box you see back there is being powered through this bicycle through a series of inverters
01:41
and all kinds of craziness and Wookie would be proud of the setup. The point I was trying to make there is ARM technology interests me in particular and my employer because of well, a number of different aspects but in the emerging server space
02:03
there's a real growth in hyperscale computing, low energy computing or any energy efficient computing and many other features where ARM just happens to fit in really well. So we see ARM as an emerging technology but we do see it as a very interesting technology
02:20
and so we're beginning to invest in development and many of us who work at Red Hat are involved in the Fedora ARM project. Now, I'll explain what Fedora ARM means later on and who's involved. It's not solely a Red Hat activity by any means but in this setup here I'm able to run a full server
02:41
by just having a bicycle attached to a generator. So there really is something to this. What I want to talk about today, I want to kind of talk a little bit about modern computer architecture. I promise not to be too boring. Talk a little bit about what happens when you have a new architecture
03:01
in terms of the hardware and software development behind that. Then I'd like to introduce you to the Fedora ARM project and I'm gonna talk through the different stages that we go through to get Fedora up and running on a new architecture. So ARM version seven, which we've already done
03:22
and AR64, which we are well into at this point. I will explain kind of where we are and then I'd like to finish up with a live demo of Fedora running on a 64-bit ARM simulator. And I'll talk a little bit more about the complexities
03:40
of doing the work without having super fast hardware when you start doing that. As far as questions go, I'm gonna leave some time at the end for questions. I'm also going to have a couple of points during the talk where you can ask questions. I think that's gonna work better in this format. So please wait until I ask for questions.
04:04
So AR64, can't really do a pirate voice so we'll just go with that. In addition to some of the technical stuff that I'll talk about as far as rationalization of the architecture, one thing ARM also did
04:21
with AR64 was rationalize the naming. So ARM is a trademark and that means that sometimes you don't wanna use ARM everywhere. It tends to devalue your name if you do that. So when you hear things like ARM version five, ARM version six, ARM version seven, ARM version eight,
04:41
these are architecture revisions. These are revisions of the core architecture. You will also hear terms like ARM, thumb, and other sort of historical terms that are now replaced by A32 and T32. These are actual instruction sets.
05:00
These are binary instructions that the machine will run when it, on one of these architecture versions. In the 64-bit case, there is a machine code known as, or instruction set known as A64. So you get this case where your machine is an AR64 machine running ARM version eight
05:21
of the architecture executing A64 instructions. It's a little convoluted. So memorize this slide because there will be a test on the way out asking you all of the different intricacies here. I guess the thing that I want you to take away from this is that there are too many different names involved. We are using AR64 in Fedora,
05:43
largely because that's the name that ARM was using. The kernel community, the Linux kernel community, took a look at that and said no, we're gonna go with ARM64. So you also get use of ARM64, ARMv8, AR64, all these different things.
06:01
Fundamentally, it's all one and the same. It's all a 64-bit architecture from ARM. So like most modern architectures, this new 64-bit architecture, it's risk-based.
06:21
Risk is kind of what everyone does these days. The instructions are 32 bits wide. That's pretty common. It's a 64-bit architecture, but every instruction actually is 32 bits wide. They modified how they encode the instructions at the binary level. So that there's more room for sort of immediate.
06:42
Got a little bit more flexibility than the previous ARM architecture revisions for accessing constants and handling memory offsets that are bigger. Lets you have fewer instructions to load from global pools and all this stuff. It's a load-store architecture,
07:01
like most modern RISC architectures. That means that you don't directly operate on memory. You load, you do some operation, you store it back. Unlike the 32-bit ARM architecture, there's a lot more general purpose registers. There are 31 in the new architecture.
07:21
And then you've got some there that have specific purposes. So unlike in the traditional case of ARM, you don't have direct access to do funky things with the program counter or manipulate the stack pointer directly. All of these things. Instead they're kind of inferred through other instructions.
07:41
You do have a couple of interesting things in the register space. You have two different, you have IP0 and IP1, which you can use to do interim, you can use both these registers together to do like a calculation to load from a procedure linkage table or something like this.
08:02
Whereas you previously would have only one to do that. So you can kind of reduce your use of instructions to load like a procedure call address or something. There's a large, flat memory model. So it's not kind of like x86. You don't have weird segmentation
08:21
on different memory models. It's a pretty simple case. And in the case of the 64-bit architecture, they also are increasingly supporting things like unaligned accesses. So in the 32-bit case, ARM systems have grown the ability to handle non-aligned memory access
08:41
in the last few years. Still expensive at the hardware level. It's better if you link your code in such a way, compile in such a way that memory accesses are aligned to certain natural alignments of whether you're accessing a word or an integer or a character address or something. But in the case of the 64-bit architecture,
09:01
you can make that access, and it will handle it at the hardware level for you. There's also various system control and architectural hinting, which is the other cool thing that most new architectures do. So you can kind of say, I'm going to some code now, but it's not really a procedure call.
09:21
So if you are trying to optimize with some kind of branch predictor inside the CPU, you know that actually I'm not really planning to come back from that. Or I am planning to come back from that. This is a function call. And so you give some hints. You can actually use instructions that will kind of hint to the underlying architecture what it is that you are doing
09:41
so that it can perform various optimizations for you. Other things that are specific to ARM version eight and not just sort of modern computer architectures. Arnd Bergman, who is one of the core Linux kernel developers involved in the ARM community,
10:01
is Arnd here? No, it's just possible that he might have been in this room that would have been funny. He called ARM version eight or AR64 a very boring architecture in a nice way, right? So there's very little there that's kind of surprising. You get this new A64 instruction set
10:20
that joins the traditional ARM instruction set, which is now called A32, and the traditional thumb, which is now called T32. It is a new instruction set, so you do have some things that you have to learn. You do have to read through some documentation to use it. The assembly language that they implement
10:41
follows the sort of conventional unified assembly language that ARM came up with a few years ago. So it looks like regular ARM assembly code, even though in fact it's doing some things behind the scenes. The instructions are different, but the assembly language looks very similar. They removed coprocessors from the architecture.
11:04
So in the traditional case on ARM systems, you had these kind of special coprocessors built into the CPU. Things like the memory management unit. It wasn't a core part of the architecture. It was a special coprocessor that lived somewhere off to one side, at least notionally,
11:22
and you would poke it and program it. That never really worked very well. The idea was that people could make their own coprocessors. If they implemented an ARM CPU, they could put some funky extra feature there. But if you try to shove things into an existing CPU, that generally doesn't end very well. So they've removed that from the architecture,
11:42
and instead they have just general instructions for things like memory management control, cache invalidation, all of these things. They're just regular instructions in the new architecture. They did do away with a lot of the sort of weird register banking. Those of you who've actually touched 32-bit ARM assembly
12:03
know what I mean there. Instead they have a pretty clean, pretty clean, flat layout. They have these exception levels they introduce into the 64-bit architecture. So every, well, almost every 64-bit ARM CPU
12:20
will boot into what is called EL3, which is a secure state. It's a special mode of execution where you can put your trusted bootloader validation code, or if you are a bank, making software for a smartphone, because these chips won't just be in servers.
12:40
They'll be in all kinds of different use cases. If you're making some kind of payment processing code, you can put all kinds of stuff in EL3 that's magical and kind of hidden from the OS. What will happen there is that will transition down various levels, down to EL2 for a hypervisor, EL1 for a regular kernel, and EL0, which is where regular user space
13:03
runs on these systems. You don't have predicated execution, so in traditional ARM you had various bits at the beginning of each instruction that said I might execute this instruction, I might not. You don't have that in the 64-bit architecture. You have just regular conditional instructions.
13:23
And you don't have the weird kind of exposure of the pipeline. You don't have addresses that might be off by four or eight because of exposing the pipeline design. Instead you just use the direct PC addresses.
13:42
Yeah, and they've cleaned up some other things that you can read through the ARM documentation to see what they changed. So, broadly speaking, when you're working on a new architecture, you have two kind of different things you have to worry about. One is the hardware development,
14:01
and the other one is the software co-development. Because you're kind of doing both at the same time. What you don't do in the case of modern architecture is kind of do all the hardware and then say, okay, now we're gonna do all the software. That doesn't work because what you wanna do is you wanna get as much software as you can
14:22
up and running very early on, test things, and see, okay, well how well does this actually work, and maybe I should make this change. Maybe I should change whatever part of the architecture before I've locked it down forever more. So it tends these days, not just with ARM, but with things like OpenRisk as well. Some of these other architectures,
14:42
it tends to be more evolutionary. You start with some good ideas, you try some things out, you get some code running, you see how well that performs, and you iterate on that a little bit more than you did 20 years ago. So it's an iterative process fed from modeling, usually open source code, because there's a large body of code,
15:01
you can compile, and you can test things out. First stage of most hardware development is you will write some kind of emulator, some kind of software model that describes how the system is gonna work, so you can start trying out some ideas. OpenRisk, which is another architecture I like to play with,
15:21
it has its own simulator called ORK1Sim. If you guys are interested in purely open source computer architectures, take a look at OpenRisk, it's kind of fun. In the case of AR64, there are three different models, software models available so far. There's a model from ARM called the FAST models
15:41
that you can get under a certain license. There's a foundation model, which I'm gonna show you later, which is a free download, so you can just go download that. It's kind of like when you are, well, it's not really, but here's a rough analogy. You know, if you're running like a VMware virtual machine
16:01
and you don't wanna change anything, you just wanna run it, that's kind of your foundation model. You can run that AR64 environment, but you're not changing the layout of the hardware or making tweaks to it, which you could do with the FAST models. There's also, as of this week, a simulator available from Cavium.
16:20
They're one of the first hardware manufacturers of V8 chips, and they've just announced they have an SDK that I've been playing with. I think I was quoted this week saying that it was faster than the foundation model in some testing we did. So there will be more, and some of them are freely available,
16:41
some of them are available under various licenses. So once you've got kind of a software emulator, the next thing that you do, kind of one team's gonna go off and they're gonna start working on hardware designs. So they're using RTL, they're using hardware description languages to start actually implementing
17:01
both synthesizable and non-synthesizable logic. That's kind of a electrical engineering term you can ignore. But you're starting at that point to have a team going off and actually implementing the architecture decisions you've made into hardware. And what they're doing is twofold. They're both implementing just the core CPU,
17:23
the core architecture, can I add two numbers, can I jump to addresses, can I make procedure calls? And then they're also looking at the broader system components, the server on chip, system on chip components, and integrating this core CPU into an overall design
17:42
that's gonna include IO devices and network interfaces and all these other things that you have in modern systems. Once they have some of the early hardware design, they have the equivalent of what we would use in software to model that. They can actually run that hardware
18:03
as a model. And it kind of behaves like real hardware. Depending on how you do it, you're either going for very accurate and very slow, or less accurate and still very slow. But these hardware guys are more interested in does my design work?
18:21
And then you can get to a point where once you've got a rough design, you can take an FPGA, a field programmable gate array, and you can take this design and you can kind of run it on a very expensive chip that can run that at a few megahertz maybe. So I have a chip, I have a system in the lab
18:42
that's an ARMv8, that's an ARCH64 system on an FPGA. It only runs at 10 megahertz, let's say, on a good day. But it's the fastest hardware that anybody has sitting around at this point. So what we can do with that is we can use a combination of software models
19:05
to just run code and from a software point of view, we don't care as long as it behaves the same. We don't care if we're on real hardware or we're on a software model, we're just trying to build a Linux distribution. But we might also want to test a little bit more than that. What would it be like on actual hardware?
19:22
So having access to an FPGA lets us get a little bit more close to what it would be like on real hardware. Later on in the process, you get actual test chips made, you get actual tape out, which is when you say, okay, I have a design, I'm gonna go make some silicon chips now. And some guys are kind of working toward that goal,
19:44
let's say, and we hope to have some silicon in the coming months, in the coming year that we can play around with. So stages of software development. This is kind of happening roughly in tandem with the early stages of hardware development.
20:01
Take the initial software model on which you can execute code and you can, at that point, you don't even have an assembler. You can't compile code for it, but you have a model. You have to have a couple of pieces there to really proceed. You have to have a base or core system ABI.
20:21
You have to define, how do my different pieces work together? When I've compiled some code, how do I link that? What does the result look like? How do I link two functions together? How does one function call another? How can I pre-compile a software library and expect it to work with another software library? Those are all defined
20:41
in procedure calling standard documents that every architecture has. And in the case of ARM, they have what they call the AAPCS, which I mention later. At this point, you're also then working on the initial tool chain. So you have to get an assembler. You have to get a compiler. You have to be able to assemble some code. You have to have GCC to start compiling C code,
21:03
all of these good things. You're gonna start by writing some simple test programs, just to prove correctness, just output, you know, text. Depending on how you're proceeding, some other architectures will then work with U-boot or something, or try to get a boot loader up and running
21:21
to play around with. And then at this point, you're gonna start thinking about the kernel port. So what you're trying to do there is get something that can talk to a UART, put something out on a serial console that you see the system booting, but you're not really targeting, you know, all different hardware devices.
21:42
You're more targeting, can I compile some code that works against this architecture? Can I boot what we call the ASM generic, the very basic Linux kernel port on that? And then, you know, we start in the software distribution world,
22:00
in the OS space, we start working on user space, let's say. Let's just, you know, broader user space. So you'll start with some very simple pieces, busy blocks, open embedded, all these have been done. And then, you know, later on you're working on full distributions. So Fedora is well and truly into its bootstrap right now.
22:23
Other distributions are working on their bootstrap. I know Debian are fairly heavily into theirs. Ubuntu are into their distribution bootstrap, and so on, right? So it's not just Fedora. Although this is all about Fedora, so I'm gonna ignore the rest of them for the rest of this talk.
22:40
And once you've got the basic distribution up and running, you're then bootstrapping the higher level software components. And I'm gonna talk through the different stages that we use there as well. You know, I think I've talked a lot about the different models we have. You know, there are the fast models, the foundation model,
23:01
Cavium have a model. There's a QMU port that is beginning within Linaro. So we hope to have a QMU port that can target a system model eventually. You know, we first wanna run user space code, but eventually we want to have a full system model
23:21
of the architecture as well. Again, when you're running software models, you know, you're running on a different architecture, you're not exactly running at gigahertz speeds, right? But you are running maybe hundreds of megahertz, maybe enough to do something useful. I think I mentioned that there's various standards behind, you know, how you build the software
23:41
in addition to arms documentation on the core instruction set. There's also supporting documentation that says how to compile programs, how they're linked together, what the register passing conventions are, what the minimum stack alignment needs to be in order for things to work,
24:01
things like that. They define some other pieces in ARMv8 that are quite nice. There's a dedicated platform register we can use to store pointers we might want to have easy access to for things like TLS and things like that. And the basic ABI is gonna cover, you know,
24:21
your core system languages, C, C++. There's a talk coming up after this one where I'm staring at one guy right now, but there's two guys from Red Hat are gonna talk about, Andrew and Andrew are gonna talk about porting Java to the new architecture, so I'll let them kind of cover those pieces.
24:42
There's a bad joke here, or a very weird, nerdy joke. Interlinking means something else, but the point is we actually are not interested in Fedora in compiling code that can link 64 and 32-bit together,
25:00
and that's not actually technically possible either, so that's great. We don't really want a 32-bit story when it comes to this new architecture. So, you know, following the upstream development, I'm gonna keep an eye on the time and skip through some of this. You know, the first piece of the toolchain that you get ported, you get your assembler ported.
25:22
You need to get the ELF file handle, ELF object support added, so, you know, ARMv8 use, AR64 uses ELF64. It has some new relocations it defines so that when you want to relocate precompiled code, you can go through and you can fix that up
25:41
with the correct addresses at runtime. If you get the link reported, so you can actually link that together, and then there's some dependencies later on against the kernel and glibc for things like TLS support and threading. GCC has various different internal backends,
26:02
different intermediate languages it uses, and it turns out doing a C or a C++ port is kind of, you know, a good starting point. If you want to have other backend languages, then other language support, then you need to target different intermediate representations within GCC. And some of that work is still ongoing.
26:23
The initial kernel port was posted along with the toolchain last year, kind of started in the summer and then went on into the fall, and various pieces of this are now getting into upstream projects. In the case of the kernel, it's a 64-bit clean modern kernel port
26:42
targeting what ARM Bergman has created, the ASM generic tree, which is like a reference. This is how all kernel ports should be done, and the goal there really is every time someone does one, it has to at least be as good as the last one, and then they'll find new rules and new restrictions and new things that should be done the next time, and more things get added, more requirements get added.
27:03
But Catalin Marinus and Will Deacon at ARM have done a good job taking their initial work, reworking it against ASM generic, and having a very, very good comprehensive port. Current code base supports booting either at ELT, which is a hypervisor level, so you can later on have support for KVM or Xen,
27:22
and then it can drop down into EL1, which is the state at which we run the kernel, usually. It doesn't support the full 48-bit physical address space yet. In fact, in ARMv8, it could in theory go up to 56 bits if we ever got there, but at this point, nobody has a system
27:42
that has that much physical memory, so it's kind of academic trying to support it. And in the same way that on x86, we've changed the page table layout, we've changed how we do memory on x86 in the past, our systems have got bigger, we've said, okay, all right, we better change that, rework that a little bit
28:00
to handle the increase in physical address space. The same kind of thing will happen when it comes to 64-bit ARM. We are still playing around with a few decisions there around page table sizes. So in the case of ARMv8, we have at least 4K and 64K pages possible.
28:20
32-bit systems use 4K pages, generally. So what this means is, if we don't care about 32-bit compatibility, which we're advocating not to, we can go to a 64K page size as the granule size and get various benefits from that. So we're currently running through some thoughts around what we want to do there, but that's likely to be where we go.
28:44
And the initial port didn't support things like lazy FPU, floating point, save, restore, some of these optimizations that can get added later, and that work's going on now. It also did not provide an SoC target platform, so it wasn't trying to provide a generic base
29:03
on which everybody could support many different shipping SoCs. There weren't any shipping SoCs at that point, any shipping chips with other devices on them. That work's ongoing now, so that some of these guys that are building the first 64-bit ARM chips have a target that they can work with.
29:23
And the goal here really is, when it comes to 64-bit ARM systems, we'll have one binary kernel, we'll be able to boot, and then as we do on x86, we'll be able to figure out at runtime, we'll be able to enumerate the devices, we'll be able to make the right decisions and boot the system
29:41
without having to have 10 different kernels. So, the user space pieces, which is the main Fedora piece, I guess. If you go to the wiki address at the bottom there, you can look later at your leisure, and these slides will be available
30:01
through the FASTM organizers. I'll make them available later on. We posted an initial stage one at that URL. You can go there and you can read through kind of how to use it, how to download it. And I'm gonna talk through now kind of the different stages we go, what stage one means, what stage five means,
30:22
and the different processes that we're going through. So, as I mentioned earlier, Fedora ARM is kind of an all-encompassing term. It's a collaboration, really, between Red Hat engineers and members of the Fedora and open source Linux communities.
30:42
And today we support a number of different 32-bit ARM devices. So, you can go download an image for an SD card, you can run DD, put your SD card in, DD something onto it, put it into your ARM laptop, this is a Chromebook running Fedora here, and then boot it. And we support things like the Panda board,
31:02
trim slice, Raspberry Pi, Raspberry Pi as a remix, which is a technical term. It's not an officially supported target because there's no upstream Linux kernel. But we support many different 32-bit targets, including several of the 32-bit servers. So, the Calzada high bank server,
31:21
and there's also a remix that targets the Marvell MVEBU, which somebody else is talking about later on. So, you can hear more about porting to Marvell architectures later on. We are targeting 64-bit server support later this year in a more comprehensive way,
31:40
and various stages along the way over the coming months. Current status, ARM version five, which is the older architecture, we are phasing that out. So, Fedora 18, which is just in release now, that will be the last version of Fedora that supports anything less than an ARM version seven,
32:02
officially. ARM version seven devices are available for $40 on the internet. The Raspberry Pi, unfortunately, is an ARM version six device. It's very popular, and so a separate group is going to maintain a port to ARM version six for at least as long as the Pi is really popular.
32:22
But if you're not buying a Pi for about the same money or a little bit more, you can buy an ARM version seven computer that we run on today. In that case, we run what we call the hard float build of Fedora. So, ARM made some changes in version seven. Principally, we can now assume the presence
32:41
of a floating point unit. So, given that we can do that now, we can, we switched to a newer version of their ABI standard that had some performance improvements, and as part of that, we kind of intentionally broke compatibility.
33:00
So, for a while, we kept what we call the soft float ARM version five build alive, and now we're gonna kill that off because really, there's only a few people using that today in Fedora, and that's not enough to keep it alive. ARM version eight is where the fun is in terms of bootstrap and making bits available, and we will have a final release of Fedora 18 for ARM devices next week.
33:22
That's our plan. You can download the beta today. You can download the release candidate images. We're just at the point of copying them over and making the announcement, but you'll find an announcement for Fedora 18 within the coming days. I've got some pictures here of various ARM devices that Fedora supports.
33:41
You can look at them in the slides. So, when we're trying to get Fedora up and running on a new architecture, like the 64-bit ARM architecture, we're concerned about building natively. So, Fedora is a native built distribution. That means we use the hardware
34:02
for the architecture concerned to build for that architecture. So, chicken and egg problem there because there isn't any hardware that we run Fedora on today to build Fedora on 64-bit ARM systems. So, the thing we have to do is a bootstrap, which is to get the world built from the ground up.
34:23
Once we have things built, which I'm gonna explain a little bit more about in a moment, once we have that built, we use a system called Koji, which is like other distributions. SUSE have OBS and Debian have their build-dees and various other pieces at Ubuntu economical that I don't even understand.
34:42
I'm sorry, I don't keep up with exactly how Ubuntu gets built, but everybody has kind of a build system. We have one called Koji, web-driven. You can go and look at a software package, see when it was last built, instigated build, all of these good things, and it's 100% reproducible because every time we do a build, we generate a build route using a tool called Mock
35:02
containing known versions of every piece of software that was needed to build that. We have to make the bits available that we're gonna use in Mock, which is where the bootstrap comes in. So, there are five stages that we came up with to do a bootstrap. And what happened was, in Fedora 15,
35:21
which is now a couple years ago, we did this bootstrap for ARM version seven. And we did this bootstrap because I kind of knew that V8 was coming, and not many people did, but we knew it was coming, and we wanted to just kind of figure out how we're gonna do this. So we treated ARM version seven
35:41
as a brand new architecture, knowing that we'd have to do this for the 64-bit release. So, we go through these five different stages. The first one is to kind of cross-compile. A minimal set of bits using an x86-64 machine, just a regular PC desktop, to build the tools that target,
36:02
that will run on a 64-bit ARM machine. Once we've done that, we take those, we put them into one of these software models that does a 64-bit ARM system, and we have just enough built to kind of run that environment and then use those tools to rebuild themselves, so they're built natively, and then to build the next level of pieces, stage three,
36:25
which gets us to a point of using our standard packaging tools, our standard RPM, RPM build, these things that Fedora packagers will know very well. After that, we then rebuild everything we've built so far as RPMs,
36:40
and once we've done that, we take those and we generate, we build more and more dependencies so that we can run a mock environment, and then the final stage is to get to a point where we can run Koji, which is our standard build system. So it's a fairly contrived and involved process.
37:03
We've currently completed stage one and two for the 64-bit architecture, largely because software models are not incredibly fast, and there are still a few patches and bugs that have to get worked out. You can go to our wiki, and you can actually download the bootstrap scripts that we use.
37:20
They're in a Git repo, and you can download the root file system that we're working on, which I kind of came up with this idea in version seven. Why don't we treat the entire file system for our bootstrap as a Git repo? Because Git is a file system, so we have an eight gigabyte Git repository that you can clone,
37:40
just to prove I'm not making that up. I know we're running low on time, so I'm gonna be very quick here. V8 root FS. So that's the file system as it stands today, and I can literally run Git log inside my file system,
38:01
and I can see what package got built, who changed it, how it was committed. It's a horrible, horrible hack, but I came up with this in version seven, in the version seven bootstrap, and then you can NFS share this across your network, and you can have different people building different parts of the file system. What could go wrong? Nothing could go wrong with this,
38:20
and then you get people sending you Git pull requests. Here's a new part of your file system sending you. It's kind of weird, but it works, so why not use it again? Yeah, so you can go to our wiki. If you want to find the wiki,
38:40
A, you'll find these slides, because I'll make them available, but B, you can type in fedora arm into Google, and you will find these links. You can actually see a list of all the available Git repos, and you can follow along, and it's really kind of fun. I'll skip through some of the stages here, but stage one, you know, we're cross-building the world. The main thing we have to do at stage one
39:01
is figure out what is it we're targeting, what are the things that we are turning on and off. So here is the kind of arm version seven, set of choices we made, and in brackets, the arm version eight ones, and largely you can see with the 64-bit architecture, we really don't set very much. We say, well, we're not targeting a specific CPU.
39:26
We are targeting AR64, we are targeting having an FPU, and that's about it, so we set some of these things. We decide the triplet, the name we're going to use for this architecture, so we had arm version seven, HL, hard-float, little-endian, red-hat,
39:43
Linux, GNU, E-ABI for 32-bit, and in 64-bit, we call it AR64, red-hat, Linux, GNU, because that's what we do. These are the kind of things you're building in stage one which we have built, obviously.
40:01
Stage two, they go straight into a Git repo, so you can track the history forevermore. We build additional dependencies. We can then mount those over NFS or as a file system image, which I'm going to show you in a few moments, and we can get RPM working, which is where we are right now.
40:22
So here's a kind of example output of where the work-in-progress, what the directory structure looks like. It's a standard Fedora system with a few extra directories in there, which can be booted. We don't have an init daemon yet. I won't rant about systemd,
40:41
although people know I love to do that. It unfortunately requires the entire gnome stack to be built before you can build it, so we're not booting with that today. We're booting using Busybox and a few other hacks just to get something that boots, but the rest of the pieces are standard RPMs. So when you see this boot, you're not gonna see a standard Fedora boot process.
41:02
We don't have everything built yet, and systemd is gonna be much later, but we'll get there. So stage three, which is being worked on right now, we've rebuilt everything we have so far as RPMs. We're still configuring things out. We turn off documentation. We say, well, that's nice, but that's gonna require we have 300 other packages built
41:22
to have tech or whatever it is we need to do to process documentation, so no, we'll turn that off. So we're still turning things off, and in fact, Debian have these sort of bootstrap dependencies that I quite like, and there's a few things I like other distributes have done that I want us to learn from and sort of take back into how we package
41:42
so we can always bootstrap the world. I think Wookiee did a great job in Debian with that. I also think we can inspire others to do horrible Git-based hacks to track your file system. We will continue with this process in stage three, so we will be using these software models to build RPMs and get a full set of minimal RPMs
42:04
out of that sufficient to generate these mock routes we can use to build higher stages. In stage four, we're gonna have mock. We're gonna have all these standard features, and when we did this for the 32-bit architecture, it was really cool because we solved all the really weird
42:21
this tool chain piece isn't working because blah. We solved all of that, and, well, most of that, and we made available a file system image to just everybody in the community, and we said, okay, you don't have to be an architecture expert. You could download this image, and if you have a system, you can run this very minimal, not Koji,
42:42
but very minimal set of scripts we wrote. It's gonna download a package, build it on your system, and push it back up, and so we kind of farmed out the bootstrap, and that's roughly what we're gonna get to with 64-bit. If we get hardware in time, we might do something else. If we don't, if you have a model and wanna help us, then actually you'll be able to use your system
43:02
to donate some compute cycles to the process, and then we'll look at using Amazon EC2 and other ways of getting a lot of simulated time. Once we get out of stage four, we have this kind of giant set of RPMs. The whole thing's been built, more or less.
43:21
The whole basic install has been built, but it's still not official. It's still not been built using our standard build system, so we then build it one more time. In the case of ARCH64, we'll probably build it two more times because we wanna make sure that everything's been built correctly once and then used again to build everything correctly a second time,
43:40
so we just love compute cycles, and we'll use a lot of them, so it's all about correctness, and at the end of the day, we will release these bits and everybody will be happy forevermore. So I've got some slides here that kind of summarize what we're doing. We've done stage one.
44:01
We've done stage two. We've validated stage two on an FPGA. It booted. It actually works on real hardware, and we are currently in stage three and should have that completed in the coming months. Yeah, and at that point, we're gonna start asking people
44:21
to really broaden this out. It started as something where a small set of people could really make a lot of progress, and it's gonna really expand out. I'm gonna skip to, kind of running out of time, I'm gonna skip to questions because I want a couple of minutes here to let people spend five minutes asking me any questions you have about what we're doing
44:41
and how you can get involved, so if you have a question, maybe, yeah? Is the microphone coming around? Oh, actually, it's here, isn't it? Sorry, you wanna? It's on, yeah. Time for a couple of questions.
45:03
Will there be support for transparent huge pages? Yes, there will. Not on day one. Not right now, but there will be. That work is going on. In fact, there's a, hands up here if you've not heard of Lanaro. Good, okay, so Lanaro is a nonprofit
45:21
spun off by ARM that is all about advancing upstream progress and support in the Linux community and open source community for ARM systems, so it's kind of like a nonprofit engineering center that does all kinds of cool work for ARM systems. We founded a group in there called Lanaro Enterprise Group. It's very 64-bit heavy,
45:41
and finally, I got Red Hat to join Lanaro as well, so we're now a member of Lanaro, and we're all working together in that context to solve some of these heavy lifting problems like that may be done by other folks, but that's the kind of problem where we would leverage that resource to solve the problem. Other question?
46:03
Yeah. What about cyclic build dependencies outside the initial toolchain? So we've been a little lucky in that we can sort of turn things off or we can hack around certain things.
46:21
Over time, every Linux distro is becoming more and more and more and more complex. We looked in the 32-bit timeframe. There were some things like Python dependencies. Well, maybe you can run that Python code on a host system that's x36-64 if it's just needing the output from running some Python code or something.
46:41
There are hacks you can use, but I think we've been lucky so far. I think that it is necessary to have an aggressive sort of rebootstrap the world mentality. It is necessary to have a rule that distro must always be bootstrappable, and that's something I actually plan to raise with Fesco, which is our sort of engineering steering group, to say we have to make that mandatory
47:01
because it's nice that, as far as I'm concerned, I'm an armed fanboy, and it's the last architecture I care about, but I'm sure other people will care about other architectures in the future, and in general, it's a good thing to make sure you can rebootstrap the world if you need to. Wookie, excellent.
47:21
We should probably all be there and talk about our problems together.