openSUSE on ARM
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Title of Series | ||
Number of Parts | 90 | |
Author | ||
License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
Identifiers | 10.5446/40310 (DOI) | |
Publisher | ||
Release Date | ||
Language |
Content Metadata
Subject Area | ||
Genre | ||
Abstract |
|
FOSDEM 201331 / 90
2
5
8
10
12
13
14
15
17
19
21
24
25
28
29
31
32
34
36
39
40
43
44
46
50
51
52
54
55
57
58
62
65
66
67
78
79
87
88
00:00
Software developerSoftwareSupport vector machineServer (computing)PowerPCArmWhiteboardGoogolOpen setArmGoodness of fitSoftware developerMereologyExtension (kinesiology)Compilation albumProjective planeGraph (mathematics)Open sourcePower (physics)INTEGRALTouchscreenPasswordGoogolConnectivity (graph theory)Virtual machineComputer hardwareWorkloadCoprocessorGame controllerWhiteboardRight angleServer (computing)IntelBitTablet computerLie groupMedical imagingLecture/Conference
02:54
WorkloadArmCore dumpServer (computing)LaptopComputer architectureCoprocessorBefehlsprozessorLecture/Conference
03:56
Server (computing)ArmComputer hardwarePhysical systemHand fanArmWorkloadCuboidType theoryGame controllerRight angleOpen setVirtual memoryServer (computing)Ferry Corsten
05:09
Information securityMultiplication signArmSoftware maintenanceLevel (video gaming)Software bugFrequencyBuildingLecture/Conference
05:46
WhiteboardLevel (video gaming)MereologyRight angleCoprocessor
06:42
Kernel (computing)WhiteboardExpressionWhiteboardArmKernel (computing)ExpressionPivot elementSpacetimeCumulantBootstrap aggregatingLie groupTheory of relativityMechanism designEmulatorLecture/ConferenceJSON
07:22
FirmwareBootingArmSpacetimeFraction (mathematics)Kernel (computing)WhiteboardSoftware development kitArmArithmetic meanConnectivity (graph theory)Type theoryAbstractionPhysical systemKernel (computing)BootingSpacetimeComputer hardwareVideo projectorTouchscreenProjective planeSinc functionWhiteboardLogicBuildingLimit of a functionDifferent (Kate Ryan album)System callPower (physics)Video gameMereologyEuler anglesNormal (geometry)BitPlastikkarteRootFraction (mathematics)Exception handlingMiniDiscCore dumpLine (geometry)Device driver2 (number)FirmwareBlock (periodic table)Right angleRandomizationStreaming mediaMathematicsStructural loadLecture/ConferenceComputer animation
13:25
WhiteboardSoftware development kitPhysical systemRootData managementDemo (music)MathematicsWindowElectronic visual displayArmKernel (computing)Ferry CorstenArithmetic progressionBlock (periodic table)Medical imagingService (economics)Connectivity (graph theory)BuildingNormal (geometry)Procedural programmingVideo gameCuboidDescriptive statisticsSource codeSoftware frameworkBinary codeBootingProjective planeOpen setPlastikkarteWhiteboardTrailRepository (publishing)Software development kitResultantLecture/ConferenceComputer animation
16:07
Computer-generated imageryWhiteboardKernel (computing)BootingEmulationDemo (music)MereologySpacetimeNormal (geometry)ArmDecision theoryBitDirectory serviceArmSoftware bugQueue (abstract data type)Kernel (computing)CodeCuboidPlastikkartePatch (Unix)Cycle (graph theory)Electronic mailing listRoute of administrationComputer fileNetwork topologyUser interfaceBuildingWhiteboardSource codeMedical imagingBinary codeBlack boxDecision theoryBootingPhysical systemComputer hardwareFactory (trading post)Streaming mediaService (economics)WindowMereologySpacetimeMultiplication signSingle-precision floating-point formatState of matterMiniDiscNamespaceProjective planeDesign by contractLatent heatRevision controlPhysicalismWeb pageDescriptive statistics
22:07
ArmComputer-generated imageryBuildingComputer hardwareDecision theoryRepository (publishing)Software repositoryMalwareProcess (computing)Latent heatKernel (computing)Service (economics)Electronic signatureBuildingVirtual machineBinary codeMedical imagingComputer hardwareArmPhysical systemTraffic reportingDifferent (Kate Ryan album)Shape (magazine)WhiteboardCuboidEmulatorProjective planeSI-EinheitenStatisticsRight anglePolygonNetwork topologyConstraint (mathematics)FrequencyFactory (trading post)MathematicsFitness functionPowerPCComputer architectureRevision controlInformation securityProper mapLecture/ConferenceComputer animation
27:35
ArmBuildingFactory (trading post)Multiplication signBuildingSimulationComputer hardwareWhiteboardArmBinary codeRepository (publishing)Software repositoryWordTrailCodePoint (geometry)Network topologySoftware bugEmulatorRevision controlPatch (Unix)Physical systemStatisticsMiniDiscRight angleClosed setProcess (computing)RootMathematicsFactory (trading post)Operating systemSlide ruleGoodness of fitHacker (term)Mereology1 (number)Wechselseitige InformationExpandierender GraphJSONComputer animation
32:49
Software repositoryStandard deviationPosition operatorImage resolutionArmFloating pointRepository (publishing)TouchscreenBinary codeVirtual machineFactory (trading post)Point (geometry)Computer animation
34:17
Factory (trading post)ArchitectureComputerBootstrap aggregatingBinary codeRepository (publishing)Service (economics)RootEmulatorInstance (computer science)Bus (computing)Process (computing)Multiplication signLogicEndliche ModelltheorieVirtual machineBootingFile systemMereologyArmBootstrap aggregatingSlide ruleHacker (term)BuildingJSONXML
35:16
Data modelPhysical systemScripting languageSoftware testingComputer hardwareWritingWeb pageWikiComputer-generated imageryArmPersonal identification numberBuildingArithmetic progressionForcing (mathematics)Process (computing)Multiplication signOnline helpProjective planeEmailDistribution (mathematics)Physical systemActive contour modelDirectory serviceSoftware developerScripting languageEmulatorPatch (Unix)Exception handlingRepository (publishing)Bootstrap aggregatingElectronic mailing listLevel (video gaming)ArmWikiBuildingBinary codeCompilerComputer hardwareCodeVirtual machineSource codeBitMathematicsRootSoftware testingWhiteboardService (economics)Factory (trading post)ResultantIsing-ModellConfiguration spaceMeta elementForm (programming)Proper mapBasis <Mathematik>Medical imagingLibrary (computing)Disk read-and-write head
39:23
Virtual machinePhysical systemTraffic reportingEmulatorArmSoftware repositoryBootingAxiom of choiceLinearizationComputing platformRoot1 (number)Bootstrap aggregatingSoftware maintenanceRepository (publishing)Medical imagingKernel (computing)Standard deviationCodeComputer fileWhiteboardFile systemHuffman codingElectronic mailing listNetwork topologyMereologyRoutingCumulantProcess (computing)Power (physics)Design by contractMusical ensembleLecture/Conference
41:59
Repository (publishing)Integrated development environmentMedical imagingFactory (trading post)Directory serviceComputer fileMathematicsRootHidden Markov modelComputer animation
42:46
Service (economics)Instance (computer science)Multiplication sign2 (number)FingerprintPhysical systemProduct (business)MathematicsAndroid (robot)Server (computing)Kernel (computing)WordIntegrated development environmentException handlingNormal (geometry)Process (computing)Derivation (linguistics)BuildingSystem callDrag (physics)Scripting languageDifferent (Kate Ryan album)Goodness of fitStability theoryInheritance (object-oriented programming)Control flowSet (mathematics)Electronic visual displayLecture/Conference
Transcript: English(auto-generated)
00:01
Hello, good afternoon. Welcome to OpenSUSE on ARM. That's a chameleon. It sits on an arm. That's what we're talking about, chameleon sitting on arms. So who am I? I'm an extender graph. I'm a software developer working for SUSE. Usually do virtualization stuff.
00:20
So this is just a hobby project going a long ways. The most interesting part in the OpenSUSE on ARM port for me so far has been improvements to QEMU, which I usually work on, so that we can compile stuff on x86. But that's a completely different story from the overall OpenSUSE on ARM part. It just means that I know quite a bit
00:40
of the integral components of OpenSUSE on ARM. So what is ARM? Who of you knows what ARM is? OK, who owns an ARM device? Who does not have his hands up right now? You're lying.
01:00
Because you definitely have an ARM device. Look at those devices. This is an ARM device. It's a Cortex-M0. You usually have those at home, or at least something remotely close to this. Raspberry Pi, really popular. That's an ARMv6 device, unfortunately. It's dirt cheap and pretty powerful. Galaxy Nexus, right?
01:21
I mean, just any tablet you want to point out. Andebor just recently got out. Can you actually see what's on the screen here? All right. Andebor recently came out a totally very powerful ARM machine, or the Google Chromebook. This is a Google Chromebook?
01:42
I actually just suspended it a couple minutes ago. And I have no idea if it turns on again. It does. So I'm going to hand this thing to you guys. If it asks for a password, it's Linux. That just basically got an OpenSUSE 12.2 image running on it.
02:02
It's an ARM device similar to the Andebor. Basically, it's the same hardware. Same hardware as the Andebor, just in a nice case, and with a screen on it. There's also really even more powerful ARM hardware. This is a picture of the HP Moonshot project, which
02:24
in between, someone got canceled and instead they went with Intel. I have no idea where money went in that project. But this is based on the Tuxedo processors that were actually made for server workloads. So slowly, we are seeing ARM devices going into even more powerful devices
02:41
than your temperature control in your house. ARM really is everywhere. So I can't imagine anybody in the developed world to not have an ARM device. Everybody essentially has at least one, usually a dozen of devices powered by at least ARM chips.
03:03
I mean, even your x86 notebook is most likely to have ARM-based chips on there. If you've got Wi-Fi, that's most likely one in ARM core. If you have, I don't know, I think even Bluetooth is usually powered on ARM. It's incredible. A lot of the devices are really, a lot of chips are in its core ARM,
03:21
even though you don't see it. But we're slowly moving on the ARM side. We're slowly moving to devices where the ARM cannot only be companion devices, companion chips, but also real CPUs, like your core processor, based on the ARM architecture. Which means, oh, and OpenSUSE.
03:43
OpenSUSE is awesome for desktop workloads. It's awesome for server workloads. Which means, combine those two, you certainly know that you need to have OpenSUSE on ARM. Or actually, we all need to have OpenSUSE on ARM, right?
04:01
For the OpenSUSE on ARM port, we are focusing on the desktop and server workloads. We don't really need OpenSUSE running on our temperature control. It might be fun to do so. But so far, the devices are not powerful enough. They usually don't have virtual memory. It's a nightmare.
04:21
You don't want to go into these really, really low-power devices. You want to go into systems that basically behave like you exit this Xbox. That's the easiest path forward. Everything else can come later. So we're targeting those type of devices, like the Chromebook. Where are we right now?
04:41
Who of you tried OpenSUSE on ARM so far? It's quite a bunch of fans. Cool. Nice. The gold master for OpenSUSE 12.2 got released in November last year, so only a couple of months ago. The release was ARMv7 only. ARMv7 is that device.
05:01
The Calxedo box I was showing you was ARMv7. The Raspberry Pi, for example, that's ARMv6, so it was not supported by that release. However, since the early beginnings, we're building for ARMv5 as well. It's just that nobody sat down to work on it, but more to that later. OpenSUSE 12.2 is a release on ARM.
05:22
It's a release that we forked off of the official gold master of 12.2, recompiled completely against ARM, fixed all the issues in it, and released it, period. We don't have rolling updates. There's no maintenance on it. If you install it, you won't get security updates. It's just exactly the same as day 12.2 was at the time
05:41
it got released, plus bug fixes for ARM. Where does it run? Down here, it actually says what the board is called.
06:01
This is a Beagle board, exactly. It's a Beagle board XM, and the right-hand side is a panda board, very good polo. Oh, it actually really does say what the boards are called. All right, that's a Beagle board, that's a panda board. Both are based on OMAP processors from TI.
06:22
If you recently read the news, OMAP is dead. It's almost dead. They essentially laid off about 2,000 people working on OMAP and still sell the parts. 2,000 out of 2,500 people working on OMAP.
06:40
So those devices work really well because they got upstream support that works even with graphics, which is a novelty in the ARM world. So we got device support for Beagle boards, panda boards, and ARM versatile express, which actually is a lie. It's just a versatile PB target right now
07:02
that runs with QEMU emulation. That's why we added it. It doesn't even run well, so don't try to run it. However, we also provide one more thing, and that's a rootFS tarball. So if you got your own kernel, if you got your own bootstrapping mechanisms, just show off our user space on it and run OpenSUSE.
07:24
To explain why we have such a bad situation in device support, let me show you what booting means on ARM and how this whole stuff actually, not in detail, but roughly how the different components work there. So on x86, the world is really, really simple. On x86, we got a device, and this device
07:42
comes with firmware. Usually you're biased. Today it's EFI, but you have firmware that abstracts hardware to you. And then you have a bootloader that really is only a bootloader. It talks to the firmware, loads your kernel, boots your kernel. That kernel talks to your firmware because the firmware abstracts hardware from you.
08:00
The kernel doesn't have to care about what type of hardware it really is running on, except for a few device drivers, but it just runs. And that kernel can easily run user space. So is there any way to get that projector high up? Down there is a chameleon smiling, saying user space.
08:23
So it's a really simple, straightforward line, firmware abstracts hardware from you, kernel abstracts hardware from you. The bootloader abstracts everything from you that you don't care about as a kernel. Happy life. Simple and easy. On ARM, we don't have firmware. The bootloader is the firmware, usually.
08:42
There are some exceptions to the rule, but in general, you have a bootloader that is also your firmware, which means your bootloader is actually device-specific. Now, since it's device-specific, you don't only have a single bootloader, but you have a dozen of bootloaders for every single device that you have out there.
09:01
Now, the same thing goes, again, with the kernel, because people figure, if my bootloader is different between different devices, I can also make my kernel different between different devices. So now we have a device-specific kernel. And since there's so many devices out there, we have a dozen of device-specific kernels. Who wants to support all of those?
09:21
I don't. So that's why we basically say OpenSUSE only supports what's upstream, what devices we have. People active in the community actually do want to support, and what is reasonably sane to support. That's the chameleon, awesome.
09:43
So that's where the counterpart comes into play. So on OpenSUSE, we can only support upstream projects. There's no way we can just take any random kernel from somebody or a binary blob to support your 3D graphics or anything else that really is not just
10:01
a normal proper way of doing it upstream, as in upstream kernel, upstream U-boot, upstream, whatever. But upstream doesn't support all the devices we care about. Upstream actually usually supports only a really tiny fraction of devices people actually do care about. The Raspberry Pi, which has been popular for almost a year now, only recently got support in the kernel.
10:22
And I don't even know if it got upstream support in U-boot at all so far. It can. There's a port of U-boot to the Raspberry Pi, so you can use it as a second stage bootloader to make the disk life easier. Oh, sorry. Question is, comment was that there's no U-boot.
10:41
There is U-boot, but you usually don't use it on the Raspberry. Got a microphone there? OK, the microphone doesn't work. So he's saying that you usually have to have the binary blob to actually boot at all,
11:02
because U-boot doesn't even include the bits to boot the board. That's true, yes. As I said, it's a hard world. So we need some way for people to enable downstream ports, to enable ports to devices that we, as a normal OpenSUSE
11:21
community, as a proper, cannot handle. Now, how do we do this? We call this thing contrip. I just learned in the last talk from Red Hat, they call it remix, for whatever reason. For us, it's contrip. When we want to support a board that we don't support, there are basically three components we care about.
11:42
There's U-boot, there's the kernel that we don't know, that we don't have upstream, some downstream U-boot, and there's the actual user space that really is the core of OpenSUSE. Now, those three building blocks don't always differentiate
12:02
between different devices. In fact, if you want to bring it up on your own device, you only have to replace the kernel and the bootloader and still keep all the user space from the OpenSUSE project to enable your own device. And if you put those two together and shove it on an SD card, you can usually
12:22
boot any device out there. That's v7 based, because that's what we released it on. The boot screen works pretty well. You can easily take any random downstream kernels, boot them, tell them root devices there, and they just boot into a full-blown OpenSUSE distro.
12:43
Of course, without graphics drivers, because these usually are binary blocks and without usually any support for HANSI hardware, but you can get a system up and running, which means we're done, right? If I have a board, that's all I care about, because now I can run OpenSUSE on my board.
13:14
The question was, if I have Ubuntu on my ARM system, can I just replace the rootFS and be done? Yes, exactly.
13:21
You can, for starters, you can even change root into it. You just download the rootFS, change root into it, and work from there. The first demos we did on ARM, we didn't even have a kernel built. We just took Ubuntu, changed root into there, ran X on Ubuntu, ran all the window manager and all those tools inside the change root, pointed the X display variable to the X
13:43
that was running on the host, on the Ubuntu system, and it looked exactly as if it was an OpenSUSE system. It was kind of hacked up, but at least we could show that we made progress. So that's enough to get yourself running, right? Because now you have a system, you can just easily run OpenSUSE on there.
14:02
Everybody's happy. Well, yes, you are happy, right? But if you want to get this stuff to anybody else out there, they don't want to go through the same tedious procedure. So if we take those three components that we have, you can find them again in the OpenSUSE build service.
14:23
Anyone ever use the OpenSUSE build service, or do I need to explain what it is? I guess I need to explain it. So if you know Koji, if you know Buildy on Fedora, Buildy on Ubuntu, it's basically the same, just a lot more powerful. It's a distro building framework
14:42
that has automatic dependency tracking and can also build images. So inside the same framework, you can say these are my packages, I want to base the rest of my distro on top of these other packages, and I also want to create an image at the end of the day, and it does all the rest in a magic box for you. You only have to describe what you really want to do, and give it the sources, obviously, to build binaries.
15:04
So in there we have a project for the ARM 12.2 port. You also have your other blobs, your other source code that is downstream, that you don't really, you cannot push into the normal ARM repository for OpenSUSE.
15:22
So if we put those into a separate repository, we basically now have a bootloader and a kernel that we could boot, that we have at least RPMs of. Easy enough. We call this OpenSUSE 12.2 ARM Condrip your name,
15:40
the name of your board. However, if we also extend this and add a Kiwi description, Kiwi is the image building toolkit for OpenSUSE. If we add an image description, we can use this image description to then build an image that we can put on an SD card,
16:02
which gets us to the same result that we had before. Happy life, huh? So if you do a Condrip project, it means that anybody out there that can download a war file and DDS on an SD card can actually run OpenSUSE on that board that you were enabling
16:21
without you having to go through tedious cycles of convincing people that having downstream source code in OpenSUSE is a good idea. It basically is in the same namespace almost. It's not an officially supported OpenSUSE distro as in full, but the user space part is supported. So that gets more people on board
16:42
of running OpenSUSE on the board. So if you are enthusiastic about having people run OpenSUSE on the board you have, just do a Condrip and they can run it. So what's the state of devices? This is what I told you before, right? In 12.2, we only have support for those three devices.
17:01
In Condrip, we however have a lot more already. So there's support for IMX53. That was funny actually. Microsoft had a raffle where they were giving away free IMX53 devices for enabling Windows CE. Turns out Windows CE didn't really get enabled on those.
17:20
There's Hibank, which is the Calxida boxes, origin boards from Samsung, QBox is a small black box that also runs on B7 on Marvel chip. Tegra, I don't think the Tegra directory actually has working code yet. There were efforts to get AC100 running, but I don't think the Condrip directory there ever produced a working image.
17:41
Arndale is pretty fun too. The Chromebook you actually just saw over here, it runs just fine. Let's take a look at the Arndale port so you have an idea what a Condrip directory really means. So this is the web interface to the OpenSUSE build service. Over here, you can see what files are
18:02
in that Condrip directory. That's really not a lot. The status of the builds, and you can see that it's called OpenSUSE 12.2 Arm Condrip Ande, because it's the Conda directory for the Ande board. In there, we have the image description, kernel, and bootloader.
18:22
That's the three pieces we need to get a system up and running. We also have a special queue in there because I did the Ande Condrip and I like KVM. So I really wanted to get the Ande board working with KVM, so I instead put a kernel in there that can run that has all the KVM patches inside. I put U-Boot in there that has all the KVM patches inside
18:42
and I put a Qemu in there that also has all the KVM patches inside. So I can just take that image, put it on my Ande board, and get KVM running. So with those pieces, thanks to the image description, we now have an image that you can just dump on any Ande board, and there you go. You have OpenSUSE with KVM running on your Ande board.
19:02
Fun, huh? It's only five different pieces to get there. You can download it from the normal repository side. It appears almost as if it was a normal build, just that it doesn't get into the official release directories of an OpenSUSE release.
19:25
So with Condrip, it's really easy to port OpenSUSE to your device. Just said that earlier. The important bit is that the OpenSUSE community is not responsible for your specific device you enabled.
19:41
So if you want to get updates into that system, if you want to get a newer kernel that has fixes in there, for example, the Chromebook currently has a kernel that blows your speakers if you enable them. So if you want to update that kernel for that Chromebook, you should basically update the kernel in there once you find a bug that won this on my to-do list.
20:03
You don't, however, have to touch anything generically in the generic parts. So for the next releases, we are obviously aiming for not only a distro that is released on a gold master that never gets updates. You obviously want to have updates as well. So if you want to enable your device,
20:22
you only have to care about your kernel and your bootloader and your packages, your overworld. But apart from that, all the rest of the distro is just generic bits. Even the image is built from the generic parts, so you don't have to reboot anything at all. It just automatically gets done by the build service. It pulls in the binaries. And you're all happy. You don't have to worry about supporting the full distro.
20:46
For the future, open to the 12.3. Currently, the current release is 12.2. 12.3 is the one that gets released soon. Where are we on 12.3? Well, right now in the 12.3 fork in the,
21:01
so usually we have this code stream called factory. And in the, as soon as there's a release nearing, we are forking off that factory tree into a specific version tree. So right now we have a 12.3 fork of the OpenSUSE tree. That one is still compiling since ARM hardware is fast.
21:24
Because on this one, we're doing something really novel. On 12.2, we were building everything on x86 boxes. For 12.3, we're building everything on ARM boxes. However, we don't have so many ARM boxes. In fact, until a couple days ago, we only had a single ARM box building it all.
21:42
So that build is not really finished so far. Lacking somewhat behind x86. But thanks to that, we don't really have any decision whether we are able to release the OpenSUSE 12.3 ARM port in time for a 12.3 x86 release.
22:01
If we don't, well, we're just gonna release it later. If we don't manage to get updates rolling, well, we're just gonna postpone that to 12.4. There's no decisions on that yet. But obviously, we are always trying to aim and align with x86 if we manage to. So this is what the 12.3 repository looks like
22:21
in the build service. Over there, you can see there's a ports repo in there that also has ARMv7 activated. And in there, this is a screenshot from this morning. We are building six jobs in parallel and have about 1,300 packages to go.
22:41
Which basically should be done by tomorrow or maybe the day after tomorrow. Unless somebody checks in gdpsi again, which just happened on Wednesday. So 12.3 is going to be the first ARM release that we don't have to fork off of to have ARM build properly. So during that 12.2 work we did,
23:02
where we had to fix a lot of packages to build properly on ARM, we took all those changes and merged them back into the factory tree. Which means 12.3, which came out of that factory tree, has all the fixes included already. So we don't have to change the factory, the distro that actually is 12.3, to really build on ARM,
23:21
because most of the stuff just already does work since we did the work earlier in the previous build. But I want images which we built on an internal build system, which doesn't have some security constraints that we have on the outside system, which I'm gonna show you later. Has already finished building, because it only takes two days to build on that system.
23:42
And there you have images that you can download and test if 12.3 works for you, if that image actually does run properly. Thanks to that internal system, we also have stats on how many packages actually fail. And as you can see, there aren't that many. Most packages just work. In fact, there's only 3% of packages that don't work.
24:04
Just really good ratio. I mean, I don't know if you ever ported a distro to a different architecture, but PowerPC is way worse off. Exactly, because PowerPC is big engine. So, how do you build all this stuff?
24:22
You see there, the OBS logo, best logo ever. Building, the OBS has workers. A worker node is basically just a box that runs build jobs. And then we have build jobs, which is this sheet of paper, that's a build job.
24:42
Now imagine somebody from the outside comes in and puts malicious code into this build job so that they would be able to take over our worker node and then create binaries with OpenSUSE signatures, for example, to take over the world and on an update read out all your data.
25:01
Once that thing gets clear to the worker and we would just execute it, hell would break loose, right? Because anybody who adds a job to the public build service where anybody can add a job, we would just run that malicious code. So what we do instead is we create a jail around it, which usually is a VM.
25:22
So we run every single build job inside of a virtual machine. That virtual machine these days is KVM, we only have a few boxes in the build system that are still running on Zen because they don't support KVM yet. As you can imagine how old those boxes are, they're not really very fast either.
25:42
So the problem that we're facing on ARM is that ARM hardware usually, until really, really, really recently could not be virtualized, at least not properly. There were ports, there was a Zen port from Fujitsu that was doing PV on ARM.
26:01
I don't think I've ever seen an upstream project that was in such a bad shape as that one. There's no way you could get that running, especially not because there's essentially no board support in there either. There's Elgast. Have you ever tried to run Elgast on ARM? So Elgast on ARM is not even in the official Elgast repository.
26:21
There's some Chinese guy that forward ports Elgast to a board that he owns, which I think was a Beagle board. And it somewhat works for him on his board on the specific kernel version that he's running, which obviously is no, I mean, we can't possibly base on that for build service. And KVM, which would be a nice fit
26:41
because everything else is running KVM and I'm a KVM guy, only runs on Cortex-A15. This device is a Cortex-A15, the ARM board is a Cortex-A15. We don't really have that many devices that have Cortex-A15s in there yet. There's essentially those two, period. And they only got released end of last year.
27:01
So the normal build process that we would want to have is this one, right? We have ARM hardware, there's a VM on top, and then we just run our build job and everybody's happy because this gives us the isolation we need. Well, we don't have KVM on almost all of the boards we own. So what can we do instead?
27:21
We run on X86 boxes. There's this cool thing called Qemu which is able to emulate ARM hardware on X86 machines. It can even emulate ARM Linux binaries on X86 Linux systems, which is faster than emulating an ARM operating system on top of an X86 system.
27:41
So here we get the isolation we need because we can run everything in KVM on X86. Then we had the emulation layer so every build gets even slower than it would on real hardware. And then we can run our ARM build job inside there as if it really was running on ARM hardware. Can anyone imagine this is slow?
28:01
All right, good. We have a few nifty hacks, mostly inspired by Jans back there. Jans wrote from a couple years ago where we replace part of this ARM build job, almost half of it, by X86 binaries and then bypass the emulation layer so that instead we execute X86 binaries
28:22
inside the ARM change root. Don't do this at home. It's a really fragile process but it was the only way we could ever get finished with an ARM build. And it's also one of the reasons we switched to native builds right now because the acceleration package that we need
28:40
to enable this simply didn't get pushed into factory because it was so hacky. Turns out we started running an emulation, I mean every build job in emulation which was about 10 times slower than you would want it to be. So 12.2 was completely built on X86. We didn't build on ARM hardware externally.
29:04
Internally we always had a nice build cluster built on panda boards that was building everything natively to compare whether a bug is a bug in a package or a failure is a failure in the package or whether it failed because of QEMU emulation.
29:21
Turns out I actually got really nice patch statistics in QEMU from that time because there were a lot of Linux users based simulation fixes that came out of that work. So we are for 12.3 aiming for native builds. This works because we now have an R&D board
29:43
which can run KVM, so that's fine. And for the next two or three weeks until we have more R&D boards, we even put the panda boards into there and only build trail three on there where we trust everybody who pushes commits. So if you own a package on trail three and you push something, please don't put any code in there that could own our cluster.
30:02
What's missing? We don't have ARMv5 releases. ARMv5 is essentially our code word for soft floating point releases. ARMv7 includes hard floating point, ARMv5 does not. ARMv5 can include hard floating point in there. I mean basically it leaves it open
30:20
to whoever produces the chip whether they want to include a floating point engine. So they may chip without a floating point engine at which point we cannot rely on a floating point engine to be existed. There are a few instructions that incompatibilities do but you can basically ignore those as well. So that one's missing. The reason for that is easy.
30:42
I'm gonna come to that in the next slide. We also have another thing that's missing and that's AR64. Anybody been to the talk from John, right? John Masters, I also get his name. Not too many. There was an interesting talk. Fedora is currently working on porting their system
31:02
to AR64 which is the next 64-bit release of ARM. So let's take a look at the old ARMS, ARMv5. I'm not gonna look into anything before version five at all because anything before version five, you don't want to run a real disk to ARM, at least not a recent one. So ARMv5 is what we would need if we wanted to run on a Raspberry Pi
31:22
which is the real shame, I mean the major shame that we don't actually support ARMv5. We always were building ARMv5 as well. Bernhard was pushing for ARMv5 support from the early days because he's got a lot of old crappy ARM devices lying around. He wanted to run OpenSUSE on.
31:40
They were only based on ARMv5. However, you have to have a certain amount of people working on a port so it gets shaped up, polished properly enough so that it actually compiles through fully and can be released as a full OpenSUSE port. We never got to that point in ARMv5.
32:02
So for ARMv5, we never really released anything but you can always just download the repositories from our tree that is based on factory and get binaries from there. We have ARMv5 available, it's just not a release. I think we need a microphone now. Guys in the back, microphone.
32:25
I can probably repeat it but since there's the third question right now, I would like to have the microphone close to me so I can jump to whoever gets the question, ask the question so I can not repeat everything all the time. So there you go. So the ARMv5 or the binaries in the OpenSUSE repositories
32:43
are compatible then with ARMv5. There are two repositories in the OpenSUSE factory ARM repo. So let me show you. It's probably the easiest way to get there.
33:10
Nice screen resolution. Now if the maximize button was actually implemented correctly. Oh, cool.
33:21
So there's build OpenSUSE.org's OpenSUSE factory ARM. That's the factory ARM repository. Is anybody downloading ISOS right now?
33:44
There you go. That's the ARM repository. You can see down here, down here we have a standard repo with ARMv7 activated where we have built a lot of packages. Those are actually built on x86 machines too. Up here we have an ARMv5 repository
34:00
where we also have a lot of packages built and that one creates binaries that run on all ARMv5 or ARMv6 or even ARMv7 devices. They just don't use the floating point engine. That's all, thanks. They are in a different repository
34:20
so you cannot use the same binaries but there are binaries available. So the more interesting part really is RH64. There was a really nice bus ride from Nuremberg to Brussels with three people working on the ARM port and a lot of borrowing time where I didn't want to write my slides.
34:41
So we turned up and actually tried to port OpenSUSE to RH64. That went really well. So we are currently in the bootstrapping process. So you can see this is an internal build service instance where we did a lot of hacks so we can enable the foundation model that ARM gives us instead of KVM to run a virtual machine.
35:02
So we just create like a small root file system on the outside where we put all the packages in. That's all logic that the build service already knows how to do. Then we boot up the emulator and the emulator then goes in and executes the build job for us. Timing-wise, the build jobs take about 100 times
35:21
as long as an x86 build job does. So if you need an hour to build something, it takes four days. But at least you make progress. That's the nice thing about it. You can see here, it's actually building so we already have nodes attached to it that are building binaries right now.
35:41
However, the C compiler doesn't work yet. We are getting to the build system. It's running, but the C compiler doesn't produce code yet because they can't find some libraries. It's a hacky nightmare that's going on right now. I'd assume that within the next couple weeks, there will be at least a bootstrap from OpenSUSE on AR64.
36:02
There's also discussions going on on the main list about it, on how we go. We are roughly about at the stage where Red Hat is because that's what we took as a basis. Now, what we did to get there is we enabled support for the emulator and the build script. However, we hacked it up.
36:21
So far, those patches to the build script are not in the public build script because you really would not want to run this build script on any system except for the one where we hacked it up unless you wanna completely shoot yourself in the foot. But this is most likely going to be
36:40
in the public build script anytime soon. So then you can even take your local system and just run OST build on any AR64 repository and you can locally build an AR64 binary even for Fedora if you like. So how can you help? Who wants to help?
37:04
All right, all right. It's really easy. It's essentially the same as in any other distribution. Test images, get contour directories up and running, enable hardware with them, fix build failures even though that's getting tricky now because if only 3% of packages fail,
37:22
that means that the heart's 3%, not the easy 3%. There are wiki pages that are always outdated because we're developers. So if you feel like writing wiki pages, that would be an awesome help. The mailing list is always there for recommendations, help, interesting discussions,
37:41
and we also have an ISE channel where everybody of you obviously is welcome to join us and idle with us. If you are a packager, anybody packages OpenSUSE packages or packages on the OpenSUSE build service? Okay, this is for you guys. OBS, thanks to all of this,
38:00
supports ARM throughout the board. So all you need to do to enable ARM builds for your package is in your project configuration which I actually wrote wrong. It's not the project conf, it's just project, OST meta project. You just add this XML blob and suddenly all your packages start building
38:20
against OpenSUSE factory form. There you go, that's it. Build service automatically does all the heavy X86 emulation lifting and acceleration bits that you really don't want to see. And at the end of the day, you get a binary or you get build results that tell you why your binary, why your source code
38:41
would not compile on ARM. And that way you can ensure that when you submit a package to factory or when you want to give somebody an ARM binary, you have all the tools available to do so. You can also build locally. So if you execute OST build on a repository that has this bit enabled for ARM,
39:02
you can, the build script would automatically call QEMU inside the change root or virtual machine for you. So you would automatically get full ARM cross emulation support on X86, at least today. I don't know how that's gonna evolve as soon as we have proper built hardware that is based on ARM, but for now
39:21
you can easily just do that. Any questions about all this stuff? I'm pretty sure there are questions. I mean, you have probably some devices you want to have enabled that you're asking, why don't we have them?
39:44
Of the contrib repos, is there some repo that works with QEMU? Yes. So you're probably asking about QEMU system emulation. So if you have an X86 machine, you don't have an ARM machine, but you still want to test the whole system,
40:01
including a kernel and the whole bootstrapping process, whether to, I mean, how to emulate that one. And QEMU also supports a couple of platforms that you could base on. QEMU supports three platforms that we also support. QEMU supports the versatile Express, that one's already in. QEMU supports high bank. You can always use the high bank support
40:21
to boot our kernel, to boot our system, that one's in the contrib. And there's a downstream QEMU from Peter Madel. Peter Madel is the ARM maintainer, but he's got really high standards for code quality. So he basically says the code that he has in his tree, he could never commit to upstream because it's so crappy code. So he has beagle board emulation in his code tree
40:43
for maybe three years by now, which he always has on his to-do list to polish and properly, cleanly get upstream. So far, it's not upstream, so you have to use the QEMU linear report to get that one. But if you have that, you can even use the QEMU linear report to just boot our beagle image.
41:00
That's basically the choices you have for contrib, I think. There's one more for Tegra. I think Andreas Faber is working on a Tegra port for QEMU emulation as well. So, soonishly, there might be a Tegra emulator.
41:20
Unfortunately, he's also the contrib maintainer for the Tegra repository in OpenSUSE. So I would hope that as soon as he pushes it in, OpenSUSE would also run on that same emulation he's writing. Yes? So here's a crazy idea. Get an OpenSUSE root file system,
41:40
at least maybe a stripped down OpenSUSE root file system running on an Android phone. Oh, that already works. How would one begin? So you would begin by- Obviously, you need a rooted phone? You need a rooted phone. And then, as you can see in that, where is it?
42:01
This is the factory repository. So I'm just gonna show them based on that one, but it's the same idea for 12.2. You can go to the image directory, and.
42:22
All right, we are going to look at 12.2 after all. If you look at the image directory, you can see down here, there's a rootFS image. RootFS image really just is a change-root environment. So on your Android phone, if you wanted to run anything on there,
42:40
you would root it, download this tarball, extract it, and then change it into the tarball, and there you go. You've got a super SUSE environment. Because the kernel is still the same, it's still a Linux kernel. It's not the same, but it's a Linux derived kernel. All the calls to the Linux kernel would still work. If you somehow find an X server that runs on Android, you could even then,
43:04
insert the change-root, set your display value to your X server inside of Android, and just push everything there. Want to pass here? I just want to say there is an X11 server for Android. It's, I think, written in Java.
43:22
Sounds fast. It, it works amazingly good. The only thing it doesn't is running wine. So, not perfectly, but I think for the most things, it works for,
43:41
yes, most likely everything except the wine. Cool, cool, that's good to hear. So, apparently it's really easy. And change-rooting is the same as any other system. It's like on x86, just change-root. More questions?
44:02
All right. Since you are anyway running with QAMU, so you already have some protection. Do you think that using Linux containers would be faster than KPM?
44:24
Or just more dense? It would be different from all the other build jobs, which means it's more work to maintain. This is not a question of how fast is the build job. If 5%, roughly, will not buy us really big improvements. If we compare that to the time it needs to actually make sure this stuff all
44:43
stays stable and works well. Because if there's a single build failure based on anything, or if a build host just completely doesn't work because we screwed up our setup, our build script, so we forgot to close braces or so. That completely nullifies all the, all the improvements you would have done.
45:03
Getting, I mean, arch64 is a different, different, different story. Because having 100 times slow down means that even smaller improvements actually help a lot. So there we are thinking about different ideas on how to improve performance. But also that one, not in the official open to the build service.
45:23
So this is happening in a separate build service instance that does not affect normal production build service use. Because we don't wanna break anybody else's builds. All right, thank you guys.