We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Powering the Jump: enabling a new delivery model for openSUSE with the Open Build Service

00:00

Formal Metadata

Title
Powering the Jump: enabling a new delivery model for openSUSE with the Open Build Service
Title of Series
Number of Parts
45
Author
License
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
This will be a keynote delivered by SUSE's Markus Noga.
28
Thumbnail
19:15
BuildingService (economics)Open setData modelHydraulic jumpExecution unitMaxima and minimaComa BerenicesService (economics)Information securityGroup actionBus (computing)Public key certificateProcess (computing)Online helpBitBuildingTouchscreenOffice suiteEndliche ModelltheorieHydraulic jumpComputer animation
Electronic data interchangeExecution unitLink (knot theory)Service (economics)Source codeInstallation artOpen setFile formatStress (mechanics)BitProcess (computing)System administratorNP-hardShape (magazine)Order (biology)WindowForm (programming)Data storage device
Menu (computing)BuildingWindows RegistryService (economics)Open setComputer-generated imageryArchitectureSource codeSoftwareChainExecution unitComputer fileCompilerTemplate (C++)MathematicsBlogLocal ringCurvatureInformation securityArmScripting languagePoint (geometry)Service (economics)CoroutineInclusion mapGastropod shellParameter (computer programming)Descriptive statisticsGoodness of fitMatrix (mathematics)Moment (mathematics)MathematicsMedical imagingData managementSource codeInsertion lossMereologyDistribution (mathematics)WindowAreaServer (computing)Network topologyOpen setLogic gateBitPower (physics)Field (computer science)Windows RegistryCore dumpComputer clusterOrder (biology)Error messageWhiteboardMetropolitan area networkForm (programming)Extension (kinesiology)ChainSoftwareInformation securityGroup actionWordConstructor (object-oriented programming)System callComputer architectureSocial classLevel (video gaming)TwitterMilitary baseTouchscreenLie groupAssembly languageOpen sourceRevision controlAddress spaceRepository (publishing)Key (cryptography)PowerPCBuildingComputing platformVulnerability (computing)Proof theoryRootDrill commandsCodeTemplate (C++)Binary codeQuicksortUniform resource locatorIntegrated development environmentArithmetic meanVariety (linguistics)Patch (Unix)Computer fileLine (geometry)Enterprise architectureBootingFunction (mathematics)Library (computing)Basis <Mathematik>Virtual machineVector potentialEntire functionChief information officerComputer animation
MathematicsMenu (computing)Gamma functionHydraulic jumpSoftware developerBitCodeDirection (geometry)MathematicsOpen setGroup actionDisk read-and-write head
Source codeSystem of linear equationsSingle-precision floating-point formatFactory (trading post)Power (physics)ArmArchitectureSoftware maintenanceHydraulic jumpInstance (computer science)Enterprise architectureDistribution (mathematics)SuSE LINUXSoftware developerData storage deviceBinary fileContent (media)Normed vector spaceComputer iconMaß <Mathematik>PlastikkarteNetwork topologyOpen setSoftware developerProjective planeLevel (video gaming)Group actionWater vaporProcess (computing)Coefficient of determinationSeries (mathematics)Arrow of timeGame theoryDirection (geometry)Goodness of fitTwitterOpen sourceWechselseitige InformationMathematicsComputer architectureEnterprise architectureException handlingQuicksortPerturbation theoryStapeldateiSelf-organizationComplex (psychology)Content (media)GoogolSoftwareServer (computing)Row (database)Instance (computer science)Compilation albumNeuroinformatikHypermediaAdditionWebsitePower (physics)Physical lawSystem callConnectivity (graph theory)UsabilityDistribution (mathematics)Limit (category theory)Shooting methodThumbnailDependent and independent variablesPublic key certificateComputer configurationInformationDataflowComa BerenicesCore dumpRing (mathematics)Total S.A.Flock (web browser)WhiteboardConfiguration spaceBinary fileFlagUniverse (mathematics)BitHydraulic jumpBinary codeService (economics)CompilerIntegrated development environmentSystem of linear equationsSource codeSystem administratorPerspective (visual)Subject indexingData storage devicePhysical systemForm (programming)Axiom of choiceRevision controlPowerPCEmailElectronic mailing listArmSoftware maintenanceStreaming mediaInteractive televisionBuildingData managementPoint (geometry)Entire functionRegulator geneSource codeComputer animation
Maxima and minimaElectronic data interchangeVector potentialElectronic program guideBoss CorporationSlide ruleBlock (periodic table)Integrated development environmentFactory (trading post)Order (biology)BitCartesian coordinate systemMereologyStack (abstract data type)Decision theoryPhysical systemCuboidSpeech synthesisCondition numberData storage devicePoint (geometry)WindowSystem administratorEnterprise architecturePower (physics)Projective planeLink (knot theory)Open sourceSystems integratorSign (mathematics)Binary fileProduct (business)QuicksortMobile appTime zoneConstraint (mathematics)Shape (magazine)Level (video gaming)Cycle (graph theory)Sweep line algorithmMusical ensembleComputer configurationDemo (music)Group actionGreen's functionDistribution (mathematics)GodArithmetic progressionNetwork topologyOpen setCausalityPerturbation theoryArithmetic meanProcess (computing)Crash (computing)Physical lawNear-ringPresentation of a group
Transcript: English(auto-generated)
Hello, and this is Marcus. Can you hear me all right? Yes, now I can see your screen. And you should see a green screen at this moment. Yes, yes, yes, I can see it. It was still loading. Fabulous. So glad that we have that sorted. A warm welcome, everybody. I'm very glad to be speaking.
I'm very honored to be speaking at this OpenSUSE deeper office conference. And the subject of my talk will be Powering the Jump, how we enable a new delivery model for what OpenSUSE is looking to achieve with Jump with the help of the OpenBuild service. My name is Marcus Noga.
I'm a relatively new joiner to SUSE. I'm vice president for solutions technology. My group looks after everything that includes solutions for end customers, as well as all the base technology, including the build service itself, but also level three support, security certifications,
and everything we do for technical partners and technical partner certification. I'm proud and honored to be here today with all of you. So let's talk a little bit about the OpenBuild service, its history and how it pertains to containers. The OpenBuild service is something
that many of us just take for granted because it's there, it's invisible in the background. It does its job and it helps us turn source code into something that users trust and administrators trust and install on their system, which is binary executables that everybody can work with.
And the OpenBuild service has historically been about openness in any shape and form, whether we target RPMs, whether we target divine images, whether we target ISOs or other new formats like containers that are upcoming and gaining a lot of traction.
So the OpenBuild service has completed a longer journey for adding container support as a first-class citizen. And that journey began in 2017, so three years ago, when container building and the first capabilities to churn out container images on the basis of packages
was added to the tool. And that led to a second milestone in 2018, when we figured out that all these containers also need to be registered somewhere, they need to become discoverable, they need to become downloadable in an easy way.
So OpenBuild service added a built-in container registry with automated publishing potential. And fast forward to 2020, I'm very happy that we've also added Helm chart support, so we can't just put out individual containers, but entire assemblies of containers into services
and into applications, and that the first V3 Helm chart is actually public and available on our registry at this very moment. The address of that registry, of the OpenBuild service registry, which you also see screenshotted on this document,
is registry under opensus.org. And at the moment, we have about 1,100 containers in this registry. Okay, checking slightly, I think it's 1,089, but that's close enough to 1,100 to make that claim. We're also running about 3,000 container builds
every single day, because the build service includes automated dependency management. So whenever a package that a container has a dependency on gets updated, OBS will automatically rebuild that container image and make it available on the registry.
For all supported architectures, whether that's ARM 32-bit, ARM 64-bit, the PowerPC solutions, Intel 64-bit, and other platforms, the OpenBuild service registry supports all flavors of container target architectures. And at the moment,
we have about 48 gigabyte of images under management with the OpenBuild service registry. It's easily accessible for you at the URL on the headline via podman commands that are also available from the command line. What makes the OpenBuild service so strong
in a container environment and in a trusted environment is that it really engenders trust in the form of a secure software supply chain. So when you double-click on any particular package, and I'll continue with the example of MariaDB,
which was also depicted on the prior patch. When you go and double-click on any particular package, you can be sure that that package is trustworthy because it's been signed using both notary and open container initiative conventions so that we have a rooted trust for where the binary comes from.
You're also able to transparently see what went into the cake. Now we will show you how the sausage is made. The build instructions for that very container are accessible from the repository. You can see the scripts, you can see the templates that created it, and you have the snapshot sources for each container
available at the point of build, and this is a very important part of software trust. Which CIOs today really have transparency about their container landscape? Which of them are able to say with certainty, all my containers are patched
against the latest vulnerability? Okay, maybe not the very latest because of the Bluetooth vulnerability that became transparent yesterday night, doesn't hit servers, but the second latest, which might be boot hole or another prominent vulnerability. How do you see that your images
are actually proof against these? It's only by having a root of trust, by having transparency about what went into building the container and by having drill down and transparency on the sources for each of these containers. And this is where the build service is extremely strong because there's no outside dependencies
and we're able to rebuild even extremely old packages based on that root of trust and open source code and build instruction repository approach that the OBS takes. How do these images get built? When we further double click
on the MariaDB example here, you can see that the build instructions for that container come in the form of key instructions, this sort of an XML description file on this page, which shows us the author, which shows us the contact details,
the specification, and then the kind of image that we use in order to create that particular binary. And the build instructions for this can support all possible targets in modern enterprise compute, whether it's Azure images, AWS images,
virtual machine images, whether that's ISOs or containers. Kivy allows you and the open build service allows you to create a multitude of targets automatically whenever sources get updated and committed based on that description and that matrix of targets and sources.
The neat thing about building with Kivy is also that you're able to create base containers, which have no from in the Docker file, which have no base container that they rely on. They can become the base of an entire system, which gets interesting. For example, with distroless approaches
to software distribution. And a lot of people in the Golang communities, for example, following static leading binary approaches, should see the strengths of this fairly immediately. Of course, Kivy is not the only path and it's not a mandatory path.
Our open build is built templates with a few extra features and capabilities that take the manual out of operating Docker. Meaning to tag your containers with build tags, that's something we have extensions for and you can assign build tags
or repositories automatically. And you're also able to choose whether in that Docker container, you would like to reuse RPMs with the zipper run command that's commented out here on line 11, or whether you would like to put files indirectly or compile files directly
with the added names or copy lines in line 14 and 15 of this example file. So you get full flexibility in an open world where it's any kind of package distribution format, any kind of container format, and the ability to transparently create from your sources, from your project,
for your package that broad variety of dependent binary outputs. Why should you do that? Because you get automated rebuilds for free, including always latest updates based on dependent packages. In a container world, it's also very convenient
to see the full dependency graph. What are the affected containers for my package update? Who's actually using this? And what do my downstream users look like for a library package, for a service package, or for anything that goes into a broader appliance-like container?
You can also very easily see and inspect images for build origins to have traceability and that secure software supply chain and auto-generated changelogs that enable you to visually and humanely inspect what's going on with container updates.
We're not alone in putting this out. So at this point, I want to give a big shout of thanks to our community. The OpenBuild Service team would explicitly like to thank Markus Hüth, Tina Möhr, Berthold Gern-Ribben, Oleg Gerko, and Neil Gompa for their strong contributions
to the OpenBuild Service. It's only with contributors and with an active and engaged community that a platform like this can continue to evolve and continue to address everybody's needs. So thank you. Very, very well done. And we really appreciate your contributions.
With that, I want to switch over a bit in the direction of building OpenSUSE and what are the changes and what are the new developments that this entails if we embark on a journey together that's currently proposed under the code name OpenSUSE Jump. So let's have a quick look at how OpenSUSE
actually gets built today. What are some challenges about that and what could we do to address that to become even better, even stronger, and dare I say, slightly greener in all of this together. So this is how OpenSUSE gets built today.
It's really a single project built. We're taking all the sources from the enterprise distributions, use of index enterprise from the OpenSUSE factory, and we take all of these sources and inject them into an OpenSUSE leap or OpenSUSE tumbleweed environment.
And there's additional layers of projects that go on top of this, for example, for the ARM environments in 32 and 64-bit or for the PowerPC environment. That's a grand total of more than 12,000 packages that get compiled with various flags
and sort of configurations into roughly 70,000, 71,000 packages per architecture. And that compile process then gets frozen once we release, and we continue to do back ports,
we continue to do maintenance updates. The back ports, particularly our service for SLE customers. They could look at this and say, well, this is very open, this is very transparent. We just take all the sources, we use that fantastic open build service engine that we have, and we churn out the binaries
that are required to run this on any architecture. What's there not to lock? Well, it turns out that there are a couple of challenges and issues around us that I want to have a look at together with you. So one of these challenges is we really don't have binary compatibility
between OpenSUSE and SLE because we're compiling slightly different changes, slightly different compiler versions, slightly different layers that get added. So there's no clear one-on-one big wise compatibility between the two. And that's actually a bit of a disadvantage here.
End users who would like to use open universe packages, for example, have a bit of a challenge doing that. And the experience for end users could be better. It's hard to explain to them on why the packages shouldn't directly binary work together. The same thing is we have a bit of duplication of effort
for the package developers, for the container maintainers and more because your needs need to configure and set things up so that they work in both environments. We also have a bit of an echo issue because we're going to compute and compile everything twice. We're going to store everything twice.
And with the many, many gigabytes under management and the many, many thousands of compiles per day, we have over 200 workers, for example, running and making these compiles. There's also a bit of an echo footprint issue. So the proposed approach with an OpenSUSE jump
makes this interlinked much, much more explicitly. We take this SUSE internal component, the binaries, and we mirror these to the public OpenSUSE instance. That's a huge win because suddenly we get binary compatibility because we can work directly with the binary. This also means that submissions to SUSE Linux Enterprise
can be triggered directly from OpenSUSE, making SLE the first enterprise distribution that can really be developed in the public, thanks to all of you. I want to say at this point that this isn't 100% possible
for the entire SLE distribution. We couldn't take that completely public at that point because of certain requirements around software certifications. It's not just the software product that gets certified for certain kinds of federal and regulated and government users. It's also the software development process
that gets certified. So we can't take the entire SLE and open it up, but this is the best step that we can take to become ever more open and to remain ever more open in an environment that brings OpenSUSE and SLE much closer together by making that jump. So what does that look like on the build service?
And what does that look like from a build approach perspective? Here's what OpenSUSE jump will look like from the build perspective. So we're going to use multiple projects and these projects will aggregate into an OpenSUSE jump on a binary level.
We can bring in SLE 15. We can bring in internal build service packages. We can bring in back ports from OpenSUSE more or less directly in a binary form as that black dotted binary stream. And at the same time, we can push requests and content
sort of in the form of sources back into all these channels. That's the red arrows going the opposite direction. And that's this sort of two-pronged interaction where we open up everything we can except for that certified software development process.
We move OpenSUSE and SLE much closer together and we achieve that binary compatibility which makes the world a lot easier for end users that consume the software for administrators that work with it and from the storage and footprint and general simplicity perspective.
And that in the end gets installed on a system that can consume updates in a consistent form from any of the channels, whether it's jump whether it's back ports or whether it's the proper SLE itself. The power of this really means that end users
have more choice with less complexity and with more ease of mind because they know everything works together. We're happy to support this thanks to the power of the build service which is running as an open build service for everything OpenSUSE and which is running as SUSE's internal version
of the build service called IBS to power the SLE distribution itself. And it's actually the same software the same build environment that powers both of them in two separate instances again for certification reasons. We'd love to make that jump together with you.
And I know it's being discussed intensively at this conference. So here's the outlook. You can learn more in additional conference sessions on LEAP and why both developers and companies should be interested in the LEAP approach. My colleague Lubos is giving these talks today and I believe tomorrow
and there should also be recordings available. The next thing you can do is to keep current by joining the OpenSUSE project mailing list where you will be involved in a vigorous discussion and involved in information flow on everything concerning OpenSUSE and the way we propose to build
and the way we propose to integrate. And last but not least, if you're at this conference because deeper offers is what interests you and what you thrive on consider becoming an OpenSUSE developer as well. It's by your contributions that OpenSUSE, the OpenBuild service and the general open source environment here in Europe
can thrive and to continue to prosper in future. Thank you very much. And I'd be very happy to open up for questions at this point.
Now you have to hit that button labeled unmute if you would like to ask a question in a big form like this. Well, I'm pretty happy to see how well this is going.
So with this seemingly interesting ambition with OpenSUSE jump, what do you hope you'll get from this in the end? What do you want to achieve with this new sleep fusion?
Yeah, I think the biggest thing is probably simplicity for end users. I'm passionate about user experience and customer experience. And whenever I meet with my teams to look at demos, to look at new things that they've created
or new things from upstream that they've incorporated and packaged, I will put myself in the shoes of the end user and ask, is this simple? Is it easy? Would my 79 year old mother with a history of being challenged with IT systems actually understand this?
Could she use this? What's the potential for tripping up? And I believe by getting binary compatibility between the distributions, we're removing a lot of potential for tripping up. In an age where more and more users are coming into Linux environments again, more and more cost constrained households
adding secondary and tertiary devices for children and more and more enterprises going towards a Linux environment and Windows administrators facing all kinds of challenges, having to retrain and relearn to, for example, deploy their enterprise applications on a Linux stack.
Making things easy, foolproof and fail safe because stuff just works together out of the box is probably the biggest price that we can shoot for. Put yourselves into the shoes of a recent college graduate half a year into the job, living in Mumbai, working for a large system integrator
and having a 20,000 dollar best practice guide to install and configure something, you're going to make mistakes. You're going to look bad in front of your boss and in front of the customer. The more stumbling blocks we move out of these people's ways, the more adoption we'll get
and that grows the power of open source for everyone. So removing these stumbling blocks, simplifying and making the consumption experience as easy as sort of downloading something from an app store. I think that's the main price that we can go for. The fact that we will also cut the echo footprint of what we do in half is a pleasant side effect.
Wow, that's, I think quite a bit more than I expected to hear on this. But yeah, no, it makes sense to me. I'm looking forward to seeing how well has this been going so far?
So right now, this is being discussed intensively and I believe the proposal is being debated in various parts of the open source project as well as in SUSE itself. I have my fingers crossed for this actually becoming reality. May I comment on this one?
Please jump in. Thank you, this is Lubos speaking. Hi, I'm very glad to have you here. And thank you for the talk, it was really interesting. So regarding the progress, my slides will have a little bit of what you are asking, Neil, I will summarize where we are, what are the problematic parts,
there are some problematic parts, there are some parts where we are fine. And there should be a sign off, let's say go now, go next week, whether we would like to proceed with the intermediate release, lead 15.2.1 and basically enroll this into production before the lead 15.3 is released or whether we postpone it to the next release.
So that may actually answer exactly what you are saying. Based on that meeting, we will see where we are. There is a bunch of conditions that we have to align with in order to actually proceed. I can share your link if you want to, but I will also make it part of my presentation. You can also see the confirmation of stakeholders
for the decision on the OpenSUSE project and factory. I believe that.