Isar Build Debian-Based Products with BitBake

Video in TIB AV-Portal: Isar Build Debian-Based Products with BitBake

Formal Metadata

Title
Isar Build Debian-Based Products with BitBake
Title of Series
Author
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
2018
Language
English
Production Year
2017

Content Metadata

Subject Area
Abstract
Building products requires putting together many components. There are source-based distributions supporting that process. Using a pre-built, pre-testedbinary distribution like Debian has its advantages. Isar is a system for image generation using binary packages. It usesmainstream tools and workflows to build multiple products, managing commonsubsystems, and maintaining components from different vendors. In this talk, Baurzhan will describe the motivation for Isar, its currentstate, advantages and challenges to overcome.
Loading...
Slide rule Presentation of a group Code Multiplication sign Similarity (geometry) Product (business) Revision control Medical imaging Mechanism design Computer hardware File system Videoconferencing Booting Firmware Physical system Distribution (mathematics) Projective plane Binary code Shared memory Planning Cartesian coordinate system Complete metric space Kernel (computing) Software Repository (publishing)
Building INTEGRAL Direction (geometry) Multiplication sign Range (statistics) Source code Medical imaging Mathematics Mechanism design Different (Kate Ryan album) Single-precision floating-point format Information security Error message Physical system Scripting language Enterprise architecture Collaborationism Arm Gradient Bit Befehlsprozessor Repository (publishing) Order (biology) Quicksort Thomas Bayes Ocean current Slide rule Implementation Device driver Metadata Product (business) Workload Term (mathematics) Gastropod shell Data structure Booting Firmware Mathematical optimization Form (programming) Standard deviation Distribution (mathematics) Projective plane Line (geometry) Directory service Software maintenance Cartesian coordinate system Software Table (information) Routing
Suite (music) Building Code Multiplication sign Source code Set (mathematics) Medical imaging Heegaard splitting Mathematics Different (Kate Ryan album) Computer configuration Single-precision floating-point format File system Modul <Datentyp> Information security Scripting language Collaborationism Enterprise architecture Meta element Arm Namespace Software developer Stress (mechanics) Parallel port Complete metric space Variable (mathematics) Software repository Repository (publishing) Order (biology) Configuration space Website Self-organization Right angle Whiteboard Cycle (graph theory) Functional (mathematics) Server (computing) Mobile app Computer file Patch (Unix) Virtual machine Branch (computer science) Product (business) Template (C++) Number Revision control Root Term (mathematics) Booting Firmware Task (computing) Fingerprint Installation art Addition Default (computer science) Distribution (mathematics) Machine vision Projective plane Division (mathematics) Directory service Line (geometry) Cartesian coordinate system Uniform resource locator Kernel (computing) Integrated development environment Personal digital assistant Musical ensemble Routing
Building Computer file Multiplication sign Virtual machine Similarity (geometry) Control flow Mereology Product (business) Wiki Mathematics Gastropod shell Compilation album Form (programming) Physical system Email Arm Software developer Keyboard shortcut Electronic mailing list Sound effect Bit Compiler Kernel (computing) Integrated development environment Repository (publishing) Theory of everything Musical ensemble Routing Resultant
my name is Lauren and in this talk I'm
going to present how to build didn't based embedded systems or products using Debian and bitbake so the plan is our motivation where we started why we develop develop the whole thing at all a brief introduction if the easel how we do this there are a couple of slides about the mechanisms I don't think we will have time for that today but I included this for a reference so that you know what to look in the documentation and a couple of videos since our last presentation on Linux conference okay so where we started we wanted to build a product and we knew that we are going to use DBM as a base system for this project actually this was back in 2004 and I don't remember whether openembedded existed at that time at least I was not using it then so what we needed we needed to have a debian sister and produce complete bootable file system and firmware images so it means including you boot and kernel and so on there should be binary Debian packages used and there should be our own application packages that are going to be included on the system what we also wanted is one command on-demand image creation and what we also didn't want is to touch every upstream package and at the end after this is a product we want to build an older version of some snapshot afterwards there are some further requirements how to handle the whole thing so we want to be able to customize upstream packages if you want to we want to build several products from one repository because they are for example similar or share components and we also would like to incorporate upstream code from software or hardware vendors in the repository and we have to maintain this for at least 10 years so if we look today at the existing art basically we have binary distributions and sauce distributions so debian is a typical binary distribution where you get precompiled binary packages and you can use them right away Yocto on the other hand is a typical sauce based distribution where you get the whole
sauces and you have to build them yourself first in order to use them so I'm not going to go through this whole table and also please don't take it very literally it's really an over simplification to give you a feeling in which direction every project goes but to summarize Debian has many standard tools is a tested as a whole distribution which is important for us has clear licensing long term maintenance or well relatively depending on the definition of long term and its small products or the whole product lines regarding Yocto in Yocto we also have clear licensing we have built bake that is a single government integration tool which is very flexible and efficient and we have collaboration policy in form of layers so that software pieces could be grouped together according to some criteria also last but not least one difference between them is that Debian focuses on certain pre-selected architectures for example if you take on then only two of them are currently available and without CPUs you have a wide range of optimizations that would be important for you so with debian you only have two grades of freedom where is the V octo after it is soft based you can fine-tune your built for your required CPU
capabilities so what is either it started 2004 as a bunch of shell scripts of course with the time we saw that it's very difficult to maintain it and we switched to bit Bay sort of either is not a distribution and is not distribution builder so we just in able to use Debian based Debian distribution as a base system for our product and we have some mechanisms to build our own applications and install it into the firmware image and we also like basically the structure and workload workflow of the Yocto project how they organize their directories and we also use this so this is what we do and what we don't basically we reuse binary packages from Debian because they are tested because they have security updates and because we don't want to reveal them them every time and we use bit bag the tool behind Yocto to bring the whole thing together and we have like in Debian we have distributed source repositories and like a yoga we have centralized meta data repositories we want also to change as little as possible and to build as little as possible this means we try to really reuse Debian packages and only where we have to then we modify them or build our own applications so this is how we this is an example how we did this in this particular product we used Debian packages as is we modified some small amount we built with them once and saved for future use so we use them also pre-built we also built our own you boot once and used it as a binary and the rest we rebuilt every time because it changes often this is an error also in this slide you boot we don't read it we build new boot every time I took it about so kernel drivers leaves and applications so how does it work okay so first we create a build change road this is a change route is a Debian installation where we build our packages that are needed like our applications then we build actually build these packages there like in this example in hello get in this bill change route so this build change route in current implementation it is an arm change route and we run it with gueermo and build the packages with standard dpkg build package to set after that after all packages are built we
multi-strap again to create root filesystem then we install the packages that we built there and create a bootable complete firmware image containing also you boot kernel and root of s so this simple five steps and they are all incorporated in one command like we know from the octo so how we build images we clone our repository we initialize the build environment and we create an image so if you look inside basically the whole thing consists of five files so called recipes and these recipes do each every small step like bill change route prepares the change route only and hello Bibi is responsible for building the wall application and these are image base and debug recipes for building the whole images this recipes consist of tasks and all the build order is defined by task dependencies and performance is achieved for one hand for on using binary packages and on the other hand through parallel execution of these tasks the whole thing is organized in two layers the meta is a whole layer and it provides basic functionality like build change route and meta either is the product template layer so that you can copy this template to your own project and modify files there so this is how it is organized you see here two methods the net whore is going to be pooled from time to time from our site when it is updated and your meta product you modify it once and maintain them you don't update it and you see that your application hello and your images are in your meta product and here you can see for example dependencies between the tasks which defines or how the whole thing will be built so how you create your project you clone either you push it as your own repo you define your product images add packages and add boards [Music] images are defined isn't in their respective recipes so that you have a copy of your recipe of your product recipe and you can customize it for example if you want to install more packages in your target root of s then you say you modify the image pre-installed variable inside it and say ok I want to have apt and I want to have OpenSSH server and so on you also might want to provide images with more functionality for example for two different products or for debugging or development tools in this case you just include the base image recipe in another recipe and add additional packages to image pre-installed like gdb or also for stress and so on so all variables in these recipes like BB and also configuration files are added to a single global namespace often there is a question why not use a single configuration file the problem is if you
want to build multiple products that share components then if we use single configuration files that we are going to repeat ourselves this is the problem and that is why this modularity and that is why we have to find a suitable place for that in the long term we want to reduce the number of configuration files and to organize them in some way maybe we need agree we have to get more experience with that so adding packages basically you should have a known project or application repository and it should be already Debian eyes after that you create a three line recipe for that referencing the URL of the repository and source division and then you say in here dpkg and this makes the whole thing built for now we have to list the dependencies of Debian packages here in the recipe again but we are looking for solutions how we can add building DSC files directly from Yocto from a big baked back-end and finally you have to add your application package name to image install in your image definition file also please note that we don't port all your tasks here we also we only have those that we need and at the time those are fetch and build so a board definition it's called machine in Yocto consists of settings that are necessary to bootstrap Debian and get a root filesystem so what you have is a reference to to the machine file it's qmo arm is really the name of the creamer arm conf reference below and you have destroy DBM Jessie it's also a reference to a file that defines the Debian Jessie distribution and distro arch is arm high F which is the architecture debian architecture name and in arm you just list the kernel image and it'll de image and pre-installed packages that are necessary to do bootstrap the root filesystem and here's the example of distro definition files so that you can install the right suite with right components from the right app source and afterwards configure it with a certain configure script if you need to create some device nodes and stuff like that so two for effective collaboration we want to organize holes code in different directories because once we put them into these directories it's not finished in a few years we have to update these files with newer versions and this is where layers are useful so we define different layers usually by vendor or by company or by division and define split also products in their own layers so that every layer has more or less it's defined lifecycle and update cycle and so on whereas maybe basically in one central repository or distributed in different git repositories it doesn't matter and you can use tools like repo for example to to clone all repositories what is important is that these layers have only to be listed in the VB layer conf file and that's it so you have after you create these layers you list them in in this in this file and they are then included in the build so one common use case is overriding an upstream package for example with custom configuration or modifications so a quick and dirty way for doing this is just to hack this in image recipe the right way that we follow is to fork the respective package but of course this means that as soon as you have one line difference in any tab you have to fork the whole this 5 in it so our vision is to have a recipe that can override the default package with some patches and then - and then that will create the sauce package and the binary package for that but this is not infinite so this is for the reference we can return to them if there are questions regarding that so what's you currently in the development branch we have added we started with arm support now we added int'l support we had some issues with Jessie that are now fixed and we also after we have done you boot and in general boot loader installation manually in recipes we are looking at image creation with week which is doctors tool to create images it has some number of dependencies and we are looking what would be the right way to incorporate that in either and we also have numerous documentation so to summarize we tried to combine the best options that we see and Debian provides a stable tested distribution with security updates doctor provides bitbake which is very flexible and efficient we provide the template
repository that you can use right off and you can use it as a template and modify for your own product and the whole thing after you get running you want to work it fast and this is last and not least why we developed this here are the references so we are on github mono is in the wiki and for mailing lists we decided to abuse they been embedded for now till we have great great amount of traffic and we could anywhere else well that's it for today any questions yeah well actually sorry in your toe there is a possibility to generate SDK do we do this or don't we need this well I personally don't need this but what you get is actually in the first step you get a complete build truth that you can use an in a virtual machine or you can use it in a change of environment so we could provide a bit break shortcut to carve this so that it is like an SDK that I can hand over with one file to a developer and that they can use it basically technically the answer is we provided but not in the form that people are used to in the octo community we could think about adding this syntactic similarity so that people find stuff where they expect it to find the same is also for their shell so you basically change route into built route and you get the development shell yes those who try to use can become freaking part of the kernel or something like this and generally they're using GPS package it's called because well actually as we started we used the question was excuse me whether we had problems with human performance for building kernels and whether we provide co-axial package to mitigate these problems the history is we started with cross compilation in a negative system so the built truth was not really an arm truth started under qmo it was a native into change route and it it ran natively but you had to cross compile your kernel package so this means basically you don't want to use your Debian kernel in your product that is why you can hack a small kernel package for you and cross build it this is not a big deal so this is how it started now we had our requirement from a customer who didn't want any cross compilation in the system and they insisted that we shipped this with with an arm build change route that is built under karma and this is where stuff gets slower so to answer your question today we haven't looked at um acceleration we might look at that however if you need performance then my suggestion would be to go to native compiler to forget it was compiled quite similar remarks effective results because they register the been empty for months or most of stop we are running to Andrew and are but or violently irani okay so we have two minutes or okay okay so we don't have time but I'm available here if anyone has questions then please [Applause] [Music]
Loading...
Feedback

Timings

  325 ms - page object

Version

AV-Portal 3.19.2 (70adb5fbc8bbcafb435210ef7d62ffee973cf172)
hidden