The State of PTXdist
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 490 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/47464 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 2020240 / 490
4
7
9
10
14
15
16
25
26
29
31
33
34
35
37
40
41
42
43
45
46
47
50
51
52
53
54
58
60
64
65
66
67
70
71
72
74
75
76
77
78
82
83
84
86
89
90
93
94
95
96
98
100
101
105
106
109
110
116
118
123
124
130
135
137
141
142
144
146
151
154
157
159
164
166
167
169
172
174
178
182
184
185
186
187
189
190
191
192
193
194
195
200
202
203
204
205
206
207
208
211
212
214
218
222
225
228
230
232
233
235
236
240
242
244
249
250
251
253
254
258
261
262
266
267
268
271
273
274
275
278
280
281
282
283
284
285
286
288
289
290
291
293
295
296
297
298
301
302
303
305
306
307
310
311
315
317
318
319
328
333
350
353
354
356
359
360
361
370
372
373
374
375
379
380
381
383
385
386
387
388
391
393
394
395
397
398
399
401
409
410
411
414
420
421
422
423
424
425
427
429
430
434
438
439
444
449
450
454
457
458
459
460
461
464
465
466
468
469
470
471
472
480
484
486
487
489
490
00:00
State of matterQuicksortMultiplication signSystems integratorHacker (term)Near-ringComputer animation
00:30
Physical systemBuildingBlock (periodic table)Revision controlRevision controlOpen setMedical imagingBuildingBlock (periodic table)CodeElectronic mailing listRootType theoryMenu (computing)Physical lawProjective planeStatisticsPhysical systemCoefficient of determinationComputing platformSymbol tableComputer animation
03:32
StatisticsChainReduced instruction set computingPower (physics)Revision controlChainArmEnterprise architecturePowerPCSlide ruleBitWebsiteFocus (optics)Menu (computing)Computing platformVideo gameWave packetUser interfaceShape (magazine)Computer animation
04:29
Computing platformMultimediaConfiguration spaceFitness functionUser interfaceText editorFormal languageWeb 2.0Menu (computing)TelecommunicationSoftware testingKernel (computing)Gastropod shellWebsiteLevel (video gaming)AeroelasticitySpacetimeUniform resource locatorRevision controlComputer fileNatural languageTable (information)CuboidComputer configurationSoftware
06:21
Data structureBuildingPatch (Unix)Local ringDirectory serviceRootLevel (video gaming)QuiltRule of inferenceRevision controlPatch (Unix)Level (video gaming)Sheaf (mathematics)Software testingComputer filePhysical systemProjective planeConfiguration spaceMenu (computing)Queue (abstract data type)Data structureInformationInstallation artRule of inferenceLibrary (computing)Directory serviceRootAreaComputing platformIndividualsoftwareQuiltCASE <Informatik>File systemKernel (computing)Electric generatorSet (mathematics)BootingInternet service providerBuildingSoftware developerQuicksortMultilaterationRight angleMedical imagingSummierbarkeitLatent heatNumberView (database)Server (computing)Web 2.0Workstation <Musikinstrument>RoutingWordPhysical lawBitStreaming mediaComputer animation
12:25
Rule of inferenceType theoryComputer-generated imageryEnterprise architectureIntegrated development environmentSystem programmingFile formatBootingPhysical systemKernel (computing)RootFiber bundleMechanism designComputing platformSheaf (mathematics)Projective planeLatent heatDefault (computer science)Computer fileTraffic reportingMedical imagingMultiplication signComputer configurationDistribution (mathematics)Computing platformMereologyCASE <Informatik>Uniform resource locatorRevision controlConfiguration spaceLibrary (computing)Source codeNumberMechanism designCore dumpLevel (video gaming)Installation artVariable (mathematics)Type theoryEmailFunctional (mathematics)BitIntegrated development environmentModel theoryDigital photographyGraphics tabletCuboidWeb pageQuicksortCurveCellular automatonAliasing
16:13
Computing platformRevision controlCuboidMedical imagingBootingPartition (number theory)Computing platformKernel (computing)ForestGroup actionPhysical lawPhysical systemXML
16:50
Computing platformBootingComputer configurationComputing platformRevision controlPatch (Unix)Medical imagingEqualiser (mathematics)ArmBitDifferent (Kate Ryan album)SoftwareSet (mathematics)Default (computer science)Computer animation
17:29
Computing platformStress (mechanics)Convex hullSoftwareComputer filePhysical systemDefault (computer science)Configuration spaceRevision controlOrder (biology)Computer animation
18:12
Computer fileComputing platformInstallation artRule of inferenceLevel (video gaming)Patch (Unix)QuiltData structureDuality (mathematics)InformationExplosionWhiteboardVariety (linguistics)AdditionRoutingProjective planeDesign by contractDemosceneConfiguration spaceAcoustic shadowRootMacro (computer science)Computer fileLevel (video gaming)Installation artLink (knot theory)BuildingInformationSoftware testingBlock (periodic table)Kernel (computing)Differenz <Mathematik>Open sourceComputing platformPatch (Unix)Symbol tableComputer scienceMechanism designSoftwareVariety (linguistics)Theory of relativityWhiteboardSummierbarkeitCASE <Informatik>Row (database)Different (Kate Ryan album)Rule of inferenceLine (geometry)Data structureRevision controlComputer configurationPoint (geometry)Ocean currentFunction (mathematics)Variable (mathematics)Probability density functionTraffic reportingMereologyBitSheaf (mathematics)Uniform resource locatorMedical imagingGraph coloringComplete metric spaceDirection (geometry)NeuroinformatikRecursionRadio-frequency identificationRight anglePhysical lawCategory of beingComputer animation
24:01
Rule of inferenceMultiplication signSymbol tableProjective planeDifferent (Kate Ryan album)Configuration spaceMedical imagingRevision controlMacro (computer science)BuildingStandard deviationXML
25:58
Point cloudOpen set
Transcript: English(auto-generated)
00:06
Yeah, welcome at this afternoon, I'm Roland, I will talk to you about PTX dist, as my previous speaker already mentioned it. I'm a systems integration hacker at Pangotronics,
00:21
we are a small company in Hildesheim, near Hanover, if you've heard about it, yeah. So what is this? Oh, sorry.
00:40
So what is PTX dist in one slide? It's basically a build system, like if you know Yocto, or open embedded, or build root, or even open WRT, so you put in things you want on your platform, on your embedded device, and then you wait some hours,
01:03
and then you get an image out of it, which you can flash on your target. And our building blocks are basically GNU Make, which we do a lot of dependency handling with it, because it's basically built for this, so we can reuse it, and we do the dependency handling
01:24
in kconfig, which also gives us a nice menu for the user to select the packages that you want, and select the sub-options, and whatnot. There is some glue code in Bash, and AWK,
01:47
which handles the dependencies between kconfig and Make, so it basically transforms the kconfig symbols, the dependencies from kconfig into the GNU Make
02:01
dependencies, and does some other things under the hood, so you don't have to type so much, yeah. The first versions of PTX list go back to at least 2003, that's the earliest version I could find, there are probably some older versions,
02:21
but I couldn't find those, yeah, so it's basically older than, I think, build root started in 2010, and Yocto is... 2001, oh, okay, then I'm wrongly informed, excuse me about that, but well, it's an old project,
02:42
and I think it isn't known so much, so basically that's what this talk is trying to change, but now we do monthly releases, so every release, every month there's a new release, which is basically just version bumps for packages, so yeah.
03:02
The overall license for PTX lists and all the recipes are GPL 2, yeah, and now we also have this shiny new logo, since I think one year ago, because we're basically just a collection of recipes,
03:20
we have this cooking penguin, and an online documentation, which is generated from re-structured text in Swings, and you can read this online. Yeah, some statistics, I did this for the previous release, 2020.01 is already out,
03:41
but my slides were prepared last month, so we have about 914 target pages, and 184 host packages, and we're also providing some pre-built tool chains for about 18 plus minus platforms,
04:02
architectures like mostly ARM, like ARM 4.4, and v7 are very big, and of course the 64-bit version of ARM, and there are also some people using it on x86, but this is not the main focus,
04:21
and of course RISC-V, and then some older PowerPC architectures, yeah. I think the basic thing which you got to see on the user interface side is this menu config. We have two big,
04:43
I say two big, but they're not very big menu configs, on the one hand we have the menu config for your user space, for your user land, which you can select, in which you can select all your target packages, like, I don't know, we have here shell and console tools,
05:02
scripting languages like Python, for example, networking tools, I don't know, IP tables, NF tables, what you need, DUSKIFIL, utilities, communication applications, well, everything that doesn't fit into the other, editors like VM and Nano,
05:25
test suites, web applications, Nginx, you name it, and graphics and multimedia like QT, Wayland, yeah, and XORC still, and on the other side,
05:41
everything that's not user land, like kernel and barebox, sorry, bootloader, which is barebox for us, but you can also build, of course, U-boot, and I don't know, Opti is in it, basically everything that's below the user land
06:04
is selectable in this platform config, so you have this bare metal side on the platform config side, and everything that's above it in the menu config,
06:22
and this is also what a typical BSP looks like on the file level, you have one folder configs, which you have here the PTX config, which is what gets selected in the menu config, and this is basically the K config configuration
06:42
from which the menu is generated, and in it, you have a platform, so this is everything you select in the platform config, and in the platform, there are usually some other packages that are also using K config for their own configuration,
07:02
like, for example, here, the kernel, I guess you have a kernel without the kernel. I don't know if anyone's ever tried it, but maybe it's possible, I don't know, but also the config for your bootloader, which in this case is barebox,
07:22
and yeah, that's basically everything in configs, and you have some rules, some package rules, if you want to build some custom software which is not already in PTX test, you can generate package rules in your BSP, and this is basically a .in file
07:42
which contains the K config menu entry, and the make file which handles everything about building the packages. Yeah, and then there's this fold in between here, project root, I will come to this later, this is basically everything which you want to overwrite,
08:02
but I will say something about this later. Yeah, and the basic structure is, the basic idea is that your BSP directory structure is structured exactly like the PTX dist upstream structure, so we have the same structure in PTX dist when you install it, so everything that's not found here
08:23
in your BSP is searched instead in the upstream PTX dist installation folder, so you can extend PTX dist however you want and use the same structure with it. Yeah, every package is built in stages,
08:43
I guess we have this in other build systems too. Yeah, it has a get stage, an extract stage, a prepare, a compile and install, and a target install stage, which basically the get stage fetches everything from the web,
09:00
and then the extract stage extracts the tar ball which you just fetched and applies the local patches which you maybe have in your BSP or in upstream PTX dist. The prepare stage runs basically Configure or CMake or Meson or whatever build system your package uses. PTX dist has already support for many of those,
09:25
and I think it is fairly easy to provide support for other build systems too. And we'll see later how this is handled in the rule file itself, in the Mac file for the package.
09:42
It's basically just a variable that you set. Yeah, then the compile stage basically just compiles the package like it is normally used. The install stage installs it into a isolated directory, so many packages install many files,
10:02
by default, which you don't use or don't need on your target system. For example, if you build a Qt, then you have a whole lot of libraries which you maybe don't want to install, so the next step is this target install stage
10:20
which cherry picks a set of files into your root file system or your IPKG package later. And you can override any one of those stages in your Mac file rule, in your package Mac file, but you can also reuse a lot of them,
10:42
which we see just now. Oh, no, just a quick information how to apply patches. You put your patch queue into a patches folder
11:01
in your BSP or in the upstream PTX test package structure if you're using a PTX test in a development version, and you name it like the version and the name of your package, in this case, BusyBox with a version number, and there is the serious file and the patch file, yeah.
11:23
And the usual thing is that you put packages for the, which, sorry, the usual thing is to put patches for your packages which are from the platform conflict
11:40
like the kernel or the bootloader or anything else in this area into this configs slash platform folder. Yeah, but you can also put it in there and have it for all the platforms which you want. So you can edit this patch deck by using quilt or git
12:04
after extracting the package and then changing to the package directory where it's extracted, yeah. Okay, this is what I wanted to tell, how a package looks like in the inside. This is basically the menu section for the package.
12:20
You just give a section in which of these one you want to have it. Project specific is the default section which is just here. And then you select basically the dependencies
12:42
that your package uses, like in this case lib USB. So lib USB gets built before your package gets built, so all the headers are found and so on. And also in this case, because we're using CMake in this example, you also have to select the CMake host tool so Pdxtest can build your package.
13:02
Yeah, and every package is a tri-state option, so you can set it to yes, no, or module which you can see, which we'll see later what that does. Then we have the makefile. As I said, it's a lot of assigning variables like the version number and the MD5
13:21
for the downloaded package version for this URL. And the source where it gets extracted and the license files. And the license files are really cool because Pdxtest can put them all together in one big PDF, so you have kind of a report
13:41
about all the licenses you're using in your package, yeah, in your image. And if you're using the default tools like CMake or AutoMake, you often don't really have to do a lot of this and can reuse all the internal Pdxtest functionality to build it.
14:02
So the next steps are really short and reuse everything except the prepare and the target install stage. And then the prepare stage, we are also only setting this config tool variable to CMake. And we are setting these config options for the CMake to maybe enable our USB functionality.
14:26
But this depends of course on the package you're using. And then in the target install stage, we set up our variables for the IPKG package which gets built later and basically we're then
14:41
only copying, oh, here should be a file name which I missed in the example. But yes, this is the cherry picking part. And we have three package types. As I already said, we have target packages which I'll get installed on the hosts
15:01
like some DBZBox core utils, whatever. And the host packages are only there for having a compatible build environment. So we try to support most of the Linux distributions that are out there. I think the earliest is something like CentOS 6, yes,
15:27
which doesn't have an up-to-date core utils so we have to build the core utils ourselves. So all the packages which are using, I don't know, tar with dash dash M time versions or something
15:42
can do that. Yeah, and the third part is image packages which are also from the platform config and you can decide there what kind of images you want to build. So I will talk about layering a bit. So PTX just started up with platforms.
16:04
So platforms are the oldest mechanism to layer things on top of each other but there are five different others. We'll start with platforms. In this example, I'm maybe using a platform for Raspberry Pi. So I have a kernel with a version 419.
16:23
I built an HD image for it and the Raspberry Pi needs a boot partition for the boot loader. So I enable this and because it already has a boot loader internally, I don't build a bare box for it and in the user land, in this example,
16:42
I built a system, a busy box and a CUPS. And if I now want to extend this support to other platforms, I can just build, add another platform like for example, a platform containing ARM v7a and ARM 64-bit v8.
17:02
And I can choose different kernel versions here and with this also, maybe a different patch stack if your platform support is not good enough upstream and I can choose different images which I need and maybe a boot loader, another boot loader
17:21
like in the other platforms. And I talked about that packages are tri-state options in kconfig. So you can set these options to maybe M, so modular. And maybe you want to have a debug collection, software collection that doesn't get installed by default
17:44
on a user's or a customer's target but you still want to be able to debug some packages in your system. So you set these debug packages to M and then choose them at a later step in a collection config file
18:02
and in the debug collection, you enable them and maybe in the release collection, you don't enable them. Yeah, these are collections. Then we have alternative config files. As I said before, we had this additional folder here,
18:22
project root and here are two config files, everstep and shadow in etc. And you can use this with packages that are using this install alternative macro in their target install stage. For example, in this example for CUPS,
18:43
CUPS is installing the etc.printcap file and you can override this by any one of these locations. So first locally in your BSP, then it gets searched follow up. Then in the project root directly
19:03
from the PTX test installer and then in the CUPS installer and then in the CUPS build here. So you have a lot of options to override config files for different configurations.
19:27
A new mechanism which exists about a year now is kconfig diffs, which is basically just diffs on, simple diffs on kconfig files. If you are using a custom kernel image, for example,
19:43
you want to have some debug symbols in there. You set the config file to something like kconfig-debug, which is living in the platform config as earlier. But you tell PTX dist this refconfig variable
20:00
and this points to another kernel config in this case. And then internally PTX dist computes the diff between what you select in this kconfig file and this kconfig file. So you do a many config for your kernel debug and then enable these three symbols
20:21
and these two symbols later get written into the kconfig diff. And at the first stage, the first line is an MD5 sum of the refconfig and so PTX just knows when to recalculate this diff.
20:44
And kconfig diffs are also the, one of the building blocks for base layers. So as I said before, we had this idea to have the same structure in our BSP, like in the upstream PTX dist installation folder
21:01
and extend locally where we need it, but we have a better idea and all problems on computer science can be solved by another level of indirections, so we use recursion. In this case, we have one BSP, which is a whole BSP by itself.
21:22
It has the same structure as before and we want to layer our BSP on top of it, so all we have to do is just create a link named base to this upper or lower layer in this case and everything you now select in the PTX config
21:41
is calculated as a diff to this PTX config on the dist record layer and you can enhance your BSP in relation to the lower layer by adding rules, for example. And this is great if you don't want to share
22:01
all your rules in one layer. For example, your customer only gets an open source layer, that gives you only an open source layer and wants to layer their own proprietary software on top of it and this is one way you can do it. Some more goodies which got added in the last stage.
22:21
There is now a command to give information about one package and one to give information about your BSP and as I already mentioned is the license report, which basically collects all the information from the package recipes
22:42
and dumps them into one big PDF and also extracts all the copyright files. If you want to try this out, you can have a look at DistroKit, which is a pre-configured BSP for a variety of dev boards like Nitrogen 6 or even the Raspberry Pi,
23:02
some versions of it, not the four versions, sadly. And you can also use it in QEMO if you want. Short things about contribution. You can send patches, patches are welcome. Currently about 85% of patches are in-house,
23:24
but maybe the changes, yeah. So are there questions?
23:50
So the my package example that you showed us has a USB in the prepare section. It has a dash D USB on.
24:04
What would you suggest if I want to have that configurable? You can add another kconfig symbol, which is dependent onto the top symbol. So you basically extend this symbol
24:22
by another sub-symbol called, like, yeah. And you can switch it on and off, and then you can use a macro here, which just looks if this kconfig symbol is switched on and off, yeah. All right, and is this something
24:41
that is halfway common in the packages? I just used a simple example here, but this is really common. And also for controlling what dependencies my package will pull in. In the CUPS example, which I showed here, you can build CUPS with many different dependencies
25:03
like PNG support or whatnot, yeah. And is that, does that have to be then set per package in the PTX config? Yeah, this is also written to the PTX config, because it is in the menu, so it gets written into the PTX config.
25:21
Yes, but does it have to be configured per package? Or is there some standardization on these, on the naming, let's say, of these sub-options so that I can set them globally for the whole image in one place?
25:42
Yeah, this is usually for the whole image, yes. Okay, then I think the time is up. Thank you.