Meson: compiling the world with Python
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 160 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/33730 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EuroPython 201794 / 160
10
14
17
19
21
32
37
39
40
41
43
46
54
57
70
73
85
89
92
95
98
99
102
103
108
113
114
115
119
121
122
130
135
136
141
142
143
146
149
153
157
158
00:00
IntelSoftwareComputer chessModule (mathematics)Infinite conjugacy class propertyComputing platformDefault (computer science)Formal languageLatent heatTime domainSystem programmingTuring testSystem programmingMultiplication signWindowComputer fileCore dumpFlow separationModulare Programmierung1 (number)Projective planeDeclarative programmingCompilerDifferent (Kate Ryan album)BuildingModule (mathematics)Quantum entanglementPoint (geometry)Internet service providerSoftware frameworkBitLine (geometry)QuicksortCuboidFormal languageExtension (kinesiology)Product (business)Computing platformDecision theoryLibrary (computing)Boilerplate (text)Rule of inferenceCASE <Informatik>Functional (mathematics)Goodness of fitElectronic mailing listRecursion10 (number)DistanceLatent heatProblemorientierte ProgrammierspracheDefault (computer science)FlagFluid staticsConfiguration spaceSource codeCase moddingScaling (geometry)Asynchronous Transfer ModeEndliche ModelltheorieMetric systemSpeech synthesisSlide ruleBriefträgerproblemClient (computing)Special unitary groupActive contour modelOrbitMassTwitterVisualization (computer graphics)Office suiteOrder (biology)Cartesian coordinate systemSurjective functionLecture/Conference
07:31
Pairwise comparisonLibrary (computing)Standard deviationTime zoneGrand Unified TheorySoftware frameworkMultimediaSystem programmingVideoconferencingComputer fileSoftware testingScale (map)Electric generatorEmailMultiplication signCuboidSystem programmingCASE <Informatik>WindowCompilation albumPlug-in (computing)Source codeProjective planeVideoconferencingSoftware testingElectric generatorBinary codeDifferent (Kate Ryan album)Point (geometry)Identity managementBit1 (number)Series (mathematics)BootingNonlinear systemEmailImplementationCore dumpProcess (computing)GUI widgetData managementMathematical optimizationFile systemBinary fileOperator (mathematics)Unit testingElectronic mailing listUser interfaceoutputJava appletLibrary (computing)Computing platformSemiconductor memoryMultimediaBefehlsprozessorComputer filePhysical lawBuildingSingle-precision floating-point formatStandard deviationLink (knot theory)Rule of inferenceServer (computing)Address spaceTraffic reportingFront and back endsWikiLetterpress printingComputer-assisted translationComplete metric space2 (number)Software development kitMatching (graph theory)Software frameworkSign (mathematics)Dimensional analysisLaptopOrder (biology)Graphical user interfaceCompilerSlide ruleSampling (statistics)Web 2.0Moment (mathematics)
14:50
Execution unitSoftware frameworkSoftware testingDefault (computer science)Integrated development environmentCompilation albumMassAndroid (robot)Goodness of fitComputer programElectric generatorVirtual machineElement (mathematics)System programmingFormal languageInternet service providerComputer fileVirtual machineTouchscreenDemo (music)MultiplicationComputer programmingInternetworkingCompilation albumSoftware testingMathematicsSource codeVariable (mathematics)Software developerIntegrated development environmentDifferent (Kate Ryan album)Near-ringQuicksortCross-platformBeta functionSystem callProjective planeComputer fileWindowBlock (periodic table)MereologyCartesian coordinate systemSuite (music)Default (computer science)Multiplication signCompilerBitLatent heatLibrary (computing)Function (mathematics)System programmingCodeParallel portShape (magazine)Execution unitArithmetic meanPerfect groupTraffic reportingBuildingAndroid (robot)Category of beingLimit (category theory)1 (number)Formal languageMassConnectivity (graph theory)Computing platformRepository (publishing)Power (physics)Extension (kinesiology)Medical imagingGoodness of fitElectric generatorNP-hardConfiguration spaceTotal S.A.outputWrapper (data mining)ArmStandard deviationStandard error
22:09
MultiplicationSoftware testingHash functionMixture modelCache (computing)WebsiteRoutingCorrelation and dependenceMultilaterationSource codeBuildingSingle-precision floating-point formatComputer fileSystem programmingHash functionLipschitz-StetigkeitProjective planeCASE <Informatik>Order (biology)Patch (Unix)Computing platformCompilation albumBitExpected valueCompilerComputer animation
24:08
Hand fanComputer fileService-oriented architecturePrice indexVariable (mathematics)Directory serviceError messageCore dumpLatent heatObject (grammar)RankingDefault (computer science)Interactive televisionFloating pointMultiplication signSheaf (mathematics)CompilerRepository (publishing)Configuration spaceFunction (mathematics)Level (video gaming)Mathematical optimizationWindowMassOnline helpDirectory serviceFunctional (mathematics)Computer animationEngineering drawingDrawing
25:48
Software testingVisualization (computer graphics)WindowError messageNumber1 (number)CompilerFreezingWage labourSource code
26:40
Configuration spaceNumberWindowLine (geometry)Computing platformBuildingCore dumpKernel (computing)System programmingMobile appGraphical user interfaceGrand Unified TheorySign (mathematics)System programmingConfiguration spaceRevision control2 (number)WindowSymbol tableCore dumpSlide ruleVirtual machineProcess (computing)BootingCartesian coordinate systemPresentation of a groupSource codeComputer fileRight angleExploit (computer security)ParsingGastropod shellVisualization (computer graphics)CodeCodeComplete metric spaceMedical imagingQuicksortFraction (mathematics)AdditionCountingDot productString (computer science)CuboidScripting languageNumberComputer programmingArray data structureCASE <Informatik>FreezingSoftware maintenanceSoftware testingMultiplication signPairwise comparisonElectronic mailing listLine (geometry)Module (mathematics)Library (computing)MereologyFile systemCross-platformFormal languageComputing platformExtension (kinesiology)Kernel (computing)Scaling (geometry)Graphical user interfaceCompilation albumBuildingPoint (geometry)Software developerBinary fileProjective planeSubgroupAffine spaceWebsiteMassMultiplicationSet (mathematics)Game theoryCategory of beingBit rateRAIDCondition numberBitService (economics)Rule of inferenceNetwork topologyVideo gameWord
35:02
Moment (mathematics)Projective planeNumberFigurate numberElectronic mailing listLecture/Conference
35:34
Library (computing)MereologySymbol tablePlug-in (computing)MultimediaPoint (geometry)Software frameworkUniqueness quantificationConnectivity (graph theory)Moment (mathematics)Sampling (statistics)TheoryTape driveDynamical systemMacro (computer science)
36:30
CompilerMicrocontrollerDifferent (Kate Ryan album)Software bugMathematicsMaxima and minimaProduct (business)WindowPatch (Unix)Mathematical optimizationSoftware testingComputer configurationRule of inferenceLevel (video gaming)CodeProjective planeProcess (computing)Formal languageMereologySoftware frameworkMultiplication signBitFunction (mathematics)StatisticsRegular graphWeb pageType theoryLecture/Conference
Transcript: English(auto-generated)
00:05
Good morning welcome to the talk My name is Yossi, and I'm here to talk to you about build systems Usually when I talk to people about build systems the reply I get is this Which is that all these build systems are terrible, and why should I care?
00:23
Well, let's look into this If you were in the keynote one of the points made there was to explicitly say that don't build any more Python extensions So with that in mind, let's build a Python 3 extension module and To make it more interesting. We're going to be using
00:40
C C++ Rust and Fortran all in the same extension module and If you've ever done this with these details I salute you But here's the height would look like if you would do the same thing with meson
01:01
so first you start the thing by defining the project which in this case is called polysnake and the list of languages that you want to use and The next step is this function where we import Python 3 this is the? Internal module system that we have in meson and this Python 3 module has all the knowledge to build Python extensions
01:27
and then we find the dependency and this What it does is goes out to the system and finds out What are all the the flags and and linker flags and all that sort of things that you need to compile Python 3 applications?
01:42
On Linux this would be going into package config and on other platforms. It does a bunch of other magic stuff Then we build a rust static library rust is a bit special, so we have to do it this way But this is very quite quite simple
02:00
And once you have that Then we can start the actual extension module building, so we call pi 3 mod dot extension module and We give it the name And then we list the source files that we want to have in it in this case the C file the CVP file and the Fortran file as we as you can see we're using the modern f90 and not the
02:22
F77 and Then we tell it that we want to link the rust library that we built earlier with this And then we say that the dependencies of this project are the this is the Python dependency that is there and This is the built definition in its entirety. This is all you need to write and
02:46
then you can just compile it and then it goes and This is so Within the design goals of this system, which were which you could see someone on the previous slide Go off it like this so there's the the platforms that we have today, which is
03:05
Linux OS 10 Windows BSDs and a bunch of other ones We want to support all of them There are several different compilers GCC clang visual studio the Intel compiler. We want to support all of those
03:20
We try to hide as much of the platform specific differences as possible In cooling compiler flags and all that sort of stuff basically just want to say I want compiler warnings to be enabled and then we do that and we take care of all the back-end stuff for you and Out of the box. We should provide support for tools that people are using today
03:42
There's very little point in having Like 7,000 line make files where you copy bits from one place to another because this worked in one project So maybe it works in this one as well, but then it doesn't and then you're just Debugging for quite a long time and The major architectural decisions of this system are not defined by what was in the default use land of San OS 3.5
04:06
circa 1987 Because that's what like for example all the tools does nice. It's it's an admirable backwards compatibility claim but like maybe it's time to let go and at the very core of of Meson is the
04:22
Domain specific language that is used for defining these builds this is the thing that we have spent most of our energy on like getting as good of a definition language as possible and Let's look at some of the design calls that we have now first we're going to look at some of the design not principles because
04:41
Your product is defined by what it does not do more than it's defined by what it actually does Especially if you have something that is Turing complete But the things that you specifically are saying that we're not gonna do help you make a lot better design decisions later on So the main design thing is that Meson is not a framework
05:02
There are a bunch of build systems which say that no, this is not a build system This is a framework with which you can build the perfect build system for your projects as if this was a good thing No, we don't do that. We have we're very specifically focused. We do this one thing we do it Well, we do it only and
05:22
the there are a bunch of build systems that are declarative languages and Declarative languages are really awesome when they work, but the problem is that when they don't work it's it's really really painful to debug and Because of this which Meson is not a declarative language
05:41
But we try to be as declarative as possible and It's not Turing complete You can't specify your own functions or recursion or any of that And the reason for this is that it makes tooling much more easier to write because it's easier to understand what the thing is actually doing As in it's not Python It's good. It hides the fact that it's implemented in Python. So it could be reimplemented in any language at all
06:09
What those were the not principles. So what are the actual design principles? It basically boils down to this is that every second you spend writing or debugging build definitions is a second wasted You're not being productive when you're writing build definition rules. It's just boilerplate stuff that you have to do
06:26
So let's do it as little of it as possible Understandability is the most important thing You should be able to look at your build definitions like one file in isolation I have a fairly good understanding of what it does and it's not like some sort of spooky action at a distance where like it
06:42
Says here that it's going to do something but actually over there. There's something else which completely changes everything So we don't want to have any of that Performance is important when it needs to scale to tens of thousands of files even more and You have to have have this sort of performance built in although the core stuff needs to be properly done
07:02
You can't really likes like bolt on afterwards some sort of performance Zen of Python Explicit is better than implicit. Awesome All that good stuff. There should only be one way to do something and it should be obvious. Awesome so we stole all of that and
07:21
basically We just steal everything that is good from anyone if there's any there's no such thing as NIH If someone has a good thing that they're doing and we are not we're gonna steal it and we're gonna be proud about it and the One thing is that the build definitions should be shorter than writing the exact compilation commands by hand So let's look at an example a hello world
07:42
Example looks like this you can define your project and then you say I want to print an executable Which has this one file and this is 54 characters and if you do it manually You said CCW all all the stuff you can pile your link and that's 54 characters So we didn't quite reach the goal. However, if you use GCC then it's 56 characters and we win
08:05
We Also want to be usable across all for every single use case and the only dependency that we have is Python 3 and Only Python 3 and the standard library anything outside of that. We're not allowed to use any of that
08:21
as an example, there are a bunch of build systems that are implemented in Java and The problem there is that there are many platforms on which Java cannot run and let there's no CPU There's not enough memory all that stuff and and it's a it's a dependency that a lot of projects can't take and and even more
08:40
projects don't want to take But we want to be able to support all of those and Python is really good because it's it's quite small It's self-contained and it's everywhere for design we've been doing the 90 slash 9 slash 1 rule which is that there's 90% case and that should work out of the box
09:01
Because almost always this is the case because you basically there's only so many ways you want to build an executable with some sources that the 9% case should be possible and for the 1% case we can just say is like What you're doing is just so weird that we can't support that and it would complicate everything else completely and usually it's like, okay
09:22
But if you do this other thing instead then we can just do that But it's it's totally okay for this like the weirdest of edge cases to say, okay We can't support that out of the box. We have to do some of your own magic In practice this has been less than 1% So, but it's still there
09:42
and There are already people using it that they're they're like 100 build systems. Usually most of them have about two users But we have managed to get some of them And one of the major users thus far has been the t-stream multimedia framework, which is
10:01
100 150 plugins and a base platform and all that good stuff and they're using it to build on Windows and and OS X and Linux and all that stuff and Then there's systemd For those of you who don't know systemd is the first project but the first process that is spawned when a modern Linux system
10:21
boots up and it's like the manager process which take care takes care of all the other processes and they actually have a Outstanding built merge proposal to remove all the tools altogether. So then starting from the next release They're probably gonna have only meson for building
10:43
The X or graphical the user interface if you are Linux laptops, you're probably using one of those Using that and if you're not you're probably using Wayland Which is the the up-and-coming new graphical GUI server and that has some built definitions as well And Western is the the reference implementation window manager for that
11:04
the GTK widget toolkit Which is pretty big and small to perform and in there for series There they they have merged the meson build definition and they're gonna I Think they're gonna some at some point get rid of the old one, but it's it's not not yet happening
11:22
a lot of ground particularly are going as well multi-less and a bunch of other ones and There's the PTV nonlinear video editor, which is also implemented in Python, it's a pretty cool project you should totally check it out and On the website. We have a list of a bunch of other projects that are also also using this
11:43
and One of the main reasons why people are switching into meson is that you get? Some fairly have to compile time improvements. So as an example System D on Linux went from 16 minutes to compile to 2 minutes and 32 seconds And if that seems weird to you, it seems weird to me
12:02
I went through these lights like four times and every single time I saw this one is like that can't be right I have to verify that and and every single time that it's still 2 minutes and 32 seconds and And so it's it's fairly heavy a difference If you are compiling Wayland on Raspberry Pi it went from six minutes to a minute and
12:25
There's the feeder re which is a debugging toolkit and their cross compilation time to iOS went from six minutes to one-half In practice what we seem to find is that projects can go up to 10 to 20 times faster if they're make-based and
12:46
Especially if you are on Windows because on Windows file system operations are like abysmal is low and Because we don't use make and and we are not going to use make ever we don't have this problem and
13:04
For incremental builds the difference is even bigger because there's a bunch of optimizations that we do where we can skip relinking of stuff and So we haven't done any large-scale tests with other ninja generators like CMake for example has a ninja backend and
13:21
But it seems that we produce slightly more efficient files, but this is like this hasn't been conclusively proven either way There was talk about Built-in support for stuff. So we support The sanitizer so address sanitizer memory sanitizer UB sanitizer all stuff You just likes toggle the switch say I want this enabled and then you have it
13:44
coverage report precompiled headers can build all the things that you would expect and Like scan build is the static analyzer that comes with clang and if you have it installed it the there's a target called scan build and it's just like run it and then it runs the thing and
14:03
Reproducible builds that's another interesting thing. This is a big thing that is going on with Debian and a bunch of other projects Where you basically want to prevent the generation of Binary packages which have laws that are not in the original source So if you can produce from the same source files the same bit for bit identical binary files
14:24
From many different people are going to do this then you can be fairly certain that the the binary that you have is actually proper and you haven't been back door and Basically want to support everything Within reason Because there's no point in everyone trying to reinvent the wheel of how do I support this particular one tool?
14:43
It just put do it once in the core system, and then everyone can use it We also have a unit testing framework for running your tests which in cool stuff like setting the environment to have set environment variables and command line arguments and things to run and all that sort of stuff and
15:02
We log Standard output standard error how long it took all of that good stuff and Excuse me It's easy to add support for stuff like Valgrind and all tests are run in parallel by default which is particularly nice if you are running Valgrind because then they're about 10 to 20 X slower than they usually would be and
15:24
You can run the test suite multiple times you want to You can run the test under GDB which is kind of nice as well And then you can just run test under GDB until they fail And then you just like leave your machine running if you have flaky tests, and then you come back to it Oh, no, there's the GDB prompt for the one the one in 1,000 runs that actually failed
15:45
Which is kind of nice as well cross compilation we all of course do A full Canadian cross if you don't know what that means it's probably not very important But if you do know what it means then it's probably quite important for you, and and we support that fully
16:03
For Linux and Linux cross compilation that's being used Lots and There are people who compile from Linux to Windows so they using the min GW packages that come with their distro So they can just generate all of their windows packaging from their Linux desktop machines very easily
16:21
Android works iOS works and also bare metal if so if you have an Arduino thing happening you can compile compile with mason and in This you need stuff like upload the target And these sorts of things are very easy to add and then when I'm doing Arduino development
16:40
I always run ninja upload which compiles and uploads the thing if it's everything works And there are very few changes in the build definitions that are actually and usually you don't need anything Yeah, and just the same build definition because all the parts that have to do with specifying the cross environment are not in the
17:01
Build files they're actually somewhere completely separate But I probably many of you were thinking like the most obvious Question which it always comes to people's minds when we talk about cross compilation which is what about bare metal embedded COBOL development and
17:20
I'm glad to say that we just we do support that so that you see There on the left you can see the COBOL program for running this External LCD screen and there's the screen running the COBOL program There and this is nice and working if you are in this that sort of thing
17:44
Other things that you could do when cross compiling is that you can specify and sort of like an exe wrapper and This is a program that you can use the or cross build files during the build time So this usually includes stuff like wine where you can run Windows programs or Qemu which you can use to run
18:03
like arm programs on an x86 machine and This ties in nicely because you can have code generators that you compile and then you're using to generate more source files during the build so you can just run the executables as they are or you can specify that some of the Executables should be built with the native compiler and not the cross compiler and then you can just run them and they probably run a bit faster
18:27
And you can also run the full test suite of your application with all this so if you're on Linux, you can run your full test suite for Windows when you're running wine and As an exe system, we're doing pretty well
18:42
It's about four and a half years old And we have more than 100 contributors at this time and which is about monthly releases And it's supported on all of the major Linux distributes So so Debbie and fedora opens usa and all of those and they have mezzanine default package libraries
19:02
sort of package repositories We run a fairly extensive test suite the docker image that we use for testing the Linux part is about two and a half gigs of all the dependencies that it's needed to run all the actual tests and the building stuff is is
19:22
Relatively simple, but dependencies are what makes things everything hard So we spend a lot of time thinking about this So the way that works is that in a mezzanine you can Build any project as a sub project of a different one and it's run in a sandbox environment
19:46
so it can access stuff from the outside and It's isolated but then from the master project you can reach into it and take stuff from there So this makes it possible to have your dependencies either come from the system as like on a Linux machine
20:03
We just used the package config files to get your dependencies or you can build them yourself and The build definition for file for both of these looks exactly the same You don't really care where your dependencies come from you just say I want this thing to be available and
20:21
This works cross platform. There's nothing specific about it. So Windows OS 10 Android all you need to do is if you're The dependency that you're using compiles with mezzanine on that platform, then you can use it this is very similar to to rusts cargo or
20:41
D has a similar thing called top and I think go has something called go get But the thing is that this definition is not tied to any specific language. It's it's completely language agnostic So you could use it for any any language dependency that you care This as a text this is not probably
21:01
Okay, sorry, and we have the this called wrap DB which hosts these dependencies online And This works in total cooperation with distro pack So if you are distro packager One of the things you hate more than anything else is that people embed the sources of their dependencies inside their packages And then they build those and then you have to rip them out and you and maybe they have changed it and all that
21:24
stuff so This works in Complete cooperation because you can just say that all I want is you're not only allowed to use system dependencies And if you are not having it those then it's a hard error. Don't try to compile your embedded source
21:40
And never download anything from the internet So if you have a thing where you must have must build all of your stuff without accessing the internet we can enforce that This is a bit dry perhaps as text so let's let's look at a dependency demo So this is a simple program that uses Lua and it opens a bunch of PNG files
22:01
So it depends on Lua the PNG and then the PNG depends on zedlet and this is what it looks like so there's the the Build definition file, and there's a single C source file, and then the sub projects folder which we'll look into a bit later
22:21
So if we can't the build file it looks like this so basically just define the project And then you say that you have these dependencies Which are Lua and libpng and the fallback says that that? Where the fallback dependency is so it's in a sub project called Lua or libpng
22:41
And then you just say I want to build an executable, and then these are dependencies that I have So and then the sub project folder looks like this and it only has is these three files Which are called wrap files? And if we count one of those then it looks like this basically
23:01
It's just an URL file for where you download the source files and the the hash code For verifying that they're actually the real thing and they haven't been MIT So there's the source file and then there's the the patch file which in this case adds the the meson build definitions because the original project doesn't have them and
23:24
Then you start run the configuring step. That's what it looks like they exceed downloads libpng and it downloads the patch and it does all the patching and As you can tell it says native dependency zedlet found yes So in this case zedlet was available in the system, so it doesn't download and build it itself
23:43
It just uses the system one, and then if you compile it then it looks like this and then there's 39 Compilation steps for Lua, libpng, zedlet and this executable and and that's it and this now works works on all the platforms where the libpng and zedlet and Lua are available on and
24:08
This is quite interesting in itself But what can we do with this? Now that we have this in, it's time. We need to go deeper We need to all go all the way down and achieve Pythonception
24:25
And ask the question Can you compile CPython with itself? Well, let's find out Here's a check out of a CPython repository which I took from the from the git master some time ago, and I wrote some of the build definitions and
24:43
there you see running the meson command and It's run the configure so you get all the things in there I'll check the functions size of things all that and Then you run the build command
25:00
Which looks like this saves about 143 two compiler warnings and Compared to the stuff that you can make this actually kind of nice because the only thing that is visible are the compiler warnings And it's not telling you that now I'm going in this directory. I'm going in this directory I'm going in this directory and I'm coming back and I'm going then and all of that goes away because you don't care
25:22
What this? And then if you run the compiled Python with this H Then you get the help output from the regular Python executable Which is it's kind of nice but compiling Python on Linux is not actually that difficult
25:43
It's it's fairly straightforward So let's go deeper. What about Windows? So here's the screenshots of running on Windows 7, I think with Visual Studio The LibreOffice in its infinite wisdom has decided to scale this picture down for some reason so it's a bit blurry
26:01
But there you can see it's running all the same tests and then if you run the compile Then you get this and this I don't know what those warnings are, but they don't seem to matter and Then if you try to run generated executable In the same way as earlier then what you get is this error message?
26:24
Which I don't know what it is because I don't really do much Windows development. It was kind of complaining about some missing DLL But ignoring that basically it works The at least fixing the rest is left as an exercise to reader
26:40
So let's look at the performance numbers of this thing Who in here knows how big the configure a C file is in? Oh, sorry, sorry On slide so on my four-year-old I3877 with eight cores the configured time on auto tools. It takes 13 seconds and
27:02
With meson takes six seconds, which is about 50% faster build time is 31 seconds Which goes down to 16 which is again about 50% faster. I also ran the tests on a Raspberry Pi 2 and Configured time from that is 3 minutes and 11 seconds and went to 131 which again about 50% faster
27:20
And the build time is is from 9 minutes to 4 1-2 Which is again about 50% so so 50% improvement overall It's this is not entirely fair because it's not building all the same things, but it's roughly about the same
27:40
So build definition size and so how many of you here know how big configure a C is on Python any guesses Anyone so the correct answer is that it's four thousand five thousand four hundred and seventy seven lines And if you haven't read it, I highly recommend that you do because it's it's both terrifying and awe-inspiring in the same
28:07
exactly the same time It has still support for stuff. You didn't even know existed and it's probably older than most people in the room and In addition to that there's the make file dot pre dots in which as you can tell is twice templated and that's
28:23
1700 lines and there's some make files in the sub triggers, but I didn't count those So in comparison the meson bill for the same is six hundred and twenty nine lines Which is like slightly less This only includes building the core and the essential modules that are parts
28:43
And this has only been tested on Linux with GCC and Windows on Visual Studio. So this probably needs needs some work once another ones But on the other hand, we still have about six thousand lines of spare until we reach the same same amount of codes as the first one, so I'm fairly confident with that. That's achievable
29:08
All right, but can we go deeper So here's something that I thought up as far as I can tell this is currently not possible But it's it's completely feasible to do so what if your
29:22
Python compilation Went something like this, so you first you compile enough court code to compile the parser right and then you compile the standard library that you have from PI files to PYC files and
29:41
Then using a fairly simple script you can pile those PIC files Into C libraries and not sorry C arrays. So a C file and a dot H file listing those things right, and then you compile the rest of the Python and all the rest of your application that you want and
30:02
Then you link the Python core plus two modules plus the standard library that you've built as C arrays statically and this gives you a Fully self-contained program dot exe and you can run it and doesn't need to unpack anything It doesn't need any file system access or anything
30:22
There's no possible way that it can read the wrong PYC files because it only Accesses them that are inside your own executable. So this would be a sub projectable Python and The things that you can do would be it's basically like freeze or PI to exe or there's a CMake Python
30:43
Which is a bit like this, but if you have it inside the build system and it's inside The the core build of the language itself then it works on all platforms and all the use cases out of the box Then you can create cross platform vacations where you embed Python because it's just one dependency and it just compile it in and start call it and and
31:06
stuff starts happening and You can ship actual exe files and there's no file extension circus anymore One of the biggest pain points in the cross-platform development in Meson that we had is
31:21
the shebang line because on Windows you must have the Py extension because otherwise it doesn't work on other platforms You mustn't have them because that's the distro will say you can't have that somewhere user bin Python points to Python 2 somewhere points to Python 3 and and it's like It's it's an enormous mess. And if the Python exe itself goes away
31:45
Then all of these problems go away as well Your applications don't require Python to be pre-installed on Windows This is especially good thing and they don't try to use the wrong version by accident You don't get weird symbols Bangling thing look up where some of the symbols come from this deal some come from that one
32:05
And then you can combine this with GTK or with beware Which is a great project for creating a multi platform GUI application And then you can ship all the platform GUI applications which you can compile from your own source into
32:21
Like full executables and just ship them and your users don't have to think about any of that Going on further so there was a presentation by I think Josh triplet how they're using Python Where they compiled and they boot directly into UFI. Well, you can do that. You can have bare-metal support inside the Python So Python would be sort of like a micro less micro Python
32:43
Where if you have a slightly bigger thing and you want to control it completely from Python Go ahead. You can do that And there are people who run containers then you can do stuff like you have your Linux kernel and you the only process that's On the system is the Python process. It's just a single process you launch that and this has interesting implications because
33:05
most exploits in the world try to Open a shell on the target machine if your system doesn't have a shell at all all of these things stop working Which is kind of nice and and there's also like sorts of scale things that you can do because you don't have your
33:22
Images that you launch are very small so winding down so in conclusion Python is set to become a core dependency for a significant fraction of the Linux you will end currently It hasn't been they have been using
33:40
Make and shell and awk and TR and said and all of those Wonderful tools that we know and some love But which ours has complete maintenance nightmares. So those are all gonna go away eventually Which is which is nice And one of the main things that comes from this is that everything will become faster. So
34:05
For those who don't know if you run a shell script and you do a string comparison What it actually does is that it spawns a new process every time you compare two strings and Especially on Windows we process spawning is slow
34:20
then this like just being able to do in process string comparisons is a massive gain and If we have these new sorts of tools with new sort of things then new things become possible Some of which we probably don't even realize yet So coming back to the original question was that all build systems are terrible and why should I care so?
34:45
In conclusion what I would like to say is that the reason you should care is because some build systems are actually fun and exciting This has been my talk. Thank you and open for questions
35:04
Well the question nobody Yes To Is this if these think you mentioned just an idea or some actual project we can look at
35:21
so Okay, okay so This one so this is not at the moment done. It's it's just that it could be done
35:43
In in theory by someone else which is how these things always go But there's nothing like inherently difficult about this It's just works that needs to be done so like you need as an example if you I'm don't I'm not entirely sure if the in Python modules the entry point is always named the same If it is then obviously you can't because all of the things go in the same
36:04
Same executable so they have to have unique names, but as an example the G streamer multimedia framework which is like 150 plugins they have all the macro magic necessary so if you're compiling Dynamically shared libraries, then they all get the same name And if they're if you're compiling them statically then they get unique names and something like this would need to be done
36:25
If there are duplicated symbols, but I'm not a fair of the like actual details of it as possible Thank you very much for talk. I'm curious. How you make sure that
36:41
Mason is still fast when you make a change so I said we have the the testing framework which is in place and The biggest so currently when we run CI it takes quite a long time, so if it gets lower We will definitely notice this
37:01
Quite quite quickly in fact and and one of the biggest problems that we currently have is that Due to some bugs in other parts of the things we have to do things a bit slower, but only in CI In in regular use it doesn't matter this is all an issue that comes up But for CI we have to do some fudging and because of that it's like to work slower
37:21
but but it's still still pretty fast and We're very much counting on the fact that people are running Building their stuff against meson master, and if we get slower they start yelling, and then we'll fix it Okay, great. Thank you very much They have another another question
37:43
So you talked about Compiling for for Windows for Mac OS X and so on and so on Consider I have to build a product and it's an embedded program, and I have some Microcontroller can I use meson to to work with some?
38:04
Not yet known compiler, and would you recommend it? So currently we only support the compilers there, so that's GCC clang ICC and then like Ten different fortran compilers which some guy contributed
38:21
And we don't you can't compile on a compiler. That's not specifically supported because we need to know how you run that But if you want to add this we are open for patches well We love patches if you if you send in and it's not actually that difficult so the the language support for D was added by one guy who had never seen the code base before in about two days and
38:45
Just adding a C compiler an existing C compiler would probably not be that much of a job so that means that you Have predefined rules for compiling what if I in a project I want to have my dedicated
39:07
Options so I Want to change? Optimization levels and what else is possible is that is it possible with with mezzan or is it impossible? Because the rules are already closed
39:20
So what we do is that this predefined? Target built optimization types, so there's an just debug this optimize for speed release and minimum size and then there's the nothing at all and then you can specify your own compiler flags and We just use those and we don't use any of our own so if you want to override that's fine another question
39:47
No, thank you so much The next talk will be done by Nicola Rossi in some minutes Thank you