Adventures in building open source software (part 2 of 2)

my voice Colonel Tom and the constant the low you know that I k so I don't wasn't successful wizard getting the haiku pit package was going on I and so I started to look around at and other platforms to to try out and so at the moment IBM is trying to do what someone's doing nearly 10 years ago and appearing friendly and offering free services and 1 of the things is at the moment they have had to develop a program where you can get access to IBM and power systems running AIX and I am I 5 and for building stuff on testing so I signed up for the for

the program and the so you basically here and you you apply on the site and they give you like to out 2 weeks lots and 1 a power sixel power several power rate system running AIX 6 7 7 1 so I started off and on AIX 7 1 power right and and try to get things of beauty now In theory this is supported by platform and package and

so the 1st thing is actually that IBM off 2 images of to try 1 of them is what they call a porting image so if reporting software that's where it you can use this image and the other 1 is a standard AIX image of what they classify standards no then if you can see but of the 4th row down as the amount of free memory now and that's how much memory from a 2 gigabytes of a system of 2 bytes of RAM allocated comes with out of the box it turns out that actually IBM and automatically turn on things like to warmly and and things like that on the image that they supply said that sits there and chews up all the brown and the other thing is is also the area ResourceManager consumes whatever else is left so you have a whole 17 megabytes of RAM and to have had to play with and so that's 1 of the things that I kind of had

several attempts at trying to get a package source code in the 2nd to last time 1 day on the on the images that they provide 90 day provide GCC but they also provide what they call their XLC compiler and so I tried to read struck package source with GCC and the truck completed fine and then so what you want to do is start to ball will process which would go about scanning the entire package source tree generating your list of dependencies and go go about building stuff so I would be instructive GCC kicked off a ball build which would stop scanning and scan would just get to like 5 thousand packages and stop now there's nothing log so this is without doubt proper into K Sh is the default shell so modified the and the settings falls and used PD the public domain case find off and I got this reality failed so was lack of RAM side freed up some random and then with you limit by the linear up to 256 megabytes fight off again and it's a and so now got this call fall out looked around and there's a canonical dbx which is there debugger which you can point to call falls turned out that was installed can because they haven't provided the dependencies so this time I thought what was that I should quite active on Twitter say comparing them questions and so was still waiting to kind of give me an answer of where the dependencies I went back and did be checked again but without GCC this summer got different error which was this no process to read data is applied and and then 1 last time I modified descriptor out of the box is used to public domain case states and again going back to the plenary for cold cold so I gave up on this and then uh my last attempt which I have trying to text I thought maybe try the other image of might have more success so this is their porting image invented tried this time so I opened up a reservation got a

new image try to bootstrap and could even bootstrap is can the whole thing blew up and got this error message and actually moving around of of for the the RID attention to this page and it turns out this is an area that was discovered back in 2011 IBM broke something in their latest and greatest service back and what you need to do is basically not downgrade from whatever version that you have to an older version and at that point I thought you know if you want I X could could have yeah that could have 5 and actually the mean the IV baffles me how as a of unit in the in the open-source space we don't have these problems and yet we compete against them and Free BSD kind of worked out of work to that issue you have the funny thing was was this 1 package in package source called as a stand-alone CSH this this package basically violates the rules of the packaging system which is it ignores the prefix everything gets installed onto and just puts the statically linked to CSH into slash which is a problem because travesti comes with its own copy of computation H and if you bulb building the ball will process would get to this package install let's remove it and then move on to the next package and now your minus society and if you're loving cellist TCS H the next time you go to connect with the right yeah but I mean like that was my login John was really funny actually my reaction to this because of my classic thing was was or why both of you have actually had to cut so as that have and so the of thing of continued to adopt free this stuff because it's just works and I can do to my remember again the and and so

often that I started looking at Solaris at but I call on a social media again and I got pointed to these guys called open Kashi and and they they have their own since the packaging system for Solaris and they have a cluster of systems and if you do not open source was still give you access to the system and for building on and so on there on the cluster that actually got Solaris 9 10 11 on X 8 6 spot and I can get package source going on Solaris 9 and on Solaris 10 X 8 6 there some issues which I couldn't get around but sigh I managed to do the bulk those on the spot versions of 10 and 11 and exits expression of 11 the problems with Solaris specifically with 10 is your user land is broken up and move multiple instances of utility so depending on what you're doing you would set up your path will go relevant components to build again and that's somewhat of a problem with when you trying to get stuff going on Solaris 10 the other thing was was that day do regular updates of the operating system without the Lipsey gets added new functionality so there you got Solaris 10 of you may be missing some functionality and older versions didn't quite critical so 1 thing that was that man Doc would build on the old version of Solaris 10 for Solaris 11 and there was a whole bunch of stuff that's kind of deprecated that you would normally expect in a politics like operating systems and of we have issues of that in package for also can bootstrap but if you switch over to their son compiler it doesn't care and just kind of pass through and builds on I'm actually tried running any of these binaries but so I'm sure there's going to be something that's the point and then after that it was on the which is really must based operating system and there's a different set of problems with this which is the version of get text is not usable for us in our package source because it's built as a shared library and we don't use the slash OPT something like slasher lived in our so get text is broken I haven't I haven't found the solution for this but 1 of them would be that maybe possibly build of discussion of get text as a static binary and then in which case we work around that but it depends on how how accepting they're going to be of accepting such actual whether it's a feasible solution to ask the user to recompile they get text as static and the BSD and so

optimistic is view a really interesting for building again because this leader cell at the box and then made the switch to the 64 bit time and I have of 1 of the maybe is the guys and rodents and had a spark 64 system so I fired off a ball built on 95 5 . 6 and those of the community and its randomness from the BEST project gain access to his the 64 books numbers on that great and the problem with the spot 64 stuff is that it's really the protocol of performance of the system isn't that good said they expect a URI they expect you to paralyze your workload you know you build with make minus j 48 whole 96 which I could really do in this situation because I only had 4 virtual CPU so that took like the best part months to complete that I am the 64 staff so in free BSD and open BST we refer to the platforms and the 64 or I 3 8 6 within package source we don't use the operating systems of signs platform then we have our own naming conventions that we use exit 664 and so there's a lot of things that are applied if you platform is x 864 and make false of packages that are not applied to the open BEST and that's kind of race other issues which have to work in a minute that rank them in the

numbers even lower but there was an interesting by but platform to test against because basically it's open BST but we've climbed as the default and compiler so it's a which I guess you could do you want from BST hide that wouldn't have the same kind of changes will be imports so that was left have and then I moved

on the ground floor obviously very much like the if the guys have this machine called monster which is done in 48 Core Opteron which they use for testing various things and this was the built the 1st attempt that was finished the interesting situation here was that so through building was a package source we found to be quite a major problems their operating system with which cause deadlocks which they fixed but the nice thing here is that it's about building would make minus j 96 which is very common and so complete gold of like WebKit's completes in less than 45 minutes long and takes up about 55 gigabytes of RAM and to the and

then so when they're going back to the IBM I think aside from AIX also offer Linux onto you can choose pursues a Red war as you might just be sure that have actually come to own picture power system with a series of 12 and this within a few weeks kind of completed this ball build on the the interesting thing is is that the power PC 64 little-endian is something relatively new within the last few years so there you have software that kind of doesn't have any problems in the code you tripped up because the motor tools if they're out of date that I'm going to pick this up and you get kind of weird issues with things on building and and that was the

single most of the time actually it wasn't dealing with the problems of code it was actually done working around issues from the water from things like that so and it would mean that the new software ecosystem and there's a there's a common set of header files and libraries that older canoe utilities conceived and if you have an issue with the with the so called utility suddenly you find that Jesus doesn't build on get textiles and build and suddenly there's not a lot you can build from that collection and actually this was 1 of the problems that's the check what file and suddenly was constant another

1 is from the water column which is a script which tries to guess your operating system and this is going back to the output PC 64 little-endian for open SSH that wasn't aware of the little engine power PC 64 so actually the open essential compile and that it wouldn't build until they patched the problem with this is if this is really a out of date you get things like you build action being held up because you're going get to this I can't detect the operating system and wait for like manual user input so there's quite a few occurrences were lost days of build time because they're just sitting there on the problem and actually when we did the bills you can turn around and we restrict your bill time based on the CPE based on CPU time so if something is taking way too long so they have more than an hour so that the builders aborted and moved on the problem is is that there is no such thing for the other resources like on the common problem is also we're bad mirrors so lots of that it is a daily routine of checking the actual build process exceeded the build hasn't happened because there's lots of mirrors that effect fetching from 1 and then not connected configured so excellent August particularly bad at this lots of lots of dead FTP mirror is all about 50 pieces that FTP sites that just how can we currently have polygon again so

actually moving on from C other scripting languages uh if they're or were behaving correctly can have worked quite a catastrophic effect more than actually what's perceived based projects would have so like a like a a a i x if parsing working package source doesn't really get very far in the Bhopal process as an action just setting up your environment before you actually do anything and before

Python you like on bit rate at the moment pump under report on the reasons that the the package low is a Python 2 . 7 and doesn't build gets and this 95 thousand packages that can't be In the problem and he had

back to include that could have

led the crazy on cohesion and track the 3rd that time

and so true actually and

then all these bottles so far actually the things that have been flagged up in other projects so man dock in a short so I went and got himself an open accounts and removed over the model posits features that we needed in man so now man doc will build on Solaris 9 onwards the patch utility in Free BSD is missing various flags that's 1 of the things that we needed was the minus so flag of that got I didn't have free BSD and the dragonflies the deadlock issues were fixed in the kernel for BST make to building on open BST flagged up an issue when they were doing men copy on overlapping buffers whereas they should been music and memory and for open SSH now as part of the workflow for release engineering they're actually updating config guests and that's testament

to the power of this class what they were just

following the you know know what you think of any part of the

program group that will do that

I think that 1 of the things we yes but only cares about the word benefit to the products of these utterances from you find that the sum of the there's there's report of the occasion part of which of them will Mitchell princess and none of the utterances of found in any area passage in the book of the products program was likely to at most but what we say we saw this yesterday and we were just following procedure like that you can see when we get a lot worse than the kinds of things that you just do this and these institutions were about half years and you want to of the project but it turns out they will be used by the federal yesterday in tutorial where I'm going to say no matter what I would say no the use of in the work or the crossed over previous schemes and usually interfere with the distribution words used uses this out the hard it's restaurants and you get the same thing going here just to try to to build a couple that's just a couple nights ago that I was going to 1 those beginner introductory procedures and that's really obvious the instructions and provide you it's a constant instruments and approaches from use condoms and he's he's just trying to get something like this is what what I was doing was used useful as going through and trying to follow the instructions in case you the areas that were not immediately obvious to you actually don't the think this is where we start what do you expect from the exact same thing in a way that I got that about suppose the user chose 15 groups people when there was 1 gun so the question I want you to know about the response was 1 that was part the test that it turned out that you know what the response that we were interested in protest at the top of the list you know things that these are like concept lets say it in and I I I really want to to have the identification of some of the things that you need to know how to use integration and I think that that's the thing that's what I'm doing is part of the the but is is there and it's an action this is 1 of the nice things and under free BSD ports you've got like upward reagent used stepped the box and then the system that we have haven't package source is more of a shell script based on but the problem is is that in some of these environments on writing as an unprivileged user so I can't have things like manifest mounts union nouns and things like that so but I think that's probably going to be for the next step is looking at something like that because it's going to be there on which variable to saturate the hardware better like window DragonFly BSD 1 it's really Jessica a building with minus Jerry 96 but the thing is is 1 packages time where it's actually I could probably such right now hardware much better if I was building multiple things will have no power parallel build can't possibly problem on the right side my heart pack I need to deal with the microscopic another so I can tell whether it's currently used to say don't think you should also try to later the library that actually we were the properties but can you think of it this the following happens all of the and that the 1st thing you think you have to like that so quiet the and we will use the word error parallelization with the right but like if you're building on a Knowledge to him of but also known all systems that kind some new operating systems and have that function and here we cannot rely on the name top of the and so there's a there's a framework of impact source code p block and literally it's this shell script that you are wrong it will set up your environment and and then I think it's forms like change roots and so does the because we know that require the knowledge of the type have actually this is the thing what why we need prior Apollo because Paul gets it there is something within that dependency lists as his pulsars polymers working on your system that you can't really do much work and there is actually the other thing is is also like on these all systems electrode of the PowerPC Macs it takes like 3 days to scan down the 4th straight at and say something about something that would save time we can we go and welcome because things were much to me and thanks to