AV-Portal 3.23.3 (4dfb8a34932102951b25870966c61d06d6b97156)

Adventures in building open source software (part 1 of 2)

Video in TIB AV-Portal: Adventures in building open source software (part 1 of 2)

Formal Metadata

Adventures in building open source software (part 1 of 2)
Title of Series
CC Attribution - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
A year of tinkering with pkgsrc and others As a pkgsrc developer ensuring a tree of previously added software builds correctly across various systems / architectures and as a "developer?" taking an existing project & applying the methodologies learnt from the *BSD project developers to improve the code base. Covering two angles of one problem (software) embarked on someone who is new to it. Almost a year ago I began to revive Darwin/PowerPC support in pkgsrc to allow up to date packages be build on PowerPC based mac's, at the start it was possible to build less than 8500 packages from the tree on OS X Tiger/PowerPC, sevan.mit.edu is about to exceed 11,427 published 32bit packages for the Darwin/x86 (Figures taken from 2014Q3 bulkbuild by Joyent). This talk will some the issues which needed to be tackled & what's yet to come over the next few months to attempt to build as many of the 15000 possible packages available from pkgsrc on this architecture along with expanding the effort to building to 10 different operating systems across 5 architectures. For the programming angle, discuss my work to clear up the coova-chili code base to use the facilities the operating system provides, introduce functionality from the OpenBSD (e.g. strlcpy) and testing building across the BSD's to improve the codebase.
Computer animation Open source Projective plane
Web page Metropolitan area network Computer animation Software Internetworking Projective plane Maxima and minima Wireless LAN Freeware Form (programming)
Functional (mathematics) System call Interface (computing) Electronic mailing list Bit Coroutine Parameter (computer programming) Open set Code System call Portable communications device Revision control Message passing Computer animation Software Interface (computing) Set (mathematics) Data structure Freeware
Functional (mathematics) Building Computer file Code Connectivity (graph theory) Direction (geometry) Modal logic Multiplication sign Source code Revision control Programmer (hardware) Fluid statics String (computer science) Operating system Software testing Computing platform Physical system Operations research Email Projective plane Total S.A. Bit Line (geometry) Computer animation System programming Daylight saving time
Laptop Operations research Building Open source Multiplication sign Bit PowerPC Mathematics Process (computing) Computer animation Hypermedia Computer hardware System programming Daylight saving time Cuboid Pressure Daylight saving time Task (computing)
Building Group action Open source Computer file Connectivity (graph theory) Multiplication sign Source code Virtual machine Set (mathematics) Parameter (computer programming) Total S.A. Revision control Fluid statics Hacker (term) Linker (computing) Core dump Operating system Physical system Module (mathematics) Email Dependent and independent variables Projective plane Database Demoscene Uniform resource locator Message passing Data management Computer animation Network topology Chain Multimedia Energy level Fiber bundle Quicksort Resultant Library (computing) Spacetime
Type theory Computer animation Open source Oval Binary code Binary file Power (physics) Physical system Computing platform Number Data type
Link (knot theory) Open source Code Mehrplatzsystem Multiplication sign Execution unit Source code Parallel port Set (mathematics) Directory service Binary file Power (physics) Neuroinformatik Mathematics Process (computing) Computer animation File system Booting Computing platform Physical system Spacetime
you have 1 on minus 7 genuine and going to talk about some of the things that working on over the last year with various open source projects predominantly package source from that PST the so to
start off with a started working on a project called the Cooper Chile which is a captive portal software for use on wireless networks of what that is is typically
when you go to a cafe or a starbucks and you connect to the public Wi-Fi usually presented some form of a webpage which you authenticate and then you granted internet access to virtually and allows you to do that so I originally reported this to Free BSD many years ago but i i left it to grow stagnant and has free BSD moved to declined in previous 10 of the packages broken so I set out to have been fix situation and it
wasn't actually any restrictions on what was anyone restrictions imposed by kind itself everything else was basically bad coding habits within the codebase which kind kind of raised to clean up the restriction that kind post was that kind doesn't support nested functions so I had to break functions out and so it should be called out logically rather than having 1 joint almost function with functions within itself which would get called out everything else after that was basically defining the parameters for the function calls rather than just having a an anti parameter list and through that's got things going
after that it was basically using deprecated functionality for setting up the network interfaces which previously finally version 10 that had been marked as deprecated back in sometime in 2002 thousand of but actually this is a good thing because the original message was you will take 3 or 4 calls to get actually set up an interface where is the new method and it's a single call with the structure and you interfaces are going on the problem is is that the codebase is also out wasn't really it's not written in with portability in mind so there's lots of horrible if they're all over the place and and and after saying I had to contribute to this a little bit in that for Linux everything is still in the old method but if your Free BSD open BST all Mac OS using new improved methods and off we go and so after that I
kind of started looking into the source code so projects like a Verity and also kind for a static code analyzer and running back against the codebase raises lots of issues and also within itself rather than using the methods that to operating system provides to develop opted to kind of create his own methods instead so 1 of the examples is these functions that approved which safe which I don't know if are safe told but instead of having something like this semester copy which is found on all the BEST the Mac OS he decided to create his own version which is just a insecure string copy functions so this was a bit more painful so much actually originally a programmer so the codebase is heavily on oriented around using 0 2 tools in the tools and so what I wanted to do was basically rupees state functions out and replace them with what's there in our lives the and then gets in the header files for imported on platforms that wouldn't have this in place so if you're building on Linux which doesn't have this this general copy the necessary header file will be pulled so traditionally if you 1 using the altar tools and stuff like that you would have your file in some common location and in your source code you would put pound include the header file to import and I thought this would be the method that you would do for all also tools but actually we do not want to do anything like that and you focus purely on tools and have tests written or totals and so when you're actually building it into tools would probe your operating system and if it doesn't find the necessary components itself with pulled out the necessary components in and so I will still quite a bit of time and due to this but thanks to Darren John from the from BST projected kind of prodded me in the right direction to doing things with and and it's literally should actually have on the thing with which 3 line check for doing the test so after that have
so these changes is still waiting to go in the support request on get out and there's more to come for this and also I moved on to have dealing with a package source so have a source that I started off trying to build a package source on old Mac OS specifically Mac OS tiger on PowerPC and he then this old hardware is kind of a very slow and it's not very powerful and I had an all laptop which arose from the duties builds on and I'm trying to build things like GCC and stuff like that was taking a considerable amount of time and putting this laptop that had and lots of pressure so I was given a Mac Mini to kind
of dedicated task and Frieda's for laptop from tortures of in building GCC which would take 48 hours here if you wanna GCC jobs support and things like this and the British Summer Time even though this is a fairly small of box it actually still puts up quite a bit of heat trying to get things compiling so I've got on Social Media and endorsed around and I ended up landing the
machine in MIT which sits there and I can just run built and it doesn't matter and so this is what is what is up here is the result of the 1st building of the entire tree on this machine MIT and at the time there was 15 thousand possible packages that could be built from sex tire there was a thousand 500 built successfully and then these were the key packages that cause the most breakages and so for methylated the reason that would build was because the toolchain that's Apple provided it wasn't able to generate the shared libraries and the message you wanted to generate so originally I thought most of that stuff is open source so I'll try and update the components myself and wrapped bundle that up as a separate package which people could use on an old version of Mac OS the problem was was that those are the source code for them what they call the CC tools is available when the hacking tossing came about when and all decided to move to Intel Macs Apple withdrew most of the and build documentation and components for building the operating system itself and so you pretty much do you have the source code for some of these components yet action on how to build it because for the documentation is being pulled so I kind of cheated and built a static version of messages to kind of get around the issue of not being able to generate shared libraries with them the chain that have been the provided for scene make a scene make was an issue the build had been built the project provides sort build falls under the project provides for building on Mac OS assumes that you're going to be building in a set location and any the other path they cannot handle so what would happen would you would build in C make from package source package source needs open SSL z lives and and some of these common core components so that would be passed to the build system and the build system would start building and then it will get to the linking stated that all I know all of the components that I need in location that Apple provides insights developed and that would blow up because it would be pulling in headers from a modern version of components and trying to link to a 15 year old version of components in location and that was quite painful for the various versions of Ruby how come back to that for the Qt and that was not really solve cheating is really big codebase and even yeah and even the smallest problem the this quite big and we could take such a long time a kind of didn't just to try and fix this things that aren't here it was in the version of GCC that was on Mac OS title was version 4 . 4 so I wanted to bring in new versions of GCC so people have modern compiler but for most of that really it would build but the problems were in package source where like when you're passing parameters dual-income the version of the linker and Mac OS cannot handle having spaces in the past past and whereas modern thinkers can so obviously this was hadn't been backported to older versions so really for me to get things going just had to remove a space from a path and things just the so removing it
turns out that Ruby has a very very decent self-documentation for their API however for the bill documentation is absolutely non-existent so when you go on and asked people how do I build really is quite common response to the actually put pointed to a package manager for your operating system and told guns these brutal it off here more young something like this which doesn't really help you if you're trying to build itself and literally the problem was was that's the database module in Ruby was unable to cope with Berkeley DB in Mac OS because on Mac OS the files a splits into 2 separate files whereas really be expected to be 1 and half and that was that was a that there was 1 comment in the source code buried somewhere in the data is you and I lost a lot of time the and is very annoying
and so by the new year it's managed to actually exceed the number of packages available package sources from the internal Intel binaries so also that I kind of started looking around that you know new platforms to play around with on package source and and every geek kind of has a soft spot for POS right so you have had high Q is still going so I thought I kind of that my kind
of play around with that but I
would like to have that was really painful and I been actually get very far because I'm not sure if this is still a this is applicable to BOS but with k there is no notion of multi-user system it's a single system and and you're magically what his unit the deceive users and from the file system there is no false system such as you have packages which contains snippets of false system and there's a Damon that starts out when the computer in when the system boots and this Damon takes you packages and union amounts all the packages that you have to form our a you of what appears to be a user like and then you have a piece of writable space on this which is your home directory so for package so as to whether he would be basically bootstrapping in your home directory and it was with a lot of work to actually trying to integrated into the system and the other problem was that parallel which gets pulled into a build quite early on in the process and not build and all it would it would build but it wouldn't link and so on the guys out the impact of managing system but the way that they've actually implemented all the changes its rather than trying to integrate with what's actually there and they're just going to to go on and started deleting stuff and and replacing and when you're looking at and changes you have these quite extensive there and that you need to kind of unpaired where like for us in package source we don't touch the Perl code reduced at build time we pass kind of expect settings file to say build problem the settings and and the bill goes off and does thing rather than actually modifying the source code had to hard-code the settings we want to build with an and so I kind of gave up on this and thought about another platform to and apply online and their their their their their their their hair yeah there