We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Sharing Reproducible Results in a Container

00:00

Formal Metadata

Title
Sharing Reproducible Results in a Container
Subtitle
A container you can build anywhere
Title of Series
Number of Parts
490
Author
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
Containers do a great job separating out different parts of a system, making sure that they don't interact unless we want them to. What happens when a colleague hands us a project they've written and we're supposed to host it for them? They're not programmers, they're scientists. Who knows what they have in their program? How can we keep it up to date and deployed with a minimum of fuss? Come and see how we've solved this problem with Guix, from rebuilding or replacing the dependencies with modern versions like a pro, having only the bare minimum required software in the container, deploying in an artisanally crafted container like a hero, and upgrading and rolling back when ready.
Multiplication signContext awarenessSystem administratorBuildingComputer animation
Presentation of a groupResultantComputer animation
Boss CorporationServer (computing)Web applicationOctaveDirectory serviceComputer fileInternetworkingTotal S.A.Functional (mathematics)User profileData managementGrass (card game)Physical systemOperations researchOperating systemMultiplication signOctaveWeb 2.0SoftwareWebsiteMobile appComputer animation
Line (geometry)OctaveScripting languageComputer animation
OctaveRevision controlComputer animation
Graph (mathematics)Lambda calculusOctaveRevision controlEntire functionService (economics)Operating systemMathematicsBootingConfiguration spaceClient (computing)SoftwareComputer fileoutputSource code
Hill differential equationPhysical systemComputer networkIntegrated development environmentExecution unitInstallation artWeb applicationLambda calculusPhysical systemScripting languageComputer fileMultiplication signGroup actionRevision controlConfiguration spaceOperating systemOctaveWeb applicationSoftwareSlide ruleBuildingPoint (geometry)RoutingGraph (mathematics)Electric generatorElectronic mailing listDirectory serviceService (economics)CASE <Informatik>Type theory
Open setPoint cloudFacebookOpen source
Transcript: English(auto-generated)
It's time for the next lightning talk. Efraim is talking about reproducible containers in the HPC context. Thank you. Okay. So, yes, I came into this really more
from the sysadmin side. I like making things build. I guess I work with the scientists, and so they have their software and need to put it out there for the papers, for other things that they can go and point to and say,
here it is, here you can see my results. So I was home working, and so we got a new app that we were gonna go and host on the website. It was being moved from old one to new one. So the old website was running on CentOS 5
with PHP 5.3, with Python 2.7, and another Python from the user's home directory, and Octave also from 2010, and just everything was from 2010. Just looking and thinking. Well tested, of course.
And it also accepts arbitrary files, upload whatever you want to it. Clearly exactly what we want running. So actually, what about it using Geeks? So with Geeks, you tell it exactly what packages go into building each package.
So you get the reproducibility in that. It works also when you have a collection of packages. Every time you install it, you get the exact same packages. You don't have cruft from installing and removing packages, and it also extends into a whole operating system or in this case, into a container.
You tell Geeks, I want exactly this inside the container, and that's it. That's exactly what you have inside. So for the web app, we have all of our old software here, and I guess going back to the container, one of the scripts I had and actually patched all the way.
Just missing, remove, rmdur. Just, I didn't tell it where to find it. I didn't put it in the container. It just wasn't there. So yeah, so had old versions of Graphviz, of Octave, all of this coming down to our final package.
So with Geeks, it was easy to go ahead and to create the old versions of the packages. We started with the version of Graphviz that was already in Geeks. We changed a couple things, mostly downgrading the inputs to older versions of them.
Octave was basically the same thing. We took Octave. Q-Haul had moved around since then. Gnu-Lib had moved around since then. So just some changes here and there to make everything actually build with older versions of everything. And then for the actual container,
and pretty much this was the entire config file for the container, it's to find the operating system. Bootloader doesn't matter, it's a container. There's no firmware, no packages, just the actual service itself and the DHCP client so it can actually find the network that you give it.
So the service gets fed a service type, a service configuration, which in our case was, can't see the mouse. So here we just have the actual package that I built at the end. I told it that it was going to be in the serve HTTP directory with that port.
Our little activation script here really just says, before you start, go ahead and run these actions so it was, if the directory exists, delete it, then recreate it, copy the stuff over and it wants to write to the directory so go ahead and make that available.
And the whole thing was just behind nginx so with the system management, the config for nginx, it was really just listed on the port that we had on the previous slide using the route that we had on the previous slide from the configuration. Then service type down here was the one
that we actually passed to the operating system config to make the container. We said, when you activation service type, when you start, run the script, you're using nginx with the config that we have and by the way, don't make sure that PHP FPM is running.
Building the container was the system container command, you feed it the actual container file and same serve HTTP folder directory that we had from before was actually mounted
in the home directory but we pulled the container, put it in that spot, I wanted the logs and give it the networking and then actually we're running on top of Debian so we just used systemd to launch it so we have the same command as before,
we have full path to geeks but geek system container, full path to the container itself, the same two shares, you need the path and give it the networking and then just go ahead and launch it. The whole thing runs under the BNW user so all the random files being uploaded
were also owned by it and as far as updating this every now and then, just ran geeks pull which pulled in new versions of, not of say graphvis or octave but new versions of PHP, new versions of nginx, all the other packages that I didn't have to go
and downgrade and then restarting the service, it just goes and rebuilds the container itself and it's there and available from the Debian, from the Debian host that this all was running on, we just went and said, point nginx at port,
what was it, it was 8880 and we got it from the outside so then, at a certain point in the future when it does go and crash and I have to go and rebuild the software, we have another command I haven't listed here, geeks list generation, geek system list generations,
that's system, you can list the generations from geeks pull, so it's geeks pull list generations, you go back to the previous generation when it did work, restart the container again and you're back up and running while you figure out just what changed this time.
So with that, we're hosting the old web app using new technology, newer technology, newer versions of everything and everything seems to be working.
Thank you again, we have time for one question.
Thank you. What's the outlook for you? How will you go about updating now that you have it on the known state so that you can move ahead, what are you planning? Right, okay, I'd have to repeat the question.
Okay, I guess I'll do it anyway just in case. So the question was how do we update the state? The actual package that we were given doesn't really get updated, it was more of making sure that there weren't anything left behind
from previous times we had restarted it. As far as the actual packages that we used to build everything, I had to downgrade GCC to GCC5, so that was one of the things that as we moved forward, I had to add that in to make sure that it worked again.
Yeah, I guess the rest of the dependency graph of everything that was running more or less because it's on top of GeeksGeeks keeps everything else up to date and then I'm just locking these versions in
at their old versions. All right, thank you Ephraim. That concludes our lightning talk, please applaud.