The high performance computing team is a central contact for high performance computing at Gantt University. So we have a big computing infrastructure, seven clusters at the moment, and one shared system between the Flemish universities.

The third tier one was number 119 at the top 500 in 2012. There are eight people on the team, they are all here today.

A big part of the work of our team is to install the scientific software for scientists. They want to do simulations, they want to do their research, and they need the software to run the cluster. They can't install it themselves, they asked us to do this.

And of course one of the modules is automate everything. So we have seven clusters, we are not going to install manually the software on each cluster again and again. We automate the build procedure. As in the previous talk I already explained, building scientific software is really fun for us.

Very good documentation, everything works flawlessly of course. No, not the case. As he said, it is written for people who need it to work for themselves.

They use hard code. If you want to use the environment modules, as we saw earlier, you have to patch it. We expect this library to be a lot faster. We need to use the environment modules. So it is very time consuming also for the HPC team.

The tools that were available are lacking. We prefer to build everything from source, so the installation is not just building the RPM and installing it. It is high performance computing, so with a CPU you want to use all the instructions.

It has to be compiled for your CPU. To do that with scientific software, you need to use Gen 2. Almost none of the packages that these researchers want are already in there, so you have to do it yourself anyway.

Why restrict yourself to one of these? There are tools out there. If you make a list of bash scripts, they just don't have to use bash. 10,000 lines is not really maintainable. Or just stand alone, start a new thing every time, but they don't try to reuse anything.

Or very OS dependent, as I said of course. And some projects, they know they are really hard to install, like Dolphin. They have a tool called Dorsall, which is where the bash scripts are used. Some people use it, it works, but only for software and it is really hard to add everything for it.

So, as I said, not all of them are packaged. And just make an RPM, yes, but then the other distros, people use another environment, you have to redo your work again. Also, current OS is really hard to install multiple versions of the program.

This is often a big requirement. Researchers all use the same system. Researcher A wants version 1.0. Researcher 2 found out there are new features in 1.0.2, so he wants to use that one.

They both have to be available in the system. And we compile them with different compilers. We compile it with the Intel compiler, GCC, and TI, because sometimes different compilers get different results. So we want to test which one generates the most performance code. I need it there.

For example, everything with Intel, mostly that is the best performance, but GCC. The researchers also want to use the same thing they have on their laptop, and then they compile it with GCC. So we want to make the code available. Say you can switch NPI stacks, change the month journals.

So what we wanted was a flexible framework. So we wanted other people to be able to join us in the effort and add features easily. So it had to be flexible, modular, pluggable. So a big part of this is a reproducible build. The scientists need to be able to reproduce their experiments.

So if you compare it with one cluster, then the system has gone up to three years, so if you want to do a little thing again, you're trying to make the same version of the software, and all it's doing is the same version.

Or it explains or exists in different versions. A big deal is also the sharing of the effort. So we put a lot of effort in it, and we want to share it with other teams all over the world. So if we share it with them, hopefully they share back, and in the future maybe we can easily install something without having to do much effort. Of course, everything has to be fully automated.

And the balancing resolution, if we install the balance sheets, so we don't want to build everything one by one, we just say build this, and all the balance sheets. So that's what EasyBuild does. It's written in Python, so people can use it.

We like Python, we use it for all our tools. EasyBuild has been developed in-house for two and a half years before we made it public. It was completely re-written, making good practices in mind, and released as a resource in development as well.

Then we finally got the state API in the developer, and since then we've been seeing multi-release systems. We're working on this continuously, because a lot of users keep testing, and other teams are requesting me to add features,

so they ask me to actually answer the questions,

and we have support from the system to use our clusters.

Especially in software, to make build and balance, to be what's in the balance that's going to push. Also, to invest and start to build, 48 hours later, all the software that was available on the old system. We tried to do comprehensive tests, and before every release,

we used the job, and we built all the software we built. Currently, what we released, 43 packages are in our public inventory.

We took 3,000 physical things, so we developed different versions of them, and changed them in order to build. Here as well, I made a list of all the packages that were in there. It was readable for about 150, so in one year time,

we added another 300.

The framework provides all of the port functionality, like the commands for patching, or pushing to git. All of this is connected to the framework.

Then we have Easyblocks, so some software, the recipe, which is not enough, because by default when you configure make, they can still use this.

We specified to do, instead of configuring, make install. This is pure Python module, so this is where you implement the entire build, so instead of documenting it. This can be generic, like for a Python package, first you install Python, and then you want to install model-tip.

It's Python, it's object oriented, so usually you're just, currently there are still text files.

So as I said, modular design, chemically extensible, so if you add these blocks,

somewhere on the right space, or load them, put them in the Python bot, in the right module, and the interactive command compiler,

which is really awesome, if you already have module files, and you want to reuse them. This was added quite recently. We test everything. If you're planning to use it, please also test things here.

We've been working hard on actually doing groups and sources, so actually it's a better test for the software, so we install the package,

and then we ask them to give us an example with that. There's also no problems with environment, what you're setting, if you have a lot of dependencies, it becomes big, and by spraying your storage system, it has to go through all the directory, so there's solutions for that,

but these are being investigated. Okay, maybe I convinced you you want to use it, well what do you need? Linux, it will work. We test this on the Fedora database,

so we have a developer meeting on a daily basis on OS X. We have no Windows support plans. This is because there's no environment modules for Windows, so we can't use it. It's not because we can't use it. Py2.4, so we run some old systems

that still have Py2.4, and we support it for now. I'm not sure for how long, but for now we're going to Py2. I see a few more questions, and then we combine to start your first build of the GCC, or the ML. Installing EasyBuild is a good step.

We'll look into providing RPMs and dev files too, because that's maybe easy to read to actually get it installed. You can install EasyBuild with EasyBuild, configure it,

try to make it heavy. So if you want to know more, we have a website, we're on GitHub, the general EasyBuild free note. Follow us on Twitter. Questions, you can also follow us.

I was in terms of specific version of GCC, but you say you do that. I understand why you do that. Can you say a little bit more about that, why you do that? I didn't like that, but I thought we'll get you sure. So we do this so we can reproduce the builds.

We build with this version of GCC later on. The researcher wants to visit on his laptop, he wants to try the same build, maybe a different version has different features or does something else, so he can actually reproduce the build

with the same version. But basically, we build GCC once, and then it will be used forever to be on your system so you don't have to rebuild it every time. Every new version of GCC comes out, you can rebuild all your packages with the newest version, and leave the old ones in place so the researcher can get paranoid.

Everything just changed, and the new version of GCC comes out, you can rebuild it for people. I have similar use cases. I've researched to come to use my systems, and they weren't the latest GCC, but people are using our system in production. They won't change it.

As a matter of fact, there's a mechanism in SUSE Linux for putting in extra compilations, extras or something like that. You put it somewhere else in there, and that makes sense. If a researcher is creating their own software to run on our machine, should I be recommending to them

to create their own EasyBuild modules for their package? A few of our users have actually known this. They can ask us to install software. We do it, but if they say here is the EasyBuild, it works in EasyBuild, then we'll just install it. A lot of them are working on several different systems, including the national C1 machine.

I'm recommending CMEK external project for that kind of thing in a lot of cases, and I'm wondering if that would be a better feature. We wrap around CMEK. Do you use CMEK? I usually configure it in install, but if there's a project already using CMEK,

just wrap around it, and also call the CMEK. Everything still works. When the sources are stored, it's like anywhere over there. Well, when you download the sources for them, some of them might be still old, right?

Exactly. So this is one of the known problems. Sources can be removed on the web. There are some efforts to actually start the source that we're allowed to distribute, to collect them somewhere on the mirror. This is a project we didn't talk about.

Yeah, that's a separate problem. We're not trying to solve it. Ultimate version of it. We have a YouTube video as well. Just on the archiving sources thing, a certain scientific package, which will not be named,

changed the sources without changing the version number, quietly changed it online. There are several ones, actually. And that's why in the new version, we added actually Jackson. Is there some part of the cluster

that was easily defined? Yes, the one I get. So when you pull the file, is there an issue with the file? So currently, these are just easy-going, so you pull all of them in.

It just gets installed quickly. So easy-going is a framework, the easy blocks of the implementations of the procedure, and then the specifications. So we call these examples. They're known to work,

and we try to share them with everyone when there are certain examples you can make. If they work, you can share them with us. Sorry. Intersecting with another vital theme of course, then, you're packaging up a file tree and practical modules. Could you make a Docker container out of this? I mean, that's it.

I'm talking about, you know, code. I'm sure you probably could, yes, but we're not going to do that. Well, I haven't looked into it yet, but it's just like basically an RPM or a dev file. Well, you can make a Docker, have a look at the applications and ship that, and I'll be able to, you know, keep it for, you know,

especially once. We've been looking into these Docker people, and Docker did a better job for me. Okay, that's it. Thank you very much, Jens.