Bestand wählen

Lmod Building a Community around an Environment Modules Tool

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Erkannte Entitäten
it is also going to start of the art in a way that I don't have a
mind and K. thank you I'm from the University of Texas at Austin
and this is going to be a very different talk although it does have a hoax into several talks succeed in this room this morning so 1st of all I will talk about what our environment modules and environment models are a special kind of module different than a Python module or all a module they're something special and been around long time although they may not be well known anybody outside the HPC community of anger talk about what things in lieu make doing a lot the interesting my experience with building a community which is the interesting thing that's not related to my project but I think his have will speak at least my experience with doing that and then I won't talk about our some technical solutions that I used to build trust could that's 1 the things I think is hard about of communities and the things we talked about in the in the 1st meeting was you know it's a small language do you trust and some conclusions OK the 1st talk I saw in this room was about geeks and geeks is a package manager and I kept looking at it and looking at looking and say well this is just another way to implement them by modules and so let me see if I can point when you have a package Mary geeks what it does is installed you get a link tree you get a you know a simile tree the points all the software that you wanna talk to and you can roll back to previous versions there and I think the way they build things they couldn't shrimp environment-a sums troop on steroids to say they only built with these things that they say we're going to build well environment modules is different but is trying to solve the same problem so the environment modules or the way that the high performance computing facility of so users to manage their software so and the invited model system is just to control the environment environment variables path settings certain variables of l The library path of those things so in the beauty of of viral modules is gives you way easy way to say here's a group of mark here's a group of their environment variables I wanna set I wanna add to the path I want to make this package available to me so chemistry packages of biological packages whatever whatever year field of science are what you wanna work on compilers things like that you choose those modules to load and that gives you that environment so is very similar to the way or is trying to solve the same problem that deixis trying to solve but in a very different way and we've been doing it for a very long time we've been doing this since 1991 but so we the beauty of environment edgels is you can unload them so you can say I want this package and make it available I can unload as a software developer and this is really beautiful because I can change compilers change libraries but in in our world is important what recall MPI which allows us to parallel programming we wanna be of a switch them and this is just absolutely phenomenal so there i i and so in my large was a bit around a long time but I don't think the original paper talked about the currently main version out there which is a set language and highly people pretty CO OK so Tcl in my mind is trying to solve the same Nietzsche or least a lot of the same these will have its embeddable was designed to be embeddable language but it's a language so people like try to limit or what is my least favorite language OK so why is this useful well I've said most of this but high-performance computers there's a single computer that thousands of users well maybe hundreds of time but thousands of users are on and then they're all got different fields of science and they want to be able do different things and so models are a convenient way to support that and as I said is a developer it's just wonderful so the is only the debugger that also works in parallel and it's useful and so you take the the and nothing happened because it's not in your past so you type modulo DDT and that changes your environment and then you go TV the he it works you unload remove the the the model it's not there anymore that's the and that's it unfinished that's the talk right that's what that's what environment tools would be to do OK now we're on Unix time to go back the telescope so what's online and I is a rewrite of the regional environment model system so I'm going to call not in anybody's heard of environment right except for this guy in the room a communal room and come with a way of sort of environment tools allow OK but so under called the original system team I just because it's a
TCL based model system then so and I can compare it without my so you know what I'm talking about the both environmental systems OK but a lot was designed to solve a particular problem which probably isn't relevant here but if you have compilers you have empowered that modules and you wanna be about SelectBoost loses a C + + library that you want a link to the change compilers you like the this the version of boosted using you'd like to change it automatically that's what was designed a song OK this is Unix and we all learned with their mother's milk that all the parent process sees which you jump processes inherit from their parents not the other around so how can I possibly run a command that's going to change the environment right that should never work well it's a trick all environmental commands do is they generate text and then they use up shall function like the 1 shown up here which says run tell a jury where 1 put output batch and given command and then generate that and evaluate so that's what it's doing it's during text which is evaluate so that you can change your current environment so the beauty of model files from a system is just system administration point of view it is we want to change the environment for people using different shells while most are users use batch or shell well most use use Bachelet to stop and doing that but that's a story on but they also used the seizure I got completely different syntax the way you specify I wanna say environment variables so the nice thing about model files is you write them in 1 language and then expands to the language you need so this 1 has 3 statements is that help function which says something about this module it sets the environment variable and prepends the past that's basically what most model files so when you run the command Italian the bash shell you right back syntax and 1 in C shell you write syntax and you can write other shells to it also work in Python and well I've never done in love but you do it but there's support for the addition of a shells and so when you of the module it figures out what things need to be unloaded and changes pulls element of the past that's why I just the left the path as it was before or you can do it in the shows as well the that so what about my doing so 1 things it's interesting about modules is this model file right here says what you do when you wanna load that's the point of a module but in fact you want the action of when it loads when it loads when you print help will you do other things there's like 7 different modes that each this model power gets evaluated on in certain things happen certain things that so when you when only the help function runs and the other the other 2 things don't do anything so I've showed you what happens when you do a lot of what happens when you do and unload its reading same model but there's no if test and then says do this when you're loading do this when you're unloading so it reverses the action so you writing you write a model file in terms of what you want to happen when load or would help and then each function gets evaluated differently depending on what mode during loading unloading asking for help so how can you handle different mode you got functions here which have to be evaluated you got a couple choices here what is you have in certain implementations do this they have they have a single function with it as if I'm doing a study in the underlying trying environment variable if I'm if I'm loading I do this of monitoring this step doing help do nothing but I like that I since pipeline had a sort of Jesus lower has will is object of phi functions are first-class objects I can redefine I can redefine what set the Indian prepend-path oranges redefined and to be is the world version of the online version it works it's not and the 3rd solution is what I call my cat class solution which does essentially the same thing but it's good use classes so the basic form of what a lot does is I've told you that I'm mode of the simplified this as much as I can you know the code goes for hundreds of white so and 1 for you that will save and load boat I build like all the master control objects and column MCP and then when I evaluate a mark of module function again from the same talk more about that later and then each of those functions that he and B or prepend-path called
the class so depending on what kind of you know what factory what what the what string I passed to the build builds me load worsen war the online version
or health or whatever and so and then the matter neutral here it's got a table it constructs a as a factor which constructs the right kind of object and then depending on so this is
the load version it says OK helps quiet prevent that maps to the 1 that really prevents path etc. 1 loading Help us quiet again but pretend that is removed that the city and the is the only 1 set that this is a really took me a long time to figure out how to do this but this is a really nice change it really clean up about 1 call to say which mode I wanna be and I just I'm in 1 mode I say that away construct a new mode and we said the other thing I really like about the Lord is sandboxing so I fountain code I mean the mailing list as straight and stuff you can find lots of is really cool sandboxing was really beautiful for me because I'm reading code the somebody else wrote and they're not as good as me anyway the using their you know their there it's untrusted code so ratings the sandbox is really a beautiful way because 1 of things I do not want to do is expose my internal functions internal Belmont functions to model files and I did that by accident but I wanna do that and the other thing that I ran into is if somebody had a bad model file it's a program in it's good and when it gets evaluated inside if I didn't use assemblages is evaluated and Armenia stack-trace as you start near code goal to remind us look like there's a bug in the LOD when there's really not as a body the model so this really save me from doing all these things and also sites can add the unconscious over my sandbox I tell you what function you can run but I give you a mechanism if the 1 at a function you could do that don't do it for the OK so 1 of the things and this is the this is not uncommon and we see a couple examples of this today where somebody picks up 1 somebody decides the current big tool in their area is a cut anymore and they're so ossified they're not gonna move so it's time to rewrite and so the mother's been around for 20 plus years and 1 things is interesting about that is that team might doesn't have a lot of documentation and it's very you know it's very terse and a word here makes all the difference and then people just experiment because it's not that complicated right a small amount our see what it does and so every site does it differently and so on but lots and lots of sites use environment models the the using the model of the sun you don't log and every news site does something completely different I learned more and every day I learned more more about this simple tool just manage the environment and so 1 of things I wanted was a way to 4 sites to control the behavior pervade each 1 does it very differently and so I had this of hoax are many user so I really love hot air I've added it with the site practicing so if you have 1 you give it you said in certain binary variables it will load it for you and users can do that making Rajesh function with books so the the as a system in a straighter of a system that uses environment as we think of software as a module that comes from we wanna know where where the 3 people that usually generate those software in the models we wanna know what margin people use so I gotta look at the clock in there which says every time the model gets loaded it it said the masses syslog which I can track but a database and say a lot of what we were using this they're not using a that it yes so I settled there you can also register site dating you could put that extra messages and he hopes to receive that in because it's really nice you just set up so you know user registers of write a function registered hoped and I just call it and was not there does not so 1 of the things that I love lower tables meaning if you get it a little little tables are just absolutely beautiful because they can be erased they can be hash tables they're just wonderful and you want to on but since this is running in the shell I have to have a way to remember the state between 1 call to Almog between the next because you run the command it goes away you don't wanna write to a file because you could have 2 shows running in the same directory so or right there so it has to go in the environment so what I do is the state is maintained a lower table well tables have quotes commas and curly braces and they just don't go nicely in the shell so what I do is I Based 64 encoded break it up into a 36 characters in right you know 1 to N 2 at the 2 6 characters but in that environment in which it there and then hold back what I need to it's it's just wonderful and it makes a and the other thing I'm competing is only stores the 2 variables or maybe a couple more but to barrels mainly just maintain so I can maintain a very complicated state and I can grow the state which is quite adding things in the environment table and people just don't know what is the just a stream characters but I can add to it changes and it's just a really nice system it so I started well I started programming in the world about the time 5 OK map so I've been doing this a while on the 1st release of modern 2009 I decided we had this problem I thought I prototype but and I thought somebody else will pick it up I didn't want take over the world I that other things to do what we say about it so I figure the team I could be said this is a great solution they would pick it up in a well you know but people get involved the project they get to see are stick and then they it goes in the maintenance they just no longer have time for the got other things to do in anyway way so I thought I'd I'd write in will was like they're a scripting language it ended be a prototype and then have to rewrite in something and I was amazed that law was passed and in fact what the problems these problems we have is that it's on parallel computers it turns out that reading 1 file is very cheap and what does this very fast but walking a directory tree is slow on parallel file systems it just isn't it's complicated and or explain why but it just is and so working 2 ways to avoid that is amazing but the prototype was fast enough so at the wedding in C would make any difference because walking the director she would just still kill the and so this is sort of how I've been building a community at 1st appointed attack and since we're not in the states you guys must have never heard of it which is fine we're 1 of the largest open source community of open-source science machines so if you got money from the National Science Foundation or National Institutes of Health you might apply for time to a system where 1 of the largest and so having my tool out there that expert got exposed to lots and lots of people say hey this is the model tool and this solves problems that I'm having so I was able to expose my tool to people who got it could use to our system said hey I 1 on mine so with that of 10 thousand counts on I get I get a lot of exposure and then I 1st and was designed really solve our problem scratch the itch of our problems and then I announced it on the ELM on the environment mailing lists tangled of attraction there and then my earlier action was that a user will find little of the element and mobile but don't know that they don't know anything about lower but that was the answer to the problems and sometimes on their new best friend I get mail from them daily and then I solve the problem and they go away they have a new best friend would not meet and but the other thing is and sometimes this means stretching l to do things that I can think and that's a good thing most the time but sometimes I have to say no 1 of things I refuse to do is be put in a little bit of artificial intelligence to say do what I need not what I said because this I
don't I do that in and the other thing is I refuse to make changes to our model help your site but all help nobody else not like in could figure 4 I can handle that but it but the core change no so sometimes I have to say no so you know building trust is just hard it just takes time were all busy people nobody wants to change unless the truth less the problem the mission told the using is like toothpicks in the ice that's just not going to do it well maybe a few very few adventure types but most people are really busy and that's your solution has to be better somehow and but they also wanna know that you're going to be around to support and somebody asked me recently you know what's the funding model how long is this project going to be around I said it's open source you know I could be hit by a bus tomorrow but we've been using it for years and is mission critical for us that's the best I can say so a traitor mailing list started getting yields few you know the more people you get on it and figure out things and ask the questions and then I presented this at various conferences important supercomputing area I as she talked about she talked about documentation is everything and I've gotten better at it but it's not it's that's everything but the other thing that's been interesting is for me is that since I'm picking up something that's already been around for a long time there is a core interface to it that I have to maintain that I don't wanna break even if I got a better way to do it maybe I keep that core maintenance I'm while I'm still we'll I feel free to add extensions but the core features should work the same break heavily with reuse with the original the so language I mean blue was a great language which work in but the you use much smaller than Python and there's been some accepts that some resistance to accepting of much it uh 6 months ago the no 3 months ago was at a conference and somebody says had a 1 learning have a look at said well the choices he l and believe me it's TCL is to fix the eyes the arm and lower where as we all know is just you see it's just beautiful but somebody else that's on his mailing list where is this thing on reported a Python but what happens is that the writing in a language that not a lot of people know is I get lots of feature requests but not a lot of code the health the I'm not sure we you can a fair amount Python right but not in the core it and what and all you don't want well Lewis doesn't seem to have that people think that think they can they know that they can how do you get in the Python they don't seem to think that law and I have no idea what because I think Lewis just mostly easier except for the colon vs . issue which you and what you get use that I put all our call them IPython it and I mean that's the view I I make the notion of year of but cellular feature requests and bug reports and that's great because x all those eyeballs are just wonderful I find lots of things and sometimes my mail was the summer genius because I make it to 1 change and put that fix it pushover push out while we started again as because a little is a nice language to work in and you can be really clean and ibid coding for a while and I've learned a few things along the way but I don't get much contributed code the of the thing that's interesting about doing this for community I mean I want to get to the every day on format for the mailing lists and some user requests up things that I don't think I'm ever going to use until they do of and sometimes users ask the right questions I was amazed that I had this restriction and I'm like a boy you with the details but there's this tight restrictions on what you what kind of models what kind of things you to have a certain models to do collections and I had the sperm the restriction that you couldn't do it do with collections in ESP something I said so should have had it wrong for years and it just made that 1 change and also they opened up the possibilities to be much more flexible and so having all these users has been really great and of and sometimes you know they just asteroid questions and sometimes they can help solve technical problems like this user this user thing that came across a case of the other things that I've done to build trust trust is I've got a technical solutions and that problem just I'm running out time right OK I'm lost what I wanna talk about is I've got a test suite that will the best test suite for l mod which is just I live and die by and you know my tests are everything every time somebody listeners bug report I got it I got a test for it's 1 of my own but the other problem I got is walking I have the problem is that you're running my software on your system and I should want to so how the hell do I D so and the skip that on the logging is just really is really the most powerful things that happen to me 1 I never found a good deleted by the 4 part of for little there I think they're out there but but I figured I do this blogging I just use it so I developed a simple water with indentation curly braces and it's always available so out there with my code you can put a couple the load something and get you know a detailed information about the structure in and the indentation makes all the difference in terms of figuring out what's going on it's just it's really wonderful of the other thing is there's lots of options to ah l mod and sometimes affects the behavior and so every time and you can write down mice mice conveying reported configuration and but I actually when I do the when I do the walking added show it but the configuration gets reported that top of that so I say please send me this command what you did and target configuration and the version of the log of a version of a lot I anyway and that's just really wonderful those 2 things have made all the difference in terms of me being responsive to my mailing lists and being able to figure out what's going on remotely because that's that's the key but I'm not going beyond a system I shouldn't be so this is the bolt the logging has been really powerful and it's been interesting ride you know I got a lot of my i put up to sourceforge and get up and people been converting it the packages on brewers on Fedor it's on that and went to and element has been much more reliable thing because the available public all those eyeballs make a difference is but a lot of fun but it's also a lot of work there's no way I'm going people you have thank you and use for yes I I I just to make it in there it is you can write on this so that people and so on yeah but it's really in a the you divided even if I well you need while you need to do something you can do it we can talk about it but yes and you can all with no you can't but you can define shell functions inside a module although it seems a bit incestuous I usually do it I cited didn't do it by having we have not strategy where you would set that use model money in your directory and the started would give you you 1 or you just what would we do testing there's a document on the BioMart moderate the docks which had assess how to do transition stuff school thank you thank you and
Gewichtete Summe
Nabel <Mathematik>
Schreiben <Datenverarbeitung>
Gesetz <Physik>
Streaming <Kommunikationstechnik>
Faktor <Algebra>
Tonnelierter Raum
Suite <Programmpaket>
Befehl <Informatik>
Gruppe <Mathematik>
Gebäude <Mathematik>
Güte der Anpassung
Rechter Winkel
Computerunterstützte Übersetzung
Lesen <Datenverarbeitung>
Tabelle <Informatik>
Klasse <Mathematik>
Virtuelle Maschine
Modul <Datentyp>
Open Source
Elektronische Publikation
Binder <Informatik>
Wort <Informatik>
Faktor <Algebra>
Neuronales Netz
Prozess <Physik>
Hintertür <Informatik>
Formale Sprache
Element <Mathematik>
Metropolitan area network
Schnitt <Graphentheorie>
Parallele Schnittstelle
Funktion <Mathematik>
Lineares Funktional
Machsches Prinzip
Konfiguration <Informatik>
Strategisches Spiel
Projektive Ebene
Explosion <Stochastik>
Web Site
Zellularer Automat
Gebäude <Mathematik>
Data Mining
Speicher <Informatik>
Leistung <Physik>
Physikalisches System
Objekt <Kategorie>
Mapping <Computergraphik>
Binder <Informatik>
Bus <Informatik>


Formale Metadaten

Titel Lmod Building a Community around an Environment Modules Tool
Serientitel FOSDEM 2016
Teil 100
Anzahl der Teile 110
Autor MCLay, Robert
Lizenz CC-Namensnennung 2.0 Belgien:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
DOI 10.5446/31014
Herausgeber FOSDEM VZW
Erscheinungsjahr 2016
Sprache Englisch

Inhaltliche Metadaten

Fachgebiet Informatik

Ähnliche Filme