Salting things up in the sysadmin's world


Formal Metadata

Salting things up in the sysadmin's world
Title of Series
Part Number
Number of Parts
Santos, Juan Manuel
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date
Production Place
Bilbao, Euskadi, Spain

Content Metadata

Subject Area
Juan Manuel Santos - Salting things up in the sysadmin's world SaltStack is a thriving configuration management system written in Python that leverages YAML and Jinja2 which, by now, probably needs no introduction. This talk will cover a brief summary of why we need configuration management tools, followed by a full dive into SaltStack, its features, pros and cons, how to use it and how to extend it. By the end of this talk you will have gone from knowing little or nothing about SaltStack, to being able to deploy your own setup. This talk will be targeted to either seasoned Python developers who are taking their first steps in the system administration world, or established system administrators who secretly love Python and prefer to stay away of configuration management systems based on other languages. Its advisable that attendees have some familiarity with Python as well as with system administration concepts. Also, this presentation will be focused on GNU/Linux systems, so it is expected that attendees are comfortable with some of its concepts.
EuroPython Conference
EP 2015
EuroPython 2015
Degree (graph theory) Email Process (computing) Red Hat Computer animation Multiplication sign System administrator System programming Information Mereology Systems engineering
Computer animation State of matter Computer Operator (mathematics) Multiplication sign Configuration management Data management Physical system
Metropolitan area network Message passing Thermodynamic system Computer animation Multiplication sign Arithmetic logic unit Configuration management
Covering space Complex (psychology) Information Code Computer Multiplication sign Line (geometry) Counting Number Arithmetic mean Computer animation Kernel (computing) Complex system Sinc function Physical system
Process (computing) Computer animation Multiplication sign Shared memory Right angle Online help Configuration management
Thermodynamic system Computer animation Online help Frustration Configuration management Formal language Physical system
Metropolitan area network Computer animation Right angle Formal language
Metropolitan area network Goodness of fit Computer animation Lecture/Conference Resultant
Enterprise architecture Thermodynamic system Computer animation State of matter Hypermedia Consistency Computer file State of matter Mass Configuration management Rule of inference
Arithmetic mean Thermodynamic system Sample (statistics) Computer animation Information Lecture/Conference Drop (liquid) Local Group
Thermodynamic system Computer animation Computer file State of matter Computer file Core dump State of matter Mereology Configuration management Data management Physical system
Point (geometry) Metropolitan area network Product (category theory) Thermodynamic system Computer file State of matter Software developer File system Computer file Point (geometry) Source code State of matter Thermodynamic system Arithmetic mean Computer animation Network topology Statement (computer science) Booting Data type
Thermodynamic system Identifiability Computer animation State of matter Right angle
Thermodynamic system Computer animation Computer file Network topology State of matter Boom (sailing) State of matter Combinational logic Parameter (computer programming) Mereology
Point (geometry) Asynchronous Transfer Mode Computer animation Computer file Lecture/Conference State of matter Boom (sailing) State of matter Configuration management
Complex (psychology) Mathematics Computer animation Computer file Real number Computer file State of matter Pattern language Template (C++)
Metropolitan area network Slide rule Fluid statics Arithmetic mean Computer animation Computer file Structural load State of matter Central processing unit Right angle Configuration management Number
Electronic data interchange Computer animation Computer file State of matter Time zone State of matter Sheaf (mathematics) Curve fitting Open set
Metropolitan area network Arithmetic mean Thermodynamic system Computer animation Computer file State of matter Multitier architecture Compiler Modul <Datentyp> Matching (graph theory) Physical system
Matching (graph theory) Computer animation State of matter Function (mathematics) Mathematics Computer file State of matter Software testing Statistics Mereology Local Group
Group action Multiplication sign Range (statistics) Combinational logic Electronic mailing list Directory service IP address Rule of inference Local Group Medical imaging Computer animation Software testing Resultant
Computer animation Software testing Local Group
Domain name Word Matching (graph theory) Dataflow Computer animation Single-precision floating-point format Local Group Domain name
Revision control Default (computer science) Computer animation Information Observational study Code Operating system Bit Information Mereology Weight
Process (computing) Thermodynamic system State of matter Mathematical singularity Expression Directory service Binary file Wave packet Number Wave packet Computer animation Central processing unit Scripting language
Thermodynamic system Computer file Multiplication sign Range (statistics) Thermodynamic system Arithmetic mean Computer animation Personal digital assistant String (computer science) Information security Subtraction Annihilator (ring theory) Booting
Arithmetic mean Computer animation Computer file State of matter Right angle Variable (mathematics)
Frequency Asynchronous Transfer Mode Computer animation Information State of matter Shape (magazine) Mereology Wave packet
Building State of matter Length Code INTEGRAL Multiplication sign Decision theory Direction (geometry) 1 (number) Client (computing) Function (mathematics) Mereology Disk read-and-write head Fault-tolerant system Information technology consulting Formal language Heegaard splitting Roundness (object) Video game Encryption File system Information security Vulnerability (computing) Exception handling Metropolitan area network Thermodynamic system Theory of relativity Product (category theory) Structural load Europeana Software developer Gradient Basis (linear algebra) Bit Staff (military) Instance (computer science) Variable (mathematics) Functional (mathematics) Connected space Category of being Arithmetic mean Message passing Software repository Data storage device Telecommunication Order (biology) Phase transition Duality (mathematics) Right angle Quicksort Data management Resultant Point (geometry) Metre Read-only memory Server (computing) Service (economics) Presentation of a group Computer file Civil engineering Auto mechanic Control flow Mass Declarative programming Template (C++) Goodness of fit Centralizer and normalizer Causality Lecture/Conference Natural number Subtraction Metropolitan area network Condition number Form (programming) Key (cryptography) Projective plane Model theory Content (media) Line (geometry) Density of states Configuration management Local Group Computer animation Password Boom (sailing) Vertex (graph theory) Dependent and independent variables Local ring Normal force
morning I think for being here this is my 1st time so please be gentle with me or if you have a
few I wouldn't enabled souls but before we dive into it ideas all of your little process the literacy show OK and system administrators we have all that's really nice was written as I OK you're both going to enjoy this so this is the boring parts I must've is really quickly when Thomas of the 1st lady said but you can find into the eye and the let's see for around you up to do you know if you use IRC I'm go like on freenode the common information system engineer and just recently I got my rehab certify degree of I to work abroad Hat starting leader and senior technical support engineer and this is something true I need to survive so if you wanna buy me a beer I'll be really grateful for
OK so the 1st question is this do you know what configuration management is and when have OK it so the 1st question would be why do we need configuration management do we need operation management and of what he's configuration management configuration management is the way that you you have to ensure that the state of your systems are consistent and not change so this may not sound like 3 glamorous or anything but you computers were like this a long
time ago right or something like that and administering these computers was really easy because you'll only just have 1 but now
the more like this and the more time goes
by there was since you have so the 1st
reason of why do we need configuration management these quantities the mosses and you have the hotter it gets to make them behave and system messages here know this for a fact the needs to be in the public public thanks so
I never remember the a really you
don't this is a long time ago computers were like this you had information stored pieces of probably the just 0 someone was really simple actually the new binary by let's go back to the future this anybody know what this number mean yeah this is how many lines of code roughly there are in this country . so we've come a long way right no more cover so that's a 2nd reason we have quantity and how complexity a more complex system is also 1 is hard to maintain and the 3rd and final reasons the most is the most companies truth of all if you're not convinced by now we want telling you I know this piece of wisdom for a fact since systems and I'm going to
share with you today the right did you don't drumroll we are lazy people
who so that's the most important
and Attilio briefly how it all
began for me would configuration management the some time ago a friend
came to me and he said I need help
at the same time he was offering me a small job but I so he
needed help with non configuration management system that
is it not being must know maybe this the most known today and you figuring out what followed was
a month-long frustration
try and the desire to
inflict harm upon myself and others so what the hell froze over
I got to thinking why was it so hard I mean it shouldn't be Saccharomyces 79 novel how
systems work and I realize that the tool will reasoning was not made in the language we all know
and love but it was actually made
in an alien language alien hostile and yet you mail out but I also think that the features of a language trickle down to the tools made with the language so it was a solution right
I mean what could we use that the not getting away there has to be
something simple right something that we can easily understand star usages right away and get to
work with a is there to like this courses I want to meet sold
not be solved this also results in almost all that and it's given
basic southern by from which is
already at I'll good thing arose I wouldn't be here but not only that
sold also uses both and ginger I mean do we have downloaded the epicenter of generative oversee Johansen OK this is some business already
familiar to you then because these 2 things are also used in general I well it's morning In the 1st step for
you to understand household works is so the terminology used soul sold works in Ramasamy architecture and the names of upon intended so as we said before configuration management is all about keeping the state of your system's consistent soul In sold everything is on in this these state files you define
what assistance from like then you have a high state I promise we'll get back to that later but it sounds really cool so I put inside I would have brains and we're also going to cover that we have matching in which is a way of storing your posts 1 more stuff by the way 1st thing I said was we have a massive media architecture so the masses basically 1 rules them all and the while the lead to serve in actually I I wish to put a small picture of the recent movie but I think because of things called copyright it cannot do so we
then have to make do with this I mean is identified by an idea they may or may not be a drop a note groups which we're going to look at that later won't work to matching
and they provide means of information so my environment that I'm using for this talk is based gentle Linux so some of the things you see may not apply to to you everyday run of the mill that we have dented is master and everybody is convenient because just workings so the 1st and
most important part of our states
and hides as I said before the states
is this they were in a system should be and this is the core of configuration management is something you're going to see regardless of the system that you're using where be propagated uncivil to add insult to you have states maybe with another name you know the population management systems unnecessary for everything sold file so they reside in the system they're called SLS files or expose state 5 in being in if you when you need to define neural uracil 5 years they files you just put something like this in a configuration file for the master
it's really really simple you just pointed to about where audio source states are going to be and you identify would with this trying also environment so you can tear down between development to a production and so what we must look the 1st is the top 5 this flight is a special state fire which is the entry point for for the environment meaning here we will define which hosts will match the environment so we're gonna know thanks to this file if I developed their developed in post should be in the environment so a and this is more or less the syntax of the top we have the environment and we have a much statement we must tell you which much that we're gonna use I know it sounds a bit confusing but would be by by
the later and then you specify the state trees and this is
also quite why looks like so this is pure yellow right the we
have we have the base environment and this is just an identifier I have another defined which example that later and we said OK you have too much the node group under apply all the states and this is just another identifier too much and another group and play so you can really grow these into more complex things maybe it's
not a subject for this but we'll see so states is where the magic
happens as said before and this is the part where you get to be creative with the environment you can write about anything you want that makes sense the you have a state trees so that the combination of states and the syntax is similar to the top dodecyl as file so this is something like this you have identified for the state the type under from where did this now so far
this is a possible state that you may have it's really really simple it
just ensures that the team expected it's installed we also deployed configuration file 14 months there will be no point in installing 2 months wouldn't have a configuration file by so this is simple and to the point think all of you are able to understand that soldiers like this the sentence really really simple and that's the magic of it but we can do more complex things so instead
of applying a verboten files sometimes we into the deploy template for example when you're dealing with Apache and so you can do more complex
things as I was saying well that's see here I just said to Saul that this despite is a change of template and I am able to
properly no replace the values in it we got from salt so in the next slide we're going to look at it and this 1 right here
our brains like I mentioned before insults they tell me for each menu and the number of CPU they have so I don't need to write about static config file for each meaning depending on the number of CPU I just write it someplace and there we are OK we we know this is another
example of how a State could look like this is a little more complex but what basically doing here is putting a small section of file that is gonna be banished by and no matter what happens to the to that section the file sold would replace it if he has changed so that's useful for example in and you holds file where you need to have a certain section of a predefined goes we are changing the whole file this lets you do that and even lets you properly marked up this section no starts and ends well we 9 mention high
state before you gotta like the name
right high status the collection of all states that must be applied to a particular system for example if we wish to apply all relevant States to all relevant systems in our petitions whole three-tier environment we'll just do this and these will apply
all states that apply to any system that is an interesting registered to the salt must so that there the ways more something like this but the meaning used when I call the they would all of the topics less file they will attempt to match against the file see where they go and then if a match is found it was done on all the modules or solid compile and execute them so by the end of it will have a happy system so that assuming
we only have the previous team that's fine we define and if we call a state were going to something that is sold going to have to leave the states that applied and it's going to say whether or not it it do it and when the start of the duration so far the next important
parts for you to understand are matching and non groups and matching is
the way the so must that has to refer to as many of they can be matched by idea which is basically a host name of the image by no rule which we're going to cover of its subnet IP address range or you can even do a compound action which is a combination of any of the previous 2 types of much you have so we can match the
single-host we can match a list of them from but much and
actually doesn't make any sense at all this is just a 1 time thing if you're matching more than 1 host we should really use know no groups are defined inside a special directory that come results which is
uh UTC so most the but and it has a similar thing that's what we're used to and now if we define the you know the previous
1 was like this so if we
define group they that 1 now we can match fighting over of instead of putting all of those names or actually this is just a special sold command but used to see the sort of from which more often than not that obviously not that's how
oral OK and we can have as many no reasons as we like and this last 1 here is an overview much using grains which I know it's coming the patient our going to cover that a and
if there's a word to the final groups I think you will be flexibility so as a said before we can match were 1 would then whole domains single host grain based matching and we can even into your groups into another another group definition sold there's actually no excuse you can apply this OK
present grains are bits and pieces of information about the meaning the how they running
the operating system the Python version and stuff like that they are generated and loaded on the you start off so there kind study information but still and and then you can list them and I you can get the values and weights there already 50 50 blast raise the ship by default what but here's the beautiful part is that you can write your own so writing your own grains basically right in Python code you just have to buy and return date that they go in a
special directory which you're seeing
the underscore grains inside the of the Maastricht Treaty of your state but this is something I just because 1 of the reasons although I needed to know the number of course a system had an right and and they're using arithmetical uh expressions in the matching process would not tried it wasn't free developed it is going along and and so asking the them of course was more than 1 wasn't feasible or moral and anything so I just made a small Python script there's a grain and it seemed returns true if the system has more than 1 CPU so after you you to replace the training data in the directory so it gets seem to dominions and you can query
it you can ask for a constant rain on it will tell you whether or not your system in this case as some to cost so that's it for range from them uh um we also have Peter and there is
also that but it's not that gets generated on its own it's 1 defined within it's storing the master and in this sense security dominions when needed so again seems everything insulted the file pillars also stormed of our system and you have a similar definition to tell sold where European files will be and I did it might be a little confusing the difference between strings and I know I get confused many many times why so during my time would sold by came up with a single room had to be all by myself grains of data from many that generated Armenians their retrieved by the master and they still things about the meaning of I'm Peter is that it gets sent to minions it's arbitrary data we're gonna have a pool of examples right now out and but that's important definitions OK so we're going to query it's your for a faithful and posts and we
have not right because we're supposed to be generating pillar that so this right of something
understand boring but please bear
with me for a 2nd yeah Peter has a top 5 to so I this is an
example of what the popular will look like it's just names variables right so where basically replacing the obvious definition of a solid state where we had the package name and the place where the file was stored and after we do so we can query the printer and we consider this already defined meaning so what does
this allow us to do with period that you can do facet of flight training no new states so T must state from previously it can look fairly you
have all the relevant information in 1 place and you can't just template your state In this way but
not also those from Latino we're avoiding information repetition because now the the important parts already in 1 place and actually need to change shape which is go to that place which is a pillar but this
is the most important part actually that the condition of the solar project is really really good uh it's basically what I used to make this presentation about from my small much sold on the euro is that when you have over there and I for K I really encourage you to do try its Python it's really easy and the ice an you should be able to learn a pretty
quickly should you need to good that you fewer no we have time for some questions and questions so this is something I have to part so the 1st part is I I kind of came from the public world to on the once you kind of know of no classifier kind of thing which is sort of a con concept from public sister pushing stuff to the grains like on that this is the the masters for example so I have no like in my template say to all of the things mentioned ranged in as much to do these things but then I realize there's a race condition for someone who could change the grains file on the end point between their could promote something to be a role that it should be between rounds because 1st it anyway so the next OK now I do with pillar make sense but it looks like the recall the most groups that you just talked about is actually present in the groups yeah but actually you cannot use the for a note group much unity press I I am currently using polarized light reduced classifier and it works but I think I'm doing it wrong so that you have I always like to find a way of doing on the 2nd question is I guess that was a common that man's splitting it to happen from when a node's configuration state changes as far as I can tell the only place that gets locked is on the menu and I haven't been able to figure out how to have on because it's old master when you run I state you get back this report on standard out that tell you what changed but I haven't seen any way in the solar masses to make that get lost on the master near you crank up the dividing up the master the products to see it changes and what the the bond issue really show you it's gonna be a lot of output but if the button doesn't show you you can run the mastering foreign and all this stuff is all by the 1st step would be to crank up the back in the log so something on the like it would be nice for so was actually for a man uh just recently including the latest release support for salt integration yeah I know I know for to and that's the thing that I really really wants to see insult still form is written in a language which I don't like but anyway and so it's coming is coming along and 1 thing I failed to mention is that soldiers and had a development but things are getting way better by the minute the documentation say is good but it sometimes it's hard to find the proper idiom to do with saying yeah civil yeah yeah yeah I understand that must Romanian grains that and practice to the talk was really to see for sold for the 1st time for me but of course if you will begin questions but the communication between must minions is unencrypted yet and and the the you substitute that the grains come from the and then the data is being sent to it yet this data on the menu is encrypted as well as the like them I will let me rephrase that if I have to say and I AWS keys or something else and in some ways in my configuration how would you do that OK well from what I understand prediabetic accent to dominions on state or high from and gets interpreted them so I actually don't know if it gets even store on the menu so it just gets and uh would really would totally you get up afterward so it's interesting which you mention but from what I understand doesn't get sort Hi um in my company would be using decision to fall off you months now and just after moving to be city to we looked into salt of course because it was a good activity I'm going to me that all of this to has the simplest relation reject souls maybe a bit simpler to implement the consultant was very variables infected now my question is what happens if on the menu in the service that is cleaning phase how do you know that the host is not responses you have somehow on the master some monitoring of the elided makes all the minions of the you have to do this manually um OK and whenever you need to interact with the meaning and the means is not responding the master has a time out and it will tell you that it couldn't contact dominion about from my experience there's no active connection I mean of course the means listening to us 0 and you will to that that's where the orange from the message gets sent by and that it's not uh activities all 4 of question you said that the connection to encrypted before that's Seattle but the wind you does not have a secure optical and as as OK I'm not as much insincere and you sorry I but but from well where you from under the condition of salt is that it is the security of secure direction so actually I don't know how the data was language the only reason why he decided to choose some households take over a puppet or after other these South thought over property that idea that the language was the main reason and the other reason was that I read about it so much the specific reason why you need public so specific reason why you hate public white do we need proper there right you hate it up and uh I never really understood it I mean there that the line is and along with ruby syntax to be honest this just curious target come back so yeah apples being the use and appurtenances both and the way I see kind of thing it seems very similar to the to hiring so is there any way in pilots to increase data because normally that kind of centralized configuration is getting Inogate repo united by their has any kind of of that functionality to include that and you can use that if you if if you mean can you place Europeana that our somewhere else other than the normal force system years candidate for example so now that that that the content that you that you put them in the united finer has any kind of them of encryption like if I wanted to store secrets in pilot black passwords toward you know ssh keys whatever is is there any kind of functionality women pilot that you can increase that so you can have that immediate repo that this public understand um OK note by itself by their just files so they have to be accessible was sort is running and needs to retrieve order and that's all the stuff happen encryption mechanisms for the flies that you have stored but you can increase the file system and you know if it ever gets eye-tracker anything but ultimately if they do gain access to the server I mean they're pretty much in the the the I seriously thing I was it is most questions she does really good really matters to the point about on interesting pillar data I think it's important to think about what the threat model is if I had sensitive data encrypted I still wouldn't want put on public get help because maybe somebody's gonna be tried before so maybe there's some vulnerability and encryption mechanism on May the 1 certain gets fired but the stuff is still out there and he has something in his head from and so let's say you do have some magic way to to the polar files you still have the decrypted on the master which means that the decryption key lives in memory which means if you're a threat model involves an attacker pounding your master server then he has access to retrieve that clear cleartext from memory so just don't get too paranoid think about some of the things so this package act uh so it looks like so this whole who is similar to what that like all the concepts of basically the same so I wonder what's up with that the difference like feature-wise and 2nd part of the Commission due to differences community was flaky this initially forage causal quality load of working models that do you have 2 sheets that it's the stuff I'm feature-wise so I I I didn't have a chance to be that much into public uh I believe it has a bit more features it's also but the basis for configuration management there for salt I really wanna check out the integration with formant with they and I really want to integration without putting for men and leaves because most of the problematic actually of performance and the community Weiss unfortunately not many people use also this also kind of like the aim of my talk to you people use also we have appeared committed how old is it sorry how this is because when I looked at 3 years ago I did fighting of the 2 presidents of 50 but it's yeah it's uh around that time or less about it's quite quite new thanks what and what we stand of the hi and like wanted to know how do you only manage dependencies we sound for example if you can make layers of states to to implement dependencies and the and how you do you manage the removal all clients for example OK you have to change a configuration OK you going to lock something new 1 and then is a user-defined that if you have to Coolidge the staff OK I well due to the the length of time we have and because I had to cover all all from the right up to what you saw our I to cover is that you can use includes in your state files to indicate dependencies between each other so you can include states and make depend on 1 another I believe that that answers the 1st question about the 2nd 1 I never actually had to remove files and the what I did do was removed the pieces of text from the files which was in a state over there you can use for a temporary I believe that was the the answer to question and the nature of yeah idea other it was yeah you can still by fires and you can require on the other the conditions for example use something that arrays and you need a certain service you can of course you have to have this service in another state file but you can require yeah but the important thing to understand is that since we're in a declarative world order inside the state file doesn't matter so you have to define your you're requirements inside the states and because it if you trust them things to be executed sequentially maybe they're gonna be out of order and something that I have 1 more question actually 2 questions that relate to their what happens it's a gradients raise an exception and handled exceptions what the sorry what happens if you grain raises an exception arrange them in the chief when you look at something it just doesn't work in something breaks because somehow the confusion local configuration of the meaning is not compatible and thing just blows up like that where we are using a custom brain that you wrote on something and how well the brain is not going to get indexed condominium using we won't have value so the question is not the grain is interpreted when the master when you issue the request always a catch when you created and it's 1 of the duality and the there is a discussion somewhere on the on the menu when you create a grains of up the grace of when the ministers but you can also forces to so it it's stored but when the code actually executed up on many of those 2 situations but it doesn't get executing normally during the minions lifespan of just gets executed or start up when the Dominion starts of meters with mustering gets the grains from the master then gets executed locally and should you choose to sing the grains manually they're they get executed at that point and now the 2nd question which is is how do you manage building let's say I want that's when my masters command is run on the meanings the result if there is an or something like this is sent to let's say some instances like century and is there an API to compute this yes there is an API it's find more about this and I really do hope to be able to come back next year you part of the presentation which stands for and if you do have more questions you can look me up in the conference of the year feel free to come to me and will talk about maybe I can show you my my life


  886 ms - page object


AV-Portal 3.11.0 (be3ed8ed057d0e90118571ff94e9ca84ad5a2265)