Integrating all Your Tools With Chef - And How we Did it at HPE

that's not going to write of this set my bag I'm here with Hewlett-Packard enterprise and I'm on the infrastructure automation team at age feet long with vector also the same team and Tiago on decomposable infrastructure to so today we're gonna be talking about integrating our I construct infrastructure and your IT infrastructure with set so acquitted a quick agenda but so we will start off with and I integration with within the technologies which includes server configuration storage configuration complete infrastructure provisioning and then we move into a few supporters chef compliance as well a little information about so integrating tools with chef we were all year avid users of chef and we've all seen the types of thing the chef tries to us there's a lot of benefits a lot of reasons why you want you chef in your environment and now receive the reasons why we wanna combined on your custom tools which self as well the so the power of automation by integrating your tools which shaft you get provide with the unified framework to interact with everything and to use these different tools so the benefits to customers see as it provides on a lab acceleration GIT provisioning and makes your tools easier use as well so we we at HP decided to take a few of our different tools and bring them with some find them with shaft to make them more powerful to make them easy for customers so in order to do this we have 1st step on process to getting to these different migrations we start stop at 1st identified identifying are different automation needs on where customers may see fit that staff would provide them acceleration inside a data center would provide them with mother greater benefits and then we determine which components of shaft by provided or use case so for example you could have resources a provisioning driver cookbook or even a less support for years custom operating system on brought into chef so we'll take a deeper look in these and then following that reconstructed helper libraries so it she has quite a few different technologies and that the plan going forward is to bring them together with different open-source tools to make them more powerful so it we decided that the best possible to do this I would be able to build libraries of can leverage in of multiple places to make them more more reusable and version of all for multiple sutra laws concluding shaft in outside and then finally we went ahead and created those components so that as we just went over and we creating easy-to-use tools and all these automation tools work seamlessly other through chat and that provide rapid configuration of your infrastructure the 1st 2 we have deals with server configuration so I'm specifically this includes of ht proprietary proprietary technology called the ILO's is the integration lights out which is a server configuration tool of we have at our company so a little bit about that on its is use in a real lights out is used for efficiency simplicity and up time when it comes to managing your server infrastructure for the built in intelligence and automation for increased server admin productivity this is a very popular on tool that our customers views and we really want enable them to be able to use this tool at scale given the power of shaft and the power of being able to automate this within your data environment to turn to turn even something as popular as me I was around for quite a while to turn this context this kind of complicated tool into code as well so we structured this into 2 different projects both based off of an industry-standard redfish shakier HE she's is the moving toward building a standard API for lotta products to make it a lot easier for these tools to be used with our customers and then on top of this API we build our on I'll STK Ruby project which is the helper libraries which gives us all of the API calls extracted out and easy to use so for the resource providers and easier integrate into a programming language and then we also built and I will research provided so moving all we take a look at it as an example on STK configuration of your I loaded into the light cycle so this is written in pure Ruby and use the gem which is called the ILO STK to configure all of Europe of resources but if the configuration for your innovator lights that so it's steps the redemption Ruby on creates 2 different lines for our 1 I want to and then it does allow fewer in on both of these 2 and build a user call test with a profit of 1 to 3 on it head I changed the password on the user sets the time set NTP and then serve reset both the servers and does this all through pure Ruby code nothing a chat just pure Ruby base off of the jam and a key which calls yet so the benefits of this a can you leverage by any technology that uses Ruby to build integration so currently we have built up a chef but this thing 11 but pop it any sort of other tool that uses the ruby library to 1 it's commands and hide all the API calls by creating released human readable resources and uses far less lines of code the manually making API calls to other classes here at the bottom you can see example of of how this works you wanna Ruby execute the executable on your workstation which is built built using the was the theory which then makes the REST API cost you right so going beyond this is 1 get the really interesting part so using this Alice SEK the we got this gem the we've created we were able to put this into a shaft resources and create a whole suite of different Scheffer resources to configure your arm i will server management consul so this code right here does the exact same thing that we just saw in that Ruby file everything everything we decide sets the user's changes NTP on that's the timezone recesses server does all that stuff but in the context of Shek this time so this is written in DSL it abstracts out all those I'll as the Cajun methods and you can basically want it on his many I was as you see fit we haven't solicitude of ILO same 1 we saw earlier and they're being 1 on each 1 these resources 1 interesting thing to notice the on note so it it does it does it through on an API call so you would run how we do all these things we don't have anything running just natively we have a call the API out from a work station and then it also goes ahead and gathers the information place that I can assess server either using data bags or using of pure files at a store this different from each other so that there is no native chef-client being 1 on these servers which is which we see as it is advantage because you can easily run on any person workstation Jefferson you run multiple times so we just saw without breaking any configuration so we designed on design the chef resources the item potent a key concept in shaft so that we run multiple times and the only bring you back to 1 end state rather than diverging into different directions so it hides all the API calls using our city as well and it executes the configuration has on multiple machines with no additional lines of code and by this we mean as you can see we have a list of ILO's provide your resources so both of both of these 2 was I will be executed against for each 1 of these resources it is provided by a simple ruby-red and below you can see an example of how does go through user chef-client and then there's a question using the API calls and make separate API calls to each 1 D-xylose but also ensures that let's say we want to add a username test it'll insure that the test user does not already exist and if it does exist toward you anything you'll see 0 out of 1 resource is updated but it did on if it doesn't exist or go ahead and create that test you and now handed over to the the top but server configuration thank you the thanks for it all right OK so as to make

spoke about idle integration would shift of work so basically we started this project because a lot of our customers they have sometimes more than 100 or more than 10 thousand more than more than that of number of layers and that on and then they have to manage their ILO's it becomes really very difficult and then they have to do a mask configuration change it as something which is very man and the tedious so we decided that real creates this automation and just make an open source so that most of our customers who are using was taken just leverage our of automation and start doing it in the 1 and as the received lot of customers are already using check so for them it becomes very easy to automate the other management so based on all know going after that what he also realize that a when a fire of G stories for about 2 to 3 by researchers a number 2 of hot-selling stories in the market right now in the military space was so we thought we create similar automation for but as well so quickly they control those are some of the key features of the virus so that people are basically this so autonomic that it be the understands the the Iowa requirements of a particular application based on that it has the intelligence to take care of the workloads hollow it has to be managed how eventually but it has to I give the eye looks to a particular application or to would it was so and also it is very efficient because of it has a lot of new features and the in the storage of has been provisioning and has a line the duplication new rich stories performance becomes really and then it provides 99 . 9 per cent data so most most of the customers are using it they really see a lot of value and of the posterior because it's highly remove it in if 1 this goes down to nothing really happens because it's fully and highly reliable storage and last but not least the Federated of feature is very very important feature because of and they have of sort of this they have studied this and they have from a flash this as well and 1 so when when they have their will to nodes which are not really required flash as storages they'd automatically moves through the the nor on this and the snow this devices so that happens automatically inside 3 but mistakes scare off your of efficiency and storage alright so this is a typical by management life cycle so if a customer buys the bus storage they have to miss you a lot of initial configuration where they have to set of CPGs that I'm not sure if you the rear of the bus services would be most of the configuration thoughts which are performed by the story than in any organization that they have to bust which OK so so right now although the integration that we have done chair of you can actually do the initial configuration ongoing management and the decommissioning management of the the stories the so I quickly go through the use cases that we have automated OK so this is this is typically the first-order that men who will have to perform a task manually and the 1 and then at the beginning takes a closer seminars 410 task on 3 the past but if you use automation that feature we have created then it may take 2 to 3 minutes to complete the same 100 task on on stories at its value and it can do the computations in a faster 1 I yeah so if 1 1 person has to perform those 100 task on 10 different stories so that it's not a fact or questions us sure on OK so the there were some of the sample of configuration dust which you can perform the using and automation so as you can see from the 1st example this is for creating a number of holes onto of of portability by so it's a you have but you have a 3 by a known 1 and and you have this this is very new neuron 1 and so you want to add all your rules which would require months from you storage then in that case you have to add those holes so as you can see it's very simple very English uh readable language would you just have to use the body this source and you just have to the by the host names that you want to add and you can also specify not of the active use like the burden opium numbers and what type of course the the the the Solaris HP-UX AIX on the next and then from this from 2nd example you can see that if you want to modify anybody guru of property for a whole is you can just use this 5 lines of code to perform the task and if you want a perform on my stability but that's the way of thinking about going to by the neuron and you can use that and you just have to use this lifeline flat of course so similarly there's this is another example where you want to create a new the so customer and rendered by new the bus is they have to create interviews and is it a lot of data to have a just 1 abuse performs all the task for them because whatever the standard-configuration they normally do manually they can actually automated and just put in 1 in the recipe and it does that for you alright so this is this is on the very similar lines on what you have seen and don't so we had a similar project we followed the same approach the and the committed to separate things in it so 1 is the but look for example that you have just seen and we have antibodies begin which basically have the entire library and that Apple around ABS which has to be God so not only the story means they don't really know how to use the ideas because they're from system and background but it is really difficult for them to use their API because there are no other programming languages so just to make it simple for that we have written this whole of the but is the gate so when they call the sources from the buckle book they don't really have to understand peers because the the thing happened in the background and then the as all the new parts of starting from 74 400 they all come with of that service ABS so when you interviewed a shift land it Doc student the recipient of all on the bus and perform the task alright and handed over to the end his our explode on 1 view and Seth thank you everyone I know that the the name of the use of the the of so his question is but that of we have a 3 by management tool that as the other 3 by CLA so I really need this automation so thing is if you want to perform a task on on us single-sorted on Monday stories you either have to connect to a management console and you have to go to each the bar and perform the same task or if you want to UCLA for that on so you have to connect to the sea and of its the bar and then run that would recommend but using this you know when you even have to do that in this case you just put everything in sight 1 recipes and you executed it talks to all the the buyers and perform the task for you so you don't have to manually connect to any of the buyers to perform any of the task this you he yes so we talk about it but actually because I show you the sources but it is coming soon we we already have a tree of you just doing fine tuning at and will make it open source resource and will also have a blog and you want there all right handed over to the other something of if talking about a complete infrastructure provisioning we the I assume that we are going from the bare metal their center to a fully operational 1 right so here we made some tough integrations the 81 view that's some of you may not be so

familiar it's so I can we could say that the 81 yields softer is up infrastructure alternation engine and also 1st 4 minutes OK so basically it's 1 view have has softer define intelligence that makes your DS interactive it's more simple and more effects OK so with fit you can deploy you infrastructure faster as said because it has a been plate bases the technology base at the that can configure your computer you're starting fabric as well that's based on hurting plates OK and also eat can simplify some lifecycle operations off your data center because we the the 1 you you can you have some Liz monitoring have online femur dates and with this you see the fight all the the these operations you were doing at the center are right and at last he provides an 8 right to even and a data-driven a interface sold basically did dates can make integration DD once because with this you can put connective H B 1 view to we you're tools and internal software and services are right so know that you know all you about state B 1 view of do the work that we are really doing doing east to create more readable interfaces and similar as the ILO we have that will best k interface for 81 you are right so it's basically the same attributes that we had on the Isle of k and with this we can integrate wheat and other softer as we we tried lots 8 their staff boots or any other rubrication you will purity 1 used and also we can't quite simple simple infrastructures that because of their SEPI the hidden With desiccate and also we can use follows line of cold to make is integration possible so here's an diagram to example there I can use it so basically you creature we deprecation that he uses that would investigate and and it to those of the called the risk also the 1 who applies alright right so here's an example that in using the desiccate you can define a date 1 the appliance if you're using sold it in the 1st block of cold you defining what's what's your 1 of you that wants to use the find some settings please don't put the user and password this way in which just lexical alright uh and this goal basically and get some great some networks for your 1 view appliance and after that it connects to the veto connects authority the heart OK so the the this can be done using the HP when you but in date in 1 view interface but it's down here use need you and you will get the k interface are right so let's get to the point the check 1 view integration so as diesel we have sold 3 layer is the layer off the recipe I ATI off 1 view their daily offbeat will be as the key and the last 1 week created this implementation of the chest integration so basically as you can see here and they just this 1 of your book a book integration use that 1 the US decay and you can create your cookbook that depends on this this cookbook quick recreating to make your false to that 1 thing so worried we did this because we suggest that we can make some the 2 more high-level the wire level integrate more been plates so with this we get even more human readable cold and also you can't accuse decimal faster you can well you know about or about the test alright so also we keep this item potent but the property and that that is we have some example here off it's far more simple than the day last long so here you can see that we can't that seem the day when we define our clients to connect with state 1 of your API and birds after that we have here in this use case we just need we was tool which these enclosure groups that we'll we'll that a need for example to support of a seasonal won't and here we find that these these things will revoke we're getting these did the connecting plates that supports handles and and we we know less about of all say that then they included that define their we want to be in this include group so when you run this with this I want what the COI site you can do this all ultimately end with 1 line of code alright so it's as seats really think weights David that like the the 1 view but it's more high-level templates the so after that we could figure it all offer infrastructure know that their center and now what we want to get our servers that are inside that encoders and configure the sources and then so basically we also have this chest driver with 1 view that after we defined song plates in 1 view of what what it once often not software the we get the same plates and then put some other configurations like you know settings off the securities that's involved some connections and we do this and after that we we get detecting cliques that went to provisions are Sara and when you run it's all done so this week an overview of what I said you goal good because when you go to 0 about there what they want to and after that you define opting plates better defined at the plates you define yourself to respect and after define yourself or you can also define the additional infrastructure you may need and better you define all off this using the check the check you run the test clients and everything is done so basically you can starts from the and 8 to 1 or sorry from the agency uh they center for very mad the center and and what we the the all the server's running and function the so here we have a we could you know off the shaft divided the the

so basically here we are a radical figured

infrastructure and we use it something plates to we are conducting plates you can see chaff web 0 1 check whether to check that the 3 In this distinct weights we're blind then on some some some servers and here you can

see that he deserves our part on and after that we will run some Jeff Jeff Jeff recipes the

in chest France this could figure out the source really

need inside the server and after that you can see that this service are on

on on the web of think the web

interface alright so right

now the that will talk about the HP-UX compliance kind the you would there's sire could the so the point and the all the integration just work the integration this works if 1 of your uh but the because it's crap inundate the feet the hard-working so the a lot that what question it's all right so this so the this can be portable birds but as we've done that we are using just did the 1 view 1 but it can be done if and other tools to so that they can they can definitely move on a different types of on provisioning things because when it come damage revision in quite a few different servers and it really doesn't on not and doesn't matter which driver using but you read and you get this so that the same velocity no matter which VM where whether 1 view it appended Haitian abstract that data so I would definitely say use these different things with VMware other tool on we specifically did 1 view because it has did have on any sort of chef integrations prior didn't have a really quick way to spend at the servers automatically because the VMware where has the knife PM where there's different schools in those spaces Israel's or other tools but this was 1 that was missing the week that would be very valuable yeah and this is the place yes or no this the best way to do this and this and this yet and it yes yes that so it's remind you of can provision bimodal machines in the invariant set so you can use several gratitude for 1 view and it can talk to the 1 you in the eyes and provision of the image machine exactly there's nothing on to the right so you can do that a new versions yeah alright so we can move to a few acts so how many of you know Lotus here all right so the the level and that's great OK so I used to be a system admin on HP-UX quite a quite a number of years of of will and 1 of the key things which I saw with experience was that most of the financial customers and the telecom customers are using it for your in and even now I still see a lot of customers about using it to be ways and we we had 1 of the biggest banks in India who wanted to use share of but they did not see any support for ship compliance the so the cut as you know compliant as 1 of the key important things for financial customers because all the financial data is at stake if they are not this system that are compliant so what we did was we tried to create so very quickly don't some of the things on HP-UX I'm sure you might be knowing about it so if you x are nowadays it comes on duty before the of the hand here's systems that use to have HP-UX and we have new words and also support on a clue of the also have a similar of hypervisor technology in its HP-UX using which you can have which in machine learning on and I'm sure of people have used this theory than they might be knowing the buyers are a reviewed by resolution of the very long ago as well and then it has high efficiency right now all of the other extending support with other security solutions uh and some of them are it's reproach and some of them are open source products and shifted 1 of those and in future lead on HP-UX only 1 of is furious running on the new generation of hiding in processes and as you know as fierce was 1 of the stable operating system so it has reduced and don't blame and it we will try to do more iterations to have listed on thanks so could you want to the next label because this is something that you already know and you know this is what will be there in the future so I don't from the compliance perspective on what we have done so chef compliance generally comes with 2 of gents so your entire urge compliance Better off you have used it so far but if it does driven by the students and as you can see from the from the 1st day on which a stranger is for making a sister nation to the point the operating system so what we did was we added the support onto the train genome so that it can connect to later you identify which operating system actors that whether it is 11 . 11 11 according to your 11 . 3 1 and then the identifier that uh part of the operating system and then give it to inspect not inspect basically has all the resources written in that so if you want to if you want to find node anybody good thing or a particle apparently the only 1 who checked particularly user onto HP-UX you can uh provide the sources inside inspect so that is what we have already done that their resources which are already available and currently a shift compliance is shipped with a CAS is Purex profiles at you might have seen in the keynote debate that we had spirit support but this was done by R. T. so this is an open source anybody can use it and uh if you have any issues please report on to inspect and our contributors will definitely try to fix that alright so and handed over to bake you talk about all these resources and these open source code it's really you can find those and to give you the links and thank you but based on that so all these different resources can by found in all the same place so they're all open source of all available right now but with except the 3 bar which we available very shortly for those if you're asking on all under the Hewlett-Packard namespace of you have to read from your laptop and check it out I'm it's up there are you had the gems as well as the actual chef resources and and for more in information about our team in the composable program go you could hit H . com slash in both so suppose the program so a little bit about our team on we provide a lot the services Towards understanding these tools and helping you Weber's these tools in your environment as well we operate in 4 different spaces was accrued includes infrastructure nation continues to live integration delivery on infrastructure monetary compliance and workload optimization so thank you I will hear booth 101 right to further door on was very question before we go ahead and you the raffle for the suites on barbecues OK yes so the question was if we integrate this in the shaft do we leave the other tools in their ever were they obsolete on the with the way this works is that other tools is still there there wasn't we really did have an actual tool for when it comes to shove formation so integrating them with Shaq provided us with the actual CD is already told the USA on on source server configuration for in the figuration so chef was kind of bridge that brought us to the to leveraging these tools in a much more automated fashion and then shipped also helped us automated of the task which was not possible before so you stood Asia approaching 18 or whatever the right and lots of shares and you know knowledge of what is needed to their day-to-day tasks so after you finish your information of so right now if any stories administrator in any organization will have to be but they just need to know dell'orco book which has all the examples in that they can just start using this simple English-like gold there only when they here's unique to understand how they will pass on the object to some objects and the parameters to the API they can just use a simple recipe and just anybody can write a from following of them are you're out of cookbooks general owning classes in the beginning of the general run 1 so that they get constantly running lecturers system containing lead to reduced and what happens is we have to restrict those who are not on cooperation which and or behind its back and do things the old way to use to her letter so so the aka they are continually run which is why we have a running I'm totally you wanted be sure they stay in that configuration converge back configurations with on we're talking about people who really or buying into chat do not want to do that kind of stuff maybe doing stuff in the background change the configuration that chef will pretty much on the track so if you don't buy in and these things are being run continuously likes in our bases on a daily basis all that configuration doing this in the back shadow IT is going to be overrun with these as well yes so so in this case of we can run the same recipe again but let's say we are trying to create something which is already there so that today and creating a land with specific details with a specific number and the land already exist then it is not clear that LAN or if you're trying do if you're trying to delete something to what to be so so we we make sure that our our stories and then when they are executing those they understand what is going inside the recipe because this these can be of and you can write and of code which can destroy something right so they should understand and be aware before the user so we train them enough to be able to understand and as you can see here I mean of decay shift shift can be learned by a system analyst to learn in buddhist it's not very difficult into to the yes we have had some services so the solution is a onto furniture years of again I out of 35 Shiffrin as well on behalf the chef community Thank you athletes argument given run a + have the about