Deploying Cloudstack with Chef
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 199 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/32507 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 2014106 / 199
2
3
10
11
12
13
14
17
18
19
23
24
25
27
29
45
46
47
48
49
50
51
52
55
56
57
58
65
67
68
70
72
74
75
76
77
78
79
81
82
84
86
87
89
90
93
94
100
101
102
103
104
106
107
109
110
111
112
113
114
115
119
122
123
124
126
127
128
129
137
139
141
143
145
147
150
154
155
158
161
163
164
165
166
168
171
174
175
176
179
182
183
185
188
190
191
192
194
195
196
00:00
AlgorithmRule of inferenceForm (programming)Monster groupProcess (computing)CASE <Informatik>Source codeNatural numberSocial classAreaMultiplication signCovering spaceContext awarenessWordInstance (computer science)Right angleService (economics)Sound effectPhysical systemPoint cloudExtension (kinesiology)Open sourceDigital rights managementSlide ruleE-bookWebsiteAdventure gameComplex (psychology)Demo (music)Computer configurationArithmetic progressionCodeRevision controlIntegrated development environmentInstallation artOperator (mathematics)Endliche ModelltheorieDatabaseConnectivity (graph theory)Video gameComputer animation
03:19
MereologyMedical imagingSummierbarkeitStandard deviationWebsiteRevision controlProcess (computing)CuboidWordOperator (mathematics)NumberDerivation (linguistics)InternetworkingDigital rights managementCategory of beingDatabaseConfiguration managementStapeldateiInstallation artOperating systemType theoryConnectivity (graph theory)Physical systemComputing platformSource code
05:27
InternetworkingMultiplication signArithmetic progressionConnected spaceSoftwareSoftware developerIntegrated development environmentSource code
06:16
SoftwareSummierbarkeitData structureUniverse (mathematics)QuicksortVector spaceStress (mechanics)Connected spaceTime zonePoint cloudBitDemo (music)VirtualizationVirtual machineComputer animation
07:07
BitTemplate (C++)Plug-in (computing)InformationScripting languageIntegrated development environmentCore dumpProcess (computing)Group actionConfiguration spaceConfiguration managementVariable (mathematics)Parameter (computer programming)Time zoneSoftware developerComputer fileIP addressServer (computing)Utility softwareUniverse (mathematics)Interactive televisionSocial classRow (database)Right angleVideo gameMultiplication signNatural numberSystem callCycle (graph theory)TheorySource code
09:24
Connectivity (graph theory)Point (geometry)Source code
09:43
Multiplication signForm (programming)Group actionPattern languageRight angleContext awarenessPasswordCovering spaceEncryptionKey (cryptography)Process (computing)Object (grammar)Lecture/Conference
11:41
CASE <Informatik>Multiplication signInternetworkingSoftwareScripting languageQuery languagePasswordDatabaseCoroutineConfiguration managementModulare ProgrammierungServer (computing)Source code
13:18
Point (geometry)Instance (computer science)Goodness of fitSoftware frameworkIntegrated development environmentContinuous integrationCodePoint cloudProcess (computing)INTEGRALCASE <Informatik>Installation artSoftware developerProduct (business)BitOpen sourceMathematicsTheory of relativityServer (computing)Physical systemSet (mathematics)Cloud computingSoftware testingDigital rights managementUtility softwareStack (abstract data type)Mobile appInternet service providerOperator (mathematics)Plug-in (computing)TwitterAxiom of choiceMultiplication signContent (media)Virtual machineData conversionLevel (video gaming)ConsistencyFrustrationBlogFrequencyMereologyElement (mathematics)Right angleComputer virusEndliche ModelltheoriePattern languageSummierbarkeitDisk read-and-write headAreaWater vaporRange (statistics)Social classNatural numberWordClosed setReduction of orderCore dumpFunctional (mathematics)Variety (linguistics)Physical lawArithmetic meanSource code
20:52
Data dictionaryStaff (military)WindowWordRight angleNumberShared memoryCodeStack (abstract data type)Open sourceData conversionBitXML
Transcript: English(auto-generated)
00:06
So, we're going to do a talk on Deploying Class Actors Chef. There's my perfect, I'm the good father, or the nickname is the good father. If you don't know what that means, come by the conflict management campus again and you can find out.
00:21
Otherwise, just start taking a hold of that and then you'll find out. I am not a chef, but I work for a chef. I noticed that when we make these slides nowadays and our company name is now Chef instead of OpsCode, it gets very confusing when you just click Chef and a couple of things have been moved down. So, some background on me. I did Linux engineering for a long time and I guess still do to an extent.
00:47
I work in operations environments for a company called Orbitz. They run e-bookers here in Europe, but also travel sites in the U.S. as well. I've done performance and complexity planning, cloud architects, done a lot with automation,
01:03
and like I said, I currently work with Chef. So, hopefully everyone in the room knows what CloudStack is, but I'm going to tell you anyway. CloudStack is an open source cloud management layer. It was started by a company called cloud.com. They open source about 95% of their code under GPLv3.
01:24
Kind of a common misconception is that CloudStack was never open source. CloudStack was actually pretty much open source from the beginning when cloud.com released it. So, in May 2010, they released it. Shortly thereafter, they got bought by Citrix.
01:41
And Citrix is actually, as some of you guys might know, trying to develop more of an open model in the things that they do, things like Zen and CloudStack. Cloud.com, like I said, was bought, and then Citrix donated the CloudStack's code to the Apache Software Foundation in April of 2012. And then shortly thereafter, CloudStack 4.0 came out, and that was the first release
02:04
that was actually incubated from the Apache Software Foundation. It is now under the Apache license and no longer in adventure. So, this talk may be a little haphazard, and the reason why it's a little haphazard was
02:22
I wanted a month to prepare for this, and I got through it in like three days. And, well, because, you know, life happens, right? So, I'm happy to jump in. I'm not complaining. I just wanted to lay the groundwork of why it's a little haphazard. So, the basics. So, what are the basic components that we need to essentially install CloudStack?
02:46
So, this doesn't matter if you're using Puppet, if you're using CF engine, if you're using whatever, if you're doing it manually. You need to install the database. You need to set up NFS optionally. You need to install CloudStack. And then, step four in all of these things, right?
03:00
The last step is to prop it. So, I'll preface this with do as I say, not as I do. And you'll see what that means as we kind of progress throughout this. So, let's do a quick demo. And I totally forgot that I was supposed to have this running the best kind of time.
03:24
Because this takes like five minutes to run. So, I can very easily just do a big run on CloudStack. And it's going to kick off the box. And it's going to go through the process of actually installing CloudStack. I have to run the right command.
03:46
I know everybody in the room that's giggling when I type that. It's like, what is this guy doing? So, this is going to come up. And if you're not extremely familiar with Chef, I'll kind of explain what's going on as this kind of scrolls in the background.
04:00
Before I do that, essentially what I have set up is I have a couple recipes. So, in this case, this recipe only works for a Red Hat-based operating system. It hasn't been extended for domain-based operating systems or other operating systems. And essentially, we're just going to take the template resource and add in the YUM repos that we need for CloudStack.
04:25
So, the nice thing about CloudStack is they provide binaries for everything. So, for whatever particular platform you need, you can either advocate install or a YUM install. So, I'll set up my template, and then I'm going to install the CloudStack management piece. And then we're going to go and kind of do all of the ancillary components.
04:43
You could stop right here, and then you could kind of do the rest of the configuration management yourself if you wanted to. But what you really want to do is you want to go in and you want to create PaaS or NFS. Of course, you always want to shell out the batch and your configuration management.
05:01
You don't. You want to initialize the database and so forth, right? This is actually a recipe that was contributed by a guy named Pierre Dion, I think his name is. He's from Montreal, and he works for a company called CloudOps. And they deployed CloudStack as a living property.
05:22
So, this one's available on our community site, and it works fairly well. Obviously. Oh, yeah, I had this problem earlier. So, we'll try it one more time. And this is always the danger in doing a live demo, is that it just doesn't work.
05:44
And we've rebooted the network again. No, here we go. I was doing this earlier, and the network came out from underneath me, and then all the APs went away. And because I'm doing things like going out to YUM, then of course you need to have an internet connection if you don't have it.
06:08
There we go. So, we got further that time. So, we'll just let this run and bake in the background and come back to it here. So, this kind of base recipe that Pierre had, or this base cookbook, works fairly well.
06:27
But if you want to take it and extend it a little bit further, then you can make it better in the following ways. So, you can set up the base infrastructure. So, CloudStack has this concept of zones and pods and networking that you need to set up so that you can actually make it something useful.
06:41
So, each, basically, tenant would have their own network, so you can segregate virtual machines on the CloudStack cloud. So, you want to set up base infrastructure, and then you want to set up hosts as well. In this case, this doesn't actually set up any hosts that have virtualization running on them, like Zen or KVM hosts.
07:01
So, in this case, we want to make it better, and then I would demo that again if I was smart enough to get the demo running in the first place. Ah, so it's booting now. And so, to basically make it a little bit better, there's this plugin called, or this command line utility called CloudLine. And what this allows you to do is essentially interact with CloudStack's API via the command line, right?
07:28
So, basically what ends up happening in your development lifecycle of configuration management is you get it working with the command line utilities, and you wrap configuration management around it, and then you go and make it
07:40
better, and you do it the right way by interacting with the API correctly. So, what I did here is essentially, in this case, we're going to create another template, and this template's going to pull a lot of data. And so, this is one of the things that I love about Chef is all of the data that you have available to yourself in your recipes and so forth.
08:04
And so, I'm going to create a template, which is actually just a script, and Sean, if you were in Sean's talk earlier, would probably be trolling me right now. But he probably is, I just don't know it. And then I'm going to actually execute this script that I run. And these variables will feed into this template, so this is kind of a core concept of Chef.
08:22
You're not familiar with Chef. All of these variables then can be used inside of this template when this template is parsed by the Chef group. And so, what I'm going to get here is a configuration file that I'll actually show you with all of my data. So, there's all of my commands that come in, or all of my parameters that come
08:44
in, and then I can actually walk through the process of actually creating the actual zones. So, all that data comes from is from what's called our environment. So, we don't necessarily need more than one CloudStack environment.
09:04
You don't want more than one CloudStack server running maybe in one environment. So, you can actually feed in data to that script that I have to actually configure all of the IP address information and things like that. So, all of this data gets fed into that script that we just saw, and then that actually executes the CloudMonkey pieces.
09:25
If I flip back, this is actually working. And so, in this case, we're actually installing the CloudStack components at this point. Any questions at this point? Yes.
10:01
Yes. So, what Chef has is something called encrypted data bags, and in this case, this recipe doesn't use encrypted data bags.
10:27
But what data bags essentially are are basically data objects that you can store structured data, right? And in this case, it's JSON structured data objects. And so, essentially, you can create these data structures.
10:41
So, go home or go out in the hallway, or you can do it now while I'm talking to you. So, Google, OpsCode, Chef, encrypted data bags. And the example we actually give is how to store a MySQL password in an encrypted data bag. And essentially, what ends up happening is then your cookbook or your recipe can actually query that data and use it.
11:03
It'll have the pre-shared secret key to encrypt that data or decrypt that data bag. But that's the way that you solve the passwords. What's actually happening in this recipe, which isn't really bad. And so, I'll kind of cover why this recipe was written in a little bit.
11:20
But what they're actually doing is just putting it in an attribute. And so, if anybody can actually go and query and see that data, and that's not really what you want with your password for databases, right? Right? So, in this case, there's actually lots of recipes in this cookbook that does the actual installation process.
11:47
So, in this case, this is initializing the database. And if I can pull it up, you'll see all the stuff that it's doing. So, in this case, you see it's querying the server for a password.
12:02
And then it actually runs the initialization routines. So, in this case, this is kind of a problem with some, I guess, software packages. I thought I was getting hooked off already. I'm like, wait, I have time.
12:23
The problem with some of these software things that you might download is that they have this kind of script already to initialize the databases. So, if you were in the talk earlier when we did the configuration management 101, this isn't something that's, you know, item coded, right?
12:41
So, in this case, what he's doing is he's using some pieces of Chef to actually go in and query and say, did we already initialize the database? Because if we already initialized the database, and if so, if that's one, then I don't need to do this again, right? And so, he's trying to make it...
13:03
Do you like that? So, he's initializing the database only if he needs to initialize the database. So, this recipe can run. So, if we go and check on this, we have a problem because the Internet's having fun.
13:26
And so, let's go and talk about why this sucks. So, back in November, I think it was, I had a talk with a guy on my team called Matt Ray.
13:43
Matt Ray has been working almost exclusively... He works for Chef, but he's been working almost exclusively with what we call our cloud integration team, which the cloud integration team has actually just met. So, he's been running this. And he's been working almost exclusively with OpenStack.
14:01
And what's really kind of awesome, I'm not going to get into a CloudStack versus OpenStack debate. We can later. But what's really awesome and what Matt has done is developed really good content along with the OpenStack community to deploy OpenStack in kind of a well-known way that's consistent and things like that.
14:22
It's been used so good, or it's been curated so well, that it's actually used to run a lot of the continuous integration environments that OpenStack uses. So, when somebody merges a change in, that's picked up, and essentially a shop process gets picked off to build an OpenStack environment
14:43
with those changes and things like that. So, what's happening with CloudStack is we have fun things like this. So, search for CloudStack Cookbook. And you're like, great, I have lots of choices. Which one should I use? Well, I could use this one, which is from a provider in the Netherlands
15:03
who does great works. And it looks like, oh, well, they even contributed a cookbook, good for you guys. Not being facetious, really good for you guys. And then they have a nice little blog post about how they're using Vagrant and CloudStack in Chef Cookbooks. And then, oh, but this one looks good
15:20
because it's on the ops code side, so this must be the official supportive thing, right? And look, there it is, it's been downloaded seven times. The other thing is like, it's nice and it's fresh, right? So, you want to make sure that it's recent and somebody's updated it. It works maybe on the distro that I need.
15:42
Well, let's dig down a little bit more. And then, here's the CloudStack Cookbook 1. And this was contributed by Hugo. Is Hugo in the room? Hugo already knows how to do this, so he didn't show up. But, you know, it's four months old, so it's not too terribly old. It might be fresh.
16:01
And then, I have this other one, which was contributed by Citrix. Oh, wait, but Citrix, CloudStack, they're related. This should be a good luck test here. That one's 11 months old. So, it's good, I've got a lot of choice, right?
16:21
But, you know, it's kind of like the yak walking into the bathroom and he realizes, well, I need to shave. And so, you start to download the cookbooks and the next thing you know, none of these are good. Because they've been written for somebody's specific environment, right? So, the cookbook that I was showing earlier
16:42
is actually developed for a very good use case So, the CloudOps team develops pieces for CloudStack as well and contributes that in. Well, their developers need environments to actually test their code on. And so, what's good about that cookbook
17:00
is that the developer can actually do a big enough and he gets his own little environment on his machine and he can test his CloudStack code, right? But, it's not developed for production. The Schuberk-Philis guy stuff, well, that's developed for their environment and it's great and you guys have contributed something to the community but I don't know what path I need to go down, right?
17:21
Which one should I choose? And so, everybody who's worked with open source for a long time knows what I'm talking about, right? It's the reason why I'm not in the desktop room right now, right? Because after years of frustration I said fuck it and I went and bought a map, right?
17:42
And I'll run open source on the servers but, you know, I just want something that works, right? And I don't want to have to shake the app all the time. And then, this continues, right? So, there's a command line utility called Knife on Stack and it's really cool. You know, there's plug-ins for our tool called Knife
18:02
and what Knife allows you to do, if you're not familiar with it, is one, you can interact with the Chef server and you can change things, change settings and bits and pieces, upload your cookbooks, all that fun stuff. But then there's plug-ins. And so, Knife essentially provides a framework and you can provision instances in EC2 or OpenStack
18:21
or you can manage your CloudStack cloud. And then we get into the exact same thing again. It's like, alright, well this one is CloudStack extras and looks really all official and everything. But then there's this other one and, you know, which one should I use? And so, once again, the whole premise of this talk is
18:48
one, I hope you got something useful out of it. There is a path to actually deploy CloudStack with Chef. The problem is it's not well curated by the community yet, right?
19:00
And so, I have this conversation with Map and essentially what we want to try and do is we want to develop better content for the community to have a well set path so you're not having these questions of where do I put my password, right? Have something that's really well known and it can be repeatable.
19:21
It can be for lab environments, for developers, it can be for production environments as well, right? And so, really what I'm kind of here to say is talk to me afterwards, reach out to me on Twitter and let's get together as a community and kind of really curate this content for CloudStack. CloudStack is a great cloud system.
19:43
It provides a lot of things that an ease of use of path that other cloud provider or cloud management systems don't have. It's well proven in many production deployments and we can make it a lot better.
20:00
The reason why I came up with this as the premise for my talk and to prove I'm just not bolting you guys just on my experience is I was talking to somebody at a big bank and they use Chef and they also are wanting to use CloudStack
20:20
and he's like, hey, I was going to install CloudStack with Chef and I found this cookbook and I found this cookbook and I don't know which one to use and I told him we'll use these two and he's like, well, what about this CloudStack Knife fog stuff and all this other stuff, right? So this is like real pain that our users have and what we've kind of come up with this theme with for Chef is this, right?
20:45
So our kind of, and you can see it on my shirt, which this is Morse code and I think it's the wrong Morse code. We have two t-shirts. It's the wrong Morse code and then the right Morse code. But it's make it fucking delightful because this is not a delightful process, right?
21:02
And as you develop any of your open source stuff and contribute it back into the community and put it up on GitHub, make sure that you're making it fucking delightful for someone who's going to go and download it later because the number one kind of complaint about GitHub is there's just so much shit up there these days, right? You can't go and grab something without making sure that it's going to work, right?
21:23
So we're sharing, but we're not making it a delightful sharing experience. So that's my talk. What questions do you guys have? And if there are some questions about actually deploying on Stack with Chef and other experiences and how people get things to work,
21:42
come and talk to me as well because we can have those conversations as well a little bit more in depth. Nothing? Thank you very much.