Test Driving your Rails Infrastructure with Chef
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Part Number | 26 | |
| Number of Parts | 94 | |
| Author | ||
| License | CC Attribution - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/30710 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
RailsConf 201526 / 94
1
4
7
8
9
10
11
13
14
16
17
19
21
24
25
29
30
33
34
35
36
37
39
40
42
47
48
49
50
51
53
54
55
58
59
61
62
64
65
66
67
68
70
71
77
79
81
82
85
86
88
92
94
00:00
Right angleSpring (hydrology)View (database)WordArithmetic meanSoftware developerSet (mathematics)Source codeReal numberState of matterDialectDependent and independent variablesMultiplication signAreaVideo gameTheoryInternetworkingData storage deviceProcess (computing)ResultantRow (database)Revision controlBitVideoconferencingPoint (geometry)1 (number)Endliche ModelltheorieOrder (biology)Musical ensembleTwitterVirtual machineLink (knot theory)Cartesian coordinate systemPhysical systemShared memoryConfiguration managementComputing platformCodeSelf-organizationSystem administratorCodeRegulärer Ausdruck <Textverarbeitung>Term (mathematics)Characteristic polynomialDescriptive statisticsSlide ruleOvalPerfect groupWater vaporComputer animation
07:41
LaptopProcess (computing)Cartesian coordinate systemDifferent (Kate Ryan album)Real numberCodeView (database)Data structureOpen sourceLine (geometry)Information securityMedical imagingRevision controlIntegrated development environmentVirtual machineComputer fileSystem administratorArithmetic meanNumberScaling (geometry)Point (geometry)Source codeMultiplication signSoftware testingSlide ruleComputing platformData storage deviceServer (computing)Basis <Mathematik>Complex (psychology)Configuration managementConnectivity (graph theory)Physical systemBackupDatabaseSoftware frameworkMetropolitan area networkProduct (business)Repository (publishing)SubsetMereologyMathematicsPoint cloudSoftwareWeb 2.0Electronic mailing listKeyboard shortcutComputer programmingQuicksortState of matterTask (computing)Client (computing)Reading (process)VideoconferencingRight angleExecution unitGroup actionService (economics)AreaInstance (computer science)1 (number)Power (physics)ResultantSelf-organizationComputer hardwareOrder (biology)BitSet (mathematics)Functional (mathematics)WebsiteUniverse (mathematics)Computer animation
16:05
Web 2.0PlastikkarteElectronic mailing listServer (computing)QuicksortCartesian coordinate systemLastteilungComputer filePoisson-KlammerSlide ruleConfiguration managementVirtual machinePasswordCuboidIP addressText editorPhysical systemWindows RegistrySet (mathematics)Multiplication signRadical (chemistry)InternetworkingStructural loadBootingGraph coloringInstallation artWindowService (economics)CodeSubject indexingProcess (computing)Goodness of fitState of matterView (database)Type theoryBitWorkstation <Musikinstrument>Uniform resource locatorLine (geometry)Universe (mathematics)Level (video gaming)TheoryNatural languageIntegrated development environmentCASE <Informatik>NumberResultantAxiomMultiplicationMedical imagingPhysical lawMaxima and minimaRule of inferenceData structureWordMereologyBounded variationEndliche ModelltheorieCross-correlationSource codeRadiusPoint (geometry)Pattern recognitionFood energyExergieProxy serverContent (media)Web serviceRight angleComputer animation
23:53
Virtual machineWordSocial engineering (security)Test-driven developmentSystem administratorPasswordType theorySpacetimeAxiom of choiceMereologySensitivity analysisAuthenticationPhysical systemCASE <Informatik>HypermediaRight angleTouch typingMultiplication signDialectUniverse (mathematics)TheoryComputer animationLecture/Conference
26:33
NeuroinformatikSoftware developerInstallation artDefault (computer science)PasswordWorkstation <Musikinstrument>State of matterVirtual machineGoodness of fitKey (cryptography)Arrow of timeCovering spaceDirectory serviceAbstractionData managementCodeStandard deviationBookmark (World Wide Web)TouchscreenImage resolutionType theoryComputer fileAdditionRight anglePower (physics)Configuration managementRepository (publishing)BitMereologySystem administratorText editorSoftware development kitCASE <Informatik>Uniform resource locatorPhysical systemMonster groupPattern languageSet (mathematics)Shared memoryMultiplication signEstimatorForm (programming)Disk read-and-write headBit rateError messageMassCore dumpFitness functionQuicksortWave packetWeightDivisorAreaGraph (mathematics)WordGroup actionOntologyFunctional (mathematics)Cellular automatonComputer animationLecture/Conference
34:05
Error messageFormal languageTerm (mathematics)Bit rateVirtual machineWordTouchscreenMereologyHypermediaInternetworkingResultantPhysical systemMultiplication signGroup actionAssociative propertyComputer configurationCASE <Informatik>Right angleNumberInformationCategory of beingPattern languageWorkstation <Musikinstrument>Theory of relativityIntegrated development environmentCodierung <Programmierung>Instance (computer science)Software testingExecution unitWave packetState of matterUniverse (mathematics)Atomic numberPower (physics)Cartesian coordinate systemGraph coloringMathematical singularityLie groupLimit (category theory)Type theoryCodeFeedbackSlide ruleClient (computing)Electric generatorPoint cloudServer (computing)Endliche ModelltheorieConfidence intervalProcess (computing)Mobile appSoftware repositoryProduct (business)Default (computer science)Projective planePerfect groupHydraulic jumpBitCore dumpComplete metric spaceAbstractionSoftware developerWritingLaptopControl flowGeneric programming
42:39
Server (computing)Product (business)Point cloudInstance (computer science)Computer fileConfiguration managementDefault (computer science)Suite (music)CodeError messageSoftware testingDirectory serviceComputer cluster2 (number)Different (Kate Ryan album)Patch (Unix)Computing platformCombinational logicPhysical systemCuboidElectronic mailing listComputer programmingMultiplication signLine (geometry)Operating systemIP addressClient (computing)Cloud computingLocal ringState of matterOrder (biology)Complete metric spaceSemiconductor memoryMultiplicationAttribute grammarGroup actionAlgorithmEndliche ModelltheorieWordIntegrated development environmentRow (database)Inverse elementMereologyMatching (graph theory)MassRevision controlRight angleContext awarenessReading (process)MeasurementNumberAreaSound effectExistenceMessage passingOrientation (vector space)SummierbarkeitSet (mathematics)Core dumpEvent horizonPlanningMaxima and minimaSocial classOrder of magnitudeComputer animation
51:12
Electronic mailing listSlide ruleComputer fileDomain nameAreaSpecial unitary groupMassSingle-precision floating-point formatMultiplication signComputer animation
52:22
Social classQuicksortRight angleSoftware testingOperating systemWorkstation <Musikinstrument>Sampling (statistics)Bit rateGreatest elementBitMachine vision
53:56
WordWorkstation <Musikinstrument>LoginGoodness of fitElectronic mailing listSoftware testingSuite (music)Instance (computer science)Metropolitan area networkResultantUniverse (mathematics)Order (biology)
55:10
Visualization (computer graphics)Software testingLoginProduct (business)Server (computing)Client (computing)Ferry CorstenInstallation artDefault (computer science)Workstation <Musikinstrument>Integrated development environmentVertex (graph theory)Metropolitan area networkAssociative propertyVideo gameComputer animation
56:56
WordHypermediaRing (mathematics)Multiplication signServer (computing)Task (computing)Variety (linguistics)Software frameworkFamilyRevision controlExecution unitClient (computing)CodeComputer fileElectronic mailing listSoftware testingXMLUML
58:35
Metropolitan area networkSingle-precision floating-point formatSystem callSoftware testingDependent and independent variablesLoginInstance (computer science)Computer animation
59:37
Default (computer science)State of matterSoftware testingServer (computing)WritingWorkstation <Musikinstrument>INTEGRALMultiplication signBitFormal verificationAbsolute valueOpen setCodeSelectivity (electronic)Computer fileDirectory serviceType theoryRevision controlBlock (periodic table)Point (geometry)Goodness of fit2 (number)Ferry CorstenString (computer science)QuicksortResultantLinearizationTask (computing)Formal languageLine (geometry)Metropolitan area networkInternetworkingPower (physics)Water vaporMassFinitismusDisk read-and-write headAreaAngleComputer animation
01:05:10
Inverse elementPresentation of a groupGradientInteractive televisionLine (geometry)VarianceSoftware developerMultiplication signMereologyMessage passingExecution unitEndliche ModelltheorieRow (database)ResultantWordNumberUniverse (mathematics)Principal ideal domainProcess (computing)Keyboard shortcutCodeSoftware testingFormal verificationThumbnailGoodness of fitServer (computing)2 (number)Software repositoryComputer fileInstance (computer science)Default (computer science)Electronic mailing listOpen setCartesian coordinate systemComputer animation
01:11:25
System callEmailAreaRevision controlIterationMultiplication signLinearizationServer (computing)FeedbackSoftware repositoryCodeSoftware frameworkUnit testingSoftware testingPoint cloudMereologyComputer animation
01:12:33
Ring (mathematics)Disk read-and-write headMereologyAreaLimit (category theory)Form (programming)Physical systemBit rateVolume (thermodynamics)Unit testingString (computer science)Software testingRight angleMultiplication signError messageSoftware repositoryINTEGRALCodeCartesian coordinate systemComputer animation
01:13:50
PlastikkarteServer (computing)Software developerSoftware frameworkWhiteboardInformationLine (geometry)Type theoryBitTouchscreenCartesian coordinate systemMobile appQuicksortSoftware testingMultiplication signMathematicsView (database)System administratorBranch (computer science)Right angleSoftware repositoryWebsiteLattice (order)Slide ruleLoop (music)State of matterUnit testingPhysical systemVirtual machineCodebuchINTEGRALGUI widgetCodeGroup actionMassPoint cloudDefault (computer science)Open source10 (number)FacebookStapeldateiHypermediaExecution unitSocial classConcentricResultantService (economics)CASE <Informatik>Data miningForm (programming)Level (video gaming)Set (mathematics)AreaWordMereologyEndliche ModelltheoriePhysical lawSpherical capReal numberSemiconductor memoryField (computer science)Bit rateComputer animation
Transcript: English(auto-generated)
00:12
What's up? Welcome to Infrastructure Coding with Chef. We're going to introduce ourselves and then we're going to get into the presentation.
00:22
Also, if you want to click that link there, you can go and grab all of the slides and code that we're going to walk through right now. It's difficult to click it from your chair, I understand that, but give it your best shot. Is it a Microsoft thing? Yes. Alright, so this is great. Why don't you introduce yourself while I figure out why am I selected?
00:45
Certainly. Awesome. I'm Nell Shamrell-Harrington. I'm a software development engineer with Chef. If you've seen me speak before, you may know about my undying love of regular expressions. I'm not going to talk about those today, but if you're interested, I've got plenty of work on it, and I'm always happy to talk about it.
01:04
You've got other bullet points you can say. I do, indeed. The main thing I work on at Chef is the supermarket product. It's a Ruby on Rails application. That is the application where we keep all the community cookbooks. We upload community cookbooks, download them, install them directly from the supermarket.
01:21
It's at the supermarket.chef.io if you want to check it out. The final thing, I'm a Naginata practitioner. Naginata is a Japanese martial art. If you're interested in learning more about that, just let me know. Over to you, Nathan. Hi, everyone. My name is Nathan Harvey. I'm a community director at Chef, which means that if I tell you how to run your infrastructure,
01:44
you should not believe me because I don't manage infrastructure. I manage a community and work with people. I used to manage infrastructure, so maybe I used to have credibility, but I wouldn't trust me much anymore. It's cool, though. The other thing I do is I co-host the Food Fight Show, which is a podcast that's all about DevOps and Chef.
02:03
I'd love it if you would subscribe and listen to that. You might be disappointed, but it will make me feel better if you tell me that you listen to it, so that's cool. I occasionally farm, and I love eggs. If you were to click on either of those links, you would understand that I actually never farm, and you probably understand why I no longer like eggs.
02:21
You can follow me on Twitter if you like. You can also email me. Yeah, that's me. Let's go ahead and get started. I'd like to know a little bit about you. If you were a system administrator, could you raise your hand? Or if you've ever been one in the past, if you're related to that. How many of you are developers?
02:41
Awesome, like the rest of you. Even some of those systems administrators raised their hands. That's a crazy thought. How many of you are DevOps? Awesome. That's good. Good stuff. I don't want to put my water on my phone. Business people? How many of you are business people? Bullshit. Raise your hand.
03:00
You're all business people. All of us. If you write software, you're a business person. If you're a system administrator, you're a business person. You have a job because you work at a business or an organization that cares about customers or people that use your stuff. You have a vested interest in that. You are a business person. How many of you are experienced with infrastructure as code, as like a thing,
03:22
or with configuration management, like a software, a framework, a platform that will do that? Awesome. How many of those are experienced with Chef? You can lie. I'd rather you didn't lie, but it's cool for you. Awesome. When you're experienced with Chef and you're here, what I always tell folks when we do introductory talks
03:41
is when we lie, your job is to keep your mouth shut. Don't let anyone else in on the secret that what we're doing is telling them the hard way. It's a better way to learn, I'm sure. I want to start with a story, a quick story here. I grew up in the 80s. A little while ago, I was watching this show on National 3D graphic that was called
04:02
The 80s, The Decade That Made Us. This show actually defined DevOps in a really clean way for me because, I'll be honest, I've been to a lot of developer conferences and a lot of developers come up to me and they say, I hate DevOps. I don't like to DevOps.
04:21
I think DevOps is stupid. The problem when they say that is that you just don't understand what DevOps is. Or rather, we don't have a common understanding of what the definition of that term is. I want to share how this amazing show on National Geographic explained DevOps for me. In order to do so, I have to take you back in your mind to the 80s.
04:42
Some of you, this is a completely foreign time because you were very, very young or not yet with us. I was definitely there. Here's one of the things that happened back then. There were bands on the radio. You could listen to them. You couldn't listen to them on the internet yet because they hadn't discovered it.
05:00
Al Gore hadn't invented it, actually. We couldn't do that. The bands, they were defining characteristics of the bands. Sometimes there were rock bands and there were also rap bands. The two would never collaborate with one another ever. Ever, ever, ever. One of my favorite bands back in the day was Run DMC.
05:21
This guy had this crazy idea that what if Run DMC made a rock song? Run DMC, if it's not clear, they were a rap band. As I said, they would never, never think about making a rock song. Some guy said, hey, you should totally do this. I want you to listen to this song from this rock band and I'd like you to cover it. They went and they listened to it and they said, oh, hell no.
05:41
This is Hillbilly Jimberish. They did not understand what was happening. The minute I heard that, myself as an assistant administrator, I related to that. I said, it's like me with the developer's handy code. I don't understand this code. How am I supposed to run this in production? They talked to the rock band, who happened to be Aerosmith, whose song they were going to cover.
06:03
They said, yeah, we want to put you together with Run DMC and sing this song. Steven Tyler said, what the hell are they doing to my song? Which, frankly, sounded like a developer coming up to me and saying, hey, works fine on my machine. What's your problem? But this amazing thing happened. This producer, Rick Rubin, put them both into the same studio,
06:23
put them together, and in his words, it was crazy good band. These guys came together. The video is a perfect description of DevOps also, because in the rock video, you start off with the rockers on one side and the rappers on the other side, and there was a literal wall between them. During the video, they tear down that wall and they start working together.
06:42
Now, the thing about DevOps is they didn't ask Run DMC to start being rockers, and they didn't ask Aerosmith to start being rappers. They said, keep what you're good at. Just work together to make the audience happy. And that's what DevOps is. So I'd like to propose a definition.
07:01
Whenever you hear someone saying DevOps, I want you to think about it as a cultural and professional movement that's focused on how to build and operate high-velocity organizations born from the experiences of its practitioners. Now, this is how I define DevOps, and so when you say to me that you hate DevOps,
07:21
I say we don't have a common understanding of what that is. So I would propose that this is our common understanding of DevOps, and I hope you'll agree, but it doesn't matter because I'm on the stage, so now you all agree this is our common definition of DevOps. Are we good? All right, let's talk about some infrastructure as code. Sorry, I had to just rant a little bit about DevOps, but I hope that was a fun little journey.
07:42
Okay, so infrastructure as code, we start treating our infrastructure as code for many, many different reasons. Of course, what we're doing is we're automating the infrastructure that runs our applications. As it turns out, applications that work on your laptop are not of much value to anyone other than you.
08:03
We want to take our applications and put them into some real infrastructure, whether that's infrastructure on the cloud, whether that's infrastructure in your data center. And by doing so, or as we do so, we want to automate that entire process. When we think about, well, when do I need to automate?
08:21
Isn't Git push Heroku? Am I DevOpsing? Sure, of course you're DevOpsing if you're Git pushing to Heroku. You're a high-velocity organization, but at some point you may want to put your applications on some other infrastructure. And when that time comes, you'll know it because you will have reached some scale. And that scale doesn't necessarily mean
08:42
the number of machines you have. It could be the number of people that you have that are building out your infrastructure or building out your applications. It could be the complexity of those applications and so forth. So you really need an automation platform that can handle your automation needs, that can handle your infrastructure and really allow you to treat it as code.
09:02
The thing I'm not going to do is read you all of the bullets on this slide. The thing I have already done is make all of these slides available to you so you can read them for yourself. But automation platform, it's a thing that you want. You can use any automation platform that you like. It should be the automation platform that works well for you and your team.
09:22
And there is one other criteria that you may disagree with. The automation platform that you use, you should not roll it yourself. You should use a framework that's out there in open source already on the market. So infrastructure is code. What does that mean? It means that you can programmatically provision and configure the components
09:40
within your infrastructure. Whether it's hardware, whether it's a cloud instance, it doesn't really matter. It means that you treat your infrastructure code like any other code base. Now when I say treat something like any other code base, what do you think I mean? This is the audience participation part. Source control. You put your infrastructure code into a version control system.
10:02
What version control system do you like? Git. You can use any version control system you like. It should start with GIT. What else does it mean to treat your software like code or treat your infrastructure like any other code base? Test. We should write some tests. We kind of gave it away in the name of the talk.
10:22
Maybe we're going to talk about some testing. So yes, testing. Why do we put our things in source code control? Why do we test our infrastructure code? There were three answers at once. Let me just distill them all down. Because code is a continual experiment.
10:41
We're never done with it. And your infrastructure code is no different than your application code. Over time you're going to refactor it. Your requirements are going to change. Therefore your infrastructure is going to change. Therefore your code is going to change. You four that answered my question, that's basically what you said, right? Yeah, pretty much. Again, you can lie to me, it's fine.
11:02
I'll never know. With treating your infrastructure as code, you can reconstruct your entire business with three things. With your code repository, a backup of your data, and with compute resources. I will tell you that I used to be a system administrator and before, like when I first became a system administrator,
11:20
I was a terrible system administrator. And that's a longer story that I'm happy to tell you after this talk. But there was one day in my journey as a system administrator where I realized that my next change to my production environment was going to start with a commit to a git repository. That day I became a free man. No longer was I logging into production and banging on the keyboard
11:43
and hoping for the best. This is where I want you all to be, in the free world that is infrastructure as code. Alright, that's infrastructure as code. With Chef, and with many of these infrastructure as code platforms, what you're working with is a policy-based system.
12:02
So in your code, you're going to capture the policy for your infrastructure. What are the requirements of your infrastructure? What does that look like? And each one of these, and of course we're talking mostly about Chef today, but each one of these will have some sort of application, some sort of program that will make sure that the policy
12:20
is being continually applied within your infrastructure. So over time, as your needs change, you can change your policy and your infrastructure will be updated. Which is great, except when you change your policy and then all of your infrastructure gets updated. Because maybe you wrote the wrong policy and you just brought down all of your infrastructure, that would be bad.
12:41
So we test. Let's see. So if we take a very simple infrastructure, we've got some Graphite, some Nagios, some Rails, a little database. Let's say you have a security firm come in and do an audit on your infrastructure. How many have been through a security audit on your infrastructure before?
13:01
All right, so this story will ring very true to you. So a security team comes in, they say, you know what, this infrastructure is not secure and here's why. SSH is listening on port 22. So the way that we're going to secure this infrastructure is we're going to have SSH listen on a different port. Because that's security, right?
13:20
You guys are laughing because you've been through the security audits and you know that that's true, that's how we secure things. When we think about what does that mean for our infrastructure, it's really one line in one file on these machines. Like who could edit six lines of code or one line of code six times by hand?
13:40
You should all raise your hand again. It's like the business person question, you can all do this. And it's totally a thing that you could do. But of course, let's say we were working with golden images, I have six different images and then I have to replace each one of those instances. And if I'm in the cloud, I come up with new IP addresses, which means that now my configurations are all worked
14:00
and I have to fix them up again and that can be a real challenge and a real headache. So you need a platform that can help solve that for you. And of course, Chef is a platform that can help solve that for you. You probably saw where I was going there from the beginning, so here we are. Let's talk about Chef. What is Chef? Chef is an open source framework
14:20
that will help you manage your infrastructure as code through a policy-based system. Chef is also a company and it is also a community of professionals. So welcome to the Chef community. If you didn't know you were part of the Chef community, guess what? You are. You are now also part of the Chef community. So now, when you leave here today, if nothing else, you'll be able to be a card-carrying member
14:43
of both the Rails community and the Chef community. That's pretty awesome. Don't you think? So Chef has a bunch of components. We're actually not going to spend a whole lot of time on the components because that starts to feel like I'm trying to sell you something. And what I really want to get to is the hands-on part where we write some tasks.
15:00
Is everybody good with that? If you want to learn more about all these components, come by our Chef booth. We'll be here tomorrow and Thursday as well. You can also maybe win a drone if you come by there. Just saying. So the Chef server itself stores your policy and the state of your infrastructure. Let's just, at a very high level, how does Chef work?
15:21
So on our node, in the machines within our infrastructure, we have a little application called the Chef client. And it runs on a regular basis. And what it does, when it executes, it will ask the Chef server, what is my policy? What policy should I follow? The Chef server itself stores all of your policies. So your policies are things like, here's what it means to be an application server.
15:41
Here's what it means to be a database server. Here's what it means to manage users on that machine. Each one of your nodes within your infrastructure will only use a subset of those policies. We call that subset of the policies the run list. So the server stores a run list that says, oh, you are a web server. So what you need to do is run NTP
16:01
because we should all synchronize our clocks. That's an important thing that you do when you DevOps. We should manage some users and you should become a web server. Those policies come down to the node and the node will bring itself in line with those policies, which really means it will inspect those policies and say, am I following the policy? If I'm not following the policy,
16:21
I'll take the steps that are necessary to come in line with that policy. Now, so that's the policy half of storing policy and state within your Chef server. So when we talk about state, the Chef server itself also has a searchable index of data about every node within your infrastructure. So the Chef server has all of that information for you.
16:42
And this can be extremely helpful in a scenario where, say, you're doing some load balancing. So here in this example, I have HAProxy as my load balancer. And if you've never used HAProxy, it doesn't really matter. It's a load balancer, requests come in, and it forwards requests back to application services behind it. But when you're configuring your HAProxy
17:02
or any other load balancer, you have to tell your load balancer who can receive traffic requests that you get. So who can you send those requests back to? Well, there's a collection of web servers. The problem is your infrastructure is dynamic. It's elastic. Over time, you're adding and removing web servers. So when you sit down to write a policy for your load balancer,
17:22
you don't know exactly which hosts it's going to have behind it. So with Chef, because it has a view of your entire infrastructure, the HAProxy server can actually ask the Chef server, who are the web servers in my environment? The Chef server will consult its search index,
17:40
and it will pull back a list of web servers that sit within this environment, send that down to the HAProxy server, who then can rewrite its own configuration file such that it's pointing to all of the application servers that sit behind it. And then the next time HAProxy runs the Chef client, it will go through the same process, who are the web servers now?
18:01
It's like a young kid who keeps asking the same question, are we there yet, are we there yet, are we there yet? Who are the web servers now? Who are the web servers now? And it will continually update its configurations. So let's talk about what questions you have before I talk about anything else. So what questions do you have?
18:24
Is this all pretty clear? How many of you knew everything that I said already when you walked in the room? It's cool, you can raise your hand, don't be shy. All right, good. So maybe you learned one thing or two, or maybe more. That's awesome. Let's talk about building out your policy. So how do we actually take Chef and use it?
18:43
What does that look like? What does the Chef code look like? At its very core, this sort of nugget of Chef is this thing that we call a resource. So resources within Chef are pieces of your system. It's a piece of the system and its desired state. So let's just look at some code
19:00
and some examples of what that is. It might be a package that should be installed or a service that should be running and configured to restart on reboot of the machine. It could be a file that should be managed, like this file should live here, it should have this content, this ownership, and so forth. Sometimes you're in cron jobs on your systems.
19:22
You should manage that. Basically, anything on your system that you can change, you should manage as code within the Chef recipes that you're writing. Each one of these different things is a resource. Jam, it works on Windows, so here you can see a desired state configuration resource or a registry key.
19:40
Not many Linux implementations, distros that I know, use a registry and have registry keys like this. So these are resources, a piece of the system and its desired state. Those two key bits. So this is a workshop. We're going to do labs. So how many of you want to type some Chef code? You can also lie to me now and be like,
20:02
yes, what I want to do is type some Chef code. That's awesome. So I'm going to give you a workstation that you can play around with and write some Chef code. And with each one of our labs, we're kind of going to set it up like this. We have a problem and success criteria. So in our first lab, our problem is that our workstation
20:20
doesn't have a text editor installed on it yet, and you should probably have a text editor if you're going to write some code. So we're going to solve that. And this is where the magic trick comes in. I lied. It's not a trick, but you do need a card if you want to do the labs. So what I'd like you to do,
20:40
if you're participating in the labs, you should go to this URL here, and what you will find is it's a gist, and it has a bunch of cards. You should look at the back of your card and sort out if your card is a blue card or a yellow card. Those of you that want to argue that it's really orange or it's really amber or it's not really yellow, you're probably right, but I called it yellow.
21:03
And then you have a blue card. If you don't have a card and want to do the labs or just want to take a card home as a souvenir, raise your hand so I can get you a card. All right. Seems like the left half of the room came in late. It didn't get cards. It's cool.
21:22
Cards. So what I'd like you to do, you will need something with which you can, well, so first you'll need an internet connection, which RailsConf has been pretty good about so far. Second, what you'll need is something with which you can SSH.
21:44
So if you have a Mac or a Linux box, you can just use terminal whether you're set. If you're on a Windows box, hopefully you have putty installed. If not, you should go install putty, but I hope everyone has something with which they can SSH with right now.
22:01
Okay, so then what you have to do is you have to find the color of your card and the value and the suit and match it up to an IP address. Once you've done that, you can SSH into the box. And I've given you the credentials there. Your username is going to be Chef. The password is not square bracket redacted square bracket. Some of you have already sorted that out.
22:21
The password is Chef.io. I didn't want to tell you that in advance because we'll put it on the slides because then you would know what everyone else's password is. Okay, so I'm going to do the same because I'm going to do this lab along with you so that you can see me do it.
22:40
And let's see, I'm going to go... Actually, I have it somewhere here. Oh, it's here. All right, I'm going to grab my card.
23:05
So what I'm giving you is a workstation. What would typically happen is that your machine in front of you... Can we see my screen? Can I drag it over? There's the slides.
23:39
What I'd like to show is not the slides.
23:43
Not the slides. Right, so it should be just pushing this to... All right, there we go. You can see that now, right? Okay, so what I'm going to do,
24:01
I'm going to SSH to my machine, but I'm not going to do it like that. You should always do what I mean, not necessarily what I type. So I'm going to SSH as Chef to that machine. And then here's your first DevOps lesson of the day. It's not really a DevOps lesson. It's a system administrator lesson.
24:21
How do you verify the authenticity of a host? I'm sorry? Yes, you type in the letters Y-E-S, and then you get the magic of it. This is how you do it. This is how professionals do it, honestly. Remember, I told you I'm a community director.
24:40
So then we asked you for your password. What's your password? Chef.io. And see, social engineering is the way that you will crack into any system. I just got you all to tell me your password. Now I'd like you to do something, and this honestly is the most important lab, the most important part of the lab that we're going to do for the rest of the afternoon.
25:00
I'd like you to type in the word touch, and then I'd like you to hit the space bar. And then I would like you to, all as one thing together, type in your first name and last name, but not the characters Y-O-U-R-F-I-R-S-T-N-A-M-E-A-N-D-L-A-S-T-N-A-M-E. I want you to take whatever your first name is.
25:21
So I, not you, but I am going to say my first name. It's Nathan, and my dad spelled it wrong, so I'm going to spell it wrong also. And then type in your last name without any spaces, and I'm going to do that also. And then the other thing that I didn't tell you before is you should never hit the magic button until either Nell or myself tell you to press the magic button.
25:40
And the magic button, of course, is the one all the way on the right, kind of in the middle. It either says return or enter. You can press the magic button now, so I'd like you to press the magic button. What if I still wanted to capitalize from there? It's okay if you capitalize. I don't care if you do capitals. You can case sensitivity is like, that's a personal choice. It's totally up to you, because DevOps is also inclusive.
26:02
So now, if you're doing the labs, and you have typed this, I'd like you to raise your hand. Okay, now I'd like you to put your hands down. And now, if you are doing the labs, and you have not yet completed this, I'd like you to raise your hand. Yeah, I do.
26:22
It's bit.ly slash rails dash chef dash TDD. Did that work for you? Bit.ly. L-Y. I could just show you the whole URL instead of trying to spell it.
26:42
That would be much nicer of me, wouldn't it? Is that better now?
27:00
While he's sorting this out, what questions can I answer for you? Yes? Were each of these nodes configured with Chef? They were configured with Chef. Indeed. Good question. Configured with a tool we're going to talk about later. Also, back, yes. I'll even show you how they were configured.
27:21
At least some of them. I will show you all of my secrets. That would be ridiculous. Pseudo-works. Yes, pseudo-works. So, what he really asked there, you couldn't hear his question, let me repeat the question.
27:41
What did you do to these boxes? What power do I have here? So let me tell you about these boxes. It's a vanilla, I don't know, it's centos, I think. Is it centos? So it's a vanilla, see how much I know? It's a vanilla centos AMI, which is an Amazon machine image, so it's a machine that's running off on EC2 right now.
28:00
I've done a couple of things to it. I gave you pseudo-access, passwordless pseudo-access, which is different than pseudo-less password access, as it turns out. So you have passwordless pseudo-access. I've installed a thing called the Chef DK, or Chef Development Kit. I've done some other things too, but I won't tell you what those are yet. I created a username, Chef.
28:20
I allowed that user to log in with username and password, which you would never, ever, ever actually do in real life. Why wouldn't you ever log into a machine with a username and password? What would you use instead? Yeah, you could use your SSH keys. And those of you that said you weren't sysadmins before,
28:41
you were clearly lying. Have you logged in? Have you touched your name? Have you touched yourself? No, that's not the right question. Did you touch me? Okay, so now we've all done this. We've all done this. Now comes the really important part. I'd like you to type in the characters L and S altogether, and then I'd like you to hit the magic key.
29:02
And you should see three things here. You should see a file called git.rb. You should see a directory called railsconf2015-chef. And you should see your name and your name only. If you see someone else's name, someone else said, oh, he just gave me the username and password to all of these machines,
29:21
and I have sudo access, I'm going to mess with one of my fellow attendees. So you should find that person and maybe get a different machine. All right, everybody's good? We all have just our own machine, right? All right, good. As yet, we've done nothing with Chef.
29:40
But what we have done is set up a problem for you. So a text editor is not installed on this workstation. We're treating this machine as if it were your developer's workstation, but would typically be the computer that's currently on your lap. What's your favorite text editor from a command line? Vim. Vim. Raider Vim.
30:01
What's that? Raider Vim. Raider Vim, okay. That's what I said. That's cool. All right, so Vim sounds like the clear winner. We don't have any Emacs users in the room? Actually, no. Yeah. Do you know how you'll know if someone's an Emacs user? They'll tell you.
30:22
So anyhow, Vim. Vim seems like a good thing that we should do. So what we're going to do here is we're going to install the text editor, Vim. But of course we're not going to use the standard package manager to do so because that wouldn't be very Chef-y. We're going to use Chef to do that. So if we think about Chef, it's all about resources.
30:41
A resource is a thing in its desired state. So we have this thing, a package, and we want to have this package installed. So we're going to use a Chef tool to do this, and this is the fastest way to introduce you to some of Chef's tooling. It's a tool called Chef Apply. So you're going to type this along with me. First we're going to use sudo.
31:01
We're going to type chef-apply. Make sure you type everything I mean to type, not necessarily what I actually type. Then we're going to do a dash e, which is difficult for you to see because of screen resolutions. Let me see if I can fix this a little.
31:21
That looks even better, right? At least it's all on a screen. Do what you want for me. So we have sudo chef-apply dash e, and then another space, and we're going to do a double quote, and we're going to say package, and then a space, and then a single quote, and then vim, and then close off that single quote,
31:41
and close off the double quote. And if you've typed this exactly as I have, you can then hit the magic key. If you prefer emacs, you can replace the vim bit with emacs. If you prefer nano, you can replace the vim bit with nano. If you prefer greater than, maybe the package repository has that. I don't know. Put the right package name in there.
32:00
And then we're going to hit enter. And what happens when you hit enter? No, nothing. Look, nothing is happening. What's going on? It's taking a minute here because in addition to giving you free workstations to play with, they give you really small workstations to play with, so they have no power, so it takes a minute.
32:20
So what's working here is you can see that Chef has sorted out that this machine is running CentOS, so it's going to use the yum package manager, and it's going to install a package called vim. If we were to read the docs, like, really? Who does that? We would see that the package has a default desired state of install.
32:41
So really, if we were to say this code to Chef in English, what we would say is our policy is that the package named vim should be installed. And so Chef is going through the system right now and verifying that that's the case. How many of you did a which vim first to make sure that vim wasn't actually here?
33:00
Yeah, those of you that are like, yeah, I don't believe in anything that dude says anymore. He's been lying all afternoon. So it goes through and it does a yum install in the background. The nice thing is, I don't need to know, and I already admitted that I didn't, if this was a CentOS machine or an Ubuntu machine, Chef sorted that out for me, and based on that,
33:20
it knew which package manager it used to install that package. So it used yum because this is a CentOS machine. If this had been an Ubuntu machine, what would have been used? It would have done apt-get under the covers. But I don't need to worry about that. It abstracts all those details, all that new stuff away from me. And now it's installed the package for me.
33:42
So which vim is going to show me that, in fact, there is a vim installed? Now, as a thought exercise, what would happen if you hit the up arrow until it said sudo chef-apply-e package vim and then hit the magic key again? What would happen in that case?
34:02
It would tell you that it's already up to date. That's exactly right. See, you guys, you're all so smart. We have slides that walk you through all this, tell you some of the lies that I told you. Let's get all caught up here. Yeah, so if we were to run it again, and you can run it again, I'm not going to.
34:21
I'm just going to go back to the slides. What it would tell you is that the package vim is already installed. It's up to date. Chef doesn't need to do anything with this. In Chef terminology, we use really big mathy words to describe this. And you should forget those really big mathy words. I'm going to tell you how Chef works. It works in what I call a test and repair process. So all of our resources follow test and repair model.
34:43
So when we see package vim, Chef executes a test. Is vim installed? Is that package installed on the system? If it is, Chef will stop and take no further action with that resource. If it's not installed, then the system is out of policy. It's broken. It needs to be repaired.
35:02
So Chef will install it, and that's the repair half of our test and repair cycle. If you understand this about Chef, you'll understand everything that you actually need to know about Chef. Resources work in a test and repair fashion. Everything about Chef is abstractions and things to make it easier for you to work with resources.
35:25
This is the core bit of Chef. Understanding that resources are a thing. You describe the resource, the thing that you want, and its desired state. So in this case, the default desired state for a package is that it is installed.
35:41
What questions can I answer? We're working on a Chef client that will do that. So what you're working on, Chef client is a separate application. So we ran Chef apply. Chef apply is a little tool that comes with the Chef development kit. It's an easy way for you to explore resources. We haven't really applied a full policy.
36:01
We just applied a singular resource here. So because this is a workstation, really I think about that as my machine, your laptop where you're going to write the code. It's not necessarily a machine that's eventually going to run Chef client and be part of your infrastructure. Great. Thank you for the question.
36:21
Yeah, words. There are lots and lots of resources that come with Chef. You'll find them all on the Docs site. Here are just some examples of them. Also, you can create your own custom resources, which we won't get to today because we do have a limited amount of time. But it's totally something that you can do.
36:43
So we have this thing in Chef called recipes. A recipe is really just a collection of resources. So we just applied a singular resource, a package. But usually you're going to apply multiple resources in the same policy. When you put multiple resources together in a file, we call that a recipe.
37:02
As you might guess, we take our recipes and we put them into cookbooks, and those become our distribution unit. We can think of a cookbook as something similar to a gem because it might have other stuff. And with that, I'm going to switch over and let Nel take over for test-driven infrastructure and start writing through some tests.
37:21
I think we might have a question in the back there. Yes, awesome. It seems like there's overlap in my mind anyway, maybe not real, between nodes and resources. How are you doing? Sure, so let me try to help resolve that overlap. A node is a machine within your infrastructure.
37:43
And by machine, I'm sure you've got the controls, because it might be a physical machine, it might be a cloud instance like running off on PC2 or DigitalOcean. So it's the full machine. A resource is just a thing on that machine that you can manipulate. Okay, perfect. Okay, great.
38:00
While we get switched over here, what other questions can I answer for you? All right, as Nel takes us through some of these labs, it's going to be my job to run around and help you when you need help. So when you need help, make sure that I know. You can raise your hand, you can shout out my name, whatever it takes to get my attention.
38:21
The other thing I will say is that when you have errors, it's actually better, because you can learn more from errors and mistakes than you touched everything exactly the way Nel told you, and it worked. We kind of know that's going to happen. So if you have errors, we can learn more together. Nel, over to you. All right, let's talk about test driving our infrastructure.
38:43
So our process so far has been to write a policy, a policy which would install an editor, then apply the policy on our workstation, then verify the policy manually. Now this is fine for a simple case, but just like Ruby and Rails code, testing our infrastructure code manually quickly becomes untenable.
39:04
I don't think I need to spend too much time preaching to this crowd about the benefits of automated tests. For application code, having tests around our infrastructure code will let us speed up those feedback loops and let us push to production with confidence.
39:21
So the ultimate purpose of tests is to tell us whether or not our code is safe to deploy. With our Chef code, there are four main questions we need to answer before we can deploy to production. Did the Chef client complete successfully? Did the recipe put the note in the desired state? Are the resources properly defined? And does the code follow our style guide?
39:42
So let's jump into answering these questions. We're going to use a very simple scenario which will let us explore the many facets of test driving infrastructure code. And that actually is the alarm, which means it is time for the 10-minute break. Go ahead and take a break for 10 minutes in between part 1 and part 2.
40:02
Please do come back. And I will take you through an exercise where you will write code test driving actual Chef code. So we will see you in 10 minutes. All right, let's bring it back. We'll definitely save time for questions at the end. So we're going to use a very simple scenario to test drive our infrastructure
40:22
following that test-first, test-driven model. So our scenario is that we want a simple Rails app deployed on a server. We only have time to build some of it in today's workshop, but you will leave the workshop today with working code. And all of our instructions and slides are in that GitHub repo, so if you don't get through everything,
40:40
you're welcome to go through that at your leisure and work through it. So make sure you're back on your workstation, and let's go ahead and create a new Chef repo for your Chef project. So go ahead and write Chef generic repo Chef repo. And I'm going to leave my code up on the slides just because there's some extensive code.
41:01
I also have what I call someone is watching me typing syndrome, which means it's really hard for me to type when someone's watching me. So Nathan is going to run the code, and he's going to let me know whenever the commands are completed.
41:31
So now CDE into our cookbook. Let's first CD into that Chef repo and then CD into our cookbooks directory. This is where we're going to do the majority of our work.
41:41
So CD Chef repo slash cookbook should get you there. And next, we're going to generate an actual cookbook. You do this by letting Chef generate cookbook cookbook slash Apache. This cookbook will allow us to install
42:01
and configure Apache. So when you use Chef generate cookbook, the Chef DK has some magic built into it. It's a lot like using the Rails generates. And what it'll do is it'll generate some scaffolding to get you started and put it together your cookbook. So let's go ahead and run that. So just one quick thing.
42:20
If you're in the cookbook directory, should this be Chef generate cookbook Apache? Apologies. So if you're in the cookbook directory, use Chef generate cookbook Apache. Do you have everyone a second for that?
42:42
So the first question we need to answer is did our Chef client complete successfully? Fortunately, this is the easiest thing for us to test. In order to do this, we need to verify that, one, we have a place to store the cookbook artifact, two, that we have a Chef client with access to that cookbook,
43:03
and three, that we have a target server which is running the same OS as our production server, either is running or will be running. So to do this, we're going to use a tool called Test Kitchen. Test Kitchen is a delightful tool that allows us to spin up a VMware container,
43:21
configure it with our Chef code, then verify that that VMware container is in the state we expect it to be in after our Chef run is complete. It's created and maintained by Fletcher Nichol, who also works at Chef. So one of the nicest things about Test Kitchen is that we can test our Chef code
43:40
on multiple operating systems at the same time. I can test whether the same cookbook will work on both Ubuntu 12.04 and CentOS 6.4. I can then define which recipes in the cookbook I want to test and test it on both those platforms. If I have multiple recipes within the same cookbook,
44:01
let's say I have a default recipe in my Apache cookbook, and I also have a recipe for configuring SSL with Apache, I can test them on both platforms using Test Kitchen. I'm in another operating system and one of those same suites on that operating system. Isn't this delightful? I'm not.
44:24
So we configure what operating systems and what testing suites we want to test using our .kitchen.yaml file within your cookbook. So go ahead and open up the .kitchen.yaml file within your Apache cookbook directory that you just created.
44:45
And then there's that file. Let's go through it step by step about what it does. So the first thing we do is we declare what driver we want to use. This defines where Test Kitchen will tend to spin up that instance to test our cookbooks.
45:00
Now this can be vagrant. It's a very common one if you want to run everything locally. It can be a cloud provider. I tend to use Digital Ocean just because it's nice and fast. Then next is our provisioner. This is how we declare how we're going to simulate a Chef client run on that Test Kitchen instance. When we use Test Kitchen, those Test Kitchen instances
45:21
are not connected to an actual Chef server. So we can use either Chef0, as we do here, to simulate that Chef client, or we can use Chef Solo, which is a way of revisiting Chef nodes without using a Chef server. So I like to use Chef0 because it runs in memory on my Test Kitchen instance, and it runs nice and fast.
45:45
So next thing is platforms. Here we define what OSes we want to test our Chef code on. And then finally we come to suites. Suites are ways to group recipes to test. The run list of a suite defines which recipes will be applied
46:02
to our Test Kitchen instance when we run that suite. Now attributes allow us to add in values we expect to find on the server, but can't know until the server is actually spun up. An attribute I like to use a lot is IP address. When I'm spinning up a server on a test kitchen instance on a cloud provider, chances are I'm not going to know
46:21
what the IP address is going to be until I actually spin up that server. So I provide a mock IP address as an attribute, and that way I can test that my Chef code finds that IP address and handles it in the ways that I expect it to. So a test suite can run multiple recipes. So we can have our default suite,
46:42
which will run our default recipe. We have another suite which will test a recipe called SSL, which will configure SSL for patching. Finally, if I want to add another operating system, it's as simple as adding it in under platforms. Let's say I want to test my code on Ubuntu 14.04
47:02
along with CentOS 6.4 and Ubuntu 12.04. Things behave a little bit differently. Certain things do on Ubuntu 14.04 versus Ubuntu 12.04, so it's good to know that my cookbooks work on both. So I'm going to pause here for a second. Is everyone still with me?
47:21
No, you don't need to type. Are you in your Apache cookbook directory? OK, did you open up .kitchen.yaml? It doesn't have anything in it? It does, but you've got to have it now. All right, hold off on typing for right now. We'll be typing in just a little bit. I'm sorry I wasted some keystrokes for you. Oh, no, no, no.
47:43
All right, so make sure you're still in your Apache cookbook directory. It should be if you just looked at your .kitchen.yaml file. And now we're going to edit .kitchen.yaml. So we're going to first update our driver. All the cool kids are using Docker these days, so we're going to be cool.
48:00
We're going to use Docker to run our test kitchen instance. And then we're going to leave the provisioner to Chef Zero so it stays nice and fast. And for right now, the only platform we're going to test this on is Ubuntu 12.04, so go ahead and take out the CentOS line and Ubuntu 14.04 if you have it in there.
48:21
Finally, we're going to define our suites. So this should already be in there. We're just going to run the default suite at first, and we're going to run the Apache recipe within that default suite. So I gave everyone a couple of seconds to get that typed in. Is it necessary to have that apparently?
48:42
It's auto-generated. I don't think it's necessary to run to the cookbook, though. So you can take it out if you want to. All right, so raise your hand if you're still typing.
49:01
Raise your foot if you're still typing. It's our only way to ensure that we stop typing. Raise both hands if you're still typing. All right, so go ahead and save and quit that .kitchen.yaml file. Hit on the command kitchen list,
49:21
and go ahead and hit that magic Enter button. Now when I run kitchen list, it shows all the test kitchens we have already configured. In this case, it's only one. We're only running our cookbook on one operating system and only using one recipe. If I were to add in back in Ubuntu 14.04 or CentOS 6.4,
49:45
or whatever it was, you would see that in this list. It would show each of the possible combinations you could run, OS and testing suite. So with that, we're testing suite in .kitchen.yaml,
50:02
and it looks like we have a question. Yes, sir? I'm getting an error that ownChefChefRefo.kitchen.yaml does not exist. Okay, I'll go ahead and submit for an overview. Is anyone else getting that? You get it unless you're going to get the actual drug. All right, you get it unless you're going to get the drug for your pet. All right, so we've defined it in our .kitchen.yaml,
50:28
but we still have to actually create the Docker container to run our tests into. So you do this by running the kitchen create command. And this is going to take a few minutes, so I'm going to go ahead and oh, it's already started running it,
50:42
and let's give it a little time to run. Are there any questions while this is running? Yes, sir? You should create the things surrounding, for example, if you specify a program provider, it will run a program. Exactly. Yep, it will run the spinup maker boxes.
51:02
In the meantime, we have one recipe on our run list right now. I'd like to show you the community MySQL cookbook and all the recipes that are in that run list.
51:23
And you don't need to follow along with this if you don't want to. There's a supermarket, here's your MySQL, and let's look for their source list. It runs. It runs on.
51:57
Bump that up a little bit. But they want to make sure the MySQL cookbook will operate correctly on.
52:03
And then here are all the sweets. And it just keeps going and going and going. So your .kitchen file is the master file for your tests, essentially.
52:22
All right, go back to the slides. Nathan, has your kitchen create completed? My kitchen create is not yet complete. Oh, right. It's on step nine. Okay. Doctor tells me which steps it's on. Which is pretty nice. Just so that we all know, sort of, if I'm faster or slower than you.
52:42
Kitchen is finished. My kitchen is done. All right. It sounds like we have a question there. Yes, sir. Does doctor tell you how many steps there will be? Question is, does doctor tell you how many steps there will be? I do not know that answer. Does anyone know that answer here? Yes, it does. When it says kitchen finished, you use your scroll bar to scroll back up and you look at the last step.
53:01
That's the way it is. All right, now that we run kitchen create, Yes. Let's take a look at a little visual now that our kitchen is up. So we have our workstation. That's what your SSH did to you right now. And we're running a Docker container with our kitchen, that sample operating system that we're going to test our Chef cookbook on.
53:22
And that's running in the workstation itself. One of the nice things about Docker is you run containers within your own workstation. If you install Chef DK, that sets everything up for you. I don't think Docker comes with Chef DK. You need to install that separately.
53:43
If you want to use vagrant, it will set up everything so you can use vagrant. But the test kitchen comes with that? Test kitchen does come with the Chef DK. All right, so if your kitchen create is complete, one of the nicest things about test kitchen is that we can log in to that container we're running
54:03
and run commands for that container. So go ahead and run kitchen login from your workstation. And when you're prompted for a password, use the word kitchen.
54:27
Big one. Oh, and you would specify the instance. You would specify kitchen. When there's only one, you only have to do kitchen login. When there's more than one, you have to do kitchen login, Ubuntu, Forteo4, this test suite.
54:41
So that's where kitchen list comes in very handy. All right. Yes, sir? Was Docker already installed on this? Or did it just install Docker? We pre-installed Docker. Docker needs to be installed separately from the Chef DK. All right, so now we're in that Docker container on our workstation.
55:03
Raise your hand if you're not seeing this command prompt. All right, looks good. Oh, certainly.
55:21
So let's demonstrate this visually. Again, we have our workstation with our container running, with our kitchen container running kitchen inside our workstation. And what we did was we ran kitchen login from our workstation. Kitchen login SSHed into our kitchen container. And now we're able to run commands from within that kitchen container.
55:47
So we still need to verify that our Chef client runs successfully. Remember that we need a target server running the same OSS production and a Chef client with access to that cookbook. We now have that target server set up. We just need a Chef client with access to that cookbook.
56:03
So the first lab we're going to do, our problem for this lab is that we have not applied our policy to our test environment. So test criteria means the default Apache recipe will be applied to this test kitchen environment. And by environment, I mean the test kitchen container we configured. So go ahead and run exit to get out of your test kitchen
56:23
and make extra sure you're still in your Apache cookbooks directory.
56:40
We're going to run the command kitchen converge. Go ahead and put that command in and hit that magic enter button. Now kitchen converge will install Chef if it's not already installed. On the test kitchen container, it will install Chef.
57:00
It will also upload all the cookbooks and then apply the run list we configured in our .kitchen.yaml file. So I'm going to go ahead and let that run. Nathan, let me know when that's complete. Are there any questions while that's running? So our couple of hands kind of go halfway up.
57:22
Why this command isn't around when you are doing kitchen grade? Why this last command isn't around when you are pressing kitchen grade?
57:42
It's around, but why doesn't it do it right when you hit kitchen grade? Because sometimes people want to just spin it up or create the kitchen first before they install their Chef recipe on it. It's a little quicker that way. It only doesn't create a command. You could run create and then converge. Later on, I'll show you a command where you can run all the kitchen commands at the same time.
58:10
When our kitchen converge runs successfully, we can answer that first question. The Chef client run did indeed complete. So congratulations. You've just completed your first test-driven Chef run
58:21
without running any tests or Chef code at all. So now let's move on to the next question. We need to verify whether Chef put our node, the server, in our desired state. So we could do this through manually testing our container.
58:40
So go ahead and run kitchen login again. Ask for kitchen, and that will get us into our test kitchen instance. So one manual way to test whether Apache is working is to run curl localhost.
59:07
So within your test kitchen instance, go ahead and run that command. Hit the magic enter button. And get back a successful response.
59:28
That would mean Apache was running. But when we run this, we get this error back. It can't connect to the host, which tells us that Apache is not running. Another visual just to remind you where you are.
59:40
It's sometimes easy when you're working with containers to lose sense of where you are, whether you're in a workstation or container. Remember, we're working that curl command from inside our test kitchen container. So our next lab is we need to manually verify the state of the test node. Well, the problem is that manually verifying the state of the test node
01:00:00
is tedious and error-prone. Our success criteria for this lab is that the end state of the node will be able to be automatically tested. No, I'm sorry. Yes, sir. I think selecting is the point of less exercise. Is there a way to get that to work correctly? Not yet. We're going to make it work. OK. It's not that we did it incorrectly. No, it's not that we did it incorrectly. It's to test it, setting up the failure first. OK, good.
01:00:23
So to use our automated test, we're going to use a tool called server spec. Server spec allows you to write tests to verify the state of your servers. And it's not just limited to Chef. You can use it if you're using Puppet, if you're using Ansible, Salt, just about whatever have you. And just like Chef, it defines resource types
01:00:41
that we test whether that resource is in the state we expect it to be in. It works extremely well in Test Kitchen. And if you want to get an idea of all the sorts of resource types you can use, go ahead and go to that URL, serverspec.org. So if you're still in your kitchen instance,
01:01:01
go ahead and exit out of it, which will take you back to your workstation. And make extra sure you're still in your Apache Cookbook directory. Now, when we ran Chef generator cookbook,
01:01:22
Chef TK automatically created a server spec test file for our default recipe. So go ahead and open it. It's test slash integration slash default slash server spec slash default underscore spec dot rb. It's a bit of a mouthful. Are you testing the previous slide? Absolutely. Just making sure that you're in your Apache Cookbook
01:01:40
directory. So when you open this file, if you're an RSpec user, this syntax should look very familiar. Raise your hand if you don't have this file open yet.
01:02:02
Is server spec also going to be installed with DK? Server spec is automatically installed with DK, yes. So let's take a closer look at that syntax. Just like RSpec, we're using a describe block to define the subject of the test. In our case, it will be the default recipe of our Apache Cookbook. We use it blocks to describe what we expect that subject
01:02:23
to do or be doing. And we use that same expect syntax that RSpec uses, later versions of RSpec, at least, to define what we expect the results of our code to be. So let's add in our real test. Yes, sir, three. Does the describe string need to match with DK,
01:02:40
or do we just do that for? You know, I don't know on that, actually. But I can find out. It doesn't need to match. But when we put in the generator, we had to pick something, and matching that seemed reasonable. You could totally change it, too.
01:03:01
All right. So let's add in a real spec. The describe block, the one that describes the default recipe of our Apache Cookbook. Let's add in a test to ensure that our recipe is awesome. When our recipe is awesome, we are awesome. So go ahead and add that in. We're using it is awesome.
01:03:20
That's our it block. And we're expecting true to equal true.
01:03:41
A couple more seconds to get that in. Go ahead and move on. Unless anyone raises their hand.
01:04:04
So save and close that file. Then from your workstation, run the command kitchen verify. Actual tests on our kitchen container. Now, this is going to take a little bit to run at first. But ultimately, we should see a success, that our recipe is indeed awesome.
01:04:23
So I'm going to give that just a second to run through. It always takes a little longer the first time you run the test. All right.
01:04:47
Do you have an exception? It's because you're writing code on your phone.
01:05:05
Do we have a question? Can you open that test again? Absolutely.
01:05:23
Go ahead and give me a thumbs up when you're ready. Money and zines up here. Oh, thank you.
01:06:09
All right. Let's go ahead and move forward. So we ran kitchen verify and saw our test pass, ensuring that our recipe is indeed awesome.
01:06:22
Therefore, we are awesome. But this isn't a real test. Science interaction exercise. What would you test to make sure that Apache is running on your test kitchen instance? Curl localhost, that's one. There's a few others.
01:06:42
PS, yep, that'll work. Which Apache? Check the package with your G package. Yes, sir, in the blue. Check the process file, the PID file. Excellent.
01:07:01
All right. So for now, we're going to add in a test to make sure Apache is installed. Open up your server server file and add in this test at the bottom, which is checking whether Apache 2 is installed. We're using an expect syntax, just saying that we expect the package Apache 2 to be installed. Server spec takes care of the rest for us.
01:07:46
Give everyone a few more seconds to edit that in. That's good.
01:08:06
Hearing less keyboard clicking. If you'd like me to wait for going on, go ahead and raise your hand. Was there a hand in the back? Okay. So let's run kitchen verify again, and this time it'll run much faster,
01:08:22
and we'll also see a failure, telling us that Apache 2 package is not installed. And this is good. It's always good to see a failure when we first run a test. So you probably already know this, but just like our application code, we test our infrastructure code by writing that test,
01:08:42
watching it fail, writing just enough code to make it pass, then repeating. So we've written a test and watched it fail. Now let's go ahead and make it pass. And to do this with Chef, it only takes one line of code. Go ahead and open up your default recipe. If you're within your Apache cookbook directory,
01:09:01
that's recipe slash default dot rb. And that one line package we're defining our Chef resource, then in quotation marks, Apache 2.
01:09:34
All right, after that, we're gonna converge the node again, so go ahead and run kitchen converge.
01:09:51
Sure. Then after you converge, converge supplying that Chef recipe to our node.
01:10:04
Then you would run kitchen verify and watch that spec pass. We have a question. You can do your kitchen list. I don't know if I'll tell you whether you haven't converged yet from your previous converge,
01:10:21
but if you haven't converged at all, you'll be able to see that way. All right, Jim.
01:10:44
It's because you're using a full-size keyboard instead of a phone. All right, we're starting to run short on time, and we want to make sure we leave plenty of time for Q&A, so I'm gonna go ahead and speed through the next slides.
01:11:00
Let's go ahead and skip ahead. All of this is available in the GitHub repo we showed. So your kitchen workflow is doing kitchen create, kitchen converge, kitchen verify to actually run the test, then kitchen destroy, destroy that test kitchen instance or container.
01:11:21
I'm gonna speed through this. Oh, I'm gonna skip ahead. Test kitchen is wonderful, and test kitchen server spec are wonderful for integration-style tests,
01:11:42
but the problem is you can see you have to converge your changes, rerun those on the node, spin up the node if you haven't spun it up yet, just to get back your feedback. With Docker, this takes about two to three minutes. With a cloud provider, it would probably take even longer. So there's another testing framework we can use, and this one specifically for unit tests.
01:12:01
It's called ChefSpec. This allows you to test before you converge. So if I were to write my code and just have a typo in that code, it would fail and let me know about that typo before I did the actual converge. We're not gonna run through that exercise, but feel free to check out the GitHub repo
01:12:20
and run through that on your own time. And there's one more framework I want to introduce you to. ChefSpec is just RSpec, so you run it just using RSpec spec. And that is Food Critic. So sometimes after we've verified
01:12:40
our integration test work, our unit test work, we also want to verify whether it follows our style guidelines, whether we're using any deprecated features of Chef or Ruby, and whether there's any syntax errors in it. So Food Critic is what's called a limiting tool. It doesn't actually execute the code, but it does go through the code looking for errors.
01:13:02
So a way we would test this is changing our package name to be a string called packageName, setting that to Apache 2, then trying to interpolate that string into another string when we actually declare the package resource. Food Critic would tell us, hey, the string is promotion. It doesn't actually break anything,
01:13:20
but it's not required, and it makes the code unnecessarily complex. So it's a nice layer on top of your code, a way to check that front layer just where your code looks. And with that, I'm going to skip ahead again. What we would have done is install Ruby on the system, then configure that full Rails application. Please take a look at that GitHub repo
01:13:41
and configure it on your own time. And now it's time to wrap up, and I'll turn it back to Nathan. All right. Thanks, now. So what I want to do, I have some wrap-up slides that I'm not going to use. What I'm going to do instead is just jump over to that GitHub repo really quick and give you the two-minute 50-cent tour
01:14:00
of that repo. It's not on there. It's not on there. No, it's not on there. But that's not on there. So this repo
01:14:27
has a bunch of branches here, as you can see. You can probably see it better than I can. But it basically walks you through each branch and builds on the previous branch. And at the end, you'll have the full working application.
01:14:41
So there's a lab called Widget World Ruby that has a codebook with all of its unit tests and integration tests that will install Ruby 2.2 on that particular machine. From there, you can do Postgres, passenger, get Apache installed, and then finally a working Rails application. All of these work pretty well
01:15:02
with a cloud provider. They won't all work with Docker as your test kitchen, though. When you install Postgres in the default Docker containers, it's not going to work so well. So I'm actually using Digital Ocean locally, and when I go through into the labs Widget World application,
01:15:21
what you'll see there is that there's one this is fun to I'm using that. It's all good. I can deal with it. So there's this Widget World application that will actually pull a Rails app from GitHub, put it into the machine, run the db migrate, and at the end, you'll have
01:15:41
that application that you can pull up, which is pretty cool. What I'm going to do is look at my screen and type the URL into the R over there while you ask questions before we wrap up. Actually, before you start asking questions, Nell and I are going to host a birds of a feather session at 8 o'clock. If you want to talk more about Chef
01:16:01
or more about DevOps, please join us for that. It's in a room. Which room, you might ask? I might ask the same question, but if you look at the birds of a feather It's AB. AB. Nell knows. She's smarter than me. So the birds of a feather board is out there, so you should check that. And now, while I'm doing something
01:16:21
over here, you can ask questions. Please do. If there's anything, we've got a couple more minutes left. Yes, ma'am. A quick summary of what makes Chef better or why you like Chef better than Puppet?
01:16:40
I can answer that personally. I personally like Chef better than Puppet because Chef takes a developer view. I came from a development background. SysAdmin did not make any sense to me until I found Chef because I could create my Chef recipes with the same Ruby code that I used and in the same kind of developer mindset. Puppet very much comes from the SysAdmin mindset.
01:17:01
It's usable, but to me it's not intuitive. Also a nice thing, even Puppet is also written in Ruby. You write your Chef recipes in actual Ruby so something like a block or in each loop that will look like an actual Ruby in each loop. As far as Ansible goes, Ansible works well for small deploys.
01:17:21
I have nothing against it. Chef works very well at scale. That's why we have Facebook as a customer because they run tens of thousands of servers and other customers with massive infrastructures. It's a great way to have that Chef server be able to deploy small changes out to your entire fleet. Testing frameworks, you can use them with the other frameworks
01:17:41
that are really much more of a Chef, which is nice. Yes, sir? I went to a talk and I came to make the impression that a lot of the tools that I already knew I didn't really have anything to talk about.
01:18:02
It looks to me like things have settled down. There's sort of one official line in how to do all these things. You will find any sort of framework. You'll find people who do it a different way. Most commonly, stuff at Chef we use server spec and Chef spec and test kitchen most commonly.
01:18:25
Some of them are not open source but we try to maintain contact with the open source creators and make sure that we're working together on it so we don't break their stuff and break our stuff. Server spec is created by a Japanese
01:18:42
developer. I don't remember his name off the top of my head but if you go to the server spec website you can probably find out. Any other questions? Yes? Right. I will turn to Nathan for that one. Sure. If you want to remove a resource
01:19:01
the way that Chef works is it manages the resources in your policy and it will make sure that those resources follow the desired state of each one of those resources as laid out in your policy. So for example, if we were to change our Apache recipe and just take the package Apache 2 out of that recipe Chef will
01:19:21
stop managing that resource which means that if it's on the system it's going to be on the system. Chef isn't going to do anything with it. However, Chef remember resources also have a desired state. So your desired state could be that that resource does not exist on that system or is removed from that system. So while the package has a default
01:19:41
desired state of install it also has a desired state that you can specify. It's called an action. And that would be to remove it. So you could say that the action is to remove it. And then Chef will follow the same test and repair. Is the Apache package installed? If so, remove it. If not, our following policy can continue on. So we are at time. I want to respect
01:20:02
everyone's time. However, Nathan there's nothing in here after us. We're happy to hang around a little bit if you have more questions. We'll also be at the Chef booth tomorrow. Both of us if you want to talk Chef. Talk Rails with Chef. Whatever you like. I like bunnies. You can talk about bunnies with me. That's right. We have Chef stickers and little info cards up here which you can also get
01:20:21
at the booth tomorrow. Thanks very much. Thanks for meeting me.