Kubernetes & Habitat
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Title of Series | ||
Number of Parts | 45 | |
Author | ||
License | CC Attribution - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
Identifiers | 10.5446/34592 (DOI) | |
Publisher | ||
Release Date | ||
Language |
Content Metadata
Subject Area | |
Genre |
ChefConf 201716 / 45
1
4
5
9
12
16
18
20
21
22
24
25
27
28
35
37
39
42
00:00
Internet forumLattice (order)BlogReading (process)Presentation of a groupLaptopInformationProduct (business)Data conversionMultiplicationOpen sourceRight anglePoint (geometry)Multiplication signForcing (mathematics)Degree (graph theory)Operator (mathematics)Software developerInternetworkingFamilyProjective planeInheritance (object-oriented programming)Computer scienceLink (knot theory)JSONXMLUML
02:17
2 (number)Entire functionRule of inferencePresentation of a groupMathematicsSlide ruleVideo gameSpherical capLevel (video gaming)VideoconferencingAutomationBitEuler anglesEnterprise architectureElectric generatorRight angleVacuumFamilyDifferent (Kate Ryan album)Coefficient of determinationKnowledge-based configurationGoodness of fitInternetworkingComputer-assisted translationTouchscreenConfiguration spaceData centerState of matterRow (database)Category of beingDialectOnline helpComputer animation
05:03
Electric currentShift operatorAreaFocus (optics)BuildingRight angleSoftware developerMultiplication signLink (knot theory)Proper mapComputer programmingCartesian coordinate systemConfiguration spaceComputing platformTwitterNewsletterInstance (computer science)SpacetimePhysicalismMaxima and minimaVirtualizationLine (geometry)Human migrationEmailData managementComputer filePerspective (visual)Level (video gaming)Different (Kate Ryan album)Open setProduct (business)Traffic reporting1 (number)Goodness of fitQuery languageCoefficient of determinationService (economics)Point cloudScaling (geometry)Physical systemMetrePoint (geometry)Peer-to-peerStatisticsAreaArchaeological field surveyBitForm (programming)Channel capacityComplex (psychology)Inheritance (object-oriented programming)Cloud computingOperating systemType theoryInsertion lossCondition numberLibrary (computing)System callCategory of beingSound effectDirected graphDigital librarySolid geometrySocial classFile formatGame theorySummierbarkeitStandard deviationSoftwareOptical disc driveInternet service providerVirtual machinePressurePlanningMedical imagingQuicksortComputer animation
12:57
Interface (computing)Source codeIndependence (probability theory)Run time (program lifecycle phase)Computer configurationInstallation artConfiguration spaceTime domainOperating systemCartesian coordinate systemSoftwareEnterprise architectureOperator (mathematics)Loop (music)WritingComputing platformRight angleLibrary (computing)AreaDisk read-and-write headLine (geometry)Computer architectureBuildingService (economics)Standard deviationServer (computing)Perspective (visual)Configuration managementConfiguration spaceLaptopCycle (graph theory)Device driverMultiplication signSoftware testingSource codeMedical imagingQuicksortPatch (Unix)Programming paradigmIndependence (probability theory)BitProduct (business)Bus (computing)File formatPhysical systemDifferent (Kate Ryan album)Run time (program lifecycle phase)PlanningDistribution (mathematics)NeuroinformatikLimit (category theory)Social classVideo gameGroup actionEmailComputer configurationFormal languageLatent heatIntegrated development environmentScaling (geometry)MathematicsPoint (geometry)Figurate numberArmInterpreter (computing)Directory serviceSound cardComputer fileEndliche Modelltheorie1 (number)SpacetimeLevel (video gaming)Computer animation
20:51
Binary filePhysical systemService (economics)Vertex (graph theory)Data managementComponent-based software engineeringMetric systemSet (mathematics)DemonData storage deviceDefault (computer science)Scheduling (computing)Radical (chemistry)Fault-tolerant systemProcess (computing)Demo (music)Spring (hydrology)Sample (statistics)BootingBlogWorkstation <Musikinstrument>Term (mathematics)Core dumpTask (computing)UDP <Protokoll>TopologyPoint cloudLoginEmailInstance (computer science)Interior (topology)Time zoneStandard deviationRight angleService (economics)Context awarenessIntegrated development environmentDifferent (Kate Ryan album)Form (programming)Cartesian coordinate systemFlow separationComputing platformPeer-to-peerNumberMereologyIP addressGamma functionNetwork topologySemiconductor memoryBitData managementConfiguration spaceTraffic reportingArmRadical (chemistry)Endliche ModelltheorieQuicksortFigurate numberLevel (video gaming)Video gameSet (mathematics)Process (computing)Instance (computer science)Point (geometry)InformationSoftware developerScaling (geometry)Connectivity (graph theory)File formatSlide ruleDemo (music)Link (knot theory)CodeBootstrap aggregatingLibrary (computing)Computer fileTheory of relativityDependent and independent variablesLine (geometry)Variety (linguistics)Ring (mathematics)Wave packetAreaInstallation artBinary codeCoprocessorGodGoodness of fitBuildingFunction (mathematics)Loop (music)Disk read-and-write headHydraulic jumpLatent heatDirection (geometry)WaveSpacetimeLoginSinc functionDemonOperating systemComputer animation
28:46
Meta elementTopologyUDP <Protokoll>Service (economics)DemonRead-only memoryBefehlsprozessorCheat <Computerspiel>Ring (mathematics)WindowDemonComputer animation
29:19
BlogProxy serverDialectServer (computing)Set (mathematics)DemonMaxima and minimaError messageFlynn's taxonomyDuality (mathematics)Convex hullNumberGroup actionSlide ruleService (economics)BitRing (mathematics)LoginData managementDependent and independent variablesDemonStability theoryComputer animation
30:13
BootingScale (map)Service (economics)Local GroupDemonPattern languageComputer-generated imagerySelf-organizationComputer filePoint (geometry)Multiplication signNetwork topologyDatabaseType theory1 (number)Medical imagingService (economics)Demo (music)FlagCartesian coordinate systemDifferent (Kate Ryan album)Ring (mathematics)Peer-to-peerRight angleTheoremSpherical capResultantLine (geometry)Data managementSelf-organizationLastteilungWordInterpreter (computing)Polarization (waves)Declarative programmingWeightExtension (kinesiology)Computer animation
33:28
DemonSupremumTopologyService (economics)Peer-to-peerSpring (hydrology)Sample (statistics)Demo (music)BootingProxy serverServer (computing)DialectPoint cloudPlastikkarteCore dumpBlogGateway (telecommunications)Configuration spaceDefault (computer science)Process (computing)Read-only memoryMaß <Mathematik>Inclusion mapTemplate (C++)Replication (computing)Partition (number theory)Computer networkPasswordSynchronizationAuthenticationMotion blurExecution unitDivision (mathematics)Right angleService (economics)Variable (mathematics)Line (geometry)AuthenticationComputer fileRing (mathematics)Integrated development environmentMultiplicationConfiguration spaceCartesian coordinate systemPoint (geometry)Instance (computer science)ResultantNetwork topologyData managementLoop (music)Connected spaceMappingDependent and independent variablesDirectory servicePower (physics)Existential quantificationMathematicsRevision controlPhysical systemWindowCodeWeightState of matterSeries (mathematics)Form (programming)MereologyDifferent (Kate Ryan album)Moving averageFigurate numberIRIS-TAsynchronous Transfer ModeRoutingMobile appStatement (computer science)FlagComputer animation
39:14
TopologyPeer-to-peerConfiguration spaceProcess (computing)Cache (computing)Replication (computing)BlogPersonal identification numberSynchronizationRead-only memoryService (economics)Extension (kinesiology)Server (computing)Error messageExecution unitCAN busDivision (mathematics)Maxima and minimaNormed vector spaceSelf-organizationComputer fileTerm (mathematics)Service (economics)Mathematical analysisPersonal digital assistantCartesian coordinate systemPhysical lawRight angleScaling (geometry)Instance (computer science)LoginRing (mathematics)Computer animation
40:20
Service (economics)Gamma functionMaxima and minimaComputer wormJukeboxVariable (mathematics)Configuration spaceIntegrated development environmentRight angleRepresentational state transferComputer fileCartesian coordinate systemLine (geometry)BitPoint (geometry)QuicksortTopological vector spaceMereologyComputer animation
41:34
TelecommunicationDirected setOperator (mathematics)Lattice (order)Internet forumDemo (music)BlogQuicksortOperator (mathematics)Service (economics)InformationProjective planeDirection (geometry)Multiplication signTelecommunicationFile formatAreaCodeOrder (biology)Ring (mathematics)Electronic mailing listHacker (term)Time zonePoint (geometry)Data managementConfiguration spaceLink (knot theory)Computer fileComputer animation
43:40
JSONXML
Transcript: English(auto-generated)
00:05
So I'll start by telling a story because that's what you're supposed to do in presentation school, right? So we had Kelsey who I guess kind of stole my thunder this morning. I Will not be deploying it. Well. I will try at the end to deploy something via my phone. We'll see how it goes
00:22
But the interesting thing that he said was you know when you walk up to the booth And you see someone and you automatically assume they're in marketing because they're in the booth, and that's what they should be right And of course a lot of you are just like oh great product marketing that's gonna come and tell me something technical, right? Let me make sure this person's not in the room
00:43
So I had an individual last night at dinner, and we he's like what do you do? And I'm like I do product marketing and I mainly focus on habitat and then we start getting into a conversation and we're talking about the thing and way habitat works and some of the things that we've been doing and Other more technical topics, and he's like wow you're really technical for someone who's in marketing
01:04
And I'm like well. Yeah, I do actually have you know multiple computer science degrees And so I've actually worked in operations, and I've done a lot of the things that I'm going to talk about So even though I have this title of marketing, please you know let's all include each other as brother and sisters here
01:21
in the technical world What was that? Idea you don't look at my get-up All right so a couple links for you of course a lot of people have laptops open you might have your phone open If you're any point in time you want to kind of look up something that I'm talking about or learn more about Habitat these are kind of the best places to go and look for information about habitat
01:45
Start by joining the habitat slack team. It's a very active community Every thing that we do development wise is developed in the open so all of the development work that people you've heard this week Adam Jamie Fletcher And everyone else they're all working in public in this slack team, so please join it and ask any questions
02:05
And then once again, it's a totally safe community if you're a super beginner And you don't know feel free to ask and don't be intimidated because there's a lot of things that we may not necessarily know yet Either it's a kind of a relatively new open source project So At I was at Kuba Khan and at Kuba Khan they
02:23
Someone talked about how there's different generations of technologies And she compared the different generations of technologies from Like the broom is like when we used to configure systems by hand on bare metal right anyone do that Hopefully like two of you Wow
02:41
And then the vacuum cleaner is like where we have VMS And it's a little bit more automation, and then you start using something like shuffle puppet To automate the deployment of those VMS and configuration of them And then the Roomba is kind of like kubernetes and containers And it's just magic and automatic and it's beautiful and your life changes and your life changes kind of like this So everyone's saying have a cat, right?
03:03
So have a cat is Something that we've come up with so this is kind of like have a cat just running around and that's enterprise IT being like hey You shouldn't be using containers or kubernetes or whatnot and have a cat just like get out of my way And then you know you you'd start using it more and more and the the experience is really good
03:22
So it's really powerful and people want to follow you around and you get a little bit. Maybe a little bit of an attitude and you start dressing up as sharks and Then of course enterprise IT sees that what you're doing isn't going to go away And so they kind of build you a container solution
03:41
That's not really a container solution, but they they dress it up like a container solution But automation is not all it's cracked up to be So this is a real video from the internet I'm just taking screen caps because it's easier than playing the video ruin the second rule of presentation club is you don't show videos
04:00
in your presentations, so There's a dog and of course the dog isn't being wasn't maybe wasn't let outside that morning and it's having a little bit of an accident and Of course as dogs do they're inspecting it which is all important for all of you good colon health is something That's very important. I've had family members with colon cancer, so you always should look down when you're done
04:24
and here comes automation and The room is just doing what it's told right and it's going to go clean up And I was showing these slides to our CMO Ken Cheney who you saw earlier on the main stage And he's like I totally have had this happen to me before so like this is real this happens
04:42
It's more common than you think and then the automation is just going to go pick it up And then the next thing you know your automations went and took and spread shit all over your house All that being said not to knock on kubernetes or containers, but if you're not careful It's very easy to go and easily spread shit throughout your entire data center
05:03
So and this is kind of encapsulated by something that we've been talking about all week Is this idea of this learning cliff right where in development containers seem really really cool And really awesome, but then when you go and actually try and deploy it out into production That's when you get into this learning cliff of containers, and there's all these complicated things well not complicated
05:24
There's a lot of these things are being solved and have started being solved as containers become more common but there's all these things that you have to think about right and One of the biggest ones is this idea that containers aren't a VM right this is kind of one of the foundational
05:40
principles that Companies like Docker and so forth have preached for a really long time and Kelsey gave a great example of Treating the container as a VM when he showed how you went and built that container and that Docker file on the main stage Right Kelsey has another great example that you can look it up Where it's building a minimal container image?
06:00
And that's basically where you have about four lines inside that Docker file, and you're not including and shipping a whole operating system So containers aren't VMS are the you know the best practice, but are you really sure about that? So how many people are using containers just like they use the M's and the room you don't need to raise your hand We don't need to shame you in front of your peers
06:23
But you can look at a few statistics So I forget where the first one came from I want to say the first one came from a survey that Right scale did it was either right scale or data dog And they looked at you know usage of containers of their customers and They found that there was about a four to one container to host ratio
06:43
So what does this tell us a four to one container to host ratio if anyone's done any form of capacity planning? Them's VMS. Thank you Nick. I could I knew I could rely on you. I've got a 20 for you later And then another one which comes from somebody by the name of Gareth Rush Grove
07:05
If you don't know who Gareth is Gareth works for puppet But Gareth is a really nice guy I was looking I was looking for a little bit of more like Because we're feeling sorry for Gareth, right Gareth does some excellent work in the DevOps space if you don't follow him you should totally follow on Twitter
07:23
He also does something called DevOps weekly where he sends a newsletter out Every week with kind of like top links and stories in the world of DevOps and tooling and other stuff like that So definitely check out DevOps weekly. He did a talk at config management camp and his talk at config management camp
07:40
He was basically doing github queries so all of github's data is up in BigQuery and you can query it And do different reports and stuff like that and what he was looking at is that he basically went and found That 75% of containers Container a full OS. I just realized there's a typo there, but I read it the way it was
08:02
So containers container a full OS right and so like are we using containers the way they're supposed to be Probably not no But the thing is about the way the container ecosystem works And really I guess kind of with Docker files in general, and I've always had a little bit of an issue with Docker files
08:22
Mainly because it put us back into the world of bash scripting again But then of course what I'm about to show you with habitat puts us back there as well. I Think we've accepted our fate at this point But it's super easy to just pull in that full operating system that you really don't need anymore in a containerized world So how can we start to begin to build containers better?
08:44
And then when we build these better containers, how can then we ship them out to actually go and run them? And if you think about it There's a new program that a very popular container company just started that's called modernized traditional Applications and the modernized traditional applications are all about
09:02
Lifting and shifting your technical debt for fun and profit, so they're just packaged up who's ever done a p2v migration Physical to virtual migration right and what did you do you picked up all that crap that was on that physical machine? That's been Accumulating for the last two or three years and you moved it onto a VM because that was somehow going to be good
09:24
for you No offense to VMware, I think there's someone with a VMware shirt back there But that doesn't actually solve the actual problem of managing that system any better than what you managed it from To begin with and then what you end up having is you're just going to have more and more and more VMs and your
09:42
Complexity problem just grows and grows and grows So let's talk about what modern applications mean so that's kind of some of the problems set in the container world Let's talk about modern applications and what we mean by modern applications So modern applications need to basically be service or the way you Communicate and work with a modern application. It should be API first right, so it shouldn't be some
10:06
particular tooling that you can't expose the API in a very You know rest based API is kind of the standard And that should be your interface in and out and then you build your tooling after you've already went and built your API to interact And manage the application right this is a very common thing that you see with a lot of cloud
10:24
Providers right where they're going to go and provide that API and then they backfill in and provide the SDKs the interface In or the CLI that interfaces in with that And the thing that you have to think about when you look at this world at API first is it's all about consumption
10:41
So what you're trying to do is you're trying to actually solve an economic problem to make it easy to consume, right? And that's why API's are great because if you have a very good API that can be accessed programmatically It's easier to consume it and then thus it's easier to use that service in the proper way, right? You shot a small area concern so for each individual application
11:02
You should try and minimize the footprint for what it's actually managing and trying to do right And this is the one of the things that you can see in micro services based principles, right? They should be ephemeral or they should be able to be Managed in a stateless way and of course we see this with things like platform as a service cloud foundry other services like that
11:22
Of course, don't be fooled. You always have to go and store data somewhere a Lot of platforms make you think that your data just where you they don't answer the question of where your data goes So your data needs to be stored somewhere Hopefully accessible via an API and that would be something like maybe a data service that you get from one of your cloud providers
11:42
like RDS for instance and Then they should focus on artifacts and the building and deploying of artifacts And so Kelsey talked about Something one time at a conference. It was called container world and his opening line was is you wouldn't have a conference called RPM world
12:00
Would you? Let that sink in for a minute, right? Like containers are just a packaging format, right? It's just a way to generate an artifact, right? I was gonna say you wouldn't have something called VM world. Would you and I was like, well wait you would All right, and so kubernetes if you look at some of the foundational principles behind kubernetes and habitat
12:24
It's really about this idea of running modern applications, right So what do we mean by that? So habitat is built around this idea of API first. So How many people think that an API is just limited to how you communicate with something over a network?
12:43
Good nobody raised their hand what if we always forget that the first like API's were in like your header files Right, and that's actually defining an API of how you interface with a library, right? And so from an API perspective you have API's when you look at What you do with configuration management, right?
13:02
And when you write resources in chef or when you write testing and inspect that's essentially writing towards an API or writing towards a spec Right and so with habitat we do it from two perspectives So in the very first thing that we do in habitat is we define an API of how you build your software artifacts And then the second thing we do is we have an API of how you run those artifacts in production
13:27
And then thus we focus on building those artifacts and this is really important and I'm gonna dive into this a little bit here In a second, so I'm gonna skip that for now And then the other thing that we do is we eliminate the dependency on the operating system And this is really important because when you go back to this idea of how you should properly run a container
13:45
Where you're not pulling in a full-fledged operating system, but minimizing your user space as much as possible So you just are able to do troubleshooting things like Kelsey mentioned on the main stage today It's very important to think about how we can start to eliminate the operating system because the operating system is
14:01
The source of most of our problems how many people have to patch servers Why do we have to patch servers? Because the way that we've defined or designed the operating system paradigm and the application paradigm So this boils down to this idea of operable application containers, and of course I didn't start my timer
14:23
All right So yeah, I don't know if I'm pacing myself right though so Thank you So it boils down to this idea of operable application containers or even operable applications right so applications should be isolated from one another
14:42
Basically meaning that if you modify or do work on that particular application You're not necessarily going to destroy the entire stack right so if you think of the Netflix model of things turning on and off based upon the availability of that service and sometimes you log in the Netflix and you don't get recommendations, or you don't get the I always hate it when they're like continue watching is like four lines down
15:03
I'm like why is continue watching not always up at the top, and then I understand how the architecture of Netflix works, and I'm like They should be immutable so the application artifacts themselves should be immutable, and this is one of the areas where chef and Kind of the traditionalists in the container world might butt our heads every now
15:23
And then and when I say chef the company because even though the application artifact should be immutable Which should still be mutable is the configuration of the application right because the configuration of the application is going to change as you promote something Through different environments right and what you don't want to have is you don't want to have container images for five different
15:42
Environments right and having to rebuild that container image because you're baking a lot of configuration into that container artifact We should have a common API for monitoring health We feel that you should rebuild from source, and you'll see this Using a common packaging technology and then having runtime independence and when we say runtime independence what we mean is compute runtime
16:04
so just like we now have different language runtimes like JVMS and Ruby interpreters and node VMS as well you also have different compute runtimes now right so we have bare metal We have VMS you have cloud-based systems you have container based systems
16:22
And then within containers you have a lot of different container formats as well And so you should be able to take this artifact that you generate and run it on any one of those platforms with these So who's ever done this? Like I like I've been I was thinking of the last couple days as I was waiting until the last minute to write my talk
16:45
About some of the problems that we solved in habitat, and I was also Talking with someone about like when did you get started with Linux and what was your first Linux distribution? And so like I would want to say it was either 97 or 98 so it's either my 19th or 20th anniversary of using Linux
17:03
No applause And I got started with slackware And then I also played around with gen 2 right and you kind of go through that cycle of figuring out what distribution you want to play with But then when you wanted a piece of software And this was probably most common with like things like audio drivers at the time right and like trying to get your audio card working
17:24
On your laptop was well, what was moved me to a Mac as soon as I had OS 10 But you kept following this cycle of you would download the package you'd untar it you would some Some logically correct asshole right now in the room is saying he forgot the CD
17:46
And then you run dot slash configure, and then usually what happens with dot slash configure It fails and why does it fail because you're missing a dependency of some sort What?
18:01
Because I didn't CD into the directory. I knew there would be one in the room It fails because you're missing the dependency of another library right, and then you're like shit, okay I gotta go find where the header files are for this particular library And then you download that one and you install it and you put it in the right place you actually do this exact same cycle For that library you just downloaded and then you find another ones missing and then probably when you went to go download one of them
18:25
You would realize you were missing a whole bunch of other stuff that that depends on right and you know then your wife Which my wife is here in the back is wondering why you were up until 3 a.m.. Working on playing on your computer
18:42
And so that's essentially what habitat does is one piece of habitat Automates that cycle right and when it automates that cycle if we think backed about like Why are you stuck in that cycle? You're stuck in that cycle because you're using an operating system and the operating system vendor has a release cycle and
19:02
As soon as you decide that you want to use that particular operating system You're tied to the vendors release cycle right how many people still manage Red Hat 5? Nobody wants to raise their hand oh there we go So how many people still run Red Hat 4 right? It's because you have some sort of dependency in your application on something in the operating system
19:22
And usually what happens of why you get stuck in that loop is because there's newer software that you want to use That your operating system vendor doesn't have the features that you want and if you're running on something like Red Hat 6 Well usually or Red Hat 7 or even a lot of other platforms. What's the first thing that you're going to install is?
19:40
You're going to have the apt cookbook configure Apple right because you need newer software It's extra package for Enterprise Alliance And that's because you need newer software to run your application and when you think about what actually goes into building an operating system Or building a system that runs something is you have the operating system vendor provided libraries you have any
20:03
Configuration changes that you want to make to what the operating system lays down you have application specific Libraries that you're going to install and then you have the the application itself, right? Habitat really kind of flips this on its head So with habitat what we can do is basically what we call deterministic builds and determine X to the build of containers as well
20:25
And so we've we've gotten rid of this blurred line between what is an operating system shipped library And what is an application shipped library? Because it's it's crap and it really needs to end right and there was a reason why we had it at that point in
20:41
Time, but it creates so many problems of managing systems at scale that it Really, it's just not worth it for us to go down this path anymore So with habitat what you do is you define your application you define what that dot slash configure? Make make install loop is for your application, and then you can go and once you've built that artifact
21:02
You can say you want to export it, and you can export it from Habitat package and you can export it into a variety of different formats And I think that's on my next slide no my next next slide so the idea is that You want to try and move on this sliding scale as much as possible to the left
21:21
I think I'm pointing in the right direction. Yes to the left And so basically if you think about what goes into a container You want to try and minimize as much as possible how much you're putting in in the operating system space Right and you want to try and keep it as much of application specific Code or libraries as possible, right?
21:43
Now if you look at the example that Kelsey gave if you're using something like go And you can compile to a statically linked binary Then there's very very little than you that you need in that actual container itself, right? So with habitat what you can do is?
22:01
Post-processed packages, so there's a whole bunch of different more than a whole bunch of different I guess it's three a whole bunch of different There's a few and I'm looking at my wife because she's always likes to clarify the definition of a couple and a few and some And we have kids so like that comes up often So there's a few
22:21
Is that right? There's a few different post processors So what you can do is you can take that habitat binary and what habitat will do is basically do this so it'll figure out What dependencies does that application artifact need? It'll say what transient dependencies does the dependencies need and I'll walk that dependency tree for you
22:42
So you're not going and downloading all these tar balls and figuring out what goes with what? It does it automatically for you It'll take the supervisor and then walk the dependency tree for the supervisor and package all of that up And that's the process that we saw on the main stage today, right? So I assume everyone saw that so I'm not going to go and do it again
23:01
They're kind of explaining what that output was that we saw and then it'll package that all up in your exported format, right? So then I can take that and I can go and run it on any platform that I want Another one that we have that's not necessarily listed here is a tar ball So you can also export it as a tar ball
23:21
So you get everything that you want and then you can have this artifact that's locked That you can then go and ship and have all the dependencies you need in that one particular tar ball So what we think of when we think of habitat and kubernetes and how not just kubernetes But things like mesos and swarm play with one another is that habitat is
23:42
Super good as I just talked about For building that container right and generating those application artifacts that you would then go and take to run on something like kubernetes or mesos or and the like the way that we think of the separations of duties is that Habitats responsible for the application itself and
24:03
Configuring the application and standing it up and then kubernetes is responsible for the underlying infrastructure that you need, right? And by infrastructure, we mean things like kubernetes concepts such as replica sets and pods and the services, right? Then make an application that you can actually do something useful with
24:22
So what do applications need to run? So let's talk about what we think applications need to run and then I'll jump in and I'll give a little bit of a demo Hopefully my demo goes as well as the one this morning Hopefully the demo gods are with me if not, we'll punt and we'll figure something else out. All right, so
24:41
If you think about what applications need to ring it need need to run is they need awareness Right and they need awareness of a couple different things. They need awareness of the environment that they're in And they also need awareness of their peers, right? So they need to know who's like me in this place and What is the running configuration of that thing? That's like me, right?
25:03
Because if you're like me, I should probably have the same configuration Most likely there might be one or two lines in your file in a config file That would might be unique to the particular host But generally speaking you want all of your application instances to be configured correctly, right? Everyone can agree with that Yep So what you need to be able to do is form this ring where you can communicate and have the application instances
25:26
communicate with one another And so what we did is we came up with this concept of creating the daemon set on Kubernetes And this is what this daemon set allows you to do is basically form this ring So as applications come up They have a common endpoint that they can go and talk to and find out information about what's running in that environment, right?
25:47
It all depends upon how you run Kubernetes. There's I don't know are they in? Yes, there is one in the room that I know of So there's a company in the Netherlands and what they do is they basically give, it's Schubert Phyllis. Schubert, wave your hand Just one of you, Christian's not in here
26:03
So what they do is they basically have a Kubernetes cluster and they give that Kubernetes cluster for development teams, right? And so this would work really well in that you have that daemon set come up and anyone who's running Applications on that cluster and since it's per dev team, it's a pretty isolated environment They have this common bootstrapping point to bring up their application instances. So let's see what that actually looks like
26:27
Sorry, so that's what a daemon set is and then Habitat containers as a daemon set is what I just described So sorry, I always get ahead of myself on my slides because I have them memorized in my head
26:42
All right, so You can see here. Let me just go over to Sublime. I have this folder and I'll have the link to this code at the end And so I have this thing called a Habitat Management Supervisor and if I look at what that YAML looks like
27:01
So we're gonna create this service called Habitat Management Supervisor. I don't know why I didn't shorten soup Supervisor, but I shortened management and as I've been typing this out, I was like, oh I should have shortened that as well So if I go in and look you can see what's going to happen is is we're going to spin up this container And it's one that we've built with Habitat and it has just basically what we call an empty supervisor in it
27:25
and then the other thing we're going to do is we're going to give it an endpoint of This Habitat Management Supervisor service that we created Then the other thing we're going to do is we're going to bring it up in a leader follower topology Let me just do this real quick. Is that 15 minutes or six minutes?
27:56
Like six minutes. How do I only have six minutes left?
28:00
Okay So I should see this service now are these pods up. Yep And then I can also go and look at the logs as well But probably the thing that's most interesting is if I do this
28:21
compute I don't know why that didn't work. I think because this terminal has been open for a while
28:40
Okay I just need to get one of these IP addresses just because I have how the service is running and then I can go and look at this Okay, so this is the supervisor that's running so I brought up a ring of supervisors
29:05
I can also go look over in Kubernetes as well and I can look at my daemon set so you can see there's my daemon set That was an old window. I told you it wasn't going to go as well as it did with him
29:25
All right And you can see the daemon set that we spun up and I can go and look at the logs of this as well And then I can also go and see that it's coming up and it's having a little bit of stability problems But it'll fix itself here in a second. And then I can also go and look and see
29:44
Look at the actual ring itself. So if I go and look at butterfly I Can see that sure enough I have a whole bunch of members and I can see the health of each one of those Members, so I spun up eight and I have eight in my management ring, right? So this is what you want
30:01
And then you can also see what services are running as well service groups and other things like that So this is the API that habitat gives you to interact with these services so now the second thing that I want to do is Let me go back to my slides real quick Right. So this is the idea of having this management service So this is going to run and now as I bring up applications
30:21
You notice I had that peer flag as I continue to bring up more applications I can go and just point it at this ring and they'll be able to find their peers very easily and then bring Up their application correctly, right? So what else do applications need to run? So the other thing that applications need to run is Applications aren't always deployed where they all are equal, right?
30:44
sometimes when you deploy applications you have leader follower type topologies that you need to go and Deploy to of course, we all know that they were called originally master slave type scenarios But we don't use that words anymore, right? So leader follower type topologies and what you do with leader follower type topologies is
31:05
Somebody needs to become the leader or the master or the one that's going to write all the data usually in a database type Scenario and then the other ones come the followers of the replica sets Right or the replicas where you can actually go and read the data from at any point in time And of course we use that for scaling we use it for a whole bunch of different reasons
31:24
A cap theorem is probably the main reason why we do that So when we talk about self organizing applications with habitat The way that you can do it with so the way it's traditionally done is that you typically bake the intent into the container
31:41
So a good example of that is if you look at the kubernetes get book application If you look at the kubernetes kind of demo application the guest book application you have a redis container. That is Declared the leader That's explicitly like baked into the container that it's the leader and then explicitly baked into the other container image that it's the follower
32:01
Right, so you have two container images, of course, you know you think to yourself. Well, it's only two container images It's not that hard to manage, right? But do you just always have two container images that you have to manage? No, you're typically going to have hundreds of container images that you might have to manage and then dust where you're gonna have to do is build a build lifecycle around
32:21
Those container images and so forth, right and so it introduces this concept of image sprawl So what habitat allows you to do is basically build one container image You can launch container images with knowledge as appears and then the application will self organize So they'll automatically do an election and they'll say I need to be this person and I need to or I need to have
32:41
This role and then I need to have this role. I'm not going to say that they're sentient and that they're people And When you do this though, you need knowledge of the application organization So kubernetes has a great thing in it called services which basically allow you to front in different application services
33:00
So API driven again, so like very easily via an API it's declarative You can go and put a load balancer in front of all of your applications with a couple lines of yaml And that's what we did when we created that have management supervisor service So what you can do is when that election takes place you can actually notify kubernetes that the election took place and then
33:21
Go and configure those services accordingly based upon that the results of that election So let me show you this real quick so if I go and look at my redis example So in this yaml, the first thing we're going to do is we're going to create a service
33:40
you notice that The name is redis master. That's not what I want Sorry, I believe this is the one I want. I had one that was modified. Sorry. Oh because I'm on the wrong
34:00
Anyone know why? No, I'm on the wrong sublime window. This is like someone else someone else's code and then this is my code which I Changed it to the leader follower. That's why I knew I was in the wrong file So I create a service for the followers, and you notice we have this thing called app
34:23
What application I'm running and then role so this called a selector so kubernetes will do is figure out who I should route traffic to Based upon how I put a label on the pods and if you want to think of what pods are pods are just Simply analogous to containers But you can have multiple containers running in one pod that was the example that Kelsey showed earlier where you had a pod
34:43
With multiple containers and one was a sidecar container inside of it And then we have one for the leader you can see that we have that selector on it as well And then the other thing you can see is we have a deployment So we're going to deploy in a leader follower topology The peer is going to be the habitat management supervisor ring that I created earlier
35:03
And then I just need to pass in some environment variables as well So what this will do is we'll actually turn off protected mode and redis Which you probably don't want to do you probably want to have authentication or something so don't actually do this at home Well, you can do it at home. Don't do it at work All right, so all I need to do is run kubectl on that
35:37
So very easily it creates it for me, and then I can say
35:43
kubectl get pods And you can see there's my redis pod, so let's look at the logs real quick And so you can see what's going on is that there is a connection
36:02
so the first thing that happens is The supervisor starts so what happens is is when you export a container using Habitat what is the first thing that you as I said you have the supervisors that's exported so PID 0 is going to be the supervisor You can see that the supervisor is running and you can see those lines there
36:23
And then the supervisor is recompiling some things and updating things and then it realizes that it has to Do an election and so it's waiting to reconfigure the application and so as a result of the election It goes and reconfigures the application according to
36:40
what it's specified in the config file and the way you do that is you go and to the right directory and So let me just show you the configuration file So I have a couple different configuration files that I use here and that Habitat will regenerate for me So the first one is the redis config and if I go and look for
37:02
Right here you can see what the lines are that basically says that if I'm a follower So this is all you have to do in your config file, right? So you basically just says service me follower or you could say service me leader. It'll go and write out this particular line If I'm a leader then I'm not going to go and write that line
37:23
Right, and then the other thing that it does is that it'll generate this configuration file for me as well So it says if my role equals leader and if my role equals far if my if I'm a leader role equals leader And it's like I can't read my own if statement else. I'm going to be a follower right make sense pretty simple
37:43
Right, and then what will happen is is kubernetes or and I'm sorry habitat will automatically generate these files Based upon that election that you saw took place All you have to do is specify that dash dash topology leader flag and have your configuration files written Automatically and those kind of distributed system problems that you have to think
38:01
about when you're deploying an application will be taken care of for you So if I go over to kubernetes now Well, I can refresh butterfly and if I refresh the butterfly endpoint here You can see that I've got more members. How many members should I have now? I launched five rediff instances Plus eight would be 13, right? I just seen that you're awake
38:24
Getting people to do math wouldn't you know and then I can see the service that I have running here Let me make this larger for you You can see this hab This this redis service that I have I can see all the members of this service as well I can find out what configuration they're running what version of the application and so forth, right?
38:43
Now over in kubernetes what I can look at is if I go and look at the services You can see there's the leader and follower services that we created via the YAML and What has happened is when the application starts up? Yeah Redis instances will automatically go and tag themselves correctly in kubernetes based upon that file that we generated that labels file
39:03
Right, so you can see I've got one here. That's the leader If I go and look over at the follower, you can see everyone else have identified themselves correctly as the followers, right? So what I can actually go and do here is Modify this YAML and I can make Tim. I know I'm supposed to be doing this via Google Assistant
39:26
Don't tell Kelsey I'm a failure And it's complaining actually I need to replace And the deployment was replaced the services weren't that's fine. But if I go and refresh this I
39:50
Now have my additional instances that just came up and they were automatically added they joined the ring automatically So if I go and look at their logs You'll notice that they've joined the ring and they came up and they came up as a follower because there was already a leader
40:07
Creator right so I can scale my application like that if I go and look at the dashboard I can see that as well Does that make sense? So let me just show you real quick how that works So if I go and look at the wrong file and then the right file
40:25
I created inside of my habitat package for K8s Redis Basically, I have this line right here. So if I have this labels file Then I want to cap those labels into a variable and then I just run kubectl on that
40:42
And I want to say I want to label the pod of pod name and you notice when I went and launched I Set an environment variable called pod name, so we're just using that pod name environment variable So I told kubernetes to please label myself based upon the configuration that I just generated
41:01
Alright, so the other one which Kelsey kind of mentioned and I just wanted to expand on it a little bit I don't actually have it implemented So we have this other risk-based API and you've seen me hitting it and with this rest API You can ask questions and one of the questions you can ask is is the application healthy? So all you need to do is add into your YAML a
41:22
Stugment like this or a stanza like this Where you're going to hit this endpoint, so all Habitat applications Would have an endpoint like this so I can go and ask for the services and
41:42
I can see a list of all the services that are running and then for any one of those services you can drill down And get specific information about that service, so it would just be Let's have And I know we're almost out of time
42:02
And I don't have any configuration, so it's actually not showing in me anything, but you can also ask other questions So this is all on the habitat documentation. Let me get to a wrap-up point Okay, so what else do we need to do? So some things if you wanted to work on there in the hack day if you're familiar with kubernetes So it'd be really awesome if we had an export format for kubernetes to where we can generate the necessary YAML that we need
42:25
Maybe generate any other services as well So that could be something that you could easily contribute into habitat Also having some sort of direct communication between the kubernetes API and habitat supervisor So when a container comes up it has all of the information of where that kubernetes API is
42:43
So you can more easily ask questions. You could also do the config thing that Kelsey showed this morning and then also something for a third-party operator Which we still are kind of up in the air of what that would actually look like But if you have suggestions I'd be happy to talk to you about that because that's a project that I'm going to start
43:01
Figuring out how we get underway to get development up or behind If you want the code for this demo or for this YAML files You can play around with it in the concept of a management supervisor ring. You can get it here And then also once again any of the useful links so we're out of time for this room
43:22
If you have questions, there's the habitat zone upstairs And I'm going to package pack up my stuff or package up my stuff because I'm so used to this habitat And I'm going to head up there. So if you have any questions, I'm happy to answer any questions while we're up there Thank you. Thank you