Habitat in Production

Video in TIB AV-Portal: Habitat in Production

Formal Metadata

Habitat in Production
Title of Series
CC Attribution - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
The Habitat Supervisor is responsible for deploying, managing, and choreographing running Habitat services. This session will explore a number of the operational concerns that the supervisor enables. See how to manage secrets, store configuration changes in version control systems, update running applications, and choreograph application upgrades. This is the talk for anyone who is ready to run Habitat services in a production environment.
Point (geometry) Slide rule Information
Service (economics) Mobile app Service (economics) Distribution (mathematics) Run time (program lifecycle phase) Weight Determinism Run-time system Revision control Web service Software Videoconferencing Configuration space Reaktives System Configuration space Physical system Row (database) Physical system
Execution unit
Laptop Time zone Multiplication sign Staff (military) Right angle Quicksort Image registration
Point (geometry) Topology Keyboard shortcut Mobile app Scheduling (computing) Group action Building Implementation Run time (program lifecycle phase) Service (economics) State of matter Multiplication sign 1 (number) Control flow Inequality (mathematics) Replication (computing) Web service Strategy game Bus (computing) Flag Software testing Booting Traffic reporting Task (computing) Computer architecture Form (programming) Covering space Default (computer science) Key (cryptography) Block (periodic table) Keyboard shortcut Bit Database Cryptography Connected space Process (computing) Ring (mathematics) Web service Network topology Configuration space Finite-state machine Right angle Quicksort Arithmetic progression
Web service Group action Service (economics) Different (Kate Ryan album) Physicalism Bit
Keyboard shortcut Group action Building Run time (program lifecycle phase) Interior (topology) State of matter Multiplication sign View (database) Combinational logic 1 (number) Function (mathematics) Mereology Subset Web 2.0 Medical imaging Web service Strategy game Different (Kate Ryan album) Moving average Flag Error message Physical system Mapping Keyboard shortcut Stress (mechanics) Shared memory Bit Instance (computer science) Type theory Message passing Arithmetic mean Process (computing) Ring (mathematics) Configuration space Website Pattern language Quicksort Whiteboard Arithmetic progression Boiling point Point (geometry) Mobile app Service (economics) Virtual machine Graph coloring Theory Rechentafel Revision control Goodness of fit Term (mathematics) Booting Computer architecture Default (computer science) Default (computer science) Matching (graph theory) Information Gender Surface Counting Database Line (geometry) Limit (category theory) Cartesian coordinate system System call Symbol table Word Software Integrated development environment Personal digital assistant Web service Network topology Calculation Strategy game Library (computing)
Point (geometry) Slide rule Dataflow Game controller Divisor Computer file Multiplication sign View (database) Gene cluster Division (mathematics) Control flow Web browser Product (business) Subset Revision control Frequency Mathematics Sign (mathematics) Web service Strategy game Term (mathematics) Automation Software testing Gamma function Covering space Default (computer science) Key (cryptography) Mapping Bit Instance (computer science) Cartesian coordinate system Public-key cryptography Graphical user interface Data management Process (computing) Integrated development environment Personal digital assistant Logic File archiver Video game MiniDisc Lastteilung Summierbarkeit Quicksort Boiling point Spacetime
Group action Scheduling (computing) Run time (program lifecycle phase) Code State of matter Multiplication sign Direction (geometry) Execution unit 1 (number) Mereology Web service Different (Kate Ryan album) Symmetric-key algorithm Physical system Personal identification number Constraint (mathematics) Computer file Bit SchlĂĽsselverteilung Variable (mathematics) Public-key cryptography Type theory Message passing Process (computing) Ring (mathematics) Telecommunication Order (biology) Configuration space Pattern language Text editor Encryption Quicksort Arc (geometry) Asynchronous Transfer Mode Point (geometry) Server (computing) Functional (mathematics) Electronic data processing Service (economics) Divisor Computer file Canonical ensemble 2 (number) Number Revision control Workload Goodness of fit Term (mathematics) Operator (mathematics) Ring (mathematics) Integrated development environment Integer Configuration space Absolute value Booting Metropolitan area network Default (computer science) Default (computer science) Time zone Key (cryptography) Content (media) Line (geometry) Directory service Cryptography Uniform resource locator Integrated development environment Personal digital assistant Web service Password Compact Cassette Revision control Charge carrier Video game
right and so that this is going to be in a talk the I hope people will find a value but if at some point you like but it's like a lot of information going by just no I'm also doing this knowing that there's gonna be recording of this later so if all you heard was like there is that we think we're retirement leader that had a leader election and just know that you build a pick this up after the fact and all that the slides and everything up so and so I'm going make some
assumptions in this talk 1 is that you are a kind have an idea what habitat is I'm going to further make the assumption you're ingenious talk just immediately prior and that doesn't necessarily have to be true but you to recording watch video 1st and and all weight and then come back and in and in my talk so and that's that's gonna work and so on and do what is habitat and so it's a runtime system and for your apps through the services it gives you a way to inject real-time configured at to service it is a reactive system that can react to other services and configuration to kind of do the right thing for the behavior and it and packages after that determines the build system and there's a distributed build service to help reveal those packages and it loves your apps and services and that's that's habitat I think I hope that the record for the quickest and least explanation version of at so the alert here
is and that that is can again be
will will level said that some that might of made some sense and and and
so I think this is from Jurassic Park like
like I know this I know units like so we're just going assume that I kind of know habitat and
and there's like the supervisor and everything in the
gossiper and so on it's not it's all right if you're still kind of the follows the
sinking and you could just think of it as if team had 40
more minutes and and we keep talking that's
kind was going to do here and so having said that if I a m in danger of going
too long or there's too many questions or if it's if it's easier for you to sort
of see it working on a laptop please come to the Habitat zone that so when you
went to the registration desk is just behind that down the hall and on the on the rights of the little contact away I told there's beer and so I'm I'm really banking on that because that's where I'm going to go immediately after this and I will just stay as long as I need to answer questions and we've got that basically staff for the whole time with I'm
including some schedule like toxin them also and so on the cover here are some kind of things you need to think about the things it and if you know about them you kind of know what the habitat building blocks are when you're composing architectures and is the runtime so all talk about our asses topology is the service bindings and update strategies release channels and so the key thing cryptography stuff and service configuration and and and that's it and so poor topologies when we were talking what apologies and were talking about are and if I had an an app and it's all the same package that I'm going so maybe it's a real that and if I'm putting it in the same bus service group kind of by definition of that there they all kind of cluster and self men and will work together to do what they need to do and so there are there are some kinds of apps that if if your stateless or if it's sort of the shared nothing architecture where they you know each of those cluster members can work independently and then actually our our topology strategy is not a ton of use free which is great because like you prior to getting good at that point and so there is a topology what this is when you start the service you give the service a topology under the supervisor then knows how to run so it the stand alone is our our kind of implicit default the topology so if you've ever kind of done have stock reports of something going just know that you're running on a stand-alone topology so if you put other ones and their supervisors knew about each other and form a ring and you have a stand-alone apology and if you need a little bit more than that and this immediately becomes useful when the state starts to matter or when you have an app it's that kind of AP were 1 needs to be different from the rest so if it's a database maybe you need 1 that to to to take the rights and the other 1 could take the reads and or you may be I have a little service that's basically a while doing a periodic task and that's cool and you might want you don't running just to make sure that the task happens but you only ever won 1 of them running at any 1 time and this is where like that leader topology can be a handy and all you're really doing is adding this test topology flag with leader and then and all the services that are running again with the same the name of the package and service groups and with within the ring they're kind of form and had this leader topology so and so little dark in here we'll see how this how this goes and what what I'm what I'm gonna do here is just boots and 3 supervisors and to keep things simple each supervisors only going to manage 1 service and so when we the 1st time and inequalities in the leader topology so what that what the supervisors going to do is make sure I have a softer for that package around and then it's going to go and check to see do I have other members in my service group already and because if we want to have a leader for this group we need enough kind of voting members and so some like minimum quorum and and right now in our implementation just 3 because you can please break ties in that kind of thing and so what the surprise is going to do is I'm in the service state machine it will get to the point of knowing that OIA I other voting members in the service group so the supervisors basically going to be there in his waiting Spain you'll see a understand like waiting for former something like that and and what that means is when you boot the 2nd 1 up it will come up and search these 2 will be discovering each other and realizing OK now we have to get to a 3 we still can't continue the low and and it's only until like the 3rd 1 comes
in and now there's enough and but the members that the connection hold an election so during all this time and building and you the service it was underneath that let's say it was red as I'm at this point no red is process is started so the supervisors like waiting to make progress and so that that's service the boots has like all that needs to be successful and and if the intention was that you have maybe leader that takes the right followers there they're just doing read-only replication you don't necessarily want to have to do those things if they just gonna be restarted on a really weird state so that's kind of why we're doing that and this can always in with findings which will talk about a little bit in a little bit but this point and we can
we can supervisors can have a take of a leader election and and the details now a little bit more important because I used to build just say that the leaders will have an election and then all the a leader and what's really happening is that the supervisor itself on behalf of the service group they're going to have a leader election for those members of that that service groups all those little residences and then you will get a leader and you can kind of think of it as like this supervisor so had the crown of the leader and we're just going to
bestow upon that service so that that service member is the is the leader and which means if it by your running 2 or 3 different services on each of the supervisors there might be a leader in different like physical nodes as supervisors for each them and that's that's the idea so the yeah to so the other thing that
is but also kind of place and this and that that helps of when you kind of like brooding all these things up together is this notion of of service bindings or the the bind flag and which you talked on and is in the top previous and so there are there are words technically 2 kinds of bindings and it's the 1 that normally years setting up in your package and it said the word bind and that is now like a I would call it a required binding and all that means is and as so for this example I just have a just a fictional like the Rails application so um commonly limit allowed so that it's it's supposed to be an abacus company so like those crazy calculated things and and and habitat T we can elect puns or tongue-twisters so we've got an origin a company that I called at the have for the advocacy and habitat and it's where that does all of the abacus online stuff is going to be whether it's so have have whether I do and and I will just assume that I use the ah you ever scaffolding that that gender showed so 1 of the things that that'll do for a like a default Rails apps and if you haven't had the PG GM is it's going to detect that no you probably get a database going on your real that probably not in the boot without a database and I also know it's a PostgreSQL database and so what it does when the packet gets me is it makes a now it's a required binding of beside it was originally required binding and which meant if you started this out and you permitted that dashed that bind and the the supervisory tell you you have required binding like need fulfill and and you could think of the part before that the columns of the database of words word is kind of a it's a label or a role or it's the it's the the requirements the need of myself as that real that's the thing that I need I need database and something will provide the database and the way that we fulfill that is by telling the this line the part after the colon is a is another a service groups so that's basically like into this database cluster that is going to be your that's can fulfill union of database and and what that's going to mean is that this will work whether or not there that that particular service group exists in in the ring and which means if you haven't booted your post database cluster yet and B is for the real that concerned there is no real it's toward we find what the supervisors doing again is going out to see OK we're supposed stress 2 will not default service group there is none yet so I'm gonna wait until I actually see it and you'll see now and in a supervisor output that I don't know what the messages but it's basically like I'm waiting for the service trip to exist and because of that point we can again make the forward progress towards that being in place and and it's it's a combination of yeah of of this and the topologies the the mean were creating closer to a world where if if everything was in in habitat packages that's what you had with services that can start up and they cannot communicate with the ring and for my like when my Rails web stack with like a maybe Radisson elastic I can literally just do all of these at the same time with the supervisor but they will start the actual process until their needs are are met so until the user there are met then they will start and then they can turn around and fulfill the needs of whoever is depending on them so for me this is like a beautiful Mark Burgess promise theory but I know just enough of them because I I've read this book in a resort with a lot of running and it was really clicking and what helps is working on this stuff so it kind of makes these things of like being able to good everything and have dependent surface water loading actually work which is something I was trying to do for a long time and shaft it's the thing I kind 1 of could articulate what I wanted so that the optional it's another way when you make a package you can say that but instead of a required mind it's an optional by so what happens is if you add on the command line and just like before it's treated as the was a required by so then it'll it'll weighted mean still and in the forward progress but if you will need it and it's not going to fail or anything so this is currently the way our Ruby scaffolding is actually using the optional blind and which means you can point PostgreSQL database and habitat or if you leave the bind off it's going to and it's In in our and it hopes so before the the rails at boots it's going to keep dying with an error message saying you need to also add a host and port like configure and find those those bits of information with with what you already had to provide that means you could pointed at like an existing or new like RDS instance or if you wanna look to see your urine at team and you want trucks have thing but you already have an an acceptance like database you can just point out that database and consider like yet symbols codebase inhabit package works the same points in today's so and I think they'll probably be more useful it is optional binds note we store have and the hidden away also the way we want this is the way I wanted it an now I get half that at the end yeah you so it works
and this is this just a bit of a visual for the for the binding site and I was talking to Jamie I think he's affected my brain that like I can think of it a little bit of a power socket as well and that is as long as like the the contract of like your type data after whatever match and you can like draw that requirement down of whatever the wire to devise so and but so I guess this this would be in the middle like our our Ruby at Rails apps and it looks like we have to binding and dependencies 1 both stress in and rest so another way to think about bindings is in build system at build time you might have Bill dependencies sold you need to build something at run time you might need softer also installed alongside like libraries this is kind of like you're runtime dependencies so like uses things when I boot I'm going to go talk to them and like that's kind were bindings come and on when you're like mapping of the architecture and white board as well it's probably the lines between like your app during your database steer that that's probably like in the habitat well promises findings um so our update strategies is going to be depending on your environment you use case in your world view in your company and so we have some everybody I think on the team says we're to update strategy that's wrong we have 3 because as 1 it's not and that to counts that the default so and you can you can say strategy none we find all this means is that when when habitat supervisor is gonna run your you like packages and and search service and it's just not going up to users can keep running at an and that's pretty good default behavior and if you're if your body into like the of mutable infrastructure or um like versioning images and that kind of thing and you have no more extra work to do those bits that were laid down as you going to get no more bits at least as far as habitat packages and and you can kind just move on and however the and that's the default however if your in an in an environment where and you might be on virtual machines are bare-metal hosts or even in containers and and we we can think of of the mutability in terms of habitats after a little differently but the packages themselves are immutable atomic and just not in the container sense that like there's more bits on this so you could just think of it we just think about immutability differently so I would still argue that what we're doing could be still considered new but and the at 1 strategy is basically that the simple version that when it starts and what the supervisors going to do is just check to make sure the version I have installed is that in fact the latest 1 so make a call to the depot and check and and if it's good then it just keeps going and once it's what is running and it basically just has a like a timeout blue that's the keeps painting the photospheric is the new Virginia and I think it's something like 60 a 2nd so you know even amended to it there's a new version published and then it would we would pull down the update but and this is kind of like a Wild West and and again if it yeah share nothing in the state was this could be great they're all this going to be independent actors even though there might be in the same service here and and if you need a little bit more control and like a um uh there's a lot of to this pattern that if if it's not the like all at once it's like you non-incremental role 1 at a time or and what we kind of what we realized on habitat team is a lot of those cases kind of boil down to the lake due 1 at a time is like the simplest version of all those other cases and the the other ones are like you do percentage-wise is an incremental rollout exponentially and so those I think will probably be just b here attack tack on to this idea of a rolling strategy and so our default behavior is for rolling right now is was 1 at a time so what we've got is a history supervisors of inner ring so they're all talking to each other and we've got a but right now is that this is 1 crown and it makes maybe should be on the surface are in a different color crown this is an update later so and the when all members in that service group are in in a rolling up its strategy they have sort a different election and for this update leader in its job is just to keeping the depot to see presenting their all the other members don't have to do that but which saves us from habitat came from having an awful lot of just like pain and it saves your network and it makes a lot more sense goods we want Cornix anyways and and so it a nice new magical package shows up and whenever that time out and check happens that update leaders can see that and and then was going to do is
start this process of doing up but what it's going to do is have every other member and do its update 1st in itself last and so it's the update we it it's gonna sit tell everybody like this is the version that were organized like we have all agreed with the goal of this version and and so this would be 1 of the 1 of the non update leader members so 1 of the update followers against which and would would do its update and once it's done it kind of sky like a relay race where it's like I had the time and my job's done and I had the time after like the next member the next member go and finally the the the update leader of NGOs last and the and it just kind of continues going on so if you have this if this is kind of like more the cost of this after the service that you sort need that more controlled rollout or that involves maybe you can't deal with an you know with the at 1 strategy might be taking outages on on like those things coming up and down it's harder control so this is the least the way that the you've got more control over maybe how many health checks on a load balancer might be and In flapping in in an updating so that's kind of why we started with the simplest case we can think of I these these things are all kind of basically they're the patterns for and how to run applications and and clusters so we've we've tried to give it a reasonable name and have a student as possible and hope that we cover the vast majority cases so there could be something out there that doesn't cover like the none at once a rolling and with a little bit of you know behavior tweaker tunable and if anybody you know knows about that it would be great to have chapters were always sort of making sure were world kinase hit as many use cases we possibly can so next I can touch quickly on release channels but I think I might use a very similar slide to work at judicial before and and it's full that's small 1 here but the so in my mind the will with our update strategy flows completely a logically into uh the need for channels and so the way I think about these 2 things is is the way that like pretty much all modern web browsers work like Firefox and Chrome I think and even Microsoft edge browser works and I would imagine most the people this room have is 1 of those and I imagine almost everybody has no idea what version that is have absolutely no idea even and even across like versions there's there's stuff anymore because I don't think about it and all I know is I installed that and by default I got like the the updates so I guess I'm unlike the stable channel and it updates itself in then at direct I I thought those questions and it but I know I can change my browser and if there's like features that I want I can I can might change then I can go on the unstable channel and then I'll get new features and break once in a while so & and based on that that might be the frequency that my suffers updating so that's kind of the Union Yang that we've got with the update strategy is like the update in the poll that the channels is the way to sort of control push for the release of those those packages so and what we have in our in our people and this is reasonably new with the launch of a builder and a lot of work we had a divine since the kind of uh get this realizes something we knew we wanted and we knew we wanted a few other things so there's a lot of work and fight for this and so I channel you can basically have a channel it's like scope cheer origin so like per from for myself as as ethnical archive origin which means I already have some channels and I could make other channels if I wanted and so everybody gets like the stable and unstable and know their names that for the at that you can't change can delete so that allows us in you the user tooling and experience and some logic to have to least make some assumptions and so we can do stuff like maybe by default what you're downloading from this stable and we're uploading to at some point may be unstable so that if you need to test something of that the package a pull it into another environment you don't necessarily need to like effectively upload it life reproduction and I mean that the the fun factors up to this point that's what we've been doing and for our own service because we have channels support so sometimes uploading softer has been done with a little more thought and effort on our part because if we if we do something kind wrong everyone else is just consuming our stuff so everyone is effectively life testing what we've uploaded which is really fun but I'm I'm really happy that have channels that can help our workflow and if anyone's then it it was a chef automate user or if you've seen delivery you and that's just the old term for now automated and workflow and then you've seen those environments for things go through like the staging in union stuff you could kind of picture like a similar channel for each of those and and all were doing when things succeeding is were promoting that package and in every other package that depends on and for like 2 to run with is were promoting that from 1 channel to another so from and maybe from acceptance to production affirmed dev too unstable or on whatever it is and like really under the hood it's the same hot files the same 1 artifact and the channels it's kind of like a materialized database view this is the logical construct and to select a bunch of them so means promotion and channels are relatively free speaking like we're not we're not doubling a disk space by adding another channel would be scary but the the the nicest thing is it's so much closer to what I understand that continuous delivery of a build artifact like an actual artifacts and and that's something I'm really happy about is once you've built that thing that is the same artifact that you're moving and is bad you build the next 1 and that release candidates can like not used anymore so you're not you're not like rebuilding packages you go you just promoting them the and so I'll talk a little bit about keys and this is kind of I guess a reasonable overview for like what we have here for various key so there are a few and habitat and what we've tried to do is to do the most right thing we can do for ourselves and for users and we end at the same time not having these user being completely overburdened with key management and all that kind of thing but this sometimes stuff that you just can't boil out like again something have something or times for for these things and so we do have a few things going on here so I'm just going to say that the sum of these 2 so you can have a sense of of what's what in these all end up mapping to like it have the the thing key generate most likely and so what the origin key if you've ever built a packet before and you've probably seen a curious in the tutorial it's a thing that you generate that has your name or your company's name and really under that the cover if you've seen stage and he pair it's basically like a public secret key pair that they were generating and we're using that so that when you're the habitat packages built it's gonna sign that package with your private key and then your public is going to go up to the people so anytime anybody anywhere is gonna install that package that public he needs to be present on that host because it's going to the person is going to do is verify like fish that the sign check some kind of thing and so that it's hard to know if I can even trust this hot or not so that's kind of what we're trying to do is to build that in and not have be like the additional step that like now that the 100 packaging now I I can add the signing on to like you just there's no way not to have that and so that's where like that's that's how we're trying to do about it is right by everybody we can the end kind of as we go forward
this is where things become slightly more optional and it is based on you use case and this is where like the work becomes a little bit harder so and there's also this but I'm sorry this is an example of just generating an origin key and then I'm just calling our had package which is and using the studio a studio instance to do a build on just not entering it and this is 1 of the ways you can say I wanna use this origin key on in my studio sign against and so that the next key we
have is is is a ranking and this is kind of a and it's kind of like the password it's a secret password to get into a party and so this is a key that you would use on all your supervisors when they start and it's a symmetric key so it's what they used to like in crypt they're basically the wire and communication and has said the decrepit so like there's not a whole lot to it and and when we were talking about this and when this kind of came up this is where we sort of had to articulate the name of what the ring was because it was too hard to explain eating trip the traffic all around the whatever and so we came up with rain and the unfortunately like Beyonce's song was in my brain and it never let so whenever I generate a rink he it's it's always the answer because if if you really like hearing it put ranking on on all this but at the and and related so you set the supervisory then you're giving it this time ring flying what that's really saying under the hood is it's going to look in its key cash among others to provide house and look for that need to be on key and and and use that and so that it's encrypting basically all the the gossiping messages and so and that the last line here is using an arc in Fig apply at runtime and to inject some configuration but again because that sorta like tapping into the gossip system and time supervisors and you need the same might call secret password to get the party of the habitat ring otherwise but you don't get to inject so and that sort of got for that and serviced and going to go over the the little bit quicker because these are on these were features that are this is more there than not the user key is actually a bit more like feature for and but the idea is they kind of go together so the idea this service key is it's another public private key pair and it goes on like this the supervisor for the service is running and it's it's targeting a service group and and that's gonna allow us and in the future to use a another key pair and which we call user compare so like news and operator or as like a CI services some trust the thing I can in crypt and some configurations and target a specific and service group and do the lighting of signing with the but what is the public secret the public and undercut the other way around anyways and then Diffie-Hellman Key Exchange kind of fun stuff and so the service key is like the the server side and then and the user keys is is on the user side now we don't we actually had the command for this and I had to look at that looking at code collect item using if in a long time and I was there no wonder like it is in non wired up and editor that no it's not wired up yet because of the big uses public key is something we're gonna want the depot to fetch so that the supervisor can fetch that uses public key 1 instituted corruption and so this is kind of like part of that when we launched habitat and still doing work were still in discovery mode and still figuring out where things that are important to people what features are worth implementing which ones aren't so this is just been 1 that has come up really so we haven't really looked at it again and and all that takes is somebody to be expressing interest or 1 work on it or has an interesting problem where they work and they if they want and absent habitat incomes in their house and we can then things feature wrote and that's kind of how we've been doing this is really a with the community and kind of learning together and so I think I'm going to be a a little short on time so I will try to quickly cover this Service Configuration so the way you when I have a package I've got some defaults and that that Jamie showed for like changing the pork that kind of thing and I believe the use Fig apply so that's 1 way to like override the default and for service groups group and we basically got 3 ways and I would say this has come from least preferential to most but that's also little pin needed and so 1 way is in the 12 factors style from the environment you can you can have an environment variable in the pattern is it's just have underscore and the name of the package so mine was whether cassettes have whether this and the contents is told that that's kind of a hard part told not great at like in mind the 1 mind and that you can do it and and what you're doing is you're giving it to the supervisor when it starts and because it's the supervisors process that he that environment stuff and so I'm just making that distinction but if you happen to be if you happen to want to use this on a host running and you like and then add the so this service and then inject the environment like that supervisor process is already running you can't it's it's kind of hard to like mutate the environment of a running process and however this works perfectly in a container situation where what your starting and in that container that schedule of workload is the supervisor so like doing that the
doctor dashed e environment direction that's kind exists is further is is for that kind of use case and that's probably and that works for you and that's as bright when you roll it and the 1 that I believe we might not even have fully documented which we will we don't this questions come up and is you can also you can also write a file a or drop a file in a in a location that also will work to override defaults so it goes it always goes in the same place and so would habitat most of the past 2 where things are are always in known absolute path and that's for many many reasons but also so that you don't have to guess and so it goes in this like slash have slashed SPC which is a service directory it's it's a directory that all the running services if they need state or anything and there's like there's a market for that and then just need the package name so in this have SPC whether because I can put a user . tonal and for life again why we chose where user on prior was the best and that again think think tonal file when the supervisor starts up it will if it sees that there and that the canonical location it'll it'll pull it in and merge the thing down and there is 1 caveat right now and this file when it gets updated while the supervisors running were actually not picking up and updating it that's just frankly just an oversight and we didn't have the easy like FS watch functionality for a long time we did for that and then we get rid of it and are multi-service stuff uses some of us watching so the plumbing is all there and just nobody's got into the actually adding that so somebody feels rusty I'm and are looking for contribution and this could be a great 1 and the idea for this 1 is this place ruinous of configuration and the lecture from and then that kind of thing that if you write the file that kind of describes and what you need and that means logically prior 1 updated so that's why I put the carrier that but shifted to amend the following this and but it might not update when you expected to the 1 that actually will and will always work and is pretty nice is is ah configure apply so this is using the the ring to gossip and the configuration itself and which lets us do some really interesting things so for the last 2 but with the environment variable I need to like directly starts providing that service or at least at some point with services but in the supervisor running and with the file 1 I need a host with that directory or existing so it'll eventually go this 1 you can target a service group that doesn't yet exist and and what's going to happen the supervisor going to keep that around knowing that if somebody comes up and that service group they're going to get that is there like the state of the world and all I got from that applies to me and I applied to themselves so this allows for a really awesome behavior like having a few of these and configurations for your various service groups having at least 1 supervisor running just apply them all and they're in the ring and then you as you put the services they basically like when they but they they get their constraints it's kind like proceedings and which means again the timing problem of what is the order that I do everything this gets me close to the world i want which is like if I get 1 thing I didn't think in Fig stuff minute told him to boot up and with that like with all this stuff before this is can like the medical system that I've been wanting to have forever and like we're getting close and I'm just showing a couple ways it takes a tonal files an argument but also accepts standard and is it's a good Unix thing to do so I will often just echo my 1 thing and pipe and configure apply and the 1 thing that Jimmy did mention and and is the same term I do a monotonically increasing integer for incarnation that's just the so it there in the being different versions of configu we need to pick a winner and down so we just use an integer for that and what I've started to do is just actually is the date the units state command and all that does is that just gives me the number of seconds so the next epoch and the whatever integer types and easily handles the so this is going to be fine for like a good long time some so long as you have like few things like this if there's like a CI worker that might be doing this and then uh bumping that as long as you're not doing that for 1 service group more than once a 2nd fine and if you are and will confuse what's going on so and and I might say this for the the habitat zone it's kind of the idea of the taking those files that you could use conveyed apply and put it in a get repo and with like a um on emerged to master is the like which trouble parts changed and based on the name I can get serviced group and I will just to have something apply so it can Alexis have a version configure and for for rain i'm because it's really nice when everything's encode including configuration and so I'm going man we get I heard I heard 1 so so we it and under the heading of upstairs support for that to the habitat zone for anything and remaining and thank you very
much the pH