Automated, modularized and versioned infrastructure with Terraform and Terragrunt
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 94 | |
| Author | ||
| License | CC Attribution 4.0 International: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/45586 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Open sourceFreewareThumbnailXMLUMLComputer animationLecture/Conference
00:32
DialectGoodness of fitScherbeanspruchungLecture/ConferenceComputer animation
01:03
Operations researchSoftware developerSystem administratorProcess (computing)Point cloudOperator (mathematics)Physical systemMereologyVirtual machineScripting languageCodeXMLLecture/Conference
01:59
CodeCodeSymbol tableRevision controlError messageFormal languageConfiguration spaceLevel (video gaming)Social classVariable (mathematics)BitLecture/ConferenceComputer animation
02:29
Service (economics)Endliche ModelltheorieBitSoftware developerLecture/Conference
03:04
Uniqueness quantificationService (economics)Data modelServer (computing)Gastropod shellConfiguration spaceUniqueness quantificationLaptopService (economics)Computer-assisted translationUniverse (mathematics)Error messageComputer animation
04:00
Service (economics)Data modelNumberService (economics)Endliche ModelltheorieException handlingSoftware developerState of matterConfiguration spaceNumberServer (computing)GradientVirtual machineComputer animation
04:44
Server (computing)Data managementConfiguration managementEnterprise architectureUniqueness quantificationProjective planeProduct (business)CodeConfiguration space
05:20
CodeService (economics)CodeEndliche ModelltheorieWikiGoodness of fitSystems engineeringMedical imagingTerm (mathematics)Server (computing)State of matterComputer animation
06:27
Formal languageSubsetBuffer overflowStack (abstract data type)State of matterOrder (biology)SubsetMereologyFormal languageProjective planeConfiguration spaceSystem callPerfect groupDeclarative programmingComputer fileDisk read-and-write headForm (programming)Lecture/ConferenceComputer animation
08:12
Internet service providerInternet service providerKey (cryptography)Fluid staticsLine (geometry)Positional notationVariable (mathematics)CASE <Informatik>XML
08:48
Internet service providerUser profileInternet service providerConfiguration spaceVariable (mathematics)Integrated development environmentKey (cryptography)Parameter (computer programming)Computer fileAsynchronous Transfer ModeStandard deviationXMLComputer animation
09:43
Scale (map)GoogolComputing platformPoint cloudDirectory serviceStack (abstract data type)Random numberInternet service providerHost Identity ProtocolDirect numerical simulationSoftware as a serviceInternet service providerMereologyElectronic mailing listRoutingConfiguration spacePoint cloudIP addressService (economics)Cloud computingPower (physics)Computer animation
10:53
Variable (mathematics)Default (computer science)Wechselseitige InformationInstance (computer science)Line (geometry)Variable (mathematics)Type theoryIntegrated development environmentWechselseitige InformationXML
11:28
MereologyInterpolationPlanningRevision controlForm (programming)Arithmetic meanXML
12:06
Uniform boundedness principleInterpolationSocial classSign (mathematics)Variable (mathematics)Declarative programmingParallel portInterpolationIntrusion detection systemInformation securityLine (geometry)Set (mathematics)Computer fileInstance (computer science)Functional (mathematics)ArmGroup actionNeuroinformatikError messageIP addressGradientExistenceFunction (mathematics)
13:36
Function (mathematics)Instance (computer science)BackupVariable (mathematics)Function (mathematics)IP addressInstance (computer science)Scripting languageMereologyCodeLocal ringBitHeegaard splittingComputer fileVariable (mathematics)Data structureState of matterXMLUML
14:18
MetreLocal GroupInstance (computer science)Information securityGroup actionInclusion mapRootBlock (periodic table)Address spaceIntegral domainSource codeComputer networkInterface (computing)Time zoneInstance (computer science)Multiplication signGradientRow (database)Structural loadSoftware testingConfiguration spaceCASE <Informatik>Complete metric spaceBitPlanning
15:06
MetreCASE <Informatik>Directory serviceFront and back endsState of matterStructural loadPlanningInstance (computer science)CodeFunction (mathematics)Software testingDistanceComputer fileSource codeComputer animation
15:47
MetreLocal GroupDeclarative programmingModul <Datentyp>Point cloudBitConfiguration spaceSoftware as a serviceGradientInternet service providerCloud computingSource codeComputer animation
16:27
Modul <Datentyp>AbstractionModule (mathematics)Group actionInformation securityRule of inferenceBitAbstractionRippingMultiplicationQuantum stateInstance (computer science)Endliche ModelltheorieGradientComputer animation
17:21
Instance (computer science)Variable (mathematics)Modul <Datentyp>Module (mathematics)Source codeInformation securityRule of inferenceFunction (mathematics)Module (mathematics)Directory serviceInformation securityGroup actionEndliche ModelltheorieProjective planeComputer fileVolume (thermodynamics)Instance (computer science)Variable (mathematics)Attribute grammarGoodness of fitSource codeParameter (computer programming)XML
18:07
Local GroupModule (mathematics)Source codeModul <Datentyp>PixelInformation securityParameter (computer programming)BitDirectory serviceRule of inferenceSource codeModule (mathematics)Default (computer science)Group actionECosOffice suiteConfiguration spaceProjective planeRevision controlPixelRepository (publishing)Virtuelles privates NetzwerkWindows RegistryXMLUML
19:07
Module (mathematics)Windows RegistryModul <Datentyp>Constraint (mathematics)Revision controlSource codeNumber theoryModule (mathematics)Constraint (mathematics)CodeWindows RegistryRevision controlMereologyLocal ringXMLComputer animationSource code
19:38
Revision controlSource codeWindows RegistryModule (mathematics)Local ringModul <Datentyp>AbstractionEncapsulation (object-oriented programming)Projective planeModule (mathematics)DatabaseConfiguration spaceRevision controlAbstractionEncapsulation (object-oriented programming)Computer animation
20:28
State of matterState of matterBlock (periodic table)Module (mathematics)Remote procedure callFunction (mathematics)Data storage deviceJSONXMLUML
21:10
Configuration spaceState of matterConfiguration spaceInternet service providerRemote procedure callState of matterMereologyModule (mathematics)Function (mathematics)XMLUMLComputer animation
21:48
Wrapper (data mining)Configuration spaceModul <Datentyp>Time zoneInternet service providerVariable (mathematics)Directory serviceConfiguration spaceWrapper (data mining)Module (mathematics)Video game consoleForm (programming)State of matterComputer animationSource codeXML
22:22
Modul <Datentyp>Time zoneInternet service providerVariable (mathematics)Module (mathematics)RoutingCodeDirectory serviceMultiplication signData structureoutputVariable (mathematics)Time zoneLocal ringEndliche Modelltheorie
22:59
DatabaseDifferent (Kate Ryan album)Web applicationModule (mathematics)CodeState of matterFront and back endsConfiguration spaceType theoryComputer animation
23:40
Level (video gaming)Data storage deviceService (economics)Time zoneConfiguration spaceDirectory serviceRepository (publishing)Video gameData structureModule (mathematics)CASE <Informatik>Set (mathematics)DialectLevel (video gaming)Direction (geometry)Remote procedure callKey (cryptography)Service (economics)Configuration spaceVariable (mathematics)RoutingProduct (business)Endliche ModelltheorieData storage deviceTime zoneComputer file19 (number)Source codeXMLComputer animation
25:14
Configuration spaceInheritance (object-oriented programming)Source codeModule (mathematics)Front and back endsModule (mathematics)State of matterConfiguration spaceDynamical systemTable (information)Key (cryptography)Endliche ModelltheorieLevel (video gaming)Directory serviceRevision controlMereologyWeb 2.0Product (business)MathematicsData structureXMLUML
26:23
Level (video gaming)Product (business)Video gameWeb 2.0Module (mathematics)Repository (publishing)Revision controlControl flowLecture/Conference
27:13
Inheritance (object-oriented programming)Source codeModule (mathematics)Integrated development environmentConfiguration spaceWeb 2.0RoutingRemote procedure callState of matterModule (mathematics)Inheritance (object-oriented programming)Data storage deviceMereologyRootInstance (computer science)CountingType theoryoutputXMLUMLComputer animation
27:55
Integrated development environmentFunction (mathematics)Module (mathematics)Modul <Datentyp>CodeDirectory serviceInheritance (object-oriented programming)Cache (computing)Module (mathematics)Form (programming)Function (mathematics)Local ringSystem callMereologySoftware repositoryMathematicsComputer animationXML
28:53
Function (mathematics)BitDirectory serviceLevel (video gaming)Software developerFunction (mathematics)Module (mathematics)Web 2.0Web serviceDatabase
29:37
Variable (mathematics)Configuration spaceLatent heatIntegrated development environmentModule (mathematics)Variable (mathematics)Projective planeComputer fileConfiguration spaceVarianceIntegrated development environmentLatent heatGradientLecture/ConferenceComputer animation
30:10
Configuration spaceComputer fileDialectInheritance (object-oriented programming)Form (programming)Configuration spaceIntegrated development environmentDialectWeb 2.0Level (video gaming)CASE <Informatik>Parameter (computer programming)VarianceModule (mathematics)Inheritance (object-oriented programming)Service (economics)Computer filePlanningView (database)Data structureVariable (mathematics)XML
31:40
Level (video gaming)Service (economics)Data storage deviceIntegrated development environmentModule (mathematics)Configuration spaceVariable (mathematics)Web 2.0Level (video gaming)Latent heatComputer fileService (economics)
32:26
Modul <Datentyp>CodeState of matterCommon Language InfrastructureFlagWrapper (data mining)Configuration spaceState of matterWeb 2.0Server (computing)MereologyLevel (video gaming)CodePlotterFlow separationWritingRemote procedure callFlagWrapper (data mining)DatabaseModule (mathematics)Revision controlComputer animation
33:20
Module (mathematics)Configuration spaceInstance (computer science)Software repositoryStandard deviationHeegaard splittingProjective planeCore dumpInternet service providerMathematicsControl flowWechselseitige InformationFunction (mathematics)CodeGradientData storage deviceBitWeb serviceOpen sourcePerturbation theoryVideo gameCycle (graph theory)Product (business)Physical systemRevision controlOrder (biology)Ultraviolet photoelectron spectroscopyOperator (mathematics)PlanningSelf-organizationSoftware developerLecture/Conference
38:17
Open sourceFreewareCartesian closed categoryControl flowComputer animationLecture/ConferenceXML
Transcript: English(auto-generated)
00:07
Let's begin. I will show you Terraform and TerraGrant, two excellent tools for infrastructure automation. So, hi at first. I love to have my lectures more interactive.
00:21
So if anything is unclear, just ask in between or maybe I give you some questions. And I speak German and English. I do this talk in English because you wouldn't understand me in German with my dialect. I'm from Austria.
00:40
So you can ask any question in English or in German. That's no problem. Good. I'm Emily. Just for my comfortness, I'm a non-binary person. My pronouns are in English, sai and sier, or in German, sif or sier, if you talk about me. So just know them.
01:02
Yeah. I work at PixelArt. We are a digital agency and my job title is called Engineering and Operations. It's just titles. Basically, I do DevOps, many DevOps system administration, cloud administration, internal development.
01:24
That's the engineering part. We could have called it DevOps. Yeah, Wayne. So my everyday work is getting up infrastructure, modify them, destroy them, getting up, destroy them, getting up, destroy them. So I don't want to do this by hand.
01:42
So I have searched for tools, have learned these tools, and I always use them. Even if I have to get up one machine, I code it. Because if any developer destroys something on this machine, I just take it down and recreate it by script. A little disclaimer.
02:01
These code examples are written for terraform below or equals 0.11 and terragrant below or equals 0.18. Both tools have released a new version with a new configuration language which is a bit different, has a first class level of variables and so on. So if you take these code examples and use the newest terraform version, 0.12, you have to adapt them a bit.
02:30
So the status quo. Who is automating the infrastructure? The rest of you, have you to do something with infrastructure?
02:45
Yeah, no. The other persons, developers? Okay. Yeah. So there's this famous model, pets versus cattle.
03:00
Let's look a bit into it as an intro. So the pet service model. When you have a pet at home, what do you do with them? Pets have unique names, like Ekron, example.com. My notebook is called Ekron. It's, if I remember correctly, one of the four Greek rivers of, yeah, one of the four rivers of Greek mythology.
03:31
And your pet, they are cared, you like lovingly hand raised and cared for. Like your cat at home, you care for your cat. And if you have a server as a pet, you raise it by hand too.
03:45
You install your packages by hand on the shell. You create a configuration by hand. Yeah. And when they get ill, you nurse them back to health. Like you have some configuration error, you fix it by hand, you get it running back. All by your hand, like, don't worry, I'm from tech support.
04:04
On the other side, you have the cattle service model. Like all your servers, all your cattle are given numbers. Like MGNL, 05, 06, 07, etc. They are almost identical to each other cattle. Of course, because they are not hand raised, you automate them to grade it.
04:24
And when they get ill, you get another one. Like I said before, I destroy them, I regrade it. I just don't give any mind if something is in bad configuration state, if something is destroyed by our developers. I just destroy the whole machine and regrade it.
04:43
There are a few exceptions, like your magical unicorn pet server, which can still be under configuration management. You have projects where you have only one server because enterprise thingy and license costs of 10,000 euros, you have that.
05:02
So, you have this unique pet, but it's still graded by configuration management, infrastructure management, but I can't destroy it, because it's the only production server running. I still have to fix some things by hand. Yeah. So, we have infrastructure as code.
05:24
This is the cattle service model or your special pet. Infrastructure as code. Who has never heard about this term? Good. Just to repeat it, infrastructure as code is reproducible,
05:41
it's plain defined infrastructure, because you have code. It's defined, it has a state, it should have a state, and it's a foundation for DevOps. You can't do DevOps stuff without coded infrastructure. Imagine few servers, all configured by hand.
06:01
One is different because one of the system engineers hasn't read the wiki exactly and missed a step. So, infrastructure as code is a really important foundation for DevOps. And you gain visibility across your infrastructure, because you just look in your infrastructure code,
06:22
you know, oh, I have ten servers, or I use this image, I use these packages. And for all of this, I use Terraform. Yeah, it's really Terraform.io. You can download it, really great documentation,
06:40
and the documentation is really, really great. I have used tools where the docs are, yeah, in a really bad state, or you're down, or Stack Overflow are the documentation, no. So, Terraform, it's made by HashiCorp. Who has heard of HashiCorp? Yeah, they make Vagrant, they make Pekka, Consul, yeah.
07:04
The nicest part about Terraform, it has a declarative language. If you compare it to Chef or Ansible, which have a broad-set style of infrastructure coding, Terraform is declarative like a puppet. You define your resources,
07:20
and they define the order of execution by themselves. And the HashiCorp configuration language is like a superset of JSON. If you paste JSON in your HashiCorp file, you still have to modify the syntax, but the nice part about it, as a superset of JSON, you can write your Terraform config in JSON2,
07:43
which I use in a tool to automatically create infrastructure for tenants, I use JSON, scripted by Terraform, and it's done. Projects are raised up and down by simple API calls.
08:00
Who of you has never used Terraform? Perfect. So I give you some introduction about Terraform. Terraform has the notation of providers. In this case, I use the provider AWS. We use AWS, I'm used to it, so I use this in my examples.
08:22
In the next talk, in this lecture room, you get a Terraform talk which uses Kubernetes. Yeah, cool. So I create the foundation for you. Yeah, I have a provider, I use variables, AWS access key and secret key. I can provide the keys on the comment line.
08:44
Don't use static credentials on the comment line. So I can configure the provider only with the region, can supply the secret key by environment variables. Every provider has a good documentation which environment variables
09:01
are read in instead of the configuration parameters, or that's not really better. So it depends on the provider how you can configure it. Like AWS provider, you have your .aws credentials file, which is also read by Terraform.
09:21
So I have my access key, I have my secret key, and Terraform will read in that file. Standard AWS configuration. Kubernetes has many other configuration modes too. Yeah, every provider has its own documentation, so you have to read it if you switch providers.
09:41
Even on, yeah. Using each provider has different resources, so it's not exactly exchangeable. If you have scripted your AWS infrastructure, you can't exactly use it for Google Cloud or Azure. But the big part about Terraform is
10:01
you have many, many providers. Azure, you have an HTTP provider. I don't know what that does to you. You have a DNS provider to read in. CNAMEs, Bitbucket, Brightbox, InfluxDB if you want. Maybe many cloud providers and service tools. Like, do I see it? No.
10:23
Yeah. I was searching for a few DNS providers. I don't see it on the list bar. It's nice. Ah, PowerDNS. If you configure your infrastructure, don't want to use Route 53, but you can configure IP addresses into PowerDNS. So, with Terraform, you're not only configuring one provider,
10:42
you can configure your infrastructure across many cloud providers, many cloud SaaS tools. So, let's continue with the introduction. So, we have resources. One resource, AWS instance, called my instance. I define an AMI image, the instance types, and few tags.
11:03
You can have variables in Terraform. With default, AWS region, AWS AMI. These variables can be read in from a TFVAS file, or from the comment line, or even from environment variables in a special syntax.
11:21
But to advance in the doc, you don't really need to know this. Yeah. Terraform usage is really simple. You can plan your Terraform run, you can apply your Terraform run, and you can destroy it. I don't know, I think, starting from version 0.10,
11:41
apply and destroy has the plan integrated. Before that, you always had to run Terraform plan to know what will Terraform do to your infrastructure. But now, if you run Terraform apply, it will show you the plan, which will be executed, and then you can say yes. And it will be applied, it will be destroyed, and so on.
12:03
The most important part of Terraform is interpolation, which means, like, I use the variables to interpolate, which is then in 0.12, a first class variable, without the quotes and dollar sign around.
12:23
I can read in files and many other functions, JSON encoding, yeah. So, I have two resources defined, and these two resources, can you see the black lines?
12:41
Can you see them, or more hard than them? So, yeah, here I use these two resources again, and this is called interpolation, and with this interpolation, Terraform knows which infrastructure has to be graded first. So, Terraform will first grade your AWS keeper
13:00
and your security group, before it is grading your instance, because you couldn't grade your instance if the security group doesn't exist. And through this declarative language, Terraform computes which is executed. Some resources are executed in parallel, some resources like security group IDs
13:23
can be executed in parallel, and you can get an error, because you use the same set of IP address and port, for example, but that all depends on your resources you use. So, you can have outputs in Terraform, you need the public IP of your instance,
13:41
define the output, and can you use it? If you only change your outputs in the Terraform code, you call refresh, and then Terraform output, you don't need to apply, and the best part is you can Terraform output public IP and use it in your script, any other script, like you need to execute.
14:02
Typically, layout of Terraform are .tf files. Yeah, for Terraform, I have a main .tf where I define mostly of my infrastructure, a local, local variables, variable, state, provider, bit of split up from the Terraform infrastructure. So, this is how a Terraform run looks.
14:24
I simply run Terraform apply. This is the older recording. It doesn't show you the Terraform plan, which is executing. This is from a G-meter fleet configuration where I created, in this case, a smaller G-meter fleet
14:44
to execute load testing. So, I have called them crew and minions. It's a bit funny. Creation complete, creation complete, and two instances are still grading as its infrastructure, AWS, takes its time
15:03
to grade instances. And then, when I've done load testing, I just destroy this infrastructure. I said old recording, no plan output before.
15:22
It's refreshing the state to see if you already have destroyed your instances by yourself. Could be the case. All this state Terraform knows about is saved in a Terraform TF state file. If you don't configure a backend for the Terraform TF state file,
15:41
it's saved locally in your Terraform directory where you execute this code. You can configure a remote state too, which is recommendable if you work with teams. Okay, let's recap that a bit. Terraform is a simply declarative modular tool.
16:02
It uses many cloud providers and SaaS providers and its coded infrastructure. Easy peasy, huh? Yeah? I said a bit more interactive. Okay, let's introduce the next step because you don't want to grade
16:21
your infrastructure in one big Terraform configuration. It's not nice, so Terraform has the possibility of using Terraform modules. With Terraform modules, they are like a container for multiple resources, like the instances I had before.
16:41
They could be contained in a module or like security group rules could be contained in a model. They grade lightweight abstractions because if you grade a VPC in AWS, it can get really complicated. So you may want to grade a user module for VPC creation,
17:01
which abstracts you all the hard bits away. And Terraform modules increase overall usability, which means, okay, you have one VPC, but you may have many security group rules to use them. So Terraform modules can be graded and used in many ways.
17:22
You can have local modules, like I have in my projects. I have an internal directory and I have many modules like the data volume, the instance, I have security group rule and security group rules. These security group rules models use the security group rule, so modules can use modules.
17:42
Each module has its own files, main TF, outputs and variables, so input, output and what is executed. Local modules are referenced by the module attributes. You give them any name you want, but use a good name,
18:03
like SG for security group dot underscore SSH. Then the source parameter defines which module is referenced. I'm a bit nervous still. This is a local module, so I go one of the directory above, use security rules,
18:20
ISMP default, because ISMP for AWS has many rules in it, for echo, for fragments, et cetera. Internal modules, like something VPN, you don't want to copy it into every Terraform config. Our VPN configuration is the same for every project.
18:42
I have a great GitHub repository, Terraform pixel art VPN. It's versionized, so I can increase. We have a new subnet in our local office. I just add it to the module, push a new git tag and every project can use the 1.6 version.
19:05
The greatest is the module registry, registry.terraform.io, with public modules, verified and unverified modules. Verified modules are verified by Terraform, that they run, that they have good code.
19:20
Still, unverified modules are good too. The best part, which is not super for local modules, is the same version constraints. For VPC, as said, you want to use a module, terraform-aws-modules-vpc-aws, and I want to use every version,
19:41
starting from 2.9 below 3.0. Public subnets, private subnets, database subnets, it's all specific to the module, but it's nice to reuse public modules, which are really good. Otherwise, I would go insane, if I have to write the whole VPC configuration
20:01
in every project. VPCs are not so easy. Some recap, Terraform modules are what? Reusable, versatile, repeatable, they provide abstraction and encapsulation, because I can reuse them in every project.
20:21
It's the same one. The abstraction, like the VPC module, it's easy to configure it. So, putting them all together, I have to introduce you to a new concept of Terraform, the remote state. As said before, with the Terraform remote state,
20:45
you can save the state like in S3, like in Azure block storage, which is important if you work with a team, but you should also use that as a single developer, because if you split all your modules up
21:00
later with Terra grant, you need the remote state to reference all your outputs of other modules. And I do this with the data Terraform remote state, which is provided by the Terraform provider. The configuration really looks the same. I say the backend, and I say the config.
21:23
And with the Terraform remote state, I have a data usage to read out my outputs of other modules. And these two parts are important for Terra grant. I like that I can. So, grantwork-io-terragrant.
21:41
It's on GitHub. It's a simple Go tool, too. Go binary, download it, use it. So, what is Terra grant? It's a thin wrapper for Terraform. So, instead of calling Terraform on the console, you call Terra grant. It keeps your configuration dry.
22:00
Don't repeat yourself. It allows you to work with multiple modules, and as I said before, it manages your remote state. Let's look how you work with Terra grant. First, you need an infrastructure modules directory called infrastructure modules, or whatever you want to call it.
22:22
There we have our internal module, as seen before. I have an internal module. In this internal directory are all my local sub-modules, and I have many other modules defined. I have a module for AuroraDB, I have a module for monitoring, a module for Route 53 zone, which can be used multiple times.
22:42
And in your module, you have the same structure as you would have in a whole Terraform code. Your main TF, which is important, you have your variables, you have your outputs, because in your model, you need input and output. And the difference to writing
23:01
a whole big bunch of Terraform code, and now split it up in smaller modules, it's smaller, encapsulated, lightweight, and reusable, because if you split it up, I have an Aurora module, it doesn't have anything to do with your web application node.
23:21
It's just for your database. So they are lightweight. A difference, remote state defined in your TerraGrant module, infrastructure module, is just the backend type. The rest is configured by TerraGrant, with the main configuration we will see in the two next steps.
23:41
So, beside your infrastructure module repository, you need a git repository for that, you can define your infrastructure life directory. This directory structure is how I use it. You are completely free to define your own directory structure. In this case, I use the account name
24:01
of our AWS account as first level, then I have the regions, like in AWS there are global things without regions like Route 53 zones, IAM, and I have most of my stuff in EU Central 1. Next level is, are your environments, dev, stage, production,
24:23
and then I have the services. I have decided to use a two level service structure. So I have a direct service VPC, I have sorted it into services, storage, and below them you have a terraform.tfvas file, where your module is configured,
24:40
which module you use, which variables you give into the input, and below them, in the directory of the account name, you have your terraform.tfvas for global configuration. Starting from terragrand 0.19, it's called terraform.hcl.
25:05
The global config looks simple, looks like our remote set before, except we have the terragrand global config key. For terragrand, we now define our remote set there with the backend, with the config. The bucket and the dynamic DB table locks.
25:23
It's the same as before, but it's only defined in your one global configuration. So your remote state is already dry. The module config still uses a terragrand key too, and the biggest notion is which module you use.
25:42
You now use the infrastructure modules in your git and with a reference for your tag. And that's the biggest part. You can versionize your infrastructure. Like this, I have a web module at version 1.9.5, which is used in dev stage and production. Now I have any change,
26:01
maybe it's a PC breaking change, and I have a version 2.0.0. In dev, I can reference this version, run my infrastructure for the web, and stage production is not touched. With this directory structure,
26:21
you can increase your infrastructure by versions. Sorry? Yeah. Which means you can upgrade your dev. You can later upgrade your stage, as you may be used, but with one look into infrastructure live repository,
26:43
you know at which version your infrastructure is. You look into the dev web module. Oh, it's at 2.0. You look in your stage web module. Oh, it's still at 1.9. I should upgrade it. You look into your production web module. And luckily, I have not upgraded it yet,
27:01
because it's Friday. Maybe you don't want to do a PC break in infrastructure on Friday, but that depends on your workflow. I have include, which is relevant for Terragran itself, which means finding parent folders finds the root terraform tfvars, which gets your remote state configuration.
27:21
That's where you keep your remote state dry. And the coolest part is you can define dependencies. The web module depends on VPC, it depends on the RO storage, it depends on the DB storage, which is later needed if you want to run your whole infrastructure at once.
27:41
Of course, in your modules, you write your inputs, like this module has a count and instance type of T3 medium. Yeah, it's pretty simple. And instead of calling terraform, as I said, you now call Terragran, get, plan, apply, output,
28:01
every command which terraform supers you can call through Terragran, and you need to because it copies your terraform infrastructure modules in a temporary cache directory and uses your code and combined with your tfvars. Of course, you want to work locally.
28:20
You don't want to push every change with attack. That's relatively easy to call Terragran, apply with Terragran SUSE. Use your local modules directory and the most important part is double slash here because infrastructure modules is your directory and this is the submodule in your whole repo.
28:41
This is needed by terraform and Terragran to know exactly which submodule to use and resolve all internally locally used modules correctly. And just to run all the things, you can use Terragran apply all, output all, destroy all in your dev directory.
29:04
You can go to your stage directory and run all the infrastructure you have defined below this directory. Your web service, your database, your VPC can be created at once because if I destroy the whole dev infrastructure because anything is damaged, I don't go into every directory,
29:22
into every module down and call terraform destroy. I just use terraform destroy all, apply all and get maybe one coffee or two coffee, talk a bit with other developers. Depending on the size of your infrastructure, it takes a while. Yeah, keeping your variables dry.
29:40
If you had noticed, I only have one global config and behind every module usage, I have a terraform tfvas, but I have global variables, like I use the AWS account, I use a PX project name, which I don't want to copy in every configuration. So I move these common variables into the global config file.
30:02
I create a var file for region-specific config and I create a var file for environment-specific config. This is excellent supported by Terragran. So now I have a configuration for terraform itself, like I want to supply extra arguments
30:22
to every terraform comment and the comment is called vars. Which comment? The parameter is called vars, sorry. And to which comment I want to supply all these comments which need variables, like apply, like plan, destroy needs two,
30:43
but I don't need to know this. Terragran knows this and gives it to me. I define a few required variable files like the parent terraform tfvas, which is this same local config, where we always seem to, and the module terraform vars itself.
31:02
And I set a few optional variable files, like these two for environment and these two for regions. That I have two configuration is just the case because I use a two-level folder structure, like I have to be sealed and I have a services web level.
31:22
So I have to set them optional and define two levels. Terragran is intelligent enough to only use those optional which exists. Yeah, this is the resulting layout. So I still have my global config here.
31:40
Now I have my region and environment variables file. As said before, our services web needs to go two levels up to reach the environment tfvas and our VPC module has to go only one level up to reach the environment. The same for tfvas. The same for the region.
32:02
It's a really simplistic file layout and now all your variables are dry too because all your variables for the region like which you use and set where you put in the region, you put your environment specific variables into n.tfvas file.
32:23
Now your configuration is dry too. And that should be called Terragran. So a quick recap. You have dry modules, dry terraform code, a dry remote state and dry CLI flag with it
32:41
instead of big configurations which you can't really destroy. If you put all your database, all your web server configuration and all your VPC configuration in one big terraform configuration, you can't call terraform destroy if you only want to destroy your web nodes. And the biggest part for me
33:01
is the versionized infrastructure. I can increase my infrastructure code for dev stage and plot separately. And Terragran is still just a thin wrapper if you don't think about you have to write configuration for Terragran. Yeah, okay.
33:21
That's all so far. Any questions? Matt? I'm not sure if I missed it. Is Terragran also developed by HashiCorp or? No, it's developed by GrandWork
33:40
which is like a DevOps agency. And they had to, yeah, they have worked with terraform and wanted to get their configuration dry, developed some concepts for it and after they had the concept they developed Terragran as open source project for everyone to use it.
34:00
Okay, thanks. Anyone else? Can you give it up, please? So you were basically always applying the changes by hand. Is there also some kind of GitOps way
34:21
where you basically commit your main config into a repo which is then being applied? I don't like to run terraform and Terragran through CI because then anyone is able to push anyone who is allowed to push code to the repo
34:41
could do terrible things to the infrastructure or just destroy it, it doesn't work. Another thing is all those dependencies. The web service depends on the storage and so on. I have put it into CI but it always makes problems.
35:02
So I have done this and I don't like it. I prefer to run it by myself because I have the plan output. I need to know which infrastructure is modified, is destroyed, is graded. There are a few things like in AWS if the AMI changes for the instance,
35:22
terraform wants to re-grade it. I don't want to re-grade it. I don't want to re-grade the instance because it may be the production instance. So it's also a bit of a thing about trust in your developers, in other DevOps peoples. For our organizations, I am the only one
35:41
which is doing DevOps and all the others are simple developers. It's a matter of trust too. I don't trust them with infrastructure. I've got another question. How often do you run into configuration problems?
36:02
If I see it right, any kind of what I would like to configure on the systems need to be provided by a module or provider in terraform and how stable is this? Or is it, I want to be dry so I like to steer away
36:21
from complicated configuration, hot fixes, workarounds, whatever. What's your impression with terraform over the years you use it? Terraform is really stable. Altaf uses zero versions. It's really stable. They made a split from getting the providers out of the core
36:43
so the providers evolve separately, like new resources for AWS are added separately, have their own versioning, have their own PC breaks with major versions, luckily. So it's really stable, new resources are added fast.
37:01
Configurations problems you mostly get because you just need to know, like if you use AWS, you need to know AWS. You can't have two S3 buckets with the same name. You just need to know this. If you recreate the S3 bucket,
37:21
you either use name prefix or you have first destroyed it. The standard configuration change in terraform is to destroy the resource first and then re-grade it. You can change the life cycle for it, like grade before destroy. And if you have something like grade before destroy
37:41
for S3 bucket with the same name, it won't work out because terraform then wants to grade the S3 bucket with the same name first, and AWS says, nay. So the most configuration problems I have because I didn't notice restriction on the AWS side.
38:08
Anyone else? Okay, that was a bit too quick today, but yeah, big break for you.
38:22
Thank you.