We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Manage GeoServer configuration with Terraform

00:00

Formal Metadata

Title
Manage GeoServer configuration with Terraform
Title of Series
Number of Parts
156
Author
License
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
This presentation presents how to manage GeoServer configuration with a custom made Terraform provider. It will focus on the different resources available in the provider and the updates made since the last FOSS4G. Different use cases will be explained to show how you can use Terraform capabilities.
Keywords
127
Process (computing)Streaming mediaState of matterInternet service providerOpen sourcePoint cloudRevision controlSource codeRAIDCodeAddress spaceConfiguration spaceInstance (computer science)Complex (psychology)Internet service providerOpen sourceDuality (mathematics)Data storage deviceData managementType theoryRepresentational state transferView (database)Server (computing)Computer fileInstance (computer science)Attribute grammarCASE <Informatik>CodeDifferent (Kate Ryan album)MathematicsCache (computing)Revision controlDoubling the cubeModule (mathematics)Variable (mathematics)InjektivitätDefault (computer science)Integrated development environmentGroup actionEmoticonOrder (biology)InformationSampling (statistics)Projective planeFlow separationConfiguration spaceMultiplication signSoftware developerWeb 2.0Connectivity (graph theory)Product (business)Client (computing)Library (computing)Key (cryptography)MappingPhysical systemLangevin-GleichungGeometryArithmetic progressionSet (mathematics)Point cloudReliefComputer animation
Address spaceConfiguration spaceInternet service providerInstance (computer science)Complex (psychology)State of matterAttribute grammarRevision controlView (database)Data structureVariable (mathematics)Block (periodic table)InformationTerm (mathematics)CodeFunction (mathematics)Template (C++)Gamma functionSubstitute goodFormal languageIterationElectronic mailing listIntegrated development environmentData storage deviceComputer iconVideo gameDatabaseLine (geometry)Data storage deviceType theoryMathematicsCycle (graph theory)Declarative programmingServer (computing)Row (database)Mechanism designMultiplication signTranslation (relic)Asynchronous Transfer ModeShared memoryComputer fileRule of inferenceFlow separationBlock (periodic table)Revision controlAttribute grammarElectronic mailing listInternet service providerConfiguration spaceTable (information)Medical imagingGraph (mathematics)Variable (mathematics)Physical systemIntegrated development environmentData managementState of matterComputer iconIterationTemplate (C++)Functional (mathematics)Instance (computer science)Substitute goodResultantLevel (video gaming)Software developerSoftware bugComputer programmingSet (mathematics)System callInternet forumComputer animationLecture/Conference
Integrated development environmentBlock (periodic table)State of matterInternet service providerData storage deviceComputer iconSpacetimeDew pointFraction (mathematics)Scale (map)Type theoryTranslation (relic)Computer fileSpherical capTemplate (C++)Rule of inferenceComputer animation
State of matterInternet service providerGroup actionElement (mathematics)Complete metric spacePhotographic mosaicType theoryInternet service providerTesselationMoment (mathematics)Server (computing)GeometryHypermediaComputer animationLecture/Conference
Computer-assisted translationLeast squaresComputer animation
Transcript: English(auto-generated)
So first I am Working for come to come we have an open-source IT service company, which is based in Switzerland France and Germany We have committers and contributors In several open source project like your server, your server cloud, your questra, your mapfish and so on
And today I would like to talk about some A Terraform provider, we maintained for a couple of years now And do some update on what we have done during the past the past year
so for So Terraform is an infrastructure as code tool That you build change and version include and on-premise resource safely and efficiently. It was developed by a RC Corp Until last year it was under Mozilla public license and they have moved to BSL last year
Why develop a Terraform provider? So it's for only for the configuration of the of your server not deploying your server by itself we use it because compared to over solution there is
already management of references between the resources of Between the different resources of your server configuration You have a so by default management of dependencies between resources When you
Update your configuration you will only modify the the resource which are required and not over there is templating variables and secret injections and And you have also the possibility to organize your configuration into modules It's also
something which is very that in some companies that with a big IT department that they are used to Terraform so they can They are confident if we come with this kind of solution So We the the code base is divided in two
two components the go library which wraps the Joe server REST API and which could be used in another project in go and the provider itself which use a SDK of Terraform and the client library to to bring the
the features We use it mainly for the Nexus system which We build for the French French firefighters We have your server configuration with 15 different data store 100 layers style layer styles some layer groups for providing base maps
We also use your web cache for the layer groups and for caching external WMS Services we do continuous deployment for all the merge request of all the developers. So there we have in development 40
Generally 40 environments in parallel to deploy each time we do a new feature we also have a Doesn't also you 80 environments and we also have the production now, which which is used and all their use For the
Joseph our component that you will use the Terraform provider to deploy easily the the configuration Eric we can have a sample of the module and how the all the resources of Terraform are glued together to to know how in which order to create and update the information so for
OSM module that there are You have so We start with the workspace, which as a base tool We have a GND data store which reference the workspace and will be created after we use ourselves a pre generalized data store and so we have before
XML resource which is with the rest API There is a layer reference the two data stores and so they will be created after we have the styles or so All our to be our glued in a layer group and the layer group is reference in a
geoweb cache instance Together with the blob store and a grid set that we push. So all vcs are all our Terraform resources So what have we done during last year is the current session in the
OO20 We have new resources to manage cascade cascading WMS server So we have a geoserver dual MS store geoserver double MS layer. We also had The resource to manage five blob store in geoweb cache
We also changes geoserver feature type to support automatic definition of attributes based on the Real Storage I will come back later on that and we also use some new use case we validated to use the geoserver feature type resource with a
SQL view So it works so currently we have not all the geoserver resources, but some so we have a resource for deploying specific files in the data here We have the workspace data store where we have tested with gdbc gndi with the general
Pre-generalized features stored in Prosegress and with CQL you we manage feature types styles and layer group double MS store and double MS layer and on geoweb cache we manage we manage grid set S3 blob store and five blob store and
in double MS layer Terraform comes with a Terraform key so mainly to Plan and apply a deployment, but you can also use it for
You can also use it to do nice stuff. So for example, there is If you have something which is already defined in a geoserver configure an existing just ever instance and you want to prepare Terraform configuration to be able to deploy it in a over a step in an over environment
You can use a terraform import command which allow to generate resource manifest from existing geoserver resources. So you You just have to configure a provider file choose the existing instance you use terraform in it and then terraform import by pointing the resource you want to
create You we have at the end the terraform state that we contain the definition of the feature type and so you only have to Copy paste this configuration in a terraform configuration file and you can use it after to deploy it in a in an over
In an over environment There is so terraform graph command which can be used to generate Graphic on the resources and to help you for the documentation of your system So you just combine that with a dot
command to get this kind of images We have also so we met some challenges this year to manage attributes in feature types with Automatic or custom list. Let's let's see the feature. So in the first version of the provider we
We did We expected we wanted to list all the expected attributes of the feature That's
One of the program is that When you have in your database a table with 40 columns it makes quite a time to create your resource and moreover by doing so Joe server will consider the feature type as a one with custom attribute list
in Joe server to 24-0 there was for some time a bug which made WFS not working with a custom attribute list feature type which is which was a issue 1 1 2 3 6 and so to
We have for a short time I have to change the provider to be able to To say To Joe server that you take the list of the data store like when you you don't provide So it was finally in good things because we we were able to we no longer have to declare
All the attribute up front in terraform. It's computed by your server and so from the version O16 on the side If there is no attribute, we create a feature type with no attribute in the rest IP call. So Joe server detect it and
If we declared at least one attribute we will turn the we create a feature type with custom attribute list But it bring another another issue
If we use the Automatic attribute list mode We are not able to Terraform won't update won't tell Joe server to update the attribute list If the database had changed for example
So for us it was a kind of forum what we have done to solve that is to use Terra some Terraform mechanism to do so so we have now database version in in the Terraform configuration
We use a Terraform data resource to store the version in the Terraform state and We had on the database. We know we will change often lifecycle block To
Which Enable us to do to do the to trigger a replace of the resource when the database version is change So just some snipe set. So at the top row you have the database version that we can for example
inject from From deployment in Kubernetes or that even we can ask it on the command line to the user Then you have the Terraform data which store it in the Terraform state and At the end you have an example of a life cycle declaration to say when the internal version
of the the variable into television change, please update is a feature type and this way it will Replace create
Recreate the feature type in the database in the joy server configuration by reading the new list of attributes You can also with a templating function you can also Use use it. I have a customer as a developer team of customer which use it to
Generate its style definition So The there is a template file function into Terraform Which can do some variable substitution from on a template file and give you back the result
There is some Feature like a simple variable substitution you can iterate on list and you can also do some map iteration and You can also a
Cold several template file inside of a template size. So you have example of list iteration in my map iteration and for example for for Microsoft the development team use it to adapt the credential of the Icon storage to the target environment because they don't store it in the same storage on S3 for example
They use it to have only one command marker style definition and They change only the SVG file by templating the the definition then also manage the translation of
The of the style into the manifest and there so share rule blocks between several styles So for example This is style definition that they generate they call a first template file with you can see a lot of lot of
Modification Type you find the translation for example, and there is a so rule that they import for label labeling and so this this way they use they generate all their definition for their different
Markers So to conclude What are the challenges that we expect and that we have First geo server rest API is not complete complete crude API. So there is some
for example for the layer API you Create feature types or over stuff which will create layers But you generally have to delete layers if you want to have a clean For example, if you want to cascade correctly the deletion in layer group
So for some terraform resources, it makes more difficult to implement the stuff I recently discovered the magic world of image mosaic and so I Start to think of how could I implement such a things with my
Terraform so I for example, it will be there is some existing provider for Listing s3 element in back in S3 tiles in a bucket and so perhaps I can do you that
I have the question of migrating to open tofu Which is a media fork of terraform So, I don't know for a moment what we will do because for for most side it's not compatible of the
BSL is no Incompatible with what we do as business. So I don't know what we will do in this way So that's it. Thank you for attention. Don't hesitate To to try yeah Try to put some documentation or so together with a provider that you can read it
And if you have any question, you're welcome