Infrastructure as Code, Terraform and You

Contents

• Intro
• Pillars of Infrastructure of Code
  • 1. Reproduce Systems Easily
  • 2. Idempotence
  • 3. Modular Processes
  • 4. Disposable Systems
  • 5. Source Control
• The Tools: Terraform
• Terraform + You
• Resources

Intro

Consider this: you are starting a fresh project - a brand new AWS, Azure or Google environment. You have documented everything perfectly, you’ve created perfectly separated but mirrored production and development environments, and everything has been successfully handed over to the customer. All is well…

But a year passes, maybe two, maybe more! Suddenly the prod environment has had a few fixes, changes, additions and is unrecognisable from that first implementation - you have drift. Not only that, the development environment is a sprawling mess and no longer is representative of the production environment it once mirrored.

If you’ve ever come across the above scenario - then you need Infrastructure as Code. Infrastructure as Code evolved to solve the problem of environment drift in the release pipeline. Without IaC, teams must maintain the settings of individual deployment environments. Over time, each environment becomes a snowflake, that is, a unique configuration that cannot be reproduced automatically. Inconsistency among environments leads to issues during deployments. With snowflakes, administration and maintenance of infrastructure involves manual processes which were hard to track and contributed to errors.

Pillars of Infrastructure of Code

1. Reproduce Systems Easily

IaC allows you to recreate your environments entirely from scratch without much effort or complexity. Everything involved can be coded into declarative definitions (scripts, tools and resource provisioning) that allow you to deploy your full environment without human interaction

2. Idempotence

Idempotency means no matter how many times you run your IaC and, what your starting state is, you will end up with the same end state. For example, if you apply a script to a running server this is often an additive process. In Infrastructure as Code we avoid this by defining the end state. This simplifies the provisioning of Infrastructure and reduces the chances of inconsistent results.

3. Modular Processes

IaC demands that the system administrators think in automation, scripts and repeatability. Tasks must be broken down or joined together into repeatable processes which can be codified in declarative files.

4. Disposable Systems

IaC acutely depends on reliable and resilient software to make hardware reliability irrelevant to system operations. In the cloud era, where the underlying hardware may or may not be reliable, organisations cannot let hardware failures disrupt their businesses. Dynamic infrastructure that can be created, destroyed, resized, moved, and replaced sit at the core of IaC. It should handle infrastructure changes like resizing and expansions gracefully.

5. Source Control

It goes without saying that everything should be in source control. Even a script that you run occasionally to fix an issue, as well as the pipeline used to provision your infrastructure and deploy your software. I have been at places where no one knows where a script that runs in production lives, who created it, and history of changes. This is a situation you don’t want to be in.

The showdown: Hashicorp Terraform vs AWS CloudFormation

The Tools: Terraform

Terraform is an IaC tool like many others - Azure RM, CloudFormation, Deployment Manager, etc. - but it’s biggest advantage is that it is completely platform agnostic. It uses it’s own language developed by Hashicorp (called HCL - Hashicorp Configuration Language) for all of its definition files.

From a single file you could define resources, configurations and settings for all your cloud environments (AWS, Google Cloud, Azure), your GitHub, your on-premises VMWare environment - you name it!

Why is this an advantage over other tools? I believe it’s for the reasons the IaC pillars listed above cover. Infrastructure as Code gives you a level of visibility and repeatability over all of your infrastructure and Terraform allows you to do that beyond the scope of a single cloud environment.

And on top of that, Terraform has a lively community and is open source, there is a massive community developing around this tool. Many people are already using it, and it’s easier to find people who know how to use it, plugins, extensions, professional support, etc. This also means Terraform is evolving at a much faster rate.

High level overview of IaC deployment

Terraform + You

The core Terraform workflow has three steps:

1. Write - Author infrastructure as code.

   1. You write Terraform configuration just like you write your other scripts: by firing up VSCode. It’s common practice to store your work in a version controlled repository even when you’re just operating as an individuals so we always recommend this.

2. Plan - Preview changes before applying.

   1. Running terraform plan will show you what the outcomes of your actions will be.

3. Apply - Provision reproducible infrastructure.

   1. Running terraform apply will take the code (or a plan file that you’ve developed in step 2) and create the changes in the environment according to the plan. It will keep a state file updated with what changes were made and what the environment currently looks like for any future comparisons.

When it’s time to make a change - loop through these steps again from the top.

Again - if the IaC principles are to be followed here we should be doing all of these from CI/CD. We like to use Google Cloud Build for this - but all CI/CD tools are available for this purpose. At a high level this looks like this:

1. Create a branch called production-plan

2. A CI trigger should be associated with Pull Requests made from this branch that will run a terraform plan which will evaluate that your Terraform code is correct and show you what the outcomes will be

3. Push the changes in the production-plan branch to the production branch - this branch should be protected so that only Pull Requests can be raised against it and that someone needs to approve the changes.

4. A CD trigger should be associated with this production branch that then takes those changes and deploys them into the environment associated with the branch - in this case: Production

Resources

It’s time to get stuck in! But you don’t know where? That’s fair enough. It’s sometimes tricky to understand or how to get started with Terraform and Infrastructure as Code. The best answer that I’ve found to this is: just do it! If you have to create something, do it in IaC (and in particular - Terraform). If you have to solve a problem, think about how to weave it into a repeatable pattern in IaC. This is the best way to start.

The good news for those that don’t know exactly where to go is that we currently are using Terraform for all of our Google Cloud deployments and we know of and have also developed some useful resources for deploying to GCP.

Introducing the Terraform Infrastructure Bootstrap

https://github.com/withriley/terraform-infrastructure-bootstrap

This bootstrap repo helps you get started with deploying resources into your existing cloud accounts. Simply define your environments, and the buckets where you want your state to live and go from there!

Don’t have your environment yet? Don’t fear! For setting up your new Google environment - organisations, projects, networks, secrets, etc. - Google provide an example foundation that can be used to bootstrap your environments:

https://github.com/terraform-google-modules/terraform-example-foundation

So get stuck into it and start your Terraform journey today! Here are some handy links for all the deployments you wish to do:

• https://github.com/terraform-google-modules

• https://github.com/terraform-aws-modules

• https://github.com/terraform-azurerm-modules