

Discover more from CoderCo
Terraform Best Practices Series - Lessons from the Battlefield: Part 1
Strategic Insights from the Terraform Trenches Part 1
Throughout my journey of DevOps/Infrastructure spanning nearly five years, I've been deep in the trenches of Terraform, crafting cloud infrastructure from startups to sprawling enterprises alike. From rookie missteps to triumphant victories, I've had a front-row seat to the power of Terraform's best practices.
In the early days, I learned the hard way that hasty changes can lead to production hiccups. But these experiences, though challenging, became invaluable lessons. They drove home the importance of meticulous planning and the significance of thorough testing before hitting that apply button. Here is my list of top Terraform best practices I have learned through experience and from others.
Table of Contents
Building a clean and organised codebase
Version control
Using modules
Environment-specific configurations
Directory structure & naming
Effective variable management
Using TF vars
Using CLI for vars
State Management & Security
Remote backend
State locking
Storing sensitive data
Plan & Automation
Always planning first
Automating your IaC
Using IAM roles in your IaC
Tagging strategies
Versioning practices
Using CodeOwners file
Releasing tagged versions
Pinning minor provider versions
Checking in .terraform.lock.hcl file
1. Building a Clean and Organised Codebase
When building complex infrastructure, maintaining a clean and organized codebase is crucial. Terraform offers a few strategies to achieve this:
1.1 - Use Version Control
Store your Terraform code in version control like GitHub/GitLab repositories to track changes, collaborate effectively, and enable rollbacks when needed.
1.2 - Use Modules for Reusability
Break down your infrastructure into reusable modules to encapsulate logic and make your code more maintainable. 👨💻
module "vpc" {
source = "./modules/vpc"
# module arguments
}
1.3 - Environment-Specific Configurations
Organize your configurations based on environments (e.g., dev, staging, prod) for clear separation and easy resource management. 🌐
terraform/
├── dev/
│ ├── main.tf
│ ├── variables.tf
├── staging/
│ ├── main.tf
│ ├── variables.tf
├── prod/
│ ├── main.tf
│ ├── variables.tf
1.4 - Follow a good directory structure & use descriptive naming
Organize your codebase using a consistent directory structure that reflects the layout of your infrastructure components. Choose meaningful and descriptive names for resources, modules, and variables to enhance clarity and understanding.
terraform/
├── modules/
│ ├── network/
│ │ ├── main.tf
│ │ ├── variables.tf
│ ├── database/
│ │ ├── main.tf
│ │ ├── variables.tf
├── environments/
│ ├── dev/
│ │ ├── main.tf
│ │ ├── variables.tf
│ ├── prod/
│ │ ├── main.tf
│ │ ├── variables.tf
2. Effective Variable Management
Effective variable management simplifies configuration and allows for flexibility:
2.1 Use tfvars Files:
Store your variable definitions in separate .tfvars files for consistency and better predictability. 📁
# terraform.tfvars
region = "us-west1"
instance_type = "t2.micro"
2.2 Avoid Command-Line Variables
Instead of using command-line options, stick to default variable files for better consistency.
Example of CLI var (not recommended!!)
terraform apply -var 'region=us-east-1'
Using default vars file (recommended)
# variables/defaults.tfvars
region = "us-west-1"
instance_type = "t2.micro"
---
To run:
terraform apply -var-file=variables/defaults.tfvars
3. State Management and Security
3.1 - Leverage a Remote Backend
Use remote state backends (e.g., AWS S3, Google Cloud Storage) to store state files securely and enable teamwork 🔐
terraform {
backend "s3" {
bucket = "my-terraform-state"
key = "terraform.tfstate"
region = "us-west-1"
encrypt = true
dynamodb_table = "terraform-lock"
}
}
3.2 - Lock Your State
Implement state locking to prevent conflicts when multiple users or processes attempt to modify infrastructure concurrently. 🔑
terraform {
backend "s3" {
bucket = "my-terraform-state"
key = "terraform.tfstate"
region = "us-west-1"
encrypt = true
dynamodb_table = "terraform-lock" >> state locking here
}
}
3.3 - Store Sensitive Data Securely
Avoid storing sensitive information in your Terraform state, and instead, use external secret management tools like Secret Manager and data sources instead 🔐
resource "aws_instance" "coderco_ec2" {
ami = "ami-0c55b159cbfafe1f0"
instance_type = "t2.micro"
tags = {
Name = "ExampleInstance"
}
user_data = <<-EOF
#!/bin/bash
echo "API_KEY=${data.aws_secretsmanager_secret_version.api_key.secret_string}" >> /etc/environment
EOF
}
data "aws_secretsmanager_secret_version" "api_key" {
secret_id = "my-api-key-secret-id"
}
4. Plan and Automation
4.1 - Always Plan First
Generate a plan and review it before applying changes to ensure understanding and prevent unintended consequences. 📝
# Generate a plan
terraform plan -out=tfplan
# Review the plan
terraform show -json tfplan
4.2 - CICD - Automate with Pipelines
Implement automated pipelines using tools like Jenkins, GitHub Actions, Cloud Build, or Terraform Cloud for consistent execution. 🛠️
name: Terraform CI/CD using GitHub Actions
on:
push:
branches:
- main
jobs:
terraform:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Set up Terraform
uses: hashicorp/setup-terraform@v1
with:
terraform_version: 1.0.0 # Specify the desired Terraform version
- name: Initialize Terraform
run: terraform init
- name: Generate Terraform plan
run: terraform plan -out=tfplan
- name: Apply Terraform changes
run: terraform apply -auto-approve tfplan
4.3 - Use Service Account/IAM roles Credentials
In your CI/CD pipeline, inherit service account/IAM roles credentials from the executing service for secure automation.
4.4 - Implement Tagging Strategies
Employ a consistent tagging strategy for resources to facilitate organization, tracking, and cost allocation. 🏷️
5. Versioning practices
5.1 - Include an Owners File
Include an owner’s file in your repository to specify the responsible individuals or teams for each Terraform module or configuration. Create a CODEOWNERS file in your GitHub repo or GitLab repo and add these.
# OWNERS
# Responsible Teams for Terraform Modules
modules/aws-vpc: @network-team
modules/aws-eks: @infra-team
modules/aws-lambda: @dev-team
5.2 - Release Tagged Versions
When sharing your Terraform modules, release tagged versions to provide stability and enable others to reference specific versions.
git tag v1.0.0
git push origin v1.0.0
5.3 - Pin to Minor Provider Versions
When specifying provider versions, pin to minor versions to benefit from bug fixes and improvements while avoiding breaking changes.
provider "aws" {
version = "~> 2.0"
region = "us-west-1"
}
## In this example, we've specified that we want to use version 2.0 of the AWS provider, allowing for any bug fix releases or minor updates within the 2.x series.
5.4 - Check in .terraform.lock.hcl
file
Include the .terraform.lock.hcl
file in your source control to track provider version selections for your configuration.
Here are just some of the practices I have picked up in my journey of learning Terraform. This is just the start and there is more to learn.
To be continued in part 2….
Terraform Best Practices Series - Lessons from the Battlefield: Part 1
incredible work! Thorough and easy to follow along