Writing reusable Terraform modules

Are you using Terraform? Deploying to multiple environments? Terraform code duplication? If you answered yes, in this blog post I am going to look at creating reusable Terraform modules including the basics, benefits and why you should be using them and finish with two examples.

Deploying and managing Terraform over time in a single file with all your resources etc, quite a few things can begin to happen:

• Complexity: An interesting topic in any Infrastructure as Code (IaC), you want to reduce complexity as much as possible
• Maintainability: Can this Terraform code still be maintained? Trying to map values and resources when one large file is used
• Reusability: Deploying to multiple environments? Can I reuse what I have created previously? Possible, but probably will involve duplication

Three key areas mentioned about, writing reusable Terraform modules can definitely assist with this.

Why create Terraform modules?

• Increases maintainability

Deploying the same Terraform to multiple environments, with modules; it allows you to increase maintainability of your Terraform code. With a module, when you update it – that same update can be rolled out to multiple environments without having to update in several areas. (more on this later)

• Navigation is easier

Creating modules allows it easier to understand what is being deployed, rather than one single file or folder. You can filter and search within a module. For example, an Azure Virtual Network module will contain all required resources to deploy an Azure Virtual Network. Allowing you to navigate straight to the module if you want to potentially amend or make a change to the Azure Virtual Network

• Provide consistency

Terraform modules can begin to be a source of truth, include them to provide consistency within your environment. Updating the module can begin to keep all environments in sync, moving onto my next point of reducing environment drift

• Reduces environment drift

Providing consistency really does allow you to reduce environment drift. Creating terraform modules which all environments use as a central resource will certainly reduce environment drift. Updating the module can take effect within all environments. Rather than modifying a resource within each environment.

• Code Organisation

More organised code, when using modules – as mentioned, having a series of modules allows you to navigate and organise your Terraform deployment a lot easier. Over time, the structured organisation will really show

• Begin to implement code structure and define best practices

Like any code/IaC – structuring is really recommended from a development and also supportability perspective. Lay out your Terraform modules with structure in mind. Best practices, remember these when creating and defining modules – over time, you will see the benefits of this!

Structuring your Terraform modules

Terraform files and modules can be structured in so many ways, some tips to assist you when structuring and creating your terraform module

• Avoid one big file

Really top of my list is this, structure your terraform module(s), utilising the potential of separation of concerns , keep to separate files, including:

• Variables
• Distinct Azure resources: you may be combining a module with multiple resources, I recommend splitting into each specific files
• Separate your local values, if using alot – move into its own file
• Any additional terraform resources, potentially a file per basis

• Plan from the start

Plan your Terraform deployment from the start; think of it like mapping a road trip, which points do you want to include and where? It will be alot easier, working towards a plan – you can then even look at it from an Agile format, breaking into various epics and stories etc

• Separation of concerns

Create modules that are going to be creating different Azure resources or areas, don’t create one huge module. I do recommend as mentioned above, split into various areas or resources.

• Hub Network module: Includes hub vnet, azure firewall, ingress etc
• Virtual Machine module: All the resources required to create and deploy a virtual machine

• Simplicity, no need for complexity (most of the time 🙂 )

Keep your modules simple; yet effective. Remember when creating modules, you are creating them for reusability and think of it from a support aspect also. Someone may come along in the further and want to amend or modify what you have created.

• Folder structure

Include an accurate representation of folder structure within your terraform module deployments, similar to separation to concerns, define components and environments inside folders.

Terraform module example

In my example of a terraform module, I will be deploying an Azure Virtual Network and relevant subnets. This will be a simple module, to reference folder structure and setup. Sample repository here

Folder structure:

terraform-module-example
    └── modules
       └── vnet
          └── vnet.tf
          └── outputs.tf
          └── variables.tf
    └── main.tf
    └── providers.tf
    └── variables.tf

If we dive into modules/vnet

vnet.tf contains the required resources to create an Azure Virtual Network and relevant subnets, including Resource Group.

resource "azurerm_resource_group" "vnet_resource_group" {
  name     = "${var.name}-rg"
  location = var.location
  
  tags = {
    Environment = var.environment
  }
}

resource "azurerm_virtual_network" "virtual_network" {
  name = var.name
  location = var.location
  resource_group_name = azurerm_resource_group.vnet_resource_group.name
  address_space = [var.network_address_space]

  tags = {
    Environment = var.environment
  }

}

resource "azurerm_subnet" "aks_subnet" {
  name = var.aks_subnet_address_name
  resource_group_name  = azurerm_resource_group.vnet_resource_group.name
  virtual_network_name = azurerm_virtual_network.virtual_network.name
  address_prefixes = [var.aks_subnet_address_prefix]
}

resource "azurerm_subnet" "appgw_subnet" {
  name = var.appgw_subnet_address_name
  resource_group_name  = azurerm_resource_group.vnet_resource_group.name
  virtual_network_name = azurerm_virtual_network.virtual_network.name
  address_prefixes = [var.appgw_subnet_address_prefix]
}

variables.tf contains the relevant variables the module requires:

variable "name" {
}

variable "location" {
  default = "uksouth"
}

variable "network_address_space" {
}

variable "aks_subnet_address_prefix" {
}

variable "aks_subnet_address_name" {
}

variable "appgw_subnet_address_prefix" {
}

variable "appgw_subnet_address_name" {
}

variable "environment" {
}

outputs.tf contains outputs that I can return to my Terraform configuration.

output "aks_subnet_id" {
  value = azurerm_subnet.aks_subnet.id
}

output "appgw_subnet_id" {
  value = azurerm_subnet.appgw_subnet.id
}

output "vnet_id" {
  value = azurerm_virtual_network.virtual_network.id
}

output "vnet_name" {
  value = azurerm_virtual_network.virtual_network.name
}

output "resource_group" {
  value = azurerm_resource_group.vnet_resource_group.name
}

output "resource_group_id" {
  value = azurerm_resource_group.vnet_resource_group.id
}

You can reference outputs within your main Terraform configuration.

Looking at root/main.tf , known as the root module – you can declare the child module referenced above:

module "vnet" {
  source                      = "./modules/vnet"
  name                        = var.vnet_name
  location                    = var.location
  network_address_space       = var.network_address_space
  aks_subnet_address_prefix   = var.aks_subnet_address_prefix
  aks_subnet_address_name     = var.aks_subnet_address_name
  appgw_subnet_address_prefix = var.appgw_subnet_address_prefix
  appgw_subnet_address_name   = var.appgw_subnet_address_name
  environment                 = var.environment
}

providers.tf declare terraform providers and backend information

provider "azurerm" {
    version = "~> 2.0"
    features {}
}

terraform {
    backend "azurerm" {
      resource_group_name = "platopsacad-tf-rg"   
      storage_account_name = "platopsacadazuredevops"
      container_name = "terraform.tfstate`"
    }
}

data "azurerm_client_config" "current" {}

variables.tf again, variables – but now for the whole environment, adding more modules – you would include further variables in this file

variable "location" {
  type        = string
  description = "Location of Resources"
}

variable "vnet_name" {
  type        = string
  description = "Virtual Network Name"
}

variable "network_address_space" {
  type        = string
  description = "Virtual Network Address Space"
}

variable "aks_subnet_address_prefix" {
  type        = string
  description = "AKS Subnet Address Prefix"
}

variable "aks_subnet_address_name" {
  type        = string
  description = "AKS Subnet Name"
}

variable "appgw_subnet_address_prefix" {
  type        = string
  description = "AppGW Subnet Address Prefix"
}

variable "appgw_subnet_address_name" {
  type        = string
  description = "AppGW Subnet Name"
}

variable "environment" {
  type        = string
  description = "Environment"
}

Terraform Modules at scale

Above, I showed the breakdown of how to reference a module, lets now looking at deploying multiple modules. In this example, I will be touching only on folder structure.

See here for further sample repository setup of this!

In my example, I am going to be deploying into Azure:

• Log Analytics
• Virtual Network
• Azure Kubernetes Service (AKS)
• Azure Container Registry (ACR)
• Role permissions (IAM)
• Application Insights

Lets scale, we have a basic configuration – looking at the folder structure, notice its being reused?

Also check out environments/environments/development.tfvars environments/environments/production.tfvars using .tfvars are really good, you can use them to customise variable naming per environment(s)

terraform-full
    └── environments
        └── development.tfvars
        └── production.tfvars
    └── modules
       └── acr
          └── acr.tf
          └── outputs.tf
          └── variables.tf
       └── aks
          └── vnet.tf
          └── outputs.tf
          └── variables.tf
       └── appinsights
          └── appinsights.tf
          └── variables.tf
       └── keyvault
          └── keyvault.tf
          └── variables.tf
       └── log-analytics
          └── la.tf
          └── outputs.tf
          └── variables.tf
       └── vnet
          └── vnet.tf
          └── outputs.tf
          └── variables.tf
    └── main.tf
    └── providers.tf
    └── variables.tf

Reusability is key! Please note, in my example – I have terraform modules structured in a mono repo model , to show folder structure. Feel free to have separate repositories per module 🙂

Taking a small snippet from root/main.tf (root module) – Notice the reference to module outputs module.acr.resource_group_id module.aks.kubelet_object_id ? As mentioned above, showing outputs within a module – you can reference this!

module "acr" {
  source      = "./modules/acr"
  name        = var.acr_name
  location    = var.location
  environment = var.environment
}

resource "azurerm_role_assignment" "aks-acr-rg" {
  scope                = module.acr.resource_group_id
  role_definition_name = "Acrpull"
  principal_id         = module.aks.kubelet_object_id

  depends_on = [
    module.aks,
    module.acr
  ]
}

How awesome 🙂 – hopefully this blog post has given you an insight into why and how to create reusable modules, they really are great!

Some further reading I recommend you to look at:

• Terraform modules overview
• Build and use a local module
• Creating reusable terraform with terraform modules