Setting up Azure DevOps to begin deploying Terraform and configuring an Azure Storage Account for Terraform remote state

Within this blog post I am going to show how to setup Azure DevOps and configuring an Azure Storage Account for Terraform remote state. I write numerous blog posts that do reference this scenario quite often; rather than repeating myself within each post I am creating this base post of which I will be referencing in any future blog posts that use this setup.

Where is the Terraform remote state file stored?

When deploying Terraform there is a requirement that it must store a state file; this file is used by Terraform to map Azure Resources to your configuration that you want to deploy, keeps track of meta data and can also assist with improving performance for larger Azure Resource deployments.

The terraform state file for Azure DevOps deployments are stored remoted within an Azure Storage Account

Why store it remotely?

Ability to share the state and accessible to all users; whether it be an Azure DevOps pipeline, GitHub Action or potentially another colleague within your team. It is certainly preferable to store the state file remotely during any sort of integration and I only really recommend running terraform in local state if you are quickly testing a new resource on your own sandbox account. Any additional interaction from another colleague; certainly should be looking at storing your terraform state remotely!

Creating Azure Storage Account & blob container to store remote state file!

Using Azure CLI – I will:-

  • Set Azure subscription
  • Create resource group: thomasthorntoncloud
  • Create storage account: thomasthorntontfstate
  • Create Blob container: sampleremotestate
# Set Azure Subscription
az account set -s thomasthorntoncloud

# Create resource group
az group create -l uksouth -n thomasthorntoncloud

# Create storage account
az storage account create -n thomasthorntontfstate -g thomasthorntoncloud -l uksouth --sku Standard_LRS --kind StorageV2

# Create blob container
az storage container create --account-name thomasthorntontfstate --name exampletfstate

Successfully running the above, you will create similar to the below:

Azure DevOps Project creation

Deploying Terraform using Azure DevOps, requires some sort of project; follow the setup of this here on how to do this!

My project is called: thomasthorntoncloud

Azure DevOps Service connection creation

Once the project is setup, its time to create a service connection – the service connection is what connects your project to your Azure subscription(s)

To begin creation, within your newly created Azure DevOps Project – select Project Settings

Select Service Connections

Select Create Service Connection -> Azure Resource Manager -> Service Principal (Automatic)

For scope level I selected Subscription and then entered as below, for Resource Group I selected thomasthorntoncloud which I created earlier

Set role assignment for service connection

Once the service connection has been created, its time to set the role assignment of the service connection. When you create service connection – it will deploy a service principal within the Azure AD tenant. This service principal is used to deploy any Azure resources using Terraform; I will show an example of a pipeline later!

Select your new service connection as below

Select Manage Service Principal and it will load into your service principal within the Azure Portal

Make note of this service principal name (it can be changed to something more meaningful)

Within the subscription to which you will be using this service connection, select Access control (AIM)

Give adequate permissions to this service principal – in this example I have given Contributor access

Time to deploy an Azure resource using this setup

Create Azure DevOps Repo

In your newly created project, create an Azure DevOps repo, I have created AzureDevOpsSetupExample

Directory setup within Azure DevOps repo:

AzureDevOpsSetupExample
    └──terraform
      └──main.tf
    └──azure-pipeline.yaml

Sample terraform code main.tf below:

  • Notice the use of provider azurerm
  • Backend is set to use azurerm
  • Creating resource group: tamops-tf
  • Creating Storage Account: tamopssatftest
provider "azurerm" {
    # The "feature" block is required for AzureRM provider 2.x.
    # If you're using version 1.x, the "features" block is not allowed.
    version = "~>2.0"
    features {}
}
 
terraform {
  backend "azurerm" {}
}
 
data "azurerm_client_config" "current" {}
 
resource "azurerm_resource_group" "tamopsrg" {
  name     = "tamops-tf"
  location = "uksouth"
}
 
resource "azurerm_storage_account" "tamopssa" {
  name                     = "tamopssatftest"
  resource_group_name      = azurerm_resource_group.tamopsrg.name
  location                 = azurerm_resource_group.tamopsrg.location
  account_tier             = "Standard"
  account_replication_type = "LRS"
}

Install Terraform extension

I use the below Terraform extension within my pipelines

https://marketplace.visualstudio.com/items?itemName=ms-devlabs.custom-terraform-tasks

Create Azure DevOps Pipeline

azure-pipeline.yaml as below – this pipeline will run the Terraform (main.tf)

This is a sample Terraform pipeline, that has two stages:

  • Terraform Plan & Apply
  • Terraform Destroy

Setting Pipeline to run by selecting Pipelines & New Pipeline & follow instructions to your Azure Repo and run pipeline

terraform plan, apply or destroy is triggered by selection one of the below Actions during runtime

azure-pipeline.yaml

name: $(BuildDefinitionName)_$(date:yyyyMMdd)$(rev:.r)

trigger: none

pr: none

parameters:

  - name: Action
    displayName: Action
    type: string
    default: 'Plan'
    values:
    - Plan
    - Apply
    - Destroy

variables:
  - name: backendServiceArm
    value: 'thomasthorntoncloud'
  - name: backendAzureRmResourceGroupName
    value: 'thomasthorntoncloud'
  - name: backendAzureRmStorageAccountName
    value: 'thomasthorntontfstate'
  - name: backendAzureRmContainerName
    value: 'exampletfstate'
  - name: backendAzureRmKey
    value: 'terraform.tfstate'
  - name: terraform_version
    value: '1.0.10'
  - name: action
    value: ${{ parameters.Action }}

stages :   
  - stage: terraform_plan_apply
    condition: ne('${{ parameters.Action }}', 'Destroy')
    jobs:
      - job: terraform_apply
        steps:
          - task: TerraformInstaller@0
            displayName: 'install'
            inputs:
              terraformVersion: '${{ variables.terraform_version }}'
          - task: TerraformTaskV2@2
            displayName: 'init'
            inputs:
              provider: 'azurerm'
              command: 'init'
              backendServiceArm: '${{ variables.backendServiceArm }}'
              backendAzureRmResourceGroupName: '${{ variables.backendAzureRmResourceGroupName }}'
              backendAzureRmStorageAccountName: '${{ variables.backendAzureRmStorageAccountName }}'
              backendAzureRmContainerName: '${{ variables.backendAzureRmContainerName }}'
              backendAzureRmKey: '${{ variables.backendAzureRmKey }}' 
              workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'
          - task: TerraformTaskV2@2
            displayName: 'plan'
            condition: and(succeeded(), eq(variables['Action'], 'Plan'))
            inputs:
              provider: 'azurerm'
              command: 'plan'
              environmentServiceNameAzureRM: '${{ variables.backendServiceArm }}'
              workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'
          - task: TerraformTaskV2@2
            displayName: 'apply'
            condition: and(succeeded(), eq(variables['Action'], 'Apply'))
            inputs:
              provider: 'azurerm'
              command: 'apply'
              environmentServiceNameAzureRM: '${{ variables.backendServiceArm }}'
              workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'

  - stage: terraform_destroy
    condition: contains('${{ parameters.Action }}', 'Destroy')
    jobs:
      - job: terraform_destroy
        steps:
          - task: TerraformTaskV2@2
            displayName: 'init'
            inputs:
              provider: 'azurerm'
              command: 'init'
              backendServiceArm: '${{ variables.backendServiceArm }}'
              backendAzureRmResourceGroupName: '${{ variables.backendAzureRmResourceGroupName }}'
              backendAzureRmStorageAccountName: '${{ variables.backendAzureRmStorageAccountName }}'
              backendAzureRmContainerName: '${{ variables.backendAzureRmContainerName }}'
              backendAzureRmKey: '${{ variables.backendAzureRmKey }}' 
              workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'
          - task: TerraformTaskV2@2
            displayName: 'destroy'
            condition: and(succeeded(), eq(variables['action'], 'Destroy'))
            inputs:
              provider: 'azurerm'
              command: 'destroy'
              environmentServiceNameAzureRM: '${{ variables.backendServiceArm }}'
              workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'

You may be asking; where is the reference to the Terraform state storage account and container & Service connection – review variables:

  • backendServiceArm is the Service Connection name
  • backendAzureRmResourceGroupName is Resource Group name of where Storage Account is located
  • backendAzureRmStorageAccountName is Storage Account name of where the Terraform state will be stored
  • backendAzureRmContainerName is the blob container of where the Terraform state will be stored
variables:
  - name: backendServiceArm
    value: 'thomasthorntoncloud'
  - name: backendAzureRmResourceGroupName
    value: 'thomasthorntoncloud'
  - name: backendAzureRmStorageAccountName
    value: 'thomasthorntontfstate'
  - name: backendAzureRmContainerName
    value: 'exampletfstate'

Running the above pipeline with action Apply will show successful output

Checking in Azure Portal, you will see the newly created Resource Group & Storage Account

Awesome 🙂 – reviewing the Storage Account container; you will also see the newly created tfstate file

Awesome – you have now setup Azure DevOps and configuring an Azure Storage Account for Terraform remote state. In any of my blog posts showing Azure DevOps pipelines & Terraform, this is the initial setup I use

GitHub containing code referenced above

1 comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s