Within this blog post I am going to show how to setup Azure DevOps and configuring an Azure Storage Account for Terraform remote state. I write numerous blog posts that do reference this scenario quite often; rather than repeating myself within each post I am creating this base post of which I will be referencing in any future blog posts that use this setup.
Where is the Terraform remote state file stored?
When deploying Terraform there is a requirement that it must store a state file; this file is used by Terraform to map Azure Resources to your configuration that you want to deploy, keeps track of meta data and can also assist with improving performance for larger Azure Resource deployments.
The terraform state file for Azure DevOps deployments are stored remoted within an Azure Storage Account
Why store it remotely?
Ability to share the state and accessible to all users; whether it be an Azure DevOps pipeline, GitHub Action or potentially another colleague within your team. It is certainly preferable to store the state file remotely during any sort of integration and I only really recommend running terraform in local state if you are quickly testing a new resource on your own sandbox account. Any additional interaction from another colleague; certainly should be looking at storing your terraform state remotely!
Creating Azure Storage Account & blob container to store remote state file!
Using Azure CLI – I will:-
- Set Azure subscription
- Create resource group:
thomasthorntoncloud - Create storage account:
thomasthorntontfstate - Create Blob container:
sampleremotestate
# Set Azure Subscription
az account set -s thomasthorntoncloud
# Create resource group
az group create -l uksouth -n thomasthorntoncloud
# Create storage account
az storage account create -n thomasthorntontfstate -g thomasthorntoncloud -l uksouth --sku Standard_LRS --kind StorageV2
# Create blob container
az storage container create --account-name thomasthorntontfstate --name exampletfstate
Successfully running the above, you will create similar to the below:
Azure DevOps Project creation
Deploying Terraform using Azure DevOps, requires some sort of project; follow the setup of this here on how to do this!
My project is called: thomasthorntoncloud
Azure DevOps Service connection creation
Once the project is setup, its time to create a service connection – the service connection is what connects your project to your Azure subscription(s)
To begin creation, within your newly created Azure DevOps Project – select Project Settings
Select Service Connections
Select Create Service Connection -> Azure Resource Manager -> Service Principal (Automatic)
For scope level I selected Subscription and then entered as below, for Resource Group I selected thomasthorntoncloud which I created earlier
Set role assignment for service connection
Once the service connection has been created, its time to set the role assignment of the service connection. When you create service connection – it will deploy a service principal within the Azure AD tenant. This service principal is used to deploy any Azure resources using Terraform; I will show an example of a pipeline later!
Select your new service connection as below
Select Manage Service Principal and it will load into your service principal within the Azure Portal
Make note of this service principal name (it can be changed to something more meaningful)
Within the subscription to which you will be using this service connection, select Access control (AIM)
Give adequate permissions to this service principal – in this example I have given Contributor access
Time to deploy an Azure resource using this setup
Create Azure DevOps Repo
In your newly created project, create an Azure DevOps repo, I have created AzureDevOpsSetupExample
Directory setup within Azure DevOps repo:
AzureDevOpsSetupExample
└──terraform
└──main.tf
└──azure-pipeline.yaml
Sample terraform code main.tf below:
- Notice the use of provider
azurerm - Backend is set to use
azurerm - Creating resource group:
tamops-tf - Creating Storage Account:
tamopssatftest
provider "azurerm" {
# The "feature" block is required for AzureRM provider 2.x.
# If you're using version 1.x, the "features" block is not allowed.
version = "~>2.0"
features {}
}
terraform {
backend "azurerm" {}
}
data "azurerm_client_config" "current" {}
resource "azurerm_resource_group" "tamopsrg" {
name = "tamops-tf"
location = "uksouth"
}
resource "azurerm_storage_account" "tamopssa" {
name = "tamopssatftest"
resource_group_name = azurerm_resource_group.tamopsrg.name
location = azurerm_resource_group.tamopsrg.location
account_tier = "Standard"
account_replication_type = "LRS"
}
Install Terraform extension
I use the below Terraform extension within my pipelines
https://marketplace.visualstudio.com/items?itemName=ms-devlabs.custom-terraform-tasks
Create Azure DevOps Pipeline
azure-pipeline.yaml as below – this pipeline will run the Terraform (main.tf)
This is a sample Terraform pipeline, that has two stages:
- Terraform Plan & Apply
- Terraform Destroy
Setting Pipeline to run by selecting Pipelines & New Pipeline & follow instructions to your Azure Repo and run pipeline
terraform plan, apply or destroy is triggered by selection one of the below Actions during runtime
azure-pipeline.yaml
name: $(BuildDefinitionName)_$(date:yyyyMMdd)$(rev:.r)
trigger: none
pr: none
parameters:
- name: Action
displayName: Action
type: string
default: 'Plan'
values:
- Plan
- Apply
- Destroy
variables:
- name: backendServiceArm
value: 'thomasthorntoncloud'
- name: backendAzureRmResourceGroupName
value: 'thomasthorntoncloud'
- name: backendAzureRmStorageAccountName
value: 'thomasthorntontfstate'
- name: backendAzureRmContainerName
value: 'exampletfstate'
- name: backendAzureRmKey
value: 'terraform.tfstate'
- name: terraform_version
value: '1.0.10'
- name: action
value: ${{ parameters.Action }}
stages :
- stage: terraform_plan_apply
condition: ne('${{ parameters.Action }}', 'Destroy')
jobs:
- job: terraform_apply
steps:
- task: TerraformInstaller@0
displayName: 'install'
inputs:
terraformVersion: '${{ variables.terraform_version }}'
- task: TerraformTaskV2@2
displayName: 'init'
inputs:
provider: 'azurerm'
command: 'init'
backendServiceArm: '${{ variables.backendServiceArm }}'
backendAzureRmResourceGroupName: '${{ variables.backendAzureRmResourceGroupName }}'
backendAzureRmStorageAccountName: '${{ variables.backendAzureRmStorageAccountName }}'
backendAzureRmContainerName: '${{ variables.backendAzureRmContainerName }}'
backendAzureRmKey: '${{ variables.backendAzureRmKey }}'
workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'
- task: TerraformTaskV2@2
displayName: 'plan'
condition: and(succeeded(), eq(variables['Action'], 'Plan'))
inputs:
provider: 'azurerm'
command: 'plan'
environmentServiceNameAzureRM: '${{ variables.backendServiceArm }}'
workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'
- task: TerraformTaskV2@2
displayName: 'apply'
condition: and(succeeded(), eq(variables['Action'], 'Apply'))
inputs:
provider: 'azurerm'
command: 'apply'
environmentServiceNameAzureRM: '${{ variables.backendServiceArm }}'
workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'
- stage: terraform_destroy
condition: contains('${{ parameters.Action }}', 'Destroy')
jobs:
- job: terraform_destroy
steps:
- task: TerraformTaskV2@2
displayName: 'init'
inputs:
provider: 'azurerm'
command: 'init'
backendServiceArm: '${{ variables.backendServiceArm }}'
backendAzureRmResourceGroupName: '${{ variables.backendAzureRmResourceGroupName }}'
backendAzureRmStorageAccountName: '${{ variables.backendAzureRmStorageAccountName }}'
backendAzureRmContainerName: '${{ variables.backendAzureRmContainerName }}'
backendAzureRmKey: '${{ variables.backendAzureRmKey }}'
workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'
- task: TerraformTaskV2@2
displayName: 'destroy'
condition: and(succeeded(), eq(variables['action'], 'Destroy'))
inputs:
provider: 'azurerm'
command: 'destroy'
environmentServiceNameAzureRM: '${{ variables.backendServiceArm }}'
workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'
You may be asking; where is the reference to the Terraform state storage account and container & Service connection – review variables:
backendServiceArmis the Service Connection namebackendAzureRmResourceGroupNameis Resource Group name of where Storage Account is locatedbackendAzureRmStorageAccountNameis Storage Account name of where the Terraform state will be storedbackendAzureRmContainerNameis the blob container of where the Terraform state will be stored
variables:
- name: backendServiceArm
value: 'thomasthorntoncloud'
- name: backendAzureRmResourceGroupName
value: 'thomasthorntoncloud'
- name: backendAzureRmStorageAccountName
value: 'thomasthorntontfstate'
- name: backendAzureRmContainerName
value: 'exampletfstate'
Running the above pipeline with action Apply will show successful output
Checking in Azure Portal, you will see the newly created Resource Group & Storage Account
Awesome 🙂 – reviewing the Storage Account container; you will also see the newly created tfstate file
Awesome – you have now setup Azure DevOps and configuring an Azure Storage Account for Terraform remote state. In any of my blog posts showing Azure DevOps pipelines & Terraform, this is the initial setup I use
Thomas Thornton - Microsoft Azure MVP - GitHub Star - HashiCorp Ambassador 
1 comment