In this blog post, I am going to show how you can deploy Terraform using Azure DevOps with a Build Artifact that is created during the Terraform plan stage.
Why Build Artifacts for Terraform?
Working in a busy environment, you may be wanting multiple iterations of the Terraform pipeline; these iterations may require an approval process or requirement to be approved prior to being released into a specific environment. Multiple people are updating Terraform code into the master branch; these approvals backup and now the current branch version is different to the original approval requirement..
This is where build artifacts will help! In a later blog, I will show, how you can configure approval gates.
How can Build Artifacts help?
After you run a Terraform plan against the environment, you can .zip these contents along with the additional configurations stored in git into a build artifact on that pipeline.
So, when approval has been granted, you can run the exact version that was planned – whether this be hours or days later! The build artifact is stored within the pipeline, so when Terraform Apply is ran, it will download and unzip this artifact – using this Terraform configuration rather than pulling directly from master branch for the newer version of the Terraform code
Recommended reading
If you are unsure how to run and configure Terraform using Azure DevOps, I recommend my blog post Deploy Terraform using Azure DevOps prior to this blog post, it will go into detail the pre-requirements etc required to successfully run Terraform using Azure DevOps
I want to artifact, what is my Azure pipeline process now?
Lets have a look at the tasks that will be used now during the terraform_plan stage:-
Git Checkout:- Normal process, checkout required branch
Terraform Init:- Initial Terraform using Task:- TerraformTaskV1@0
Terraform Plan:- Terraform Plan using Task:- TerraformTaskV1@0
Archive Files:- Archive the directory that has both the git content and the Terraform Plan
Now the terraform_deploy stage:-
Git Checkout:- Checkout none, this stage will not checkout the required branch. It will use the code that was archived and zipped as part of the above stage
Download Build Artifacts:- Download artifacts from previous step using DownloadBuildArtifacts@0 Task
Extract Build Artifacts:- Extract the build artifacts using ExtractFiles@1 Task
Terraform Init:- Initial Terraform using Task:- TerraformTaskV1@0
Terraform Plan:- Terraform Plan using Task:- TerraformTaskV1@0 that will display the plan taken from above stage
Terraform Apply:- Terraform Apply using Task:- TerraformTaskV1@0 that applies required additions/changes from the Terraform Plan
Stages
└──terraform_plan
└──Git Checkout
└──Terraform Init
└──Terraform Plan
└──Archive Files
└──Publish Build Artifact
└──Delete Files (Cleanup)
└──terraform_apply
└──Checkout: None
└──Download Build Artifacts
└──Extract Build Artifacts
└──Terraform Init
└──Terraform Plan
└──Terraform Apply
Lets look at the above inside Azure Pipelines as code
Stage:- terraform_plan
Note:- I have included Terraform Installer task on each stage, so you can run this code directly using Azure DevOps Agent
- stage: terraform_plan
jobs:
- job: terraform_plan
steps:
- checkout: self
- task: TerraformInstaller@0
displayName: 'install'
inputs:
terraformVersion: '0.13.4'
- task: TerraformTaskV1@0
displayName: 'init'
inputs:
provider: 'azurerm'
command: 'init'
backendServiceArm: 'tamopstf'
backendAzureRmResourceGroupName: 'tamopstfstates'
backendAzureRmStorageAccountName: 'tfstatedevops'
backendAzureRmContainerName: 'azureterraformbuildartifacts'
backendAzureRmKey: 'terraform.tfstate'
workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'
- task: TerraformTaskV1@0
displayName: 'plan'
inputs:
provider: 'azurerm'
command: 'plan'
commandOptions: '-input=false -var-file="../vars/production/production.tfvars"'
environmentServiceNameAzureRM: 'tamopstf'
workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'
- task: ArchiveFiles@2
inputs:
rootFolderOrFile: '$(Build.SourcesDirectory)'
includeRootFolder: false
archiveType: 'tar'
tarCompression: 'gz'
archiveFile: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).tgz'
replaceExistingArchive: true
displayName: 'Create Plan Artifact'
- task: PublishBuildArtifacts@1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: '$(Build.BuildId)-tfplan'
publishLocation: 'Container'
displayName: 'Publish Plan Artifact'
- task: DeleteFiles@1
displayName: 'Remove unneeded files'
inputs:
contents: |
.terraform
tfplan
Stage:- terraform_apply
- stage: terraform_apply
dependsOn: [terraform_plan]
condition: succeeded('terraform_plan')
jobs:
- job: terraform_apply
steps:
- checkout: none
- task: DownloadBuildArtifacts@0
inputs:
artifactName: '$(Build.BuildId)-tfplan'
displayName: 'Download Plan Artifact'
- task: ExtractFiles@1
inputs:
archiveFilePatterns: '$(System.ArtifactsDirectory)/$(Build.BuildId)-tfplan/$(Build.BuildId).tgz'
destinationFolder: '$(System.DefaultWorkingDirectory)/'
cleanDestinationFolder: false
displayName: 'Extract Terraform Plan Artifact'
- task: TerraformInstaller@0
displayName: 'install'
inputs:
terraformVersion: '0.13.4'
- task: TerraformTaskV1@0
displayName: 'init'
inputs:
provider: 'azurerm'
command: 'init'
backendServiceArm: 'tamopstf'
backendAzureRmResourceGroupName: 'tamopstfstates'
backendAzureRmStorageAccountName: 'tfstatedevops'
backendAzureRmContainerName: 'azureterraformbuildartifacts'
backendAzureRmKey: 'terraform.tfstate'
workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'
- task: TerraformTaskV1@0
displayName: 'plan'
inputs:
provider: 'azurerm'
command: 'plan'
commandOptions: '-input=false -var-file="../vars/production/production.tfvars"'
environmentServiceNameAzureRM: 'tamopstf'
workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'
- task: TerraformTaskV1@0
displayName: 'apply'
inputs:
provider: 'azurerm'
command: 'apply'
commandOptions: '-input=false -auto-approve -var-file="../vars/production/production.tfvars"'
environmentServiceNameAzureRM: 'tamopstf'
workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'
The full pipeline
name: $(BuildDefinitionName)_$(date:yyyyMMdd)$(rev:.r)
trigger: none
pr: none
stages :
- stage: terraform_plan
jobs:
- job: terraform_plan
steps:
- checkout: self
- task: TerraformInstaller@0
displayName: 'install'
inputs:
terraformVersion: '0.13.4'
- task: TerraformTaskV1@0
displayName: 'init'
inputs:
provider: 'azurerm'
command: 'init'
backendServiceArm: 'tamopstf'
backendAzureRmResourceGroupName: 'tamopstfstates'
backendAzureRmStorageAccountName: 'tfstatedevops'
backendAzureRmContainerName: 'azureterraformbuildartifacts'
backendAzureRmKey: 'terraform.tfstate'
workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'
- task: TerraformTaskV1@0
displayName: 'plan'
inputs:
provider: 'azurerm'
command: 'plan'
commandOptions: '-input=false -var-file="../vars/production/production.tfvars"'
environmentServiceNameAzureRM: 'tamopstf'
workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'
- task: ArchiveFiles@2
inputs:
rootFolderOrFile: '$(Build.SourcesDirectory)'
includeRootFolder: false
archiveType: 'tar'
tarCompression: 'gz'
archiveFile: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).tgz'
replaceExistingArchive: true
displayName: 'Create Plan Artifact'
- task: PublishBuildArtifacts@1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: '$(Build.BuildId)-tfplan'
publishLocation: 'Container'
displayName: 'Publish Plan Artifact'
- task: DeleteFiles@1
displayName: 'Remove unneeded files'
inputs:
contents: |
.terraform
tfplan
- stage: terraform_apply
dependsOn: [terraform_plan]
condition: succeeded('terraform_plan')
jobs:
- job: terraform_apply
steps:
- checkout: none
- task: DownloadBuildArtifacts@0
inputs:
artifactName: '$(Build.BuildId)-tfplan'
displayName: 'Download Plan Artifact'
- task: ExtractFiles@1
inputs:
archiveFilePatterns: '$(System.ArtifactsDirectory)/$(Build.BuildId)-tfplan/$(Build.BuildId).tgz'
destinationFolder: '$(System.DefaultWorkingDirectory)/'
cleanDestinationFolder: false
displayName: 'Extract Terraform Plan Artifact'
- task: TerraformInstaller@0
displayName: 'install'
inputs:
terraformVersion: '0.13.4'
- task: TerraformTaskV1@0
displayName: 'init'
inputs:
provider: 'azurerm'
command: 'init'
backendServiceArm: 'tamopstf'
backendAzureRmResourceGroupName: 'tamopstfstates'
backendAzureRmStorageAccountName: 'tfstatedevops'
backendAzureRmContainerName: 'azureterraformbuildartifacts'
backendAzureRmKey: 'terraform.tfstate'
workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'
- task: TerraformTaskV1@0
displayName: 'plan'
inputs:
provider: 'azurerm'
command: 'plan'
commandOptions: '-input=false -var-file="../vars/production/production.tfvars"'
environmentServiceNameAzureRM: 'tamopstf'
workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'
- task: TerraformTaskV1@0
displayName: 'apply'
inputs:
provider: 'azurerm'
command: 'apply'
commandOptions: '-input=false -auto-approve -var-file="../vars/production/production.tfvars"'
environmentServiceNameAzureRM: 'tamopstf'
workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'
Reviewing the Pipeline in Azure DevOps
We can download the artifact to review, I have downloaded the artifact and screenshot taken below:-
If we review the steps in terraform_apply stage, we can see Checkout was ignored
Awesome, I have deployed terraform using the artifact created during the terraform_plan stage
Github Repo here containing required Terraform configuration & Pipelines to deploy the above
Thomas Thornton - Microsoft Azure MVP - GitHub Star - HashiCorp Ambassador 
Great article. Please keep on posting such stuff. Really appreciate it.
Hi Lakshay, really happy to hear that feedback! Glad you enjoyed it. Feel free to subscribe to my blog and check out my other posts
Thanks
Thomas
Maybe I’m missing something, but shouldn’t you refer to the tfplan file when you do terraform apply?
To be sure that no external changes has been done while being between stages?
Hi Martin,
In this example and mentioned in the blog – it will be running terraform apply against the plan that is created within the build artifact. Can help with the approval process; depending on the additional requirements and sometimes; the actual time it takes for approval – you will want to apply what has been agreed and approved.
It can be down to the actual requirements from a customer/environment perspective, this is why I blogged about build artifacts and then approval gates.
In relation to “external changes”, it should be done via code and if using the same pipeline it will be added to the next “build”
Thanks
Thomas
Hi Thomas,
Great article; thanks so much!
I’m having a few issues with the Azure-managed build agent actually utilising the artifact. It keeps failing at the Terraform Apply stage, saying that “Error: stat 39-tfplan.tgz: no such file or directory”. I’ve changed my EXTRACT FILES step to extract the file into the same directory as the rest of the Terraform files, in case that was it, but nothing.
Any ideas?
Hi Jonno,
Could you email me your pipeline in a readable format and errors from pipeline?
Thanks
Thomas