Azure DevOps pipeline templates allow you to create multiple types of templates that you can define and reuse in multiple pipelines. In this blog post I am going to show how you can create template jobs! Each stage will have its own templated job that has multiple tasks. In this example I will be templating a Terraform validate, plan and apply stages
Why create templates?
Some reasons into why I consider creating templates as part of my Azure DevOps pipelines:-
- Reusability
- Avoids duplication
- Reduces complexity and size of creating a single pipeline
- Set structure and pattern that I can follow throughout my pipelines
- Used as a guide to create further stages
- Saves time and allows me to create generic templates that I can use on multiple projects
I recommend checking out this blog post, Azure DevOps Pipelines – Keeping your pipelines DRY (Don’t Repeat Yourself) – it is following my thoughts and suggestions of keeping your pipelines DRY!
Pipeline Structure
Below is the folder & pipeline structure that I will be using:-
- pipelines folder:- The main pipeline along with the three templated jobs for terraform validate, plan & apply
- terraform folder:- Terraform resources that I want to deploy
- vars folder:- .tfvars for various environments, in this example it will just be production.tfvars
Notice the multiple references of .yaml?
pipeline.yaml ( pipeline.yaml GitHub )will be the main pipeline that I will be running within Azure DevOps and the templates/* are the templated pipeline jobs that I will be running within each stage:-
Breakdown of the main pipeline used in Azure DevOps
Lets look at the pipeline that I will be running within Azure DevOps – pipeline.yaml
name: $(BuildDefinitionName)_$(date:yyyyMMdd)$(rev:.r)
trigger: none
pr: none
variables:
backendServiceArm: 'tamopstf2'
backendAzureRmResourceGroupName: 'tamopstfstates'
backendAzureRmStorageAccountName: 'tfstatedevops'
backendAzureRmContainerName: 'azure-devops-template-pipelines'
backendAzureRmKey: 'terraform.tfstate'
environment: production
stages :
- stage: terraform_validate
jobs:
- template: templates/terraform-validate.yaml
parameters:
backendServiceArm: ${{ variables.backendServiceArm }}
backendAzureRmResourceGroupName: ${{ variables.backendAzureRmResourceGroupName }}
backendAzureRmStorageAccountName: ${{ variables.backendAzureRmStorageAccountName }}
backendAzureRmContainerName: ${{ variables.backendAzureRmContainerName }}
backendAzureRmKey: ${{ variables.backendAzureRmKey }}
environment: ${{ variables.environment }}
- stage: terraform_plan
dependsOn: [terraform_validate]
condition: succeeded('terraform_validate')
jobs:
- template: templates/terraform-plan.yaml
parameters:
backendServiceArm: ${{ variables.backendServiceArm }}
backendAzureRmResourceGroupName: ${{ variables.backendAzureRmResourceGroupName }}
backendAzureRmStorageAccountName: ${{ variables.backendAzureRmStorageAccountName }}
backendAzureRmContainerName: ${{ variables.backendAzureRmContainerName }}
backendAzureRmKey: ${{ variables.backendAzureRmKey }}
environment: ${{ variables.environment }}
- stage: terraform_apply
dependsOn: [terraform_plan]
condition: succeeded('terraform_plan')
jobs:
- template: templates/terraform-apply.yaml
parameters:
backendServiceArm: ${{ variables.backendServiceArm }}
backendAzureRmResourceGroupName: ${{ variables.backendAzureRmResourceGroupName }}
backendAzureRmStorageAccountName: ${{ variables.backendAzureRmStorageAccountName }}
backendAzureRmContainerName: ${{ variables.backendAzureRmContainerName }}
backendAzureRmKey: ${{ variables.backendAzureRmKey }}
environment: ${{ variables.environment }}
Within this example, I do not want the pipeline to run during a pull-request(pr) or to be triggered so I have added these references:-
trigger: none
pr: none
Notice in my referenced blog post above Azure DevOps Pipelines – Keeping your pipelines DRY (Don’t Repeat Yourself) that I reference the use of variables and to avoid duplication?
I’ve used variables that I reference within each templated job as below:-
variables:
backendServiceArm: 'tamopstf2'
backendAzureRmResourceGroupName: 'tamopstfstates'
backendAzureRmStorageAccountName: 'tfstatedevops'
backendAzureRmContainerName: 'azure-devops-template-pipelines'
backendAzureRmKey: 'terraform.tfstate'
environment: production
Within each template I make reference to these variables, lets have a look at this. Checking out any of the stages, they are created using the same process. I am going to show a breakdown of the terraform_plan stage.
Notice the use of variables (${{ variables.backendServiceArm }} etc) as these are now referenced as parameters?
- stage: terraform_plan
dependsOn: [terraform_validate]
condition: succeeded('terraform_validate')
jobs:
- template: templates/terraform-plan.yaml
parameters:
backendServiceArm: ${{ variables.backendServiceArm }}
backendAzureRmResourceGroupName: ${{ variables.backendAzureRmResourceGroupName }}
backendAzureRmStorageAccountName: ${{ variables.backendAzureRmStorageAccountName }}
backendAzureRmContainerName: ${{ variables.backendAzureRmContainerName }}
backendAzureRmKey: ${{ variables.backendAzureRmKey }}
environment: ${{ variables.environment }}
Referencing another pipeline/template is down via
- template: templates/terraform-plan.yaml
Why parameters? Lets check out a template!
Within each of my templates I’ve created – I can make reference to these variables from the main pipeline!
Looking at the below terraform-plan.yaml you can see reference to this:-
jobs:
- job: terraform_plan
steps:
- task: TerraformInstaller@0
displayName: 'install'
inputs:
terraformVersion: '0.13.4'
- task: TerraformTaskV1@0
displayName: 'init'
inputs:
provider: 'azurerm'
command: 'init'
backendServiceArm: '${{ parameters.backendServiceArm }}'
backendAzureRmResourceGroupName: '${{ parameters.backendAzureRmResourceGroupName }}'
backendAzureRmStorageAccountName: '${{ parameters.backendAzureRmStorageAccountName }}'
backendAzureRmContainerName: '${{ parameters.backendAzureRmContainerName }}'
backendAzureRmKey: '${{ parameters.backendAzureRmKey }}'
workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'
- task: TerraformTaskV1@0
displayName: 'plan'
inputs:
provider: 'azurerm'
command: 'plan'
commandOptions: '-input=false -var-file="../vars/${{ parameters.environment }}/${{ parameters.environment }}.tfvars"'
environmentServiceNameAzureRM: '${{ parameters.backendServiceArm }}'
workingDirectory: '$(System.DefaultWorkingDirectory)/terraform/'
Awesome isn’t it?
What is being deployed?
In this example, I am deploying a resource group:- templates-rg , check out production.tfvars – you can template further, per environment! Notice how we can reuse templates? Constantly trying to keep them pipelines DRY!
Round-up
Hope you have enjoyed this blog post regarding Templating Azure DevOps Pipeline Jobs and how you can reference multiple .yaml files within the same pipeline!
The code used has been uploaded to my Github repo here
