Create an Azure DevOps self-hosted linux agent virtual machine using Terraform

Using Azure DevOps & wanting to deploy an Azure DevOps self-hosted agent using Terraform? In this blog I will show you how by creating all the required Azure resources for IaaS self-hosted agent!

Why use an Azure DevOps self-hosted agent?

Numerous reasons to why you may want to use an self-hosted agent > Microsoft hosted agent; I will actually look at blogging this at a later stage. Some potential reasons & benefits:-

• Microsoft hosted agents have a predefined 60 minute use per run
• Self-hosted agents can be built to your specification of hardware
• Custom software/packages etc can be pre-deployed; speeding up the build/deployment process
• You own the self-hosted agent; patching/security updates are to be maintained

What will be deployed via Terraform?

• Resource Group to where all related Azure DevOps self-hosted agent resources will be situated
• Virtual Network (vNet)
• Key Vault & Secret Generated (In this example; I will use password authentication for ssh access)
• Virtual Machine & additional configuration for setup of Azure DevOps self-hosted agent
• Network Security Group (NSG)

Prerequisites

Prior to running the Terraform deployment; there is some prerequisites that are required to be setup/configured

1. Generate a PAT token

Documented here & for now, store the token locally

2. Create new agent pool

In this example; I am going to be creating a new agent pool to where the self-hosted agent will be situated

Select Agent Pools within Organisation settings

Add pool similar to the below, I will be using thomasthorntoncloud

Time to Terraform!

Pull Terraform repository from here

Review script.sh, update PAT token as hi-lighted below (as Terraform is going to be ran locally – if it is going to be ran remote – we would need to review how this PAT token would be stored)

This script.sh – will:

• Creates directory & download ADO agent install files
• Perform unattended install
• Configure the agent pool to run as a service (using user account: thomasthorntoncloud)
• Start the service

#!/bin/sh

# Creates directory & download ADO agent install files
mkdir myagent && cd myagent
wget https://vstsagentpackage.azureedge.net/agent/2.186.1/vsts-agent-linux-x64-2.186.1.tar.gz
tar zxvf vsts-agent-linux-x64-2.186.1.tar.gz

# Unattended install
./config.sh --unattended \
  --agent "${AZP_AGENT_NAME:-$(hostname)}" \
  --url "https://dev.azure.com/thomasthorntoncloud" \
  --auth PAT \
  --token "<INSERT_TOKEN_HERE>" \
  --pool "thomasthorntoncloud" \
  --replace \
  --acceptTeeEula & wait $!

cd /home/thomasthorntoncloud/
#Configure as a service
sudo ./svc.sh install thomasthorntoncloud

#Start svc
sudo ./svc.sh start

Running Terraform locally!

Ensure you are in the Terraform Folder (review plan prior to accepting the apply)

tamops.tfvars will store the .tfvars required to build the environment

tamops.tfvars

# Create Resource Group
rg_name = "thomasthorntoncloud-ado-agent"

# Create Virtual Network
vnet_name                     = "thomasthorntoncloud-ado-agent-vnet"
vnet_ip_address               = ["192.168.0.0/16"]
subnet_name_ado_agent         = "ado-agent-subnet"
subnet_name_ado_agent_address = "192.168.0.0/24"

# Key Vault
key_vault_name = "thomasthorntoncloud-ado"

# Create Virtual Machine
vm_name               = "thomasthorntoncloud-ado-vm"
vm_private_ip_address = "192.168.0.4"
vm_pip_name           = "thomasthorntoncloud-ado-agent-pip"
vm_username           = "thomasthorntoncloud"
vm_osdisk_name        = "thomasthorntoncloud-ado-vm-osdisk"

# Network Security Group
nsg_name = "thomasthorntoncloud-ado-vm-nsg"

terraform init
terraform apply --var-file tamops.tfvars

The Azure required resources will deploy

Reviewing the Azure DevOps Agent Pool thomasthorntoncloud – a new agent has been configured and showing as active – time to test with an example pipeline!

Example Azure DevOps pipeline (Notice the reference to pool:)

name: $(BuildDefinitionName)_$(date:yyyyMMdd)$(rev:.r)

pool: thomasthorntoncloud

# Don't run against PRs
pr: none

stages :
  - stage: Test
    jobs:
    - job: testexample
      continueOnError: false
      steps:
        - task: Bash@3
          displayName: 'Test Echo'
          inputs:
            targetType: inline
            script: |
              echo "Testing thomasthornton.cloud ADO Agent"

The successful pipeline output

Awesome! Successfully deployed via Terraform an Azure DevOps self-hosted agent IaaS VM!

GitHub Repository for all code used