In my previous blog post, I detailed how to import 3 Azure resources until Terraform state, a manual approach that is fine for a few resources; what happens if there are 100s of the same resource already configured in Azure?
Recently in my current project I was wanting to import 300+ private DNS A Records into terraform state, this couldn’t have been a manual approach as it would have taken so long!
What did I decide to do? Create a script to do this using bash/AZ CLI
I’ve decided to blog about it as this type of script can be used for numerous scenarios – in total I imported 300+ private DNS A records using this approach!
What I was importing – some A Records were not previously created, my script had to check if the resource was already there and if it was to import it into terraform state; if not – just to echo so I can check at a later stage and import it during my terraform apply.
In this blog post, I am going to import a number of subnets within a virtual network, the script will check if the subnet is already created or not.
Folder structure for this example
Terraform-Edit-State-At-Scale
└──main.tf
└──provider.tf
└──script.sh
Terraform init
terraform init with remote state in provider.tf
provider.tf
terraform {
required_version = ">= 0.13.0"
backend "azure" {
resource_group_name = "tamopstfstates"
storage_account_name = "tfstatedevops"
container_name = "terraformstateatscale"
key = "terraform.tfstate"
}
}
main.tf (Note i’ve copy/pasted “azurerm_subnet” multiple times for simplicity – in actual Terraform I would have used a for_each
# Create Resource Group
resource "azurerm_resource_group" "tamops" {
name = "tamops"
location = "eastus2"
}
# Create Virtual Network
resource "azurerm_virtual_network" "vnet" {
name = "tamops-vnet"
address_space = ["192.168.0.0/16"]
location = "eastus2"
resource_group_name = azurerm_resource_group.tamops.name
}
# Create Subnet 1
resource "azurerm_subnet" "subnet1" {
name = "subnet1"
resource_group_name = azurerm_resource_group.tamops.name
virtual_network_name = azurerm_virtual_network.vnet.name
address_prefix = "192.168.1.0/24"
}
# Create Subnet 2
resource "azurerm_subnet" "subnet2" {
name = "subnet2"
resource_group_name = azurerm_resource_group.tamops.name
virtual_network_name = azurerm_virtual_network.vnet.name
address_prefix = "192.168.2.0/24"
}
# Create Subnet 3
resource "azurerm_subnet" "subnet3" {
name = "subnet3"
resource_group_name = azurerm_resource_group.tamops.name
virtual_network_name = azurerm_virtual_network.vnet.name
address_prefix = "192.168.3.0/24"
}
# Create Subnet 4
resource "azurerm_subnet" "subnet4" {
name = "subnet4"
resource_group_name = azurerm_resource_group.tamops.name
virtual_network_name = azurerm_virtual_network.vnet.name
address_prefix = "192.168.4.0/24"
}
# Create Subnet 5
resource "azurerm_subnet" "subnet5" {
name = "subnet5"
resource_group_name = azurerm_resource_group.tamops.name
virtual_network_name = azurerm_virtual_network.vnet.name
address_prefix = "192.168.5.0/24"
}
# Create Subnet 6
resource "azurerm_subnet" "subnet6" {
name = "subnet6"
resource_group_name = azurerm_resource_group.tamops.name
virtual_network_name = azurerm_virtual_network.vnet.name
address_prefix = "192.168.6.0/24"
}
# Create Subnet 7
resource "azurerm_subnet" "subnet7" {
name = "subnet7"
resource_group_name = azurerm_resource_group.tamops.name
virtual_network_name = azurerm_virtual_network.vnet.name
address_prefix = "192.168.7.0/24"
}
resource "azurerm_subnet" "subnet8" {
name = "subnet8"
resource_group_name = azurerm_resource_group.tamops.name
virtual_network_name = azurerm_virtual_network.vnet.name
address_prefix = "192.168.8.0/24"
}
resource "azurerm_subnet" "subnet9" {
name = "subnet9"
resource_group_name = azurerm_resource_group.tamops.name
virtual_network_name = azurerm_virtual_network.vnet.name
address_prefix = "192.168.9.0/24"
}
Terraform Importing
Lets check to see what subnets are deployed already in Azure
For this example, I have previously imported into Terraform state both the resource group and virtual network
Running another plan, we can Terraform wants to import all subnets, whereas there is already 7 subnets configured in the Virtual Network
terraform plan
Acquiring state lock. This may take a few moments...
azurerm_resource_group.tamops: Refreshing state... [id=/subscriptions/XXXXXXXXXXXXXXXXXXXXX/resourceGroups/tamops]
azurerm_virtual_network.vnet: Refreshing state... [id=/subscriptions/XXXXXXXXXXXXXXXXXXXXX/resourceGroups/tamops/providers/Microsoft.Network/virtualNetworks/tamops-vnet]
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# azurerm_subnet.subnet1 will be created
+ resource "azurerm_subnet" "subnet1" {
+ address_prefix = "192.168.1.0/24"
+ address_prefixes = (known after apply)
+ enforce_private_link_endpoint_network_policies = false
+ enforce_private_link_service_network_policies = false
+ id = (known after apply)
+ name = "subnet1"
+ resource_group_name = "tamops"
+ virtual_network_name = "tamops-vnet"
}
# azurerm_subnet.subnet2 will be created
+ resource "azurerm_subnet" "subnet2" {
+ address_prefix = "192.168.2.0/24"
+ address_prefixes = (known after apply)
+ enforce_private_link_endpoint_network_policies = false
+ enforce_private_link_service_network_policies = false
+ id = (known after apply)
+ name = "subnet2"
+ resource_group_name = "tamops"
+ virtual_network_name = "tamops-vnet"
}
# azurerm_subnet.subnet3 will be created
+ resource "azurerm_subnet" "subnet3" {
+ address_prefix = "192.168.3.0/24"
+ address_prefixes = (known after apply)
+ enforce_private_link_endpoint_network_policies = false
+ enforce_private_link_service_network_policies = false
+ id = (known after apply)
+ name = "subnet3"
+ resource_group_name = "tamops"
+ virtual_network_name = "tamops-vnet"
}
# azurerm_subnet.subnet4 will be created
+ resource "azurerm_subnet" "subnet4" {
+ address_prefix = "192.168.4.0/24"
+ address_prefixes = (known after apply)
+ enforce_private_link_endpoint_network_policies = false
+ enforce_private_link_service_network_policies = false
+ id = (known after apply)
+ name = "subnet4"
+ resource_group_name = "tamops"
+ virtual_network_name = "tamops-vnet"
}
# azurerm_subnet.subnet5 will be created
+ resource "azurerm_subnet" "subnet5" {
+ address_prefix = "192.168.5.0/24"
+ address_prefixes = (known after apply)
+ enforce_private_link_endpoint_network_policies = false
+ enforce_private_link_service_network_policies = false
+ id = (known after apply)
+ name = "subnet5"
+ resource_group_name = "tamops"
+ virtual_network_name = "tamops-vnet"
}
# azurerm_subnet.subnet6 will be created
+ resource "azurerm_subnet" "subnet6" {
+ address_prefix = "192.168.6.0/24"
+ address_prefixes = (known after apply)
+ enforce_private_link_endpoint_network_policies = false
+ enforce_private_link_service_network_policies = false
+ id = (known after apply)
+ name = "subnet6"
+ resource_group_name = "tamops"
+ virtual_network_name = "tamops-vnet"
}
# azurerm_subnet.subnet7 will be created
+ resource "azurerm_subnet" "subnet7" {
+ address_prefix = "192.168.7.0/24"
+ address_prefixes = (known after apply)
+ enforce_private_link_endpoint_network_policies = false
+ enforce_private_link_service_network_policies = false
+ id = (known after apply)
+ name = "subnet7"
+ resource_group_name = "tamops"
+ virtual_network_name = "tamops-vnet"
}
# azurerm_subnet.subnet8 will be created
+ resource "azurerm_subnet" "subnet8" {
+ address_prefix = "192.168.8.0/24"
+ address_prefixes = (known after apply)
+ enforce_private_link_endpoint_network_policies = false
+ enforce_private_link_service_network_policies = false
+ id = (known after apply)
+ name = "subnet8"
+ resource_group_name = "tamops"
+ virtual_network_name = "tamops-vnet"
}
# azurerm_subnet.subnet9 will be created
+ resource "azurerm_subnet" "subnet9" {
+ address_prefix = "192.168.9.0/24"
+ address_prefixes = (known after apply)
+ enforce_private_link_endpoint_network_policies = false
+ enforce_private_link_service_network_policies = false
+ id = (known after apply)
+ name = "subnet9"
+ resource_group_name = "tamops"
+ virtual_network_name = "tamops-vnet"
}
Plan: 9 to add, 0 to change, 0 to destroy.
As mentioned, when I had 300+ A Records, a manual approach to terraform importing wasn’t possible – I decided to script it using bash/AZ CLI
The script below:-
- Passed in both resource group & virtual network name
- Created an array for subnets
- Each subnet in $SUBNETS will be checked to see if it is already configured in Azure
- If present in Azure – import into terraform state
- If not present, echo subnet is not created
#!/bin/bash
set -e
RESOURCE_GROUP="tamops"
VNET_NAME="tamops-vnet"
SUBSCRIPTION="XXXXXXXXXXXXXXXXXX"
SUBNETS=()
SUBNETS+=("subnet1" "subnet2" "subnet3" "subnet4" "subnet5" "subnet6" "subnet7" "subnet8" "subnet9")
for i in "${!SUBNETS[@]}"; do
SUBNET_CHECK=()
SUBNET_CHECK=($(az network vnet subnet list -g $RESOURCE_GROUP --vnet-name $VNET_NAME --query "[?name=='${SUBNETS[i]}']" -o tsv))
if [ -n "$SUBNET_CHECK" ]
then
terraform import azurerm_subnet.${SUBNETS[i]} /subscriptions/$SUBSCRIPTION/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.Network/virtualNetworks/$VNET_NAME/subnets/${SUBNETS[i]}
else
echo "${SUBNETS[i]} is not created"
fi
done
As mentioned, similar was created for importing 300+ A records, I also automated the initial array also – I wasn’t going to add 300+ records manually into an array 🙂
Example output from the script
Acquiring state lock. This may take a few moments...
azurerm_subnet.subnet7: Importing from ID "/subscriptions/XXXXXXXXXXXXXXXXXX/resourceGroups/tamops/providers/Microsoft.Network/virtualNetworks/tamops-vnet/subnets/subnet7"...
azurerm_subnet.subnet7: Import prepared!
Prepared azurerm_subnet for import
azurerm_subnet.subnet7: Refreshing state... [id=/subscriptions/XXXXXXXXXXXXXXXXXX/resourceGroups/tamops/providers/Microsoft.Network/virtualNetworks/tamops-vnet/subnets/subnet7]
Import successful!
The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.
subnet8 is not created
subnet9 is not created
With the script ran, running a terraform plan – only the subnets not present in Azure will be created
tamops@Synth:/mnt/c/Users/thomast/Documents/thomasthorntoncloud-examples-new/Terraform-Edit-State-At-Scale$ terraform plan
Acquiring state lock. This may take a few moments...
azurerm_resource_group.tamops: Refreshing state... [id=/subscriptions/XXXXXXXXXXXXXXXXXX/resourceGroups/tamops]
azurerm_virtual_network.vnet: Refreshing state... [id=/subscriptions/XXXXXXXXXXXXXXXXXX/resourceGroups/tamops/providers/Microsoft.Network/virtualNetworks/tamops-vnet]
azurerm_subnet.subnet4: Refreshing state... [id=/subscriptions/XXXXXXXXXXXXXXXXXX/resourceGroups/tamops/providers/Microsoft.Network/virtualNetworks/tamops-vnet/subnets/subnet4]
azurerm_subnet.subnet2: Refreshing state... [id=/subscriptions/XXXXXXXXXXXXXXXXXX/resourceGroups/tamops/providers/Microsoft.Network/virtualNetworks/tamops-vnet/subnets/subnet2]
azurerm_subnet.subnet5: Refreshing state... [id=/subscriptions/XXXXXXXXXXXXXXXXXX/resourceGroups/tamops/providers/Microsoft.Network/virtualNetworks/tamops-vnet/subnets/subnet5]
azurerm_subnet.subnet6: Refreshing state... [id=/subscriptions/XXXXXXXXXXXXXXXXXX/resourceGroups/tamops/providers/Microsoft.Network/virtualNetworks/tamops-vnet/subnets/subnet6]
azurerm_subnet.subnet1: Refreshing state... [id=/subscriptions/XXXXXXXXXXXXXXXXXX/resourceGroups/tamops/providers/Microsoft.Network/virtualNetworks/tamops-vnet/subnets/subnet1]
azurerm_subnet.subnet3: Refreshing state... [id=/subscriptions/XXXXXXXXXXXXXXXXXX/resourceGroups/tamops/providers/Microsoft.Network/virtualNetworks/tamops-vnet/subnets/subnet3]
azurerm_subnet.subnet7: Refreshing state... [id=/subscriptions/XXXXXXXXXXXXXXXXXX/resourceGroups/tamops/providers/Microsoft.Network/virtualNetworks/tamops-vnet/subnets/subnet7]
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# azurerm_subnet.subnet8 will be created
+ resource "azurerm_subnet" "subnet8" {
+ address_prefix = "192.168.8.0/24"
+ address_prefixes = (known after apply)
+ enforce_private_link_endpoint_network_policies = false
+ enforce_private_link_service_network_policies = false
+ id = (known after apply)
+ name = "subnet8"
+ resource_group_name = "tamops"
+ virtual_network_name = "tamops-vnet"
}
# azurerm_subnet.subnet9 will be created
+ resource "azurerm_subnet" "subnet9" {
+ address_prefix = "192.168.9.0/24"
+ address_prefixes = (known after apply)
+ enforce_private_link_endpoint_network_policies = false
+ enforce_private_link_service_network_policies = false
+ id = (known after apply)
+ name = "subnet9"
+ resource_group_name = "tamops"
+ virtual_network_name = "tamops-vnet"
}
Plan: 2 to add, 0 to change, 0 to destroy.
Awesome! Hopefully you’ve enjoyed this blog post and an insight into how you can begin Importing Terraform state at scale in Azure!
Thomas Thornton - Microsoft Azure MVP - HashiCorp Ambassador 
Hi Thomas, thanks for sharing this – I’m wondering if there isn’t a simple way to import all resources from a resource group into terraform?
I have a resource group with many resources inside, can’t I just tell terraform to import everything that’s part of the rg? Your solution is interesting but it still requires some manual considerations of the different components to import. I didn’t find a solution to “blindly” import the entire content of a resource group.
Would you have a solution to that?
Cheers
Guillaume
Hi Guillaume,
Afraid there isn’t something standard for this. As Terraform recognises each Azure resource type as a different component.
You need to write the appropriate terraform code before you try to import
Thanks
Thomas