Ensuring Terraform State Security with Ephemeral Values and Write-Only Outputs


Ephemeral resources are Terraform resources that are essentially temporary. Ephemeral resources have a unique lifecycle, and Terraform does not store information about ephemeral resources in state or plan files. Each ephemeral block describes one or more ephemeral resources, such as a temporary password or connection to another system. (you must use Terraform v.1.11 or later and use a resource that supports write-only arguments)

In this blog post, I will cover what are ephmeral resources, why write-only arguments should be used and an example both being used in Azure to store and retrieve sensitive values without them being stored in the state file.

Lets start by covering what are ephmeral resources and write-only arguments

What are Ephemeral resources?

Ephemeral resources are Terraform resources that are essentially temporary. Ephemeral resources have a unique lifecycle, and Terraform does not store information about ephemeral resources in state or plan files. Each ephemeral block describes one or more ephemeral resources, such as a temporary password or connection to another system.

Ephemeral resources in Terraform are, in simple terms, temporary by design. They have a distinct lifecycle and – here’s the key part – Terraform doesn’t save any details about them in the state or plan files. Each ephemeral block defines one or more short-lived resources, like a one-time-use password or a fleeting connection to another system. Think of them as “now you see it, now you don’t” resources – useful, secure, and gone when no longer needed.

What are write-only arguments?

Terraform doesn’t keep write-only arguments in the state file, which means it has no idea if their values change over time. Since it can’t track them, it plays it safe and sends those values to the provider every single time it runs – whether they’ve changed or not.

Now, because these write-only values aren’t stored in plan files either, Terraform can’t show you any differences (aka plan diffs) when they change. To work around this, most providers include a handy version argument alongside the write-only value. This version is saved in the state, so Terraform can detect changes and trigger updates when needed.

It’s worth noting: how this works can vary between providers.

Show me the Terraform code!

Reviewing terraform state file with a non write-only sensitive value

Lets review the terraform state file when creating a new Key Vault secret called secret-password, using azurerm_key_vault_secret when non write-only value is used.

Example Terraform:

resource "azurerm_key_vault_secret" "tamops-secret" {
  name         = "secret-password"
  value        = "secret-password-value"
  key_vault_id = azurerm_key_vault.tamops-kv.id
}

Reviewing the below Terraform state file snippet, we can see the secret value is viewable:

    {
      "mode": "managed",
      "type": "azurerm_key_vault_secret",
      "name": "tamops-secret",
      "provider": "provider[\"registry.terraform.io/hashicorp/azurerm\"]",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "content_type": "",
            "expiration_date": null,
            "id": "https://tamops-keyvault.vault.azure.net/secrets/secret-password/9d0aa0dd7f5f402ebb71041b994e564e",
            "key_vault_id": "/subscriptions/04109105-f3ca-44ac-a3a7-66b4936112c3/resourceGroups/tamops-resources/providers/Microsoft.KeyVault/vaults/tamops-keyvault",
            "name": "secret-password",
            "not_before_date": null,
            "resource_id": "/subscriptions/04109105-f3ca-44ac-a3a7-66b4936112c3/resourceGroups/tamops-resources/providers/Microsoft.KeyVault/vaults/tamops-keyvault/secrets/secret-password/versions/9d0aa0dd7f5f402ebb71041b994e564e",
            "resource_versionless_id": "/subscriptions/04109105-f3ca-44ac-a3a7-66b4936112c3/resourceGroups/tamops-resources/providers/Microsoft.KeyVault/vaults/tamops-keyvault/secrets/secret-password",
            "tags": null,
            "timeouts": null,
            "value": "secret-password-value",
            "value_wo": null,
            "value_wo_version": 0,
            "version": "9d0aa0dd7f5f402ebb71041b994e564e",
            "versionless_id": "https://tamops-keyvault.vault.azure.net/secrets/secret-password"
          },

Ouch! Lets try the same approach with a write only Terraform value

Review terraform state file with a write-only sensitive value

Similar approach to the approach, same secret & value – but using value_wo & value_wo_version

resource "azurerm_key_vault_secret" "tamops-secret" {
  name         = "secret-password"
  value_wo        = "secret-password-value"
  value_wo_version = "3"
  key_vault_id = azurerm_key_vault.tamops-kv.id
}

Some more context on the above using write only & write only version – Terraform does not store write-only arguments in state files, so Terraform has no way of knowing if a write-only argument value has changed. Because Terraform cannot track write-only argument values, it sends write-only arguments to the provider during every operation.

Terraform can’t show plan differences for write-only arguments because those values never make it into the plan files. So, how does it keep track of changes? That’s where version arguments come in. Most providers pair write-only arguments with a version argument – think of it as a change counter. While the sensitive value itself stays out of the state file, the version number is stored, letting Terraform spot when you’ve updated it.

Why does it matter?

The version argument acts as a signal: when you bump the version, Terraform knows to send the new write-only value to the provider. This clever pairing lets you control when a provider should use a fresh secret or password, even though Terraform never remembers what that secret actually was. Just remember, the exact way this works can vary from one provider to another, so it’s always smart to check the Terraform Registry for the details that apply to your chosen provide

Now lets review the terraform state, the value of the secret is not present – great!

      "mode": "managed",
      "type": "azurerm_key_vault_secret",
      "name": "tamops-secret",
      "provider": "provider[\"registry.terraform.io/hashicorp/azurerm\"]",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "content_type": "",
            "expiration_date": null,
            "id": "https://tamops-keyvault.vault.azure.net/secrets/secret-password/3e4713318b944f5aabec3a7163238d8e",
            "key_vault_id": "/subscriptions/04109105-f3ca-44ac-a3a7-66b4936112c3/resourceGroups/tamops-resources/providers/Microsoft.KeyVault/vaults/tamops-keyvault",
            "name": "secret-password",
            "not_before_date": null,
            "resource_id": "/subscriptions/04109105-f3ca-44ac-a3a7-66b4936112c3/resourceGroups/tamops-resources/providers/Microsoft.KeyVault/vaults/tamops-keyvault/secrets/secret-password/versions/3e4713318b944f5aabec3a7163238d8e",
            "resource_versionless_id": "/subscriptions/04109105-f3ca-44ac-a3a7-66b4936112c3/resourceGroups/tamops-resources/providers/Microsoft.KeyVault/vaults/tamops-keyvault/secrets/secret-password",
            "tags": {},
            "timeouts": null,
            "value": "",
            "value_wo": null,
            "value_wo_version": 3,
            "version": "3e4713318b944f5aabec3a7163238d8e",
            "versionless_id": "https://tamops-keyvault.vault.azure.net/secrets/secret-password"
          },

Set and store an ephemeral password

I touched briefly on Ephemeral resources in Terraform at the start of the blog, lets look at using this resource to set and store an ephemeral password.

What the below example is doing?

  1. Create secret secret-password in azure key vault
  2. Retrieve secret-password value using ephemeral resource
  3. Create secret-password4 with the ephemeral resource value
resource "azurerm_key_vault_secret" "tamops-secret" {
  name         = "secret-password"
  value_wo        = "secret-password-value"
  value_wo_version = "3"
  key_vault_id = azurerm_key_vault.tamops-kv.id
}

ephemeral "azurerm_key_vault_secret" "tamops-secret" {
  name         = azurerm_key_vault_secret.tamops-secret.name
  key_vault_id = azurerm_key_vault.tamops-kv.id
}

resource "azurerm_key_vault_secret" "tamops-secret2" {
  name         = "secret-password4"
  value_wo        = ephemeral.azurerm_key_vault_secret.tamops-secret.value
  value_wo_version = "1"
  key_vault_id = azurerm_key_vault.tamops-kv.id
}

Terraform state snippet below of tamops-secret2, no reference of the value – great!

    {
      "mode": "managed",
      "type": "azurerm_key_vault_secret",
      "name": "tamops-secret2",
      "provider": "provider[\"registry.terraform.io/hashicorp/azurerm\"]",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "content_type": "",
            "expiration_date": null,
            "id": "https://tamops-keyvault.vault.azure.net/secrets/secret-password4/e57af1dbb14f48c8afa0c43dae431e73",
            "key_vault_id": "/subscriptions/04109105-f3ca-44ac-a3a7-66b4936112c3/resourceGroups/tamops-resources/providers/Microsoft.KeyVault/vaults/tamops-keyvault",
            "name": "secret-password4",
            "not_before_date": null,
            "resource_id": "/subscriptions/04109105-f3ca-44ac-a3a7-66b4936112c3/resourceGroups/tamops-resources/providers/Microsoft.KeyVault/vaults/tamops-keyvault/secrets/secret-password4/versions/e57af1dbb14f48c8afa0c43dae431e73",
            "resource_versionless_id": "/subscriptions/04109105-f3ca-44ac-a3a7-66b4936112c3/resourceGroups/tamops-resources/providers/Microsoft.KeyVault/vaults/tamops-keyvault/secrets/secret-password4",
            "tags": null,
            "timeouts": null,
            "value": "",
            "value_wo": null,
            "value_wo_version": 1,
            "version": "e57af1dbb14f48c8afa0c43dae431e73",
            "versionless_id": "https://tamops-keyvault.vault.azure.net/secrets/secret-password4"
          },

Checking in Azure, can confirm the value is correct

Wrapping up

With cloud security making headlines and compliance requirements tightening, keeping secrets out of your state files isn’t just a “nice to have”

The good news? With ephemeral resources and write-only arguments, you’re ahead of the game.

So, next time you’re writing Terraform code, ask yourself: “Where are my secrets going?” If the answer is “my state file,” it’s time to switch things up. Try out ephemeral resources and write-only outputs. Your future self (and your security team) will thank you.

Some additional reading on the topic:

GitHub repo containing example code above


Leave a Reply

Discover more from Thomas Thornton Blog

Subscribe now to keep reading and get access to the full archive.

Continue reading

Discover more from Thomas Thornton Blog

Subscribe now to keep reading and get access to the full archive.

Continue reading