Menu

Virtual Geek

Tales from real IT system administrators world and non-production environment

Configure Azure Storage Account Blob as Terraform backend to store tfstate file

In the HashiCorp Terraform context state file is a crucial file when deploying resources through terraform configuration tf file. State file's extension is .tfstate. It is a JSON information/documentation that stores and records details about your organization infrastructure and its configuration. This contains resource IDs, attributes, and dependencies. Terraform uses this data to monitor resource states, handles dependencies, and ensure the actual infrastructure aligns with the desired configuration.

In Terraform world, Backend is a configuration block in tf configuration file. It defines how and where you can keep and store state file. There are two types of backend where you can store state file - Local and Remote

When no backend block is defined inside you tf file, after initial initialization with terraform init, will generally look for remote backend configuration, download providers plugins. If it doesn't find such configuration will do nothing. But as you run command terraform apply in the directory, it will create local backend terraform.tfstate file.

As you can see below in the screenshot terraform.tfstate file is generated and the json structure of the applied configuration and settings.

Microsoft Azure Terraform hashicorp  terraform init successfully initialized output function subnet all functions tutorial step by step plan apply auto approve configuration tf file.png

I added below backend connection block to the my tf file. For this testing I already have created Resource Group, Storage account (with blob access AD role permissions) and container.

terraform {
  backend "azurerm" {
    resource_group_name  = "vcloud-lab.com"            # Can be passed via `-backend-config=`"resource_group_name=<resource group name>"` in the `init` command.
    storage_account_name = "vcloudlabtfstate"          # Can be passed via `-backend-config=`"storage_account_name=<storage account name>"` in the `init` command.
    container_name       = "tfstate"                   # Can be passed via `-backend-config=`"container_name=<container name>"` in the `init` command.
    key                  = "example1.terraform.tfstate" # Can be passed via `-backend-config=`"key=<blob key name>"` in the `init` command. File name of terraform.tfstate file
    use_azuread_auth     = true                        # Can also be set via `ARM_USE_AZUREAD` environment variable.
  }
}

By keeping remote backend for the Terraform state file offers numerous advantages:

  1. Security:
    • Encryption: Remote backends frequently support encryption at rest, safeguarding that sensitive/restricted data is protected.
    • Central Management: Only authorized users can access the state file. Access can be restricted. Keeping the state file in a remote backend permits for centralized management of sensitive data.
  2. Consistency and Reliability:
    • Durability: With cloud storage services for the backend guarantees high Accessibility, resilience and availability of the state file.
    • Backups: Remote backends usually offer integral mechanisms for backup and recovery of the state file.
  3. Automation:
    • CI/CD Integration: Storing the state file in a remote backend permits for unified incorporation with CI/CD pipelines, allowing automated infrastructure provisioning and management.
  4. Collaboration:
    • Locking: Many remote backends support state locking, which stops multiple Terraform processes from being run concurrently, dropping the risk of conflicts and corruption.
    • Shared State: When multiple team members are working on the same infrastructure, having the state file in a remote backend ensures that everyone is working with the same data. This prevents conflicts and ensures consistency.

After adding backend block and running apply command. I see it is failing with error with reason that azurerm is backend configured. To adopt new changes and migrate state file, it will need to run terraform init with either -reconfigure or -migrate-state argument again. It will copy the state to remote backend. If you haven't provided any argument flag it will ask if you want to proceed with yes.

Once new configuration is successfully initialized. You can see state is copied to remote backend and local terraform.tfstate file is emptied.

Microsoft Azure Terraform hashcorp terraform init copy existing state to the new backend provider configuration pre-existing state aquiring state lock tfstate apply terraform hashicorp no changes added destroyed changed infrastructure.png

Related Article:  Create storage account and Service Principal using PowerShell for Terraform Azure Backend

You can verify inside Azure Storage account under assigned blob container filename example.terraform.tfstate is created and it has same content of local file. (local files content will be blanked out once state is moved to remote)

Microsoft Azure storage account blob example terraform tfstate backend configuration snapshots terraform backend cloud configuration hcl json tfstate backend provider edit file.png

Multiple Authentication steps to the backend.
You can authenticate to backend using one of the following methods:

  • Environment Variables

This is the most common way to Set the below environment variables to authenticate using a service principal: 

export ARM_CLIENT_ID="<your-service-principal-client-id>"
export ARM_CLIENT_SECRET="<your-service-principal-client-secret>"
export ARM_SUBSCRIPTION_ID="<your-subscription-id>"
export ARM_TENANT_ID="<your-tenant-id>"

terraform init

For Windows OS, use $env in PowerShell or use environment variables box to configure the settings.

  • Azure CLI

This is also another common method, If you have authenticated using Azure CLI (e.g., az login), Terraform can use the CLI session to authenticate.

az login
terraform init
  • Managed Identity

If running Terraform from an Azure VM or other Azure service with a managed identity, ensure the managed identity has the necessary permissions to the storage account.

terraform init
  • Optional Backend Configuration using -backend-config flag

You can also pass backend configuration parameters at the time of initialization using the -backend-config flag:

terraform init \
  -backend-config="resource_group_name=StorageAccount-ResourceGroup" \
  -backend-config="storage_account_name=abcd1234" \
  -backend-config="container_name=tfstate" \
  -backend-config="key=prod.terraform.tfstate"

This is my entire code, basically it has general in use terraform functions examples, You can download it here Terraform_Azure_Backend_Example.tf or it is also available on github.com.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
terraform {
  backend "azurerm" {
    resource_group_name  = "vcloud-lab.com"            # Can be passed via `-backend-config=`"resource_group_name=<resource group name>"` in the `init` command.
    storage_account_name = "vcloudlabtfstate"          # Can be passed via `-backend-config=`"storage_account_name=<storage account name>"` in the `init` command.
    container_name       = "tfstate"                   # Can be passed via `-backend-config=`"container_name=<container name>"` in the `init` command.
    key                  = "example.terraform.tfstate" # Can be passed via `-backend-config=`"key=<blob key name>"` in the `init` command.
    use_azuread_auth     = true                        # Can also be set via `ARM_USE_AZUREAD` environment variable.
  }
}

locals {
  # String Functions
  upper_string   = upper("hello")                # "HELLO"
  lower_string   = lower("HELLO")                # "hello"
  replace_string = replace("hello", "e", "a")    # "hallo"
  substr_string  = substr("hello", 1, 3)         # "ell"
  length_string  = length("hello")               # 5
  format_string  = format("Hello, %s!", "World") # "Hello, World!"

  # Collection Functions
  list         = ["one", "two", "three"]
  length_list  = length(local.list)                 # 3
  merge_map    = merge({ a = 1, b = 2 }, { c = 3 }) # {a = 1, b = 2, c = 3}
  concat_list  = concat(["a", "b"], ["c", "d"])     # ["a", "b", "c", "d"]
  flatten_list = flatten([["a", "b"], ["c", "d"]])  # ["a", "b", "c", "d"]
  sort_list    = sort(["b", "a", "c"])              # ["a", "b", "c"]

  # Numeric Functions
  add_numbers      = 1 + 2        # 3
  subtract_numbers = 5 - 3        # 2
  multiply_numbers = 2 * 3        # 6
  divide_numbers   = 10 / 2       # 5
  min_number       = min(1, 2, 3) # 1
  max_number       = max(1, 2, 3) # 3
  abs_number       = abs(-5)      # 5

  # Logical Functions
  bool_and = true && false # false
  bool_or  = true || false # true
  not_bool = !true         # false

  # Date and Time Functions
  timestamp      = timestamp()                           # Current UTC time in RFC 3339 format
  formatted_date = formatdate("YYYY-MM-DD", timestamp()) # e.g., "2023-07-01"

  # IP Address Functions
  cidr    = "192.168.0.0/16"
  subnet1 = cidrsubnet(local.cidr, 8, 1) // "192.168.1.0/24" #cidrsubnet(prefix, newbits, netnum)
  subnet2 = cidrsubnet(local.cidr, 7, 2) // "192.168.1.0/24" #cidrsubnet(prefix, newbits, netnum)
  host    = cidrhost(local.subnet1, 5)   # "192.168.1.5"

  # Type Conversion Functions
  bool_to_string   = tostring(true)  # "true"
  number_to_string = tostring(123)   # "123"
  string_to_number = tonumber("123") # 123

  # Control Structures and Conditional Functions
  conditional_value = true ? "yes" : "no"              # "yes"
  element_value     = element(["a", "b", "c"], 1)      # "b"
  lookup_value      = lookup({ a = 1, b = 2 }, "b", 0) # 2

  # Validation Functions
  contains_example = contains(["a", "b", "c"], "b") # true

  # Additional Examples
  environment        = upper("dev") # "DEV"
  servers            = ["web1", "web2", "web3"]
  server_count       = length(local.servers) # 3
  server_name_prefix = local.server_count > 2 ? "large-cluster" : "small-cluster"
  server_names       = [for server in local.servers : format("%s-%s", local.server_name_prefix, server)]
  deployment_date    = formatdate("YYYY-MM-DD", timestamp()) # e.g., "2023-07-01"
  network_cidr       = "192.168.0.0/16"
  public_subnet      = cidrsubnet(local.network_cidr, 8, 1) # "192.168.1.0/24"
}

output "functions" {
  value = local.subnet2
}

Following is the text output after backend configuration initialization and applied the configuration.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
PS D:\Projects\Terraform\\Azure_Backend> terraform apply --auto-approve

 Error: Backend initialization required, please run "terraform init"
 
 Reason: Initial configuration of the requested backend "azurerm"
 
 The "backend" is the interface that Terraform uses to store state,
 perform operations, etc. If this message is showing up, it means that the
 Terraform configuration you're using is using a custom configuration for
 the Terraform backend.
 
 Changes to backend configurations require reinitialization. This allows
 Terraform to set up the new configuration, copy existing state, etc. Please run
 "terraform init" with either the "-reconfigure" or "-migrate-state" flags to
 use the current configuration.
 
 If the change reason above is incorrect, please verify your configuration
 hasn't changed and try again. At this point, no changes to your existing
 configuration or state have been made.

PS D:\Projects\Terraform\\Azure_Backend> 
PS D:\Projects\Terraform\\Azure_Backend> terraform init
Initializing the backend...
Acquiring state lock. This may take a few moments...
Do you want to copy existing state to the new backend?
  Pre-existing state was found while migrating the previous "local" backend to the
  newly configured "azurerm" backend. No existing state was found in the newly
  configured "azurerm" backend. Do you want to copy this state to the new "azurerm"
  backend? Enter "yes" to copy and "no" to start with an empty state.

  Enter a value: yes

Releasing state lock. This may take a few moments...

Successfully configured the backend "azurerm"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
PS D:\Projects\Terraform\\Azure_Backend> 
PS D:\Projects\Terraform\\Azure_Backend> terraform apply --auto-approve
Acquiring state lock. This may take a few moments...

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.
Releasing state lock. This may take a few moments...

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

functions = "192.168.1.0/24"
PS D:\Projects\Terraform\\Azure_Backend> 

For more information check official documentation https://developer.hashicorp.com/terraform/language/settings/backends/azurerm.

Useful Articles
Terraform variable validation example
Terraform create Azure Virtual Machines from map of objects
Terraform refactoring moved block example
Terraform create Azure Virtual Machines from map of objects
Hashicorp Terraform map and object inside module and variable example
Terraform one module deploy null or multiple resources based on input
Terraform A reference to a resource type must be followed by at least one attribute access, specifying the resource name
Terraform fore_each for loop filter with if condition example
Terraform remote-exec provisioner with ssh connection in null_resource
Terraform count vs for_each for examples with map of objects
Terraform one module deploy null or multiple resources based on input (nested for loop) Example of Terraform functions flatten() and coalesce()
Terraform Azure Create Private Endpoint to existing Storage Account with Custom Private DNS zone record link
Creating a Private Endpoint for Azure Storage Account with required sub services using Terraform Example Terraform functions of lookup() and lower()
Using element function with count meta argument example Terraform Azure subnets Example Terraform functions of element()count() and sum()
Terraform create Azure Virtual Network subnets from map of object and show name in the header Header name change in the output
Creating a Private Endpoint for Azure Storage Account with Terraform example 2 Example of for_each with toset() function
Creating a Private Endpoint for Azure Storage Account with Terraform example 3

Go Back

Comment

Blog Search

Page Views

11955117

Follow me on Blogarama