In the HashiCorp Terraform context state file is a crucial file when deploying resources through terraform configuration tf file. State file's extension is .tfstate. It is a JSON information/documentation that stores and records details about your organization infrastructure and its configuration. This contains resource IDs, attributes, and dependencies. Terraform uses this data to monitor resource states, handles dependencies, and ensure the actual infrastructure aligns with the desired configuration.
In Terraform world, Backend is a configuration block in tf configuration file. It defines how and where you can keep and store state file. There are two types of backend where you can store state file - Local and Remote.
When no backend block is defined inside you tf file, after initial initialization with terraform init, will generally look for remote backend configuration, download providers plugins. If it doesn't find such configuration will do nothing. But as you run command terraform apply in the directory, it will create local backend terraform.tfstate file.
As you can see below in the screenshot terraform.tfstate file is generated and the json structure of the applied configuration and settings.
I added below backend connection block to the my tf file. For this testing I already have created Resource Group, Storage account (with blob access AD role permissions) and container.
terraform { backend "azurerm" { resource_group_name = "vcloud-lab.com" # Can be passed via `-backend-config=`"resource_group_name=<resource group name>"` in the `init` command. storage_account_name = "vcloudlabtfstate" # Can be passed via `-backend-config=`"storage_account_name=<storage account name>"` in the `init` command. container_name = "tfstate" # Can be passed via `-backend-config=`"container_name=<container name>"` in the `init` command. key = "example1.terraform.tfstate" # Can be passed via `-backend-config=`"key=<blob key name>"` in the `init` command. File name of terraform.tfstate file use_azuread_auth = true # Can also be set via `ARM_USE_AZUREAD` environment variable. } }
By keeping remote backend for the Terraform state file offers numerous advantages:
- Security:
- Encryption: Remote backends frequently support encryption at rest, safeguarding that sensitive/restricted data is protected.
- Central Management: Only authorized users can access the state file. Access can be restricted. Keeping the state file in a remote backend permits for centralized management of sensitive data.
- Consistency and Reliability:
- Durability: With cloud storage services for the backend guarantees high Accessibility, resilience and availability of the state file.
- Backups: Remote backends usually offer integral mechanisms for backup and recovery of the state file.
- Automation:
- CI/CD Integration: Storing the state file in a remote backend permits for unified incorporation with CI/CD pipelines, allowing automated infrastructure provisioning and management.
- Collaboration:
- Locking: Many remote backends support state locking, which stops multiple Terraform processes from being run concurrently, dropping the risk of conflicts and corruption.
- Shared State: When multiple team members are working on the same infrastructure, having the state file in a remote backend ensures that everyone is working with the same data. This prevents conflicts and ensures consistency.
After adding backend block and running apply command. I see it is failing with error with reason that azurerm is backend configured. To adopt new changes and migrate state file, it will need to run terraform init with either -reconfigure or -migrate-state argument again. It will copy the state to remote backend. If you haven't provided any argument flag it will ask if you want to proceed with yes.
Once new configuration is successfully initialized. You can see state is copied to remote backend and local terraform.tfstate file is emptied.
Related Article: Create storage account and Service Principal using PowerShell for Terraform Azure Backend
You can verify inside Azure Storage account under assigned blob container filename example.terraform.tfstate is created and it has same content of local file. (local files content will be blanked out once state is moved to remote)
Multiple Authentication steps to the backend.
You can authenticate to backend using one of the following methods:
- Environment Variables
This is the most common way to Set the below environment variables to authenticate using a service principal:
export ARM_CLIENT_ID="<your-service-principal-client-id>" export ARM_CLIENT_SECRET="<your-service-principal-client-secret>" export ARM_SUBSCRIPTION_ID="<your-subscription-id>" export ARM_TENANT_ID="<your-tenant-id>" terraform init
For Windows OS, use $env in PowerShell or use environment variables box to configure the settings.
- Azure CLI
This is also another common method, If you have authenticated using Azure CLI (e.g., az login), Terraform can use the CLI session to authenticate.
az login terraform init
- Managed Identity
If running Terraform from an Azure VM or other Azure service with a managed identity, ensure the managed identity has the necessary permissions to the storage account.
terraform init
- Optional Backend Configuration using -backend-config flag
You can also pass backend configuration parameters at the time of initialization using the -backend-config flag:
terraform init \ -backend-config="resource_group_name=StorageAccount-ResourceGroup" \ -backend-config="storage_account_name=abcd1234" \ -backend-config="container_name=tfstate" \ -backend-config="key=prod.terraform.tfstate"
This is my entire code, basically it has general in use terraform functions examples, You can download it here Terraform_Azure_Backend_Example.tf or it is also available on github.com.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 |
terraform { backend "azurerm" { resource_group_name = "vcloud-lab.com" # Can be passed via `-backend-config=`"resource_group_name=<resource group name>"` in the `init` command. storage_account_name = "vcloudlabtfstate" # Can be passed via `-backend-config=`"storage_account_name=<storage account name>"` in the `init` command. container_name = "tfstate" # Can be passed via `-backend-config=`"container_name=<container name>"` in the `init` command. key = "example.terraform.tfstate" # Can be passed via `-backend-config=`"key=<blob key name>"` in the `init` command. use_azuread_auth = true # Can also be set via `ARM_USE_AZUREAD` environment variable. } } locals { # String Functions upper_string = upper("hello") # "HELLO" lower_string = lower("HELLO") # "hello" replace_string = replace("hello", "e", "a") # "hallo" substr_string = substr("hello", 1, 3) # "ell" length_string = length("hello") # 5 format_string = format("Hello, %s!", "World") # "Hello, World!" # Collection Functions list = ["one", "two", "three"] length_list = length(local.list) # 3 merge_map = merge({ a = 1, b = 2 }, { c = 3 }) # {a = 1, b = 2, c = 3} concat_list = concat(["a", "b"], ["c", "d"]) # ["a", "b", "c", "d"] flatten_list = flatten([["a", "b"], ["c", "d"]]) # ["a", "b", "c", "d"] sort_list = sort(["b", "a", "c"]) # ["a", "b", "c"] # Numeric Functions add_numbers = 1 + 2 # 3 subtract_numbers = 5 - 3 # 2 multiply_numbers = 2 * 3 # 6 divide_numbers = 10 / 2 # 5 min_number = min(1, 2, 3) # 1 max_number = max(1, 2, 3) # 3 abs_number = abs(-5) # 5 # Logical Functions bool_and = true && false # false bool_or = true || false # true not_bool = !true # false # Date and Time Functions timestamp = timestamp() # Current UTC time in RFC 3339 format formatted_date = formatdate("YYYY-MM-DD", timestamp()) # e.g., "2023-07-01" # IP Address Functions cidr = "192.168.0.0/16" subnet1 = cidrsubnet(local.cidr, 8, 1) // "192.168.1.0/24" #cidrsubnet(prefix, newbits, netnum) subnet2 = cidrsubnet(local.cidr, 7, 2) // "192.168.1.0/24" #cidrsubnet(prefix, newbits, netnum) host = cidrhost(local.subnet1, 5) # "192.168.1.5" # Type Conversion Functions bool_to_string = tostring(true) # "true" number_to_string = tostring(123) # "123" string_to_number = tonumber("123") # 123 # Control Structures and Conditional Functions conditional_value = true ? "yes" : "no" # "yes" element_value = element(["a", "b", "c"], 1) # "b" lookup_value = lookup({ a = 1, b = 2 }, "b", 0) # 2 # Validation Functions contains_example = contains(["a", "b", "c"], "b") # true # Additional Examples environment = upper("dev") # "DEV" servers = ["web1", "web2", "web3"] server_count = length(local.servers) # 3 server_name_prefix = local.server_count > 2 ? "large-cluster" : "small-cluster" server_names = [for server in local.servers : format("%s-%s", local.server_name_prefix, server)] deployment_date = formatdate("YYYY-MM-DD", timestamp()) # e.g., "2023-07-01" network_cidr = "192.168.0.0/16" public_subnet = cidrsubnet(local.network_cidr, 8, 1) # "192.168.1.0/24" } output "functions" { value = local.subnet2 } |
Following is the text output after backend configuration initialization and applied the configuration.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 |
PS D:\Projects\Terraform\\Azure_Backend> terraform apply --auto-approve ╷ │ Error: Backend initialization required, please run "terraform init" │ │ Reason: Initial configuration of the requested backend "azurerm" │ │ The "backend" is the interface that Terraform uses to store state, │ perform operations, etc. If this message is showing up, it means that the │ Terraform configuration you're using is using a custom configuration for │ the Terraform backend. │ │ Changes to backend configurations require reinitialization. This allows │ Terraform to set up the new configuration, copy existing state, etc. Please run │ "terraform init" with either the "-reconfigure" or "-migrate-state" flags to │ use the current configuration. │ │ If the change reason above is incorrect, please verify your configuration │ hasn't changed and try again. At this point, no changes to your existing │ configuration or state have been made. ╵ PS D:\Projects\Terraform\\Azure_Backend> PS D:\Projects\Terraform\\Azure_Backend> terraform init Initializing the backend... Acquiring state lock. This may take a few moments... Do you want to copy existing state to the new backend? Pre-existing state was found while migrating the previous "local" backend to the newly configured "azurerm" backend. No existing state was found in the newly configured "azurerm" backend. Do you want to copy this state to the new "azurerm" backend? Enter "yes" to copy and "no" to start with an empty state. Enter a value: yes Releasing state lock. This may take a few moments... Successfully configured the backend "azurerm"! Terraform will automatically use this backend unless the backend configuration changes. Initializing provider plugins... Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work. If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary. PS D:\Projects\Terraform\\Azure_Backend> PS D:\Projects\Terraform\\Azure_Backend> terraform apply --auto-approve Acquiring state lock. This may take a few moments... No changes. Your infrastructure matches the configuration. Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed. Releasing state lock. This may take a few moments... Apply complete! Resources: 0 added, 0 changed, 0 destroyed. Outputs: functions = "192.168.1.0/24" PS D:\Projects\Terraform\\Azure_Backend> |
For more information check official documentation https://developer.hashicorp.com/terraform/language/settings/backends/azurerm.
Useful Articles
Terraform variable validation example
Terraform create Azure Virtual Machines from map of objects
Terraform refactoring moved block example
Terraform create Azure Virtual Machines from map of objects
Hashicorp Terraform map and object inside module and variable example
Terraform one module deploy null or multiple resources based on input
Terraform A reference to a resource type must be followed by at least one attribute access, specifying the resource name
Terraform fore_each for loop filter with if condition example
Terraform remote-exec provisioner with ssh connection in null_resource
Terraform count vs for_each for examples with map of objects
Terraform one module deploy null or multiple resources based on input (nested for loop) Example of Terraform functions flatten() and coalesce()
Terraform Azure Create Private Endpoint to existing Storage Account with Custom Private DNS zone record link
Creating a Private Endpoint for Azure Storage Account with required sub services using Terraform Example Terraform functions of lookup() and lower()
Using element function with count meta argument example Terraform Azure subnets Example Terraform functions of element(), count() and sum()
Terraform create Azure Virtual Network subnets from map of object and show name in the header Header name change in the output
Creating a Private Endpoint for Azure Storage Account with Terraform example 2 Example of for_each with toset() function
Creating a Private Endpoint for Azure Storage Account with Terraform example 3