While working on earlier article Configure Azure Storage Account Blob as Terraform backend to store tfstate file, I wanted to test terraform backend configuration with Azure Service Principal (App registrations) account. To configure initial resources on Azure for backend, I have automated all the steps using PowerShell with Az module. With this PowerShell script it helps me to create new Storage Account with Blob container and new Service Principal in Azure Intra ID app registrations. Once both the resources are created successfully, Service Account will get access (Assigned permissions) to Storage Account to store and modify tf state file.
To start first I imported az module in the PowerShell and then Connected to Az account selecting correct Azure Subscription name no (It will ask you to authenticate first to Azure portal and you will need to provide valid username and password to connect it).
Once all the resources are created successfully, it will give you terraform backend block template with the information. Which you can copy in your existing terraform configuration file in the terraform backend section. Highlighted yellow part need to be moved to azurerm provider block for storage account backend authentication.
Download this script bundle Terraform_Azure_Backend_Configure_with_service_principal.zip here or it is also available github.com.
Highlighted code in the Green - you can run it once. In Yellow highlighted part provide the information of new Storage Account and Service Principal. Information such as Name, Blob Container, Sku and etc you need to provide in the variables. This is very basic setup, next In the Role definition there are 2 roles related to blob, which you can configure, I am selecting Storage Blob Data Owner.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 |
######### Run once to login to Azure ########## <# Import-Module Az Connect-AzAccount -Subscription Sponsership-by-Microsoft Select-AzContext #> ######### Update Below Values ########## $resourceGroupName = 'vcloud-lab.com' #existing resource group $storageAccountName = 'vcloudlabterraform' $storageAccountSku = 'Standard_LRS' $blobContainerName = 'tfstate' #$storageAccountUser = '[email protected]' $roleDefinition = 'Storage Blob Data Owner' $servicePrincipalName = 'tfapp' #Get-AzRoleDefinition | where-object {$_.Name -match 'blob'} | Select-Object Name, Id, Description <# Name Id Description ---- -- ----------- Storage Blob Data Contributor ba92xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx Allows for read, write and delete access to Azure Storage blob containers and data Storage Blob Data Owner b7e6xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx Allows for full access to Azure Storage blob containers and data, including assigning POSIX access control. Storage Blob Data Reader 2a2bxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx Allows for read access to Azure Storage blob containers and data Storage Blob Delegator db58xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx Allows for generation of a user delegation key which can be used to sign SAS tokens #> ######### Do not touch below code ########## try { $resourceGroup = Get-AzResourceGroup -Name $resourceGroupName -ErrorAction Stop "Found Existing Resource Group - $($resourceGroup.ResourceGroupName)" $storageAccount = New-AzStorageAccount -Name $storageAccountName -Location $resourceGroup.Location -ResourceGroupName $resourceGroup.ResourceGroupName -SkuName $storageAccountSku -ErrorAction Stop "Storage Account created - $($storageAccount.StorageAccountName)" $storageAccountContext = New-AzStorageContext -StorageAccountName $storageAccount.StorageAccountName -ErrorAction Stop $blobContainer = New-AzStorageContainer -Name $blobContainerName -Context $storageAccountContext -ErrorAction Stop "Blob Container created - $($blobContainer.Name)" $servicePrincipal = New-AzADServicePrincipal -DisplayName $servicePrincipalName "Service Principal created - $($servicePrincipal.DisplayName)" #New-AzADServicePrincipalCredential #Create new SP credential secret #Assign role assignment to ad user #$roleAssignment = New-AzRoleAssignment -SignInName $user.DisplayName -RoleDefinitionName $roleDefinition -Scope $storageAccount.Id -ErrorAction Stop #New-AzADServicePrincipalAppRoleAssignment $roleAssignment = New-AzRoleAssignment -ApplicationId $servicePrincipal.AppId -Scope $storageAccount.Id -RoleDefinitionName $roleDefinition "Role Assigned to Service Principal over Storage Account - $($roleAssignment.RoleAssignmentName)" $azContext = Get-AzContext #"{0,-20} = {1,-42} {2}" -f 'resource_group_name', $($resourceGroup.ResourceGroupName), '# Can be passed via `-backend-config=`"resource_group_name=<resource group name>"` in the `init` command.' $tfScript = "{0,-20} = `"{1,-42} {2}`n" -f 'resource_group_name', "$($resourceGroup.ResourceGroupName)`"", '# Can be passed via `-backend-config=`"resource_group_name=<resource group name>"` in the `init` command.' $tfScript += " {0,-20} = `"{1,-42} {2}`n" -f 'storage_account_name', "$($storageAccount.StorageAccountName)`"", '# Can be passed via `-backend-config=`"storage_account_name=<storage account name>"` in the `init` command.' $tfScript += " {0,-20} = `"{1,-42} {2}`n" -f 'container_name', "$($blobContainer.Name)`"", '# Can be passed via `-backend-config=`"container_name=<container name>"` in the `init` command.' $tfScript += " {0,-20} = `"{1,-42} {2}`n" -f 'key', 'example.terraform.tfstate"', '# Can be passed via `-backend-config=`"key=<blob key name>"` in the `init` command.' $tfScript += " {0,-20} = `"{1,-42} {2}`n" -f 'client_id', "$($servicePrincipal.Id)`"", '# Can also be set via `ARM_CLIENT_ID` environment variable.' $tfScript += " {0,-20} = `"{1,-42} {2}`n" -f 'client_secret', "$($servicePrincipal.PasswordCredentials.SecretText)`"", '# Can also be set via `ARM_CLIENT_SECRET` environment variable.' $tfScript += " {0,-20} = `"{1,-42} {2}`n" -f 'subscription_id', "$($azContext.Subscription)`"", '# Can also be set via `ARM_SUBSCRIPTION_ID` environment variable.' $tfScript += " {0,-20} = `"{1,-42} {2}" -f 'tenant_id', "$($azContext.Tenant)`"", '# Can also be set via `ARM_TENANT_ID` environment variable.' $terraformBackEndConf = @" `n `n `e[41m##############################################################################################################`e[0m `e[41m## Use Below configuration in your Terraform file to configure Azure Storage Account Backend. ##`e[0m `e[41m##############################################################################################################`e[0m `e[44m################################################## Start #################################################`e[0m terraform { backend "azurerm" { $tfScript } } `e[44m################################################## End ##################################################`e[0m `n `n "@ $terraformBackEndConf } catch { Write-Error $Error[0].Exception.Message } <# #Delete reosource for testing Remove-AzStorageAccount -Name $storageAccountName -ResourceGroupName $resourceGroupName -Force Remove-AzADServicePrincipal -ApplicationId $servicePrincipal.AppId # -ServicePrincipalName $ServicePrincipal.ServicePrincipalName #> |
Once all the resources are created, verify them in the Azure portal, whether they are created successfully.
Also verify on the Storage Account Access Control (IAM) page that Role assignment is done with correct privileges to Service Principal.
Once PowerShell script execution is successful, Provide the generated information on the terraform tf configuration for Azure backend setting, As mentioned below.
Download this script bundle Terraform_Azure_Backend_Configure_with_service_principal.zip here or it is also available github.com.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 |
variable "backend_client_id" { default = "27b9xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" } variable "backend_client_secret" { default = "XPM8xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" } variable "backend_subscription_id" { default = "9e22xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" } variable "backend_tenant_id" { default = "3b80xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" } variable "login_client_id" { default = "1395xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" } variable "login_client_secret" { default = "rA.8xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" } variable "login_subscription_id" { default = "9e22xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" } variable "login_tenant_id" { default = "3b80xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" } terraform { backend "azurerm" { resource_group_name = "vcloud-lab.com" # Can be passed via `-backend-config=`"resource_group_name=<resource group name>"` in the `init` command. storage_account_name = "vcloudlabterraform" # Can be passed via `-backend-config=`"storage_account_name=<storage account name>"` in the `init` command. container_name = "tfstate" # Can be passed via `-backend-config=`"container_name=<container name>"` in the `init` command. key = "example.terraform.tfstate" # Can be passed via `-backend-config=`"key=<blob key name>"` in the `init` command. } } provider "azurerm" { features {} alias = "backend" client_id = var.backend_client_id # Can also be set via `ARM_CLIENT_ID` environment variable. client_secret = var.backend_client_secret # Can also be set via `ARM_CLIENT_SECRET` environment variable. subscription_id = var.backend_subscription_id # Can also be set via `ARM_SUBSCRIPTION_ID` environment variable. tenant_id = var.backend_tenant_id # Can also be set via `ARM_TENANT_ID` environment variable. } #For normal access to deploy the resources without backend provider "azurerm" { features {} client_id = var.login_client_id client_secret = var.login_client_secret subscription_id = var.login_subscription_id tenant_id = var.login_tenant_id } data "azurerm_resource_group" "dev" { name = "test-rg" } resource "azurerm_storage_account" "name" { name = "vcloudlabtestsa01" location = data.azurerm_resource_group.dev.location resource_group_name = data.azurerm_resource_group.dev.name account_tier = "Standard" account_replication_type = "LRS" } |
Here is the output after running command terraform init. it successfully connected and configured backend azurerm on the given Storage Account.
Below is the complete output of deployment of the resources after running terraform apply command..
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 |
terraform init Initializing the backend... Successfully configured the backend "azurerm"! Terraform will automatically use this backend unless the backend configuration changes. Initializing provider plugins... - Finding latest version of hashicorp/azurerm... - Installing hashicorp/azurerm v3.114.0... - Installed hashicorp/azurerm v3.114.0 (signed by HashiCorp) Terraform has created a lock file .terraform.lock.hcl to record the provider selections it made above. Include this file in your version control repository so that Terraform can guarantee to make the same selections by default when you run "terraform init" in the future. Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work. If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary. terraform apply --auto-approve Acquiring state lock. This may take a few moments... data.azurerm_resource_group.dev: Reading... data.azurerm_resource_group.dev: Read complete after 1s [id=/subscriptions/9e22xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/test-rg] Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # azurerm_storage_account.name will be created + resource "azurerm_storage_account" "name" { + access_tier = (known after apply) + account_kind = "StorageV2" + account_replication_type = "LRS" + account_tier = "Standard" + allow_nested_items_to_be_public = true + cross_tenant_replication_enabled = true + default_to_oauth_authentication = false + dns_endpoint_type = "Standard" + enable_https_traffic_only = (known after apply) + https_traffic_only_enabled = (known after apply) + id = (known after apply) + infrastructure_encryption_enabled = false + is_hns_enabled = false + large_file_share_enabled = (known after apply) + local_user_enabled = true + location = "eastus" + min_tls_version = "TLS1_2" + name = "vcloudlabtestsa01" + nfsv3_enabled = false + primary_access_key = (sensitive value) + primary_blob_connection_string = (sensitive value) + primary_blob_endpoint = (known after apply) + primary_blob_host = (known after apply) + primary_blob_internet_endpoint = (known after apply) + primary_blob_internet_host = (known after apply) + primary_blob_microsoft_endpoint = (known after apply) + primary_blob_microsoft_host = (known after apply) + primary_connection_string = (sensitive value) + primary_dfs_endpoint = (known after apply) + primary_dfs_host = (known after apply) + primary_dfs_internet_endpoint = (known after apply) + primary_dfs_internet_host = (known after apply) + primary_dfs_microsoft_endpoint = (known after apply) + primary_dfs_microsoft_host = (known after apply) + primary_file_endpoint = (known after apply) + primary_file_host = (known after apply) + primary_file_internet_endpoint = (known after apply) + primary_file_internet_host = (known after apply) + primary_file_microsoft_endpoint = (known after apply) + primary_file_microsoft_host = (known after apply) + primary_location = (known after apply) + primary_queue_endpoint = (known after apply) + primary_queue_host = (known after apply) + primary_queue_microsoft_endpoint = (known after apply) + primary_queue_microsoft_host = (known after apply) + primary_table_endpoint = (known after apply) + primary_table_host = (known after apply) + primary_table_microsoft_endpoint = (known after apply) + primary_table_microsoft_host = (known after apply) + primary_web_endpoint = (known after apply) + primary_web_host = (known after apply) + primary_web_internet_endpoint = (known after apply) + primary_web_internet_host = (known after apply) + primary_web_microsoft_endpoint = (known after apply) + primary_web_microsoft_host = (known after apply) + public_network_access_enabled = true + queue_encryption_key_type = "Service" + resource_group_name = "test-rg" + secondary_access_key = (sensitive value) + secondary_blob_connection_string = (sensitive value) + secondary_blob_endpoint = (known after apply) + secondary_blob_host = (known after apply) + secondary_blob_internet_endpoint = (known after apply) + secondary_blob_internet_host = (known after apply) + secondary_blob_microsoft_endpoint = (known after apply) + secondary_blob_microsoft_host = (known after apply) + secondary_connection_string = (sensitive value) + secondary_dfs_endpoint = (known after apply) + secondary_dfs_host = (known after apply) + secondary_dfs_internet_endpoint = (known after apply) + secondary_dfs_internet_host = (known after apply) + secondary_dfs_microsoft_endpoint = (known after apply) + secondary_dfs_microsoft_host = (known after apply) + secondary_file_endpoint = (known after apply) + secondary_file_host = (known after apply) + secondary_file_internet_endpoint = (known after apply) + secondary_file_internet_host = (known after apply) + secondary_file_microsoft_endpoint = (known after apply) + secondary_file_microsoft_host = (known after apply) + secondary_location = (known after apply) + secondary_queue_endpoint = (known after apply) + secondary_queue_host = (known after apply) + secondary_queue_microsoft_endpoint = (known after apply) + secondary_queue_microsoft_host = (known after apply) + secondary_table_endpoint = (known after apply) + secondary_table_host = (known after apply) + secondary_table_microsoft_endpoint = (known after apply) + secondary_table_microsoft_host = (known after apply) + secondary_web_endpoint = (known after apply) + secondary_web_host = (known after apply) + secondary_web_internet_endpoint = (known after apply) + secondary_web_internet_host = (known after apply) + secondary_web_microsoft_endpoint = (known after apply) + secondary_web_microsoft_host = (known after apply) + sftp_enabled = false + shared_access_key_enabled = true + table_encryption_key_type = "Service" + blob_properties (known after apply) + network_rules (known after apply) + queue_properties (known after apply) + routing (known after apply) + share_properties (known after apply) } Plan: 1 to add, 0 to change, 0 to destroy. azurerm_storage_account.name: Creating... azurerm_storage_account.name: Still creating... [10s elapsed] azurerm_storage_account.name: Still creating... [20s elapsed] azurerm_storage_account.name: Still creating... [30s elapsed] azurerm_storage_account.name: Still creating... [40s elapsed] azurerm_storage_account.name: Still creating... [50s elapsed] azurerm_storage_account.name: Still creating... [1m0s elapsed] azurerm_storage_account.name: Still creating... [1m10s elapsed] azurerm_storage_account.name: Creation complete after 1m12s [id=/subscriptions/9e22xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/test-rg/providers/Microsoft.Storage/storageAccounts/vcloudlabtestsa01] Releasing state lock. This may take a few moments... Apply complete! Resources: 1 added, 0 changed, 0 destroyed. |
Once Terraform configuration apply is successful, verify new tfstate file is generated and it has all the information about new deployed resources.
For more information check official documentation https://developer.hashicorp.com/terraform/language/settings/backends/azurerm.
Addition if you wish to keep backend information separate file use below parameters.
-backend=false | Disable backend or HCP Terraform initialization for this configuration and use what was previously initialized instead. aliases: -cloud=false |
-backend-config=path | Configuration to be merged with what is in the configuration file's 'backend' block. This can be either a path to an HCL file with key/value assignments (same format as terraform.tfvars) or a 'key=value' format, and can be specified multiple times. The backend type must be in the configuration itself. |
Useful Articles
Terraform variable validation example
Terraform create Azure Virtual Machines from map of objects
Terraform refactoring moved block example
Terraform create Azure Virtual Machines from map of objects
Hashicorp Terraform map and object inside module and variable example
Terraform one module deploy null or multiple resources based on input
Terraform A reference to a resource type must be followed by at least one attribute access, specifying the resource name
Terraform fore_each for loop filter with if condition example
Terraform remote-exec provisioner with ssh connection in null_resource
Terraform count vs for_each for examples with map of objects
Terraform one module deploy null or multiple resources based on input (nested for loop) Example of Terraform functions flatten() and coalesce()
Terraform Azure Create Private Endpoint to existing Storage Account with Custom Private DNS zone record link
Creating a Private Endpoint for Azure Storage Account with required sub services using Terraform Example Terraform functions of lookup() and lower()
Using element function with count meta argument example Terraform Azure subnets Example Terraform functions of element(), count() and sum()
Terraform create Azure Virtual Network subnets from map of object and show name in the header Header name change in the output
Creating a Private Endpoint for Azure Storage Account with Terraform example 2 Example of for_each with toset() function
Creating a Private Endpoint for Azure Storage Account with Terraform example 3