Menu

Virtual Geek

Tales from real IT system administrators world and non-production environment

Create storage account and Service Principal using PowerShell for Terraform Azure Backend

While working on earlier article Configure Azure Storage Account Blob as Terraform backend to store tfstate file, I wanted to test terraform backend configuration with Azure Service Principal (App registrations) account. To configure initial resources on Azure for backend, I have automated all the steps using PowerShell with Az module. With this PowerShell script it helps me to create new Storage Account with Blob container and new Service Principal in Azure Intra ID app registrations. Once both the resources are created successfully, Service Account will get access (Assigned permissions) to Storage Account to store and modify tf state file.

To start first I imported az module in the PowerShell and then Connected to Az account selecting correct Azure Subscription name no (It will ask you to authenticate first to Azure portal and you will need to provide valid username and password to connect it).

Microsoft Azure sponsership subscription free mccp id tenant terraform storage account service principal app registrations configuration terraform backend automated update-azconfig Import-Module connect-azaccount login.png

Once all the resources are created successfully, it will give you terraform backend block template with the information. Which you can copy in your existing terraform configuration file in the terraform backend section. Highlighted yellow part need to be moved to azurerm provider block for storage account backend authentication.

Microsoft Azure Powershell connect-azaccount module az azurerm terraform backend resource group storage account container name blob key client_id client_secret subscription_id tenant_id backend config arm configuration terraform.png

Download this script bundle Terraform_Azure_Backend_Configure_with_service_principal.zip here or it is also available github.com.

Highlighted code in the Green - you can run it once. In Yellow highlighted part provide the information of new Storage Account and Service Principal. Information such as Name, Blob Container, Sku and etc you need to provide in the variables. This is very basic setup, next In the Role definition there are 2 roles related to blob, which you can configure, I am selecting Storage Blob Data Owner.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
######### Run once to login to Azure ##########
<#
Import-Module Az
Connect-AzAccount -Subscription Sponsership-by-Microsoft
Select-AzContext
#>

######### Update Below Values ##########

$resourceGroupName = 'vcloud-lab.com' #existing resource group
$storageAccountName = 'vcloudlabterraform'
$storageAccountSku = 'Standard_LRS'
$blobContainerName = 'tfstate'

#$storageAccountUser = '[email protected]'
$roleDefinition = 'Storage Blob Data Owner'
$servicePrincipalName = 'tfapp'

#Get-AzRoleDefinition | where-object {$_.Name -match 'blob'} | Select-Object Name, Id, Description
<#
Name                          Id                                   Description
----                          --                                   -----------
Storage Blob Data Contributor ba92xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx Allows for read, write and delete access to Azure Storage blob containers and data
Storage Blob Data Owner       b7e6xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx Allows for full access to Azure Storage blob containers and data, including assigning POSIX access control.
Storage Blob Data Reader      2a2bxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx Allows for read access to Azure Storage blob containers and data
Storage Blob Delegator        db58xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx Allows for generation of a user delegation key which can be used to sign SAS tokens
#>

######### Do not touch below code ##########

try 
{
    $resourceGroup = Get-AzResourceGroup -Name $resourceGroupName -ErrorAction Stop
    "Found Existing Resource Group - $($resourceGroup.ResourceGroupName)"
    $storageAccount = New-AzStorageAccount -Name $storageAccountName -Location $resourceGroup.Location -ResourceGroupName $resourceGroup.ResourceGroupName -SkuName $storageAccountSku -ErrorAction Stop
    "Storage Account created - $($storageAccount.StorageAccountName)"
    $storageAccountContext = New-AzStorageContext -StorageAccountName $storageAccount.StorageAccountName -ErrorAction Stop
    $blobContainer = New-AzStorageContainer -Name $blobContainerName -Context $storageAccountContext -ErrorAction Stop
    "Blob Container created - $($blobContainer.Name)"


    $servicePrincipal = New-AzADServicePrincipal -DisplayName $servicePrincipalName
    "Service Principal created - $($servicePrincipal.DisplayName)"
    #New-AzADServicePrincipalCredential #Create new SP credential secret
    
    #Assign role assignment to ad user
    #$roleAssignment = New-AzRoleAssignment -SignInName $user.DisplayName -RoleDefinitionName $roleDefinition -Scope $storageAccount.Id  -ErrorAction Stop
    #New-AzADServicePrincipalAppRoleAssignment   
    $roleAssignment = New-AzRoleAssignment -ApplicationId $servicePrincipal.AppId -Scope $storageAccount.Id -RoleDefinitionName $roleDefinition
    "Role Assigned to Service Principal over Storage Account - $($roleAssignment.RoleAssignmentName)"

    $azContext = Get-AzContext

    #"{0,-20} = {1,-42} {2}" -f 'resource_group_name', $($resourceGroup.ResourceGroupName), '# Can be passed via `-backend-config=`"resource_group_name=<resource group name>"` in the `init` command.'
    $tfScript = "{0,-20} = `"{1,-42} {2}`n" -f 'resource_group_name', "$($resourceGroup.ResourceGroupName)`"", '# Can be passed via `-backend-config=`"resource_group_name=<resource group name>"` in the `init` command.'
    $tfScript += "    {0,-20} = `"{1,-42} {2}`n" -f 'storage_account_name', "$($storageAccount.StorageAccountName)`"", '# Can be passed via `-backend-config=`"storage_account_name=<storage account name>"` in the `init` command.'
    $tfScript += "    {0,-20} = `"{1,-42} {2}`n" -f 'container_name', "$($blobContainer.Name)`"",  '# Can be passed via `-backend-config=`"container_name=<container name>"` in the `init` command.'
    $tfScript += "    {0,-20} = `"{1,-42} {2}`n" -f 'key', 'example.terraform.tfstate"', '# Can be passed via `-backend-config=`"key=<blob key name>"` in the `init` command.'
    $tfScript += "    {0,-20} = `"{1,-42} {2}`n" -f 'client_id', "$($servicePrincipal.Id)`"", '# Can also be set via `ARM_CLIENT_ID` environment variable.'
    $tfScript += "    {0,-20} = `"{1,-42} {2}`n" -f 'client_secret', "$($servicePrincipal.PasswordCredentials.SecretText)`"", '# Can also be set via `ARM_CLIENT_SECRET` environment variable.'
    $tfScript += "    {0,-20} = `"{1,-42} {2}`n" -f 'subscription_id', "$($azContext.Subscription)`"", '# Can also be set via `ARM_SUBSCRIPTION_ID` environment variable.'
    $tfScript += "    {0,-20} = `"{1,-42} {2}" -f 'tenant_id', "$($azContext.Tenant)`"", '# Can also be set via `ARM_TENANT_ID` environment variable.'

$terraformBackEndConf = @"
`n
`n
`e[41m##############################################################################################################`e[0m
`e[41m##       Use Below configuration in your Terraform file to configure Azure Storage Account Backend.         ##`e[0m
`e[41m##############################################################################################################`e[0m
`e[44m##################################################   Start   #################################################`e[0m

terraform {
  backend "azurerm" {
    $tfScript
  }
}

`e[44m##################################################   End   ##################################################`e[0m
`n
`n
"@
$terraformBackEndConf
}
catch 
{
    Write-Error $Error[0].Exception.Message
}

<#
#Delete reosource for testing
Remove-AzStorageAccount -Name $storageAccountName -ResourceGroupName $resourceGroupName -Force
Remove-AzADServicePrincipal -ApplicationId $servicePrincipal.AppId # -ServicePrincipalName $ServicePrincipal.ServicePrincipalName
#>

Once all the resources are created, verify them in the Azure portal, whether they are created successfully.

Microsoft Azure storage account storage browser blob containers file shares queues tables storage mover intra id azure active directory ad application client id object id directory tenant id app registrations service principal.png

Also verify on the Storage Account Access Control (IAM) page that Role assignment is done with correct privileges to Service Principal.

Microsoft azure terraform storage account backend tf configuration state file tfstate access control iam service principal client id secret configuration tags storage browser hcl lock terraform.png

Once PowerShell script execution is successful, Provide the generated information on the terraform tf configuration for Azure backend setting, As mentioned below.

Download this script bundle Terraform_Azure_Backend_Configure_with_service_principal.zip here or it is also available github.com.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
variable "backend_client_id" { default = "27b9xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" }
variable "backend_client_secret" { default = "XPM8xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" }
variable "backend_subscription_id" { default = "9e22xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" }
variable "backend_tenant_id" { default = "3b80xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" }

variable "login_client_id" { default = "1395xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" }
variable "login_client_secret" { default = "rA.8xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" }
variable "login_subscription_id" { default = "9e22xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" }
variable "login_tenant_id" { default = "3b80xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" }


terraform {
  backend "azurerm" {
    resource_group_name  = "vcloud-lab.com"            # Can be passed via `-backend-config=`"resource_group_name=<resource group name>"` in the `init` command.
    storage_account_name = "vcloudlabterraform"        # Can be passed via `-backend-config=`"storage_account_name=<storage account name>"` in the `init` command.
    container_name       = "tfstate"                   # Can be passed via `-backend-config=`"container_name=<container name>"` in the `init` command.
    key                  = "example.terraform.tfstate" # Can be passed via `-backend-config=`"key=<blob key name>"` in the `init` command.
  }
}

provider "azurerm" {
  features {}
  alias           = "backend"
  client_id       = var.backend_client_id       # Can also be set via `ARM_CLIENT_ID` environment variable.
  client_secret   = var.backend_client_secret   # Can also be set via `ARM_CLIENT_SECRET` environment variable.
  subscription_id = var.backend_subscription_id # Can also be set via `ARM_SUBSCRIPTION_ID` environment variable.
  tenant_id       = var.backend_tenant_id       # Can also be set via `ARM_TENANT_ID` environment variable.
}

#For normal access to deploy the resources without backend
provider "azurerm" {
  features {}
  client_id       = var.login_client_id
  client_secret   = var.login_client_secret
  subscription_id = var.login_subscription_id
  tenant_id       = var.login_tenant_id
}

data "azurerm_resource_group" "dev" {
  name = "test-rg"
}

resource "azurerm_storage_account" "name" {
  name                     = "vcloudlabtestsa01"
  location                 = data.azurerm_resource_group.dev.location
  resource_group_name      = data.azurerm_resource_group.dev.name
  account_tier             = "Standard"
  account_replication_type = "LRS"
}

Here is the output after running command terraform init. it successfully connected and configured backend azurerm on the given Storage Account.

Microsoft Azure backend terraform hashicorp azurerm lock.hcl successfully initialized terraform init plan apply --auto-approve cloud terraform configuration tf file tfstate locked.png

Below is the complete output of deployment of the resources after running terraform apply command..

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
terraform init
Initializing the backend...

Successfully configured the backend "azurerm"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...
- Finding latest version of hashicorp/azurerm...
- Installing hashicorp/azurerm v3.114.0...
- Installed hashicorp/azurerm v3.114.0 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.


terraform apply --auto-approve
Acquiring state lock. This may take a few moments...
data.azurerm_resource_group.dev: Reading...
data.azurerm_resource_group.dev: Read complete after 1s [id=/subscriptions/9e22xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/test-rg]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # azurerm_storage_account.name will be created
  + resource "azurerm_storage_account" "name" {
      + access_tier                        = (known after apply)
      + account_kind                       = "StorageV2"
      + account_replication_type           = "LRS"
      + account_tier                       = "Standard"
      + allow_nested_items_to_be_public    = true
      + cross_tenant_replication_enabled   = true
      + default_to_oauth_authentication    = false
      + dns_endpoint_type                  = "Standard"
      + enable_https_traffic_only          = (known after apply)
      + https_traffic_only_enabled         = (known after apply)
      + id                                 = (known after apply)
      + infrastructure_encryption_enabled  = false
      + is_hns_enabled                     = false
      + large_file_share_enabled           = (known after apply)
      + local_user_enabled                 = true
      + location                           = "eastus"
      + min_tls_version                    = "TLS1_2"
      + name                               = "vcloudlabtestsa01"
      + nfsv3_enabled                      = false
      + primary_access_key                 = (sensitive value)
      + primary_blob_connection_string     = (sensitive value)
      + primary_blob_endpoint              = (known after apply)
      + primary_blob_host                  = (known after apply)
      + primary_blob_internet_endpoint     = (known after apply)
      + primary_blob_internet_host         = (known after apply)
      + primary_blob_microsoft_endpoint    = (known after apply)
      + primary_blob_microsoft_host        = (known after apply)
      + primary_connection_string          = (sensitive value)
      + primary_dfs_endpoint               = (known after apply)
      + primary_dfs_host                   = (known after apply)
      + primary_dfs_internet_endpoint      = (known after apply)
      + primary_dfs_internet_host          = (known after apply)
      + primary_dfs_microsoft_endpoint     = (known after apply)
      + primary_dfs_microsoft_host         = (known after apply)
      + primary_file_endpoint              = (known after apply)
      + primary_file_host                  = (known after apply)
      + primary_file_internet_endpoint     = (known after apply)
      + primary_file_internet_host         = (known after apply)
      + primary_file_microsoft_endpoint    = (known after apply)
      + primary_file_microsoft_host        = (known after apply)
      + primary_location                   = (known after apply)
      + primary_queue_endpoint             = (known after apply)
      + primary_queue_host                 = (known after apply)
      + primary_queue_microsoft_endpoint   = (known after apply)
      + primary_queue_microsoft_host       = (known after apply)
      + primary_table_endpoint             = (known after apply)
      + primary_table_host                 = (known after apply)
      + primary_table_microsoft_endpoint   = (known after apply)
      + primary_table_microsoft_host       = (known after apply)
      + primary_web_endpoint               = (known after apply)
      + primary_web_host                   = (known after apply)
      + primary_web_internet_endpoint      = (known after apply)
      + primary_web_internet_host          = (known after apply)
      + primary_web_microsoft_endpoint     = (known after apply)
      + primary_web_microsoft_host         = (known after apply)
      + public_network_access_enabled      = true
      + queue_encryption_key_type          = "Service"
      + resource_group_name                = "test-rg"
      + secondary_access_key               = (sensitive value)
      + secondary_blob_connection_string   = (sensitive value)
      + secondary_blob_endpoint            = (known after apply)
      + secondary_blob_host                = (known after apply)
      + secondary_blob_internet_endpoint   = (known after apply)
      + secondary_blob_internet_host       = (known after apply)
      + secondary_blob_microsoft_endpoint  = (known after apply)
      + secondary_blob_microsoft_host      = (known after apply)
      + secondary_connection_string        = (sensitive value)
      + secondary_dfs_endpoint             = (known after apply)
      + secondary_dfs_host                 = (known after apply)
      + secondary_dfs_internet_endpoint    = (known after apply)
      + secondary_dfs_internet_host        = (known after apply)
      + secondary_dfs_microsoft_endpoint   = (known after apply)
      + secondary_dfs_microsoft_host       = (known after apply)
      + secondary_file_endpoint            = (known after apply)
      + secondary_file_host                = (known after apply)
      + secondary_file_internet_endpoint   = (known after apply)
      + secondary_file_internet_host       = (known after apply)
      + secondary_file_microsoft_endpoint  = (known after apply)
      + secondary_file_microsoft_host      = (known after apply)
      + secondary_location                 = (known after apply)
      + secondary_queue_endpoint           = (known after apply)
      + secondary_queue_host               = (known after apply)
      + secondary_queue_microsoft_endpoint = (known after apply)
      + secondary_queue_microsoft_host     = (known after apply)
      + secondary_table_endpoint           = (known after apply)
      + secondary_table_host               = (known after apply)
      + secondary_table_microsoft_endpoint = (known after apply)
      + secondary_table_microsoft_host     = (known after apply)
      + secondary_web_endpoint             = (known after apply)
      + secondary_web_host                 = (known after apply)
      + secondary_web_internet_endpoint    = (known after apply)
      + secondary_web_internet_host        = (known after apply)
      + secondary_web_microsoft_endpoint   = (known after apply)
      + secondary_web_microsoft_host       = (known after apply)
      + sftp_enabled                       = false
      + shared_access_key_enabled          = true
      + table_encryption_key_type          = "Service"

      + blob_properties (known after apply)

      + network_rules (known after apply)

      + queue_properties (known after apply)

      + routing (known after apply)

      + share_properties (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.
azurerm_storage_account.name: Creating...
azurerm_storage_account.name: Still creating... [10s elapsed]
azurerm_storage_account.name: Still creating... [20s elapsed]
azurerm_storage_account.name: Still creating... [30s elapsed]
azurerm_storage_account.name: Still creating... [40s elapsed]
azurerm_storage_account.name: Still creating... [50s elapsed]
azurerm_storage_account.name: Still creating... [1m0s elapsed]
azurerm_storage_account.name: Still creating... [1m10s elapsed]
azurerm_storage_account.name: Creation complete after 1m12s [id=/subscriptions/9e22xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/test-rg/providers/Microsoft.Storage/storageAccounts/vcloudlabtestsa01]
Releasing state lock. This may take a few moments...

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Once Terraform configuration apply is successful, verify new tfstate file is generated and it has all the information about new deployed resources.

Microsoft azure powershell terraform tfstate container iam access control state file hcl lock lineage subscription free tier azure portal account managed by instance backend terraform client id service principal.png

For more information check official documentation https://developer.hashicorp.com/terraform/language/settings/backends/azurerm.

Addition if you wish to keep backend information separate file use below parameters.

  -backend=falseDisable backend or HCP Terraform initialization for this configuration and use what was previously initialized instead.
aliases: -cloud=false
  -backend-config=pathConfiguration to be merged with what is in the configuration file's 'backend' block. This can be either a path to an HCL file with key/value assignments (same format as terraform.tfvars) or a 'key=value' format, and can be specified multiple times. The backend type must be in the configuration itself.

Useful Articles
Terraform variable validation example
Terraform create Azure Virtual Machines from map of objects
Terraform refactoring moved block example
Terraform create Azure Virtual Machines from map of objects
Hashicorp Terraform map and object inside module and variable example
Terraform one module deploy null or multiple resources based on input
Terraform A reference to a resource type must be followed by at least one attribute access, specifying the resource name
Terraform fore_each for loop filter with if condition example
Terraform remote-exec provisioner with ssh connection in null_resource
Terraform count vs for_each for examples with map of objects
Terraform one module deploy null or multiple resources based on input (nested for loop) Example of Terraform functions flatten() and coalesce()
Terraform Azure Create Private Endpoint to existing Storage Account with Custom Private DNS zone record link
Creating a Private Endpoint for Azure Storage Account with required sub services using Terraform Example Terraform functions of lookup() and lower()
Using element function with count meta argument example Terraform Azure subnets Example Terraform functions of element()count() and sum()
Terraform create Azure Virtual Network subnets from map of object and show name in the header Header name change in the output
Creating a Private Endpoint for Azure Storage Account with Terraform example 2 Example of for_each with toset() function
Creating a Private Endpoint for Azure Storage Account with Terraform example 3

Go Back

Comment

Blog Search

Page Views

12086028

Follow me on Blogarama