Menu

Virtual Geek

Tales from real IT system administrators world and non-production environment

Deploying Azure ARM templates with Terraform and terraform templatefile example

This article is a revised article of Create CPU quota usage alerts for subscription using Azure ARM templates. Initially, I encountered a challenge while attempting to incorporate ARM variables or parameters into Kusto Query Language (KQL) within an Azure Resource Manager (ARM) template. Despite the rest of the template functioning as expected, the hard-coded KQL value used for defining alerts posed a significant hurdle, as I couldn't find a straightforward method to automate or utilize parameters within the KQL itself.

Solution
To resolve this issue, I leveraged Terraform templates in conjunction with KQL to parameterize values effectively. By utilizing the templatefile feature, I was able to successfully work around the limitations and achieve the desired outcome.

Key Takeaways

  • Azure ARM templates have limitations when it comes to parameterizing KQL queries.
  • Terraform templates can be used to overcome these limitations and enable parameterization.
  • The templatefile function is a valuable tool in achieving this workaround.
  • This approach allows for greater flexibility and automation in managing KQL queries within Azure ARM templates, making it easier to manage and maintain complex alerting systems.

Below is the complete output how the configuration is successful after deployment.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
 
.\terraform_azure_deployment_ARM_template> terraform apply --auto-approve
data.azurerm_subscription.subscriptioninfo: Reading...
data.azurerm_resource_group.rginfo: Reading...
data.azurerm_subscription.subscriptioninfo: Read complete after 1s [id=/subscriptions/9exxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx]
data.azurerm_resource_group.rginfo: Read complete after 1s [id=/subscriptions/9exxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/vcloud-lab.com]
data.azurerm_user_assigned_identity.uaiinfo: Reading...
data.azurerm_user_assigned_identity.uaiinfo: Read complete after 1s [id=/subscriptions/9exxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/vcloud-lab.com/providers/Microsoft.ManagedIdentity/userAssignedIdentities/dev]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # azurerm_resource_group_template_deployment.resources will be created
  + resource "azurerm_resource_group_template_deployment" "resources" {
      + deployment_mode     = "Incremental"
      + id                  = (known after apply)
      + name                = "vcloud-lab.com"
      + output_content      = (known after apply)
      + parameters_content  = jsonencode(
            {
              + Action_Group_Name      = {
                  + value = "Action_Group01"
                }
              + Action_Group_ShortName = {
                  + value = "ag01"
                }
              + Dimensions_Location    = {
                  + value = "eastus"
                }
              + Dimensions_Quota_Name  = {
                  + value = "cores"
                }
              + Dimensions_Type        = {
                  + value = "microsoft.compute/locations/usages"
                }
              + Email_Address          = {
                  + value = "[email protected]"
                }
              + Quota_Alert_Name       = {
                  + value = "Quota_Alert01"
                }
              + Scopes                 = {
                  + value = "/subscriptions/9exxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
                }
              + User_Managed_Identity  = {
                  + value = "/subscriptions/9exxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/vcloud-lab.com/providers/Microsoft.ManagedIdentity/userAssignedIdentities/dev"
                }
            }
        )
      + resource_group_name = "vcloud-lab.com"
      + template_content    = jsonencode(
            {
              + "$schema"      = "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#"
              + contentVersion = "1.0.0.0"
              + outputs        = {
                  + exampleOutput = {
                      + type  = "string"
                      + value = "[resourceId('microsoft.insights/scheduledqueryrules', parameters('Quota_Alert_Name'))]"
                    }
                }
              + parameters     = {
                  + Action_Group_Name      = {
                      + defaultValue = "Action_Group01"
                      + metadata     = {
                          + description = "Name of the Action Group"
                        }
                      + type         = "string"
                    }
                  + Action_Group_ShortName = {
                      + defaultValue = "ag01"
                      + metadata     = {
                          + description = "Name of the Action Group Short Name"
                        }
                      + type         = "string"
                    }
                  + Dimensions_Location    = {
                      + defaultValue = "eastus"
                      + type         = "string"
                    }
                  + Dimensions_Quota_Name  = {
                      + defaultValue = "cores"
                      + type         = "string"
                    }
                  + Dimensions_Type        = {
                      + defaultValue = "microsoft.compute/locations/usages"
                      + type         = "string"
                    }
                  + Email_Address          = {
                      + defaultValue = "[email protected]"
                      + type         = "string"
                    }
                  + Quota_Alert_Name       = {
                      + defaultValue = "Quota_Alert01"
                      + type         = "string"
                    }
                  + Scopes                 = {
                      + type = "string"
                    }
                  + User_Managed_Identity  = {
                      + metadata = {
                          + description = "ID of user managed identity"
                        }
                      + type     = "string"
                    }
                }
              + resources      = [
                  + {
                      + apiVersion = "2023-09-01-preview"
                      + location   = "Global"
                      + name       = "[parameters('Action_Group_Name')]"
                      + properties = {
                          + armRoleReceivers           = [
                              + {
                                  + name                 = "EmailARMRole"
                                  + roleId               = "8exxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
                                  + useCommonAlertSchema = true
                                },
                            ]
                          + automationRunbookReceivers = []
                          + azureAppPushReceivers      = []
                          + azureFunctionReceivers     = []
                          + emailReceivers             = [
                              + {
                                  + emailAddress         = "[parameters('Email_Address')]"
                                  + name                 = "EmailAction"
                                  + useCommonAlertSchema = true
                                },
                            ]
                          + enabled                    = true
                          + eventHubReceivers          = []
                          + groupShortName             = "[parameters('Action_Group_ShortName')]"
                          + itsmReceivers              = []
                          + logicAppReceivers          = []
                          + smsReceivers               = []
                          + voiceReceivers             = []
                          + webhookReceivers           = []
                        }
                      + type       = "microsoft.insights/actionGroups"
                    },
                  + {
                      + apiVersion = "2023-03-15-preview"
                      + identity   = {
                          + type                   = "UserAssigned"
                          + userAssignedIdentities = {
                              + "[parameters('User_Managed_Identity')]" = {}
                            }
                        }
                      + location   = "[parameters('Dimensions_Location')]"
                      + name       = "[parameters('Quota_Alert_Name')]"
                      + properties = {
                          + actions             = {
                              + actionGroups = [
                                  + "[resourceId('microsoft.insights/actionGroups', parameters('Action_Group_Name'))]",
                                ]
                            }
                          + criteria            = {
                              + allOf = [
                                  + {
                                      + dimensions          = [
                                          + {
                                              + name     = "type"
                                              + operator = "Include"
                                              + values   = [
                                                  + "[parameters('Dimensions_Type')]",
                                                ]
                                            },
                                          + {
                                              + name     = "location"
                                              + operator = "Include"
                                              + values   = [
                                                  + "[parameters('Dimensions_Location')]",
                                                ]
                                            },
                                          + {
                                              + name     = "quotaName"
                                              + operator = "Include"
                                              + values   = [
                                                  + "[parameters('Dimensions_Quota_Name')]",
                                                ]
                                            },
                                        ]
                                      + failingPeriods      = {
                                          + minFailingPeriodsToAlert  = 1
                                          + numberOfEvaluationPeriods = 1
                                        }
                                      + metricMeasureColumn = "usagePercent"
                                      + operator            = "GreaterThanOrEqual"
                                      + query               = "arg(\"\").QuotaResources | where subscriptionId =~ '9exxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' | where type =~ 'microsoft.compute/locations/usages' | where isnotempty(properties) | mv-expand propertyJson = properties.value limit 400 | extend usage = propertyJson.currentValue, quota = propertyJson['limit'], quotaName = tostring(propertyJson['name'].value) | extend usagePercent = toint(usage)*100 / toint(quota) | project-away properties | where location in~ ('eastus') | where quotaName in~ ('cores')"
                                      + threshold           = 80
                                      + timeAggregation     = "Maximum"
                                    },
                                ]
                            }
                          + enabled             = true
                          + evaluationFrequency = "PT15M"
                          + scopes              = [
                              + "[parameters('Scopes')]",
                            ]
                          + severity            = 4
                          + windowSize          = "PT15M"
                        }
                      + type       = "microsoft.insights/scheduledqueryrules"
                    },
                ]
              + variables      = {}
            }
        )
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + arm_example_output = (known after apply)
  + name               = {
      + display_name          = "Sponsership-by-Microsoft"
      + id                    = "/subscriptions/9exxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
      + location_placement_id = "Public_2014-09-01"
      + quota_id              = "MPN_2014-09-01"
      + spending_limit        = "Off"
      + state                 = "Enabled"
      + subscription_id       = "9exxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
      + tags                  = {}
      + tenant_id             = "3bxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
      + timeouts              = null
    }
azurerm_resource_group_template_deployment.resources: Creating...
azurerm_resource_group_template_deployment.resources: Still creating... [10s elapsed]
azurerm_resource_group_template_deployment.resources: Still creating... [20s elapsed]
azurerm_resource_group_template_deployment.resources: Still creating... [30s elapsed]
azurerm_resource_group_template_deployment.resources: Still creating... [40s elapsed]
azurerm_resource_group_template_deployment.resources: Creation complete after 43s [id=/subscriptions/9exxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/vcloud-lab.com/providers/Microsoft.Resources/deployments/vcloud-lab.com]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Outputs:

arm_example_output = "/subscriptions/9exxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/vcloud-lab.com/providers/microsoft.insights/scheduledqueryrules/Quota_Alert01"
name = {
  "display_name" = "Sponsership-by-Microsoft"
  "id" = "/subscriptions/9exxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
  "location_placement_id" = "Public_2014-09-01"
  "quota_id" = "MPN_2014-09-01"
  "spending_limit" = "Off"
  "state" = "Enabled"
  "subscription_id" = "9exxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
  "tags" = tomap({})
  "tenant_id" = "3bxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
  "timeouts" = null /* object */
}
.\terraform_azure_deployment_ARM_template>

Download this complete code terraform_azure_deployment_ARM_template_tftpl_templatefile.zip here or it is also available on github.com.

In below configuration I have defined terraform locals variables which will be used in template_content tempate files.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
 
terraform {
  required_version = "1.9.2"
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "4.4"
    }
  }
}

##################################

provider "azurerm" {
  features {}
  subscription_id = "9exxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
}

##################################

variable "information" {
  type = object({
    action_group_name      = string
    action_group_shortname = string
    email_address          = string
    quota_alert_name       = string
    user_managed_identity  = string
    dimensions_type        = string
    dimensions_location    = string
    dimensions_quota_name  = string
  })
  default = {
    action_group_name      = "Action_Group01"
    action_group_shortname = "ag01"
    email_address          = "[email protected]"
    quota_alert_name       = "Quota_Alert01"
    user_managed_identity  = "dev"
    dimensions_type        = "microsoft.compute/locations/usages"
    dimensions_location    = "eastus"
    dimensions_quota_name  = "cores"
  }
}

##################################

data "azurerm_subscription" "subscriptioninfo" {}

data "azurerm_resource_group" "rginfo" {
  name = "vcloud-lab.com"
}

data "azurerm_user_assigned_identity" "uaiinfo" {
  name                = var.information.user_managed_identity
  resource_group_name = data.azurerm_resource_group.rginfo.name
}

##################################

locals {
  #case sensitive
  subscription_id     = data.azurerm_subscription.subscriptioninfo.subscription_id #"subscription_id" = "9exxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
  scopes              = data.azurerm_subscription.subscriptioninfo.id              #"id" = "/subscriptions/9exxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
  dimensions_location = data.azurerm_resource_group.rginfo.location                # location for dimensions and scheduledqueryresource
}

##################################

resource "azurerm_resource_group_template_deployment" "resources" {
  name                = "vcloud-lab.com"
  resource_group_name = "vcloud-lab.com"
  deployment_mode     = "Incremental"
  parameters_content = jsonencode({
    "Action_Group_Name" = {
      value = var.information.action_group_name
    },
    "Action_Group_ShortName" = {
      value = var.information.action_group_shortname
    },
    "Email_Address" = {
      value = var.information.email_address
    },
    "Quota_Alert_Name" = {
      value = var.information.quota_alert_name
    },
    "User_Managed_Identity" = {
      value = data.azurerm_user_assigned_identity.uaiinfo.id
    },
    "Scopes" = {
      value = local.scopes
    },
    "Dimensions_Type" = {
      value = var.information.dimensions_type
    },
    "Dimensions_Location" = {
      value = local.dimensions_location
    },
    "Dimensions_Quota_Name" = {
      value = var.information.dimensions_quota_name
    }
  })
  template_content = templatefile("${path.module}/arm_deployment.json.tftpl", {
    QUERY_SUBSCRIPTION_ID = local.subscription_id,
    QUERY_LOCATION        = local.dimensions_location,
    QUERY_QUOTA_NAME      = var.information.dimensions_quota_name
  })

  // NOTE: whilst we show an inline template here, we recommend
  // sourcing this from a file for readability/editor support
}

output "arm_example_output" {
  value = jsondecode(azurerm_resource_group_template_deployment.resources.output_content).exampleOutput.value
}

output "name" {
  value = data.azurerm_subscription.subscriptioninfo
}

I have created separate .tftpl extension file with ARM json code and defined the variables information. Once the terraform configuration is applied KUSTO language values will be replaced in template file and deployed using terraform.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
 
{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "variables": {},
    "parameters": {
        "Action_Group_Name": {
            "defaultValue": "Action_Group01",
            "type": "string",
            "metadata": {
                "description": "Name of the Action Group"
            }
        },
        "Action_Group_ShortName": {
            "defaultValue": "ag01",
            "type": "string",
            "metadata": {
                "description": "Name of the Action Group Short Name"
            }
        },
        "Email_Address": {
            "defaultValue": "[email protected]",
            "type": "string"
        },
        "Quota_Alert_Name": {
            "defaultValue": "Quota_Alert01",
            "type": "string"
        },
        "User_Managed_Identity": {
            "type": "string",
            "metadata": {
                "description": "ID of user managed identity"
            }
        },
        "Scopes": {
            "type": "string"
        },
        "Dimensions_Type": {
            "defaultValue": "microsoft.compute/locations/usages",
            "type": "string"
        },
        "Dimensions_Location": {
            "defaultValue": "eastus",
            "type": "string"
        },
        "Dimensions_Quota_Name": {
            "defaultValue": "cores",
            "type": "string"
        }
    },
    "resources": [
        {
            "type": "microsoft.insights/actionGroups",
            "apiVersion": "2023-09-01-preview",
            "name": "[parameters('Action_Group_Name')]",
            "location": "Global",
            "properties": {
                "groupShortName": "[parameters('Action_Group_ShortName')]",
                "enabled": true,
                "emailReceivers": [
                    {
                        "name": "EmailAction",
                        "emailAddress": "[parameters('Email_Address')]",
                        "useCommonAlertSchema": true
                    }
                ],
                "smsReceivers": [],
                "webhookReceivers": [],
                "eventHubReceivers": [],
                "itsmReceivers": [],
                "azureAppPushReceivers": [],
                "automationRunbookReceivers": [],
                "voiceReceivers": [],
                "logicAppReceivers": [],
                "azureFunctionReceivers": [],
                "armRoleReceivers": [
                    {
                        "name": "EmailARMRole",
                        "roleId": "8e3af657-a8ff-443c-a75c-2fe8c4bcb635",
                        "useCommonAlertSchema": true
                    }
                ]
            }
        },
        {
            "type": "microsoft.insights/scheduledqueryrules",
            "apiVersion": "2023-03-15-preview",
            "name": "[parameters('Quota_Alert_Name')]",
            "location": "[parameters('Dimensions_Location')]",
            "identity": {
                "type": "UserAssigned",
                "userAssignedIdentities": {
                    "[parameters('User_Managed_Identity')]": {}
                }
            },
            "properties": {
                "severity": 4,
                "enabled": true,
                "evaluationFrequency": "PT15M",
                "scopes": [
                    "[parameters('Scopes')]"
                ],
                "windowSize": "PT15M",
                "criteria": {
                    "allOf": [
                        {
                            "query": "arg(\"\").QuotaResources | where subscriptionId =~ '${QUERY_SUBSCRIPTION_ID}' | where type =~ 'microsoft.compute/locations/usages' | where isnotempty(properties) | mv-expand propertyJson = properties.value limit 400 | extend usage = propertyJson.currentValue, quota = propertyJson['limit'], quotaName = tostring(propertyJson['name'].value) | extend usagePercent = toint(usage)*100 / toint(quota) | project-away properties | where location in~ ('${QUERY_LOCATION}') | where quotaName in~ ('${QUERY_QUOTA_NAME}')",
                            "timeAggregation": "Maximum",
                            "metricMeasureColumn": "usagePercent",
                            "dimensions": [
                                {
                                    "name": "type",
                                    "operator": "Include",
                                    "values": [
                                        "[parameters('Dimensions_Type')]"
                                    ]
                                },
                                {
                                    "name": "location",
                                    "operator": "Include",
                                    "values": [
                                        "[parameters('Dimensions_Location')]"
                                    ]
                                },
                                {
                                    "name": "quotaName",
                                    "operator": "Include",
                                    "values": [
                                        "[parameters('Dimensions_Quota_Name')]"
                                    ]
                                }
                            ],
                            "operator": "GreaterThanOrEqual",
                            "threshold": 80,
                            "failingPeriods": {
                                "numberOfEvaluationPeriods": 1,
                                "minFailingPeriodsToAlert": 1
                            }
                        }
                    ]
                },
                "actions": {
                    "actionGroups": [
                        "[resourceId('microsoft.insights/actionGroups', parameters('Action_Group_Name'))]"
                    ]
                }
            }
        }
    ],
    "outputs": {
        "exampleOutput": {
            "type": "string",
            "value": "[resourceId('microsoft.insights/scheduledqueryrules', parameters('Quota_Alert_Name'))]"
        }
    }
}

Useful Articles
GitHub repository integration with Terraform Cloud to Deploy and Manage Azure
Resolved Terraform Error: POST https api.github.com user repos 401 Requires authentication
Azure DevOps Enable creation of classic build release pipelines grayed out
Adding parameters in Azure DevOps pipelines examples Part 1
Azure Web App Containers Cannot perform credential operations for providers Microsoft.ContainerRegistry ad admin user is disabled, enable it
Create CPU quota usage alerts for subscription using Azure ARM templates and PowerShell
Configure CPU quota usage alerts for subscription using Azure Bicep templates and Azure CLI
Deploy CPU quota usage alerts for subscription using Terraform azapi provider
Azure resource group deployments with ARM JSON templates in Subscription with PowerShell
Deploying Azure ARM templates using Terraform

Go Back

 Posted in Devops
  Tags terraform azure arm devops

Comment

Blog Search

Page Views

12386170

Archive

Comments

Follow me on Blogarama