Menu

Virtual Geek

Tales from real IT system administrators world and non-production environment

Azure Terraform fixed Availibility Zones on Virtual Machine Scale Set

While working previously on Create an Azure virtual machine scale set and load balancer using Terraform, I was trying to deploy Virtual Machine scale sets in availibility zones and i was receiving below error. To deploy VMSS in availaibility zones (1 and 2), I was using argument zone = ["1", "2"] under azurerm_virtual_machine_scale set resource, but the terraform didn't not like it. Whenever I was removing Azure VMs from availability zones parameter script, deployment was getting successful.

My strict requirement was VMSS must be deployed in the zones. I read complete error and this line I concentrated to troubleshoot- NetworkInterface or LoadBalancer has a different zone constraint Regional.

Error: compute.VirtualMachineScaleSetsClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="ComputeResourceZoneConstraintDoesNotMatchPublicIPAddressZoneConstraint" Message="Compute resource /subscriptions/9e22xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx/resourceGroups/VMSS/providers/Microsoft.Compute/virtualMachineScaleSets/VMScaleSet has a zone constraint 1 but the PublicIPAddress /subscriptions/9e22xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx/resourceGroups/VMSS/providers/Microsoft.Network/publicIPAddresses/vmss-public-ip used by the compute resource via NetworkInterface or LoadBalancer has a different zone constraint Regional." Details=[] InnerError={"errorDetail":"{\r\n  \"error\": {\r\n    \"code\": \"ComputeResourceZoneConstraintDoesNotMatchPublicIPAddressZoneConstraint\",\r\n    \"message\": \"Compute resource /subscriptions/9e22xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx/resourceGroups/VMSS/providers/Microsoft.Compute/virtualMachineScaleSets/VMScaleSet has a zone constraint 1 but the PublicIPAddress /subscriptions/9e22xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx/resourceGroups/VMSS/providers/Microsoft.Network/publicIPAddresses/vmss-public-ip used 
by the compute resource via NetworkInterface or LoadBalancer has a different zone constraint Regional.\",\r\n    \"details\": [],\r\n    \"innerError\": \"Source: Nrp.Frontend.Contract. Microsoft.WindowsAzure.Networking.Nrp.Frontend.Common.ValidationException: Compute resource /subscriptions/9e22xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx/resourceGroups/VMSS/providers/Microsoft.Compute/virtualMachineScaleSets/VMScaleSet has a zone constraint 
1 but the PublicIPAddress /subscriptions/9e22xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx/resourceGroups/VMSS/providers/Microsoft.Network/publicIPAddresses/vmss-public-ip used by the compute resource via NetworkInterface or LoadBalancer has a different zone constraint Regional.\\r\\n   at Microsoft.WindowsAzure.Networking.Nrp.Frontend.Contract.Csm.Public.PublicIpAddress.ValidateZonalConstraints(HashSet`1 publicIps, VM virtualMachine, VMScaleSet vmscaleSet, Resource referencingResource, Subscription subscription, ILogger logger) in X:\\\\bt\\\\1086953\\\\repo\\\\src\\\\sources\\\\Frontend\\\\FrontendContract\\\\Csm.Public\\\\PublicIPAddress.cs:line 365\\r\\n   at Microsoft.WindowsAzure.Networking.Nrp.Frontend.Contract.Internal.VMScaleSet.Validate(VMScaleSet existingResource, Subscription subscription, Dictionary`2 referencedResources, String apiVersion, Subscription computeSubscription, ILogger logger, String clientAppId) in X:\\\\bt\\\\1086953\\\\repo\\\\src\\\\sources\\\\Frontend\\\\FrontendContract\\\\Internal\\\\VMScaleSet.cs:line 1079\\r\\n   at Microsoft.WindowsAzure.Networking.Nrp.Frontend.Operations.Internal.ValidateVMScaleSetOperation.\u003cExecuteAsync\u003ed__31.MoveNext() in X:\\\\bt\\\\1086953\\\\repo\\\\src\\\\sources\\\\Frontend\\\\FrontEndOperations\\\\Internal\\\\ValidateVMScaleSetOperation.cs:line 322\\r\\n--- End of stack trace from previous location where exception was thrown ---\\r\\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\\r\\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\\r\\n   at Microsoft.WindowsAzure.Networking.Nrp.Frontend.Operations.OperationBase`1.\u003cRunAsync\u003ed__64.MoveNext() in X:\\\\bt\\\\1086953\\\\repo\\\\src\\\\sources\\\\Frontend\\\\FrontEndOperations\\\\OperationBase.cs:line 0\"\r\n  }\r\n}"}

Microsoft Azure Terraform Hashicorp configure VMSS virtual machine scale set load balancer azure provider private ip availibility zones sku standard network interface.png

To troubleshoot the issue I read registry.terraform.io documentation for resource virtual_machine_scale_set and I found a argument referent zone which I had already defined as array ["1", "2"]. Which was not very helpful line with no examples.

  • zones - (Optional) A collection of availability zones to spread the Virtual Machines over.

After few manual deployment on the Azure Portal, I got to know that to resolve this issue my Azure Virtual Machine Scale set is deployed into a specific zone in a region, whereas my Public IP address and Load Balancer isn't. So its a regular (basic) Public IP address and Load Balancer. as a result there was a need to create a Public Ip address and Load Balancer in the same zone as the VM scale set is in and link the VMSS to that compatible Public IP and Load Balancer. 

In - short PIP and LB are deployed as in basic sku by default if not mentioned and they are not supported with Virtual Machine or VMSS scale sets with zones (Basic instances does not support zones). To make them zone based and link with VMSS I need to set SKU of PIP and LB to standard. So only Standard SKU resources can be zone pinned. 

This information can be found on azurerm_public_ip or azure-lb terraform resource documentation - Availability Zones are only supported with a Standard SKU and in select regions at this time. Standard SKU Public IP Addresses and Load Balancer that do not specify a zone are zone redundant by default. 

By adding sku = "standard" argument to azurerm_public_ip and azurerm_lb resources my issue resolved in the tf file.

Download complete Terraform VMSS deployment script here or it is available on github.com.

microsoft azure provider terraform azurerm sku standard availability zones array load balancer public ip domain name label resource group name public ip address virtual machine scale set vmss.png

Once the terraform apply is complete without any error, I checked the results on the Azure Portal LB and PIP sku are standard and the main resource VMSS is configured in the availibility zone successfully.

Microsoft Azure Portal terraform deploy apply complete azurerm resources added changes destroyed sku availbility standard zone location subscription resource group sponsered tags.png

Useful Articles
Create an Azure App registrations in Azure Active Directory using PowerShell & AzureCLI
Get started and configure with certificate-based authentication in Azure
Create a Virtual machine on Microsoft Azure
PowerShell List All Azure Resverations
Powershell get the list of Azure Reservations Virtual Machines instances
Get the list Azure Reservation Catalog with PowerShell and AzureCLI
Azure automation account DSC for On-Premise Virtual Machine on boarding
Azure Powershell : Operation returned an invalid status code 'BadRequest'
Get Azure virtual machine backup reports using Powershell

Go Back

Comment

Blog Search

Page Views

5937898

Follow me on Blogarama