Zobrazují se příspěvky se štítkemdatacentrum. Zobrazit všechny příspěvky
Zobrazují se příspěvky se štítkemdatacentrum. Zobrazit všechny příspěvky

2021-09-01

Azure Application Gateway change subnet

It is possible to change subnet in which is azure application gateway running without need to delete the whole instance and do all configuration from the beggining. It would be faster with recreation than going via process described below. This require quite long service break. I had reserved 2 hours and with little bit of trial-and-fix I managed to fit in.

  

Web UI way

Step 1 set application gateway to be manually scaled.

Step 2 stop application gateway via console command "az network application-gateway stop"

wait to proceeed

Step 3 open appgw page in portal.azure.com

modify URL from

https://portal.azure.com/#@CUSTOMER.onmicrosoft.com/resource/subscriptions/SUBSCRIPTIONID/resourceGroups/RESOURCEGROUP/providers/Microsoft.Network/applicationGateways/APPLICATIONGATEWAY/overview

modify URL to

https://resources.azure.com/#@CUSTOMER.onmicrosoft.com/resource/subscriptions/SUBSCRIPTIONID/resourceGroups/RESOURCEGROUP/providers/Microsoft.Network/applicationGateways/APPLICATIONGATEWAY

Step 4 open modified page

Step 5 click get and search in page "subnet". We are going to modifying this value only.

Step 6 switch to read/write mode on top of the page

Step 7 click on button edit

Step 8 find again "subnet" and modify name or path to subnet which you want to use

Step 9 click press PUT and wait for green tick on the screen

Step 10 if there is an orange warning, scroll to bottom of the page and there is additional text area explaining what is causing problem. Fix it and repeat.

Step 11 wait till update is done. There is no need to start application gateway again, because update will start it.

Azure Cloud Shell

Alternatively it's possible to do it with commands via Azure cloud shell or similar.

Check settings

az network application-gateway show --subscription <SUBSCRIPTIONID> --resource-group <RESOURCEGROUP> --name <APPLICATIONGATEWAY> --query gatewayIpConfigurations

Stop AppGw

az network application-gateway stop --subscription <SUBSCRIPTIONID> --resource-group <RESOURCEGROUP> --name <APPLICATIONGATEWAY>

Get JSON 

az network application-gateway show --subscription <SUBSCRIPTIONID> --resource-group <RESOURCEGROUP> --name <APPLICATIONGATEWAY> --output json

Prepare JSON by modify the value of subnet. Next stop shrink it to one line (visual studio code and shift+tab) and put as one liner. 

Post it

az network application-gateway update --subscription <SUBSCRIPTIONID> --resource-group <RESOURCEGROUP> --name  <APPLICATIONGATEWAY> --set gatewayIpConfigurations[0].subnet.id='[{"id": "/subscriptions/<SUBSCRIPTIONID>/resourceGroups/<RESOURCEGROUP>/providers/Microsoft.Network/applicationGateways/ <APPLICATIONGATEWAY>/gatewayIPConfigurations/appGatewayFrontendIP","name": "appGatewayFrontendIP","provisioningState": "Succeeded","resourceGroup": "<RESOURCEGROUP>","subnet": {"id": "/subscriptions/<SUBSCRIPTIONID>/resourceGroups/<RESOURCEGROUP>/providers/Microsoft.Network/virtualNetworks/<VNET>/subnets/<SUBNET>","resourceGroup": "<RESOURCEGROUP>"},"type": "Microsoft.Network/applicationGateways/gatewayIPConfigurations"}]'

Start

az network application-gateway start --subscription <SUBSCRIPTIONID> --resource-group <RESOURCEGROUP> --name <APPLICATIONGATEWAY>

Beware

There is some leftover network device after application gateway migration, so you cannot easily delete subnet you used for migration. See output of trial of deleting that subnet:

az network vnet subnet delete --ids /subscriptions/<SUBSCRIPTIONID>/resourceGroups/<RESOURCEGROUP>/providers/Microsoft.Network/virtualNetworks/<VNET>/subnets/<SUBNET>

(InUseSubnetCannotBeDeleted) Subnet <SUBNET> is in use by subscriptions/<SUBSCRIPTIONID>/resourceGroups/<WEIRDRESOURCEGROUPARMRG>/providers/Microsoft.Network/networkInterfaces/|providers|Microsoft.Compute|virtualMachineScaleSets|appgw|virtualMachines|1|networkInterfaces|custnic and cannot be deleted. In order to delete the subnet, delete all the resources within the subnet. See aka.ms/deletesubnet.

This I could resolve without help of Microsoft Support, so I created a ticket. Here is solution summary:

It was due to an error i.e. when the App GW was moved to other subnet from these subnet it was not completely removed in the backend due to which you were not able to delete the subnet   

  • Initially I informed you that all the delegations and the service endpoints need to be removed before proceeding with the deletion of the subnet.
  • You confirmed that you removed all the delegations and the service endpoints and I could see the same from my end that they got removed.
  • I suggested you to try the deletion again but the deletion still kept failing.
  • I asked for the error messages that you were receiving for reference.
  • Upon further troubleshooting we could see that the traces of Network Interface card of the AppGW that was previously present in those subnet were not deleted properly in the backend.
  • We engaged the backend team to do a manual cleanup of the traces left.
  • PG team completed the manual cleanup after which I requested you to try the deletion again and you confirmed that the deletion was successful.
  • You informed us that the ticket can be archived. 
It was quite exhausing process in the end, but thankfully working even it took several days to close it.

2021-06-29

Azure WAF set Custom rule Header name

WAF_v2 in Azure's Application Gateway is quite strong tool how to work with incoming traffic on web app. I had to do some exception on traffic filtering based on HTTP headers and use AZ CLI for that.

Structure of work is following: Application Gateway WAF policy set Match variable RequestHeaders correct Header name. There is no direct command under creation sequence of az network application-gateway waf-policy custom-rule create AND az network application-gateway waf-policy custom-rule match-condition add.

You can check existing settings with get command:

az network application-gateway waf-policy custom-rule show --name <RULE_NAME> --policy-name <WAF_NAME> -g <RG> --query matchConditions[].matchVariables -o tsv

To do a change you need to use following set command:

az network application-gateway waf-policy custom-rule update --name <RULE_NAME> --policy-name <WAF_NAME> --set matchConditions[0].matchVariables[0].selector=Referer

After that review the change again and that's all.

2021-06-10

Azure VM start failure KeyVault does not exist

Long time ago I migrated Azure virtual machine from classic model to ARM and later cleaned leftover items like machine specific Key Vault object. I didn't notify reference in current configuration, so one day I deallocated the VM I got error message.

Start VM 'Failed' - Provisioning Failed, Key Vault Does Not Exist

I couldn't recognize the problem, so I went to resources.azure.com and there under osProfile found reference to my deleted keyvault. 

My first (and wrong) idea was to change that value to some random and existing keyvault, so I collect original values by command

az vm show -g <RG> --name <VM> --query osProfile.secrets[].sourceVault.id -o tsv
az vm show -g <RG> --name <VM> --query osProfile.secrets[].vaultCertificates[].certificateUrl -o tsv

I tried to update them by calling

az vm update -g <RG> --name <VM> --set osProfile.secrets[0].vaultCertificates[0].certificateUrl=https://<VAULTNAME>.vault.azure.net/secrets/<NAME>/<ID>

but then I got error "Failed to start virtual machine. Error: The data retrieved from is not deserializable into JSON." I guess because the page itself was showing "{"error":{"code":"Unauthorized","message":"AKV10000: Request is missing a Bearer or PoP token."}}"

So this was a wrong way. I found correct command to remove those messages after some browsing on internet and it was:

Get-AzureRmVM -ResourceGroupName "<RG>" -Name "<VM>" | Remove-AzureRmVMSecret | Update-AzureRmVM


My virtual machine was able to start again after the mentioned steps. Note: there was also invalid value in osProfile.windowsConfiguration.winRm.listeners[].certificateUrl but this one has no problem for virtual machine start and just blocking WinRM functionality I guess.

I hope it helped a bit.

2021-01-10

Azure Site2Site VPN (ARM to Classic)

Issue:
Unable to get the new connection from classic to ARM VPN Gateway connected

Cause:
Authentication failure

Resolution:
Use the below commands to get the existing pre-shared key on the classic VPN gateway and then try to update the same on either sides.

Get-AzureVNetGatewayKey
   -VNetName <String>
   -LocalNetworkSiteName <String>
   [-Profile <AzureSMProfile>]
   [<CommonParameters>]

More Information:

PowerShell installation to use classic commands:

Install-Module -Name PowerShellGet -Force
Install-Module Azure
Install-Module Azure -AllowClobber
Import-Module Azure

Note:
While trying to use the classic subscription via PowerShell, you might have not seen the subscription listed from the below commands:

Add-AzureAccount
Get-AzureSubscription

To see your classic subscription here, you would have to add yourself as a co-administrator for the Classic Account which is what you did in this case.

Documentations for future references:

Azure Classic Select-AzureSubscription Error

No default subscription has been designated. Use Select-AzureSubscription -Default <subscriptionName> to set the default subscription.

Select-AzureSubscription : The subscription id doesn't exist.
Select-AzureSubscription : The subscription name doesn't exist.
Select-AzureSubscription : Parameter set cannot be resolved using the specified named parameters.

Add-AzureAccount : No subscriptions are associated with the logged in account in Azure Service Management (RDFE). This means that the logged in user is not an administrator or co-administrator for any account

Add-AzureAccount : AADSTS50074: Strong Authentication is required.

I had the following problems with Setting the default subscription because of two reasons. For this you need to have set co-administrator rights

Solution: Add classic administrator role inside AAD for user you are using to log in. Azure Portal > subscriptions > subscription - Access control (IAM) > Classic administrators > Add > Add co-administrator and try again!

Non responding Azure VM

It might happen to your virtual machine in Azure Cloud too. It gets stuck without responding to enabled services like SSH, HTTP even pings; in Microsoft words "VM was not responding to any means of communication".

Symptoms: There is no answer from the public IP range as same as from a private network from a machine on the same VNET and subnet. A tricky part is a machine in the portal looks up and running, stopping, and restarting. 

I opened the M$ support case because it was another occasion of the same behavior and I wanted to know the answer. We went through a classic scenario: Restart, deallocation, and redeploy via the portal. The extra task was the restart VM's from the serial console but the serial console did not come up even after a reboot.
Short answer: one of the disks was incorrectly mounted. One of the logs was containing crucial information as "Reached target Emergency Mode" followed by
Failed to mount /var/lib/docker. See 'systemctl status var-lib-docker.mount' for details. Dependency failed for Local File Systems.
That failed mount is preventing VM to boot, it needs to be fixed as described below. We had to create a rescue VM for which we used the OS disk of the impacted VM to create a new VM and it worked.

Solution: add to mount point /etc/fstab an item -nofail. Save and exit. Detach drive and do OS swap for the machine. The reboot should be OK and the machine should be online.

How to rescue the VM
  • Take the snapshot of OS disk - a full snapshot 
    • In disks – Created new disk using source path as a snapshot.
    • Verify the size of the disk and the type of disk used and used the same size and type to create a new disk.
  • Attach the disk to an existing Redhat VM, swap the disk, and mount it.
These few hints might help you to get rid of the troubles.

2020-07-15

Azure network peering when not authorized to access linked subscription

I got stacked for several hours on the case with network peering across different subscriptions with different tenantsRelated documentation. This scenario is usually when you do network peering with customer's Azure or between Azure with different owners. My case was easy since it was between Azure's managed by me, but still, there was a weird error. Luckily it got resolved by the way which you want to try too.

Azure Portal WebUI error message
Error: The client has permission to perform action 'Microsoft.Network/virtualNetworks/peer/action' on scope '<rg>/providers/Microsoft.Network/virtualNetworks/<vnet>/virtualNetworkPeerings/<peeringName>', however the current tenant 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' is not authorized to access linked subscription 'yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy'.

That sounds like you are missing some access rights. I tried despite I was subscription owner in IAM configuration to add a minimal needed role "Network Contributor".
az role assignment create --assignee name.surmame@company.com --role "Network Contributor" --scope /subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/<rg>/providers/Microsoft.Network/virtualNetworks/<vnet>

and also the same right for the other side
az role assignment create --assignee name.surmame@company.com --role "Network Contributor" --scope /subscriptions/yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy/resourceGroups/<rg>/providers/Microsoft.Network/virtualNetworks/<vnet>

It should be now working, but it wasn't. The portal's authorize button in peering configuration opened a window where is usually a login page, but it stayed forever on redirection. URL was https://rc.portal.azure.com/tokenauthorize#access_token=XXX. Using different web browser was not enough to resolve.

What next? Let's try to use Azure CLI via Cloud shell console.
az network vnet peering create --name vnetPeeringName --resource-group rgName --vnet-name vnetName --remote-vnet-id /subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/<rg>/providers/Microsoft.Network/virtualNetworks/<vnet> --allow-vnet-access

But an error out of this was the opposite subscription not found. So what next? WebUI is not working, the Azure command line is not working in this scenario? What's left? Yes, it's Powershell!
1) + 7)
Connect-AzAccount
Set-AzContext -SubscriptionId xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx -TenantId zzzzzzzz-zzzz-zzzz-zzzz-zzzzzzzzzzzz

2)
New-AzRoleAssignment -SignInName name.surmame@company.com -RoleDefinitionName "Network Contributor" -Scope /subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/<rg>/providers/Microsoft.Network/virtualNetworks/<vnet>

8)
$vNetA=Get-AzVirtualNetwork -Name vnetName -ResourceGroupName rgName
Add-AzVirtualNetworkPeering -Name vnetPeeringName -VirtualNetwork $vNetA -RemoteVirtualNetworkId "/subscriptions/yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy/resourceGroups/<rg2>/providers/Microsoft.Network/virtualNetworks/<vnet2>"


3) + 9)
Disconnect-AzAccount

4) + 10)
Connect-AzAccount
Set-AzContext -SubscriptionId yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy -TenantId qqqqqqqq-qqqq-qqqq-qqqq-qqqqqqqqqqqq

5)
New-AzRoleAssignment -SignInName name.surmame@company.com -RoleDefinitionName "Network Contributor" -Scope /subscriptions/yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy/resourceGroups/<rg2>/providers/Microsoft.Network/virtualNetworks/<vnet2>

6)
Disconnect-AzAccount

11)
$vNetB=Get-AzVirtualNetwork -Name vnetName2 -ResourceGroupName rgName2
Add-AzVirtualNetworkPeering -Name vnetPeeringName2 -VirtualNetwork $vNetB -RemoteVirtualNetworkId "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/<rg>/providers/Microsoft.Network/virtualNetworks/<vnet>"

12)
Disconnect-AzAccount

Network peering is enstablished after this procedure and thing which I don't understand is why it works only in PowerShell console and not in both other options. Well, at least it's working now.