Azure VM start failure KeyVault does not exist

Long time ago I migrated Azure virtual machine from classic model to ARM and later cleaned leftover items like machine specific Key Vault object. I didn't notify reference in current configuration, so one day I deallocated the VM I got error message.

Start VM 'Failed' - Provisioning Failed, Key Vault Does Not Exist

I couldn't recognize the problem, so I went to resources.azure.com and there under osProfile found reference to my deleted keyvault. 

My first (and wrong) idea was to change that value to some random and existing keyvault, so I collect original values by command

az vm show -g <RG> --name <VM> --query osProfile.secrets[].sourceVault.id -o tsv
az vm show -g <RG> --name <VM> --query osProfile.secrets[].vaultCertificates[].certificateUrl -o tsv

I tried to update them by calling

az vm update -g <RG> --name <VM> --set osProfile.secrets[0].vaultCertificates[0].certificateUrl=https://<VAULTNAME>.vault.azure.net/secrets/<NAME>/<ID>

but then I got error "Failed to start virtual machine. Error: The data retrieved from is not deserializable into JSON." I guess because the page itself was showing "{"error":{"code":"Unauthorized","message":"AKV10000: Request is missing a Bearer or PoP token."}}"

So this was a wrong way. I found correct command to remove those messages after some browsing on internet and it was:

Get-AzureRmVM -ResourceGroupName "<RG>" -Name "<VM>" | Remove-AzureRmVMSecret | Update-AzureRmVM

My virtual machine was able to start again after the mentioned steps. Note: there was also invalid value in osProfile.windowsConfiguration.winRm.listeners[].certificateUrl but this one has no problem for virtual machine start and just blocking WinRM functionality I guess.

I hope it helped a bit.