A quick blog post to show the fix I implemented in relation to receiving this error:-
failed to update user-assigned identities on node aks-nodepool-12345-vmss (add [19], del [76], update[0]), error: failed to update identities for aks-nodepool-12345-vmss in MC_aks_cluster_uksouth, error: azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for request to https://management.azure.com/subscriptions/b72ab7b7-723f-4b18-b6f6-03b0f2c6a1bb/resourceGroups/MC_aks_cluster_uksouth/providers/Microsoft.Compute/virtualMachineScaleSets/aks-nodepool-12345-vmss?api-version=2019-07-01: StatusCode=404 -- Original Error: adal: Refresh request failed. Status Code = '404'. Response body: getting assigned identities for pod default/mic-675d649b7b-zqbc2 in CREATED state failed after 16 attempts, retry duration [5]s, error: <nil>
With the initial error of error: <nil> there wasn’t much further logging available.
Here is the full output of log found under mic-* pod as part of aad-pod-identity
I1023 13:36:50.382862 1 crd.go:512] creating assigned id test/service3-service-65bd5969c7-9tsqc-test-test
I1023 13:36:50.382921 1 crd.go:512] creating assigned id test/test-service-auth-provider-69884f6757-d65t8-test-test
I1023 13:45:23.065236 1 cloudprovider.go:196] updating user-assigned identities on aks-nodepool-12345-vmss, assign [3], unassign [0]
E1023 13:53:55.140630 1 mic.go:1077] failed to update user-assigned identities on node aks-nodepool-12345-vmss (add [19], del [76], update[0]), error: failed to update identities for aks-nodepool-12345-vmss in MC_aks_cluster_uksouth, error: azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for request to https://management.azure.com/subscriptions/b72ab7b7-723f-4b18-b6f6-03b0f2c6a1bb/resourceGroups/MC_aks_cluster_uksouth/providers/Microsoft.Compute/virtualMachineScaleSets/aks-nodepool-12345-vmss?api-version=2019-07-01: StatusCode=404 -- Original Error: adal: Refresh request failed. Status Code = '404'. Response body: getting assigned identities for pod default/mic-675d649b7b-zqbc2 in CREATED state failed after 16 attempts, retry duration [5]s, error: <nil>
E1023 14:02:27.221163 1 mic.go:1080] failed to get a list of user-assigned identites from node aks-nodepool-12345-vmss, error: identity info is nil
I1023 14:02:27.221200 1 mic.go:511] work done: true. Found 19 pods, 8 ids, 8 bindings
I1023 14:02:27.221207 1 mic.go:512] total work cycles: 2, out of which work was done in: 2
I1023 14:02:27.221217 1 stats.go:99] ** stats collected **
I1023 14:02:27.221223 1 stats.go:82] Pod listing: 162.602µs
I1023 14:02:27.221232 1 stats.go:82] ID listing: 18.4µs
I1023 14:02:27.221237 1 stats.go:82] Binding listing: 25.1µs
I1023 14:02:27.221242 1 stats.go:82] Assigned ID listing: 226.803µs
I1023 14:02:27.221247 1 stats.go:82] System: 698.408µs
I1023 14:02:27.221252 1 stats.go:82] CacheSync: 5.2µs
I1023 14:02:27.221257 1 stats.go:82] Cloud provider get: 0s
I1023 14:02:27.221261 1 stats.go:82] Cloud provider put: 0s
I1023 14:02:27.221266 1 stats.go:82] Assigned ID addition: 1.325185467s
I1023 14:02:27.221271 1 stats.go:82] Assigned ID deletion: 0s
I1023 14:02:27.221276 1 stats.go:89] Number of cloud provider PUT: 0
I1023 14:02:27.221281 1 stats.go:89] Number of cloud provider GET: 0
I1023 14:02:27.221285 1 stats.go:89] Number of assigned ids created in this sync cycle: 0
I1023 14:02:27.221290 1 stats.go:89] Number of assigned ids updated in this sync cycle: 0
I1023 14:02:27.221294 1 stats.go:89] Number of assigned ids deleted in this sync cycle: 0
I1023 14:02:27.221298 1 stats.go:82] Find assigned ids to create: 31µs
I1023 14:02:27.221312 1 stats.go:82] Find assigned ids to delete: 76.801µs
I1023 14:02:27.221317 1 stats.go:82] Total time to assign or remove IDs: 0s
I1023 14:02:27.221321 1 stats.go:82] Event recording: 0s
I1023 14:02:27.221326 1 stats.go:82] Total: 25m36.840632035s
I1023 14:02:27.221331 1 stats.go:129] *********************
I1023 14:02:27.423396 1 mic.go:1006] processing node aks-nodepool-12345-vmss, add [22], del [82], update [0]
I1023 14:02:27.423461 1 crd.go:512] creating assigned id test/test-service-auth-provider-69884f6757-srr8v-test-test
I1023 14:02:27.423501 1 crd.go:512] creating assigned id service1/service1-web-default-84775546f6-n8rcf-service1-service1
I1023 14:02:27.423528 1 crd.go:512] creating assigned id service1/service1-web-default-84775546f6-v49gn-service1-service1
I1023 14:02:27.423563 1 crd.go:512] creating assigned id test/service3-service-65bd5969c7-gtph6-test-test
I1023 14:02:27.423655 1 crd.go:512] creating assigned id service1/service1-web-default-84775546f6-rbq49-service1-service1
I1023 14:02:27.423657 1 crd.go:512] creating assigned id service1/service1-web-default-84775546f6-qdmb8-service1-service1
I1023 14:02:27.423744 1 crd.go:512] creating assigned id test/test-service-auth-provider-69884f6757-7pc6h-test-test
I1023 14:02:27.423747 1 crd.go:512] creating assigned id service1/service1-web-default-84775546f6-znfbq-service1-service1
I1023 14:02:27.423750 1 crd.go:512] creating assigned id test/service3-service-65bd5969c7-pxjfn-test-test
I1023 14:11:00.320372 1 cloudprovider.go:196] updating user-assigned identities on aks-nodepool-12345-vmss, assign [3], unassign [0]
E1023 14:19:32.403082 1 mic.go:1077] failed to update user-assigned identities on node aks-nodepool-12345-vmss (add [22], del [82], update[0]), error: failed to update identities for aks-nodepool-12345-vmss in MC_aks_cluster_uksouth, error: azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for request to https://management.azure.com/subscriptions/b72ab7b7-723f-4b18-b6f6-03b0f2c6a1bb/resourceGroups/MC_aks_cluster_uksouth/providers/Microsoft.Compute/virtualMachineScaleSets/aks-nodepool-12345-vmss?api-version=2019-07-01: StatusCode=404 -- Original Error: adal: Refresh request failed. Status Code = '404'. Response body: getting assigned identities for pod default/mic-675d649b7b-zqbc2 in CREATED state failed after 16 attempts, retry duration [5]s, error: <nil>
E1023 14:28:04.482948 1 mic.go:1080] failed to get a list of user-assigned identites from node aks-nodepool-12345-vmss, error: identity info is nil
The fix for this, is to add and deploy this .yaml found here
mic-exception.yaml
apiVersion: "aadpodidentity.k8s.io/v1"
kind: AzurePodIdentityException
metadata:
name: mic-exception
namespace: default
spec:
podLabels:
app: mic
component: mic
---
apiVersion: "aadpodidentity.k8s.io/v1"
kind: AzurePodIdentityException
metadata:
name: aks-addon-exception
namespace: kube-system
spec:
podLabels:
kubernetes.azure.com/managedby: aks
After adding this mic-exception.yaml file along with removing and redeploying the aad-pod-identity pods, the required managed identities were assigned to the Virtual Machine scaleset.
Thanks for reading, hopefully this blog post has assisted you if you have been producing the same error!
Thomas Thornton - Microsoft Azure MVP - HashiCorp Ambassador 
I had the same issue getting aad-pod-identity working with azure-key-vault-to-kubernetes. It’s great once aad-pod-identity is all working though; no more managing Service Principals!
https://github.com/SparebankenVest/azure-key-vault-to-kubernetes is worth checking out too.
Yeah definitely it’s great removing the need for service principals and as you said, no more managing service principals! 😁