Azure Workload Identities

This guide covers creating and configuring managed identities for Kubernetes workloads running on AKS. Workload Identity enables your pods to securely access Azure services without managing static credentials.

How Workload Identity Works

Azure Workload Identity uses federated identity credentials to allow Kubernetes service accounts to authenticate as Azure AD applications:

1. AKS issues OIDC tokens for service accounts

2. Azure AD trusts tokens from the AKS OIDC issuer

3. Pods exchange service account tokens for Azure AD tokens

4. Azure resources validate the Azure AD token and grant access

This eliminates the need for client secrets and provides fine-grained, pod-level access control.

---

Prerequisites

• AKS cluster with OIDC issuer and Workload Identity enabled

• Permissions to create managed identities and role assignments

• Your cluster's OIDC issuer URL (found in the AKS portal or via Azure CLI)

---

Creating a Managed Identity

Step 1: Get Your AKS OIDC Issuer URL

# Get the OIDC issuer URL for your cluster
az aks show \
  --resource-group <RESOURCE_GROUP> \
  --name <CLUSTER_NAME> \
  --query "oidcIssuerProfile.issuerUrl" \
  --output tsv

# Example output: https://uksouth.oic.prod-aks.azure.com/00000000-0000-0000-0000-000000000000/00000000-0000-0000-0000-000000000000/

Step 2: Create the Managed Identity

az identity create \
  --resource-group <RESOURCE_GROUP> \
  --name my-app-identity \
  --location <LOCATION>

# Get the client ID for later use
az identity show \
  --resource-group <RESOURCE_GROUP> \
  --name my-app-identity \
  --query "clientId" \
  --output tsv

Step 3: Create the Federated Credential

Link the managed identity to your Kubernetes service account:

az identity federated-credential create \
  --name my-app-federated-credential \
  --identity-name my-app-identity \
  --resource-group <RESOURCE_GROUP> \
  --issuer <OIDC_ISSUER_URL> \
  --subject "system:serviceaccount:<NAMESPACE>:<SERVICE_ACCOUNT_NAME>" \
  --audiences "api://AzureADTokenExchange"

💡 Multiple Service Accounts

You can create multiple federated credentials for the same managed identity if different service accounts need the same permissions:

# Create additional federated credential for another namespace
az identity federated-credential create \
  --name my-app-federated-credential-staging \
  --identity-name my-app-identity \
  --resource-group <RESOURCE_GROUP> \
  --issuer <OIDC_ISSUER_URL> \
  --subject "system:serviceaccount:staging:my-app-sa" \
  --audiences "api://AzureADTokenExchange"

Step 4: Assign Azure Roles

Grant the managed identity access to Azure resources:

# Get the managed identity's principal ID
PRINCIPAL_ID=$(az identity show \
  --resource-group <RESOURCE_GROUP> \
  --name my-app-identity \
  --query "principalId" \
  --output tsv)

# Assign a role (example: Key Vault Secrets User)
az role assignment create \
  --assignee-object-id $PRINCIPAL_ID \
  --assignee-principal-type ServicePrincipal \
  --role "Key Vault Secrets User" \
  --scope /subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.KeyVault/vaults/<VAULT_NAME>

Step 5: Configure Your Application

Add the identity annotations to your ServiceAccount configuration:

pgt-application:
  serviceAccount:
    name: my-app-sa
    annotations:
      azure.workload.identity/client-id: "00000000-0000-0000-0000-000000000000"
      azure.workload.identity/tenant-id: "00000000-0000-0000-0000-000000000000"

  pod:
    metadata:
      labels:
        azure.workload.identity/use: "true"

---

Common Azure Role Assignments

Key Vault Secrets User

For applications using External Secrets to fetch secrets from Key Vault:

az role assignment create \
  --assignee-object-id $PRINCIPAL_ID \
  --assignee-principal-type ServicePrincipal \
  --role "Key Vault Secrets User" \
  --scope /subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.KeyVault/vaults/<VAULT_NAME>

Storage Blob Data Reader

For applications that need to read from Azure Blob Storage:

az role assignment create \
  --assignee-object-id $PRINCIPAL_ID \
  --assignee-principal-type ServicePrincipal \
  --role "Storage Blob Data Reader" \
  --scope /subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.Storage/storageAccounts/<STORAGE_ACCOUNT>

Storage Blob Data Contributor

For applications that need to read and write to Azure Blob Storage:

az role assignment create \
  --assignee-object-id $PRINCIPAL_ID \
  --assignee-principal-type ServicePrincipal \
  --role "Storage Blob Data Contributor" \
  --scope /subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.Storage/storageAccounts/<STORAGE_ACCOUNT>

Service Bus Data Receiver

For ScaledJobs processing Service Bus messages:

az role assignment create \
  --assignee-object-id $PRINCIPAL_ID \
  --assignee-principal-type ServicePrincipal \
  --role "Azure Service Bus Data Receiver" \
  --scope /subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.ServiceBus/namespaces/<NAMESPACE>

Service Bus Data Sender

For applications that need to send messages to Service Bus:

az role assignment create \
  --assignee-object-id $PRINCIPAL_ID \
  --assignee-principal-type ServicePrincipal \
  --role "Azure Service Bus Data Sender" \
  --scope /subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.ServiceBus/namespaces/<NAMESPACE>

Cosmos DB Data Contributor

For applications using Cosmos DB:

az role assignment create \
  --assignee-object-id $PRINCIPAL_ID \
  --assignee-principal-type ServicePrincipal \
  --role "Cosmos DB Built-in Data Contributor" \
  --scope /subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.DocumentDB/databaseAccounts/<ACCOUNT_NAME>

🔒 Least Privilege

Always assign roles at the most specific scope possible (e.g., specific Key Vault or storage account rather than resource group or subscription level).

---

Terraform Example

If you manage infrastructure with Terraform, here's a complete example:

data "azurerm_kubernetes_cluster" "cluster" {
  name                = var.cluster_name
  resource_group_name = var.resource_group_name
}

resource "azurerm_user_assigned_identity" "app_identity" {
  name                = "my-app-identity"
  resource_group_name = var.resource_group_name
  location            = var.location
}

resource "azurerm_federated_identity_credential" "app_credential" {
  name                = "my-app-federated-credential"
  resource_group_name = var.resource_group_name
  parent_id           = azurerm_user_assigned_identity.app_identity.id
  audience            = ["api://AzureADTokenExchange"]
  issuer              = data.azurerm_kubernetes_cluster.cluster.oidc_issuer_url
  subject             = "system:serviceaccount:${var.namespace}:${var.service_account_name}"
}

resource "azurerm_role_assignment" "keyvault_secrets_user" {
  scope                = var.keyvault_id
  role_definition_name = "Key Vault Secrets User"
  principal_id         = azurerm_user_assigned_identity.app_identity.principal_id
}

---

Troubleshooting

Authentication Failures

Error: AADSTS70021: No matching federated identity record found

• Verify the federated credential subject matches exactly: system:serviceaccount:<namespace>:<service-account-name>

• Check the OIDC issuer URL in the federated credential matches your AKS cluster

• Ensure the audience is set to api://AzureADTokenExchange

Access Denied to Azure Resources

Error: AuthorizationFailed or 403 Forbidden

• Verify the role assignment exists and is at the correct scope

• Check that the role includes the required permissions

• Ensure the role assignment has had time to propagate (can take a few minutes)

Token Exchange Failures

Error: Failed to exchange token

• Verify the managed identity exists and hasn't been deleted

• Check that Workload Identity is enabled on your AKS cluster

• Ensure the pod has the azure.workload.identity/use: "true" label

Debugging in Argo CD

Check Pod Logs:

1. Navigate to your application in the Argo CD UI

2. Expand the application tree to find your Pod

3. Click on the Pod resource

4. Select the Logs tab to view container output

5. Look for Azure SDK errors or authentication failures (e.g., AADSTS error codes)

Check ServiceAccount Configuration:

1. In the application tree, click on the ServiceAccount resource

2. Select the Live Manifest tab

3. Verify the azure.workload.identity/client-id and azure.workload.identity/tenant-id annotations are present and correct

Check Pod Labels and Environment:

1. Click on the Pod resource

2. Select the Live Manifest tab

3. Verify the azure.workload.identity/use: "true" label is present under metadata.labels

4. Check for the AZURE_CLIENT_ID, AZURE_TENANT_ID, and AZURE_FEDERATED_TOKEN_FILE environment variables (injected automatically)

5. Verify the projected service account token volume is mounted

Check Events:

1. Click on the Pod or Deployment resource

2. Select the Events tab

3. Look for warnings related to service account tokens or Azure authentication

---

Best Practices

🔒 Security Recommendations

• One identity per application: Create dedicated managed identities for each workload to isolate permissions

• Use specific scopes: Assign roles at the resource level, not resource group or subscription

• Namespace scoping: Include the namespace in federated credential subjects

• Regular audits: Review and remove unused identities and role assignments

• Use built-in roles: Prefer Azure built-in roles over custom role definitions where possible

• Tag resources: Tag managed identities with application and team identifiers for cost allocation and auditing

---

Related Documentation

• PGT Application Deployment - ServiceAccount configuration

• PGT Secrets - Using Workload Identity with External Secrets

• PGT CronJob - ServiceAccount for scheduled jobs

• PGT ScaledJob - ServiceAccount for event-driven jobs

• AWS IAM Roles (IRSA) - AWS equivalent