Skip to main content

Secret Management with AKS and Azure Key Vault

Securely managing sensitive information (Secrets) such as database connection strings, API keys, and certificates is crucial in containerized applications. In Azure Kubernetes Service (AKS), integrating with Azure Key Vault enables robust and scalable secret management.

Why Use Azure Key Vault?​

By default, Kubernetes native Secret resources are stored in etcd merely as Base64 encoded strings (although encryption at rest can be configured, it requires management). Using Azure Key Vault provides the following benefits:

  • Centralized Management: Manage application secrets centrally using Azure's managed service.
  • Access Control: Finely control who (which apps) can access which secrets using Azure RBAC or Access Policies.
  • Audit Logging: Log and monitor access history to secrets.
  • Lifecycle Management: Utilize features like automatic certificate renewal and secret rotation.

The standard and recommended way to use Azure Key Vault with AKS today is via the Azure Key Vault Provider for Secrets Store CSI Driver.

How It Works​

This driver mounts Secrets, Keys, and Certificates from Azure Key Vault into Pods as a CSI (Container Storage Interface) volume.

  1. Mount: Upon Pod startup, secrets are retrieved from Key Vault and mounted as files within the Pod's file system.
  2. Sync: Optionally, mounted secrets can be synced as Kubernetes native Secret resources (useful when you need to use them as environment variables).

Authentication: Microsoft Entra Workload ID​

For authenticating access to Key Vault, using Microsoft Entra Workload ID (formerly Azure AD Workload Identity) is strongly recommended. This allows Kubernetes Service Accounts to federate with Azure Managed Identities, enabling secure, passwordless access to Key Vault.

Implementation Steps Overview​

  1. Enable CSI Driver and Workload Identity on AKS Cluster:

    az aks update -n <cluster-name> -g <resource-group> --enable-secret-rotation --enable-oidc-issuer --enable-workload-identity

  2. Create Azure Key Vault and Secrets: Store the necessary sensitive information in Key Vault.

  3. Create Managed Identity and Assign Permissions: Create a Managed Identity with access rights (e.g., Key Vault Secrets User) to the Key Vault.

  4. Configure Service Account and Federated Credential: Create a Kubernetes Service Account and link it with the Azure Managed Identity.

  5. Create SecretProviderClass: Create a YAML file defining which secrets to retrieve from which Key Vault.

    apiVersion: secrets-store.csi.x-k8s.io/v1
    kind: SecretProviderClass
    metadata:
      name: azure-kv-name
    spec:
      provider: azure
      parameters:
        usePodIdentity: "false"
        useVMManagedIdentity: "false"
        clientID: "<managed-identity-client-id>"
        keyvaultName: "<key-vault-name>"
        objects:  |
          array:
            - |
              objectName: db-password
              objectType: secret
        tenantId: "<tenant-id>"

  6. Mount to Pod: Define volumes and volumeMounts in your Deployment manifest, referencing the SecretProviderClass.

Best Practices​

1. Use Workload Identity​

Avoid using legacy Pod Identity or Service Principals (Client Secrets). Use Workload Identity to minimize credential management overhead and leakage risks.

2. Principle of Least Privilege​

Grant the Managed Identity only the necessary permissions (e.g., read-only) to the required Key Vaults. When using Azure RBAC, the Key Vault Secrets User role is appropriate.

3. Enable Auto Rotation​

By enabling the auto-rotation feature of the Secrets Store CSI Driver (--enable-secret-rotation), mounted contents in Pods or Kubernetes Secrets can be periodically updated when secrets are updated in Key Vault.

  • Note: If used as environment variables, Pod restarts may be required (can be automated with tools like reloader).

4. Coexistence with GitOps​

Never commit the actual secrets to your Git repository (Helm charts or manifests). Instead, manage only the SecretProviderClass definitions (metadata like secret names and Key Vault names). This allows you to adhere to GitOps practices while keeping sensitive information securely separated.

5. Minimize Sync to Kubernetes Secrets​

Use the CSI Driver's secretObjects feature to sync to Kubernetes Secrets only when necessary, such as for environment variables or Ingress controller certificates. If file mounting is sufficient, avoiding sync reduces writes to etcd and lowers security risks.