AWS IRSA

This guide covers creating and configuring IAM roles for Kubernetes workloads running on EKS. IRSA enables your pods to securely access AWS services without managing static credentials.

How IRSA Works

IRSA uses OpenID Connect (OIDC) federation to allow Kubernetes service accounts to assume IAM roles:

1. EKS exposes an OIDC provider endpoint for your cluster

2. IAM trusts tokens issued by this OIDC provider

3. Pods use projected service account tokens to authenticate

4. AWS STS exchanges these tokens for temporary IAM credentials

This eliminates the need for long-lived access keys and provides fine-grained, pod-level access control.

---

Prerequisites

• EKS cluster with OIDC provider configured (enabled by default on new clusters)

• IAM permissions to create roles and policies

• Your cluster's OIDC provider URL (found in the EKS console or via aws eks describe-cluster)

---

Creating an IAM Role

Step 1: Get Your OIDC Provider Information

# Get the OIDC provider URL for your cluster
aws eks describe-cluster --name <cluster-name> --query "cluster.identity.oidc.issuer" --output text

# Example output: https://oidc.eks.eu-west-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE

Step 2: Create the IAM Trust Policy

Create a trust policy that allows the service account to assume the role. Replace the placeholders with your values:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::<ACCOUNT_ID>:oidc-provider/oidc.eks.<REGION>.amazonaws.com/id/<OIDC_ID>"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "oidc.eks.<REGION>.amazonaws.com/id/<OIDC_ID>:sub": "system:serviceaccount:<NAMESPACE>:<SERVICE_ACCOUNT_NAME>",
          "oidc.eks.<REGION>.amazonaws.com/id/<OIDC_ID>:aud": "sts.amazonaws.com"
        }
      }
    }
  ]
}

💡 Multiple Service Accounts

To allow multiple service accounts to assume the same role, use StringLike with a wildcard pattern:

"Condition": {
  "StringLike": {
    "oidc.eks.<REGION>.amazonaws.com/id/<OIDC_ID>:sub": "system:serviceaccount:<NAMESPACE>:*"
  }
}

Use this carefully - it grants access to all service accounts in the namespace.

Step 3: Create the IAM Role

aws iam create-role \
  --role-name my-app-role \
  --assume-role-policy-document file://trust-policy.json \
  --description "IAM role for my-app workload"

Step 4: Attach Policies to the Role

Attach the required policies for your application's AWS access:

# Attach a managed policy
aws iam attach-role-policy \
  --role-name my-app-role \
  --policy-arn arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess

# Or attach a custom inline policy
aws iam put-role-policy \
  --role-name my-app-role \
  --policy-name my-app-policy \
  --policy-document file://policy.json

Step 5: Configure Your Application

Add the role ARN annotation to your ServiceAccount configuration:

pgt-application:
  serviceAccount:
    name: my-app-sa
    annotations:
      eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/my-app-role

---

Common IAM Policies

AWS Secrets Manager (Read-Only)

For applications using External Secrets to fetch secrets:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue",
        "secretsmanager:DescribeSecret"
      ],
      "Resource": [
        "arn:aws:secretsmanager:<REGION>:<ACCOUNT_ID>:secret:<PREFIX>/*"
      ]
    }
  ]
}

S3 Bucket Access

For applications that need to read/write to S3:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::<BUCKET_NAME>",
        "arn:aws:s3:::<BUCKET_NAME>/*"
      ]
    }
  ]
}

SQS Queue Access

For ScaledJobs processing SQS messages:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "sqs:ReceiveMessage",
        "sqs:DeleteMessage",
        "sqs:GetQueueAttributes",
        "sqs:GetQueueUrl"
      ],
      "Resource": [
        "arn:aws:sqs:<REGION>:<ACCOUNT_ID>:<QUEUE_NAME>"
      ]
    }
  ]
}

DynamoDB Table Access

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem",
        "dynamodb:UpdateItem",
        "dynamodb:DeleteItem",
        "dynamodb:Query",
        "dynamodb:Scan"
      ],
      "Resource": [
        "arn:aws:dynamodb:<REGION>:<ACCOUNT_ID>:table/<TABLE_NAME>",
        "arn:aws:dynamodb:<REGION>:<ACCOUNT_ID>:table/<TABLE_NAME>/index/*"
      ]
    }
  ]
}

🔒 Least Privilege

Always scope policies to specific resources using ARNs rather than wildcards (*). This limits the blast radius if credentials are compromised.

---

Terraform Example

If you manage infrastructure with Terraform, here's a complete example:

data "aws_eks_cluster" "cluster" {
  name = var.cluster_name
}

data "aws_iam_openid_connect_provider" "cluster" {
  url = data.aws_eks_cluster.cluster.identity[0].oidc[0].issuer
}

locals {
  oidc_provider_id = replace(data.aws_iam_openid_connect_provider.cluster.url, "https://", "")
}

data "aws_iam_policy_document" "app_assume_role" {
  statement {
    effect = "Allow"

    principals {
      type        = "Federated"
      identifiers = [data.aws_iam_openid_connect_provider.cluster.arn]
    }

    actions = ["sts:AssumeRoleWithWebIdentity"]

    condition {
      test     = "StringEquals"
      variable = "${local.oidc_provider_id}:sub"
      values   = ["system:serviceaccount:${var.namespace}:${var.service_account_name}"]
    }

    condition {
      test     = "StringEquals"
      variable = "${local.oidc_provider_id}:aud"
      values   = ["sts.amazonaws.com"]
    }
  }
}

resource "aws_iam_role" "app_role" {
  name               = "my-app-role"
  assume_role_policy = data.aws_iam_policy_document.app_assume_role.json
}

data "aws_iam_policy_document" "app_policy" {
  statement {
    effect = "Allow"

    actions = [
      "secretsmanager:GetSecretValue",
      "secretsmanager:DescribeSecret",
    ]

    resources = [
      "arn:aws:secretsmanager:${var.region}:${var.account_id}:secret:${var.secret_prefix}/*"
    ]
  }
}

resource "aws_iam_role_policy" "app_policy" {
  name   = "my-app-policy"
  role   = aws_iam_role.app_role.id
  policy = data.aws_iam_policy_document.app_policy.json
}

---

Troubleshooting

Pod Cannot Assume Role

Error: An error occurred (AccessDenied) when calling the AssumeRoleWithWebIdentity operation

• Verify the trust policy includes the correct OIDC provider ARN

• Check the service account name and namespace in the trust policy condition match exactly

• Ensure the aud condition is set to sts.amazonaws.com

• Verify the ServiceAccount annotation matches the role ARN exactly

Access Denied to AWS Resources

Error: An error occurred (AccessDenied) when calling the <Action> operation

• Verify the IAM policy is attached to the role

• Check resource ARNs in the policy match your actual resources

• Ensure the policy actions include all required permissions

• Check for any SCPs (Service Control Policies) that might be blocking access

Token Expired or Invalid

Error: ExpiredTokenException or InvalidIdentityToken

• Ensure the OIDC provider is correctly configured in IAM

• Verify the EKS cluster OIDC issuer URL matches the IAM OIDC provider

• Check that the pod's service account token is being projected correctly

Debugging in Argo CD

Check Pod Logs:

1. Navigate to your application in the Argo CD UI

2. Expand the application tree to find your Pod

3. Click on the Pod resource

4. Select the Logs tab to view container output

5. Look for AWS SDK errors or authentication failures

Check ServiceAccount Configuration:

1. In the application tree, click on the ServiceAccount resource

2. Select the Live Manifest tab

3. Verify the eks.amazonaws.com/role-arn annotation is present and correct

Check Pod Environment:

1. Click on the Pod resource

2. Select the Live Manifest tab

3. Look for the AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE environment variables (injected automatically)

4. Verify the projected service account token volume is mounted

Check Events:

1. Click on the Pod or Deployment resource

2. Select the Events tab

3. Look for warnings related to service account tokens or IAM authentication

---

Best Practices

🔒 Security Recommendations

• One role per application: Create dedicated roles for each workload to isolate permissions

• Use specific resource ARNs: Avoid wildcards in resource specifications

• Namespace scoping: Include the namespace in trust policy conditions

• Regular audits: Review and remove unused roles and policies

• Use conditions: Add additional conditions (e.g., source IP, MFA) where appropriate

• Tag resources: Tag IAM roles with application and team identifiers for cost allocation and auditing

---

Related Documentation

• PGT Application Deployment - ServiceAccount configuration

• PGT Secrets - Using IRSA with External Secrets

• PGT CronJob - ServiceAccount for scheduled jobs

• PGT ScaledJob - ServiceAccount for event-driven jobs

• Azure Workload Identity - Azure equivalent