Skip to main content

AWS ECS → Vertex AI (Workload Identity Federation)

Secure, keyless authentication from AWS ECS to Google Vertex AI using Workload Identity Federation (WIF).

  • No service account keys or long-lived secrets
  • Short-lived, auto-rotating tokens
  • IAM-based, auditable access

Overview​

Components (in order of the authentication flow)​

  1. AWS ECS Task — Starts the container with an attached IAM role.

  2. WIF Config File — Configuration that defines the token exchange path: pool, provider, and target service account.

  3. Google SDK — Reads the config file and orchestrates the full authentication flow.

  4. AWS STS — Provides temporary AWS credentials when called by the SDK (via boto3).

  5. GCP Workload Identity Pool — Receives the AWS identity and initiates validation.

  6. AWS Identity Provider — Validates the AWS account and role against allowed values.

  7. IAM Credentials API — Exchanges the federated token for a short-lived service account token.

  8. GCP Service Account (vertex-caller-sa) — The impersonated identity with roles/aiplatform.user permission.

  9. Vertex AI — Receives API calls authenticated with the service account token.

How It Works​

AWS IAM Role → WIF Pool → AWS Identity Provider → GCP Service Account → Vertex AI

Security Requirements​

To authenticate, an ECS task must:

  1. Come from an allowed AWS account.
  2. Use an approved IAM role.
  3. Be accepted by the AWS Identity Provider in GCP.
  4. Be allowed to impersonate the target GC vertex-caller-sa service account.

The GCP vertex-caller-sa service account must have roles/aiplatform.user.

AWS Configuration​

Each ECS task must set:

VariableDescription
VERTEX_PROJECT_NAMETarget GCP project
GOOGLE_APPLICATION_CREDENTIALSExternal account (WIF) config file

Config files (configs/gcp_binding_prod.json, configs/gcp_binding_dev.json) contain no secrets.

Why We Patch google-auth​

The Problem​

The google-auth library expects AWS credentials from the EC2 metadata service (169.254.169.254). This works on EC2 instances but fails on ECS/Fargate — where credentials are provided via a different mechanism (task role credentials endpoint).

The Solution​

At app startup, we monkey-patch google-auth to fetch AWS credentials using boto3 instead. boto3 automatically detects the correct credential source for any AWS environment (EC2, ECS, Lambda, etc.).

When the Patch Applies​

EnvironmentPatch AppliedReason
Local devNoUses local GCP credentials or service account key
GCP (Cloud Run)NoUses native GCP metadata service
AWS ECS/FargateYesFixes credential fetching for WIF

Setting Up the Credential File (for replacement if needed)​

1. Download from GCP Console​

  1. Go to IAM & Admin → Workload Identity Federation
  2. Select your identity pool
  3. Click Grant Access (or select an existing connected service account)
  4. Choose Download Config and select the AWS provider
  5. Save the file to configs/gcp_binding_prod.json or configs/gcp_binding_dev.json

2. Add to Code​

Add the config file to your repository and make sure the GOOGLE_APPLICATION_CREDENTIALS point to it based on the desired env:

Credential File Format​

{
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/{PROJECT_NUMBER}/locations/global/workloadIdentityPools/{POOL_ID}/providers/aws-provider",
  "subject_token_type": "urn:ietf:params:aws:token-type:aws4_request",
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/{SERVICE_ACCOUNT_EMAIL}:generateAccessToken",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "environment_id": "aws1",
    "region_url": "http://169.254.169.254/latest/meta-data/placement/availability-zone",
    "url": "http://169.254.169.254/latest/meta-data/iam/security-credentials",
    "regional_cred_verification_url": "https://sts.{region}.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15"
  }
}