How to Fix Terraform Permission Denied on Google Cloud Run

Troubleshooting Guide: Terraform “Permission Denied” on Google Cloud Run

As a Senior DevOps Engineer, encountering “Permission Denied” errors during Terraform deployments to Google Cloud Run is a common scenario. This guide will walk you through diagnosing and resolving these issues, ensuring your Infrastructure as Code (IaC) pipelines run smoothly.

1. The Root Cause: Why this happens on Google Cloud Run

“Permission Denied” errors from Terraform when targeting Google Cloud Run resources fundamentally stem from insufficient Identity and Access Management (IAM) permissions granted to the identity Terraform is using to interact with your Google Cloud Project.

Terraform, when executing terraform plan or terraform apply, acts on behalf of a specific principal:

  • Your User Account: If you’re running Terraform locally and have authenticated via gcloud auth application-default login.
  • A Service Account: If you’re running Terraform in a CI/CD pipeline (e.g., GitLab CI, GitHub Actions, Cloud Build) or on a VM/container that has been authenticated using a service account key or Workload Identity.

The “Permission Denied” message indicates that this principal lacks one or more of the necessary IAM roles or permissions to perform the requested actions (e.g., run.services.create, run.services.update, iam.serviceAccounts.actAs) on the Google Cloud Run service or related resources within your specified Google Cloud project.

Common scenarios include:

  • The service account or user has only Viewer permissions.
  • Required permissions for specific actions (like creating new services or updating IAM policies) are missing.
  • The Cloud Run service itself requires a service account for its runtime, and the Terraform principal lacks the iam.serviceAccounts.actAs permission to set this.

2. Quick Fix (CLI)

For a rapid resolution, you can grant broader (but temporary, if aiming for least privilege) permissions to the principal Terraform is using. Always aim to narrow these down later.

First, identify the principal Terraform is using:

  • For local development (your user): user:YOUR_EMAIL@example.com
  • For CI/CD (service account): serviceAccount:your-service-account@your-project-id.iam.gserviceaccount.com

Then, execute the following gcloud commands:

  1. Grant Cloud Run Admin role: This provides comprehensive control over Cloud Run services.

    gcloud projects add-iam-policy-binding YOUR_GCP_PROJECT_ID \
  --member="PRINCIPAL_IDENTIFIER" \
  --role="roles/run.admin"
    • Replace YOUR_GCP_PROJECT_ID with your actual Google Cloud Project ID.
    • Replace PRINCIPAL_IDENTIFIER with your user email or service account email.

  2. Grant Service Account User role (if your Cloud Run service uses a specific runtime service account): This allows your Terraform principal to assign and act as other service accounts.

    gcloud projects add-iam-policy-binding YOUR_GCP_PROJECT_ID \
  --member="PRINCIPAL_IDENTIFIER" \
  --role="roles/iam.serviceAccountUser"

    Example for a user running locally:

    gcloud projects add-iam-policy-binding my-dev-project-12345 \
  --member="user:devops-engineer@example.com" \
  --role="roles/run.admin"
gcloud projects add-iam-policy-binding my-dev-project-12345 \
  --member="user:devops-engineer@example.com" \
  --role="roles/iam.serviceAccountUser"

    Example for a CI/CD service account:

    gcloud projects add-iam-policy-binding my-prod-project-67890 \
  --member="serviceAccount:terraform-sa@my-prod-project-67890.iam.gserviceaccount.com" \
  --role="roles/run.admin"
gcloud projects add-iam-policy-binding my-prod-project-67890 \
  --member="serviceAccount:terraform-sa@my-prod-project-67890.iam.gserviceaccount.com" \
  --role="roles/iam.serviceAccountUser"

3. Configuration Check

After applying the quick fix, or if the issue persists, review your Terraform and Google Cloud configurations.

3.1. Terraform GCP Provider Block

Ensure your google provider block is correctly configured.

  • Project ID: Confirm the project attribute matches the Google Cloud Project ID where you’re deploying.

  • Region: Ensure region is set correctly for Cloud Run deployments.

  • Authentication (Optional but important to check):

    • Local User: If running locally, ensure you’re authenticated with gcloud auth application-default login.
    • Service Account Key: If using a service account key file (less recommended for long-term automation), ensure GOOGLE_APPLICATION_CREDENTIALS environment variable points to the correct JSON key file path.
    • Service Account Impersonation: For CI/CD, if using impersonation, verify the impersonate_service_account is configured correctly and your executing identity has the roles/iam.serviceAccountTokenCreator on the impersonated service account.
    # main.tf or versions.tf
terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 4.0" # Use an appropriate version
    }
  }
}

provider "google" {
  project = "YOUR_GCP_PROJECT_ID"
  region  = "us-central1" # Or your desired region
  # credentials = file("path/to/your/service-account-key.json") # Less recommended
  # impersonate_service_account = "terraform-sa@your-project-id.iam.gserviceaccount.com" # Recommended for CI/CD
}

3.2. Google Cloud IAM Policies

Verify the exact permissions held by the principal Terraform is using.

  1. Check current IAM policy for your project:

    gcloud projects get-iam-policy YOUR_GCP_PROJECT_ID \
  --format="table(bindings.role,bindings.members)" \
  --flatten="bindings[].members"

    Review the output to confirm if your principal (user or service account) is listed under the roles/run.admin (or other relevant roles) for the specified project.

  2. Principle of Least Privilege: While roles/run.admin provides a quick fix, it’s highly recommended to refine permissions to the minimum necessary for security. For Cloud Run, a more granular set of permissions typically includes:

    • roles/run.developer (Allows managing Cloud Run services, but not IAM policies on them)
    • roles/iam.serviceAccountUser (If the Cloud Run service itself needs to run as a specific service account)
    • roles/editor (Only if truly necessary and understood implications, very broad)
    • roles/artifactregistry.reader (If your Cloud Run service pulls images from Artifact Registry)
    • roles/logging.viewer (To view Cloud Run logs)
    • roles/monitoring.viewer (To view Cloud Run metrics)

    You would then use gcloud projects remove-iam-policy-binding to take away the run.admin role and add the more specific roles.

4. Verification

Once you’ve made the necessary adjustments, verify the fix.

  1. Run Terraform Plan: This performs a dry run and checks if Terraform can read the current state and plan changes without permission issues.

    terraform plan

    If successful, you should see the proposed changes without “Permission Denied” errors.

  2. Run Terraform Apply: If terraform plan is clean, proceed with terraform apply to deploy the changes.

    terraform apply

    Monitor the output for successful resource creation/update.

  3. Check Google Cloud Console:

    • Navigate to the Cloud Run services in your GCP project.
    • Confirm that the service managed by Terraform is deployed and configured as expected.
    • Check the IAM section for the project and specific resources to ensure the permissions are now correctly reflected.

  4. Review Cloud Audit Logs: If the terraform apply still fails with “Permission Denied,” dive deeper into Cloud Audit Logs for granular details.

    gcloud logging read "protoPayload.serviceName=cloudresourcemanager.googleapis.com AND protoPayload.status.message:PERMISSION_DENIED AND resource.type=project" \
  --limit=5 --format=json

    Look for specific methods (methodName) that failed and the associated permissions that were missing. This will often reveal the exact permission string (e.g., run.services.create) that your principal requires.

By systematically addressing the identity, roles, and Terraform configuration, you’ll effectively resolve “Permission Denied” errors when deploying to Google Cloud Run with Terraform. Remember to prioritize the Principle of Least Privilege for long-term security.

⚡ Related Troubleshooting Guides