← All guides AI Release
RU Subscribe to @ai_release1

How to Access OpenAI Models via Google Cloud: Step-by-Step Setup

Updated: 21.10.2026 · AI Release · @ai_release1
How to Access OpenAI Models via Google Cloud: Step-by-Step Setup

TL;DR

Scope note: Google Cloud does not resell OpenAI API keys. The pattern below runs your workload inside Google Cloud and calls the OpenAI API from it, keeping the key in Secret Manager instead of source code or environment files. Whether a given OpenAI model is also offered as a managed endpoint in Vertex AI Model Garden changes over time — verify current partner-model availability against the documentation before choosing that path.

Requirements and setup

Illustration: Requirements and setup
Illustration: Requirements and setup

Example: a working starting state looks like this.

$ gcloud version
Google Cloud SDK 5xx.x.x
$ gcloud config list
[core]
project = ai-openai-gw
account = you@example.com

Why this works: the gcloud CLI stores the active project and account per user, so every later command applies to the right project without repeating the --project flag.

Step 1. Create the project baseline and enable the APIs

What to do: create (or select) the project, link billing, and enable the three APIs the rest of the guide depends on: Secret Manager, Cloud Run, and Cloud Build.

gcloud projects create ai-openai-gw --name="OpenAI via GCP"
gcloud config set project ai-openai-gw

# Find your billing account ID
gcloud beta billing accounts list

# Link it — replace the placeholder with the ID from the previous command
gcloud beta billing projects link ai-openai-gw \
  --billing-account=<BILLING_ACCOUNT_ID>

gcloud services enable \
  secretmanager.googleapis.com \
  run.googleapis.com \
  cloudbuild.googleapis.com

Placeholders: <BILLING_ACCOUNT_ID> is the value in the ACCOUNT_ID column, formatted XXXXXX-XXXXXX-XXXXXX. Replace ai-openai-gw with your own project ID if it is already taken — project IDs are globally unique.

Why: no Google Cloud API is callable until it is enabled on the project, and a project without a billing account rejects every billable call. Enabling only three services keeps the surface small and the audit log readable.

How to check: list what is actually enabled.

gcloud services list --enabled \
  --filter="config.name:(secretmanager.googleapis.com OR run.googleapis.com OR cloudbuild.googleapis.com)"

Expected output — three rows, each with STATE: ENABLED:

NAME                             TITLE
cloudbuild.googleapis.com        Cloud Build API
run.googleapis.com               Cloud Run Admin API
secretmanager.googleapis.com     Secret Manager API

If a row is missing, the enable call failed; rerun it and read the error text.

Step 2. Store the OpenAI key in Secret Manager and grant access

Illustration: Step 2. Store the OpenAI key in Secret Manager and grant access
Illustration: Step 2. Store the OpenAI key in Secret Manager and grant access

What to do: upload the OpenAI key as a secret version, create a dedicated service account, and bind only the Secret Accessor role to it.

# 1. Read the key into a shell variable without echoing it
read -rs OPENAI_API_KEY

# 2. Create the secret and its first version
printf '%s' "$OPENAI_API_KEY" \
  | gcloud secrets create openai-api-key \
      --data-file=- \
      --replication-policy=automatic

# 3. Create a least-privilege identity for the workload
gcloud iam service-accounts create openai-runner \
  --display-name="OpenAI runner"

# 4. Allow that identity to read this one secret
gcloud secrets add-iam-policy-binding openai-api-key \
  --member="serviceAccount:openai-runner@ai-openai-gw.iam.gserviceaccount.com" \
  --role="roles/secretmanager.secretAccessor"

Then deploy a container that receives the secret as a reference, not as a literal value:

gcloud run deploy openai-proxy \
  --image=<REGION>-docker.pkg.dev/ai-openai-gw/apps/openai-proxy:latest \
  --region=<REGION> \
  --service-account=openai-runner@ai-openai-gw.iam.gserviceaccount.com \
  --set-secrets=OPENAI_API_KEY=openai-api-key:latest

Replace <REGION> with a region you enabled, for example us-central1.

Why: --set-env-vars OPENAI_API_KEY=sk-... writes the literal key into the revision configuration, where anyone holding roles/viewer on the project can read it. --set-secrets stores only a reference (openai-api-key:latest), and the container fetches the value at runtime using the service account's identity. A dedicated service account means a compromised container can read exactly one secret and nothing else.

How to check: confirm the secret exists, has one version, and that the stored value looks like a key.

gcloud secrets versions list openai-api-key
gcloud secrets versions access latest --secret=openai-api-key | head -c 7

Expected: one row in STATE: enabled, then sk- (or sk-proj for project-scoped keys) with no trailing newline. If you see the key truncated mid-string or followed by a blank line, you uploaded it with echo instead of printf — recreate the version.

If you plan to route traffic to several providers behind one endpoint, the pattern in How to combine multiple AI providers behind one OpenAI-compatible router extends this step cleanly: one secret per provider, one service account per deployment.

Step 3. Verify the key end to end and lock down access

Illustration: Step 3. Verify the key end to end and lock down access
Illustration: Step 3. Verify the key end to end and lock down access

What to do: run two checks — one directly against the OpenAI API from your machine, one through the deployed Cloud Run service.

# Check A: the stored secret is a valid OpenAI credential
export OPENAI_API_KEY="$(gcloud secrets versions access latest --secret=openai-api-key)"
curl -sS https://api.openai.com/v1/models \
  -H "Authorization: Bearer $OPENAI_API_KEY" | head -c 300

Expected output starts with {"object":"list","data":[{"id":"... — a JSON list of model IDs available to your account. Note the IDs from that list rather than hard-coding a model name in your app; availability depends on your OpenAI account tier.

# Check B: the deployed service is up and bound to the right identity
gcloud run services describe openai-proxy \
  --region=<REGION> \
  --format="value(status.url,status.conditions[0].status)"

Expected: the service URL followed by True. A False here means the revision never became ready — read the logs with gcloud run services logs read openai-proxy --region=<REGION>.

Then send one request through your service:

SERVICE_URL=<YOUR_SERVICE_URL>
curl -sS "$SERVICE_URL/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{"model":"<MODEL_ID_FROM_STEP_3_CHECK_A>",
       "messages":[{"role":"user","content":"Reply with the single word: ok"}]}'

Why: Check A isolates the credential from your infrastructure — if it fails, the problem is the key, not Google Cloud. Check B isolates the deployment from the credential. Running them in that order means a failure points at exactly one layer.

How to check: success looks like an HTTP 200 with a choices array in the body:

{"id":"chatcmpl-...","object":"chat.completion",
 "choices":[{"index":0,"message":{"role":"assistant","content":"ok"},
 "finish_reason":"stop"}]}

A 403 from the Cloud Run URL means the service requires authentication — add -H "Authorization: Bearer $(gcloud auth print-identity-token)".

For running OpenAI-compatible endpoints from a coding agent instead of curl, see How to connect a local LLM to Claude Code: llama-server and an OpenAI-compatible API — the request shape is the same, only the base URL changes.

Troubleshooting

Problem. The service account cannot read the secret

gcloud secrets get-iam-policy openai-api-key
gcloud secrets add-iam-policy-binding openai-api-key \
  --member="serviceAccount:openai-runner@ai-openai-gw.iam.gserviceaccount.com" \
  --role="roles/secretmanager.secretAccessor"

Problem. OpenAI rejects the key that works in the dashboard

printf '%s' "$OPENAI_API_KEY" | gcloud secrets versions add openai-api-key --data-file=-
gcloud secrets versions access latest --secret=openai-api-key | wc -c

The byte count must equal the key length exactly — no extra 1 for a newline.

Problem. Requests fail with a quota error while the key is valid

Example: a minimal retry wrapper that only retries on quota/rate errors:

import time, openai

def chat(client, **kwargs):
    for attempt in range(4):
        try:
            return client.chat.completions.create(**kwargs)
        except openai.RateLimitError:
            time.sleep(2 ** attempt)
    raise RuntimeError("quota still exhausted after 4 attempts")

Summary and next steps

You now have the OpenAI key stored as a Secret Manager version, a least-privilege service account that can read only that secret, and a Cloud Run service verified end to end with a live chat completion. Next, add alerting on secretmanager.versions.access audit logs so you notice unexpected reads, and if you add a second provider, front both with the router guide linked above rather than duplicating credentials in each client.

FAQ

Do I need a Google Cloud account to use the OpenAI API?

No. The key is issued by OpenAI and billed by OpenAI; Google Cloud is used here only to store the key securely and to host the code that calls it.

Where should I put the API key in Google Cloud?

In Secret Manager, mounted into the workload as a reference via --set-secrets. Never in --set-env-vars, a Dockerfile, or a committed .env file.

Can I use several OpenAI keys for different environments?

Yes — create one secret per environment (openai-api-key-dev, openai-api-key-prod) and bind each service account to exactly one of them. That way a staging compromise cannot spend production quota.

How do I rotate the key without downtime?

Add a new version with gcloud secrets versions add openai-api-key --data-file=-, redeploy or restart the service, then disable the old version with gcloud secrets versions disable <VERSION> --secret=openai-api-key. The :latest reference picks up the new version on the next cold start.

🤖 AI summary

Related guides

How to Connect an AI Agent to Chrome: Browser Automation Setup

Пошаговый гайд по подключению ИИ-агентов к Chrome через расширения и API для автоматизации действий в фоновых вкладках.

How to connect a local LLM to Claude Code: llama-server and an OpenAI-compatible API

Пошаговая инструкция, как поднять llama-server или Ollama и заставить Claude Code работать с локальной моделью вместо пл

How to combine multiple AI providers behind one OpenAI-compatible router

Практический гайд по роутингу LLM-запросов: один эндпоинт и ключ для GPT, Claude, Gemini и локальных моделей с фолбэками