Codex on Azure: Foundry setup, APIM routing, streaming failures, and capacity design

Scope, dates, and how to read this guide

This article consolidates five engineering documents: a gateway-options comparison, a direct Foundry/Codex setup, an API Management load-balancing implementation, a capacity-design memo, and a June 26, 2026 mid-stream rate-limit experiment. The setup records refer to June 2026 model availability and Codex CLI 0.142.2. The capacity memo is a planning analysis, not a completed production rollout.

Observed successes, availability lists, prices, retry defaults, and design assumptions below belong to those records. They are not a promise of unlimited tokens, universal streaming behavior, present-day SKU support, or guaranteed availability. Recheck model availability, service documentation, organizational policies, and quotas before deployment. Commands are reference examples for an authorized environment; none were executed to create or change cloud resources for this article.

All actual subscription, principal, account, resource, publisher, endpoint, and personal-path identifiers have been removed. Bash examples require a Bash-compatible shell with Azure CLI, Node.js/npm, Python 3, curl, and the standard text utilities used below. They are not PowerShell syntax. Replace quoted YOUR_ values with your own authorized values; do not paste credentials into a published configuration.

Three ways to connect Codex to Foundry

Selection guide

• For a single developer or proof of concept, connect directly when a gateway adds no necessary control.
• For many developers or teams needing attribution, budgets, quotas, and centralized internal governance, the memo favors LiteLLM.
• For simple same-model PTU/PAYG overflow without a gateway hop, investigate native spillover.
• For Azure-side observability, governance, and separation of client credentials from model authentication, use APIM.
• The options can be combined: native spillover inside each model resource, with LiteLLM handling teams, models, regions, and attribution above it.

The overview argues that admission-time PTU spillover can avoid starting a stream on exhausted provisioned capacity. That is a design rationale, not a result of the GlobalStandard experiments below. It does not prove that every mid-stream failure is prevented, that PAYG always has capacity, or that arbitrary PTU/PAYG combinations are supported. Similarly, detecting SSE-body errors in LiteLLM was proposed for verification, not demonstrated.

Networking and price notes from the design record

The record lists inbound private endpoints for Standard v2/Premium v2, VNet injection and availability zones for Premium v2, and no built-in v2 multi-region deployment at the time; a separate global load-balancing architecture would be needed for regional HA under that assumption. These are historical SKU statements to revalidate. Its illustrative Premium v2 estimates were approximately USD 2,801/month for one unit and USD 4,200/month for a two-unit HA configuration, not a current quote. Check the Azure Pricing Calculator for region, unit count, and applicable charges. Equal underlying model usage was the comparison baseline; PTU commitments, PAYG usage, operations, and networking can change the real bill.

Direct Foundry setup: environment and deployment

The verified direct path used an AIServices account in East US, two GlobalStandard deployments of gpt-5.3-codex version 2026-02-24, 50K TPM assigned to each, Entra RBAC, and Codex CLI 0.142.2. In this guide the deployments are called A and B; their aliases are supplied through environment variables.

export SUB='YOUR_SUBSCRIPTION_ID'
export RG='YOUR_RESOURCE_GROUP'
export LOC='eastus'
export ACC='YOUR_FOUNDRY_ACCOUNT'
export DEPLOYMENT_A='YOUR_CODEX_DEPLOYMENT_A'
export DEPLOYMENT_B='YOUR_CODEX_DEPLOYMENT_B'
az login
az account set --subscription "$SUB"

# Inspect regional availability before choosing a version and SKU.
az cognitiveservices model list --location "$LOC" \
  --query "[?contains(model.name, 'codex')].{name:model.name, version:model.version, sku:model.skus[0].name}" \
  -o table

az group create -n "$RG" -l "$LOC" -o table
az cognitiveservices account create \
  --name "$ACC" --resource-group "$RG" --location "$LOC" \
  --kind AIServices --sku S0 --custom-domain "$ACC" --yes -o table

for DEPLOYMENT in "$DEPLOYMENT_A" "$DEPLOYMENT_B"; do
  az cognitiveservices account deployment create \
    --resource-group "$RG" --name "$ACC" \
    --deployment-name "$DEPLOYMENT" \
    --model-name gpt-5.3-codex --model-version 2026-02-24 --model-format OpenAI \
    --sku-name GlobalStandard --sku-capacity 50 -o table
done

az cognitiveservices account deployment list -g "$RG" -n "$ACC" \
  --query "[].{name:name, model:properties.model.name, ver:properties.model.version, sku:sku.name, cap:sku.capacity}" \
  -o table

In the June inventory, the Codex family included gpt-5-codex (2025-09-15), gpt-5.1-codex, gpt-5.1-codex-mini, gpt-5.2-codex, and gpt-5.3-codex (2026-02-24), with GlobalStandard listed. The sample query projects the first SKU entry rather than proving it is the only supported SKU. Account creation used --custom-domain for token authentication and the account's OpenAI endpoint behavior.

The original setup started at --sku-capacity 10, but even one CLI turn hit a rate limit. Raising the allocation to 50 allowed a single turn to succeed in that test. For this model/SKU, those values represented 10K and 50K TPM. Reissuing deployment create with the same deployment name updated its capacity in the experiment; the attempted deployment update --sku-capacity did not reflect the change in that environment. Always read back the deployment after changing capacity rather than assuming every CLI version behaves identically.

Entra authentication when local keys are disabled

The tested subscription enforced disableLocalAuth=true through Azure Policy. A key-list request failed with BadRequest: Failed to list key. disableLocalAuth is set to be true. An attempted resource update to false was reset to true by policy. This was a troubleshooting observation, not a recommendation to weaken or evade organizational policy. Disabled local authentication should not be confused with proof that no key material exists; the relevant fact is that key authentication was unavailable.

az cognitiveservices account show -n "$ACC" -g "$RG" \
  --query properties.disableLocalAuth -o tsv

ACC_ID=$(az cognitiveservices account show -n "$ACC" -g "$RG" --query id -o tsv)
MY_OID=$(az ad signed-in-user show --query id -o tsv)
az role assignment create \
  --assignee-object-id "$MY_OID" --assignee-principal-type User \
  --role 'Cognitive Services OpenAI User' --scope "$ACC_ID" -o table

export ENDPOINT="https://${ACC}.cognitiveservices.azure.com"
TOKEN=$(az account get-access-token --resource https://cognitiveservices.azure.com \
  --query accessToken -o tsv)
REQUEST=$(python3 -c 'import json,os; print(json.dumps({"model":os.environ["DEPLOYMENT_A"],"input":"Say OK in one word."}))')
curl -sS -X POST "$ENDPOINT/openai/v1/responses" \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d "$REQUEST"
unset TOKEN

The observed direct response was HTTP 200 with status: completed. For automation, supply the intended service principal or managed identity object ID and use --assignee-principal-type ServicePrincipal instead of querying a signed-in user. The Cognitive Services OpenAI User role was sufficient for data-plane inference in this setup. Do not broaden the assignment unnecessarily.

The OpenAI v1 endpoint is POST /openai/v1/responses; the request body's model field selects the Azure deployment alias. These v1 examples do not add an api-version query parameter. The bearer-token syntax shown above is a parameterized reconstruction of the intended authorization header; the source's header text had already been masked and was not copied as executable syntax.

Install and configure the direct Codex provider

node --version
npm install -g @openai/codex
codex --version

The record used Node.js 22.x and Codex CLI 0.142.2. An unpinned install may produce a different version now; verify supported configuration and retry behavior for the installed version.

Complete command-backed token helper

Save this complete inline example as $HOME/.codex/az-codex-token.sh. It obtains a Cognitive Services access token from the existing Azure CLI login and prints only the token to standard output.

#!/usr/bin/env bash
set -euo pipefail
az account get-access-token \
  --resource https://cognitiveservices.azure.com \
  --query accessToken -o tsv

chmod +x "$HOME/.codex/az-codex-token.sh"
"$HOME/.codex/az-codex-token.sh" | wc -c

The length check avoids displaying the token. Do not capture its output in shared logs. The helper requires a valid Azure CLI session and the intended identity's RBAC assignment.

Complete direct-provider TOML example

Merge this with the intended Codex configuration. Replace the deployment alias, account name, and entire YOUR_ABSOLUTE_TOKEN_HELPER_PATH value with your own values. TOML does not automatically expand the Bash variables introduced earlier; the helper path must resolve to the script you actually saved.

model = "YOUR_CODEX_DEPLOYMENT_A"
model_provider = "azure_foundry"

[model_providers.azure_foundry]
name = "Azure AI Foundry Codex"
base_url = "https://YOUR_FOUNDRY_ACCOUNT.cognitiveservices.azure.com/openai/v1"
wire_api = "responses"
request_max_retries = 4
stream_max_retries = 10
stream_idle_timeout_ms = 300000

[model_providers.azure_foundry.auth]
command = "YOUR_ABSOLUTE_TOKEN_HELPER_PATH"
timeout_ms = 10000
refresh_interval_ms = 300000

With wire_api = "responses", the client appends /responses to the base URL. Command-backed authentication runs the configured command without stdin, trims its output, and expects one nonempty token string; the result becomes the bearer credential. It is an alternative to env_key, not something to configure alongside it.

The 0.142.2 notes describe refreshing the command-backed credential every 300,000 ms (five minutes) and re-running the command after a 401. They describe refresh_interval_ms = 0 as disabling periodic refresh while retaining the 401 path. Entra access tokens commonly last about an hour, but lifetime and Azure CLI session behavior are not guaranteed by this configuration. If the Azure CLI sign-in/cache can no longer refresh credentials, authenticate again. No API key file is needed for this direct RBAC path.

Direct-path verification and sandbox behavior

codex doctor
codex exec --skip-git-repo-check --sandbox read-only -c approval_policy='"never"' \
  'Reply with exactly the text: CODEX-OK-5'
codex exec --skip-git-repo-check --sandbox read-only -c approval_policy='"never"' \
  --model "$DEPLOYMENT_B" 'Reply with exactly: CODEX-OK-53'

# Interactive use: default deployment, or explicitly select B.
codex
codex --model "$DEPLOYMENT_B"

# Only use a workspace where you intend to permit file changes.
codex exec --skip-git-repo-check --sandbox workspace-write -c approval_policy='"never"' \
  'Create a src subfolder and a simple Node.js API application.'

The direct tests returned CODEX-OK-5 with 17,735 tokens used and CODEX-OK-53 with 14,381 tokens used. A read-only sandbox intentionally cannot create files: receiving instructions instead of a written application is not an inference failure. The workspace-write variant permits writes under the workspace. If an authorized test needs package-installation network access, the recorded additional setting was -c sandbox_workspace_write.network_access=true. The source described interactive Codex's default as workspace-write with approval prompts; check the current local configuration rather than assuming that default. The noninteractive examples explicitly choose no approval prompts and should only be used in a controlled test workspace.

The tested deployment aliases produced warnings about missing model metadata/fallback metadata and failure to refresh available models. The inference tests still succeeded. Treat those particular observations as benign only when the actual request succeeds; not every catalog or authentication warning is necessarily harmless.

Why one small prompt can exceed capacity

A single Codex turn in the test included the system prompt and approximately ten built-in tool definitions, creating a request body of roughly 67 KB and approximately 14K–18K tokens. A 10K-TPM deployment could therefore reject even a short user prompt. The streaming response contained an error of this shape:

{"type":"error","error":{"type":"too_many_requests","code":"rate_limit_exceeded","message":"Rate limit exceeded for the requested deployment."}}

The message is generalized to avoid environment identifiers; the error types and code are retained. Codex reported a disconnected stream and backed off, with up to ten stream retries in the setup configuration. More retries cannot create token capacity. The successful 50K-TPM allocation was a lab result, not a production sizing recommendation.

Keep an agent session on the same model and version

A user turn is not necessarily one Responses call. Tool calls and results lead to further sequential requests, and reasoning.encrypted_content carries reasoning-continuity information. The original experiment first mixed gpt-5-codex and gpt-5.3-codex across requests. Each model could create files when called directly, but the mixed-model APIM path intermittently produced only a short preamble and ended without invoking file-writing tools.

Replacing the pool with two deployments of gpt-5.3-codex version 2026-02-24 produced successful workspace-write file creation in 3 of 3 tests. The operational rule drawn from this evidence is to distribute an agent session only across compatible deployments of the same model and version. This is not proof that every possible cross-resource state artifact is portable; test the actual session and state mechanism. Routing different stateless, single-shot tasks to different models is a separate decision. Splitting allocated quota across deployments can distribute load but does not double the subscription's total entitlement.

APIM architecture and the critical streaming distinction

Codex CLI
  | Ocp-Apim-Subscription-Key
  v
APIM Standard v2
  | choose deployment A or B; rewrite body.model
  | managed identity obtains Cognitive Services bearer token
  v
Foundry /openai/v1/responses
  |-- deployment A: same model and version
  `-- deployment B: same model and version

The client authenticates to APIM with a subscription key, while APIM's system-assigned managed identity authenticates to Foundry using Cognitive Services OpenAI User. Model credentials are not sent to clients. The example uses a generic API path codex and operation POST /openai/v1/responses, producing a client base URL ending in /codex/openai/v1.

In one direct streaming probe with deployment B reduced to 1K TPM, five consecutive responses started with HTTP 200 and then contained rate-limit errors. That is a measured failure mode, not a claim that every streaming throttle must always arrive as HTTP 200. The configured HTTP-429 retry still matters when a real 429 status is returned before a stream is committed.

Create APIM and assign its managed identity

The source's Azure CLI version did not accept StandardV2 in az apim create; its allowed list was Developer, Standard, Premium, Basic, Consumption, and Isolated. The experiment therefore used ARM REST with API version 2023-05-01-preview. It reported provisioning in a few minutes, versus roughly 30–45 minutes for classic SKU provisioning. Those timings and CLI limitations are historical, not a deployment-time promise.

export APIM='YOUR_APIM_SERVICE'
export PUBLISHER_EMAIL='YOUR_PUBLISHER_EMAIL'
export PUBLISHER_NAME='YOUR_PUBLISHER_NAME'
export API_ID='YOUR_APIM_API_ID'
export OPERATION_ID='YOUR_APIM_OPERATION_ID'
export APIM_SUBSCRIPTION_ID='YOUR_APIM_SUBSCRIPTION_ID'
export APIM_RESOURCE="https://management.azure.com/subscriptions/$SUB/resourceGroups/$RG/providers/Microsoft.ApiManagement/service/$APIM"

python3 - > apim.json <<'PY'
import json, os
print(json.dumps({
    "location": os.environ["LOC"],
    "sku": {"name": "StandardV2", "capacity": 1},
    "identity": {"type": "SystemAssigned"},
    "properties": {
        "publisherEmail": os.environ["PUBLISHER_EMAIL"],
        "publisherName": os.environ["PUBLISHER_NAME"]
    }
}))
PY

az rest --method put \
  --url "$APIM_RESOURCE?api-version=2023-05-01-preview" \
  --body @apim.json --headers 'Content-Type=application/json'
az rest --method get \
  --url "$APIM_RESOURCE?api-version=2023-05-01-preview" \
  --query '{state:properties.provisioningState, gw:properties.gatewayUrl, mi:identity.principalId}'

# Continue only after successful provisioning.
MI_OID=$(az rest --method get \
  --url "$APIM_RESOURCE?api-version=2023-05-01-preview" --query identity.principalId -o tsv)
ACC_ID=$(az cognitiveservices account show -n "$ACC" -g "$RG" --query id -o tsv)
az role assignment create \
  --assignee-object-id "$MI_OID" --assignee-principal-type ServicePrincipal \
  --role 'Cognitive Services OpenAI User' --scope "$ACC_ID"

The JSON generation above is a parameterized rendering of the complete inline ARM request, with proper JSON escaping for supplied values. The managed-identity object ID is read from the intended APIM resource rather than embedding the original principal ID.

Create the API and operation; understand the missing policy artifact

az apim api create --resource-group "$RG" --service-name "$APIM" \
  --api-id "$API_ID" --display-name 'Codex Load Balancer' \
  --path codex --protocols https --subscription-required true
az apim api operation create --resource-group "$RG" --service-name "$APIM" \
  --api-id "$API_ID" --operation-id "$OPERATION_ID" --display-name responses \
  --method POST --url-template '/openai/v1/responses'

The implementation document refers to a complete XML policy, but that file is not present in the inventoried source folders, and its full contents are not embedded in the documents. This article does not invent or distribute a replacement. API/operation creation alone does not install routing. You need an independently reviewed policy implementing the behavior below before running the APIM verification procedures.

Policy behavior described by the implementation record

1. In inbound, use authentication-managed-identity resource="https://cognitiveservices.azure.com" to authenticate with APIM's identity.
2. Set the Foundry backend base URL to the host only, such as https://YOUR_FOUNDRY_ACCOUNT.cognitiveservices.azure.com. The operation adds /openai/v1/responses. Including /openai/v1 in both places caused a duplicated path and HTTP 404.
3. Choose a primary deployment from A and B, both the same model/version, and make the other the secondary.
4. In backend, a retry block tests HTTP status 429, allows one retry, and uses first-fast-retry. Before each forward-request, rewrite the JSON body's model field to the selected deployment: primary on the first attempt, secondary on the retry.
5. In outbound, emit x-lb-deployment for the final selected deployment and x-lb-attempts for the number of backend attempts.

The initial implementation describes round-robin selection. The separate June 26 experiment describes an operational random-selection policy that was temporarily replaced with a hardcoded primary. These are different recorded policy states; the missing XML prevents proving a single universal implementation. Alternating or mixed deployment headers demonstrate distribution in a short sample, not mathematically strict round-robin behavior.

Applying an independently supplied policy

The following is the original ARM wrapping/application procedure with parameterized identifiers. It intentionally refuses to proceed without your XML file. YOUR_REVIEWED_POLICY.xml is a required external input, not a download supplied by this article.

export POLICY_XML='YOUR_REVIEWED_POLICY.xml'
if [ ! -f "$POLICY_XML" ]; then
  printf '%s\n' 'A reviewed policy XML file is required; no policy is included here.' >&2
else
  python3 - > policybody.json <<'PY'
import json, os
from pathlib import Path
xml = Path(os.environ["POLICY_XML"]).read_text(encoding="utf-8")
print(json.dumps({"properties": {"format": "xml", "value": xml}}))
PY
  az rest --method put \
    --url "$APIM_RESOURCE/apis/$API_ID/operations/$OPERATION_ID/policies/policy?api-version=2023-05-01-preview" \
    --body @policybody.json --headers 'Content-Type=application/json'
fi

Errors recorded while writing the XML included unescaped double quotes in expression attributes, unescaped ampersands in &&, and literal less-than signs. In the XML file, encode embedded double quotes as ", && as &&, and less-than as <. The C# block used by set-body, including generic syntax such as As<JObject>, was placed inside <![CDATA[ @{ ... } ]]>. These notes are authoring guidance, not the absent complete policy.

Issue an API-scoped subscription and configure Codex

Only retrieve credentials for your own authorized APIM instance. Store the subscription key privately; do not print it, commit it, or expose it in client-side website content.

python3 - > subscription.json <<'PY'
import json, os
print(json.dumps({"properties": {
    "displayName": "Codex CLI subscription",
    "scope": "/apis/" + os.environ["API_ID"],
    "state": "active"
}}))
PY
az rest --method put \
  --url "$APIM_RESOURCE/subscriptions/$APIM_SUBSCRIPTION_ID?api-version=2023-05-01-preview" \
  --body @subscription.json --headers 'Content-Type=application/json'
KEY=$(az rest --method post \
  --url "$APIM_RESOURCE/subscriptions/$APIM_SUBSCRIPTION_ID/listSecrets?api-version=2023-05-01-preview" \
  --query primaryKey -o tsv)
printf 'Key length: %s\n' "${#KEY}"
export APIM_URL="https://${APIM}.azure-api.net/codex/openai/v1/responses"

The lab key length was 32 characters; length alone is not a validity check. The source's client example used a literal subscription key in http_headers. The complete provider block below preserves that structure but has no actual credential. Supply a key only in a private local configuration. A supported environment-backed header or secret-store integration is preferable when available; it is not implemented by this placeholder.

[model_providers.azure_apim]
name = "Azure APIM Codex Load Balancer"
base_url = "https://YOUR_APIM_SERVICE.azure-api.net/codex/openai/v1"
wire_api = "responses"
http_headers = { "Ocp-Apim-Subscription-Key" = "YOUR_APIM_SUBSCRIPTION_KEY" }
request_max_retries = 4
stream_max_retries = 10
stream_idle_timeout_ms = 300000

This provider does not need the direct provider's auth block or env_key: APIM handles backend RBAC. The policy overwrites the request's model field with its chosen Azure deployment alias, so a client-supplied alias is not a routing selector for this particular policy. Keep the actual pool homogeneous regardless of that client value. To make APIM the default, set the top-level model_provider = "azure_apim"; otherwise select it explicitly.

codex exec --skip-git-repo-check --sandbox read-only -c approval_policy='"never"' \
  -c model_provider=azure_apim 'Reply with exactly: APIM-LB-OK'
codex -c model_provider=azure_apim

# Optional file-writing verification in an authorized test workspace.
codex exec --skip-git-repo-check --sandbox workspace-write -c approval_policy='"never"' \
  -c model_provider=azure_apim \
  'Create a src subfolder and a simple Node.js API application.'

The recorded inference result was APIM-LB-OK with 17,759 tokens used. After switching to two same-model deployments, the separate file-writing test passed 3/3. A read-only run cannot validate file creation. Storing the subscription key in TOML is still plaintext credential storage; central rotation, private local permissions, a secret store, or an appropriately designed Entra/JWT ingress policy may be required for sensitive environments.

APIM verification: distribution, failover, and stream limits

Prerequisites are the earlier variables, an authenticated Azure CLI session in the intended subscription, a provisioned APIM instance, its assigned backend RBAC role, the actual reviewed routing policy, an API-scoped key in KEY, and two healthy 50K-TPM deployments. The following examples use local files in the current working directory. Run only in an isolated test environment: changing a live deployment's capacity to 1 can disrupt other consumers.

Test 1: basic call and distribution

REQUEST=$(python3 -c 'import json,os; print(json.dumps({"model":os.environ["DEPLOYMENT_A"],"input":"say OK"}))')
for i in $(seq 1 6); do
  resp=$(curl -sS -D - -o response.json -X POST "$APIM_URL" \
    -H "Ocp-Apim-Subscription-Key: $KEY" -H 'Content-Type: application/json' \
    -d "$REQUEST")
  http=$(printf '%s\n' "$resp" | grep -i '^HTTP' | tail -1 | tr -d '\r')
  dep=$(printf '%s\n' "$resp" | grep -i '^x-lb-deployment:' | tr -d '\r' | cut -d' ' -f2)
  served=$(python3 -c 'import json; print("body.model=" + str(json.load(open("response.json")).get("model")))')
  printf 'req%s: %s | lb=%s | %s\n' "$i" "$http" "$dep" "$served"
done

The source observed HTTP 200, both deployment aliases appearing in x-lb-deployment, and matching response body.model values. This is the expected observation for that policy, not a universal promise that every API reports deployment aliases in the same response field.

Test 2: non-streaming HTTP-429 failover

# Reduce only the isolated test deployment B.
az cognitiveservices account deployment create -g "$RG" -n "$ACC" \
  --deployment-name "$DEPLOYMENT_B" \
  --model-name gpt-5.3-codex --model-version 2026-02-24 --model-format OpenAI \
  --sku-name GlobalStandard --sku-capacity 1

python3 - > large-request.json <<'PY'
import json, os
print(json.dumps({
    "model": os.environ["DEPLOYMENT_A"],
    "input": "explain quantum computing in great detail. " * 400
}))
PY
for i in $(seq 1 8); do
  resp=$(curl -sS -D - -o response.json -X POST "$APIM_URL" \
    -H "Ocp-Apim-Subscription-Key: $KEY" -H 'Content-Type: application/json' \
    --data-binary @large-request.json)
  http=$(printf '%s\n' "$resp" | grep -i '^HTTP' | tail -1 | tr -d '\r')
  dep=$(printf '%s\n' "$resp" | grep -i '^x-lb-deployment:' | tr -d '\r' | cut -d' ' -f2)
  att=$(printf '%s\n' "$resp" | grep -i '^x-lb-attempts:' | tr -d '\r' | cut -d' ' -f2)
  printf 'req%s: %s | lb=%s attempts=%s\n' "$i" "$http" "$dep" "$att"
done

# Restore B even if a test fails or must be interrupted.
az cognitiveservices account deployment create -g "$RG" -n "$ACC" \
  --deployment-name "$DEPLOYMENT_B" \
  --model-name gpt-5.3-codex --model-version 2026-02-24 --model-format OpenAI \
  --sku-name GlobalStandard --sku-capacity 50

The recorded result was 8/8 HTTP 200 responses. When B was initially selected and returned 429, the final serving deployment was A with x-lb-attempts=2; a request initially sent to healthy A had attempts=1. This demonstrates transparent failover in that eight-request non-streaming test. It does not establish 100% production availability or imply that an alternate deployment cannot also throttle.

Test 3: reproduce a streaming error behind HTTP 200

az cognitiveservices account deployment create -g "$RG" -n "$ACC" \
  --deployment-name "$DEPLOYMENT_B" \
  --model-name gpt-5.3-codex --model-version 2026-02-24 --model-format OpenAI \
  --sku-name GlobalStandard --sku-capacity 1
TOKEN=$(az account get-access-token --resource https://cognitiveservices.azure.com \
  --query accessToken -o tsv)
python3 - > stream-request.json <<'PY'
import json, os
print(json.dumps({
    "model": os.environ["DEPLOYMENT_B"],
    "stream": True,
    "input": "explain quantum computing in detail. " * 400
}))
PY
curl -sS -o stream-response.txt -w 'stream HTTP=%{http_code}\n' \
  -X POST "$ENDPOINT/openai/v1/responses" \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  --data-binary @stream-request.json
grep -E 'rate_limit_exceeded|response.failed|too_many_requests' stream-response.txt | head -1
unset TOKEN

# Restore B after the probe.
az cognitiveservices account deployment create -g "$RG" -n "$ACC" \
  --deployment-name "$DEPLOYMENT_B" \
  --model-name gpt-5.3-codex --model-version 2026-02-24 --model-format OpenAI \
  --sku-name GlobalStandard --sku-capacity 50

The direct-backend probe returned stream HTTP=200 with a rate-limit error inside the SSE body. APIM's status-based retry cannot infer that from a 200 header. After both deployments are restored, repeat the APIM-LB-OK Codex command as Test 4 to verify the complete client-to-gateway-to-managed-identity-to-Foundry path. Always read back both capacity values and remove local test outputs containing sensitive data when finished.

June 26 experiment: what Codex retries actually do

This separate experiment used Codex CLI 0.142.2 with wire_api = responses and stream_max_retries = 5, rather than the setup examples' explicit value of 10. Deployment A was reduced from capacity 50 to 1, while B remained at 50. Random backend selection was disabled and primary was hardcoded to A, forcing every first attempt to hit the throttled deployment. One streaming codex exec call was observed through client logs and Foundry metrics. Non-streaming had already been tested; buffering SSE was excluded for performance reasons.

Observed log sequence

Initial POST: HTTP 200, attempt 0
SSE event: response.failed; stream disconnected before completion
Retry 1/5 after 192 ms
Retry 2/5 after 414 ms
Retry 3/5 after 762 ms
Retry 4/5 after 1.636 s
Retry 5/5 after 2.904 s
Final error: stream disconnected before completion: response.failed event received

There were six HTTP POSTs: the initial call plus five retries. All six had HTTP status 200, and all six bodies reported failed status with too_many_requests/rate_limit_exceeded. The logs therefore disprove the simplistic explanation that Codex never retries this situation. It did retry, but every new gateway request was routed back to the same blocked backend.

Independent metric cross-check

For the approximately seven-minute observation window, Foundry metrics grouped by RatelimitKey, the deployment dimension in this record, showed:

The six failed backend calls match the client's six attempts, while the secondary received none. Within an APIM policy cycle, retry(StatusCode == 429) never activated because the header was 200. At the client level, response.failed caused the separate stream-disconnection retry path. A provider's status-code retry_429 setting is not the same as stream_max_retries.

Why random routing changes the probability, not the guarantee

Restoring random selection means every CLI retry is a new APIM call and another backend choice. Under the assumptions of independent, equally likely choices between two backends, one continuously healthy backend, and six total attempts, the probability of reaching the healthy backend at least once is:

1 - (1/2)^6 = 63/64 = 98.4375%
Probability of choosing the blocked backend all six times = 1/64 = 1.5625%

This is a mathematical scenario, not an observed 98.4% success measurement or an SLA. Weighted routing, correlated choices, shared quota pressure, errors unrelated to throttling, and both backends being exhausted invalidate the simple assumptions. The hardcoded-primary experiment measured six failures, not the random policy's long-run reliability.

Retry settings recorded for version 0.142.2

The record states that the original random-selection policy was restored and its MD5 matched the saved original, A was restored to 50, B remained at 50, and subscription keys were masked with no plaintext occurrences in the reviewed logs. No policy file, checksum value, or raw logs are supplied here.

Operational implications and unverified alternatives

1. Address real capacity first: provision adequate PTU or obtain sufficient PAYG quota. Neither gateway retries nor distribution raises the underlying quota ceiling.
2. Use enough same-model/version capacity for the actual agent request size; in the lab even a small turn carried 14K–18K tokens of total context/tool overhead.
3. Make retry routing intentional when possible, for example by excluding a backend known to have failed the request. The notes propose sticky-avoid or weighted-exclude strategies but provide no implementation.
4. For non-streaming requests that actually return HTTP 429, the demonstrated policy can retry a secondary transparently. It still needs a healthy target and retry budget.
5. Consider more deployments or regions and APIM backend pools/circuit breakers for applicable HTTP/URL-based failures. A status-based circuit breaker does not automatically inspect SSE error events inside HTTP 200.

Two more invasive ideas were recorded but not validated: force the backend request to stream:false, fail over on HTTP 429, then convert the successful response back into SSE compatible with Codex; or inspect/buffer early SSE events before committing client headers so an early error can trigger a different backend. Both raise latency, implementation, and client-compatibility questions. Once HTTP headers or part of a response have been sent, they cannot simply be retroactively changed into a new status or safely replayed without protocol-aware handling. The June experiment explicitly ruled out SSE buffering for performance reasons. These are not turnkey APIM capabilities supplied by this article.

Capacity planning: three models behind an existing LiteLLM gateway

The planning memo assumed an existing on-premises LiteLLM gateway, not APIM, to handle authentication, authorization, logging, routing, and fallback. It considered gpt-5.3-codex, gpt-5.4, and gpt-5.5 for 500–1,000 developers using Codex CLI or similar coding agents. The requirement was an endpoint in Korea Central, with global model processing assumed acceptable.

The demand estimate was 10 million tokens per person per day, potentially 20 million. At 1,000 users that is 10–20 billion tokens per day. Concentrating that load into eight hours gives roughly 20.8–41.7 million TPM, summarized in the memo as 20–40M TPM. At 500 users, the same per-person assumptions produce half those totals. The memo explicitly questions whether these are measured numbers or inflated assumptions. Daily volume is not the binding capacity metric: peak TPM, workload shape, and concurrency matter.

Model/SKU inventory recorded for Korea Central

The memo reports GlobalStandard PAYG and GlobalProvisionedManaged PTU availability for all three in Korea Central; no DataZoneProvisionedManaged for the checked models in Korea Central, Japan East, or Southeast Asia; and no Azure OpenAI model region in Korea South. This is an inventory observation at the time of analysis, not a current deployment guarantee or proof of allocated quota/capacity. The source does not supply raw model-list output for this capacity inventory.

Resolve data residency before selecting capacity

The source's Korea-only/zone wording is not sufficient evidence that DataZoneStandard meets a Korea-only processing requirement. Confirm the applicable data zone with Microsoft and organizational compliance owners before choosing a deployment. The lack of a second Korean model region in the observed inventory also constrained an in-country multi-region design. Global processing and a Korea-based resource do not eliminate regional endpoint-failure considerations.

The proposed PTU/PAYG structure

For each model, provision one GlobalProvisionedManaged deployment for the measured baseline and one GlobalStandard deployment for best-effort bursts: three pairs, six deployments. Keep the model and version the same within each pair. LiteLLM routes by selected model and distributes or falls back only within that model's compatible endpoints.

Codex or another coding client
  v
On-premises LiteLLM: authentication, attribution, budgets, routing, logs
  |-- Codex model group: gpt-5.3-codex PTU --overflow--> same-version PAYG
  |-- GPT-5.4 group:     gpt-5.4 PTU       --overflow--> same-version PAYG
  `-- GPT-5.5 group:     gpt-5.5 PTU       --overflow--> same-version PAYG

The memo characterizes PTU as the provisioned baseline and PAYG as burst absorption. Its shorthand of no 429 within PTU capacity should not be read as an unconditional service guarantee: actual workload sizing, supported model behavior, admission rules, and other failure modes still matter. PAYG remains best effort and subject to quota and service availability. Native spillover, where supported, and gateway-managed fallback are distinct mechanisms; no LiteLLM configuration or spillover-property command was included in the source.

What actually increases headroom

• The memo treats quota as scoped by subscription, region, model, and deployment SKU. Each model therefore has its own relevant pool; confirm the exact quota dimensions for the target service.
• Splitting the same region/model/SKU quota among more deployments is zero-sum for that quota. Per-deployment allocations can improve distribution and isolate a deployment's assigned bucket, but do not create an extra entitlement.
• PTU and PAYG are separate capacity/quota categories in the proposed design; using both can add headroom compared with only one, provided each allocation is actually available.
• Separate accounts in the same subscription/region/model/SKU are primarily an isolation and availability measure, not a way to multiply the shared quota.
• If one model exhausts the approved Korea Central capacity, an additional deployment in another supported region may provide another regional quota pool, subject to quota approval, capacity availability, and residency constraints.

Assign models by department only after validating substitutability

The memo proposes assigning one department to gpt-5.3-codex, another to gpt-5.5, and another to gpt-5.4, each with its own PTU/PAYG pair. LiteLLM can map an authenticated department to its default model. This distributes demand across model quota pools without switching models in the middle of an agent session.

That only works if the department's tasks accept the assigned model's code quality and tool-loop behavior. Workflows requiring Codex-specific multi-step patching stability should remain with a suitable Codex model. The memo notes that Codex CLI need not be limited to a model with a -codex suffix and proposes testing general gpt-5.4/gpt-5.5 models through its model setting.

The qualitative assessment was that newer general models may match or exceed single-shot code quality, while Codex variants are tuned for the Codex agent harness, including apply_patch, exec, and multi-step persistence. No comparative benchmark was supplied for that claim. Before adopting department-level model allocation, run the same tasks on each candidate and compare file-creation success, patch success, sustained tool use, and output quality. Do not treat a model-name suffix as a measured quality ranking.

Expanding to another region

Japan East and Southeast Asia were examples for possible same-model expansion beyond Korea Central. Under the recorded inventory, a second Azure OpenAI region inside Korea was not available. Any such expansion therefore requires a fresh residency and endpoint-location decision. Although clients may see only the LiteLLM address, that abstraction does not automatically satisfy a requirement that every backend API endpoint must be in Korea. Global-processing approval and permission to use a non-Korean backend endpoint must be clarified separately.

Single-region resilience with two accounts

The overview's strongest proposed single-region topology combined a highly available on-premises LiteLLM gateway with two Foundry accounts, each using same-model PTU-to-PAYG spillover pairs. LiteLLM would group the two compatible endpoints per model and fail over between accounts, while virtual keys and logs provide department/user attribution.

Codex clients
  v
LiteLLM replicas (at least two) + PostgreSQL
  |-- Foundry account A: model-specific PTU --spillover--> PAYG
  `-- Foundry account B: model-specific PTU --spillover--> PAYG

Gateway: model groups, virtual keys, budgets/rate limits, logs
Observability suggested in the memo: PostgreSQL + Grafana

• Within an account, supported native spillover handles the proposed PTU/PAYG admission overflow.
• Across accounts, gateway routing adds resource-level isolation and fallback, subject to the actual failure boundaries and available spare capacity.
• At the gateway, virtual keys, budgets, rate controls, and logging support governance and attribution.
• At least two LiteLLM replicas and a suitably resilient supporting data layer are necessary to avoid simply introducing a new gateway single point of failure.
• Two accounts do not double a shared subscription/region/model quota, and this topology does not survive every regional or shared-service failure. The original no-extra-capacity caveat remains essential.

Size from measurements, not an unlimited-TPM requirement

The memo warns that PTU is a capacity commitment: reserving a large amount against an unverified estimate can waste money. Its recommended sequence is to start largely with GlobalStandard and request sufficient quota, collect two to four weeks of actual per-model/per-department usage in LiteLLM logs, and then reserve PTU for the sustained baseline while leaving PAYG for peaks.

Ten million daily tokens per user describes a heavy agent workload, and twenty million suggests very heavy or near-continuous use. The memo calls 10–20 billion tokens/day an unusually large enterprise-scale demand. Cost depends on model mix, input/output ratio, caching, utilization, and the applicable billing model; the sources do not provide a completed cost model.

1. Does Korea location mean only the resource endpoint, or also the processing boundary?
2. Is 10 million tokens per user per day measured, a maximum, or an assumption?
3. How concentrated is traffic: eight working hours, continuous usage, and what actual peak TPM?
4. Can departments accept different models, and what proportion requires Codex-specific agent behavior?
5. What provisioned-capacity budget and baseline commitment are acceptable?
6. If a data zone is mandatory, what exact geographic boundary is supported and approved?

No gateway option makes token limits disappear. Quota increases, real provisioned capacity, permitted regional expansion, and justified model-level distribution are the meaningful capacity levers.

Restore test settings and clean up deliberately

After an induced-throttle experiment, restore any changed deployment to its intended capacity, restore the reviewed routing policy, and verify both. The 50K-TPM value below is the lab's restoration value; use the actual pre-test value for your environment. Removing infrastructure is optional and destructive; inspect the target resource group and dependencies first.

az cognitiveservices account deployment list -g "$RG" -n "$ACC" \
  --query '[].{name:name,capacity:sku.capacity}' -o table
unset KEY TOKEN

# Optional, destructive: remove only the intended APIM instance.
az rest --method delete \
  --url "$APIM_RESOURCE?api-version=2023-05-01-preview"

# Alternative full teardown: deletes the entire selected resource group.
# Run only if this group contains exclusively disposable test resources.
az group delete -n "$RG" --yes --no-wait

Included examples and source gaps

All five Markdown documents in the Codex folder were read in full. The folder inventory contained no companion XML policy, LiteLLM configuration, deployment template bundle, logs, or other downloadable source files. The complete inline helper, provider configurations, management-request bodies, and substantive verification commands have been translated and parameterized in this article. They remain inline; no fabricated companion files are offered.

Repeated setup/test commands were consolidated. Already-masked authorization headers were repaired into standard variable-based bearer syntax rather than copied as broken commands. Original personal paths and resource names are replaced by variables or YOUR_ placeholders. The source's round-robin versus random-routing descriptions are explicitly separated, and the absent full XML policy is called out as a reproducibility gap rather than silently recreated. Historical model/SKU/pricing statements, capacity guarantees asserted in planning prose, and probabilistic outcomes are distinguished from measured results.

Resources

No standalone companion files are distributed for this topic. Use the in-page examples and review the editorial notes below for source availability.

Editorial notes

This page consolidates the following source documents into an English technical guide:

• Overview — Comparing LiteLLM, Native Spillover, and API Management for Codex
• Azure AI Foundry Codex Setup and Local CLI Integration
• API Management Load Balancing Across Two Codex Deployments
• Capacity Design — LiteLLM with Three Models and Mixed PTU/PAYG
• Codex CLI and APIM — Mid-Stream 429 Retry Experiment, June 26, 2026

• Actual subscription, identity, resource, deployment-alias, publisher, account, endpoint, and personal-path identifiers were removed or parameterized. No private repository links or identifier-bearing original filenames are included.
• The referenced full APIM policy XML was absent from the folder inventory and not embedded in the source. Its documented behavior and ARM application procedure are preserved, but no replacement policy was invented.
• No LiteLLM configuration, native-spillover deployment command, raw logs, or companion binaries existed in the scoped folders. No fabricated downloads are offered; complete available inline examples are included in the article.
• Duplicate commands were consolidated. The failed local-authentication policy-change attempt is described historically rather than promoted as a step to weaken policy; already-masked bearer headers were repaired to valid variable-based syntax.
• Claims of unlimited capacity, universal mid-stream prevention, no-429 guarantees, and 100 percent availability were qualified as design assumptions or limited test findings. Historical regional SKU, price, and CLI-default statements were not independently revalidated.

No original credentials or private repository links are included. Do not put populated configuration files or copied production outputs back into this public site.