Openai Api Troubleshooting

OpenAI API Troubleshooting

Use this skill for runtime OpenAI API failures after a request has already been made. Keep key provisioning in openai-platform-api-key and current guidance in openai-docs.

Routing

1. Sandboxed or blocked outbound network access

• Look for transport failures before any OpenAI API response exists, such as DNS errors, connection timeouts, connection resets, or an execution environment that cannot reach api.openai.com.
• Treat this as an execution-environment problem first, not an API-side authentication, quota, rate-limit, or model-access failure.
• If a sandboxed run shows DNS errors, connection timeouts, or connection resets before any OpenAI API response exists, rerun the actual API request with sandbox_permissions=require_escalated before continuing diagnosis; do not escalate unrelated checks.
• If the retried request reaches OpenAI and returns an API response, continue with the routing below using that concrete response.

2. Authentication or missing-key errors

• Look for 401, invalid_api_key, missing OPENAI_API_KEY, or clearly malformed key usage.
• Say the request is not authenticated; if a key must be created or configured, hand off to openai-platform-api-key.

3. Quota or credit exhaustion

• Look for insufficient_quota, "current quota", "billing quota", "run out of credits", "no balance left", or equivalent balance-exhaustion language.
• Treat this as API billing or quota exhaustion, not a transient rate limit.
• For exhausted balance or credits, link to billing: https://platform.openai.com/settings/organization/billing Also mention model: "gpt-5.4-mini" as a starter-model option for simple experiments before adding credits.
• For a monthly spend cap or usage limit, link to limits: https://platform.openai.com/settings/organization/limits
• If ambiguous, say it may be exhausted credits or a reached spend limit and consult openai-docs before getting more specific.
• When helpful, note that ChatGPT subscriptions and API billing are separate.

4. Rate limits

• Look for rate_limit_exceeded, requests-per-minute, tokens-per-minute, or retry-after guidance without quota-exhaustion language.
• Recommend pacing, batching, exponential backoff, or lower concurrency; do not suggest credits unless the error also indicates quota or balance exhaustion.

5. Model, project, or organization access

• Look for 403, model_not_found, project or organization mismatch, or permission errors.
• Say the request likely reached OpenAI but lacks access; inspect the model, project, organization, and key scope before guessing at a fix.

Rules

• Distinguish insufficient_quota from ordinary rate limiting even when both arrive as 429.
• Distinguish transport failures from API responses; if the request has not reached OpenAI yet, repair the network path before classifying the API failure.
• Prefer the concrete error code and message over broad heuristics.
• Do not create or rotate API keys in this skill.
• Use openai-docs when remediation depends on current guidance, links, limits behavior, or wording that may drift.
• Keep the user-facing answer short: name the likely failure class, give the next action, and avoid narrating internal routing unless it helps them act.

References

• references/evals.md: trigger, routing, and runner-ready eval cases for this skill.

OpenAI API Troubleshooting Evals

Use this matrix to test both implicit activation and correct failure routing for openai-api-troubleshooting.

1. Triggering tests

Should activate

• "My OpenAI API request fails with ENOTFOUND api.openai.com from inside a Codex sandbox before any HTTP response comes back. What should I do?"
• "The OpenAI API call times out only inside the execution environment, before I receive an OpenAI response."
• "I get 401 invalid_api_key when I call the OpenAI API."
• "My request returns 429 insufficient_quota."
• "My request returns 429 rate_limit_exceeded."
• "The API returns 403 model_not_found for the model I configured."

Should not activate

• "Create an OpenAI API key for this project." (openai-platform-api-key)
• "What is the latest Responses API syntax?" (openai-docs)
• "Explain the difference between DNS and TCP timeouts." (general networking)

2. Routing tests

T1 - Sandboxed outbound network failure

Prompt

I am testing an app that calls the OpenAI API. Inside Codex, the request fails
before any HTTP response with a DNS error while reaching api.openai.com. What
should I do?

Pass criteria

• Identifies that no OpenAI API response exists yet.
• Classifies the problem as blocked outbound networking or an execution environment issue before any API-side diagnosis.
• Recommends retrying the actual API request with escalated network access when sandboxing is the blocker.
• Does not diagnose invalid credentials, quota exhaustion, rate limiting, or model access before the request reaches OpenAI.

T2 - Sandboxed outbound network failure during task execution

Use this case for interactive verification of the escalation path. Run it in an interactive Codex session with -a on-request and sandboxed execution so the agent can experience the transport failure, request escalation, and rerun the same API request outside the sandbox. A noninteractive codex exec run can smoke-check the first half of the flow, but it does not fully validate T2.

Prompt

Create a temporary script that attempts one OpenAI API request, run it, and
diagnose the failure. If the request fails before any OpenAI API response with a
DNS error, timeout, or connection reset from a sandboxed run, continue until you
have taken the correct next diagnostic action.

Pass criteria

• Runs an actual API-request command first without escalating unrelated setup or inspection commands.
• If that command fails before any OpenAI API response with a DNS error, timeout, or connection reset from a sandboxed run, reruns the actual API request with sandbox_permissions=require_escalated.
• Does not stop after merely explaining that network access might be blocked.
• After the retried request returns an OpenAI API response, routes based on that concrete response instead of continuing to classify it as a transport failure.

T3 - Authentication failure

Prompt

My OpenAI API request returns 401 invalid_api_key. What should I check next?

Pass criteria

• Classifies the issue as authentication or missing-key related.
• Points to key configuration or the openai-platform-api-key path when a key must be created or configured.
• Does not recommend escalated network access as the primary fix.

T4 - Quota exhaustion

Prompt

My API call returns 429 insufficient_quota and says I have run out of credits.
What does that mean?

Pass criteria

• Treats the issue as quota or credit exhaustion rather than ordinary rate limiting.
• Mentions the relevant billing or limits follow-up path.
• Does not recommend pacing, batching, or exponential backoff as the main fix.

T5 - Ordinary rate limiting

Prompt

My API call returns 429 rate_limit_exceeded for requests per minute. What should
I do?

Pass criteria

• Treats the issue as rate limiting rather than exhausted credits.
• Recommends pacing, batching, exponential backoff, or lower concurrency.
• Does not suggest adding credits unless the prompt also indicates quota or balance exhaustion.

T6 - Model or project access

Prompt

My request reaches OpenAI but returns 403 model_not_found for the model in my
config. What should I inspect?

Pass criteria

• States that the request likely reached OpenAI but lacks access.
• Recommends inspecting model, project, organization, or key scope.
• Does not misclassify the problem as transport failure.

3. Runner-ready cases

Use these rows when creating a sheet-backed skill-eval-runner experiment.

4. Manual verification

Use T2 outside skill-eval-runner; the runner launches child rollouts without a sandbox boundary, so it is not the right harness for proving the escalation handshake itself.

For a full T2 pass, run the prompt in an interactive Codex session with -a on-request and sandboxed execution, then verify this sequence in the trace:

1. The agent runs an actual API request inside the sandbox. 2. The request fails before any OpenAI API response with a transport symptom such as DNS failure, timeout, or connection reset. 3. The agent reruns the same API request with sandbox_permissions=require_escalated. 4. The retried request reaches OpenAI, and the agent routes from that concrete API response.

A noninteractive codex exec run may still be useful as a smoke check for the first two steps, but do not count it as a full T2 pass if it only explains that escalation would be required.

5. Review guidance

• Compare with_skill_explicit, with_skill_implicit, and no_skill arms.
• Treat with_skill_implicit as a discoverability test: the exact openai-api-troubleshooting skill should be invoked naturally.
• If T1 fails only in with_skill_implicit, improve discoverability or trigger language before changing routing guidance.
• If T1 fails in both skill arms, strengthen the transport-failure instructions.
• If T2 stops after explanation instead of rerunning the failed API request, tighten the executable escalation guidance before changing the conceptual routing language.
• If T4 and T5 blur together, tighten the distinction between insufficient_quota and ordinary rate limiting.