
Opencode Qa
- 93 installs
- 67.2k repo stars
- Updated August 4, 2026
- code-yeongyu/oh-my-openagent
QA opencode itself: verify CLI commands, prove plugin hooks fired via the SSE event stream, smoke-test the TUI under tmux, and inspect sessions in its SQLite DB.
About
A skill that QAs the opencode agent per case, mapping each need to a tested helper script: CLI verification, proving a plugin hook fired via the SSE event stream, tmux TUI smoke tests, and SQLite session investigation. A developer uses it to QA, smoke-test, or debug opencode's CLI, server, hooks, or TUI.
- Uses an isolated XDG sandbox so QA never writes junk sessions into the real DB
- Every helper script ships a --self-test that doubles as a regression check
Opencode Qa by the numbers
- 93 all-time installs (skills.sh)
- Ranked #1,022 of 2,153 Testing & QA skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/code-yeongyu/oh-my-openagent --skill opencode-qaAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 93 |
|---|---|
| repo stars | ★ 67.2k |
| Last updated | August 4, 2026 |
| Repository | code-yeongyu/oh-my-openagent ↗ |
What it does
QA opencode itself: verify CLI commands, prove plugin hooks fired via the SSE event stream, smoke-test the TUI under tmux, and inspect sessions in its SQLite DB.
Files
opencode QA
QA the opencode coding agent itself. This skill maps each QA need to a tested helper script and a deep reference. Every script ships a --self-test that asserts its scenario against the live machine, so the scripts are both the QA tools and their own regression checks.
Verified against opencode v1.17.7 (bun 1.3.12, macOS). Confirm the installed version with opencode --version; the surface is stable but always sanity check a flag with opencode <cmd> --help.
Golden rules (read before running anything)
- READS of the live DB are safe and intended. Investigating sessions (Case D)
only reads ~/.local/share/opencode/opencode.db.
- Anything that SPAWNS opencode (serve, run, the TUI) must use an isolated XDG
sandbox so QA never writes junk sessions into the real DB. The bundled scripts already do this; if you run opencode by hand for QA, set XDG_DATA_HOME / XDG_CONFIG_HOME / XDG_STATE_HOME / XDG_CACHE_HOME to temp dirs first.
- Global text search over the
parttable is a multi-GB scan. Always scope it
(--session, --recent, or --since). The text script refuses an unbounded scan on purpose.
- The opencode source repo (
packages/opencode) tests itself withbun test
and CANNOT run tests from the repo root. See references/testing-harness.md.
Setup
Scripts live next to this file under scripts/. Invoke them from this skill directory (or with their absolute path):
cd <this-skill-dir> # .agents/skills/opencode-qa
bash scripts/lib/common.sh --self-check # confirm the harness + depsDocker is the default QA surface. Run QA inside a disposable container that has the latest opencode and a copy of your config, with the host untouched: script/agent/qa-docker.sh (see references/docker-qa.md). The local scripts below are the fallback for when Docker is unavailable or on Windows.
common.sh provides the shared harness (DB path, SQL escaping, isolated XDG sandbox, free port, server start/stop, and an EXIT-trap cleanup). It requires opencode, sqlite3, curl, jq, and tmux on PATH.
Router: pick your case
| You want to... | Case | Script | Reference |
|---|---|---|---|
| Run opencode non-interactively / check a CLI command | A | opencode run --format json (inline) | references/cli-commands.md |
| Find a session by its id | D | scripts/db-session-by-id.sh <ses_id> | references/db-investigation.md |
| Find sessions by title/name | D | scripts/db-session-by-name.sh "<text>" | references/db-investigation.md |
| Find sessions by message text | D | scripts/db-session-by-text.sh --recent N "<text>" | references/db-investigation.md |
| Export a whole session as JSON | D | scripts/export-roundtrip.sh <ses_id> | references/db-investigation.md |
| Check the HTTP server / an endpoint | B | scripts/server-smoke.sh | references/server-api.md |
| Prove a hook / action / event fired | B | scripts/sse-hook-probe.sh | references/events-hooks.md |
| Prove serve-topology wake runner-split (reproduced/fixed) | B | `scripts/serve-wake-split-probe.sh --expect reproduced\ | fixed --evidence-dir DIR (self-test: --self-test; fake LLM: scripts/lib/fake-openai-server.mjs`) |
| Smoke-test the TUI | C | scripts/tui-smoke.sh | references/tui-tmux.md |
| Write/run a test in the opencode source | - | (bun test) | references/testing-harness.md |
| Drive opencode from a Bun/TS script | - | (SDK) | references/sdk.md |
Case A: CLI / terminal works
The canonical scriptable, non-interactive entry is opencode run. JSON mode emits one event per line so you can assert on it.
# stream structured events (types: text, tool_use, step_start, step_finish, reasoning, error)
opencode run "list files in src" --format json
# run a slash command
opencode run --command commit
# resume the last session
opencode run -c "continue"
# target an already-running server instead of booting one
opencode run "explain auth" --attach http://127.0.0.1:4096 -p "$OPENCODE_SERVER_PASSWORD"Other QA-useful commands: opencode db path, opencode debug paths, opencode session list --format json, opencode models --verbose. Full flag detail in references/cli-commands.md.
Case B: a specific hook, action, or event
opencode publishes lifecycle events over Server-Sent Events at GET /event. Plugins observe the same events via the event hook, so seeing an event on the wire proves a hook would fire.
# prove the SSE plumbing works (isolated server, asserts server.connected)
bash scripts/sse-hook-probe.sh --self-test
# watch a REAL server for a specific event while you trigger an action
bash scripts/sse-hook-probe.sh --attach http://127.0.0.1:4096 \
--password "$OPENCODE_SERVER_PASSWORD" --directory "$PWD" \
--event message.part.updated --timeout 30Trigger an action over HTTP (fire-and-forget so the stream is not blocked):
curl -X POST -u opencode:$OPENCODE_SERVER_PASSWORD -H 'Content-Type: application/json' \
-d '{"parts":[{"type":"text","text":"say hi"}]}' \
"http://127.0.0.1:4096/session/<ses_id>/prompt_async?directory=$PWD"A real prompt needs a configured provider, so run the watch-and-trigger pattern against your real server, not the isolated sandbox. Event-type catalog, the 21 plugin hook points, and how to load a local plugin: references/events-hooks.md. Server start, auth, and routes: references/server-api.md.
Case C: the TUI
bash scripts/tui-smoke.sh --self-testThis launches the TUI under tmux in an isolated sandbox, confirms it renders (capture-pane), confirms send-keys reaches the composer, tears the tmux session down, and verifies the real DB session count is unchanged.
Honest verdict: tmux is fine for SMOKE (did it boot, render, accept a key) but fragile for asserting conversation output (the TUI is a 60fps full-screen app). For real behavior assertions use Case A (opencode run), Case B (server API + SSE), or the TUI control HTTP API (POST /tui/append-prompt, POST /tui/submit-prompt, POST /tui/execute-command). Details and the manual tmux recipe: references/tui-tmux.md.
Case D: investigate sessions in the DB
Read-only against the live SQLite DB. The session table is small (title and id lookups are instant); message text lives in the multi-GB part table, so text search must be scoped.
# by id
bash scripts/db-session-by-id.sh ses_3a4ee6335ffedFB8f76BPU1Eb3
# by title / name (newest first; second arg = limit)
bash scripts/db-session-by-name.sh "auth refactor" 20
# by message text - scope with --session, --recent N, or --since "<window>"
bash scripts/db-session-by-text.sh --session ses_3a4e... "ULTRAWORK"
bash scripts/db-session-by-text.sh --recent 50 "permission denied"
bash scripts/db-session-by-text.sh --since "7 days" --limit 50 "TODO"
# export an entire session as clean JSON
bash scripts/export-roundtrip.sh ses_3a4e... > session.jsonAd hoc queries: opencode db "<SQL>" --format json. Schema, tested query shapes with timings, the legacy message/part vs V2 session_message distinction, and the 25 GB caveat: references/db-investigation.md.
Scripts index
Run any script with --self-test to verify it against the live machine, or -h for usage. DB-read scripts are read-only; serve/sse/tui scripts use an isolated sandbox and clean up on exit.
| Script | Case | Self-test asserts |
|---|---|---|
scripts/lib/common.sh --self-check | - | deps present, DB path resolves, SQL escaping, free port, sandbox auto-removed |
scripts/db-session-by-id.sh | D | id round-trips for a real session |
scripts/db-session-by-name.sh | D | a derived title needle returns >=1 row |
scripts/db-session-by-text.sh | D | scoped search hits; unbounded scan refused; bounded search <30s |
scripts/export-roundtrip.sh | D | export stdout is valid JSON and .info.id round-trips |
scripts/server-smoke.sh | B | /global/health healthy, /doc >=100 paths, no-auth -> 401 |
scripts/sse-hook-probe.sh | B | /event opens and delivers server.connected |
scripts/tui-smoke.sh | C | TUI renders under tmux, tears down, real DB untouched |
Risks and caveats
- 25 GB part table: never run an unbounded text scan. Use
--session,
--recent, or --since. A naive JOIN ... WHERE session.time_created >= X scans oldest-first and can take ~50s; the scripts use an IN-subquery on the newest sessions (~20ms).
opencode exportwrites its banner to STDERR; pipe with2>/dev/nullbefore
jq or you will get a parse error.
- The server enforces auth only when
OPENCODE_SERVER_PASSWORDis set;
otherwise it runs unsecured. Authenticated calls use -u opencode:$PASS. Unauthenticated calls to a secured server return HTTP 401.
- Installed binary vs dev source: cite dev source paths for internals but
verify flags against the installed opencode <cmd> --help.
- Isolation: any QA that spawns opencode must use an isolated XDG sandbox so it
never pollutes the real DB. Prove it by comparing sqlite3 "$(opencode db path)" "SELECT count(*) FROM session" before and after.
- TUI output assertions are fragile; use the API for real assertions.
References
references/cli-commands.md- every QA-relevant opencode subcommand and flagreferences/db-investigation.md- DB schema, tested queries, the 25 GB caveatreferences/server-api.md- server start, auth, route catalog, /docreferences/events-hooks.md- SSE endpoints, event types, plugin hooksreferences/tui-tmux.md- tmux recipe, isolation, TUI control APIreferences/testing-harness.md- how opencode tests itself (bun test)references/sdk.md- the @opencode-ai/sdk client (reference only)references/docker-qa.md- run QA in a disposable Docker container (default; local is the fallback)
opencode CLI for QA (Case A: terminal works)
The installed binary is opencode (v1.15.13). From the source repo you can also run bun run --conditions=browser ./src/index.ts <cmd> inside packages/opencode. The canonical non-interactive QA entry is opencode run --format json.
Global flags
--print-logs
--log-level DEBUG|INFO|WARN|ERROR
--pure (run without external plugins)
-h, --help
-v, --versionopencode run (non-interactive QA core)
Positional message. Key flags:
-m, --model <provider/model>
--agent <name>
-s, --session <ses_...>
-c, --continue
--fork
--format default|json
-f, --file <path>
--title
--attach <url>
-p, --password
-u, --username
--dir
--variant
--thinking
-i, --interactive
--dangerously-skip-permissions
--command <slash-cmd>--format json emits NDJSON, one JSON object per line, each shaped like:
{"type":"...", "timestamp":<ms>, "sessionID":"ses_...", ...}type is one of: text, tool_use, step_start, step_finish, reasoning, error. The process exits when the session goes idle.
Validation rules:
-icannot combine with--commandor--format json--forkneeds-cor-s
Examples:
opencode run "list the files in src" --format jsonopencode run --command commitopencode run -c "continue the previous task"opencode run "explain auth" --attach http://127.0.0.1:4096 -p "$OPENCODE_SERVER_PASSWORD"opencode db (database tools)
opencode db pathPrints the active DB file path.
opencode db "<SQL>" --format jsonRuns a query and prints JSON rows. Use --format tsv (default) for TSV output.
opencode dbOpens an interactive sqlite3 shell.
opencode db migrateMigrates legacy JSON storage into SQLite.
Bundled scripts for session investigation:
scripts/db-session-by-id.shscripts/db-session-by-name.shscripts/db-session-by-text.shscripts/export-roundtrip.sh
Full detail in references/db-investigation.md.
opencode session
opencode session list --format jsonLists sessions as JSON.
opencode session delete <ses_id>Deletes one session.
opencode export [sessionID]
Prints "Exporting session: ..." to STDERR and a clean JSON document {info:{...}, messages:[...]} to STDOUT. Always redirect STDERR before piping to jq.
Example:
opencode export ses_3a4e... 2>/dev/null | jq '.info.id'Bundled wrapper:
scripts/export-roundtrip.sh <ses_id>opencode serve
Starts a headless HTTP server.
Flags:
--port (0 = pick 4096 then a free port)
--hostname (default 127.0.0.1)
--mdns
--mdns-domain
--corsOn start it prints:
opencode server listening on http://<host>:<port>Set OPENCODE_SERVER_PASSWORD to require auth. See references/server-api.md.
Bundled smoke test:
scripts/server-smoke.shopencode debug
Useful subcommands:
opencode debug paths # data/config/cache/state dirs
opencode debug info # version, OS, terminal, pluginsOthers: config, lsp, ripgrep, file, skill, snapshot, agent, v2, wait.
Other commands
opencode models [provider] --verbose
opencode stats
opencode providers list # alias: auth
opencode mcp list
opencode generate # prints the OpenAPI JSON specInstalled binary vs dev source (IMPORTANT note box)
The installed opencode (v1.15.13) matches the dev source in packages/opencode. When citing internals, cite dev source paths but always verify a flag against opencode <cmd> --help on the installed binary, since the dev branch can drift ahead.
For DB internals see references/db-investigation.md; for the HTTP server see references/server-api.md.
Investigating opencode sessions in the DB (Case D)
Table of Contents
- Where the data lives
- Access methods
- Schema (the tables that matter)
- Time conversion
- Tested query patterns
- The 25 GB caveat
- Verifying read-only
Where the data lives
Active DB path: opencode db path (on this machine ~/.local/share/opencode/opencode.db).
Derived from XDG data dir + "opencode" + "opencode.db" (or "opencode-<channel>.db" on non-stable channels). Override via env OPENCODE_DB (:memory:, absolute, or relative-to-data).
It is large (tens of GB) because the part table stores tool output. The session table is small (~21k rows; full scans are milliseconds).
Access methods
Preferred: opencode db "<SQL>" --format json (WAL-safe; resolves the active DB). --format tsv default. Bare opencode db opens an interactive sqlite3 shell. opencode db path prints the file.
Raw fallback for EXPLAIN/perf: sqlite3 "$(opencode db path)" "<SQL>". Reads are safe alongside a running opencode (WAL allows concurrent readers).
Schema (the tables that matter)
Note the ACTIVE storage in v1.15.13 is the LEGACY pair message + part; the V2 session_message table exists but is EMPTY in this version (a recent session showed 43 message rows, 169 part rows, 0 session_message). Document both but make clear message/part is what holds current data.
session
| Column | Notes |
|---|---|
| id | PK, 'ses_' prefix |
| project_id | FK |
| parent_id | |
| slug | |
| directory | |
| title | NOT NULL |
| version | |
| agent | |
| model | JSON {providerID, modelID} |
| cost | |
| tokens_input | |
| tokens_output | |
| tokens_reasoning | |
| tokens_cache_read | |
| tokens_cache_write | |
| metadata | JSON |
| time_created | epoch MILLISECONDS |
| time_updated | epoch MILLISECONDS |
| time_archived |
Indexes: project_id, parent_id, workspace_id. NO index on title or time_created.
message (legacy)
| Column | Notes |
|---|---|
| id | 'msg_' prefix |
| session_id | FK -> session, cascade |
| time_created | |
| time_updated | |
| data | JSON: {role, time:{created}, summary:{title}, agent, model:{providerID,modelID}, variant} |
part (legacy)
| Column | Notes |
|---|---|
| id | 'prt_' prefix |
| message_id | FK -> message, cascade |
| session_id | denormalized; index part_session_idx |
| data | JSON |
Part types seen: text, reasoning, tool, step-start, step-finish. A text part is {"type":"text","text":"..."}.
Other tables
session_message (V2, currently empty), todo, project, permission, session_share, workspace, event.
Time conversion
time_created/time_updated are epoch milliseconds. Convert:
datetime(time_created/1000,'unixepoch')Tested query patterns
1. By id (instant)
Script: scripts/db-session-by-id.sh <ses_id>
SELECT
id,
slug,
title,
directory,
agent,
json_extract(model,'$.modelID') AS model,
json_extract(model,'$.providerID') AS provider,
cost,
tokens_input,
tokens_output,
datetime(time_created/1000,'unixepoch') AS created,
datetime(time_updated/1000,'unixepoch') AS updated
FROM session
WHERE id='<ses_id>'2. By name/title (0.006s over 21k rows)
Script: scripts/db-session-by-name.sh "<substr>" [limit]
SELECT
id,
title,
datetime(time_created/1000,'unixepoch') AS created
FROM session
WHERE title LIKE '%<substr>%'
ORDER BY time_created DESC
LIMIT <N>3. By message text
Script: scripts/db-session-by-text.sh (--session <id>|--recent <N>|--since "<window>") [--limit N] "<text>"
CRITICAL performance note: text lives in part.data JSON, and part is the multi-GB table, so an UNBOUNDED text scan is refused by the script. Always scope it.
Scoped within one session (indexed, ~0.017s)
SELECT
p.session_id,
p.id,
substr(json_extract(p.data,'$.text'),1,120)
FROM part p
WHERE p.session_id='<id>'
AND json_extract(p.data,'$.type')='text'
AND json_extract(p.data,'$.text') LIKE '%<text>%'
LIMIT 50Bounded to the N most-recent sessions (worst-case ~0.02s)
SELECT
p.session_id,
p.id,
substr(json_extract(p.data,'$.text'),1,120)
FROM part p
WHERE p.session_id IN (
SELECT id FROM session ORDER BY time_created DESC LIMIT <N>
)
AND json_extract(p.data,'$.type')='text'
AND json_extract(p.data,'$.text') LIKE '%<text>%'
LIMIT 50AVOID this naive form (took ~50s)
A JOIN FROM session s JOIN part p ON p.session_id=s.id WHERE s.time_created >= X ... scans oldest sessions first. The IN-subquery (newest-first, drives part_session_idx) is the right shape because it lets SQLite use the index on part.session_id with a small, ordered set of recent session IDs, rather than scanning the entire part table from the oldest sessions upward.
4. Full export
Script: scripts/export-roundtrip.sh <ses_id> wraps opencode export <id> 2>/dev/null -> clean JSON {info:{id,slug,projectID,directory,title,tokens,time,...}, messages:[...]} (banner goes to stderr).
5. Listing recent sessions
SELECT
id,
title,
datetime(time_created/1000,'unixepoch') created
FROM session
ORDER BY time_created DESC
LIMIT 100The 25 GB caveat
Global text search over all parts is a full scan of the largest table and can take a long time. The bundled script refuses it; you must pass --session, --recent, or --since. Title search (session table) is always cheap.
Verifying read-only
All Case D operations are reads. To prove a QA pass did not mutate the DB, compare before and after:
sqlite3 "$(opencode db path)" "SELECT count(*) FROM session"These queries are exactly what the scripts/db-*.sh helpers run; each ships a --self-test.
Docker QA (default path)
Run opencode QA inside a DISPOSABLE container so the host is never touched and you always test against the latest opencode. The container itself is the sandbox: latest released opencode + codex are baked in, a COPY of your local config is loaded, and the container is removed on exit (docker run --rm). This is the DEFAULT; fall back to running the scripts locally (see SKILL.md) only when Docker is unavailable or on Windows.
Use it
qa-docker.sh brings up a disposable box (builds omo-dev then omo-qa on first use, reused after) and either drops you into it or serves opencode to your host. From the repo root:
# a shell inside the box: just type `opencode ...` or `codex ...`
script/agent/qa-docker.sh
script/agent/qa-docker.sh shell
# serve opencode's HTTP API to the host, then drive it from OUTSIDE:
script/agent/qa-docker.sh serve 4096 # terminal 1 (Ctrl-C stops + removes)
curl http://127.0.0.1:4096/global/health # host -> {"healthy":true,"version":"1.17.7"}
opencode run "hi" --attach http://127.0.0.1:4096 # a real turn against the box
# one-off command, or a skill's own self-test, inside:
script/agent/qa-docker.sh exec opencode --version
script/agent/qa-docker.sh exec bash .agents/skills/opencode-qa/scripts/server-smoke.sh --self-test
script/agent/qa-docker.sh --no-config exec opencode --version # skip the config copy
script/agent/qa-docker.sh --clean # remove the QA imagesomo-qa is omo-dev (.devcontainer/Dockerfile) plus the latest opencode-ai and @openai/codex npm packages and sqlite3 jq curl rsync. Pin with --build-arg OMO_OPENCODE_VERSION=... on the qa.Dockerfile for a specific release.
Why the container is the sandbox
The local scripts isolate by pointing XDG_* at temp dirs so they never pollute the real ~/.local/share/opencode/opencode.db. In Docker the whole container is throwaway, so isolation is structural: your host DB and config are never written. qa-docker.sh mounts ~/.config/opencode (and ~/.codex) READ-ONLY at /mnt/host/*; the entrypoint copies them into the container's writable home (heavy caches excluded) so QA runs against a COPY.
Credentials
Secrets are never baked into the image. Provide them at run time only:
- a gitignored
.envor.env.localat the repo root (auto-sourced by
script/agent/setup.sh and script/agent/qa-sandbox.sh),
- GitHub Codespaces secrets, or
- the devcontainer
remoteEnvpassthrough.
The host config is mounted read-only, so auth that already lives in ~/.config/opencode rides along without copying secrets into any image layer.
Fish caveat: if your shell sets OPENCODE_CONFIG_DIR (for example a profiles/today override), export it before calling qa-docker.sh so the container resolves the same profile (the runner forwards it with -e).
Fallback: local / Windows
qa-docker.sh exits 3 with guidance when Docker is unavailable or on Windows. There, run the scripts directly on the host (the rest of this skill); they isolate via temp XDG_*. Windows has no Docker QA path here by design.
Cleanup
Each run auto-removes its container (--rm). The omo-dev / omo-qa images persist for fast re-runs; drop them with script/agent/qa-docker.sh --clean.
QAing opencode hooks, actions, and events (Case B)
opencode publishes lifecycle events over Server-Sent Events. Plugins observe the SAME events via the event hook, so confirming an event on the wire proves a hook would fire. The bundled probe is scripts/sse-hook-probe.sh.
Table of Contents
- The two SSE endpoints
- Watch the stream
- Important event types
- Hook-fired recipe (watch + trigger + assert)
- Plugin hooks (the 21 hook points a plugin can implement)
- Loading a local plugin for QA
The two SSE endpoints
- GET /event?directory=<dir> - per-instance stream; the FIRST event is
server.connected, aserver.heartbeatarrives every 10s, and the stream ends onserver.instance.disposed. - GET /global/event - all events, no instance filter.
- Frames look like
data: {"type":"...","properties":{...}}(one per line). Consume withcurl -N.
Watch the stream
curl -N -u opencode:$PASS "http://127.0.0.1:4096/event?directory=$PWD"Bundled, with assertions + auto-teardown:
scripts/sse-hook-probe.sh --self-test(spawns an isolated server, asserts server.connected)
scripts/sse-hook-probe.sh --attach http://127.0.0.1:4096 --password "$PASS" --directory "$PWD" --event message.part.updated --timeout 30(watch your real server for a specific event)
Important event types (type - properties)
session.created/session.updated/session.deleted(sessionID, info)message.updated(sessionID, info)message.removed(sessionID, messageID)message.part.updated(sessionID, part, time)message.part.delta(sessionID, messageID, partID, field, delta)message.part.removedpermission.asked(id, sessionID, permission, tool?)permission.repliedsession.error(sessionID?, error)session.diff(sessionID, diff)question.asked/question.replied/question.rejectedfile.watcher.updated(file, event)project.updatedlsp.updatedpty.created/pty.updated/pty.exited/pty.deletedserver.connectedserver.heartbeatserver.instance.disposedglobal.disposedplugin.added
Hook-fired recipe (watch + trigger + assert)
Two-shell pattern (or use the script):
# shell 1: watch (kill with Ctrl-C when done)
curl -N -u opencode:$PASS "http://127.0.0.1:4096/event?directory=$PWD" \
| grep --line-buffered '"type":"message.part.updated"'
# shell 2: trigger an action (fire-and-forget)
curl -X POST -u opencode:$PASS -H 'Content-Type: application/json' \
-d '{"parts":[{"type":"text","text":"say hi"}]}' \
"http://127.0.0.1:4096/session/<ses_id>/prompt_async?directory=$PWD"A message.part.updated (text/tool) confirms the prompt action drove the model and any tool/permission hook path. Note: a real prompt requires a configured provider/auth, so this runs against your real server, not the isolated sandbox (the sandbox only proves the SSE plumbing via server.connected).
Plugin hooks (the 21 hook points a plugin can implement)
event, config, tool, auth, provider, chat.message, chat.params, chat.headers, permission.ask, command.execute.before, tool.execute.before, tool.execute.after, tool.definition, shell.env, experimental.chat.messages.transform, experimental.chat.system.transform, experimental.session.compacting, experimental.compaction.autocontinue, experimental.text.complete.
- A plugin is a module default-exporting
{ id?, server: (input, options) => Promise<Hooks> }. - Minimal example implementing
eventandtool.execute.beforethat console.log the activity:
export default {
id: "qa-logger",
async server(input, options) {
return {
event: async (event) => {
console.log("[event]", event.type, event.properties);
},
"tool.execute.before": async (tool, args, context) => {
console.log("[tool.before]", tool.name, args);
},
};
},
};Loading a local plugin for QA
- Add an absolute path or npm spec to the opencode config
plugin/plugin_originsarray (project.opencode/config or user config), then restart opencode. On load it emitsplugin.added. - To QA a hook: load the plugin, watch /event (or the plugin's own logging), trigger the relevant action, and assert.
---
Pair this with references/server-api.md (how to start the server, auth, prompt routes).
opencode SDK (@opencode-ai/sdk) - reference only
A TypeScript/Bun way to drive opencode for QA. Prefer the tested CLI/curl scripts for portability; reach for the SDK when you want typed access from a Bun script.
IMPORTANT: method signatures differ between SDK versions and between the published docs and the generated client. ALWAYS check the installed version's types (node_modules/@opencode-ai/sdk) before relying on a signature, and verify against GET /doc (the OpenAPI spec the SDK is generated from).Entry points and exports
Package @opencode-ai/sdk subpath exports:
.(src/index.ts)./client./server./v2(src/v2/index.ts)./v2/client./v2/server./v2/gen/client
Root and v2 entries export createOpencode(), createOpencodeClient(...), createOpencodeServer(...).
createOpencodeServer() spawns opencode serve ... and waits for the startup line. createOpencodeClient({ baseUrl }) wraps the generated client, rewrites directory/workspace headers, installs error interception.
Two ways to connect
import { createOpencodeClient, createOpencodeServer } from "@opencode-ai/sdk/v2"
// A) embedded server (spawns opencode serve)
const server = await createOpencodeServer()
const client = createOpencodeClient({ baseUrl: server.url })
// ... use client ...
server.close()
// B) connect to an already-running server
const client2 = createOpencodeClient({ baseUrl: "http://127.0.0.1:4096" })Client namespaces
Top-level on OpencodeClient:
auth, app, global, event, config, experimental, tool, worktree, find, file, instance, path, vcs, command, lsp, formatter, mcp, project, pty, question, permission, provider, session, part, sync, v2, tui.
Useful methods (shapes vary by version)
client.global.health(),client.global.event()client.app.log(...),client.app.agents(...),client.app.skills(...)client.config.get(),client.config.providers()client.event.subscribe()- SSE on /event; iteratefor await (const event of events.stream) { event.type, event.properties }client.session(legacy surface): list, create, status, get, update, delete, children, todo, diff, messages, message, deleteMessage, prompt, promptAsync, command, shell, fork, abort, init, share, unshare, summarize, revert, unrevertclient.v2.session(newer read/stream surface): list, prompt, compact, wait, context, messagesclient.part.delete(...),client.part.update(...)
Minimal QA snippet
Arg shapes may differ by version. Treat this as a starting point, not a contract.
import { createOpencodeClient, createOpencodeServer } from "@opencode-ai/sdk/v2"
const server = await createOpencodeServer()
const client = createOpencodeClient({ baseUrl: server.url })
try {
const session = await client.session.create({ title: "QA session" })
await client.session.promptAsync({
sessionID: session.id,
parts: [{ type: "text", text: "Say hello in one line." }],
})
const sessions = await client.session.list({ limit: 10 })
console.log(sessions[0]?.title)
const messages = await client.v2.session.messages({
sessionID: session.id,
limit: 20,
})
console.log(messages.items.length)
} finally {
server.close()
}Key types
- Legacy Session: id, slug, projectID, directory, title, version, time.created/updated, optional workspaceID, path, parentID, summary, cost, tokens, share, agent, model, metadata, permission, revert.
- Message = UserMessage | AssistantMessage (role "user" | "assistant"; assistant adds time.completed?, modelID, providerID, agent, tokens, finish?, error?).
- Part union: TextPart, ReasoningPart, FilePart, ToolPart, StepStartPart, StepFinishPart, SnapshotPart, PatchPart, AgentPart, RetryPart, CompactionPart, SubtaskPart.
How it is generated
packages/sdk/js/script/build.ts runs bun dev generate > openapi.json from the opencode repo, feeds it to @hey-api/openapi-ts.createClient, writes output to packages/sdk/js/src/v2/gen, patches an SSE generic, prettifies and typechecks. Regenerate with ./packages/sdk/js/script/build.ts.
---
For version-stable QA, prefer the curl/CLI scripts in this skill; cross-check any SDK call against GET /doc.
opencode HTTP server API for QA (Case B)
Table of Contents
- Start a server
- Authentication
- Per-request workspace routing
- Introspect the API
- Tested smoke calls
- Route catalog
- Triggering a prompt over HTTP
Start a server
Run the server with a fixed port and host:
opencode serve --port 4096 --hostname 127.0.0.1Output:
opencode server listening on http://127.0.0.1:4096Port 0 means the server will pick 4096, then fall back to a free port if that one is taken.
A bundled isolated smoke test is available at scripts/server-smoke.sh. It spawns an isolated server, checks /global/health, checks that /doc returns at least 100 paths, and confirms that no-auth requests get 401, then tears the server down.
Authentication
Set the environment variable OPENCODE_SERVER_PASSWORD to require authentication. If it is unset, the server runs UNSECURED and prints a warning.
The username defaults to opencode. Override it with OPENCODE_SERVER_USERNAME.
Two ways to authenticate:
1. HTTP Basic Auth: -u opencode:$PASS 2. Query parameter: ?auth_token=<base64(user:pass)>
Unauthenticated requests to protected routes return HTTP 401. This was verified.
Per-request workspace routing
Most instance routes need the target project directory. Pass it as either:
- Query parameter:
?directory=$PWD - Header:
x-opencode-directory: $PWD
Aliases also work: x-opencode-workspace header or ?workspace= query parameter.
The server resolves an instance per request, so a single serve process can handle many projects.
Introspect the API
The /doc endpoint returns the full OpenAPI spec. To list all documented paths:
curl -s -u opencode:$PASS http://127.0.0.1:4096/doc | jq '.paths | keys'On v1.15.13 this returned 113 paths. This is the source of truth for exact request and response schemas.
Tested smoke calls
curl -s -u opencode:$PASS http://127.0.0.1:4096/global/health | jq .
# {"healthy":true,"version":"1.15.13"}
curl -s -u opencode:$PASS http://127.0.0.1:4096/doc | jq '.paths|length'
# 113
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:4096/session?directory=$PWD
# 401 (no credentials)
curl -s -u opencode:$PASS "http://127.0.0.1:4096/session?directory=$PWD" | jq 'length'Route catalog
This mirrors the structure returned by /doc. Each entry is grouped as method path - purpose.
Global
GET /global/health- health checkGET /global/event- server-wide SSE event streamGET /global/config- read global configurationPATCH /global/config- update global configurationPOST /global/dispose- dispose the server instanceGET /doc- OpenAPI specification
Session
GET /session- list sessionsGET /session/status- session status overviewGET /session/:id- get session by IDGET /session/:id/children- list child sessionsGET /session/:id/todo- get session todo itemsGET /session/:id/diff- get session diffGET /session/:id/message- list session messagesGET /session/:id/message/:messageID- get a specific messagePOST /session- create a new sessionDELETE /session/:id- delete a sessionPATCH /session/:id- update a sessionPOST /session/:id/fork- fork a sessionPOST /session/:id/abort- abort a sessionPOST /session/:id/init- initialize a sessionPOST /session/:id/share- share a sessionPOST /session/:id/summarize- summarize a sessionPOST /session/:id/revert- revert a sessionPOST /session/:id/unrevert- unrevert a sessionDELETE /session/:id/share- unshare a session
Prompting
POST /session/:id/message- send a prompt; streams JSONPOST /session/:id/prompt_async- fire-and-forget prompt; returns 204POST /session/:id/command- execute a command in a sessionPOST /session/:id/shell- run a shell command in a session
Files and find
GET /find- text search via ripgrepGET /find/file- file searchGET /find/symbol- symbol searchGET /file- file metadataGET /file/content- file contentsGET /file/status- file status
Instance and app
GET /path- path resolutionGET /vcs- version control infoGET /vcs/status- VCS statusGET /vcs/diff- VCS diffGET /command- available commandsGET /agent- available agentsGET /skill- available skillsGET /lsp- LSP infoGET /formatter- formatter info
Permission and question
GET /permission- list pending permission requestsPOST /permission/:requestID/reply- reply to a permission requestGET /question- list pending questionsPOST /question/:requestID/reply- reply to a questionPOST /question/:requestID/reject- reject a question
TUI control
These endpoints drive a running TUI over HTTP.
POST /tui/append-prompt- append text to the TUI promptPOST /tui/submit-prompt- submit the current TUI promptPOST /tui/execute-command- execute a TUI commandPOST /tui/show-toast- show a toast in the TUIGET /tui/control/next- get the next TUI control eventPOST /tui/control/response- respond to a TUI control event
PTY
GET /pty- list PTY sessionsPOST /pty- create a PTY sessionGET /pty/:id- get PTY session infoDELETE /pty/:id- delete a PTY sessionPOST /pty/:id/connect-token- generate a PTY connect tokenGET /pty/:id/connect- WebSocket connection to the PTY
Event
GET /event- instance-level SSE event stream
V2 API
GET /api/session- list sessions (v2)POST /api/session/:id/prompt- prompt a session (v2)POST /api/session/:id/compact- compact a session (v2)POST /api/session/:id/wait- wait for a session (v2)GET /api/session/:id/context- get session context (v2)GET /api/session/:id/message- get session messages (v2)GET /api/model- list models (v2)GET /api/provider- list providers (v2)
Triggering a prompt over HTTP (for hook and event QA)
Use prompt_async so the event stream is not blocked.
curl -X POST -u opencode:$PASS -H 'Content-Type: application/json' \
-d '{"parts":[{"type":"text","text":"hello"}]}' \
"http://127.0.0.1:4096/session/<ses_id>/prompt_async?directory=$PWD"This returns HTTP 204. Watching events is covered in references/events-hooks.md.
---
Schemas are authoritative in GET /doc; for the event stream see references/events-hooks.md.
opencode Test Harness (how opencode QAs itself)
This is reference material for writing and running tests against the opencode source. The skill's own QA scripts (CLI, curl, sqlite) do not require this, but it is the authoritative pattern when you need a unit or integration test.
Table of Contents
1. Runner and the root guard 2. Test bootstrap (in-memory, isolated) 3. Effect-based harness (test/lib/effect.ts) 4. Instance and tmpdir fixtures (test/fixture/fixture.ts) 5. CLI subprocess harness (test/lib/cli-process.ts) 6. Fake LLM server (test/lib/llm-server.ts) 7. Representative test shapes 8. App e2e (Playwright) 9. Test style conventions
Runner and the root guard
The runner is bun test (Bun built-in, not vitest or jest).
Tests cannot run from the repo root. Two guards enforce this:
bunfig.tomlat repo root setsroot = "./do-not-run-tests-from-root"- Root
package.jsonhas"test": "echo 'do not run tests from root' && exit 1"
Run from a package directory instead:
cd packages/opencode && bun test --timeout 30000Run a single file:
bun test test/tool/read.test.tsFilter by test name:
bun test --grep "truncates large file"CI variant:
bun run test:ciTurbo dependency: opencode#test depends on ^build.
Test bootstrap (in-memory, isolated)
The preload file is packages/opencode/test/preload.ts. It is wired via packages/opencode/bunfig.toml:
[test]
preload = ["@opentui/solid/preload", "./test/preload.ts"]What it does:
- Sets
XDG_DATA_HOME,XDG_CACHE_HOME,XDG_CONFIG_HOME, andXDG_STATE_HOMEto temp directories - Sets
OPENCODE_TEST_HOME - Sets
OPENCODE_DB=":memory:"(SQLite in-memory) - Wipes all provider API keys from
process.env - Sets
OPENCODE_EXPERIMENTAL_EVENT_SYSTEM=true - Sets
OPENCODE_EXPERIMENTAL_WORKSPACES=true - Initializes
Log.init({ print: false }) - Calls
initProjectors()
Effect-based harness (test/lib/effect.ts)
The it factory wraps bun:test with three variants:
it.effect(name, body)... TestClock + TestConsole (isolated time)it.live(name, body)... real clock + TestConsoleit.instance(name, body, opts)... real clock + scoped tmpdir + a real Instance context
testEffect(layer) builds an it bound to an Effect layer:
const it = testEffect(Layer.mergeAll(readLayer(), testInstanceStoreLayer))Instance and tmpdir fixtures (test/fixture/fixture.ts)
tmpdirScoped(options?)... scoped temp directory. Optionalgit: true, optionalconfig(writesopencode.json), optionalinit.provideInstance(directory)(effect)... runs an Effect inside a real instance for that directory.withTmpdirInstance({ git?, config?, init? })(effect)... one-liner: make tmpdir, optional git init + config, provide instance.testInstanceStoreLayer... instance store with a no-op bootstrap.
CLI subprocess harness (test/lib/cli-process.ts)
cliIt.live(name, body, timeoutMs?) and cliIt.concurrent(...) spawn the real CLI (bun run --conditions=browser src/index.ts) in an isolated environment.
Exposed helpers:
opencode.run()opencode.serve()opencode.acp()expectExitparseJsonEvents
Isolation environment keys:
OPENCODE_TEST_HOMEOPENCODE_CONFIG_CONTENT(inline provider config)OPENCODE_DISABLE_PROJECT_CONFIG=1OPENCODE_PURE=1OPENCODE_DISABLE_AUTOUPDATE=1OPENCODE_DISABLE_AUTOCOMPACT=1OPENCODE_DISABLE_MODELS_FETCH=1
Real example from packages/opencode/test/cli/serve/serve-process.test.ts:
cliIt.live("spawns serve and health responds", async ({ opencode, expectExit }) => {
const server = await opencode.serve()
expect(server.port).toBeGreaterThan(0)
const res = await fetch(`${server.url}/global/health`)
expect(res.status).toBe(200)
})Fake LLM server (test/lib/llm-server.ts)
TestLLMServer is an in-process OpenAI-compatible SSE server to mock model responses deterministically.
Methods:
llm.text("hello")llm.tool("read", { filePath: "x" })llm.pushMatch(matchFn, reply)
This is how tests avoid real provider calls.
Representative test shapes
1. Tool test
From packages/opencode/test/tool/read.test.ts:
const it = testEffect(Layer.mergeAll(readLayer(), testInstanceStoreLayer))
it.instance("truncates large file over maxReadFileSize", () =>
Effect.gen(function* () {
const test = yield* TestInstance
// ... exercise the read tool, assert truncation
})
)2. Session/event test
From packages/opencode/test/session/session.test.ts:
test("session.created fires after session.create", async () => {
const deferred = Deferred.unsafeMake<void>(FiberId.none)
// ... listen for session.created event
await session.create({})
// ... assert deferred resolves
})3. Plain unit test
From packages/opencode/test/cli/run/runtime.boot.test.ts:
import { describe, expect, mock, spyOn, test } from "bun:test"
test("boots runtime without errors", () => {
// ... standard assertions, no Effect
})App e2e (Playwright)
The app lives in packages/app (SolidJS). Config is packages/app/playwright.config.ts. It starts the Vite dev server via webServer; the backend is expected at localhost:4096.
Commands (run from packages/app):
bunx playwright install chromium
bun run test:e2e:localFilter with grep:
bun run test:e2e:local -- --grep "settings"UI mode:
bun run test:e2e:uiApp unit tests:
bun run test:unitThis equals bun test --preload ./happydom.ts ./src.
Test style conventions
Per opencode AGENTS.md:
- Avoid mocks where possible. Test the real implementation. Do not duplicate logic into tests.
- Run
bun typecheckfrom the package directory (uses tsgo). Never run baretsc.
For runtime or scriptable QA without writing tests, use the opencode-qa scripts (Cases A-D in SKILL.md).
QAing the opencode TUI under tmux (Case C)
Verdict first (be honest)
- tmux CAN launch the opencode TUI and
capture-panereads the rendered frame;send-keysdelivers keystrokes to the composer. This is proven and good for SMOKE checks: did it boot, does it render, does it accept input. - The TUI is a 60fps full-screen app (built on @opentui/solid) with a custom renderer, animations, and a worker thread. Asserting on conversation OUTPUT by scraping the frame is FRAGILE and not recommended.
- For real behavior assertions prefer:
opencode run(Case A, references/cli-commands.md), the server API + SSE (Case B, references/server-api.md + events-hooks.md), or the TUI control HTTP API (below). The TUI talks to the same server, so API-level QA is equivalent to driving the screen.
Safety: isolate so QA never pollutes the real DB
Launching the real TUI would create sessions in the real ~/.local/share/opencode DB. Run it under an isolated XDG sandbox. The bundled scripts/tui-smoke.sh does exactly this and verifies the real session count is unchanged before/after.
Smoke test (bundled)
scripts/tui-smoke.sh --self-testlaunches the TUI under tmux in an isolated sandbox, polls capture-pane for a render marker (version string / "Ask anything" / footer), sends a sentinel keystroke, then kills the tmux session and confirms the real DB is untouched.
Manual tmux recipe (fenced) - for ad hoc smoke
SESS=oqa_tui_demo
DIR=$(mktemp -d)
tmux new-session -d -s "$SESS" -x 200 -y 50
# isolate XDG so no real session is written
tmux send-keys -t "$SESS" "XDG_DATA_HOME=$DIR/data XDG_CONFIG_HOME=$DIR/cfg XDG_STATE_HOME=$DIR/state XDG_CACHE_HOME=$DIR/cache OPENCODE_DISABLE_AUTOUPDATE=1 OPENCODE_DISABLE_MODELS_FETCH=1 opencode $DIR" Enter
sleep 7
tmux capture-pane -t "$SESS" -p | sed -n '1,30p' # inspect the rendered frame
tmux send-keys -t "$SESS" "hello" # type into the composer
sleep 1
tmux capture-pane -t "$SESS" -p | sed -n '1,30p'
tmux kill-session -t "$SESS" # teardown (kills the TUI)
rm -rf "$DIR"Explain: capture-pane -p prints the visible frame; send-keys injects input; kill-session tears down the process tree. Always teardown and remove the temp dir.
The reliable alternative: TUI control HTTP API
A running TUI is a client of the local server, so you can drive it over HTTP without scraping the screen:
- POST /tui/append-prompt - append text to the composer
- POST /tui/submit-prompt - submit the composer
- POST /tui/execute-command - run a TUI command
- POST /tui/show-toast - show a toast
- GET /tui/control/next + POST /tui/control/response - the control channel
Use these (with auth + ?directory=) to deterministically drive a TUI you launched, then assert via the event stream (references/events-hooks.md).
Headless component testing (for source-level TUI tests)
opencode unit-tests TUI components headlessly with @opentui/core/testing createTestRenderer (see packages/opencode/test/cli/tui/, e.g. app-lifecycle.test.ts). This is the route for asserting TUI component behavior in the source repo; see references/testing-harness.md.
Bottom line: tmux for smoke, server/SSE or /tui/* control for assertions, createTestRenderer for source unit tests.
#!/usr/bin/env bash
# db-session-by-id.sh - investigate an opencode session by its id (ses_...).
# Read-only against the LIVE opencode DB via `opencode db ... --format json`.
#
# Usage:
# db-session-by-id.sh ses_3a4ee6335ffedFB8f76BPU1Eb3
# db-session-by-id.sh --self-test
#
# Output: a JSON array with one row (id, slug, title, directory, agent, model,
# cost, token counts, human-readable created/updated times) or [] if not found.
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
. "$SCRIPT_DIR/lib/common.sh"
oqa_session_by_id() {
local id esc
id="$1"
esc="$(oqa_sql_escape "$id")"
oqa_db_query "SELECT
id, slug, title, directory, agent,
json_extract(model,'\$.modelID') AS model,
json_extract(model,'\$.providerID') AS provider,
cost, tokens_input, tokens_output,
datetime(time_created/1000,'unixepoch') AS created,
datetime(time_updated/1000,'unixepoch') AS updated
FROM session WHERE id='$esc'"
}
oqa_self_test() {
oqa_require opencode jq || return 1
local id out got
id="$(oqa_db_query "SELECT id FROM session ORDER BY time_created DESC LIMIT 1" | jq -r '.[0].id // empty')"
if [ -z "$id" ]; then
oqa_log "FAIL: no sessions in DB to test against"; return 1
fi
out="$(oqa_session_by_id "$id")"
got="$(printf '%s' "$out" | jq -r '.[0].id // empty')"
if [ "$got" = "$id" ]; then
oqa_pass "db-session-by-id round-trips id ($id)"
return 0
fi
oqa_log "FAIL: expected id '$id', got '$got'"; return 1
}
case "${1:-}" in
--self-test) oqa_self_test; exit $? ;;
-h|--help|"")
sed -n '2,12p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'
[ -z "${1:-}" ] && exit 2 || exit 0 ;;
*)
oqa_require opencode jq || exit 1
oqa_session_by_id "$1" ;;
esac
#!/usr/bin/env bash
# db-session-by-name.sh - find opencode sessions whose TITLE matches a substring.
# Read-only against the LIVE opencode DB. Title search is cheap (the session
# table is small; ~21k rows scan in milliseconds), so no bounding is needed.
#
# Usage:
# db-session-by-name.sh "auth refactor" # newest 20 matches
# db-session-by-name.sh "auth refactor" 50 # newest 50 matches
# db-session-by-name.sh --self-test
#
# Output: JSON array of {id, title, created, updated} ordered newest first.
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
. "$SCRIPT_DIR/lib/common.sh"
oqa_session_by_name() {
local needle limit esc
needle="$1"
limit="${2:-20}"
case "$limit" in (*[!0-9]*|"") limit=20 ;; esac
esc="$(oqa_sql_escape "$needle")"
oqa_db_query "SELECT
id, title,
datetime(time_created/1000,'unixepoch') AS created,
datetime(time_updated/1000,'unixepoch') AS updated
FROM session
WHERE title LIKE '%$esc%'
ORDER BY time_created DESC
LIMIT $limit"
}
oqa_self_test() {
oqa_require opencode jq || return 1
# Derive a guaranteed-present needle: the first 5 chars of a real title.
local needle out n
needle="$(oqa_db_query "SELECT substr(title,1,5) AS t FROM session WHERE length(title)>=5 ORDER BY time_created DESC LIMIT 1" | jq -r '.[0].t // empty')"
if [ -z "$needle" ]; then
oqa_log "FAIL: could not derive a title needle"; return 1
fi
out="$(oqa_session_by_name "$needle" 5)"
n="$(printf '%s' "$out" | jq 'length')"
if [ "${n:-0}" -ge 1 ]; then
oqa_pass "db-session-by-name found $n row(s) for needle '$needle'"
return 0
fi
oqa_log "FAIL: expected >=1 row for needle '$needle', got '$n'"; return 1
}
case "${1:-}" in
--self-test) oqa_self_test; exit $? ;;
-h|--help|"")
sed -n '2,14p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'
[ -z "${1:-}" ] && exit 2 || exit 0 ;;
*)
oqa_require opencode jq || exit 1
oqa_session_by_name "$1" "${2:-20}" ;;
esac
#!/usr/bin/env bash
# db-session-by-text.sh - find opencode message TEXT by content.
#
# Message text lives inside the `part` table as JSON blobs
# (json_extract(data,'$.text') for text parts). The `part` table holds the bulk
# of the DB (tens of GB of tool output), so an UNBOUNDED text scan is refused.
# You MUST scope the search to a bounded, recent set of sessions:
# --session ses_... one session (indexed, instant)
# --recent N the N most-recent sessions (default 25)
# --since "<window>" sessions created within a window (e.g. "7 days"),
# capped at the 200 most-recent in that window
#
# All bounded modes use `part.session_id IN (SELECT id FROM session ORDER BY
# time_created DESC LIMIT ...)`, which drives the part_session_idx on exactly
# the newest sessions. (A naive JOIN with `WHERE session.time_created >= X`
# scans oldest-first and can take ~50s; the IN-subquery form returns in ~20ms.)
#
# Usage:
# db-session-by-text.sh --session ses_3a4e... "ULTRAWORK"
# db-session-by-text.sh --recent 50 "permission denied"
# db-session-by-text.sh --since "7 days" --limit 50 "TODO"
# db-session-by-text.sh --self-test
#
# Output: JSON array of {session_id, part_id, snippet} (snippet = first 120
# chars of the matching text part).
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
. "$SCRIPT_DIR/lib/common.sh"
oqa_text_scoped() {
local sid needle limit esc_id esc_txt
sid="$1"; needle="$2"; limit="${3:-50}"
esc_id="$(oqa_sql_escape "$sid")"
esc_txt="$(oqa_sql_escape "$needle")"
oqa_db_query "SELECT
p.session_id AS session_id,
p.id AS part_id,
substr(json_extract(p.data,'\$.text'),1,120) AS snippet
FROM part p
WHERE p.session_id='$esc_id'
AND json_extract(p.data,'\$.type')='text'
AND json_extract(p.data,'\$.text') LIKE '%$esc_txt%'
LIMIT $limit"
}
oqa_text_recent() {
local n needle limit esc_txt
n="$1"; needle="$2"; limit="${3:-50}"
case "$n" in (*[!0-9]*|"") n=25 ;; esac
esc_txt="$(oqa_sql_escape "$needle")"
oqa_db_query "SELECT
p.session_id AS session_id,
p.id AS part_id,
substr(json_extract(p.data,'\$.text'),1,120) AS snippet
FROM part p
WHERE p.session_id IN (SELECT id FROM session ORDER BY time_created DESC LIMIT $n)
AND json_extract(p.data,'\$.type')='text'
AND json_extract(p.data,'\$.text') LIKE '%$esc_txt%'
LIMIT $limit"
}
oqa_text_since() {
local window needle limit esc_win esc_txt
window="$1"; needle="$2"; limit="${3:-50}"
esc_win="$(oqa_sql_escape "$window")"
esc_txt="$(oqa_sql_escape "$needle")"
# Cap at the 200 most-recent sessions inside the window and drive
# part_session_idx via an IN-subquery (newest-first) to stay fast.
oqa_db_query "SELECT
p.session_id AS session_id,
p.id AS part_id,
substr(json_extract(p.data,'\$.text'),1,120) AS snippet
FROM part p
WHERE p.session_id IN (
SELECT id FROM session
WHERE time_created >= (strftime('%s','now','-$esc_win') * 1000)
ORDER BY time_created DESC LIMIT 200)
AND json_extract(p.data,'\$.type')='text'
AND json_extract(p.data,'\$.text') LIKE '%$esc_txt%'
LIMIT $limit"
}
oqa_text_main() {
local sid="" window="" recent="" limit=50 needle=""
while [ $# -gt 0 ]; do
case "$1" in
--session) sid="$2"; shift 2 ;;
--recent) recent="$2"; shift 2 ;;
--since) window="$2"; shift 2 ;;
--limit) limit="$2"; shift 2 ;;
*) needle="$1"; shift ;;
esac
done
if [ -z "$needle" ]; then
oqa_log "error: missing search text"; return 2
fi
if [ -n "$sid" ]; then
oqa_text_scoped "$sid" "$needle" "$limit"; return 0
fi
if [ -n "$recent" ]; then
oqa_text_recent "$recent" "$needle" "$limit"; return 0
fi
if [ -n "$window" ]; then
oqa_text_since "$window" "$needle" "$limit"; return 0
fi
oqa_log "error: refusing an unbounded global text scan over the multi-GB part table."
oqa_log " scope it with --session <ses_id>, --recent <N>, or --since \"<N days|hours>\"."
return 2
}
oqa_self_test() {
oqa_require opencode jq || return 1
local fails=0
# 1) scoped: find a recent session with text parts, derive a needle from one.
local sid needle out n
sid="$(oqa_db_query "SELECT session_id AS s FROM part WHERE json_extract(data,'\$.type')='text' ORDER BY rowid DESC LIMIT 1" | jq -r '.[0].s // empty')"
if [ -z "$sid" ]; then oqa_log "FAIL: no text parts found"; return 1; fi
needle="$(oqa_db_query "SELECT substr(json_extract(data,'\$.text'),1,8) AS t FROM part WHERE session_id='$(oqa_sql_escape "$sid")' AND json_extract(data,'\$.type')='text' AND length(json_extract(data,'\$.text'))>=8 LIMIT 1" | jq -r '.[0].t // empty')"
if [ -z "$needle" ]; then oqa_log "FAIL: could not derive a text needle"; return 1; fi
out="$(oqa_text_main --session "$sid" "$needle")"
n="$(printf '%s' "$out" | jq 'length')"
if [ "${n:-0}" -ge 1 ]; then oqa_pass "scoped text search found $n row(s) in $sid"; else oqa_log "FAIL: scoped search empty for '$needle' in $sid"; fails=$((fails+1)); fi
# 2) unbounded refusal: no --session/--since must exit 2.
oqa_text_main "$needle" >/dev/null 2>&1
if [ "$?" -eq 2 ]; then oqa_pass "unbounded global scan refused (exit 2)"; else oqa_log "FAIL: unbounded scan was not refused"; fails=$((fails+1)); fi
# 3) bounded --recent search completes well under a hard 30s budget, even in
# the worst case (no early match) because the IN-subquery caps the scan to
# the newest N sessions and drives part_session_idx.
local t0 t1
t0=$(date +%s)
oqa_text_main --recent 25 --limit 5 "oqa_no_match_$(date +%s)_zzz" >/dev/null 2>&1
t1=$(date +%s)
if [ $((t1 - t0)) -le 30 ]; then
oqa_pass "bounded --recent 25 worst-case search completed in $((t1-t0))s (<=30s)"
else
oqa_log "FAIL: bounded --recent search took $((t1-t0))s (>30s)"; fails=$((fails+1))
fi
# also prove --recent returns real matches for the derived needle
out="$(oqa_text_main --recent 25 --limit 5 "$needle" 2>/dev/null)"
n="$(printf '%s' "$out" | jq 'length')"
if [ "${n:-0}" -ge 1 ]; then oqa_pass "bounded --recent 25 found $n row(s) for '$needle'"; else oqa_log "FAIL: --recent found no rows for '$needle'"; fails=$((fails+1)); fi
[ "$fails" -eq 0 ] && { oqa_pass "db-session-by-text"; return 0; }
oqa_log "db-session-by-text had $fails failure(s)"; return 1
}
case "${1:-}" in
--self-test) oqa_self_test; exit $? ;;
-h|--help|"")
sed -n '2,24p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'
[ -z "${1:-}" ] && exit 2 || exit 0 ;;
*)
oqa_require opencode jq || exit 1
oqa_text_main "$@" ;;
esac
#!/usr/bin/env bash
# export-roundtrip.sh - export a session as clean JSON and verify it round-trips.
#
# `opencode export <id>` prints a human line ("Exporting session: ...") to
# STDERR and the JSON document to STDOUT, so suppress stderr before piping to jq.
# The JSON shape is { info: {id, slug, projectID, directory, title, tokens,
# time, ...}, messages: [...] }.
#
# Usage:
# export-roundtrip.sh ses_3a4e22ad5ffebMKLt0tL7exPjZ # prints clean JSON
# export-roundtrip.sh --self-test
#
# Tip: redirect to a file for archival: export-roundtrip.sh <id> > session.json
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
. "$SCRIPT_DIR/lib/common.sh"
oqa_export() {
local id="$1" out
out="$(mktemp -t oqa-export.XXXXXX)" || return 1
OQA_TMPDIRS+=("$out")
oqa_export_to_file "$id" "$out" || return 1
oqa_validate_export_file "$id" "$out" || return 1
cat "$out"
}
oqa_extract_json_stdout() {
local raw="$1" clean="$2"
awk '
BEGIN { started = 0 }
started { print; next }
{
pos = index($0, "{")
if (pos > 0) {
print substr($0, pos)
started = 1
}
}
' "$raw" >"$clean"
}
oqa_export_to_file() {
local id="$1" out="$2" raw
if [ -z "$id" ]; then
oqa_log "FAIL: missing session id"
return 2
fi
raw="$(mktemp -t oqa-export-raw.XXXXXX)" || return 1
OQA_TMPDIRS+=("$raw")
opencode export "$id" >"$raw" 2>/dev/null || return 1
oqa_extract_json_stdout "$raw" "$out"
}
oqa_validate_export_file() {
local id="$1" file="$2" got title msgtype
if ! jq -e . "$file" >/dev/null 2>&1; then
oqa_log "FAIL: export stdout is not valid JSON for $id"
return 1
fi
got="$(jq -r '.info.id // empty' "$file")"
if [ "$got" != "$id" ]; then
oqa_log "FAIL: export .info.id '$got' != '$id'"
return 1
fi
title="$(jq -r '.info.title|type' "$file" 2>/dev/null)"
msgtype="$(jq -r '.messages|type' "$file" 2>/dev/null)"
if [ "$title" = "string" ] && { [ "$msgtype" = "array" ] || [ "$msgtype" = "null" ]; }; then
return 0
fi
oqa_log "FAIL: unexpected shape (title=$title messages=$msgtype)"
return 1
}
oqa_self_test() {
oqa_require opencode jq || return 1
local id out
id="$(oqa_db_query "SELECT id FROM session ORDER BY time_created DESC LIMIT 1" | jq -r '.[0].id // empty')"
if [ -z "$id" ]; then oqa_log "FAIL: no sessions to export"; return 1; fi
out="$(mktemp -t oqa-export.XXXXXX)" || return 1
OQA_TMPDIRS+=("$out")
oqa_export_to_file "$id" "$out" || return 1
oqa_validate_export_file "$id" "$out" || return 1
oqa_pass "export round-trips $id (info.id matches, valid JSON)"
}
case "${1:-}" in
--self-test) oqa_self_test; exit $? ;;
-h|--help|"")
sed -n '2,16p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'
[ -z "${1:-}" ] && exit 2 || exit 0 ;;
*)
oqa_require opencode jq || exit 1
oqa_export "$1" ;;
esac
#!/usr/bin/env bash
# common.sh - shared helpers for opencode-qa scripts.
#
# Source it from a sibling script:
# SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# . "$SCRIPT_DIR/lib/common.sh"
#
# SAFETY MODEL (read this):
# - DB-read helpers (oqa_db_path / oqa_db_query) hit the LIVE opencode DB
# READ-ONLY. That is safe and intended for session investigation.
# - Anything that SPAWNS opencode (serve / run / tui) must run under an
# ISOLATED XDG sandbox (oqa_mk_isolated_xdg) so QA never writes junk
# sessions into the real ~/.local/share/opencode DB.
# - oqa_cleanup runs on EXIT and tears down servers, tmux sessions, curl
# watchers, and every temp dir created via the helpers.
# No `set -e`: these scripts deliberately probe failure paths (401, refused).
set -uo pipefail
OQA_TMPDIRS=()
OQA_TMUX_SESSIONS=()
OQA_CURL_PIDS=()
OQA_SERVER_PID=""
oqa_log() { printf '%s\n' "$*" >&2; }
oqa_pass() { printf 'PASS: %s\n' "$*"; }
oqa_fail() { printf 'FAIL: %s\n' "$*" >&2; return 1; }
# oqa_require <bin>... -> 0 if all present, else 1 (names the missing ones).
oqa_require() {
local missing=0 b
for b in "$@"; do
if ! command -v "$b" >/dev/null 2>&1; then
oqa_log "missing dependency: $b"
missing=1
fi
done
return "$missing"
}
# Absolute path of the active opencode DB (resolves channel / OPENCODE_DB).
oqa_db_path() {
opencode db path 2>/dev/null | head -1
}
# Escape a value for safe embedding inside a single-quoted SQL literal.
oqa_sql_escape() {
printf '%s' "$1" | sed "s/'/''/g"
}
# Run a read-only SQL query against the active DB; emit JSON rows.
# Usage: oqa_db_query "SELECT ... LIMIT 5"
oqa_db_query() {
opencode db "$1" --format json 2>/dev/null
}
# Preserve HOME-based opencode shims after HOME is sandboxed. Some installed
# opencode wrappers resolve the real binary via "$HOME/.opencode/bin/opencode";
# after oqa_mk_isolated_xdg rewrites HOME, that path must still exist.
oqa_preserve_home_opencode_bin() {
local real_home="$1" sandbox_home="$2"
[ -d "$real_home/.opencode/bin" ] || return 0
mkdir -p "$sandbox_home/.opencode" || return 1
ln -s "$real_home/.opencode/bin" "$sandbox_home/.opencode/bin" 2>/dev/null || return 1
}
# Create an isolated XDG sandbox so a spawned opencode never touches the real
# DB. Sets globals OQA_XDG_ROOT + OQA_PROJ and exports XDG_*; registers the
# root for cleanup.
#
# IMPORTANT: call this DIRECTLY, never via $(...). Command substitution runs in
# a subshell, which would discard the exports and the cleanup registration.
# oqa_mk_isolated_xdg # good
# root="$OQA_XDG_ROOT" # read the global afterwards
oqa_mk_isolated_xdg() {
local root real_home
root="$(mktemp -d -t oqa-xdg.XXXXXX)" || return 1
real_home="$HOME"
OQA_TMPDIRS+=("$root")
mkdir -p "$root/data" "$root/config" "$root/cache" "$root/state" "$root/home" "$root/proj"
oqa_preserve_home_opencode_bin "$real_home" "$root/home" || return 1
export OQA_XDG_ROOT="$root"
export HOME="$root/home"
export OPENCODE_TEST_HOME="$root/home"
export XDG_DATA_HOME="$root/data"
export XDG_CONFIG_HOME="$root/config"
export XDG_CACHE_HOME="$root/cache"
export XDG_STATE_HOME="$root/state"
export OQA_PROJ="$root/proj"
# keep the sandbox offline + fast
export OPENCODE_DISABLE_AUTOUPDATE=1
export OPENCODE_DISABLE_MODELS_FETCH=1
}
# Print a free TCP port on 127.0.0.1.
oqa_free_port() {
if command -v python3 >/dev/null 2>&1; then
python3 -c 'import socket; s=socket.socket(); s.bind(("127.0.0.1",0)); print(s.getsockname()[1]); s.close()'
elif command -v bun >/dev/null 2>&1; then
bun -e 'const s=Bun.listen({hostname:"127.0.0.1",port:0,socket:{data(){}}});console.log(s.port);s.stop()'
else
# last resort: a high random port (small race window)
printf '%s' "$(( (RANDOM % 20000) + 40000 ))"
fi
}
# Poll an HTTP url until it accepts a connection (any status) or times out.
# Usage: oqa_wait_http <url> [user:pass] [timeout_s]
oqa_wait_http() {
local url="$1" auth="${2:-}" timeout="${3:-25}" deadline
deadline=$(( $(date +%s) + timeout ))
while [ "$(date +%s)" -lt "$deadline" ]; do
if [ -n "$auth" ]; then
curl -s -o /dev/null -u "$auth" "$url" && return 0
else
curl -s -o /dev/null "$url" && return 0
fi
sleep 0.2
done
return 1
}
# Start an isolated, password-protected headless server.
# Sets globals OQA_SERVER_URL / OQA_SERVER_PASS / OQA_SERVER_PORT / OQA_SERVER_PID.
# Returns 1 if it never becomes ready.
#
# IMPORTANT: call this DIRECTLY, never via $(...). The PID + exports must land
# in the caller's shell so oqa_cleanup can kill the server on exit.
# oqa_start_server || { oqa_log "no server"; exit 1; }
# url="$OQA_SERVER_URL"
oqa_start_server() {
oqa_mk_isolated_xdg || return 1
local port pass
port="$(oqa_free_port)"
pass="oqa-${RANDOM}${RANDOM}"
OPENCODE_SERVER_PASSWORD="$pass" opencode serve --port "$port" --hostname 127.0.0.1 \
>"$XDG_STATE_HOME/serve.log" 2>&1 &
OQA_SERVER_PID=$!
disown "$OQA_SERVER_PID" 2>/dev/null || true
export OQA_SERVER_PORT="$port"
export OQA_SERVER_PASS="$pass"
export OQA_SERVER_URL="http://127.0.0.1:$port"
if ! oqa_wait_http "$OQA_SERVER_URL/global/health" "opencode:$pass" 30; then
oqa_log "server failed to start; log follows:"
cat "$XDG_STATE_HOME/serve.log" >&2 2>/dev/null || true
return 1
fi
}
# Teardown everything the helpers created. Safe to call multiple times.
oqa_cleanup() {
if [ -n "${OQA_SERVER_PID:-}" ]; then
kill "$OQA_SERVER_PID" 2>/dev/null || true
sleep 0.3
kill -0 "$OQA_SERVER_PID" 2>/dev/null && kill -9 "$OQA_SERVER_PID" 2>/dev/null || true
OQA_SERVER_PID=""
fi
local s p d
for s in "${OQA_TMUX_SESSIONS[@]:-}"; do
[ -n "$s" ] && tmux kill-session -t "$s" 2>/dev/null || true
done
for p in "${OQA_CURL_PIDS[@]:-}"; do
[ -n "$p" ] && kill "$p" 2>/dev/null || true
[ -n "$p" ] && sleep 0.1
[ -n "$p" ] && kill -0 "$p" 2>/dev/null && kill -9 "$p" 2>/dev/null || true
done
for d in "${OQA_TMPDIRS[@]:-}"; do
[ -n "$d" ] && rm -rf "$d" 2>/dev/null || true
done
OQA_TMPDIRS=()
OQA_TMUX_SESSIONS=()
OQA_CURL_PIDS=()
}
trap oqa_cleanup EXIT
# ---- self-check ------------------------------------------------------------
# Run: bash scripts/lib/common.sh --self-check
oqa__self_check() {
local fails=0
if oqa_require opencode sqlite3 curl jq tmux; then
oqa_pass "dependencies present (opencode sqlite3 curl jq tmux)"
else
oqa_log "FAIL: missing dependencies"; fails=$((fails+1))
fi
local dbp; dbp="$(oqa_db_path)"
if [ -n "$dbp" ] && [ -f "$dbp" ]; then
oqa_pass "oqa_db_path -> $dbp"
else
oqa_log "FAIL: oqa_db_path returned '$dbp'"; fails=$((fails+1))
fi
local esc; esc="$(oqa_sql_escape "a'b'c")"
if [ "$esc" = "a''b''c" ]; then
oqa_pass "oqa_sql_escape quotes single quotes"
else
oqa_log "FAIL: oqa_sql_escape -> '$esc'"; fails=$((fails+1))
fi
local port; port="$(oqa_free_port)"
if [ "$port" -gt 0 ] 2>/dev/null; then
oqa_pass "oqa_free_port -> $port"
else
oqa_log "FAIL: oqa_free_port -> '$port'"; fails=$((fails+1))
fi
# isolation + trap teardown: an inner shell creates a sandbox (calling the
# helper DIRECTLY so the cleanup registration survives) and exits; the EXIT
# trap must remove it. We pass the sandbox path out via a marker file.
local marker isodir home test_home
marker="$(mktemp -t oqa-marker.XXXXXX)"
bash -c '. "'"${BASH_SOURCE[0]}"'"; oqa_mk_isolated_xdg; printf "%s\n%s\n%s\n" "$OQA_XDG_ROOT" "$HOME" "$OPENCODE_TEST_HOME" > "'"$marker"'"'
isodir="$(sed -n '1p' "$marker" 2>/dev/null)"
home="$(sed -n '2p' "$marker" 2>/dev/null)"
test_home="$(sed -n '3p' "$marker" 2>/dev/null)"
rm -f "$marker"
if [ -n "$isodir" ] && [ ! -d "$isodir" ]; then
oqa_pass "isolated XDG sandbox auto-removed on exit ($isodir)"
else
oqa_log "FAIL: sandbox not cleaned: '$isodir' (exists=$([ -d "$isodir" ] && echo yes || echo no))"; fails=$((fails+1))
fi
if [ -n "$isodir" ] && [ "$home" = "$isodir/home" ] && [ "$test_home" = "$isodir/home" ]; then
oqa_pass "isolated HOME points inside sandbox"
else
oqa_log "FAIL: sandbox HOME not isolated (HOME='$home' OPENCODE_TEST_HOME='$test_home' root='$isodir')"; fails=$((fails+1))
fi
local shim_marker shim_result
shim_marker="$(mktemp -t oqa-shim.XXXXXX)"
bash -c '
set -u
. "'"${BASH_SOURCE[0]}"'"
fake_home="$(mktemp -d -t oqa-fake-home.XXXXXX)"
mkdir -p "$fake_home/.local/bin" "$fake_home/.opencode/bin"
printf "%s\n" "#!/usr/bin/env bash" "exec \"\$HOME/.opencode/bin/opencode\" \"\$@\"" > "$fake_home/.local/bin/opencode"
printf "%s\n" "#!/usr/bin/env bash" "exec \"\$HOME/.opencode/bin/opencode-real\" \"\$@\"" > "$fake_home/.opencode/bin/opencode"
printf "%s\n" "#!/usr/bin/env bash" "printf fake-opencode-ok" > "$fake_home/.opencode/bin/opencode-real"
chmod +x "$fake_home/.local/bin/opencode" "$fake_home/.opencode/bin/opencode" "$fake_home/.opencode/bin/opencode-real"
HOME="$fake_home"
PATH="$fake_home/.local/bin:$PATH"
oqa_mk_isolated_xdg
opencode > "'"$shim_marker"'"
oqa_cleanup
rm -rf "$fake_home"
'
shim_result="$(cat "$shim_marker" 2>/dev/null)"
rm -f "$shim_marker"
if [ "$shim_result" = "fake-opencode-ok" ]; then
oqa_pass "isolated HOME preserves HOME-based opencode shim"
else
oqa_log "FAIL: HOME-based opencode shim returned '$shim_result'"; fails=$((fails+1))
fi
if [ "$fails" -eq 0 ]; then
oqa_pass "common.sh self-check"
return 0
fi
oqa_log "common.sh self-check had $fails failure(s)"
return 1
}
if [ "${1:-}" = "--self-check" ]; then
oqa__self_check
exit $?
fi
export const branchCounts = {
title: 0,
"parent-tool-call": 0,
"parent-hold": 0,
child: 0,
wake: 0,
default: 0,
}
export const latches = {
parentToolCallIssued: false,
parentHoldIssued: false,
}
export function hasToolResult(inputStr) {
return (
inputStr.includes('"type":"function_call_output"') ||
inputStr.includes('"type": "function_call_output"') ||
inputStr.includes('"type":"tool_result"') ||
inputStr.includes('"type": "tool_result"') ||
inputStr.includes('"role":"tool"') ||
inputStr.includes('"role": "tool"')
)
}
export function selectBranch(inputStr) {
const isTitle = inputStr.includes("Generate a title")
const isSplitProbe = inputStr.includes("Run the split probe")
const isChild = inputStr.includes("SPLIT_CHILD_TASK")
const isWake = inputStr.includes("[BACKGROUND TASK")
const hasResult = hasToolResult(inputStr)
if (isTitle) return "title"
if (isChild && !isSplitProbe) return "child"
if (isWake) return "wake"
if (isSplitProbe && !hasResult && !latches.parentToolCallIssued) return "parent-tool-call"
if (isSplitProbe && (hasResult || latches.parentToolCallIssued) && !latches.parentHoldIssued) return "parent-hold"
return "default"
}
import fs from "node:fs"
export function completedUsage() {
return {
input_tokens: 10,
output_tokens: 5,
input_tokens_details: { cached_tokens: 0 },
output_tokens_details: { reasoning_tokens: 0 },
}
}
export function sendSse(res, events) {
res.writeHead(200, {
"content-type": "text/event-stream; charset=utf-8",
"cache-control": "no-cache",
connection: "keep-alive",
})
for (const event of events) {
res.write(`data: ${JSON.stringify(event)}\n\n`)
}
res.write("data: [DONE]\n\n")
res.end()
}
export function textEvents(callCount, text) {
const id = `resp_${callCount}`
const item = `msg_${callCount}`
return [
{
type: "response.created",
response: { id, created_at: Math.floor(Date.now() / 1000), model: "gpt-fake" },
},
{
type: "response.output_item.added",
output_index: 0,
item: { type: "message", id: item },
},
{
type: "response.output_text.delta",
item_id: item,
output_index: 0,
delta: text,
},
{
type: "response.output_item.done",
output_index: 0,
item: { type: "message", id: item },
},
{
type: "response.completed",
response: { usage: completedUsage() },
},
]
}
export function toolCallEvents(callCount, name, callId, argsObj) {
const id = `resp_${callCount}`
const fcId = `fc_${callCount}`
const argsStr = JSON.stringify(argsObj)
return [
{
type: "response.created",
response: { id, created_at: Math.floor(Date.now() / 1000), model: "gpt-fake" },
},
{
type: "response.output_item.added",
output_index: 0,
item: {
type: "function_call",
id: fcId,
call_id: callId,
name,
arguments: "",
},
},
{
type: "response.function_call_arguments.delta",
item_id: fcId,
output_index: 0,
delta: argsStr,
},
{
type: "response.output_item.done",
output_index: 0,
item: {
type: "function_call",
id: fcId,
call_id: callId,
name,
arguments: argsStr,
status: "completed",
},
},
{
type: "response.completed",
response: { usage: completedUsage() },
},
]
}
export function appendLog(logFile, line) {
try {
fs.appendFileSync(logFile, line)
} catch {
}
}
#!/usr/bin/env node
import http from "node:http"
import fs from "node:fs"
import path from "node:path"
import os from "node:os"
import { sendSse, textEvents, toolCallEvents, appendLog } from "./fake-openai-events.mjs"
import { branchCounts, latches, selectBranch } from "./fake-openai-branches.mjs"
const requestedPort = Number(process.env.FAKE_OPENAI_PORT ?? 0)
const logFile = process.env.FAKE_LLM_LOG ?? path.join(os.tmpdir(), "fake-llm.log")
let callCount = 0
function logBranch(branch, extra = {}) {
const now = new Date().toISOString()
const line = `[${now}] branch=${branch} call=${callCount}${Object.keys(extra).length ? " " + JSON.stringify(extra) : ""}\n`
appendLog(logFile, line)
process.stdout.write(line)
}
function readBody(req) {
return new Promise((resolve, reject) => {
const chunks = []
req.on("data", (chunk) => chunks.push(chunk))
req.on("end", () => resolve(Buffer.concat(chunks).toString("utf8")))
req.on("error", reject)
})
}
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms))
}
const server = http.createServer(async (req, res) => {
if (req.method === "GET" && req.url === "/health") {
res.writeHead(200, { "content-type": "text/plain" }).end("ok")
return
}
if (req.method !== "POST" || !req.url?.includes("/responses")) {
res.writeHead(404, { "content-type": "application/json" }).end(JSON.stringify({ error: "not found" }))
return
}
callCount++
const raw = await readBody(req)
let body
try { body = JSON.parse(raw) } catch { body = {} }
const inputStr = JSON.stringify(body.input ?? body.messages ?? body)
const branch = selectBranch(inputStr)
branchCounts[branch] = (branchCounts[branch] ?? 0) + 1
logBranch(branch)
if (branch === "title") {
sendSse(res, textEvents(callCount, "wake split probe session"))
return
}
if (branch === "child") {
sendSse(res, textEvents(callCount, "DONE"))
return
}
if (branch === "wake") {
await sleep(3000)
sendSse(res, textEvents(callCount, `WAKE_ACK ${callCount}`))
return
}
if (branch === "parent-tool-call") {
latches.parentToolCallIssued = true
sendSse(res, toolCallEvents(callCount, "task", `call_agent_${callCount}`, {
description: "split probe child",
prompt: "SPLIT_CHILD_TASK: reply exactly DONE",
subagent_type: "explore",
run_in_background: true,
load_skills: [],
}))
return
}
if (branch === "parent-hold") {
latches.parentHoldIssued = true
sendSse(res, toolCallEvents(callCount, "bash", `call_bash_${callCount}`, {
command: "i=0; while [ $i -lt 8 ]; do i=$((i+1)); sleep 1; done",
description: "hold turn",
}))
return
}
if (inputStr.includes("say exactly: TUI_NOREG_OK")) {
sendSse(res, textEvents(callCount, "TUI_NOREG_OK"))
return
}
sendSse(res, textEvents(callCount, `fake response ${callCount}`))
})
function logFinalCounts() {
const summary = Object.entries(branchCounts).map(([k, v]) => `${k}=${v}`).join(" ")
const line = `[${new Date().toISOString()}] FINAL_COUNTS ${summary}\n`
appendLog(logFile, line)
process.stdout.write(line)
}
server.listen(requestedPort, "127.0.0.1", () => {
const addr = server.address()
const port = typeof addr === "object" && addr !== null ? addr.port : requestedPort
try {
fs.mkdirSync(path.dirname(logFile), { recursive: true })
appendLog(logFile, `[${new Date().toISOString()}] START port=${port}\n`)
} catch {}
process.stdout.write(`fake-openai listening on ${port}\n`)
})
process.on("SIGTERM", () => { logFinalCounts(); server.close(() => process.exit(0)) })
process.on("SIGINT", () => { logFinalCounts(); server.close(() => process.exit(0)) })
#!/usr/bin/env bash
# serve-wake-split-probe.sh
# Serve-topology wake runner-split QA harness.
#
# Proves whether omo's plugin-origin promptAsync (parent-wake bg notifications)
# forks a second concurrent LLM runner in opencode serve topology (REPRODUCED)
# or routes correctly through the live listener (FIXED).
#
# Two assertion modes:
# --expect reproduced exit 0 if terminal_stops>1 OR child_task_sessions>1 OR mechanism arm true
# --expect fixed exit 0 if terminal_stops==1, child_task_sessions==1,
# fixed branch counts hold, and route logs show live dispatch
#
# Usage:
# serve-wake-split-probe.sh [--expect reproduced|fixed] [--evidence-dir DIR]
# [--self-test] [--help]
#
# Env:
# OMO_SANDBOX_OMO_CONFIG JSON string; when set, deep-merged over the base
# agent overrides (env keys win) and written to
# $XDG_CONFIG_HOME/opencode/oh-my-openagent.json
# before the server starts (flag-disabled control).
# FAKE_OPENAI_PORT Force the fake-LLM to bind a specific port
# (default: random). Port 1 triggers a startup
# failure test used by the self-test failure path.
#
# Exit codes:
# 0 expectation met (or --self-test OK)
# 1 expectation NOT met, or internal harness error
# 2 usage / bad arguments
set -uo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
. "$SCRIPT_DIR/lib/common.sh"
# ---- defaults ----------------------------------------------------------------
EXPECT_MODE="" # reproduced | fixed
EVIDENCE_DIR=""
SELF_TEST=0
FAKE_SERVER_PID=""
FAKE_SERVER_PORT=""
FAKE_LLM_LOG="" # set after evidence dir is known
# ---- argument parsing --------------------------------------------------------
while [ $# -gt 0 ]; do
case "$1" in
--expect)
if [ $# -lt 2 ] || [ "${2#--}" != "$2" ]; then
printf 'error: --expect requires reproduced or fixed\n' >&2
exit 2
fi
EXPECT_MODE="$2"
shift 2
;;
--evidence-dir)
if [ $# -lt 2 ] || [ "${2#--}" != "$2" ]; then
printf 'error: --evidence-dir requires a directory\n' >&2
exit 2
fi
EVIDENCE_DIR="$2"
shift 2
;;
--self-test)
SELF_TEST=1
shift
;;
-h|--help)
sed -n '2,26p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'
exit 0
;;
*)
printf 'unknown option: %s\n' "$1" >&2
exit 2
;;
esac
done
if [ "$SELF_TEST" -eq 0 ] && [ -z "$EXPECT_MODE" ]; then
EXPECT_MODE="reproduced"
fi
if [ -n "$EXPECT_MODE" ] && [ "$EXPECT_MODE" != "reproduced" ] && [ "$EXPECT_MODE" != "fixed" ]; then
printf 'error: --expect must be reproduced or fixed\n' >&2
exit 2
fi
# ---- helpers -----------------------------------------------------------------
swsp_log() { printf '%s\n' "$*" >&2; }
swsp_info() { printf '[swsp] %s\n' "$*" >&2; }
swsp_tail_log_since_offset() {
local offset="$1"
local log_path="$2"
if [ ! -f "$log_path" ]; then
return 0
fi
local current_size
current_size="$(wc -c <"$log_path" 2>/dev/null | tr -d ' ')" || current_size=0
current_size="${current_size:-0}"
if [ "$offset" -gt "$current_size" ] 2>/dev/null; then
return 0
else
tail -c "+$((offset + 1))" "$log_path" 2>/dev/null || true
fi
}
# Start the fake-LLM server; sets FAKE_SERVER_PID + FAKE_SERVER_PORT.
swsp_start_fake_llm() {
local log_file="$1"
local port_file
port_file="$(mktemp -t swsp-port.XXXXXX)"
OQA_TMPDIRS+=("$port_file")
FAKE_LLM_LOG="$log_file" FAKE_OPENAI_PORT="${FAKE_OPENAI_PORT:-0}" \
bun run --bun "$SCRIPT_DIR/lib/fake-openai-server.mjs" >"$port_file.stdout" 2>&1 &
FAKE_SERVER_PID=$!
disown "$FAKE_SERVER_PID" 2>/dev/null || true
# Poll for the port line (fake-openai listening on <port>)
local deadline
deadline=$(( $(date +%s) + 10 ))
while [ "$(date +%s)" -lt "$deadline" ]; do
if grep -q "^fake-openai listening on " "$port_file.stdout" 2>/dev/null; then
FAKE_SERVER_PORT="$(grep "^fake-openai listening on " "$port_file.stdout" | head -1 | awk '{print $NF}')"
break
fi
if ! kill -0 "$FAKE_SERVER_PID" 2>/dev/null; then
swsp_log "FAIL: fake-openai server process died immediately"
cat "$port_file.stdout" >&2 2>/dev/null || true
return 1
fi
sleep 0.3
done
OQA_TMPDIRS+=("$port_file.stdout")
if [ -z "$FAKE_SERVER_PORT" ]; then
swsp_log "FAIL: fake-openai server did not report port within 10s"
cat "$port_file.stdout" >&2 2>/dev/null || true
kill "$FAKE_SERVER_PID" 2>/dev/null || true
return 1
fi
# Verify it's up
local hdeadline=0
hdeadline=$(( $(date +%s) + 5 ))
while [ "$(date +%s)" -lt "$hdeadline" ]; do
if curl -sf "http://127.0.0.1:${FAKE_SERVER_PORT}/health" >/dev/null 2>&1; then
swsp_info "fake-openai listening on port $FAKE_SERVER_PORT"
return 0
fi
sleep 0.2
done
swsp_log "FAIL: fake-openai /health did not respond within 5s on port $FAKE_SERVER_PORT"
kill "$FAKE_SERVER_PID" 2>/dev/null || true
return 1
}
swsp_stop_fake_llm() {
if [ -n "$FAKE_SERVER_PID" ]; then
kill "$FAKE_SERVER_PID" 2>/dev/null || true
sleep 0.3
kill -0 "$FAKE_SERVER_PID" 2>/dev/null && kill -9 "$FAKE_SERVER_PID" 2>/dev/null || true
FAKE_SERVER_PID=""
fi
}
# Write the sandbox omo config: base agent overrides (explore/librarian -> the
# fake provider, required for child model resolution) deep-merged with
# OMO_SANDBOX_OMO_CONFIG when set (jq '.[0] * .[1]'; env keys win).
# Args: sandbox_config_dir
swsp_write_omo_config() {
local cfg_dir="$1"
local omo_cfg="$cfg_dir/opencode/oh-my-openagent.json"
local base='{"agents":{"explore":{"model":"openai/gpt-fake"},"librarian":{"model":"openai/gpt-fake"}}}'
mkdir -p "$cfg_dir/opencode"
if [ -n "${OMO_SANDBOX_OMO_CONFIG:-}" ]; then
if ! printf '%s\n%s\n' "$base" "$OMO_SANDBOX_OMO_CONFIG" | jq -s '.[0] * .[1]' >"$omo_cfg" 2>/dev/null; then
swsp_log "FAIL: OMO_SANDBOX_OMO_CONFIG is not valid JSON"
return 1
fi
swsp_info "wrote merged OMO_SANDBOX_OMO_CONFIG to $omo_cfg"
else
printf '%s\n' "$base" >"$omo_cfg"
swsp_info "wrote agent overrides to $omo_cfg"
fi
}
# Write the sandbox opencode.jsonc with the fake provider + local plugin.
# Args: sandbox_config_dir fake_port
swsp_write_opencode_config() {
local cfg_dir="$1"
local fake_port="$2"
local repo_root
repo_root="$(cd "$SCRIPT_DIR/../../../.." && pwd)"
mkdir -p "$cfg_dir/opencode"
cat >"$cfg_dir/opencode/opencode.jsonc" <<JSONC
{
"plugin": ["file://${repo_root}/packages/omo-opencode/src/index.ts"],
"model": "openai/gpt-fake",
"provider": {
"openai": {
"options": {
"apiKey": "fake-key",
"baseURL": "http://127.0.0.1:${fake_port}/v1",
"timeout": 30000
},
"models": {
"gpt-fake": {
"tool_call": true,
"limit": {
"context": 200000,
"output": 8192
}
}
}
}
},
"permission": {
"bash": "allow",
"call_omo_agent": "allow"
}
}
JSONC
swsp_info "opencode.jsonc written to $cfg_dir/opencode/opencode.jsonc"
}
# Poll the sandbox DB for parent assistant step metrics on a message matching a LIKE pattern.
# Args: db_path like_pattern timeout_s
# Outputs: "<parent_assistant_messages> <parent_tool_call_turns> <terminal_stops> <child_task_sessions>" on stdout
swsp_poll_db_metrics() {
local db="$1"
local like_pat="$2"
local timeout_s="${3:-90}"
local deadline
local metrics_query="
WITH target AS (
SELECT m.id AS user_id, m.session_id
FROM message m
JOIN part p ON p.message_id = m.id
WHERE json_extract(m.data, '\$.role') = 'user'
AND json_extract(p.data, '\$.type') = 'text'
AND json_extract(p.data, '\$.text') LIKE '${like_pat}'
),
counts AS (
SELECT
count(a.id) AS parent_assistant_messages,
sum(CASE WHEN json_extract(a.data, '\$.finish') = 'tool-calls' THEN 1 ELSE 0 END) AS parent_tool_call_turns,
sum(CASE WHEN json_extract(a.data, '\$.finish') = 'stop' THEN 1 ELSE 0 END) AS terminal_stops
FROM target t
LEFT JOIN message a
ON a.session_id = t.session_id
AND json_extract(a.data, '\$.parentID') = t.user_id
GROUP BY t.user_id
),
child_task_sessions AS (
SELECT count(DISTINCT m.session_id) AS child_task_sessions
FROM message m
JOIN part p ON p.message_id = m.id
WHERE json_extract(m.data, '\$.role') = 'user'
AND json_extract(p.data, '\$.type') = 'text'
AND json_extract(p.data, '\$.text') LIKE '%SPLIT_CHILD_TASK:%'
)
SELECT printf('%d %d %d %d',
coalesce((SELECT max(parent_assistant_messages) FROM counts), 0),
coalesce((SELECT max(parent_tool_call_turns) FROM counts), 0),
coalesce((SELECT max(terminal_stops) FROM counts), 0),
coalesce((SELECT child_task_sessions FROM child_task_sessions), 0)
);
"
deadline=$(( $(date +%s) + timeout_s ))
while [ "$(date +%s)" -lt "$deadline" ]; do
if [ ! -f "$db" ]; then
sleep 0.5
continue
fi
local result
result="$(sqlite3 "$db" "$metrics_query" 2>/dev/null)" || true
local parent_assistant_messages parent_tool_call_turns terminal_stops child_task_sessions
parent_assistant_messages="$(printf '%s' "$result" | awk '{print $1}')"
parent_tool_call_turns="$(printf '%s' "$result" | awk '{print $2}')"
terminal_stops="$(printf '%s' "$result" | awk '{print $3}')"
child_task_sessions="$(printf '%s' "$result" | awk '{print $4}')"
# Return once we have at least 1 stop (parent session finished)
if [ -n "$terminal_stops" ] && [ "${terminal_stops:-0}" -ge 1 ] 2>/dev/null; then
printf '%s %s %s %s' "$parent_assistant_messages" "$parent_tool_call_turns" "$terminal_stops" "$child_task_sessions"
return 0
fi
sleep 0.5
done
# Return whatever we have on timeout
local result
result="$(sqlite3 "$db" "$metrics_query" 2>/dev/null)" || true
printf '%s' "${result:-0 0 0 0}"
}
# Wait until a session is no longer in the server's active status map.
# Args: server_url pass session_id timeout_s
swsp_wait_session_idle() {
local url="$1" pass="$2" ses_id="$3" timeout_s="${4:-120}"
local deadline
deadline=$(( $(date +%s) + timeout_s ))
while [ "$(date +%s)" -lt "$deadline" ]; do
local status_json
status_json="$(curl -sf -u "opencode:${pass}" "${url}/session/status" 2>/dev/null)" || true
if [ -z "$status_json" ] || ! printf '%s' "$status_json" | grep -q "$ses_id" 2>/dev/null; then
return 0
fi
sleep 0.5
done
swsp_log "WARNING: session $ses_id did not go idle within ${timeout_s}s; proceeding with current DB state"
return 0
}
# Count plugin_inits from the omo log since a byte offset, filtered to sandbox dir.
# Args: log_offset sandbox_dir
swsp_count_plugin_inits() {
local offset="$1"
local sandbox_dir="$2"
local log_path="${TMPDIR:-/tmp}/oh-my-opencode.log"
if [ ! -f "$log_path" ]; then
printf '0'
return 0
fi
swsp_tail_log_since_offset "$offset" "$log_path" \
| grep "ENTRY - plugin loading" \
| awk -v sandbox_dir="$sandbox_dir" 'index($0, sandbox_dir) { count += 1 } END { print count + 0 }'
}
# Detect WAKE_DISPATCHED_DURING_PARENT_TURN:
# true iff the omo log (since offset) contains a [prompt-async-gate] promptAsync dispatching
# line with source containing "parent-wake", AND that line's timestamp is within the
# parent-hold window (between branch=parent-hold line and next non-wake completion).
# Args: log_offset fake_llm_log sandbox_dir
swsp_detect_wake_during_parent() {
local offset="$1"
local fake_log="$2"
local sandbox_dir="$3"
local omo_log="${TMPDIR:-/tmp}/oh-my-opencode.log"
# Find parent-hold timestamp from fake-llm.log
local hold_ts
hold_ts="$(grep "branch=parent-hold" "$fake_log" 2>/dev/null | head -1 | grep -o '\[.*\]' | tr -d '[]')" || true
if [ -z "$hold_ts" ]; then
# parent-hold never fired — cannot determine
printf 'false'
return 0
fi
# Check for gate dispatch log line with parent-wake source since offset
local dispatch_line
dispatch_line="$(swsp_tail_log_since_offset "$offset" "$omo_log" \
| grep "promptAsync dispatching" \
| grep -i "parent-wake\|background-agent-parent-wake" \
| head -1)" || true
if [ -z "$dispatch_line" ]; then
printf 'false'
return 0
fi
# Extract timestamp from dispatch line (ISO 8601 in brackets or as prefix)
local dispatch_ts
dispatch_ts="$(printf '%s' "$dispatch_line" | grep -o '[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]T[0-9][0-9]:[0-9][0-9]:[0-9][0-9]' | head -1)" || true
if [ -z "$dispatch_ts" ]; then
# Can't compare timestamps — fall back to presence check
printf 'true'
return 0
fi
# Simple lexicographic timestamp compare (ISO 8601 sorts correctly)
# The hold_ts is the start of the hold window; if dispatch happened after it, signal is true
if [ "$dispatch_ts" \> "$hold_ts" ] || [ "$dispatch_ts" = "$hold_ts" ]; then
printf 'true'
else
printf 'false'
fi
}
swsp_has_session_live_dispatch() {
local route_prov="$1"
local session_id="$2"
[ -n "$session_id" ] || return 1
printf '%s\n' "$route_prov" \
| grep -F "dispatch via live listener" \
| grep -F "\"sessionID\":\"${session_id}\"" >/dev/null 2>&1
}
swsp_is_nonnegative_int() {
case "${1:-}" in
''|*[!0-9]*) return 1 ;;
*) return 0 ;;
esac
}
swsp_fixed_topology_observed() {
local parent_assistant_messages="$1"
local parent_tool_call_turns="$2"
local terminal_stops="$3"
local child_task_sessions="$4"
local parent_tool_call_branches="$5"
local parent_hold_branches="$6"
local child_branches="$7"
local wake_branches="$8"
local default_branches="$9"
local has_live_dispatch="${10}"
swsp_is_nonnegative_int "$parent_assistant_messages" || return 1
swsp_is_nonnegative_int "$parent_tool_call_turns" || return 1
swsp_is_nonnegative_int "$terminal_stops" || return 1
swsp_is_nonnegative_int "$child_task_sessions" || return 1
swsp_is_nonnegative_int "$parent_tool_call_branches" || return 1
swsp_is_nonnegative_int "$parent_hold_branches" || return 1
swsp_is_nonnegative_int "$child_branches" || return 1
swsp_is_nonnegative_int "$wake_branches" || return 1
swsp_is_nonnegative_int "$default_branches" || return 1
[ "$has_live_dispatch" = "true" ] || return 1
if [ "${terminal_stops:-0}" -ne 1 ] \
|| [ "${child_task_sessions:-0}" -ne 1 ] \
|| [ "${parent_tool_call_turns:-0}" -ne 2 ] \
|| [ "${parent_assistant_messages:-0}" -ne 3 ] \
|| [ "${parent_tool_call_branches:-0}" -ne 1 ] \
|| [ "${parent_hold_branches:-0}" -ne 1 ] \
|| [ "${child_branches:-0}" -ne 1 ] \
|| [ "${default_branches:-0}" -lt 1 ] \
|| [ "${wake_branches:-0}" -ne 0 ] 2>/dev/null; then
return 1
fi
return 0
}
swsp_collect_route_provenance() {
local offset="$1"
local log_path="$2"
local project_dir="$3"
local session_id="$4"
local output_file="$5"
local all_output_file="$6"
local timeout_s="${7:-0}"
local deadline
deadline=$(( $(date +%s) + timeout_s ))
while :; do
local route_prov_all="" route_prov=""
if [ -f "$log_path" ]; then
route_prov_all="$(swsp_tail_log_since_offset "$offset" "$log_path" \
| grep -E "live-server-route" || true)"
route_prov="$(printf '%s\n' "$route_prov_all" \
| awk -v dir="$project_dir" -v sid="\"sessionID\":\"${session_id}\"" 'index($0, dir) || index($0, sid)' || true)"
fi
printf '%s' "$route_prov" >"$all_output_file"
printf '%s' "$route_prov" >"$output_file"
if swsp_has_session_live_dispatch "$route_prov" "$session_id"; then
return 0
fi
if [ "$timeout_s" -le 0 ] || [ "$(date +%s)" -ge "$deadline" ]; then
return 1
fi
sleep 0.25
done
}
# Verify branch-count guard: all required branches fired.
# Returns 0 if OK, 1 if any required branch missing (also sets RESULT=HARNESS_ERROR).
swsp_check_branch_counts() {
local fake_log="$1"
local mode="${2:-}"
local ptc pc cc wc
ptc="$(grep -c "branch=parent-tool-call" "$fake_log" 2>/dev/null | tr -d '[:space:]')"; ptc="${ptc:-0}"
pc="$(grep -c "branch=parent-hold" "$fake_log" 2>/dev/null | tr -d '[:space:]')"; pc="${pc:-0}"
cc="$(grep -c "branch=child" "$fake_log" 2>/dev/null | tr -d '[:space:]')"; cc="${cc:-0}"
wc="$(grep -c "branch=wake" "$fake_log" 2>/dev/null | tr -d '[:space:]')"; wc="${wc:-0}"
ptc="${ptc%%[!0-9]*}"; pc="${pc%%[!0-9]*}"; cc="${cc%%[!0-9]*}"; wc="${wc%%[!0-9]*}"
ptc="${ptc:-0}"; pc="${pc:-0}"; cc="${cc:-0}"; wc="${wc:-0}"
swsp_info "branch counts: parent-tool-call=$ptc parent-hold=$pc child=$cc wake=$wc"
if [ "$ptc" -lt 1 ] || [ "$pc" -lt 1 ] || [ "$cc" -lt 1 ]; then
printf 'RESULT=HARNESS_ERROR branch_counts parent-tool-call=%s parent-hold=%s child=%s wake=%s\n' \
"$ptc" "$pc" "$cc" "$wc"
return 1
fi
return 0
}
# ---- self-test ---------------------------------------------------------------
swsp_self_test() {
swsp_info "running self-test..."
local fails=0
# Deps
oqa_require opencode sqlite3 curl jq bun || { swsp_log "FAIL: missing dependencies"; fails=$((fails+1)); }
# Start fake-LLM
local st_log
st_log="$(mktemp -t swsp-st-llm.XXXXXX)"
OQA_TMPDIRS+=("$st_log")
if ! swsp_start_fake_llm "$st_log"; then
swsp_log "FAIL: fake-LLM did not start (port=${FAKE_OPENAI_PORT:-dynamic})"
fails=$((fails+1))
else
swsp_info "fake-LLM started on port $FAKE_SERVER_PORT"
# Health check
if curl -sf "http://127.0.0.1:${FAKE_SERVER_PORT}/health" >/dev/null 2>&1; then
swsp_info "PASS: fake-LLM /health 200"
else
swsp_log "FAIL: fake-LLM /health did not return 200"
fails=$((fails+1))
fi
fi
local missing_expect_out missing_expect_err missing_evidence_out missing_evidence_err
missing_expect_out="$(mktemp -t swsp-missing-expect-out.XXXXXX)"
missing_expect_err="$(mktemp -t swsp-missing-expect-err.XXXXXX)"
missing_evidence_out="$(mktemp -t swsp-missing-evidence-out.XXXXXX)"
missing_evidence_err="$(mktemp -t swsp-missing-evidence-err.XXXXXX)"
OQA_TMPDIRS+=("$missing_expect_out" "$missing_expect_err" "$missing_evidence_out" "$missing_evidence_err")
if bash "${BASH_SOURCE[0]}" --expect >"$missing_expect_out" 2>"$missing_expect_err"; then
swsp_log "FAIL: missing --expect operand unexpectedly succeeded"
fails=$((fails+1))
elif grep -q "error: --expect requires reproduced or fixed" "$missing_expect_err"; then
swsp_info "PASS: missing --expect operand fails with usage error"
else
swsp_log "FAIL: missing --expect operand did not emit usage error"
fails=$((fails+1))
fi
if bash "${BASH_SOURCE[0]}" --evidence-dir --self-test >"$missing_evidence_out" 2>"$missing_evidence_err"; then
swsp_log "FAIL: missing --evidence-dir operand unexpectedly succeeded"
fails=$((fails+1))
elif grep -q "error: --evidence-dir requires a directory" "$missing_evidence_err"; then
swsp_info "PASS: missing --evidence-dir operand fails with usage error"
else
swsp_log "FAIL: missing --evidence-dir operand did not emit usage error"
fails=$((fails+1))
fi
# Sandbox + opencode serve
if ! oqa_start_server; then
swsp_log "FAIL: opencode serve did not start"
swsp_stop_fake_llm
fails=$((fails+1))
else
swsp_info "PASS: opencode serve started at $OQA_SERVER_URL"
# /global/health check
local health_code
health_code="$(curl -so /dev/null -w "%{http_code}" -u "opencode:${OQA_SERVER_PASS}" \
"${OQA_SERVER_URL}/global/health" 2>/dev/null)" || true
if [ "$health_code" = "200" ]; then
swsp_info "PASS: /global/health 200"
else
swsp_log "FAIL: /global/health returned $health_code"
fails=$((fails+1))
fi
# OMO_SANDBOX_OMO_CONFIG env contract assertion: merge keeps base overrides
local omo_cfg_path="$XDG_CONFIG_HOME/opencode/oh-my-openagent.json"
OMO_SANDBOX_OMO_CONFIG='{"_probe":true}' swsp_write_omo_config "$XDG_CONFIG_HOME"
local probe_val explore_model
probe_val="$(jq -r '._probe' "$omo_cfg_path" 2>/dev/null)"
explore_model="$(jq -r '.agents.explore.model' "$omo_cfg_path" 2>/dev/null)"
if [ "$probe_val" = "true" ] && [ "$explore_model" = "openai/gpt-fake" ]; then
swsp_info "PASS: OMO_SANDBOX_OMO_CONFIG merge assertion (env key + base overrides both present)"
else
swsp_log "FAIL: omo config merge wrong: _probe='$probe_val' explore_model='$explore_model'"
fails=$((fails+1))
fi
fi
local route_fixture
route_fixture='[2026-06-19T00:00:00.000Z] [live-server-route] dispatch via live listener {"sessionID":"ses_other","source":"background-agent-parent-wake"}
[2026-06-19T00:00:01.000Z] [live-server-route] dispatch via live listener {"sessionID":"ses_probe","source":"background-agent-parent-wake"}'
if swsp_has_session_live_dispatch "$route_fixture" "ses_probe" \
&& ! swsp_has_session_live_dispatch "$route_fixture" "ses_missing"; then
swsp_info "PASS: live dispatch detection is scoped to the probe session"
else
swsp_log "FAIL: live dispatch detection accepted an unrelated session"
fails=$((fails+1))
fi
local branch_log
branch_log="$(mktemp -t swsp-branch-log.XXXXXX)"
OQA_TMPDIRS+=("$branch_log")
{
printf '[2026-06-19T00:00:00.000Z] branch=parent-tool-call\n'
printf '[2026-06-19T00:00:01.000Z] branch=parent-hold\n'
printf '[2026-06-19T00:00:02.000Z] branch=child\n'
} >"$branch_log"
if swsp_check_branch_counts "$branch_log" reproduced >/dev/null 2>&1; then
swsp_info "PASS: reproduced branch guard accepts mechanism-only evidence"
else
swsp_log "FAIL: reproduced branch guard still requires wake branch"
fails=$((fails+1))
fi
if swsp_fixed_topology_observed 3 2 1 1 1 1 1 0 1 true; then
swsp_info "PASS: fixed topology accepts scoped live dispatch plus deterministic DB/provider evidence"
else
swsp_log "FAIL: fixed topology rejected scoped live dispatch plus deterministic DB/provider evidence"
fails=$((fails+1))
fi
if swsp_fixed_topology_observed 3 2 1 1 1 1 1 0 1 false; then
swsp_log "FAIL: fixed topology accepted missing scoped live dispatch"
fails=$((fails+1))
else
swsp_info "PASS: fixed topology rejects missing scoped live dispatch"
fi
if swsp_fixed_topology_observed bad 2 1 1 1 1 1 0 1 true; then
swsp_log "FAIL: fixed topology accepted malformed numeric evidence"
fails=$((fails+1))
else
swsp_info "PASS: fixed topology rejects malformed numeric evidence"
fi
if swsp_fixed_topology_observed 3 2 2 1 1 1 1 0 1 true; then
swsp_log "FAIL: fixed topology accepted duplicate terminal stop"
fails=$((fails+1))
else
swsp_info "PASS: fixed topology rejects duplicate terminal stop"
fi
local stale_log stale_scoped stale_all
stale_log="$(mktemp -t swsp-stale-log.XXXXXX)"
stale_scoped="$(mktemp -t swsp-stale-scoped.XXXXXX)"
stale_all="$(mktemp -t swsp-stale-all.XXXXXX)"
OQA_TMPDIRS+=("$stale_log" "$stale_scoped" "$stale_all")
printf '[2026-06-19T00:00:00.000Z] [live-server-route] dispatch via live listener {"sessionID":"ses_unrelated","source":"background-agent-parent-wake"}\n' >"$stale_log"
swsp_collect_route_provenance 999999 "$stale_log" "/probe" "ses_probe" "$stale_scoped" "$stale_all" 0 || true
if [ ! -s "$stale_scoped" ] && [ ! -s "$stale_all" ]; then
swsp_info "PASS: stale log offset does not persist unrelated route provenance"
else
swsp_log "FAIL: stale log offset persisted unrelated route provenance"
fails=$((fails+1))
fi
local scoped_log scoped_out scoped_all
scoped_log="$(mktemp -t swsp-scoped-log.XXXXXX)"
scoped_out="$(mktemp -t swsp-scoped-out.XXXXXX)"
scoped_all="$(mktemp -t swsp-scoped-all.XXXXXX)"
OQA_TMPDIRS+=("$scoped_log" "$scoped_out" "$scoped_all")
{
printf '[2026-06-19T00:00:00.000Z] [live-server-route] dispatch via live listener {"sessionID":"ses_other","source":"background-agent-parent-wake"}\n'
printf '[2026-06-19T00:00:01.000Z] [live-server-route] dispatch via live listener {"sessionID":"ses_probe","source":"background-agent-parent-wake"}\n'
} >"$scoped_log"
swsp_collect_route_provenance 0 "$scoped_log" "/probe" "ses_probe" "$scoped_out" "$scoped_all" 0 || true
if grep -q "ses_probe" "$scoped_all" && ! grep -q "ses_other" "$scoped_all"; then
swsp_info "PASS: route provenance artifact excludes unrelated sessions"
else
swsp_log "FAIL: route provenance artifact included unrelated sessions"
fails=$((fails+1))
fi
local route_wait_log route_wait_scoped route_wait_all
route_wait_log="$(mktemp -t swsp-route-wait-log.XXXXXX)"
route_wait_scoped="$(mktemp -t swsp-route-wait-scoped.XXXXXX)"
route_wait_all="$(mktemp -t swsp-route-wait-all.XXXXXX)"
OQA_TMPDIRS+=("$route_wait_log" "$route_wait_scoped" "$route_wait_all")
printf '[2026-06-19T00:00:00.000Z] [live-server-route] registered {"directory":"/probe","hasServerUrl":true}\n' >"$route_wait_log"
(
sleep 0.5
printf '[2026-06-19T00:00:01.000Z] [live-server-route] dispatch via live listener {"sessionID":"ses_wait","source":"background-agent-parent-wake"}\n' >>"$route_wait_log"
) &
local route_wait_pid=$!
if swsp_collect_route_provenance 0 "$route_wait_log" "/probe" "ses_wait" "$route_wait_scoped" "$route_wait_all" 3; then
swsp_info "PASS: route provenance waits for delayed session dispatch"
else
swsp_log "FAIL: route provenance did not wait for delayed session dispatch"
fails=$((fails+1))
fi
wait "$route_wait_pid" 2>/dev/null || true
local st_fake_pid="$FAKE_SERVER_PID"
swsp_stop_fake_llm
if [ -n "$st_fake_pid" ] && ! kill -0 "$st_fake_pid" 2>/dev/null; then
swsp_info "PASS: fake-openai server process stopped"
else
swsp_log "FAIL: fake-openai server process still running"
fails=$((fails+1))
fi
if [ "$fails" -eq 0 ]; then
printf 'SELF-TEST OK\n'
return 0
fi
printf 'SELF-TEST FAILED (%d failure(s))\n' "$fails" >&2
return 1
}
# ---- main probe run ----------------------------------------------------------
swsp_run_probe() {
local evidence_dir="${EVIDENCE_DIR:-$(mktemp -d -t swsp-evidence.XXXXXX)}"
mkdir -p "$evidence_dir"
OQA_TMPDIRS+=("$evidence_dir") 2>/dev/null || true # only auto-clean if we created it
# Override: if caller gave --evidence-dir, don't delete it
if [ -n "$EVIDENCE_DIR" ]; then
# Remove from cleanup list (last element we added)
unset 'OQA_TMPDIRS[${#OQA_TMPDIRS[@]}-1]' 2>/dev/null || true
fi
swsp_info "evidence dir: $evidence_dir"
local fake_llm_log="$evidence_dir/fake-llm.log"
local harness_log="$evidence_dir/harness.log"
local serve_stdout="$evidence_dir/opencode-serve.stdout"
local serve_stderr="$evidence_dir/opencode-serve.stderr"
# Step 1: Record real-DB session count (read-only)
local real_db_path real_db_count_before
real_db_path="$(opencode db path 2>/dev/null | head -1 || echo "")"
if [ -n "$real_db_path" ] && [ -f "$real_db_path" ]; then
real_db_count_before="$(sqlite3 "$real_db_path" 'SELECT count(*) FROM session' 2>/dev/null || echo "0")"
else
real_db_count_before="0"
real_db_path="(not found)"
fi
swsp_info "real DB session count before: $real_db_count_before"
printf 'real_db=%s before=%s\n' "$real_db_path" "$real_db_count_before" >"$evidence_dir/isolation-receipt.txt"
# Capture omo log byte offset
local omo_log="${TMPDIR:-/tmp}/oh-my-opencode.log"
local omo_log_offset
if [ -f "$omo_log" ]; then
omo_log_offset="$(wc -c <"$omo_log" 2>/dev/null | tr -d ' ')" || omo_log_offset=0
else
omo_log_offset=0
fi
# Step 2: Start fake-openai server
swsp_info "starting fake-openai server..."
if ! swsp_start_fake_llm "$fake_llm_log"; then
swsp_log "HARNESS_ERROR: fake-openai server failed to start"
printf 'RESULT=HARNESS_ERROR fake_llm_start_failed\n' | tee -a "$harness_log"
return 1
fi
swsp_info "fake-openai on port $FAKE_SERVER_PORT"
printf 'fake_llm_port=%s\n' "$FAKE_SERVER_PORT" >>"$evidence_dir/isolation-receipt.txt"
# Step 3: Create isolated sandbox and write opencode.jsonc
# oqa_mk_isolated_xdg sets XDG_CONFIG_HOME, XDG_DATA_HOME, etc.
oqa_mk_isolated_xdg
swsp_info "sandbox: $OQA_XDG_ROOT"
swsp_write_omo_config "$XDG_CONFIG_HOME"
swsp_write_opencode_config "$XDG_CONFIG_HOME" "$FAKE_SERVER_PORT"
local sandbox_db="$XDG_DATA_HOME/opencode/opencode.db"
# Step 4: Start opencode serve (using oqa_start_server internals but with our config)
# oqa_start_server includes another oqa_mk_isolated_xdg call which would reset XDG vars.
# Instead, start server directly using the already-set XDG vars.
swsp_info "starting opencode serve..."
local port pass
port="$(oqa_free_port)"
pass="oqa-${RANDOM}${RANDOM}"
OPENCODE_SERVER_PASSWORD="$pass" opencode serve --port "$port" --hostname 127.0.0.1 \
>"$serve_stdout" 2>"$serve_stderr" &
OQA_SERVER_PID=$!
disown "$OQA_SERVER_PID" 2>/dev/null || true
export OQA_SERVER_PORT="$port"
export OQA_SERVER_PASS="$pass"
export OQA_SERVER_URL="http://127.0.0.1:$port"
if ! oqa_wait_http "$OQA_SERVER_URL/global/health" "opencode:$pass" 30; then
swsp_log "HARNESS_ERROR: opencode serve failed to start"
cat "$serve_stderr" >&2 2>/dev/null || true
printf 'RESULT=HARNESS_ERROR opencode_serve_start_failed\n' | tee -a "$harness_log"
swsp_stop_fake_llm
return 1
fi
swsp_info "opencode serve ready at $OQA_SERVER_URL"
# Encode the working directory for use in URL
local enc_dir
enc_dir="$(python3 -c 'import urllib.parse,sys;print(urllib.parse.quote(sys.argv[1],safe=""))' "$OQA_PROJ" 2>/dev/null \
|| printf '%s' "$OQA_PROJ" | sed 's|/|%2F|g')"
# Step 5: Create a session
swsp_info "creating session..."
local ses_response ses_id
ses_response="$(curl -sS -u "opencode:${pass}" \
-X POST "${OQA_SERVER_URL}/session?directory=${enc_dir}" \
-H 'content-type: application/json' \
-d '{"title":"wake split probe"}' 2>/dev/null)" || ses_response=""
ses_id="$(printf '%s' "$ses_response" | jq -r '.id // .sessionID // empty' 2>/dev/null)" || ses_id=""
if [ -z "$ses_id" ]; then
swsp_log "HARNESS_ERROR: could not create session (response: $ses_response)"
printf 'RESULT=HARNESS_ERROR session_create_failed\n' | tee -a "$harness_log"
swsp_stop_fake_llm
return 1
fi
swsp_info "session: $ses_id"
printf 'session_id=%s\n' "$ses_id" >>"$evidence_dir/isolation-receipt.txt"
# Step 6: Send the split probe prompt
swsp_info "sending split probe prompt..."
local prompt_response
prompt_response="$(curl -sS -u "opencode:${pass}" \
-X POST "${OQA_SERVER_URL}/session/${ses_id}/prompt_async?directory=${enc_dir}" \
-H 'content-type: application/json' \
-d '{"parts":[{"type":"text","text":"Run the split probe: call call_omo_agent exactly once as instructed, then run the bash hold command."}]}' \
2>/dev/null)" || prompt_response=""
swsp_info "prompt_async response: $prompt_response"
swsp_info "polling DB for wake-split metrics (up to 120s)..."
local metrics
metrics="$(swsp_poll_db_metrics "$sandbox_db" '%Run the split probe:%' 120)"
local parent_assistant_messages parent_tool_call_turns terminal_stops child_task_sessions
parent_assistant_messages="$(printf '%s' "$metrics" | awk '{print $1}')"
parent_tool_call_turns="$(printf '%s' "$metrics" | awk '{print $2}')"
terminal_stops="$(printf '%s' "$metrics" | awk '{print $3}')"
child_task_sessions="$(printf '%s' "$metrics" | awk '{print $4}')"
parent_assistant_messages="${parent_assistant_messages:-0}"
parent_tool_call_turns="${parent_tool_call_turns:-0}"
terminal_stops="${terminal_stops:-0}"
child_task_sessions="${child_task_sessions:-0}"
swsp_info "DB metrics: parent_assistant_messages=$parent_assistant_messages parent_tool_call_turns=$parent_tool_call_turns terminal_stops=$terminal_stops child_task_sessions=$child_task_sessions"
# Wait for parent session to go idle
swsp_info "waiting for parent session to go idle..."
swsp_wait_session_idle "$OQA_SERVER_URL" "$pass" "$ses_id" 60
# Re-read metrics after idle
metrics="$(swsp_poll_db_metrics "$sandbox_db" '%Run the split probe:%' 10)"
parent_assistant_messages="$(printf '%s' "$metrics" | awk '{print $1}')"
parent_tool_call_turns="$(printf '%s' "$metrics" | awk '{print $2}')"
terminal_stops="$(printf '%s' "$metrics" | awk '{print $3}')"
child_task_sessions="$(printf '%s' "$metrics" | awk '{print $4}')"
parent_assistant_messages="${parent_assistant_messages:-0}"
parent_tool_call_turns="${parent_tool_call_turns:-0}"
terminal_stops="${terminal_stops:-0}"
child_task_sessions="${child_task_sessions:-0}"
swsp_info "final DB metrics: parent_assistant_messages=$parent_assistant_messages parent_tool_call_turns=$parent_tool_call_turns terminal_stops=$terminal_stops child_task_sessions=$child_task_sessions"
printf 'parent_assistant_messages=%s parent_tool_call_turns=%s terminal_stops=%s child_task_sessions=%s\n' \
"$parent_assistant_messages" "$parent_tool_call_turns" "$terminal_stops" "$child_task_sessions" \
>"$evidence_dir/marker-metrics.txt"
# Step 8: Plugin-init count
local plugin_inits
plugin_inits="$(swsp_count_plugin_inits "$omo_log_offset" "$OQA_PROJ")"
plugin_inits="${plugin_inits:-0}"
swsp_info "plugin_inits: $plugin_inits"
printf '%s\n' "$plugin_inits" >"$evidence_dir/plugin-init-count.txt"
# Step 9: Route provenance
local route_prov=""
swsp_collect_route_provenance \
"$omo_log_offset" \
"$omo_log" \
"$OQA_PROJ" \
"$ses_id" \
"$evidence_dir/route-provenance.log" \
"$evidence_dir/route-provenance-all.log" \
10 || true
route_prov="$(cat "$evidence_dir/route-provenance.log" 2>/dev/null || true)"
swsp_info "route-provenance lines: $(printf '%s' "$route_prov" | wc -l | tr -d ' ')"
# WAKE_DISPATCHED_DURING_PARENT_TURN mechanism signal
local wake_during_parent
wake_during_parent="$(swsp_detect_wake_during_parent "$omo_log_offset" "$fake_llm_log" "$OQA_PROJ")"
swsp_info "WAKE_DISPATCHED_DURING_PARENT_TURN=$wake_during_parent"
local real_db_count_after=""
if [ -n "$real_db_path" ] && [ "$real_db_path" != "(not found)" ] && [ -f "$real_db_path" ]; then
real_db_count_after="$(sqlite3 "$real_db_path" 'SELECT count(*) FROM session' 2>/dev/null || echo "0")"
else
real_db_count_after="$real_db_count_before"
fi
printf 'after=%s unchanged=%s\n' \
"$real_db_count_after" \
"$([ "$real_db_count_after" = "$real_db_count_before" ] && echo yes || echo NO)" \
>>"$evidence_dir/isolation-receipt.txt"
swsp_info "isolation: real DB before=$real_db_count_before after=$real_db_count_after"
sqlite3 "$sandbox_db" ".backup '$evidence_dir/sandbox-opencode.db'" 2>/dev/null || true
# Step 10: Branch-count guard
if ! swsp_check_branch_counts "$fake_llm_log" "$EXPECT_MODE" >&2; then
# Branch counts not met — HARNESS_ERROR
local ptc pc cc wc
ptc="$(grep -c "branch=parent-tool-call" "$fake_llm_log" 2>/dev/null || true)"; ptc="${ptc:-0}"
pc="$(grep -c "branch=parent-hold" "$fake_llm_log" 2>/dev/null || true)"; pc="${pc:-0}"
cc="$(grep -c "branch=child" "$fake_llm_log" 2>/dev/null || true)"; cc="${cc:-0}"
wc="$(grep -c "branch=wake" "$fake_llm_log" 2>/dev/null || true)"; wc="${wc:-0}"
local verdict_line
verdict_line="RESULT=HARNESS_ERROR parent_assistant_messages=${parent_assistant_messages} parent_tool_call_turns=${parent_tool_call_turns} terminal_stops=${terminal_stops} child_task_sessions=${child_task_sessions} plugin_inits=${plugin_inits} WAKE_DISPATCHED_DURING_PARENT_TURN=${wake_during_parent} branch_counts=parent-tool-call:${ptc},parent-hold:${pc},child:${cc},wake:${wc}"
printf '%s\n' "$verdict_line" | tee -a "$harness_log"
swsp_stop_fake_llm
return 1
fi
# Step 11: Determine verdict
local result="INCONCLUSIVE"
local exit_code=1
local ptc pc cc wc
ptc="$(grep -c "branch=parent-tool-call" "$fake_llm_log" 2>/dev/null || true)"; ptc="${ptc:-0}"
pc="$(grep -c "branch=parent-hold" "$fake_llm_log" 2>/dev/null || true)"; pc="${pc:-0}"
cc="$(grep -c "branch=child" "$fake_llm_log" 2>/dev/null || true)"; cc="${cc:-0}"
wc="$(grep -c "branch=wake" "$fake_llm_log" 2>/dev/null || true)"; wc="${wc:-0}"
local dc
dc="$(grep -c "branch=default" "$fake_llm_log" 2>/dev/null || true)"; dc="${dc:-0}"
if [ "${terminal_stops:-0}" -gt 1 ] || [ "${child_task_sessions:-0}" -gt 1 ] 2>/dev/null; then
result="REPRODUCED"
fi
# Arm 2: Mechanism signal (wake dispatched during parent turn + in-process path)
# in-process path: no live-server-route dispatch line for this wake
local has_live_dispatch=false
if swsp_has_session_live_dispatch "$route_prov" "$ses_id"; then
has_live_dispatch=true
fi
if [ "$wake_during_parent" = "true" ] && [ "$has_live_dispatch" = "false" ]; then
result="REPRODUCED"
fi
if [ "$result" = "INCONCLUSIVE" ] \
&& swsp_fixed_topology_observed \
"$parent_assistant_messages" \
"$parent_tool_call_turns" \
"$terminal_stops" \
"$child_task_sessions" \
"$ptc" \
"$pc" \
"$cc" \
"$wc" \
"$dc" \
"$has_live_dispatch"; then
result="FIXED"
fi
local verdict_line
verdict_line="RESULT=${result} parent_assistant_messages=${parent_assistant_messages} parent_tool_call_turns=${parent_tool_call_turns} terminal_stops=${terminal_stops} child_task_sessions=${child_task_sessions} plugin_inits=${plugin_inits} WAKE_DISPATCHED_DURING_PARENT_TURN=${wake_during_parent} route_live_dispatch=${has_live_dispatch} branch_counts=parent-tool-call:${ptc},parent-hold:${pc},child:${cc},wake:${wc},default:${dc}"
printf '%s\n' "$verdict_line" | tee -a "$harness_log"
# Determine exit code based on expected mode
if [ -n "$EXPECT_MODE" ]; then
if [ "$EXPECT_MODE" = "reproduced" ] && [ "$result" = "REPRODUCED" ]; then
exit_code=0
elif [ "$EXPECT_MODE" = "fixed" ] && [ "$result" = "FIXED" ]; then
exit_code=0
else
exit_code=1
fi
else
exit_code=0 # no expectation: just report
fi
# Cleanup receipt
local stopped_fake_pid="$FAKE_SERVER_PID"
swsp_stop_fake_llm
printf 'fake_llm=stopped opencode_serve=stopping\n' >"$evidence_dir/cleanup-receipt.txt"
if [ -n "$stopped_fake_pid" ] && kill -0 "$stopped_fake_pid" 2>/dev/null; then
swsp_log "WARNING: fake-openai server process $stopped_fake_pid still running after cleanup"
printf 'fake_llm_pid_alive=yes pid=%s\n' "$stopped_fake_pid" >>"$evidence_dir/cleanup-receipt.txt"
else
printf 'fake_llm_pid_alive=no pid=%s\n' "$stopped_fake_pid" >>"$evidence_dir/cleanup-receipt.txt"
fi
return "$exit_code"
}
# ---- dispatch ----------------------------------------------------------------
if [ "$SELF_TEST" -eq 1 ]; then
swsp_self_test
exit $?
fi
swsp_run_probe
exit $?
#!/usr/bin/env bash
# server-smoke.sh - boot an ISOLATED opencode HTTP server and verify the core
# API surface end to end. Uses an isolated XDG sandbox + a random password, so
# it never touches the real ~/.local/share/opencode DB, and tears the server
# down on exit.
#
# Checks:
# 1. GET /global/health -> {"healthy":true,"version":...}
# 2. GET /doc -> OpenAPI spec with >=100 paths
# 3. GET /session (no credentials) -> HTTP 401 (auth is enforced)
#
# Usage:
# server-smoke.sh # run the smoke test
# server-smoke.sh --self-test # same thing (alias for the QA sweep)
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
. "$SCRIPT_DIR/lib/common.sh"
oqa_server_smoke() {
oqa_require opencode curl jq || return 1
if ! oqa_start_server; then
oqa_log "FAIL: server did not become ready"; return 1
fi
local url="$OQA_SERVER_URL" auth="opencode:$OQA_SERVER_PASS" fails=0
local healthy version
healthy="$(curl -s -u "$auth" "$url/global/health" | jq -r '.healthy // false')"
version="$(curl -s -u "$auth" "$url/global/health" | jq -r '.version // "?"')"
if [ "$healthy" = "true" ]; then
oqa_pass "GET /global/health healthy=true version=$version ($url)"
else
oqa_log "FAIL: /global/health healthy=$healthy"; fails=$((fails+1))
fi
local npaths
npaths="$(curl -s -u "$auth" "$url/doc" | jq '.paths | length' 2>/dev/null)"
if [ "${npaths:-0}" -ge 100 ]; then
oqa_pass "GET /doc lists $npaths documented paths (>=100)"
else
oqa_log "FAIL: /doc path count=$npaths"; fails=$((fails+1))
fi
local code
code="$(curl -s -o /dev/null -w '%{http_code}' "$url/session?directory=$OQA_PROJ")"
if [ "$code" = "401" ]; then
oqa_pass "unauthenticated GET /session rejected with HTTP 401"
else
oqa_log "FAIL: unauthenticated /session returned $code (expected 401)"; fails=$((fails+1))
fi
if [ "$fails" -eq 0 ]; then
oqa_pass "server-smoke"
return 0
fi
oqa_log "server-smoke had $fails failure(s)"; return 1
}
case "${1:-}" in
-h|--help)
sed -n '2,18p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'
exit 0 ;;
*)
oqa_server_smoke; exit $? ;;
esac
#!/usr/bin/env bash
# sse-hook-probe.sh - QA opencode's event stream (the plumbing behind hooks).
#
# opencode publishes lifecycle events over Server-Sent Events at GET /event
# (per-instance) and GET /global/event. Plugins observe the same events via the
# `event` hook, so confirming an event on the wire is how you prove a hook
# would have fired.
#
# Two modes:
# (default / --self-test) Spawn an ISOLATED server and assert the stream
# opens with a `server.connected` event. No real DB
# is touched. This proves the SSE plumbing works.
# --attach <url> Watch an ALREADY-RUNNING server's /event stream for
# a specific event type (default: server.connected).
# Use this against your real server to verify a hook
# or action. Pair it with a prompt in another shell:
# curl -X POST -u opencode:$PASS \
# -H 'Content-Type: application/json' \
# -d '{"parts":[{"type":"text","text":"hi"}]}' \
# "<url>/session/<ses_id>/prompt_async?directory=<dir>"
# then watch for e.g. message.part.updated.
#
# --attach options:
# --password <p> server password (user defaults to "opencode")
# --user <u> server username (default: opencode)
# --directory <d> instance directory (default: $PWD)
# --event <type> event type to wait for (default: server.connected)
# --timeout <s> seconds to wait (default: 15)
#
# Usage:
# sse-hook-probe.sh --self-test
# sse-hook-probe.sh --attach http://127.0.0.1:4096 --password secret \
# --directory "$PWD" --event message.part.updated --timeout 30
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
. "$SCRIPT_DIR/lib/common.sh"
# Watch an SSE stream for an event type. Args: url auth directory event timeout
# Returns 0 if seen, 1 otherwise. Always kills its curl watcher.
oqa_sse_watch() {
local url="$1" auth="$2" dir="$3" want="$4" timeout="${5:-15}"
local out cpid found="" deadline
out="$(mktemp -t oqa-sse.XXXXXX)"; OQA_TMPDIRS+=("$out")
if [ -n "$auth" ]; then
curl -sN -u "$auth" "$url/event?directory=$dir" >"$out" 2>/dev/null &
else
curl -sN "$url/event?directory=$dir" >"$out" 2>/dev/null &
fi
cpid=$!; OQA_CURL_PIDS+=("$cpid")
disown "$cpid" 2>/dev/null || true
deadline=$(( $(date +%s) + timeout ))
while [ "$(date +%s)" -lt "$deadline" ]; do
if grep -q "\"$want\"" "$out" 2>/dev/null; then found=1; break; fi
kill -0 "$cpid" 2>/dev/null || break
sleep 0.2
done
kill "$cpid" 2>/dev/null || true
sleep 0.1
kill -0 "$cpid" 2>/dev/null && kill -9 "$cpid" 2>/dev/null || true
if [ -n "$found" ]; then
printf 'first matching event: '
grep -m1 "\"$want\"" "$out" | sed 's/^data: //' | jq -c '{type: .type}' 2>/dev/null || true
return 0
fi
oqa_log "stream head (no '$want' within ${timeout}s):"; head -5 "$out" >&2
return 1
}
oqa_self_test() {
oqa_require opencode curl jq || return 1
if ! oqa_start_server; then oqa_log "FAIL: server did not start"; return 1; fi
if oqa_sse_watch "$OQA_SERVER_URL" "opencode:$OQA_SERVER_PASS" "$OQA_PROJ" "server.connected" 15; then
oqa_pass "SSE /event opened and delivered server.connected"
return 0
fi
oqa_log "FAIL: did not observe server.connected"; return 1
}
oqa_attach_mode() {
local url="" user="opencode" pass="" dir="$PWD" event="server.connected" timeout=15
shift # drop --attach
url="$1"; shift || true
while [ $# -gt 0 ]; do
case "$1" in
--password) pass="$2"; shift 2 ;;
--user) user="$2"; shift 2 ;;
--directory) dir="$2"; shift 2 ;;
--event) event="$2"; shift 2 ;;
--timeout) timeout="$2"; shift 2 ;;
*) oqa_log "unknown option: $1"; shift ;;
esac
done
[ -n "$url" ] || { oqa_log "error: --attach requires a URL"; return 2; }
local auth=""; [ -n "$pass" ] && auth="$user:$pass"
oqa_log "watching $url/event?directory=$dir for '$event' (<=${timeout}s)"
if oqa_sse_watch "$url" "$auth" "$dir" "$event" "$timeout"; then
oqa_pass "observed '$event' on $url"
return 0
fi
oqa_log "FAIL: '$event' not observed"; return 1
}
case "${1:-}" in
--attach) oqa_attach_mode "$@"; exit $? ;;
-h|--help)
sed -n '2,34p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'
exit 0 ;;
*) oqa_self_test; exit $? ;;
esac
#!/usr/bin/env bash
# tui-smoke.sh - launch the opencode TUI under tmux in an ISOLATED sandbox,
# confirm it renders, prove send-keys reaches the composer, then tear down.
#
# This is a feasibility/smoke check, NOT a functional assertion harness. The
# TUI is a 60fps full-screen app; reading its frame is fine for "did it boot
# and accept a keystroke", but brittle for asserting conversation output. For
# real behavior assertions use `opencode run` (Case A) or the server API /
# SSE probe (Case B). See references/tui-tmux.md.
#
# Safety: runs opencode under an isolated XDG sandbox so no session is written
# to the real ~/.local/share/opencode DB; the tmux session is always killed.
#
# Usage:
# tui-smoke.sh # run the smoke test
# tui-smoke.sh --self-test # same
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
. "$SCRIPT_DIR/lib/common.sh"
oqa_tui_smoke() {
oqa_require opencode tmux jq sqlite3 || return 1
local before after realdb ver sess cap found="" i fails=0
# Capture the REAL DB path + count BEFORE isolation. We must read it with
# sqlite3 directly: once oqa_mk_isolated_xdg exports XDG_DATA_HOME, `opencode
# db` would resolve the empty sandbox DB instead of the real one.
realdb="$(oqa_db_path)"
before="$(sqlite3 "$realdb" "SELECT count(*) FROM session" 2>/dev/null)"
ver="$(opencode --version 2>/dev/null | head -1 | tr -d '[:space:]')"
oqa_mk_isolated_xdg
sess="oqa_tui_${$}_${RANDOM}"
OQA_TMUX_SESSIONS+=("$sess")
tmux new-session -d -s "$sess" -x 200 -y 50
# launch the TUI inside the pane, carrying the isolated sandbox env
tmux send-keys -t "$sess" \
"HOME='$HOME' OPENCODE_TEST_HOME='$OPENCODE_TEST_HOME' XDG_DATA_HOME='$XDG_DATA_HOME' XDG_CONFIG_HOME='$XDG_CONFIG_HOME' XDG_CACHE_HOME='$XDG_CACHE_HOME' XDG_STATE_HOME='$XDG_STATE_HOME' OPENCODE_DISABLE_AUTOUPDATE=1 OPENCODE_DISABLE_MODELS_FETCH=1 opencode '$OQA_PROJ'" Enter
# poll for a render marker (version string, composer placeholder, or footer)
for ((i=0; i<50; i++)); do
cap="$(tmux capture-pane -t "$sess" -p 2>/dev/null)"
if printf '%s' "$cap" | grep -Eq "${ver}|Ask anything|ctrl\+p|agents"; then found=1; break; fi
sleep 0.5
done
if [ -n "$found" ]; then
oqa_pass "TUI rendered under tmux (marker found; version ${ver:-?})"
else
oqa_log "FAIL: TUI did not render a known marker in 25s; pane was:"; printf '%s\n' "$cap" | head -8 >&2
fails=$((fails+1))
fi
# prove send-keys reaches the composer: type a sentinel, expect it on screen
if [ -n "$found" ]; then
tmux send-keys -t "$sess" "oqaXYZ"
sleep 1
cap="$(tmux capture-pane -t "$sess" -p 2>/dev/null)"
if printf '%s' "$cap" | grep -q "oqaXYZ"; then
oqa_pass "send-keys reached the TUI composer (sentinel echoed)"
else
oqa_log "WARN: sentinel not visible (TUI may have remapped input); render still proven"
fi
fi
# teardown + verify
tmux kill-session -t "$sess" 2>/dev/null || true
sleep 0.5
if tmux has-session -t "$sess" 2>/dev/null; then
oqa_log "FAIL: tmux session survived teardown"; fails=$((fails+1))
else
oqa_pass "tmux session torn down (has-session false)"
fi
after="$(sqlite3 "$realdb" "SELECT count(*) FROM session" 2>/dev/null)"
if [ "$before" = "$after" ]; then
oqa_pass "real DB untouched (session count $before unchanged)"
else
oqa_log "FAIL: real DB session count changed $before -> $after"; fails=$((fails+1))
fi
[ "$fails" -eq 0 ] && { oqa_pass "tui-smoke"; return 0; }
oqa_log "tui-smoke had $fails failure(s)"; return 1
}
case "${1:-}" in
-h|--help)
sed -n '2,20p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'
exit 0 ;;
*) oqa_tui_smoke; exit $? ;;
esac