Agent Tui

agent-tui

Quick start

• Verify install: agent-tui --version
• Install with npm i -g agent-tui (or pnpm add -g agent-tui, bun add -g agent-tui).
• Alternate install: curl -fsSL https://raw.githubusercontent.com/pproenca/agent-tui/master/install.sh -o /tmp/agent-tui-install.sh && sh /tmp/agent-tui-install.sh or cargo install --git https://github.com/pproenca/agent-tui.git --path cli/crates/agent-tui.
• If you used the install script, ensure ~/.local/bin is on your PATH.
• Start a session: agent-tui run --format json <command> -- <args...>
• Observe: agent-tui screenshot --format json
• Act: agent-tui press Enter or agent-tui type "text"
• Wait or verify: agent-tui wait "Expected text" --assert or agent-tui wait --stable
• Cleanup: agent-tui kill

Core workflow

1. Run the app with agent-tui run and capture session_id from JSON output. 2. Take a fresh snapshot with agent-tui screenshot or agent-tui screenshot --format json. 3. Decide the next action based on the latest snapshot. 4. Act with press or type. 5. Synchronize with wait --assert or wait --stable. 6. Repeat from step 2 until the task finishes. 7. Clean up with agent-tui kill.

Reliability rules

• Re-snapshot after every action that could change the UI.
• Never act on a changing screen; wait for stability first.
• Verify outcomes with wait --assert instead of assuming success.
• Always end runs with kill or sessions cleanup.

Session handling

• Use --session <id> for every command if more than one session exists.
• If you lose the session id, run agent-tui sessions and agent-tui sessions show <id>.

Live preview (optional)

• Ask whether a live preview is desired.
• Start preview: agent-tui live start --open
• Stop preview when done: agent-tui live stop

Deep-dive references

• Full CLI coverage and options: references/command-atlas.md
• JSON output contract: references/output-contract.md
• End-to-end command sequences: references/flows.md
• Quick command selection: references/decision-tree.md
• Session lifecycle and concurrency: references/session-lifecycle.md
• Assertions and test oracles: references/assertions.md
• Failure recovery playbook: references/recovery.md
• Safety and confirmation prompts: references/safety.md
• Clarification checklist: references/clarifications.md
• Test plan template: references/test-plan.md
• Demo script: references/demo.md
• User prompt templates: references/prompt-templates.md
• Minimal command sets by use case: references/use-cases.md
• Explorer/replay automation skill: ../tui-explorer/SKILL.md

Assertions and Test Oracles

Use this file when translating requirements into checks.

Turn Requirements into Assertions

• Convert each requirement into observable UI state (text or stability).
• Prefer wait --assert for pass/fail semantics.
• Collect evidence with a final screenshot before cleanup.

Assertion Patterns

• Text presence: agent-tui wait "Expected" --assert
• Text gone: agent-tui wait "Expected" --gone --assert
• Stability: agent-tui wait --stable --assert

Validation Strategy

• For static UI: text snapshot + wait is enough.
• For dynamic UI: insert wait --stable before each action.

Reporting

• Report: scenario, actions taken, assertions run, and final evidence (screenshot output).
• If an assertion fails, include the last screenshot and the expected condition.

Clarification Checklist

Use this file when requirements are incomplete or ambiguous.

Ask Before Running

• App command, args, and working directory (what to run, where).
• Required environment variables or config files.
• Expected UI outcomes (texts, states, or tree hints to assert).
• Inputs and sequencing (keys, text, timing constraints).
• Stop condition (what indicates the test is done).
• Terminal size requirements (cols/rows) if layout-sensitive.
• Output format preference: JSON for automation or text for readability.

Live Preview (always ask)

• "Do you want a live preview while I run the test?"
• If yes: run agent-tui live start --open before agent-tui run ... and stop it afterward.

Safety and Data Handling

• Ask for credentials or test accounts if login is required; never guess secrets.
• Confirm if it is safe to execute actions that modify data (delete, submit, confirm).
• If the app can perform destructive actions, request a dry-run or a test environment.

Session Scope

• Ask whether multiple sessions will be running; if yes, plan to use --session <id>.

Command Atlas (Full)

Use this file when you need complete CLI coverage and exact options.

Global Flags

• --session <id>: target a specific session (default: most recent).
• --format <text|json>: output format.
• --json: shorthand for --format json.
• --no-color: disable color (also respects NO_COLOR).

Core Commands

Run

• agent-tui run <command> [-- args...]
• Options:
• -d, --cwd <dir>: working directory.
• --cols <n>: terminal columns (default 120).
• --rows <n>: terminal rows (default 40).

Screenshot

• agent-tui screenshot
• Options:
• --region <name>: limit capture to region (if supported).
• --strip-ansi: remove ANSI color codes.
• --include-cursor: include cursor position.

Resize / Restart

• agent-tui resize --cols <n> --rows <n>
• agent-tui restart

Press / Type

• agent-tui press <key...> [--hold|--release]
• agent-tui type "text"
• Keys: Enter, Tab, Escape, Backspace, Delete, Arrow keys, Home, End, PageUp, PageDown, F1-F12
• Modifiers: Ctrl+<key>, Alt+<key>, Shift+<key>

Wait

• agent-tui wait <text>
• agent-tui wait --stable
• Modifiers:
• -g, --gone: wait for text to disappear.
• -t, --timeout <ms>: timeout in milliseconds (default 30000).
• --assert: exit code 1 on timeout (0 on success).

Kill

• agent-tui kill

Sessions

• agent-tui sessions (list)
• agent-tui sessions list
• agent-tui sessions show <id>
• agent-tui sessions switch <id>
• agent-tui sessions attach (use -s <id> to target)
• -T, --no-tty: stream only.
• --detach-keys <keys>: custom detach sequence (env: AGENT_TUI_DETACH_KEYS).
• agent-tui sessions cleanup [--all]

Live Preview

• agent-tui live start [--open] [--browser <cmd>]
• agent-tui live status
• agent-tui live stop
• Options:
• --open: open UI in browser (uses AGENT_TUI_UI_URL if set).
• --browser <cmd>: override $BROWSER.

Daemon

• agent-tui daemon start
• agent-tui daemon stop [--force]
• agent-tui daemon restart

Utilities

• agent-tui env
• agent-tui version
• agent-tui help

Shell Completions

• agent-tui completions <bash|zsh|fish|elvish>

Environment Variables

• NO_COLOR: disable colored output.
• AGENT_TUI_DETACH_KEYS: default detach keys for sessions attach.
• AGENT_TUI_WS_LISTEN: daemon WS bind address (default: 127.0.0.1:0).
• AGENT_TUI_WS_ALLOW_REMOTE: allow non-loopback bind (boolean).
• AGENT_TUI_WS_STATE: state file path (default: ~/.agent-tui/api.json).
• AGENT_TUI_WS_DISABLED: disable daemon WS server (boolean).
• AGENT_TUI_WS_MAX_CONNECTIONS: max live connections.
• AGENT_TUI_WS_QUEUE: outbound WS queue size.
• AGENT_TUI_UI_URL: base URL to open with live start --open.
• AGENT_TUI_SESSION_STORE: session metadata log path (default: ~/.agent-tui/sessions.jsonl).
• AGENT_TUI_LOG: log file path (optional).
• AGENT_TUI_LOG_FORMAT: log format (text or json; default: text).
• AGENT_TUI_LOG_STREAM: log output stream (stderr or stdout; default: stderr).
• BROWSER: browser command (overridden by --browser).
• PORT: Bun web preview server port (web/server.ts), when using standalone web UI.

Decision Tree

Use this file when you need a quick command choice based on intent.

Snapshot Strategy

• Need raw text? Use screenshot.
• Need machine-readable output? Use screenshot --json.

Waiting Strategy

• Waiting for text to appear: wait "text" --assert.
• Waiting for text to disappear: wait "text" --gone --assert.
• Waiting for UI to settle: wait --stable.

Action Strategy

• Navigate/confirm: press (keys or sequences).
• Enter text: type "text".

Reliability

• Re-snapshot after any action that could change the UI.
• Prefer wait --stable before acting on dynamic screens.
• Verify outcomes with wait ... --assert.

Live Demo Script

Goal: show a real terminal app launch, capture screenshots, send keystrokes, and verify outcomes in real-time.

Suggested Demo (htop)

Step 1: Launch

agent-tui run htop

Step 2: Observe

agent-tui screenshot

Say: "Here is the current terminal screen. I'll keep re-snapshotting after each action."

Step 3: Interaction

agent-tui press F10

Say: "I'm sending a keypress to exit."

Step 4: Verify

agent-tui wait "Quit" --gone

Say: "I wait for the expected text to disappear to confirm the app closed."

Step 5: Cleanup

agent-tui kill

Narration Tips

• Emphasize the observe/act/wait loop.
• Call out that every action is followed by a fresh snapshot.
• Highlight using wait --stable when the UI is changing.

Full Flows and Command Sequences

Use this file when you need a complete, end-to-end command sequence.

Standard CLI Regression Test (human-like flow)

1) Start the app under test:

• agent-tui run <your-cli> -- <args>

2) Capture session id from JSON output (use --session <id> for the rest). 3) First snapshot (text or structure):

• agent-tui --session <id> screenshot --format json

4) Act based on the latest screen:

• agent-tui --session <id> type "value"
• agent-tui --session <id> press Enter

5) Wait for expected state:

• agent-tui --session <id> wait "Expected text" --assert

6) Repeat steps 3-5 until the flow finishes. 7) Cleanup:

• agent-tui --session <id> kill

Form Interaction Flow

1) agent-tui run <app> 2) agent-tui --session <id> screenshot --format json 3) Focus the input (if needed): agent-tui --session <id> press Tab 4) Type value: agent-tui --session <id> type "my-value" 5) Submit: agent-tui --session <id> press Enter 6) Wait for success: agent-tui --session <id> wait "Success" --assert 7) Cleanup: agent-tui --session <id> kill

Dynamic UI / Flaky Rendering Flow

1) Start: agent-tui run <app> 2) Stabilize: agent-tui --session <id> wait --stable 3) Snapshot: agent-tui --session <id> screenshot --format json 4) Act: agent-tui --session <id> press Enter or agent-tui --session <id> type "text" 5) Re-stabilize: agent-tui --session <id> wait --stable 6) Re-snapshot and continue. 7) Cleanup: agent-tui --session <id> kill

Live Preview Flow (optional)

1) Ask user if they want live preview. 2) If yes: agent-tui live start --open 3) Run the app: agent-tui run <app> 4) Continue normal flow (snapshot/act/wait). 5) Stop preview when done: agent-tui live stop 6) Cleanup: agent-tui kill

Debug/Attach Flow (existing session)

1) List sessions: agent-tui sessions 2) Switch active session if needed: agent-tui sessions switch <id> 3) Attach to active session: agent-tui sessions attach 4) Interact/observe, then detach (Ctrl-P Ctrl-Q). 5) Cleanup: agent-tui kill or agent-tui sessions cleanup

Smoke Test Example (htop)

1) agent-tui run htop 2) agent-tui --session <id> screenshot 3) Verify UI text (e.g., "F1 Help"). 4) Quit: agent-tui --session <id> press F10 5) Confirm quit: agent-tui --session <id> wait "Quit" --gone 6) Cleanup: agent-tui --session <id> kill

JSON Output Contract

Use this file when parsing --format json output.

Spawn / Run

• agent-tui run ... returns:

  { "session_id": "<id>", "pid": 123 }

Screenshot (text)

• agent-tui screenshot ... returns:

  {
    "session_id": "<id>",
    "screenshot": "<string>",
    "cursor": { "row": 0, "col": 0, "visible": true },
    "rendered": "<optional>"
  }

Wait

• agent-tui wait ... returns:

  { "found": true, "elapsed_ms": 1200 }

Resize

• agent-tui resize ... returns:

  { "success": true, "session_id": "<id>", "cols": 120, "rows": 40 }

Restart / Kill

• restart returns:

  { "old_session_id": "<id>", "new_session_id": "<id>", "command": "<cmd>", "pid": 123 }

• kill returns:

  { "success": true, "session_id": "<id>" }

Sessions

• agent-tui sessions returns:

  {
    "sessions": [
      {
        "id": "<id>",
        "command": "<command>",
        "pid": 123,
        "running": true,
        "created_at": "<timestamp>",
        "size": { "cols": 120, "rows": 40 }
      }
    ],
    "active_session": "<id>"
  }

Prompt Templates

Use these verbatim when you need to ask the user for missing details.

Minimum Info Request

• "Please share: (1) command + args to run, (2) expected UI text/state to assert, (3) inputs/steps, (4) any env vars, and (5) whether you want live preview while I run it."

Live Preview

• "Do you want a live preview while I run the test? If yes, I'll start agent-tui live start --open before running the app."

Safety / Environment

• "Is this a test environment, and is it safe to submit or modify data during this run?"

Credentials

• "Does the flow require login? If so, please provide test credentials or a fixture account."

Clarify Assertions

• "What exact text or UI state should I treat as success?"
• "Are there any error states I should explicitly check for?"

Timeouts and Stability

• "Should I use a longer timeout or wait for stability (wait --stable) before actions?"

Completion Check

• "What should indicate the test is finished (specific screen, message, or exit state)?"

Results Summary Template

• "I ran the flow, executed these actions: <actions>, and asserted: <assertions>. The last snapshot shows: <evidence>. Want me to extend coverage or add more assertions?"

Failure Recovery

Use this file when runs are flaky, stalled, or inconsistent.

Common Failures and Fixes

• Text not found: re-run screenshot, wait for stability, then re-check expected text.
• Target not visible: use press navigation keys (for example PageDown, arrows), then re-snapshot.
• Wait timeout: increase --timeout, use wait --stable, then re-snapshot.
• No active session: sessions to list; re-run run if needed.
• Daemon not running: daemon start.
• Version mismatch: daemon restart.
• Unresponsive session: kill, then re-run.
• Layout missing/overflow: resize --cols --rows, re-snapshot.

Retry Budget

• Allow 3-5 retries for transient UI changes.
• If still failing, stop and ask the user for guidance or updated expectations.

Escalation

• Last resort: daemon stop --force then daemon start.
• Warn that daemon restart terminates all sessions.

Safety and Confirmation Prompts

Use this file when actions can mutate data or require credentials.

Always Confirm

• Destructive actions (delete, reset, overwrite, submit).
• Actions against production or real user data.
• Use of real credentials.

Prompt Examples

• "Is this a test environment, and is it safe to submit or delete data?"
• "Do you want me to use live preview while running the test?"
• "Can you provide test credentials or a fixture account?"

Safe Defaults

• Prefer dry-run or read-only flows when available.
• Avoid submitting forms unless explicitly requested.
• Avoid sharing secrets in logs or outputs.

Session Lifecycle and Concurrency

Use this file when managing multiple sessions or debugging stuck runs.

Lifecycle Overview

1) run creates a session and returns session_id. 2) Commands without --session target the most recent session. 3) Use --session <id> when multiple sessions exist. 4) End sessions with kill or sessions cleanup.

Recommended Pattern

• Capture session_id from run JSON output.
• Pass --session <id> to every subsequent command in the flow.
• When running multiple apps concurrently, never rely on the default session.

Inspect and Attach

• sessions: list active sessions.
• sessions show <id>: show session details.
• sessions switch <id>: set the active session.
• sessions attach: attach in TTY mode (detach with Ctrl-P Ctrl-Q). Use -s <id> to target a specific session.
• -s <id> sessions attach -T: stream output only (no TTY).

Cleanup and Recovery

• kill: terminate the current session.
• sessions cleanup: remove dead/orphaned sessions.
• sessions cleanup --all: remove all sessions (including active).

Restart Behavior

• restart: restart the current session command.
• daemon restart: restarts daemon and terminates all sessions.

Utilities

• env: show environment configuration affecting the CLI.
• version: show CLI + daemon version info.

Test Plan Template

Use this file when writing step-by-step tests.

Inputs to Collect

• Command and args to run.
• Terminal size (cols/rows) if layout-sensitive.
• Expected text or stability checkpoints.
• Credentials or fixtures (if needed).

Step Structure

• Step 1: screenshot (or screenshot --json)
• Step 2: Action (press, type)
• Step 3: wait --assert for expected text or wait --stable
• Step 4: Repeat until done
• Step 5: Cleanup (kill)

Example

1) agent-tui run <app> 2) agent-tui --session <id> screenshot --format json 3) agent-tui --session <id> press Tab 4) agent-tui --session <id> type "value" 5) agent-tui --session <id> press Enter 6) agent-tui --session <id> wait "Success" --assert 7) agent-tui --session <id> kill

Use Cases

Use this file when selecting a minimal command set for a task.

Basic Automation Loop

• Commands: run -> screenshot -> press/type -> wait --assert -> kill.
• Re-snapshot after each action.

Form Entry

• Commands: run, screenshot --json, press Tab, type "value", press Enter, wait "Success" --assert.

Stabilization Before Acting

• Commands: wait --stable, screenshot, then press/type.

Live Preview Support

• Commands: live start --open -> run -> normal flow -> live stop.