Cmux Browser

Browser Automation with cmux

Use this skill for browser tasks inside cmux webviews.

Core Workflow

1. Open or target a browser surface. 2. Verify navigation with get url before waiting or snapshotting. 3. Snapshot (--interactive) to get fresh element refs. 4. Act with refs (click, fill, type, select, press). 5. Wait for state changes. 6. Re-snapshot after DOM/navigation changes.

cmux --json browser open https://example.com
# use returned surface ref, for example: surface:7

cmux browser surface:7 get url
cmux browser surface:7 wait --load-state complete --timeout-ms 15000
cmux browser surface:7 snapshot --interactive
cmux browser surface:7 fill e1 "hello"
cmux --json browser surface:7 click e2 --snapshot-after
cmux browser surface:7 snapshot --interactive

Surface Targeting

# identify current context
cmux identify --json

# open routed to a specific topology target
cmux browser open https://example.com --workspace workspace:2 --window window:1 --json

Notes:

• CLI output defaults to short refs (surface:N, pane:N, workspace:N, window:N).
• UUIDs are still accepted on input; only request UUID output when needed (--id-format uuids|both).
• Keep using one surface:N per task unless you intentionally switch.

Wait Support

cmux supports wait patterns similar to agent-browser:

cmux browser <surface> wait --selector "#ready" --timeout-ms 10000
cmux browser <surface> wait --text "Success" --timeout-ms 10000
cmux browser <surface> wait --url-contains "/dashboard" --timeout-ms 10000
cmux browser <surface> wait --load-state complete --timeout-ms 15000
cmux browser <surface> wait --function "document.readyState === 'complete'" --timeout-ms 10000

Common Flows

Form Submit

cmux --json browser open https://example.com/signup
cmux browser surface:7 get url
cmux browser surface:7 wait --load-state complete --timeout-ms 15000
cmux browser surface:7 snapshot --interactive
cmux browser surface:7 fill e1 "Jane Doe"
cmux browser surface:7 fill e2 "jane@example.com"
cmux --json browser surface:7 click e3 --snapshot-after
cmux browser surface:7 wait --url-contains "/welcome" --timeout-ms 15000
cmux browser surface:7 snapshot --interactive

Clear an Input

cmux browser surface:7 fill e11 "" --snapshot-after --json
cmux browser surface:7 get value e11 --json

Stable Agent Loop (Recommended)

# navigate -> verify -> wait -> snapshot -> action -> snapshot
cmux browser surface:7 get url
cmux browser surface:7 wait --load-state complete --timeout-ms 15000
cmux browser surface:7 snapshot --interactive
cmux --json browser surface:7 click e5 --snapshot-after
cmux browser surface:7 snapshot --interactive

If get url is empty or about:blank, navigate first instead of waiting on load state.

Deep-Dive References

Ready-to-Use Templates

Limits (WKWebView)

These commands currently return not_supported because they rely on Chrome/CDP-only APIs not exposed by WKWebView:

• viewport emulation
• offline emulation
• trace/screencast recording
• network route interception/mocking
• low-level raw input injection

Use supported high-level commands (click, fill, press, scroll, wait, snapshot) instead.

Troubleshooting

js_error on snapshot --interactive or eval

Some complex pages can reject or break the JavaScript used for rich snapshots and ad-hoc evaluation.

Recovery steps:

cmux browser surface:7 get url
cmux browser surface:7 get text body
cmux browser surface:7 get html body

• Use get url first so you know whether the page actually navigated.
• Fall back to get text body or get html body when snapshot --interactive or eval returns js_error.
• If the page is still failing, navigate to a simpler intermediate page, then retry the task from there.

Authentication Patterns

Login flows, session persistence, OAuth, and 2FA patterns for cmux browser surfaces.

Related: session-management.md, SKILL.md

Contents

• Basic Login Flow
• Saving Authentication State
• Restoring Authentication
• OAuth / SSO Flows
• Two-Factor Authentication
• Cookie-Based Auth
• Token Refresh Handling
• Security Best Practices

Basic Login Flow

cmux browser open https://app.example.com/login --json
cmux browser surface:7 wait --load-state complete --timeout-ms 15000

cmux browser surface:7 snapshot --interactive
# [ref=e1] email, [ref=e2] password, [ref=e3] submit

cmux browser surface:7 fill e1 "user@example.com"
cmux browser surface:7 fill e2 "$APP_PASSWORD"
cmux browser surface:7 click e3 --snapshot-after --json
cmux browser surface:7 wait --url-contains "/dashboard" --timeout-ms 20000

Saving Authentication State

After logging in, save state for reuse:

cmux browser surface:7 state save ./auth-state.json

State includes cookies, localStorage, sessionStorage, and open tab metadata for that surface.

Restoring Authentication

cmux browser open https://app.example.com --json
cmux browser surface:8 state load ./auth-state.json
cmux browser surface:8 goto https://app.example.com/dashboard
cmux browser surface:8 snapshot --interactive

OAuth / SSO Flows

cmux browser open https://app.example.com/auth/google --json
cmux browser surface:7 wait --url-contains "accounts.google.com" --timeout-ms 30000
cmux browser surface:7 snapshot --interactive

cmux browser surface:7 fill e1 "user@gmail.com"
cmux browser surface:7 click e2 --snapshot-after --json

cmux browser surface:7 wait --url-contains "app.example.com" --timeout-ms 45000
cmux browser surface:7 state save ./oauth-state.json

Two-Factor Authentication

cmux browser open https://app.example.com/login --json
cmux browser surface:7 snapshot --interactive
cmux browser surface:7 fill e1 "user@example.com"
cmux browser surface:7 fill e2 "$APP_PASSWORD"
cmux browser surface:7 click e3

# complete 2FA manually in the webview, then:
cmux browser surface:7 wait --url-contains "/dashboard" --timeout-ms 120000
cmux browser surface:7 state save ./2fa-state.json

Cookie-Based Auth

cmux browser surface:7 cookies set session_token "abc123xyz"
cmux browser surface:7 goto https://app.example.com/dashboard

Token Refresh Handling

#!/usr/bin/env bash
set -euo pipefail

STATE_FILE="./auth-state.json"
SURFACE="surface:7"

if [ -f "$STATE_FILE" ]; then
  cmux browser "$SURFACE" state load "$STATE_FILE"
fi

cmux browser "$SURFACE" goto https://app.example.com/dashboard
URL=$(cmux browser "$SURFACE" get url)

if printf '%s' "$URL" | grep -q '/login'; then
  cmux browser "$SURFACE" snapshot --interactive
  cmux browser "$SURFACE" fill e1 "$APP_USERNAME"
  cmux browser "$SURFACE" fill e2 "$APP_PASSWORD"
  cmux browser "$SURFACE" click e3
  cmux browser "$SURFACE" wait --url-contains "/dashboard" --timeout-ms 20000
  cmux browser "$SURFACE" state save "$STATE_FILE"
fi

Security Best Practices

1. Never commit state files (they include auth tokens). 2. Use environment variables for credentials. 3. Clear state/cookies after sensitive tasks:

cmux browser surface:7 cookies clear
rm -f ./auth-state.json

Command Reference (cmux Browser)

This maps common agent-browser usage to cmux browser usage.

Direct Equivalents

• agent-browser open <url> -> cmux browser open <url>
• agent-browser goto|navigate <url> -> cmux browser <surface> goto|navigate <url>
• agent-browser snapshot -i -> cmux browser <surface> snapshot --interactive
• agent-browser click <ref> -> cmux browser <surface> click <ref>
• agent-browser fill <ref> <text> -> cmux browser <surface> fill <ref> <text>
• agent-browser type <ref> <text> -> cmux browser <surface> type <ref> <text>
• agent-browser select <ref> <value> -> cmux browser <surface> select <ref> <value>
• agent-browser get text <ref> -> cmux browser <surface> get text <ref-or-selector>
• agent-browser get url -> cmux browser <surface> get url
• agent-browser get title -> cmux browser <surface> get title

Core Command Groups

Navigation

cmux browser open <url>                        # opens in caller's workspace (uses CMUX_WORKSPACE_ID)
cmux browser open <url> --workspace <id|ref>   # opens in a specific workspace
cmux browser <surface> goto <url>
cmux browser <surface> back|forward|reload
cmux browser <surface> get url|title

Workspace context: browser open targets the workspace of the terminal where the command is run (via CMUX_WORKSPACE_ID), even if a different workspace is currently focused. Use --workspace to override.

Snapshot and Inspection

cmux browser <surface> snapshot --interactive
cmux browser <surface> snapshot --interactive --compact --max-depth 3
cmux browser <surface> get text body
cmux browser <surface> get html body
cmux browser <surface> get value "#email"
cmux browser <surface> get attr "#email" --attr placeholder
cmux browser <surface> get count ".row"
cmux browser <surface> get box "#submit"
cmux browser <surface> get styles "#submit" --property color
cmux browser <surface> eval '<js>'

Interaction

cmux browser <surface> click|dblclick|hover|focus <selector-or-ref>
cmux browser <surface> fill <selector-or-ref> [text]   # empty text clears
cmux browser <surface> type <selector-or-ref> <text>
cmux browser <surface> press|keydown|keyup <key>
cmux browser <surface> select <selector-or-ref> <value>
cmux browser <surface> check|uncheck <selector-or-ref>
cmux browser <surface> scroll [--selector <css>] [--dx <n>] [--dy <n>]

Wait

cmux browser <surface> wait --selector "#ready" --timeout-ms 10000
cmux browser <surface> wait --text "Done" --timeout-ms 10000
cmux browser <surface> wait --url-contains "/dashboard" --timeout-ms 10000
cmux browser <surface> wait --load-state complete --timeout-ms 15000
cmux browser <surface> wait --function "document.readyState === 'complete'" --timeout-ms 10000

Session/State

cmux browser <surface> cookies get|set|clear ...
cmux browser <surface> storage local|session get|set|clear ...
cmux browser <surface> tab list|new|switch|close ...
cmux browser <surface> state save|load <path>

Diagnostics

cmux browser <surface> console list|clear
cmux browser <surface> errors list|clear
cmux browser <surface> highlight <selector>
cmux browser <surface> screenshot
cmux browser <surface> download wait --timeout-ms 10000

Agent Reliability Tips

• Use --snapshot-after on mutating actions to return a fresh post-action snapshot.
• Re-snapshot after navigation, modal open/close, or major DOM changes.
• Prefer short handles in outputs by default (surface:N, pane:N, workspace:N, window:N).
• Use --id-format both only when a UUID must be logged/exported.

Known WKWebView Gaps (not_supported)

• browser.viewport.set
• browser.geolocation.set
• browser.offline.set
• browser.trace.start|stop
• browser.network.route|unroute|requests
• browser.screencast.start|stop
• browser.input_mouse|input_keyboard|input_touch

See also:

• snapshot-refs.md
• authentication.md
• session-management.md

Proxy Support

How proxy behavior works for cmux browser automation.

Related: commands.md, SKILL.md

Contents

• Current Behavior
• What Is Not Exposed via CLI
• Workarounds
• Verification

Current Behavior

cmux browser uses WKWebView networking. Proxy behavior follows macOS/system networking and app process environment.

What Is Not Exposed via CLI

There is currently no first-class cmux browser proxy ... command for per-surface proxy routing.

Why: WKWebView does not provide CDP-style per-context proxy controls equivalent to Chrome automation stacks.

Workarounds

1. Configure system/network-level proxy for the environment where cmux runs. 2. Route traffic through an upstream gateway you control. 3. Validate behavior with explicit IP checks.

Verification

cmux browser open https://httpbin.org/ip --json
cmux browser surface:7 get text body

Compare returned IP against expected proxy egress.

Session Management

cmux uses isolated browser contexts per surface. Treat each browser surface as its own session.

Related: authentication.md, SKILL.md

Contents

• Surface-Based Sessions
• Isolation Properties
• State Persistence
• Common Patterns
• Cleanup
• Best Practices

Surface-Based Sessions

# session A
cmux browser open https://app.example.com/login --json
# -> surface:7

# session B
cmux browser open https://example.com --json
# -> surface:8

cmux browser surface:7 get url
cmux browser surface:8 get url

Isolation Properties

Each surface has independent:

• cookies
• localStorage/sessionStorage
• tab list and active tab
• navigation history

State Persistence

Save State

cmux browser surface:7 state save /tmp/auth-state.json

Load State

cmux browser surface:8 state load /tmp/auth-state.json
cmux browser surface:8 goto https://app.example.com/dashboard

Common Patterns

Reuse Auth Across New Surface

cmux browser open https://app.example.com/login --json
# login on surface:7 ...
cmux browser surface:7 state save /tmp/auth.json

cmux browser open https://app.example.com --json
# assume surface:8
cmux browser surface:8 state load /tmp/auth.json
cmux browser surface:8 goto https://app.example.com/dashboard

Parallel Multi-Site Tasks

cmux browser open https://site-a.example --json
cmux browser open https://site-b.example --json
cmux browser open https://site-c.example --json

cmux browser surface:11 get text body > /tmp/a.txt
cmux browser surface:12 get text body > /tmp/b.txt
cmux browser surface:13 get text body > /tmp/c.txt

Cleanup

cmux close-surface --surface surface:7
cmux close-surface --surface surface:8
rm -f /tmp/auth-state.json

Best Practices

1. Name/log surfaces in your script output so actions stay attributable. 2. Keep one task per surface to avoid ref churn. 3. Save state after successful auth milestones. 4. Re-snapshot after switching tabs/pages inside a surface.

Snapshot and Refs

Element refs from snapshots make browser automation compact and reliable.

Related: commands.md, SKILL.md

Contents

• How Refs Work
• The Snapshot Command
• Using Refs
• Ref Lifecycle
• Best Practices
• Troubleshooting

How Refs Work

Classic flow:

full DOM/HTML -> selector guessing -> action

cmux flow:

snapshot -> refs (e1/e2/...) -> direct action

The Snapshot Command

cmux browser surface:7 snapshot
cmux browser surface:7 snapshot --interactive
cmux browser surface:7 snapshot --interactive --compact --max-depth 3

Using Refs

cmux browser surface:7 click e6
cmux browser surface:7 fill e10 "user@example.com"
cmux browser surface:7 fill e11 "password123"
cmux browser surface:7 click e12

Ref Lifecycle

Refs are invalidated when page structure changes.

cmux browser surface:7 snapshot --interactive
# e1 is "Next"

cmux browser surface:7 click e1

# page changed, take a fresh snapshot
cmux browser surface:7 snapshot --interactive

Best Practices

1. Snapshot before interacting. 2. Re-snapshot after navigation/modal/open-close flows. 3. Use --snapshot-after on mutating actions. 4. Scope snapshots with --selector for very large pages.

Troubleshooting

not_found / stale ref

cmux browser surface:7 snapshot --interactive

Element missing due visibility/timing

cmux browser surface:7 wait --selector "#target" --timeout-ms 10000
cmux browser surface:7 scroll --dy 400
cmux browser surface:7 snapshot --interactive

Too many elements

cmux browser surface:7 snapshot --selector "form#checkout" --interactive

Video Recording

Status and alternatives for capturing browser automation evidence in cmux.

Related: commands.md, SKILL.md

Contents

• Current Status
• Recommended Alternatives
• Use Cases
• Best Practices

Current Status

cmux browser currently does not expose a built-in video recording command.

Why: cmux browser automation runs on WKWebView, and the agent-browser style recording pipeline is Chrome/CDP-specific.

Recommended Alternatives

1. Step Screenshots

cmux browser surface:7 screenshot > /tmp/step1.b64
cmux browser surface:7 click e3 --snapshot-after --json
cmux browser surface:7 screenshot > /tmp/step2.b64

2. Snapshot Timeline

cmux browser surface:7 snapshot --interactive > /tmp/snap-1.txt
cmux browser surface:7 click e3 --snapshot-after --json > /tmp/action-1.json
cmux browser surface:7 snapshot --interactive > /tmp/snap-2.txt

3. macOS Window Capture (external)

Use an external screen recorder if full-motion capture is required.

Use Cases

• Debug flaky browser automation.
• Produce artifacts for CI logs.
• Document flow changes between releases.

Best Practices

1. Capture snapshot before and after each mutating action. 2. Add --snapshot-after on clicks/fills/types that change state. 3. Keep artifacts grouped by timestamp/run id.