Rch

RCH — Remote Compilation Helper

Use this skill for remote compilation offload, worker fleet health checks, and hook incident recovery.

Quick Start

rch check
rch status --workers --jobs
rch workers probe --all
rch hook status
rch diagnose --dry-run "cargo check --workspace --all-targets"
rch exec -- env CARGO_TARGET_DIR=${TMPDIR:-/tmp}/rch_target_<name> cargo check --workspace --all-targets

IMPORTANT: Always use ${TMPDIR:-/tmp} for target dirs, never hardcode /tmp or /var/tmp. On fleet machines TMPDIR=/data/tmp (disk-backed). Hardcoding /tmp wastes tmpfs RAM; /var/tmp survives reboots and accumulates silently.

If rch exec -- ... succeeds, remote offload is healthy and remaining failures are likely project/toolchain specific.

If rch status shows storage pressure, always check both / and /tmp on the worker before deciding what to fix:

ssh ubuntu@<host> 'df -h / /tmp && free -h && cat /proc/pressure/memory && cat /proc/pressure/io'

---

Fast Triage Order

Run in this order and stop at the first failing stage:

1. Availability

rch check
rch status --workers --jobs
rch workers probe --all
rch queue

2. Config + socket consistency

rch config show --sources
rch --json config get general.socket_path
rch --json daemon status

3. Hook integration

rch hook status
rch agents status
rch hook install

4. Command classification + path closure

rch diagnose "cargo build --release"
rch diagnose --dry-run "cargo test --workspace"

5. Remote compile proof

rch exec -- env CARGO_TARGET_DIR=${TMPDIR:-/tmp}/rch_target_<name> cargo check --workspace --all-targets

6. If sync fails or storage looks bad, inspect the worker directly

ssh ubuntu@<host> 'df -h / /tmp'
ssh ubuntu@<host> 'du -sh /tmp/rch-* /tmp/rch_target_* 2>/dev/null | sort -h'
ssh ubuntu@<host> 'find /data/projects -maxdepth 2 -type d \\( -name "target_rch_*" -o -name "target_*" -o -name "target-*" -o -name target \\) -exec du -sh {} + 2>/dev/null | sort -h | tail'

---

Quick Fixes

---

Reference Index

Use these files for full depth:

• Runbooks + operational playbooks: references/OPERATIONS.md
• Troubleshooting flow + failure signatures: references/TROUBLESHOOTING.md
• Worker lifecycle operations: references/WORKERS.md
• Config hierarchy + environment controls: references/CONFIGURATION.md
• PreToolUse hook protocol and behavior: references/HOOKS.md
• Workers config template: assets/workers-template.toml
• Project docs: https://github.com/Dicklesworthstone/remote_compilation_helper

Command Reference

Core Commands

Daemon Commands

# Start (choose one)
rchd &                           # Background
rchd --foreground                # Foreground (for debugging)
systemctl --user start rchd      # Systemd (Linux)
launchctl load ~/Library/LaunchAgents/com.rch.daemon.plist  # macOS

# Stop
systemctl --user stop rchd
launchctl unload ~/Library/LaunchAgents/com.rch.daemon.plist

# Logs
journalctl --user -u rchd -f     # Linux
tail -f ~/.config/rch/logs/daemon.log  # macOS

Debug Commands

# Verbose diagnostics
rch doctor --verbose

# Export diagnostics
rch doctor --json > diagnostic.json

# Test hook directly
echo '{"tool":"Bash","input":{"command":"cargo check"}}' | rch hook

# Dry run (logs without execution)
RCH_DRY_RUN=1 cargo check

# Debug logging
RCH_LOG=debug cargo build
RCH_LOG=trace cargo build  # Maximum detail

Worker Probe Output

rch workers probe worker1 --verbose

Shows:

• SSH connectivity (port 22)
• Detected toolchains (rustc, cargo, bun, gcc, clang)
• Disk space (/tmp)
• System load

Environment Variables

Config Files

RCH Configuration Reference

Precedence and File Locations

RCH resolves settings in this order (highest to lowest):

1. CLI flags (--json, --verbose, etc.) 2. Environment variables (RCH_*) 3. Profile defaults (RCH_PROFILE) 4. .env / .rch.env 5. Project config (.rch/config.toml) 6. User config (~/.config/rch/config.toml) 7. Built-in defaults

Primary files:

• User config: ~/.config/rch/config.toml
• Worker config: ~/.config/rch/workers.toml
• Project override: .rch/config.toml
• Optional transfer excludes: .rchignore

---

Main Config (~/.config/rch/config.toml)

[general]
enabled = true
force_local = false
force_remote = false
log_level = "info"                    # trace, debug, info, warn, error, off
socket_path = "~/.cache/rch/rch.sock" # default resolves from runtime/cache path

[compilation]
confidence_threshold = 0.85
min_local_time_ms = 2000
remote_speedup_threshold = 1.2
build_slots = 4
test_slots = 8
check_slots = 2
build_timeout_sec = 300
test_timeout_sec = 1800
bun_timeout_sec = 600
external_timeout_enabled = true

[transfer]
compression_level = 3
remote_base = "/tmp/rch"
adaptive_compression = true
verify_artifacts = false
exclude_patterns = [
  "target/",
  ".git/objects/",
  "node_modules/",
]

[selection]
strategy = "fair_fastest"

[output]
visibility = "summary"                # none, summary, verbose
first_run_complete = true

[self_healing]
hook_starts_daemon = true
daemon_installs_hooks = true
auto_start_timeout_secs = 3

Socket path default behavior:

• First choice: $XDG_RUNTIME_DIR/rch.sock
• Fallback: ~/.cache/rch/rch.sock
• Last resort: /tmp/rch.sock

---

Workers Config (~/.config/rch/workers.toml)

[[workers]]
id = "worker-name"
host = "203.0.113.20"
user = "ubuntu"
identity_file = "~/.ssh/id_ed25519"
total_slots = 16
priority = 100
tags = ["rust", "bun", "fast"]

Slot guidance:

• Start with ~2x physical CPU cores for mixed workloads.
• Reduce slots if workers hit CPU steal, swap pressure, or I/O saturation.
• Increase priority for faster/more reliable workers.

---

Environment Variables

Common overrides:

---

Hook Configuration (Claude Code)

Location: ~/.claude/settings.json

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash",
        "hooks": [
          {
            "type": "command",
            "command": "/absolute/path/to/rch"
          }
        ]
      }
    ]
  }
}

Recommended management commands:

rch hook install
rch hook status
rch hook uninstall

---

Validation and Diagnostics

rch config show --sources
rch config validate
rch config lint
rch config doctor
rch check

---

Runtime Data Paths

Hook Integration

Execution Flow

Claude Code PreToolUse -> rch (no subcommand = hook mode)
                               |
                               +-- Non-Bash tool -> allow unchanged
                               +-- Bash non-compilation -> allow unchanged
                               +-- Bash compilation -> allow with modified command:
                                   "rch exec -- <original command>"

Important behavior:

• Hook returns quickly (classification path), then remote execution happens via rch exec -- ....
• On unsafe/unavailable remote conditions, hook fails open and allows local execution.
• For successful or failed remote runs, RCH preserves command semantics by rewriting to true or exit <code> as needed.

---

Installation and Status

rch hook install
rch hook status
rch hook test
rch hook uninstall

For multi-agent installs:

rch agents list
rch agents status
rch agents install-hook claude-code
rch agents uninstall-hook claude-code

Installed Claude settings entry:

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash",
        "hooks": [
          { "type": "command", "command": "/absolute/path/to/rch" }
        ]
      }
    ]
  }
}

---

Hook Protocol (Current)

Input on stdin

{
  "tool_name": "Bash",
  "tool_input": {
    "command": "cargo build --release",
    "description": "Build project"
  },
  "session_id": "optional"
}

Output on stdout

1. Allow unchanged command: empty stdout 2. Allow with command rewrite (transparent interception):

{
  "hookSpecificOutput": {
    "hookEventName": "PreToolUse",
    "permissionDecision": "allow",
    "updatedInput": {
      "command": "rch exec -- cargo build --release"
    }
  }
}

3. Deny command (rare, policy-level):

{
  "hookSpecificOutput": {
    "hookEventName": "PreToolUse",
    "permissionDecision": "deny",
    "permissionDecisionReason": "reason text"
  }
}

---

What Gets Intercepted

Common intercepted command families:

• Rust: cargo build, cargo check, cargo clippy, cargo doc, cargo test, cargo nextest run, cargo bench, rustc
• Bun/TypeScript: bun test, bun typecheck
• C/C++: gcc, g++, clang, clang++
• Build systems: make, cmake --build, ninja, meson compile

Commonly not intercepted:

• Local-mutating package commands (cargo install, cargo clean, bun install, bun add, etc.)
• Interactive/dev commands (bun run, bun dev, bun build, bunx)
• Piped/redirected/backgrounded shell forms where deterministic offload is unsafe

---

Quick Hook Tests

# Direct protocol test (compilation command)
printf '%s\n' \
  '{"tool_name":"Bash","tool_input":{"command":"cargo build --release"}}' | rch

# Direct protocol test (non-compilation command should usually return empty stdout)
printf '%s\n' \
  '{"tool_name":"Bash","tool_input":{"command":"ls -la"}}' | rch

# Built-in integration test
rch hook test

Debugging:

RCH_LOG_LEVEL=debug rch hook test
RCH_LOG_LEVEL=debug rch diagnose "cargo test --workspace"

---

Performance and Safety Notes

• Non-compilation decisions target sub-millisecond latency.
• Compilation decisions target low-millisecond latency.
• Hook mode always keeps stdout protocol-clean; diagnostics go to stderr.
• If parsing/config/daemon selection fails, RCH allows local execution instead of blocking.

RCH Operations

Baseline Runbook

Use this sequence for most production incidents.

1) Confirm current posture

rch check
rch status --workers --jobs
rch workers probe --all
rch queue

2) Validate config and daemon wiring

rch config show --sources
rch --json config get general.socket_path
rch --json daemon status
rch config doctor

3) Validate hook routing

rch hook status
rch agents status
rch hook test

4) Validate offload path directly

rch diagnose "cargo check --workspace --all-targets"
rch exec -- env CARGO_TARGET_DIR=${TMPDIR:-/tmp}/rch_target_<name> cargo check --workspace --all-targets

If step 4 succeeds, RCH infrastructure is healthy and remaining failures are project/toolchain specific.

5) If workers show storage pressure, inspect the right filesystem

RCH pressure warnings often come from / while the immediate churn lives in /tmp. Check both before deciding whether the host actually needs intervention:

ssh ubuntu@<host> 'df -h / /tmp'
ssh ubuntu@<host> 'free -h && cat /proc/pressure/memory && cat /proc/pressure/io'

Then inspect the usual large artifact surfaces:

ssh ubuntu@<host> 'du -sh /tmp/rch-* /tmp/rch_target_* 2>/dev/null | sort -h'
ssh ubuntu@<host> 'find /data/projects -maxdepth 2 -type d \( -name "target_rch_*" -o -name "target_*" -o -name "target-*" -o -name target \) -exec du -sh {} + 2>/dev/null | sort -h | tail -n 20'

Before removing anything, verify the candidate is inactive:

ssh ubuntu@<host> 'sudo lsof +D /tmp/rch_target_<name>'
ssh ubuntu@<host> 'sudo lsof +D /data/projects/<repo>/target_rch_<name>'

Only treat empty lsof results as a low-risk stale-artifact cleanup signal.

6) If rch exec fails at sync time, verify remote mirror ownership

When the canonical worker mirror under /data/projects/<repo> is owned by root or another account, rsync fails with Permission denied or Operation not permitted.

Check:

ssh ubuntu@<host> "stat -c '%U:%G %a %n' /data/projects/<repo>"

Fix:

ssh ubuntu@<host> 'sudo chown -R ubuntu:ubuntu /data/projects/<repo> && sudo chmod 775 /data/projects/<repo>'

After the fix, rerun:

rch diagnose --dry-run "cargo build --release"
rch exec -- cargo build --release

---

Worker Fleet Lifecycle

Discovery and setup

rch workers discover
rch workers discover --probe
rch workers discover --add --yes
rch workers setup --all

Runtime management

rch workers list --speedscore
rch workers capabilities --refresh
rch workers benchmark
rch workers drain <worker> -y
rch workers enable <worker>
rch workers disable <worker> --reason "maintenance" --drain -y

Toolchain/binary synchronization

rch workers sync-toolchain --all
rch workers deploy-binary --all

---

Fleet Deploy/Rollback

rch fleet status
rch fleet deploy --verify
rch fleet deploy --canary 25 --canary-wait 60 --verify
rch fleet rollback --verify
rch fleet history --limit 20

For large fleets:

• Prefer canary first, then full rollout.
• Use --dry-run before disruptive operations.
• Use --drain-first if workers are heavily loaded.

---

Path-Dependency and Multi-Repo Notes

RCH now supports dependency-closure planning and canonical topology handling, but path-based workspaces still require worker-accessible sibling repos.

Recommended checks:

rch diagnose --dry-run "cargo test --workspace"
rch exec -- env CARGO_TARGET_DIR=${TMPDIR:-/tmp}/rch_target_<name> cargo test --workspace --no-fail-fast

If remote path dependencies are missing:

• Ensure required sibling repos exist on worker hosts under canonical project roots.
• Re-run rch workers setup --all and then retry the rch exec -- ... command.

---

Transfer Stability (Rsync/Artifact Churn)

If sync fails due to active artifact churn, extend transfer excludes:

[transfer]
exclude_patterns = [
  "target/",
  "target_*/",
  "target-*/",
  ".cargo-target/",
  ".cargo-target-*/",
]

Then reload daemon config:

rch daemon reload
rch config show --sources

Operational note:

• If you need a manual target dir for Rust builds, prefer /tmp/rch_target_<name>.
• If the working tree itself cannot sync because the remote canonical mirror is broken, either repair ownership on the worker or temporarily build from a clean directory under /data/projects, not /tmp, because RCH canonical-root normalization expects /data/projects.

---

Queue and Cancellation Operations

rch queue
rch queue --watch
rch cancel <build-id>
rch cancel --all --yes

Use cancellation when builds are wedged or backlog pressure is starving high-priority work.

---

Anti-Patterns

---

Debug Command Pack

RCH_LOG_LEVEL=debug rch diagnose "cargo build --release"
RCH_LOG_LEVEL=debug rch check
rch doctor --json > /tmp/rch-doctor.json
rch --json workers probe --all > /tmp/rch-workers-probe.json
rch daemon logs -n 200

RCH Troubleshooting

Diagnostic Flow

Compilation running locally instead of remotely?
│
├─ Quick health gate:
│  $ rch check
│  │
│  ├─ Not ready/degraded?
│  │   ├─ Check daemon:
│  │   │  $ rch --json daemon status
│  │   │
│  │   ├─ Check workers:
│  │   │  $ rch workers probe --all
│  │   │
│  │   └─ Check hook install:
│  │      $ rch hook status
│  │
│  └─ Ready?
│      continue below
│
└─ Ready but behavior is wrong?
   ├─ Socket alignment:
   │  $ rch --json config get general.socket_path
   │  $ rch --json daemon status
   │
   ├─ Explain routing decision:
   │  $ rch diagnose "cargo build --release"
   │
   ├─ Validate hook protocol path:
   │  $ rch hook test
   │
   └─ Force direct offload proof:
      $ rch exec -- cargo check --workspace --all-targets

---

Common Errors

Daemon not running / check says not ready

Cause: daemon process absent or startup failure.

rch daemon start
rch --json daemon status
rch daemon logs -n 200

Socket mismatch between config and daemon

Cause: general.socket_path differs from active daemon socket.

rch --json config get general.socket_path
rch --json daemon status
# then align and restart:
rch daemon restart -y

"No workers available" / probe failures

Cause: no workers configured, SSH/auth failures, or workers are disabled/drained.

rch workers list
rch workers probe --all
rch workers discover --probe
rch workers discover --add --yes
rch workers setup --all

"rustup: not found" / "cargo: not found" on worker

Cause: missing toolchain on one or more workers.

rch workers sync-toolchain --all
rch workers capabilities --refresh

If still failing, SSH to the specific worker and validate rustup, cargo, and PATH.

Hook not intercepting

Cause: hook missing, wrong binary path, or command classified as local.

rch hook status
rch hook install
rch hook test
rch diagnose "cargo build --release"

Sync/transfer fails under active target churn

Cause: build artifacts changing during rsync.

# Add target-like excludes in ~/.config/rch/config.toml [transfer].exclude_patterns
rch daemon reload
rch config show --sources

Also inspect the worker directly:

ssh ubuntu@<host> 'df -h / /tmp'
ssh ubuntu@<host> 'du -sh /tmp/rch-* /tmp/rch_target_* 2>/dev/null | sort -h'

If cleanup is needed, verify inactivity first:

ssh ubuntu@<host> 'sudo lsof +D /tmp/rch_target_<name>'

If the directory is inactive, prefer targeted stale-artifact cleanup over broad cache deletion.

Sync fails with Permission denied or Operation not permitted inside /data/projects/<repo>

Cause: the canonical mirror on the worker is not writable by the SSH user. This commonly happens when a repo under /data/projects was created or updated as root.

Check:

ssh ubuntu@<host> "stat -c '%U:%G %a %n' /data/projects/<repo>"

Fix:

ssh ubuntu@<host> 'sudo chown -R ubuntu:ubuntu /data/projects/<repo> && sudo chmod 775 /data/projects/<repo>'

Then retry:

rch exec -- cargo check --workspace --all-targets

rch exec fails open for workdirs outside /data/projects

Cause: canonical-root normalization rejects workdirs outside the configured project root.

Symptoms include errors mentioning input resolves outside canonical root.

Fix:

pwd
rch diagnose --dry-run "cargo build --release"

Then run the build from a workspace under /data/projects. If you need a clean copy for testing, stage it under /data/projects/<temp-repo> instead of /tmp/<temp-repo>.

Worker shows storage pressure even after cleanup

Cause: telemetry lag, large ballast allocation, or active live build churn.

Check:

rch status --workers --jobs
ssh ubuntu@<host> 'df -h / /tmp && free -h'
ssh ubuntu@<host> 'journalctl -u sbh -n 50 --no-pager'

Interpretation:

• If df is healthy but rch status still warns, give telemetry a minute and refresh.
• If /tmp is healthy but / is still low, inspect large project target_* trees under /data/projects.
• If sbh is active but repeatedly logging scan channel saturated or scan timed out, inspect stale build artifacts and verify the host is running the current sbh binary and the narrowed worker config.

Path dependency missing remotely (../.../Cargo.toml)

Cause: required sibling repositories are not available in worker topology.

rch diagnose --dry-run "cargo test --workspace"
rch exec -- env CARGO_TARGET_DIR=${TMPDIR:-/tmp}/rch_target_<name> cargo check --workspace --all-targets

Then ensure sibling repos exist on workers under canonical roots and retry.

---

Debug Mode

RCH_LOG_LEVEL=debug rch check
RCH_LOG_LEVEL=debug rch diagnose "cargo test --workspace"
RCH_LOG_LEVEL=debug rch exec -- cargo check --workspace --all-targets

Protocol-level hook test:

RCH_LOG_LEVEL=debug printf '%s\n' \
  '{"tool_name":"Bash","tool_input":{"command":"cargo check"}}' | rch

---

Safe Reset Sequence

rch daemon restart -y
rch config validate
rch config doctor
rch workers probe --all
rch hook status
rch hook test
rch check

If still failing, capture artifacts for escalation:

rch doctor --json > /tmp/rch-doctor.json
rch --json daemon status > /tmp/rch-daemon-status.json
rch --json workers probe --all > /tmp/rch-workers-probe.json

Worker Management

Worker Lifecycle

1) Discover and add workers

rch workers discover
rch workers discover --probe
rch workers discover --add --yes

2) Complete setup

rch workers setup --all

This performs the standard bootstrap path (binary/toolchain setup, validation) for configured workers.

3) Validate runtime health

rch workers list --speedscore
rch workers probe --all
rch workers capabilities --refresh
rch check

---

Add a Worker Manually

Edit ~/.config/rch/workers.toml:

[[workers]]
id = "new-worker"
host = "203.0.113.20"
user = "ubuntu"
identity_file = "~/.ssh/new_worker_ed25519"
total_slots = 16
priority = 90
tags = ["rust", "bun"]

Then validate and setup:

rch config validate
rch workers probe new-worker
rch workers setup new-worker

---

Drain / Disable / Enable

Use these for maintenance windows and incident isolation.

rch workers drain <worker> -y
rch workers disable <worker> --reason "maintenance" --drain -y
rch workers enable <worker>

State model:

• HEALTHY: accepting jobs
• DRAINING: finishing active jobs, no new jobs
• DRAINED: idle and not accepting jobs
• DISABLED: explicitly offline from scheduler

---

Toolchain and Binary Management

rch workers sync-toolchain --all
rch workers deploy-binary --all

Use --dry-run before broad changes:

rch workers sync-toolchain --all --dry-run
rch workers deploy-binary --all --dry-run

---

Fleet-Level Rollout Commands

rch fleet status
rch fleet deploy --verify
rch fleet deploy --canary 25 --canary-wait 60 --verify
rch fleet rollback --verify
rch fleet history --limit 20

---

Worker Selection Notes

Selection favors availability and execution quality signals (slot capacity, health, and policy strategy).

Operational guidance:

• Keep total_slots realistic for CPU and memory limits.
• Prefer explicit priority shaping for known fast/reliable workers.
• Drain before disruptive operations.
• Keep worker toolchains synchronized to avoid fallback churn.

---

SSH Verification Shortcuts

Single worker:

rch workers probe <worker>

All workers with machine-readable output:

rch --json workers probe --all

If probes fail:

1. Verify identity_file exists and permissions are restrictive. 2. Verify worker host reachability and SSH service. 3. Re-run rch workers setup <worker> after connectivity is restored.