Fleet Auditor

Fleet Auditor: Cross-Platform Agent Token Waste Auditor

Detects installed agent systems, collects token usage data, identifies waste patterns, and recommends fixes with dollar savings estimates. Everyone tracks. Nobody coaches. Until now.

Use when: Running multiple agent systems, spending $2-5/day on agents, suspecting idle heartbeats are burning tokens, or want a cross-system cost audit.

---

Phase 0: Initialize

1. Resolve runtime and fleet.py path (works for both skill and plugin installs):

RUNTIME="${TOKEN_OPTIMIZER_RUNTIME:-}"
if [ -z "$RUNTIME" ]; then
  if [ -n "$CLAUDE_PLUGIN_ROOT" ] || [ -n "$CLAUDE_PLUGIN_DATA" ]; then
    RUNTIME="claude"
  elif [ -n "$CODEX_HOME" ] || [ -d "$HOME/.codex" ]; then
    RUNTIME="codex"
  else
    RUNTIME="claude"
  fi
fi

FLEET_PY=""
for f in "$HOME/.codex/skills/fleet-auditor/scripts/fleet.py" \
         "$HOME/.codex/plugins/cache"/*/token-optimizer/*/skills/fleet-auditor/scripts/fleet.py \
         "$HOME/.claude/skills/fleet-auditor/scripts/fleet.py" \
         "$HOME/.claude/plugins/cache"/*/token-optimizer/*/skills/fleet-auditor/scripts/fleet.py; do
  [ -f "$f" ] && FLEET_PY="$f" && break
done
[ -z "$FLEET_PY" ] && { echo "[Error] fleet.py not found. Is Fleet Auditor installed?"; exit 1; }
echo "Using: $FLEET_PY"
export TOKEN_OPTIMIZER_RUNTIME="$RUNTIME"

Use $FLEET_PY for all subsequent fleet.py calls.

2. Detect systems:

python3 "$FLEET_PY" detect --json

Parse the JSON output. Report what was found.

If nothing detected, explain: "No agent systems found. Fleet Auditor supports: Claude Code, Codex, OpenClaw, NanoClaw, Hermes, OpenCode, IronClaw."

---

Phase 1: Scan

Collect token usage data from detected systems:

python3 "$FLEET_PY" scan --days 30

Report how many runs were collected per system. If this is the first scan, it may take a moment to parse all session files.

---

Phase 2: Audit

Run waste pattern detection:

python3 "$FLEET_PY" audit --json

Parse the JSON output. Present findings ordered by severity and monthly savings.

If no waste found: "Your fleet looks clean. No significant waste patterns detected."

For Codex findings, be careful with dollars: if a model's pricing is not in the local pricing table, present token waste confidently and say dollar impact depends on current model pricing instead of inventing a cost.

OpenClaw Security + Cost Check (March 2026):

• If OpenClaw is detected and version < 2026.3.12: flag as HIGH severity. 31 CVEs including ClawJacked (full agent takeover). Compromised instances run rogue agents that burn tokens undetected. Recommend immediate upgrade.
• Check for ClawHub skills matching ClawHavoc campaign patterns (1,184+ malicious skills). These skills exfiltrate data via extra API calls, inflating token spend.
• Flag if gateway rate limiting is disabled (CVE-2026-25253). Without it, brute-force attacks can spawn unlimited agent sessions. Recommend: openclaw config set security.rateLimit.enabled true

---

Phase 3: Present Findings

[Fleet Auditor Results]

SYSTEMS DETECTED
- Claude Code: X runs ($Y.YY)
- Codex: X runs ($Y.YY)
- OpenClaw: X runs ($Y.YY)

WASTE PATTERNS FOUND
1. [SEVERITY] Description
   Est. savings: $X.XX/month
   Fix: recommendation

2. [SEVERITY] Description
   ...

TOTAL POTENTIAL SAVINGS: $X.XX/month

Ready to act? I can:
1. Show detailed fix snippets for each finding
2. Generate the fleet dashboard for visual analysis
3. Run /token-optimizer for deeper Claude Code optimization

---

Phase 4: Dashboard (optional)

If user wants visual analysis:

python3 "$FLEET_PY" dashboard

This generates ~/.claude/_backups/token-optimizer/fleet-dashboard.html in Claude Code, or ~/.codex/_backups/token-optimizer/fleet-dashboard.html when TOKEN_OPTIMIZER_RUNTIME=codex.

---

Phase 5: Deep Dive (optional)

For Claude Code specifically, offer /token-optimizer for full audit (CLAUDE.md, skills, MCP, hooks, etc.).

For Codex specifically, offer token-optimizer for full audit (AGENTS.md, Codex memories, plugin skills, MCP, balanced hooks, compact prompt, status line).

For other systems, show the fix snippets from the audit and guide the user through implementing them.

---

Reference Files

---

Error Handling

• No systems detected: Report cleanly, list supported systems
• Empty scan results: System detected but no session data in window. Suggest increasing --days
• Permission errors: Report which files couldn't be read, continue with available data
• Corrupted data: Skip bad files, report count of skipped files
• fleet.py not found: Check both skill and plugin install paths

---

Core Rules

• Quantify everything in dollars AND tokens
• Never read or expose message content (privacy-first)
• Report confidence levels alongside findings
• Suppress findings below 0.4 confidence threshold
• Always show fix snippets with recommendations
• Frame savings as monthly recurring, not one-time

Fleet Systems: Data Format Specifications

Reference file for Fleet Auditor. Loaded on demand for adapter development.

---

Claude Code

Data Location: ~/.claude/projects/ Format: JSONL (one JSON object per line) Structure: Each project gets a directory named with a path-encoded slug (e.g., -Users-alex-myproject/). Session files are {uuid}.jsonl. Subagent files live in {uuid}/subagents/{sub-uuid}.jsonl.

JSONL Record Types

• type: "user" - User messages with message.content (string or array of blocks)
• type: "assistant" - Assistant messages with message.content (array of tool_use and text blocks), message.usage (token data), message.model
• type: "result" - Tool results

Token Fields (in message.usage)

{
  "input_tokens": 12345,
  "output_tokens": 678,
  "cache_read_input_tokens": 9000,
  "cache_creation_input_tokens": 3000
}

Key Extraction Points

• Version: First record with version field
• Slug: First record with slug field
• Timestamp: timestamp field (ISO-8601 with Z suffix)
• Model: message.model in assistant records
• Tools: tool_use blocks in message.content array

---

OpenClaw

Data Location: ~/.openclaw/agents/ (also ~/.clawdbot/, ~/.moltbot/) Format: JSON index + JSONL transcripts

Key Files

• sessions.json - Index of all sessions with metadata
• agents/{agent-name}/sessions/{session-id}.jsonl - Individual session transcripts
• cron/ - Heartbeat/cron configuration and logs
• config.json - Global config including model settings, pricing overrides

Token Fields

{
  "inputTokens": 12345,
  "outputTokens": 678,
  "totalTokens": 13023
}

Note: OpenClaw does NOT expose cache read/write breakdown. Use inputTokens for total input.

---

NanoClaw

Data Location: ~/.nanoclaw/ or container volume mounts Format: SQLite (messages table) + Claude Agent SDK response objects Built on: Anthropic Claude Agent SDK

Token Fields (from SDK response)

response.usage.input_tokens
response.usage.output_tokens
response.usage.cache_creation_input_tokens
response.usage.cache_read_input_tokens

SQLite Schema

-- messages table stores conversations
-- Token data in SDK response metadata, not always in DB
CREATE TABLE messages (
    id TEXT PRIMARY KEY,
    session_id TEXT,
    role TEXT,
    content TEXT,
    created_at TIMESTAMP
);

Container Considerations

• Data may be inside Docker/container volumes
• Check for exported/mounted data directories
• Container isolation means direct DB access may not be possible

---

Hermes

Data Location: ~/.hermes/ Format: SQLite (state.db) + JSONL session logs

Key Files

• state.db - Primary state database
• sessions/{date}/{session-id}.jsonl - Session transcripts

Token Fields

{
  "tokens": {
    "input": 12345,
    "output": 678
  }
}

---

OpenCode

Data Location: ~/.local/share/opencode/ or $OPENCODE_DATA_DIR Format: JSON per-message + SQLite (v1.2+)

Token Fields

{
  "usage": {
    "input": 12345,
    "output": 678
  },
  "cacheRead": 9000,
  "cacheWrite": 3000
}

File Structure

• sessions/{session-id}/ - Per-session directories
• sessions/{session-id}/messages.json - Array of messages
• storage.db - SQLite database (v1.2+, replaces JSON)

---

IronClaw

Data Location: ~/.ironclaw/ Format: PostgreSQL or libSQL (requires connection config)

Token Fields

{
  "max_tokens": 4096,
  "total_tokens_used": 12345
}

Access Pattern

• Requires database connection string from config
• Phase 1: detect only, no scan
• Phase 2+: support exported data files or direct DB connection

Waste Patterns: Detection Algorithms and Thresholds

Reference file for Fleet Auditor. Loaded on demand for detector development.

---

Tier 1: Static Config Analysis

These detectors run against configuration files and don't need session data.

1. Heartbeat Model Waste

Signal: Cron/heartbeat agent configured with opus or sonnet Threshold: Any heartbeat using non-haiku model with >$0.10/month cost False positive check: Some heartbeats legitimately need reasoning (e.g., triage bots) Confidence: 0.9 (high, easy to verify from config)

2. Heartbeat Over-Frequency

Signal: 3+ consecutive heartbeat runs with <5 min interval Threshold: Average interval < 300 seconds across 3+ runs False positive check: Burst patterns (3 quick then long gap) are OK Confidence: 0.7 (intervals can be irregular)

3. Skill Bloat

Signal: >10 skills loaded per agent Threshold: 10+ skills = medium, 20+ = high Cost model: ~100 tokens/skill/API call x 20 calls/session x 30 sessions/month False positive check: Power users with 15 skills they all actively use Confidence: 0.8

4. Tool Definition Bloat

Signal: MCP tool definitions consuming >15% of 200K context Threshold: Estimated tool tokens > 30K Cost model: Rough (150 tokens/eager tool, 15/deferred, ~10 tools/server) False positive check: All servers could be actively used Confidence: 0.6 (rough estimate)

5. Memory/Config Overhead

Signal: CLAUDE.md or MEMORY.md exceeding 5,000 tokens Threshold: >5K = medium, >10K = high Cost model: Tokens x 20 calls/session x 30 sessions/month False positive check: Large CLAUDE.md might be legitimately needed Confidence: 0.9

6. Stale Cron Configurations

Signal: Cron/hook commands referencing non-existent paths Threshold: Any dead path reference False positive check: Paths with variables ($HOME, etc.) that we can't resolve Confidence: 0.5 (can't always determine validity)

---

Tier 2: Session Log Analysis

These detectors require parsed session data (AgentRun objects from fleet.db).

7. Empty Heartbeat Runs (THE #1 WASTE PATTERN)

Signal: Input > 5K tokens, output < 100 tokens, messages <= 4 Confirmation: Input > 10K OR outcome == "empty" Threshold: 2+ confirmed empty runs in the window Cost model: Actual cost from token data False positive check: Legitimate "nothing to do" checks with small context Confidence: 0.85

8. Session History Bloat

Signal: Sessions with 30+ messages and 500K+ input tokens Interpretation: Context growing monotonically without compaction Savings estimate: ~40% of bloated input (conservative compaction savings) False positive check: Some sessions legitimately process large codebases Confidence: 0.6

9. Loop Detection

Signal: High input:output ratio (>20:1) in sessions with 10+ messages Interpretation: Agent reading lots of context but producing little output = stuck Threshold: 2+ suspected loop sessions, >$0.50/month waste False positive check: Exclude "empty" outcome runs (caught by detector 7) Confidence: 0.5 (heuristic, needs JSONL deep-parse for confirmation)

10. Abandoned Sessions

Signal: 1-2 messages, >3K input tokens, manual run type Interpretation: User started a session, loaded full context, then left Threshold: 3+ abandoned sessions, >$0.20/month waste False positive check: Quick "check something" sessions are normal Confidence: 0.7

---

Phase 2+ Detectors (Not Yet Implemented)

11. Retry Storms

Signal: Same tool called 3+ times consecutively with similar inputs Implementation: Requires JSONL deep-parse for tool call sequences

12. Model Downgrade Opportunities

Signal: Sessions using opus/sonnet with low complexity indicators Complexity indicators: Short messages, few tool calls, simple patterns Implementation: Requires session content analysis

---

Severity Levels

Confidence Levels