
Doctor
- 42 installs
- 213 repo stars
- Updated August 4, 2026
- yonatangross/skillforge-claude-plugin
Helps with ai & agent building tasks.
About
doctor is a Claude Code skill for ai & agent building. It helps solo builders move faster with AI-assisted coding.
- doctor
- AI & Agent Building
- AI-coding skill
Doctor by the numbers
- 42 all-time installs (skills.sh)
- Ranked #8,070 of 16,546 AI & Agent Building skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/yonatangross/skillforge-claude-plugin --skill doctorAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 42 |
|---|---|
| repo stars | ★ 213 |
| Last updated | August 4, 2026 |
| Repository | yonatangross/skillforge-claude-plugin ↗ |
What it does
Helps with ai & agent building tasks.
Files
OrchestKit Health Diagnostics
Argument Resolution
FLAGS = "$ARGUMENTS" # Full argument string, e.g., "--verbose" or "--json"
FLAG = "$ARGUMENTS[0]" # First token: -v, --verbose, --json, --category=X
# $ARGUMENTS[0], $ARGUMENTS[1] for indexed access (CC 2.1.59)STEP 0: Choose Scope (AskUserQuestion — M118 #1464)
A full doctor run takes ~20s. Most invocations only need one slice. Ask the user up-front so voice-flow shortcuts ("just the MCPs") map cleanly:
# Skip the prompt when an explicit scope arg or env override is present:
# /ork:doctor cc → skip, use cc-only
# /ork:doctor mcp → skip, use mcp-only
# /ork:doctor plugin → skip, use plugin-only
# ORK_DOCTOR_SCOPE=all (or any of the above) → skip, use the env value
#
# Otherwise, ask:
AskUserQuestion(questions=[{
"question": "What should doctor check?",
"header": "Scope",
"options": [
{"label": "Everything (default)", "description": "Full system health — ~20s; runs all 14 categories"},
{"label": "CC version & features only", "description": "Categories 10 + 13 + 14; ~3s — for 'is my CC up to date?'"},
{"label": "MCP servers only", "description": "Category 12 (incl. pinning sub-check); ~5s — for 'are MCPs working?'"},
{"label": "Plugin health only", "description": "Categories 0-3 + 5 (skills, agents, hooks, build); ~8s — for 'after npm run build'"}
]
}])Skip the prompt entirely when the scope is unambiguous from the invocation. The fast scopes (3-8s) are 3-7× faster than the full run — voice users say "just the MCPs" and get a 5s answer.
Overview
The /ork:doctor command performs comprehensive health checks on your OrchestKit installation. It auto-detects installed plugins and validates 14 categories:
1. Installed Plugins - Detects ork plugin 2. Skills Validation - Frontmatter, references, token budget (dynamic count) 3. Agents Validation - Frontmatter, tool refs, skill refs (dynamic count) 4. Hook Health - Registration, bundles, async patterns 5. Permission Rules - Detects unreachable rules 6. Schema Compliance - Validates JSON files against schemas 7. Coordination System - Checks lock health and registry integrity 8. Context Budget - Monitors token usage against budget 9. Memory System - Graph memory health 10. Claude Code Version & Channel - Validates CC >= 2.1.170 (supported floor), detects release channel (stable/beta/alpha), recommends 2.1.154+ for Opus 4.8 / xhigh effort, /ultrareview, stream-json plugin_errors 11. External Dependencies - Checks optional tool availability (agent-browser) 12. MCP Status - Active vs disabled vs misconfigured, API key presence for paid MCPs. CC 2.1.110: detects duplicate definitions across config scopes. Sub-check warns when HIGH-tier servers resolve to @latest in .mcp.json (closes #1462) 13. Plugin Validate - Runs claude plugin validate for official CC frontmatter + hooks.json validation (CC >= 2.1.77) 14. Effort/Model Compatibility - Warns when xhigh effort is requested without Opus 4.8 (silent fallback otherwise)
When to Use
- After installing or updating OrchestKit
- When hooks aren't firing as expected
- Before deploying to a team environment
- When debugging coordination issues
- After running
npm run build
Quick Start
/ork:doctor # Standard health check
/ork:doctor -v # Verbose output
/ork:doctor --json # Machine-readable for CICLI Options
| Flag | Description |
|---|---|
-v, --verbose | Detailed output per check |
--json | JSON output for CI integration |
--category=X | Run only specific category |
Health Check Categories
Detailed check procedures: Load Read("${CLAUDE_SKILL_DIR}/rules/diagnostic-checks.md") for bash commands and validation logic per category.>
MCP-specific checks: Load Read("${CLAUDE_SKILL_DIR}/rules/mcp-status-checks.md") for credential validation and misconfiguration detection.>
Output examples: Load Read("${CLAUDE_SKILL_DIR}/references/health-check-outputs.md") for sample output per category.Categories 0-3: Core Validation
| Category | What It Checks | Reference |
|---|---|---|
| 0. Installed Plugins | Auto-detects ork plugin, counts skills/agents | load ${CLAUDE_SKILL_DIR}/rules/diagnostic-checks.md |
| 1. Skills | Frontmatter, context field, token budget, links, activation-channel reachability (no orphaned user-invocable skills) | load ${CLAUDE_SKILL_DIR}/references/skills-validation.md |
| 2. Agents | Frontmatter, model, skill refs, tool refs | load ${CLAUDE_SKILL_DIR}/references/agents-validation.md |
| 3. Hooks | hooks.json schema, bundles, async patterns | load ${CLAUDE_SKILL_DIR}/references/hook-validation.md |
Activation-channel orphans (repo / pre-release): a user-invocable skill should be reachable by more than a human typing it — via a chain (another skill references/ork:<skill>), a subagent grant (skills:insrc/agents/*.md), or a background trigger. A skill with none is an "island" that silently rots. In a repo checkout, runnpm run test:manifests:channels(gated in CI viatest:manifests). Fix an island by wiring any one channel, or add it toSTANDALONE_ALLOWLISTwith a justification.
Categories 4-5: System Health
| Category | What It Checks | Reference |
|---|---|---|
| 4. Memory | .claude/memory/ exists, decisions.jsonl integrity, queue depth | load ${CLAUDE_SKILL_DIR}/references/memory-health.md |
| 5. Build | plugins/ sync with src/, manifest counts, orphans | load ${CLAUDE_SKILL_DIR}/rules/diagnostic-checks.md |
Categories 6-9: Infrastructure
| Category | What It Checks |
|---|---|
| 6. Permission Rules | Unreachable rules detection |
| 7. Schema Compliance | JSON files against schemas |
| 8. Coordination | Multi-worktree lock health, stale locks, sparse paths config |
| 9. Context Budget | Token usage against budget |
Categories 10-14: Environment
| Category | What It Checks | Reference |
|---|---|---|
| 10. CC Version & Channel | Runtime version against minimum required, release channel (stable/beta/alpha) | load ${CLAUDE_SKILL_DIR}/references/version-compatibility.md |
| 11. External Deps | Optional tools (agent-browser, portless) | load ${CLAUDE_SKILL_DIR}/rules/diagnostic-checks.md |
| 12. MCP Status | Enabled/disabled state, credential checks, HIGH-tier `@latest` pinning warn | load ${CLAUDE_SKILL_DIR}/rules/mcp-status-checks.md + ${CLAUDE_SKILL_DIR}/references/mcp-pinning-check.md |
| 13. Plugin Validate | Official CC frontmatter + hooks.json validation (CC >= 2.1.77) | load ${CLAUDE_SKILL_DIR}/rules/diagnostic-checks.md |
| 14. Effort/Model | Detects xhigh effort configured without Opus 4.8 — see below | inline |
Category 14: Effort/Model Compatibility (CC 2.1.111+)
CC 2.1.111 added xhigh effort (Opus 4.8; since CC 2.1.154 it defaults to high and takes xhigh for the hardest tasks). Using it with a model that doesn't support it silently falls back to high — producing no error but losing the extra deepening pass documented in the affected skills.
Detection:
- If the active model does NOT support
xhigh(i.e. not Opus 4.8), check whether/effortis set toxhigh: - Read
.claude/settings.json→effortfield - Read
$ORCHESTKIT_EFFORTenv var (populated by the effort-detector hook) - Check for any skill invocation under
.claude/chain/*.jsonthat explicitly seteffort: xhighwith a non-Opus-4.8 model in scope
Warning format:
WARNING: xhigh effort requires Opus 4.8.
Current model: <model-id>
Configured effort: xhigh
Impact: Skills fall back to high — xhigh's extra deepening pass is lost silently.
Fix: Either switch to Opus 4.8 (`claude --model opus-4-8`) or lower effort to `high`.Exit code: Non-zero in --json mode; soft warning in interactive mode.
Report Format
Load Read("${CLAUDE_SKILL_DIR}/references/report-format.md") for ASCII report templates, JSON CI output schema, and exit codes.Interpreting Results & Troubleshooting
Load Read("${CLAUDE_SKILL_DIR}/references/remediation-guide.md") for the full results interpretation table and troubleshooting steps for common failures (skills validation, build sync, memory).Bisect with `--safe-mode` (CC 2.1.169+): when doctor findings don't explain a misbehaving session, restart withclaude --safe-mode(orCLAUDE_CODE_SAFE_MODE=1) — it disables ALL customizations (CLAUDE.md, plugins incl. ork, skills, hooks, MCP). If the problem disappears, it's a customization; re-enable halves to isolate. If it persists, it's CC itself — file upstream.
After you fix an issue
CC 2.1.69+: Run /reload-plugins to activate plugin changes in the current session without restarting.>
CC 2.1.116+:/reload-pluginsand background plugin auto-update now auto-install missing plugin dependencies from marketplaces you've already added. Ifork:doctorflagged a plugin-load failure due to a missing dep,/reload-pluginsresolves it in place — no manualplugin installstep needed.
>
CC 2.1.152+: For non-plugin skills in a skill directory (~/.claude/skills/or.claude/skills/), run/reload-skillsto re-scan without restarting — the skill analogue of/reload-plugins.
Chain: Deeper Audit
After a clean health report, audit the observability pipeline itself:
>
```
/ork:telemetry-inspect
```
>
doctorvalidates structure (manifests, hooks, skills, agents);/ork:telemetry-inspectvalidates the data plane — every telemetry writer's row count, schema lock, growth trend, and orphaned analytics files that structural checks don't cover.
Related Skills
ork:configure- Configure plugin settingsork:telemetry-inspect- Audit the telemetry/analytics pipeline after a clean structural checkork:quality-gates- CI/CD integrationsecurity-scanning- Comprehensive audits
References
Load on demand with Read("${CLAUDE_SKILL_DIR}/references/<file>") or Read("${CLAUDE_SKILL_DIR}/rules/<file>"):
| File | Content |
|---|---|
rules/diagnostic-checks.md | Bash commands and validation logic per category |
rules/mcp-status-checks.md | Credential validation and misconfiguration detection |
references/remediation-guide.md | Results interpretation and troubleshooting steps |
references/health-check-outputs.md | Sample output per category |
references/skills-validation.md | Skills frontmatter and structure checks |
references/agents-validation.md | Agents frontmatter and tool ref checks |
references/hook-validation.md | Hook registration and bundle checks |
references/memory-health.md | Memory system integrity checks |
references/permission-rules.md | Permission rule detection |
references/schema-validation.md | JSON schema compliance |
references/report-format.md | ASCII report templates and JSON CI output |
references/version-compatibility.md | CC version and channel validation |
references/mcp-pinning-check.md | HIGH-tier MCP @latest warning logic + tier source-of-truth |
Agents Validation
Overview
OrchestKit includes 30 specialized agents validated against CC 2.1.69 frontmatter format.
Agent Structure
src/agents/
├── backend-system-architect.md
├── code-quality-reviewer.md
├── frontend-ui-developer.md
└── ... (38 total)Validation Checks
1. Frontmatter Fields
Required fields:
name- Agent identifier (used in Task subagent_type)description- Purpose and auto-mode keywordsmodel- opus, sonnet, or haikutools- Array of allowed CC toolsskills- Array of skill names to auto-inject
Optional fields:
context- fork or inheritcolor- Display colorhooks- Agent-specific hooks
2. Model Validation
Only valid models:
# Check model values
grep "^model:" src/agents/*.md | sort | uniq -cExpected: opus, sonnet, haiku
3. Skills References
All skills in agent frontmatter must exist:
# Check skill references
for agent in src/agents/*.md; do
grep -A100 "^skills:" "$agent" | grep "^ - " | \
sed 's/.*- //' | while read skill; do
[ -d "src/skills/$skill" ] || echo "Missing: $agent -> $skill"
done
done4. Tools Validation
Valid CC tools:
- Bash, Read, Write, Edit, MultiEdit
- Grep, Glob
- Task, Skill
- WebFetch, WebSearch
- NotebookEdit
- AskUserQuestion
- TaskCreate, TaskUpdate, TaskGet, TaskList
Quick Validation
# Run full agent validation
npm run test:agents
# Or directly
./tests/agents/test-agent-frontmatter.shCommon Issues
Invalid model
model: sonnet # Valid: opus, sonnet, haikuMissing skill reference
Ensure skill exists in src/skills/ directory.
Invalid tool name
Check tool spelling matches CC tool names exactly.
Agent Registration Check (CC 2.1.50+)
Run claude agents to list all registered agents and compare against the expected count from manifests.
Gate: Only run if CC >= 2.1.50 (feature: claude_agents_cli). Skip with a note if version is older.
# Check registered agent count matches expected
expected_count=$(grep -c '"agents/' manifests/ork.json 2>/dev/null || echo 0)
registered_count=$(claude agents 2>/dev/null | wc -l | tr -d ' ')
if [ "$registered_count" -ne "$expected_count" ]; then
echo "WARN: Agent count mismatch — expected $expected_count, got $registered_count"
# List missing agents for investigation
claude agents 2>/dev/null | sort > /tmp/ork-registered.txt
ls src/agents/*.md 2>/dev/null | xargs -I{} basename {} .md | sort > /tmp/ork-expected.txt
echo "Missing agents:"
comm -23 /tmp/ork-expected.txt /tmp/ork-registered.txt
fiCheck: claude agents | wc -l should match expected agent count (38).
Model Routing
See docs/model-routing.md for per-agent model assignment rationale and version history.
Fail action: List missing agents for manual investigation. Common causes:
- Plugin not installed or not rebuilt after adding agents
- Agent frontmatter parse error preventing registration
- CC version too old (< 2.1.50) to support
claude agentsCLI
Health Check Output Examples
Reference output examples for each /ork:doctor check category.
Channel Detection
Stable:
Channel: stable (v7.0.0)Beta:
Channel: beta (v7.0.0-beta.3)
⚠ You are on the beta channel. Report issues at github.com/yonatangross/orchestkit/issuesAlpha:
Channel: alpha (v7.0.0-alpha.1)
⚠ You are on the alpha channel. Expect breaking changes. Report issues at github.com/yonatangross/orchestkit/issuesDetection logic: 1. Read version from .claude-plugin/plugin.json (version field) or version.txt 2. If version contains -alpha → alpha channel 3. If version contains -beta → beta channel 4. Otherwise → stable channel
Installed Plugins
Installed Plugins: 1
- ork: 111 skills, 37 agents, 173 hook entriesSkills Validation
Skills: 79/79 valid
- User-invocable: 18 commands
- Reference skills: 61Agents Validation
Agents: 38/38 valid
- Models: 12 sonnet, 15 haiku, 8 opus
- All skill references validHook Health
Hooks: 95/95 entries valid (12 bundles)
- Global: 34, Agent-scoped: 54, Skill-scoped: 7
- Async hooks: 9 (native async)
- Error Rate: 0.3%Memory System
Memory System: healthy
- Graph Memory: 42 decisions, 0 corrupt, queue depth 3Build System
Build System: in sync
- Skills: 69 src/ = 69 plugins/
- Agents: 38 src/ = 38 plugins/
- Last build: 2 minutes agoPermission Rules
Permission Rules: 12/12 reachableSchema Compliance
Schemas: 15/15 compliantCoordination System
Coordination: healthy
- Active instances: 1
- Stale locks: 0Context Budget
Context Budget: 1850/2200 tokens (84%)Claude Code Version
OK:
Claude Code: 2.1.47 (OK)
- Minimum required: 2.1.47
- All 15 features availableDegraded:
Claude Code: 2.1.44 (DEGRADED)
- Minimum required: 2.1.47
- Missing: last_assistant_message, added_dirs, Windows hooks, worktree discovery
- Upgrade: npm install -g @anthropic-ai/claude-code@latestExternal Dependencies
Installed (agent-browser ≥ 0.26 — structured doctor):
External Dependencies:
agent-browser: OK (chrome=128.0.6613.137, net=✓)Installed but degraded:
External Dependencies:
agent-browser: DEGRADED (chrome=128.0.6613.137, net=✗)
↳ HIGH: daemon not reachable on port 9222 — try `agent-browser daemon restart`Installed (agent-browser < 0.26 — legacy fuzzy probe):
External Dependencies:
agent-browser: installed (legacy probe — upgrade to 0.26+ for structured doctor)Not installed:
External Dependencies:
agent-browser: NOT INSTALLED (optional — install via vercel-labs/agent-browser ≥ 0.26)Plugin Validate
Pass (CC >= 2.1.77):
Plugin Validate: PASSED
- claude plugin validate: 0 errors, 0 warningsFail (CC >= 2.1.77):
Plugin Validate: FAILED
- claude plugin validate: 2 errors
- src/skills/broken/SKILL.md: missing required field "description"
- src/hooks/hooks.json: invalid matcher pattern at hooks[3]
- Fix errors and re-run: npm run build && claude plugin validateSkipped (CC < 2.1.77):
Plugin Validate: SKIPPED (requires CC >= 2.1.77)
- Falling back to OrchestKit custom validation onlyManaged Settings Policy (CC >= 2.1.92)
OK — forceRemoteSettingsRefresh with endpoint:
Managed Settings: OK
- forceRemoteSettingsRefresh: enabled
- Remote endpoint configuredWarning — forceRemoteSettingsRefresh without endpoint:
Managed Settings: WARNING
- forceRemoteSettingsRefresh: enabled but no remote settings endpoint detected
- This will block startup if network is unavailable
- Configure a remote endpoint or remove forceRemoteSettingsRefreshInfo — not set:
Managed Settings: OK (default)
- forceRemoteSettingsRefresh: not set (falls back to cached settings)MCP connector conflict (CC >= 2.1.92 fix):
MCP Servers: WARNING
- Plugin MCP server "{name}" duplicates a claude.ai connector
- Prior to CC 2.1.92 this caused stuck "connecting" state
- Consider setting ENABLE_CLAUDEAI_MCP_SERVERS=false or renaming the plugin serverHook Validation
Overview
OrchestKit uses 143 global hook entries across 27 event types, compiled into 11 bundles. This reference explains how to validate and troubleshoot hooks.
Hook Architecture
hooks.json (63 global + 22 agent-scoped + 1 skill-scoped entries)
↓
12 TypeScript bundles (dist/*.mjs)
↓
9 async hooks use fire-and-forget patternPlatform Support
| Platform | Hook Status | Notes |
|---|---|---|
| macOS | Full support | Native execution |
| Linux | Full support | Native execution |
| Windows | Full support (CC 2.1.47+) | Uses Git Bash instead of cmd.exe |
Windows support history:
- Before CC 2.1.47: All hooks silently failed on Windows (cmd.exe incompatible)
- CC 2.1.47: Fixed by executing hooks via Git Bash instead of cmd.exe
- PR #645: OrchestKit added Windows-safe spawning (no console flashing, no ENAMETOOLONG)
Cross-platform safety measures in OrchestKit hooks:
paths.tsprovides cross-platform path handling (os.homedir(),os.tmpdir(),path.join())- CRLF normalization (
\r\n→\n) in subagent-validator, decision-history, common.ts - Windows backslash path normalization in structure-location-validator
- Windows-specific test cases for paths, CRLF, and permission handling
Hook Categories
| Event Type | Count | Purpose |
|---|---|---|
| PreToolUse | 14 | Before tool execution |
| SubagentStart | 7 | Before agent spawn |
| SubagentStop | 7 | After agent completes |
| PostToolUse | 6 | After tool execution |
| Setup | 6 | Plugin initialization |
| SessionStart | 5 | Session initialization |
| UserPromptSubmit | 5 | Prompt enhancement |
| PermissionRequest | 3 | Auto-approval logic |
| SessionEnd | 3 | Session cleanup |
| TeammateIdle | 3 | Teammate idle handling |
| Stop | 2 | Conversation end |
| Notification | 2 | Desktop/sound alerts |
| PostToolUseFailure | 1 | Failed tool handling |
| PreCompact | 1 | Before context compaction |
| TaskCompleted | 1 | Task completion handling |
| Total Global | 66 |
Bundle Structure
| Bundle | Handlers | Async |
|---|---|---|
| agent.mjs | Agent hooks | No |
| hooks.mjs | Core dispatching | No |
| lifecycle.mjs | Session lifecycle | Yes |
| notification.mjs | Alerts | Yes |
| permission.mjs | Auto-approve | No |
| posttool.mjs | Post-execution | Yes |
| pretool.mjs | Pre-execution | No |
| prompt.mjs | Prompt enhancement | Yes |
| setup.mjs | Initialization | Yes |
| skill.mjs | Skill-scoped hooks | No |
| stop.mjs | Conversation end | Yes |
| subagent.mjs | Agent lifecycle | No |
Validation Checks
1. hooks.json Schema
# Validate hooks.json structure
cat src/hooks/hooks.json | jq '.hooks | keys'2. Bundle Existence
# Check all bundles exist
ls -la src/hooks/dist/*.mjs3. Async Hook Pattern
Async hooks use fire-and-forget scripts:
# 9 fire-and-forget scripts required (updated CC 2.1.47)
ls src/hooks/bin/*-fire-and-forget.mjs4. Matcher Syntax
Valid matcher patterns:
{
"matcher": "Bash", // Exact tool name
"matcher": "Write|Edit", // Multiple tools
"matcher": "*", // All tools
"matcher": "mcp__*" // Wildcard prefix
}Debug Mode (Issue #243)
Silent hooks run in detached background processes. Enable debug mode to monitor them.
Enable Debug Logging
Create .claude/hooks/debug.json:
{
"enabled": true,
"verbose": false,
"includeInput": false,
"hookFilters": []
}View Debug Logs
# Recent background hook activity
tail -f .claude/logs/background-hooks.log
# Filter by hook name
grep "unified-dispatcher" .claude/logs/background-hooks.logHook Metrics
Execution metrics tracked in .claude/hooks/metrics.json:
{
"hooks": {
"posttool/unified-dispatcher": {
"totalRuns": 42,
"successCount": 41,
"errorCount": 1,
"avgDurationMs": 150
}
}
}PID Tracking
# Check for orphaned processes
for f in .claude/hooks/pids/*.pid; do
pid=$(jq -r '.pid' "$f" 2>/dev/null)
if [ -n "$pid" ] && ! kill -0 "$pid" 2>/dev/null; then
echo "Orphaned: $f"
rm "$f"
fi
doneTroubleshooting
Hook not firing
1. Check matcher pattern matches tool name 2. Verify bundle exists in dist/ 3. Check hooks.json registration
Hook timing out
Default timeout: 120s (bash), 600s (CC 2.1.3+)
Exit Codes
| Code | Meaning |
|---|---|
| 0 | Success / Allow |
| 1 | Error (logged, continues) |
| 2 | Block (stops execution) |
MCP Pinning Check (Category 12 sub-check)
Warns when .mcp.json resolves HIGH-tier MCP servers to @latest. HIGH-tier upstream packages are pre-1.0 or beta-surface — a breaking change can land on any npx -y fetch with no signal.
Closes: #1462. Split from #1446 (MCP audit).
Tier Source-of-Truth
src/skills/mcp-patterns/references/mcp-version-matrix.md
| Tier | Criteria | Packages |
|---|---|---|
| HIGH | Pre-1.0 upstream, API may change without notice | @21st-dev/magic, agentation-mcp |
| MEDIUM | Active upstream, semver, used in many skills | @upstash/context7-mcp, tavily-mcp, fal-ai-mcp |
| LOW | Stable API, calendar-versioned | @modelcontextprotocol/server-sequential-thinking, @modelcontextprotocol/server-memory |
Detection
The check parses .mcp.json and for each non-disabled entry extracts the npm package + version specifier from args (handles both npx -y pkg@x and sh -c "... npx -y pkg@x" shapes). Local servers (e.g., node ./server.mjs) are skipped.
For each enabled remote server:
| Spec | Tier | Behavior |
|---|---|---|
pkg@<concrete> | any | OK |
pkg or pkg@latest | HIGH | WARN (exit 1) |
pkg or pkg@latest | MEDIUM | informational only (exit 0) |
pkg or pkg@latest | LOW | silent (exit 0) |
Output
HIGH-tier @latest detected:
MCP pinning: 2 HIGH-tier server(s) resolve to @latest
⚠ 21st-dev-magic (@21st-dev/magic)
⚠ agentation (agentation-mcp)
→ Consider pinning to concrete versions in .mcp.json
→ See src/skills/mcp-patterns/references/mcp-version-matrix.mdMEDIUM-tier only:
MCP pinning: 3 MEDIUM-tier server(s) at @latest (informational)
ℹ context7 (@upstash/context7-mcp)
ℹ tavily (tavily-mcp)
ℹ fal (fal-ai-mcp)OK:
MCP pinning: OK (no HIGH-tier @latest entries)Script
src/skills/doctor/scripts/check-mcp-pinning.sh — invokable standalone or via /ork:doctor Category 12.
# Standalone
src/skills/doctor/scripts/check-mcp-pinning.sh
# JSON for CI
src/skills/doctor/scripts/check-mcp-pinning.sh --json
# Test fixture
src/skills/doctor/scripts/check-mcp-pinning.sh --mcp-json /tmp/test.jsonExit codes: 0 = OK or absent, 1 = HIGH-tier @latest found, 2 = usage error.
Tests
tests/skills/test-mcp-pinning-check.sh — 18 assertions across 7 fixtures covering HIGH/MEDIUM/LOW tier classification, disabled entries, local servers, JSON output shape, and missing-file handling.
Memory Health
Overview
OrchestKit uses graph memory for knowledge persistence. Doctor validates it with automated checks.
Automated Health Check
The memory-health.ts library provides checkMemoryHealth() which returns a MemoryHealthReport:
interface MemoryHealthReport {
overall: 'healthy' | 'degraded' | 'unavailable';
timestamp: string;
graph: {
status: TierStatus;
memoryDir: boolean; // .claude/memory/ exists
decisions: FileHealth; // decisions.jsonl analysis
graphQueue: FileHealth; // graph-queue.jsonl depth
};
}Each FileHealth includes: exists, lineCount, corruptLines, sizeBytes, lastModified.
Status Meanings
| Status | Meaning |
|---|---|
healthy | Graph memory operational, no issues |
degraded | Working but with issues (corrupt data, high queue depth) |
unavailable | Not configured or missing critical components |
Graph Memory
Local knowledge graph stored in .claude/memory/.
Validation Commands
# Check directory exists
ls -la .claude/memory/
# Count decisions
wc -l .claude/memory/decisions.jsonl
# Check graph queue depth
wc -l .claude/memory/graph-queue.jsonl 2>/dev/null
# Validate JSONL integrity (each line must be valid JSON)
while IFS= read -r line; do
echo "$line" | python3 -m json.tool > /dev/null 2>&1 || echo "CORRUPT: $line"
done < .claude/memory/decisions.jsonlHealth Indicators
- Directory exists:
.claude/memory/ decisions.jsonl: Valid JSONL, no corrupt linesgraph-queue.jsonl: Queue depth < 50 (high depth = sync backlog)- No corruption: every line parses as valid JSON
Degraded Conditions
- Corrupt lines: One or more JSONL lines fail to parse
- High queue depth: >50 pending graph operations (sync backlog)
- Missing directory: Graph memory never initialized
Troubleshooting
Graph memory missing
# Initialize graph
mkdir -p .claude/memoryThe first /ork:remember call will create decisions.jsonl.
Corrupt JSONL lines
# Find corrupt lines
python3 -c "
import json, sys
with open('.claude/memory/decisions.jsonl') as f:
for i, line in enumerate(f, 1):
try: json.loads(line)
except: print(f'Line {i}: {line.strip()[:80]}')
"High queue depth
Queue items accumulate if the stop dispatcher doesn't run (e.g., session crash). The queue-recovery hook processes orphaned queues on the next session start.
Permission Rules Analysis
Overview
CC 2.1.3 added detection for unreachable permission rules. This reference explains how to diagnose and fix permission issues.
Common Issues
1. Unreachable Rules
A rule is unreachable when a more general rule already matches:
// PROBLEM: Second rule never matches
{
"permissions": [
{ "path": "**/*.md", "action": "allow" },
{ "path": "README.md", "action": "deny" } // Unreachable!
]
}Fix: Order rules from specific to general:
{
"permissions": [
{ "path": "README.md", "action": "deny" },
{ "path": "**/*.md", "action": "allow" }
]
}2. Shadowed Rules
When two rules match the same pattern with different actions:
// PROBLEM: Both match, but first wins
{
"permissions": [
{ "matcher": "Bash", "action": "allow" },
{ "matcher": "Bash", "commands": ["rm"], "action": "deny" }
]
}Fix: Use more specific matchers:
{
"permissions": [
{ "matcher": "Bash", "commands": ["rm", "rm -rf"], "action": "deny" },
{ "matcher": "Bash", "action": "allow" }
]
}3. Invalid Patterns
Glob patterns that will never match:
// PROBLEM: Typo in pattern
{
"permissions": [
{ "path": "**.md", "action": "allow" } // Should be **/*.md
]
}Validation Commands
# Check for unreachable rules
jq '.permissions // [] | to_entries | map(select(.value.action == "allow"))' \
.claude/settings.json
# List all permission matchers
jq '.permissions // [] | map(.matcher) | unique' .claude/settings.jsonBest Practices
1. Order rules from most specific to least specific 2. Use explicit deny rules before catch-all allow rules 3. Test rules with actual tool invocations 4. Review rules after plugin updates
Remediation Guide
Quick remediation steps for common /ork:doctor findings.
Interpreting Results
| Status | Meaning | Action |
|---|---|---|
| All checks pass | Plugin healthy | None required |
| Skills warning | Invalid frontmatter | Run npm run test:skills |
| Agents warning | Invalid frontmatter | Run npm run test:agents |
| Hook error | Missing/broken hook | Check hooks.json and bundles |
| Memory warning | Graph unavailable | Check .claude/memory/ directory |
| Build warning | Out of sync | Run npm run build |
| Permission warning | Unreachable rules | Review .claude/settings.json |
| Plugin validate error | CC frontmatter/hooks.json invalid | Run claude plugin validate and fix reported errors |
Troubleshooting
"Skills validation failed"
# Run skill structure tests
npm run test:skills
./tests/skills/structure/test-skill-md.sh"Build out of sync"
# Rebuild plugins from source
npm run build"Memory unavailable"
# Check graph memory
ls -la .claude/memory/"Plugin validate failed"
# Run CC's official validator (requires CC >= 2.1.77)
claude plugin validate
# Fix reported errors, then rebuild and re-validate
npm run build
claude plugin validate"Stale plugin paths in PATH"
CC ≤ 2.1.127 occasionally left installed_plugins.json entries pointing at deleted cache directories, polluting PATH for subprocesses. CC 2.1.128+ scrubs those automatically — no maintenance needed at our floor (2.1.168). If you see plugin commands failing with command not found after uninstalling a plugin on CC < 2.1.128, upgrade.
"Auto mode unable to evaluate"
CC 2.1.128+ adds a hint to the auto-mode classifier failure path. When you see:
Auto mode could not evaluate this action.
hint: retry, use /compact, or run with --debugPick whichever applies — /compact if context is full, --debug to see the classifier's reasoning, retry if it was transient.
"Logged out after laptop wake"
If multiple CC sessions all logged themselves out at the same moment after the laptop woke from sleep, that is the pre-2.1.129 OAuth refresh race — concurrent wake-time refreshes invalidated the active token across every running session.
Fix: upgrade to CC ≥ 2.1.129 (our floor is 2.1.170, so anyone on the supported window is already fixed). Recover the session with:
claude /loginThen claude --resume the affected session(s). Checkpoint state in .claude/pipeline-state.json survives — see checkpoint-resume skill for resume semantics.
If logouts after wake persist on CC ≥ 2.1.129, the cause is no longer the race — investigate the refresh token (expired, keychain ACL changed, 1Password locked) instead.
"All concurrent sessions 401 simultaneously"
If you had multiple CC sessions open (worktree-isolated agents, parallel /ork:implement chains, multi-tab work) and every one of them dead-ended at 401 Unauthorized at the same instant — that is the pre-2.1.133 parallel-session refresh-token race. A refresh-token rotation fired in one session, the other sessions raced against it, and they all wound up holding the now-invalidated old token.
Fix: upgrade to CC ≥ 2.1.133 (our floor is 2.1.170, so the supported window is already past this). Recover the stuck sessions with:
claude /loginThen claude --resume the affected sessions. Worktree state in each agent's branch is unaffected.
If "all sessions 401 at once" still reproduces on CC ≥ 2.1.133, the cause is no longer this race — check the refresh token itself (expired, revoked by the IdP, keychain ACL changed, 1Password locked). See also the "Logged out after laptop wake" entry above for the related 2.1.129 wake-from-sleep race.
"EnterWorktree drops my unpushed commits"
If a worktree spawned via EnterWorktree, --worktree, or an agent run with isolation: "worktree" is missing commits you made locally but never pushed — and git log in the new worktree starts from origin/<default-branch> instead of your current HEAD — that is CC 2.1.133's new worktree.baseRef default at work. CC 2.1.133 added the setting with default "fresh", which branches new worktrees from origin/<default> rather than local HEAD (the 2.1.128–2.1.132 behavior).
Fix: set worktree.baseRef: "head" in .claude/settings.json (project) or ~/.claude/settings.json (user):
{
"worktree": {
"baseRef": "head"
}
}After adding the setting, spawn a fresh worktree — the new one will branch from local HEAD and include your unpushed commits. Existing worktrees that were created without the setting need to be recreated; you can recover their work by cherry-picking commits from the original branch first.
See ${CLAUDE_SKILL_DIR}/../chain-patterns/references/worktree-agent-pattern.md for the full agent-isolation context, and ${CLAUDE_SKILL_DIR}/../configure/references/cc-version-settings.md (CC 2.1.133 section) for the upstream change description.
Doctor Report Format
ASCII Report
Full ork plugin:
+===================================================================+
| OrchestKit Health Report |
+===================================================================+
| Version: {version} | CC: {cc_version} | Plugins: ork |
+===================================================================+
| Skills | 67/67 valid |
| Agents | 37/37 valid |
| Hooks | 87/87 entries (12 bundles) |
| Memory | Graph memory healthy |
| MCP | context7 ✓ memory ✓ tavily ○ agentation ○ |
| Permissions | 12/12 reachable |
| Schemas | 15/15 compliant |
| Context | 1850/2200 tokens (84%) |
| Coordination | 0 stale locks |
| CC Version | {cc_version} (OK) |
| Plugin Validate | PASSED (0 errors) |
+===================================================================+
| Status: HEALTHY (11/11 checks passed) |
+===================================================================+Note:{version}is read frompackage.jsonat runtime.{cc_version}is detected from Claude Code. Counts reflect installed plugin — dynamic, not hardcoded.
JSON Output (CI Integration)
/ork:doctor --json{
"version": "{version}",
"claudeCode": "{cc_version}",
"status": "healthy",
"plugins": {
"installed": ["ork"],
"count": 1
},
"checks": {
"skills": {"passed": true, "count": 67, "perPlugin": {"ork": 67}},
"agents": {"passed": true, "count": 37, "perPlugin": {"ork": 37}},
"hooks": {"passed": true, "entries": 87, "bundles": 12, "source": "ork"},
"memory": {"passed": true, "available": ["graph"]},
"mcp": {"passed": true, "servers": {"context7": "enabled", "memory": "enabled", "sequential-thinking": "disabled", "tavily": "disabled", "agentation": "disabled"}},
"permissions": {"passed": true, "count": 12},
"schemas": {"passed": true, "count": 15},
"context": {"passed": true, "usage": 0.84},
"coordination": {"passed": true, "staleLocks": 0},
"ccVersion": {"passed": true, "version": "2.1.47"},
"pluginValidate": {"passed": true, "errors": 0, "warnings": 0, "skipped": false}
},
"exitCode": 0
}Exit Codes
| Code | Meaning |
|---|---|
| 0 | All checks pass |
| 1 | One or more checks failed |
Schema Validation
Overview
OrchestKit uses JSON schemas to validate configuration files. This reference explains how to validate and fix schema issues.
Schemas
Located in .claude/schemas/:
| Schema | Validates |
|---|---|
plugin.schema.json | plugin.json |
skill files | All SKILL.md files |
context.schema.json | Context protocol files |
coordination.schema.json | Work registry and decision log |
Validation Commands
Validate All
./tests/schemas/validate-all.shValidate Specific File
# Using ajv
npx ajv validate \
-s .claude/schemas/skill files \
-d .claude/skills/doctor/SKILL.md
# Using jq for basic structure check
jq empty .claude/skills/doctor/SKILL.mdCommon Schema Errors
Missing Required Field
// ERROR: Missing "description"
{
"name": "my-skill",
"version": "1.0.0"
}Fix: Add all required fields:
{
"$schema": "../../schemas/skill files",
"name": "my-skill",
"version": "1.0.0",
"description": "Description of the skill",
"capabilities": ["capability-1"]
}Invalid Type
// ERROR: capabilities must be array
{
"capabilities": "single-capability"
}Fix: Use correct type:
{
"capabilities": ["single-capability"]
}Pattern Mismatch
// ERROR: version must match semver
{
"version": "1.0"
}Fix: Use proper semver:
{
"version": "1.0.0"
}Batch Validation
# Validate all SKILL.md files
for category in skills/*/.claude/skills; do for f in "$category"/*/SKILL.md; do
npx ajv validate \
-s .claude/schemas/skill files \
-d "$f" || echo "INVALID: $f"
doneCreating Valid Files
Use schema as a template:
# View required fields
jq '.required' .claude/schemas/skill files
# View property types
jq '.properties | to_entries | map({key: .key, type: .value.type})' \
.claude/schemas/skill filesSkills Validation
Overview
OrchestKit includes 111 skills validated against frontmatter requirements and content standards.
Skill Types
| Type | Count | Frontmatter |
|---|---|---|
| User-invocable | 18 | user-invocable: true |
| Internal | 61 | user-invocable: false |
Validation Checks
1. Frontmatter Fields
Required fields:
name- Skill identifierdescription- Brief description with triggersuser-invocable- Boolean for command availability
Optional fields:
context- fork (recommended), inheritagent- Associated agent nametags- Keywords for discoveryversion- Semver versionallowedTools- Tool whitelist
2. Token Budget
Skills must stay within token limits:
- Minimum: 300 tokens
- Maximum: 5000 tokens
# Check skill size
wc -c src/skills/*/SKILL.md | sort -n3. Reference Links
All internal links must resolve:
# Check for broken references
for skill in src/skills/*/SKILL.md; do
grep -o 'references/[^)]*' "$skill" | while read ref; do
dir=$(dirname "$skill")
[ -f "$dir/$ref" ] || echo "Broken: $skill -> $ref"
done
done4. Related Skills
All skills in "Related Skills" section must exist:
# Validate related skill references
grep -h "^- " src/skills/*/SKILL.md | grep -v "http" | \
sed 's/.*`\([^`]*\)`.*/\1/' | sort -uQuick Validation
# Run full skill validation
npm run test:skills
# Or directly
./tests/skills/structure/test-skill-md.shCommon Issues
Missing frontmatter
---
name: my-skill
description: Does something useful
user-invocable: false
---Token budget exceeded
Split into SKILL.md + references/ directory.
Broken reference link
Ensure file exists at references/filename.md.
Doctor Rules Index
Rules for OrchestKit health diagnostics checks.
| Rule | Focus |
|---|---|
| diagnostic-checks | Detailed check procedures for each health category |
| mcp-status-checks | MCP server validation: enabled/disabled state, credential checks |
Diagnostic Check Procedures
Detailed procedures for each health check category in /ork:doctor.
---
Incorrect:
✓ Plugins OK (no actual validation — just assumed healthy)Correct:
✓ Skills: 69/69 valid (frontmatter, token budget, links)
✗ Hooks: dist/memory-writer.mjs missing — run: cd src/hooks && npm run build
✓ Agents: 38/38 CC 2.1.6 compliant0. Installed Plugins Detection
Auto-detects which OrchestKit plugins are installed:
# Detection logic:
# - Scans for .claude-plugin/plugin.json in plugin paths
# - Identifies ork plugin
# - Counts skills/agents per installed plugin---
1. Skills Validation
Validates skills in installed plugins (count varies by installation):
# Checks performed:
# - SKILL.md frontmatter (name, description, user-invocable)
# - context: fork field (required for CC 2.1.0+)
# - Token budget compliance (300-5000 tokens)
# - Internal link validation (references/ paths)
# - Related Skills references exist---
2. Agents Validation
Validates agents in installed plugins:
# Checks performed:
# - Frontmatter fields (name, description, model, tools, skills)
# - Model validation (opus, sonnet, haiku only)
# - Skills references exist in src/skills/
# - Tools are valid CC tools---
3. Hook Health
Verifies hooks are properly configured:
# Checks performed:
# - hooks.json schema valid
# - Bundle files exist (12 .mjs bundles)
# - Async hooks use fire-and-forget pattern (9 async)
# - Background hook metrics health (Issue #243)
# - Windows-safe spawning (PR #645)---
4. Memory System
Validates graph memory with file-level integrity checks:
# Automated checks:
# - Graph: .claude/memory/ exists, decisions.jsonl valid JSONL, queue depth
# Run these commands to gather memory health data:
wc -l .claude/memory/decisions.jsonl 2>/dev/null || echo "No decisions yet"
wc -l .claude/memory/graph-queue.jsonl 2>/dev/null || echo "No graph queue"
ls -la .claude/memory/ 2>/dev/null || echo "Memory directory missing"Read .claude/memory/decisions.jsonl directly to validate JSONL integrity (each line must parse as valid JSON). Count total lines, corrupt lines, and report per-category breakdown.
See Memory Health for details.
---
5. Build System
Verifies plugins/ sync with src/:
# Checks performed:
# - plugins/ generated from src/
# - Manifest counts match actual files
# - No orphaned skills/agents---
6. Permission Rules
Leverages CC 2.1.3's unreachable permission rules detection.
---
7. Schema Compliance
Validates JSON files against schemas.
---
8. Coordination System
Checks multi-worktree coordination health (active instances, stale locks).
---
9. Context Budget
Monitors token usage against budget.
---
10. Claude Code Version
Validates runtime version against the Version Compatibility Matrix.
---
11. External Dependencies
Checks optional tool availability:
# agent-browser (vercel-labs/agent-browser)
# Prefer the structured `agent-browser doctor --json` from 0.26.0+ (CC 2.1.121+).
# Falls back to the fuzzy "is the binary on PATH + symlink present?" probe on older versions.
if command -v agent-browser >/dev/null 2>&1; then
if agent-browser doctor --json >/tmp/ab-doctor.json 2>/dev/null; then
# Structured snapshot: surface only high-severity issues + a one-line health summary.
jq -r '
"agent-browser: " +
(if (.daemon.status // "unknown") == "running" then "OK" else "DEGRADED" end) +
" (chrome=" + (.chrome.version // "?") +
", net=" + (if .network.reachable then "✓" else "✗" end) + ")"
' /tmp/ab-doctor.json
# Promote any high-severity issue into doctor's findings stream.
jq -r '.issues[]? | select(.severity == "high") | " ↳ HIGH: " + .message' /tmp/ab-doctor.json
else
# Fallback for agent-browser < 0.26 (no `doctor` subcommand)
test -L "$HOME/.claude/skills/agent-browser" \
&& echo "agent-browser: installed (legacy probe — upgrade to 0.26+ for structured doctor)" \
|| echo "agent-browser: SYMLINK MISSING at ~/.claude/skills/agent-browser"
fi
else
echo "agent-browser: NOT INSTALLED (optional — install via vercel-labs/agent-browser ≥ 0.26)"
fi
# portless: stable named localhost URLs for local dev
# which portless 2>/dev/null && portless list 2>/dev/null
# If missing: RECOMMEND "npm i -g portless" for stable local dev URLs
# If installed but not running: WARN "portless is installed but no services registered"
# tailscale (M127 #1561): only relevant if user has used /ork:dev --share / --funnel / --live.
# Detected by inspecting .claude/state/dev-stack.json for share != null.
# if [[ -f .claude/state/dev-stack.json ]] && jq -e '.share != null' .claude/state/dev-stack.json >/dev/null 2>&1; then
# command -v tailscale >/dev/null 2>&1 \
# && echo "tailscale: OK (share mode in use: $(jq -r '.share.mode' .claude/state/dev-stack.json))" \
# || echo "tailscale: SHARE MODE ACTIVE BUT TAILSCALE CLI MISSING (Install: brew install tailscale)"
# fi
# Live demos older than 24h (M127 #1565): warns about sprawl from /ork:dev --live.
# live_log=".claude/state/live-demos.jsonl"
# if [[ -f "$live_log" ]]; then
# now_ts=$(date -u +%s)
# while IFS= read -r line; do
# expires=$(printf '%s' "$line" | jq -r '.expiresAt // empty')
# [[ -z "$expires" ]] && continue
# expires_ts=$(date -j -u -f '%Y-%m-%dT%H:%M:%SZ' "$expires" +%s 2>/dev/null \
# || date -u -d "$expires" +%s 2>/dev/null)
# if [[ -n "$expires_ts" && "$expires_ts" -lt "$now_ts" ]]; then
# age_h=$(( (now_ts - expires_ts) / 3600 ))
# echo "live demo: EXPIRED ${age_h}h ago — branch=$(printf '%s' "$line" | jq -r '.branch')"
# fi
# done < "$live_log"
# fiWhy structured doctor: agent-browser 0.26.0 added doctor --json returning a snapshot of chrome, daemon, network, config, security, and providers. Wiring it in turns the previous "agent-browser broken" failure into actionable per-subsystem findings (Chrome version, daemon status, network reachability, high-severity issues), unblocking debug sessions where the user can't tell us what's wrong.
---
13. Plugin Validate (CC >= 2.1.77)
Runs claude plugin validate for official CC validation of frontmatter and hooks.json. This complements OrchestKit's custom checks (categories 1-3) with CC's built-in validator.
# Check CC version supports plugin validate (>= 2.1.77)
# If CC < 2.1.77, skip with: "Plugin validate: SKIPPED (requires CC >= 2.1.77)"
# Run official validation from plugin root
claude plugin validate
# Checks performed by CC:
# - SKILL.md frontmatter schema (required fields, types, allowed values)
# - hooks.json schema (event types, matchers, command paths)
# - Agent frontmatter schema (model, tools, skills fields)
# - File path resolution (command paths in hooks exist)Relationship to OrchestKit checks: claude plugin validate performs structural/schema validation at the CC level. OrchestKit's categories 1-3 perform deeper semantic checks (token budgets, cross-references, async patterns) that CC does not cover. Both should pass for a fully healthy plugin.
---
14. Stale Project State (CC >= 2.1.126, #1582, fixed in #1587)
CC 2.1.126 added claude project purge [path] — deletes all CC state (transcripts, tasks, file history, config entry) for a project. Surface this as an info-severity diagnostic when canonical project paths no longer exist on disk.
# Check CC version supports project purge (>= 2.1.126)
# If CC < 2.1.126, skip silently (suggestion would be unactionable)
# Detect stale project state via the authoritative source.
# Use `claude project purge --dry-run --all` because the directory-name encoding
# under ~/.claude/projects/ is lossy (both `/` and `.` collapse to `-`, so the
# original path cannot be reconstructed deterministically — `my-project` is
# indistinguishable from `my.project` or `my/project`). The CLI's dry-run
# output emits canonical paths from ~/.claude.json which IS lossless.
#
# claude project purge --dry-run --all 2>/dev/null \
# | grep -oE 'projects\["[^"]+"\]' \
# | sed -E 's/^projects\["//; s/"\]$//' \
# | while IFS= read -r p; do
# [ -n "$p" ] && [ ! -d "$p" ] && echo "$p"
# done
#
# Example output (info severity, never blocking):
# ℹ Stale project state: 3 canonical paths reference directories that no longer exist on disk.
# Suggested cleanup (always preview first):
# claude project purge --dry-run --all
# claude project purge --interactive # confirm each projectWhy info, never warn or fail: the user may have moved a project rather than deleted it; aggressive removal would lose transcript history. Always recommend --dry-run first. Mirrors the pattern from claude plugin prune (Category 13b).
Why not parse `~/.claude/projects/` directly: the directory naming is a lossy collapse of the original path (/ and . both become -). A naive sed 's|-|/|g' decode produces ambiguous results — ~/.claude/projects/Users-me-my-project could be /Users/me/my-project, /Users/me/my/project, /Users/me/my.project, or other combinations. The dry-run output above is the canonical source.
MCP Status Checks
Validates .mcp.json entries for enabled/disabled state and required credentials.
Check Procedure
# Checks performed:
# - Parse .mcp.json, list each server with enabled/disabled state
# - For tavily: check TAVILY_API_KEY env var OR op CLI availability
# - For memory: check MEMORY_FILE path is writable
# - For agentation: check agentation-mcp package is installed (npx --yes dry-run)
# - Flag any enabled MCP whose process would likely fail at startup
# - HIGH-tier @latest pinning: see references/mcp-pinning-check.md
# (script: scripts/check-mcp-pinning.sh — exit 1 on HIGH-tier @latest)
# - alwaysLoad audit (CC 2.1.121+, #1541): warn when memory, context7, or
# sequential-thinking lack `"alwaysLoad": true` — these are universally
# used and per-skill ToolSearch probes are wasted work without it.
# Skip the warning on CC < 2.1.121 (key would be silently ignored).
# - claude plugin orphans (CC 2.1.121+, #1544): suggest `claude plugin prune`
# when `claude plugin list --json` shows orphaned auto-installed deps.Incorrect:
MCP Servers: all OKCorrect:
MCP Servers:
- context7: enabled ✓
- tavily: enabled ✗ TAVILY_API_KEY not set — will fail at startupOutput Examples
Healthy:
MCP Servers:
- context7: enabled ✓
- memory: enabled ✓
- sequential-thinking: disabled ○
- tavily: disabled ○ (enable: set TAVILY_API_KEY, see /ork:configure)
- agentation: disabled ○Misconfigured (Tavily enabled but no key):
MCP Servers:
- context7: enabled ✓
- memory: enabled ✓
- tavily: enabled ✗ TAVILY_API_KEY not set — MCP will fail at startup
Fix: set TAVILY_API_KEY or set "disabled": true in .mcp.jsonMisconfigured (agentation enabled but not installed):
MCP Servers:
- agentation: enabled ✗ agentation-mcp package not found
Fix: npm install -D agentation-mcp or set "disabled": true#!/bin/bash
# Generated by OrchestKit Claude Plugin
# Created: 2026-04-25
# Check MCP version pinning in .mcp.json
#
# Warns when HIGH-tier MCP servers (pre-1.0 upstream) resolve to @latest.
# Tier source-of-truth: src/skills/mcp-patterns/references/mcp-version-matrix.md
#
# Usage: ./check-mcp-pinning.sh [--mcp-json PATH] [--json]
#
# Exit codes:
# 0 = no HIGH-tier @latest pins (or .mcp.json absent)
# 1 = at least one HIGH-tier @latest pin found
# 2 = usage / parse error
set -euo pipefail
# =============================================================================
# Tier classification — keep in sync with mcp-version-matrix.md
# =============================================================================
# A package is HIGH-tier when upstream is pre-1.0 or beta-surface; @latest on
# these can break agents silently on next `npx -y` fetch.
HIGH_TIER_PACKAGES=(
"@21st-dev/magic"
"agentation-mcp"
)
MEDIUM_TIER_PACKAGES=(
"@upstash/context7-mcp"
"tavily-mcp"
"fal-ai-mcp"
)
# =============================================================================
# Argument parsing
# =============================================================================
MCP_JSON=""
OUTPUT_FORMAT="table"
while [[ $# -gt 0 ]]; do
case "$1" in
--mcp-json)
MCP_JSON="$2"
shift 2
;;
--json)
OUTPUT_FORMAT="json"
shift
;;
--help|-h)
sed -n '2,9p' "$0"
exit 0
;;
*)
echo "Error: Unknown option '$1'." >&2
exit 2
;;
esac
done
# Default to repo-root .mcp.json
if [[ -z "$MCP_JSON" ]]; then
dir="$(pwd)"
while [[ "$dir" != "/" ]]; do
if [[ -f "$dir/.mcp.json" ]]; then
MCP_JSON="$dir/.mcp.json"
break
fi
dir="$(dirname "$dir")"
done
fi
if [[ -z "$MCP_JSON" || ! -f "$MCP_JSON" ]]; then
if [[ "$OUTPUT_FORMAT" == "json" ]]; then
echo '{"status":"skipped","reason":".mcp.json not found","high_tier_unpinned":[],"medium_tier_unpinned":[]}'
else
echo "MCP pinning: skipped (.mcp.json not found)"
fi
exit 0
fi
# =============================================================================
# Parse .mcp.json and detect @latest / unpinned entries
# =============================================================================
# Output one line per server: "<server-name>\t<package>\t<version-spec>"
# version-spec is one of: "latest" | "pinned" | "local" | "unknown"
SERVERS=()
while IFS= read -r line; do
[[ -n "$line" ]] && SERVERS+=("$line")
done < <(python3 - "$MCP_JSON" <<'PY'
import json, re, sys
with open(sys.argv[1]) as f:
data = json.load(f)
for name, cfg in (data.get("mcpServers") or {}).items():
if cfg.get("disabled"):
continue
args = cfg.get("args") or []
cmd = cfg.get("command") or ""
# find first npm-style package token in args (handles `npx -y pkg@x` and
# `sh -c "... npx -y pkg@x"` variants)
candidate = None
for token in args:
# normalize: split on whitespace inside shell -c strings too
for sub in re.split(r"\s+", token):
if sub.startswith("@") and "/" in sub: # scoped: @scope/pkg[@ver]
candidate = sub
break
if re.match(r"^[a-z0-9][a-z0-9._-]*(@.+)?$", sub) and (
"mcp" in sub or sub.endswith("@latest")
):
candidate = sub
break
if candidate:
break
if not candidate:
# local server (e.g., node ./server.mjs) — skip
if cmd in ("node", "python", "python3") or any(
t.endswith(".mjs") or t.endswith(".js") or t.endswith(".py") for t in args
):
print(f"{name}\t<local>\tlocal")
else:
print(f"{name}\t<unknown>\tunknown")
continue
# extract pkg + version spec
if candidate.startswith("@"):
# scoped: @scope/name[@ver]
m = re.match(r"^(@[^/]+/[^@]+)(?:@(.+))?$", candidate)
if m:
pkg, ver = m.group(1), m.group(2)
else:
pkg, ver = candidate, None
else:
m = re.match(r"^([^@]+)(?:@(.+))?$", candidate)
if m:
pkg, ver = m.group(1), m.group(2)
else:
pkg, ver = candidate, None
spec = "latest" if (ver is None or ver == "latest") else "pinned"
print(f"{name}\t{pkg}\t{spec}")
PY
)
# =============================================================================
# Classify
# =============================================================================
HIGH_UNPINNED=()
MEDIUM_UNPINNED=()
LOW_UNPINNED=()
PINNED=()
contains() {
local needle="$1"; shift
for hay in "$@"; do
[[ "$hay" == "$needle" ]] && return 0
done
return 1
}
if [[ ${#SERVERS[@]} -gt 0 ]]; then
for line in "${SERVERS[@]}"; do
IFS=$'\t' read -r name pkg spec <<<"$line"
case "$spec" in
local|unknown)
continue
;;
pinned)
PINNED+=("$name ($pkg)")
;;
latest)
if contains "$pkg" "${HIGH_TIER_PACKAGES[@]}"; then
HIGH_UNPINNED+=("$name ($pkg)")
elif contains "$pkg" "${MEDIUM_TIER_PACKAGES[@]}"; then
MEDIUM_UNPINNED+=("$name ($pkg)")
else
LOW_UNPINNED+=("$name ($pkg)")
fi
;;
esac
done
fi
# =============================================================================
# Output
# =============================================================================
EXIT_CODE=0
[[ ${#HIGH_UNPINNED[@]} -gt 0 ]] && EXIT_CODE=1
if [[ "$OUTPUT_FORMAT" == "json" ]]; then
join_q() {
local first=1
printf '['
for x in "$@"; do
[[ $first -eq 1 ]] || printf ','
printf '"%s"' "$x"
first=0
done
printf ']'
}
printf '{"status":"%s",' "$(if [[ $EXIT_CODE -eq 0 ]]; then echo ok; else echo warning; fi)"
printf '"high_tier_unpinned":'
join_q "${HIGH_UNPINNED[@]+"${HIGH_UNPINNED[@]}"}"
printf ',"medium_tier_unpinned":'
join_q "${MEDIUM_UNPINNED[@]+"${MEDIUM_UNPINNED[@]}"}"
printf ',"low_tier_unpinned":'
join_q "${LOW_UNPINNED[@]+"${LOW_UNPINNED[@]}"}"
printf ',"pinned":'
join_q "${PINNED[@]+"${PINNED[@]}"}"
printf '}\n'
else
if [[ ${#HIGH_UNPINNED[@]} -gt 0 ]]; then
echo "MCP pinning: ${#HIGH_UNPINNED[@]} HIGH-tier server(s) resolve to @latest"
for s in "${HIGH_UNPINNED[@]}"; do
echo " ⚠ $s"
done
echo " → Consider pinning to concrete versions in .mcp.json"
echo " → See src/skills/mcp-patterns/references/mcp-version-matrix.md"
elif [[ ${#MEDIUM_UNPINNED[@]} -gt 0 ]]; then
echo "MCP pinning: ${#MEDIUM_UNPINNED[@]} MEDIUM-tier server(s) at @latest (informational)"
for s in "${MEDIUM_UNPINNED[@]}"; do
echo " ℹ $s"
done
else
echo "MCP pinning: OK (no HIGH-tier @latest entries)"
fi
fi
exit $EXIT_CODE
#!/bin/bash
# Check Plugin Health
# Validates OrchestKit plugin ecosystem: manifests, skills, hooks, and build output.
#
# Usage: ./check-plugin-health.sh [OPTIONS]
#
# Options:
# --json Output results as JSON
# --help Show this help message
#
# Exit codes:
# 0 = all checks pass
# 1 = one or more checks failed
# 2 = usage error (not in repo root)
set -euo pipefail
# =============================================================================
# CONFIGURATION
# =============================================================================
OUTPUT_FORMAT="table"
PROJECT_ROOT=""
# Detect project root (walk up to find manifests/)
find_project_root() {
local dir
dir=$(pwd)
while [[ "$dir" != "/" ]]; do
if [[ -d "$dir/manifests" && -d "$dir/src/skills" ]]; then
echo "$dir"
return 0
fi
dir=$(dirname "$dir")
done
return 1
}
# =============================================================================
# ARGUMENT PARSING
# =============================================================================
while [[ $# -gt 0 ]]; do
case "$1" in
--json)
OUTPUT_FORMAT="json"
shift
;;
--help|-h)
echo "Check Plugin Health"
echo ""
echo "Usage: $0 [OPTIONS]"
echo ""
echo "Options:"
echo " --json Output results as JSON"
echo " --help Show this help message"
echo ""
echo "Checks:"
echo " 1. plugins/ directory populated"
echo " 2. Manifest files are valid JSON"
echo " 3. Skill count matches manifest expectations"
echo " 4. Hook dist/ compiled output exists"
echo " 5. Agent definitions present"
echo ""
echo "Exit codes:"
echo " 0 = all healthy"
echo " 1 = issues found"
echo " 2 = usage error"
exit 0
;;
*)
echo "Error: Unknown option '$1'. Use --help for usage." >&2
exit 2
;;
esac
done
# =============================================================================
# DETECT PROJECT ROOT
# =============================================================================
PROJECT_ROOT=$(find_project_root) || {
echo "Error: Could not find OrchestKit project root (manifests/ + src/skills/)." >&2
echo "Run this script from within the OrchestKit repository." >&2
exit 2
}
# =============================================================================
# HEALTH CHECKS
# =============================================================================
declare -a CHECK_NAMES=()
declare -a CHECK_STATUSES=()
declare -a CHECK_DETAILS=()
add_result() {
CHECK_NAMES+=("$1")
CHECK_STATUSES+=("$2")
CHECK_DETAILS+=("$3")
}
OVERALL_STATUS=0
# --- Check 1: plugins/ directory populated ---
if [[ -d "$PROJECT_ROOT/plugins" ]]; then
plugin_count=$(find "$PROJECT_ROOT/plugins" -name "*.md" -o -name "*.json" 2>/dev/null | wc -l | tr -d ' ')
if [[ "$plugin_count" -gt 0 ]]; then
add_result "plugins/ populated" "OK" "$plugin_count files found"
else
add_result "plugins/ populated" "FAIL" "Directory exists but is empty (run npm run build)"
OVERALL_STATUS=1
fi
else
add_result "plugins/ populated" "FAIL" "Directory missing (run npm run build)"
OVERALL_STATUS=1
fi
# --- Check 2: Manifest files valid JSON ---
manifest_ok=0
manifest_fail=0
manifest_errors=""
for manifest in "$PROJECT_ROOT"/manifests/*.json; do
if [[ -f "$manifest" ]]; then
manifest_name=$(basename "$manifest")
if python3 -c "import json, sys; json.load(open(sys.argv[1]))" "$manifest" 2>/dev/null; then
manifest_ok=$((manifest_ok + 1))
else
manifest_fail=$((manifest_fail + 1))
manifest_errors="${manifest_errors}${manifest_name}, "
OVERALL_STATUS=1
fi
fi
done
if [[ $manifest_fail -eq 0 ]]; then
add_result "Manifests valid JSON" "OK" "$manifest_ok manifest(s) validated"
else
add_result "Manifests valid JSON" "FAIL" "Invalid: ${manifest_errors%, }"
fi
# --- Check 3: Skill count matches manifest ---
src_skill_count=$(find "$PROJECT_ROOT/src/skills" -maxdepth 1 -mindepth 1 -type d 2>/dev/null | wc -l | tr -d ' ')
# Check ork.json skill count
if [[ -f "$PROJECT_ROOT/manifests/ork.json" ]]; then
manifest_skill_count=$(python3 -c "
import json, sys
with open(sys.argv[1]) as f:
data = json.load(f)
skills = data.get('skills', [])
print(len(skills))
" "$PROJECT_ROOT/manifests/ork.json" 2>/dev/null || echo "0")
if [[ "$src_skill_count" -eq "$manifest_skill_count" ]]; then
add_result "Skill count (ork)" "OK" "$src_skill_count skills in src/ match $manifest_skill_count in manifest"
else
add_result "Skill count (ork)" "WARN" "src/ has $src_skill_count skills, ork.json lists $manifest_skill_count"
fi
else
add_result "Skill count (ork)" "FAIL" "manifests/ork.json not found"
OVERALL_STATUS=1
fi
# --- Check 4: Hooks compiled ---
if [[ -d "$PROJECT_ROOT/src/hooks" ]]; then
if [[ -d "$PROJECT_ROOT/src/hooks/dist" ]]; then
hook_files=$(find "$PROJECT_ROOT/src/hooks/dist" -name "*.js" -o -name "*.mjs" 2>/dev/null | wc -l | tr -d ' ')
if [[ "$hook_files" -gt 0 ]]; then
add_result "Hooks compiled" "OK" "$hook_files compiled files in dist/"
else
add_result "Hooks compiled" "WARN" "dist/ exists but no compiled files found"
fi
else
add_result "Hooks compiled" "WARN" "src/hooks/dist/ not found (run: cd src/hooks && npm run build)"
fi
if [[ -f "$PROJECT_ROOT/src/hooks/hooks.json" ]]; then
hook_count=$(python3 -c "
import json, sys
with open(sys.argv[1]) as f:
data = json.load(f)
print(len(data.get('hooks', data) if isinstance(data, dict) else data))
" "$PROJECT_ROOT/src/hooks/hooks.json" 2>/dev/null || echo "?")
add_result "hooks.json entries" "OK" "$hook_count hook entries"
else
add_result "hooks.json entries" "WARN" "hooks.json not found"
fi
else
add_result "Hooks directory" "WARN" "src/hooks/ not found"
fi
# --- Check 5: Agent definitions ---
if [[ -d "$PROJECT_ROOT/src/agents" ]]; then
agent_count=$(find "$PROJECT_ROOT/src/agents" -name "*.md" 2>/dev/null | wc -l | tr -d ' ')
if [[ "$agent_count" -gt 0 ]]; then
add_result "Agent definitions" "OK" "$agent_count agents found"
else
add_result "Agent definitions" "WARN" "No agent .md files found"
fi
else
add_result "Agent definitions" "FAIL" "src/agents/ not found"
OVERALL_STATUS=1
fi
# =============================================================================
# OUTPUT
# =============================================================================
if [[ "$OUTPUT_FORMAT" == "json" ]]; then
echo "{"
echo " \"status\": \"$(if [[ $OVERALL_STATUS -eq 0 ]]; then echo "healthy"; else echo "unhealthy"; fi)\","
echo " \"checks\": ["
for i in "${!CHECK_NAMES[@]}"; do
comma=""
if [[ $i -lt $((${#CHECK_NAMES[@]} - 1)) ]]; then
comma=","
fi
echo " {\"component\": \"${CHECK_NAMES[$i]}\", \"status\": \"${CHECK_STATUSES[$i]}\", \"details\": \"${CHECK_DETAILS[$i]}\"}${comma}"
done
echo " ]"
echo "}"
else
echo "============================================================================"
echo " OrchestKit Plugin Health Check"
echo "============================================================================"
echo ""
printf "%-30s %-8s %s\n" "Component" "Status" "Details"
echo "----------------------------------------------------------------------------"
for i in "${!CHECK_NAMES[@]}"; do
status="${CHECK_STATUSES[$i]}"
# Color output if terminal supports it
if [[ -t 1 ]]; then
case "$status" in
OK) status=$'\033[0;32mOK\033[0m ' ;;
WARN) status=$'\033[1;33mWARN\033[0m ' ;;
FAIL) status=$'\033[0;31mFAIL\033[0m ' ;;
esac
fi
printf "%-30s %-8s %s\n" "${CHECK_NAMES[$i]}" "$status" "${CHECK_DETAILS[$i]}"
done
echo ""
echo "============================================================================"
if [[ $OVERALL_STATUS -eq 0 ]]; then
echo " All checks passed"
else
echo " Issues found - review above"
fi
echo "============================================================================"
fi
exit $OVERALL_STATUS
{
"skill": "doctor",
"version": "1.0.0",
"testCases": [
{
"id": "basic-orkdoctor",
"rule": "",
"query": "/ork:doctor",
"expectedBehavior": [
"Auto-detects installed ork plugin and its version from the manifest",
"Validates all 13 health check categories in sequence",
"Checks skills frontmatter, agents validation, hook registration",
"Verifies permission rules for unreachable entries",
"Checks Claude Code version meets minimum >= 2.1.59",
"Reports MCP server status: active, disabled, or misconfigured",
"Produces formatted health report with pass/warn/fail per category"
]
},
{
"id": "edge-orkdoctor-categoryhooks",
"rule": "",
"query": "/ork:doctor --category=hooks",
"expectedBehavior": [
"Runs only the Hook Health category check (category 3)",
"Validates hooks.json schema compliance against expected structure and fields",
"Checks hook bundles exist and are compiled",
"Verifies async hook patterns are correct",
"Skips all other 12 categories for faster execution",
"Reports hook-specific pass/warn/fail results in a formatted summary"
]
},
{
"id": "negative-add-a-new-rest",
"rule": "",
"query": "Add a new REST endpoint for fetching user profiles.",
"expectedBehavior": [
"Claude does NOT invoke the doctor skill",
"Treats this as a feature implementation task",
"Uses standard code generation and API design patterns",
"Doctor is for plugin health diagnostics, not application development"
]
},
{
"id": "diagnostic-checks",
"rule": "diagnostic-checks",
"query": "What exactly does each diagnostic check in /ork:doctor validate?",
"expectedBehavior": [
"Validates skills frontmatter, token budget compliance, and cross-reference link integrity",
"Checks that hook bundles exist as compiled files and are properly registered in hooks.json",
"Verifies agent definitions comply with Claude Code 2.1 format requirements",
"Reports specific pass, warn, or fail status per category with actionable fix suggestions"
]
},
{
"id": "mcp-status-checks",
"rule": "mcp-status-checks",
"query": "How does /ork:doctor check MCP server health and configuration?",
"expectedBehavior": [
"Parses .mcp.json to list each server with its enabled or disabled state",
"Checks required environment variables and credentials for each MCP server",
"Verifies tavily server has TAVILY_API_KEY available in environment or via op CLI",
"Flags enabled MCP servers whose startup process would likely fail due to missing configuration"
]
},
{
"id": "agent-browser-doctor-structured",
"rule": "diagnostic-checks",
"query": "How does /ork:doctor check the agent-browser tool when version 0.26+ is installed?",
"expectedBehavior": [
"Detects agent-browser binary on PATH via `command -v agent-browser`",
"Invokes `agent-browser doctor --json` to get a structured snapshot",
"Parses chrome version, daemon status, and network reachability into a one-line health summary",
"Promotes any high-severity issue from the issues array into doctor's findings stream"
]
},
{
"id": "agent-browser-doctor-fallback",
"rule": "diagnostic-checks",
"query": "What does /ork:doctor do for agent-browser versions older than 0.26?",
"expectedBehavior": [
"Falls back to the legacy fuzzy probe when `agent-browser doctor --json` exits non-zero or is missing",
"Checks for the symlink at ~/.claude/skills/agent-browser to confirm installation",
"Recommends upgrading to agent-browser 0.26+ for structured doctor output",
"Reports `agent-browser: NOT INSTALLED` only when the binary is genuinely absent from PATH"
]
}
]
}