Diagnostic Issue Resolver

Diagnostic Issue Resolver

Diagnose and fix common TTS + Telegram bot issues through systematic symptom collection, automated diagnostics, and targeted fixes.

Platform: macOS (Apple Silicon)

---

Self-Evolving Skill: This skill improves through use. If instructions are wrong, parameters drifted, or a workaround was needed — fix this file immediately, don't defer. Only update for real, reproducible issues.

When to Use This Skill

• TTS audio is not playing or sounds wrong
• Telegram bot is not responding to messages
• Kokoro engine errors or timeouts
• Lock file appears stuck
• Audio plays twice (race condition)
• MLX Metal acceleration is not working
• Queue appears full or backed up

---

Requirements

• Access to ~/.claude/automation/claude-telegram-sync/ (bot source)
• Access to ~/.local/share/kokoro/ (Kokoro engine)
• Access to ~/.local/state/launchd-logs/telegram-bot/ (launchd logs)
• Access to ~/.claude/automation/claude-telegram-sync/logs/audit/ (NDJSON audit)

---

Known Issue Table

---

Workflow Phases

Phase 1: Symptom Collection

Use AskUserQuestion to understand what the user is experiencing. Key questions:

• What happened? (no audio, wrong audio, bot silent, error message)
• When did it start? (after upgrade, suddenly, always)
• What were you doing? (clipboard read, Telegram notification, manual TTS)

Phase 2: Automated Diagnostics

Based on symptoms, run the relevant subset of these checks:

# Lock state
ls -la /tmp/kokoro-tts.lock 2>/dev/null && stat -f "%Sm" /tmp/kokoro-tts.lock || echo "No lock file"

# Audio processes
pgrep -la afplay; pgrep -la say

# Bot process
pgrep -la 'bun.*src/main.ts'

# Kokoro health
~/.local/share/kokoro/.venv/bin/python -c "from mlx_audio.tts.utils import load_model; print('MLX-Audio OK')"

# Recent errors in audit log
tail -20 ~/.claude/automation/claude-telegram-sync/logs/audit/*.ndjson 2>/dev/null | grep -i error

# Recent bot console output
tail -50 /private/tmp/telegram-bot.log 2>/dev/null | grep -i -E '(error|fail|timeout)'

Phase 3: Root Cause Analysis

Map diagnostic output to the Known Issue Table above. Common patterns:

• Lock file exists + mtime > 30s ago + no afplay = stale lock
• No bot PID found = bot crashed
• from mlx_audio.tts.utils import load_model fails = MLX-Audio broken
• Multiple afplay PIDs = race condition

Phase 4: Fix Application

Apply the targeted fix from the Known Issue Table. Always use the least disruptive fix first.

Phase 5: Verification

After applying the fix, verify the issue is resolved:

# Quick TTS test
~/.local/share/kokoro/.venv/bin/python ~/.local/share/kokoro/tts_generate.py \
  --text "Diagnostic test complete" --voice af_heart --lang en-us --speed 1.0 \
  --output /tmp/kokoro-tts-diag-test.wav && afplay /tmp/kokoro-tts-diag-test.wav && echo "OK"

# Full health check
~/eon/cc-skills/plugins/tts-tg-sync/scripts/kokoro-install.sh --health

---

TodoWrite Task Templates

1. [Symptoms] Collect symptoms via AskUserQuestion
2. [Triage] Map symptoms to likely causes
3. [Lock] Check TTS lock state (mtime, PID, stale detection)
4. [Process] Check bot process and audio processes
5. [Kokoro] Verify Kokoro venv and MLX-Audio availability
6. [Logs] Check recent audit logs for errors
7. [Fix] Apply targeted fix for identified root cause
8. [Verify] Run health check to confirm resolution

---

Post-Change Checklist

• [ ] Root cause identified and documented
• [ ] Fix applied successfully
• [ ] Health check passes
• [ ] Test audio plays correctly
• [ ] No stale locks or orphan processes remain

Troubleshooting

This skill IS the troubleshooting skill. If the standard diagnostics do not identify the issue:

1. Check the full bot console log: cat /private/tmp/telegram-bot.log 2. Check all NDJSON audit logs: ls -lt ~/.claude/automation/claude-telegram-sync/logs/audit/ 3. Check system audio: afplay /System/Library/Sounds/Tink.aiff (if this fails, it is a macOS audio issue, not TTS) 4. Run a manual Kokoro generation outside the bot to isolate the problem 5. If all else fails, do a full teardown and reinstall using clean-component-removal then full-stack-bootstrap

---

Reference Documentation

• Common Issues -- Expanded diagnostic procedures for each known issue
• Lock Debugging -- Deep dive into the two-layer lock mechanism
• Evolution Log -- Change history for this skill

Post-Execution Reflection

After this skill completes, reflect before closing the task:

0. Locate yourself. — Find this SKILL.md's canonical path (Glob for this skill's name) before editing. All corrections target THIS file and its sibling references/ — never other documentation. 1. What failed? — Fix the instruction that caused it. If it could recur, add it as an anti-pattern. 2. What worked better than expected? — Promote it to recommended practice. Document why. 3. What drifted? — Any script, reference, or external dependency that no longer matches reality gets fixed now. 4. Log it. — Every change gets an evolution-log entry with trigger, fix, and evidence.

Do NOT defer. The next invocation inherits whatever you leave behind.

---

---

Common Issues -- Expanded Diagnostic Procedures

Detailed step-by-step procedures for diagnosing and resolving each known issue.

---

1. No Audio Output

Symptom: TTS generates silently -- no sound is heard.

Diagnostic Steps:

# Step 1: Check if lock file is blocking playback
ls -la /tmp/kokoro-tts.lock 2>/dev/null
stat -f "%Sm %N" /tmp/kokoro-tts.lock 2>/dev/null

# Step 2: Check if any audio process is active
pgrep -la afplay
pgrep -la say

# Step 3: Check macOS audio output (is sound muted?)
osascript -e 'output volume of (get volume settings)'

# Step 4: Test raw audio playback
afplay /System/Library/Sounds/Tink.aiff

Resolution Tree:

• Lock file exists + stale mtime (>30s) + no audio process --> Remove lock: rm -f /tmp/kokoro-tts.lock
• Lock file exists + fresh mtime --> Another TTS is in progress, wait for it to finish
• No lock + no audio + system sound works --> Check bot logs for generation errors
• System sound does not play --> macOS audio issue (check Sound preferences, output device)

---

2. Bot Not Responding

Symptom: Telegram messages are sent but bot does not reply.

Diagnostic Steps:

# Step 1: Check if bot process is running
pgrep -la 'bun.*src/main.ts'

# Step 2: Check recent log output
tail -30 /private/tmp/telegram-bot.log 2>/dev/null

# Step 3: Check if bun is available
which bun && bun --version

# Step 4: Check network (Telegram API reachable)
curl -s -o /dev/null -w "%{http_code}" https://api.telegram.org/

Resolution Tree:

• No process found --> Restart: cd ~/.claude/automation/claude-telegram-sync && bun --watch run src/main.ts &
• Process running but not responding --> Check logs for error loops, consider restart
• Network unreachable --> Check internet connectivity
• Bun not found --> mise install in the bot directory

---

3. Kokoro Timeout

Symptom: TTS generation hangs or times out after TTS_GENERATE_TIMEOUT_MS (default 15s).

Diagnostic Steps:

# Step 1: Check if model is cached
ls -la ~/.cache/huggingface/hub/models--mlx-community--Kokoro-82M-bf16/ 2>/dev/null

# Step 2: Test manual generation with verbose output
time ~/.local/share/kokoro/.venv/bin/python ~/.local/share/kokoro/tts_generate.py \
  --text "Test" --voice af_heart --lang en-us --speed 1.0 \
  --output /tmp/kokoro-tts-timeout-test.wav

# Step 3: Check MLX-Audio is importable
~/.local/share/kokoro/.venv/bin/python -c "from mlx_audio.tts.utils import load_model; print('MLX OK')"

Resolution Tree:

• Model not cached --> First run downloads from HuggingFace. Wait or run kokoro-install.sh --install
• MLX-Audio not importable --> kokoro-install.sh --upgrade to reinstall dependencies
• Generation works manually but times out from bot --> Increase TTS_GENERATE_TIMEOUT_MS in mise.toml

---

4. Queue Full / Backed Up

Symptom: New TTS requests are dropped with "Dropped stale item" in logs.

Diagnostic Steps:

# Step 1: Check audit log for queue events
grep -h 'tts.drop\|tts.enqueue\|tts.drain' \
  ~/.claude/automation/claude-telegram-sync/logs/audit/*.ndjson 2>/dev/null | tail -20

# Step 2: Check current queue config
grep TTS_MAX_QUEUE_DEPTH ~/.claude/automation/claude-telegram-sync/mise.toml
grep TTS_STALE_TTL_MS ~/.claude/automation/claude-telegram-sync/mise.toml

Resolution Tree:

• Frequent drops --> Increase TTS_MAX_QUEUE_DEPTH in mise.toml (default: 5)
• Items going stale --> Decrease TTS_STALE_TTL_MS or investigate why generation is slow
• Burst of notifications --> Normal during rapid prompting; queue is working as designed

---

5. Lock Stuck Forever

Symptom: TTS never starts; lock file never disappears.

See Lock Debugging for the full protocol. Quick resolution:

# Check lock state
stat -f "%Sm" /tmp/kokoro-tts.lock 2>/dev/null
pgrep -x afplay
pgrep -x say

# If lock is stale (>30s) AND no audio process: safe to remove
rm -f /tmp/kokoro-tts.lock

---

6. Slow MLX Metal Acceleration

Symptom: TTS generation is slow (~5-10s instead of ~1-2s).

Diagnostic Steps:

# Step 1: Check MLX-Audio is working
~/.local/share/kokoro/.venv/bin/python -c "
from mlx_audio.tts.utils import load_model
from importlib.metadata import version
print('mlx-audio version:', version('mlx-audio'))
print('MLX OK')
"

# Step 2: Check Python version
~/.local/share/kokoro/.venv/bin/python --version

# Step 3: Check hardware
uname -m  # Should be arm64

Resolution Tree:

• Not arm64 --> MLX-Audio requires Apple Silicon (M1+). No Intel/Linux fallback.
• Wrong Python version --> Must be 3.14. Rebuild venv: kokoro-install.sh --uninstall && kokoro-install.sh --install
• MLX OK but still slow --> Check if other GPU-heavy processes are running, or if the model needs re-download

---

7. Double Audio Playback

Symptom: The same text plays twice, or two different TTS outputs overlap.

Diagnostic Steps:

# Step 1: Check for multiple audio processes
pgrep -la afplay
pgrep -la say

# Step 2: Check for lock file
ls -la /tmp/kokoro-tts.lock 2>/dev/null

# Step 3: Check audit log for race conditions
grep -h 'tts.play.start' \
  ~/.claude/automation/claude-telegram-sync/logs/audit/*.ndjson 2>/dev/null | tail -10

Resolution Tree:

• Multiple afplay processes --> Kill all: pkill -x afplay, then check what triggered them
• Bot + shell script racing --> The lock protocol should prevent this. Check if both are acquiring locks properly
• Same notification processed twice --> Check bot logs for duplicate webhook deliveries

diagnostic-issue-resolver Evolution Log

Lock Debugging -- Two-Layer Lock Mechanism

Deep dive into the TTS lock protocol shared between shell scripts and the Telegram bot.

---

Overview

The TTS system uses a shared lock file at /tmp/kokoro-tts.lock to prevent audio overlap between:

• Shell scripts (tts_kokoro.sh, tts_read_clipboard.sh, etc.)
• Telegram bot (kokoro-client.ts)

Both writers and both readers use the same two-layer protocol.

---

Two-Layer Lock Protocol

Layer 1: Lock File Mtime Freshness (Heartbeat)

The lock holder writes its PID to the lock file and starts a background heartbeat that touches the lock every 5 seconds.

Shell scripts (via tts-common.sh):

acquire_tts_lock() {
    echo "$$" > "$TTS_LOCK"
    # Background heartbeat: touch lock every 5s while parent is alive
    (
        while kill -0 $$ 2>/dev/null; do
            touch "$TTS_LOCK" 2>/dev/null || true
            sleep 5
        done
    ) &
    _TTS_HEARTBEAT_PID=$!
}

Bot (via kokoro-client.ts):

function acquireTtsLock(): () => void {
  fs.writeFileSync(TTS_LOCK_FILE, String(process.pid));
  return () => {
    fs.unlinkSync(TTS_LOCK_FILE);
  };
}

Note: The bot does NOT run a heartbeat because its lock duration is bounded by the afplay subprocess -- it acquires before afplay and releases immediately after.

Layer 2: Active Audio Process Check (Defense-in-Depth)

Even if the lock mtime is stale (>30s), the system checks whether an audio process (afplay or say) is actually running before removing the lock.

Bot (via waitForTtsLock() in kokoro-client.ts):

// Only removes lock if BOTH:
// 1. Lock mtime is stale (no update for 30s = heartbeat died)
// 2. No afplay/say process is running (no active audio)

This prevents a race where:

• Script A is playing audio via afplay
• Script A's heartbeat process died (orphaned lock)
• Bot sees stale lock and removes it
• Bot starts its own afplay, causing overlap

With Layer 2, the bot sees afplay is still running and waits.

---

Stale Detection Logic

A lock is considered stale when:

1. Lock file exists 2. Lock mtime is older than 30 seconds (no heartbeat update) 3. No afplay or say process is running

If all three conditions are met, the lock is safe to remove.

Lock exists?
  |
  No --> Proceed (no contention)
  |
  Yes --> Check mtime
           |
           Fresh (<30s) --> Wait and re-check
           |
           Stale (>30s) --> Check audio processes
                             |
                             Running --> Wait (Layer 2 safety)
                             |
                             Not running --> Remove lock, proceed

---

Diagnostic Commands

Check Lock State

# Does the lock exist?
ls -la /tmp/kokoro-tts.lock 2>/dev/null || echo "No lock file"

# What PID holds it?
cat /tmp/kokoro-tts.lock 2>/dev/null || echo "No lock"

# When was it last touched (heartbeat)?
stat -f "Last modified: %Sm" /tmp/kokoro-tts.lock 2>/dev/null

# How old is it in seconds?
if [ -f /tmp/kokoro-tts.lock ]; then
  lock_mtime=$(stat -f %m /tmp/kokoro-tts.lock)
  now=$(date +%s)
  echo "Lock age: $(( now - lock_mtime )) seconds"
fi

Check Audio Processes

# Is afplay running?
pgrep -la afplay || echo "No afplay"

# Is say running?
pgrep -la say || echo "No say"

Check Lock Holder

# Is the PID in the lock file still alive?
if [ -f /tmp/kokoro-tts.lock ]; then
  lock_pid=$(cat /tmp/kokoro-tts.lock)
  if kill -0 "$lock_pid" 2>/dev/null; then
    echo "Lock holder PID $lock_pid is alive"
    ps -p "$lock_pid" -o pid,command
  else
    echo "Lock holder PID $lock_pid is DEAD (orphaned lock)"
  fi
fi

---

Common Lock Scenarios

Scenario 1: Normal Operation

Shell script starts -> acquires lock -> heartbeat every 5s -> plays audio -> releases lock

Lock mtime stays fresh. Other TTS requests wait. No intervention needed.

Scenario 2: Orphaned Lock (Heartbeat Died)

Shell script crashes -> heartbeat subprocess dies -> lock mtime goes stale -> no afplay running

Both Layer 1 (stale mtime) and Layer 2 (no audio) confirm it is safe to remove. The bot's waitForTtsLock() handles this automatically after 30s.

Manual fix: rm -f /tmp/kokoro-tts.lock

Scenario 3: Stale Lock But Audio Still Playing

Shell script crashes -> heartbeat dies -> lock mtime stale -> BUT afplay is still playing the last chunk

Layer 1 says "stale" but Layer 2 says "audio active". The bot waits. This is correct behavior -- removing the lock would cause audio overlap.

Manual: Do NOT remove the lock. Wait for afplay to finish, then the bot will clean up.

Scenario 4: Lock Race Between Bot and Shell

Bot checks: no lock -> Bot creates lock -> Shell checks: lock exists -> Shell waits

This is the normal mutual exclusion path. The 500ms poll interval in waitForTtsLock() means worst-case audio gap between bot and shell is ~500ms.

---

Configuration

---

Key Source Files