
Asciinema Streaming Backup
- 119 installs
- 62 repo stars
- Updated August 3, 2026
- terrylica/cc-skills
Use asciinema-streaming-backup for development tasks
About
asciinema-streaming-backup: A skill for development. This provides functionality for development workflows.
- asciinema-streaming-backup
Asciinema Streaming Backup by the numbers
- 119 all-time installs (skills.sh)
- Ranked #2,847 of 4,347 Backend & APIs skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/terrylica/cc-skills --skill asciinema-streaming-backupAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 119 |
|---|---|
| repo stars | ★ 62 |
| Last updated | August 3, 2026 |
| Repository | terrylica/cc-skills ↗ |
What it does
Use asciinema-streaming-backup for development tasks
Files
asciinema-streaming-backup
Complete system for streaming asciinema recordings to GitHub with automatic brotli archival. Uses idle-detection for intelligent chunking, zstd for concatenatable streaming compression, and GitHub Actions for final brotli recompression.
Self-Evolving Skill: This skill improves through use. If instructions are wrong, parameters drifted, or a workaround was needed — fix this file immediately, don't defer. Only update for real, reproducible issues.
When to Use This Skill
Use this skill when:
- Setting up real-time backup of asciinema recordings to GitHub
- Configuring idle-detection chunking for recordings
- Creating orphan branch infrastructure for recording storage
- Integrating GitHub Actions for brotli recompression
Platform: macOS, Linux
Isolation: Uses Git orphan branch (separate history, cannot pollute main)
---
Architecture Overview
┌─────────────────┐ zstd chunks ┌─────────────────┐ Actions ┌─────────────────┐
│ asciinema rec │ ──────────────────▶ │ GitHub Orphan │ ───────────────▶ │ brotli archive │
│ + idle-chunker │ (concatenatable) │ gh-recordings │ │ (300x compress)│
└─────────────────┘ └─────────────────┘ └─────────────────┘
│ │
│ Idle ≥30s triggers chunk │ Separate history
▼ │ Cannot PR to main
~/asciinema_recordings/ ▼
└── repo-name/ .github/workflows/
└── chunks/*.zst └── recompress.yml---
Requirements
| Component | Required | Installation | Version |
|---|---|---|---|
| asciinema CLI | Yes | brew install asciinema | 3.0+ (Rust) |
| zstd | Yes | brew install zstd | Any |
| brotli | Yes | brew install brotli | Any |
| git | Yes | Pre-installed | 2.20+ |
| gh CLI | Yes | brew install gh | Any |
| fswatch | Optional | brew install fswatch | For real-time |
---
Workflow Phases
Phase 0: Preflight Validation
Verify all tools installed, offer self-correction if missing. Run the preflight check script, then AskUserQuestion to offer installation for missing tools.
See Setup Scripts for the complete preflight-check.sh script.
Self-Correction: If tools are missing, generate installation command and offer to run it.
---
Phase 1: GitHub Account Detection
Detect available GitHub accounts from 5 sources (SSH config, SSH keys, gh CLI, mise env, git config) and let user choose which to use.
See Account & Repository Detection for the detection script, scoring logic, and AskUserQuestion flow.
---
Phase 1.5: Current Repository Detection
Detect current git repository context (CURRENT_REPO_OWNER, CURRENT_REPO_NAME, DETECTED_FROM) to provide intelligent defaults for Phase 2.
See Account & Repository Detection for the detection script.
---
Phase 2: Core Configuration
Gather essential configuration: repository URL (with auto-detection from Phase 1.5), recording directory, and branch name.
See Configuration Reference for all AskUserQuestion sequences and URL normalization logic.
---
Phase 3: Advanced Configuration
Allow customization of compression and behavior parameters:
| Parameter | Default | Range |
|---|---|---|
| Idle threshold | 30s | 5-300s |
| zstd level | 3 | 1-22 |
| Brotli level | 9 | 1-11 |
| Auto-push | Yes | Yes/No |
| Poll interval | 5s | 2s, 5s, 10s |
See Configuration Reference for all AskUserQuestion sequences.
---
Phase 4: Orphan Branch Setup
Create or configure the orphan branch with GitHub Actions workflow. Checks for existing branch first and offers clone/reset/verify options.
Key actions:
1. Check if branch exists on remote via git ls-remote 2. If exists: AskUserQuestion for clone/reset/verify 3. If new: Create orphan branch, add workflow + directory structure, push
See Setup Scripts for the complete setup-orphan-branch.sh script. See GitHub Workflow for the full recompress.yml Actions workflow.
---
Phase 5: Local Environment Setup
Configure local directory and generate customized idle-chunker.sh with user parameters embedded from Phase 3.
Key actions:
1. Clone orphan branch to ~/asciinema_recordings/<repo>/ 2. Generate idle-chunker.sh with embedded configuration 3. Display configuration summary and usage instructions
See Setup Scripts for local setup scripts. See Idle Chunker for the complete chunker implementation.
---
Phase 6: Autonomous Validation
Claude executes 8 tests autonomously, displaying formatted results. Only 2 tests require user action (recording test, chunker live test).
| Test Category | Count | Autonomous? |
|---|---|---|
| Tool preflight | 5 | Yes |
| Compression round-trip | 3 | Yes |
| Repository validation | 4 | Yes |
| GitHub Actions trigger | 1 | Yes |
| Recording test | 1 | No (USER) |
| Chunker live test | 1 | No (USER) |
See Autonomous Validation for the complete validation script, user-required test flows, and troubleshooting table.
---
Quick Start
First-Time Setup
/usr/bin/env bash << 'PREFLIGHT_EOF'
# 1. Check requirements
for tool in asciinema zstd brotli git gh; do
command -v "$tool" &>/dev/null && echo "$tool: OK" || echo "$tool: MISSING"
done
# 2. Create orphan branch (replace with your repo)
REPO="git@github.com:YOUR/REPO.git"
./setup-orphan-branch.sh "$REPO"
# 3. Validate setup
./validate-setup.sh "$HOME/asciinema_recordings/REPO"
PREFLIGHT_EOFRecording Session
/usr/bin/env bash << 'SKILL_SCRIPT_EOF'
# Terminal 1: Start recording
WORKSPACE=$(basename "$PWD")
asciinema rec $PWD/tmp/${WORKSPACE}_$(date +%Y-%m-%d_%H-%M).cast
# Terminal 2: Start idle-chunker
~/asciinema_recordings/REPO/idle-chunker.sh $PWD/tmp/${WORKSPACE}_*.cast
SKILL_SCRIPT_EOF---
Key Design Decisions
| Decision | Rationale |
|---|---|
| zstd for streaming | Supports frame concatenation (brotli doesn't) |
| brotli for archival | Best compression ratio (~300x for .cast files) |
| Orphan branch | Complete isolation, can't pollute main history |
| Idle-based chunking | Semantic breakpoints, not mid-output splits |
| Shallow clone | Minimal disk usage, can't accidentally access main |
| 30s idle threshold | Balances chunk frequency vs semantic completeness |
---
Troubleshooting
See Troubleshooting Guide for common issues and fixes.
---
Post-Change Checklist
After modifying this skill:
1. [ ] Orphan branch creation scripts use heredoc wrapper 2. [ ] All bash blocks compatible with zsh (no declare -A, no grep -P) 3. [ ] GitHub Actions workflow validates brotli recompression 4. [ ] Idle chunker handles both macOS and Linux stat syntax 5. [ ] Detection flow outputs parseable key=value format 6. [ ] References validate links to external documentation
---
Reference Documentation
- Account & Repository Detection - GitHub account detection + repo context
- Configuration Reference - AskUserQuestion sequences + task templates
- Idle Chunker Script - Complete chunker implementation
- GitHub Workflow - Full Actions workflow
- Setup Scripts - All setup and validation scripts
- Autonomous Validation - Validation script and user-required tests
- Troubleshooting Guide - Common issues and fixes
- asciinema 3.0 Docs
- zstd Frame Format
- Git Orphan Branches
Post-Execution Reflection
After this skill completes, reflect before closing the task:
0. Locate yourself. — Find this SKILL.md's canonical path before editing. 1. What failed? — Fix the instruction that caused it. 2. What worked better than expected? — Promote to recommended practice. 3. What drifted? — Fix any script, reference, or dependency that no longer matches reality. 4. Log it. — Evolution-log entry with trigger, fix, and evidence.
Do NOT defer. The next invocation inherits whatever you leave behind.
Skill: asciinema-streaming-backup
Account & Repository Detection
Scripts for detecting GitHub accounts and current repository context.
---
Phase 1: GitHub Account Detection
Probe 5 sources to detect GitHub accounts:
| Source | Command | What it finds |
|---|---|---|
| SSH config | grep -A5 "Host github" ~/.ssh/config | Match directives with IdentityFile |
| SSH keys | ls ~/.ssh/id_ed25519_* | Account-named keys (e.g., id_ed25519_terrylica) |
| gh CLI | gh auth status | Authenticated accounts |
| mise env | grep GH_ACCOUNT .mise.toml | GH_ACCOUNT variable |
| git config | git config user.name | Global git username |
Detection Script
/usr/bin/env bash << 'DETECT_ACCOUNTS_EOF'
# detect-github-accounts.sh - Probe all sources for GitHub accounts
# Uses portable parallel arrays (works in bash 3.2+ and when wrapped for zsh)
ACCOUNT_NAMES=()
ACCOUNT_SOURCES=()
log() { echo "[detect] $*"; }
# Helper: add account with source (updates existing or appends new)
add_account() {
local account="$1" source="$2"
local idx
for idx in "${!ACCOUNT_NAMES[@]}"; do
if [[ "${ACCOUNT_NAMES[$idx]}" == "$account" ]]; then
ACCOUNT_SOURCES[$idx]+="$source "
return
fi
done
ACCOUNT_NAMES+=("$account")
ACCOUNT_SOURCES+=("$source ")
}
# 1. SSH config Match directives
if [[ -f ~/.ssh/config ]]; then
while IFS= read -r line; do
if [[ "$line" =~ IdentityFile.*id_ed25519_([a-zA-Z0-9_-]+) ]]; then
add_account "${BASH_REMATCH[1]}" "ssh-config"
fi
done < ~/.ssh/config
fi
# 2. SSH key filenames
for keyfile in ~/.ssh/id_ed25519_*; do
if [[ -f "$keyfile" && "$keyfile" != *.pub ]]; then
account=$(basename "$keyfile" | sed 's/id_ed25519_//')
add_account "$account" "ssh-key"
fi
done
# 3. gh CLI authenticated accounts
if command -v gh &>/dev/null; then
while IFS= read -r account; do
[[ -n "$account" ]] && add_account "$account" "gh-cli"
done < <(gh auth status 2>&1 | grep -oE 'Logged in to github.com account [a-zA-Z0-9_-]+' | awk '{print $NF}')
fi
# 4. mise env GH_ACCOUNT
if [[ -f .mise.toml ]]; then
account=$(grep -E 'GH_ACCOUNT\s*=' .mise.toml 2>/dev/null | sed 's/.*=\s*"\([^"]*\)".*/\1/')
[[ -n "$account" ]] && add_account "$account" "mise-env"
fi
# 5. git config user.name
git_user=$(git config user.name 2>/dev/null)
[[ -n "$git_user" ]] && add_account "$git_user" "git-config"
# Score and display
log "=== Detected GitHub Accounts ==="
RECOMMENDED=""
MAX_SOURCES=0
for idx in "${!ACCOUNT_NAMES[@]}"; do
account="${ACCOUNT_NAMES[$idx]}"
sources="${ACCOUNT_SOURCES[$idx]}"
count=$(echo "$sources" | wc -w | tr -d ' ')
log "$account: $count sources ($sources)"
if (( count > MAX_SOURCES )); then
MAX_SOURCES=$count
RECOMMENDED="$account"
RECOMMENDED_SOURCES="$sources"
fi
done
echo ""
echo "RECOMMENDED=$RECOMMENDED"
echo "SOURCES=$RECOMMENDED_SOURCES"
DETECT_ACCOUNTS_EOFAskUserQuestion
AskUserQuestion:
question: "Which GitHub account should be used for recording storage?"
header: "GitHub Account Selection"
options:
- label: "${RECOMMENDED} (Recommended)"
description: "Detected via: ${SOURCES}"
# Additional detected accounts appear here dynamically
- label: "Enter manually"
description: "Type a GitHub username not listed above"Post-Selection
If user selects an account, ensure gh CLI is using that account:
/usr/bin/env bash << 'POST_SELECT_EOF'
# Ensure gh CLI is authenticated as selected account
SELECTED_ACCOUNT="${1:?Usage: provide selected account}"
if ! gh auth status 2>&1 | grep -q "Logged in to github.com account $SELECTED_ACCOUNT"; then
echo "Switching gh CLI to account: $SELECTED_ACCOUNT"
gh auth switch --user "$SELECTED_ACCOUNT" 2>/dev/null || \
echo "Warning: Could not switch accounts. Manual auth may be needed."
fi
POST_SELECT_EOF---
Phase 1.5: Current Repository Detection
Detect current git repository context to provide intelligent defaults for Phase 2 questions.
Detection Script
/usr/bin/env bash << 'DETECT_REPO_EOF'
# Detect current repository context for intelligent defaults
CURRENT_REPO_URL=""
CURRENT_REPO_OWNER=""
CURRENT_REPO_NAME=""
DETECTED_FROM=""
# Check if we're in a git repository
if git rev-parse --git-dir &>/dev/null; then
# Try origin remote first
if git remote get-url origin &>/dev/null; then
CURRENT_REPO_URL=$(git remote get-url origin)
DETECTED_FROM="origin remote"
# Fallback to first available remote
elif [[ -n "$(git remote)" ]]; then
REMOTE=$(git remote | head -1)
CURRENT_REPO_URL=$(git remote get-url "$REMOTE")
DETECTED_FROM="$REMOTE remote"
fi
# Parse owner and name from URL (SSH or HTTPS)
if [[ -n "$CURRENT_REPO_URL" ]]; then
if [[ "$CURRENT_REPO_URL" =~ github\.com[:/]([^/]+)/([^/.]+) ]]; then
CURRENT_REPO_OWNER="${BASH_REMATCH[1]}"
CURRENT_REPO_NAME="${BASH_REMATCH[2]%.git}"
fi
fi
fi
# Output for Claude to parse
echo "CURRENT_REPO_URL=$CURRENT_REPO_URL"
echo "CURRENT_REPO_OWNER=$CURRENT_REPO_OWNER"
echo "CURRENT_REPO_NAME=$CURRENT_REPO_NAME"
echo "DETECTED_FROM=$DETECTED_FROM"
DETECT_REPO_EOFClaude Action: Store detected values (CURRENT_REPO_OWNER, CURRENT_REPO_NAME, DETECTED_FROM) for use in subsequent AskUserQuestion calls. If no repo detected, proceed without defaults.
Skill: asciinema-streaming-backup
Autonomous Validation Reference
Purpose: Validation tests that Claude Code executes autonomously, plus user-required test flows.
Key Principle: Claude executes 8 tests autonomously and displays results in CLI. User interaction is only required for 2 tests that need terminal control.
---
Test Categories
| Test | Autonomous? | Reason |
|---|---|---|
| Tool preflight | YES | Bash checks tools |
| zstd round-trip | YES | Synthetic test data |
| Brotli round-trip | YES | Synthetic test data |
| zstd concatenation | YES | Critical for streaming |
| Git/gh auth check | YES | Query auth status |
| Orphan branch validation | YES | Check remote/local |
| Workflow file check | YES | Read file contents |
| GitHub Actions trigger | YES | gh workflow run + watch |
| Recording test | NO (USER) | Requires starting asciinema |
| Chunker live test | NO (USER) | Requires active recording |
---
Autonomous Validation Script
Execute this script via Bash tool to run all autonomous tests:
/usr/bin/env bash << 'PREFLIGHT_EOF'
#!/usr/bin/env bash
# autonomous-validation.sh - Claude runs this automatically
# Usage: autonomous-validation.sh <repo_dir> <repo_url> [branch_name]
set -euo pipefail
REPO_DIR="${1:?Usage: autonomous-validation.sh <repo_dir> <repo_url> [branch_name]}"
REPO_URL="${2:?Usage: autonomous-validation.sh <repo_dir> <repo_url> [branch_name]}"
BRANCH_NAME="${3:-gh-recordings}"
PASSED=0
FAILED=0
log_pass() { echo " ✓ $1"; ((PASSED++)); }
log_fail() { echo " ✗ $1"; ((FAILED++)); }
log_run() { echo "[RUN] $1..."; }
echo "╔════════════════════════════════════════════════════════════════╗"
echo "║ AUTONOMOUS VALIDATION - Claude Code Executes All Tests ║"
echo "╠════════════════════════════════════════════════════════════════╣"
# ─────────────────────────────────────────────────────────────────────
# Phase 1: Tool Check
# ─────────────────────────────────────────────────────────────────────
echo ""
echo " Phase 1: Tool Check"
echo " ─────────────────"
for tool in asciinema zstd brotli git gh; do
log_run "Checking $tool"
if command -v "$tool" &>/dev/null; then
VERSION=$("$tool" --version 2>&1 | head -1 | grep -oE '[0-9]+\.[0-9]+(\.[0-9]+)?' | head -1 || echo "?")
log_pass "$tool installed (v$VERSION)"
else
log_fail "$tool MISSING"
fi
done
# ─────────────────────────────────────────────────────────────────────
# Phase 2: Compression Tests
# ─────────────────────────────────────────────────────────────────────
echo ""
echo " Phase 2: Compression Tests"
echo " ────────────────────────"
log_run "zstd round-trip"
TEST_DATA="test-$(date +%s)"
if echo "$TEST_DATA" | zstd -3 2>/dev/null | zstd -d 2>/dev/null | grep -q "$TEST_DATA"; then
log_pass "zstd round-trip PASSED"
else
log_fail "zstd round-trip FAILED"
fi
log_run "brotli round-trip"
if echo "$TEST_DATA" | brotli 2>/dev/null | brotli -d 2>/dev/null | grep -q "$TEST_DATA"; then
log_pass "brotli round-trip PASSED"
else
log_fail "brotli round-trip FAILED"
fi
log_run "zstd concatenation (CRITICAL for streaming)"
TMP=$(mktemp -d)
trap 'rm -rf "$TMP"' EXIT
echo "chunk1" | zstd -3 > "$TMP/a.zst" 2>/dev/null
echo "chunk2" | zstd -3 > "$TMP/b.zst" 2>/dev/null
cat "$TMP/a.zst" "$TMP/b.zst" > "$TMP/combined.zst"
RESULT=$(zstd -d -c "$TMP/combined.zst" 2>/dev/null || true)
if [[ "$RESULT" == $'chunk1\nchunk2' ]]; then
log_pass "zstd concatenation PASSED"
else
log_fail "zstd concatenation FAILED"
fi
# ─────────────────────────────────────────────────────────────────────
# Phase 3: Repository Validation
# ─────────────────────────────────────────────────────────────────────
echo ""
echo " Phase 3: Repository Validation"
echo " ─────────────────────────────"
log_run "Checking gh auth"
if gh auth status &>/dev/null; then
ACCOUNT=$(gh api user --jq '.login' 2>/dev/null || echo "unknown")
log_pass "authenticated as $ACCOUNT"
else
log_fail "gh not authenticated"
fi
log_run "Checking orphan branch on remote"
if git ls-remote --heads "$REPO_URL" "$BRANCH_NAME" 2>/dev/null | grep -q "$BRANCH_NAME"; then
log_pass "$BRANCH_NAME exists on remote"
else
log_fail "$BRANCH_NAME NOT found on remote"
fi
log_run "Checking local clone"
if [[ -d "$REPO_DIR" ]]; then
log_pass "local directory exists: $REPO_DIR"
else
log_fail "local directory NOT found: $REPO_DIR"
fi
log_run "Checking workflow file"
if [[ -f "$REPO_DIR/.github/workflows/recompress.yml" ]]; then
log_pass "recompress.yml present"
else
log_fail "recompress.yml MISSING"
fi
# ─────────────────────────────────────────────────────────────────────
# Phase 4: GitHub Actions Test
# ─────────────────────────────────────────────────────────────────────
echo ""
echo " Phase 4: GitHub Actions Test"
echo " ─────────────────────────────"
# Extract owner/repo from URL for gh commands
OWNER_REPO=""
if [[ "$REPO_URL" =~ github\.com[:/]([^/]+)/([^/.]+) ]]; then
OWNER_REPO="${BASH_REMATCH[1]}/${BASH_REMATCH[2]}"
fi
if [[ -z "$OWNER_REPO" ]]; then
log_fail "Could not parse owner/repo from URL: $REPO_URL"
else
log_run "Triggering workflow_dispatch"
if gh workflow run recompress -R "$OWNER_REPO" --ref "$BRANCH_NAME" 2>/dev/null; then
log_pass "workflow triggered"
sleep 5
log_run "Fetching run status"
RUN_ID=$(gh run list -R "$OWNER_REPO" -w recompress --limit 1 --json databaseId -q '.[0].databaseId' 2>/dev/null || true)
if [[ -n "$RUN_ID" ]]; then
STATUS=$(gh run view "$RUN_ID" -R "$OWNER_REPO" --json status -q '.status' 2>/dev/null || echo "unknown")
echo " ⏳ Run #$RUN_ID: $STATUS"
# Wait for completion (max 60s)
COMPLETED=false
for _ in {1..12}; do
STATUS_FULL=$(gh run view "$RUN_ID" -R "$OWNER_REPO" --json status,conclusion -q '.status + " " + .conclusion' 2>/dev/null || true)
if [[ "$STATUS_FULL" == "completed "* ]]; then
CONCLUSION="${STATUS_FULL#completed }"
if [[ "$CONCLUSION" == "success" ]]; then
log_pass "workflow completed successfully"
else
log_fail "workflow completed with: $CONCLUSION"
fi
COMPLETED=true
break
fi
sleep 5
done
if [[ "$COMPLETED" == "false" ]]; then
echo " ⏳ Run still in progress after 60s (check manually)"
log_pass "workflow triggered (completion pending)"
fi
else
log_fail "could not fetch run ID"
fi
else
log_fail "workflow trigger failed (workflow_dispatch may not be enabled)"
fi
fi
# ─────────────────────────────────────────────────────────────────────
# Summary
# ─────────────────────────────────────────────────────────────────────
echo ""
echo "║ ║"
echo "╠═════════════════════════════════════════════════════════════════╣"
echo "║ AUTONOMOUS TESTS: $PASSED passed, $FAILED failed"
echo "╚═════════════════════════════════════════════════════════════════╝"
[[ $FAILED -eq 0 ]] && exit 0 || exit 1
PREFLIGHT_EOF---
User-Required Tests
These tests require user action in a terminal. Use AskUserQuestion to guide the user.
Recording Validation
AskUserQuestion:
question: "Ready to test recording? This requires you to start asciinema in another terminal."
header: "Recording Test"
options:
- label: "Guide me through it (Recommended)"
description: "I'll show step-by-step instructions"
- label: "Skip this test"
description: "I trust the setup works"
- label: "I've already verified recording works"
description: "Mark as passed"If user selects "Guide me through it", display:
╔════════════════════════════════════════════════════════════════╗
║ USER ACTION REQUIRED: Recording Test ║
╠════════════════════════════════════════════════════════════════╣
║ ║
║ In a NEW terminal, run: ║
║ ┌────────────────────────────────────────────────────────┐ ║
║ │ asciinema rec ~/asciinema_recordings/test_session.cast │ ║
║ └────────────────────────────────────────────────────────┘ ║
║ ║
║ Then: ║
║ 1. Type a few commands (ls, echo "hello", etc.) ║
║ 2. Exit with Ctrl+D or type 'exit' ║
║ 3. Come back here when done ║
║ ║
╚════════════════════════════════════════════════════════════════╝After user confirms, Claude validates autonomously:
/usr/bin/env bash << 'PREFLIGHT_EOF_2'
# Claude runs after user confirms
CAST_FILE="$HOME/asciinema_recordings/test_session.cast"
if [[ -f "$CAST_FILE" ]]; then
echo " ✓ test_session.cast exists"
# Check JSON header
if head -1 "$CAST_FILE" | jq -e '.version' &>/dev/null; then
echo " ✓ Valid JSON header"
else
echo " ✗ Invalid JSON header"
fi
# Check line count (at least header + some events)
LINE_COUNT=$(wc -l < "$CAST_FILE")
if [[ $LINE_COUNT -gt 1 ]]; then
echo " ✓ $LINE_COUNT events recorded"
else
echo " ✗ No events recorded"
fi
else
echo " ✗ test_session.cast NOT found"
fi
PREFLIGHT_EOF_2Live Chunker Test (Optional)
AskUserQuestion:
question: "Ready to test live chunking? This requires running recording + chunker simultaneously."
header: "Chunker Test (Optional)"
options:
- label: "Guide me through it"
description: "Full end-to-end test with two terminals"
- label: "Skip - I trust the setup"
description: "Chunker test is optional"If user selects "Guide me through it", display:
╔════════════════════════════════════════════════════════════════╗
║ USER ACTION REQUIRED: Live Chunker Test ║
╠════════════════════════════════════════════════════════════════╣
║ ║
║ TERMINAL 1 (Recording): ║
║ ┌────────────────────────────────────────────────────────┐ ║
║ │ asciinema rec ~/asciinema_recordings/chunker_test.cast │ ║
║ └────────────────────────────────────────────────────────┘ ║
║ ║
║ TERMINAL 2 (Chunker): ║
║ ┌────────────────────────────────────────────────────────┐ ║
║ │ ~/asciinema_recordings/<repo>/idle-chunker.sh \ │ ║
║ │ ~/asciinema_recordings/chunker_test.cast │ ║
║ └────────────────────────────────────────────────────────┘ ║
║ ║
║ In Terminal 1: ║
║ 1. Type some commands ║
║ 2. Wait 30+ seconds (idle threshold) ║
║ 3. Type more commands ║
║ 4. Exit with Ctrl+D ║
║ ║
║ Watch Terminal 2 for chunk creation messages. ║
║ ║
╚════════════════════════════════════════════════════════════════╝After user confirms, Claude validates:
/usr/bin/env bash << 'PREFLIGHT_EOF_3'
# Check if chunks were created
REPO_DIR="$HOME/asciinema_recordings/<repo>"
CHUNKS=$(find "$REPO_DIR/chunks" -name "*.zst" 2>/dev/null | wc -l)
if [[ $CHUNKS -gt 0 ]]; then
echo " ✓ $CHUNKS chunk(s) created"
# Check if git tracked
cd "$REPO_DIR"
if git status --porcelain chunks/ 2>/dev/null | grep -q .; then
echo " ✓ Chunks staged for commit"
fi
else
echo " ✗ No chunks found in $REPO_DIR/chunks/"
fi
PREFLIGHT_EOF_3---
Troubleshooting
Common failures and resolutions:
| Failure | Cause | Resolution |
|---|---|---|
asciinema MISSING | Not installed | brew install asciinema (macOS) or pipx install asciinema |
zstd MISSING | Not installed | brew install zstd (macOS) or apt install zstd (Linux) |
brotli MISSING | Not installed | brew install brotli (macOS) or apt install brotli (Linux) |
gh not authenticated | No GitHub login | Run gh auth login and follow prompts |
gh-recordings NOT found on remote | Branch not pushed | Run orphan branch setup from Phase 4 of skill |
local directory NOT found | Clone failed | Check repo URL and permissions, re-run clone |
recompress.yml MISSING | Workflow not created | Re-run orphan branch setup to create workflow |
workflow trigger failed | No workflow_dispatch | Add workflow_dispatch: trigger to workflow |
zstd concatenation FAILED | zstd version issue | Update zstd: brew upgrade zstd |
brotli round-trip FAILED | brotli corrupted | Reinstall: brew reinstall brotli |
---
Execution Instructions
For Claude Code: After running the setup phases, execute autonomous validation:
1. Save the script to a temp file or run inline via Bash tool 2. Execute with: bash <script> "$REPO_DIR" "$REPO_URL" "$BRANCH_NAME" 3. Display formatted output to user 4. If any test fails, show relevant troubleshooting row 5. After autonomous tests, prompt for user-required tests via AskUserQuestion 6. Report final summary: "X/Y autonomous tests passed, user tests: <status>"
Skill: asciinema-streaming-backup
Configuration Reference
All AskUserQuestion sequences for core and advanced configuration, plus URL normalization and task templates.
---
Phase 2: Core Configuration
2.1 Repository URL
If current repo detected (from Phase 1.5):
AskUserQuestion:
question: "Which repository should store the recordings?"
header: "Repository"
options:
- label: "${CURRENT_REPO_OWNER}/${CURRENT_REPO_NAME} (Recommended)"
description: "Current repo detected from ${DETECTED_FROM}"
- label: "Create dedicated repo: ${GITHUB_ACCOUNT}/asciinema-recordings"
description: "Separate repository for all recordings"
- label: "Enter different repository"
description: "Specify another repository (user/repo format)"If no current repo detected:
AskUserQuestion:
question: "Enter the GitHub repository URL for storing recordings:"
header: "Repository URL"
options:
- label: "Create dedicated repo: ${GITHUB_ACCOUNT}/asciinema-recordings"
description: "Separate repository for all recordings (Recommended)"
- label: "Enter repository manually"
description: "SSH (git@github.com:user/repo.git), HTTPS, or shorthand (user/repo)"URL Normalization
Handles multiple formats:
/usr/bin/env bash << 'NORMALIZE_URL_EOF'
# Normalize to SSH format for consistent handling
normalize_repo_url() {
local url="$1"
# Shorthand: user/repo -> git@github.com:user/repo.git
if [[ "$url" =~ ^[a-zA-Z0-9_-]+/[a-zA-Z0-9_.-]+$ ]]; then
echo "git@github.com:${url}.git"
# HTTPS: https://github.com/user/repo -> git@github.com:user/repo.git
elif [[ "$url" =~ ^https://github\.com/([^/]+)/([^/]+)/?$ ]]; then
echo "git@github.com:${BASH_REMATCH[1]}/${BASH_REMATCH[2]%.git}.git"
# Already SSH format
else
echo "$url"
fi
}
URL="${1:?Usage: provide URL to normalize}"
normalize_repo_url "$URL"
NORMALIZE_URL_EOFConfirmation for Free-Form Input
If user selected "Enter different/manually":
AskUserQuestion:
question: "You entered '${USER_INPUT}'. Normalized to: ${NORMALIZED_URL}. Is this correct?"
header: "Confirm Repository"
options:
- label: "Yes, use ${NORMALIZED_URL}"
description: "Proceed with this repository"
- label: "No, let me re-enter"
description: "Go back to repository selection"2.2 Recording Directory
AskUserQuestion:
question: "Where should recordings be stored locally?"
header: "Recording Directory"
options:
- label: "~/asciinema_recordings/${RESOLVED_REPO_NAME} (Recommended)"
description: "Example: ~/asciinema_recordings/alpha-forge"
- label: "Custom path"
description: "Enter a different directory path"Note: ${RESOLVED_REPO_NAME} is the actual repo name from Phase 1.5 or Phase 2.1, not a variable placeholder. Display the concrete path to user.
2.3 Branch Name
AskUserQuestion:
question: "What should the orphan branch be named?"
header: "Branch Name"
options:
- label: "asciinema-recordings (Recommended)"
description: "Matches ~/asciinema_recordings/ parent directory pattern"
- label: "gh-recordings"
description: "GitHub-prefixed alternative (gh = GitHub storage)"
- label: "recordings"
description: "Minimal name"
- label: "Custom"
description: "Enter a custom branch name"Naming Convention: The default asciinema-recordings matches the parent directory ~/asciinema_recordings/ for consistency.
---
Phase 3: Advanced Configuration
Configuration Parameters
| Parameter | Default | Options |
|---|---|---|
| Idle threshold | 30s | 15s, 30s (Recommended), 60s, Custom (5-300) |
| zstd level | 3 | 1 (fast), 3 (Recommended), 6, Custom (1-22) |
| Brotli level | 9 | 6, 9 (Recommended), 11, Custom (1-11) |
| Auto-push | Yes | Yes (Recommended), No |
| Poll interval | 5s | 2s, 5s (Recommended), 10s |
3.1 Idle Threshold
AskUserQuestion:
question: "How long should the chunker wait before creating a chunk?"
header: "Idle Threshold"
options:
- label: "15 seconds"
description: "More frequent chunks, smaller files"
- label: "30 seconds (Recommended)"
description: "Balanced chunk size and frequency"
- label: "60 seconds"
description: "Larger chunks, less frequent uploads"
- label: "Custom (5-300 seconds)"
description: "Enter a custom threshold"3.2 zstd Compression Level
AskUserQuestion:
question: "What zstd compression level for streaming chunks?"
header: "zstd Level"
options:
- label: "1 (Fast)"
description: "Fastest compression, larger files"
- label: "3 (Recommended)"
description: "Good balance of speed and compression"
- label: "6 (Better compression)"
description: "Slower but smaller chunks"
- label: "Custom (1-22)"
description: "Enter a custom level"3.3 Brotli Compression Level
AskUserQuestion:
question: "What brotli compression level for final archives?"
header: "Brotli Level"
options:
- label: "6"
description: "Faster archival, slightly larger files"
- label: "9 (Recommended)"
description: "Great compression with reasonable speed"
- label: "11 (Maximum)"
description: "Best compression, slowest (may timeout on large files)"
- label: "Custom (1-11)"
description: "Enter a custom level"3.4 Auto-Push
AskUserQuestion:
question: "Should chunks be automatically pushed to GitHub?"
header: "Auto-Push"
options:
- label: "Yes (Recommended)"
description: "Push immediately after each chunk"
- label: "No"
description: "Manual push when ready"3.5 Poll Interval
AskUserQuestion:
question: "How often should the chunker check for idle state?"
header: "Poll Interval"
options:
- label: "2 seconds"
description: "More responsive, slightly higher CPU"
- label: "5 seconds (Recommended)"
description: "Good balance"
- label: "10 seconds"
description: "Lower resource usage"---
TodoWrite Task Templates
Template: Full Setup
1. [Preflight] Validate all tools installed (asciinema, zstd, brotli, git, gh)
2. [Preflight] AskUserQuestion: offer installation for missing tools
3. [Account] Detect GitHub accounts from 5 sources
4. [Account] AskUserQuestion: select GitHub account
5. [Config] AskUserQuestion: repository URL
6. [Config] AskUserQuestion: recording directory
7. [Config] AskUserQuestion: branch name
8. [Advanced] AskUserQuestion: idle threshold
9. [Advanced] AskUserQuestion: zstd level
10. [Advanced] AskUserQuestion: brotli level
11. [Advanced] AskUserQuestion: auto-push
12. [Advanced] AskUserQuestion: poll interval
13. [Branch] Check if orphan branch exists on remote
14. [Branch] AskUserQuestion: handle existing branch
15. [Branch] Create orphan branch if needed
16. [Branch] Create GitHub Actions workflow with embedded parameters
17. [Local] Clone orphan branch to ~/asciinema_recordings/
18. [Local] Generate idle-chunker.sh with embedded parameters
19. [Validate] Run autonomous validation (8 tests)
20. [Validate] AskUserQuestion: recording test (user action)
21. [Validate] AskUserQuestion: chunker live test (user action)
22. [Guide] Display configuration summary and usage instructionsTemplate: Recording Session
1. [Context] Detect workspace from $PWD
2. [Context] Generate datetime for filename
3. [Context] Ensure tmp/ directory exists
4. [Command] Generate asciinema rec command
5. [Command] Generate idle-chunker command
6. [Guide] Display two-terminal workflow instructionsEvolution Log
Convention: Reverse chronological order (newest on top, oldest at bottom). Prepend new entries.
---
2026-02-26: Initial Evolution Log
Status: Skill is in use and maintained. Track improvements here.
Purpose
This evolution log tracks updates to the asciinema-streaming-backup skill. Each entry should note:
- What changed (content, structure, tooling)
- Why it changed (bug fix, feature request, best practice)
- Files affected
How to Use
1. When updating SKILL.md or references, add an entry here with the date 2. Keep entries reverse-chronological (newest first) 3. Link to ADRs or GitHub issues when relevant 4. Reference specific line changes when helpful
---
Skill: asciinema-streaming-backup
GitHub Actions Workflow
Complete GitHub Actions workflow for recompressing zstd chunks to brotli archives.
recompress.yml
# .github/workflows/recompress.yml
# Lives in the gh-recordings orphan branch
name: Recompress to Brotli
on:
push:
branches: [gh-recordings]
paths: ["chunks/**/*.zst"]
workflow_dispatch:
inputs:
force:
description: "Force recompress even if no new chunks"
required: false
default: "false"
jobs:
recompress:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Checkout orphan branch
uses: actions/checkout@v4
with:
ref: gh-recordings
fetch-depth: 1
- name: Install compression tools
run: |
sudo apt-get update
sudo apt-get install -y zstd brotli
- name: Check for chunks
id: check
run: |
if compgen -G "chunks/*.zst" > /dev/null 2>&1; then
echo "has_chunks=true" >> $GITHUB_OUTPUT
echo "chunk_count=$(ls -1 chunks/*.zst 2>/dev/null | wc -l)" >> $GITHUB_OUTPUT
else
echo "has_chunks=false" >> $GITHUB_OUTPUT
echo "chunk_count=0" >> $GITHUB_OUTPUT
fi
- name: Display chunk info
if: steps.check.outputs.has_chunks == 'true'
run: |
echo "=== Chunks to process ==="
ls -lh chunks/*.zst
echo ""
echo "Total chunks: ${{ steps.check.outputs.chunk_count }}"
- name: Concatenate and recompress
if: steps.check.outputs.has_chunks == 'true'
run: |
mkdir -p archives
# Generate archive name with timestamp
ARCHIVE_NAME="session_$(date +%Y%m%d_%H%M%S).cast.br"
echo "Processing ${{ steps.check.outputs.chunk_count }} chunks..."
# Concatenate all zstd chunks in order, decompress, recompress to brotli
# Sort by filename to ensure correct order
ls -1 chunks/*.zst | sort | xargs cat | zstd -d | brotli -9 -o "archives/$ARCHIVE_NAME"
# Get sizes for logging
CHUNKS_SIZE=$(du -sh chunks/*.zst | tail -1 | cut -f1)
ARCHIVE_SIZE=$(ls -lh "archives/$ARCHIVE_NAME" | awk '{print $5}')
echo ""
echo "=== Compression Results ==="
echo "Input chunks: ${{ steps.check.outputs.chunk_count }} files"
echo "Output archive: archives/$ARCHIVE_NAME"
echo "Archive size: $ARCHIVE_SIZE"
# Cleanup chunks after successful archival
rm -f chunks/*.zst
echo "Cleaned up processed chunks"
# Export for commit message
echo "ARCHIVE_NAME=$ARCHIVE_NAME" >> $GITHUB_ENV
echo "ARCHIVE_SIZE=$ARCHIVE_SIZE" >> $GITHUB_ENV
- name: Verify archive integrity
if: steps.check.outputs.has_chunks == 'true'
run: |
echo "Verifying archive..."
brotli -d -c "archives/${{ env.ARCHIVE_NAME }}" | head -5
echo "..."
echo "Archive verified successfully"
- name: Commit archive
if: steps.check.outputs.has_chunks == 'true'
uses: stefanzweifel/git-auto-commit-action@v5
with:
commit_message: |
chore: archive recording to brotli
Archive: ${{ env.ARCHIVE_NAME }}
Size: ${{ env.ARCHIVE_SIZE }}
Chunks processed: ${{ steps.check.outputs.chunk_count }}
file_pattern: "archives/*.br chunks/"
branch: gh-recordings
- name: No chunks to process
if: steps.check.outputs.has_chunks != 'true'
run: echo "No chunks found to process"How It Works
Trigger Conditions
1. Push to gh-recordings: Only triggers when .zst files are added/modified in chunks/ 2. Manual dispatch: Can be triggered manually with optional force flag
Processing Pipeline
chunks/*.zst → sort by name → cat → zstd -d → brotli -9 → archives/*.br1. Sort chunks: Ensures correct order (chunk_001, chunk_002, etc.) 2. Concatenate: Uses zstd's frame concatenation feature 3. Decompress: Single pass through zstd decoder 4. Recompress: Brotli -9 for ~300x total compression 5. Cleanup: Removes processed chunks
Output
- Archive name:
session_YYYYMMDD_HHMMSS.cast.br - Location:
archives/directory - Commit message includes size and chunk count
Customization
Change Brotli Level
For faster compression (less ratio):
brotli -6 -o "archives/$ARCHIVE_NAME"For maximum compression (slower):
brotli -11 -o "archives/$ARCHIVE_NAME" # May fail on very large filesKeep Chunks (No Cleanup)
Remove the cleanup line:
# rm -f chunks/*.zst # Comment out to keep chunksAdd Slack Notification
- name: Notify Slack
if: steps.check.outputs.has_chunks == 'true'
uses: slackapi/slack-github-action@v1
with:
payload: |
{
"text": "Recording archived: ${{ env.ARCHIVE_NAME }} (${{ env.ARCHIVE_SIZE }})"
}
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK }}Permissions Required
The workflow requires contents: write permission to:
1. Read chunks from the repository 2. Write archives to the repository 3. Delete processed chunks 4. Push commits back to gh-recordings
This is specified in the job:
permissions:
contents: writeTroubleshooting
"Permission denied" on push
Ensure the workflow has write permissions:
1. Go to repo Settings → Actions → General 2. Under "Workflow permissions", select "Read and write permissions"
"No chunks found" but chunks exist
Check the path pattern:
paths: ["chunks/**/*.zst"] # Must match your chunk locationArchive is corrupted
Verify chunks are sequential (no gaps or overlaps):
/usr/bin/env bash << 'GITHUB_WORKFLOW_SCRIPT_EOF'
for f in chunks/*.zst; do
echo "=== $f ==="
zstd -d -c "$f" | head -1
done
GITHUB_WORKFLOW_SCRIPT_EOFWorkflow not triggering
Check the branch filter:
branches: [gh-recordings] # Must match your orphan branch nameSkill: asciinema-streaming-backup
Idle Chunker Script (DEPRECATED)
DEPRECATED: This inline chunker has been superseded by the launchd daemon architecture.
See ADR 2025-12-26 for rationale.
>
Use instead: /asciinema-tools:daemon-setup to configure the background daemon.---
Historical Reference: The following documents the v9.3 inline chunker approach for reference only.
idle-chunker.sh (LEGACY)
/usr/bin/env bash << 'PREFLIGHT_EOF'
#!/usr/bin/env bash
# idle-chunker.sh - Creates zstd chunks during recording idle periods
#
# Usage: idle-chunker.sh <cast_file> <recordings_dir> [idle_threshold]
#
# Arguments:
# cast_file - Path to the active .cast recording file
# recordings_dir - Path to the orphan branch clone (e.g., ~/asciinema_recordings/repo-name)
# idle_threshold - Seconds of inactivity before chunking (default: 30)
#
# Environment:
# CHUNK_PREFIX - Prefix for chunk filenames (default: chunk)
# PUSH_ENABLED - Set to "false" to disable auto-push (default: true)
# VERBOSE - Set to "true" for debug output (default: false)
set -euo pipefail
# Arguments
CAST_FILE="${1:?Usage: idle-chunker.sh <cast_file> <recordings_dir> [idle_threshold]}"
RECORDINGS_DIR="${2:?Usage: idle-chunker.sh <cast_file> <recordings_dir> [idle_threshold]}"
IDLE_THRESHOLD="${3:-30}"
# Configuration
CHUNK_PREFIX="${CHUNK_PREFIX:-chunk}"
PUSH_ENABLED="${PUSH_ENABLED:-true}"
VERBOSE="${VERBOSE:-false}"
ZSTD_LEVEL="${ZSTD_LEVEL:-3}"
POLL_INTERVAL="${POLL_INTERVAL:-5}"
# State
last_chunk_pos=0
chunk_count=0
log() {
echo "[$(date +%H:%M:%S)] $*"
}
debug() {
[[ "$VERBOSE" == "true" ]] && log "DEBUG: $*"
}
# Validate inputs
if [[ ! -d "$RECORDINGS_DIR" ]]; then
log "ERROR: Recordings directory not found: $RECORDINGS_DIR"
exit 1
fi
if [[ ! -d "$RECORDINGS_DIR/chunks" ]]; then
log "Creating chunks directory..."
mkdir -p "$RECORDINGS_DIR/chunks"
fi
cd "$RECORDINGS_DIR"
log "Idle chunker started"
log " Monitoring: $CAST_FILE"
log " Chunks to: $RECORDINGS_DIR/chunks/"
log " Idle threshold: ${IDLE_THRESHOLD}s"
log " Auto-push: $PUSH_ENABLED"
log ""
log "Waiting for recording to start..."
# Wait for file to exist
while [[ ! -f "$CAST_FILE" ]]; do
sleep 2
done
log "Recording detected, monitoring for idle periods..."
# Get file modification time (cross-platform)
get_mtime() {
local file="$1"
if [[ "$(uname)" == "Darwin" ]]; then
stat -f%m "$file" 2>/dev/null || echo 0
else
stat -c%Y "$file" 2>/dev/null || echo 0
fi
}
# Get file size (cross-platform)
get_size() {
local file="$1"
if [[ "$(uname)" == "Darwin" ]]; then
stat -f%z "$file" 2>/dev/null || echo 0
else
stat -c%s "$file" 2>/dev/null || echo 0
fi
}
# Main loop
while true; do
# Check if file still exists (recording might have ended)
if [[ ! -f "$CAST_FILE" ]]; then
log "Recording file removed, creating final chunk..."
break
fi
# Check idle time
file_mtime=$(get_mtime "$CAST_FILE")
now=$(date +%s)
idle_seconds=$((now - file_mtime))
debug "Idle: ${idle_seconds}s, Threshold: ${IDLE_THRESHOLD}s"
if (( idle_seconds >= IDLE_THRESHOLD )); then
current_size=$(get_size "$CAST_FILE")
if (( current_size > last_chunk_pos )); then
chunk_count=$((chunk_count + 1))
chunk_name="${CHUNK_PREFIX}_$(date +%Y%m%d_%H%M%S)_${chunk_count}.cast"
new_bytes=$((current_size - last_chunk_pos))
log "Idle detected (${idle_seconds}s) - creating chunk..."
# Extract only new bytes since last chunk (no overlap!)
tail -c +"$((last_chunk_pos + 1))" "$CAST_FILE" > "chunks/$chunk_name"
# Compress with zstd
zstd -${ZSTD_LEVEL} --rm "chunks/$chunk_name"
log "Created: chunks/${chunk_name}.zst (${new_bytes} bytes, chunk #${chunk_count})"
# Push to GitHub
if [[ "$PUSH_ENABLED" == "true" ]]; then
if git add chunks/ && git commit -m "chunk #${chunk_count}: $(date +%H:%M)" 2>/dev/null; then
if git push 2>/dev/null; then
log "Pushed to GitHub"
else
log "WARNING: Push failed (will retry next chunk)"
fi
fi
fi
# Update position tracker
last_chunk_pos=$current_size
# Reset idle detection (wait for new content)
sleep $POLL_INTERVAL
fi
fi
sleep $POLL_INTERVAL
done
# Final chunk if there's remaining data
if [[ -f "$CAST_FILE" ]]; then
current_size=$(get_size "$CAST_FILE")
if (( current_size > last_chunk_pos )); then
chunk_count=$((chunk_count + 1))
chunk_name="${CHUNK_PREFIX}_$(date +%Y%m%d_%H%M%S)_final.cast"
tail -c +"$((last_chunk_pos + 1))" "$CAST_FILE" > "chunks/$chunk_name"
zstd -${ZSTD_LEVEL} --rm "chunks/$chunk_name"
log "Created final chunk: chunks/${chunk_name}.zst"
if [[ "$PUSH_ENABLED" == "true" ]]; then
git add chunks/ && git commit -m "chunk #${chunk_count}: final" && git push
log "Pushed final chunk to GitHub"
fi
fi
fi
log "Idle chunker finished (${chunk_count} chunks created)"
PREFLIGHT_EOFUsage Examples
Basic Usage
# Start recording in terminal 1
asciinema rec ~/project/tmp/session.cast
# Start chunker in terminal 2
~/asciinema_recordings/my-repo/idle-chunker.sh ~/project/tmp/session.cast ~/asciinema_recordings/my-repoWith Custom Threshold
# Chunk after 15 seconds of idle (more frequent)
idle-chunker.sh session.cast ~/asciinema_recordings/repo 15
# Chunk after 60 seconds of idle (less frequent)
idle-chunker.sh session.cast ~/asciinema_recordings/repo 60Debug Mode
VERBOSE=true idle-chunker.sh session.cast ~/asciinema_recordings/repoDisable Auto-Push (Manual Control)
PUSH_ENABLED=false idle-chunker.sh session.cast ~/asciinema_recordings/repo
# Push manually when ready
cd ~/asciinema_recordings/repo && git pushHow It Works
1. File Monitoring: Watches the .cast file's modification time 2. Idle Detection: When file hasn't been modified for IDLE_THRESHOLD seconds 3. Chunk Extraction: Uses tail -c +N to extract only new bytes (no overlap) 4. Compression: zstd -3 provides ~10x compression with speed 5. Git Push: Commits and pushes to orphan branch 6. Position Tracking: Remembers last chunk position to avoid duplication
Key Design: No Overlap
The script tracks last_chunk_pos to ensure chunks are sequential, not overlapping:
File: [AAAAAABBBBBBCCCCCC]
^ ^ ^
Chunk 1: [AAAAAA] (bytes 0-5)
Chunk 2: [BBBBBB] (bytes 6-11)
Chunk 3: [CCCCCC] (bytes 12-17)This allows zstd concatenation to work correctly:
cat chunk_1.zst chunk_2.zst chunk_3.zst > combined.zst
zstd -d combined.zst # Produces original fileSkill: asciinema-streaming-backup
Setup Scripts
Table of Contents
Complete setup and validation scripts for the asciinema streaming backup system.
preflight-check.sh
Validates all required tools are installed.
/usr/bin/env bash << 'PREFLIGHT_EOF'
#!/usr/bin/env bash
# preflight-check.sh - Validates all requirements with self-correction
#
# Usage: preflight-check.sh [--fix]
# --fix Attempt to install missing tools via Homebrew
set -euo pipefail
FIX_MODE="${1:-}"
MISSING=()
WARNINGS=()
log() { echo "[preflight] $*"; }
warn() { WARNINGS+=("$*"); }
fail() { MISSING+=("$*"); }
# Check each required tool
check_tool() {
local tool="$1"
local install_cmd="${2:-brew install $tool}"
if command -v "$tool" &>/dev/null; then
log "$tool: OK ($(command -v "$tool"))"
else
fail "$tool"
log "$tool: MISSING"
log " Install: $install_cmd"
fi
}
log "=== Checking required tools ==="
check_tool "asciinema" "brew install asciinema"
check_tool "zstd" "brew install zstd"
check_tool "brotli" "brew install brotli"
check_tool "git" "xcode-select --install"
check_tool "gh" "brew install gh"
log ""
log "=== Checking optional tools ==="
if command -v fswatch &>/dev/null; then
log "fswatch: OK (enables real-time monitoring)"
else
log "fswatch: NOT INSTALLED (optional)"
log " Install: brew install fswatch"
fi
# Check asciinema version
if command -v asciinema &>/dev/null; then
log ""
log "=== Checking versions ==="
ASCIINEMA_VERSION=$(asciinema --version 2>&1 | grep -oE '[0-9]+\.[0-9]+' | head -1)
if [[ -n "$ASCIINEMA_VERSION" ]]; then
MAJOR="${ASCIINEMA_VERSION%%.*}"
if (( MAJOR >= 3 )); then
log "asciinema: v$ASCIINEMA_VERSION (Rust version, recommended)"
else
warn "asciinema: v$ASCIINEMA_VERSION (Python version, upgrade recommended)"
log " Upgrade: brew upgrade asciinema"
fi
fi
fi
# Check gh authentication
if command -v gh &>/dev/null; then
log ""
log "=== Checking GitHub CLI auth ==="
if gh auth status &>/dev/null; then
log "gh: Authenticated"
else
warn "gh: Not authenticated"
log " Run: gh auth login"
fi
fi
# Summary
log ""
log "=== Summary ==="
if [[ ${#MISSING[@]} -gt 0 ]]; then
log "Missing tools: ${MISSING[*]}"
if [[ "$FIX_MODE" == "--fix" ]]; then
log ""
log "Attempting to install missing tools..."
brew install "${MISSING[@]}"
log "Installation complete. Re-run preflight to verify."
else
log ""
log "To install all missing tools:"
log " brew install ${MISSING[*]}"
log ""
log "Or run: $0 --fix"
exit 1
fi
else
log "All required tools installed"
fi
if [[ ${#WARNINGS[@]} -gt 0 ]]; then
log ""
log "Warnings:"
for w in "${WARNINGS[@]}"; do
log " - $w"
done
fi
PREFLIGHT_EOFsetup-orphan-branch.sh
Creates the orphan branch with GitHub Actions workflow.
/usr/bin/env bash << 'PREFLIGHT_EOF_2'
#!/usr/bin/env bash
# setup-orphan-branch.sh - Creates gh-recordings orphan branch
#
# Usage: setup-orphan-branch.sh <repo_url>
# repo_url SSH or HTTPS URL (e.g., git@github.com:user/repo.git)
#
# Creates:
# - Orphan branch 'gh-recordings' with separate history
# - GitHub Actions workflow for brotli recompression
# - Local clone at ~/asciinema_recordings/<repo-name>/
set -euo pipefail
REPO_URL="${1:?Usage: setup-orphan-branch.sh <repo_url>}"
BRANCH="gh-recordings"
BROTLI_LEVEL="${BROTLI_LEVEL:-9}"
# Extract repo name from URL
REPO_NAME=$(basename "$REPO_URL" .git)
LOCAL_DIR="$HOME/asciinema_recordings/$REPO_NAME"
log() { echo "[setup] $*"; }
# Detect GitHub account from gh auth
detect_github_account() {
log "Detecting GitHub accounts..."
ACCOUNTS=$(gh auth status 2>&1 | grep -oE 'Logged in to github.com account [^ ]+' | awk '{print $NF}' || true)
if [[ -z "$ACCOUNTS" ]]; then
log "ERROR: No GitHub accounts found. Run 'gh auth login' first."
exit 1
fi
ACTIVE_ACCOUNT=$(gh auth status 2>&1 | grep -A1 'github.com' | grep 'Active account: true' -B1 | head -1 | awk '{print $NF}' || echo "$ACCOUNTS" | head -1)
log "Active GitHub account: $ACTIVE_ACCOUNT"
# Check if correct account for this repo
REPO_OWNER=$(echo "$REPO_URL" | sed -E 's|.*github.com[:/]([^/]+)/.*|\1|')
if [[ "$ACTIVE_ACCOUNT" != "$REPO_OWNER" ]]; then
log "Switching to account: $REPO_OWNER"
if ! gh auth switch --user "$REPO_OWNER" 2>/dev/null; then
log "WARNING: Could not switch to $REPO_OWNER, using $ACTIVE_ACCOUNT"
fi
fi
SELECTED_ACCOUNT="${REPO_OWNER:-$ACTIVE_ACCOUNT}"
}
# Get SSH key for selected account
get_ssh_key() {
local account="$1"
local key_path="$HOME/.ssh/id_ed25519_${account}"
if [[ -f "$key_path" ]]; then
echo "$key_path"
elif [[ -f "$HOME/.ssh/id_ed25519" ]]; then
echo "$HOME/.ssh/id_ed25519"
else
echo ""
fi
}
detect_github_account
SSH_KEY=$(get_ssh_key "$SELECTED_ACCOUNT")
if [[ -n "$SSH_KEY" ]]; then
export GIT_SSH_COMMAND="ssh -i $SSH_KEY"
log "Using SSH key: $SSH_KEY"
fi
log "Repository: $REPO_URL"
log "Branch: $BRANCH"
log "Local directory: $LOCAL_DIR"
log ""
# Check if branch already exists
if git ls-remote --heads "$REPO_URL" "$BRANCH" 2>/dev/null | grep -q "$BRANCH"; then
log "Orphan branch '$BRANCH' already exists on remote"
if [[ -d "$LOCAL_DIR" ]]; then
log "Local clone already exists: $LOCAL_DIR"
log "Pulling latest..."
git -C "$LOCAL_DIR" pull
else
log "Cloning to: $LOCAL_DIR"
mkdir -p "$(dirname "$LOCAL_DIR")"
git clone --single-branch --branch "$BRANCH" --depth 1 "$REPO_URL" "$LOCAL_DIR"
fi
log "Setup complete"
exit 0
fi
log "Creating orphan branch..."
# Create temporary clone for setup
TEMP_DIR=$(mktemp -d)
trap "rm -rf $TEMP_DIR" EXIT
git clone --depth 1 "$REPO_URL" "$TEMP_DIR"
cd "$TEMP_DIR"
# Create orphan branch
git checkout --orphan "$BRANCH"
git rm -rf .
# Setup directory structure
mkdir -p .github/workflows chunks archives
# Create GitHub Actions workflow (brotli level embedded at creation time)
cat > .github/workflows/recompress.yml << WORKFLOW_EOF
name: Recompress to Brotli
on:
push:
branches: [gh-recordings]
paths: ['chunks/**/*.zst']
workflow_dispatch:
jobs:
recompress:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v4
- name: Install tools
run: sudo apt-get update && sudo apt-get install -y zstd brotli
- name: Recompress chunks
run: |
if compgen -G "chunks/*.zst" > /dev/null; then
mkdir -p archives
ARCHIVE="session_\$(date +%Y%m%d_%H%M%S).cast.br"
ls -1 chunks/*.zst | sort | xargs cat | zstd -d | brotli -${BROTLI_LEVEL} -o "archives/\$ARCHIVE"
rm -f chunks/*.zst
echo "ARCHIVE=\$ARCHIVE" >> \$GITHUB_ENV
fi
- name: Commit
if: env.ARCHIVE != ''
uses: stefanzweifel/git-auto-commit-action@v5
with:
commit_message: "chore: archive to brotli (\${{ env.ARCHIVE }})"
file_pattern: 'archives/*.br chunks/'
WORKFLOW_EOF
# Create placeholder files
cat > chunks/README.md << 'EOF'
# Chunks
Streaming zstd-compressed recording chunks.
Auto-deleted after archival to brotli.
EOF
cat > archives/README.md << 'EOF'
# Archives
Final brotli-compressed recordings.
~300x compression ratio.
EOF
# Create main README
cat > README.md << 'EOF'
# Recording Storage
Orphan branch for asciinema recording backups.
Completely isolated from main codebase history.
## Structure
- `chunks/` - Streaming zstd chunks (temporary)
- `archives/` - Brotli archives (permanent)
## Workflow
1. Local idle-chunker creates zstd chunks
2. Chunks pushed to this branch
3. GitHub Action recompresses to brotli
4. Chunks deleted, archives retained
## Isolation
This is an orphan branch with no shared history.
Git refuses to merge with main: "refusing to merge unrelated histories"
EOF
# Initial commit
git add .
git commit -m "init: recording storage (orphan branch)"
# Push
log "Pushing orphan branch to remote..."
git push -u origin "$BRANCH"
# Clone to local recordings directory
cd -
mkdir -p "$(dirname "$LOCAL_DIR")"
git clone --single-branch --branch "$BRANCH" --depth 1 "$REPO_URL" "$LOCAL_DIR"
# Copy idle-chunker script
cat > "$LOCAL_DIR/idle-chunker.sh" << 'CHUNKER_EOF'
#!/usr/bin/env bash
# idle-chunker.sh - See references/idle-chunker.md for full version
CAST_FILE="${1:?Usage: idle-chunker.sh <cast_file>}"
IDLE_THRESHOLD="${2:-30}"
cd "$(dirname "$0")"
last_pos=0
echo "Monitoring: $CAST_FILE (idle threshold: ${IDLE_THRESHOLD}s)"
while [[ -f "$CAST_FILE" ]] || sleep 2; do
[[ -f "$CAST_FILE" ]] || continue
mtime=$(stat -f%m "$CAST_FILE" 2>/dev/null || stat -c%Y "$CAST_FILE")
idle=$(($(date +%s) - mtime))
size=$(stat -f%z "$CAST_FILE" 2>/dev/null || stat -c%s "$CAST_FILE")
if (( idle >= IDLE_THRESHOLD && size > last_pos )); then
chunk="chunks/chunk_$(date +%Y%m%d_%H%M%S).cast"
tail -c +$((last_pos + 1)) "$CAST_FILE" > "$chunk"
zstd -3 --rm "$chunk"
git add chunks/ && git commit -m "chunk $(date +%H:%M)" && git push
last_pos=$size
echo "[$(date +%H:%M:%S)] Created: ${chunk}.zst"
fi
sleep 5
done
CHUNKER_EOF
chmod +x "$LOCAL_DIR/idle-chunker.sh"
log ""
log "=== Setup Complete ==="
log "Local directory: $LOCAL_DIR"
log "Idle chunker: $LOCAL_DIR/idle-chunker.sh"
log ""
log "To start recording:"
log " 1. asciinema rec /path/to/session.cast"
log " 2. $LOCAL_DIR/idle-chunker.sh /path/to/session.cast"
PREFLIGHT_EOF_2validate-system.sh
Complete system validation with self-correction.
/usr/bin/env bash << 'PREFLIGHT_EOF_3'
#!/usr/bin/env bash
# validate-system.sh - Full system validation
#
# Usage: validate-system.sh <repo_url> [--fix]
set -euo pipefail
REPO_URL="${1:?Usage: validate-system.sh <repo_url> [--fix]}"
FIX_MODE="${2:-}"
REPO_NAME=$(basename "$REPO_URL" .git)
LOCAL_DIR="$HOME/asciinema_recordings/$REPO_NAME"
ERRORS=()
FIXES=()
log() { echo "[validate] $*"; }
error() { ERRORS+=("$*"); log "ERROR: $*"; }
fix() { FIXES+=("$*"); }
log "=== Validating Streaming Backup System ==="
log "Repository: $REPO_URL"
log "Local: $LOCAL_DIR"
log ""
# 1. Check tools
log "--- Tools ---"
for tool in asciinema zstd brotli git gh; do
if command -v "$tool" &>/dev/null; then
log "$tool: OK"
else
error "$tool: MISSING"
fix "brew install $tool"
fi
done
# 2. Check orphan branch exists
log ""
log "--- Remote Branch ---"
if git ls-remote --heads "$REPO_URL" gh-recordings 2>/dev/null | grep -q gh-recordings; then
log "gh-recordings: EXISTS"
else
error "gh-recordings: NOT FOUND"
fix "./setup-orphan-branch.sh $REPO_URL"
fi
# 3. Check local clone
log ""
log "--- Local Clone ---"
if [[ -d "$LOCAL_DIR" ]]; then
log "Directory: EXISTS"
# Check it's correct branch
BRANCH=$(git -C "$LOCAL_DIR" branch --show-current 2>/dev/null || echo "")
if [[ "$BRANCH" == "gh-recordings" ]]; then
log "Branch: OK (gh-recordings)"
else
error "Branch: WRONG ($BRANCH)"
fix "cd $LOCAL_DIR && git checkout gh-recordings"
fi
# Check workflow exists
if [[ -f "$LOCAL_DIR/.github/workflows/recompress.yml" ]]; then
log "Workflow: EXISTS"
else
error "Workflow: MISSING"
fix "Regenerate workflow"
fi
# Check directories
[[ -d "$LOCAL_DIR/chunks" ]] && log "chunks/: EXISTS" || error "chunks/: MISSING"
[[ -d "$LOCAL_DIR/archives" ]] && log "archives/: EXISTS" || error "archives/: MISSING"
# Check idle-chunker
if [[ -x "$LOCAL_DIR/idle-chunker.sh" ]]; then
log "idle-chunker.sh: EXISTS"
else
error "idle-chunker.sh: MISSING"
fi
else
error "Local directory: NOT FOUND"
fix "git clone --single-branch --branch gh-recordings --depth 1 $REPO_URL $LOCAL_DIR"
fi
# 4. Test compression
log ""
log "--- Compression Test ---"
TEST_DATA="test-$(date +%s)"
if echo "$TEST_DATA" | zstd -3 | zstd -d | grep -q "$TEST_DATA"; then
log "zstd round-trip: OK"
else
error "zstd round-trip: FAILED"
fi
if echo "$TEST_DATA" | brotli | brotli -d | grep -q "$TEST_DATA"; then
log "brotli round-trip: OK"
else
error "brotli round-trip: FAILED"
fi
# 5. Test zstd concatenation
log ""
log "--- Concatenation Test ---"
TMP=$(mktemp -d)
echo "chunk1" | zstd -3 > "$TMP/a.zst"
echo "chunk2" | zstd -3 > "$TMP/b.zst"
cat "$TMP/a.zst" "$TMP/b.zst" > "$TMP/combined.zst"
RESULT=$(zstd -d -c "$TMP/combined.zst")
rm -rf "$TMP"
if [[ "$RESULT" == $'chunk1\nchunk2' ]]; then
log "zstd concatenation: OK"
else
error "zstd concatenation: FAILED"
fi
# Summary
log ""
log "=== Summary ==="
if [[ ${#ERRORS[@]} -eq 0 ]]; then
log "All checks passed"
exit 0
fi
log "Errors found: ${#ERRORS[@]}"
for e in "${ERRORS[@]}"; do
log " - $e"
done
if [[ ${#FIXES[@]} -gt 0 ]]; then
log ""
log "Suggested fixes:"
for f in "${FIXES[@]}"; do
log " $f"
done
fi
exit 1
PREFLIGHT_EOF_3test-workflow.sh
Test the complete workflow end-to-end.
/usr/bin/env bash << 'VALIDATE_EOF'
#!/usr/bin/env bash
# test-workflow.sh - End-to-end workflow test
#
# Usage: test-workflow.sh <local_recordings_dir>
#
# Creates a test recording, generates chunks, and verifies round-trip
set -euo pipefail
LOCAL_DIR="${1:?Usage: test-workflow.sh <local_recordings_dir>}"
log() { echo "[test] $*"; }
log "=== Testing Streaming Backup Workflow ==="
log "Directory: $LOCAL_DIR"
# Create test .cast file
TEST_CAST=$(mktemp).cast
log ""
log "Creating test recording: $TEST_CAST"
cat > "$TEST_CAST" << 'CAST_EOF'
{"version": 2, "width": 80, "height": 24, "timestamp": 1234567890}
[0.1, "o", "$ echo hello\r\n"]
[0.2, "o", "hello\r\n"]
[0.3, "o", "$ echo world\r\n"]
[0.4, "o", "world\r\n"]
CAST_EOF
log "Test recording created ($(wc -l < "$TEST_CAST") lines)"
# Simulate chunking
log ""
log "Creating test chunks..."
cd "$LOCAL_DIR"
mkdir -p chunks
# Chunk 1: header + first command
head -3 "$TEST_CAST" > chunks/test_001.cast
zstd -3 --rm chunks/test_001.cast
log "Created: chunks/test_001.cast.zst"
# Chunk 2: remaining lines
tail -n +4 "$TEST_CAST" > chunks/test_002.cast
zstd -3 --rm chunks/test_002.cast
log "Created: chunks/test_002.cast.zst"
# Test concatenation
log ""
log "Testing concatenation..."
cat chunks/test_*.zst > /tmp/test_combined.zst
zstd -d /tmp/test_combined.zst -o /tmp/test_combined.cast
# Verify content
if diff -q "$TEST_CAST" /tmp/test_combined.cast &>/dev/null; then
log "Concatenation: PASSED (content matches)"
else
log "Concatenation: FAILED (content differs)"
diff "$TEST_CAST" /tmp/test_combined.cast
exit 1
fi
# Test brotli recompression
log ""
log "Testing brotli recompression..."
brotli -9 /tmp/test_combined.cast -o /tmp/test_archive.cast.br
brotli -d /tmp/test_archive.cast.br -o /tmp/test_final.cast
if diff -q "$TEST_CAST" /tmp/test_final.cast &>/dev/null; then
log "Brotli round-trip: PASSED"
else
log "Brotli round-trip: FAILED"
exit 1
fi
# Size comparison
ORIG_SIZE=$(wc -c < "$TEST_CAST")
ZSTD_SIZE=$(cat chunks/test_*.zst | wc -c)
BR_SIZE=$(wc -c < /tmp/test_archive.cast.br)
log ""
log "=== Size Comparison ==="
log "Original: $ORIG_SIZE bytes"
log "zstd chunks: $ZSTD_SIZE bytes ($(echo "scale=1; $ORIG_SIZE / $ZSTD_SIZE" | bc)x)"
log "brotli: $BR_SIZE bytes ($(echo "scale=1; $ORIG_SIZE / $BR_SIZE" | bc)x)"
# Cleanup test files
rm -f "$TEST_CAST" /tmp/test_*.cast /tmp/test_*.zst /tmp/test_*.br
rm -f chunks/test_*.zst
log ""
log "=== All Tests Passed ==="
VALIDATE_EOFSkill: asciinema-streaming-backup
Troubleshooting Guide
Common issues and fixes for the streaming backup system.
---
"Cannot push to orphan branch"
Cause: Authentication or permissions issue.
Fix:
# Check gh auth status
gh auth status
# Re-authenticate if needed
gh auth login"Chunks not being created"
Cause: Idle threshold not reached, or file not growing.
Fix:
- Verify recording is active:
tail -f $CAST_FILE - Lower threshold:
IDLE_THRESHOLD=15 - Check file permissions
"GitHub Action not triggering"
Cause: Workflow file missing or wrong branch filter.
Fix:
# Verify workflow exists
cat ~/asciinema_recordings/REPO/.github/workflows/recompress.yml
# Check branch filter includes gh-recordings
grep -A2 "branches:" ~/asciinema_recordings/REPO/.github/workflows/recompress.yml"Brotli archive empty or corrupted"
Cause: zstd chunks not concatenating properly (overlapping data).
Fix: Ensure idle-chunker uses last_chunk_pos to avoid overlap:
/usr/bin/env bash << 'PREFLIGHT_EOF_2'
# Check for overlaps - each chunk should be sequential
for f in chunks/*.zst; do
zstd -d "$f" -c | head -1
done
PREFLIGHT_EOF_2Validation Failure Quick Reference
| Failure | Cause | Resolution |
|---|---|---|
asciinema MISSING | Not installed | brew install asciinema (macOS) or pipx install asciinema |
zstd MISSING | Not installed | brew install zstd (macOS) or apt install zstd (Linux) |
brotli MISSING | Not installed | brew install brotli (macOS) or apt install brotli (Linux) |
gh not authenticated | No GitHub login | Run gh auth login and follow prompts |
gh-recordings NOT found on remote | Branch not pushed | Run orphan branch setup from Phase 4 of skill |
local directory NOT found | Clone failed | Check repo URL and permissions, re-run clone |
recompress.yml MISSING | Workflow not created | Re-run orphan branch setup to create workflow |
workflow trigger failed | No workflow_dispatch | Add workflow_dispatch: trigger to workflow |
zstd concatenation FAILED | zstd version issue | Update zstd: brew upgrade zstd |
brotli round-trip FAILED | brotli corrupted | Reinstall: brew reinstall brotli |