Transcript Fixer

Transcript Fixer

Two-phase correction pipeline: deterministic dictionary rules (instant, free) followed by AI-powered error detection. Corrections accumulate in ~/.transcript-fixer/corrections.db, improving accuracy over time.

What each phase is actually good at (calibration, not a rule): the dictionary shines on recurring errors — product names, common homophones, anything you've corrected before — at zero cost and zero latency. But on a fresh database, on high-quality ASR (e.g. transcripts from a strong engine like Whisper, Otter, or Feishu / Tencent-Meeting), or in specialized domains (finance, medical, legal), the dictionary often matches almost nothing — the errors that remain are proper nouns and domain terms it has never seen. There, the AI pass does essentially all the real work. Treat Stage 1 as a cheap pre-filter for known repeats, not as the primary corrector, and don't be alarmed when it changes only a handful of lines on a clean transcript.

Prerequisites

All scripts use PEP 723 inline metadata — uv run auto-installs dependencies. Requires uv (install guide).

Quick Start

# First time: Initialize database
uv run scripts/fix_transcription.py --init

# Single file
uv run scripts/fix_transcription.py --input meeting.md --stage 1

# Batch: multiple files in parallel (use shell loop)
for f in /path/to/*.txt; do
  uv run scripts/fix_transcription.py --input "$f" --stage 1
done

After Stage 1, Claude reads the output and fixes remaining ASR errors natively (no API key needed). The full method — triage by confidence, verify-don't-guess, second pass, needs-checking list — is in Native AI Correction below; read that section as the source of truth. For a quick, clean transcript it collapses to: read the whole thing → fix the obvious errors with sed → save reusable patterns to the dictionary.

See references/example_session.md for a concrete input/output walkthrough.

Alternative: API batch processing (for automation without Claude Code):

export GLM_API_KEY="<api-key>"  # From https://open.bigmodel.cn/
uv run scripts/fix_transcript_enhanced.py input.md --output ./corrected

Core Workflow

Two-phase pipeline with persistent learning:

1. Initialize (once): uv run scripts/fix_transcription.py --init 2. Add domain corrections: --add "错误词" "正确词" --domain <domain> 3. Phase 1 — Dictionary: --input file.md --stage 1 (instant, free) 4. Phase 2 — AI Correction: Claude reads output and fixes errors natively, or --stage 3 with GLM_API_KEY for API mode 5. Save stable patterns: --add "错误词" "正确词" after each session 6. Review learned patterns: --review-learned and --approve high-confidence suggestions

Domains: general, embodied_ai, finance, medical, or custom (e.g., legal, gaming) Learning: Patterns appearing ≥3 times at ≥80% confidence auto-promote from AI to dictionary

After fixing, always save reusable corrections to dictionary. This is the skill's core value — see references/iteration_workflow.md for the complete checklist.

Dictionary Addition After Fixing

After native AI correction, review all applied fixes and decide which to save. Use this decision matrix:

Batch add multiple corrections in one session:

uv run scripts/fix_transcription.py --add "错误1" "正确1" --domain tech
uv run scripts/fix_transcription.py --add "错误2" "正确2" --domain business
# Chain with && for efficiency

False Positive Prevention

Adding wrong dictionary rules silently corrupts future transcripts. Read `references/false_positive_guide.md` before adding any correction rule, especially for short words (≤2 chars) or common Chinese words that appear correctly in normal text.

Native AI Correction (Default Mode)

When running inside Claude Code, use Claude's own language understanding for Phase 2 — on high-quality ASR this is where almost all the real correction happens. Scale the effort to the transcript. A short, clean recording with no proper nouns (a quick voice memo) just needs steps 1-3 plus one obvious-fix pass; skip the verification / second-pass / subagent / needs-checking machinery below, which earns its keep on long, multi-speaker, domain-heavy, or high-stakes transcripts. Don't turn a 10-second memo into a research project.

1. Run Stage 1 (dictionary) on all files (parallel if multiple) 2. Verify Stage 1 — diff against the original. If the dictionary introduced false positives, work from the original file instead and apply your edits there 3. Read the entire transcript before proposing corrections — later context disambiguates earlier errors (a name garbled near the start often becomes obvious later). For large files, read in chunks but finish the whole thing before deciding anything 4. Triage each candidate error into one of three buckets — this triage is the part that takes judgment:

• Confident fix — non-words, obvious garbling, product-name variants you already recognize, or a homophone that's unambiguous in context (their→there where context forces it; 彭波→彭博 when every other mention already reads 彭博). Apply directly (step 5).
• Needs verification — a proper noun you can't confirm from context: a person / company / ticker / product / place name (a misheard drug name in a medical interview, a researcher's surname in a podcast, a ticker on an earnings call), or any term you can't point to a specific source for — even one you think you recognize ("I'm pretty sure" is exactly how wrong names slip in). Search it, don't guess — WebSearch, or a local grep if it's a project / personal entity. A confirmed result becomes a Confident fix; if the search can't confirm it, it drops to Uncertain. Batch these: collect the unique unknowns and look them up together, not one-by-one.
• Uncertain — you suspect an error but can't confirm it even after searching (a syllable that maps to several real entities; a structurally broken sentence). Leave the original text exactly as-is and record it in the needs-checking list (step 7). A fluent-but-wrong "fix" is harder to catch downstream than an obvious garble — silence beats a confident guess.

5. Apply the confident fixes efficiently:

• Global replacements (unique non-words like "克劳锐"→"Claude"): one sed -i '' with multiple -e flags
• Context-dependent (a word that's only wrong in one context, like "争"→"蒸" in a distillation discussion): sed with a longer surrounding phrase for uniqueness, or the Edit tool
• Re-grep each changed term afterward to confirm it landed and didn't hit look-alikes you meant to keep

6. Second pass — catch what one read missed. A single linear read reliably leaves residue: an idiom degraded into a near-homophone, a term wrong in just one spot among many correct ones, an acronym misheard as another. Always re-scan once for leftovers. For a long or high-stakes transcript, also spawn an independent subagent (Task) to re-read the corrected file cold — fresh eyes with no memory of your first pass catch what you've read past. Have it report suspected residuals with line numbers, then run each back through step-4 triage (fix / search / log). Task works when you're in the main context; if it isn't available — e.g. these instructions are themselves running inside a subagent, which can't spawn another — just do one more thorough independent re-read yourself. Never skip the second pass over a missing tool. 7. Emit a needs-checking list — in your chat summary to the human, not baked into the file — for everything still Uncertain: line number, the original text you left in place, what you suspect, and why you couldn't confirm it. This surfaces the few items that need a recording or source to resolve, instead of burying them or papering over them with guesses. If nothing is uncertain, say so. 8. Verify with diff against the file you actually edited (diff <original> <your-working-file>) — every change should trace back to a triage decision 9. Finalize: rename *_stage1.md → *.md, delete the original .txt 10. Save stable patterns to the dictionary (see "Dictionary Addition" below) 11. If you worked from corrected_stage1.md, strip any remaining Stage 1 false positives before finalizing

Common ASR Error Patterns

AI product names are frequently garbled. These patterns recur across transcripts:

Person names and company names also produce consistent ASR errors across sessions — always add confirmed name corrections to the dictionary.

Efficient Batch Fix Strategy

When fixing multiple files (e.g., 5 transcripts from one day):

1. Stage 1 in parallel: run all files through dictionary at once 2. Read all files first: build a mental model of speakers, topics, and recurring terms before fixing anything 3. Compile a global correction list: many errors repeat across files from the same session (same speakers, same topics) 4. Apply global corrections first (sed with multiple -e flags), then per-file context-dependent fixes 5. Verify all diffs, finalize all files, then do one dictionary addition pass

Parallel via Dynamic Workflow (large batches)

For a large batch (10+ files), a Dynamic Workflow — one subagent per file, running in parallel — is faster than a shell loop and gives each file full AI attention. Four rules earned the hard way; skipping any of them has caused real damage:

1. Hardcode the file list into the script — don't pass it through `args`. A Workflow args array of strings containing non-ASCII characters, brackets, or path separators can silently arrive empty: the script sees zero files, no agents spawn, and it exits instantly with something like "no files". Plain alphanumeric tokens pass fine, but file paths should go straight into a const FILES = [...] literal in the script body, guarded with if (!FILES.length) return.

2. Scope each agent to exactly one file, and forbid cross-file `grep -r` / `sed` in its prompt. Left unconstrained, an agent will turn a local fix ("this garbled term → correct term, here") into a global search-and-replace and edit unrelated files that were never part of the batch. State the single file path and an explicit "only edit this one file" instruction.

3. After the batch, verify with `git diff` before trusting it (works when the files are under version control):

• git diff --name-only against your intended list — this catches any agent that strayed outside its assigned file. git checkout to revert the strays.
• grep the deleted (-) lines for invariants that must never change. For speaker-diarized transcripts, that invariant is the speaker-label lines — an ASR fix should only ever touch spoken content, never alter or reassign who-said-what. Confirm zero speaker lines were deleted or changed.

4. Run the aggregated dictionary suggestions through the false-positive filter before saving any of them. Parallel agents collectively propose far more rules than are safe — and they don't see each other's suggestions, so duplicates and overreach pile up. Keep only unambiguous non-word → correct-term mappings. Drop anything whose "from" side is a real word in some context: a common word, or a term that's only wrong inside one domain. A global dictionary rule on a real word silently corrupts every future transcript — exactly what references/false_positive_guide.md warns about. (In one real batch, ~80 raw suggestions collapsed to ~18 safe ones after this filter.)

Enhanced Capabilities (Native Mode Only)

• Intelligent paragraph breaks: Add \n\n at logical topic transitions
• Filler word reduction: "这个这个这个" → "这个"
• Interactive review: Corrections confirmed before applying
• Context-aware judgment: Full document context resolves ambiguous errors

When to Use API Mode Instead

Use GLM_API_KEY + Stage 3 for batch processing, standalone usage without Claude Code, or reproducible automated processing.

Legacy Fallback

When the script outputs [CLAUDE_FALLBACK] (GLM API error), switch to native mode automatically.

Utility Scripts

Timestamp repair:

uv run scripts/fix_transcript_timestamps.py meeting.txt --in-place

Split transcript into sections (rebase each to 00:00:00):

uv run scripts/split_transcript_sections.py meeting.txt \
  --first-section-name "intro" \
  --section "main::<verbatim line that starts the next section>" \
  --rebase-to-zero

Word-level diff (recommended for reviewing corrections):

uv run scripts/generate_word_diff.py original.md corrected.md output.html

Output Files

• *_stage1.md — Dictionary corrections applied
• *_corrected.txt — Final version (native mode) or *_stage2.md (API mode)
• *_对比.html — Visual diff (open in browser)

Database Operations

Read `references/database_schema.md` before writing any custom query — the column names are not what you'd guess. The correction columns are `from_text` / `to_text` (not wrong_term/correct_term, not original/corrected). Guessing column names is the most common way these queries fail with "no such column".

# Inspect corrections — real column names are from_text, to_text, domain
sqlite3 ~/.transcript-fixer/corrections.db "SELECT from_text, to_text, domain FROM active_corrections;"
# Count rules per domain
sqlite3 ~/.transcript-fixer/corrections.db "SELECT domain, COUNT(*) FROM active_corrections GROUP BY domain;"
# Schema version
sqlite3 ~/.transcript-fixer/corrections.db "SELECT value FROM system_config WHERE key='schema_version';"

Stages

Bundled Resources

Scripts:

• fix_transcription.py — Core CLI (dictionary, add, audit, learning)
• fix_transcript_enhanced.py — Enhanced wrapper for interactive use
• fix_transcript_timestamps.py — Timestamp normalization and repair
• generate_word_diff.py — Word-level diff HTML generation
• split_transcript_sections.py — Split transcript by marker phrases

References (load as needed):

• Safety: false_positive_guide.md (read before adding rules), database_schema.md (read before DB ops)
• Workflow: iteration_workflow.md, workflow_guide.md, example_session.md
• CLI: quick_reference.md, script_parameters.md
• Advanced: dictionary_guide.md, sql_queries.md, architecture.md, best_practices.md
• Operations: troubleshooting.md, installation_setup.md, glm_api_setup.md, team_collaboration.md

Troubleshooting

uv run scripts/fix_transcription.py --validate checks setup health. See references/troubleshooting.md for detailed resolution.

Next Step: Structure into Meeting Minutes

After correcting a transcript, if the content is from a meeting, lecture, or interview, suggest structuring it:

Transcript corrected: [N] errors fixed, saved to [output_path].

Want to turn this into structured meeting minutes with decisions and action items?

Options:
A) Yes — run /daymade-audio:meeting-minutes-taker (Recommended for meetings/lectures)
B) Export as PDF — run /daymade-docs:pdf-creator on the corrected text
C) No thanks — the corrected transcript is all I need

Architecture Reference

Technical implementation details of the transcript-fixer system.

Table of Contents

• Module Structure
• Design Principles
• SOLID Compliance
• File Length Limits
• Module Architecture
• Layer Diagram
• Correction Workflow
• Learning Cycle
• Data Flow
• SQLite Architecture (v2.0)
• Two-Layer Data Access
• Database Schema
• ACID Guarantees
• Thread Safety
• Migration from JSON
• Module Details
• fix_transcription.py
• correction_repository.py
• correction_service.py
• CLI Integration
• dictionary_processor.py
• ai_processor.py
• learning_engine.py
• diff_generator.py
• State Management
• Database-Backed State
• Thread-Safe Access
• Error Handling Strategy
• Testing Strategy
• Performance Considerations
• Security Architecture
• Extensibility Points
• Dependencies
• Deployment
• Further Reading

Module Structure

The codebase follows a modular package structure for maintainability:

scripts/
├── fix_transcription.py        # Main entry point (~70 lines)
├── core/                       # Business logic & data access
│   ├── correction_repository.py # Data access layer (466 lines)
│   ├── correction_service.py    # Business logic layer (525 lines)
│   ├── schema.sql              # SQLite database schema (216 lines)
│   ├── dictionary_processor.py # Stage 1 processor (140 lines)
│   ├── ai_processor.py        # Stage 2 processor (199 lines)
│   └── learning_engine.py     # Pattern detection (252 lines)
├── cli/                        # Command-line interface
│   ├── commands.py            # Command handlers (180 lines)
│   └── argument_parser.py     # Argument config (95 lines)
└── utils/                      # Utility functions
    ├── diff_generator.py       # Multi-format diffs (132 lines)
    ├── logging_config.py       # Logging configuration (130 lines)
    └── validation.py          # SQLite validation (105 lines)

Benefits of modular structure:

• Clear separation of concerns (business logic / CLI / utilities)
• Easy to locate and modify specific functionality
• Supports independent testing of modules
• Scales well as codebase grows
• Follows Python package best practices

Design Principles

SOLID Compliance

Every module follows SOLID principles for maintainability:

1. Single Responsibility Principle (SRP)

• Each module has exactly one reason to change
• CorrectionRepository: Database operations only
• CorrectionService: Business logic and validation only
• DictionaryProcessor: Text transformation only
• AIProcessor: API communication only
• LearningEngine: Pattern analysis only

2. Open/Closed Principle (OCP)

• Open for extension via SQL INSERT
• Closed for modification (no code changes needed)
• Add corrections via CLI or SQL without editing Python

3. Liskov Substitution Principle (LSP)

• All processors implement same interface
• Can swap implementations without breaking workflow

4. Interface Segregation Principle (ISP)

• Repository, Service, Processor, Engine are independent
• No unnecessary dependencies

5. Dependency Inversion Principle (DIP)

• Service depends on Repository interface
• CLI depends on Service interface
• Not tied to concrete implementations

File Length Limits

All files comply with code quality standards:

Module Architecture

Layer Diagram

┌─────────────────────────────────────────┐
│   CLI Layer (fix_transcription.py)     │
│   - Argument parsing                    │
│   - Command routing                     │
│   - User interaction                    │
└───────────────┬─────────────────────────┘
                │
┌───────────────▼─────────────────────────┐
│   Business Logic Layer                  │
│                                         │
│  ┌──────────────────┐  ┌──────────────┐│
│  │ Dictionary       │  │ AI           ││
│  │ Processor        │  │ Processor    ││
│  │ (Stage 1)        │  │ (Stage 2)    ││
│  └──────────────────┘  └──────────────┘│
│                                         │
│  ┌──────────────────┐  ┌──────────────┐│
│  │ Learning         │  │ Diff         ││
│  │ Engine           │  │ Generator    ││
│  │ (Pattern detect) │  │ (Stage 3)    ││
│  └──────────────────┘  └──────────────┘│
└───────────────┬─────────────────────────┘
                │
┌───────────────▼─────────────────────────┐
│   Data Access Layer (SQLite-based)      │
│                                         │
│  ┌──────────────────────────────────┐  │
│  │ CorrectionManager (Facade)       │  │
│  │ - Backward-compatible API        │  │
│  └──────────────┬───────────────────┘  │
│                 │                       │
│  ┌──────────────▼───────────────────┐  │
│  │ CorrectionService                │  │
│  │ - Business logic                 │  │
│  │ - Validation                     │  │
│  │ - Import/Export                  │  │
│  └──────────────┬───────────────────┘  │
│                 │                       │
│  ┌──────────────▼───────────────────┐  │
│  │ CorrectionRepository             │  │
│  │ - ACID transactions              │  │
│  │ - Thread-safe connections        │  │
│  │ - Audit logging                  │  │
│  └──────────────────────────────────┘  │
└───────────────┬─────────────────────────┘
                │
┌───────────────▼─────────────────────────┐
│   Storage Layer                         │
│   ~/.transcript-fixer/corrections.db    │
│   - SQLite database (ACID compliant)    │
│   - 8 normalized tables + 3 views       │
│   - Comprehensive indexes               │
│   - Foreign key constraints             │
└─────────────────────────────────────────┘

Data Flow

Correction Workflow

1. User Input
   ↓
2. fix_transcription.py (Orchestrator)
   ↓
3. CorrectionService.get_corrections()
   ← Query from ~/.transcript-fixer/corrections.db
   ↓
4. DictionaryProcessor.process()
   - Apply context rules (regex)
   - Apply dictionary replacements
   - Track changes
   ↓
5. AIProcessor.process()
   - Split into chunks
   - Call GLM-4.6 API
   - Retry with fallback on error
   - Track AI changes
   ↓
6. CorrectionService.save_history()
   → Insert into correction_history table
   ↓
7. LearningEngine.analyze_and_suggest()
   - Query correction_history table
   - Detect patterns (frequency ≥3, confidence ≥80%)
   - Generate suggestions
   → Insert into learned_suggestions table
   ↓
8. Output Files
   - {filename}_stage1.md
   - {filename}_stage2.md

Learning Cycle

Run 1: meeting1.md
   AI corrects: "巨升" → "具身"
   ↓
   INSERT INTO correction_history

Run 2: meeting2.md
   AI corrects: "巨升" → "具身"
   ↓
   INSERT INTO correction_history

Run 3: meeting3.md
   AI corrects: "巨升" → "具身"
   ↓
   INSERT INTO correction_history
   ↓
   LearningEngine queries patterns:
   - SELECT ... GROUP BY from_text, to_text
   - Frequency: 3, Confidence: 100%
   ↓
   INSERT INTO learned_suggestions (status='pending')
   ↓
   User reviews: --review-learned
   ↓
   User approves: --approve "巨升" "具身"
   ↓
   INSERT INTO corrections (source='learned')
   UPDATE learned_suggestions (status='approved')
   ↓
   Future runs query corrections table (Stage 1 - faster!)

SQLite Architecture (v2.0)

Two-Layer Data Access (Simplified)

Design Principle: No users = no backward compatibility overhead.

The system uses a clean 2-layer architecture:

┌──────────────────────────────────────────┐
│ CLI Commands (commands.py)               │
│ - User interaction                       │
│ - Command routing                        │
└──────────────┬───────────────────────────┘
               │
┌──────────────▼───────────────────────────┐
│ CorrectionService (Business Logic)       │
│ - Input validation & sanitization        │
│ - Business rules enforcement             │
│ - Import/export orchestration            │
│ - Statistics calculation                 │
│ - History tracking                       │
└──────────────┬───────────────────────────┘
               │
┌──────────────▼───────────────────────────┐
│ CorrectionRepository (Data Access)       │
│ - ACID transactions                      │
│ - Thread-safe connections                │
│ - SQL query execution                    │
│ - Audit logging                          │
└──────────────┬───────────────────────────┘
               │
┌──────────────▼───────────────────────────┐
│ SQLite Database (corrections.db)         │
│ - 8 normalized tables                    │
│ - Foreign key constraints                │
│ - Comprehensive indexes                  │
│ - 3 views for common queries             │
└───────────────────────────────────────────┘

Database Schema (schema.sql)

Core Tables:

1. corrections (main correction storage)

• Primary key: id
• Unique constraint: (from_text, domain)
• Indexes: domain, source, added_at, is_active, from_text
• Fields: confidence (0.0-1.0), usage_count, notes

2. context_rules (regex-based rules)

• Pattern + replacement with priority ordering
• Indexes: priority (DESC), is_active

3. correction_history (audit trail for runs)

• Tracks: filename, domain, timestamps, change counts
• Links to correction_changes via foreign key
• Indexes: run_timestamp, domain, success

4. correction_changes (detailed change log)

• Links to history via foreign key (CASCADE delete)
• Stores: line_number, from/to text, rule_type, context
• Indexes: history_id, rule_type

5. learned_suggestions (AI-detected patterns)

• Status: pending → approved/rejected
• Unique constraint: (from_text, to_text, domain)
• Fields: frequency, confidence, timestamps
• Indexes: status, domain, confidence, frequency

6. suggestion_examples (occurrences of patterns)

• Links to learned_suggestions via foreign key
• Stores context where pattern occurred

7. system_config (configuration storage)

• Key-value store with type safety
• Stores: API settings, thresholds, defaults

8. audit_log (comprehensive audit trail)

• Tracks all database operations
• Fields: action, entity_type, entity_id, user, success
• Indexes: timestamp, action, entity_type, success

Views (for common queries):

• active_corrections: Active corrections only
• pending_suggestions: Suggestions pending review
• correction_statistics: Statistics per domain

ACID Guarantees

Atomicity: All-or-nothing transactions

with self._transaction() as conn:
    conn.execute("INSERT ...")  # Either all succeed
    conn.execute("UPDATE ...")  # or all rollback

Consistency: Constraints enforced

• Foreign key constraints
• Check constraints (confidence 0.0-1.0, usage_count ≥ 0)
• Unique constraints

Isolation: Serializable transactions

conn.execute("BEGIN IMMEDIATE")  # Acquire write lock

Durability: Changes persisted to disk

• SQLite guarantees persistence after commit
• Backup before migrations

Thread Safety

Thread-local connections:

def _get_connection(self):
    if not hasattr(self._local, 'connection'):
        self._local.connection = sqlite3.connect(...)
    return self._local.connection

Connection pooling:

• One connection per thread
• Automatic cleanup on close
• Foreign keys enabled per connection

Clean Architecture (No Legacy)

Design Philosophy:

• Clean 2-layer architecture (Service → Repository)
• No backward compatibility overhead
• Direct API design without legacy constraints
• YAGNI principle: Build for current needs, not hypothetical migrations

Module Details

fix_transcription.py (Orchestrator)

Responsibilities:

• Parse CLI arguments
• Route commands to appropriate handlers
• Coordinate workflow between modules
• Display user feedback

Key Functions:

cmd_init()              # Initialize ~/.transcript-fixer/
cmd_add_correction()    # Add single correction
cmd_list_corrections()  # List corrections
cmd_run_correction()    # Execute correction workflow
cmd_review_learned()    # Review AI suggestions
cmd_approve()           # Approve learned correction

Design Pattern: Command pattern with function routing

correction_repository.py (Data Access Layer)

Responsibilities:

• Execute SQL queries with ACID guarantees
• Manage thread-safe database connections
• Handle transactions (commit/rollback)
• Perform audit logging
• Convert between database rows and Python objects

Key Methods:

add_correction()          # INSERT with UNIQUE handling
get_correction()          # SELECT single correction
get_all_corrections()     # SELECT with filters
get_corrections_dict()    # For backward compatibility
update_correction()       # UPDATE with transaction
delete_correction()       # Soft delete (is_active=0)
increment_usage()         # Track usage statistics
bulk_import_corrections() # Batch INSERT with conflict resolution

Transaction Management:

@contextmanager
def _transaction(self):
    conn = self._get_connection()
    try:
        conn.execute("BEGIN IMMEDIATE")
        yield conn
        conn.commit()
    except Exception:
        conn.rollback()
        raise

correction_service.py (Business Logic Layer)

Responsibilities:

• Input validation and sanitization
• Business rule enforcement
• Orchestrate repository operations
• Import/export with conflict detection
• Statistics calculation

Key Methods:

# Validation
validate_correction_text()  # Check length, control chars, NULL bytes
validate_domain_name()      # Prevent path traversal, injection
validate_confidence()       # Range check (0.0-1.0)
validate_source()          # Enum validation

# Operations
add_correction()           # Validate + repository.add
get_corrections()          # Get corrections for domain
remove_correction()        # Validate + repository.delete

# Import/Export
import_corrections()       # Pre-validate + bulk import + conflict detection
export_corrections()       # Query + format as JSON

# Analytics
get_statistics()          # Calculate metrics per domain

Validation Rules:

@dataclass
class ValidationRules:
    max_text_length: int = 1000
    min_text_length: int = 1
    max_domain_length: int = 50
    allowed_domain_pattern: str = r'^[a-zA-Z0-9_-]+$'

CLI Integration (commands.py)

Direct Service Usage:

def _get_service():
    """Get configured CorrectionService instance."""
    config_dir = Path.home() / ".transcript-fixer"
    db_path = config_dir / "corrections.db"
    repository = CorrectionRepository(db_path)
    return CorrectionService(repository)

def cmd_add_correction(args):
    service = _get_service()
    service.add_correction(args.from_text, args.to_text, args.domain)

Benefits of Direct Integration:

• No unnecessary abstraction layers
• Clear data flow: CLI → Service → Repository
• Easy to understand and debug
• Performance: One less function call per operation

dictionary_processor.py (Stage 1)

Responsibilities:

• Apply context-aware regex rules
• Apply simple dictionary replacements
• Track all changes with line numbers

Processing Order: 1. Context rules first (higher priority) 2. Dictionary replacements second

Key Methods:

process(text) -> (corrected_text, changes)
_apply_context_rules()
_apply_dictionary()
get_summary(changes)

Change Tracking:

@dataclass
class Change:
    line_number: int
    from_text: str
    to_text: str
    rule_type: str      # "dictionary" or "context_rule"
    rule_name: str

ai_processor.py (Stage 2)

Responsibilities:

• Split text into API-friendly chunks
• Call GLM-4.6 API
• Handle retries with fallback model
• Track AI-suggested changes

Key Methods:

process(text, context) -> (corrected_text, changes)
_split_into_chunks()     # Respect paragraph boundaries
_process_chunk()         # Single API call
_build_prompt()          # Construct correction prompt

Chunking Strategy:

• Max 6000 characters per chunk
• Split on paragraph boundaries (\n\n)
• If paragraph too long, split on sentences
• Preserve context across chunks

Error Handling:

• Retry with fallback model (GLM-4.5-Air)
• If both fail, use original text
• Never lose user's data

learning_engine.py (Pattern Detection)

Responsibilities:

• Analyze correction history
• Detect recurring patterns
• Calculate confidence scores
• Generate suggestions for review
• Track rejected suggestions

Algorithm:

1. Query correction_history table
2. Extract stage2 (AI) changes
3. Group by pattern (from→to)
4. Count frequency
5. Calculate confidence
6. Filter by thresholds:
   - frequency ≥ 3
   - confidence ≥ 0.8
7. Save to learned/pending_review.json

Confidence Calculation:

confidence = (
    0.5 * frequency_score +   # More occurrences = higher
    0.3 * consistency_score + # Always same correction
    0.2 * recency_score       # Recent = higher
)

Key Methods:

analyze_and_suggest()     # Main analysis pipeline
approve_suggestion()      # Move to corrections.json
reject_suggestion()       # Move to rejected.json
list_pending()           # Get all suggestions

diff_generator.py (Stage 3)

Responsibilities:

• Generate comparison reports
• Multiple output formats
• Word-level diff analysis

Output Formats: 1. Markdown summary (statistics + change list) 2. Unified diff (standard diff format) 3. HTML side-by-side (visual comparison) 4. Inline marked ([-old-] [+new+])

Not Modified: Kept original 338-line file as-is (working well)

State Management

Database-Backed State

• All state stored in ~/.transcript-fixer/corrections.db
• SQLite handles caching and transactions
• ACID guarantees prevent corruption
• Backup created before migrations

Thread-Safe Access

• Thread-local connections (one per thread)
• BEGIN IMMEDIATE for write transactions
• No global state or shared mutable data
• Each operation is independent (stateless modules)

Soft Deletes

• Records marked inactive (is_active=0) instead of DELETE
• Preserves audit trail
• Can be reactivated if needed

Error Handling Strategy

Fail Fast for User Errors

if not skill_path.exists():
    print(f"❌ Error: Skill directory not found")
    sys.exit(1)

Retry for Transient Errors

try:
    api_call(model_primary)
except Exception:
    try:
        api_call(model_fallback)
    except Exception:
        use_original_text()

Backup Before Destructive Operations

if target_file.exists():
    shutil.copy2(target_file, backup_file)
# Then overwrite target_file

Testing Strategy

Unit Testing (Recommended)

# Test dictionary processor
def test_dictionary_processor():
    corrections = {"错误": "正确"}
    processor = DictionaryProcessor(corrections, [])
    text = "这是错误的文本"
    result, changes = processor.process(text)
    assert result == "这是正确的文本"
    assert len(changes) == 1

# Test learning engine thresholds
def test_learning_thresholds():
    engine = LearningEngine(history_dir, learned_dir)
    # Create mock history with pattern appearing 3+ times
    suggestions = engine.analyze_and_suggest()
    assert len(suggestions) > 0

Integration Testing

# End-to-end test
python fix_transcription.py --init
python fix_transcription.py --add "test" "TEST"
python fix_transcription.py --input test.md --stage 3
# Verify output files exist

Performance Considerations

Bottlenecks

1. AI API calls: Slowest part (60s timeout per chunk) 2. File I/O: Negligible (JSON files are small) 3. Pattern matching: Fast (regex + dict lookups)

Optimization Strategies

1. Stage 1 First: Test dictionary corrections before expensive AI calls 2. Chunking: Process large files in parallel chunks (future enhancement) 3. Caching: Could cache API results by content hash (future enhancement)

Scalability

Current capabilities (v2.0 with SQLite):

• File size: Unlimited (chunks handle large files)
• Corrections: Tested up to 100,000 entries (with indexes)
• History: Unlimited (database handles efficiently)
• Concurrent access: Thread-safe with ACID guarantees
• Query performance: O(log n) with B-tree indexes

Performance improvements from SQLite:

• Indexed queries (domain, source, added_at)
• Views for common aggregations
• Batch imports with transactions
• Soft deletes (no data loss)

Future improvements:

• Parallel chunk processing for AI calls
• API response caching
• Full-text search for corrections

Security Architecture

Secret Management

• API keys via environment variables only
• Never hardcode credentials
• Security scanner enforces this

Backup Security

• .bak files same permissions as originals
• No encryption (user's responsibility)
• Recommendation: Use encrypted filesystems

Git Security

• .gitignore for .bak files
• Private repos recommended
• Security scan before commits

Extensibility Points

Adding New Processors

1. Create new processor class 2. Implement process(text) -> (result, changes) interface 3. Add to orchestrator workflow

Example:

class SpellCheckProcessor:
    def process(self, text):
        # Custom spell checking logic
        return corrected_text, changes

Adding New Learning Algorithms

1. Subclass LearningEngine 2. Override _calculate_confidence() 3. Adjust thresholds as needed

Adding New Export Formats

1. Add method to CorrectionManager 2. Support new file format 3. Add CLI command

Dependencies

Required

• Python 3.8+ (from __future__ import annotations)
• httpx (for API calls)

Optional

• diff command (for unified diffs)
• Git (for version control)

Development

• pytest (for testing)
• black (for formatting)
• mypy (for type checking)

Deployment

User Installation

# 1. Clone or download skill to workspace
git clone <repo> transcript-fixer
cd transcript-fixer

# 2. Install dependencies
pip install -r requirements.txt

# 3. Initialize
python scripts/fix_transcription.py --init

# 4. Set API key
export GLM_API_KEY="KEY_VALUE"

# Ready to use!

CI/CD Pipeline (Future)

# Potential GitHub Actions workflow
test:
  - Install dependencies
  - Run unit tests
  - Run integration tests
  - Check code style (black, mypy)

security:
  - Run security_scan.py
  - Check for secrets

deploy:
  - Package skill
  - Upload to skill marketplace

Further Reading

• SOLID Principles: https://en.wikipedia.org/wiki/SOLID
• API Patterns: references/glm_api_setup.md
• File Formats: references/file_formats.md
• Testing: https://docs.pytest.org/

Best Practices

Recommendations for effective use of transcript-fixer based on production experience.

Table of Contents

• Getting Started
• Build Foundation Before Scaling
• Review Learned Suggestions Regularly
• Domain Organization
• Use Domain Separation
• Domain Selection Strategy
• Cost Optimization
• Test Dictionary Changes Before AI Calls
• Approve High-Confidence Suggestions
• Team Collaboration
• Export Corrections for Version Control
• Share Corrections via Import/Merge
• Data Management
• Database Backup Strategy
• Cleanup Strategy
• Workflow Efficiency
• File Organization
• Batch Processing
• Context Rules for Edge Cases
• Quality Assurance
• Validate After Manual Changes
• Monitor Learning Quality
• Production Deployment
• Environment Variables
• Monitoring
• Performance
• Summary

Getting Started

Build Foundation Before Scaling

Start small: Begin with 5-10 manually-added corrections for the most common errors in your domain.

# Example: embodied AI domain
uv run scripts/fix_transcription.py --add "巨升智能" "具身智能" --domain embodied_ai
uv run scripts/fix_transcription.py --add "巨升" "具身" --domain embodied_ai
uv run scripts/fix_transcription.py --add "奇迹创坛" "奇绩创坛" --domain embodied_ai

Let learning discover others: After 3-5 correction runs, the learning system will suggest additional patterns automatically.

Rationale: Manual corrections provide high-quality training data. Learning amplifies your corrections exponentially.

Review Learned Suggestions Regularly

Frequency: Every 3-5 correction runs

uv run scripts/fix_transcription.py --review-learned

Why: Learned corrections move from Stage 2 (AI, expensive) to Stage 1 (dictionary, cheap/instant).

Impact:

• 10x faster processing (no API calls)
• Zero cost for repeated patterns
• Builds domain-specific vocabulary automatically

Domain Organization

Use Domain Separation

Prevent conflicts: Same phonetic error might have different corrections in different domains.

Example:

• Finance domain: "股价" (stock price) is correct
• General domain: "股价" → "框架" (framework) ASR error

# Domain-specific corrections
uv run scripts/fix_transcription.py --add "股价" "框架" --domain general
# No correction needed in finance domain - "股价" is correct there

Available domains:

• general (default) - General-purpose corrections
• embodied_ai - Robotics and embodied AI terminology
• finance - Financial terminology
• medical - Medical terminology

Custom domains: Any string matching ^[a-z0-9_]+$ (lowercase, numbers, underscore).

Domain Selection Strategy

1. Default domain for general corrections (dates, common words) 2. Specialized domains for technical terminology 3. Project domains for company/product-specific terms

# Project-specific domain
uv run scripts/fix_transcription.py --add "我司" "奇绩创坛" --domain yc_china

Cost Optimization

Test Dictionary Changes Before AI Calls

Problem: AI calls (Stage 2) consume API quota and time.

Solution: Test dictionary changes with Stage 1 first.

# 1. Add new corrections
uv run scripts/fix_transcription.py --add "新错误" "正确词" --domain general

# 2. Test on small sample (Stage 1 only)
uv run scripts/fix_transcription.py --input sample.md --stage 1

# 3. Review output
less sample_stage1.md

# 4. If satisfied, run full pipeline on large files
uv run scripts/fix_transcription.py --input large_file.md --stage 3

Savings: Avoid wasting API quota on files with dictionary-only corrections.

Approve High-Confidence Suggestions

Check suggestions regularly:

uv run scripts/fix_transcription.py --review-learned

Approve suggestions with:

• Frequency ≥ 5
• Confidence ≥ 0.9
• Pattern makes semantic sense

Impact: Each approved suggestion saves future API calls.

Team Collaboration

Export Corrections for Version Control

Don't commit .db files to Git:

• Binary format causes merge conflicts
• Database grows over time (bloats repository)
• Not human-reviewable

Do commit JSON exports:

# Export domain dictionaries
uv run scripts/fix_transcription.py --export general_$(date +%Y%m%d).json --domain general
uv run scripts/fix_transcription.py --export embodied_ai_$(date +%Y%m%d).json --domain embodied_ai

# .gitignore
*.db
*.db-journal
*.bak

# Commit exports
git add *_corrections.json
git commit -m "Update correction dictionaries"

Share Corrections via Import/Merge

Always use `--merge` flag to combine corrections:

# Pull latest from team
git pull origin main

# Import new corrections (merge mode)
uv run scripts/fix_transcription.py --import general_20250128.json --merge
uv run scripts/fix_transcription.py --import embodied_ai_20250128.json --merge

Merge behavior:

• New corrections: inserted
• Existing corrections with higher confidence: updated
• Existing corrections with lower confidence: skipped
• Preserves local customizations

See team_collaboration.md for Git workflows and conflict handling.

Data Management

Database Backup Strategy

Automatic backups: Database creates timestamped backups before migrations:

~/.transcript-fixer/
├── corrections.db
├── corrections.20250128_140532.bak
└── corrections.20250127_093021.bak

Manual backups before bulk changes:

cp ~/.transcript-fixer/corrections.db ~/backups/corrections_$(date +%Y%m%d).db

Or use SQLite backup:

sqlite3 ~/.transcript-fixer/corrections.db ".backup ~/backups/corrections.db"

Cleanup Strategy

History retention: Keep recent history, archive old entries:

# Archive history older than 90 days
sqlite3 ~/.transcript-fixer/corrections.db "
DELETE FROM correction_history
WHERE run_timestamp < datetime('now', '-90 days');
"

# Reclaim space
sqlite3 ~/.transcript-fixer/corrections.db "VACUUM;"

Suggestion cleanup: Reject low-confidence suggestions periodically:

# Reject suggestions with frequency < 3
sqlite3 ~/.transcript-fixer/corrections.db "
UPDATE learned_suggestions
SET status = 'rejected'
WHERE frequency < 3 AND confidence < 0.7;
"

Workflow Efficiency

File Organization

Use consistent naming:

meeting_20250128.md           # Original transcript
meeting_20250128_stage1.md    # Dictionary corrections
meeting_20250128_stage2.md    # Final corrected version

Generate diff reports for review:

uv run scripts/diff_generator.py \
  meeting_20250128.md \
  meeting_20250128_stage1.md \
  meeting_20250128_stage2.md

Output formats:

• Markdown report (what changed, statistics)
• Unified diff (git-style)
• HTML side-by-side (visual review)
• Inline markers (for direct editing)

Batch Processing

Process similar files together to amplify learning:

# Day 1: Process 5 similar meetings
for file in meeting_*.md; do
  uv run scripts/fix_transcription.py --input "$file" --stage 3 --domain embodied_ai
done

# Day 2: Review learned patterns
uv run scripts/fix_transcription.py --review-learned

# Approve good suggestions
uv run scripts/fix_transcription.py --approve "常见错误1" "正确词1"
uv run scripts/fix_transcription.py --approve "常见错误2" "正确词2"

# Day 3: Future files benefit from dictionary corrections

Context Rules for Edge Cases

Use regex context rules for:

• Positional dependencies (e.g., "的" vs "地" before verbs)
• Multi-word patterns
• Traditional vs simplified Chinese

Example:

sqlite3 ~/.transcript-fixer/corrections.db

# "的" before verb → "地"
INSERT INTO context_rules (pattern, replacement, description, priority)
VALUES ('近距离的去看', '近距离地去看', '的→地 before verb', 10);

# Preserve correct usage
INSERT INTO context_rules (pattern, replacement, description, priority)
VALUES ('近距离搏杀', '近距离搏杀', '的 is correct here (noun modifier)', 5);

Priority: Higher numbers run first (use for exceptions).

Quality Assurance

Validate After Manual Changes

After direct SQL edits:

uv run scripts/fix_transcription.py --validate

After imports:

# Check statistics
uv run scripts/fix_transcription.py --list --domain general | head -20

# Verify specific corrections
sqlite3 ~/.transcript-fixer/corrections.db "
SELECT from_text, to_text, source, confidence
FROM active_corrections
WHERE domain = 'general'
ORDER BY added_at DESC
LIMIT 10;
"

Monitor Learning Quality

Check suggestion confidence distribution:

sqlite3 ~/.transcript-fixer/corrections.db "
SELECT
  CASE
    WHEN confidence >= 0.9 THEN 'high (>=0.9)'
    WHEN confidence >= 0.8 THEN 'medium (0.8-0.9)'
    ELSE 'low (<0.8)'
  END as confidence_level,
  COUNT(*) as count
FROM learned_suggestions
WHERE status = 'pending'
GROUP BY confidence_level;
"

Review examples for low-confidence suggestions:

sqlite3 ~/.transcript-fixer/corrections.db "
SELECT s.from_text, s.to_text, s.confidence, e.context
FROM learned_suggestions s
JOIN suggestion_examples e ON s.id = e.suggestion_id
WHERE s.confidence < 0.8 AND s.status = 'pending';
"

Production Deployment

Environment Variables

Set permanently in production:

# Add to /etc/environment or systemd service
GLM_API_KEY=your-production-key

Monitoring

Track usage statistics:

# Corrections by source
sqlite3 ~/.transcript-fixer/corrections.db "
SELECT source, COUNT(*) as count, SUM(usage_count) as total_usage
FROM corrections
WHERE is_active = 1
GROUP BY source;
"

# Success rate
sqlite3 ~/.transcript-fixer/corrections.db "
SELECT
  COUNT(*) as total_runs,
  SUM(CASE WHEN success = 1 THEN 1 ELSE 0 END) as successful,
  ROUND(100.0 * SUM(CASE WHEN success = 1 THEN 1 ELSE 0 END) / COUNT(*), 2) as success_rate
FROM correction_history;
"

Performance

Database optimization:

# Rebuild indexes periodically
sqlite3 ~/.transcript-fixer/corrections.db "REINDEX;"

# Analyze query patterns
sqlite3 ~/.transcript-fixer/corrections.db "ANALYZE;"

# Vacuum to reclaim space
sqlite3 ~/.transcript-fixer/corrections.db "VACUUM;"

Summary

Key principles: 1. Start small, let learning amplify 2. Use domain separation for quality 3. Test dictionary changes before AI calls 4. Export to JSON for version control 5. Review and approve learned suggestions 6. Validate after manual changes 7. Monitor learning quality 8. Backup before bulk operations

ROI timeline:

• Week 1: Build foundation (10-20 manual corrections)
• Week 2-3: Learning kicks in (20-50 suggestions)
• Month 2+: Mature vocabulary (80%+ dictionary coverage, minimal AI calls)

Database Schema Reference

MUST read this before any database operations.

Database location: ~/.transcript-fixer/corrections.db

Core Tables

corrections

Main storage for correction mappings.

Constraint: UNIQUE(from_text, domain)

context_rules

Regex-based context-aware correction rules.

learned_suggestions

AI-learned patterns pending user review.

Constraint: UNIQUE(from_text, to_text, domain)

correction_history

Audit log for all correction runs.

correction_changes

Detailed changes made in each correction run.

system_config

Key-value configuration store.

Default configs:

• schema_version: '2.0'
• api_model: 'GLM-4.6'
• learning_frequency_threshold: 3
• learning_confidence_threshold: 0.8
• history_retention_days: 90

audit_log

Comprehensive operations trail.

Views

active_corrections

Active corrections only, ordered by domain and from_text.

SELECT * FROM active_corrections;

pending_suggestions

Suggestions awaiting review, with example count.

SELECT * FROM pending_suggestions WHERE confidence > 0.8;

correction_statistics

Statistics per domain.

SELECT * FROM correction_statistics;

Common Queries

-- List all active corrections
SELECT from_text, to_text, domain FROM active_corrections;

-- Check pending high-confidence suggestions
SELECT * FROM pending_suggestions WHERE confidence > 0.8 ORDER BY frequency DESC;

-- Domain statistics
SELECT domain, total_corrections, total_usage FROM correction_statistics;

-- Recent correction history
SELECT filename, stage1_changes, stage2_changes, run_timestamp
FROM correction_history
ORDER BY run_timestamp DESC LIMIT 10;

-- Add new correction (use CLI instead for safety)
INSERT INTO corrections (from_text, to_text, domain, source, confidence, added_by)
VALUES ('错误词', '正确词', 'general', 'manual', 1.0, 'user');

-- Disable a correction
UPDATE corrections SET is_active = 0 WHERE id = ?;

Schema Version

Check current version:

SELECT value FROM system_config WHERE key = 'schema_version';

For complete schema including indexes and constraints, see scripts/core/schema.sql.

纠错词典配置指南

词典结构

纠错词典位于 fix_transcription.py 中,包含两部分:

1. 上下文规则 (CONTEXT_RULES)

用于需要结合上下文判断的替换:

CONTEXT_RULES = [
    {
        "pattern": r"正则表达式",
        "replacement": "替换文本",
        "description": "规则说明"
    }
]

示例:

{
    "pattern": r"近距离的去看",
    "replacement": "近距离地去看",
    "description": "修正'的'为'地'"
}

2. 通用词典 (CORRECTIONS_DICT)

用于直接字符串替换:

CORRECTIONS_DICT = {
    "错误词汇": "正确词汇",
}

示例:

{
    "巨升智能": "具身智能",
    "奇迹创坛": "奇绩创坛",
    "矩阵公司": "初创公司",
}

添加自定义规则

步骤1: 识别错误模式

从修复报告中识别重复出现的错误。

步骤2: 选择规则类型

• 简单替换 → 使用 CORRECTIONS_DICT
• 需要上下文 → 使用 CONTEXT_RULES

步骤3: 添加到词典

编辑 scripts/fix_transcription.py:

CORRECTIONS_DICT = {
    # 现有规则...
    "你的错误": "正确词汇",  # 添加新规则
}

步骤4: 测试

运行修复脚本测试新规则。

常见错误类型

同音字错误

"股价": "框架",
"三观": "三关",

专业术语

"巨升智能": "具身智能",
"近距离": "具身",  # 某些上下文中

公司名称

"奇迹创坛": "奇绩创坛",

优先级

1. 先应用 CONTEXT_RULES (精确匹配) 2. 再应用 CORRECTIONS_DICT (全局替换)

Example Session

Input transcript (meeting.md)

今天我们讨论了巨升智能的最新进展。
股价系统需要优化，目前性能不够好。

After Stage 1 (meeting_stage1.md)

今天我们讨论了具身智能的最新进展。  ← "巨升"→"具身" corrected
股价系统需要优化，目前性能不够好。  ← Unchanged (not in dictionary)

After Stage 2 (meeting_stage2.md)

今天我们讨论了具身智能的最新进展。
框架系统需要优化，目前性能不够好。  ← "股价"→"框架" corrected by AI

Learned pattern detected

✓ Detected: "股价" → "框架" (confidence: 85%, count: 1)
  Run --review-learned after 2 more occurrences to approve

False Positive Prevention Guide

Dictionary-based corrections are powerful but dangerous. Adding the wrong rule silently corrupts every future transcript. The --add command runs safety checks automatically, but you must understand the risks.

What is safe to add

• ASR-specific gibberish: "巨升智能" -> "具身智能" (no real word sounds like "巨升智能")
• Long compound errors: "语音是别" -> "语音识别" (4+ chars, unlikely to collide)
• English transliteration errors: "japanese 3 pro" -> "Gemini 3 Pro"

What is NEVER safe to add

• Common Chinese words: "仿佛", "正面", "犹豫", "传说", "增加", "教育" -- these appear correctly in normal text. Replacing them corrupts transcripts from better ASR models.
• Words <=2 characters: Almost any 2-char Chinese string is a valid word or part of one. "线数" inside "产线数据" becomes "产线束据".
• Both sides are real words: "仿佛->反复", "犹豫->抑郁" -- both forms are valid Chinese. The "error" is only an error for one specific ASR model.

When in doubt, use a context rule instead

Context rules use regex patterns that match only in specific surroundings, avoiding false positives:

# Instead of: --add "线数" "线束"
# Use a context rule in the database:
sqlite3 ~/.transcript-fixer/corrections.db "INSERT INTO context_rules (pattern, replacement, description, priority) VALUES ('(?<!产)线数(?!据)', '线束', 'ASR: 线数->线束 (not inside 产线数据)', 10);"

Auditing the dictionary

Run --audit periodically to scan all rules for false positive risks:

uv run scripts/fix_transcription.py --audit
uv run scripts/fix_transcription.py --audit --domain manufacturing

Forcing a risky addition

If you understand the risks and still want to add a flagged rule:

uv run scripts/fix_transcription.py --add "仿佛" "反复" --domain general --force

Storage Format Reference

This document describes the SQLite database format used by transcript-fixer v2.0.

Table of Contents

• Database Location
• Database Schema
• Core Tables
• Views
• Querying the Database
• Using Python API
• Using SQLite CLI
• Import/Export
• Export to JSON
• Import from JSON
• Backup Strategy
• Automatic Backups
• Manual Backups
• Version Control
• Best Practices
• Troubleshooting
• Database Locked
• Corrupted Database
• Missing Tables

Database Location

Path: ~/.transcript-fixer/corrections.db

Type: SQLite 3 database with ACID guarantees

Database Schema

Core Tables

corrections

Main correction dictionary storage.

Unique Constraint: (from_text, domain)

Indexes: domain, source, added_at, is_active, from_text

context_rules

Regex-based context-aware correction rules.

Indexes: priority (DESC), is_active

correction_history

Audit log for all correction runs.

Indexes: run_timestamp (DESC), domain, success

correction_changes

Detailed changes made in each run.

Foreign Key: history_id → correction_history.id (CASCADE DELETE)

Indexes: history_id, rule_type

learned_suggestions

AI-detected patterns pending review.

Unique Constraint: (from_text, to_text, domain)

Indexes: status, domain, confidence (DESC), frequency (DESC)

suggestion_examples

Example occurrences of learned patterns.

Foreign Key: suggestion_id → learned_suggestions.id (CASCADE DELETE)

Index: suggestion_id

system_config

System configuration key-value store.

Default Values:

• schema_version: "2.0"
• api_provider: "GLM"
• api_model: "GLM-4.6"
• default_domain: "general"
• auto_learn_enabled: "true"
• learning_frequency_threshold: "3"
• learning_confidence_threshold: "0.8"

audit_log

Comprehensive audit trail for all operations.

Indexes: timestamp (DESC), action, entity_type, success

Views

active_corrections

Quick access to active corrections.

SELECT id, from_text, to_text, domain, source, confidence, usage_count, last_used, added_at
FROM corrections
WHERE is_active = 1
ORDER BY domain, from_text;

pending_suggestions

Suggestions pending review with example count.

SELECT s.id, s.from_text, s.to_text, s.domain, s.frequency, s.confidence,
       s.first_seen, s.last_seen, COUNT(e.id) as example_count
FROM learned_suggestions s
LEFT JOIN suggestion_examples e ON s.id = e.suggestion_id
WHERE s.status = 'pending'
GROUP BY s.id
ORDER BY s.confidence DESC, s.frequency DESC;

correction_statistics

Statistics per domain.

SELECT domain,
       COUNT(*) as total_corrections,
       COUNT(CASE WHEN source = 'manual' THEN 1 END) as manual_count,
       COUNT(CASE WHEN source = 'learned' THEN 1 END) as learned_count,
       COUNT(CASE WHEN source = 'imported' THEN 1 END) as imported_count,
       SUM(usage_count) as total_usage,
       MAX(added_at) as last_updated
FROM corrections
WHERE is_active = 1
GROUP BY domain;

Querying the Database

Using Python API

from pathlib import Path
from core import CorrectionRepository, CorrectionService

# Initialize
db_path = Path.home() / ".transcript-fixer" / "corrections.db"
repository = CorrectionRepository(db_path)
service = CorrectionService(repository)

# Add correction
service.add_correction("错误", "正确", domain="general")

# Get corrections
corrections = service.get_corrections(domain="general")

# Get statistics
stats = service.get_statistics(domain="general")
print(f"Total: {stats['total_corrections']}")

# Close
service.close()

Using SQLite CLI

# Open database
sqlite3 ~/.transcript-fixer/corrections.db

# View active corrections
SELECT from_text, to_text, domain FROM active_corrections;

# View statistics
SELECT * FROM correction_statistics;

# View pending suggestions
SELECT * FROM pending_suggestions;

# Check schema version
SELECT value FROM system_config WHERE key = 'schema_version';

Import/Export

Export to JSON

service = _get_service()
corrections = service.export_corrections(domain="general")

# Write to file
import json
with open("export.json", "w", encoding="utf-8") as f:
    json.dump({
        "version": "2.0",
        "domain": "general",
        "corrections": corrections
    }, f, ensure_ascii=False, indent=2)

Import from JSON

import json

with open("import.json", "r", encoding="utf-8") as f:
    data = json.load(f)

service = _get_service()
inserted, updated, skipped = service.import_corrections(
    corrections=data["corrections"],
    domain=data.get("domain", "general"),
    merge=True,
    validate_all=True
)

print(f"Imported: {inserted} new, {updated} updated, {skipped} skipped")

Backup Strategy

Automatic Backups

The system maintains database integrity through SQLite's ACID guarantees and automatic journaling.

Manual Backups

# Backup database
cp ~/.transcript-fixer/corrections.db ~/backups/corrections_$(date +%Y%m%d).db

# Or use SQLite backup
sqlite3 ~/.transcript-fixer/corrections.db ".backup ~/backups/corrections.db"

Version Control

Recommended: Use Git for configuration and export files, but NOT for the database:

# .gitignore
*.db
*.db-journal
*.bak

Instead, export corrections periodically:

python scripts/fix_transcription.py --export-json corrections_backup.json
git add corrections_backup.json
git commit -m "Backup corrections"

Best Practices

1. Regular Exports: Export to JSON weekly for team sharing 2. Database Backups: Backup .db file before major changes 3. Use Transactions: All modifications use ACID transactions automatically 4. Soft Deletes: Records are marked inactive, not deleted (preserves audit trail) 5. Validate: Run --validate after manual database changes 6. Statistics: Check usage patterns via correction_statistics view 7. Cleanup: Old history can be archived (query by run_timestamp)

Troubleshooting

Database Locked

# Check for lingering connections
lsof ~/.transcript-fixer/corrections.db

# If needed, backup and recreate
cp corrections.db corrections_backup.db
sqlite3 corrections.db "VACUUM;"

Corrupted Database

# Check integrity
sqlite3 corrections.db "PRAGMA integrity_check;"

# Recover if possible
sqlite3 corrections.db ".recover" | sqlite3 corrections_new.db

Missing Tables

# Reinitialize schema (safe, uses IF NOT EXISTS)
python -c "from core import CorrectionRepository; from pathlib import Path; CorrectionRepository(Path.home() / '.transcript-fixer' / 'corrections.db')"

GLM API 配置指南

API配置

设置环境变量

在运行脚本前,设置GLM API密钥环境变量:

# Linux/macOS
export GLM_API_KEY="your-api-key-here"

# Windows (PowerShell)
$env:GLM_API_KEY="your-api-key-here"

# Windows (CMD)
set GLM_API_KEY=your-api-key-here

永久设置 (推荐):

# Linux/macOS: 添加到 ~/.bashrc 或 ~/.zshrc
echo 'export GLM_API_KEY="your-api-key-here"' >> ~/.bashrc
source ~/.bashrc

# Windows: 在系统环境变量中设置

脚本配置

脚本会自动从环境变量读取API密钥:

# 脚本会检查环境变量
if "GLM_API_KEY" not in os.environ:
    raise ValueError("请设置 GLM_API_KEY 环境变量")

os.environ["ANTHROPIC_BASE_URL"] = "https://open.bigmodel.cn/api/anthropic"
os.environ["ANTHROPIC_API_KEY"] = os.environ["GLM_API_KEY"]

# 模型配置
GLM_MODEL = "GLM-4.6"  # 主力模型
GLM_MODEL_FAST = "GLM-4.5-Air"  # 快速模型(备用)

支持的模型

注意: 模型名称大小写不敏感。

API认证

智谱GLM使用Anthropic兼容API:

headers = {
    "anthropic-version": "2023-06-01",
    "Authorization": f"Bearer {api_key}",
    "content-type": "application/json"
}

关键点:

• 使用 Authorization: Bearer 头
• 不要使用 x-api-key 头

API调用示例

def call_glm_api(prompt: str) -> str:
    url = "https://open.bigmodel.cn/api/anthropic/v1/messages"
    headers = {
        "anthropic-version": "2023-06-01",
        "Authorization": f"Bearer {os.environ.get('ANTHROPIC_API_KEY')}",
        "content-type": "application/json"
    }

    data = {
        "model": "GLM-4.6",
        "max_tokens": 8000,
        "temperature": 0.3,
        "messages": [{"role": "user", "content": prompt}]
    }

    response = httpx.post(url, headers=headers, json=data, timeout=60.0)
    return response.json()["content"][0]["text"]

获取API密钥

1. 访问 https://open.bigmodel.cn/ 2. 注册/登录账号 3. 进入API管理页面 4. 创建新的API密钥 5. 复制密钥到配置中

费用

参考智谱AI官方定价:

• GLM-4.6: 按token计费
• GLM-4.5-Air: 更便宜的选择

故障排查

401错误

• 检查API密钥是否正确
• 确认使用 Authorization: Bearer 头

超时错误

• 增加timeout参数
• 考虑使用GLM-4.5-Air快速模型

Setup Guide

Complete installation and configuration guide for transcript-fixer.

Table of Contents

• Installation
• API Configuration
• Environment Setup
• Next Steps

Installation

Dependencies

Install required dependencies using uv:

uv pip install -r requirements.txt

Or sync the project environment:

uv sync

Required packages:

• anthropic - For Claude API integration (future)
• requests - For GLM API calls
• difflib - Standard library for diff generation

Database Initialization

Initialize the SQLite database (first time only):

uv run scripts/fix_transcription.py --init

This creates ~/.transcript-fixer/corrections.db with the complete schema:

• 8 tables (corrections, context_rules, history, suggestions, etc.)
• 3 views (active_corrections, pending_suggestions, statistics)
• ACID transactions enabled
• Automatic backups before migrations

See file_formats.md for complete database schema.

API Configuration

GLM API Key (Required for Stage 2)

Stage 2 AI corrections require a GLM API key.

1. Obtain API key: Visit https://open.bigmodel.cn/ 2. Register for an account 3. Generate an API key from the dashboard 4. Set environment variable:

export GLM_API_KEY="your-api-key-here"

Persistence: Add to shell profile for permanent access:

# For bash
echo 'export GLM_API_KEY="your-key"' >> ~/.bashrc
source ~/.bashrc

# For zsh
echo 'export GLM_API_KEY="your-key"' >> ~/.zshrc
source ~/.zshrc

Verify Configuration

Run validation to check setup:

uv run scripts/fix_transcription.py --validate

Expected output:

🔍 Validating transcript-fixer configuration...

✅ Configuration directory exists: ~/.transcript-fixer
✅ Database valid: 0 corrections
✅ All 8 tables present
✅ GLM_API_KEY is set

============================================================
✅ All checks passed! Configuration is valid.
============================================================

Environment Setup

Python Environment

Required: Python 3.8+

Recommended: Use uv for all Python operations:

# Never use system python directly
uv run scripts/fix_transcription.py  # ✅ Correct

# Don't use system python
python scripts/fix_transcription.py  # ❌ Wrong

Directory Structure

After initialization, the directory structure is:

~/.transcript-fixer/
├── corrections.db              # SQLite database
├── corrections.YYYYMMDD.bak   # Automatic backups
└── (migration artifacts)

Important: The .db file should NOT be committed to Git. Export corrections to JSON for version control instead.

Next Steps

After setup: 1. Add initial corrections (5-10 terms) 2. Run first correction on a test file 3. Review learned suggestions after 3-5 runs 4. Build domain-specific dictionaries

See workflow_guide.md for detailed usage instructions.

Dictionary Iteration Workflow

The core value of transcript-fixer is building a personalized correction dictionary that improves over time.

The Core Loop

┌─────────────────────────────────────────────────┐
│  1. Fix transcript (manual or Stage 3)          │
│                    ↓                            │
│  2. Identify new ASR errors during fixing       │
│                    ↓                            │
│  3. IMMEDIATELY save to dictionary              │
│                    ↓                            │
│  4. Next time: Stage 1 auto-corrects these      │
└─────────────────────────────────────────────────┘

Key principle: Every stable, reusable ASR correction you make should be saved to the dictionary. This transforms one-time work into permanent value without polluting the database.

Workflow Checklist

Copy this checklist when correcting transcripts:

Correction Progress:
- [ ] Run correction: --input file.md --stage 3
- [ ] Review output file for remaining ASR errors
- [ ] Fix errors manually with Edit tool
- [ ] Save EACH correction to dictionary with --add
- [ ] Verify with --list that corrections were saved
- [ ] Next time: Stage 1 handles these automatically

Save Corrections Immediately

After fixing any transcript, save stable corrections:

# Single correction
uv run scripts/fix_transcription.py --add "错误词" "正确词" --domain general

# Multiple corrections - run command for each
uv run scripts/fix_transcription.py --add "片片总" "翩翩总" --domain general
uv run scripts/fix_transcription.py --add "姐弟" "结业" --domain general
uv run scripts/fix_transcription.py --add "自杀性" "自嗨性" --domain general
uv run scripts/fix_transcription.py --add "被看" "被砍" --domain general
uv run scripts/fix_transcription.py --add "单反过" "单访过" --domain general

Verify Dictionary

Always verify corrections were saved:

# List all corrections in current domain
uv run scripts/fix_transcription.py --list

# Direct database query
sqlite3 ~/.transcript-fixer/corrections.db \
  "SELECT from_text, to_text, domain FROM active_corrections ORDER BY added_at DESC LIMIT 10;"

Domain Selection

Choose the right domain for corrections:

# Domain-specific correction
uv run scripts/fix_transcription.py --add "股价系统" "框架系统" --domain embodied_ai
uv run scripts/fix_transcription.py --add "片片总" "翩翩总" --domain 火星加速器

Common ASR Error Patterns

Build your dictionary with these common patterns:

When GLM API Fails

If you see [CLAUDE_FALLBACK] output, the GLM API is unavailable.

Steps: 1. Claude Code should analyze the text directly for ASR errors 2. Fix using Edit tool 3. MUST save corrections to dictionary - this is critical 4. Dictionary corrections work even without AI

Auto-Learning Feature

After running Stage 3 multiple times:

# Check learned patterns
uv run scripts/fix_transcription.py --review-learned

# Approve high-confidence patterns
uv run scripts/fix_transcription.py --approve "错误词" "正确词"

Patterns appearing ≥3 times at ≥80% confidence are suggested for review.

Best Practices

1. Save immediately: Don't batch corrections - save each one right after fixing 2. Be specific: Use exact phrases, not partial words 3. Use domains: Organize corrections by topic for better precision 4. Verify: Always run --list to confirm saves 5. Review suggestions: Periodically check --review-learned for auto-detected patterns

What NOT to Save to Dictionary

Do not save these as reusable dictionary entries:

• Full-sentence deletions
• One-off section headers or meeting-specific boilerplate
• Context-only disambiguations such as cloud -> Claude when cloud can also be legitimate
• File-local cleanup after section splitting or timestamp rebasing

Quick Reference

Storage: transcript-fixer uses SQLite database for corrections storage.

Database location: ~/.transcript-fixer/corrections.db

Quick Start Examples

Adding Corrections via CLI

# Add a simple correction
uv run scripts/fix_transcription.py --add "巨升智能" "具身智能" --domain embodied_ai

# Add corrections for specific domain
uv run scripts/fix_transcription.py --add "奇迹创坛" "奇绩创坛" --domain general
uv run scripts/fix_transcription.py --add "矩阵公司" "初创公司" --domain general

Adding Corrections via SQL

sqlite3 ~/.transcript-fixer/corrections.db

# Insert corrections
INSERT INTO corrections (from_text, to_text, domain, source)
VALUES ('巨升智能', '具身智能', 'embodied_ai', 'manual');

INSERT INTO corrections (from_text, to_text, domain, source)
VALUES ('巨升', '具身', 'embodied_ai', 'manual');

INSERT INTO corrections (from_text, to_text, domain, source)
VALUES ('奇迹创坛', '奇绩创坛', 'general', 'manual');

# Exit
.quit

Adding Context Rules via SQL

Context rules use regex patterns for context-aware corrections:

sqlite3 ~/.transcript-fixer/corrections.db

# Add context-aware rules
INSERT INTO context_rules (pattern, replacement, description, priority)
VALUES ('巨升方向', '具身方向', '巨升→具身', 10);

INSERT INTO context_rules (pattern, replacement, description, priority)
VALUES ('巨升现在', '具身现在', '巨升→具身', 10);

INSERT INTO context_rules (pattern, replacement, description, priority)
VALUES ('近距离的去看', '近距离地去看', '的→地 副词修饰', 5);

# Exit
.quit

Adding Corrections via Python API

Save as add_corrections.py and run with uv run add_corrections.py:

#!/usr/bin/env -S uv run
from pathlib import Path
from core import CorrectionRepository, CorrectionService

# Initialize service
db_path = Path.home() / ".transcript-fixer" / "corrections.db"
repository = CorrectionRepository(db_path)
service = CorrectionService(repository)

# Add corrections
corrections = [
    ("巨升智能", "具身智能", "embodied_ai"),
    ("巨升", "具身", "embodied_ai"),
    ("奇迹创坛", "奇绩创坛", "general"),
    ("火星营", "火星营", "general"),
    ("矩阵公司", "初创公司", "general"),
    ("股价", "框架", "general"),
    ("三观", "三关", "general"),
]

for from_text, to_text, domain in corrections:
    service.add_correction(from_text, to_text, domain)
    print(f"✅ Added: '{from_text}' → '{to_text}' (domain: {domain})")

# Close connection
service.close()

Bulk Import Example

Use the provided bulk import script for importing multiple corrections:

uv run scripts/examples/bulk_import.py

Querying the Database

View Active Corrections

sqlite3 ~/.transcript-fixer/corrections.db "SELECT from_text, to_text, domain FROM active_corrections;"

View Statistics

sqlite3 ~/.transcript-fixer/corrections.db "SELECT * FROM correction_statistics;"

View Context Rules

sqlite3 ~/.transcript-fixer/corrections.db "SELECT pattern, replacement, priority FROM context_rules WHERE is_active = 1 ORDER BY priority DESC;"

See Also

• references/file_formats.md - Complete database schema documentation
• references/script_parameters.md - CLI command reference
• SKILL.md - Main user documentation

Script Parameters Reference

Detailed command-line parameters and usage examples for transcript-fixer Python scripts.

Table of Contents

• fix_transcription.py - Main correction pipeline
• Setup Commands
• Correction Management
• Correction Workflow
• Learning Commands
• fix_transcript_timestamps.py - Normalize/repair speaker timestamps
• split_transcript_sections.py - Split transcript into named sections
• diff_generator.py - Generate comparison reports
• Common Workflows
• Exit Codes
• Environment Variables

---

fix_transcription.py

Main correction pipeline script supporting three processing stages.

Syntax

python scripts/fix_transcription.py --input <file> --stage <1|2|3> [--output <dir>]

Parameters

• --input, -i (required): Input Markdown file path
• --stage, -s (optional): Stage to execute (default: 3)
• 1 = Dictionary corrections only
• 2 = AI corrections only (requires Stage 1 output file)
• 3 = Both stages sequentially
• --output, -o (optional): Output directory (defaults to input file directory)

Usage Examples

Run dictionary corrections only:

python scripts/fix_transcription.py --input meeting.md --stage 1

Output: meeting_阶段1_词典修复.md

Run AI corrections only:

python scripts/fix_transcription.py --input meeting_阶段1_词典修复.md --stage 2

Output: meeting_阶段2_AI修复.md

Note: Requires Stage 1 output file as input.

Run complete pipeline:

python scripts/fix_transcription.py --input meeting.md --stage 3

Outputs:

• meeting_阶段1_词典修复.md
• meeting_阶段2_AI修复.md

Custom output directory:

python scripts/fix_transcription.py --input meeting.md --stage 3 --output ./corrections

Exit Codes

• 0 - Success
• 1 - Missing required parameters or file not found
• 2 - GLM_API_KEY environment variable not set (Stage 2 or 3 only)
• 3 - API request failed

fix_transcript_timestamps.py

Normalize speaker timestamp lines such as 说话人A 00:21 or Speaker 7 01:31:10.

Syntax

python scripts/fix_transcript_timestamps.py <file> [--output FILE | --in-place | --check]

Key Parameters

• --format {hhmmss,preserve}: output timestamp style
• --rebase-to-zero: reset the first detected speaker timestamp to 00:00:00
• --rollover-backjump-seconds: threshold for treating 59:58 -> 00:05 as a new hour
• --jitter-seconds: tolerated small backward jitter before flagging anomaly

Usage Examples

# Normalize mixed MM:SS / HH:MM:SS
python scripts/fix_transcript_timestamps.py meeting.txt --in-place

# Rebase a split transcript so it starts at 00:00:00
python scripts/fix_transcript_timestamps.py workshop-class.txt --in-place --rebase-to-zero

# Only inspect anomalies, do not write
python scripts/fix_transcript_timestamps.py meeting.txt --check

split_transcript_sections.py

Split a transcript into named sections using marker phrases. Useful for workshop transcripts that include setup chat, class, and debrief in one file.

Syntax

python scripts/split_transcript_sections.py <file> \
  --first-section-name <name> \
  --section "Name::Marker" \
  --section "Name::Marker"

Usage Example

python scripts/split_transcript_sections.py workshop.txt \
  --first-section-name "课前聊天" \
  --section "正式上课::好，无缝切换嘛。对。那个曹总连上了吗？那个网页。" \
  --section "课后复盘::我们复盘一下。" \
  --rebase-to-zero

generate_diff_report.py

Multi-format diff report generator for comparing correction stages.

Syntax

python scripts/generate_diff_report.py --original <file> --stage1 <file> --stage2 <file> [--output-dir <dir>]

Parameters

• --original (required): Original transcript file path
• --stage1 (required): Stage 1 correction output file path
• --stage2 (required): Stage 2 correction output file path
• --output-dir (optional): Output directory for diff reports (defaults to original file directory)

Usage Examples

Basic usage:

python scripts/generate_diff_report.py \
    --original "meeting.md" \
    --stage1 "meeting_阶段1_词典修复.md" \
    --stage2 "meeting_阶段2_AI修复.md"

Custom output directory:

python scripts/generate_diff_report.py \
    --original "meeting.md" \
    --stage1 "meeting_阶段1_词典修复.md" \
    --stage2 "meeting_阶段2_AI修复.md" \
    --output-dir "./reports"

Output Files

The script generates four comparison formats:

1. Markdown summary (*_对比报告.md)

• High-level statistics and change summary
• Word count changes per stage
• Common error patterns identified

2. Unified diff (*_unified.diff)

• Traditional Unix diff format
• Suitable for command-line review or version control

3. HTML side-by-side (*_对比.html)

• Visual side-by-side comparison
• Color-coded additions/deletions
• Recommended for human review

4. Inline marked (*_行内对比.txt)

• Single-column format with inline change markers
• Useful for quick text editor review

Exit Codes

• 0 - Success
• 1 - Missing required parameters or file not found
• 2 - File format error (non-Markdown input)

Common Workflows

Testing Dictionary Changes

Test dictionary updates before running expensive AI corrections:

# 1. Update CORRECTIONS_DICT in scripts/fix_transcription.py
# 2. Run Stage 1 only
python scripts/fix_transcription.py --input meeting.md --stage 1

# 3. Review output
cat meeting_阶段1_词典修复.md

# 4. If satisfied, run Stage 2
python scripts/fix_transcription.py --input meeting_阶段1_词典修复.md --stage 2

Batch Processing

Process multiple transcripts in sequence:

for file in transcripts/*.md; do
    python scripts/fix_transcription.py --input "$file" --stage 3
done

Quick Review Cycle

Generate and open comparison report immediately after correction:

# Run corrections
python scripts/fix_transcription.py --input meeting.md --stage 3

# Generate and open diff report
python scripts/generate_diff_report.py \
    --original "meeting.md" \
    --stage1 "meeting_阶段1_词典修复.md" \
    --stage2 "meeting_阶段2_AI修复.md"

open meeting_对比.html  # macOS
# xdg-open meeting_对比.html  # Linux
# start meeting_对比.html  # Windows

SQL Query Reference

Database location: ~/.transcript-fixer/corrections.db

Basic Operations

Add Corrections

-- Add a correction
INSERT INTO corrections (from_text, to_text, domain, source)
VALUES ('巨升智能', '具身智能', 'embodied_ai', 'manual');

INSERT INTO corrections (from_text, to_text, domain, source)
VALUES ('奇迹创坛', '奇绩创坛', 'general', 'manual');

View Corrections

-- View all active corrections
SELECT from_text, to_text, domain, source, usage_count
FROM active_corrections
ORDER BY domain, from_text;

-- View corrections for specific domain
SELECT from_text, to_text, usage_count, added_at
FROM active_corrections
WHERE domain = 'embodied_ai';

Context Rules

Add Context-Aware Rules

-- Add regex-based context rule
INSERT INTO context_rules (pattern, replacement, description, priority)
VALUES ('巨升方向', '具身方向', '巨升→具身', 10);

INSERT INTO context_rules (pattern, replacement, description, priority)
VALUES ('近距离的去看', '近距离地去看', '的→地 副词修饰', 5);

View Rules

-- View all active context rules (ordered by priority)
SELECT pattern, replacement, description, priority
FROM context_rules
WHERE is_active = 1
ORDER BY priority DESC;

Statistics

-- View correction statistics by domain
SELECT * FROM correction_statistics;

-- Count corrections by source
SELECT source, COUNT(*) as count, SUM(usage_count) as total_usage
FROM corrections
WHERE is_active = 1
GROUP BY source;

-- Most frequently used corrections
SELECT from_text, to_text, domain, usage_count, last_used
FROM corrections
WHERE is_active = 1 AND usage_count > 0
ORDER BY usage_count DESC
LIMIT 10;

Learning and Suggestions

View Suggestions

-- View pending suggestions
SELECT * FROM pending_suggestions;

-- View high-confidence suggestions
SELECT from_text, to_text, domain, frequency, confidence
FROM learned_suggestions
WHERE status = 'pending' AND confidence >= 0.8
ORDER BY confidence DESC, frequency DESC;

Approve Suggestions

-- Insert into corrections
INSERT INTO corrections (from_text, to_text, domain, source, confidence)
SELECT from_text, to_text, domain, 'learned', confidence
FROM learned_suggestions
WHERE id = 1;

-- Mark as approved
UPDATE learned_suggestions
SET status = 'approved', reviewed_at = CURRENT_TIMESTAMP
WHERE id = 1;

History and Audit

-- View recent correction runs
SELECT filename, domain, stage1_changes, stage2_changes, run_timestamp
FROM correction_history
ORDER BY run_timestamp DESC
LIMIT 10;

-- View detailed changes for a specific run
SELECT ch.line_number, ch.from_text, ch.to_text, ch.rule_type
FROM correction_changes ch
JOIN correction_history h ON ch.history_id = h.id
WHERE h.filename = 'meeting.md'
ORDER BY ch.line_number;

-- Calculate success rate
SELECT
    COUNT(*) as total_runs,
    SUM(CASE WHEN success = 1 THEN 1 ELSE 0 END) as successful,
    ROUND(100.0 * SUM(CASE WHEN success = 1 THEN 1 ELSE 0 END) / COUNT(*), 2) as success_rate
FROM correction_history;

Maintenance

-- Deactivate (soft delete) a correction
UPDATE corrections
SET is_active = 0
WHERE from_text = '错误词' AND domain = 'general';

-- Reactivate a correction
UPDATE corrections
SET is_active = 1
WHERE from_text = '错误词' AND domain = 'general';

-- Update correction confidence
UPDATE corrections
SET confidence = 0.95
WHERE from_text = '巨升' AND to_text = '具身';

-- Delete old history (older than 90 days)
DELETE FROM correction_history
WHERE run_timestamp < datetime('now', '-90 days');

-- Reclaim space
VACUUM;

System Configuration

-- View system configuration
SELECT key, value, description FROM system_config;

-- Update configuration
UPDATE system_config
SET value = '5'
WHERE key = 'learning_frequency_threshold';

-- Check schema version
SELECT value FROM system_config WHERE key = 'schema_version';

Export

-- Export corrections as CSV
.mode csv
.headers on
.output corrections_export.csv
SELECT from_text, to_text, domain, source, confidence, usage_count, added_at
FROM active_corrections;
.output stdout

For JSON export, use Python script with service.export_corrections() instead.

See Also

• references/file_formats.md - Complete database schema documentation
• references/quick_reference.md - CLI command quick reference
• SKILL.md - Main user documentation

Team Collaboration Guide

This guide explains how to share correction knowledge across teams using export/import and Git workflows.

Table of Contents

• Export/Import Workflow
• Export Corrections
• Import from Teammate
• Team Workflow Example
• Git-Based Collaboration
• Initial Setup
• Team Members Clone
• Ongoing Sync
• Handling Conflicts
• Selective Domain Sharing
• Finance Team
• AI Team
• Individual imports specific domains
• Git Branching Strategy
• Feature Branches
• Domain Branches (Alternative)
• Automated Sync (Advanced)
• macOS/Linux Cron
• Windows Task Scheduler
• Backup and Recovery
• Backup Strategy
• Recovery from Backup
• Recovery from Git
• Team Best Practices
• Integration with CI/CD
• GitHub Actions Example
• Troubleshooting
• Import Failed
• Git Sync Failed
• Merge Conflicts Too Complex
• Security Considerations
• Further Reading

Export/Import Workflow

Export Corrections

Share your corrections with team members:

# Export specific domain
python scripts/fix_transcription.py --export team_corrections.json --domain embodied_ai

# Export general corrections
python scripts/fix_transcription.py --export team_corrections.json

Output: Creates a standalone JSON file with your corrections.

Import from Teammate

Two modes: merge (combine) or replace (overwrite):

# Merge (recommended) - combines with existing corrections
python scripts/fix_transcription.py --import team_corrections.json --merge

# Replace - overwrites existing corrections (dangerous!)
python scripts/fix_transcription.py --import team_corrections.json

Merge behavior:

• Adds new corrections
• Updates existing corrections with imported values
• Preserves corrections not in import file

Team Workflow Example

Person A (Domain Expert):

# Build correction dictionary
python fix_transcription.py --add "巨升" "具身" --domain embodied_ai
python fix_transcription.py --add "奇迹创坛" "奇绩创坛" --domain embodied_ai
# ... add 50 more corrections ...

# Export for team
python fix_transcription.py --export ai_corrections.json --domain embodied_ai
# Send ai_corrections.json to team via Slack/email

Person B (Team Member):

# Receive ai_corrections.json
# Import and merge with existing corrections
python fix_transcription.py --import ai_corrections.json --merge

# Now Person B has all 50+ corrections!

Git-Based Collaboration

For teams using Git, version control the entire correction database.

Initial Setup

Person A (First User):

cd ~/.transcript-fixer
git init
git add corrections.json context_rules.json config.json
git add domains/
git commit -m "Initial correction database"

# Push to shared repo
git remote add origin git@github.com:org/transcript-corrections.git
git push -u origin main

Team Members Clone

Person B, C, D (Team Members):

# Clone shared corrections
git clone git@github.com:org/transcript-corrections.git ~/.transcript-fixer

# Now everyone has the same corrections!

Ongoing Sync

Daily workflow:

# Morning: Pull team updates
cd ~/.transcript-fixer
git pull origin main

# During day: Add corrections
python fix_transcription.py --add "错误" "正确"

# Evening: Push your additions
cd ~/.transcript-fixer
git add corrections.json
git commit -m "Added 5 new embodied AI corrections"
git push origin main

Handling Conflicts

When two people add different corrections to same file:

cd ~/.transcript-fixer
git pull origin main

# If conflict occurs:
# CONFLICT in corrections.json

# Option 1: Manual merge (recommended)
nano corrections.json  # Edit to combine both changes
git add corrections.json
git commit -m "Merged corrections from teammate"
git push

# Option 2: Keep yours
git checkout --ours corrections.json
git add corrections.json
git commit -m "Kept local corrections"
git push

# Option 3: Keep theirs
git checkout --theirs corrections.json
git add corrections.json
git commit -m "Used teammate's corrections"
git push

Best Practice: JSON merge conflicts are usually easy - just combine the correction entries from both versions.

Selective Domain Sharing

Share only specific domains with different teams:

Finance Team

# Finance team exports their domain
python fix_transcription.py --export finance_corrections.json --domain finance

# Share finance_corrections.json with finance team only

AI Team

# AI team exports their domain
python fix_transcription.py --export ai_corrections.json --domain embodied_ai

# Share ai_corrections.json with AI team only

Individual imports specific domains

# Alice works on both finance and AI
python fix_transcription.py --import finance_corrections.json --merge
python fix_transcription.py --import ai_corrections.json --merge

Git Branching Strategy

For larger teams, use branches for different domains or workflows:

Feature Branches

# Create branch for major dictionary additions
git checkout -b add-medical-terms
python fix_transcription.py --add "医疗术语" "正确术语" --domain medical
# ... add 100 medical corrections ...
git add domains/medical.json
git commit -m "Added 100 medical terminology corrections"
git push origin add-medical-terms

# Create PR for review
# After approval, merge to main

Domain Branches (Alternative)

# Separate branches per domain
git checkout -b domain/embodied-ai
# Work on AI corrections
git push origin domain/embodied-ai

git checkout -b domain/finance
# Work on finance corrections
git push origin domain/finance

Automated Sync (Advanced)

Set up automatic Git sync using cron/Task Scheduler:

macOS/Linux Cron

# Edit crontab
crontab -e

# Add daily sync at 9 AM and 6 PM
0 9,18 * * * cd ~/.transcript-fixer && git pull origin main && git push origin main

Windows Task Scheduler

# Create scheduled task
$action = New-ScheduledTaskAction -Execute "git" -Argument "pull origin main" -WorkingDirectory "$env:USERPROFILE\.transcript-fixer"
$trigger = New-ScheduledTaskTrigger -Daily -At 9am
Register-ScheduledTask -Action $action -Trigger $trigger -TaskName "SyncTranscriptCorrections"

Backup and Recovery

Backup Strategy

# Weekly backup to cloud
cd ~/.transcript-fixer
tar -czf transcript-corrections-$(date +%Y%m%d).tar.gz corrections.json context_rules.json domains/
# Upload to Dropbox/Google Drive/S3

Recovery from Backup

# Extract backup
tar -xzf transcript-corrections-20250127.tar.gz -C ~/.transcript-fixer/

Recovery from Git

# View history
cd ~/.transcript-fixer
git log corrections.json

# Restore from 3 commits ago
git checkout HEAD~3 corrections.json

# Or restore specific version
git checkout abc123def corrections.json

Team Best Practices

1. Pull Before Push: Always git pull before starting work 2. Commit Often: Small, frequent commits better than large infrequent ones 3. Descriptive Messages: "Added 5 finance terms" better than "updates" 4. Review Process: Use PRs for major dictionary changes (100+ corrections) 5. Domain Ownership: Assign domain experts as reviewers 6. Weekly Sync: Schedule team sync meetings to review learned suggestions 7. Backup Policy: Weekly backups of entire ~/.transcript-fixer/

Integration with CI/CD

For enterprise teams, integrate validation into CI:

GitHub Actions Example

# .github/workflows/validate-corrections.yml
name: Validate Corrections

on:
  pull_request:
    paths:
      - 'corrections.json'
      - 'domains/*.json'

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: Validate JSON
        run: |
          python -m json.tool corrections.json > /dev/null
          for file in domains/*.json; do
            python -m json.tool "$file" > /dev/null
          done

      - name: Check for duplicates
        run: |
          python scripts/check_duplicates.py corrections.json

Troubleshooting

Import Failed

# Check JSON validity
python -m json.tool team_corrections.json

# If invalid, fix JSON syntax errors
nano team_corrections.json

Git Sync Failed

# Check remote connection
git remote -v

# Re-add if needed
git remote set-url origin git@github.com:org/corrections.git

# Verify SSH keys
ssh -T git@github.com

Merge Conflicts Too Complex

# Nuclear option: Keep one version
git checkout --ours corrections.json  # Keep yours
# OR
git checkout --theirs corrections.json  # Keep theirs

# Then re-import the other version
python fix_transcription.py --import other_version.json --merge

Security Considerations

1. Private Repos: Use private Git repositories for company-specific corrections 2. Access Control: Limit who can push to main branch 3. Secret Scanning: Never commit API keys (already handled by security_scan.py) 4. Audit Trail: Git history provides full audit trail of who changed what 5. Backup Encryption: Encrypt backups if containing sensitive terminology

Further Reading

• Git workflows: https://git-scm.com/book/en/v2/Git-Branching-Branching-Workflows
• JSON validation: https://jsonlint.com/
• Team Git practices: https://github.com/git-guides

Troubleshooting Guide

Solutions to common issues and error conditions.

Table of Contents

• API Authentication Errors
• GLM_API_KEY Not Set
• Invalid API Key
• Learning System Issues
• No Suggestions Generated
• Database Issues
• Database Not Found
• Database Locked
• Corrupted Database
• Missing Tables
• Common Pitfalls
• 1. Stage Order Confusion
• 2. Overwriting Imports
• 3. Ignoring Learned Suggestions
• 4. Testing on Large Files
• 5. Manual Database Edits Without Validation
• 6. Committing .db Files to Git
• Validation Commands
• Quick Health Check
• Detailed Diagnostics
• Getting Help

API Authentication Errors

GLM_API_KEY Not Set

Symptom:

❌ Error: GLM_API_KEY environment variable not set
   Set it with: export GLM_API_KEY='your-key'

Solution:

# Check if key is set
echo $GLM_API_KEY

# If empty, export key
export GLM_API_KEY="your-api-key-here"

# Verify
uv run scripts/fix_transcription.py --validate

Persistence: Add to shell profile (.bashrc or .zshrc) for permanent access.

See glm_api_setup.md for detailed API key management.

Invalid API Key

Symptom: API calls fail with 401/403 errors

Solutions: 1. Verify key is correct (copy from https://open.bigmodel.cn/) 2. Check for extra spaces or quotes in the key 3. Regenerate key if compromised 4. Verify API quota hasn't been exceeded

Learning System Issues

No Suggestions Generated

Symptom: Running --review-learned shows no suggestions after multiple corrections.

Requirements:

• Minimum 3 correction runs with consistent patterns
• Learning frequency threshold ≥3 (default)
• Learning confidence threshold ≥0.8 (default)

Diagnostic steps:

# Check correction history count
sqlite3 ~/.transcript-fixer/corrections.db "SELECT COUNT(*) FROM correction_history;"

# If 0, no corrections have been run yet
# If >0 but <3, run more corrections

# Check suggestions table
sqlite3 ~/.transcript-fixer/corrections.db "SELECT * FROM learned_suggestions;"

# Check system configuration
sqlite3 ~/.transcript-fixer/corrections.db "SELECT key, value FROM system_config WHERE key LIKE 'learning%';"

Solutions: 1. Run at least 3 correction sessions 2. Ensure patterns repeat (same error → same correction) 3. Verify database permissions (should be readable/writable) 4. Check correction_history table has entries

Database Issues

Database Not Found

Symptom:

⚠️  Database not found: ~/.transcript-fixer/corrections.db

Solution:

uv run scripts/fix_transcription.py --init

This creates the database with the complete schema.

Database Locked

Symptom:

Error: database is locked

Causes:

• Another process is accessing the database
• Unfinished transaction from crashed process
• File permissions issue

Solutions:

# Check for processes using the database
lsof ~/.transcript-fixer/corrections.db

# If processes found, kill them or wait for completion

# If database is corrupted, backup and recreate
cp ~/.transcript-fixer/corrections.db ~/.transcript-fixer/corrections_backup.db
sqlite3 ~/.transcript-fixer/corrections.db "VACUUM;"

Corrupted Database

Symptom: SQLite errors, integrity check failures

Solutions:

# Check integrity
sqlite3 ~/.transcript-fixer/corrections.db "PRAGMA integrity_check;"

# If corrupted, attempt recovery
sqlite3 ~/.transcript-fixer/corrections.db ".recover" | sqlite3 ~/.transcript-fixer/corrections_new.db

# Replace database with recovered version
mv ~/.transcript-fixer/corrections.db ~/.transcript-fixer/corrections_corrupted.db
mv ~/.transcript-fixer/corrections_new.db ~/.transcript-fixer/corrections.db

Missing Tables

Symptom:

❌ Database missing tables: ['corrections', ...]

Solution: Reinitialize schema (safe, uses IF NOT EXISTS):

python -c "from core import CorrectionRepository; from pathlib import Path; CorrectionRepository(Path.home() / '.transcript-fixer' / 'corrections.db')"

Or delete database and reinitialize:

# Backup first
cp ~/.transcript-fixer/corrections.db ~/corrections_backup_$(date +%Y%m%d).db

# Reinitialize
uv run scripts/fix_transcription.py --init

Common Pitfalls

1. Stage Order Confusion

Problem: Running Stage 2 without Stage 1 output.

Solution: Use --stage 3 for full pipeline, or run stages sequentially:

# Wrong: Stage 2 on raw file
uv run scripts/fix_transcription.py --input file.md --stage 2  # ❌

# Correct: Full pipeline
uv run scripts/fix_transcription.py --input file.md --stage 3  # ✅

# Or sequential stages
uv run scripts/fix_transcription.py --input file.md --stage 1
uv run scripts/fix_transcription.py --input file_stage1.md --stage 2

2. Overwriting Imports

Problem: Using --import without --merge overwrites existing corrections.

Solution: Always use --merge flag:

# Wrong: Overwrites existing
uv run scripts/fix_transcription.py --import team.json  # ❌

# Correct: Merges with existing
uv run scripts/fix_transcription.py --import team.json --merge  # ✅

3. Ignoring Learned Suggestions

Problem: Not reviewing learned patterns, missing free optimizations.

Impact: Patterns detected by AI remain expensive (Stage 2) instead of cheap (Stage 1).

Solution: Review suggestions every 3-5 runs:

uv run scripts/fix_transcription.py --review-learned
uv run scripts/fix_transcription.py --approve "错误" "正确"

4. Testing on Large Files

Problem: Testing dictionary changes on large files wastes API quota.

Solution: Start with --stage 1 on small files (100-500 lines):

# Test dictionary changes first
uv run scripts/fix_transcription.py --input small_sample.md --stage 1

# Review output, adjust corrections
# Then run full pipeline
uv run scripts/fix_transcription.py --input large_file.md --stage 3

5. Manual Database Edits Without Validation

Problem: Direct SQL edits might violate schema constraints.

Solution: Always validate after manual changes:

sqlite3 ~/.transcript-fixer/corrections.db
# ... make changes ...
.quit

# Validate
uv run scripts/fix_transcription.py --validate

6. Committing .db Files to Git

Problem: Binary database files in Git cause merge conflicts and bloat repository.

Solution: Use JSON exports for version control:

# .gitignore
*.db
*.db-journal
*.bak

# Export for version control instead
uv run scripts/fix_transcription.py --export corrections_$(date +%Y%m%d).json
git add corrections_*.json

Validation Commands

Quick Health Check

uv run scripts/fix_transcription.py --validate

Detailed Diagnostics

# Check database integrity
sqlite3 ~/.transcript-fixer/corrections.db "PRAGMA integrity_check;"

# Check table counts
sqlite3 ~/.transcript-fixer/corrections.db "
SELECT 'corrections' as table_name, COUNT(*) as count FROM corrections
UNION ALL
SELECT 'context_rules', COUNT(*) FROM context_rules
UNION ALL
SELECT 'learned_suggestions', COUNT(*) FROM learned_suggestions
UNION ALL
SELECT 'correction_history', COUNT(*) FROM correction_history;
"

# Check configuration
sqlite3 ~/.transcript-fixer/corrections.db "SELECT * FROM system_config;"

Getting Help

If issues persist:

1. Run --validate to collect diagnostic information 2. Check correction_history and audit_log tables for errors 3. Review references/file_formats.md for schema details 4. Check references/architecture.md for component details 5. Verify Python and uv versions are up to date

For database corruption, automatic backups are created before migrations. Check for .bak files in ~/.transcript-fixer/.