Interview Simulator

Interview Simulator

Platform architecture and coaching system for realistic mock interview practice. This skill serves two purposes: (1) it coaches candidates on how to structure effective practice sessions, and (2) it specifies the full-stack architecture for building an automated interview simulation platform with voice AI, collaborative whiteboard, gaze-tracking proctoring, and mobile companion.

The other 7 interview skills define WHAT to practice. This skill defines HOW to practice it -- with realistic conditions, adaptive difficulty, and measurable progress.

---

When to Use

Use for:

• Designing or building a mock interview simulation platform
• Configuring realistic practice sessions with voice, whiteboard, and proctoring
• Implementing adaptive difficulty that targets weaknesses automatically
• Building a scoring and debrief system that tracks progress across sessions
• Setting up spaced repetition for concept review and story rehearsal
• Establishing a daily/weekly practice protocol
• Cost analysis and optimization for practice infrastructure

NOT for:

• Practicing a specific round type in isolation (use the round-specific skill)
• Building a prep timeline or study plan (use interview-loop-strategist)
• Resume or career narrative work (use cv-creator or career-biographer)
• Salary negotiation or offer evaluation
• Conference talk preparation (different evaluation criteria)

---

System Architecture

graph TB
    subgraph Client["Client Layer"]
        MOBILE["Mobile App<br/>React Native + Expo<br/>Flash cards, voice drills,<br/>progress dashboard"]
        DESKTOP["Desktop Web<br/>Next.js<br/>Full sessions, whiteboard,<br/>proctoring"]
    end

    subgraph Engines["Engine Layer"]
        VOICE["Voice Engine<br/>Hume AI EVI<br/>Emotion-sensitive<br/>interviewer voice"]
        BOARD["Whiteboard Engine<br/>tldraw + Claude Vision<br/>Diagram evaluation<br/>and scoring"]
        PROCTOR["Proctor Engine<br/>MediaPipe Face Mesh<br/>Gaze tracking,<br/>attention monitoring"]
    end

    subgraph Orchestrator["Session Orchestrator — Node.js"]
        ROUND["Round Selector<br/>Weakness-weighted<br/>random selection"]
        ADAPT["Adaptive Difficulty<br/>Performance-based<br/>question scaling"]
        DEBRIEF["Debrief Generator<br/>Transcript + emotion +<br/>proctor + whiteboard<br/>scored rubric"]
        SM2["SM-2 Scheduler<br/>Spaced repetition<br/>for concepts and stories"]
    end

    subgraph Data["Data Layer — Supabase"]
        SESSIONS[("sessions<br/>recordings, transcripts")]
        SCORES[("scores<br/>per-dimension breakdowns")]
        STORIES[("story_bank<br/>STAR-L entries")]
        CARDS[("flash_cards<br/>SM-2 intervals")]
    end

    MOBILE --> Orchestrator
    DESKTOP --> Orchestrator
    Orchestrator --> VOICE
    Orchestrator --> BOARD
    Orchestrator --> PROCTOR
    Orchestrator --> Data
    VOICE --> DEBRIEF
    BOARD --> DEBRIEF
    PROCTOR --> DEBRIEF

Component Selection Rationale

flowchart TD
    V{Voice AI?}
    V -->|"Emotion detection needed"| HUME["Hume AI EVI<br/>Emotion callbacks,<br/>adaptive persona,<br/>WebSocket streaming"]
    V -->|"Voice only, no emotion"| ELEVEN["ElevenLabs<br/>Fallback: high-quality<br/>TTS, no affect reading"]
    V -->|"Cost-constrained"| OPENAI_RT["OpenAI Realtime API<br/>Cheaper per minute,<br/>no emotion detection"]

    W{Whiteboard?}
    W -->|"React ecosystem, extensible"| TLDRAW["tldraw<br/>MIT license, React native,<br/>rich API, snapshot export"]
    W -->|"Simpler, self-hosted"| EXCALI["Excalidraw<br/>Good but harder to<br/>integrate programmatic<br/>screenshot capture"]

    P{Proctoring?}
    P -->|"Privacy-first, free"| MEDIAPIPE["MediaPipe Face Mesh<br/>Browser-based, 468 landmarks,<br/>iris tracking, no cloud"]
    P -->|"Commercial accuracy"| COMMERCIAL["Commercial proctoring<br/>Expensive, privacy concerns,<br/>overkill for self-practice"]

    style HUME fill:#2d5016,stroke:#333,color:#fff
    style TLDRAW fill:#2d5016,stroke:#333,color:#fff
    style MEDIAPIPE fill:#2d5016,stroke:#333,color:#fff

Why Hume over OpenAI Realtime API: Hume's EVI provides emotion callbacks (nervousness, confidence, hesitation) that enable adaptive interviewer behavior. OpenAI's Realtime API is voice-only with no affect detection. For interview simulation, emotion awareness is the differentiator -- a real interviewer adjusts based on your emotional state.

Why tldraw over Excalidraw: tldraw is a React component with a rich programmatic API. You can call editor.getSnapshot() to capture the canvas state, export to image, and send to Claude Vision for evaluation. Excalidraw's API is more limited for programmatic interaction.

Why MediaPipe over commercial proctoring: This is self-practice, not exam proctoring. MediaPipe runs entirely in the browser (no cloud), processes 468 face landmarks including iris position for gaze estimation, and costs nothing. Commercial proctoring (ProctorU, ExamSoft) is designed for adversarial exam settings with privacy trade-offs that make no sense for personal practice.

---

Session Flow

sequenceDiagram
    participant U as User
    participant O as Orchestrator
    participant V as Voice Engine
    participant W as Whiteboard
    participant P as Proctor
    participant D as Debrief

    U->>O: Start session
    O->>O: Select round type<br/>(weakness-weighted)
    O->>U: Confirm: ML Design, Difficulty 3/5,<br/>Persona: Collaborative
    U->>O: Accept / override

    O->>V: Initialize interviewer persona
    O->>P: Activate gaze tracking
    alt Design or Coding Round
        O->>W: Open whiteboard
    end

    loop During Session (30-45 min)
        V->>U: Ask question / follow-up
        U->>V: Respond (voice)
        V->>O: Emotion data (confidence, hesitation)
        O->>V: Adjust difficulty / tone
        P->>O: Gaze flags (second monitor, notes)
        alt Design Round
            W-->>O: Periodic screenshot (every 30s active)
            O-->>W: Evaluate diagram (Claude Vision)
        end
    end

    U->>O: End session
    O->>D: Compile transcript + emotion<br/>timeline + proctor flags +<br/>whiteboard evaluations
    D->>U: Scored debrief with<br/>strengths, weaknesses,<br/>specific improvement actions
    O->>O: Update weakness tracker,<br/>adjust next session focus

Session Configuration Options

---

Daily Practice Protocol

Morning Mobile Session (10 minutes)

07:00  Open mobile app
07:00  3 flash cards — spaced repetition surfaces weakest concepts
       (ML concepts, system design patterns, Anthropic-specific topics)
07:05  1 behavioral story rehearsal — voice, 3 minutes max
       App plays the prompt, you respond aloud, app records duration
07:08  Quick self-check — rate confidence 1-5 on today's cards
07:10  Done — push notification schedules evening session

Evening Desktop Session (30-60 minutes, 3-4x/week)

19:00  Open desktop app, orchestrator selects round type
19:02  Configure: confirm round, set proctor to Training mode
19:05  Session begins — voice AI drives conversation
       Whiteboard opens for design rounds
       Proctor tracks gaze, flags second monitor use
19:35  Session ends (30 min) or 19:50 (45 min)
19:35  Debrief displays: scored rubric, emotion timeline,
       proctor flags, whiteboard evaluation (if applicable)
19:45  Review debrief — spend 1/3 of practice time here
19:55  Update story bank with any new insights
20:00  Done — weakness tracker updated automatically

Weekend Loop Simulation (2 hours, 1x/week)

10:00  Full loop: 2-3 back-to-back rounds (different types)
       5-minute breaks between rounds (no phone, no notes)
       Proctor set to Simulation (strict) mode
11:30  Energy management practice — track cognitive fatigue
11:45  Cross-round story coherence review
       Did you tell the same project consistently across rounds?
12:00  Comprehensive weekly debrief — pattern analysis across sessions

---

Scoring and Progress Tracking

Per-Session Scoring Dimensions

Progress Visualization

Track these metrics over time on the dashboard:

• Composite score per session (0-100) with trend line
• Dimension radar chart showing strengths and weaknesses
• Streak tracker (consecutive days with at least one practice activity)
• Weakness heat map showing which round types and dimensions lag
• Story readiness gauge per story in bank (how many follow-up levels prepared)
• Spaced repetition coverage (percentage of flash cards at "mature" interval)

---

Setup Guide

Prerequisites

First-Run Experience

# 1. Clone the simulator repo
git clone <your-simulator-repo>
cd interview-simulator

# 2. Install dependencies
npm install

# 3. Configure environment
cp .env.example .env.local
# Edit .env.local with your API keys:
#   HUME_API_KEY=...
#   HUME_SECRET_KEY=...
#   ANTHROPIC_API_KEY=...
#   NEXT_PUBLIC_SUPABASE_URL=...
#   SUPABASE_SERVICE_KEY=...

# 4. Initialize database
npx supabase db push

# 5. Run first calibration session
npm run dev
# Navigate to localhost:3000/calibrate
# 10-minute session to establish baseline scores

Calibration Session

The first session is a calibration round: 10 minutes, mixed questions across all round types, no proctoring, friendly persona. This establishes baseline scores for each dimension so the adaptive difficulty has a starting point. Without calibration, the system defaults to difficulty 3 for all dimensions.

---

Cost Analysis

Cost Optimization Strategies

1. Session length caps: Hard-stop at configured time to prevent runaway voice costs 2. Whiteboard eval batching: Evaluate every 30s during active drawing, every 2min during discussion (not continuously) 3. Debrief caching: If same question type + similar transcript, reuse rubric structure with specific details swapped 4. Mobile voice: Use ElevenLabs (cheaper) for morning drills where emotion detection is unnecessary 5. Free tier Supabase: Sufficient for single-user practice; upgrade only for multi-user or heavy recording storage

---

Anti-Patterns

Practice Without Proctoring

Novice: Practices with notes open on a second monitor, browser tabs with answers visible, phone in hand for quick lookups. Builds false confidence from sessions where external resources masked knowledge gaps. In the real interview, stripped of supports, performance drops 30-40%.

Expert: Activates proctoring from the first session, even in Training (lenient) mode. Treats every practice as an approximation of real conditions. Clears desk, closes irrelevant tabs, puts phone face-down. Uses strict Simulation mode for weekend loop simulations. Understands that the discomfort of being watched IS the training.

Detection: Session history shows zero proctor flags across all sessions (impossibly clean), or proctor is consistently set to "Off." Compare self-reported confidence to actual debrief scores -- large gap indicates practice conditions are too easy.

Comfort Zone Looping

Novice: Manually selects the same round type repeatedly -- always behavioral (because stories are polished), always coding (because it feels productive), always the round they are already good at. Avoids design rounds because whiteboard evaluation is harsh. Avoids values rounds because deep follow-ups are uncomfortable.

Expert: Lets the orchestrator select rounds based on weakness analysis. Trusts the SM-2 algorithm to surface the uncomfortable topics at optimal intervals. When manually selecting, deliberately picks the lowest-scoring round type. Tracks round type distribution in the progress dashboard and rebalances if any type exceeds 40% of sessions.

Detection: Session history shows >50% of sessions are the same round type. Weakness heat map has persistent cold spots that never improve. Flash card review skips entire categories.

Feedback Ignored

Novice: Runs sessions back-to-back without reviewing debriefs. Treats mock interviews as reps to complete rather than learning opportunities. Session count is high but scores plateau. The debrief tab has a <50% read rate. Improvement actions from debriefs are never attempted.

Expert: Spends one-third of total practice time on debrief review. After each session, reads the full scored rubric, highlights one specific improvement action, and practices that action in the next session. Reviews weekly pattern analysis to identify cross-session trends. Keeps a "lessons learned" document updated after every debrief.

Detection: Debrief read rate below 50% (tracked via time-on-page). Same weaknesses flagged in debriefs 3+ sessions in a row without improvement. No improvement actions logged.

---

Integration with Round-Specific Skills

The simulator does not contain round-type content. It delegates to the 7 specialist skills for questions, rubrics, and evaluation criteria.

---

Reference Files

Mobile App Architecture

Architecture guide for the React Native + Expo companion app that handles morning drills, flash card review, story rehearsal, push notification scheduling, and progress tracking.

---

Stack Decisions

Why ElevenLabs for Mobile (Not Hume)

Mobile morning drills are 3-minute story rehearsals. The value is in speaking aloud under time pressure, not in emotion-adaptive interviewer behavior. ElevenLabs provides:

• Simpler SDK integration for React Native
• Lower per-minute cost (~$0.05/min vs ~$0.07/min)
• Faster connection setup (REST API vs WebSocket)
• No camera/face detection needed (saves battery)

Desktop evening sessions use Hume for the full experience. Mobile is the lightweight companion.

---

App Structure

interview-simulator-mobile/
├── app/                          # Expo Router pages
│   ├── _layout.tsx               # Root layout with tab navigation
│   ├── (tabs)/
│   │   ├── home.tsx              # Dashboard: streak, next session, quick actions
│   │   ├── cards.tsx             # Flash card review (SM-2)
│   │   ├── stories.tsx           # Story bank manager
│   │   ├── progress.tsx          # Progress dashboard, charts
│   │   └── settings.tsx          # API keys, notifications, sync
│   ├── drill/
│   │   ├── flash-card.tsx        # Flash card drill session
│   │   └── story-rehearsal.tsx   # Timed story rehearsal with voice
│   └── session/
│       └── [id].tsx              # View past session debrief
├── src/
│   ├── components/
│   │   ├── FlashCard.tsx         # Card display with flip animation
│   │   ├── StoryTimer.tsx        # Countdown timer for rehearsals
│   │   ├── StreakDisplay.tsx     # Daily streak visualization
│   │   ├── RadarChart.tsx        # Dimension score radar
│   │   └── WeaknessHeatmap.tsx   # Round-type weakness heat map
│   ├── hooks/
│   │   ├── useSM2.ts            # SM-2 spaced repetition logic
│   │   ├── useSync.ts           # Supabase sync with offline queue
│   │   ├── useVoice.ts          # ElevenLabs voice interaction
│   │   └── useNotifications.ts  # Push notification scheduling
│   ├── stores/
│   │   ├── cardStore.ts         # Flash card state (Zustand)
│   │   ├── storyStore.ts        # Story bank state
│   │   ├── sessionStore.ts      # Session history state
│   │   └── syncStore.ts         # Sync queue state
│   ├── db/
│   │   ├── schema.ts            # SQLite schema definitions
│   │   ├── migrations.ts        # Schema migration runner
│   │   └── queries.ts           # Typed query helpers
│   └── utils/
│       ├── sm2.ts               # SM-2 algorithm implementation
│       └── scoring.ts           # Score computation helpers
├── assets/                       # Icons, sounds, fonts
└── app.config.ts                 # Expo configuration

---

SM-2 Spaced Repetition Algorithm

The SuperMemo 2 (SM-2) algorithm schedules review intervals based on recall quality. Used for flash cards AND story rehearsal scheduling.

Algorithm Implementation

// SM-2 algorithm implementation
// Reference: https://www.supermemo.com/en/archives1990-2015/english/ol/sm2

interface SM2Card {
  id: string;
  content: string;          // Question or story prompt
  answer?: string;          // Expected answer (flash cards only)
  category: CardCategory;
  // SM-2 state
  easeFactor: number;       // Starts at 2.5, minimum 1.3
  interval: number;         // Days until next review
  repetitions: number;      // Consecutive correct reviews
  nextReviewDate: string;   // ISO date string
  lastReviewDate: string;
  // Metadata
  createdAt: string;
  roundType?: string;       // Which interview round this relates to
}

type CardCategory =
  | 'ml_concepts'           // ML theory, algorithms, math
  | 'system_design'         // System design patterns, tradeoffs
  | 'anthropic_specific'    // Constitutional AI, RLHF, interpretability
  | 'behavioral_stories'    // STAR-L story prompts
  | 'coding_patterns'       // Data structures, Python idioms
  | 'company_knowledge';    // Company-specific facts and culture

interface SM2ReviewResult {
  quality: 0 | 1 | 2 | 3 | 4 | 5;
  // 0: complete blackout
  // 1: incorrect, remembered upon seeing answer
  // 2: incorrect, but answer seemed easy to recall
  // 3: correct with serious difficulty
  // 4: correct after hesitation
  // 5: perfect response
}

function sm2(card: SM2Card, result: SM2ReviewResult): SM2Card {
  const { quality } = result;
  let { easeFactor, interval, repetitions } = card;

  if (quality >= 3) {
    // Correct response
    if (repetitions === 0) {
      interval = 1;
    } else if (repetitions === 1) {
      interval = 6;
    } else {
      interval = Math.round(interval * easeFactor);
    }
    repetitions += 1;
  } else {
    // Incorrect response: reset
    repetitions = 0;
    interval = 1;
  }

  // Update ease factor
  easeFactor = easeFactor + (0.1 - (5 - quality) * (0.08 + (5 - quality) * 0.02));
  easeFactor = Math.max(1.3, easeFactor); // Minimum 1.3

  const nextReviewDate = addDays(new Date(), interval).toISOString().split('T')[0];

  return {
    ...card,
    easeFactor,
    interval,
    repetitions,
    nextReviewDate,
    lastReviewDate: new Date().toISOString().split('T')[0],
  };
}

function addDays(date: Date, days: number): Date {
  const result = new Date(date);
  result.setDate(result.getDate() + days);
  return result;
}

SM-2 React Hook

import { useSQLiteContext } from 'expo-sqlite';

interface UseSM2Return {
  dueCards: SM2Card[];
  reviewCard: (cardId: string, quality: SM2ReviewResult['quality']) => Promise<void>;
  addCard: (card: Omit<SM2Card, 'easeFactor' | 'interval' | 'repetitions' | 'nextReviewDate' | 'lastReviewDate'>) => Promise<void>;
  stats: {
    totalCards: number;
    dueToday: number;
    mature: number;      // interval > 21 days
    learning: number;    // interval <= 21 days
    newToday: number;    // reviewed for first time today
  };
}

function useSM2(category?: CardCategory): UseSM2Return {
  const db = useSQLiteContext();
  const [dueCards, setDueCards] = useState<SM2Card[]>([]);
  const [stats, setStats] = useState<UseSM2Return['stats']>({
    totalCards: 0, dueToday: 0, mature: 0, learning: 0, newToday: 0,
  });

  const loadDueCards = useCallback(async () => {
    const today = new Date().toISOString().split('T')[0];
    const query = category
      ? `SELECT * FROM flash_cards WHERE next_review_date <= ? AND category = ? ORDER BY next_review_date ASC LIMIT 20`
      : `SELECT * FROM flash_cards WHERE next_review_date <= ? ORDER BY next_review_date ASC LIMIT 20`;

    const params = category ? [today, category] : [today];
    const rows = await db.getAllAsync(query, params);
    setDueCards(rows.map(rowToCard));
  }, [db, category]);

  const reviewCard = useCallback(async (cardId: string, quality: SM2ReviewResult['quality']) => {
    const card = dueCards.find(c => c.id === cardId);
    if (!card) return;

    const updated = sm2(card, { quality });

    await db.runAsync(
      `UPDATE flash_cards SET ease_factor = ?, interval = ?, repetitions = ?,
       next_review_date = ?, last_review_date = ? WHERE id = ?`,
      [updated.easeFactor, updated.interval, updated.repetitions,
       updated.nextReviewDate, updated.lastReviewDate, cardId]
    );

    // Queue for sync
    await queueSync('flash_cards', cardId, 'update');

    await loadDueCards();
  }, [db, dueCards, loadDueCards]);

  useEffect(() => { loadDueCards(); }, [loadDueCards]);

  return { dueCards, reviewCard, addCard, stats };
}

---

Flash Card Categories

Pre-built Card Decks

The app ships with starter decks that users can customize:

const STARTER_DECKS: Record<CardCategory, Array<{ content: string; answer: string }>> = {
  ml_concepts: [
    { content: "What is the bias-variance tradeoff?", answer: "High bias = underfitting (model too simple). High variance = overfitting (model too complex). Sweet spot minimizes total error = bias^2 + variance + irreducible noise." },
    { content: "Explain attention mechanism in transformers", answer: "Q, K, V matrices. Attention(Q,K,V) = softmax(QK^T / sqrt(d_k)) * V. Self-attention lets each token attend to all others. Multi-head allows attending to different representation subspaces." },
    { content: "What is catastrophic forgetting?", answer: "Neural networks lose previously learned information when trained on new data. Mitigations: EWC (elastic weight consolidation), progressive nets, replay buffers, multi-task learning." },
    // ... 50+ cards per category
  ],
  system_design: [
    { content: "When would you choose eventual consistency over strong consistency?", answer: "Eventual: high availability, partition tolerance, read-heavy (DNS, social feeds, caches). Strong: financial transactions, inventory counts, anything where stale reads have business cost. CAP theorem forces the tradeoff." },
    { content: "Explain the leaky bucket vs token bucket rate limiting", answer: "Leaky bucket: requests drain at constant rate, excess queued or dropped. Smooth output. Token bucket: tokens added at rate r, bucket holds b. Allows bursts up to b, then rate-limited to r. More flexible for bursty traffic." },
    // ... 50+ cards
  ],
  anthropic_specific: [
    { content: "What is Constitutional AI (CAI)?", answer: "Train AI to follow principles (constitution) instead of human ratings for every output. Two phases: (1) Self-critique with principles to generate revisions (2) RLAIF - use AI feedback instead of human feedback for RL. Reduces human labeling cost, makes values explicit and auditable." },
    { content: "What is the difference between RLHF and RLAIF?", answer: "RLHF: human ranks outputs, reward model trained on rankings, policy optimized via RL. RLAIF: AI model ranks outputs per constitutional principles. RLAIF advantages: scalable, consistent, transparent (constitution is readable). RLAIF risks: reward hacking, constitution design is hard." },
    // ... 30+ cards
  ],
  behavioral_stories: [
    { content: "Tell me about a time you failed.", answer: "[Your STAR-L story here. This card prompts rehearsal -- record yourself responding in under 3 minutes.]" },
    { content: "When did you disagree with your manager?", answer: "[STAR-L story. Focus on how you handled the disagreement, not just that you were right.]" },
    // ... 12 cards matching story bank categories
  ],
  coding_patterns: [
    { content: "When would you use a deque vs a list in Python?", answer: "deque: O(1) append/pop from both ends. Use for queues, sliding windows, BFS. list: O(1) append/pop from end only. O(n) insert/delete from front. Use deque when you need fast operations on both ends." },
    // ... 40+ cards
  ],
  company_knowledge: [
    { content: "What are Anthropic's core product offerings?", answer: "Claude (consumer chat), Claude API (developer), Claude for Enterprise (SSO, admin, audit), MCP (Model Context Protocol for tool use). Revenue primarily from API and Enterprise." },
    // ... 20+ cards per target company
  ],
};

---

Push Notification Scheduling

Notification Strategy

import * as Notifications from 'expo-notifications';

interface NotificationSchedule {
  morningDrill: { hour: number; minute: number; enabled: boolean };
  eveningReminder: { hour: number; minute: number; daysOfWeek: number[]; enabled: boolean };
  weekendLoop: { hour: number; minute: number; dayOfWeek: number; enabled: boolean };
  staleStoryReminder: { intervalDays: number; enabled: boolean };
}

const DEFAULT_SCHEDULE: NotificationSchedule = {
  morningDrill: { hour: 7, minute: 0, enabled: true },
  eveningReminder: { hour: 18, minute: 30, daysOfWeek: [1, 2, 3, 4], enabled: true }, // Mon-Thu
  weekendLoop: { hour: 10, minute: 0, dayOfWeek: 6, enabled: true }, // Saturday
  staleStoryReminder: { intervalDays: 4, enabled: true },
};

async function scheduleNotifications(schedule: NotificationSchedule): Promise<void> {
  // Cancel all existing scheduled notifications
  await Notifications.cancelAllScheduledNotificationsAsync();

  // Morning drill
  if (schedule.morningDrill.enabled) {
    await Notifications.scheduleNotificationAsync({
      content: {
        title: '3 flash cards + 1 story',
        body: getDueCardsSummary(), // e.g., "5 ML concepts due, 2 stories need rehearsal"
        data: { screen: 'drill/flash-card' },
      },
      trigger: {
        type: 'daily',
        hour: schedule.morningDrill.hour,
        minute: schedule.morningDrill.minute,
      },
    });
  }

  // Evening session reminder
  if (schedule.eveningReminder.enabled) {
    for (const day of schedule.eveningReminder.daysOfWeek) {
      await Notifications.scheduleNotificationAsync({
        content: {
          title: 'Evening mock session',
          body: getWeaknessBasedSuggestion(), // e.g., "Your ML Design scores are lowest -- focus there tonight"
          data: { screen: 'home' },
        },
        trigger: {
          type: 'weekly',
          weekday: day,
          hour: schedule.eveningReminder.hour,
          minute: schedule.eveningReminder.minute,
        },
      });
    }
  }

  // Weekend loop reminder
  if (schedule.weekendLoop.enabled) {
    await Notifications.scheduleNotificationAsync({
      content: {
        title: 'Weekend loop simulation',
        body: '2-hour full loop practice. Clear your schedule and close all other apps.',
        data: { screen: 'home' },
      },
      trigger: {
        type: 'weekly',
        weekday: schedule.weekendLoop.dayOfWeek,
        hour: schedule.weekendLoop.hour,
        minute: schedule.weekendLoop.minute,
      },
    });
  }
}

Stale Story Reminders

Track when each story was last rehearsed and nudge when it goes stale:

async function checkStaleStories(intervalDays: number): Promise<void> {
  const cutoffDate = addDays(new Date(), -intervalDays).toISOString().split('T')[0];
  const staleStories = await db.getAllAsync(
    `SELECT * FROM stories WHERE last_rehearsed < ? OR last_rehearsed IS NULL`,
    [cutoffDate]
  );

  if (staleStories.length > 0) {
    const storyNames = staleStories.slice(0, 3).map(s => s.title).join(', ');
    await Notifications.scheduleNotificationAsync({
      content: {
        title: 'Stories getting stale',
        body: `You haven't practiced: ${storyNames}. Rehearse one today.`,
        data: { screen: 'stories' },
      },
      trigger: null, // Immediate
    });
  }
}

---

Offline Mode

What Works Offline

Sync Strategy

import { createClient } from '@supabase/supabase-js';

interface SyncQueueEntry {
  id: string;
  table: string;
  recordId: string;
  operation: 'insert' | 'update' | 'delete';
  payload: any;
  createdAt: string;
  synced: boolean;
}

class SyncManager {
  private supabase;
  private db;

  constructor(supabaseUrl: string, supabaseKey: string, db: SQLiteDatabase) {
    this.supabase = createClient(supabaseUrl, supabaseKey);
    this.db = db;
  }

  // Queue a change for sync when online
  async queueChange(table: string, recordId: string, operation: string, payload: any): Promise<void> {
    await this.db.runAsync(
      `INSERT INTO sync_queue (table_name, record_id, operation, payload, synced)
       VALUES (?, ?, ?, ?, 0)`,
      [table, recordId, operation, JSON.stringify(payload)]
    );
  }

  // Process queue when connection is available
  async processQueue(): Promise<{ synced: number; failed: number }> {
    const pending = await this.db.getAllAsync(
      `SELECT * FROM sync_queue WHERE synced = 0 ORDER BY created_at ASC`
    );

    let synced = 0;
    let failed = 0;

    for (const entry of pending) {
      try {
        const payload = JSON.parse(entry.payload);

        switch (entry.operation) {
          case 'insert':
            await this.supabase.from(entry.table_name).insert(payload);
            break;
          case 'update':
            await this.supabase.from(entry.table_name).update(payload).eq('id', entry.record_id);
            break;
          case 'delete':
            await this.supabase.from(entry.table_name).delete().eq('id', entry.record_id);
            break;
        }

        await this.db.runAsync(`UPDATE sync_queue SET synced = 1 WHERE id = ?`, [entry.id]);
        synced++;
      } catch (error) {
        console.warn(`Sync failed for ${entry.table_name}/${entry.record_id}:`, error);
        failed++;
      }
    }

    return { synced, failed };
  }

  // Pull updates from server (desktop sessions, debrief scores)
  async pullFromServer(lastSyncTimestamp: string): Promise<void> {
    // Pull session scores created on desktop
    const { data: sessions } = await this.supabase
      .from('sessions')
      .select('*')
      .gt('updated_at', lastSyncTimestamp);

    if (sessions?.length) {
      for (const session of sessions) {
        await this.db.runAsync(
          `INSERT OR REPLACE INTO sessions (id, round_type, composite_score, debrief, created_at, updated_at)
           VALUES (?, ?, ?, ?, ?, ?)`,
          [session.id, session.round_type, session.composite_score, JSON.stringify(session.debrief), session.created_at, session.updated_at]
        );
      }
    }

    // Update last sync timestamp
    await this.db.runAsync(
      `UPDATE app_state SET value = ? WHERE key = 'last_sync'`,
      [new Date().toISOString()]
    );
  }
}

Conflict Resolution

Desktop and mobile may modify the same data (e.g., flash card review on both). Resolution strategy:

---

Progress Dashboard

Key Visualizations

Streak Display

interface StreakData {
  currentStreak: number;      // Consecutive days with activity
  longestStreak: number;      // All-time best
  todayComplete: boolean;     // Has user done anything today?
  weekActivity: boolean[];    // Last 7 days: [Mon, Tue, ..., Sun]
}

Dimension Radar Chart

Shows scores across all 7 scoring dimensions (from session-orchestration.md), updated after each desktop session. Uses the last 5 sessions' moving average for stability.

Weakness Heat Map

              Coding  Design  Behavioral  Presentation  HM   Technical
Technical     ████    ██░░    ████████    ███░░░        ██   ███████
Communication ███░    █████   ██████░░    ████████      ███  ████░░░
Time Mgmt     ██░░    ███░░   ███████░    █████░░       ██░  █████░░
Structure     ████    ██████  █████░░░    ██████░░      ███  ████░░░
Composure     █████   ███░░   ████████    ███░░░░░      ███  ██████░
Questions     ███░░   ████░░  █████████   █████░░░      ████ ████░░░

Green = strong (80+), Yellow = moderate (60-79), Red = weak (<60). Empty cells = no data yet for that combination.

Session History

Scrollable list of past sessions with:

• Date and round type
• Composite score with trend indicator (up/down/stable)
• Top 1 strength and top 1 weakness from debrief
• Tap to view full debrief

Proctor Engine Setup

Integration guide for MediaPipe Face Mesh as a browser-based proctoring engine that tracks gaze direction, detects attention drift, and flags suspicious behavior during mock interview sessions.

---

Why MediaPipe

MediaPipe Face Mesh runs entirely in the browser via TensorFlow.js. No video is sent to any server. This is critical for a self-practice tool:

1. Privacy-first: All processing is local. No video frames leave the device. 2. Free: No per-minute or per-session cost. Runs on client hardware. 3. 468 face landmarks: Enough precision for reliable gaze estimation, including iris tracking. 4. Real-time: 30+ FPS on modern hardware (even on mid-range laptops). 5. No installation: Works in Chrome/Edge/Firefox via WebAssembly.

Commercial proctoring solutions (ProctorU, ExamSoft, Respondus) are designed for adversarial exam settings. They record video, flag to human reviewers, and cost $10-30 per session. For self-practice, this is overkill with unacceptable privacy trade-offs.

---

Setup

Installation

# MediaPipe Face Mesh via TensorFlow.js
npm install @mediapipe/face_mesh @mediapipe/camera_utils
npm install @tensorflow/tfjs-core @tensorflow/tfjs-backend-webgl

Browser Permissions

The proctor requires camera access. Prompt the user clearly about why:

async function requestCameraPermission(): Promise<MediaStream | null> {
  try {
    const stream = await navigator.mediaDevices.getUserMedia({
      video: {
        width: { ideal: 640 },   // Don't need high resolution
        height: { ideal: 480 },
        facingMode: 'user',       // Front camera
        frameRate: { ideal: 15 }, // 15 FPS is sufficient for gaze tracking
      },
      audio: false,  // Audio is handled by the voice engine
    });
    return stream;
  } catch (error) {
    console.warn('Camera access denied. Proctoring disabled for this session.');
    return null;
  }
}

---

Face Mesh Initialization

import { FaceMesh, Results } from '@mediapipe/face_mesh';
import { Camera } from '@mediapipe/camera_utils';

class ProctorEngine {
  private faceMesh: FaceMesh;
  private camera: Camera | null = null;
  private videoElement: HTMLVideoElement;
  private flags: ProctorFlag[] = [];
  private config: ProctorConfig;
  private gazeHistory: GazeVector[] = [];
  private facePresent: boolean = false;
  private lastFaceSeenTs: number = Date.now();

  constructor(videoElement: HTMLVideoElement, config: ProctorConfig) {
    this.videoElement = videoElement;
    this.config = config;

    this.faceMesh = new FaceMesh({
      locateFile: (file) => {
        return `https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh/${file}`;
      },
    });

    this.faceMesh.setOptions({
      maxNumFaces: 2,            // Detect up to 2 faces (flag if >1)
      refineLandmarks: true,     // Enable iris tracking for precise gaze
      minDetectionConfidence: 0.5,
      minTrackingConfidence: 0.5,
    });

    this.faceMesh.onResults((results) => this.processResults(results));
  }

  async start(): Promise<void> {
    this.camera = new Camera(this.videoElement, {
      onFrame: async () => {
        await this.faceMesh.send({ image: this.videoElement });
      },
      width: 640,
      height: 480,
    });
    await this.camera.start();
  }

  stop(): void {
    this.camera?.stop();
  }

  private processResults(results: Results): void {
    const now = Date.now();

    // Multiple face detection
    if (results.multiFaceLandmarks.length > 1) {
      this.addFlag('multiple_faces', 'Multiple faces detected', now);
    }

    // No face detection
    if (results.multiFaceLandmarks.length === 0) {
      this.facePresent = false;
      const absenceDuration = now - this.lastFaceSeenTs;
      if (absenceDuration > this.config.absenceThresholdMs) {
        this.addFlag('face_absent', `Face absent for ${Math.round(absenceDuration / 1000)}s`, now);
      }
      return;
    }

    this.facePresent = true;
    this.lastFaceSeenTs = now;

    const landmarks = results.multiFaceLandmarks[0];
    const gaze = this.estimateGaze(landmarks);
    this.gazeHistory.push({ ...gaze, timestamp: now });

    // Gaze deviation check
    this.checkGazeDeviation(gaze, now);
  }

  getFlags(): ProctorFlag[] {
    return this.flags;
  }

  getGazeHistory(): GazeVector[] {
    return this.gazeHistory;
  }
}

---

Gaze Vector Calculation

MediaPipe Face Mesh provides 468 face landmarks. For gaze estimation, we use the iris landmarks (introduced with refineLandmarks: true) plus head pose estimation.

Key Landmark Indices

// MediaPipe Face Mesh landmark indices for gaze estimation
const LANDMARKS = {
  // Left eye
  LEFT_EYE_INNER: 133,
  LEFT_EYE_OUTER: 33,
  LEFT_EYE_TOP: 159,
  LEFT_EYE_BOTTOM: 145,

  // Right eye
  RIGHT_EYE_INNER: 362,
  RIGHT_EYE_OUTER: 263,
  RIGHT_EYE_TOP: 386,
  RIGHT_EYE_BOTTOM: 374,

  // Iris (with refineLandmarks enabled)
  LEFT_IRIS_CENTER: 468,    // Center of left iris
  LEFT_IRIS_TOP: 469,
  LEFT_IRIS_BOTTOM: 471,
  LEFT_IRIS_LEFT: 470,
  LEFT_IRIS_RIGHT: 472,

  RIGHT_IRIS_CENTER: 473,   // Center of right iris
  RIGHT_IRIS_TOP: 474,
  RIGHT_IRIS_BOTTOM: 476,
  RIGHT_IRIS_LEFT: 475,
  RIGHT_IRIS_RIGHT: 477,

  // Head pose reference points
  NOSE_TIP: 1,
  CHIN: 152,
  LEFT_CHEEK: 234,
  RIGHT_CHEEK: 454,
  FOREHEAD: 10,
};

Gaze Estimation Algorithm

interface GazeVector {
  horizontalAngle: number;  // Degrees: negative=left, positive=right
  verticalAngle: number;    // Degrees: negative=down, positive=up
  confidence: number;        // 0-1 confidence in the estimate
  timestamp: number;
}

function estimateGaze(landmarks: NormalizedLandmark[]): Omit<GazeVector, 'timestamp'> {
  // Step 1: Get iris positions relative to eye boundaries
  const leftIris = landmarks[LANDMARKS.LEFT_IRIS_CENTER];
  const leftEyeInner = landmarks[LANDMARKS.LEFT_EYE_INNER];
  const leftEyeOuter = landmarks[LANDMARKS.LEFT_EYE_OUTER];
  const leftEyeTop = landmarks[LANDMARKS.LEFT_EYE_TOP];
  const leftEyeBottom = landmarks[LANDMARKS.LEFT_EYE_BOTTOM];

  const rightIris = landmarks[LANDMARKS.RIGHT_IRIS_CENTER];
  const rightEyeInner = landmarks[LANDMARKS.RIGHT_EYE_INNER];
  const rightEyeOuter = landmarks[LANDMARKS.RIGHT_EYE_OUTER];

  // Step 2: Calculate horizontal iris position within eye (0=inner, 1=outer)
  const leftEyeWidth = distance2D(leftEyeInner, leftEyeOuter);
  const leftIrisHoriz = (leftIris.x - leftEyeInner.x) / (leftEyeOuter.x - leftEyeInner.x);

  const rightEyeWidth = distance2D(rightEyeInner, rightEyeOuter);
  const rightIrisHoriz = (rightIris.x - rightEyeInner.x) / (rightEyeOuter.x - rightEyeInner.x);

  // Average both eyes for robustness
  const avgHorizontalRatio = (leftIrisHoriz + rightIrisHoriz) / 2;

  // Step 3: Calculate vertical iris position (0=top, 1=bottom)
  const leftEyeHeight = distance2D(leftEyeTop, leftEyeBottom);
  const leftIrisVert = (leftIris.y - leftEyeTop.y) / (leftEyeBottom.y - leftEyeTop.y);

  // Step 4: Convert ratios to angles
  // Center position is ~0.5. Deviation from center maps to gaze angle.
  // Empirical calibration: 0.1 deviation ≈ 15 degrees
  const horizontalAngle = (avgHorizontalRatio - 0.5) * 150; // degrees
  const verticalAngle = (leftIrisVert - 0.5) * 120;          // degrees

  // Step 5: Add head pose compensation
  const headYaw = estimateHeadYaw(landmarks);
  const compensatedHorizontal = horizontalAngle + headYaw;

  // Step 6: Confidence based on face detection quality
  const confidence = calculateConfidence(landmarks);

  return {
    horizontalAngle: compensatedHorizontal,
    verticalAngle,
    confidence,
  };
}

function estimateHeadYaw(landmarks: NormalizedLandmark[]): number {
  // Use nose tip and cheek landmarks to estimate head rotation
  const noseTip = landmarks[LANDMARKS.NOSE_TIP];
  const leftCheek = landmarks[LANDMARKS.LEFT_CHEEK];
  const rightCheek = landmarks[LANDMARKS.RIGHT_CHEEK];

  const leftDist = distance2D(noseTip, leftCheek);
  const rightDist = distance2D(noseTip, rightCheek);

  // Asymmetry in nose-to-cheek distance indicates head rotation
  const ratio = leftDist / (leftDist + rightDist);
  return (ratio - 0.5) * 90; // Approximate degrees
}

function distance2D(a: { x: number; y: number }, b: { x: number; y: number }): number {
  return Math.sqrt((a.x - b.x) ** 2 + (a.y - b.y) ** 2);
}

function calculateConfidence(landmarks: NormalizedLandmark[]): number {
  // Higher confidence when face is frontal and well-lit
  // Lower confidence with extreme angles or partial occlusion
  const headYaw = Math.abs(estimateHeadYaw(landmarks));
  if (headYaw > 45) return 0.3;  // Extreme angle
  if (headYaw > 30) return 0.5;  // Moderate angle
  if (headYaw > 15) return 0.7;  // Slight angle
  return 0.9;                     // Frontal
}

---

Suspicion Thresholds

Two modes with different sensitivities:

Training Mode (Lenient)

For building good habits without constant interruption. Flags are logged but don't interrupt the session.

const TRAINING_CONFIG: ProctorConfig = {
  mode: 'training',
  absenceThresholdMs: 10_000,        // 10s before flagging absence
  gazeDeviationAngle: 35,            // Degrees off-center before flagging
  gazeDeviationDurationMs: 5_000,    // Must sustain deviation for 5s
  multipleFaceAction: 'log',         // Just log, don't interrupt
  interruptOnFlag: false,            // Never interrupt the session
  maxFlagsBeforeWarning: 10,         // Warn in debrief if >10 flags
  cooldownBetweenFlagsMs: 30_000,    // Minimum 30s between same flag type
};

Simulation Mode (Strict)

For realistic exam conditions during weekend loop simulations.

const SIMULATION_CONFIG: ProctorConfig = {
  mode: 'simulation',
  absenceThresholdMs: 5_000,         // 5s before flagging
  gazeDeviationAngle: 25,            // Tighter angle threshold
  gazeDeviationDurationMs: 3_000,    // Flag after 3s sustained deviation
  multipleFaceAction: 'flag_severe', // Severe flag for multiple faces
  interruptOnFlag: true,             // Voice AI acknowledges: "I noticed you looked away"
  maxFlagsBeforeWarning: 5,          // Stricter limit
  cooldownBetweenFlagsMs: 15_000,    // More frequent flagging allowed
};

Gaze Deviation Detection

function checkGazeDeviation(gaze: GazeVector, now: number): void {
  const deviationAngle = Math.sqrt(
    gaze.horizontalAngle ** 2 + gaze.verticalAngle ** 2
  );

  if (deviationAngle > this.config.gazeDeviationAngle && gaze.confidence > 0.5) {
    // Check if deviation is sustained
    const recentGazes = this.gazeHistory
      .filter(g => now - g.timestamp < this.config.gazeDeviationDurationMs);

    const sustainedDeviation = recentGazes.every(g => {
      const angle = Math.sqrt(g.horizontalAngle ** 2 + g.verticalAngle ** 2);
      return angle > this.config.gazeDeviationAngle;
    });

    if (sustainedDeviation && recentGazes.length > 3) {
      // Determine likely cause
      const direction = gaze.horizontalAngle > 0 ? 'right' : 'left';
      const severity = deviationAngle > 45 ? 'high' : 'moderate';

      this.addFlag(
        'gaze_deviation',
        `Sustained gaze ${direction} (${Math.round(deviationAngle)} deg) for ${this.config.gazeDeviationDurationMs / 1000}s`,
        now,
        severity,
      );
    }
  }
}

Common Gaze Patterns and What They Mean

Important: Looking up and to the side while thinking is NORMAL and should not be flagged. The deviation must be sustained and directionally consistent to suggest note-reading or second-monitor reference.

---

Multiple Face Detection

private handleMultipleFaces(faceCount: number, now: number): void {
  if (faceCount <= 1) return;

  const flag: ProctorFlag = {
    type: 'multiple_faces',
    message: `${faceCount} faces detected in camera frame`,
    timestamp: now,
    severity: this.config.multipleFaceAction === 'flag_severe' ? 'high' : 'low',
  };

  this.flags.push(flag);

  // In simulation mode, the voice AI can address this
  if (this.config.interruptOnFlag) {
    // Trigger voice prompt: "I noticed someone else is visible.
    // In a real interview, please make sure you're in a private space."
  }
}

---

Absence Detection

Track periods when the face is not detected:

interface AbsencePeriod {
  startTs: number;
  endTs: number | null;  // null = still absent
  duration: number;
  flagged: boolean;
}

private absencePeriods: AbsencePeriod[] = [];
private currentAbsence: AbsencePeriod | null = null;

private trackAbsence(facePresent: boolean, now: number): void {
  if (!facePresent && !this.currentAbsence) {
    // Start absence period
    this.currentAbsence = {
      startTs: now,
      endTs: null,
      duration: 0,
      flagged: false,
    };
  } else if (facePresent && this.currentAbsence) {
    // End absence period
    this.currentAbsence.endTs = now;
    this.currentAbsence.duration = now - this.currentAbsence.startTs;
    this.absencePeriods.push(this.currentAbsence);
    this.currentAbsence = null;
  } else if (!facePresent && this.currentAbsence) {
    // Update ongoing absence
    const duration = now - this.currentAbsence.startTs;
    if (duration > this.config.absenceThresholdMs && !this.currentAbsence.flagged) {
      this.addFlag('face_absent', `Absent for ${Math.round(duration / 1000)}s`, now, 'moderate');
      this.currentAbsence.flagged = true;
    }
  }
}

---

Privacy Configuration

All proctor data stays local by default. Users must explicitly opt in to any storage.

interface PrivacyConfig {
  storeVideoFrames: false;           // NEVER store raw video by default
  storeGazeVectors: true;            // Numeric data only, no PII
  storeFlagTimestamps: true;         // When flags occurred
  storeAbsencePeriods: true;         // Duration of absences
  storeLandmarkData: false;          // Face landmark coordinates (opt-in)
  exportGazeHeatmap: boolean;        // Generate anonymized gaze heatmap for debrief
  retentionDays: 30;                 // Auto-delete proctor data after 30 days
}

const DEFAULT_PRIVACY: PrivacyConfig = {
  storeVideoFrames: false,
  storeGazeVectors: true,
  storeFlagTimestamps: true,
  storeAbsencePeriods: true,
  storeLandmarkData: false,
  exportGazeHeatmap: true,
  retentionDays: 30,
};

Key privacy principles: 1. No video storage: Raw video frames are processed in-memory and discarded. Only derived data (gaze angles, flag events) is stored. 2. No cloud processing: MediaPipe runs in the browser via WebAssembly. No network requests for face processing. 3. Explicit opt-in for landmarks: Storing 468 face landmark coordinates is unnecessary for proctor functionality. Only offered for advanced debugging. 4. Auto-deletion: Proctor data expires after 30 days. Session scores and flags persist longer (controlled by session orchestrator).

---

Integration with Session Orchestrator

Flag Format for Debrief

interface ProctorFlag {
  type: 'gaze_deviation' | 'face_absent' | 'multiple_faces';
  message: string;
  timestamp: number;
  severity: 'low' | 'moderate' | 'high';
}

// Sent to debrief generator
interface ProctorSummary {
  sessionId: string;
  mode: 'training' | 'simulation';
  totalFlags: number;
  flagsByType: Record<string, number>;
  flagsByMinute: Array<{ minute: number; count: number }>; // When were flags concentrated?
  absenceTotalSeconds: number;
  gazeDeviationPercent: number; // % of session with gaze outside threshold
  complianceScore: number;     // 0-100: higher = fewer flags
}

function computeProctorSummary(flags: ProctorFlag[], gazeHistory: GazeVector[], sessionDurationMs: number): ProctorSummary {
  const flagsByType: Record<string, number> = {};
  flags.forEach(f => {
    flagsByType[f.type] = (flagsByType[f.type] || 0) + 1;
  });

  // Gaze deviation percentage
  const deviatedFrames = gazeHistory.filter(g => {
    const angle = Math.sqrt(g.horizontalAngle ** 2 + g.verticalAngle ** 2);
    return angle > 25;  // Using simulation threshold for scoring
  });
  const gazeDeviationPercent = (deviatedFrames.length / gazeHistory.length) * 100;

  // Compliance score: start at 100, deduct per flag
  const deductions = {
    gaze_deviation: 5,
    face_absent: 10,
    multiple_faces: 15,
  };
  let complianceScore = 100;
  flags.forEach(f => {
    complianceScore -= deductions[f.type] || 5;
  });
  complianceScore = Math.max(0, complianceScore);

  // Flags by minute for timeline visualization
  const sessionMinutes = Math.ceil(sessionDurationMs / 60_000);
  const flagsByMinute = Array.from({ length: sessionMinutes }, (_, i) => {
    const minuteStart = i * 60_000;
    const minuteEnd = (i + 1) * 60_000;
    const count = flags.filter(f => f.timestamp >= minuteStart && f.timestamp < minuteEnd).length;
    return { minute: i, count };
  });

  return {
    sessionId: '',  // Set by orchestrator
    mode: 'training',
    totalFlags: flags.length,
    flagsByType,
    flagsByMinute,
    absenceTotalSeconds: 0,  // Computed from absence periods
    gazeDeviationPercent,
    complianceScore,
  };
}

Voice AI Integration in Simulation Mode

When interruptOnFlag is true, the proctor communicates with the voice engine:

// In session orchestrator
async function handleProctorFlag(flag: ProctorFlag): Promise<void> {
  if (!config.interruptOnFlag) return;

  const prompts: Record<string, string> = {
    gaze_deviation: "I noticed you were looking away from the screen. In a real interview, your interviewer would notice this. Let's continue -- where were we?",
    face_absent: "It looks like you stepped away. Everything okay? Let's pick up where we left off.",
    multiple_faces: "I noticed someone else in the frame. For the most realistic practice, try to be in a private space.",
  };

  const prompt = prompts[flag.type];
  if (prompt) {
    await voiceEngine.injectPrompt(prompt);
  }
}

---

Performance Considerations

CPU Usage

MediaPipe Face Mesh at 15 FPS uses approximately:

• M1/M2/M3/M4 Mac: 5-8% CPU (negligible)
• Modern Windows laptop (i7/Ryzen 7): 8-12% CPU
• Budget laptop: 15-25% CPU (may need to reduce to 10 FPS)

Frame Rate Configuration

// Adaptive frame rate based on device capability
function getOptimalFrameRate(): number {
  const cores = navigator.hardwareConcurrency || 4;
  if (cores >= 8) return 15;  // High-end: 15 FPS
  if (cores >= 4) return 10;  // Mid-range: 10 FPS
  return 5;                    // Low-end: 5 FPS (still usable for gaze tracking)
}

Memory

MediaPipe Face Mesh model is ~2MB loaded. Gaze history for a 45-minute session at 15 FPS is approximately:

• 45 min 60 sec 15 frames = 40,500 entries
• Each entry: ~100 bytes (angles, timestamp, confidence)
• Total: ~4MB per session (negligible)

Session Orchestration

The session orchestrator is the central nervous system of the interview simulator. It selects rounds, adapts difficulty, tracks performance, generates debriefs, and schedules future practice. This document covers the algorithms, data models, and prompts that drive it.

---

Round Selection Algorithm

Weakness-Weighted Random Selection

The orchestrator does not simply pick a random round type. It biases selection toward the candidate's weakest areas, with a minimum variety constraint to prevent tunnel vision.

interface RoundWeight {
  roundType: RoundType;
  baseWeight: number;       // Inversely proportional to recent score
  varietyBonus: number;     // Bonus if this type hasn't been practiced recently
  finalWeight: number;      // baseWeight + varietyBonus
}

type RoundType =
  | 'coding'
  | 'ml_design'
  | 'behavioral'
  | 'tech_presentation'
  | 'hiring_manager'
  | 'anthropic_technical';

function selectRound(history: SessionHistory[], candidateOverride?: RoundType): RoundType {
  if (candidateOverride) return candidateOverride;

  const recentScores = getRecentScores(history, 10); // Last 10 sessions
  const roundTypes: RoundType[] = [
    'coding', 'ml_design', 'behavioral',
    'tech_presentation', 'hiring_manager', 'anthropic_technical',
  ];

  const weights: RoundWeight[] = roundTypes.map(roundType => {
    // Base weight: inverse of average score (lower score = higher weight)
    const scores = recentScores.filter(s => s.roundType === roundType);
    const avgScore = scores.length > 0
      ? scores.reduce((sum, s) => sum + s.compositeScore, 0) / scores.length
      : 50; // Default 50 for untried round types (medium priority)

    // Invert: score of 80 -> weight 20, score of 30 -> weight 70
    const baseWeight = 100 - avgScore;

    // Variety bonus: days since last practice of this type
    const lastSession = history
      .filter(h => h.roundType === roundType)
      .sort((a, b) => b.timestamp - a.timestamp)[0];

    const daysSinceLast = lastSession
      ? (Date.now() - lastSession.timestamp) / (1000 * 60 * 60 * 24)
      : 30; // 30 days if never practiced

    // Variety bonus: +5 per day since last practice, capped at 30
    const varietyBonus = Math.min(30, daysSinceLast * 5);

    return {
      roundType,
      baseWeight,
      varietyBonus,
      finalWeight: baseWeight + varietyBonus,
    };
  });

  // Weighted random selection
  const totalWeight = weights.reduce((sum, w) => sum + w.finalWeight, 0);
  let random = Math.random() * totalWeight;

  for (const w of weights) {
    random -= w.finalWeight;
    if (random <= 0) return w.roundType;
  }

  return weights[0].roundType; // Fallback
}

Minimum Variety Constraint

Even if one round type is very weak, the candidate should not practice it more than 40% of sessions. This prevents burnout and ensures all round types stay fresh.

function enforceVarietyConstraint(
  selected: RoundType,
  recentHistory: SessionHistory[],
  maxRatio: number = 0.4,
): RoundType {
  const last10 = recentHistory.slice(-10);
  const selectedCount = last10.filter(h => h.roundType === selected).length;

  if (selectedCount / Math.max(last10.length, 1) >= maxRatio) {
    // This type is over-represented. Pick the least-practiced type instead.
    const typeCounts = new Map<RoundType, number>();
    last10.forEach(h => {
      typeCounts.set(h.roundType, (typeCounts.get(h.roundType) || 0) + 1);
    });

    let leastPracticed: RoundType = selected;
    let minCount = Infinity;

    for (const rt of ALL_ROUND_TYPES) {
      const count = typeCounts.get(rt) || 0;
      if (count < minCount) {
        minCount = count;
        leastPracticed = rt;
      }
    }

    return leastPracticed;
  }

  return selected;
}

---

Adaptive Difficulty

Difficulty adjusts based on performance trends over the last 5 sessions of the same round type. The algorithm smooths noise by using a moving average and only adjusts when the trend is clear.

interface DifficultyState {
  roundType: RoundType;
  currentLevel: 1 | 2 | 3 | 4 | 5;
  recentScores: number[];     // Last 5 composite scores for this round type
  trend: 'improving' | 'stable' | 'declining';
}

function adjustDifficulty(state: DifficultyState): DifficultyState {
  const { recentScores, currentLevel } = state;

  if (recentScores.length < 3) {
    // Not enough data to adjust. Stay at current level.
    return { ...state, trend: 'stable' };
  }

  // Calculate trend: compare first half to second half of recent scores
  const midpoint = Math.floor(recentScores.length / 2);
  const firstHalf = recentScores.slice(0, midpoint);
  const secondHalf = recentScores.slice(midpoint);

  const firstAvg = firstHalf.reduce((a, b) => a + b, 0) / firstHalf.length;
  const secondAvg = secondHalf.reduce((a, b) => a + b, 0) / secondHalf.length;
  const delta = secondAvg - firstAvg;

  // Significant improvement (delta > 10): increase difficulty
  if (delta > 10 && currentLevel < 5) {
    return {
      ...state,
      currentLevel: (currentLevel + 1) as DifficultyState['currentLevel'],
      trend: 'improving',
    };
  }

  // Significant decline (delta < -10): decrease difficulty
  if (delta < -10 && currentLevel > 1) {
    return {
      ...state,
      currentLevel: (currentLevel - 1) as DifficultyState['currentLevel'],
      trend: 'declining',
    };
  }

  // Consistently high scores (avg > 85): increase difficulty
  const overallAvg = recentScores.reduce((a, b) => a + b, 0) / recentScores.length;
  if (overallAvg > 85 && currentLevel < 5) {
    return {
      ...state,
      currentLevel: (currentLevel + 1) as DifficultyState['currentLevel'],
      trend: 'improving',
    };
  }

  // Consistently low scores (avg < 40): decrease difficulty
  if (overallAvg < 40 && currentLevel > 1) {
    return {
      ...state,
      currentLevel: (currentLevel - 1) as DifficultyState['currentLevel'],
      trend: 'declining',
    };
  }

  return { ...state, trend: 'stable' };
}

What Difficulty Levels Control

---

Performance Tracking Schema

Database Schema (Supabase / SQLite)

-- Core session record
CREATE TABLE sessions (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID REFERENCES auth.users(id),
  round_type TEXT NOT NULL CHECK (round_type IN (
    'coding', 'ml_design', 'behavioral',
    'tech_presentation', 'hiring_manager', 'anthropic_technical'
  )),
  difficulty_level INTEGER NOT NULL CHECK (difficulty_level BETWEEN 1 AND 5),
  interviewer_persona TEXT NOT NULL CHECK (interviewer_persona IN (
    'friendly', 'neutral', 'adversarial', 'socratic'
  )),
  proctor_mode TEXT CHECK (proctor_mode IN ('off', 'training', 'simulation')),
  session_length_minutes INTEGER NOT NULL,
  started_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  ended_at TIMESTAMPTZ,
  composite_score NUMERIC(5,2),
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

-- Per-dimension scores for each session
CREATE TABLE session_scores (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  session_id UUID REFERENCES sessions(id) ON DELETE CASCADE,
  dimension TEXT NOT NULL CHECK (dimension IN (
    'technical_accuracy', 'communication_clarity', 'time_management',
    'structured_thinking', 'composure_under_pressure',
    'question_handling', 'proctor_compliance'
  )),
  score NUMERIC(5,2) NOT NULL CHECK (score BETWEEN 0 AND 100),
  weight NUMERIC(3,2) NOT NULL,
  evidence TEXT,  -- Specific examples from the session
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

-- Full debrief for each session
CREATE TABLE debriefs (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  session_id UUID REFERENCES sessions(id) ON DELETE CASCADE,
  transcript TEXT,              -- Full session transcript
  emotion_timeline JSONB,      -- Array of {timestamp, emotions}
  proctor_summary JSONB,       -- ProctorSummary object
  whiteboard_evaluations JSONB, -- Array of evaluation results
  strengths TEXT[],
  weaknesses TEXT[],
  improvement_actions TEXT[],
  debrief_read BOOLEAN DEFAULT FALSE,
  debrief_read_at TIMESTAMPTZ,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

-- Audio/video recordings
CREATE TABLE recordings (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  session_id UUID REFERENCES sessions(id) ON DELETE CASCADE,
  recording_type TEXT NOT NULL CHECK (recording_type IN ('audio', 'video')),
  storage_path TEXT NOT NULL,   -- Supabase storage path
  duration_seconds INTEGER,
  file_size_bytes BIGINT,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

-- Story bank for behavioral rounds
CREATE TABLE stories (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID REFERENCES auth.users(id),
  title TEXT NOT NULL,
  project_name TEXT NOT NULL,
  category TEXT NOT NULL CHECK (category IN (
    'failure', 'disagreement', 'changed_opinion', 'ethical_tradeoff',
    'mentorship', 'ambiguity', 'someone_else_right', 'mission'
  )),
  situation TEXT,
  task TEXT,
  action TEXT,
  result TEXT,
  learning TEXT,
  follow_up_depth JSONB,       -- Prepared answers for 6 levels of follow-up
  last_rehearsed TIMESTAMPTZ,
  rehearsal_count INTEGER DEFAULT 0,
  -- SM-2 fields for story rehearsal scheduling
  ease_factor NUMERIC(3,2) DEFAULT 2.5,
  interval_days INTEGER DEFAULT 1,
  repetitions INTEGER DEFAULT 0,
  next_review_date DATE,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

-- Flash cards with SM-2 scheduling
CREATE TABLE flash_cards (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID REFERENCES auth.users(id),
  category TEXT NOT NULL CHECK (category IN (
    'ml_concepts', 'system_design', 'anthropic_specific',
    'behavioral_stories', 'coding_patterns', 'company_knowledge'
  )),
  content TEXT NOT NULL,        -- Question
  answer TEXT,                  -- Expected answer
  round_type TEXT,              -- Associated interview round
  -- SM-2 fields
  ease_factor NUMERIC(3,2) DEFAULT 2.5,
  interval_days INTEGER DEFAULT 1,
  repetitions INTEGER DEFAULT 0,
  next_review_date DATE DEFAULT CURRENT_DATE,
  last_review_date DATE,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

-- Weakness tracker (aggregated from sessions)
CREATE TABLE weakness_tracker (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID REFERENCES auth.users(id),
  round_type TEXT NOT NULL,
  dimension TEXT NOT NULL,
  rolling_avg_score NUMERIC(5,2),  -- 5-session rolling average
  trend TEXT CHECK (trend IN ('improving', 'stable', 'declining')),
  sessions_counted INTEGER DEFAULT 0,
  last_updated TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  UNIQUE(user_id, round_type, dimension)
);

-- Indexes for common queries
CREATE INDEX idx_sessions_user_round ON sessions(user_id, round_type, started_at DESC);
CREATE INDEX idx_flash_cards_due ON flash_cards(user_id, next_review_date);
CREATE INDEX idx_stories_review ON stories(user_id, next_review_date);
CREATE INDEX idx_weakness_tracker ON weakness_tracker(user_id);

---

SM-2 Algorithm Details

The SuperMemo 2 algorithm is used for two purposes: 1. Flash card scheduling: When to review each concept card 2. Story rehearsal scheduling: When to practice each behavioral story

Algorithm Flow

Input: card state (ease_factor, interval, repetitions), quality grade (0-5)

If quality >= 3 (correct response):
  if repetitions == 0: interval = 1 day
  if repetitions == 1: interval = 6 days
  if repetitions >= 2: interval = round(interval * ease_factor)
  repetitions += 1
Else (incorrect response):
  repetitions = 0
  interval = 1 day

Update ease_factor:
  ease_factor += 0.1 - (5 - quality) * (0.08 + (5 - quality) * 0.02)
  ease_factor = max(1.3, ease_factor)

next_review_date = today + interval days

Quality Grade Definitions

For flash cards:

• 5: Perfect recall, no hesitation
• 4: Correct after brief hesitation
• 3: Correct with difficulty
• 2: Incorrect, but answer seemed familiar
• 1: Incorrect, vague memory of answer
• 0: Complete blackout, no idea

For story rehearsals:

• 5: Delivered story smoothly in <3 min, hit all STAR-L beats, no filler words
• 4: Delivered correctly but with some hesitation or minor structural issues
• 3: Covered the main points but missed beats or went over time
• 2: Struggled to recall key details, story was incomplete
• 1: Could only recall the general topic, not the specific details
• 0: Could not recall the story at all

Interval Progression Example

A card starting with ease_factor=2.5 and perfect recall (quality=5) every time:

A card with quality=3 (difficulty) reduces ease_factor, leading to shorter intervals:

A card with a single failure (quality=1) resets to interval=1 day but retains the (reduced) ease factor.

---

Debrief Generation

What the Debrief Generator Receives

After each session, the orchestrator compiles all available data and sends it to Claude for debrief generation:

interface DebriefInput {
  // Session metadata
  roundType: RoundType;
  difficultyLevel: number;
  interviewerPersona: string;
  sessionLengthMinutes: number;

  // Voice engine data
  transcript: string;                    // Full conversation transcript
  emotionTimeline: EmotionCallback[];    // Timestamped emotion data

  // Proctor data (if enabled)
  proctorSummary: ProctorSummary | null;

  // Whiteboard data (if design round)
  whiteboardEvaluations: Array<{
    timestamp: number;
    evaluation: PeriodicCheckResult | FinalEvaluation;
  }> | null;

  // Historical context
  recentScores: Array<{
    roundType: RoundType;
    compositeScore: number;
    date: string;
  }>;
  knownWeaknesses: string[];
}

Debrief Generation Prompt

const DEBRIEF_PROMPT = `You are an expert interview coach generating a post-session debrief.

## Session Data

Round type: {ROUND_TYPE}
Difficulty: {DIFFICULTY}/5
Persona: {PERSONA}
Duration: {DURATION} minutes

## Transcript
{TRANSCRIPT}

## Emotion Timeline
{EMOTION_DATA}

## Proctor Summary
{PROCTOR_SUMMARY}

## Whiteboard Evaluations
{WHITEBOARD_EVALS}

## Known Weaknesses from Previous Sessions
{KNOWN_WEAKNESSES}

---

Generate a comprehensive debrief. Be specific and actionable. Reference exact moments from the transcript. Do not be generically positive.

Respond with JSON:
{
  "composite_score": <0-100>,
  "dimension_scores": {
    "technical_accuracy": {
      "score": <0-100>,
      "evidence": "Specific moment from transcript...",
      "compared_to_last": "improved | stable | declined"
    },
    "communication_clarity": { ... },
    "time_management": { ... },
    "structured_thinking": { ... },
    "composure_under_pressure": { ... },
    "question_handling": { ... },
    "proctor_compliance": { ... }
  },
  "strengths": [
    "Specific strength with transcript reference (at minute X, you said...)"
  ],
  "weaknesses": [
    "Specific weakness with transcript reference"
  ],
  "improvement_actions": [
    {
      "action": "Practice X by doing Y",
      "priority": "high | medium | low",
      "estimated_sessions_to_improve": 3,
      "related_dimension": "technical_accuracy"
    }
  ],
  "emotion_insights": [
    "At minute 12, nervousness spiked when asked about distributed systems. This suggests a knowledge gap rather than a performance issue."
  ],
  "proctor_notes": [
    "2 gaze deviations in the first 10 minutes, then clean for the remainder. Opening jitters appear to cause note-checking behavior."
  ],
  "whiteboard_notes": [
    "Strong component diagram. Missing monitoring layer until prompted at minute 25. Data flow arrows added late."
  ],
  "next_session_recommendation": {
    "round_type": "ml_design",
    "difficulty": 3,
    "focus_area": "monitoring and observability discussion",
    "persona": "neutral"
  }
}

IMPORTANT:
- Reference specific timestamps and quotes from the transcript
- Do not say "good job" without evidence
- Every weakness must have a corresponding improvement action
- The next session recommendation must be based on data, not rotation
- If emotion data shows a pattern (e.g., anxiety increases during specific topics), call it out explicitly`;

Debrief Generation Implementation

async function generateDebrief(input: DebriefInput): Promise<Debrief> {
  const prompt = DEBRIEF_PROMPT
    .replace('{ROUND_TYPE}', input.roundType)
    .replace('{DIFFICULTY}', String(input.difficultyLevel))
    .replace('{PERSONA}', input.interviewerPersona)
    .replace('{DURATION}', String(input.sessionLengthMinutes))
    .replace('{TRANSCRIPT}', truncateTranscript(input.transcript, 8000))
    .replace('{EMOTION_DATA}', summarizeEmotions(input.emotionTimeline))
    .replace('{PROCTOR_SUMMARY}', JSON.stringify(input.proctorSummary, null, 2))
    .replace('{WHITEBOARD_EVALS}', JSON.stringify(input.whiteboardEvaluations, null, 2))
    .replace('{KNOWN_WEAKNESSES}', input.knownWeaknesses.join('\n- '));

  const response = await anthropic.messages.create({
    model: 'claude-sonnet-4-20250514',
    max_tokens: 2000,
    messages: [{ role: 'user', content: prompt }],
  });

  const debrief = JSON.parse(response.content[0].text);

  // Store the debrief
  await supabase.from('debriefs').insert({
    session_id: input.sessionId,
    transcript: input.transcript,
    emotion_timeline: input.emotionTimeline,
    proctor_summary: input.proctorSummary,
    whiteboard_evaluations: input.whiteboardEvaluations,
    strengths: debrief.strengths,
    weaknesses: debrief.weaknesses,
    improvement_actions: debrief.improvement_actions,
  });

  return debrief;
}

// Truncate transcript to fit in context window while preserving key moments
function truncateTranscript(transcript: string, maxChars: number): string {
  if (transcript.length <= maxChars) return transcript;

  // Keep first 5 minutes, last 5 minutes, and sample from middle
  const lines = transcript.split('\n');
  const totalLines = lines.length;

  const headLines = lines.slice(0, Math.floor(totalLines * 0.2));
  const tailLines = lines.slice(Math.floor(totalLines * 0.8));

  // Sample evenly from middle
  const middleLines = lines.slice(
    Math.floor(totalLines * 0.2),
    Math.floor(totalLines * 0.8)
  );
  const middleSample = middleLines.filter((_, i) => i % 3 === 0); // Every 3rd line

  const truncated = [
    ...headLines,
    '\n[...transcript truncated for debrief generation...]\n',
    ...middleSample,
    '\n[...]\n',
    ...tailLines,
  ].join('\n');

  return truncated.slice(0, maxChars);
}

// Summarize emotion timeline into readable format
function summarizeEmotions(timeline: EmotionCallback[]): string {
  if (timeline.length === 0) return 'Emotion data unavailable (fallback voice engine used)';

  // Compute per-minute summaries
  const minuteBuckets = new Map<number, EmotionCallback[]>();
  const sessionStart = timeline[0].timestamp;

  timeline.forEach(entry => {
    const minute = Math.floor((entry.timestamp - sessionStart) / 60_000);
    if (!minuteBuckets.has(minute)) minuteBuckets.set(minute, []);
    minuteBuckets.get(minute)!.push(entry);
  });

  const summaries: string[] = [];
  minuteBuckets.forEach((entries, minute) => {
    const avgEmotions = computeAverageEmotions(entries);
    const dominant = avgEmotions.sort((a, b) => b.score - a.score)[0];
    if (dominant.score > 0.4) {
      summaries.push(`Minute ${minute}: dominant emotion "${dominant.name}" (${(dominant.score * 100).toFixed(0)}%)`);
    }
  });

  return summaries.join('\n');
}

---

Weakness Detection

Cross-Session Pattern Analysis

Run weekly (or after every 5 sessions) to identify persistent weaknesses:

interface WeaknessPattern {
  roundType: RoundType;
  dimension: string;
  avgScore: number;
  trend: 'improving' | 'stable' | 'declining';
  sessionsAnalyzed: number;
  specificExamples: string[];   // From debrief improvement_actions
  recommendedAction: string;
}

async function detectWeaknessPatterns(userId: string): Promise<WeaknessPattern[]> {
  // Get all session scores from last 30 days
  const { data: scores } = await supabase
    .from('session_scores')
    .select(`
      *,
      sessions!inner(round_type, started_at, user_id)
    `)
    .eq('sessions.user_id', userId)
    .gte('sessions.started_at', thirtyDaysAgo())
    .order('sessions.started_at', { ascending: false });

  // Group by round_type + dimension
  const groups = new Map<string, typeof scores>();
  scores?.forEach(score => {
    const key = `${score.sessions.round_type}:${score.dimension}`;
    if (!groups.has(key)) groups.set(key, []);
    groups.get(key)!.push(score);
  });

  const patterns: WeaknessPattern[] = [];

  groups.forEach((groupScores, key) => {
    const [roundType, dimension] = key.split(':');
    const numericScores = groupScores.map(s => Number(s.score));

    if (numericScores.length < 3) return; // Need at least 3 data points

    const avgScore = numericScores.reduce((a, b) => a + b, 0) / numericScores.length;

    // Only flag as weakness if below 65
    if (avgScore >= 65) return;

    // Calculate trend
    const midpoint = Math.floor(numericScores.length / 2);
    const firstHalfAvg = numericScores.slice(0, midpoint).reduce((a, b) => a + b, 0) / midpoint;
    const secondHalfAvg = numericScores.slice(midpoint).reduce((a, b) => a + b, 0) / (numericScores.length - midpoint);
    const trend = secondHalfAvg - firstHalfAvg > 5 ? 'improving'
      : secondHalfAvg - firstHalfAvg < -5 ? 'declining'
      : 'stable';

    patterns.push({
      roundType: roundType as RoundType,
      dimension,
      avgScore,
      trend,
      sessionsAnalyzed: numericScores.length,
      specificExamples: [], // Populated from debriefs
      recommendedAction: generateRecommendation(roundType as RoundType, dimension, avgScore, trend),
    });
  });

  // Sort by severity: declining + low score first
  patterns.sort((a, b) => {
    const severityA = (100 - a.avgScore) + (a.trend === 'declining' ? 20 : 0);
    const severityB = (100 - b.avgScore) + (b.trend === 'declining' ? 20 : 0);
    return severityB - severityA;
  });

  // Update weakness_tracker table
  for (const pattern of patterns) {
    await supabase.from('weakness_tracker').upsert({
      user_id: userId,
      round_type: pattern.roundType,
      dimension: pattern.dimension,
      rolling_avg_score: pattern.avgScore,
      trend: pattern.trend,
      sessions_counted: pattern.sessionsAnalyzed,
      last_updated: new Date().toISOString(),
    }, { onConflict: 'user_id,round_type,dimension' });
  }

  return patterns;
}

function generateRecommendation(
  roundType: RoundType,
  dimension: string,
  avgScore: number,
  trend: string,
): string {
  const recommendations: Record<string, Record<string, string>> = {
    technical_accuracy: {
      coding: 'Practice 2 additional coding problems per week from the problem archetypes list. Focus on tracing through examples aloud.',
      ml_design: 'Review the 7-stage framework and practice filling each stage with specific numbers and trade-offs.',
      behavioral: 'Ensure STAR-L stories have concrete metrics and technical details, not just narrative.',
      default: 'Spend 20 minutes reviewing fundamentals before each practice session.',
    },
    communication_clarity: {
      default: 'Record yourself answering questions and listen for filler words, long pauses, and unclear transitions. Practice the "intent before code" narration pattern.',
    },
    time_management: {
      coding: 'Use the 5/20/10/5 minute budget strictly. Set a timer for each phase.',
      ml_design: 'Use the 5/3/7/5/8/8/5 minute budget. Practice with a visible timer.',
      default: 'Set explicit time targets for each section and practice with a visible countdown.',
    },
    structured_thinking: {
      ml_design: 'Always start with the 7-stage framework, even if the problem seems to need a different structure.',
      default: 'Before answering, state your structure: "I will cover three things: first X, then Y, then Z."',
    },
    composure_under_pressure: {
      default: 'Practice at one difficulty level higher than comfortable. The discomfort IS the training. Use the adversarial persona more often.',
    },
    question_handling: {
      behavioral: 'Practice the follow-up ladder to level 6 for each story. If you can only reach level 3, the story is not ready.',
      default: 'When asked a question you do not know, practice saying "I am not sure about X, but here is how I would approach figuring it out."',
    },
    proctor_compliance: {
      default: 'Practice with proctor in simulation mode. Clear your desk, close extra tabs, and put your phone face-down before starting.',
    },
  };

  const dimRecs = recommendations[dimension] || {};
  return dimRecs[roundType] || dimRecs['default'] || 'Review the relevant skill reference files for this round type.';
}

---

Preparation Plan Adjustment

Weakness data feeds back into the practice schedule. The orchestrator generates a weekly plan that prioritizes weak areas:

interface WeeklyPlan {
  week: string;  // ISO week
  sessions: Array<{
    dayOfWeek: number;  // 0=Sun, 1=Mon, ...
    roundType: RoundType;
    difficulty: number;
    focus: string;
    estimatedMinutes: number;
  }>;
  morningDrills: {
    cardCategories: CardCategory[];  // Prioritized by weakness
    storiesToRehearse: string[];     // Story IDs due for rehearsal
  };
  rationale: string;
}

function generateWeeklyPlan(
  weaknesses: WeaknessPattern[],
  availableDays: number[],  // Which days the user practices
  sessionsPerWeek: number,
): WeeklyPlan {
  const sessions = [];

  // Allocate sessions based on weakness severity
  // Top weakness gets 40% of sessions, second gets 30%, rest split evenly
  const topWeakness = weaknesses[0];
  const secondWeakness = weaknesses[1];

  const topCount = Math.ceil(sessionsPerWeek * 0.4);
  const secondCount = Math.ceil(sessionsPerWeek * 0.3);
  const otherCount = sessionsPerWeek - topCount - secondCount;

  // Assign top weakness sessions
  for (let i = 0; i < topCount && i < availableDays.length; i++) {
    sessions.push({
      dayOfWeek: availableDays[i],
      roundType: topWeakness.roundType,
      difficulty: topWeakness.trend === 'declining' ? 2 : 3,
      focus: topWeakness.dimension,
      estimatedMinutes: 45,
    });
  }

  // Assign second weakness sessions
  if (secondWeakness) {
    for (let i = topCount; i < topCount + secondCount && i < availableDays.length; i++) {
      sessions.push({
        dayOfWeek: availableDays[i],
        roundType: secondWeakness.roundType,
        difficulty: 3,
        focus: secondWeakness.dimension,
        estimatedMinutes: 45,
      });
    }
  }

  // Fill remaining with variety
  const remainingTypes = ALL_ROUND_TYPES.filter(
    rt => rt !== topWeakness?.roundType && rt !== secondWeakness?.roundType
  );

  for (let i = topCount + secondCount; i < sessionsPerWeek && i < availableDays.length; i++) {
    sessions.push({
      dayOfWeek: availableDays[i],
      roundType: remainingTypes[i % remainingTypes.length],
      difficulty: 3,
      focus: 'general',
      estimatedMinutes: 30,
    });
  }

  return {
    week: getCurrentISOWeek(),
    sessions,
    morningDrills: {
      cardCategories: weaknesses
        .map(w => roundTypeToCardCategory(w.roundType))
        .filter(Boolean) as CardCategory[],
      storiesToRehearse: [], // Populated from SM-2 schedule
    },
    rationale: `Focus on ${topWeakness?.roundType} (${topWeakness?.dimension}: avg ${topWeakness?.avgScore?.toFixed(0)}, ${topWeakness?.trend}) and ${secondWeakness?.roundType || 'variety'}.`,
  };
}

---

Session Lifecycle Summary

1. User opens app → Orchestrator selects round type (weakness-weighted)
2. User confirms or overrides → Orchestrator sets difficulty (adaptive)
3. Session starts:
   a. Voice engine connects (Hume AI or ElevenLabs fallback)
   b. Proctor activates (if enabled)
   c. Whiteboard opens (if design round)
4. During session:
   a. Voice AI drives conversation using round-type-specific prompt
   b. Emotion data streams from Hume → orchestrator adjusts tone/pace
   c. Whiteboard snapshots captured periodically → Claude Vision evaluates
   d. Proctor tracks gaze → flags logged silently (training) or announced (simulation)
5. Session ends:
   a. Final whiteboard capture and evaluation
   b. Emotion timeline compiled
   c. Proctor summary generated
   d. All data sent to debrief generator (Claude)
6. Debrief generated:
   a. Per-dimension scores with evidence
   b. Strengths, weaknesses, improvement actions
   c. Emotion insights, proctor notes, whiteboard notes
   d. Next session recommendation
7. Post-session:
   a. Weakness tracker updated
   b. SM-2 intervals adjusted for related flash cards
   c. Weekly plan recalculated if significant change
   d. Data synced to mobile app

Voice Engine Setup

Integration guide for Hume AI EVI as the primary voice engine, with ElevenLabs as a fallback for cost-constrained or mobile scenarios.

---

Hume AI EVI Integration

Why Hume AI

Hume's Empathic Voice Interface (EVI) provides two capabilities no other voice API offers in combination: 1. Real-time emotion detection from user speech (nervousness, confidence, hesitation, frustration) 2. Expression-aware TTS that adjusts the interviewer voice based on conversation state

For interview simulation, this enables the core differentiator: an AI interviewer that responds to your emotional state the way a real interviewer does -- pressing harder when you seem overconfident, offering encouragement when you are clearly struggling.

API Setup

# Install the Hume SDK
npm install hume

# Environment variables
HUME_API_KEY=your_api_key
HUME_SECRET_KEY=your_secret_key

WebSocket Connection

EVI uses a persistent WebSocket for real-time voice streaming. The connection lifecycle:

import { HumeClient } from 'hume';

interface EmotionScore {
  name: string;    // "Nervousness", "Confidence", "Hesitation", etc.
  score: number;   // 0.0 - 1.0
}

interface EmotionCallback {
  emotions: EmotionScore[];
  timestamp: number;
  text: string;  // Transcribed user speech
}

class VoiceEngine {
  private client: HumeClient;
  private socket: WebSocket | null = null;
  private persona: InterviewerPersona;
  private emotionHistory: EmotionCallback[] = [];

  constructor(apiKey: string, secretKey: string) {
    this.client = new HumeClient({ apiKey, secretKey });
    this.persona = PERSONAS.neutral;
  }

  async connect(sessionId: string): Promise<void> {
    // Authenticate and get a WebSocket URL
    const auth = await this.client.expressionMeasurement.stream.connect({
      config: {
        language: { granularity: 'sentence' },
        prosody: {},  // Enable prosody (tone) analysis
        face: {},     // Enable facial expression (if video enabled)
      }
    });

    this.socket = auth.socket;

    this.socket.onmessage = (event) => {
      const data = JSON.parse(event.data);
      if (data.type === 'expression_measurement') {
        this.handleEmotionData(data, sessionId);
      }
    };
  }

  private handleEmotionData(data: any, sessionId: string): void {
    const emotions: EmotionScore[] = data.prosody?.predictions?.[0]?.emotions || [];
    const callback: EmotionCallback = {
      emotions,
      timestamp: Date.now(),
      text: data.transcript || '',
    };

    this.emotionHistory.push(callback);
    this.adjustPersonaBehavior(callback);
  }

  private adjustPersonaBehavior(latest: EmotionCallback): void {
    const nervousness = latest.emotions.find(e => e.name === 'Anxiety')?.score || 0;
    const confidence = latest.emotions.find(e => e.name === 'Determination')?.score || 0;
    const hesitation = latest.emotions.find(e => e.name === 'Doubt')?.score || 0;

    // Adaptive interviewer behavior
    if (nervousness > 0.7 && this.persona.adaptiveMode) {
      // High nervousness: slow down, offer scaffolding
      this.setTone('encouraging');
      this.setFollowUpDelay(3000); // 3-second pause before follow-up
    } else if (confidence > 0.8 && hesitation < 0.2) {
      // Overconfident: press harder, ask for specifics
      this.setTone('probing');
      this.setFollowUpDelay(500); // Quick follow-up
    } else if (hesitation > 0.6) {
      // Struggling: give more time, rephrase question
      this.setTone('supportive');
      this.setFollowUpDelay(5000); // 5-second pause
    }
  }

  getEmotionTimeline(): EmotionCallback[] {
    return this.emotionHistory;
  }

  async disconnect(): Promise<void> {
    if (this.socket) {
      this.socket.close();
      this.socket = null;
    }
  }

  private setTone(tone: 'encouraging' | 'probing' | 'supportive' | 'neutral'): void {
    // Update the system prompt sent with the next TTS request
    this.persona.currentTone = tone;
  }

  private setFollowUpDelay(ms: number): void {
    this.persona.followUpDelayMs = ms;
  }
}

Interviewer Persona Prompts

Each persona controls the system prompt that drives the voice AI's conversational behavior.

Friendly Persona

const PERSONAS = {
  friendly: {
    name: 'Friendly',
    adaptiveMode: true,
    currentTone: 'encouraging' as const,
    followUpDelayMs: 2000,
    systemPrompt: `You are a supportive interviewer conducting a mock interview.
Your goal is to help the candidate practice while building their confidence.

Behavior:
- Start with a warm greeting and explain what you'll cover
- After each answer, acknowledge what was strong before asking follow-ups
- If the candidate struggles, rephrase the question or offer a starting point
- Use phrases like "That's a good start, let me ask you to go deeper on..."
- Smile through your voice -- warm, encouraging tone
- Still push for depth, but frame it as curiosity, not testing
- If time is running short, say "Let's move to the next area" rather than cutting off

Do NOT:
- Accept surface-level answers without follow-up
- Be so encouraging that the candidate doesn't get realistic feedback
- Skip difficult topics to avoid discomfort`,
  },

  neutral: {
    name: 'Neutral',
    adaptiveMode: true,
    currentTone: 'neutral' as const,
    followUpDelayMs: 1500,
    systemPrompt: `You are a professional interviewer conducting a mock interview.
Your tone is neutral and businesslike -- neither warm nor cold.

Behavior:
- Brief introduction, then directly into questions
- After each answer, move to the next question or follow-up without extensive commentary
- Ask follow-ups that probe depth: "Can you be more specific about X?"
- Allow natural silence (2-3 seconds) after the candidate finishes before responding
- If the candidate gives a vague answer, ask for a concrete example
- Keep time management tight -- redirect if answers run long

Do NOT:
- Provide coaching or feedback during the session
- React emotionally to good or bad answers
- Hint at whether an answer was correct`,
  },

  adversarial: {
    name: 'Adversarial',
    adaptiveMode: false,  // No adaptation -- stays tough regardless
    currentTone: 'probing' as const,
    followUpDelayMs: 500,
    systemPrompt: `You are a rigorous interviewer who pushes candidates to their limits.
Your goal is to find the edges of the candidate's knowledge.

Behavior:
- Minimal pleasantries, get right to it
- After every answer, ask "Why?" or "What about [edge case]?" or "That seems over-engineered"
- Challenge assumptions: "Are you sure about that number?"
- If the candidate says something incorrect, press: "I'm not sure that's right. Can you walk me through it again?"
- Interrupt long-winded answers: "Let me stop you there -- what's the core point?"
- Ask follow-ups that require the candidate to defend trade-offs
- Maintain professional respect -- tough, not rude

Do NOT:
- Be personally disrespectful or demeaning
- Accept "I don't know" without asking "What would you do to find out?"
- Let the candidate redirect away from a question they're struggling with`,
  },

  socratic: {
    name: 'Socratic',
    adaptiveMode: true,
    currentTone: 'neutral' as const,
    followUpDelayMs: 2000,
    systemPrompt: `You are a Socratic interviewer who guides through questions rather than evaluating answers.
Your goal is to help the candidate discover gaps in their own thinking.

Behavior:
- Ask open-ended questions that expose assumptions
- When the candidate makes a claim, ask "What would need to be true for that to work?"
- Use "What if..." scenarios to probe edge cases
- When they identify a good trade-off, ask "How would you decide between those options?"
- Build on their answers: "You mentioned X -- how does that interact with Y?"
- Allow long thinking pauses without filling the silence
- Guide them toward insights rather than telling them the answer

Do NOT:
- Give the answer directly
- Say "that's wrong" -- instead ask questions that reveal the issue
- Rush through topics -- depth over breadth`,
  },
};

Round-Type Specific Voice Behavior

The interviewer persona is further specialized based on the round type:

Coding Round Voice Behavior

const CODING_ROUND_OVERLAY = {
  additionalPrompt: `During this coding interview:
- Start by reading the problem statement clearly
- After reading, pause 5 seconds for the candidate to process
- When they start coding, reduce interruptions -- let them work
- If they go silent for >60 seconds, ask "What are you thinking about?"
- When they finish a section, ask "Walk me through what this does"
- For follow-up extensions, say "Now let's add [requirement]"
- If they're stuck, offer ONE hint: "What data structure would give you O(1) lookup?"
- Do NOT give implementation hints -- only structural hints`,
  voiceSettings: {
    speed: 0.9,  // Slightly slower for clarity when reading problems
    stability: 0.8,
  },
};

Behavioral Round Voice Behavior

const BEHAVIORAL_ROUND_OVERLAY = {
  additionalPrompt: `During this behavioral interview:
- Ask the initial question, then LISTEN
- Allow the candidate to speak for 2-3 minutes before any follow-up
- Follow the depth ladder: Surface -> Context -> Decision -> Tradeoff -> Meta-Reflection -> Worldview
- Use silence as a tool -- a 3-second pause after they finish often prompts deeper reflection
- If they give a surface answer, ask "Can you tell me more about what was going through your mind?"
- If they avoid the negative framing, gently redirect: "I appreciate the positive outcome, but I'm curious about what was hard about this"
- Track which level of the follow-up ladder they've reached`,
  voiceSettings: {
    speed: 1.0,
    stability: 0.6,  // More natural variation for conversational tone
  },
};

System Design Round Voice Behavior

const DESIGN_ROUND_OVERLAY = {
  additionalPrompt: `During this system design interview:
- Present the problem, then ask "How would you approach this?"
- This is collaborative -- you are working together, not testing
- When they draw on the whiteboard, narrate what you see: "I see you're adding a cache layer here..."
- Ask probing questions about each component: "Why did you choose X over Y?"
- If they skip a critical component (monitoring, data pipeline), prompt: "What happens after deployment?"
- At the halfway mark, summarize what they've covered and what's remaining
- In the last 5 minutes, ask "What would you change if you had more time?"`,
  voiceSettings: {
    speed: 1.0,
    stability: 0.7,
  },
};

---

Emotion-Adaptive Logic

Emotion Categories Tracked

Hume AI provides scores for 48 emotion categories. For interview simulation, these are the most relevant:

Composite State Detection

Individual emotions are noisy. The engine computes composite states for more reliable adaptation:

interface CompositeState {
  state: 'flow' | 'struggling' | 'overconfident' | 'anxious' | 'neutral';
  confidence: number;
  signals: string[];
}

function computeCompositeState(emotions: EmotionScore[]): CompositeState {
  const anxiety = getScore(emotions, 'Anxiety');
  const determination = getScore(emotions, 'Determination');
  const doubt = getScore(emotions, 'Doubt');
  const concentration = getScore(emotions, 'Concentration');
  const frustration = getScore(emotions, 'Frustration');
  const excitement = getScore(emotions, 'Excitement');

  // Flow: high concentration + excitement, low anxiety
  if (concentration > 0.6 && excitement > 0.4 && anxiety < 0.3) {
    return { state: 'flow', confidence: 0.8, signals: ['high focus', 'engaged'] };
  }

  // Struggling: high doubt + frustration, low determination
  if ((doubt > 0.5 || frustration > 0.5) && determination < 0.3) {
    return { state: 'struggling', confidence: 0.7, signals: ['high doubt', 'low confidence'] };
  }

  // Overconfident: high determination, low doubt, fast speech
  if (determination > 0.7 && doubt < 0.2 && anxiety < 0.2) {
    return { state: 'overconfident', confidence: 0.6, signals: ['very confident', 'low self-doubt'] };
  }

  // Anxious: high anxiety, moderate to high doubt
  if (anxiety > 0.6) {
    return { state: 'anxious', confidence: 0.75, signals: ['high anxiety'] };
  }

  return { state: 'neutral', confidence: 0.5, signals: [] };
}

---

ElevenLabs Fallback

When Hume AI is unavailable (API outage, cost constraints, mobile use), fall back to ElevenLabs for voice-only interaction without emotion detection.

Setup

npm install elevenlabs

# Environment variable
ELEVENLABS_API_KEY=your_api_key

Fallback Implementation

import { ElevenLabsClient } from 'elevenlabs';

class ElevenLabsFallback {
  private client: ElevenLabsClient;
  private voiceId: string;

  constructor(apiKey: string) {
    this.client = new ElevenLabsClient({ apiKey });
    // "Rachel" voice -- professional, clear, neutral tone
    this.voiceId = '21m00Tcm4TlvDq8ikWAM';
  }

  async speak(text: string): Promise<ReadableStream> {
    const audio = await this.client.generate({
      voice: this.voiceId,
      text,
      model_id: 'eleven_multilingual_v2',
      voice_settings: {
        stability: 0.7,
        similarity_boost: 0.8,
        style: 0.3,
      },
    });
    return audio;
  }

  // No emotion detection -- persona adaptation is disabled
  getEmotionTimeline(): EmotionCallback[] {
    return []; // Empty -- debrief will note "emotion data unavailable"
  }
}

When to Use Each

---

Cost Analysis and Optimization

Hume AI Pricing (as of early 2026)

• ~$0.07/minute of streaming voice interaction
• Emotion analysis included in the per-minute rate
• WebSocket connection has no idle cost (only active voice time billed)

Cost Reduction Strategies

1. Hard session time limits: Enforce the configured session length server-side. A 45-minute session costs ~$3.15 in Hume fees. An unbounded session that runs 90 minutes costs double.

2. Silence detection: Pause billing during long thinking pauses (>10s). Hume charges for active audio streaming -- detect silence client-side and mute the stream.

3. Morning drills on ElevenLabs: 30 morning drills at 3 min each = 90 min. On Hume: $6.30. On ElevenLabs: ~$4.50. Save $1.80/month and emotion detection adds little value for 3-minute story rehearsals.

4. Cache common interviewer responses: For standard phrases ("Tell me about a time when...", "Can you go deeper on that?", "Let's move to the next question"), pre-generate audio with ElevenLabs and serve from cache. Only use live Hume for adaptive responses.

5. Session recording for review: Record sessions locally. Candidates can replay and self-evaluate without running a new voice session. Cost: $0 for replay vs $3+ for a new live session.

Whiteboard Engine Setup

Integration guide for tldraw as the collaborative whiteboard, with Claude Vision API for automated diagram evaluation and scoring.

---

tldraw React Integration

Why tldraw

tldraw is an open-source (MIT license) React drawing component with a rich programmatic API. Key advantages for interview simulation:

1. React-native component: Drops into a Next.js app with zero friction 2. Programmatic snapshot: editor.getSnapshot() captures full canvas state; exportToBlob() exports to PNG 3. Shape API: Can programmatically add shapes, annotations, and evaluation overlays 4. Collaboration-ready: Built-in multiplayer support if you want an AI that draws alongside 5. Persistence: Canvas state serializes to JSON for session replay

Installation

npm install tldraw

# tldraw peer dependencies
npm install @tldraw/tldraw

Basic Integration

import { Tldraw, Editor, TLStoreSnapshot } from 'tldraw';
import 'tldraw/tldraw.css';
import { useRef, useCallback } from 'react';

interface WhiteboardProps {
  sessionId: string;
  onSnapshot: (imageBlob: Blob, canvasState: TLStoreSnapshot) => void;
}

export function InterviewWhiteboard({ sessionId, onSnapshot }: WhiteboardProps) {
  const editorRef = useRef<Editor | null>(null);

  const handleMount = useCallback((editor: Editor) => {
    editorRef.current = editor;

    // Configure for interview-style drawing
    editor.updateInstanceState({
      isGridMode: false,       // Free-form drawing, not snapped
      isDebugMode: false,
    });

    // Set default tool to draw (pencil)
    editor.setCurrentTool('draw');
  }, []);

  return (
    <div style={{ width: '100%', height: '600px', border: '2px solid #333' }}>
      <Tldraw
        onMount={handleMount}
        persistenceKey={`interview-${sessionId}`}
      />
    </div>
  );
}

Toolbar Configuration

For interview simulation, restrict the toolbar to tools that match real whiteboard constraints:

const INTERVIEW_TOOLS = [
  'select',   // Pointer for moving shapes
  'draw',     // Freehand drawing (primary tool)
  'rectangle', // Boxes for components
  'arrow',    // Arrows for data flow
  'text',     // Labels
  'eraser',   // Fix mistakes
];

// Hide tools that wouldn't exist on a real whiteboard
const HIDDEN_TOOLS = [
  'image',    // No image imports in a real interview
  'embed',    // No embeds
  'frame',    // Unnecessary complexity
];

---

Periodic Screenshot Strategy

The whiteboard engine captures snapshots at intervals for Claude Vision evaluation. The frequency adapts to user activity.

Screenshot Timing

Implementation

class WhiteboardCapture {
  private editor: Editor;
  private captureInterval: NodeJS.Timeout | null = null;
  private lastActivityTs: number = Date.now();
  private snapshots: WhiteboardSnapshot[] = [];
  private isDrawing: boolean = false;

  constructor(editor: Editor) {
    this.editor = editor;
    this.setupActivityTracking();
  }

  private setupActivityTracking(): void {
    // Track drawing activity via store changes
    this.editor.store.listen((entry) => {
      this.lastActivityTs = Date.now();
      this.isDrawing = true;

      // Reset drawing flag after 2 seconds of inactivity
      setTimeout(() => {
        if (Date.now() - this.lastActivityTs > 2000) {
          this.isDrawing = false;
        }
      }, 2000);
    });
  }

  startCapturing(sessionId: string): void {
    // Adaptive capture loop
    const captureLoop = async () => {
      const timeSinceActivity = Date.now() - this.lastActivityTs;

      // Skip if idle for >30 seconds
      if (timeSinceActivity > 30_000) {
        this.captureInterval = setTimeout(captureLoop, 5000); // Check again in 5s
        return;
      }

      // Determine interval based on activity
      const interval = this.isDrawing ? 30_000 : 120_000;

      await this.captureSnapshot(sessionId);
      this.captureInterval = setTimeout(captureLoop, interval);
    };

    captureLoop();
  }

  private async captureSnapshot(sessionId: string): Promise<void> {
    // Export canvas to PNG blob
    const shapeIds = this.editor.getCurrentPageShapeIds();
    if (shapeIds.size === 0) return; // Skip empty canvas

    const blob = await this.editor.exportToBlob({
      format: 'png',
      ids: [...shapeIds],
      padding: 20,
    });

    const snapshot: WhiteboardSnapshot = {
      sessionId,
      timestamp: Date.now(),
      imageBlob: blob,
      shapeCount: shapeIds.size,
      canvasState: this.editor.getSnapshot(),
    };

    this.snapshots.push(snapshot);
  }

  async captureFinal(sessionId: string): Promise<WhiteboardSnapshot> {
    await this.captureSnapshot(sessionId);
    return this.snapshots[this.snapshots.length - 1];
  }

  stopCapturing(): void {
    if (this.captureInterval) {
      clearTimeout(this.captureInterval);
      this.captureInterval = null;
    }
  }

  getSnapshots(): WhiteboardSnapshot[] {
    return this.snapshots;
  }
}

interface WhiteboardSnapshot {
  sessionId: string;
  timestamp: number;
  imageBlob: Blob;
  shapeCount: number;
  canvasState: any;
}

---

Claude Vision API Evaluation

Evaluation Strategy

Two types of evaluations happen during a session:

1. Periodic checks (during session): Quick assessment of diagram progress, looking for missing components or structural issues. Used to prompt the voice AI: "I notice you haven't drawn a data pipeline yet -- how does data flow into the system?"

2. Final evaluation (post-session): Comprehensive scoring of the complete diagram against a rubric specific to the round type.

Periodic Check Prompt

const PERIODIC_CHECK_PROMPT = `You are evaluating a system design whiteboard diagram during a mock interview.

The candidate is designing: {PROBLEM_DESCRIPTION}

Analyze this diagram snapshot and respond with JSON:

{
  "components_present": ["list of identifiable system components"],
  "components_missing": ["critical components not yet drawn"],
  "structural_issues": ["any unclear connections, missing arrows, ambiguous labels"],
  "progress_assessment": "on_track | falling_behind | ahead",
  "suggested_prompt": "A question the interviewer should ask based on what's drawn or missing. Empty string if diagram looks good."
}

Be concise. This is a mid-session check, not a final evaluation.
Focus on what's MISSING that should be present at this stage of the interview.`;

async function periodicCheck(
  imageBlob: Blob,
  problemDescription: string,
  elapsedMinutes: number,
  totalMinutes: number,
): Promise<PeriodicCheckResult> {
  const base64 = await blobToBase64(imageBlob);

  const response = await anthropic.messages.create({
    model: 'claude-sonnet-4-20250514',
    max_tokens: 500,
    messages: [{
      role: 'user',
      content: [
        {
          type: 'image',
          source: { type: 'base64', media_type: 'image/png', data: base64 },
        },
        {
          type: 'text',
          text: PERIODIC_CHECK_PROMPT
            .replace('{PROBLEM_DESCRIPTION}', problemDescription)
            + `\n\nElapsed: ${elapsedMinutes}/${totalMinutes} minutes.`,
        },
      ],
    }],
  });

  return JSON.parse(response.content[0].text);
}

Final Evaluation Prompt and Rubric

const FINAL_EVALUATION_PROMPT = `You are a senior ML system design interviewer evaluating a candidate's whiteboard diagram.

Problem: {PROBLEM_DESCRIPTION}
Session length: {SESSION_LENGTH} minutes
Round type: {ROUND_TYPE}

Evaluate this final diagram using the rubric below. Score each dimension 1-5 and provide specific evidence.

## Scoring Rubric

### Component Completeness (weight: 25%)
5 - All critical components present with clear labels and connections
4 - Most components present, minor omissions (e.g., monitoring but no alerting)
3 - Core components present but missing important supporting components
2 - Several critical components missing (e.g., no data pipeline, no monitoring)
1 - Only 1-2 components drawn, diagram is skeletal

### Scalability Patterns (weight: 20%)
5 - Explicit scaling annotations (QPS, latency targets, sharding strategy)
4 - Scaling mentioned for key components, some quantification
3 - Generic scaling awareness ("we'd shard this") without specifics
2 - Scaling not addressed for most components
1 - No consideration of scale

### Data Flow Clarity (weight: 20%)
5 - All data flows clearly shown with arrows, labeled with format/protocol
4 - Main flows clear, minor ambiguities in secondary paths
3 - Primary data flow visible but secondary flows unclear or missing
2 - Arrows present but unlabeled, flow direction ambiguous
1 - No clear data flow, disconnected components

### Trade-off Awareness (weight: 15%)
5 - Explicit trade-off annotations on diagram (e.g., "chose X over Y because Z")
4 - Trade-offs mentioned verbally but visible in design choices
3 - Some awareness of trade-offs but not explicitly marked
2 - Design suggests default choices without trade-off consideration
1 - No evidence of trade-off thinking

### Production Readiness (weight: 10%)
5 - Monitoring, logging, alerting, rollback, and failure modes all represented
4 - Monitoring and basic ops present, some gaps
3 - Monitoring box exists but details sparse
2 - No operational components
1 - Design is purely theoretical

### Visual Organization (weight: 10%)
5 - Clean layout, logical grouping, readable labels, consistent style
4 - Generally organized with minor clutter
3 - Readable but disorganized, hard to follow flow
2 - Cluttered, overlapping elements, hard to read
1 - Illegible or nonsensical layout

Respond with JSON:
{
  "scores": {
    "component_completeness": { "score": 1-5, "evidence": "..." },
    "scalability_patterns": { "score": 1-5, "evidence": "..." },
    "data_flow_clarity": { "score": 1-5, "evidence": "..." },
    "tradeoff_awareness": { "score": 1-5, "evidence": "..." },
    "production_readiness": { "score": 1-5, "evidence": "..." },
    "visual_organization": { "score": 1-5, "evidence": "..." }
  },
  "composite_score": <weighted average 0-100>,
  "strengths": ["top 2-3 things done well"],
  "critical_gaps": ["top 2-3 things missing or wrong"],
  "improvement_actions": ["specific things to practice"]
}`;

Cost Per Evaluation

For a 45-minute design session with 8 periodic checks and 1 final evaluation:

• Periodic: 8 x $0.01 = $0.08
• Final: 1 x $0.03 = $0.03
• Total per session: ~$0.11

Cost Optimization

1. Skip periodic checks when canvas is unchanged: Compare shape count and positions between snapshots. If nothing moved, skip the Vision API call.

2. Use Haiku for periodic checks: Periodic checks need less nuance than final evaluation. Switch to claude-3-5-haiku for mid-session checks (~$0.003 each) and reserve Sonnet for the final evaluation.

3. Batch snapshot evaluation: Instead of evaluating every snapshot, evaluate every 3rd snapshot during active drawing. Most incremental changes don't require evaluation.

4. Reduce image resolution: tldraw exports at screen resolution by default. For Vision API evaluation, 1024x768 is sufficient. Higher resolutions add token cost without improving evaluation quality.

const blob = await editor.exportToBlob({
  format: 'png',
  ids: [...shapeIds],
  padding: 20,
  // Constrain export dimensions
  scale: 0.5,  // Half resolution
});

---

Diagram Replay

Store canvas state (JSON) at each snapshot for post-session replay. This lets candidates review their diagram evolution without re-running a session.

class DiagramReplay {
  private snapshots: Array<{
    timestamp: number;
    canvasState: any;
    evaluation?: PeriodicCheckResult;
  }>;

  constructor(snapshots: WhiteboardSnapshot[]) {
    this.snapshots = snapshots.map(s => ({
      timestamp: s.timestamp,
      canvasState: s.canvasState,
      evaluation: undefined, // Attached after evaluation
    }));
  }

  // Restore canvas to any point in time
  restoreToTimestamp(editor: Editor, timestamp: number): void {
    const snapshot = this.snapshots.find(s => s.timestamp <= timestamp);
    if (snapshot) {
      editor.loadSnapshot(snapshot.canvasState);
    }
  }

  // Get evaluation timeline overlay
  getEvaluationTimeline(): Array<{
    timestamp: number;
    compositeScore: number;
    gaps: string[];
  }> {
    return this.snapshots
      .filter(s => s.evaluation)
      .map(s => ({
        timestamp: s.timestamp,
        compositeScore: s.evaluation!.composite_score || 0,
        gaps: s.evaluation!.critical_gaps || [],
      }));
  }
}

---

Integration with Voice Engine

The whiteboard engine communicates with the voice engine to enable interviewer prompting based on diagram state:

// In the session orchestrator
class SessionOrchestrator {
  private voiceEngine: VoiceEngine;
  private whiteboardCapture: WhiteboardCapture;

  async onPeriodicCheck(result: PeriodicCheckResult): Promise<void> {
    // If the diagram is missing a critical component, prompt the interviewer to ask about it
    if (result.suggested_prompt && result.progress_assessment === 'falling_behind') {
      await this.voiceEngine.injectPrompt(result.suggested_prompt);
      // Voice AI will naturally incorporate this into the conversation
      // e.g., "I notice your diagram doesn't show how data gets to the model.
      //         Can you walk me through the data pipeline?"
    }
  }
}

---

tldraw Customization for Interview Context

Custom Shape: Component Box

Add a custom shape type optimized for system architecture diagrams:

import { BaseBoxShapeUtil, TLBaseShape, Rectangle2d } from 'tldraw';

type ComponentShape = TLBaseShape<'component', {
  label: string;
  techStack: string;
  w: number;
  h: number;
}>;

class ComponentShapeUtil extends BaseBoxShapeUtil<ComponentShape> {
  static override type = 'component' as const;

  getDefaultProps(): ComponentShape['props'] {
    return { label: 'Component', techStack: '', w: 200, h: 80 };
  }

  getGeometry(shape: ComponentShape): Rectangle2d {
    return new Rectangle2d({
      width: shape.props.w,
      height: shape.props.h,
      isFilled: true,
    });
  }

  component(shape: ComponentShape) {
    return (
      <div style={{
        width: shape.props.w,
        height: shape.props.h,
        border: '2px solid #333',
        borderRadius: 8,
        padding: 8,
        background: '#f8f8f8',
        display: 'flex',
        flexDirection: 'column',
        alignItems: 'center',
        justifyContent: 'center',
      }}>
        <strong>{shape.props.label}</strong>
        {shape.props.techStack && (
          <small style={{ color: '#666', marginTop: 4 }}>
            {shape.props.techStack}
          </small>
        )}
      </div>
    );
  }

  indicator(shape: ComponentShape) {
    return (
      <rect
        width={shape.props.w}
        height={shape.props.h}
        rx={8}
      />
    );
  }
}

Evaluation Overlay

After each periodic check, overlay evaluation annotations on the canvas:

function addEvaluationOverlay(
  editor: Editor,
  result: PeriodicCheckResult,
): void {
  // Add a small indicator for missing components
  result.components_missing.forEach((component, index) => {
    editor.createShape({
      type: 'text',
      x: editor.getViewportScreenBounds().maxX - 250,
      y: 20 + (index * 30),
      props: {
        text: `Missing: ${component}`,
        color: 'red',
        size: 's',
      },
    });
  });

  // Auto-remove overlay after 10 seconds (or on next draw action)
  setTimeout(() => {
    // Remove overlay shapes
  }, 10_000);
}