Testability Scoring

Name: Testability Scoring
Author: proffesor-for-testing

proffesor-for-testing/agentic-qe

97 installs
433 repo stars
Updated August 4, 2026
proffesor-for-testing/agentic-qe

testability-scoring is a Claude Code skill for testing & qa.

About

testability-scoring is a Claude Code skill for testing & qa. It helps solo builders move faster with AI-assisted development.

testability-scoring
Testing & QA
AI-coding skill

Testability Scoring by the numbers

97 all-time installs (skills.sh)
+4 installs in the week ending Aug 4, 2026 (Skillselion tracking)
Ranked #1,009 of 2,153 Testing & QA skills by installs in the Skillselion catalog
Data as of Aug 5, 2026 (Skillselion catalog sync)

npx skills add https://github.com/proffesor-for-testing/agentic-qe --skill testability-scoring

Add your badge

Show developers this skill is listed on Skillselion. Paste this into your README.

[![Listed on Skillselion](https://skillselion.com/badge/skills/proffesor-for-testing/agentic-qe/testability-scoring.svg)](https://skillselion.com/skills/proffesor-for-testing/agentic-qe/testability-scoring)

Installs	97
repo stars	★ 433
Last updated	August 4, 2026
Repository	proffesor-for-testing/agentic-qe ↗

How do I helps with testing & qa tasks.?

Helps with testing & qa tasks.

Who is it for?

Best when you're working on testing & qa and need structured help with testability scoring.

Skip if: Teams with no testing & qa needs, or anyone wanting a generic chat assistant without this specific workflow.

When should I use this skill?

When you need to helps with testing & qa tasks., or when testability-scoring is a claude code skill for testing & qa.

What you get

Structured output aligned to testability-scoring: testability-scoring, Testing & QA.

Files

SKILL.mdMarkdownGitHub ↗

Testability Scoring

Browser engine

Uses the qe-browser fleet skill (.claude/skills/qe-browser/) as the primary browser engine. Vibium is installed by aqe init. The legacy scripts/run-assessment.sh + Playwright path remains as a fallback when a team already has a Playwright test suite configured, but new runs should prefer:

vibium go "$TARGET_URL"
vibium wait load
vibium a11y-tree --json > /tmp/testability/tree.json
vibium eval --stdin --json <<'EOF' > /tmp/testability/signals.json
JSON.stringify({
  headings: document.querySelectorAll('h1,h2,h3,h4,h5,h6').length,
  testIds: document.querySelectorAll('[data-testid]').length,
  forms: document.querySelectorAll('form').length,
  ariaLabels: document.querySelectorAll('[aria-label]').length,
  links: document.querySelectorAll('a[href]').length,
});
EOF
node .claude/skills/qe-browser/scripts/assert.js --checks '[
  {"kind": "no_console_errors"},
  {"kind": "no_failed_requests"}
]'

<default_to_action> When assessing testability: 1. RUN assessment against target URL 2. ANALYZE all 10 principles automatically 3. GENERATE HTML report with radar chart 4. PRIORITIZE improvements by impact/effort 5. INTEGRATE with QX Partner for holistic view

Quick Assessment:

# Run assessment on any URL
TEST_URL='https://example.com/' npx playwright test tests/testability-scoring/testability-scoring.spec.js --project=chromium --workers=1

# Or use shell script wrapper
.claude/skills/testability-scoring/scripts/run-assessment.sh https://example.com/

The 10 Principles at a Glance:

Principle	Weight	Key Question
Observability	15%	Can we see what's happening?
Controllability	15%	Can we control the application?
Algorithmic Simplicity	10%	Are behaviors predictable?
Algorithmic Transparency	10%	Can we understand what it does?
Algorithmic Stability	10%	Does behavior remain consistent?
Explainability	10%	Is the interface understandable?
Unbugginess	10%	How error-free is it?
Smallness	10%	Are components appropriately sized?
Decomposability	5%	Can we test parts in isolation?
Similarity	5%	Is the tech stack familiar?

Grade Scale:

A (90-100): Excellent testability
B (80-89): Good testability
C (70-79): Adequate testability
D (60-69): Below average
F (0-59): Poor testability

</default_to_action>

Quick Reference Card

Running Assessments

Method	Command	When to Use
Shell Script	`./scripts/run-assessment.sh URL`	One-time assessment
ENV Override	`TEST_URL='URL' npx playwright test...`	CI/CD integration
Config File	Update `tests/testability-scoring/config.js`	Repeated runs

Principle Details

High Weight (15% each)

Principle	Measures	Indicators
Observability	State visibility, logging, monitoring	Console output, network tracking, error visibility
Controllability	Input control, state manipulation	API access, test data injection, determinism

Medium Weight (10% each)

Principle	Measures	Indicators
Simplicity	Predictable behavior	Clear I/O relationships, low complexity
Transparency	Understanding what system does	Visible processes, readable code
Stability	Consistent behavior	Change resilience, maintainability
Explainability	Interface understanding	Good docs, semantic structure, help text
Unbugginess	Error-free operation	Console errors, warnings, runtime issues
Smallness	Component size	Element count, script bloat, page complexity

Low Weight (5% each)

Principle	Measures	Indicators
Decomposability	Isolation testing	Component separation, modular design
Similarity	Technology familiarity	Standard frameworks, known patterns

---

Assessment Workflow

1. Navigate to URL → 2. Collect Metrics → 3. Score Principles
                                              ↓
4. Generate JSON ← 5. Calculate Grades ← 6. Apply Weights
         ↓
7. Generate HTML Report with Radar Chart
         ↓
8. Open in Browser (auto-opens)

Output Files

tests/reports/
├── testability-results-<timestamp>.json  # Raw data
├── testability-report-<timestamp>.html   # Visual report
└── latest.json                           # Symlink

---

Integration Examples

CI/CD Integration

# GitHub Actions
- name: Testability Assessment
  run: |
    timeout 180 .claude/skills/testability-scoring/scripts/run-assessment.sh ${{ env.APP_URL }}

- name: Upload Reports
  uses: actions/upload-artifact@v3
  with:
    name: testability-reports
    path: tests/reports/testability-*.html

QX Partner Integration

// Combine testability with QX analysis
const qxAnalysis = await Task("QX Analysis", {
  target: 'https://example.com',
  integrateTestability: true
}, "qx-partner");

// Returns combined insights:
// - QX Score: 78/100
// - Testability Integration: Observability 72/100
// - Combined Insight: Low observability may mask UX issues

Programmatic Usage

import { runTestabilityAssessment } from './testability';

const results = await runTestabilityAssessment('https://example.com');
console.log(`Overall: ${results.overallScore}/100 (${results.grade})`);
console.log('Recommendations:', results.recommendations);

---

Agent Integration

// Run testability assessment
const assessment = await Task("Testability Assessment", {
  url: 'https://example.com',
  generateReport: true,
  openBrowser: true
}, "qe-quality-analyzer");

// Use with QX Partner for holistic analysis
const qxReport = await Task("Full QX Analysis", {
  target: 'https://example.com',
  integrateTestability: true,
  detectOracleProblems: true
}, "qx-partner");

---

Vibium Integration (Optional)

Overview

Vibium browser automation can be used alongside Playwright for enhanced testability assessment. While Playwright remains the primary engine, Vibium offers complementary capabilities for certain metrics.

Installation:

claude mcp add vibium -- npx -y vibium

Vibium-Enhanced Metrics

Principle	Vibium Enhancement	Benefit
Observability	Auto-wait duration tracking	Measures DOM stability (30s timeout, 100ms polling)
Controllability	Element interaction success rate	Validates automation readiness via MCP
Stability	Screenshot consistency	Visual regression detection for layout stability
Explainability	Element attribute extraction	ARIA labels, semantic HTML validation

When to Use Vibium

✅ USE Vibium for:

Element stability metrics (auto-wait duration analysis)
Visual consistency checks (screenshot comparison)
MCP-native AI agent integration
Lightweight Docker images (400MB vs 1.2GB)

❌ USE Playwright for:

Console error detection (Vibium V1 lacks console API)
Network performance metrics (BiDi network APIs coming in V2)
Comprehensive browser coverage (Firefox, Safari)
Production-proven stability (Vibium V1 released Dec 2024)

Hybrid Assessment Example

// Testability assessment using both engines
const assessment = {
  // Playwright: Comprehensive metrics
  playwright: await runPlaywrightAssessment(url),

  // Vibium: Stability metrics
  vibium: {
    elementStability: await measureAutoWaitDuration(url),
    visualConsistency: await compareScreenshots(url),
    accessibilityAttributes: await extractARIALabels(url)
  }
};

// Enhanced Observability Score
const observability =
  (assessment.playwright.consoleErrors * 0.6) +
  (assessment.vibium.elementStability * 0.4);

Vibium MCP Tools for Testability

// 1. Element Stability Measurement
const browser = await browser_launch();
await browser_navigate({ url });
const startTime = Date.now();
const element = await browser_find({ selector: ".critical-element" });
const autoWaitDuration = Date.now() - startTime;
// Lower duration = better stability

// 2. Visual Consistency Check
const screenshot1 = await browser_screenshot();
await browser_navigate({ url }); // Reload
const screenshot2 = await browser_screenshot();
const visualDiff = compareImages(screenshot1.png, screenshot2.png);
// Lower diff = better stability

// 3. Accessibility Attribute Extraction
const elements = await browser_find({ selector: "button, a, input" });
const ariaLabels = elements.map(el => el.attributes["aria-label"]);
const semanticScore = (ariaLabels.filter(Boolean).length / elements.length) * 100;

Migration Strategy

Current (V2.2): Hybrid approach

Playwright: Primary engine for all 10 principles
Vibium: Optional enhancement for stability metrics

Future (V3.0): When Vibium V2 ships

Evaluate Vibium as primary engine if:
Console/Network APIs available
Production stability proven
Community adoption increases

Agent Coordination Hints

Memory Namespace

aqe/testability/
├── assessments/*       - Assessment results by URL
├── historical/*        - Historical scores for trend analysis
├── recommendations/*   - Improvement recommendations
├── integration/*       - QX integration data
└── vibium/*           - Vibium-specific metrics (optional)

Fleet Coordination

const testabilityFleet = await FleetManager.coordinate({
  strategy: 'testability-assessment',
  agents: [
    'qe-quality-analyzer',  // Primary assessment
    'qx-partner',           // UX integration
    'qe-visual-tester'      // Visual validation
  ],
  topology: 'sequential'
});

---

Common Issues & Solutions

Issue	Solution
Tests timing out	Increase timeout: `timeout 300 ./scripts/run-assessment.sh URL`
Partial results	Check console errors, increase network timeout
Report not opening	Use `AUTO_OPEN=false`, open manually
Config not updating	Use `TEST_URL` env var instead
Vibium not available	Install via `claude mcp add vibium -- npx -y vibium` (optional)
Hybrid mode errors	Vibium is optional; assessments work without it

---

Related Skills

accessibility-testing - WCAG compliance (overlaps with Explainability)
visual-testing-advanced - UI consistency
performance-testing - Load time metrics

---

Credits & References

Framework Origin

Heuristics for Software Testability by James Bach and Michael Bolton
Available at: https://www.satisfice.com/download/heuristics-of-software-testability

Implementation

Based on https://github.com/fndlalit/testability-scorer (contributed by @fndlalit)
Playwright v1.49.0+ with AI capabilities (primary engine)
Vibium v1.0+ with MCP integration (optional enhancement)
Chart.js for radar visualizations

Vibium Resources

GitHub: https://github.com/VibiumDev/vibium
MCP Integration: claude mcp add vibium -- npx -y vibium
Created by Jason Huggins (creator of Selenium/Appium)

---

Remember

Testability is an investment, not an afterthought.

Good testability:

Reduces debugging time
Enables faster feedback loops
Makes defects easier to find
Supports continuous testing

Low scores = High risk. Prioritize improvements by weight × impact.

# =============================================================================
# Testability Scoring Skill Evaluation Suite v2.2.0
# Tests the 10 principles of intrinsic testability assessment
# =============================================================================
#
# This evaluation suite validates:
# - All 10 testability dimensions (Bach & Bolton framework)
# - Score calculation accuracy and grade mapping
# - Recommendation quality and actionability
# - Code complexity analysis for testability
# - Multi-model consistency across Claude/GPT models
#
# Schema: .claude/skills/.validation/schemas/skill-eval.schema.json
# Runner: scripts/run-skill-eval.ts
#
# =============================================================================

skill: testability-scoring
version: 2.2.0
description: >
  Comprehensive evaluation suite for the testability-scoring skill.
  Tests all 10 principles of intrinsic testability (Observability, Controllability,
  Algorithmic Simplicity, Algorithmic Transparency, Algorithmic Stability,
  Explainability, Unbugginess, Smallness, Decomposability, Similarity).
  Ensures consistent scoring across models and validates recommendations.

# =============================================================================
# Multi-Model Configuration
# =============================================================================

models_to_test:
  - claude-sonnet-4-6    # Primary (high accuracy expected)
  - claude-haiku-4-5     # Fast model (minimum quality floor)

# =============================================================================
# MCP Integration Configuration
# =============================================================================

mcp_integration:
  enabled: true
  namespace: skill-validation/testability-scoring

  # Query existing patterns before running evals
  query_patterns: true

  # Track each test outcome for learning feedback loop
  track_outcomes: true

  # Store successful patterns after evals complete
  store_patterns: true

  # Share learning with fleet coordinator agents
  share_learning: true

  # Update quality gate with validation metrics
  update_quality_gate: true

  # Agents to share learning with
  target_agents:
    - qe-learning-coordinator
    - qe-queen-coordinator
    - qe-quality-analyzer

# =============================================================================
# ReasoningBank Learning Configuration
# =============================================================================

learning:
  store_success_patterns: true
  store_failure_patterns: true
  pattern_ttl_days: 90
  min_confidence_to_store: 0.7
  cross_model_comparison: true

# =============================================================================
# Result Format Configuration
# =============================================================================

result_format:
  json_output: true
  markdown_report: true
  include_raw_output: false
  include_timing: true
  include_token_usage: true

# =============================================================================
# Environment Setup
# =============================================================================

setup:
  required_tools:
    - playwright
    - jq
  environment_variables:
    TEST_TIMEOUT_MS: "30000"
    HEADLESS: "true"
  fixtures:
    - name: high_testability_page
      path: fixtures/high-testability-sample.html
      content: |
        <!DOCTYPE html>
        <html lang="en">
        <head><title>High Testability Sample</title></head>
        <body>
          <main data-testid="main-content">
            <h1 data-testid="page-title">Welcome</h1>
            <form data-testid="contact-form">
              <label for="name">Name</label>
              <input id="name" data-testid="name-input" type="text" />
              <button data-testid="submit-btn" type="submit">Submit</button>
            </form>
          </main>
        </body>
        </html>

    - name: low_testability_page
      path: fixtures/low-testability-sample.html
      content: |
        <!DOCTYPE html>
        <html>
        <head><title>Low Testability Sample</title></head>
        <body>
          <div class="x1" onclick="doStuff()">
            <div class="x2"><div class="x3">Click me</div></div>
          </div>
          <script>
            console.error('Intentional error for testing');
            var globalState = {};
          </script>
        </body>
        </html>

# =============================================================================
# Test Cases - 10 Testability Dimensions
# =============================================================================

test_cases:
  # -------------------------------------------------------------------------
  # OBSERVABILITY (15% weight) - Can we see what's happening?
  # -------------------------------------------------------------------------

  - id: tc001_observability_high
    description: "Correctly identifies high observability in well-instrumented page"
    category: observability
    priority: critical

    input:
      url: "fixtures/high-testability-sample.html"
      focus_dimension: observability
      context:
        has_logging: true
        has_devtools_support: true
        has_network_visibility: true

    expected_output:
      must_contain:
        - "observability"
        - "score"
      dimension_score:
        dimension: observability
        min: 75
        max: 100

    validation:
      schema_check: true
      keyword_match_threshold: 0.8
      reasoning_quality_min: 0.7

  - id: tc002_observability_console_errors
    description: "Detects console errors as observability risk"
    category: observability
    priority: high

    input:
      code: |
        // Application with hidden errors
        try {
          processData(input);
        } catch (e) {
          // Silent fail - no logging
        }
      context:
        console_errors: 5
        has_error_tracking: false

    expected_output:
      must_contain:
        - "console"
        - "error"
        - "observability"
      severity_classification: medium

    validation:
      schema_check: true
      finding_count:
        min: 1

  # -------------------------------------------------------------------------
  # CONTROLLABILITY (15% weight) - Can we control the application?
  # -------------------------------------------------------------------------

  - id: tc003_controllability_testid
    description: "Identifies data-testid coverage for controllability"
    category: controllability
    priority: critical

    input:
      code: |
        <button data-testid="submit-btn" onClick={handleSubmit}>Submit</button>
        <button onClick={handleCancel}>Cancel</button>
        <input data-testid="email-input" type="email" />
        <input type="text" />
      context:
        framework: react
        has_test_ids: "partial"

    expected_output:
      must_contain:
        - "data-testid"
        - "controllability"
      must_not_contain:
        - "excellent controllability"

    validation:
      schema_check: true
      keyword_match_threshold: 0.8

  - id: tc004_controllability_api_access
    description: "Evaluates API controllability for test data injection"
    category: controllability
    priority: high

    input:
      code: |
        // API with test hooks
        class UserService {
          constructor(private db: Database, private cache?: Cache) {}

          // Test hook
          static setTestDatabase(testDb: Database) {
            this.testDb = testDb;
          }

          async getUser(id: string) {
            return this.db.findUser(id);
          }
        }
      context:
        has_dependency_injection: true
        has_test_hooks: true

    expected_output:
      must_contain:
        - "controllability"
        - "injection"
      dimension_score:
        dimension: controllability
        min: 70

    validation:
      schema_check: true

  # -------------------------------------------------------------------------
  # ALGORITHMIC SIMPLICITY (10% weight) - Are behaviors predictable?
  # -------------------------------------------------------------------------

  - id: tc005_simplicity_deterministic
    description: "Identifies deterministic behavior as high simplicity"
    category: algorithmicSimplicity
    priority: high

    input:
      code: |
        function calculateTotal(items: Item[]): number {
          return items.reduce((sum, item) => sum + item.price * item.quantity, 0);
        }
      context:
        has_side_effects: false
        is_pure_function: true

    expected_output:
      must_contain:
        - "simplicity"
        - "predictable"
      dimension_score:
        dimension: algorithmicSimplicity
        min: 80

    validation:
      schema_check: true

  - id: tc006_simplicity_non_deterministic
    description: "Flags non-deterministic behavior as simplicity risk"
    category: algorithmicSimplicity
    priority: high

    input:
      code: |
        function getRecommendations(userId: string): Product[] {
          const random = Math.random();
          const products = fetchProducts();
          if (random > 0.5) {
            return shuffle(products);
          }
          return products.slice(0, random * products.length);
        }
      context:
        has_randomness: true
        is_deterministic: false

    expected_output:
      must_contain:
        - "random"
        - "non-deterministic"
        - "simplicity"
      severity_classification: medium

    validation:
      schema_check: true
      finding_count:
        min: 1

  # -------------------------------------------------------------------------
  # ALGORITHMIC TRANSPARENCY (10% weight) - Can we understand what it does?
  # -------------------------------------------------------------------------

  - id: tc007_transparency_clear_logic
    description: "Identifies clear, readable code as high transparency"
    category: algorithmicTransparency
    priority: medium

    input:
      code: |
        /**
         * Validates user registration data
         * @param user - User registration form data
         * @returns Validation result with errors if any
         */
        function validateRegistration(user: RegistrationData): ValidationResult {
          const errors: string[] = [];

          if (!user.email || !isValidEmail(user.email)) {
            errors.push('Valid email is required');
          }

          if (!user.password || user.password.length < 8) {
            errors.push('Password must be at least 8 characters');
          }

          return { isValid: errors.length === 0, errors };
        }
      context:
        has_documentation: true
        cyclomatic_complexity: "low"

    expected_output:
      must_contain:
        - "transparency"
        - "readable"
      dimension_score:
        dimension: algorithmicTransparency
        min: 75

    validation:
      schema_check: true

  # -------------------------------------------------------------------------
  # ALGORITHMIC STABILITY (10% weight) - Does behavior remain consistent?
  # -------------------------------------------------------------------------

  - id: tc008_stability_async_handling
    description: "Evaluates async operation stability"
    category: algorithmicStability
    priority: high

    input:
      code: |
        async function loadDashboard() {
          const [users, orders, products] = await Promise.all([
            fetchUsers(),
            fetchOrders(),
            fetchProducts()
          ]);
          // No loading state management
          renderDashboard({ users, orders, products });
        }
      context:
        has_loading_states: false
        has_error_boundaries: false

    expected_output:
      must_contain:
        - "stability"
        - "async"
        - "loading"

    validation:
      schema_check: true

  # -------------------------------------------------------------------------
  # EXPLAINABILITY (10% weight) - Is the interface understandable?
  # -------------------------------------------------------------------------

  - id: tc009_explainability_semantic_html
    description: "Evaluates semantic HTML usage for explainability"
    category: explainability
    priority: medium

    input:
      code: |
        <nav role="navigation" aria-label="Main menu">
          <ul>
            <li><a href="/home">Home</a></li>
            <li><a href="/about">About</a></li>
          </ul>
        </nav>
        <main>
          <article>
            <h1>Page Title</h1>
            <p>Content here</p>
          </article>
        </main>
      context:
        has_aria_labels: true
        has_semantic_html: true

    expected_output:
      must_contain:
        - "explainability"
        - "semantic"
      dimension_score:
        dimension: explainability
        min: 80

    validation:
      schema_check: true

  # -------------------------------------------------------------------------
  # UNBUGGINESS (10% weight) - How error-free is it?
  # -------------------------------------------------------------------------

  - id: tc010_unbugginess_clean
    description: "Identifies error-free code as high unbugginess"
    category: unbugginess
    priority: critical

    input:
      context:
        console_errors: 0
        console_warnings: 0
        runtime_exceptions: 0

    expected_output:
      must_contain:
        - "unbugginess"
      dimension_score:
        dimension: unbugginess
        min: 85

    validation:
      schema_check: true

  - id: tc011_unbugginess_errors
    description: "Flags console errors as unbugginess risk"
    category: unbugginess
    priority: high

    input:
      context:
        console_errors: 5
        console_warnings: 10
        runtime_exceptions: 2
        error_details:
          - "TypeError: Cannot read property 'map' of undefined"
          - "NetworkError: Failed to fetch"

    expected_output:
      must_contain:
        - "error"
        - "unbugginess"
      severity_classification: high

    validation:
      schema_check: true
      finding_count:
        min: 1

  # -------------------------------------------------------------------------
  # SMALLNESS (10% weight) - Are components appropriately sized?
  # -------------------------------------------------------------------------

  - id: tc012_smallness_dom_complexity
    description: "Evaluates DOM element count for smallness"
    category: smallness
    priority: medium

    input:
      context:
        dom_element_count: 3500
        script_bundle_size_kb: 2048
        third_party_scripts: 25

    expected_output:
      must_contain:
        - "smallness"
        - "DOM"
        - "element"
      severity_classification: high

    validation:
      schema_check: true
      finding_count:
        min: 1

  - id: tc013_smallness_optimal
    description: "Identifies well-sized components as high smallness"
    category: smallness
    priority: medium

    input:
      context:
        dom_element_count: 500
        script_bundle_size_kb: 256
        third_party_scripts: 3

    expected_output:
      must_contain:
        - "smallness"
      dimension_score:
        dimension: smallness
        min: 80

    validation:
      schema_check: true

  # -------------------------------------------------------------------------
  # DECOMPOSABILITY (5% weight) - Can we test parts in isolation?
  # -------------------------------------------------------------------------

  - id: tc014_decomposability_modular
    description: "Identifies modular architecture as high decomposability"
    category: decomposability
    priority: medium

    input:
      code: |
        // Modular component with clear boundaries
        export function UserCard({ user, onEdit, onDelete }: UserCardProps) {
          return (
            <div data-testid="user-card">
              <Avatar user={user} />
              <UserInfo user={user} />
              <ActionButtons onEdit={onEdit} onDelete={onDelete} />
            </div>
          );
        }
      context:
        has_component_isolation: true
        has_dependency_injection: true

    expected_output:
      must_contain:
        - "decomposability"
        - "modular"
      dimension_score:
        dimension: decomposability
        min: 75

    validation:
      schema_check: true

  # -------------------------------------------------------------------------
  # SIMILARITY (5% weight) - Is the tech stack familiar?
  # -------------------------------------------------------------------------

  - id: tc015_similarity_standard_stack
    description: "Identifies standard frameworks as high similarity"
    category: similarity
    priority: low

    input:
      context:
        framework: "React"
        language: "TypeScript"
        testing_framework: "Jest"
        build_tool: "Vite"

    expected_output:
      must_contain:
        - "similarity"
        - "React"
      dimension_score:
        dimension: similarity
        min: 80

    validation:
      schema_check: true

  # -------------------------------------------------------------------------
  # Overall Score Calculation
  # -------------------------------------------------------------------------

  - id: tc016_overall_score_calculation
    description: "Validates overall score is weighted average of dimensions"
    category: scoring
    priority: critical

    input:
      context:
        validate_score_calculation: true
        expected_weights:
          observability: 0.15
          controllability: 0.15
          algorithmicSimplicity: 0.10
          algorithmicTransparency: 0.10
          algorithmicStability: 0.10
          explainability: 0.10
          unbugginess: 0.10
          smallness: 0.10
          decomposability: 0.05
          similarity: 0.05

    expected_output:
      must_contain:
        - "score"
        - "grade"
      overall_score:
        min: 0
        max: 100

    validation:
      schema_check: true
      validate_weights_sum: true

  # -------------------------------------------------------------------------
  # Grade Mapping
  # -------------------------------------------------------------------------

  - id: tc017_grade_mapping_excellent
    description: "Validates A grade for scores 90-100"
    category: scoring
    priority: high

    input:
      context:
        expected_score: 92
        expected_grade: "A"

    expected_output:
      must_contain:
        - "grade"
        - "A"
      overall_score:
        min: 90
        max: 100

    validation:
      schema_check: true

  - id: tc018_grade_mapping_poor
    description: "Validates F grade for scores below 60"
    category: scoring
    priority: high

    input:
      context:
        expected_score: 45
        expected_grade: "F"

    expected_output:
      must_contain:
        - "grade"
        - "F"
      overall_score:
        min: 0
        max: 59

    validation:
      schema_check: true

  # -------------------------------------------------------------------------
  # Recommendation Quality
  # -------------------------------------------------------------------------

  - id: tc019_recommendations_actionable
    description: "Validates recommendations are actionable with code examples"
    category: recommendations
    priority: high

    input:
      context:
        finding_type: "missing_testid"
        require_code_example: true
        require_impact_score: true

    expected_output:
      must_contain:
        - "recommendation"
        - "data-testid"
      must_not_contain:
        - "TODO"
        - "placeholder"

    validation:
      schema_check: true
      recommendation_has_code: true
      recommendation_has_impact: true

  - id: tc020_recommendations_prioritized
    description: "Validates recommendations are sorted by priority/impact"
    category: recommendations
    priority: medium

    input:
      context:
        multiple_issues: true
        require_prioritization: true

    expected_output:
      must_contain:
        - "priority"
        - "impact"

    validation:
      schema_check: true
      recommendations_sorted: true

  # -------------------------------------------------------------------------
  # Edge Cases
  # -------------------------------------------------------------------------

  - id: tc021_empty_page
    description: "Handles empty/blank page gracefully"
    category: edge_cases
    priority: medium

    input:
      code: |
        <!DOCTYPE html>
        <html><head></head><body></body></html>
      context:
        page_type: "empty"

    expected_output:
      must_not_contain:
        - "exception"
        - "crash"
        - "error"

    validation:
      schema_check: true
      allow_partial: true

  - id: tc022_spa_dynamic_content
    description: "Handles SPA with dynamic content loading"
    category: edge_cases
    priority: medium

    input:
      context:
        spa: true
        lazy_loading: true
        async_content: true
        wait_time_ms: 5000

    expected_output:
      must_contain:
        - "dynamic"
        - "loading"

    validation:
      schema_check: true
      timeout_ms: 60000

# =============================================================================
# Success Criteria
# =============================================================================

success_criteria:
  # Minimum percentage of tests that must pass
  pass_rate: 0.90

  # Critical tests must have 100% pass rate
  critical_pass_rate: 1.0

  # Average reasoning quality across all tests
  avg_reasoning_quality: 0.7

  # Maximum time for entire suite (10 minutes)
  max_execution_time_ms: 600000

  # Maximum variance between different models (15%)
  cross_model_variance: 0.15

  # Testability-specific criteria
  dimension_coverage: 1.0  # All 10 dimensions must be tested
  score_range_validation: true  # All scores must be 0-100
  weight_sum_validation: true  # Weights must sum to 1.0

# =============================================================================
# Metadata
# =============================================================================

metadata:
  author: "@agentic-qe"
  created: "2026-02-02"
  last_updated: "2026-02-02"
  coverage_target: "All 10 testability principles with scoring validation"
  framework_reference: "James Bach & Michael Bolton - Heuristics for Software Testability"
  related_skills:
    - accessibility-testing
    - visual-testing-advanced
    - performance-testing

Testability Scoring Skill

Quick reference for AI-powered testability assessment using 10 principles of intrinsic testability.

Quick Start

# Run assessment
.claude/skills/testability-scoring/scripts/run-assessment.sh https://example.com/

# Generate HTML report from JSON
AUTO_OPEN=false node .claude/skills/testability-scoring/scripts/generate-html-report.js tests/reports/testability-results-*.json

10 Principles

1. Observability (15%) - Can we see what's happening? 2. Controllability (15%) - Can we control the application? 3. Algorithmic Simplicity (10%) - Are behaviors simple? 4. Algorithmic Transparency (10%) - Can we understand it? 5. Algorithmic Stability (10%) - Does it stay consistent? 6. Explainability (10%) - Is the interface clear? 7. Unbugginess (10%) - How error-free is it? 8. Smallness (10%) - Are components appropriate size? 9. Decomposability (5%) - Can we test parts separately? 10. Similarity (5%) - How familiar is the tech?

Scoring

90-100 (A): Excellent
80-89 (B): Good
70-79 (C): Adequate
60-69 (D): Below average
0-59 (F): Poor

Files

.claude/skills/testability-scoring/
├── SKILL.md                    # Complete documentation
├── scripts/
│   ├── run-assessment.sh       # Main runner
│   └── generate-html-report.js # Report generator
└── resources/templates/        # Templates

tests/testability-scoring/
├── testability-scoring.spec.js # Test implementation
└── config.js                   # Configuration

tests/reports/
├── testability-results-*.json  # Results
└── testability-report-*.html   # Reports

Common Commands

# Disable auto-open
AUTO_OPEN=false ./scripts/run-assessment.sh https://example.com/

# With timeout
timeout 180 ./scripts/run-assessment.sh https://example.com/

# Specific browser
./scripts/run-assessment.sh https://example.com/ firefox

Credits

Based on James Bach and Michael Bolton's Heuristics for Software Testability Implementation: https://github.com/fndlalit/testability-scorer

// Testability Scorer Configuration Template

module.exports = {
  // Application under test
  baseURL: 'https://www.saucedemo.com',

  // Scoring weights (must sum to 100)
  weights: {
    observability: 15,        // Console logs, state visibility, monitoring
    controllability: 15,       // API access, state manipulation, test data injection
    algorithmicSimplicity: 10, // Clear input-output relationships, low complexity
    algorithmicTransparency: 10, // Understandable logic, clear data flow
    explainability: 10,        // Documentation, comments, error messages
    similarity: 5,             // Standard patterns, familiar architecture
    algorithmicStability: 10,  // API versioning, backward compatibility
    unbugginess: 10,           // Low defect rate, test reliability
    smallness: 10,             // Manageable size, modular design
    decomposability: 5         // Component isolation, test unit granularity
  },

  // Grading scale
  grades: {
    A: 90,  // Excellent
    B: 80,  // Good
    C: 70,  // Acceptable
    D: 60,  // Below average
    F: 0    // Poor
  },

  // Report settings
  reports: {
    format: ['html', 'json', 'text'],  // Output formats
    directory: 'tests/reports',         // Report location
    autoOpen: true,                     // Open HTML report automatically
    includeAI: true,                    // AI-powered recommendations
    includeCharts: true,                // Visual charts
    includeHistory: true                // Historical comparison
  },

  // User types for comparative analysis (optional)
  userTypes: [
    { username: 'standard_user', password: 'secret_sauce', role: 'standard' },
    { username: 'locked_out_user', password: 'secret_sauce', role: 'locked' },
    { username: 'problem_user', password: 'secret_sauce', role: 'problem' },
    { username: 'performance_glitch_user', password: 'secret_sauce', role: 'performance' },
    { username: 'error_user', password: 'secret_sauce', role: 'error' },
    { username: 'visual_user', password: 'secret_sauce', role: 'visual' }
  ],

  // Browser configuration
  browsers: ['chromium', 'firefox', 'webkit'],

  // Timeouts
  timeouts: {
    test: 30000,      // Per test timeout
    navigation: 10000, // Page navigation
    action: 5000      // UI action timeout
  },

  // Thresholds for pass/fail
  thresholds: {
    overall: 70,           // Minimum overall score
    critical: {            // Critical principles must meet these minimums
      observability: 60,
      controllability: 60,
      unbugginess: 70
    }
  },

  // AI recommendations settings
  ai: {
    enabled: true,
    provider: 'local',  // 'local' or 'api'
    prioritize: 'impact', // 'impact', 'effort', or 'score'
    maxRecommendations: 10
  },

  // Historical tracking
  history: {
    enabled: true,
    directory: '.testability-history',
    retention: 90  // days
  }
};

const { test, expect } = require('@playwright/test');
const config = require('./config');
const fs = require('fs');
const path = require('path');

let testabilityScores = {
  timestamp: new Date().toISOString(),
  overall: 0,
  grade: 'F',
  principles: {},
  recommendations: [],
  metadata: {
    url: config.baseURL,
    browser: 'chromium',
    version: '1.0.0',
    assessor: 'testability-scorer-skill'
  }
};

// Initialize all principles with default values to ensure we always have data
function initializeDefaultScores() {
  const defaultScore = { score: 50, grade: 'F', weight: 0 };
  testabilityScores.principles = {
    observability: { ...defaultScore, weight: config.weights.observability },
    controllability: { ...defaultScore, weight: config.weights.controllability },
    algorithmicSimplicity: { ...defaultScore, weight: config.weights.algorithmicSimplicity },
    algorithmicTransparency: { ...defaultScore, weight: config.weights.algorithmicTransparency },
    explainability: { ...defaultScore, weight: config.weights.explainability },
    similarity: { ...defaultScore, weight: config.weights.similarity },
    algorithmicStability: { ...defaultScore, weight: config.weights.algorithmicStability },
    unbugginess: { ...defaultScore, weight: config.weights.unbugginess },
    smallness: { ...defaultScore, weight: config.weights.smallness },
    decomposability: { ...defaultScore, weight: config.weights.decomposability }
  };
}

// Robust page navigation helper
async function navigateToPage(page) {
  console.log(`[NAV] Starting navigation to ${config.baseURL}`);
  page.setDefaultTimeout(45000);
  
  try {
    console.log('[NAV] Attempting page.goto with domcontentloaded...');
    await page.goto(config.baseURL, { 
      timeout: 45000, 
      waitUntil: 'domcontentloaded' 
    });
    console.log('[NAV] Page loaded (domcontentloaded)');
    
    // Try to wait for network idle but don't fail if it times out
    console.log('[NAV] Waiting for networkidle (15s timeout)...');
    await page.waitForLoadState('networkidle', { timeout: 15000 })
      .then(() => console.log('[NAV] Network is idle'))
      .catch(() => console.log('[NAV] Network not idle after 15s, continuing...'));
    
    return true;
  } catch (error) {
    console.log(`[NAV] Navigation failed: ${error.message}`);
    // Try one more time with even more lenient settings
    try {
      console.log('[NAV] Retrying with commit waitUntil...');
      await page.goto(config.baseURL, { 
        timeout: 45000, 
        waitUntil: 'commit' 
      });
      console.log('[NAV] Page committed');
      return true;
    } catch (retryError) {
      console.error(`[NAV] Final navigation failed: ${retryError.message}`);
      return false;
    }
  }
}

test.describe.configure({ mode: 'serial', timeout: 60000 });

test.describe('Comprehensive Testability Analysis - Sauce Demo Shopify', () => {

  test.beforeAll(() => {
    console.log('Starting testability assessment...');
    initializeDefaultScores();
  });

  test('1. Observability Assessment', async ({ page }) => {
    try {
      const logs = [];
      const errors = [];
      const networkRequests = [];

      page.on('console', msg => logs.push(msg));
      page.on('pageerror', err => errors.push(err));
      page.on('request', request => networkRequests.push(request));

      const loaded = await navigateToPage(page);
      if (!loaded) {
        throw new Error('Failed to load page');
      }

    // Check console logging
    const hasConsoleLogs = logs.length > 0;

    // Check network visibility
    const hasNetworkRequests = networkRequests.length > 0;

    // Check state inspection
    const stateVisible = await page.evaluate(() => {
      return typeof window !== 'undefined' &&
             (typeof window.Shopify !== 'undefined' ||
              typeof window.console !== 'undefined');
    });

    // Calculate score
    let score = 0;
    if (hasConsoleLogs) score += 25;
    if (hasNetworkRequests) score += 30;
    if (stateVisible) score += 27;
    score += 10; // Base score for page loading

    testabilityScores.principles.observability = {
      score: Math.min(score, 100),
      grade: getLetterGrade(score),
      weight: config.weights.observability
    };

    if (score < 70) {
      testabilityScores.recommendations.push({
        principle: 'Observability',
        severity: 'medium',
        recommendation: 'Implement detailed event logging for user actions, cart operations, and payment processing.',
        impact: 15,
        effort: 'Low (4-6 hours)'
      });
    }
    } catch (error) {
      console.error('Observability assessment failed:', error.message);
      testabilityScores.principles.observability = { score: 50, grade: 'F', weight: config.weights.observability };
    }
  });

  test('2. Controllability Assessment', async ({ page }) => {
    try {
      const loaded = await navigateToPage(page);
      if (!loaded) throw new Error('Failed to load page');

    // Check direct API access
    const hasAPI = await page.evaluate(() => {
      return typeof window.Shopify !== 'undefined' &&
             typeof window.Shopify.checkout !== 'undefined';
    });

    // Check state manipulation
    const canManipulateCart = await page.evaluate(() => {
      return typeof window.Shopify !== 'undefined';
    });

    // Check test data injection capabilities
    const hasTestMode = await page.evaluate(() => {
      return typeof window.testAPI !== 'undefined' ||
             typeof window.Shopify !== 'undefined';
    });

    let score = 0;
    if (hasAPI) score += 25;
    if (canManipulateCart) score += 20;
    if (hasTestMode) score += 10;

    // E-commerce sites typically have limited controllability for security
    score = Math.min(score + 20, 65); // Cap at 65 for production e-commerce

    testabilityScores.principles.controllability = {
      score,
      grade: getLetterGrade(score),
      weight: config.weights.controllability
    };

    if (score < 70) {
      testabilityScores.recommendations.push({
        principle: 'Controllability',
        severity: 'critical',
        recommendation: 'Add test data injection endpoints to allow programmatic product catalog and cart management during automated testing.',
        impact: 38,
        effort: 'Medium (8-12 hours)'
      });
    }
    } catch (error) {
      console.error('Controllability assessment failed:', error.message);
      testabilityScores.principles.controllability = { score: 50, grade: 'F', weight: config.weights.controllability };
    }
  });

  test('3. Algorithmic Simplicity Assessment', async ({ page }) => {
    try {
      const loaded = await navigateToPage(page);
      if (!loaded) throw new Error('Failed to load page');

    // Measure complexity through interaction patterns
    const interactions = [];

    try {
      // Test product browsing
      const products = await page.locator('[data-product], .product, .product-item').count();
      interactions.push({ action: 'browse', steps: products > 0 ? 2 : 5 });

      // Test cart interaction
      const cartExists = await page.locator('[data-cart], .cart, #cart').count() > 0;
      interactions.push({ action: 'cart', steps: cartExists ? 2 : 4 });

      // Test checkout flow
      const checkoutExists = await page.locator('[href*="checkout"], .checkout-button').count() > 0;
      interactions.push({ action: 'checkout', steps: checkoutExists ? 3 : 6 });
    } catch (e) {
      interactions.push({ action: 'default', steps: 5 });
    }

    const avgComplexity = interactions.reduce((sum, i) => sum + i.steps, 0) / interactions.length;

    // Shopify is well-structured, so base score is high
    const score = Math.max(70, Math.min(100 - (avgComplexity * 5), 95));

    testabilityScores.principles.algorithmicSimplicity = {
      score: Math.round(score),
      grade: getLetterGrade(score),
      weight: config.weights.algorithmicSimplicity
    };
    } catch (error) {
      console.error('Algorithmic Simplicity assessment failed:', error.message);
      testabilityScores.principles.algorithmicSimplicity = { score: 50, grade: 'F', weight: config.weights.algorithmicSimplicity };
    }
  });

  test('4. Algorithmic Transparency Assessment', async ({ page }) => {
    try {
      const loaded = await navigateToPage(page);
      if (!loaded) throw new Error('Failed to load page');

    // Check code readability indicators
    const hasReadableClasses = await page.evaluate(() => {
      const elements = document.querySelectorAll('[class]');
      let readableCount = 0;
      elements.forEach(el => {
        const classes = el.className.toString();
        if (classes.includes('product') || classes.includes('cart') ||
            classes.includes('checkout') || classes.includes('price')) {
          readableCount++;
        }
      });
      return readableCount > 10;
    });

    // Check data attributes
    const hasDataAttributes = await page.evaluate(() => {
      return document.querySelectorAll('[data-product], [data-cart], [data-price]').length > 0;
    });

    let score = 60; // Base score for Shopify structure
    if (hasReadableClasses) score += 10;
    if (hasDataAttributes) score += 10;

    testabilityScores.principles.algorithmicTransparency = {
      score: Math.min(score, 100),
      grade: getLetterGrade(score),
      weight: config.weights.algorithmicTransparency
    };
    } catch (error) {
      console.error('Algorithmic Transparency assessment failed:', error.message);
      testabilityScores.principles.algorithmicTransparency = { score: 50, grade: 'F', weight: config.weights.algorithmicTransparency };
    }
  });

  test('5. Explainability Assessment', async ({ page }) => {
    try {
      const loaded = await navigateToPage(page);
      if (!loaded) throw new Error('Failed to load page');

    // Check for help text and documentation
    const hasHelpText = await page.locator('[aria-label], [title], .help-text').count() > 0;

    // Check for clear error messages
    const hasErrorHandling = await page.evaluate(() => {
      return document.querySelectorAll('[role="alert"], .error, .message').length >= 0;
    });

    // Check for tooltips and guidance
    const hasGuidance = await page.locator('[data-tooltip], .tooltip').count() > 0;

    let score = 50; // Base score
    if (hasHelpText) score += 15;
    if (hasErrorHandling) score += 10;
    if (hasGuidance) score += 7;

    testabilityScores.principles.explainability = {
      score: Math.min(score, 100),
      grade: getLetterGrade(score),
      weight: config.weights.explainability
    };

    if (score < 70) {
      testabilityScores.recommendations.push({
        principle: 'Explainability',
        severity: 'medium',
        recommendation: 'Document API contracts, add OpenAPI specifications, and improve error message clarity for checkout process.',
        impact: 25,
        effort: 'Medium (6-8 hours)'
      });
    }
    } catch (error) {
      console.error('Explainability assessment failed:', error.message);
      testabilityScores.principles.explainability = { score: 50, grade: 'F', weight: config.weights.explainability };
    }
  });

  test('6. Similarity to Known Technology Assessment', async ({ page }) => {
    try {
      const loaded = await navigateToPage(page);
      if (!loaded) throw new Error('Failed to load page');

    // Shopify is a well-known platform
    const usesShopify = await page.evaluate(() => {
      return typeof window.Shopify !== 'undefined' ||
             document.documentElement.innerHTML.includes('Shopify');
    });

    // Check for standard frameworks
    const usesStandardFrameworks = await page.evaluate(() => {
      return typeof window.jQuery !== 'undefined' ||
             document.querySelector('[data-react-root]') !== null ||
             typeof window.Vue !== 'undefined';
    });

    let score = 85; // High base score for Shopify
    if (usesShopify) score += 10;
    if (usesStandardFrameworks) score += 5;

    testabilityScores.principles.similarity = {
      score: Math.round(score),
      grade: getLetterGrade(score),
      weight: config.weights.similarity
    };
    } catch (error) {
      console.error('Similarity assessment failed:', error.message);
      testabilityScores.principles.similarity = { score: 50, grade: 'F', weight: config.weights.similarity };
    }
  });

  test('7. Algorithmic Stability Assessment', async ({ page }) => {
    try {
      const loaded = await navigateToPage(page);
      if (!loaded) throw new Error('Failed to load page');

    // Check for versioning
    const hasVersioning = await page.evaluate(() => {
      return typeof window.Shopify !== 'undefined';
    });

    // Check page consistency
    const pageLoadsConsistently = await page.evaluate(() => {
      return document.readyState === 'complete';
    });

    let score = 60; // Base score
    if (hasVersioning) score += 15;
    if (pageLoadsConsistently) score += 10;

    testabilityScores.principles.algorithmicStability = {
      score: Math.min(score, 100),
      grade: getLetterGrade(score),
      weight: config.weights.algorithmicStability
    };
    } catch (error) {
      console.error('Algorithmic Stability assessment failed:', error.message);
      testabilityScores.principles.algorithmicStability = { score: 50, grade: 'F', weight: config.weights.algorithmicStability };
    }
  });

  test('8. Unbugginess Assessment', async ({ page }) => {
    try {
      const errors = [];
      const warnings = [];

      page.on('console', msg => {
        if (msg.type() === 'error') errors.push(msg);
        if (msg.type() === 'warning') warnings.push(msg);
      });

      page.on('pageerror', err => errors.push(err));

      const loaded = await navigateToPage(page);
      if (!loaded) throw new Error('Failed to load page');

      // Score based on errors
      let score = 95; // Start high
      score -= errors.length * 5;
      score -= warnings.length * 2;
      score = Math.max(score, 0);

      testabilityScores.principles.unbugginess = {
        score: Math.min(score, 100),
        grade: getLetterGrade(score),
        weight: config.weights.unbugginess
      };
    } catch (error) {
      console.error('Unbugginess assessment failed:', error.message);
      testabilityScores.principles.unbugginess = { score: 50, grade: 'F', weight: config.weights.unbugginess };
    }
  });

  test('9. Smallness Assessment', async ({ page }) => {
    try {
      const loaded = await navigateToPage(page);
      if (!loaded) throw new Error('Failed to load page');

      // Measure page size indicators
      const elementCount = await page.evaluate(() => document.querySelectorAll('*').length);
      const scriptCount = await page.evaluate(() => document.querySelectorAll('script').length);
      const styleCount = await page.evaluate(() => document.querySelectorAll('style, link[rel="stylesheet"]').length);

      // Smaller is better
      let score = 100;
      if (elementCount > 1000) score -= 10;
      if (elementCount > 2000) score -= 10;
      if (scriptCount > 20) score -= 5;
      if (styleCount > 10) score -= 5;

      testabilityScores.principles.smallness = {
        score: Math.min(score, 100),
        grade: getLetterGrade(score),
        weight: config.weights.smallness
      };
    } catch (error) {
      console.error('Smallness assessment failed:', error.message);
      testabilityScores.principles.smallness = { score: 50, grade: 'F', weight: config.weights.smallness };
    }
  });

  test('10. Decomposability Assessment', async ({ page }) => {
    try {
      const loaded = await navigateToPage(page);
      if (!loaded) throw new Error('Failed to load page');

    // Check for modular components
    const hasModularStructure = await page.evaluate(() => {
      const hasComponents = document.querySelectorAll('[data-component], [data-module], .component, .module').length > 0;
      const hasSections = document.querySelectorAll('section, [role="region"]').length > 0;
      return hasComponents || hasSections;
    });

    // Check for isolated features
    const hasIsolatedFeatures = await page.evaluate(() => {
      const hasCart = document.querySelector('[data-cart], .cart') !== null;
      const hasProduct = document.querySelector('[data-product], .product') !== null;
      return hasCart && hasProduct;
    });

    let score = 50; // Base score
    if (hasModularStructure) score += 20;
    if (hasIsolatedFeatures) score += 15;

    testabilityScores.principles.decomposability = {
      score: Math.min(score, 100),
      grade: getLetterGrade(score),
      weight: config.weights.decomposability
    };

    if (score < 70) {
      testabilityScores.recommendations.push({
        principle: 'Decomposability',
        severity: 'high',
        recommendation: 'Extract product catalog, shopping cart, and checkout into separate microservices for better isolation and testability.',
        impact: 30,
        effort: 'High (16-24 hours)'
      });
    }
    } catch (error) {
      console.error('Decomposability assessment failed:', error.message);
      testabilityScores.principles.decomposability = { score: 50, grade: 'F', weight: config.weights.decomposability };
    }
  });

  test.afterAll('Calculate Overall Score & Generate Report', async () => {
    // Calculate weighted average
    const principles = testabilityScores.principles;
    const weights = config.weights;

    let totalScore = 0;
    Object.keys(principles).forEach(key => {
      const weight = weights[key] / 100;
      totalScore += principles[key].score * weight;
    });

    testabilityScores.overall = Math.round(totalScore);
    testabilityScores.grade = getLetterGrade(testabilityScores.overall);
    testabilityScores.metadata.duration = Date.now() - new Date(testabilityScores.timestamp).getTime();

    // Sort recommendations by impact
    testabilityScores.recommendations.sort((a, b) => b.impact - a.impact);

    // Save JSON report
    const timestamp = Date.now();
    const jsonPath = path.join(config.reports.directory, `testability-results-${timestamp}.json`);
    const htmlPath = path.join(config.reports.directory, `testability-report-${timestamp}.html`);

    fs.writeFileSync(jsonPath, JSON.stringify(testabilityScores, null, 2));
    fs.writeFileSync(path.join(config.reports.directory, 'latest.json'), JSON.stringify(testabilityScores, null, 2));

    console.log(`\n✅ Assessment Complete!`);
    console.log(`\n📊 Overall Testability Score: ${testabilityScores.overall}/100 (${testabilityScores.grade})`);
    console.log(`\n📄 JSON Report saved: ${jsonPath}`);
    console.log(`\n🎯 Generating HTML report with Chrome auto-launch...`);

    // Generate HTML report using the enhanced script
    const { exec } = require('child_process');
    exec(`node .claude/skills/testability-scorer/scripts/generate-html-report.js "${jsonPath}" "${htmlPath}"`,
      (error, stdout, stderr) => {
        if (error) {
          console.error(`Error generating HTML: ${error.message}`);
          return;
        }
        if (stderr) {
          console.error(`stderr: ${stderr}`);
        }
        console.log(stdout);
      });
  });
});

function getLetterGrade(score) {
  if (score >= 90) return 'A';
  if (score >= 80) return 'B';
  if (score >= 70) return 'C';
  if (score >= 60) return 'D';
  return 'F';
}

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://agentic-qe.dev/schemas/testability-scoring-output.json",
  "title": "Testability Scoring Output Schema",
  "description": "Schema for testability assessment output based on the 10 principles of intrinsic testability (James Bach & Michael Bolton). Validates scoring structure, testability dimensions, and improvement recommendations.",
  "type": "object",
  "required": ["skillName", "version", "timestamp", "status", "trustTier", "output"],
  "properties": {
    "skillName": {
      "type": "string",
      "const": "testability-scoring",
      "description": "Must be 'testability-scoring'"
    },
    "version": {
      "type": "string",
      "pattern": "^\\d+\\.\\d+\\.\\d+(-[a-zA-Z0-9]+)?$",
      "description": "Semantic version of the skill"
    },
    "timestamp": {
      "type": "string",
      "format": "date-time",
      "description": "ISO 8601 timestamp of output generation"
    },
    "status": {
      "type": "string",
      "enum": ["success", "partial", "failed", "skipped"],
      "description": "Overall execution status"
    },
    "trustTier": {
      "type": "integer",
      "minimum": 0,
      "maximum": 3,
      "description": "Trust tier level (3 for full validation)"
    },
    "output": {
      "type": "object",
      "required": ["summary", "score", "categories"],
      "properties": {
        "summary": {
          "type": "string",
          "minLength": 50,
          "maxLength": 2000,
          "description": "Human-readable summary of testability assessment"
        },
        "score": {
          "$ref": "#/$defs/testabilityScore",
          "description": "Overall testability score"
        },
        "findings": {
          "type": "array",
          "items": {
            "$ref": "#/$defs/testabilityFinding"
          },
          "maxItems": 100,
          "description": "Testability issues discovered"
        },
        "recommendations": {
          "type": "array",
          "items": {
            "$ref": "#/$defs/testabilityRecommendation"
          },
          "maxItems": 50,
          "description": "Actionable recommendations for improving testability"
        },
        "metrics": {
          "$ref": "#/$defs/testabilityMetrics",
          "description": "Quantitative testability metrics"
        },
        "categories": {
          "type": "object",
          "required": [
            "observability",
            "controllability",
            "algorithmicSimplicity",
            "algorithmicTransparency",
            "algorithmicStability",
            "explainability",
            "unbugginess",
            "smallness",
            "decomposability",
            "similarity"
          ],
          "properties": {
            "observability": {
              "$ref": "#/$defs/testabilityDimension",
              "description": "Transparency of product states and behavior - Can we see what's happening?"
            },
            "controllability": {
              "$ref": "#/$defs/testabilityDimension",
              "description": "Capacity to provide any input and invoke any state - Can we control the application?"
            },
            "algorithmicSimplicity": {
              "$ref": "#/$defs/testabilityDimension",
              "description": "Clear relationships between inputs and outputs - Are behaviors predictable?"
            },
            "algorithmicTransparency": {
              "$ref": "#/$defs/testabilityDimension",
              "description": "Understanding how the product produces output - Can we understand what it does?"
            },
            "algorithmicStability": {
              "$ref": "#/$defs/testabilityDimension",
              "description": "Changes do not disturb logic - Does behavior remain consistent?"
            },
            "explainability": {
              "$ref": "#/$defs/testabilityDimension",
              "description": "Design understandable to outsiders - Is the interface understandable?"
            },
            "unbugginess": {
              "$ref": "#/$defs/testabilityDimension",
              "description": "Minimal defects that slow testing - How error-free is it?"
            },
            "smallness": {
              "$ref": "#/$defs/testabilityDimension",
              "description": "Less product means less to examine - Are components appropriately sized?"
            },
            "decomposability": {
              "$ref": "#/$defs/testabilityDimension",
              "description": "Parts can be separated for testing - Can we test parts in isolation?"
            },
            "similarity": {
              "$ref": "#/$defs/testabilityDimension",
              "description": "Resemblance to known technology - Is the tech stack familiar?"
            }
          },
          "additionalProperties": false,
          "description": "Breakdown by 10 testability principles"
        },
        "artifacts": {
          "type": "array",
          "items": {
            "$ref": "#/$defs/artifact"
          },
          "maxItems": 20,
          "description": "Generated artifacts (reports, data files)"
        },
        "timeline": {
          "type": "array",
          "items": {
            "$ref": "#/$defs/timelineEvent"
          },
          "description": "Execution timeline for debugging"
        }
      }
    },
    "metadata": {
      "type": "object",
      "properties": {
        "executionTimeMs": {
          "type": "integer",
          "minimum": 0,
          "maximum": 600000,
          "description": "Execution time in milliseconds"
        },
        "toolsUsed": {
          "type": "array",
          "items": {
            "type": "string"
          },
          "description": "Tools/utilities used"
        },
        "agentId": {
          "type": "string",
          "description": "ID of executing agent"
        },
        "modelUsed": {
          "type": "string",
          "description": "LLM model used"
        },
        "inputHash": {
          "type": "string",
          "pattern": "^[a-f0-9]{64}$",
          "description": "SHA-256 hash of input"
        },
        "targetUrl": {
          "type": "string",
          "format": "uri",
          "description": "Target URL assessed"
        },
        "environment": {
          "type": "string",
          "enum": ["development", "staging", "production", "ci"],
          "description": "Environment where skill was executed"
        },
        "retryCount": {
          "type": "integer",
          "minimum": 0,
          "maximum": 10,
          "description": "Number of retries"
        }
      }
    },
    "validation": {
      "type": "object",
      "properties": {
        "schemaValid": {
          "type": "boolean"
        },
        "contentValid": {
          "type": "boolean"
        },
        "confidence": {
          "type": "number",
          "minimum": 0,
          "maximum": 1
        },
        "warnings": {
          "type": "array",
          "items": {
            "type": "string"
          }
        },
        "errors": {
          "type": "array",
          "items": {
            "type": "string"
          }
        },
        "validatorVersion": {
          "type": "string"
        }
      }
    },
    "learning": {
      "type": "object",
      "properties": {
        "patternsDetected": {
          "type": "array",
          "items": {
            "type": "string"
          },
          "description": "Pattern names detected during assessment"
        },
        "reward": {
          "type": "number",
          "minimum": 0,
          "maximum": 1,
          "description": "Reward signal for learning"
        },
        "feedbackLoop": {
          "type": "object",
          "properties": {
            "previousRunId": {
              "type": "string"
            },
            "improvement": {
              "type": "number",
              "minimum": -1,
              "maximum": 1
            }
          }
        }
      }
    }
  },
  "$defs": {
    "testabilityScore": {
      "type": "object",
      "required": ["value", "max"],
      "properties": {
        "value": {
          "type": "number",
          "minimum": 0,
          "maximum": 100,
          "description": "Overall testability score (0-100)"
        },
        "max": {
          "type": "number",
          "const": 100,
          "description": "Maximum possible score (always 100)"
        },
        "grade": {
          "type": "string",
          "pattern": "^[A-F][+-]?$",
          "description": "Letter grade (A=90-100, B=80-89, C=70-79, D=60-69, F=0-59)"
        },
        "percentile": {
          "type": "number",
          "minimum": 0,
          "maximum": 100,
          "description": "Percentile ranking compared to baseline"
        },
        "trend": {
          "type": "string",
          "enum": ["improving", "stable", "declining", "unknown"],
          "description": "Trend compared to previous assessments"
        }
      }
    },
    "testabilityDimension": {
      "type": "object",
      "required": ["score", "weight"],
      "properties": {
        "score": {
          "type": "number",
          "minimum": 0,
          "maximum": 100,
          "description": "Dimension score (0-100)"
        },
        "weight": {
          "type": "number",
          "minimum": 0,
          "maximum": 1,
          "description": "Weight in overall score (must sum to 1.0 across all dimensions)"
        },
        "description": {
          "type": "string",
          "maxLength": 500,
          "description": "Explanation of what this dimension measures"
        },
        "grade": {
          "type": "string",
          "pattern": "^[A-F][+-]?$",
          "description": "Letter grade for this dimension"
        },
        "findingCount": {
          "type": "integer",
          "minimum": 0,
          "description": "Number of findings in this dimension"
        }
      }
    },
    "testabilityFinding": {
      "type": "object",
      "required": ["id", "title", "severity", "category"],
      "properties": {
        "id": {
          "type": "string",
          "pattern": "^TEST-\\d{3,6}$",
          "description": "Unique finding identifier (e.g., TEST-001)"
        },
        "title": {
          "type": "string",
          "minLength": 10,
          "maxLength": 200,
          "description": "Finding title"
        },
        "description": {
          "type": "string",
          "maxLength": 2000,
          "description": "Detailed description"
        },
        "severity": {
          "type": "string",
          "enum": ["critical", "high", "medium", "low", "info"],
          "description": "Severity level"
        },
        "category": {
          "type": "string",
          "enum": [
            "observability",
            "controllability",
            "algorithmicSimplicity",
            "algorithmicTransparency",
            "algorithmicStability",
            "explainability",
            "unbugginess",
            "smallness",
            "decomposability",
            "similarity"
          ],
          "description": "Which testability dimension this finding relates to"
        },
        "location": {
          "type": "object",
          "properties": {
            "url": {
              "type": "string",
              "format": "uri"
            },
            "selector": {
              "type": "string"
            },
            "file": {
              "type": "string"
            },
            "line": {
              "type": "integer"
            }
          }
        },
        "evidence": {
          "type": "string",
          "maxLength": 5000,
          "description": "Evidence supporting the finding"
        },
        "remediation": {
          "type": "string",
          "maxLength": 2000,
          "description": "How to fix this finding"
        },
        "confidence": {
          "type": "number",
          "minimum": 0,
          "maximum": 1,
          "description": "Confidence in this finding (0.0-1.0)"
        }
      }
    },
    "testabilityRecommendation": {
      "type": "object",
      "required": ["id", "title", "priority"],
      "properties": {
        "id": {
          "type": "string",
          "pattern": "^REC-\\d{3,6}$",
          "description": "Unique recommendation identifier"
        },
        "title": {
          "type": "string",
          "minLength": 10,
          "maxLength": 200,
          "description": "Recommendation title"
        },
        "description": {
          "type": "string",
          "maxLength": 2000,
          "description": "Detailed recommendation"
        },
        "priority": {
          "type": "string",
          "enum": ["critical", "high", "medium", "low"],
          "description": "Priority level"
        },
        "effort": {
          "type": "string",
          "enum": ["trivial", "low", "medium", "high", "major"],
          "description": "Estimated effort"
        },
        "impact": {
          "type": "integer",
          "minimum": 1,
          "maximum": 10,
          "description": "Expected improvement impact (1-10 points)"
        },
        "relatedFindings": {
          "type": "array",
          "items": {
            "type": "string",
            "pattern": "^TEST-\\d{3,6}$"
          },
          "description": "IDs of related findings"
        },
        "codeExample": {
          "type": "string",
          "maxLength": 5000,
          "description": "Example code for implementation"
        },
        "resources": {
          "type": "array",
          "items": {
            "type": "object",
            "required": ["title", "url"],
            "properties": {
              "title": {
                "type": "string"
              },
              "url": {
                "type": "string",
                "format": "uri"
              }
            }
          },
          "maxItems": 5,
          "description": "External resources for learning more"
        }
      }
    },
    "testabilityMetrics": {
      "type": "object",
      "properties": {
        "total": {
          "type": "integer",
          "minimum": 0,
          "description": "Total checks performed"
        },
        "passed": {
          "type": "integer",
          "minimum": 0,
          "description": "Checks that passed"
        },
        "failed": {
          "type": "integer",
          "minimum": 0,
          "description": "Checks that failed"
        },
        "skipped": {
          "type": "integer",
          "minimum": 0,
          "description": "Checks skipped"
        },
        "coverage": {
          "type": "number",
          "minimum": 0,
          "maximum": 100,
          "description": "Percentage of principles assessed"
        },
        "duration": {
          "type": "integer",
          "minimum": 0,
          "description": "Assessment duration in milliseconds"
        },
        "custom": {
          "type": "object",
          "properties": {
            "domElementCount": {
              "type": "integer",
              "minimum": 0,
              "description": "Total DOM elements on page"
            },
            "consoleErrorCount": {
              "type": "integer",
              "minimum": 0,
              "description": "JavaScript console errors"
            },
            "consoleWarningCount": {
              "type": "integer",
              "minimum": 0,
              "description": "JavaScript console warnings"
            },
            "dataTestIdCoverage": {
              "type": "number",
              "minimum": 0,
              "maximum": 100,
              "description": "Percentage of interactive elements with data-testid"
            },
            "semanticHtmlScore": {
              "type": "number",
              "minimum": 0,
              "maximum": 100,
              "description": "Semantic HTML usage score"
            },
            "loadTimeMs": {
              "type": "integer",
              "minimum": 0,
              "description": "Page load time in milliseconds"
            },
            "thirdPartyScriptCount": {
              "type": "integer",
              "minimum": 0,
              "description": "Number of third-party scripts"
            },
            "bundleSizeKb": {
              "type": "number",
              "minimum": 0,
              "description": "Total JavaScript bundle size in KB"
            }
          },
          "description": "Custom testability-specific metrics"
        }
      }
    },
    "artifact": {
      "type": "object",
      "required": ["type", "path"],
      "properties": {
        "type": {
          "type": "string",
          "enum": ["report", "data", "screenshot", "video", "log"],
          "description": "Artifact type"
        },
        "path": {
          "type": "string",
          "maxLength": 500,
          "description": "Path to artifact"
        },
        "format": {
          "type": "string",
          "enum": ["json", "html", "md", "txt", "png", "jpg", "webm", "mp4"],
          "description": "File format"
        },
        "description": {
          "type": "string",
          "maxLength": 500,
          "description": "Artifact description"
        },
        "sizeBytes": {
          "type": "integer",
          "minimum": 0,
          "description": "File size in bytes"
        }
      }
    },
    "timelineEvent": {
      "type": "object",
      "required": ["timestamp", "event"],
      "properties": {
        "timestamp": {
          "type": "string",
          "format": "date-time"
        },
        "event": {
          "type": "string",
          "maxLength": 200
        },
        "type": {
          "type": "string",
          "enum": ["start", "checkpoint", "warning", "error", "complete"]
        },
        "durationMs": {
          "type": "integer",
          "minimum": 0
        }
      }
    }
  }
}

#!/usr/bin/env node
/**
 * Testability Scorer - HTML Report Generator
 *
 * Generates professional HTML reports with Chart.js visualizations
 * matching the style from https://github.com/fndlalit/testability-scorer
 */

const fs = require('fs');
const path = require('path');

// Parse command line arguments
const args = process.argv.slice(2);
let reportData = args[0] ? JSON.parse(fs.readFileSync(args[0], 'utf8')) : null;

if (!reportData) {
  console.error('Usage: node generate-html-report.js <results.json>');
  process.exit(1);
}

/**
 * Normalize report data format
 * Handles both legacy format (overall/principles) and new format (overallScore/categories)
 * Also maps generic categories to 10 Testability Principles when needed
 */
function normalizeReportData(data) {
  const normalized = { ...data };
  
  // Normalize overall score
  if (data.overallScore !== undefined && data.overall === undefined) {
    normalized.overall = data.overallScore;
  }
  
  // Map generic categories to 10 Testability Principles if needed
  if (data.categories && !data.principles) {
    const categories = data.categories;
    
    // Check if this is generic quality assessment (functionality, usability, etc.)
    // vs testability principles (observability, controllability, etc.)
    const hasGenericCategories = categories.functionality || categories.usability || categories.performance;
    
    if (hasGenericCategories) {
      // Map to 10 Testability Principles with proper weights
      normalized.principles = {
        observability: {
          score: Math.round((categories.functionality?.score || 75) * 0.9),
          weight: 0.15,
          description: 'Transparency of product states and behavior'
        },
        controllability: {
          score: Math.round((categories.functionality?.score || 75) * 0.95),
          weight: 0.15,
          description: 'Capacity to provide any input and invoke any state'
        },
        algorithmicSimplicity: {
          score: Math.round((categories.maintainability?.score || 75) * 0.9),
          weight: 0.10,
          description: 'Clear relationships between inputs and outputs'
        },
        algorithmicTransparency: {
          score: Math.round((categories.maintainability?.score || 75) * 0.95),
          weight: 0.10,
          description: 'Understanding how the product produces output'
        },
        explainability: {
          score: Math.round((categories.usability?.score || 75) * 0.9),
          weight: 0.10,
          description: 'Design understandable to outsiders'
        },
        similarity: {
          score: Math.round((categories.maintainability?.score || 75) * 0.85),
          weight: 0.05,
          description: 'Resemblance to known technology'
        },
        algorithmicStability: {
          score: Math.round((categories.maintainability?.score || 75) * 0.92),
          weight: 0.10,
          description: 'Changes do not disturb logic'
        },
        unbugginess: {
          score: Math.round((categories.functionality?.score || 75) * 0.88),
          weight: 0.10,
          description: 'Minimal defects that slow testing'
        },
        smallness: {
          score: Math.round((categories.performance?.score || 75) * 0.9),
          weight: 0.10,
          description: 'Less product means less to examine'
        },
        decomposability: {
          score: Math.round((categories.maintainability?.score || 75) * 0.87),
          weight: 0.05,
          description: 'Parts can be separated for testing'
        }
      };
    } else {
      // Already has testability principles, just use them
      normalized.principles = categories;
    }
  }
  
  // Ensure timestamp exists
  if (!normalized.timestamp) {
    normalized.timestamp = data.metadata?.assessmentDate || new Date().toISOString();
  }
  
  // Normalize recommendations to ensure all have required fields
  if (Array.isArray(normalized.recommendations)) {
    normalized.recommendations = normalized.recommendations.map((rec, index) => {
      // If recommendation is just a string, convert to object
      if (typeof rec === 'string') {
        return {
          principle: 'General',
          recommendation: rec,
          severity: index < 3 ? 'critical' : index < 6 ? 'high' : 'medium',
          impact: Math.max(1, 5 - Math.floor(index / 3)),
          effort: 'Medium'
        };
      }
      
      // Ensure all required fields exist
      return {
        principle: rec.principle || 'General',
        recommendation: rec.recommendation || rec.text || 'No description provided',
        severity: rec.severity || 'medium',
        impact: rec.impact !== undefined ? rec.impact : 3,
        effort: rec.effort || 'Medium'
      };
    });
  }
  
  return normalized;
}

// Normalize the data to handle different formats
reportData = normalizeReportData(reportData);

/**
 * Get color for grade
 */
function getGradeColor(score) {
  if (score >= 90) return '#28a745'; // Green (A)
  if (score >= 80) return '#20c997'; // Teal (B)
  if (score >= 70) return '#ffc107'; // Yellow (C)
  if (score >= 60) return '#fd7e14'; // Orange (D)
  return '#dc3545'; // Red (F)
}

/**
 * Get letter grade
 */
function getLetterGrade(score) {
  if (score >= 90) return 'A';
  if (score >= 80) return 'B';
  if (score >= 70) return 'C';
  if (score >= 60) return 'D';
  return 'F';
}

/**
 * Format principle name
 */
function formatPrincipleName(name) {
  return name
    .replace(/([A-Z])/g, ' $1')
    .replace(/^./, str => str.toUpperCase())
    .trim();
}

/**
 * Generate HTML report
 */
function generateHTML(data) {
  const timestamp = new Date(data.timestamp).toLocaleString();
  const overall = data.overall || 0;
  const grade = getLetterGrade(overall);
  const gradeColor = getGradeColor(overall);

  const principles = data.principles || {};
  const recommendations = data.recommendations || [];
  const metadata = data.metadata || {};

  // Prepare chart data
  const principleNames = Object.keys(principles).map(formatPrincipleName);
  const principleScores = Object.values(principles).map(p => p.score || p);

  return `<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Testability Assessment Report - ${overall}/100 (${grade})</title>
  <script src="https://cdn.jsdelivr.net/npm/chart.js@4.4.0/dist/chart.umd.min.js"></script>
  <style>
    * {
      margin: 0;
      padding: 0;
      box-sizing: border-box;
    }

    body {
      font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, sans-serif;
      line-height: 1.6;
      color: #333;
      background: #f5f7fa;
      padding: 20px;
    }

    .container {
      max-width: 1200px;
      margin: 0 auto;
      background: white;
      border-radius: 12px;
      box-shadow: 0 2px 8px rgba(0,0,0,0.1);
      overflow: hidden;
    }

    header {
      background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
      color: white;
      padding: 40px;
      text-align: center;
    }

    h1 {
      font-size: 2.5em;
      margin-bottom: 10px;
      font-weight: 700;
    }

    .subtitle {
      font-size: 1.1em;
      opacity: 0.9;
    }

    .overall-score {
      background: white;
      margin: -30px 40px 0;
      padding: 30px;
      border-radius: 12px;
      box-shadow: 0 4px 12px rgba(0,0,0,0.15);
      text-align: center;
      position: relative;
    }

    .score-display {
      font-size: 5em;
      font-weight: 700;
      color: ${gradeColor};
      line-height: 1;
    }

    .grade-badge {
      display: inline-block;
      background: ${gradeColor};
      color: white;
      padding: 8px 20px;
      border-radius: 20px;
      font-size: 1.5em;
      font-weight: 700;
      margin-top: 10px;
    }

    .metadata {
      display: flex;
      justify-content: space-around;
      margin-top: 20px;
      padding-top: 20px;
      border-top: 2px solid #f0f0f0;
      font-size: 0.9em;
      color: #666;
    }

    .metadata-item {
      text-align: center;
    }

    .metadata-label {
      font-weight: 600;
      color: #333;
      display: block;
      margin-bottom: 5px;
    }

    .content {
      padding: 40px;
    }

    .section {
      margin-bottom: 40px;
    }

    h2 {
      font-size: 1.8em;
      margin-bottom: 20px;
      color: #2c3e50;
      border-bottom: 3px solid #667eea;
      padding-bottom: 10px;
    }

    .chart-container {
      max-width: 600px;
      margin: 0 auto 40px;
      padding: 20px;
      background: #f8f9fa;
      border-radius: 8px;
    }

    .principles-grid {
      display: grid;
      grid-template-columns: repeat(auto-fit, minmax(280px, 1fr));
      gap: 20px;
      margin-top: 30px;
    }

    .principle-card {
      background: #fff;
      border: 2px solid #e0e0e0;
      border-radius: 8px;
      padding: 20px;
      transition: all 0.3s ease;
    }

    .principle-card:hover {
      box-shadow: 0 4px 12px rgba(0,0,0,0.1);
      transform: translateY(-2px);
    }

    .principle-header {
      display: flex;
      justify-content: space-between;
      align-items: center;
      margin-bottom: 15px;
    }

    .principle-name {
      font-size: 1.1em;
      font-weight: 600;
      color: #2c3e50;
    }

    .principle-score {
      font-size: 1.8em;
      font-weight: 700;
    }

    .principle-grade {
      display: inline-block;
      padding: 4px 12px;
      border-radius: 4px;
      color: white;
      font-weight: 600;
      font-size: 0.9em;
      margin-left: 8px;
    }

    .progress-bar {
      height: 8px;
      background: #e0e0e0;
      border-radius: 4px;
      overflow: hidden;
      margin: 10px 0;
    }

    .progress-fill {
      height: 100%;
      background: linear-gradient(90deg, #667eea 0%, #764ba2 100%);
      border-radius: 4px;
      transition: width 1s ease;
    }

    .recommendations {
      margin-top: 30px;
    }

    .breakdown-table {
      width: 100%;
      border-collapse: collapse;
      margin: 30px 0;
      background: white;
      border-radius: 8px;
      overflow: hidden;
      box-shadow: 0 2px 8px rgba(0,0,0,0.1);
    }

    .breakdown-table thead {
      background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
      color: white;
    }

    .breakdown-table th {
      padding: 15px;
      text-align: left;
      font-weight: 600;
      font-size: 0.95em;
    }

    .breakdown-table td {
      padding: 12px 15px;
      border-bottom: 1px solid #f0f0f0;
    }

    .breakdown-table tbody tr:last-child td {
      border-bottom: none;
    }

    .breakdown-table tbody tr:hover {
      background: #f8f9fa;
    }

    .breakdown-table .grade-cell {
      font-size: 1.1em;
      font-weight: 600;
    }

    .breakdown-table .score-cell {
      font-weight: 700;
      font-size: 1.1em;
    }

    .breakdown-table .status-cell {
      font-size: 0.9em;
      color: #666;
    }

    .recommendation-card {
      background: #fff;
      border-left: 4px solid #667eea;
      padding: 20px;
      margin-bottom: 20px;
      border-radius: 4px;
      box-shadow: 0 2px 4px rgba(0,0,0,0.05);
    }

    .recommendation-card.critical {
      border-left-color: #dc3545;
      background: #fff5f5;
    }

    .recommendation-card.high {
      border-left-color: #fd7e14;
      background: #fff8f0;
    }

    .recommendation-card.medium {
      border-left-color: #ffc107;
      background: #fffbf0;
    }

    .recommendation-card.low {
      border-left-color: #20c997;
      background: #f0fdf9;
    }

    .recommendation-header {
      display: flex;
      justify-content: space-between;
      align-items: center;
      margin-bottom: 10px;
    }

    .recommendation-title {
      font-size: 1.2em;
      font-weight: 600;
      color: #2c3e50;
    }

    .severity-badge {
      padding: 4px 12px;
      border-radius: 4px;
      color: white;
      font-weight: 600;
      font-size: 0.85em;
      text-transform: uppercase;
    }

    .severity-critical { background: #dc3545; }
    .severity-high { background: #fd7e14; }
    .severity-medium { background: #ffc107; color: #333; }
    .severity-low { background: #20c997; }

    .recommendation-text {
      color: #555;
      line-height: 1.8;
      margin-top: 10px;
    }

    .recommendation-impact {
      margin-top: 15px;
      padding: 10px;
      background: rgba(102, 126, 234, 0.1);
      border-radius: 4px;
      font-size: 0.9em;
    }

    .impact-label {
      font-weight: 600;
      color: #667eea;
    }

    .grade-distribution {
      display: flex;
      justify-content: space-around;
      margin: 30px 0;
      padding: 20px;
      background: #f8f9fa;
      border-radius: 8px;
    }

    .grade-item {
      text-align: center;
    }

    .grade-count {
      font-size: 2em;
      font-weight: 700;
      display: block;
      margin-bottom: 5px;
    }

    .grade-label {
      color: #666;
      font-size: 0.9em;
    }

    footer {
      background: #2c3e50;
      color: white;
      padding: 30px;
      text-align: center;
      margin-top: 40px;
    }

    .footer-links {
      margin-top: 15px;
    }

    .footer-links a {
      color: #667eea;
      text-decoration: none;
      margin: 0 15px;
    }

    .footer-links a:hover {
      text-decoration: underline;
    }

    @media print {
      body {
        background: white;
        padding: 0;
      }

      .container {
        box-shadow: none;
      }

      .principle-card:hover {
        transform: none;
      }
    }

    .emoji {
      font-size: 1.2em;
      margin-right: 8px;
    }

    .timestamp {
      color: #666;
      font-size: 0.9em;
      margin-top: 10px;
    }

    .status-icon {
      font-size: 1.5em;
      margin-left: 10px;
    }
  </style>
</head>
<body>
  <div class="container">
    <header>
      <h1>🎯 Testability Assessment Report</h1>
      <p class="subtitle">Comprehensive analysis using 10 principles of intrinsic testability</p>
    </header>

    <div class="overall-score">
      <div>
        <span class="score-display">${overall}<span style="font-size: 0.5em; color: #999;">/100</span></span>
        <div class="grade-badge">${grade}</div>
      </div>

      <div class="metadata">
        <div class="metadata-item">
          <span class="metadata-label">📅 Date</span>
          ${timestamp}
        </div>
        <div class="metadata-item">
          <span class="metadata-label">🌐 URL</span>
          ${metadata.targetURL || metadata.url || 'N/A'}
        </div>
        <div class="metadata-item">
          <span class="metadata-label">🖥️ Browser</span>
          ${metadata.browser || 'Chromium'}
        </div>
        <div class="metadata-item">
          <span class="metadata-label">⏱️ Duration</span>
          ${typeof metadata.duration === 'number' ? Math.round(metadata.duration / 1000) + 's' : metadata.duration || 'N/A'}
        </div>
      </div>
    </div>

    <div class="content">
      <!-- Grade Distribution -->
      <section class="section">
        <h2>📊 Grade Distribution</h2>
        <div class="grade-distribution">
          ${['A', 'B', 'C', 'D', 'F'].map(g => {
            const count = Object.values(principles).filter(p => {
              const s = p.score || p;
              return getLetterGrade(s) === g;
            }).length;
            return `
              <div class="grade-item">
                <span class="grade-count" style="color: ${getGradeColor(g === 'A' ? 95 : g === 'B' ? 85 : g === 'C' ? 75 : g === 'D' ? 65 : 55)}">${count}</span>
                <span class="grade-label">Grade ${g}</span>
              </div>
            `;
          }).join('')}
        </div>
      </section>

      <!-- Radar Chart -->
      <section class="section">
        <h2>📈 Testability Radar</h2>
        <div class="chart-container">
          <canvas id="radarChart"></canvas>
        </div>
      </section>

      <!-- Principle Scores -->
      <section class="section">
        <h2>🎯 Principle Scores</h2>
        <div class="principles-grid">
          ${Object.entries(principles).map(([key, value]) => {
            const score = value.score || value;
            const grade = getLetterGrade(score);
            const color = getGradeColor(score);
            // Match status icons to grade scale: A/B = ✓, C = ●, D/F = ✗
            // Use colored circles for clearer visual distinction
            let statusIcon = '';
            if (score >= 80) {
              statusIcon = `<span style="color: ${color}; font-weight: bold;">✓</span>`;
            } else if (score >= 70) {
              statusIcon = `<span style="color: ${color}; font-weight: bold;">●</span>`;
            } else {
              statusIcon = `<span style="color: ${color}; font-weight: bold;">✗</span>`;
            }

            return `
              <div class="principle-card">
                <div class="principle-header">
                  <div class="principle-name">${formatPrincipleName(key)}</div>
                  <span class="status-icon">${statusIcon}</span>
                </div>
                <div>
                  <span class="principle-score" style="color: ${color}">${score}</span>
                  <span class="principle-grade" style="background: ${color}">${grade}</span>
                </div>
                <div class="progress-bar">
                  <div class="progress-fill" style="width: ${score}%; background: ${color}"></div>
                </div>
              </div>
            `;
          }).join('')}
        </div>
      </section>

      <!-- Principle Breakdown Table -->
      <section class="section">
        <h2>📋 Principle Breakdown</h2>
        <table class="breakdown-table">
          <thead>
            <tr>
              <th>Grade</th>
              <th>Principle</th>
              <th>Score</th>
              <th>Status</th>
            </tr>
          </thead>
          <tbody>
            ${Object.entries(principles)
              .map(([key, value]) => ({
                key,
                score: value.score || value,
                grade: getLetterGrade(value.score || value),
                color: getGradeColor(value.score || value)
              }))
              .sort((a, b) => b.score - a.score)
              .map(item => {
                let gradeEmoji = '';
                let statusText = '';
                let statusIcon = '';
                
                if (item.score >= 90) {
                  gradeEmoji = '🟢 A';
                  statusText = '✓ Excellent';
                  statusIcon = '✓';
                } else if (item.score >= 80) {
                  gradeEmoji = '🟢 B';
                  statusText = '✓ Good';
                  statusIcon = '✓';
                } else if (item.score >= 70) {
                  gradeEmoji = '🟡 C';
                  statusText = '● Needs improvement';
                  statusIcon = '●';
                } else if (item.score >= 60) {
                  gradeEmoji = '🟠 D';
                  statusText = '✗ Poor';
                  statusIcon = '✗';
                } else {
                  gradeEmoji = '🔴 F';
                  statusText = '✗ Critical';
                  statusIcon = '✗';
                }

                return `
                  <tr>
                    <td class="grade-cell">${gradeEmoji}</td>
                    <td><strong>${formatPrincipleName(item.key)}</strong></td>
                    <td class="score-cell" style="color: ${item.color}">${item.score}</td>
                    <td class="status-cell">${statusText}</td>
                  </tr>
                `;
              }).join('')}
          </tbody>
        </table>
      </section>

      <!-- Recommendations -->
      ${recommendations.length > 0 ? `
      <section class="section recommendations">
        <h2>💡 Improvement Recommendations</h2>
        <p style="color: #666; margin-bottom: 20px;">
          ${recommendations.length} recommendation${recommendations.length > 1 ? 's' : ''} based on assessment results
        </p>

        ${recommendations.map((rec, index) => `
          <div class="recommendation-card ${rec.severity}">
            <div class="recommendation-header">
              <span class="recommendation-title">
                ${rec.principle}
              </span>
              <span class="severity-badge severity-${rec.severity}">${rec.severity}</span>
            </div>
            <div class="recommendation-text">
              ${rec.recommendation}
            </div>
            ${rec.impact ? `
              <div class="recommendation-impact">
                <span class="impact-label">Expected Impact:</span>
                +${rec.impact} points improvement
              </div>
            ` : ''}
            ${rec.effort ? `
              <div style="margin-top: 10px; color: #666; font-size: 0.9em;">
                <strong>Effort:</strong> ${rec.effort}
              </div>
            ` : ''}
          </div>
        `).join('')}
      </section>
      ` : ''}

      <!-- Summary -->
      <section class="section">
        <h2>📝 Summary</h2>
        <div style="background: #f8f9fa; padding: 20px; border-radius: 8px; line-height: 1.8;">
          ${overall >= 90 ? `
            <p><strong style="color: #28a745;">Excellent testability!</strong> Your application demonstrates outstanding testability across all principles. Continue maintaining these high standards.</p>
          ` : overall >= 70 ? `
            <p><strong style="color: #ffc107;">Good testability with room for improvement.</strong> Your application has solid fundamentals but could benefit from addressing the recommendations above.</p>
          ` : overall >= 50 ? `
            <p><strong style="color: #fd7e14;">Acceptable testability but needs work.</strong> Focus on critical and high-priority recommendations to significantly improve test automation capabilities.</p>
          ` : `
            <p><strong style="color: #dc3545;">Poor testability - urgent improvements needed.</strong> Implementing the critical recommendations will dramatically improve your ability to test this application effectively.</p>
          `}

          <div style="margin-top: 20px; padding-top: 20px; border-top: 2px solid #e0e0e0;">
            <strong>Next Steps:</strong>
            <ol style="margin-top: 10px; padding-left: 20px;">
              <li>Review and prioritize the recommendations above</li>
              <li>Implement critical and high-priority fixes first</li>
              <li>Re-run assessment after improvements</li>
              <li>Track progress over time</li>
            </ol>
          </div>
        </div>
      </section>
    </div>

    <footer>
      <p><strong>Generated by Testability Scorer</strong></p>
      <p style="margin-top: 10px; font-size: 0.9em; opacity: 0.8;">
        Based on 10 Principles of Intrinsic Testability
      </p>
      <div class="footer-links">
        <a href="https://github.com/fndlalit/testability-scorer" target="_blank">Original Framework</a>
        <a href="https://playwright.dev/" target="_blank">Powered by Playwright</a>
        <a href="https://github.com/ruvnet/claude-flow" target="_blank">Agentic QE Fleet</a>
      </div>
      <p class="timestamp" style="margin-top: 20px;">
        Report generated: ${new Date().toLocaleString()}
      </p>
    </footer>
  </div>

  <script>
    // Radar Chart
    const ctx = document.getElementById('radarChart').getContext('2d');
    new Chart(ctx, {
      type: 'radar',
      data: {
        labels: ${JSON.stringify(principleNames)},
        datasets: [{
          label: 'Testability Score',
          data: ${JSON.stringify(principleScores)},
          borderColor: '#667eea',
          backgroundColor: 'rgba(102, 126, 234, 0.2)',
          borderWidth: 3,
          pointBackgroundColor: '#667eea',
          pointBorderColor: '#fff',
          pointBorderWidth: 2,
          pointRadius: 5,
          pointHoverRadius: 7
        }]
      },
      options: {
        responsive: true,
        maintainAspectRatio: true,
        scales: {
          r: {
            min: 0,
            max: 100,
            beginAtZero: true,
            ticks: {
              stepSize: 20,
              font: {
                size: 12
              }
            },
            pointLabels: {
              font: {
                size: 13,
                weight: '600'
              }
            },
            grid: {
              color: 'rgba(0, 0, 0, 0.1)'
            }
          }
        },
        plugins: {
          legend: {
            display: false
          },
          tooltip: {
            callbacks: {
              label: function(context) {
                return context.parsed.r + '/100 (' + getLetterGrade(context.parsed.r) + ')';
              }
            }
          }
        }
      }
    });

    function getLetterGrade(score) {
      if (score >= 90) return 'A';
      if (score >= 80) return 'B';
      if (score >= 70) return 'C';
      if (score >= 60) return 'D';
      return 'F';
    }

    // Animate progress bars on load
    window.addEventListener('load', () => {
      document.querySelectorAll('.progress-fill').forEach(bar => {
        const width = bar.style.width;
        bar.style.width = '0%';
        setTimeout(() => {
          bar.style.width = width;
        }, 100);
      });
    });
  </script>
</body>
</html>`;
}

// Generate and save HTML report
const html = generateHTML(reportData);
const outputPath = args[1] || `tests/reports/testability-report-${Date.now()}.html`;

// Ensure directory exists
const dir = path.dirname(outputPath);
if (!fs.existsSync(dir)) {
  fs.mkdirSync(dir, { recursive: true });
}

fs.writeFileSync(outputPath, html);

console.log(`✓ HTML report generated: ${outputPath}`);
console.log(`✓ Overall score: ${reportData.overall}/100 (${getLetterGrade(reportData.overall)})`);

// Auto-open by default (disable with AUTO_OPEN=false)
if (process.env.AUTO_OPEN !== 'false') {
  console.log(`\n🌐 Starting HTTP server...`);

  const { exec } = require('child_process');
  const http = require('http');
  const fs = require('fs');
  const absolutePath = path.resolve(outputPath);
  const reportDir = path.dirname(absolutePath);
  const reportFile = path.basename(absolutePath);

  // Find a free port starting from 8080
  const findFreePort = (startPort = 8080) => {
    return new Promise((resolve) => {
      const server = http.createServer();
      server.listen(startPort, () => {
        const port = server.address().port;
        server.close(() => resolve(port));
      }).on('error', () => {
        resolve(findFreePort(startPort + 1));
      });
    });
  };

  findFreePort().then(port => {
    // Create Node.js HTTP server (more reliable than Python in containers)
    const server = http.createServer((_req, res) => {
      const filePath = path.join(reportDir, reportFile);
      fs.readFile(filePath, (err, data) => {
        if (err) {
          res.writeHead(500);
          res.end('Error loading report');
        } else {
          res.writeHead(200, { 'Content-Type': 'text/html' });
          res.end(data);
        }
      });
    });

    server.listen(port, '0.0.0.0', () => {
      const reportUrl = `http://localhost:${port}`;

      // Display prominent, clickable URL
      console.log(`\n┌─────────────────────────────────────────────────────────────┐`);
      console.log(`│                                                             │`);
      console.log(`│  ✅ HTTP Server Running on Port ${port}                       │`);
      console.log(`│                                                             │`);
      console.log(`│  📊 CLICK HERE TO OPEN REPORT:                              │`);
      console.log(`│                                                             │`);
      console.log(`│     ${reportUrl}                              │`);
      console.log(`│                                                             │`);
      console.log(`│  (VS Code will forward this port automatically)            │`);
      console.log(`│                                                             │`);
      console.log(`└─────────────────────────────────────────────────────────────┘\n`);
      console.log(`💡 Server will keep running until you stop it (Ctrl+C)\n`);

      // Auto-open in browser (works in VS Code Dev Container)
      setTimeout(() => {
        const openCommand = process.platform === 'darwin' ? 'open' :
                           process.platform === 'win32' ? 'start' :
                           'xdg-open';
        exec(`${openCommand} ${reportUrl}`, (err) => {
          if (err) {
            console.log(`⚠️  Could not auto-open browser: ${err.message}`);
            console.log(`📌 Please manually open: ${reportUrl}`);
          } else {
            console.log(`🚀 Browser opened automatically!`);
          }
        });
      }, 1000);
    });

    server.on('error', (err) => {
      console.error(`❌ Server error: ${err.message}`);
      console.log(`\n📄 Report saved to: ${absolutePath}`);
    });
  }).catch(err => {
    console.error(`❌ Failed to start server: ${err.message}`);
    console.log(`\n📄 Report saved to: ${absolutePath}`);
  });
} else {
  console.log(`\nView report:`);
  console.log(`  google-chrome ${outputPath}`);
  console.log(`  # or`);
  console.log(`  open ${outputPath}`);
}

// Don't exit immediately - let server keep running
if (process.env.AUTO_OPEN !== 'false') {
  console.log(`\n💡 Tip: Set AUTO_OPEN=false to disable automatic browser opening`);
  console.log(`💡 Server is running in background. Kill process to stop.`);
}

#!/bin/bash
set -e

# Testability Scoring Assessment Runner
# Usage: ./run-assessment.sh <URL> [browser]

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../../../.." && pwd)"
TEST_FILE="$PROJECT_ROOT/tests/testability-scoring/testability-scoring.spec.js"

# Colors
GREEN='\033[0;32m'
BLUE='\033[0;34m'
YELLOW='\033[1;33m'
RED='\033[0;31m'
NC='\033[0m' # No Color

# Check if URL provided
if [ -z "$1" ]; then
    echo -e "${RED}Error: URL required${NC}"
    echo "Usage: $0 <URL> [browser]"
    echo "Example: $0 https://example.com chromium"
    exit 1
fi

TARGET_URL="$1"
BROWSER="${2:-chromium}"

echo -e "${BLUE}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
echo -e "${GREEN}🎯 Testability Assessment${NC}"
echo -e "${BLUE}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
echo ""
echo -e "  ${YELLOW}URL:${NC}     $TARGET_URL"
echo -e "  ${YELLOW}Browser:${NC} $BROWSER"
echo ""
echo -e "${BLUE}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
echo ""

# Change to project root
cd "$PROJECT_ROOT"

# Run Playwright tests with TEST_URL environment variable
echo -e "${YELLOW}⏳ Running assessment...${NC}\n"

TEST_URL="$TARGET_URL" npx playwright test "$TEST_FILE" --project="$BROWSER" --workers=1

# Check if tests passed
if [ $? -eq 0 ]; then
    echo ""
    echo -e "${GREEN}✅ Assessment completed successfully!${NC}"
    
    # Find the latest JSON report
    LATEST_JSON=$(ls -t "$PROJECT_ROOT/tests/reports/testability-results-"*.json 2>/dev/null | head -1)
    
    if [ -n "$LATEST_JSON" ]; then
        # Generate HTML report
        echo -e "\n${YELLOW}📊 Generating HTML report...${NC}\n"
        node "$SCRIPT_DIR/generate-html-report.js" "$LATEST_JSON"
        
        echo ""
        echo -e "${GREEN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
        echo -e "${GREEN}✓ Complete! Check your browser for the report.${NC}"
        echo -e "${GREEN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
    else
        echo -e "${YELLOW}⚠️  JSON report not found${NC}"
    fi
else
    echo -e "\n${RED}❌ Assessment failed${NC}"
    exit 1
fi

{
  "skillName": "testability-scoring",
  "skillVersion": "2.2.0",
  "requiredTools": [
    "jq"
  ],
  "optionalTools": [
    "node",
    "ajv",
    "jsonschema",
    "python3"
  ],
  "schemaPath": "schemas/output.json",
  "requiredFields": [
    "skillName",
    "status",
    "output",
    "output.score",
    "output.categories"
  ],
  "requiredNonEmptyFields": [
    "output.summary",
    "output.score.value"
  ],
  "mustContainTerms": [
    "testability",
    "score"
  ],
  "mustNotContainTerms": [
    "TODO",
    "placeholder",
    "REPLACE_WITH"
  ],
  "enumValidations": {
    ".status": [
      "success",
      "partial",
      "failed",
      "skipped"
    ]
  }
}

Related skills

TddFollow test-driven development with a strict red-green-refactor loop when creating reliable features or fixing bugs.544k196k

Test Driven DevelopmentEnforce writing failing tests before any production implementation code.181k264k

QaRun conversational QA sessions that turn user-reported bugs into well-written, domain-aware GitHub issues without manual ticket writing.179k196k

Migrate To ShoehornAutomatically update TypeScript test files that rely on unsafe `as` type assertions by replacing them with type-safe partial objects from @total-typescript/shoehorn.164k196k

Webapp TestingVerify frontend behavior, debug UI issues, capture screenshots, and inspect logs of a running local web application using Playwright.123k165k

Playwright CliRun browser automation, generate element snapshots, inspect DOM attributes, and execute Playwright tests from the terminal.102k12.3k

FAQ

What does testability-scoring do?

testability-scoring is a Claude Code skill for testing & qa.

When should I use testability-scoring?

When you need to helps with testing & qa tasks., or when testability-scoring is a claude code skill for testing & qa.

What are the main capabilities?

testability-scoring; Testing & QA; AI-coding skill.

Testing & QAtesting

About

Testability Scoring by the numbers

Add your badge

How do I helps with testing & qa tasks.?

Who is it for?

When should I use this skill?

What you get

Files

Testability Scoring

Browser engine

Quick Reference Card

Running Assessments

Principle Details

High Weight (15% each)

Medium Weight (10% each)

Low Weight (5% each)

Assessment Workflow

Output Files

Integration Examples

CI/CD Integration

QX Partner Integration

Programmatic Usage

Agent Integration

Vibium Integration (Optional)

Overview

Vibium-Enhanced Metrics

When to Use Vibium

Hybrid Assessment Example

Vibium MCP Tools for Testability

Migration Strategy

Agent Coordination Hints

Memory Namespace

Fleet Coordination

Common Issues & Solutions

Related Skills

Credits & References

Framework Origin

Implementation

Vibium Resources

Remember

Testability Scoring Skill

Quick Start

10 Principles

Scoring

Files

Common Commands

Credits

Related skills

FAQ

What does testability-scoring do?

When should I use testability-scoring?

What are the main capabilities?

This week in AI coding