End To End Testing

Name: End To End Testing
Author: aws-samples

aws-samples/sample-agent-skills-for-builders

28 installs
41 repo stars
Updated July 6, 2026
aws-samples/sample-agent-skills-for-builders

end-to-end-testing is a Claude skill that runs a systematic E2E testing workflow with evidence capture, defect documentation, and reporting.

About

This skill runs a systematic end-to-end testing workflow for validating complete user flows. A developer uses it to plan test cases, execute them, capture evidence such as screenshots and logs, document defects, and generate reports including an HTML viewer. It enforces a mandatory pre-reporting validation phase before results are reported.

Systematic 5-phase E2E testing workflow with evidence capture
Captures screenshots and logs, documents defects, generates reports
Mandatory pre-reporting validation phase before results are shared

End To End Testing by the numbers

28 all-time installs (skills.sh)
Ranked #1,360 of 2,153 Testing & QA skills by installs in the Skillselion catalog
Data as of Jul 30, 2026 (Skillselion catalog sync)

At a glance

end-to-end-testing capabilities & compatibility

Capabilities: code review · security audit
Use cases: testing · debugging
Pricing: Free

From the docs

What end-to-end-testing says it does

Systematic E2E testing workflow with evidence capture and reporting.

SKILL.md

Systematic E2E testing workflow for complete system validation with evidence capture.

SKILL.md

Phase 4: Pre-Reporting Validation (CRITICAL)

SKILL.md

npx skills add https://github.com/aws-samples/sample-agent-skills-for-builders --skill end-to-end-testing

Add your badge

Show developers this skill is listed on Skillselion. Paste this into your README.

[![Listed on Skillselion](https://skillselion.com/badge/skills/aws-samples/sample-agent-skills-for-builders/end-to-end-testing.svg)](https://skillselion.com/skills/aws-samples/sample-agent-skills-for-builders/end-to-end-testing)

Installs	28
repo stars	★ 41
Last updated	July 6, 2026
Repository	aws-samples/sample-agent-skills-for-builders ↗

What it does

Run a systematic E2E testing workflow that validates user flows, captures evidence, and generates test reports.

Who is it for?

Developers running structured integration and E2E test passes that need evidence and reports.

When should I use this skill?

When running integration or E2E tests, validating complete user flows, or generating test reports.

What you get

A validated test run with captured evidence, documented defects, and reports including an HTML viewer.

Test_Report_Viewer.html
test-case-reports
defect-reports

By the numbers

5-phase testing workflow
mandatory pre-reporting validation phase

Files

SKILL.mdMarkdownGitHub ↗

End-to-End Testing

Systematic E2E testing workflow for complete system validation with evidence capture.

When to Apply

Reference this skill when:

Running integration or E2E tests
Validating complete user flows
Capturing test evidence (screenshots, logs)
Generating test reports
Documenting defects systematically

How It Works

Phase 1: Preparation

Gather environment information
Verify test credentials
Set up test data

Phase 2: Planning & Design

Smoke testing
Test case design
Priority assignment

Phase 3: Execution

Execute test cases
Capture evidence (screenshots)
Document defects

Phase 4: Pre-Reporting Validation (CRITICAL)

MANDATORY before reporting
Verify all evidence captured
Validate defect documentation
Cross-check test results

Phase 5: Reporting

Generate comprehensive reports
Create HTML viewer
Archive evidence

Output Structure

test-reports/
└── YYYY-MM-DD_HHMMSS/
    ├── Test_Report_Viewer.html
    ├── test-case-reports/
    ├── defect-reports/
    ├── test-summary-report.md
    ├── screenshots/
    └── logs/

Usage

# Generate HTML report
python scripts/generate-html-report.py --input test-results/

# Validate test data
./scripts/validate-test-data.sh

Templates

Defect Report Template
Test Case Report Template
Test Summary Template

References

Preparation
Planning & Design
Execution
Validation
Reporting

🐞 Defect Report

---

📌 1. Essential Metadata

Field	Value
Defect ID	[D-MODULE-YYYYMMDD-NNN]
Summary	[[Module] + [Action] + [Unexpected Result]]
Reporter	[Your Name (email@company.com)]
Date Found	[YYYY-MM-DD HH:MM]
Status	[New / Assigned / In Progress / To Be Retested / Closed / Resolved]
Assigned To	[Developer Name (email@company.com) - Team Name]
Module	[Specific functional module or component]
Test Version	[Build number or version identifier]

---

📝 2. Detailed Description

Steps to Reproduce

1. [Clear, atomic step 1] 2. [Clear, atomic step 2] 3. [Clear, atomic step 3] 4. [Continue as needed...]

Expected Result

[Objective and specific description of correct system behavior according to requirements]

Actual Result

[Objective description of observed incorrect behavior - avoid subjective judgments]

Environment

Component	Details
Browser/App Version	[e.g., Chrome 120.0.6099.109]
Operating System	[e.g., Windows 11, macOS 14.1, Ubuntu 22.04]
Test URL	[e.g., https://test.example.com]
Database Version	[if applicable]
Network Conditions	[if relevant, e.g., VPN, cellular]
Other	[Any other relevant environment details]

Attachments

[ ] Screenshot showing the defect (required)
[ ] Browser console logs (if frontend issue)
[ ] Server/application logs (if backend issue)
[ ] Network trace/API response data (if API issue)
[ ] Screen recording (for complex interactions)
[ ] Additional evidence: [list any other supporting files]

---

🔥 3. Severity & Priority

Field	Value	Justification
Severity	[S1 (Critical) / S2 (Major) / S3 (Minor) / S4 (Trivial)]	[Explain the technical impact on the system]
Priority	[P1 (Urgent) / P2 (High) / P3 (Medium) / P4 (Low)]	[Explain the business urgency]

Severity Guidelines

S1 (Critical): System crash, data loss, security breach, or complete application failure
S2 (Major): Core functionality unavailable or severely impaired; no workaround exists
S3 (Minor): Non-core functionality impaired; workaround available
S4 (Trivial): Cosmetic issues, UI/text errors, minor inconveniences

Priority Guidelines

P1 (Urgent): Must fix immediately; blocks testing or production release
P2 (High): Must fix in next iteration; significant business impact
P3 (Medium): Can be scheduled for fix in upcoming sprints
P4 (Low): Fix when time permits; minimal impact

---

📎 Additional Notes

[Any additional observations, related test cases, potential root causes, or relevant information]

---

✅ Verification Steps (for retesting after fix)

1. [Step to verify the fix] 2. [Additional verification steps] 3. [Regression testing to ensure no side effects]

<!DOCTYPE html>
<html lang="zh-CN">
<!--
    ========================================
    Test Execution Report Viewer Template
    ========================================

    This HTML template serves as a reference for generating interactive test reports.

    USAGE:
    1. Use the automated script: python skills/end-to-end-testing/scripts/generate-html-report.py <test-reports-dir>
    2. Or manually populate the template with test data from markdown files

    DIRECTORY STRUCTURE:
    The HTML file should be placed in: test-reports/YYYY-MM-DD_HHMMSS/Test_Report_Viewer.html

    ASSET PATHS:
    - Screenshots: screenshots/ (relative to this HTML file)
    - Defect reports: defect-reports/ (relative to this HTML file)
    - Logs: logs/ (relative to this HTML file)

    DATA SOURCES:
    - Executive summary: test-summary-report.md Section 1
    - Test statistics: test-case-reports/*.md (count by status)
    - Test cases: test-case-reports/*.md
    - Defects: defect-reports/*.md
    - Screenshots: Auto-discovered from screenshots/ directory
    - Logs: Auto-discovered from logs/ directory

    NOTE: This is an example template with hardcoded data. Replace with dynamic content from actual test run.
    ========================================
-->
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Test Execution Report Viewer</title>
    <style>
        * {
            margin: 0;
            padding: 0;
            box-sizing: border-box;
        }

        body {
            font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, 'Helvetica Neue', Arial, sans-serif;
            background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
            min-height: 100vh;
            padding: 20px;
        }

        .container {
            max-width: 1400px;
            margin: 0 auto;
            background: white;
            border-radius: 12px;
            box-shadow: 0 20px 60px rgba(0, 0, 0, 0.3);
            overflow: hidden;
        }

        .header {
            background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
            color: white;
            padding: 40px;
            text-align: center;
        }

        .header h1 {
            font-size: 2.5em;
            margin-bottom: 10px;
            font-weight: 700;
        }

        .header p {
            font-size: 1.1em;
            opacity: 0.95;
        }

        .stats-grid {
            display: grid;
            grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
            gap: 20px;
            padding: 30px;
            background: #f8f9fa;
        }

        .stat-card {
            background: white;
            padding: 25px;
            border-radius: 8px;
            box-shadow: 0 2px 8px rgba(0, 0, 0, 0.1);
            text-align: center;
            transition: transform 0.2s;
        }

        .stat-card:hover {
            transform: translateY(-5px);
            box-shadow: 0 4px 12px rgba(0, 0, 0, 0.15);
        }

        .stat-card .number {
            font-size: 3em;
            font-weight: 700;
            margin-bottom: 10px;
        }

        .stat-card .label {
            font-size: 0.9em;
            color: #666;
            text-transform: uppercase;
            letter-spacing: 1px;
        }

        .stat-card.passed .number { color: #28a745; }
        .stat-card.failed .number { color: #dc3545; }
        .stat-card.blocked .number { color: #ffc107; }
        .stat-card.total .number { color: #667eea; }

        .tabs {
            display: flex;
            background: #f8f9fa;
            border-bottom: 2px solid #dee2e6;
            padding: 0 30px;
            overflow-x: auto;
        }

        .tab {
            padding: 15px 30px;
            cursor: pointer;
            border: none;
            background: none;
            font-size: 1em;
            color: #666;
            transition: all 0.3s;
            position: relative;
            white-space: nowrap;
        }

        .tab:hover {
            color: #667eea;
            background: rgba(102, 126, 234, 0.05);
        }

        .tab.active {
            color: #667eea;
            font-weight: 600;
        }

        .tab.active::after {
            content: '';
            position: absolute;
            bottom: -2px;
            left: 0;
            right: 0;
            height: 2px;
            background: #667eea;
        }

        .content {
            padding: 30px;
        }

        .tab-content {
            display: none;
        }

        .tab-content.active {
            display: block;
            animation: fadeIn 0.3s;
        }

        @keyframes fadeIn {
            from { opacity: 0; transform: translateY(10px); }
            to { opacity: 1; transform: translateY(0); }
        }

        .test-case {
            background: white;
            border: 1px solid #dee2e6;
            border-radius: 8px;
            margin-bottom: 20px;
            overflow: hidden;
            box-shadow: 0 2px 4px rgba(0, 0, 0, 0.05);
        }

        .test-case-header {
            padding: 20px;
            background: #f8f9fa;
            cursor: pointer;
            display: flex;
            justify-content: space-between;
            align-items: center;
            transition: background 0.2s;
        }

        .test-case-header:hover {
            background: #e9ecef;
        }

        .test-case-title {
            font-size: 1.2em;
            font-weight: 600;
            display: flex;
            align-items: center;
            gap: 15px;
        }

        .status-badge {
            padding: 6px 12px;
            border-radius: 20px;
            font-size: 0.85em;
            font-weight: 600;
            text-transform: uppercase;
            letter-spacing: 0.5px;
        }

        .status-badge.pass {
            background: #d4edda;
            color: #155724;
        }

        .status-badge.blocked {
            background: #fff3cd;
            color: #856404;
        }

        .test-case-body {
            padding: 20px;
            display: none;
        }

        .test-case.expanded .test-case-body {
            display: block;
        }

        .test-case.expanded .toggle-icon::before {
            content: '▼';
        }

        .toggle-icon::before {
            content: '▶';
            display: inline-block;
            transition: transform 0.3s;
        }

        .info-grid {
            display: grid;
            grid-template-columns: repeat(auto-fit, minmax(250px, 1fr));
            gap: 15px;
            margin-bottom: 20px;
        }

        .info-item {
            padding: 15px;
            background: #f8f9fa;
            border-radius: 6px;
        }

        .info-item .label {
            font-weight: 600;
            color: #495057;
            margin-bottom: 5px;
            font-size: 0.9em;
        }

        .info-item .value {
            color: #212529;
            font-size: 1em;
        }

        .screenshot-grid {
            display: grid;
            grid-template-columns: repeat(auto-fill, minmax(300px, 1fr));
            gap: 20px;
            margin-top: 20px;
        }

        .screenshot-card {
            background: white;
            border: 1px solid #dee2e6;
            border-radius: 8px;
            overflow: hidden;
            box-shadow: 0 2px 4px rgba(0, 0, 0, 0.05);
            cursor: pointer;
            transition: transform 0.2s, box-shadow 0.2s;
        }

        .screenshot-card:hover {
            transform: translateY(-3px);
            box-shadow: 0 4px 12px rgba(0, 0, 0, 0.15);
        }

        .screenshot-card img {
            width: 100%;
            height: 200px;
            object-fit: cover;
            border-bottom: 1px solid #dee2e6;
        }

        .screenshot-card .caption {
            padding: 15px;
            font-size: 0.9em;
            color: #495057;
            background: #f8f9fa;
        }

        .defect-card {
            background: #fff3cd;
            border: 2px solid #ffc107;
            border-radius: 8px;
            padding: 25px;
            margin-bottom: 20px;
        }

        .defect-header {
            display: flex;
            align-items: center;
            gap: 15px;
            margin-bottom: 15px;
        }

        .defect-id {
            font-size: 1.5em;
            font-weight: 700;
            color: #856404;
        }

        .severity-badge {
            padding: 6px 12px;
            border-radius: 20px;
            font-size: 0.85em;
            font-weight: 600;
            background: #dc3545;
            color: white;
        }

        .defect-description {
            line-height: 1.6;
            color: #856404;
            margin-bottom: 15px;
        }

        .defect-details {
            background: white;
            padding: 15px;
            border-radius: 6px;
            margin-top: 15px;
        }

        .modal {
            display: none;
            position: fixed;
            top: 0;
            left: 0;
            width: 100%;
            height: 100%;
            background: rgba(0, 0, 0, 0.9);
            z-index: 1000;
            justify-content: center;
            align-items: center;
            padding: 20px;
        }

        .modal.active {
            display: flex;
        }

        .modal img {
            max-width: 95%;
            max-height: 95vh;
            border-radius: 8px;
            box-shadow: 0 10px 40px rgba(0, 0, 0, 0.5);
        }

        .modal-close {
            position: absolute;
            top: 20px;
            right: 20px;
            font-size: 2em;
            color: white;
            cursor: pointer;
            background: rgba(0, 0, 0, 0.5);
            width: 50px;
            height: 50px;
            border-radius: 50%;
            display: flex;
            align-items: center;
            justify-content: center;
            transition: background 0.2s;
        }

        .modal-close:hover {
            background: rgba(0, 0, 0, 0.8);
        }

        .summary-section {
            background: #f8f9fa;
            padding: 25px;
            border-radius: 8px;
            margin-bottom: 20px;
        }

        .summary-section h3 {
            color: #667eea;
            margin-bottom: 15px;
            font-size: 1.3em;
        }

        .summary-section p {
            line-height: 1.8;
            color: #495057;
            margin-bottom: 10px;
        }

        .recommendation-box {
            background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
            color: white;
            padding: 30px;
            border-radius: 8px;
            margin: 20px 0;
        }

        .recommendation-box h3 {
            font-size: 1.5em;
            margin-bottom: 15px;
        }

        .recommendation-box ul {
            margin-left: 20px;
            line-height: 2;
        }

        table {
            width: 100%;
            border-collapse: collapse;
            margin: 20px 0;
        }

        th, td {
            padding: 12px;
            text-align: left;
            border-bottom: 1px solid #dee2e6;
        }

        th {
            background: #f8f9fa;
            font-weight: 600;
            color: #495057;
        }

        tr:hover {
            background: #f8f9fa;
        }

        .log-viewer {
            background: #282c34;
            color: #abb2bf;
            padding: 20px;
            border-radius: 8px;
            font-family: 'Courier New', monospace;
            font-size: 0.9em;
            overflow-x: auto;
            line-height: 1.6;
            margin: 20px 0;
        }

        /* Inline screenshots in test case body */
        .test-screenshots {
            margin-top: 25px;
            padding-top: 20px;
            border-top: 2px solid #e9ecef;
        }

        .test-screenshots h4 {
            color: #667eea;
            margin-bottom: 15px;
            font-size: 1.1em;
        }

        .inline-screenshot-grid {
            display: grid;
            grid-template-columns: repeat(auto-fill, minmax(250px, 1fr));
            gap: 15px;
            margin-top: 15px;
        }

        .inline-screenshot {
            background: white;
            border: 1px solid #dee2e6;
            border-radius: 6px;
            overflow: hidden;
            box-shadow: 0 2px 4px rgba(0, 0, 0, 0.05);
            cursor: pointer;
            transition: transform 0.2s, box-shadow 0.2s;
        }

        .inline-screenshot:hover {
            transform: translateY(-3px);
            box-shadow: 0 4px 12px rgba(0, 0, 0, 0.15);
        }

        .inline-screenshot img {
            width: 100%;
            height: 150px;
            object-fit: cover;
            border-bottom: 1px solid #dee2e6;
        }

        .inline-screenshot .caption {
            padding: 10px;
            font-size: 0.85em;
            color: #495057;
            background: #f8f9fa;
            text-align: center;
        }

        /* API error logs in test case body */
        .api-error-log {
            margin-top: 20px;
            padding-top: 20px;
            border-top: 2px solid #ffc107;
        }

        .api-error-log h4 {
            color: #dc3545;
            margin-bottom: 15px;
            font-size: 1.1em;
            display: flex;
            align-items: center;
            gap: 10px;
        }

        .api-error-log h4::before {
            content: '⚠️';
            font-size: 1.2em;
        }

        .api-log-viewer {
            background: #fff3cd;
            border: 2px solid #ffc107;
            color: #856404;
            padding: 15px;
            border-radius: 6px;
            font-family: 'Courier New', monospace;
            font-size: 0.85em;
            overflow-x: auto;
            line-height: 1.6;
            margin-top: 10px;
        }

        .api-log-section {
            margin-bottom: 15px;
        }

        .api-log-section:last-child {
            margin-bottom: 0;
        }

        .api-log-label {
            font-weight: 700;
            color: #856404;
            margin-bottom: 5px;
            font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
        }

        .alert {
            padding: 15px 20px;
            border-radius: 6px;
            margin: 20px 0;
            display: flex;
            align-items: center;
            gap: 10px;
        }

        .alert.warning {
            background: #fff3cd;
            border-left: 4px solid #ffc107;
            color: #856404;
        }

        .alert.success {
            background: #d4edda;
            border-left: 4px solid #28a745;
            color: #155724;
        }

        .alert-icon {
            font-size: 1.5em;
        }

        @media (max-width: 768px) {
            .header h1 {
                font-size: 1.8em;
            }

            .stats-grid {
                grid-template-columns: repeat(2, 1fr);
            }

            .screenshot-grid {
                grid-template-columns: 1fr;
            }

            .tabs {
                padding: 0 15px;
            }

            .tab {
                padding: 12px 15px;
                font-size: 0.9em;
            }

            .content {
                padding: 15px;
            }
        }
    </style>
</head>
<body>
    <div class="container">
        <div class="header">
            <h1>🤖 Game AI Bot v1.3.0</h1>
            <p>User Acceptance Testing (UAT) - Final Test Execution Report</p>
            <p style="opacity: 0.8; margin-top: 10px;">Execution Date: 2025-11-12 | AWS China North 1 Region</p>
        </div>

        <div class="stats-grid">
            <div class="stat-card total">
                <div class="number">8</div>
                <div class="label">Total Tests</div>
            </div>
            <div class="stat-card passed">
                <div class="number">4</div>
                <div class="label">Passed</div>
            </div>
            <div class="stat-card failed">
                <div class="number">0</div>
                <div class="label">Failed</div>
            </div>
            <div class="stat-card blocked">
                <div class="number">4</div>
                <div class="label">Blocked</div>
            </div>
        </div>

        <div class="tabs">
            <button class="tab active" onclick="switchTab('summary')">📊 Executive Summary</button>
            <button class="tab" onclick="switchTab('passed')">✅ Passed Tests (4)</button>
            <button class="tab" onclick="switchTab('blocked')">⚠️ Blocked Tests (4)</button>
            <button class="tab" onclick="switchTab('defects')">🐞 Defects (1)</button>
            <button class="tab" onclick="switchTab('screenshots')">📸 Screenshots (26)</button>
            <button class="tab" onclick="switchTab('logs')">📝 Logs</button>
        </div>

        <div class="content">
            <!-- Summary Tab -->
            <div id="summary" class="tab-content active">
                <div class="alert warning">
                    <span class="alert-icon">⚠️</span>
                    <strong>Status: CONDITIONAL APPROVAL</strong> - Critical defect blocks 50% of test suite. DO NOT RELEASE until DEFECT-001 is resolved.
                </div>

                <div class="summary-section">
                    <h3>Test Execution Overview</h3>
                    <p><strong>Total Execution Time:</strong> ~27.5 minutes (09:04:30 UTC - 09:32:00 UTC)</p>
                    <p><strong>Test Environment:</strong> AWS Account 123456789012, Region cn-north-1, EKS Cluster ai-bot-cluster</p>
                    <p><strong>Test User:</strong> testuser@example.com</p>
                    <p><strong>Authentication:</strong> OIDC via Authing (successful)</p>
                </div>

                <div class="summary-section">
                    <h3>Module Test Coverage</h3>
                    <table>
                        <thead>
                            <tr>
                                <th>Module</th>
                                <th>Tests</th>
                                <th>Executed</th>
                                <th>Passed</th>
                                <th>Blocked</th>
                                <th>Coverage</th>
                            </tr>
                        </thead>
                        <tbody>
                            <tr>
                                <td>Game Package Management</td>
                                <td>2</td>
                                <td>2</td>
                                <td>2</td>
                                <td>0</td>
                                <td>✅ 100%</td>
                            </tr>
                            <tr>
                                <td>Game Environment Management</td>
                                <td>2</td>
                                <td>0</td>
                                <td>0</td>
                                <td>2</td>
                                <td>❌ 0%</td>
                            </tr>
                            <tr>
                                <td>System Settings</td>
                                <td>1</td>
                                <td>1</td>
                                <td>1</td>
                                <td>0</td>
                                <td>✅ 100%</td>
                            </tr>
                            <tr>
                                <td>Ray Cluster Management</td>
                                <td>1</td>
                                <td>1</td>
                                <td>1</td>
                                <td>0</td>
                                <td>✅ 100%</td>
                            </tr>
                            <tr>
                                <td>Model Training</td>
                                <td>1</td>
                                <td>0</td>
                                <td>0</td>
                                <td>1</td>
                                <td>❌ 0%</td>
                            </tr>
                            <tr>
                                <td>Testing & Evaluation</td>
                                <td>1</td>
                                <td>0</td>
                                <td>0</td>
                                <td>1</td>
                                <td>❌ 0%</td>
                            </tr>
                        </tbody>
                    </table>
                </div>

                <div class="recommendation-box">
                    <h3>🎯 Recommendations</h3>
                    <p><strong>Immediate Actions (P1 - Urgent):</strong></p>
                    <ul>
                        <li>Resolve DEFECT-001: Fix EKS node group labeling (node-group=x86-ng-v2)</li>
                        <li>Verify nodes are properly joined to cluster and ready</li>
                        <li>Confirm backend service account has correct Kubernetes RBAC permissions</li>
                        <li>Re-execute blocked test cases after fix</li>
                    </ul>
                    <p style="margin-top: 15px;"><strong>Short-term Improvements (P2):</strong></p>
                    <ul>
                        <li>Enhance error messages with detailed Kubernetes API errors</li>
                        <li>Add pre-flight validation for node group availability</li>
                        <li>Implement infrastructure monitoring and alerts</li>
                    </ul>
                </div>
            </div>

            <!-- Passed Tests Tab -->
            <div id="passed" class="tab-content">
                <div class="alert success">
                    <span class="alert-icon">✅</span>
                    <strong>4 tests passed successfully</strong> - Game Package Management, System Settings, and Ray Cluster modules are production-ready.
                </div>

                <div class="test-case">
                    <div class="test-case-header" onclick="toggleTestCase(this)">
                        <div class="test-case-title">
                            <span class="toggle-icon"></span>
                            <span>TC-1.1: Upload Headless Game Package</span>
                        </div>
                        <span class="status-badge pass">✅ PASS</span>
                    </div>
                    <div class="test-case-body">
                        <div class="info-grid">
                            <div class="info-item">
                                <div class="label">Module</div>
                                <div class="value">Game Package Management</div>
                            </div>
                            <div class="info-item">
                                <div class="label">Priority</div>
                                <div class="value">P1 (High)</div>
                            </div>
                            <div class="info-item">
                                <div class="label">Execution Time</div>
                                <div class="value">2025-11-12 09:07 UTC</div>
                            </div>
                            <div class="info-item">
                                <div class="label">Duration</div>
                                <div class="value">150 seconds</div>
                            </div>
                        </div>
                        <p><strong>Summary:</strong> Successfully uploaded Headless game package "Match3-Headless-Test v1.0.0" with automatic Docker image build completion.</p>
                        <p><strong>Key Results:</strong></p>
                        <ul style="margin-left: 20px; line-height: 2;">
                            <li>✅ File upload completed (33 MB)</li>
                            <li>✅ Package created in list</li>
                            <li>✅ Docker build automatically triggered</li>
                            <li>✅ Build completed successfully in 53 seconds</li>
                            <li>✅ Docker image pushed to ECR</li>
                        </ul>

                        <!-- Inline Screenshots Section -->
                        <div class="test-screenshots">
                            <h4>📸 Test Execution Screenshots</h4>
                            <div class="inline-screenshot-grid">
                                <div class="inline-screenshot" onclick="openModal('screenshots/04_game_packages_page.png')">
                                    <img src="screenshots/04_game_packages_page.png" alt="Empty Packages">
                                    <div class="caption">04: Empty Packages List</div>
                                </div>
                                <div class="inline-screenshot" onclick="openModal('screenshots/05_create_package_form.png')">
                                    <img src="screenshots/05_create_package_form.png" alt="Create Form">
                                    <div class="caption">05: Create Package Form</div>
                                </div>
                                <div class="inline-screenshot" onclick="openModal('screenshots/06_form_filled_ready_submit.png')">
                                    <img src="screenshots/06_form_filled_ready_submit.png" alt="Form Ready">
                                    <div class="caption">06: Form Ready to Submit</div>
                                </div>
                                <div class="inline-screenshot" onclick="openModal('screenshots/07_tc11_package_list_success.png')">
                                    <img src="screenshots/07_tc11_package_list_success.png" alt="Package Created">
                                    <div class="caption">07: Package Created</div>
                                </div>
                                <div class="inline-screenshot" onclick="openModal('screenshots/08_tc11_package_details.png')">
                                    <img src="screenshots/08_tc11_package_details.png" alt="Build In Progress">
                                    <div class="caption">08: Build IN_PROGRESS</div>
                                </div>
                                <div class="inline-screenshot" onclick="openModal('screenshots/09_tc11_build_succeeded.png')">
                                    <img src="screenshots/09_tc11_build_succeeded.png" alt="Build Succeeded">
                                    <div class="caption">09: Build SUCCEEDED</div>
                                </div>
                            </div>
                        </div>
                    </div>
                </div>

                <div class="test-case">
                    <div class="test-case-header" onclick="toggleTestCase(this)">
                        <div class="test-case-title">
                            <span class="toggle-icon"></span>
                            <span>TC-1.2: Upload Headed Game Package</span>
                        </div>
                        <span class="status-badge pass">✅ PASS</span>
                    </div>
                    <div class="test-case-body">
                        <div class="info-grid">
                            <div class="info-item">
                                <div class="label">Module</div>
                                <div class="value">Game Package Management</div>
                            </div>
                            <div class="info-item">
                                <div class="label">Priority</div>
                                <div class="value">P1 (High)</div>
                            </div>
                            <div class="info-item">
                                <div class="label">Execution Time</div>
                                <div class="value">2025-11-12 09:10 UTC</div>
                            </div>
                            <div class="info-item">
                                <div class="label">Duration</div>
                                <div class="value">120 seconds</div>
                            </div>
                        </div>
                        <p><strong>Summary:</strong> Successfully uploaded Headed game package "Match3-Headed-Test v1.0.0" without Docker build (correct behavior).</p>
                        <p><strong>Key Results:</strong></p>
                        <ul style="margin-left: 20px; line-height: 2;">
                            <li>✅ File upload completed (35 MB)</li>
                            <li>✅ Package created in list</li>
                            <li>✅ No Docker build triggered (expected for Headed packages)</li>
                            <li>✅ Package uploaded to S3 successfully</li>
                            <li>✅ Both packages now available for environment creation</li>
                        </ul>
                        <p style="margin-top: 15px;"><strong>Related Screenshots:</strong> 10-13</p>
                    </div>
                </div>

                <div class="test-case">
                    <div class="test-case-header" onclick="toggleTestCase(this)">
                        <div class="test-case-title">
                            <span class="toggle-icon"></span>
                            <span>TC-6.1: Configure System Settings</span>
                        </div>
                        <span class="status-badge pass">✅ PASS</span>
                    </div>
                    <div class="test-case-body">
                        <div class="info-grid">
                            <div class="info-item">
                                <div class="label">Module</div>
                                <div class="value">System Settings</div>
                            </div>
                            <div class="info-item">
                                <div class="label">Priority</div>
                                <div class="value">P2 (Medium)</div>
                            </div>
                            <div class="info-item">
                                <div class="label">Execution Time</div>
                                <div class="value">2025-11-12 09:25 UTC</div>
                            </div>
                            <div class="info-item">
                                <div class="label">Duration</div>
                                <div class="value">240 seconds</div>
                            </div>
                        </div>
                        <p><strong>Summary:</strong> Successfully modified system configuration for Headed instance types and verified changes reflected in UI.</p>
                        <p><strong>Test Actions:</strong></p>
                        <ul style="margin-left: 20px; line-height: 2;">
                            <li>✅ Modified "Headed Instance Types" to add "g4dn.xlarge"</li>
                            <li>✅ API update returned HTTP 200</li>
                            <li>✅ Changes persisted in database</li>
                            <li>✅ Verified new instance type appears in Environment creation dropdown</li>
                        </ul>
                        <p style="margin-top: 15px;"><strong>Related Screenshots:</strong> 18-21</p>
                    </div>
                </div>

                <div class="test-case">
                    <div class="test-case-header" onclick="toggleTestCase(this)">
                        <div class="test-case-title">
                            <span class="toggle-icon"></span>
                            <span>TC-4.1: Deploy Ray Cluster</span>
                        </div>
                        <span class="status-badge pass">✅ PASS</span>
                    </div>
                    <div class="test-case-body">
                        <div class="info-grid">
                            <div class="info-item">
                                <div class="label">Module</div>
                                <div class="value">Ray Cluster Management</div>
                            </div>
                            <div class="info-item">
                                <div class="label">Priority</div>
                                <div class="value">P1 (High)</div>
                            </div>
                            <div class="info-item">
                                <div class="label">Execution Time</div>
                                <div class="value">2025-11-12 09:30 UTC</div>
                            </div>
                            <div class="info-item">
                                <div class="label">Duration</div>
                                <div class="value">180 seconds</div>
                            </div>
                        </div>
                        <p><strong>Summary:</strong> Successfully deployed Ray cluster "TestRayCluster" with head node running and workers provisioning.</p>
                        <p><strong>Cluster Details:</strong></p>
                        <ul style="margin-left: 20px; line-height: 2;">
                            <li><strong>Cluster ID:</strong> 8d5b10f16c35457f97a4e2f8993bc5b5</li>
                            <li><strong>Ray Version:</strong> 2.38.0</li>
                            <li><strong>Head Node:</strong> g4dn.2xlarge (running) - i-0dafa662e3724d09c</li>
                            <li><strong>Worker Nodes:</strong> 2 × c6i.2xlarge (provisioning)</li>
                            <li><strong>Network:</strong> VPC vpc-09a28fe8e82cb4fca, 2 subnets, security group configured</li>
                            <li><strong>Storage:</strong> Head node 1000 GB, Workers 500 GB each</li>
                        </ul>
                        <p style="margin-top: 15px;"><strong>Related Screenshots:</strong> 22-26</p>
                    </div>
                </div>
            </div>

            <!-- Blocked Tests Tab -->
            <div id="blocked" class="tab-content">
                <div class="alert warning">
                    <span class="alert-icon">⚠️</span>
                    <strong>4 tests blocked by DEFECT-001</strong> - Environment creation API returns HTTP 500 error due to infrastructure issues.
                </div>

                <div class="test-case">
                    <div class="test-case-header" onclick="toggleTestCase(this)">
                        <div class="test-case-title">
                            <span class="toggle-icon"></span>
                            <span>TC-2.1: Create Headless Game Environment</span>
                        </div>
                        <span class="status-badge blocked">⚠️ BLOCKED</span>
                    </div>
                    <div class="test-case-body">
                        <div class="info-grid">
                            <div class="info-item">
                                <div class="label">Module</div>
                                <div class="value">Game Environment Management</div>
                            </div>
                            <div class="info-item">
                                <div class="label">Priority</div>
                                <div class="value">P1 (High)</div>
                            </div>
                            <div class="info-item">
                                <div class="label">Blocked By</div>
                                <div class="value">DEFECT-001</div>
                            </div>
                            <div class="info-item">
                                <div class="label">API Error</div>
                                <div class="value">HTTP 500 Internal Server Error</div>
                            </div>
                        </div>
                        <p><strong>Test Attempted:</strong> 2025-11-12 09:22 UTC</p>
                        <p><strong>Issue:</strong> API endpoint POST /api/create_game_env returned error message: "Error creating service"</p>
                        <p><strong>Form Data Submitted:</strong></p>
                        <ul style="margin-left: 20px; line-height: 2;">
                            <li>Environment Name: HeadlessEnv-Test</li>
                            <li>Package Type: Headless</li>
                            <li>Game: Match3-Headless-Test 1.0.0</li>
                            <li>Node Group: x86-ng-v2</li>
                            <li>Replicas: 10</li>
                            <li>CPU: 100 millicores, Memory: 64 MiB</li>
                        </ul>

                        <!-- API Error Log Section -->
                        <div class="api-error-log">
                            <h4>API Error Details</h4>
                            <div class="api-log-viewer">
                                <div class="api-log-section">
                                    <div class="api-log-label">REQUEST:</div>
POST /api/create_game_env HTTP/1.1
Host: <API_GATEWAY_ID>.execute-api.cn-north-1.amazonaws.com.cn
Content-Type: application/json
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGci...

{
  "gameEnvName": "HeadlessEnv-Test",
  "gameType": "Headless",
  "gameId": "example-game-id-placeholder",
  "gameName": "Match3-Headless-Test",
  "version": "1.0.0",
  "headlessGameEnvInfo": {
    "imageUri": "123456789012.dkr.ecr.cn-north-1.amazonaws.com.cn/ai-bot-aibotgameecr...",
    "nodeGroup": "x86-ng-v2",
    "replicasNum": 10,
    "cpu": 100,
    "memory": 64
  }
}
                                </div>
                                <div class="api-log-section">
                                    <div class="api-log-label">RESPONSE:</div>
HTTP/1.1 500 Internal Server Error
Content-Type: application/json; charset=utf-8
Date: Tue, 12 Nov 2025 09:22:15 GMT

{
  "message": "Error creating service",
  "timestamp": "2025-11-12T09:22:15.234Z",
  "path": "/api/create_game_env"
}
                                </div>
                                <div class="api-log-section">
                                    <div class="api-log-label">LOG FILE:</div>
See logs/api_error_tc21_create_environment.json for full details
                                </div>
                            </div>
                        </div>

                        <!-- Inline Screenshots Section -->
                        <div class="test-screenshots">
                            <h4>📸 Test Execution Screenshots</h4>
                            <div class="inline-screenshot-grid">
                                <div class="inline-screenshot" onclick="openModal('screenshots/14_tc21_env_list_empty.png')">
                                    <img src="screenshots/14_tc21_env_list_empty.png" alt="Empty Env List">
                                    <div class="caption">14: Empty Environment List</div>
                                </div>
                                <div class="inline-screenshot" onclick="openModal('screenshots/15_tc21_env_create_form.png')">
                                    <img src="screenshots/15_tc21_env_create_form.png" alt="Create Env Form">
                                    <div class="caption">15: Create Environment Form</div>
                                </div>
                                <div class="inline-screenshot" onclick="openModal('screenshots/16_tc21_env_form_complete.png')">
                                    <img src="screenshots/16_tc21_env_form_complete.png" alt="Form Complete">
                                    <div class="caption">16: Form Ready to Submit</div>
                                </div>
                                <div class="inline-screenshot" onclick="openModal('screenshots/17_tc21_submit_status.png')">
                                    <img src="screenshots/17_tc21_submit_status.png" alt="Submit Failed">
                                    <div class="caption">17: HTTP 500 Error</div>
                                </div>
                            </div>
                        </div>
                    </div>
                </div>

                <div class="test-case">
                    <div class="test-case-header" onclick="toggleTestCase(this)">
                        <div class="test-case-title">
                            <span class="toggle-icon"></span>
                            <span>TC-2.2: Create Headed Game Environment</span>
                        </div>
                        <span class="status-badge blocked">⚠️ BLOCKED</span>
                    </div>
                    <div class="test-case-body">
                        <p><strong>Status:</strong> Not attempted due to same API endpoint failure as TC-2.1.</p>
                        <p><strong>Reason:</strong> Both Headless and Headed environment creation use the same API endpoint which is currently failing.</p>
                    </div>
                </div>

                <div class="test-case">
                    <div class="test-case-header" onclick="toggleTestCase(this)">
                        <div class="test-case-title">
                            <span class="toggle-icon"></span>
                            <span>TC-3.1: Model Training with SageMaker</span>
                        </div>
                        <span class="status-badge blocked">⚠️ BLOCKED</span>
                    </div>
                    <div class="test-case-body">
                        <p><strong>Status:</strong> Cannot execute because it depends on TC-2.1.</p>
                        <p><strong>Dependency:</strong> Model training requires a running Headless game environment which cannot be created due to DEFECT-001.</p>
                    </div>
                </div>

                <div class="test-case">
                    <div class="test-case-header" onclick="toggleTestCase(this)">
                        <div class="test-case-title">
                            <span class="toggle-icon"></span>
                            <span>TC-5.1: Comparative Testing Between Models</span>
                        </div>
                        <span class="status-badge blocked">⚠️ BLOCKED</span>
                    </div>
                    <div class="test-case-body">
                        <p><strong>Status:</strong> Cannot execute because it depends on TC-3.1.</p>
                        <p><strong>Dependency Chain:</strong> TC-5.1 → TC-3.1 → TC-2.1 (all blocked by DEFECT-001)</p>
                    </div>
                </div>
            </div>

            <!-- Defects Tab -->
            <div id="defects" class="tab-content">
                <div class="defect-card">
                    <div class="defect-header">
                        <div class="defect-id">DEFECT-001</div>
                        <span class="severity-badge">S2 (Major)</span>
                        <span class="severity-badge" style="background: #dc3545;">P1 (Urgent)</span>
                    </div>
                    <div class="defect-description">
                        <strong>[Environment Creation] API returns 500 error when creating Headless game environment</strong>
                    </div>
                    <p><strong>Reporter:</strong> Claude Code (Automated Testing)</p>
                    <p><strong>Date Found:</strong> 2025-11-12 09:22 UTC</p>
                    <p><strong>Module:</strong> Game Environment Management - Environment Creation API</p>

                    <div class="defect-details">
                        <h4>Description</h4>
                        <p>API endpoint POST /api/create_game_env returns HTTP 500 Internal Server Error when attempting to create Headless game environments. Error message: <code>{"message":"Error creating service"}</code></p>

                        <h4 style="margin-top: 20px;">Root Cause (Suspected)</h4>
                        <ol style="margin-left: 20px; line-height: 1.8;">
                            <li><strong>Node Label Mismatch:</strong> The x86-ng-v2 node group exists and is ACTIVE, but nodes may not have the correct <code>node-group=x86-ng-v2</code> label that the pod scheduler is looking for.</li>
                            <li><strong>Node Availability:</strong> Although the node group has desiredSize=2, the nodes might not be ready or properly joined to the cluster yet.</li>
                            <li><strong>Service Account / RBAC Permissions:</strong> The backend service may lack sufficient Kubernetes permissions to create deployments/services in the target namespace.</li>
                        </ol>

                        <h4 style="margin-top: 20px;">Investigation Results</h4>
                        <div class="log-viewer">
# Check for nodes with x86-ng-v2 label
$ kubectl get nodes -l node-group=x86-ng-v2
No resources found

# Check EKS node group status
$ aws eks describe-nodegroup --cluster-name ai-bot-cluster --nodegroup-name x86-ng-v2
{
  "status": "ACTIVE",
  "scalingConfig": {
    "minSize": 1,
    "maxSize": 10,
    "desiredSize": 2
  }
}
                        </div>

                        <h4 style="margin-top: 20px;">Impact</h4>
                        <ul style="margin-left: 20px; line-height: 1.8;">
                            <li><strong>User Impact:</strong> HIGH - Users cannot create any game environments, which is a core feature</li>
                            <li><strong>Workflow Impact:</strong> CRITICAL - Blocks model training workflow</li>
                            <li><strong>Test Coverage Impact:</strong> Blocks TC-2.1, TC-2.2, TC-3.1, TC-5.1 (4 out of 8 test cases = 50%)</li>
                        </ul>

                        <h4 style="margin-top: 20px;">Recommendations</h4>
                        <ol style="margin-left: 20px; line-height: 1.8;">
                            <li><strong>Immediate:</strong> Check backend application logs for detailed stack trace</li>
                            <li><strong>Immediate:</strong> Verify node group labels match what the API expects</li>
                            <li><strong>Immediate:</strong> Confirm Kubernetes service account has correct permissions</li>
                            <li><strong>Short-term:</strong> Improve error messages to include actionable details</li>
                            <li><strong>Short-term:</strong> Add API endpoint validation before creating Kubernetes resources</li>
                            <li><strong>Long-term:</strong> Implement pre-flight checks in UI to validate node group availability</li>
                        </ol>

                        <p style="margin-top: 20px;"><strong>Reproduction Rate:</strong> 100% (2 out of 2 attempts failed with same error)</p>
                        <p><strong>Full Report:</strong> <a href="defect-reports/DEFECT-001-Environment-Creation-500-Error.md" target="_blank">DEFECT-001-Environment-Creation-500-Error.md</a></p>
                    </div>
                </div>
            </div>

            <!-- Screenshots Tab -->
            <div id="screenshots" class="tab-content">
                <h3 style="margin-bottom: 20px;">Test Execution Screenshots (26 total)</h3>

                <h4 style="margin: 20px 0;">Authentication & Initial Setup</h4>
                <div class="screenshot-grid">
                    <div class="screenshot-card" onclick="openModal('screenshots/01_portal_home.png')">
                        <img src="screenshots/01_portal_home.png" alt="Portal Home">
                        <div class="caption">01: Initial portal home page (login required)</div>
                    </div>
                    <div class="screenshot-card" onclick="openModal('screenshots/02_after_oidc_click.png')">
                        <img src="screenshots/02_after_oidc_click.png" alt="OIDC Page">
                        <div class="caption">02: OIDC authentication page (Authing)</div>
                    </div>
                    <div class="screenshot-card" onclick="openModal('screenshots/03_after_login_home.png')">
                        <img src="screenshots/03_after_login_home.png" alt="After Login">
                        <div class="caption">03: Portal after successful login</div>
                    </div>
                </div>

                <h4 style="margin: 30px 0 20px;">TC-1.1: Headless Package Upload ✅</h4>
                <div class="screenshot-grid">
                    <div class="screenshot-card" onclick="openModal('screenshots/04_game_packages_page.png')">
                        <img src="screenshots/04_game_packages_page.png" alt="Empty Packages">
                        <div class="caption">04: Empty Game Packages list (0–0 of 0)</div>
                    </div>
                    <div class="screenshot-card" onclick="openModal('screenshots/05_create_package_form.png')">
                        <img src="screenshots/05_create_package_form.png" alt="Create Form">
                        <div class="caption">05: Create Headless package form</div>
                    </div>
                    <div class="screenshot-card" onclick="openModal('screenshots/06_form_filled_ready_submit.png')">
                        <img src="screenshots/06_form_filled_ready_submit.png" alt="Form Ready">
                        <div class="caption">06: Form filled and ready to submit</div>
                    </div>
                    <div class="screenshot-card" onclick="openModal('screenshots/07_tc11_package_list_success.png')">
                        <img src="screenshots/07_tc11_package_list_success.png" alt="Package Created">
                        <div class="caption">07: Headless package created in list (1–1 of 1)</div>
                    </div>
                    <div class="screenshot-card" onclick="openModal('screenshots/08_tc11_package_details.png')">
                        <img src="screenshots/08_tc11_package_details.png" alt="Build In Progress">
                        <div class="caption">08: Package details - Build IN_PROGRESS</div>
                    </div>
                    <div class="screenshot-card" onclick="openModal('screenshots/09_tc11_build_succeeded.png')">
                        <img src="screenshots/09_tc11_build_succeeded.png" alt="Build Succeeded">
                        <div class="caption">09: Package details - Build SUCCEEDED (53s)</div>
                    </div>
                </div>

                <h4 style="margin: 30px 0 20px;">TC-1.2: Headed Package Upload ✅</h4>
                <div class="screenshot-grid">
                    <div class="screenshot-card" onclick="openModal('screenshots/10_tc12_headed_form.png')">
                        <img src="screenshots/10_tc12_headed_form.png" alt="Headed Form">
                        <div class="caption">10: Create Headed package form</div>
                    </div>
                    <div class="screenshot-card" onclick="openModal('screenshots/11_tc12_headed_ready_submit.png')">
                        <img src="screenshots/11_tc12_headed_ready_submit.png" alt="Headed Ready">
                        <div class="caption">11: Headed form ready to submit</div>
                    </div>
                    <div class="screenshot-card" onclick="openModal('screenshots/12_tc12_both_packages_list.png')">
                        <img src="screenshots/12_tc12_both_packages_list.png" alt="Both Packages">
                        <div class="caption">12: Both packages in list (1–2 of 2)</div>
                    </div>
                    <div class="screenshot-card" onclick="openModal('screenshots/13_tc12_headed_details_no_build.png')">
                        <img src="screenshots/13_tc12_headed_details_no_build.png" alt="No Build">
                        <div class="caption">13: Headed package - No build info (correct)</div>
                    </div>
                </div>

                <h4 style="margin: 30px 0 20px;">TC-2.1: Environment Creation ❌ (BLOCKED)</h4>
                <div class="screenshot-grid">
                    <div class="screenshot-card" onclick="openModal('screenshots/14_tc21_env_list_empty.png')">
                        <img src="screenshots/14_tc21_env_list_empty.png" alt="Empty Env List">
                        <div class="caption">14: Empty Game Environments list</div>
                    </div>
                    <div class="screenshot-card" onclick="openModal('screenshots/15_tc21_env_create_form.png')">
                        <img src="screenshots/15_tc21_env_create_form.png" alt="Create Env Form">
                        <div class="caption">15: Environment creation form</div>
                    </div>
                    <div class="screenshot-card" onclick="openModal('screenshots/16_tc21_env_form_complete.png')">
                        <img src="screenshots/16_tc21_env_form_complete.png" alt="Form Complete">
                        <div class="caption">16: Complete form ready for submission</div>
                    </div>
                    <div class="screenshot-card" onclick="openModal('screenshots/17_tc21_submit_status.png')">
                        <img src="screenshots/17_tc21_submit_status.png" alt="Submit Failed">
                        <div class="caption">17: Form after failed submission (HTTP 500)</div>
                    </div>
                </div>

                <h4 style="margin: 30px 0 20px;">TC-6.1: System Settings ✅</h4>
                <div class="screenshot-grid">
                    <div class="screenshot-card" onclick="openModal('screenshots/18_tc61_system_settings_initial.png')">
                        <img src="screenshots/18_tc61_system_settings_initial.png" alt="Settings Initial">
                        <div class="caption">18: System Settings initial state</div>
                    </div>
                    <div class="screenshot-card" onclick="openModal('screenshots/19_tc61_modified_headed_instance_types.png')">
                        <img src="screenshots/19_tc61_modified_headed_instance_types.png" alt="Settings Modified">
                        <div class="caption">19: Modified Headed instance types (+g4dn.xlarge)</div>
                    </div>
                    <div class="screenshot-card" onclick="openModal('screenshots/20_tc61_update_success.png')">
                        <img src="screenshots/20_tc61_update_success.png" alt="Update Success">
                        <div class="caption">20: Settings update successful</div>
                    </div>
                    <div class="screenshot-card" onclick="openModal('screenshots/21_tc61_headed_instance_types_verified.png')">
                        <img src="screenshots/21_tc61_headed_instance_types_verified.png" alt="Instance Types Verified">
                        <div class="caption">21: New instance type verified in dropdown</div>
                    </div>
                </div>

                <h4 style="margin: 30px 0 20px;">TC-4.1: Ray Cluster Deployment ✅</h4>
                <div class="screenshot-grid">
                    <div class="screenshot-card" onclick="openModal('screenshots/22_tc41_ray_cluster_list.png')">
                        <img src="screenshots/22_tc41_ray_cluster_list.png" alt="Empty Cluster List">
                        <div class="caption">22: Empty Ray Cluster list</div>
                    </div>
                    <div class="screenshot-card" onclick="openModal('screenshots/23_tc41_create_form.png')">
                        <img src="screenshots/23_tc41_create_form.png" alt="Cluster Create Form">
                        <div class="caption">23: Ray Cluster creation form</div>
                    </div>
                    <div class="screenshot-card" onclick="openModal('screenshots/24_tc41_form_complete.png')">
                        <img src="screenshots/24_tc41_form_complete.png" alt="Cluster Form Complete">
                        <div class="caption">24: Complete form ready to submit</div>
                    </div>
                    <div class="screenshot-card" onclick="openModal('screenshots/25_tc41_cluster_created.png')">
                        <img src="screenshots/25_tc41_cluster_created.png" alt="Cluster Created">
                        <div class="caption">25: Cluster created (INITIALIZING status)</div>
                    </div>
                    <div class="screenshot-card" onclick="openModal('screenshots/26_tc41_cluster_details.png')">
                        <img src="screenshots/26_tc41_cluster_details.png" alt="Cluster Details">
                        <div class="caption">26: Cluster details - Head node running</div>
                    </div>
                </div>
            </div>

            <!-- Logs Tab -->
            <div id="logs" class="tab-content">
                <h3 style="margin-bottom: 20px;">Test Execution Logs</h3>

                <div class="summary-section">
                    <h4>Portal Home Page Accessibility Tree</h4>
                    <p>Initial page snapshot captured using Chrome DevTools accessibility tree</p>
                    <div class="log-viewer">uid=1_0 RootWebArea url="https://<API_GATEWAY_ID>.execute-api.cn-north-1.amazonaws.com.cn/auth/signin"
  uid=1_1 main
    uid=1_2 heading "Sign in to Game AI Bot" level="5"
    uid=1_3 StaticText "Welcome, please sign in to continue"
    uid=1_4 button "Sign In With OIDC"
  uid=1_5 StaticText "/auth/signin"</div>
                </div>

                <div class="summary-section">
                    <h4>API Request - Environment Creation (Failed)</h4>
                    <p><strong>Endpoint:</strong> POST /api/create_game_env</p>
                    <p><strong>Status:</strong> HTTP 500 Internal Server Error</p>
                    <div class="log-viewer"># Request
POST /api/create_game_env HTTP/1.1
Host: <API_GATEWAY_ID>.execute-api.cn-north-1.amazonaws.com.cn
Content-Type: application/json
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGci...

{
  "gameEnvName": "HeadlessEnv-Test",
  "gameType": "Headless",
  "gameId": "example-game-id-placeholder",
  "gameName": "Match3-Headless-Test",
  "version": "1.0.0",
  "gameEnvId": "",
  "createTime": "2025-11-12T09:11:23.154Z",
  "status": "Available",
  "headlessGameEnvInfo": {
    "imageUri": "123456789012.dkr.ecr.cn-north-1.amazonaws.com.cn/ai-bot-aibotgameecr61a88526-gwtg3vem2hdm:example-game-id-placeholder",
    "nodeGroup": "x86-ng-v2",
    "replicasNum": 10,
    "cpu": 100,
    "memory": 64,
    "namespace": "",
    "headlessGameEnvPodInfoList": []
  }
}

# Response
HTTP/1.1 500 Internal Server Error
Content-Type: application/json; charset=utf-8

{
  "message": "Error creating service"
}</div>
                </div>

                <div class="summary-section">
                    <h4>Kubernetes Investigation</h4>
                    <div class="log-viewer"># Check for nodes with x86-ng-v2 label
$ kubectl get nodes -l node-group=x86-ng-v2
No resources found

# Check EKS node group status via AWS CLI
$ aws eks describe-nodegroup --cluster-name ai-bot-cluster --nodegroup-name x86-ng-v2
{
  "nodegroup": {
    "nodegroupName": "x86-ng-v2",
    "status": "ACTIVE",
    "scalingConfig": {
      "minSize": 1,
      "maxSize": 10,
      "desiredSize": 2
    },
    "instanceTypes": ["t3.medium"],
    "subnets": ["subnet-0d695b06bbca96178", "subnet-0f16cd113b5f39552"]
  }
}

# Check existing deployments in namespace
$ kubectl get deployments -n game-ai-bot
No resources found in game-ai-bot namespace.</div>
                </div>

                <div class="summary-section">
                    <h4>Test Execution Timeline</h4>
                    <table>
                        <thead>
                            <tr>
                                <th>Time (UTC)</th>
                                <th>Activity</th>
                                <th>Status</th>
                            </tr>
                        </thead>
                        <tbody>
                            <tr>
                                <td>09:04:30</td>
                                <td>Environment validation started</td>
                                <td>✅ Pass</td>
                            </tr>
                            <tr>
                                <td>09:05:00</td>
                                <td>Portal login via OIDC</td>
                                <td>✅ Pass</td>
                            </tr>
                            <tr>
                                <td>09:06:30</td>
                                <td>TC-1.1: Headless Package Upload started</td>
                                <td>✅ Pass</td>
                            </tr>
                            <tr>
                                <td>09:07:59</td>
                                <td>Docker build started (CodeBuild)</td>
                                <td>✅ Pass</td>
                            </tr>
                            <tr>
                                <td>09:08:52</td>
                                <td>Docker build completed (53 seconds)</td>
                                <td>✅ Pass</td>
                            </tr>
                            <tr>
                                <td>09:09:00</td>
                                <td>TC-1.2: Headed Package Upload started</td>
                                <td>✅ Pass</td>
                            </tr>
                            <tr>
                                <td>09:11:00</td>
                                <td>TC-2.1: Environment Creation started</td>
                                <td>❌ Failed</td>
                            </tr>
                            <tr>
                                <td>09:22:00</td>
                                <td>API returned HTTP 500 error</td>
                                <td>❌ Failed</td>
                            </tr>
                            <tr>
                                <td>09:25:00</td>
                                <td>TC-6.1: System Settings started</td>
                                <td>✅ Pass</td>
                            </tr>
                            <tr>
                                <td>09:29:00</td>
                                <td>TC-4.1: Ray Cluster Deployment started</td>
                                <td>✅ Pass</td>
                            </tr>
                            <tr>
                                <td>09:30:05</td>
                                <td>Ray cluster created (INITIALIZING)</td>
                                <td>✅ Pass</td>
                            </tr>
                            <tr>
                                <td>09:32:00</td>
                                <td>Test execution completed</td>
                                <td>⚠️ Partial</td>
                            </tr>
                        </tbody>
                    </table>
                </div>
            </div>
        </div>
    </div>

    <!-- Image Modal -->
    <div id="imageModal" class="modal" onclick="closeModal()">
        <span class="modal-close">&times;</span>
        <img id="modalImage" src="" alt="Screenshot">
    </div>

    <script>
        function switchTab(tabName) {
            // Hide all tab contents
            document.querySelectorAll('.tab-content').forEach(content => {
                content.classList.remove('active');
            });

            // Remove active class from all tabs
            document.querySelectorAll('.tab').forEach(tab => {
                tab.classList.remove('active');
            });

            // Show selected tab content
            document.getElementById(tabName).classList.add('active');

            // Add active class to clicked tab
            event.target.classList.add('active');
        }

        function toggleTestCase(header) {
            const testCase = header.parentElement;
            testCase.classList.toggle('expanded');
        }

        function openModal(imageSrc) {
            const modal = document.getElementById('imageModal');
            const modalImg = document.getElementById('modalImage');
            modal.classList.add('active');
            modalImg.src = imageSrc;
        }

        function closeModal() {
            const modal = document.getElementById('imageModal');
            modal.classList.remove('active');
        }

        // Close modal on Escape key
        document.addEventListener('keydown', function(event) {
            if (event.key === 'Escape') {
                closeModal();
            }
        });
    </script>
</body>
</html>

Detailed Test Case Report

Project: [Project Name] Test Phase: [Phase Name] Prepared By: [Tester Name] Date: [YYYY-MM-DD]

---

Test Case: [Case ID]

Field	Value
Case ID	[SYS-MODULE-TC-NNN]
Test Module	[Module Name]
Requirement ID	[REQ-XXX]
Case Type	[Functional/Performance/Security/Integration]
Title/Objective	[Clear statement of what is being tested]
Priority	[P1 (High) / P2 (Medium) / P3 (Low)]

Pre-Conditions

[Required state 1]
[Required state 2]
[Required state 3]

Test Steps

Step	Action	Expected Behavior
1	[Detailed action step 1]	[Expected outcome 1]
2	[Detailed action step 2]	[Expected outcome 2]
3	[Detailed action step 3]	[Expected outcome 3]

Test Data

Data Field	Value
[Field 1]	[Value 1]
[Field 2]	[Value 2]
[Field 3]	[Value 3]

Expected Result

[Comprehensive description of correct system behavior per requirements. Be specific, measurable, and verifiable.]

Actual Result

[To be filled during test execution - describe observed behavior]

Status

[ ] Pass - Actual result matches expected result
[ ] Fail - Actual result differs from expected result (create defect report)
[ ] Blocked - Cannot execute due to dependency or environment issue
[ ] Not Run - Test has not been executed yet

Notes

[Any additional observations, screenshots, or relevant information]

---

Test Case: [Case ID]

[Repeat the above template for additional test cases]

# .test.env - Test Environment Configuration Template
#
# Instructions: Fill in all values before starting test execution
# This file should be kept secure and never committed to version control

# AWS CLI Configuration
AWS_PROFILE=your-aws-profile
AWS_REGION=us-east-1

# CloudFormation Stack
STACK_NAME=your-cloudformation-stack-name

# Application Portal
PORTAL_URL=https://your-portal-url.example.com

# Test User Credentials
# WARNING: Replace placeholders with actual credentials. Never commit real credentials.
USER=testuser@example.com
PASSWORD=<YOUR_PASSWORD_HERE>

# Optional: Additional Environment Variables
# API_ENDPOINT=https://api.example.com
# DATABASE_URL=postgresql://localhost:5432/testdb

System Test Summary Report

Project: [Project Name] Test Phase: [Phase/Sprint Name] Report Date: [YYYY-MM-DD] Prepared By: [Tester Name] Version Tested: [Build/Version Number]

---

1. Executive Summary

[Brief overview of testing scope, objectives, and key findings. 2-3 paragraphs summarizing the test execution results and overall system quality assessment.]

---

2. Test Execution Statistics

Overall Test Metrics

Metric	Count	Percentage
Total Test Cases	[N]	100%
Executed	[N]	[%]
Passed	[N]	[%]
Failed	[N]	[%]
Blocked	[N]	[%]
Not Run	[N]	[%]

Test Execution Timeline

Phase	Start Date	End Date	Duration
Preparation	[YYYY-MM-DD]	[YYYY-MM-DD]	[X days]
Planning & Design	[YYYY-MM-DD]	[YYYY-MM-DD]	[X days]
Execution	[YYYY-MM-DD]	[YYYY-MM-DD]	[X days]
Evaluation & Reporting	[YYYY-MM-DD]	[YYYY-MM-DD]	[X days]
Total	[YYYY-MM-DD]	[YYYY-MM-DD]	[X days]

---

3. Test Case Results by Module

Module	Total	Passed	Failed	Blocked	Not Run	Pass Rate
[Module 1]	[N]	[N]	[N]	[N]	[N]	[%]
[Module 2]	[N]	[N]	[N]	[N]	[N]	[%]
[Module 3]	[N]	[N]	[N]	[N]	[N]	[%]
Total	[N]	[N]	[N]	[N]	[N]	[%]

---

4. Defect Summary

Defect Statistics

Metric	Count
Total Defects Found	[N]
Critical (S1)	[N]
Major (S2)	[N]
Minor (S3)	[N]
Trivial (S4)	[N]

Defect Status Distribution

Status	Count	Percentage
New	[N]	[%]
Assigned	[N]	[%]
In Progress	[N]	[%]
To Be Retested	[N]	[%]
Resolved	[N]	[%]
Closed	[N]	[%]

Critical & Major Defects

Defect ID	Module	Summary	Severity	Priority	Status
[D-XXX-001]	[Module]	[Brief description]	[S1/S2]	[P1/P2]	[Status]
[D-XXX-002]	[Module]	[Brief description]	[S1/S2]	[P1/P2]	[Status]

---

5. Test Coverage Analysis

Functional Coverage

Requirement Area	Test Cases	Coverage Status
[Area 1]	[N test cases]	[Fully Covered / Partially Covered / Not Covered]
[Area 2]	[N test cases]	[Fully Covered / Partially Covered / Not Covered]
[Area 3]	[N test cases]	[Fully Covered / Partially Covered / Not Covered]

Coverage Gaps

[List any requirements or functional areas that were not adequately tested and explain why]

---

6. Environment & Infrastructure

Test Environment Details

Component	Configuration
AWS Profile	[profile-name]
AWS Region	[region]
CloudFormation Stack	[stack-name]
Stack Status	[status]
Portal URL	[url]
Database Version	[version]

Environment Issues

[Document any environment-related issues encountered during testing]

---

7. Risk Assessment

High-Risk Areas

1. [Risk Area 1]

Issue: [Description of the risk]
Impact: [Business/technical impact]
Mitigation: [Recommended action]

2. [Risk Area 2]

Issue: [Description of the risk]
Impact: [Business/technical impact]
Mitigation: [Recommended action]

---

8. Recommendations

Immediate Actions Required

1. [Critical action item 1 with justification] 2. [Critical action item 2 with justification]

Future Improvements

1. [Improvement suggestion 1] 2. [Improvement suggestion 2] 3. [Improvement suggestion 3]

Test Process Enhancements

[Suggestions for improving the testing process, automation opportunities, or test coverage]

---

9. Conclusion

Overall Assessment

[Provide a clear assessment of system quality and readiness. Include:]

Overall system stability
Critical functionality status
Recommendation for release/go-live decision
Any prerequisites for production deployment

Sign-Off

Role	Name	Date
Test Lead	[Name]	[YYYY-MM-DD]
QA Manager	[Name]	[YYYY-MM-DD]
Project Manager	[Name]	[YYYY-MM-DD]

---

10. Appendices

Appendix A: Test Case Details

[Link to detailed test case report]

Appendix B: Defect Reports

[Links to individual defect reports]

Appendix C: Test Evidence

[Links to screenshots, logs, recordings, and other test artifacts]

End-to-End Testing Skill

Overview

The End-to-End Testing skill provides a systematic workflow for running comprehensive integration and E2E tests with built-in evidence capture and professional reporting. It guides you through test preparation, planning, execution, validation, and report generation with an organized output structure.

Installation

npx skills add https://github.com/aws-samples/sample-agent-skills-for-builders --skill end-to-end-testing

When to Use

Reference this skill when:

Running integration or E2E tests
Validating complete user flows
Capturing test evidence (screenshots, logs)
Generating test reports
Documenting defects systematically

Quick Start

# Validate your test environment setup
./scripts/validate-test-data.sh

# Generate an HTML report from test results
python scripts/generate-html-report.py --input test-results/

# View the generated report in your browser
open test-reports/YYYY-MM-DD_HHMMSS/Test_Report_Viewer.html

File Structure

end-to-end-testing/
├── SKILL.md                          # Skill definition and workflow
├── README.md                         # This file
├── scripts/
│   ├── generate-html-report.py      # Convert test results to HTML
│   └── validate-test-data.sh        # Verify test environment
├── assets/
│   └── templates/
│       ├── defect-report.md         # Template for bug documentation
│       ├── test-case-report.md      # Template for test results
│       └── test-summary-report.md   # Template for test summaries
└── references/
    ├── preparation.md               # Environment setup guide
    ├── planning-design.md           # Test planning strategies
    ├── execution.md                 # Test execution best practices
    ├── validation.md                # Pre-reporting validation checklist
    └── reporting.md                 # Report generation guide

Prerequisites

Before using this skill, ensure you have:

A test environment that can be accessed and configured
Test data and credentials available (keep these secure)
Python 3.7+ (for HTML report generation)
Bash shell access (for validation scripts)
Screenshot/evidence capture tools for your target platform
At least one test framework set up (Selenium, Playwright, Cypress, etc.)

Workflow Overview

Phase 1: Preparation

Gather environment information, verify test credentials, and set up test data.

Reference: See preparation.md for detailed steps.

Phase 2: Planning & Design

Conduct smoke testing, design test cases, and assign priorities.

Reference: See planning-design.md for strategies.

Phase 3: Execution

Execute test cases, capture evidence, and document any defects found.

Reference: See execution.md for best practices.

Phase 4: Pre-Reporting Validation (CRITICAL)

Before generating reports, verify all evidence is captured and defects are properly documented.

Reference: See validation.md for the validation checklist.

Phase 5: Reporting

Generate comprehensive reports with HTML viewer and archive all evidence.

Reference: See reporting.md for detailed instructions.

Output Structure

Test reports are automatically organized with timestamps:

test-reports/
└── 2024-04-24_143022/
    ├── Test_Report_Viewer.html       # Interactive HTML report
    ├── test-case-reports/            # Individual test case results
    ├── defect-reports/               # Bug documentation
    ├── test-summary-report.md        # Markdown summary
    ├── screenshots/                  # Evidence capture
    └── logs/                         # Test execution logs

Templates

Use these templates when documenting test results:

[Defect Report](./assets/templates/defect-report.md) - Standard format for bug documentation
[Test Case Report](./assets/templates/test-case-report.md) - Format for individual test results
[Test Summary](./assets/templates/test-summary-report.md) - Overall testing summary

Usage Examples

Generate a Test Report

python scripts/generate-html-report.py --input test-results/

Validate Test Environment

./scripts/validate-test-data.sh

Document a Defect

1. Copy the defect report template 2. Fill in defect details, severity, steps to reproduce 3. Attach screenshots and logs 4. Place in test-reports/[TIMESTAMP]/defect-reports/

Additional Resources

For detailed guidance on each workflow phase, see:

Preparation Guide
Planning & Design Guide
Execution Guide
Validation Guide
Reporting Guide

License

MIT - See LICENSE file for details.

Test Execution Phase

Overview

Execute test cases systematically and document defects using standardized reporting formats. This ensures efficient communication with development teams and proper issue tracking throughout the defect lifecycle.

Defect Report Standards

A Defect Report communicates quality issues to development teams. Effective reports enable rapid problem resolution by providing complete, reproducible information.

Report Structure

Defect reports contain three sections:

1. Essential Metadata

Quick identification and classification fields:

Field	Purpose	Requirements
Defect ID	Unique identifier for the report	Auto-generated by defect management tools (e.g., Jira, Bugzilla)
Summary/Title	Concise description of the defect	Brief and accurate; typically follows "[Module] + [Action] + [Result]" structure
Reporter	Test engineer who submitted the report	Name or user ID
Date Found	Date and time when defect was discovered	Auto-recorded
Status	Current stage in the defect lifecycle	Options: New, Assigned, In Progress, To Be Retested, Closed, Resolved
Assigned To	Developer responsible for fixing the defect	Specify the responsible developer
Module	Specific functional module or component where defect occurs	Examples: User Registration, Payment Gateway, Frontend UI
Test Version	Software version/build where defect was found	Build number or version identifier

2. Detailed Description

Core information for issue reproduction:

Field	Purpose	Requirements
Steps to Reproduce	Step-by-step instructions to trigger the defect	Atomic, clearly numbered steps; must be reproducible by anyone following the steps
Expected Result	Correct system behavior according to requirements	Objective and specific description of intended functionality
Actual Result	Observed incorrect behavior or phenomenon	Objective description avoiding subjective judgments
Environment	System environment where defect occurred	Browser/App version, OS, test URL, database version, network conditions, etc.
Attachments	Supporting evidence for troubleshooting	Screenshots (required), log files, screen recordings, API response data

3. Severity & Priority

Classification for fix urgency and impact:

Field	Description	Common Classifications
Severity	Impact level of the defect on the system (technical perspective)	S1 (Critical): System crash/data loss<br>S2 (Major): Core functionality unavailable<br>S3 (Minor): Non-core functionality impaired<br>S4 (Trivial): UI/text errors
Priority	Urgency of fixing the defect (business perspective)	P1 (Urgent): Must fix immediately; blocks progress<br>P2 (High): Must fix in next iteration<br>P3 (Medium): Can be scheduled for fix<br>P4 (Low): Fix when time permits

---

Defect Report Template

Use this template when documenting defects:

# 🐞 Defect Report

---

### 📌 1. Essential Metadata

| Field | Value |
|:------|:------|
| **Defect ID** | D-PAY-20251111-042 |
| **Summary** | [Payment Module] Order status not updated to "Paid" after Alipay payment completion |
| **Reporter** | Jane Smith (jane.smith@company.com) |
| **Date Found** | 2025-11-11 10:30 AM |
| **Status** | New |
| **Assigned To** | John Doe (john.doe@company.com) - Payment Gateway Team |
| **Module** | Payment Integration Service (Alipay) |
| **Test Version** | Build 1.3.5 |

---

### 🚨 2. Severity & Priority

| Field | Classification | Description |
|:------|:---------------|:------------|
| **Severity** | **S2 (Major)** | Core business process impaired; users cannot proceed with order fulfillment after payment |
| **Priority** | **P1 (Urgent)** | Directly impacts revenue and user experience; requires immediate fix |

---

### 📝 3. Detailed Description

#### Steps to Reproduce

1. User (`user001`) logs into the e-commerce platform and adds any product to the shopping cart.
2. Click "Checkout" button to proceed to the payment page.
3. Select **Alipay** as the payment method.
4. Click "Confirm Payment" button; system redirects to Alipay sandbox environment.
5. Complete payment in Alipay interface.
6. Wait for automatic redirect back to the order details page.

#### Expected Result

According to the requirements document, the system should exhibit the following correct behavior:

> 1. Order details page displays order status as **"Paid"**.
> 2. In the backend admin system, order status is synchronized and updated to "Paid".

#### Actual Result

The observed incorrect behavior is:

> 1. Page redirects back to the order details page, but **order status still shows "Pending Payment"**.
> 2. Attempting to click "Pay" button again, system displays error message "This order has already been paid, please do not repeat the operation", but status remains unchanged.

#### Environment Information

* **Operating System:** Windows 11
* **Browser Version:** Google Chrome 108.0
* **Test URL:** `https://test.mall.com/order/D12345`
* **Test Data:** Order ID: D12345, User ID: user001
* **Database:** MySQL 8.0.32
* **API Version:** Payment Gateway API v2.3

---

### 🖼️ 4. Attachments

* `screenshot_order_status_bug.png` - Order details page screenshot showing "Pending Payment" status
* `server_log_20251111_1030.txt` - Backend service log excerpt showing payment callback processing exception
* `api_response_alipay_callback.json` - Alipay callback response data

---

### 💡 5. Additional Notes

* Issue appears to be intermittent; affects approximately 15% of Alipay transactions
* Database query shows payment record was successfully created with transaction ID `TXN-20251111-001234`
* Suspected root cause: Race condition in payment callback handler or database transaction rollback

---

Execution Workflow

When executing test cases and discovering defects:

1. Execute systematically: Follow test cases in planned sequence 2. Document immediately: Record defects as soon as discovered 3. Gather evidence: Capture screenshots, logs, and API responses 4. Reproduce consistently: Verify defect can be triggered repeatedly 5. Classify accurately: Assign appropriate severity and priority 6. Continue testing: Don't halt entire test suite for single defect 7. Track status: Update defect lifecycle states promptly

File Organization During Execution

All test artifacts must be saved to the timestamped test reports directory created in Phase 2 (Planning & Design).

Directory Structure

test-reports/YYYY-MM-DD_HHMMSS/
├── test-case-reports/       # Test case execution results
├── defect-reports/          # Defect documentation
├── screenshots/             # Test evidence screenshots
└── logs/                    # Log files and data

File Naming Conventions

Test Case Reports:

Format: TC-[ID]-[Name].md
Examples: TC-001-Login.md, TC-002-UserRegistration.md
Location: test-reports/YYYY-MM-DD_HHMMSS/test-case-reports/

Defect Reports:

Format: DEFECT-[Number]-[ShortDescription].md
Examples: DEFECT-001-Login-Failure.md, DEFECT-002-Payment-Status-Bug.md
Location: test-reports/YYYY-MM-DD_HHMMSS/defect-reports/

Screenshots:

Format: [Number]_[description].png (sequential numbering)
Examples: 01_portal_home.png, 02_login_page.png, 03_error_message.png
Location: test-reports/YYYY-MM-DD_HHMMSS/screenshots/
Sequential numbering ensures logical ordering in the HTML report viewer

Log Files:

Format: [type]_[description].[extension]
Examples: api_response.json, kubernetes_logs.txt, execution_timeline.log
Location: test-reports/YYYY-MM-DD_HHMMSS/logs/

Evidence Collection Guidelines

Screenshots:

Capture at least one screenshot per test case (even for passed tests)
For defects, capture multiple screenshots showing:
Initial state before action
Error message or incorrect behavior
Expected vs actual comparison (if applicable)
Use sequential numbering (01_, 02_, 03_) for chronological ordering
Include test case ID in filename if linking to specific test: 05_TC-003_payment_error.png

Log Files:

Save API request/response logs for failed test cases
Include Kubernetes pod logs if investigating infrastructure issues
Save database query results if data validation fails
Include browser console logs for frontend errors
Store execution timelines for performance analysis

API Error Log Collection (Critical for Failed Tests)

When a test case fails due to an API error, immediately capture and save the complete API transaction details:

Required Information: 1. Request Details:

HTTP method and full URL
Request headers (sanitize sensitive tokens)
Complete request body (JSON/XML payload)
Timestamp of request

2. Response Details:

HTTP status code
Response headers
Complete response body
Response timestamp

3. Additional Context:

Test case ID that triggered the request
User/session context
Environment details (region, cluster, etc.)

File Naming Convention:

Format: api_error_[TEST-CASE-ID]_[description].json
Examples:
api_error_TC-021_create_environment.json
api_error_TC-031_start_training.json
Location: test-reports/YYYY-MM-DD_HHMMSS/logs/

Log File Structure:

{
  "testCaseId": "TC-021",
  "timestamp": "2025-11-12T09:22:15.234Z",
  "request": {
    "method": "POST",
    "url": "/api/create_game_env",
    "headers": {
      "Content-Type": "application/json",
      "Authorization": "Bearer eyJ... (truncated)"
    },
    "body": {
      "gameEnvName": "HeadlessEnv-Test",
      "gameType": "Headless",
      "nodeGroup": "x86-ng-v2"
    }
  },
  "response": {
    "statusCode": 500,
    "statusText": "Internal Server Error",
    "headers": {
      "Content-Type": "application/json; charset=utf-8"
    },
    "body": {
      "message": "Error creating service",
      "timestamp": "2025-11-12T09:22:15.234Z",
      "path": "/api/create_game_env"
    }
  },
  "error": {
    "type": "APIError",
    "message": "Server returned 500 status code"
  }
}

Why This Matters:

Developers need exact request/response to reproduce issues
API logs are automatically displayed in the HTML report for failed test cases
Complete logs enable faster root cause analysis
Logs serve as evidence for defect reports

How to Capture: 1. Use browser DevTools Network tab → Right-click request → "Copy as cURL" or "Save all as HAR" 2. Use API testing tools (Postman, curl) with -v flag for verbose output 3. Capture from application logs if available 4. Use browser console: console.log(JSON.stringify(response, null, 2))

Best Practices:

✅ Capture logs immediately when error occurs (before retry attempts)
✅ Sanitize sensitive data (passwords, full tokens, API keys)
✅ Keep the full error response body (even if it seems uninformative)
✅ Include correlation IDs or request IDs if present
❌ Don't skip API logging even for "simple" errors
❌ Don't just save a screenshot of the error - save the actual API data

Relative Path References:

When writing defect reports or test case reports, reference screenshots using relative paths
Example in markdown: ![Error Screenshot](../screenshots/05_payment_error.png)
Reference API logs: See logs/api_error_tc21_create_environment.json for details
This ensures links work when the HTML report is generated

Quality Checklist

Before submitting a defect report, verify:

[ ] Steps to reproduce are clear and numbered
[ ] Expected vs actual results are explicitly stated
[ ] Environment information is complete
[ ] Screenshots/logs are attached
[ ] Severity and priority are justified
[ ] Issue is reproducible on your end
[ ] Related defects are cross-referenced

Common Pitfalls to Avoid

Vague descriptions: "System doesn't work" → Specify exact behavior
Missing context: Always include environment and test data
Subjective language: Use objective observations only
Incomplete steps: Ensure anyone can reproduce from your steps
Wrong classification: Severity ≠ Priority; assess both independently
Single screenshot: Capture before/after states and error messages

Planning & Design Phase

Overview

Validate test environment readiness through smoke testing and transform user-provided test cases into detailed, executable test case reports. This phase ensures environment stability and test case completeness before execution.

Phase Activities

1. Smoke Testing

Perform quick validation checks to confirm basic system functionality.

Required Smoke Tests

Test	Command/Action	Success Criteria
AWS CLI Access	`aws sts get-caller-identity --profile <profile>`	Returns account ID and user ARN
CloudFormation Stack	`aws cloudformation describe-stacks --stack-name <name>`	Stack status is `CREATE_COMPLETE` or `UPDATE_COMPLETE`
Portal Accessibility	Navigate to portal URL in browser	Page loads with HTTP 200, login page visible
Basic Authentication	Login with test credentials	Successful authentication and redirect

Smoke Test Checklist:

[ ] AWS CLI commands execute without errors
[ ] Stack resources are accessible
[ ] Portal responds within acceptable time
[ ] Test user can authenticate
[ ] No critical infrastructure issues detected

---

2. Detailed Test Case Design

Transform high-level test cases into comprehensive, executable test case reports following standardized format.

Detailed Test Case Report Format

The Detailed Test Case Report guides test execution with clear, traceable, and atomic test cases. Each test case must be:

Atomic: Tests single functional point or scenario
Traceable: Links to specific requirements
Reproducible: Anyone can execute following the steps
Complete: Includes all required fields

Required Fields

Field	Description	Criticality	Notes
Case ID	Unique identifier with project/module prefix	Required	Format: `SYS-[MODULE]-TC-[NNN]`
Test Module	Functional module being tested	Required	Examples: User Registration, Order Management
Requirement ID	Link to SRS requirement number	Required	Enables traceability
Case Type	Testing category	Required	Functional, Performance, Security, etc.
Title/Objective	Clear statement of what is being tested	Required	Brief and specific
Priority	Test importance level	Required	P1 (High), P2 (Medium), P3 (Low)
Pre-Conditions	Required state before test execution	Required	User state, system state, data state
Test Steps	Detailed, numbered execution steps	Required	Clear, sequential, actionable
Test Data	Specific data used in test	Required	Usernames, IDs, input values
Expected Result	Correct behavior per requirements	Required	Specific, measurable, verifiable
Actual Result	Observed outcome after execution	Fill during execution	Compare with expected result
Status	Test execution status	Fill during execution	Pass / Fail / Blocked / Not Run

---

Test Case Examples

Example 1: Successful Registration (Happy Path)

Field	Value
Case ID	SYS-REG-TC-001
Test Module	User Registration
Requirement ID	SRS-USR-R1.1, SRS-USR-R1.5
Case Type	Functional Testing (Positive)
Title/Objective	Successfully create new user with valid, unique credentials
Priority	P1 (High)
Pre-Conditions	1. User is not logged in<br>2. Registration page is accessible<br>3. Email `test001@example.com` is not already registered
Test Steps	1. Navigate to registration page<br>2. Enter username: `Tester001`<br>3. Enter email: `test001@example.com`<br>4. Enter password: `<VALID_PASSWORD>`<br>5. Enter confirm password: `<VALID_PASSWORD>`<br>6. Check "I agree to terms" checkbox<br>7. Click "Register" button
Test Data	Username: `Tester001`<br>Email: `test001@example.com`<br>Password: `<VALID_PASSWORD>`<br>Confirm Password: `<VALID_PASSWORD>`
Expected Result	1. System displays "Registration successful" message<br>2. Page redirects to user dashboard or login page<br>3. New user record exists in database with encrypted password<br>4. Confirmation email sent to user
Actual Result	_(Fill during execution)_
Status	Not Run

---

Example 2: Registration with Existing Email (Negative Path)

Field	Value
Case ID	SYS-REG-TC-002
Test Module	User Registration
Requirement ID	SRS-USR-R1.2
Case Type	Functional Testing (Negative)
Title/Objective	Verify system prevents registration with already registered email
Priority	P1 (High)
Pre-Conditions	1. User is not logged in<br>2. Registration page is accessible<br>3. Email `existing@example.com` is already registered in system
Test Steps	1. Navigate to registration page<br>2. Enter username: `NewUser123`<br>3. Enter email: `existing@example.com`<br>4. Enter password: `<VALID_PASSWORD>`<br>5. Enter confirm password: `<VALID_PASSWORD>`<br>6. Check "I agree to terms" checkbox<br>7. Click "Register" button
Test Data	Username: `NewUser123`<br>Email: `existing@example.com` (already registered)<br>Password: `<VALID_PASSWORD>`
Expected Result	1. System displays error: "Email already registered"<br>2. Registration is blocked<br>3. User remains on registration page<br>4. No duplicate user record created
Actual Result	_(Fill during execution)_
Status	Not Run

---

Example 3: Registration with Weak Password (Boundary Test)

Field	Value
Case ID	SYS-REG-TC-003
Test Module	User Registration
Requirement ID	SRS-USR-R1.3
Case Type	Functional Testing (Boundary)
Title/Objective	Verify password strength validation rejects weak passwords
Priority	P2 (Medium)
Pre-Conditions	1. User is not logged in<br>2. Registration page is accessible<br>3. Password policy requires: 8+ chars, uppercase, lowercase, number, special char
Test Steps	1. Navigate to registration page<br>2. Enter username: `TestUser456`<br>3. Enter email: `weak@example.com`<br>4. Enter password: `<WEAK_PASSWORD>` (weak password)<br>5. Enter confirm password: `<WEAK_PASSWORD>`<br>6. Attempt to check "I agree to terms"<br>7. Observe validation feedback
Test Data	Username: `TestUser456`<br>Email: `weak@example.com`<br>Password: `<WEAK_PASSWORD>` (fails complexity requirements)
Expected Result	1. Password field shows validation error<br>2. Error message explains password requirements<br>3. "Register" button remains disabled or shows error<br>4. Registration is blocked
Actual Result	_(Fill during execution)_
Status	Not Run

---

Test Case Design Guidelines

Priority Classification

P1 (High): Core functionality, critical user flows, security features
P2 (Medium): Important features, common user scenarios
P3 (Low): Edge cases, nice-to-have features, cosmetic issues

Test Coverage Types

Include diverse test case types:

1. Positive/Happy Path: Valid inputs, expected user behavior 2. Negative: Invalid inputs, error handling 3. Boundary: Min/max values, edge cases 4. Security: Authentication, authorization, injection attacks 5. Performance: Load times, response times under stress 6. Integration: Cross-module functionality

Quality Checklist

Before finalizing test case report, verify:

[ ] All test cases have unique Case IDs
[ ] Each case links to specific requirement(s)
[ ] Test steps are numbered and atomic
[ ] Expected results are specific and measurable
[ ] Test data includes all necessary values
[ ] Priority is justified based on business impact
[ ] Coverage includes positive, negative, and boundary cases

Deliverable

Output: Detailed Test Case Report containing all test cases in standardized format, ready for execution phase.

Next Steps

Once test case design is complete: 1. Review test cases for completeness and clarity 2. Confirm smoke tests passed successfully 3. Proceed to Execution phase to execute test cases systematically

Preparation Phase

Overview

Gather all necessary environment credentials and test case documentation before proceeding with test execution. This phase ensures you have proper access and clear testing objectives.

Required Inputs

1. Test Environment Configuration

Users must provide environment information in the following format:

# .test.env

# AWS CLI Configuration
AWS_PROFILE=test
AWS_REGION=us-east-1

# CloudFormation Stack
STACK_NAME=my-application-stack

# Application Portal
PORTAL_URL=https://test.example.com

# Test User Credentials
USER=testuser@example.com
PASSWORD=Sec0reP@ss1

Validation Checklist:

[ ] AWS credentials are valid (test with aws sts get-caller-identity)
[ ] CloudFormation stack exists and is in stable state
[ ] Portal URL is accessible
[ ] Test user credentials are confirmed working

---

2. Test Case Suite

Users must provide test cases in structured format. Below is the standard template and an example.

Test Case Template

# Case [N]: [Test Case Title]

## Pre-Conditions
- [Condition 1: State that must exist before testing]
- [Condition 2: Required setup or configuration]

## Test Steps
1. [Action 1: First step to perform]
2. [Action 2: Second step to perform]
3. [Action 3: Third step to perform]

## Test Data
- [Data Field 1]: [Specific value to use]
- [Data Field 2]: [Specific value to use]

## Expected Result
- [Expected Outcome 1: What should happen]
- [Expected Outcome 2: Additional expected behavior]

---

Example: Successful Login Test Case

# Case 1: Successful Login with Valid Credentials

## Pre-Conditions
- User has completed registration and is not currently logged in
- Login page is accessible

## Test Steps
1. Navigate to the login page
2. Enter username in the username field
3. Enter password in the password field
4. Click the "Login" button

## Test Data
- Username: user@example.com
- Password: Sec0reP@ss1

## Expected Result
- Login succeeds without errors
- Page automatically redirects to the home dashboard
- User session is established (verify cookie/token)

---

Input Validation

Before proceeding to Planning & Design phase, confirm:

1. Environment Access

AWS CLI commands execute successfully
Portal URL responds (HTTP 200)
Stack resources are deployed

2. Test Case Completeness

Each test case has all required sections
Pre-conditions are clearly stated
Test steps are atomic and sequential
Expected results are specific and verifiable

3. Credentials Security

Test credentials are for non-production environments
Sensitive data is properly protected
Access permissions are appropriate for testing

Common Issues

Issue	Solution
AWS credentials expired	Refresh credentials using SSO or IAM
Portal URL not accessible	Verify VPN connection, security groups, DNS
Stack not found	Confirm stack name and region are correct
Test cases lack detail	Request clarification on steps and expected results

Next Steps

Once all inputs are validated: 1. Store environment configuration securely 2. Review test cases for completeness 3. Proceed to Planning & Design phase for smoke testing and detailed test case design

Evaluation & Reporting Phase

Overview

Generate comprehensive test summary reports that provide stakeholders with clear insights into test execution results, system quality assessment, and actionable recommendations. This phase synthesizes all testing activities into executive-level documentation.

Phase Activities

1. Data Collection & Analysis

Gather all test execution data before report generation:

Required Data Sources:

Detailed test case reports with execution status
All defect reports with current status
Test environment configuration logs
Test execution timeline and effort tracking
Coverage analysis against requirements

Analysis Tasks:

Calculate test execution statistics
Analyze defect distribution by severity and module
Assess test coverage completeness
Identify high-risk areas based on failure patterns
Evaluate environment stability issues

---

2. Test Summary Report Structure

The System Test Summary Report is the primary deliverable of this phase. Use the template at assets/templates/test-summary-report.md as the foundation.

Required Sections

Section 1: Executive Summary

Purpose: Provide non-technical stakeholders with quick assessment Content:

Testing scope and objectives (2-3 sentences)
Key findings and overall quality assessment
Go/no-go recommendation with justification

Guidelines:

Keep it concise (200-300 words maximum)
Avoid technical jargon
Lead with the conclusion, then support with evidence

Section 2: Test Execution Statistics

Purpose: Quantify testing effort and results Content:

Total test cases vs. executed test cases
Pass/fail/blocked/not run counts and percentages
Timeline showing start/end dates for each phase

Metrics to Calculate:

Pass Rate = (Passed Test Cases / Total Executed) × 100%
Execution Rate = (Executed Test Cases / Total Test Cases) × 100%
Defect Density = Total Defects Found / Total Test Cases Executed

Section 3: Test Results by Module

Purpose: Identify which functional areas are stable vs. problematic Content: Breakdown of test results per module with pass rates

Analysis Guidelines:

Modules with <80% pass rate require attention
Modules with 100% blocked tests indicate environmental issues
Compare pass rates across modules to identify patterns

Section 4: Defect Summary

Purpose: Communicate quality issues to development and management Content:

Total defects by severity (S1/S2/S3/S4)
Defect status distribution (New/Assigned/In Progress/Resolved/Closed)
Detailed table of Critical (S1) and Major (S2) defects

Criticality Assessment:

Release Blockers: Any unresolved S1 defects
High Risk: Multiple unresolved S2 defects in core functionality
Acceptable: Only S3/S4 defects remaining with workarounds

Section 5: Test Coverage Analysis

Purpose: Identify gaps in testing scope Content:

Functional coverage by requirement area
Coverage gaps with explanations
Untested scenarios due to blockers or time constraints

Coverage Assessment Criteria:

Fully Covered: All requirements have corresponding test cases, all executed
Partially Covered: Some requirements tested, others not executed or missing tests
Not Covered: Requirements without test cases or completely untested

Section 6: Environment & Infrastructure

Purpose: Document test environment configuration for reproducibility Content:

AWS configuration (profile, region, stack details)
Portal and database versions
Environment issues encountered during testing

Include:

Any infrastructure instability
Configuration changes during testing
Access or permission issues

Section 7: Risk Assessment

Purpose: Highlight areas requiring attention before release Content:

High-risk areas with unresolved defects
Business/technical impact of each risk
Recommended mitigation strategies

Risk Prioritization: 1. Critical functionality with S1/S2 defects 2. Modules with <70% pass rate 3. Untested or blocked core workflows 4. Environment instability issues

Section 8: Recommendations

Purpose: Provide actionable next steps Content:

Immediate actions required (defect fixes, retesting)
Future improvements (automation, coverage expansion)
Test process enhancements

Structure Recommendations As:

Short-term (before release): Critical defect fixes, blocked test resolution
Medium-term (next iteration): Minor defect fixes, coverage improvements
Long-term (future sprints): Automation opportunities, process improvements

Section 9: Conclusion

Purpose: Final go/no-go decision with sign-off Content:

Overall system quality assessment
Release readiness recommendation
Prerequisites for production deployment
Sign-off table for stakeholders

Decision Criteria:

Go: No S1 defects, <3 S2 defects with workarounds, >85% pass rate
Conditional Go: Few S2 defects, >80% pass rate, acceptable coverage
No-Go: Any S1 defects unresolved, <75% pass rate, or major coverage gaps

Section 10: Appendices

Purpose: Link to supporting documentation Content:

Links to detailed test case reports
Links to individual defect reports
Links to test evidence (screenshots, logs, recordings)

---

3. Metrics Calculation Guide

Basic Metrics

Pass Rate:

Pass Rate = (Number of Passed Test Cases / Number of Executed Test Cases) × 100%

Minimum acceptable: 80%
Target: 95%+

Execution Coverage:

Execution Coverage = (Executed Test Cases / Total Planned Test Cases) × 100%

Minimum acceptable: 90%
Target: 100%

Defect Density:

Defect Density = Total Defects Found / Total Test Cases Executed

Lower is better
Industry average: 0.1 - 0.5 defects per test case

Advanced Metrics

Defect Removal Efficiency (DRE):

DRE = (Defects Found in Testing / (Defects Found in Testing + Defects Found in Production)) × 100%

Target: >95% (catch most defects before production)

Test Effectiveness:

Test Effectiveness = Number of Defects Found / Number of Test Cases Executed

Indicates how well test cases uncover defects
Higher values suggest better test case design

---

4. Report Quality Checklist

Before finalizing the test summary report, verify:

[ ] All sections are complete with accurate data
[ ] Metrics are calculated correctly
[ ] Critical and major defects are highlighted
[ ] Recommendations are specific and actionable
[ ] Executive summary provides clear conclusion
[ ] All links to supporting documentation work
[ ] Sign-off section includes stakeholder names
[ ] Report is formatted consistently and professionally
[ ] Technical terms are explained for non-technical readers
[ ] Charts or graphs are included if beneficial (optional)

---

5. HTML Report Generation

After completing the test summary report, generate an interactive HTML report viewer that consolidates all test artifacts into a single, user-friendly interface.

Purpose

Provide stakeholders with an interactive, visual dashboard that:

Displays all test results in an organized, tabbed interface
Allows easy navigation between test cases, defects, screenshots, and logs
Eliminates the need to open multiple files to understand test results
Provides a professional, shareable format for test documentation

Directory Organization

Before generating the HTML report, ensure all test artifacts are properly organized in the timestamped directory:

test-reports/YYYY-MM-DD_HHMMSS/
├── test-case-reports/       # All test case markdown files
├── defect-reports/          # All defect markdown files
├── test-summary-report.md   # Overall summary (already created)
├── screenshots/             # All captured screenshots
│   ├── 01_portal_home.png
│   ├── 05_TC-003_payment_error.png  # Links to TC-003
│   └── 07_tc11_package_list.png     # Links to TC-011 or TC-11
└── logs/                    # All log files
    ├── api_error_TC-021_create_environment.json  # Links to TC-021
    ├── kubernetes_logs.txt
    └── execution_timeline.log

Important Naming Conventions for Automatic Linking:

To enable automatic linking of screenshots and API logs to test cases:

1. Screenshots - Use one of these patterns in filenames:

[number]_TC-XXX_[description].png → Links to TC-XXX
[number]_tcXX_[description].png → Links to TC-0XX (padded)
Example: 05_TC-003_payment_error.png → Displays in TC-003 test case
Example: 07_tc11_package_list.png → Displays in TC-011 test case

2. API Error Logs - Required pattern:

api_error_TC-XXX_[description].json → Links to TC-XXX
Example: api_error_TC-021_create_environment.json → Displays in TC-021 test case
Must start with `api_error_` to be recognized as API error log
Must include TC-XXX pattern to link to correct test case

HTML Report Generation Methods

Method 1: Automated Script (Recommended)

Use the provided Python script to automatically generate the HTML report:

python skills/end-to-end-testing/scripts/generate-html-report.py test-reports/YYYY-MM-DD_HHMMSS

The script will: 1. Parse test-summary-report.md for executive summary and statistics 2. Parse all files in test-case-reports/ for test case details 3. Parse all files in defect-reports/ for defect information 4. Scan screenshots/ directory for all images 5. Scan logs/ directory for all log files 6. Automatically link screenshots to test cases based on filename patterns 7. Automatically link API error logs to test cases based on filename patterns 8. Generate Test_Report_Viewer.html with all data populated

New Features:

Inline Screenshots: Screenshots are displayed directly within each test case's expanded view
API Error Logs: Failed test cases automatically display their API error logs inline
Intelligent Linking: Screenshots and logs are automatically associated with test cases by matching filename patterns (e.g., 05_TC-003_error.png links to TC-003)

Method 2: Manual Population

If the script encounters issues or customization is needed:

1. Copy the template:

   cp skills/end-to-end-testing/assets/templates/Test_Report_Viewer.html test-reports/YYYY-MM-DD_HHMMSS/

2. Extract data from markdown files:

Read test-summary-report.md sections 1-2 for executive summary and statistics
Read each test-case-reports/*.md file for test case details
Read each defect-reports/*.md file for defect information

3. Populate HTML sections:

Executive Summary: Extract from test-summary-report.md Section 1
Test Statistics: Calculate from test case status counts
Test Cases by Status: Group test cases by Pass/Fail/Blocked/Not Run
Defects: List all defects with severity, priority, and status
Screenshots: Generate thumbnail grid from screenshots/ directory
Logs: List all log files with download links

4. Update relative paths:

Screenshot paths: screenshots/filename.png
Defect report paths: defect-reports/DEFECT-XXX.md
Log file paths: logs/filename.log

HTML Report Structure

The generated HTML report should contain these interactive sections:

Tab 1: Summary

Executive summary with key findings
Test execution statistics (total, passed, failed, blocked)
Test environment details
Module-level test coverage table
Overall pass rate visualization

Tab 2: Passed Tests

List of all test cases with status "Pass"
Expandable details for each test case
Test case metadata (module, priority, execution time)

Tab 3: Blocked Tests

List of all test cases with status "Blocked"
Blocking reasons and dependencies
Required actions to unblock

Tab 4: Defects

All defects organized by severity (S1, S2, S3, S4)
Defect details with links to full defect reports
Status indicators (New, In Progress, Resolved, Closed)
Impact assessment and priority

Tab 5: Screenshots

Thumbnail grid of all captured screenshots
Click-to-expand modal for full-size viewing
Screenshots numbered sequentially (01_, 02_, 03_, etc.)
Organized by test case if naming convention includes TC-ID

Tab 6: Logs

List of all log files with descriptions
Direct links to view/download logs
Log file types: API responses, Kubernetes logs, execution timelines
Timestamps and file sizes

Data Extraction Patterns

When parsing markdown files, use these patterns:

From test-summary-report.md:

Executive Summary section → Tab 1 summary text
Test Execution Statistics table → Tab 1 statistics
Module results table → Tab 1 module coverage
Defect summary → Tab 4 defect statistics

*From test-case-reports/.md**:

File name → Test Case ID
Status field → Group into Passed/Failed/Blocked tabs
Module, Priority, Steps → Test case details

*From defect-reports/.md**:

Defect ID, Severity, Priority → Defect listing
Description → Defect details
Status → Defect status indicator

From screenshots/ directory:

List all .png, .jpg, .jpeg files
Sort by filename (numerically if numbered)
Generate thumbnail links with modal view

From logs/ directory:

List all .log, .json, .txt files
Extract filename and size
Generate download links

HTML Generation Validation

After generating the HTML report, verify:

[ ] All sections load without errors
[ ] All tabs are clickable and display correct content
[ ] Screenshot thumbnails display correctly
[ ] Screenshot modal opens and shows full-size images
[ ] All relative paths resolve correctly (no broken links)
[ ] Defect report links work (if applicable)
[ ] Statistics match the test summary report
[ ] Test case counts are accurate
[ ] HTML renders properly in multiple browsers (Chrome, Firefox, Safari)

Troubleshooting

Issue: Screenshots don't display

Solution: Verify screenshot paths use relative paths: screenshots/filename.png
Solution: Ensure screenshot files exist in the screenshots/ directory

Issue: Statistics don't match markdown reports

Solution: Recount test cases from test-case-reports/ directory
Solution: Verify all test case files have valid status fields

Issue: HTML layout is broken

Solution: Check for JavaScript errors in browser console
Solution: Verify HTML syntax is valid (no unclosed tags)

Issue: Log files not listed

Solution: Ensure log files are in the logs/ directory
Solution: Verify file permissions allow reading

---

Deliverables

1. System Test Summary Report

Format: Markdown document using assets/templates/test-summary-report.md

Location: test-reports/YYYY-MM-DD_HHMMSS/test-summary-report.md

Distribution:

Project Manager (for planning and decision-making)
Development Team Lead (for defect prioritization)
QA Manager (for process improvement)
Product Owner (for release decisions)
Stakeholders (for visibility into quality)

Timeline: Complete within 1-2 business days after test execution concludes

2. Interactive HTML Report Viewer

Format: HTML document with embedded CSS/JavaScript

Location: test-reports/YYYY-MM-DD_HHMMSS/Test_Report_Viewer.html

Purpose: Provides stakeholders with a single, interactive interface to view all test results, defects, screenshots, and logs without navigating multiple files

Distribution:

Share the timestamped directory via email, file share, or version control
Recipients can open Test_Report_Viewer.html in any modern web browser
All assets (screenshots, logs) are accessible via relative paths

Timeline: Generate immediately after completing the test summary report

---

Best Practices

1. Be Objective: Present data without bias; let metrics speak 2. Be Clear: Use simple language and visual aids when possible 3. Be Actionable: Every recommendation should have a clear owner and timeline 4. Be Complete: Include all required sections; incomplete reports delay decisions 5. Be Honest: Report quality issues transparently; hiding problems creates bigger issues later

---

Common Pitfalls to Avoid

❌ Don't: Sugarcoat quality issues to make results look better ✅ Do: Report problems honestly with mitigation plans

❌ Don't: Use vague statements like "most tests passed" ✅ Do: Provide exact numbers and percentages

❌ Don't: Skip the conclusion or recommendation section ✅ Do: Provide clear go/no-go decision with justification

❌ Don't: Submit report without stakeholder review ✅ Do: Get sign-off from key stakeholders

❌ Don't: Leave defects untracked or unresolved ✅ Do: Ensure all defects have status and assigned owners

Related skills

TddFollow test-driven development with a strict red-green-refactor loop when creating reliable features or fixing bugs.544k196k

Test Driven DevelopmentEnforce writing failing tests before any production implementation code.181k264k

QaRun conversational QA sessions that turn user-reported bugs into well-written, domain-aware GitHub issues without manual ticket writing.179k196k

Migrate To ShoehornAutomatically update TypeScript test files that rely on unsafe `as` type assertions by replacing them with type-safe partial objects from @total-typescript/shoehorn.164k196k

Webapp TestingVerify frontend behavior, debug UI issues, capture screenshots, and inspect logs of a running local web application using Playwright.123k165k

Playwright CliRun browser automation, generate element snapshots, inspect DOM attributes, and execute Playwright tests from the terminal.102k12.3k

FAQ

What does the testing workflow produce?

Comprehensive reports plus an HTML viewer, with captured screenshots, logs, and documented defects archived by timestamp.

Why is there a pre-reporting phase?

Phase 4 is a mandatory validation step that verifies all evidence is captured and defect documentation is complete before results are reported.

Testing & QAtesting