Content Evaluation Framework

Content Evaluation Framework

This skill provides a comprehensive, systematic rubric for evaluating educational book chapters and lessons with quantifiable quality standards.

Constitution Alignment: v4.0.1 emphasizing:

• Principle 1: Specification Primacy ("Specs Are the New Syntax")
• Section IIa: Panaversity 4-Layer Teaching Method
• Section IIb: AI Three Roles Framework (bidirectional co-learning)
• 8 Foundational Principles: Including Factual Accuracy, Coherent Structure, Progressive Complexity
• Nine Pillars (Section I): AI CLI, Markdown, MCP, AI-First IDEs, Cross-Platform, TDD, SDD, Composable Skills, Cloud-Native

Purpose

Evaluate educational content across 6 weighted categories to ensure:

• Technical correctness and code quality
• Effective pedagogical design and learning outcomes
• Clear, accessible writing for target audience
• Proper structure and organization
• AI-augmented learning principles (learning WITH AI, not generating FROM AI)
• Constitution compliance and standards adherence

When to Use This Skill

Invoke this evaluation framework at multiple checkpoints:

1. During Iterative Drafting - Mid-process quality checks to catch issues early 2. After Lesson/Chapter Completion - Comprehensive evaluation before moving to next content unit 3. On-Demand Review Requests - When user explicitly asks for quality assessment 4. Before Validation Phase - Part of the SDD Validate phase workflow for final sign-off

Evaluation Methodology

Scoring System

Multi-Tier Assessment:

• Excellent (90-100%) - Exceeds standards, exemplary quality
• Good (75-89%) - Meets all standards with minor improvements possible
• Needs Work (50-74%) - Meets some standards but requires significant revision
• Insufficient (<50%) - Does not meet minimum standards, requires major rework

Weighted Categories

The evaluation uses 6 categories with the following weights:

Total Weighted Score Calculation:

Final Score = (Technical × 0.30) + (Pedagogical × 0.25) + (Writing × 0.20) +
              (Structure × 0.15) + (AI-First × 0.10)

Constitution Compliance: Must achieve "Pass" status. If "Fail," content cannot proceed regardless of weighted score.

How to Conduct an Evaluation

Step 1: Prepare Context

Before evaluation, gather:

• Content being evaluated (lesson.md, chapter.md, or section file)
• Relevant spec, plan, and tasks files from specs/<feature>/
• Constitution file (.specify/memory/constitution.md)
• Learning objectives and success criteria for the content unit
• Output style template used (.claude/output-styles/lesson.md or similar)

Step 2: Load Detailed Rubric

Read the detailed tier criteria for each category:

Read: references/rubric-details.md

This file contains specific criteria defining Excellent/Good/Needs Work/Insufficient for each of the 6 categories.

Step 3: Evaluate Constitution Compliance First

Constitution compliance is a gate - if content fails constitutional requirements, it cannot proceed.

Use the constitution checklist:

Read: references/constitution-checklist.md

Assess all non-negotiable principles and requirements. Mark as Pass or Fail with specific violations noted.

If Constitution Compliance = Fail: Stop evaluation and report violations immediately. Content must be revised before proceeding.

If Constitution Compliance = Pass: Continue to weighted category evaluation.

Step 4: Score Each Weighted Category

For each of the 5 weighted categories (Technical Accuracy, Pedagogical Effectiveness, Writing Quality, Structure & Organization, AI-First Teaching):

1. Review specific criteria from rubric-details.md for that category 2. Assess content against criteria for each tier 3. Assign tier (Excellent/Good/Needs Work/Insufficient) with score range 4. Record specific evidence - Quote examples, note line numbers, cite specific passages 5. Provide improvement recommendations - Concrete, actionable feedback

Step 5: Calculate Weighted Score

Apply the weighted formula:

Final Score = (Technical × 0.30) + (Pedagogical × 0.25) + (Writing × 0.20) +
              (Structure × 0.15) + (AI-First × 0.10)

Convert tier scores to numeric values:

• Excellent: 95%
• Good: 82%
• Needs Work: 62%
• Insufficient: 40%

(Or use specific numeric score within tier range if warranted)

Step 6: Generate Evaluation Report

Use the structured evaluation template:

Read: references/evaluation-template.md

Complete all sections: 1. Executive Summary - Overall score, tier, pass/fail status 2. Category Scores - Table showing each category score, tier, and weight contribution 3. Detailed Findings - Evidence-based assessment for each category 4. Strengths - What the content does well (specific examples) 5. Areas for Improvement - Prioritized list of issues with recommendations 6. Constitution Compliance Status - Pass/Fail with specific principle checks 7. Actionable Next Steps - Concrete tasks to improve content

Step 7: Communicate Results

Present evaluation report with:

• Clear verdict - Pass/Fail and overall quality tier
• Evidence-based feedback - Specific quotes and line numbers
• Prioritized improvements - Most critical issues first
• Encouragement - Acknowledge strengths and effort

Evaluation Best Practices

Be Objective and Evidence-Based

• Quote specific passages from content being evaluated
• Reference line numbers or section headers
• Compare against objective rubric criteria, not subjective preference
• Use concrete metrics where possible (word count, readability scores, etc.)

Focus on Standards, Not Perfection

• Content rated "Good" (75-89%) is publication-ready with minor polish
• Content rated "Excellent" (90-100%) exceeds standards but is not required
• Focus improvements on moving "Needs Work" → "Good" before "Good" → "Excellent"

Provide Actionable Feedback

• Don't just say "improve clarity" - specify which sentences are unclear and suggest rewrites
• Don't just say "add examples" - suggest specific example types that would help
• Prioritize recommendations: critical (blocking issues) → important → nice-to-have

Respect the Learning Journey

• Recognize iterative improvement - drafts evolve through multiple passes
• Celebrate progress and strengths
• Frame criticism constructively as opportunities for growth
• Remember: the goal is helping create excellent educational content, not gatekeeping

Quality Gates and Thresholds

Minimum Acceptance Threshold

• Constitution Compliance: MUST be Pass (gate)
• Overall Weighted Score: MUST be ≥ 75% (Good or better)
• No category below 50%: Each individual category must achieve at least "Needs Work" tier

Recommended for Publication

• Constitution Compliance: Pass
• Overall Weighted Score: ≥ 82% (Good tier)
• Technical Accuracy: ≥ 75% (Good tier) - Critical for credibility
• Pedagogical Effectiveness: ≥ 75% (Good tier) - Critical for learning outcomes

Exemplary Content (Optional)

• Overall Weighted Score: ≥ 90% (Excellent tier)
• At least 3 categories at Excellent tier
• No categories below Good tier

Common Evaluation Scenarios

Scenario 1: Mid-Draft Check (Iterative)

Context: Writer requests feedback on partial draft Approach:

• Focus on foundational issues (structure, learning objectives, concept scaffolding)
• Flag critical issues early (technical errors, constitution violations)
• Provide guidance for remaining sections
• Don't expect polish - prioritize content completeness and correctness

Scenario 2: Completion Review

Context: Writer believes content is complete and ready for validation Approach:

• Conduct full evaluation across all 6 categories
• Calculate final weighted score
• Check all quality gates and thresholds
• Provide comprehensive report with prioritized improvements
• Determine if content meets publication standards

Scenario 3: Pre-Validation Quality Gate

Context: Content enters SDD Validate phase Approach:

• Verify constitution compliance (gate)
• Confirm minimum acceptance threshold (≥75%)
• Validate all category scores meet minimums
• Generate pass/fail recommendation with evidence
• If fails gate: return to implementation with specific revision tasks

Scenario 4: On-Demand Spot Check

Context: User asks "How's this looking?" for specific section Approach:

• Evaluate relevant categories for that section (may not be all 6)
• Provide quick feedback on specific concerns
• Highlight any critical issues
• Suggest improvements without full formal report
• Use judgment on depth based on context

Resources and References

This skill includes detailed reference materials:

• `references/rubric-details.md` - Comprehensive tier criteria for all 6 categories with specific indicators
• `references/constitution-checklist.md` - Pass/Fail checklist for constitutional compliance evaluation
• `references/evaluation-template.md` - Structured template for consistent evaluation reports

Load these references as needed during evaluation to ensure consistency and thoroughness.

---

Example Evaluation Flow

User Request: "Please evaluate this lesson draft: apps/learn-app/docs/chapter-3/lesson-2.md"

Evaluation Process:

1. Read content: apps/learn-app/docs/chapter-3/lesson-2.md 2. Load context: spec, plan, constitution, learning objectives 3. Check constitution compliance: references/constitution-checklist.md

• Result: Pass (all non-negotiables met)

4. Load detailed rubric: references/rubric-details.md 5. Evaluate each category:

• Technical Accuracy: Good (80%) - Code works, minor type hint gaps
• Pedagogical Effectiveness: Excellent (92%) - Strong scaffolding, great exercises
• Writing Quality: Good (78%) - Clear writing, minor readability improvements
• Structure & Organization: Good (85%) - Good flow, all LOs met
• AI-First Teaching: Needs Work (65%) - AI exercises present but weak guidance

6. Calculate weighted score:

• (80×0.30) + (92×0.25) + (78×0.20) + (85×0.15) + (65×0.10) = 81.55%
• Final Tier: Good (81.55%)

7. Load template: references/evaluation-template.md 8. Generate report with findings, strengths, improvements, next steps 9. Communicate verdict: "Good (81.55%) - Ready for publication with minor improvements to AI-First Teaching section"

---

Use this skill to maintain consistent, objective, evidence-based quality standards for all educational content.

Constitution Compliance Checklist

This checklist evaluates content against the CoLearning Python & Agentic AI: The AI-Driven Way project constitution (Version 2.2.0).

Result: Content must achieve PASS status on all critical items to proceed. Constitution compliance is a GATE — failures block progression regardless of weighted scores.

---

Instructions

For each section, evaluate content and mark:

• ✅ Pass — Fully compliant with constitutional requirement
• ⚠️ Partial — Partially compliant but needs minor improvement
• ❌ Fail — Non-compliant, requires significant revision

Critical Items (marked with 🔴): ANY failure results in overall FAIL status.

---

Section 1: Core Principles (11 Principles)

Principle 1: AI-First Teaching Philosophy 🔴

Requirement: Every concept, example, and exercise demonstrates AI-assisted development as primary workflow.

• [ ] AI tools used throughout content (not just mentioned once)
• [ ] Code examples show both prompt/request AND AI-generated result
• [ ] Students explicitly taught to write effective prompts
• [ ] Traditional "manual coding first" approach rejected or reframed
• [ ] AI collaboration skills integrated into every exercise

Evidence Required: Quote specific examples showing AI-first workflow

Verdict: ⬜ Pass / ⬜ Partial / ⬜ Fail

---

Principle 2: Spec-Kit Methodology as Foundation 🔴

Requirement: Spec-Kit methodology introduced progressively and applied to projects.

• [ ] Spec-Kit concepts referenced appropriately for chapter placement
• [ ] Projects use Spec → Plan → Tasks → Implementation structure (where applicable)
• [ ] Students practice specification writing WITH AI (where applicable)
• [ ] Constitution, ADR, PHR concepts explained as real artifacts (where applicable)

Evidence Required: Note how Spec-Kit is applied or referenced

Verdict: ⬜ Pass / ⬜ Partial / ⬜ Fail

---

Principle 3: Modern Python Standards (3.13+) 🔴

Requirement: All code uses Python 3.13+ with mandatory type hints.

• [ ] Python 3.13+ syntax used (no legacy patterns)
• [ ] Type hints present on ALL function signatures (100% coverage)
• [ ] Modern syntax features demonstrated (match/case, structural patterns, etc.)
• [ ] No pre-3.10 type hint styles (e.g., Optional from typing module)
• [ ] Type safety validated (mypy or pyright checks mentioned or implied)

Evidence Required: Review all code blocks for type hints and modern syntax

Verdict: ⬜ Pass / ⬜ Partial / ⬜ Fail

---

Principle 4: Test-First Mindset 🔴

Requirement: Testing integrated early and practiced throughout (not relegated to late chapter).

• [ ] Testing concepts present (appropriate for chapter position)
• [ ] Significant code examples include corresponding tests (where applicable)
• [ ] Test-writing prompts shown alongside implementation prompts (where applicable)
• [ ] TDD workflow demonstrated: write test → fail → implement → pass (where applicable)
• [ ] Coverage expectations mentioned (critical functions 100%, overall >80%) (where applicable)

Evidence Required: Note testing integration (or explain why N/A for conceptual chapters)

Verdict: ⬜ Pass / ⬜ Partial / ⬜ Fail / ⬜ N/A (conceptual chapter)

---

Principle 5: Progressive Complexity with Clear Scaffolding 🔴

Requirement: Content difficulty increases gradually with appropriate scaffolding level.

• [ ] Scaffolding appropriate for chapter position:
• Chapters 1-9: Heavy support
• Chapters 10-30: Moderate support
• Chapters 31-46: Minimal support
• [ ] Concepts introduced once, then referenced by name
• [ ] Explicit prerequisite chains documented or clear
• [ ] No forward references to unexplained concepts without "Chapter X covers this"
• [ ] Related concepts taught together, not scattered

Evidence Required: Assess complexity level and scaffolding appropriateness

Verdict: ⬜ Pass / ⬜ Partial / ⬜ Fail

---

Principle 6: Consistent Structure Across All Chapters 🔴

Requirement: Content uses shared infrastructure (skills, output styles, sub-agents) for consistency.

• [ ] Chapter/lesson follows identical structure from output-styles
• [ ] Python code follows identical formatting standards
• [ ] Exercises follow identical structure and approach (where applicable)
• [ ] Cross-chapter consistency maintained
• [ ] References same skills and constraints

Evidence Required: Confirm output style template adherence

Verdict: ⬜ Pass / ⬜ Partial / ⬜ Fail

---

Principle 7: Technical Accuracy and Currency (Always Verified) 🔴

Requirement: All technical claims verified, tools current, best practices demonstrated.

• [ ] Python 3.13+ version features verified
• [ ] Tool instructions tested (Claude Code, Gemini CLI, etc.) where applicable
• [ ] External links live and current (if present)
• [ ] Best practices demonstrated (PEP 8, Python idioms, modern patterns)
• [ ] Technical claims fact-checked and sourced
• [ ] Security practices demonstrated (no hardcoded secrets, error handling)

Evidence Required: Verify technical accuracy of all claims and code

Verdict: ⬜ Pass / ⬜ Partial / ⬜ Fail

---

Principle 8: Accessibility and Inclusivity (No Gatekeeping) 🔴

Requirement: Content welcoming and accessible to diverse learners.

• [ ] No assumed CS background (jargon explained on first use)
• [ ] No ableist language ("obviously", "simply", "just", "easy")
• [ ] Code examples include clear comments explaining intent
• [ ] Diverse example names/contexts; gender-neutral language
• [ ] Multiple explanation styles: text, code, analogies
• [ ] Platform-specific instructions where setup differs (Windows/Mac/Linux) if applicable
• [ ] Free/open-source alternatives provided if applicable

Evidence Required: Review for inclusive language and accessibility

Verdict: ⬜ Pass / ⬜ Partial / ⬜ Fail

---

Principle 9: Show-Then-Explain Pedagogy

Requirement: Teaching follows show code FIRST, then explain WHAT/HOW/WHY pattern.

• [ ] Working code presented before explanations
• [ ] WHAT the code does described (high-level overview)
• [ ] HOW it works explained (step-by-step execution)
• [ ] WHY design decisions made explained
• [ ] Variations and related patterns shown
• [ ] "Common Mistakes" section included (every chapter requirement)
• [ ] "AI Exercise" included (requirement starting Ch 3)

Evidence Required: Confirm show-then-explain pattern throughout

Verdict: ⬜ Pass / ⬜ Partial / ⬜ Fail

---

Principle 10: Real-World Project Integration

Requirement: Projects reflect realistic development scenarios, not contrived exercises.

• [ ] Projects use real tools (git, virtual environments, package managers, CI/CD concepts) where applicable
• [ ] File organization matches professional conventions (src/, tests/, history/, .env) where applicable
• [ ] Projects publishable to GitHub with README, license, docs (where applicable)
• [ ] Integration with real APIs and data sources with error handling (where applicable)
• [ ] Deployment considerations addressed (where applicable)
• [ ] Projects span multiple chapters showing iterative development (where applicable)

Evidence Required: Assess realism of projects and examples

Verdict: ⬜ Pass / ⬜ Partial / ⬜ Fail / ⬜ N/A (conceptual chapter)

---

Principle 11: Tool Diversity and Honest Comparison

Requirement: Multiple AI tools covered with honest comparison, not single-tool lock-in.

• [ ] Multiple AI tools mentioned or covered appropriately for chapter
• [ ] Each tool's strengths and use cases explained objectively (where applicable)
• [ ] Common workflows demonstrated across tools (where applicable)
• [ ] Students encouraged to experiment and find preferences
• [ ] No vendor lock-in language; tools presented as options
• [ ] Fallback strategies when tools unavailable (where applicable)

Evidence Required: Assess tool diversity and objectivity

Verdict: ⬜ Pass / ⬜ Partial / ⬜ Fail / ⬜ N/A (conceptual chapter or early foundation)

---

Section 2: Book Gaps Checklist (Required Coverage by Chapter Type)

For ALL Chapters (Regardless of Type) 🔴

• [ ] Factual Accuracy: All claims verified with inline citations (e.g., [Source, Year])
• [ ] Field Volatility: Rapidly-changing topics include maintenance triggers (e.g., "Review annually")
• [ ] Inclusive Language: No gatekeeping terms; diverse examples; gender-neutral; free alternatives
• [ ] Accessibility: Clear terminology with definitions; multiple explanation styles; content breaks; appropriate pacing
• [ ] Bias & Representation: Diverse perspectives; no stereotypes; inclusive references
• [ ] Technical Accuracy: All claims verified; best practices; no deprecated syntax/tools

Evidence Required: Review entire content for these universal requirements

Verdict: ⬜ Pass / ⬜ Partial / ⬜ Fail

---

For Technical Chapters (Code-Focused)

If chapter contains code, ALL of the following MUST be checked:

• [ ] Code Security: No hardcoded secrets; secure practices demonstrated; disclaimers for generated code
• [ ] Ethical AI Use: AI limitations framed; responsible use cases; potential biases addressed
• [ ] Testing & Quality: Every code example has tests; error cases handled; cross-platform verified
• [ ] Deployment Readiness: Environment setup; dependency management; troubleshooting; fallbacks
• [ ] Scalability Awareness: Real-world constraints mentioned (performance, memory, network)
• [ ] Real-World Context: Realistic scenarios with error handling, not toy problems
• [ ] Engagement: Opening hook; content breaks; realistic complexity progression; 5-7 min sections
• [ ] Practicality: Cross-platform setup; common pitfalls section; explicit environment requirements

Evidence Required: Review all code examples and exercises

Verdict: ⬜ Pass / ⬜ Partial / ⬜ Fail / ⬜ N/A (non-technical chapter)

---

For Conceptual/Narrative Chapters (Non-Code)

If chapter is conceptual/narrative, ALL of the following MUST be checked:

• [ ] Evidence-Based Claims: Assertions backed by data, research, examples; sources cited inline
• [ ] Diverse Perspectives: Multiple viewpoints; objections addressed; not monolithic narrative
• [ ] Real-World Relevance: Specific, concrete examples relevant to readers' context
• [ ] Narrative Flow: Engaging opening hook; natural progression; compelling storytelling
• [ ] Reflection Prompts: Thought-provoking questions; personal relevance to reader
• [ ] Contextual Grounding: Explains why NOW matters; historical parallels; forward-looking implications
• [ ] Professional Polish: No hype; realistic assessment of opportunities/risks; balanced tone
• [ ] Accessibility: Concepts explained with analogies; no jargon gatekeeping; 15-30 min reading time

Evidence Required: Review narrative structure and engagement

Verdict: ⬜ Pass / ⬜ Partial / ⬜ Fail / ⬜ N/A (technical chapter)

---

For Hybrid Chapters (Mixed Content)

If chapter mixes code and narrative:

• [ ] All items from "For ALL Chapters" checked
• [ ] Technical items checked for code-containing sections
• [ ] Narrative items checked for conceptual sections
• [ ] Clear section-type identification present

Verdict: ⬜ Pass / ⬜ Partial / ⬜ Fail / ⬜ N/A (not hybrid)

---

Section 3: Non-Negotiable Rules (ALWAYS/NEVER)

What We ALWAYS Do 🔴

Every item MUST be checked:

• [ ] Use specialized skills and subagents appropriately
• [ ] Include type hints on every function without exception (technical chapters)
• [ ] Test all code before publication (technical chapters)
• [ ] Explain WHY, not just HOW (design decisions and reasoning)
• [ ] Provide working code examples with expected output (technical chapters)
• [ ] Use Python 3.13+ modern syntax (technical chapters)
• [ ] Include "Common Mistakes" section in every chapter
• [ ] Include "AI Exercise" in every chapter (starting Ch 3)
• [ ] Validate against Constitution before publication
• [ ] Assume readers know nothing (no gatekeeping)
• [ ] Show both prompt/request and AI result
• [ ] Encourage verification and iteration with AI tools

Evidence Required: Confirm all applicable "ALWAYS" rules followed

Verdict: ⬜ Pass / ⬜ Partial / ⬜ Fail

---

What We NEVER Do 🔴

NONE of these items can be present:

• [ ] ✅ No vague gatekeeping terms without explanation ("easy", "simple", "obvious")
• [ ] ✅ No untested or broken code
• [ ] ✅ No assumptions about reader knowledge or background
• [ ] ✅ No deprecated Python syntax
• [ ] ✅ No skipped type hints for "simple" functions
• [ ] ✅ No condescension to readers
• [ ] ✅ No hardcoded secrets, tokens, API keys, passwords
• [ ] ✅ No technical claims without verification
• [ ] ✅ No placeholder text or TODOs in content
• [ ] ✅ No contradictions with earlier chapters without explicit explanation
• [ ] ✅ No single AI tool presented as mandatory

Evidence Required: Confirm NONE of the "NEVER" violations present

Verdict: ⬜ Pass / ⬜ Fail (If ANY violation, automatic FAIL)

---

Section 4: Domain Skills Application (The 9 CoLearning Skills)

Skill Application Verification

Verify appropriate application of the 9 domain skills for chapter type:

1. learning-objectives 🔴

• [ ] Clear, measurable learning outcomes stated (Bloom's taxonomy)
• [ ] 3-4 learning objectives appropriate for content scope

2. concept-scaffolding 🔴

• [ ] Complex topics broken into progressive steps
• [ ] 3-4 core concepts per lesson (optimal cognitive load)
• [ ] Clear prerequisite chains

3. code-example-generator

• [ ] High-quality, runnable code with type hints and docstrings (technical chapters)
• [ ] All code tested and verified
• [ ] N/A for conceptual chapters

4. exercise-designer

• [ ] Effective practice exercises aligned with learning objectives
• [ ] Range of difficulty levels
• [ ] Clear prompts and model solutions

5. assessment-builder

• [ ] Meaningful quizzes, Quick Checks, or evaluations embedded
• [ ] Assessments test stated learning objectives
• [ ] Multiple Bloom's taxonomy levels

6. technical-clarity 🔴

• [ ] Clear, accessible explanations free of jargon gatekeeping
• [ ] Technical terms defined on first use
• [ ] Multiple explanation styles used

7. book-scaffolding

• [ ] Content structure appropriate for book position
• [ ] Clear narrative continuity with previous chapters
• [ ] Cognitive load appropriate for scaffolding level

8. ai-augmented-teaching 🔴

• [ ] AI tools effectively integrated as learning partners
• [ ] "Learning WITH AI" philosophy demonstrated
• [ ] AI used for understanding, not just code generation

Evidence Required: Note which skills applied and how

Verdict: ⬜ Pass / ⬜ Partial / ⬜ Fail

---

Section 5: Output Style Compliance

Output Style Template Adherence 🔴

Required Elements (from `.claude/output-styles/lesson.md` or `.claude/output-styles/chapters.md`):

• [ ] YAML frontmatter with required fields (sidebar_position, title, duration/reading time)
• [ ] H1 title matches frontmatter title
• [ ] Proper heading hierarchy (H2 main sections, H3 subsections)
• [ ] No skipped heading levels
• [ ] Opening hook present (2-3 paragraphs before main content)
• [ ] Appropriate word count (2,000-2,500 for technical; 1,200-2,500 for conceptual)
• [ ] Strong closing with reflection or transition
• [ ] Consistent formatting throughout

Evidence Required: Compare content structure to output style template

Verdict: ⬜ Pass / ⬜ Partial / ⬜ Fail

---

Section 6: Structural Requirements

File Organization and Naming 🔴

Verify alignment with `specs/book/directory-structure.md`:

• [ ] File path matches directory structure specification
• [ ] File naming conventions followed
• [ ] Content placed in correct part/chapter folder
• [ ] References to other chapters use correct paths

Evidence Required: Verify file path and organization

Verdict: ⬜ Pass / ⬜ Partial / ⬜ Fail

---

Chapter Integration

Content must integrate with book structure:

• [ ] Builds on previous chapters with clear prerequisites
• [ ] Prepares for upcoming chapters (forward momentum)
• [ ] No contradictions with earlier content
• [ ] Cross-references valid and accurate
• [ ] Consistent with part-level narrative

Evidence Required: Assess integration with broader book context

Verdict: ⬜ Pass / ⬜ Partial / ⬜ Fail

---

Final Compliance Verdict

Critical Items Summary

Count critical failures (🔴 items marked Fail):

• Principle 1 (AI-First Teaching): ⬜ Pass / ⬜ Fail
• Principle 2 (Spec-Kit): ⬜ Pass / ⬜ Fail
• Principle 3 (Python 3.13+): ⬜ Pass / ⬜ Fail
• Principle 4 (Test-First): ⬜ Pass / ⬜ Fail
• Principle 5 (Progressive Complexity): ⬜ Pass / ⬜ Fail
• Principle 6 (Consistent Structure): ⬜ Pass / ⬜ Fail
• Principle 7 (Technical Accuracy): ⬜ Pass / ⬜ Fail
• Principle 8 (Accessibility): ⬜ Pass / ⬜ Fail
• Book Gaps (ALL Chapters): ⬜ Pass / ⬜ Fail
• ALWAYS Rules: ⬜ Pass / ⬜ Fail
• NEVER Rules: ⬜ Pass / ⬜ Fail
• Domain Skills Application: ⬜ Pass / ⬜ Fail
• Output Style Compliance: ⬜ Pass / ⬜ Fail
• Structural Requirements: ⬜ Pass / ⬜ Fail

---

Overall Constitution Compliance Verdict

✅ PASS — All critical items pass; content may proceed to weighted evaluation

⚠️ CONDITIONAL PASS — Minor issues in non-critical items; address before final publication

❌ FAIL — One or more critical items failed; content MUST be revised before proceeding

---

Evidence Summary

Strengths (Constitution Compliance):

• [List specific areas where content excels in constitutional adherence]

Violations or Gaps:

• [List specific constitutional violations with evidence]

Required Actions:

• [Prioritized list of revisions needed for constitutional compliance]

---

Notes for Evaluators

How to Use This Checklist:

1. Read content completely first before evaluating 2. Work through checklist systematically section by section 3. Mark each item with specific evidence from content 4. Flag critical failures immediately (🔴 items) 5. Document specific violations with quotes and line numbers 6. Provide actionable recommendations for each failure 7. Determine overall verdict based on critical items

Remember:

• Constitution compliance is a GATE — fails block all progress
• Be objective and evidence-based in assessments
• Cite specific passages for every verdict
• Prioritize critical items (🔴) first
• Provide clear, actionable guidance for revisions

---

This checklist ensures rigorous constitutional compliance for all CoLearning Python book content.

Content Evaluation Report

Content Evaluated: [File path to content] Evaluation Date: [YYYY-MM-DD] Evaluator: [Name or "AI Evaluation Framework"] Chapter/Lesson: [Chapter X, Lesson Y or Section Title] Content Type: [Technical / Conceptual / Hybrid]

---

Executive Summary

Overall Verdict: ⬜ Pass / ⬜ Conditional Pass / ⬜ Fail

Overall Weighted Score: [X.X]% — [Excellent / Good / Needs Work / Insufficient]

Constitution Compliance: ✅ Pass / ❌ Fail

Ready for Publication: ⬜ Yes / ⬜ Yes, with minor revisions / ⬜ No, requires significant revision

Quick Summary: [1-2 sentence summary of content quality and readiness]

---

Category Scores (Weighted)

Score Calculation:

Final Score = (Technical × 0.30) + (Pedagogical × 0.25) + (Writing × 0.20) +
              (Structure × 0.15) + (AI-First × 0.10)
            = ([score] × 0.30) + ([score] × 0.25) + ([score] × 0.20) +
              ([score] × 0.15) + ([score] × 0.10)
            = [X.X]%

---

Constitution Compliance Status

Overall Verdict: ✅ Pass / ❌ Fail

Critical Items Summary:

• Principle 1 (AI-First Teaching): ✅ Pass / ❌ Fail
• Principle 2 (Spec-Kit): ✅ Pass / ❌ Fail
• Principle 3 (Python 3.13+): ✅ Pass / ❌ Fail / ⬜ N/A
• Principle 4 (Test-First): ✅ Pass / ❌ Fail / ⬜ N/A
• Principle 5 (Progressive Complexity): ✅ Pass / ❌ Fail
• Principle 6 (Consistent Structure): ✅ Pass / ❌ Fail
• Principle 7 (Technical Accuracy): ✅ Pass / ❌ Fail
• Principle 8 (Accessibility): ✅ Pass / ❌ Fail
• Book Gaps Checklist: ✅ Pass / ❌ Fail
• ALWAYS/NEVER Rules: ✅ Pass / ❌ Fail
• Domain Skills Application: ✅ Pass / ❌ Fail
• Output Style Compliance: ✅ Pass / ❌ Fail

Constitutional Violations (if any):

• [List specific violations with evidence]
• [Quote passages that violate principles]
• [Note which principle(s) violated]

Constitutional Strengths:

• [List areas of strong constitutional adherence]
• [Quote exemplary passages]
• [Note which principles exemplified]

See detailed checklist in Appendix A for full constitutional evaluation.

---

Detailed Category Findings

1. Technical Accuracy (30% weight) — [Score]% ([Tier])

Overall Assessment: [2-3 sentence summary of technical accuracy]

Strengths:

• [Specific strength with evidence/quote]
• [Specific strength with evidence/quote]
• [Specific strength with evidence/quote]

Weaknesses:

• [Specific issue with evidence/quote and line number if applicable]
• [Specific issue with evidence/quote and line number if applicable]
• [Specific issue with evidence/quote and line number if applicable]

Evidence:

• Code Quality: [Assessment with examples]
• Explanations: [Assessment with examples]
• Examples: [Assessment with examples]

Recommendations: 1. [Specific, actionable recommendation] 2. [Specific, actionable recommendation] 3. [Specific, actionable recommendation]

---

2. Pedagogical Effectiveness (25% weight) — [Score]% ([Tier])

Overall Assessment: [2-3 sentence summary of pedagogical effectiveness]

Strengths:

• [Specific strength with evidence/quote]
• [Specific strength with evidence/quote]
• [Specific strength with evidence/quote]

Weaknesses:

• [Specific issue with evidence/quote and line number if applicable]
• [Specific issue with evidence/quote and line number if applicable]
• [Specific issue with evidence/quote and line number if applicable]

Evidence:

• Learning Design: [Assessment with examples]
• Concept Scaffolding: [Assessment with examples]
• Exercises & Practice: [Assessment with examples]
• Assessment: [Assessment with examples]

Recommendations: 1. [Specific, actionable recommendation] 2. [Specific, actionable recommendation] 3. [Specific, actionable recommendation]

---

3. Writing Quality (20% weight) — [Score]% ([Tier])

Overall Assessment: [2-3 sentence summary of writing quality]

Strengths:

• [Specific strength with evidence/quote]
• [Specific strength with evidence/quote]
• [Specific strength with evidence/quote]

Weaknesses:

• [Specific issue with evidence/quote and line number if applicable]
• [Specific issue with evidence/quote and line number if applicable]
• [Specific issue with evidence/quote and line number if applicable]

Evidence:

• Readability: [Assessment - Flesch-Kincaid grade level, sentence length, paragraph structure]
• Clarity & Precision: [Assessment with examples]
• Voice & Tone: [Assessment with examples]
• Inclusive Language: [Assessment with examples]

Readability Metrics (if available):

• Flesch-Kincaid Grade Level: [X.X]
• Average Sentence Length: [X] words
• Paragraph Length: [Assessment]

Recommendations: 1. [Specific, actionable recommendation] 2. [Specific, actionable recommendation] 3. [Specific, actionable recommendation]

---

4. Structure & Organization (15% weight) — [Score]% ([Tier])

Overall Assessment: [2-3 sentence summary of structure and organization]

Strengths:

• [Specific strength with evidence/quote]
• [Specific strength with evidence/quote]
• [Specific strength with evidence/quote]

Weaknesses:

• [Specific issue with evidence/quote and line number if applicable]
• [Specific issue with evidence/quote and line number if applicable]
• [Specific issue with evidence/quote and line number if applicable]

Evidence:

• Document Structure: [Assessment with examples]
• Opening Hook: [Assessment with examples]
• Content Flow: [Assessment with examples]
• Word Count: [Actual count] words ([Target range] expected)
• Closing: [Assessment with examples]

Recommendations: 1. [Specific, actionable recommendation] 2. [Specific, actionable recommendation] 3. [Specific, actionable recommendation]

---

5. AI-First Teaching (10% weight) — [Score]% ([Tier])

Overall Assessment: [2-3 sentence summary of AI-first teaching integration]

Strengths:

• [Specific strength with evidence/quote]
• [Specific strength with evidence/quote]
• [Specific strength with evidence/quote]

Weaknesses:

• [Specific issue with evidence/quote and line number if applicable]
• [Specific issue with evidence/quote and line number if applicable]
• [Specific issue with evidence/quote and line number if applicable]

Evidence:

• Learning WITH AI Philosophy: [Assessment with examples]
• AI Exercise Quality: [Assessment with examples]
• Responsible AI Use: [Assessment with examples]
• Integration: [Assessment with examples]

Recommendations: 1. [Specific, actionable recommendation] 2. [Specific, actionable recommendation] 3. [Specific, actionable recommendation]

---

Overall Strengths

What This Content Does Well:

1. [Strength Category]: [Specific strength with evidence]

• Quote: "[Exemplary passage]"
• Why this matters: [Explanation]

2. [Strength Category]: [Specific strength with evidence]

• Quote: "[Exemplary passage]"
• Why this matters: [Explanation]

3. [Strength Category]: [Specific strength with evidence]

• Quote: "[Exemplary passage]"
• Why this matters: [Explanation]

---

Areas for Improvement (Prioritized)

Critical Issues (Blocking Publication)

Issue 1: [Title] — ⚠️ HIGH PRIORITY

• Category Affected: [Category name]
• Current State: [Describe current problematic state with evidence]
• Why This Matters: [Impact on learners/quality]
• Required Action: [Specific revision needed]
• Success Criteria: [How to verify fix]

Issue 2: [Title] — ⚠️ HIGH PRIORITY

• Category Affected: [Category name]
• Current State: [Describe current problematic state with evidence]
• Why This Matters: [Impact on learners/quality]
• Required Action: [Specific revision needed]
• Success Criteria: [How to verify fix]

---

Important Issues (Should Address Before Publication)

Issue 3: [Title] — 🔶 MEDIUM PRIORITY

• Category Affected: [Category name]
• Current State: [Describe current state with evidence]
• Suggested Action: [Specific revision recommendation]
• Expected Improvement: [How this will enhance content]

Issue 4: [Title] — 🔶 MEDIUM PRIORITY

• Category Affected: [Category name]
• Current State: [Describe current state with evidence]
• Suggested Action: [Specific revision recommendation]
• Expected Improvement: [How this will enhance content]

---

Enhancement Opportunities (Nice to Have)

Enhancement 1: [Title] — 🟢 LOW PRIORITY

• Category Affected: [Category name]
• Suggestion: [Optional improvement idea]
• Benefit: [Potential enhancement to content]

Enhancement 2: [Title] — 🟢 LOW PRIORITY

• Category Affected: [Category name]
• Suggestion: [Optional improvement idea]
• Benefit: [Potential enhancement to content]

---

Actionable Next Steps

Immediate Actions (Required for Publication): 1. [ ] [Specific action with target section/line] 2. [ ] [Specific action with target section/line] 3. [ ] [Specific action with target section/line]

Short-Term Actions (Recommended Before Publication): 1. [ ] [Specific action with target section/line] 2. [ ] [Specific action with target section/line] 3. [ ] [Specific action with target section/line]

Long-Term Enhancements (Optional Improvements): 1. [ ] [Specific action with target section/line] 2. [ ] [Specific action with target section/line] 3. [ ] [Specific action with target section/line]

---

Comparison to Quality Thresholds

Minimum Acceptance Threshold (75% Good or Better)

Verdict: ⬜ Meets Minimum / ⬜ Does Not Meet Minimum

---

Recommended for Publication (82% Good or Better)

Verdict: ⬜ Recommended for Publication / ⬜ Not Yet Ready

---

Exemplary Content (90% Excellent)

Verdict: ⬜ Exemplary Content / ⬜ Good But Not Exemplary

---

Final Recommendation

Publication Readiness

Status: ⬜ Ready to Publish / ⬜ Ready with Minor Edits / ⬜ Requires Significant Revision / ⬜ Not Ready

Rationale: [2-3 sentences explaining publication readiness decision based on scores, constitution compliance, and threshold analysis]

Estimated Revision Time:

• Critical issues: [X] hours
• Important issues: [X] hours
• Total estimated revision: [X] hours

Next Steps: 1. [Most critical action] 2. [Second most critical action] 3. [Third action or re-evaluation trigger]

---

Encouragement & Recognition

What the Author/Team Should Be Proud Of: [2-3 sentences recognizing effort, highlighting specific strengths, and encouraging continued work]

Areas of Growth Demonstrated: [1-2 sentences noting improvement or progress if applicable]

Looking Forward: [1-2 sentences about next steps and encouragement for revision or next content unit]

---

Appendix A: Detailed Constitutional Checklist

[Include complete constitutional checklist from references/constitution-checklist.md with all items marked]

---

Appendix B: Detailed Rubric Reference

For detailed tier criteria for each category, refer to: references/rubric-details.md

---

Evaluation Metadata

Framework Version: Content Evaluation Framework v1.0 Constitution Version: CoLearning Python Constitution v2.2.0 Evaluation Date: [YYYY-MM-DD] Evaluation Duration: [X] minutes

Files Consulted:

• Content file: [path]
• Spec file: [path or N/A]
• Plan file: [path or N/A]
• Constitution: .specify/memory/constitution.md
• Rubric details: references/rubric-details.md
• Constitution checklist: references/constitution-checklist.md

---

End of Evaluation Report

---

Notes for Report Generators

When filling out this template:

1. Be specific: Always include quotes, line numbers, and concrete examples 2. Be objective: Base assessments on rubric criteria, not personal preference 3. Be actionable: Recommendations should be concrete and implementable 4. Be encouraging: Recognize strengths and frame criticism constructively 5. Be thorough: Complete all sections fully; don't skip or abbreviate 6. Be consistent: Use the rubric-details.md criteria for all tier assignments 7. Be evidence-based: Every score and verdict must have supporting evidence cited

Report Quality Checklist:

• [ ] All scores have supporting evidence with quotes or examples
• [ ] Constitutional violations (if any) are documented with specific passages
• [ ] Recommendations are actionable with clear success criteria
• [ ] Strengths are recognized and celebrated
• [ ] Priority levels are appropriate for impact on learners
• [ ] Thresholds are accurately calculated and compared
• [ ] Final recommendation is clear and justified
• [ ] Encouragement section is genuine and specific

Detailed Evaluation Rubric

This document provides comprehensive tier criteria for evaluating educational content across 6 weighted categories.

---

Category 1: Technical Accuracy (30% weight)

Excellent (90-100%)

Code Quality:

• All code examples are syntactically correct and execute without errors
• Complete type hints for all functions, parameters, and return values
• Follows project code standards (PEP 8 or equivalent) consistently
• Includes error handling where appropriate
• Code is production-quality, not just "demo code"

Explanations:

• Technical concepts explained with precision and depth
• Explanations are factually accurate with no misleading statements
• Complex concepts broken down clearly with accurate analogies
• Edge cases and gotchas explicitly addressed
• No outdated or deprecated information

Examples:

• 3-5 runnable, self-contained code examples provided
• Examples progressively increase in complexity
• Each example demonstrates specific concept clearly
• Examples include inline comments explaining key lines
• All examples have been tested and verified to work

Evidence:

• Cite specific code blocks that exemplify quality
• Note any technical review or validation performed
• Reference accuracy of technical terminology used

---

Good (75-89%)

Code Quality:

• All code examples execute correctly
• Type hints present for most functions (≥80% coverage)
• Generally follows code standards with minor inconsistencies
• Basic error handling included
• Code is functional and instructive

Explanations:

• Technical concepts explained accurately
• Explanations are clear and understandable
• Key concepts well-covered, minor details may be simplified
• Most important edge cases addressed
• Information is current and relevant

Examples:

• 2-4 working code examples provided
• Examples demonstrate key concepts
• Complexity progression is present
• Most examples include helpful comments
• Examples are verified to work

Evidence:

• Note which technical elements meet standards
• Identify minor gaps or areas for polish
• Confirm core technical accuracy

---

Needs Work (50-74%)

Code Quality:

• Some code examples have minor bugs or syntax errors
• Type hints incomplete (<80% coverage) or missing
• Code standards inconsistently applied
• Limited or no error handling
• Code works but lacks polish

Explanations:

• Technical concepts generally accurate but may have gaps
• Some explanations unclear or imprecise
• Important concepts under-explained
• Edge cases not consistently addressed
• Some outdated or less-than-ideal approaches used

Examples:

• 1-2 code examples provided, or examples too simple
• Examples demonstrate concepts but lack depth
• Limited complexity progression
• Minimal or no code comments
• Not all examples verified to work

Evidence:

• List specific technical errors or omissions
• Note which concepts need deeper explanation
• Identify missing or weak examples

---

Insufficient (<50%)

Code Quality:

• Code examples have significant errors or don't run
• Type hints largely missing (<50% coverage)
• Code standards not followed
• No error handling
• Code appears incomplete or hastily written

Explanations:

• Technical inaccuracies or misleading statements present
• Explanations confusing or incorrect
• Key concepts missing or poorly explained
• Edge cases ignored
• Outdated or deprecated approaches recommended

Examples:

• No examples, or examples don't work
• Examples don't clearly demonstrate concepts
• No progression in complexity
• No code comments
• Examples not tested

Evidence:

• Document critical technical errors
• List missing or incorrect concepts
• Note lack of working examples

---

Category 2: Pedagogical Effectiveness (25% weight)

Excellent (90-100%)

Learning Design:

• Clear, measurable learning objectives stated upfront (3-4 per lesson)
• Learning objectives follow Bloom's taxonomy appropriately
• All stated learning objectives fully addressed in content
• Strong alignment between objectives, content, and assessments
• Appropriate cognitive load for target audience

Concept Scaffolding:

• Exemplary "show-then-explain" pattern throughout
• Concepts introduced in logical, progressive sequence
• 3-4 core concepts per lesson (optimal cognitive load)
• Each concept builds naturally on previous concepts
• Smooth transitions between concepts with clear connections
• Concrete examples precede abstract explanations consistently

Exercises & Practice:

• 3-5 well-designed exercises that reinforce learning objectives
• Exercises range from simple recall to applied problem-solving
• Each exercise includes clear prompts and model solutions
• Exercises encourage "learning WITH AI" (not generating FROM AI)
• Practice problems are authentic and relevant to learners

Assessment:

• 2-4 Quick Check questions embedded in content
• Assessments directly test learning objectives
• Mix of question types (multiple choice, scenario-based, reflection)
• Model answers provided with explanations
• Self-assessment opportunities for learners

Evidence:

• Quote examples showing strong scaffolding
• Note how exercises align with learning objectives
• Cite specific show-then-explain instances

---

Good (75-89%)

Learning Design:

• Learning objectives stated clearly (2-4 per lesson)
• Objectives generally follow Bloom's taxonomy
• Most learning objectives addressed in content
• Good alignment between objectives and content
• Cognitive load appropriate for most learners

Concept Scaffolding:

• "Show-then-explain" pattern generally followed
• Concepts introduced in logical order
• 3-5 concepts per lesson (acceptable range)
• Concepts build on each other with minor gaps
• Most transitions are clear
• Examples usually precede explanations

Exercises & Practice:

• 2-4 exercises that relate to learning objectives
• Exercises include mix of difficulty levels
• Clear prompts provided, most have model solutions
• Some guidance on learning WITH AI present
• Practice problems are relevant

Assessment:

• 1-3 Quick Check questions included
• Assessments test key concepts
• Some variety in question types
• Model answers provided
• Basic self-assessment available

Evidence:

• Note areas where scaffolding is strong
• Identify which exercises are most effective
• Confirm objectives are generally met

---

Needs Work (50-74%)

Learning Design:

• Learning objectives present but may be vague (1-3 per lesson)
• Objectives don't consistently follow Bloom's taxonomy
• Some learning objectives not fully addressed
• Partial alignment between objectives and content
• Cognitive load may be too high or too low

Concept Scaffolding:

• "Show-then-explain" pattern inconsistently applied
• Concept order sometimes unclear or jumps around
• More than 5 concepts or fewer than 2 (suboptimal load)
• Some concepts don't clearly build on previous material
• Transitions occasionally abrupt or missing
• Examples and explanations sometimes in wrong order

Exercises & Practice:

• 1-2 exercises, or exercises too simple/complex
• Limited range of difficulty
• Some exercises lack clear prompts or solutions
• Limited or no AI learning guidance
• Practice problems may feel contrived

Assessment:

• 0-1 Quick Check questions, or questions too vague
• Assessments partially aligned with content
• Limited question variety
• Some model answers missing
• Self-assessment opportunities limited

Evidence:

• List specific scaffolding gaps
• Note which learning objectives are not fully met
• Identify weak or missing exercises

---

Insufficient (<50%)

Learning Design:

• Learning objectives missing, vague, or not measurable
• Objectives don't follow Bloom's taxonomy
• Learning objectives not addressed in content
• Poor alignment between objectives and content
• Cognitive load inappropriate for audience

Concept Scaffolding:

• "Show-then-explain" pattern not followed
• Concepts introduced without clear order or logic
• Cognitive overload (too many concepts) or underwhelming
• Concepts don't build on each other
• Missing or confusing transitions
• Explanations precede examples (backwards pattern)

Exercises & Practice:

• No exercises, or exercises don't relate to content
• No range in difficulty
• Exercises lack prompts or solutions
• No AI learning guidance
• Practice problems are irrelevant or confusing

Assessment:

• No Quick Checks or assessments
• Assessments don't align with learning objectives
• No variety in question types
• Model answers missing
• No self-assessment opportunities

Evidence:

• Document missing or inadequate learning design elements
• List learning objectives not addressed
• Note lack of exercises or assessments

---

Category 3: Writing Quality (20% weight)

Excellent (90-100%)

Readability:

• Flesch-Kincaid Grade Level: 7-9 (optimal for target audience)
• Average sentence length: 12-18 words
• Paragraphs: 3-5 sentences maximum (scannable)
• Complex ideas broken into digestible chunks
• Text flows naturally and engages reader

Clarity & Precision:

• Every technical term defined on first use with clear explanations
• No jargon without accessible analogies
• Active voice used consistently (≥90% of sentences)
• Direct address ("you," "your") used throughout
• No ambiguous pronouns or unclear references

Voice & Tone:

• Engaging, conversational tone that encourages learners
• Professional yet approachable (not condescending)
• Encouraging without being patronizing
• Enthusiasm for topic evident
• Appropriate balance of formality and accessibility

Inclusive Language:

• No gatekeeping language ("simple," "just," "obviously," "merely")
• Assumes no prior knowledge unless explicitly stated as prerequisite
• Examples and scenarios inclusive and diverse
• Avoids assumptions about learner background

Evidence:

• Cite specific passages demonstrating excellent writing
• Note readability metrics if available
• Quote examples of clear, engaging voice

---

Good (75-89%)

Readability:

• Flesch-Kincaid Grade Level: 7-10 (appropriate range)
• Average sentence length: 12-20 words
• Most paragraphs are 3-5 sentences
• Complex ideas generally well-explained
• Text is clear and understandable

Clarity & Precision:

• Most technical terms defined on first use
• Limited unexplained jargon
• Active voice used frequently (≥75% of sentences)
• Direct address generally used
• Few ambiguous references

Voice & Tone:

• Professional and clear tone
• Generally approachable
• Mostly encouraging
• Tone appropriate for audience
• Balance of formal and accessible mostly maintained

Inclusive Language:

• Minimal gatekeeping language
• Generally accessible to target audience
• Examples mostly inclusive
• Few assumptions about background

Evidence:

• Note strong writing passages
• Identify minor areas for improvement
• Confirm overall clarity

---

Needs Work (50-74%)

Readability:

• Flesch-Kincaid Grade Level: 10-12 (somewhat difficult)
• Average sentence length: 20-25 words (long)
• Some paragraphs too long (6+ sentences)
• Complex ideas not always well-explained
• Text occasionally dense or confusing

Clarity & Precision:

• Some technical terms not defined on first use
• Jargon used without sufficient explanation
• Active voice inconsistent (50-75% of sentences)
• Direct address inconsistent
• Some ambiguous references or pronouns

Voice & Tone:

• Tone inconsistent (sometimes too formal, sometimes too casual)
• Occasionally disengaging or dry
• Limited encouragement
• May feel impersonal
• Balance of formality uneven

Inclusive Language:

• Some gatekeeping language present
• Makes some assumptions about prior knowledge
• Limited diversity in examples
• Some exclusionary assumptions

Evidence:

• List specific passages needing revision
• Note readability concerns
• Identify tone inconsistencies

---

Insufficient (<50%)

Readability:

• Flesch-Kincaid Grade Level: 12+ (too difficult)
• Average sentence length: 25+ words (very long)
• Many paragraphs too long (7+ sentences)
• Complex ideas poorly explained or not explained
• Text frequently confusing or unclear

Clarity & Precision:

• Technical terms regularly undefined
• Heavy use of unexplained jargon
• Passive voice predominant (<50% active voice)
• Indirect address or inconsistent perspective
• Frequent ambiguous references

Voice & Tone:

• Tone inappropriate for audience (too formal, condescending, or too casual)
• Disengaging or boring
• Discouraging or intimidating
• Impersonal
• No balance between formal and accessible

Inclusive Language:

• Frequent gatekeeping language throughout
• Assumes significant prior knowledge not stated as prerequisite
• Non-inclusive examples
• Exclusionary assumptions common

Evidence:

• Document serious writing quality issues
• List passages requiring major revision
• Note lack of accessibility

---

Category 4: Structure & Organization (15% weight)

Excellent (90-100%)

Document Structure:

• Clear YAML frontmatter with sidebar_position (if applicable), title, and duration
• H1 title matches frontmatter title exactly
• Proper heading hierarchy (H2 for main sections, H3 for subsections)
• No skipped heading levels (H2 → H4)
• 5-8 well-organized main sections

Opening Hook:

• Compelling opening that engages immediately (2-3 paragraphs)
• Uses pattern recognition, story, question, or surprising statistic
• Captures attention before diving into technical content
• Clearly establishes relevance and importance
• Smooth transition from hook to main content

Content Flow:

• Logical progression from simple to complex
• Smooth transitions between sections with clear signposting
• Each section builds naturally on previous sections
• No jarring jumps in complexity or topic
• Cohesive narrative throughout

Word Count:

• Technical lessons: 2,000-2,500 words (8-12 min reading time)
• Conceptual sections: 1,200-2,500 words (appropriate for content type)
• Proper balance - not too brief, not overwhelming
• Depth appropriate for topic complexity

Closing:

• Strong closing with reflection prompt or transition
• Clear forward bridge to next lesson/section
• Optional summary when value-adding (not formulaic bullet points)
• Leaves reader with clear takeaways and motivation

Evidence:

• Note excellent structural elements
• Quote effective transitions
• Cite engaging opening and closing

---

Good (75-89%)

Document Structure:

• YAML frontmatter present with required fields
• H1 title matches frontmatter
• Heading hierarchy generally correct
• Few if any skipped heading levels
• 4-7 well-organized sections

Opening Hook:

• Clear opening that establishes context (2-4 paragraphs)
• Uses one or more engagement techniques
• Captures attention reasonably well
• Establishes relevance
• Adequate transition to main content

Content Flow:

• Generally logical progression
• Most transitions are clear
• Sections mostly build on each other
• Few awkward jumps
• Mostly cohesive

Word Count:

• Technical lessons: 1,800-2,700 words (reasonable range)
• Conceptual sections: 1,000-2,700 words (acceptable)
• Generally appropriate length
• Depth mostly appropriate

Closing:

• Adequate closing with some form of transition or reflection
• Some forward momentum to next content
• Summary present if appropriate
• Leaves reader with some clear takeaways

Evidence:

• Note structural strengths
• Identify minor flow issues
• Confirm appropriate length

---

Needs Work (50-74%)

Document Structure:

• YAML frontmatter incomplete or missing fields
• H1 title may not match frontmatter exactly
• Heading hierarchy has some errors
• Some skipped heading levels
• Fewer than 4 or more than 8 sections (too few/many)

Opening Hook:

• Opening present but not engaging (1-2 paragraphs or very long)
• Limited use of engagement techniques
• Doesn't strongly establish relevance
• Abrupt transition to main content
• Hook feels generic or weak

Content Flow:

• Progression sometimes unclear
• Some transitions missing or abrupt
• Some sections don't clearly build on previous material
• Several awkward jumps in complexity or topic
• Cohesion inconsistent

Word Count:

• Technical lessons: 1,500-1,800 or 2,700-3,000 words (too short/long)
• Conceptual sections: 800-1,000 or 2,700-3,000 words (too short/long)
• Length somewhat inappropriate for content
• Depth uneven

Closing:

• Closing present but weak or abrupt
• Limited forward momentum
• Summary formulaic or missing when needed
• Takeaways unclear

Evidence:

• List structural issues
• Note missing or weak transitions
• Identify length concerns

---

Insufficient (<50%)

Document Structure:

• YAML frontmatter missing or severely incomplete
• H1 title missing or doesn't match frontmatter
• Heading hierarchy incorrect throughout
• Multiple skipped heading levels
• Poorly organized sections (too few, too many, or unclear)

Opening Hook:

• No clear opening hook (jumps straight to content)
• No engagement techniques used
• Doesn't establish relevance or importance
• No transition - starts mid-thought
• Opening is confusing or missing

Content Flow:

• No clear logical progression
• Transitions frequently missing
• Sections feel disconnected
• Frequent jarring jumps
• Lacks cohesion

Word Count:

• Technical lessons: <1,500 or >3,000 words (too short/too long)
• Conceptual sections: <800 or >3,000 words (too short/too long)
• Length highly inappropriate
• Depth severely inadequate or overwhelming

Closing:

• No real closing or conclusion
• No forward momentum
• Missing summary when critical
• No clear takeaways

Evidence:

• Document major structural problems
• List missing organizational elements
• Note severe length issues

---

Category 5: AI-First Teaching (10% weight)

Excellent (90-100%)

Learning WITH AI Philosophy:

• Consistently emphasizes "learning WITH AI" not "generating FROM AI"
• Clear distinction: AI as learning mentor vs. code generator
• Students encouraged to write own code with AI as tutor
• AI used for understanding, explanation, exploration (not just answers)
• Philosophy explicitly taught and modeled throughout

AI Exercise Quality:

• 2-3 exercises specifically designed for AI-augmented learning
• Each exercise includes "Learning with AI" guidance section
• Example AI prompts provided that deepen understanding:
• ✅ "Explain why this works"
• ✅ "What does this error mean?"
• ✅ "Compare these two approaches"
• ❌ "Write this function for me"
• ❌ "Fix my code"
• Exercises explicitly warn against using AI as code generator
• Students directed to: attempt → ask AI for clarity → understand → try again

Responsible AI Use:

• Modeling effective AI collaboration for learning throughout
• Shows how to ask good questions that lead to understanding
• Demonstrates iterative learning cycle: try → fail → learn with AI → understand
• Addresses AI limitations and when to rely on own thinking
• Teaches critical evaluation of AI suggestions

Integration:

• AI-augmented learning naturally integrated into content (not bolted on)
• AI learning prompts embedded throughout (not just in exercises)
• Clear pedagogical rationale for AI use in each instance
• AI tools mentioned are appropriate and accessible to learners

Evidence:

• Quote examples showing strong AI-first pedagogy
• Note exercises that exemplify learning WITH AI
• Cite integration of AI throughout content

---

Good (75-89%)

Learning WITH AI Philosophy:

• Generally emphasizes "learning WITH AI"
• Distinction between AI as mentor vs. generator present
• Students mostly encouraged to write own code
• AI used for understanding and explanation
• Philosophy present but could be stronger

AI Exercise Quality:

• 1-2 exercises designed for AI-augmented learning
• Some "Learning with AI" guidance provided
• Some example AI prompts that deepen understanding
• Some warning against code generation
• Basic attempt → learn → try again cycle present

Responsible AI Use:

• Shows some effective AI collaboration techniques
• Some guidance on asking good questions
• Partial demonstration of iterative learning
• Some mention of AI limitations
• Some guidance on evaluating AI suggestions

Integration:

• AI-augmented learning mostly integrated
• Some AI learning prompts embedded
• Generally clear rationale for AI use
• AI tools mentioned are appropriate

Evidence:

• Note AI-first teaching elements present
• Identify areas for strengthening
• Confirm basic philosophy is clear

---

Needs Work (50-74%)

Learning WITH AI Philosophy:

• Philosophy mentioned but not consistently emphasized
• Distinction between mentor and generator unclear
• Mixed messages about writing own code vs. using AI
• AI sometimes positioned as answer-provider rather than learning tool
• Philosophy present but weakly articulated

AI Exercise Quality:

• 0-1 exercises for AI-augmented learning
• Limited "Learning with AI" guidance
• Few or weak example AI prompts
• Insufficient warning against code generation, or warnings feel pro forma
• Iterative learning cycle partially present

Responsible AI Use:

• Limited modeling of effective collaboration
• Minimal guidance on asking questions
• Limited demonstration of learning cycle
• Little mention of AI limitations
• Weak guidance on evaluating suggestions

Integration:

• AI learning feels somewhat bolted on
• Few embedded AI prompts
• Rationale for AI use sometimes unclear
• AI tools mentioned may not all be accessible

Evidence:

• List gaps in AI-first teaching
• Note weak or missing AI exercises
• Identify unclear philosophy statements

---

Insufficient (<50%)

Learning WITH AI Philosophy:

• Philosophy not articulated or missing
• No clear distinction between mentor and generator
• Encourages using AI to generate code without learning
• AI positioned as shortcut rather than learning tool
• Philosophy absent or contradictory to learning goals

AI Exercise Quality:

• No exercises for AI-augmented learning
• No "Learning with AI" guidance
• No example AI prompts, or prompts encourage code generation
• No warnings about appropriate AI use
• No iterative learning cycle

Responsible AI Use:

• No modeling of effective collaboration
• No guidance on asking questions
• No demonstration of learning cycle
• No mention of AI limitations
• No guidance on evaluating suggestions

Integration:

• AI learning not integrated or missing entirely
• No embedded AI prompts
• No rationale for AI use
• AI tools not mentioned or inappropriate

Evidence:

• Document missing AI-first teaching elements
• Note lack of appropriate AI exercises
• List ways content contradicts learning WITH AI philosophy

---

Category 6: Constitution Compliance (Pass/Fail - GATE)

This category is evaluated as Pass or Fail only. If Fail, content cannot proceed regardless of weighted scores.

Constitution Compliance Checklist

Use the detailed checklist in references/constitution-checklist.md to evaluate compliance with all constitutional principles and requirements.

Key Areas: 1. Project Vision Alignment - Content serves the stated project purpose and philosophy 2. Core Principles Adherence - All 11 core principles respected (if applicable to project) 3. Domain Skills Application - Required domain skills applied as specified 4. Code Standards - Code meets quality standards defined in constitution 5. Quality Gates - Content meets minimum quality thresholds 6. Non-Negotiable Rules - All "ALWAYS DO" and "NEVER DO" rules followed 7. Output Style Compliance - Content follows required output style template 8. Accessibility Requirements - Accessibility standards met (alt text, readability, etc.) 9. Technical Requirements - Technical specifications and dependencies satisfied 10. Structural Requirements - File organization and naming conventions followed

Pass Criteria

Content receives PASS when:

• ALL critical non-negotiable rules are followed
• No violations of "NEVER DO" rules
• ALL "ALWAYS DO" rules are satisfied
• Required domain skills appropriately applied
• Minimum quality thresholds met
• Output style template followed
• All applicable constitutional principles respected

Fail Criteria

Content receives FAIL if ANY of the following:

• Any critical non-negotiable rule violated
• Any "NEVER DO" rule violated
• Required "ALWAYS DO" rules not followed
• Required domain skills not applied or misapplied
• Minimum quality thresholds not met
• Output style template not followed
• Key constitutional principles violated

If FAIL: Content must be revised to address constitutional violations before proceeding with weighted evaluation.

---

Using This Rubric

Step-by-Step Process

1. Read the content being evaluated completely 2. Review context (spec, plan, learning objectives, constitution) 3. Evaluate Constitution Compliance first using checklist

• If FAIL: Stop and report violations immediately
• If PASS: Continue to weighted categories

4. Evaluate each weighted category using tier criteria above 5. Record specific evidence for each category score (quotes, line numbers, examples) 6. Calculate weighted score using formula 7. Generate report using evaluation template 8. Provide actionable recommendations prioritized by impact

Tips for Consistent Evaluation

• Be objective: Base scores on rubric criteria, not subjective preference
• Use evidence: Always cite specific examples from content
• Be fair: Recognize strengths as well as weaknesses
• Be constructive: Frame criticism as opportunities for improvement
• Be clear: Provide concrete, actionable recommendations
• Be consistent: Apply same standards across all content units

---

This rubric ensures rigorous, evidence-based, consistent quality evaluation for all educational content.