Practice Paper

Practice Paper Skill

Generates short mock open-book exam papers from the course binder, lets the user attempt them in .docx, and marks attempts against the course's own rubric — all in a single session per paper.

The skill reads everything from the course's course.yaml and templates/. It works with any course that follows the schema — modules, rubric criteria/weights/bands, paper config, and question-style template are all course-specific.

Step 0: Resolve the course

If a course path is given as the first arg, use it. Otherwise check the current working directory: if it has a course.yaml, that's the course. If neither — ask the user "which course?" and stop.

# load course config (just to confirm it's valid; you'll re-read fields below)
python -c "from pipeline.course import load_course; c = load_course('<course-path>'); print(c.name, c.module_codes)"

Throughout this skill, <course> means the resolved course root. Read the following from <course>/course.yaml:

• course.name — for the paper title block
• modules — list of {code, display_name}; the user picks one with the trigger arg
• rubric.criteria — list of {name, weight, descriptor}; weights sum to rubric.scale
• rubric.bands (optional) — list of {name, min}
• rubric.scale (default 100)
• paper.short_form (default 4)
• paper.essay (default 1)
• paper.target_minutes (default 60)
• paper.no_repeat_within (default 2)

Step 1: Detect mode

If multiple modules have pending papers and the user says "mark it" without specifying, list them and ask which.

---

GENERATE MODE

Step 2: Pick lectures

1. Read <course>/papers/_history.md (create empty if missing). Identify lectures used as primary source in the most recent paper.no_repeat_within papers for the chosen module. 2. List candidate lectures: <course>/vault/lectures/{module}_L*.md. 3. Pick paper.short_form + paper.essay lectures total:

• paper.essay of them get the essay question(s); essay lectures may

pull from 1 adjacent lecture

• paper.short_form get one short-form question each

4. Prefer fresh lectures: avoid any used in the recent paper.no_repeat_within papers. Allow rare repeats only if >75% of lectures have already been used recently.

Step 3: Read source material

• Read each chosen lecture vault .md file in full (<course>/vault/lectures/{module}_L{NN}_*.md)
• Read the question-style guide: <course>/templates/question_style.md

if it exists, else pipeline/templates/question_style.md from the installed package

• Reference the rubric: course.yaml rubric.criteria (each criterion's

name + weight + descriptor)

Step 4: Generate questions

Match the question patterns in the question-style guide. Mix freely; pick the pattern that fits each lecture's content best. Never write generic "Discuss X" or "Describe Y".

Per-question targets (from the style guide):

Total paper time: ~paper.target_minutes.

Step 5: Write outline answers

For each question, write outline answer per the style guide's "Outline Answer Format". The rubric guidance block in each outline must reflect the actual rubric.criteria from course.yaml — name, weight, what top-tier looks like for that criterion.

Reference scheme: sections by heading (§Section name), NEVER absolute page numbers — pages shift on rebuild.

Step 6: Render the paper docx

Write the question list as JSON:

{
  "title": "<course.name> — <module-code> Practice Paper N",
  "module": "<module-code>",
  "date": "YYYY-MM-DD",
  "questions": [
    {"number": "Q1", "type": "short", "primary_lecture": "<MODULE> L05", "text": "..."},
    {"number": "Q2", "type": "short", "primary_lecture": "<MODULE> L09", "text": "..."},
    {"number": "Q3", "type": "essay", "primary_lecture": "<MODULE> L03", "text": "..."}
  ]
}

Save to /tmp/paper-content.json, then render with the repo's renderer:

python <repo-root>/skills/practice-paper/render_paper.py \
  --json /tmp/paper-content.json \
  --out <course>/papers/YYYY-MM-DD-<MODULE>-paper-N.docx

The renderer creates: title page → instructions → each question with "Your answer:" prompt and ~10 (short) or ~25 (essay) blank paragraphs of typing space.

Step 7: Write key.md sidecar

Write <course>/papers/YYYY-MM-DD-<MODULE>-paper-N-key.md with the outline answers from Step 5 (one section per question).

Step 8: Update history

Append to <course>/papers/_history.md. Create with header if new:

# Practice paper history

| Date | File | Module | Primary lectures | Status | Mark |
|------|------|--------|------------------|--------|------|
| YYYY-MM-DD | <filename>.docx | <MODULE> | L03 (essay), L05, L09 | pending | — |

Step 9: Hand off to user

Tell the user:

• Paper saved at <course>/papers/<filename>.docx
• Display the questions inline in chat (so they can read them straight away)
• Instructions: open the docx, type answers under each "Your answer:" prompt, save in place
• When done, say "mark it" — same conversation, no /clear needed

---

MARK MODE

Step 1: Find the paper

If a path was passed: use it.

Otherwise: read <course>/papers/_history.md, find the most recent row with Status = pending. If multiple modules pending and "mark it" is ambiguous, list them and ask the user.

You'll need three things:

• <course>/papers/<paper>.docx — the user's filled attempt
• <course>/papers/<paper>-key.md — outline answers + rubric guidance
• The history row (to update at the end)

Step 2: Extract the attempt

python <repo-root>/skills/practice-paper/extract_attempt.py \
  --docx <course>/papers/<paper>.docx \
  --out /tmp/attempt.json

Produces JSON: [{number, type, primary_lecture, question_text, answer}, ...].

If extraction returns empty answers but the docx is non-trivial in size, fall back to reading the docx contents directly via unzip -p ... word/document.xml and parsing.

If a question has no attempt: mark 0/<scale> with note "no attempt"; do not penalise other questions.

Step 3: Read the key + rubric prompt

• Read <paper>-key.md to get outline answers + rubric guidance
• Read the marking-prompt scaffold:

<course>/templates/rubric_prompt.md if it exists, else pipeline/templates/rubric_prompt.md from the installed package

• Splice in the actual rubric.criteria from course.yaml

Step 4: Mark each question

Score each rubric criterion separately (each on the 0–rubric.scale range). Per-question total:

weighted_total = round(
  (score_1 * weight_1 + score_2 * weight_2 + ...) / sum_of_weights
)

If rubric.bands is defined, append band labels via the band whose min the mark meets or exceeds.

Per-question output format (one block per criterion, dynamically generated from rubric.criteria):

### Q[N] — [type] — Mark: [weighted_total]/[scale] [— band, if defined]

**[Criterion 1 name] ([criterion 1 weight]%):** [score]/[scale] [— band]
- Good: [what they got right against this criterion's descriptor]
- Gap: [what was missing or weak]
- To push higher: [specific step that would lift the next attempt]

[repeat for each criterion in course.yaml rubric.criteria]

[1-2 sentence overall comment]

Step 5: Overall summary

## Overall — N% [— band]

**Strongest:** Q[X] — [why]
**Weakest:** Q[Y] — [why]
**Top focus for next paper:** [1-2 specific things]

Where overall is the unweighted mean of per-question weighted totals (each question counts equally).

Step 6: Update history

Edit _history.md: change the row's Status to marked and Mark to the overall %.

---

EDGE CASES

PARAMETER CONVENTIONS

• N in filename: paper number for that module-date combo. If

YYYY-MM-DD-<MODULE>-paper-1.docx already exists, use paper-2. Don't overwrite.

• Module names: use the canonical code from course.yaml

modules[].code. Don't accept variants — the user's input must match one of the configured codes.

• Date format: YYYY-MM-DD. Use today's date.

CALIBRATION CAVEAT

Real examiners often mark relative to the cohort distribution. This skill marks against the rubric in absolute terms — treat marks as relative-to-self over time, not absolute predictions of real-exam outcomes.