Planning

Principles

• One question at a time. Each question should resolve a branching decision in the plan. Avoid generic or out-of-domain questions.
• Surface constraints early. If a user decision would constrain downstream options, flag it before the plan is finalized.
• Keep plans short. Only include tasks that are necessary for the user's stated goal.
• Don't ask what you already know. Check conversation history and project files before asking the user.

---

Phase 1: Brainstorming

Goal: Understand what the user wants to accomplish and identify which skills belong in the plan.

Read references/input-output-contracts.md, references/model-customization-plan.md, and references/evaluate-first-plan.md to:

• Identify which skills could be relevant to the user's stated goal.
• Check whether the user has the necessary input artifacts for each skill. If not, find the skills that generate those inputs and add them first.
• Order skills to allow a smooth transition from one to the next and avoid dead ends.
• Check if a recommended workflow matches the user's needs. If not, assess what modifications are needed and verify they are possible against the contracts table.
• Decide which skills in a matching workflow can be skipped.
• Surface limitations early — if a user decision (model choice, region, evaluation method) would constrain downstream options, mention it proactively, get user feedback, and adapt the plan accordingly.

During brainstorming:

• Workflow choice gate: Before generating any plan, determine whether the user wants the evaluate-first workflow or the direct fine-tuning workflow. If the user has explicitly chosen (e.g., "evaluate first", "skip evaluation", "already evaluated the base model"), proceed with their choice. Otherwise, present both options with brief pros/cons and ask the user to choose. Saying "fine-tune" or naming a technique alone is NOT an explicit choice to skip evaluation — the user may not know evaluate-first is an option. Do NOT present a plan until the user has chosen a path. After they choose, read ONLY the corresponding reference plan.
• Use the Restrictions column of the contracts table to flag constraints as soon as the relevant decision is made. Examples (non-comprehensive list, check contracts table for the full picture):
• User picks a Nova model → alert that deployment regions are limited.
• User picks a region → alert if it conflicts with model availability.
• If a restriction applies, check whether it requires changes to other steps in the plan.
• Do NOT ask the user about base model selection or preferences. Model selection is handled exclusively by the model-selection skill.
• Move to Phase 2 as soon as you can determine which skills and tools the plan needs.

---

Phase 2: Plan Generation

Goal: Propose a structured plan for the user to review.

Generate a plan as a numbered list of tasks. Each task has:

• A short name
• A one-sentence description of what happens
• Which skill handles it (if applicable)

Format:

Based on what you've described, here's what I propose:

1. ⬜ **[Task Name]** — [What happens]. *(Skill: [skill-name])*
2. ⬜ **[Task Name]** — [What happens]. *(Skill: [skill-name])*
3. ⬜ **[Task Name]** — [What happens]. *(Skill: [skill-name])*

Does this plan look right, or would you like to change anything?

Rules for plan generation:

• Infer ordering from the Prerequisites column in the contracts table — a skill cannot appear before its prerequisites. If unsure, consult references/skill-routing-constraints.md.
• Only offer capabilities covered by an available skill. If the user needs something no skill supports, say so.
• Tailor the plan to the user's actual intent. Not every plan needs every skill.
• If the user already has input artifacts (e.g., a trained model), skip the steps that produce them.

When the user approves the plan, write it to PLAN.md and save it under the project directory structure defined by the directory-management skill.

# Plan

1. ⬜ **[Task Name]** — [Description]. _(Skill: [skill-name])_
2. ⬜ **[Task Name]** — [Description]. _(Skill: [skill-name])_
3. ⬜ **[Task Name]** — [Description]. _(Skill: [skill-name])_

Status indicators:

• ⬜ Not Started
• 🔄 In Progress
• ✅ Completed

Update PLAN.md whenever a task's status changes.

---

Phase 3: Plan Iteration

Goal: Refine the plan until the user approves it.

• If the user suggests changes, regenerate the plan incorporating their feedback.
• If the user approves, begin execution by handing off to the first task's skill.

---

Execution

Once the plan is approved:

1. Before starting a task, update its status in PLAN.md to 🔄 (In Progress). 2. If the task maps to a skill, load that skill's full SKILL.md before doing any work. Do not attempt the task from general knowledge — always defer to the skill's instructions. 3. Execute the task by following the loaded skill's workflow. 4. When the task completes:

• Update its status in PLAN.md to ✅ (Completed). If the task generated output files (scripts, notebooks, manifests), record the file paths under the completed task:

     - [x] Fine-tune model
       - Output: `scripts/01_sft_finetuning.py`
       - Output: `manifests/sft-llama-20260515.json`

• Briefly confirm completion and move to the next task.

5. If the user interrupts with a new request mid-execution:

• Completed tasks are immutable — do NOT modify them.
• Regenerate the remaining tasks to incorporate the user's new input.
• Present the updated remainder for approval before continuing.

---

Plan Completion

When all tasks in the plan are done: Present to the user:

"We've completed everything in the plan. What would you like to do next?"

This re-enters Phase 1 (Brainstorming) for a new goal. There is no terminal state — the conversation continues as long as the user wants.

---

References

Load the reference plan that matches the customer's intent, then adjust based on their needs.

• references/evaluate-first-plan.md — The evaluate-first workflow: evaluate a base model before deciding whether to fine-tune.
• references/model-customization-plan.md — The direct fine-tuning plan. Use when the user has explicitly committed to fine-tuning.
• references/input-output-contracts.md - A table showing all skills, required inputs, produced outputs, prerequisites, and constraints.
• references/skill-routing-constraints.md — Optional supplemental resource about Mandatory inclusion rules, ordering constraints, and skill boundary rules.

Evaluate-First Plan

Recommended for: Users who want to check if finetuning is necessary to solving their problem, users who want to check which base model is best for their use case, users who don't want to commit to finetuning yet, and users who are open to suggestions

1. Define Use Case — Capture the business problem, users, and success criteria. _(Skill: use-case-specification)_ 2. Select Base Model — Choose a base model from SageMaker Hub. _(Skill: model-selection)_ 3. Verify Environment — Check SDK version, region, and execution role are configured. _(Skill: sdk-getting-started)_ 4. Evaluate Dataset — Validate the evaluation dataset (query/response format). _(Skill: dataset-evaluation)_ 5. Transform Dataset — Convert to SageMaker evaluation format if needed. _(Skill: dataset-transformation)_ 6. Evaluate Model — Run the base model against the evaluation dataset and present results against success criteria. _(Skill: model-evaluation)_ 7. Decision Gate — Present evaluation results. User decides whether to fine-tune or stop.

If the user decides to fine-tune after the decision gate, extend the plan:

<!-- markdownlint-disable MD029 -->

8. Select Finetuning Technique — Choose the appropriate finetuning technique. _(Skill: finetuning-technique)_ 9. Evaluate Training Dataset — Validate training data format. _(Skill: dataset-evaluation)_ 10. Transform Training Dataset — Convert to training format. _(Skill: dataset-transformation)_ 11. Fine-Tune Model — Train the model. _(Skill: finetuning)_ 12. Evaluate Finetuned Model — Compare against base model results. _(Skill: model-evaluation)_ 13. Deploy Model — Create an endpoint. _(Skill: model-deployment)_

At the decision gate, present data objectively against the user's success criteria. Do not recommend — let the user decide.

Input-Output Contracts

End-to-End Model Customization Plan

Recommended for: users who are certain they want to finetune a model, and users who clearly communicate that they want to finetune a model

1. Define Use Case — Capture the business problem, users, and success criteria. _(Skill: use-case-specification)_ 2. Select Base Model — Choose a base model from SageMaker Hub based on benchmarks and use case fit. _(Skill: model-selection)_ 3. Verify Environment — Check SDK version, region, and execution role are configured. _(Skill: sdk-getting-started)_ 4. Select Finetuning Technique — Choose a fine-tuning technique and validate compatibility with the selected model. _(Skill: finetuning-technique)_ 5. Evaluate Dataset — Assess data quality, completeness, and format. _(Skill: dataset-evaluation)_ 6. Transform Dataset — Convert the dataset to the required format for the selected fine-tuning technique and base model. _(Skill: dataset-transformation)_ 7. Fine-Tune Model — Train a custom model using SageMaker. _(Skill: finetuning)_ 8. Evaluate Model — Measure model performance against success criteria. _(Skill: model-evaluation)_ 9. Deploy Model — Create an endpoint for inference. _(Skill: model-deployment)_

Note: This skills package does not support data generation. Do not suggest, offer, or imply that you have the ability to generate data. If the user asks about this, make it clear that the skills do not support this ability.

Skill Routing Constraints

Plan Completeness

• Generate the complete plan upfront. The plan presented to the user

must include all steps needed to reach their goal. Do not generate a partial plan with the intent to add steps later.

• Each step must be executed by its designated skill. Do not perform

a skill's work ad-hoc or inline within another skill.

Mandatory Inclusion

• use-case-specification: Include by default in every model

customization plan unless the user explicitly declines or has an existing spec.

Evaluate-First Path

When the user chooses to evaluate the base model before fine-tuning:

• model-selection MUST run before sdk-getting-started and

dataset-evaluation.

• sdk-getting-started MUST run after model-selection and before

dataset-evaluation.

• model-selection runs before model-evaluation.
• finetuning-technique is NOT required. It only enters the plan if

the user decides to fine-tune after the decision gate.

Direct Fine-Tuning Path

When the user chooses to go straight to fine-tuning:

• model-selection MUST run before sdk-getting-started.
• sdk-getting-started MUST run after model-selection and before

dataset-evaluation.

• model-selection MUST run before finetuning-technique.
• finetuning-technique MUST run before dataset-evaluation (for

training data) and finetuning. The technique must be known before training data can be validated or training can begin.

• dataset-evaluation should run after finetuning-technique and before

finetuning, to catch format issues before training.

Ordering Constraints

• model-selection MUST run before sdk-getting-started.
• sdk-getting-started MUST run after model-selection and before the

first task that runs scripts or makes AWS API calls requiring an execution role (e.g., dataset-evaluation, finetuning, model-evaluation, model-deployment). It is not needed before conversational-only tasks like use-case-specification or model-selection.

Skill Boundaries

• All dataset format changes MUST go through dataset-transformation.

Do not write inline transformation code in other skills' notebooks.

• All model selection MUST go through model-selection.

Do not resolve model IDs ad-hoc.

• All technique selection MUST go through finetuning-technique.

Do not select techniques ad-hoc.