Competency Builder

Competency Builder Skill

Build and operate competency frameworks that produce capability—not just completion. Diagnose where competency development is stuck and guide the next step.

Core Principle

Competencies are observable capabilities, not knowledge states. If you can't watch someone demonstrate it, it's not a competency.

---

Diagnostic States

CF0: No Framework

Symptoms: Have training content but no competency structure. People complete training but can't apply it. Same questions keep getting asked.

Test:

• What decisions do people need to make with this knowledge?
• What mistakes indicate someone lacks competency?
• Can you describe what a competent person can DO?

Intervention: Start with failure modes. List mistakes you've seen, questions that shouldn't need asking, things that take too long. Each failure mode suggests a competency that would prevent it.

---

CF1: Content-First Trap

Symptoms: Started by listing all the information people need to know. Training is comprehensive but competence is low. "We trained on that" but mistakes continue.

Test:

• Can you describe what someone with this competency can DO?
• What would you watch them do to verify competency?
• Does each content piece connect to a specific competency?

Intervention: Reframe each content chunk as "what decision/action does this enable?" Kill orphan content that doesn't support a competency. Work backward from actions to required knowledge.

---

CF2: Vague Competencies

Symptoms: Competencies are knowledge states ("understands X") not capabilities ("can evaluate X against Y"). Can't tell if someone has the competency or not.

Test:

• Could two people disagree about whether someone has this competency?
• Can you observe it?
• Does it start with "Can" + action verb?

Intervention: Rewrite each competency as observable behavior. Transform:

• "Understands data policies" → "Can classify data according to policy categories"
• "Knows the approval process" → "Can determine required approval level for a given case"
• "Familiar with the tool" → "Can configure the tool to accomplish [specific task]"

---

CF3: No Scenarios

Symptoms: Competencies defined but no way to test them. Assessment is knowledge recall (quizzes, multiple choice). People pass but fail in real situations.

Test:

• What realistic situation requires this competency?
• Does assessment require judgment, or can it be answered by searching documentation?
• What would a weak vs. strong response look like?

Intervention: For each core competency, create a scenario that: 1. Presents a realistic situation 2. Includes incomplete information 3. Requires judgment (not just recall) 4. Has better and worse responses (not binary right/wrong)

Create variants: interview (generic), assessment (org-specific), ongoing (real situations).

---

CF4: Simple Scenarios

Symptoms: Scenarios exist but have artificial clarity. All information needed is provided. There's an obvious "right answer." People pass but fail in messy real situations.

Test:

• Do scenarios match the ambiguity of real situations?
• Can scenarios be answered by looking up documentation?
• Do scenarios require weighing trade-offs?

Intervention: Add ambiguity. Remove artificial clarity. Include information that might be relevant but isn't, and omit information that would make the answer obvious. Test with real people—if everyone gets the same answer immediately, it's too simple.

---

CF5: Single Audience

Symptoms: Everyone gets the same training. Specialists are bored by basics. Generalists are overwhelmed by detail. One-size-fits-none.

Test:

• Who are your actual audiences?
• What depth does each audience need?
• Does a general employee need the same competencies as a specialist?

Intervention: Define audience layers (typically General / Practitioner / Specialist). Map competencies to audiences. Layer content by depth:

• L1: Rules without extensive justification (what to do)
• L2: Principles behind rules (how to handle edge cases)
• L3: Full technical detail (how to verify, audit, configure)

---

CF6: No Progression

Symptoms: Competencies exist but no clear order. Prerequisites unclear. No skip logic. Everyone follows the same path regardless of prior knowledge.

Test:

• Which competencies require others as foundation?
• What's the minimum viable path for each role?
• Can someone with prior knowledge skip parts?

Intervention: Map dependencies. Build progression tree:

Foundation (everyone)
├── Prerequisite competencies
├─► Intermediate (builds on foundation)
└─► Role-specific branches (parallel tracks)

Define skip logic: what evidence allows skipping which modules?

---

CF7: No Verification Stakes

Symptoms: Assessment exists but doesn't gate anything. People skip or game it. No consequence for demonstrating vs. not demonstrating competency.

Test:

• What decision does verification inform?
• What happens if someone fails assessment?
• Is there a real consequence for "not demonstrated"?

Intervention: Connect each verification to a decision:

• Hiring: Does candidate advance?
• Onboarding: Ready to work independently?
• Access: Qualified for elevated permissions?
• Promotion: Has developed required competency?

If verification doesn't connect to a decision, question whether it's worth doing.

---

CF8: No Feedback Loop

Symptoms: Framework built once and never updated. Questions keep arising that weren't anticipated. No visibility into what's not working.

Test:

• How do you know what's not working?
• What mechanism surfaces gaps in training, framework, or process?
• When did this framework last change based on feedback?

Intervention: Implement feedback loop: 1. Agent/support system logs questions with context 2. Tag questions by competency/content area (or "unmapped") 3. Regular review for patterns 4. Route fixes to owners (training team, policy owners, tooling) 5. Track when patterns lead to changes

---

CF9: Static Framework

Symptoms: Framework was built months/years ago. Reality has changed but framework hasn't. Questions reveal framework doesn't match current state.

Test:

• When was this last reviewed?
• What triggers an update?
• Who owns maintenance?

Intervention: Define:

• Review triggers (policy changes, incidents, new tools, feedback patterns)
• Ownership (who updates what)
• Version tracking (people trained on V1 vs. V2)
• Cadence (minimum review frequency even without triggers)

---

CF10: Framework Operational

Symptoms: Competencies observable, scenarios tested, progression mapped, verification meaningful, feedback loop active, maintenance owned.

Indicators:

• Can answer all previous state questions affirmatively
• New hires reach competence faster
• Repeat questions decrease
• Framework has evolved based on feedback data
• Skip logic personalizes paths based on demonstrated competency

---

Diagnostic Process

When someone presents a competency development need:

1. Identify current state — What exists? Training content? Competency list? Scenarios? Assessment? 2. Apply state diagnosis — Match symptoms to states above 3. Ask clarifying questions — What decisions do people make? What failures have you seen? 4. Explain the diagnosis — Name the state and what's missing 5. Recommend next step — Point to specific template or intervention 6. Validate progress — Check if intervention resolved the state

---

Key Questions by Phase

For Competency Identification

• What decisions do people need to make with this knowledge?
• What mistakes indicate someone lacks this competency?
• What would you watch someone do to verify competency?
• What's the failure mode this competency would prevent?

For Scenario Design

• What realistic situation requires this judgment?
• What information would be incomplete or ambiguous?
• What would a weak response miss?
• What distinguishes competent from exceptional?

For Audience Mapping

• Who needs full depth? Who needs rules only?
• Can the same content serve multiple audiences at different depths?
• What's minimum viable competency for each role?

For Verification Design

• What decision does this verification inform?
• What evidence types are appropriate (scenario response, artifact, observed behavior)?
• What distinguishes "partial" from "competent" from "strong"?

For Feedback Loops

• What questions do people ask after training?
• Which questions indicate training gaps vs. framework gaps vs. process gaps?
• Who receives the signal? Who decides on fixes?

---

Anti-Patterns

---

Templates

Competency Definition Template

## [Cluster Name] Competencies

| ID | Competency | Description |
|----|------------|-------------|
| [PREFIX]-1 | [Action verb phrase] | [Observable capability starting with "Can..."] |

Scenario Template

### Scenario: [Name]

**Core decision structure:** [What judgment is being tested]

**Interview variant:**
> [Generic situation]

**Assessment variant:**
> [Organization-specific situation]

**Competencies assessed:** [IDs]

**What good looks like:**
- [Consideration]

**Red flags:**
- [Weak response indicator]

Progression Template

Foundation (Role: Everyone)
├── [COMP-1]: [Name]
└── [COMP-2]: [Name]

├─► Intermediate (Role: [Role])
│   ├── [COMP-3]: [Name] (requires: COMP-1)
│   └── [COMP-4]: [Name] (requires: COMP-2)

└─► Specialist (Role: [Role])
    └── [COMP-5]: [Name] (requires: COMP-3, COMP-4)

Feedback Loop Template

## Feedback Loop Design

**Observation mechanism:**
- How questions are logged
- What context is captured
- How they're tagged to competencies

**Analysis cadence:** [frequency]

**Pattern categories:**
- Training gap: [who handles]
- Framework gap: [who handles]
- Process gap: [who handles]
- Tooling gap: [who handles]

**Change tracking:**
- How changes are documented
- How effectiveness is measured

---

Minimum Viable Framework

If starting small:

1. 3-5 core competencies — the ones that matter most 2. 2-3 scenarios — interview + assessment variants covering core competencies 3. One layer of content — probably L2 (practitioner depth) 4. Basic rubric — not demonstrated / partial / competent / strong 5. One feedback signal — what questions do people ask after training?

Expand based on what you learn from using it.

---

Output Persistence

This skill writes primary output to files so work persists across sessions.

Output Discovery

Before doing any other work:

1. Check for context/output-config.md in the project 2. If found, look for this skill's entry 3. If not found or no entry for this skill, ask the user first:

• "Where should I save output from this competency-builder session?"
• Suggest: explorations/competency/ or a sensible location for this project

4. Store the user's preference:

• In context/output-config.md if context network exists
• In .competency-builder-output.md at project root otherwise

Primary Output

For this skill, persist:

• Diagnosed state - which competency framework state applies
• Competency definitions - derived from failure modes
• Scenario designs - test scenarios for each competency
• Framework structure - progression model and dependencies
• Feedback loop design - how gaps will be identified

Conversation vs. File

File Naming

Pattern: {domain}-competency-{date}.md Example: ai-literacy-competency-2025-01-15.md

What This Skill Does NOT Do

• Write training content — You help structure, they write content
• Prescribe specific competencies — You help them discover theirs from failure modes
• Assess whether existing training is "good" — You diagnose what's missing
• Replace subject matter expertise — You provide methodology, they provide domain knowledge

---

Health Check Questions

During competency framework development, ask:

1. Do all competencies describe observable capabilities (not knowledge states)? 2. Does each scenario require judgment that can't be looked up? 3. Is content layered appropriately for different audiences? 4. Does verification connect to real decisions? 5. Is there a mechanism to learn what's not working? 6. Has the framework changed based on feedback? 7. Can someone with prior knowledge skip parts? 8. Does everyone follow the same path, or is it personalized?

---

Integration Points

---

Example Interaction

User: "We have a 40-page security policy. Everyone 'completes' the training but keeps making mistakes."

Diagnosis: CF1 (Content-First Trap)

Questions to ask:

• What are the 3 most common mistakes people make after "completing" training?
• What decisions do people make that require this knowledge?
• When someone makes a mistake, what did they fail to recognize or do?

Guidance: "Each mistake suggests a competency gap. Let's work backward: if someone incorrectly handles sensitive data, the missing competency might be 'Can classify data according to organizational categories.' Once we have 3-5 competencies from failure modes, we'll design scenarios that test whether someone can actually apply the knowledge—not just recall it."

---

Source Framework

Derived from: frameworks/competency/competency-framework-development.md

Competency Definition Template

Use this template to define competencies. Each competency must be an observable capability.

Template

## [Cluster Name] Competencies

| ID | Competency | Description |
|----|------------|-------------|
| [PREFIX]-1 | [Action verb phrase] | Can [observable capability that you could watch someone demonstrate] |
| [PREFIX]-2 | [Action verb phrase] | Can [observable capability] |

Checklist

Before finalizing each competency, verify:

• [ ] Starts with "Can" + action verb
• [ ] Describes something you could observe
• [ ] Two people could agree/disagree whether someone has it
• [ ] Not too broad (applies to too many situations)
• [ ] Not too narrow (only applies to one exact situation)
• [ ] Connected to a real decision or action

Good vs. Bad Examples

ID Conventions

• Use 2-4 letter prefix based on cluster name (e.g., DP- for Data Privacy)
• Sequential numbers within cluster
• Keep IDs stable once assigned (don't renumber)

Deriving Competencies from Failure Modes

1. List mistakes people make 2. For each mistake, identify what someone would need to DO to avoid it 3. Write that as a competency 4. Verify it's observable

Example:

• Failure: "People put customer data in public AI tools"
• What would prevent it: Recognizing that data is sensitive + knowing which tools are approved
• Competencies:
• "Can classify data according to sensitivity levels"
• "Can identify which AI tools are approved for which data types"

Progression Model Template

Use this template to map competency dependencies and create personalized learning paths.

Template

## Progression Model

### Dependency Diagram

\`\`\`
Foundation (Role: Everyone)
├── [COMP-1]: [Name] - [Why this is foundational]
└── [COMP-2]: [Name] - [Why this is foundational]

├─► Intermediate (Role: [Role Name])
│   ├── [COMP-3]: [Name] (requires: COMP-1, COMP-2)
│   └── [COMP-4]: [Name] (requires: COMP-2)

└─► Specialist Track (Role: [Role Name])
    ├── [COMP-5]: [Name] (requires: COMP-3)
    └── [COMP-6]: [Name] (requires: COMP-3, COMP-4)
\`\`\`

### Dependency Rationale

| Competency | Requires | Why |
|------------|----------|-----|
| COMP-3 | COMP-1, COMP-2 | [Explanation of dependency] |
| COMP-4 | COMP-2 | [Explanation] |
| COMP-5 | COMP-3 | [Explanation] |

### Skip Logic

| If demonstrates... | Evidence required | Can skip... |
|--------------------|-------------------|-------------|
| COMP-1, COMP-2 in interview | Strong scenario response | Foundation training |
| Prior [domain] experience | Portfolio review | [Specific modules] |
| Certification in [X] | Verification | [Specific modules] |

### Role Paths

| Role | Required Competencies | Optional |
|------|----------------------|----------|
| [Role 1] | COMP-1, COMP-2, COMP-3 | COMP-4 |
| [Role 2] | COMP-1, COMP-2, COMP-5, COMP-6 | - |
| [Role 3] | All | - |

### Minimum Viable Path

For someone who needs to be functional quickly:

1. [COMP-1] - [Estimate: X hours] - [Why first]
2. [COMP-2] - [Estimate: X hours] - [Why second]
3. Assessment checkpoint
4. If passing, can begin work with supervision
5. Complete [COMP-3] within first [timeframe]

Design Principles

1. Dependencies Should Be Real

A dependency exists when:

• You literally can't understand B without understanding A
• A provides vocabulary/concepts that B builds on
• Attempting B without A causes common failure modes

A dependency does NOT exist just because:

• A comes before B in your documentation
• A is "simpler" than B
• You've always taught them in that order

2. Parallel Tracks Are OK

Not everyone needs everything. Role-specific branches allow:

• Faster time-to-competence for specialized roles
• Deeper expertise where needed
• No wasted training on irrelevant competencies

3. Skip Logic Respects Prior Knowledge

If someone can demonstrate competency, don't make them sit through training for it:

• Test before training
• Accept external evidence (certifications, portfolio, reference checks)
• Convert training to "verify you know our specifics" rather than full coverage

4. Minimum Viable Paths Enable Action

Define the shortest path to being useful:

• What's the absolute minimum to start working (with support)?
• What completes the foundation (independent work)?
• What leads to mastery (can teach others)?

Common Patterns

The Funnel

Wide foundation, narrowing specialization:

Foundation (everyone)
├── ├── ├──
    │
    ▼
Intermediate (most)
├── ├──
    │
    ▼
Specialist (few)
├──

The Fork

Common foundation, diverging role paths:

Foundation
├── ├──
    │
   ╱ ╲
  ▼   ▼
Path A Path B
├──    ├──

The Ladder

Linear progression with increasing depth:

Level 1 → Level 2 → Level 3 → Level 4

Validation Questions

1. Can someone complete a role path without gaps? 2. Are dependencies actually required, or just traditional? 3. Does skip logic exist for common prior knowledge? 4. Is there a minimum viable path for urgent needs? 5. Do role paths match actual job requirements?

Scenario Template

Use this template to design scenarios that test competencies through realistic situations requiring judgment.

Template

### Scenario: [Descriptive Name]

**Core decision structure:** [What judgment or trade-off is being tested]

**Interview variant:**
> [Generic situation that doesn't require knowledge of specific organization. Tests reasoning approach.]

**Assessment variant:**
> [Organization-specific version using real tools, policies, team names. Tests application of learned content.]

**Ongoing variant:** (optional)
> [Real situation that occurred, anonymized if needed. Tests continued competence.]

**Competencies assessed:** [Competency IDs this scenario tests]

**What good looks like:**
- [Key consideration a competent response would include]
- [Another consideration]
- [What distinguishes competent from strong]

**Red flags:**
- [What a weak response would miss]
- [Common mistakes]
- [Misconceptions revealed]

Scenario Design Checklist

• [ ] Realistic: Situation people will actually encounter
• [ ] Incomplete information: Like real life, not everything is specified
• [ ] Requires judgment: Can't be answered by searching documentation
• [ ] Gradable: Has better and worse responses, not just right/wrong
• [ ] Time-efficient: Can be evaluated in reasonable time
• [ ] Variant-ready: Interview version doesn't require org-specific knowledge

Testing Your Scenario

1. Too easy: If everyone gets the same answer quickly, add ambiguity 2. Too hard: If no one can engage meaningfully, provide more context 3. Not testing competency: If responses don't differentiate by competency level, redesign 4. Lookup-able: If someone could ctrl+F the answer in docs, it's testing recall not judgment

Example: Tool Evaluation Scenario

### Scenario: Third-Party Tool Evaluation

**Core decision structure:** Given incomplete vendor information, identify what you need to know and how to verify claims.

**Interview variant:**
> A team wants to use a third-party AI coding assistant. They've sent you the product page. What questions do you need answered before you can evaluate it? How would you find those answers?

**Assessment variant:**
> Team X wants to use [SpecificTool]. Here's their use case: [description]. Here's the vendor's security documentation: [link]. Evaluate this against our data policies and write up your recommendation.

**Competencies assessed:** DP-1 (Data classification), DP-2 (Service model differentiation), DP-4 (Vendor evaluation)

**What good looks like:**
- Asks about data retention, training data usage, processing location
- Distinguishes marketing claims from verifiable commitments
- Identifies what data the team would actually send
- Recognizes gaps in available information
- Knows when/how to escalate uncertainty

**Red flags:**
- Takes marketing claims at face value
- Doesn't ask about data flow
- Binary yes/no without nuance
- Can't articulate what would change the assessment

Scoring Scenarios