Mcp Sdk Tier Audit

MCP SDK Tier Audit

You are performing a comprehensive tier assessment for an MCP SDK repository against SEP-1730 (the SDK Tiering System). Your goal is to produce a definitive tier classification (Tier 1, 2, or 3) backed by evidence.

Step 0: Pre-flight Checks

Before doing anything else, verify GitHub CLI authentication:

gh auth status 2>&1

If this fails (exit code non-zero or shows "not logged in"), stop immediately and tell the user:

GitHub authentication is required for this skill. Please run gh auth login first, then re-run the skill.

Do NOT proceed to any other step if this check fails.

After parsing arguments (Step 1), also verify the conformance server is reachable:

curl -sf <conformance-server-url> -o /dev/null -w '%{http_code}' 2>&1 || true

If the server is not reachable, stop and tell the user:

Conformance server at <url> is not reachable. Make sure the everything server is running before invoking this skill.

Step 1: Parse Arguments

Extract from the user's input:

• local-path: absolute path to the SDK checkout (e.g. ~/src/mcp/typescript-sdk)
• conformance-server-url: URL where the SDK's everything server is already running (e.g. http://localhost:3000/mcp)
• client-cmd (optional): command to run the SDK's conformance client (e.g. npx tsx test/conformance/src/everythingClient.ts). If not provided, client conformance tests are skipped and noted as a gap in the report.
• branch (optional): Git branch to check on GitHub (e.g. --branch fweinberger/v1x-governance-docs). If not provided, derive from the local checkout's current branch: cd <local-path> && git rev-parse --abbrev-ref HEAD. This is passed to the tier-check CLI so that policy signal file checks use the correct branch instead of the repo's default branch.

The first two arguments are required. If either is missing, ask the user to provide it.

Derive the GitHub owner/repo from the local checkout:

cd <local-path> && git remote get-url origin | sed 's#.*github.com[:/]##; s#\.git$##'

Step 2: Run the Deterministic Scorecard

The tier-check CLI handles all deterministic checks — server conformance, client conformance, labels, triage, P0 resolution, releases, policy signals, and spec tracking. You are already in the conformance repo, so run it directly.

npm run --silent tier-check -- \
  --repo <owner/repo> \
  --branch <branch> \
  --conformance-server-url <conformance-server-url> \
  --client-cmd '<client-cmd>' \
  --output json

If no client-cmd was detected, omit the --client-cmd flag (client conformance will be skipped). The --branch flag should always be included (derived from the local checkout if not explicitly provided).

The CLI output includes server conformance pass rate, client conformance pass rate (with per-spec-version breakdown), issue triage compliance, P0 resolution times, label taxonomy, stable release status, policy signal files, and spec tracking gap. Parse the JSON output to feed into Step 4.

The conformance results now include a specVersions field on each detail entry, enabling per-version pass rate analysis. The list command also shows spec version tags: node dist/index.js list shows [2025-06-18], [2025-11-25], [draft], or [extension] next to each scenario.

Conformance Baseline Check

After running the CLI, check for an expected-failures baseline file in the SDK repo:

find <local-path> -name "baseline.yml" -o -name "expected-failures.yml" 2>/dev/null | head -5

If found, read the file. It lists known/expected conformance failures. This context is essential for interpreting raw pass rates — a 20% client pass rate due entirely to unimplemented OAuth scenarios is very different from 20% due to broken core functionality.

Step 3: Launch Parallel Evaluations

Launch 2 evaluations in parallel. Each reads the SDK from the local checkout path.

IMPORTANT: Launch both evaluations at the same time (in the same response) so they run in parallel.

Evaluation 1: Documentation Coverage

Use the prompt from references/docs-coverage-prompt.md. Pass the local path.

This evaluation checks:

• Whether all non-experimental features are documented with examples (Tier 1 requirement)
• Whether core features are documented (Tier 2 requirement)
• Produces an evidence table with file:line references

Evaluation 2: Policy Evaluation

Use the prompt from references/policy-evaluation-prompt.md. Pass the local path, the derived owner/repo, and the policy_signals section from the CLI JSON output.

The CLI has already checked which policy files exist (ROADMAP.md, DEPENDENCY_POLICY.md, dependabot.yml, VERSIONING.md, etc.). The AI evaluation reads only the files the CLI found to judge whether the content is substantive — it does NOT search for files in other locations.

This evaluation checks:

• Dependency update policy (required for Tier 1 and Tier 2)
• Published roadmap (required for Tier 1; plan-toward-Tier-1 for Tier 2)
• Clear versioning with documented breaking change policy (required for Tier 1)
• Produces evidence tables for each policy area

Step 4: Compute Final Tier

Combine the deterministic scorecard (from the CLI) with the evaluation results (docs, policies). Apply the tier logic:

Tier 1 requires ALL of:

• Server conformance test pass rate == 100% (date-versioned scenarios only; draft and extension are informational and not scored)
• Client conformance test pass rate == 100% (date-versioned scenarios only; draft and extension are informational and not scored)
• Issue triage compliance >= 90% within 2 business days
• All P0 bugs resolved within 7 days
• Stable release >= 1.0.0 with no pre-release suffix
• Clear versioning with documented breaking change policy (evaluation)
• All non-experimental features documented with examples (evaluation)
• Published dependency update policy (evaluation)
• Published roadmap with concrete steps tracking spec components (evaluation)

Tier 2 requires ALL of:

• Server conformance test pass rate >= 80% (date-versioned scenarios only)
• Client conformance test pass rate >= 80% (date-versioned scenarios only)
• Issue triage compliance >= 80% within 1 month
• P0 bugs resolved within 2 weeks
• At least one stable release >= 1.0.0
• Basic docs covering core features (evaluation)
• Published dependency update policy (evaluation)
• Published plan toward Tier 1 or explanation for remaining Tier 2 (evaluation)

Otherwise: Tier 3

If any Tier 2 requirement is not met, the SDK is Tier 3.

Important edge cases:

• If GitHub issue labels are not set up per SEP-1730, triage metrics cannot be computed. Note this as a gap. However, repos may use GitHub's native issue types instead of type labels — the CLI checks for both.
• If client conformance was skipped (no client command found), note this as a gap but do not block tier advancement based on it alone.

Conformance Breakdown:

The full suite pass rates (server total, client total) are used for tier threshold checks. To interpret them, present a single conformance matrix combining server and client results. Each detail entry in the tier-check JSON has a specVersions field; client category is derived from the scenario name (auth/ prefix = Auth, everything else = Core). Server scenarios are all Core.

Example:

Informational (not scored for tier):

The tier-scoring table only includes date-versioned scenarios. draft and extension scenarios are shown separately as informational — they do not affect tier advancement.

This immediately shows where failures concentrate. Failures clustered in Client: Auth / 2025-11-25 means "new auth features not yet implemented" — a scope gap, not a quality problem. Failures in Server or Client: Core are more concerning.

If the SDK has a baseline.yml or expected-failures file, cross-reference with the matrix to identify whether baselined failures cluster in a specific cell (e.g. all in 2025-11-25 / Client: Auth = scope gap).

P0 Label Audit Guidance:

When evaluating P0 metrics, flag potentially mislabeled P0 issues:

• If P0 count is high (>2) but other Tier 2 metrics (conformance, triage compliance, docs) are strong, this may indicate P0 labels are being used for enhancements, lower-priority work, or feature requests rather than actual critical bugs.
• In such cases, recommend a P0 label audit as a remediation action. Review open P0 issues to verify they represent genuine blocking defects vs. misclassified work.
• Document this finding in the remediation output with specific issue numbers and suggested re-triage actions.
• Do not treat high P0 count as an automatic hard blocker if the audit reveals mislabeling; instead, note it as a process improvement opportunity.

Step 5: Generate Output

Write detailed reports to files using subagents, then show a concise summary to the user.

Output files (write via subagents)

IMPORTANT: Write both report files using parallel subagents (Task tool) so the file-writing work does not pollute the main conversation thread. Launch both subagents at the same time.

Write two files to results/ in the conformance repo:

• results/<YYYY-MM-DD>-<sdk-name>-assessment.md
• results/<YYYY-MM-DD>-<sdk-name>-remediation.md

For example: results/2026-02-11-typescript-sdk-assessment.md

Assessment subagent

Pass all the gathered data (CLI scorecard JSON, docs evaluation results, policy evaluation results) to a subagent and instruct it to write the assessment file using the template from references/report-template.md. This file contains the full requirements table, conformance test details (both server and client), triage metrics, documentation coverage table, and policy evaluation evidence.

Remediation subagent

Pass all the gathered data to a subagent and instruct it to write the remediation file using the template from references/report-template.md. This file always includes both:

• Path to Tier 2 (if current tier is 3) -- what's needed to reach Tier 2
• Path to Tier 1 (always) -- what's needed to reach Tier 1

Console output (shown to the user)

After the subagents finish, output a short executive summary directly to the user:

## <sdk-name> — Tier <X>

Conformance:

|              | 2025-03-26 | 2025-06-18 | 2025-11-25 | All* | T2 | T1 |
|--------------|------------|------------|------------|------|----|----|
| Server       | —          | pass/total | pass/total | pass/total (rate%) | ✓/✗ | ✓/✗ |
| Client: Core | —          | pass/total | pass/total | pass/total (rate%) | — | — |
| Client: Auth | pass/total | pass/total | pass/total | pass/total (rate%) | — | — |
| **Client Total** | | | | **pass/total (rate%)** | **✓/✗** | **✓/✗** |

\* unique scenarios — a scenario may apply to multiple spec versions

Informational (not scored for tier):

|              | draft | extension |
|--------------|-------|-----------|
| Client: Auth | pass/total | pass/total |

If a baseline file was found, add a note below the conformance table:
> **Baseline**: {N} failures in `baseline.yml` ({list by cell, e.g. "6 in Client: Auth/2025-11-25, 2 in Client: Auth/extension"}).

Repository Health:

| Check | Value | T2 | T1 |
|-------|-------|----|----|
| Issue Triage | <rate>% (<triaged>/<total>) | ✓/✗ | ✓/✗ |
| Labels | <present>/<required> | ✓/✗ | ✓/✗ |
| P0 Resolution | <count> open | ✓/✗ | ✓/✗ |
| Spec Tracking | <days>d gap | ✓/✗ | ✓/✗ |
| Documentation | <pass>/<total> features | ✓/✗ | ✓/✗ |
| Dependency Policy | <summary> | ✓/✗ | ✓/✗ |
| Roadmap | <summary> | ✓/✗ | ✓/✗ |
| Versioning Policy | <summary> | N/A | ✓/✗ |
| Stable Release | <version> | ✓/✗ | ✓/✗ |

---

**High-Priority Fixes:**
1. <fix description>

**For Tier 2:**
1. <gap description>
2. <gap description>

**For Tier 1:**
1. <gap description>
2. <gap description>

Reports:
- results/<date>-<sdk-name>-assessment.md
- results/<date>-<sdk-name>-remediation.md

Use ✓ for pass and ✗ for fail.

High-Priority Fixes: List any issues that need urgent attention (e.g., P0 label audit if P0 count is >2 but other metrics are strong, suggesting mislabeled issues). If none, omit this section.

For Tier 2 / For Tier 1: List each gap as a separate numbered item. Use "All requirements met" if there are no gaps for that tier. Each item should be a concise action (e.g., "Re-triage mislabeled P0s", "Document 16 undocumented core features").

Reference Files

The following reference files are available in the references/ directory alongside this skill:

• references/feature-list.md -- Canonical list of 48 non-experimental + 5 experimental features (single source of truth)
• references/tier-requirements.md -- Full SEP-1730 requirements table with exact thresholds
• references/report-template.md -- Output format template for the audit report
• references/docs-coverage-prompt.md -- Evaluation prompt for documentation coverage
• references/policy-evaluation-prompt.md -- Evaluation prompt for policy review

Read these reference files when you need the detailed content for evaluation prompts or report formatting.

Usage Examples

# TypeScript SDK — server + client conformance
/mcp-sdk-tier-audit ~/src/mcp/typescript-sdk http://localhost:3000/mcp "npx tsx ~/src/mcp/typescript-sdk/test/conformance/src/everythingClient.ts"

# Python SDK — server + client conformance
/mcp-sdk-tier-audit ~/src/mcp/python-sdk http://localhost:3001/mcp "uv run python ~/src/mcp/python-sdk/.github/actions/conformance/client.py"

# Go SDK — server + client conformance
/mcp-sdk-tier-audit ~/src/mcp/go-sdk http://localhost:3002 "/tmp/go-conformance-client"

# C# SDK — server + client conformance
# Two C#-specific requirements in the client-cmd:
#   --framework net9.0    : required because the project targets net8.0/net9.0/net10.0
#   -- $MCP_CONFORMANCE_SCENARIO : the runner sets this env var and uses shell:true, so the
#                           shell expands it; dotnet passes [scenario, url] to the program
/mcp-sdk-tier-audit ~/src/mcp/csharp-sdk http://localhost:3003 "dotnet run --project ~/src/mcp/csharp-sdk/tests/ModelContextProtocol.ConformanceClient --framework net9.0 -- $MCP_CONFORMANCE_SCENARIO"

# Any SDK — server conformance only (no client)
/mcp-sdk-tier-audit ~/src/mcp/some-sdk http://localhost:3004

MCP SDK Tier Audit

Assess any MCP SDK repository against SEP-1730 (the SDK Tiering System). Produces a tier classification (1/2/3) with an evidence-backed scorecard.

Two components work together:

• `tier-check` CLI — runs deterministic checks (server + client conformance pass rate, issue triage speed, P0 resolution, labels, releases, policy signals). Works standalone, no AI needed.
• AI-assisted assessment — an agent uses the CLI scorecard plus judgment-based evaluation (documentation coverage, dependency policy, roadmap) to produce a full tier report with remediation guide.

Quick Start: CLI

The CLI is a subcommand of the MCP Conformance tool.

# Clone and build
git clone https://github.com/modelcontextprotocol/conformance.git
cd conformance
npm install
npm run build

# Authenticate with GitHub (needed for API access)
gh auth login

# Run against any MCP SDK repo (without conformance tests)
npm run --silent tier-check -- --repo modelcontextprotocol/typescript-sdk --skip-conformance

The CLI uses the GitHub API (read-only) for issue metrics, labels, and release checks. Authenticate via one of:

• GitHub CLI (recommended): gh auth login — the CLI picks up your token automatically
• Environment variable: export GITHUB_TOKEN=ghp_...
• Flag: --token ghp_...

For public repos, any authenticated token works (no special scopes needed — authentication just avoids rate limits). For a fine-grained personal access token, select Public Repositories (read-only) with no additional permissions.

CLI Options

--repo <owner/repo>              GitHub repository (required)
--branch <branch>                Branch to check
--skip-conformance               Skip conformance tests
--conformance-server-url <url>   URL of the already-running conformance server
--client-cmd <cmd>               Command to run the SDK conformance client (for client conformance tests)
--days <n>                       Limit triage analysis to last N days
--output <format>                json | markdown | terminal (default: terminal)
--token <token>                  GitHub token (defaults to GITHUB_TOKEN or gh auth token)

What the CLI Checks

Example Output

Tier Assessment: Tier 2

Repo:      modelcontextprotocol/typescript-sdk
Timestamp: 2026-02-10T12:00:00Z

Check Results:

  ✓ Server Conformance  45/45 (100%)
  ✓ Client Conformance  4/4 (100%)
  ✗ Labels         9/12 required labels
    Missing: needs confirmation, needs repro, ready for work
  ✓ Triage         92% within 2BD (150 issues, median 8h)
  ✓ P0 Resolution  0 open, 3/3 closed within 7d
  ✓ Stable Release 2.3.1
  ~ Policy Signals ✓ CHANGELOG.md, ✗ SECURITY.md, ✓ CONTRIBUTING.md, ✓ .github/dependabot.yml, ✗ ROADMAP.md
  ✓ Spec Tracking  2d gap

Use --output json to get machine-readable results, or --output markdown for a report you can paste into an issue.

Full AI-Assisted Assessment

The CLI produces a deterministic scorecard, but some SEP-1730 requirements need judgment: documentation quality, dependency policy, roadmap substance. An AI agent can evaluate these by reading the repo.

Claude Code

The skill lives in .claude/skills/ in this repo, so if you open Claude Code in the conformance repo it's already available.

1. Make sure gh auth login is done (the skill checks this upfront) 2. Start the SDK's everything server in a separate terminal 3. Run the skill:

/mcp-sdk-tier-audit <local-sdk-path> <conformance-server-url> [client-cmd]

Pass the client command as the third argument to include client conformance testing. If omitted, client conformance is skipped and noted as a gap in the report.

TypeScript SDK example:

# Terminal 1: start the everything server (build first: npm run build)
cd ~/src/mcp/typescript-sdk && npm run test:conformance:server:run

# Terminal 2: run the audit (from the conformance repo)
/mcp-sdk-tier-audit ~/src/mcp/typescript-sdk http://localhost:3000/mcp "npx tsx ~/src/mcp/typescript-sdk/test/conformance/src/everythingClient.ts"

Python SDK example:

# Terminal 1: install and start the everything server
cd ~/src/mcp/python-sdk && uv sync --frozen --all-extras --package mcp-everything-server
uv run mcp-everything-server --port 3001

# Terminal 2: run the audit (from the conformance repo)
/mcp-sdk-tier-audit ~/src/mcp/python-sdk http://localhost:3001/mcp "uv run python ~/src/mcp/python-sdk/.github/actions/conformance/client.py"

Go SDK example:

# Terminal 1: build and start the everything server
cd ~/src/mcp/go-sdk && go build -o /tmp/go-conformance-server ./conformance/everything-server
go build -o /tmp/go-conformance-client ./conformance/everything-client
/tmp/go-conformance-server -http="localhost:3002"

# Terminal 2: run the audit (from the conformance repo)
/mcp-sdk-tier-audit ~/src/mcp/go-sdk http://localhost:3002 "/tmp/go-conformance-client"

C# SDK example:

# Terminal 1: start the everything server (requires .NET SDK)
cd ~/src/mcp/csharp-sdk
dotnet run --project tests/ModelContextProtocol.ConformanceServer --framework net9.0 -- --urls http://localhost:3003

# Terminal 2: run the audit (from the conformance repo)
/mcp-sdk-tier-audit ~/src/mcp/csharp-sdk http://localhost:3003 "dotnet run --project ~/src/mcp/csharp-sdk/tests/ModelContextProtocol.ConformanceClient"

The skill derives owner/repo from git remote, runs the CLI, launches parallel evaluations for docs and policy, and writes detailed reports to results/.

Any Other AI Coding Agent

If you use a different agent (Codex, Cursor, Aider, OpenCode, etc.), give it these instructions:

1. Run the CLI to get the deterministic scorecard:

   node dist/index.js tier-check --repo <repo> --conformance-server-url <url> --output json

2. Evaluate documentation coverage — check whether MCP features (tools, resources, prompts, sampling, transports, etc.) are documented with examples. See `references/docs-coverage-prompt.md` for the full checklist.

3. Evaluate policies — check for dependency update policy, roadmap, and versioning/breaking-change policy. See `references/policy-evaluation-prompt.md` for criteria.

4. Apply tier logic — combine scorecard + evaluations against the thresholds in `references/tier-requirements.md`.

5. Generate report — use `references/report-template.md` for the output format.

Manual Review

Run the CLI for the scorecard, then review docs and policies yourself using the tier requirements as a checklist:

Running Conformance Tests

To include conformance test results, start the SDK's everything server first, then pass the URL to the CLI. To also run client conformance tests, pass --client-cmd with the command to launch the SDK's conformance client.

TypeScript SDK:

# Terminal 1: start the server (SDK must be built first)
cd ~/src/mcp/typescript-sdk && npm run build
npm run test:conformance:server:run   # starts on port 3000

# Terminal 2: run tier-check (server + client conformance)
npm run --silent tier-check -- \
  --repo modelcontextprotocol/typescript-sdk \
  --conformance-server-url http://localhost:3000/mcp \
  --client-cmd 'npx tsx ~/src/mcp/typescript-sdk/test/conformance/src/everythingClient.ts'

Python SDK:

# Terminal 1: install and start the server
cd ~/src/mcp/python-sdk
uv sync --frozen --all-extras --package mcp-everything-server
uv run mcp-everything-server --port 3001   # specify port to avoid conflicts

# Terminal 2: run tier-check (server + client conformance)
npm run --silent tier-check -- \
  --repo modelcontextprotocol/python-sdk \
  --conformance-server-url http://localhost:3001/mcp \
  --client-cmd 'uv run python ~/src/mcp/python-sdk/.github/actions/conformance/client.py'

Go SDK:

# Terminal 1: build and start the server
cd ~/src/mcp/go-sdk
go build -o /tmp/go-conformance-server ./conformance/everything-server
go build -o /tmp/go-conformance-client ./conformance/everything-client
/tmp/go-conformance-server -http="localhost:3002"

# Terminal 2: run tier-check (server + client conformance)
npm run --silent tier-check -- \
  --repo modelcontextprotocol/go-sdk \
  --conformance-server-url http://localhost:3002 \
  --client-cmd '/tmp/go-conformance-client'

C# SDK:

# Terminal 1: start the server (requires .NET SDK)
cd ~/src/mcp/csharp-sdk
dotnet run --project tests/ModelContextProtocol.ConformanceServer --framework net9.0 -- --urls http://localhost:3003

# Terminal 2: run tier-check (server + client conformance)
npm run --silent tier-check -- \
  --repo modelcontextprotocol/csharp-sdk \
  --conformance-server-url http://localhost:3003 \
  --client-cmd 'dotnet run --project ~/src/mcp/csharp-sdk/tests/ModelContextProtocol.ConformanceClient'

Other SDKs: Your SDK needs an "everything server" — an HTTP server implementing the Streamable HTTP transport with all MCP features (tools, resources, prompts, etc.). See the implementations above as reference.

Start your everything server, then pass --conformance-server-url. Pass --client-cmd if your SDK has a conformance client. If neither exists yet, use --skip-conformance — the scorecard will note this as a gap.

Reference Files

These files in `references/` contain the detailed criteria and prompts:

Documentation Coverage Subagent Prompt

You are evaluating the documentation coverage of an MCP SDK repository for the SEP-1730 tier assessment.

Input

• SDK path: {local-path} (absolute path to local SDK checkout)

Your Task

Evaluate the documentation quality and coverage of this MCP SDK against the canonical feature list. You need to determine:

1. Tier 1 compliance: Are ALL non-experimental features documented with examples? 2. Tier 2 compliance: Are core features documented (basic docs)?

Steps

1. Read the canonical feature list

Read references/feature-list.md for the definitive list of 48 non-experimental features (plus 5 experimental) to evaluate. That file is the single source of truth — use every feature listed there, in order.

2. Find all documentation sources

The SDK is available at {local-path}. Search for documentation in these locations:

• README.md (root and any subdirectory READMEs)
• docs/ directory
• documentation/ directory
• examples/ directory
• API documentation (generated or hand-written)
• CONTRIBUTING.md
• Inline code comments and docstrings on public API surfaces
• Any *.md files in the repo

# Find all markdown files
find {local-path} -name "*.md" -not -path "*/node_modules/*" -not -path "*/.git/*"

# Find example files
find {local-path} -path "*/examples/*" -not -path "*/node_modules/*"

# Find API docs
find {local-path} -path "*/docs/*" -not -path "*/node_modules/*"

3. Evaluate each feature

For each of the 48 non-experimental features in the canonical list, determine:

• Documented?: Is there documentation explaining this feature? (Yes/No)
• Where: File path and line numbers where documentation exists
• Has Examples?: Are there code examples showing how to use this feature? (Yes/No/N/A)
• Verdict: PASS (documented with examples), PARTIAL (documented but no examples), or FAIL (not documented)

Required Output Format

Produce your assessment in this exact format:

### Documentation Coverage Assessment

**SDK path**: {local-path}
**Documentation locations found**:

- {path1}: {description}
- {path2}: {description}
- ...

#### Feature Documentation Table

One row per feature from `references/feature-list.md`. Use the exact feature numbers and names from that file.

| #   | Feature                    | Documented? | Where          | Has Examples?           | Verdict           |
| --- | -------------------------- | ----------- | -------------- | ----------------------- | ----------------- |
| 1   | Tools - listing            | Yes/No      | {file}:{lines} | Yes ({N} examples) / No | PASS/PARTIAL/FAIL |
| 2   | Tools - calling            | Yes/No      | {file}:{lines} | Yes ({N} examples) / No | PASS/PARTIAL/FAIL |
| ... | ...                        | ...         | ...            | ...                     | ...               |
| 48  | JSON Schema 2020-12        | Yes/No      | {file}:{lines} | Yes ({N} examples) / No | PASS/PARTIAL/FAIL |
| —   | Tasks - get (experimental) | Yes/No      | {file}:{lines} | Yes ({N} examples) / No | INFO              |
| ... | ...                        | ...         | ...            | ...                     | ...               |

All 48 non-experimental features MUST appear in the table. Do not skip or merge rows.

#### Summary

**Total non-experimental features**: 48
**PASS (documented with examples)**: {N}/48
**PARTIAL (documented, no examples)**: {N}/48
**FAIL (not documented)**: {N}/48

**Core features documented**: {N}/{total core} ({percentage}%)
**All features documented with examples**: {N}/48 ({percentage}%)

#### Tier Verdicts

**Tier 1** (all non-experimental features documented with examples): **PASS/FAIL**

- {If FAIL: list the features missing documentation or examples}

**Tier 2** (basic docs covering core features): **PASS/FAIL**

- {If FAIL: list the core features missing documentation}

What Counts as "Documented"

A feature is "documented" only if there is prose documentation (in README, docs/, or similar) explaining what the feature does, when to use it, and how it works. The following do not count as documentation on their own:

• Example code without accompanying prose explanation
• Conformance test servers or test fixtures
• Source code, even with comments or docstrings
• Mere existence of an API (e.g., a function existing in the SDK)

Examples supplement documentation but do not replace it. A feature with a working example in examples/ but no prose explaining the feature is PARTIAL, not PASS. A feature with only a conformance server implementation and no user-facing docs is FAIL.

Verdict criteria

• PASS: Prose documentation exists explaining the feature AND at least one runnable or near-runnable code example
• PARTIAL: Either prose docs exist but no examples, OR examples exist but no prose docs
• FAIL: No prose documentation and no examples. Also use FAIL if the feature is only demonstrated in test/conformance code with no user-facing docs or examples

What counts as an "example"

• Runnable code in an examples/ directory
• Code snippets embedded in prose documentation (README, docs/\*.md)
• Go Example* test functions (these render on pkg.go.dev and are a language convention)
• Examples in test files count only if they are clearly labeled as examples or referenced from documentation

What does NOT count as an example

• Conformance test server implementations
• Internal test fixtures
• Source code of the SDK itself

Other Important Notes

• If the SDK does not implement a feature at all, mark it as "FAIL" for documentation but note "Not implemented" in the Where column.
• Be thorough: check README, docs/, examples/, API references, and inline docstrings.
• Apply these criteria consistently across all features. Do not give credit for documentation that doesn't exist.

MCP SDK Canonical Feature List

Single source of truth for all MCP features evaluated in the tier audit. 48 non-experimental features plus 5 experimental (informational only).

When updating this list, also update the total count referenced in docs-coverage-prompt.md.

Non-Experimental Features (48 total)

Core Features (36)

Transport Features (6)

Protocol Features (6)

Experimental Features (5, informational only)

Policy Evaluation Prompt

You are evaluating the governance and policy documentation of an MCP SDK repository for the SEP-1730 tier assessment.

Input

• SDK path: {local-path} (absolute path to local SDK checkout)
• Repository: {repo} (GitHub owner/repo, derived from git remote)
• CLI policy_signals: {policy_signals_json} (from the tier-check CLI output — shows which files exist)

Your Task

The CLI has already determined which policy files exist in the repository. Your job is to read and evaluate the content of the files that were found. Do NOT search for files in other locations — only evaluate what the CLI reported as present.

Three policy areas to evaluate:

1. Dependency update policy (required for Tier 1 and Tier 2) 2. Roadmap (Tier 1: published roadmap; Tier 2: published plan toward Tier 1) 3. Versioning policy (Tier 1 only: documented breaking change policy)

Steps

1. Identify which files exist from CLI output

From the policy_signals.files object in the CLI JSON output, note which files have true (exist) vs false (missing).

The CLI checks these files:

Dependency policy: DEPENDENCY_POLICY.md, docs/dependency-policy.md, .github/dependabot.yml, .github/renovate.json, renovate.json

Roadmap: ROADMAP.md, docs/roadmap.md

Versioning: VERSIONING.md, docs/versioning.md, BREAKING_CHANGES.md

General (may contain relevant sections): CONTRIBUTING.md

2. Read and evaluate files that exist

For each file that the CLI reported as present, read its content at {local-path}/{file} and evaluate:

• Is the content substantive (not just a placeholder title)?
• Does it meet the criteria below?

Do NOT search the repo for policy information in other files. If the dedicated file doesn't exist, the policy is not published.

Evaluation Criteria

Dependency Update Policy

PASS if any of these exist with substantive content:

• DEPENDENCY_POLICY.md or docs/dependency-policy.md — must describe how and when dependencies are updated
• .github/dependabot.yml or .github/renovate.json or renovate.json — automated tooling counts as a published policy in practice

FAIL if none of the above exist (per CLI output).

Roadmap

PASS for Tier 1: ROADMAP.md or docs/roadmap.md exists with concrete work items tracking MCP spec components.

PASS for Tier 2: Same file exists with at least a plan toward Tier 1, or explanation for remaining at Tier 2.

FAIL if no roadmap file exists (per CLI output).

Versioning Policy

PASS for Tier 1 if any of these exist with substantive content:

• VERSIONING.md or docs/versioning.md or BREAKING_CHANGES.md
• A clearly labeled "Versioning" or "Breaking Changes" section in CONTRIBUTING.md (only check if CONTRIBUTING.md exists per CLI output)

The content must describe: what constitutes a breaking change, how breaking changes are communicated, and the versioning scheme.

Not required for Tier 2.

FAIL if no versioning documentation found in the above files.

Required Output Format

### Policy Evaluation Assessment

**SDK path**: {local-path}
**Repository**: {repo}

---

#### 1. Dependency Update Policy: {PASS/FAIL}

| File                      | Exists (CLI) | Content Verdict                 |
| ------------------------- | ------------ | ------------------------------- |
| DEPENDENCY_POLICY.md      | Yes/No       | Substantive / Placeholder / N/A |
| docs/dependency-policy.md | Yes/No       | Substantive / Placeholder / N/A |
| .github/dependabot.yml    | Yes/No       | Configured / N/A                |
| .github/renovate.json     | Yes/No       | Configured / N/A                |

**Verdict**: **PASS/FAIL** — {one-line explanation}

---

#### 2. Roadmap: {PASS/FAIL}

| File            | Exists (CLI) | Content Verdict                 |
| --------------- | ------------ | ------------------------------- |
| ROADMAP.md      | Yes/No       | Substantive / Placeholder / N/A |
| docs/roadmap.md | Yes/No       | Substantive / Placeholder / N/A |

**Verdict**:

- **Tier 1**: **PASS/FAIL** — {one-line explanation}
- **Tier 2**: **PASS/FAIL** — {one-line explanation}

---

#### 3. Versioning Policy: {PASS/FAIL}

| File                                 | Exists (CLI) | Content Verdict                 |
| ------------------------------------ | ------------ | ------------------------------- |
| VERSIONING.md                        | Yes/No       | Substantive / Placeholder / N/A |
| docs/versioning.md                   | Yes/No       | Substantive / Placeholder / N/A |
| BREAKING_CHANGES.md                  | Yes/No       | Substantive / Placeholder / N/A |
| CONTRIBUTING.md (versioning section) | Yes/No       | Found / Not found / N/A         |

**Verdict**:

- **Tier 1**: **PASS/FAIL** — {one-line explanation}
- **Tier 2**: **N/A** — only requires stable release

---

#### Overall Policy Summary

| Policy Area              | Tier 1    | Tier 2    |
| ------------------------ | --------- | --------- |
| Dependency Update Policy | PASS/FAIL | PASS/FAIL |
| Roadmap                  | PASS/FAIL | PASS/FAIL |
| Versioning Policy        | PASS/FAIL | N/A       |

Important Notes

• Only evaluate files the CLI reported as existing. Do not search the repo for alternatives.
• If a file exists but is just a placeholder (e.g., only has a title with no content), mark it as "Placeholder" and FAIL.
• Dependabot/Renovate config files pass automatically if they exist and are properly configured.
• CHANGELOG.md showing past releases does NOT count as a roadmap.

Report Templates

Write two files to results/ in the conformance repo:

• results/<YYYY-MM-DD>-<sdk-name>-assessment.md
• results/<YYYY-MM-DD>-<sdk-name>-remediation.md

assessment.md

# MCP SDK Tier Audit: {repo}

**Date**: {date}
**Branch**: {branch}
**Auditor**: mcp-sdk-tier-audit skill (automated + subagent evaluation)

## Tier Assessment: Tier {X}

{Brief 1-2 sentence summary of the overall assessment and key factors.}

### Requirements Summary

| #   | Requirement             | Tier 1 Standard                   | Tier 2 Standard              | Current Value                     | T1?         | T2?         | Gap                |
| --- | ----------------------- | --------------------------------- | ---------------------------- | --------------------------------- | ----------- | ----------- | ------------------ |
| 1a  | Server Conformance      | 100% pass rate                    | >= 80% pass rate             | {X}% ({passed}/{total})           | {PASS/FAIL} | {PASS/FAIL} | {detail or "None"} |
| 1b  | Client Conformance      | 100% pass rate                    | >= 80% pass rate             | {X}% ({passed}/{total})           | {PASS/FAIL} | {PASS/FAIL} | {detail or "None"} |
| 2   | Issue Triage            | >= 90% within 2 biz days          | >= 80% within 1 month        | {compliance}% ({triaged}/{total}) | {PASS/FAIL} | {PASS/FAIL} | {detail or "None"} |
| 2b  | Labels                  | 12 required labels                | 12 required labels           | {present}/{required}              | {PASS/FAIL} | {PASS/FAIL} | {detail or "None"} |
| 3   | Critical Bug Resolution | All P0s within 7 days             | All P0s within 2 weeks       | {open P0 count} open              | {PASS/FAIL} | {PASS/FAIL} | {detail or "None"} |
| 4   | Stable Release          | Required + clear versioning       | At least one stable release  | {version}                         | {PASS/FAIL} | {PASS/FAIL} | {detail or "None"} |
| 4b  | Spec Tracking           | Timeline agreed per release       | Within 6 months              | {days_gap}d gap ({PASS/FAIL})     | {PASS/FAIL} | {PASS/FAIL} | {detail or "None"} |
| 5   | Documentation           | Comprehensive w/ examples         | Basic docs for core features | {pass}/{total} features           | {PASS/FAIL} | {PASS/FAIL} | {detail or "None"} |
| 6   | Dependency Policy       | Published update policy           | Published update policy      | {Found/Not found}                 | {PASS/FAIL} | {PASS/FAIL} | {detail or "None"} |
| 7   | Roadmap                 | Published roadmap                 | Plan toward Tier 1           | {Found/Not found}                 | {PASS/FAIL} | {PASS/FAIL} | {detail or "None"} |
| 8   | Versioning Policy       | Documented breaking change policy | N/A                          | {Found/Not found}                 | {PASS/FAIL} | N/A         | {detail or "None"} |

### Tier Determination

- Tier 1: {PASS/FAIL} -- {count}/8 requirements met (failing: {list})
- Tier 2: {PASS/FAIL} -- {count}/7 requirements met (failing: {list})
- **Final Tier: {X}**

---

## Server Conformance Details

Pass rate: {X}% ({passed}/{total})

| Scenario | Status      | Checks           |
| -------- | ----------- | ---------------- |
| {name}   | {PASS/FAIL} | {passed}/{total} |
| ...      | ...         | ...              |

---

## Client Conformance Details

Full suite pass rate: {X}% ({passed}/{total})

> **Suite breakdown**: Core: {core_pass}/{core_total} ({core_rate}%), Auth: {auth_pass}/{auth_total} ({auth_rate}%)
> **Baseline**: {N} known expected failures documented in `{baseline_file}` ({categories})

### Core Scenarios

| Scenario | Status      | Checks           |
| -------- | ----------- | ---------------- |
| {name}   | {PASS/FAIL} | {passed}/{total} |
| ...      | ...         | ...              |

### Auth Scenarios

| Scenario | Status      | Checks           | Notes                       |
| -------- | ----------- | ---------------- | --------------------------- |
| {name}   | {PASS/FAIL} | {passed}/{total} | {in baseline? / unexpected} |
| ...      | ...         | ...              | ...                         |

---

## Issue Triage Details

Analysis period: Last {N} issues
Labels: {present/missing list}

| Metric          | Value | T1 Req | T2 Req | Verdict   |
| --------------- | ----- | ------ | ------ | --------- |
| Compliance rate | {X}%  | >= 90% | >= 80% | {verdict} |
| Exceeding SLA   | {N}   | --     | --     | --        |
| Open P0s        | {N}   | 0      | 0      | {verdict} |

{If open P0s, list them with issue number, title, age}

---

## Documentation Coverage

{Paste subagent 1 output: feature table with Documented/Where/Examples/Verdict columns}

---

## Policy Evaluation

{Paste subagent 2 output: dependency policy, roadmap, versioning policy sections with evidence tables}

remediation.md

# Remediation Guide: {repo}

**Date**: {date}
**Current Tier**: {X}

## Path to Tier 2

{Only include this section if current tier is 3. List every requirement not met for Tier 2.}

| #   | Action        | Requirement   | Effort               | Where        |
| --- | ------------- | ------------- | -------------------- | ------------ |
| 1   | {description} | {requirement} | {Small/Medium/Large} | {file paths} |
| ... | ...           | ...           | ...                  | ...          |

## Path to Tier 1

{Always include this section. List every requirement not met for Tier 1 (including any Tier 2 gaps).}

| #   | Action        | Requirement   | Effort               | Where        |
| --- | ------------- | ------------- | -------------------- | ------------ |
| 1   | {description} | {requirement} | {Small/Medium/Large} | {file paths} |
| ... | ...           | ...           | ...                  | ...          |

## Recommended Next Steps

1. {First priority action with brief rationale}
2. {Second priority action}
3. {Third priority action}

Formatting Rules

1. Every PASS/FAIL must be based on evidence, not assumption. 2. If data is unavailable, mark as "N/A - {reason}" and note in remediation. 3. All file references must include file path and line numbers where possible. 4. Remediation items ordered by impact: tier-advancing items first. 5. Effort estimates: Small (< 1 day), Medium (1-3 days), Large (> 3 days).

SEP-1730: SDK Tier Requirements Reference

This is the authoritative reference table for MCP SDK tiering requirements, extracted from SEP-1730.

Source: modelcontextprotocol/docs/community/sdk-tiers.mdx in the spec repository

Full Requirements Table

Exact Thresholds for Automated Checking

Conformance Score Calculation

Every scenario in the conformance suite has a specVersions field indicating which spec version it targets. The valid values are defined as the SpecVersion type (as a list) in src/types.ts — run node dist/index.js list to see the current mapping of scenarios to spec versions.

Date-versioned scenarios (e.g. 2025-06-18, 2025-11-25) count toward tier scoring. draft and extension scenarios are listed separately as informational.

The --spec-version CLI flag filters scenarios cumulatively for date versions (e.g. --spec-version 2025-06-18 includes 2025-03-26 + 2025-06-18). For draft/extension, it returns exact matches only.

The tier-check output includes a per-version pass rate breakdown alongside the aggregate.

Tier Relegation Rules

• Tier 1 to Tier 2: Any conformance test fails continuously for 4 weeks
• Tier 2 to Tier 3: More than 20% of conformance tests fail continuously for 4 weeks

Issue Triage Label Taxonomy

SDK repositories must use these consistent labels to enable automated reporting.

Type Labels (pick one)

Note: Repositories using GitHub's native issue types satisfy this requirement without needing type labels.

Status Labels (pick one)

Priority Labels (only if actionable)

Total: 12 labels (3 type + 5 status + 4 priority)

Key Definitions

Issue Triage

Labeling and determining whether an issue is valid. This is NOT the same as resolving the issue. An issue is considered triaged when it receives its first label.

Critical Bug (P0)

• Security vulnerabilities with CVSS score >= 7.0 (High or Critical severity)
• Core functionality failures that prevent basic MCP operations: connection establishment, message exchange, or use of core primitives (tools, resources, prompts)

Stable Release

A published version explicitly marked as production-ready. Specifically: version 1.0.0 or higher without pre-release identifiers like -alpha, -beta, or -rc.

Clear Versioning

Following idiomatic versioning patterns with documented breaking change policies, so users can understand compatibility expectations when upgrading.

Roadmap

Outlines concrete steps and work items that track implementation of required MCP specification components (non-experimental features and optional capabilities), giving users visibility into upcoming feature support.