Wp Abilities Verify

WP Abilities Verify

Verify a WordPress plugin's Abilities API registrations. The centerpiece is the adversarial annotation correctness check: a readonly: true ability that actually writes (via $wpdb->update, update_option, a non-GET delegate, etc.) is a security and UX disaster because agents plan actions on the basis of the annotations they introspect. This skill catches those lies by reading the callback body and comparing what it does against what the annotation claims.

The skill also validates audit docs produced by wp-abilities-audit, checks permission gates and schema hygiene, and optionally executes each ability against a live environment.

When to use

• After abilities have been registered in a plugin but before a PR

lands.

• As a health-check on an already-shipped plugin (catch regressions

where a refactor turned a readonly ability into a writing one).

• To validate an audit document before handing it to an implementer.

Two modes

• Static mode — runs from the plugin checkout. No env. Enumerates

via source inspection, runs the adversarial correctness check, runs schema and permission lints, and validates audit docs.

• Runtime mode — requires a running env. Does everything static

does PLUS: wp_get_abilities() for authoritative enumeration, executes each ability with curated inputs, confirms permission roundtrip against real users, and runs a twin-invocation heuristic on idempotent: true abilities to flag candidates for review (return-value equality is a signal, not a verdict — core defines idempotent as "no additional effect on the environment").

Both modes produce the same structured report format.

A static-mode PASS means "no obvious-shape violations," not "verified write-free." For high-stakes plugins, run runtime mode before landing — it catches bootstrap-order, permission-roundtrip, and idempotency issues that static can't. See references/annotation-correctness.md for the static blind spots.

Inputs required

1. Plugin checkout path — working tree to verify. 2. Mode — static or runtime. Default to static if unspecified. 3. (Runtime only) Env-up command — read the plugin's AGENTS.md. Common patterns: npm run wp-env start, npx wp-env start, or a composer-based bring-up. Plugin families with their own dev tooling will document their own command. Do NOT assume npm run wp-env works. 4. (Optional) Audit doc path — enables cross-checks between the audit and the registered abilities, and validates the audit itself. 5. Report output path — explicit path, typically the user's vault.

Prerequisites

• wp-project-triage has been run on the plugin.
• The plugin has at least one registered ability in source. Zero hits

on wp_register_ability( → return a clear "no abilities registered" report, not an empty PASS.

Procedure

1. (If audit provided) Validate the audit doc

Read references/audit-schema-validation.md. Validate the audit against the canonical schema owned by wp-abilities-audit. Surface missing required fields, multiple reference_ability: true, and backing: null entries that aren't paired with a surfaced_gaps entry. backing: null alone is WARN (intentional gap output), not FAIL.

2. Enumerate abilities statically

Read references/static-enumeration.md. Find each wp_register_ability( call, extract the name, the annotation block, and the execute-callback location. Use a multi-line tool (rg --multiline --pcre2) — the canonical formatting splits the call across lines. Record each ability's source-file + line + annotations + callback byte range.

3. (Runtime only) Enumerate via REST + wp-cli

Read references/runtime-harness.md. Bring the env up using the command from AGENTS.md, then enumerate via wp_get_abilities() over wp-cli and cross-check against the static inventory. Source-only → FAIL (registration not firing). Runtime-only → WARN (dynamic registration path).

4. Annotation correctness (the adversarial core)

Read references/annotation-correctness.md. Read each callback body and verify it matches the annotation claim:

• readonly: true → callback must not write to the database, the

options table, post / user / term / comment data, the filesystem, cron, or via non-GET HTTP / REST delegates.

• destructive: false → callback must not delete, refund, void,

cancel, or trash.

• idempotent: true → repeated calls with the same input have no

additional effect on the environment (per the idempotent annotation's docblock in class-wp-ability.php). Static catches counter writes and per-call cron schedules; runtime adds a twin-invocation heuristic for visible state changes.

The reference lists common write patterns as a starting set, not a checklist — plugin vocabularies vary, and the agent extends with verbs specific to the plugin under verification.

False positives get suppressed via an inline // verify-ignore: <annotation> -- <reason> comment.

5. Permission roundtrip

Read references/permission-roundtrip.md. Static: classify each permission_callback against the six shapes (preferred Shape A current_user_can(...); FAIL on Shape B-bad WP_REST_Request patterns or Shape E literal true). Runtime: anon and subscriber denied; admin allowed (unless deliberately public). When an audit was provided, cross-check the registered cap against the audit's declared gate.

6. Schema lints

Read references/schema-lints.md. Six small principles applied to each ability's input_schema: object schemas declare additionalProperties; required fields have descriptions; enums non-empty; no $ref; defaults are statically constant (including (object) array()); reference abilities have no required inputs.

Cross-reference ../wp-abilities-api/references/input-schema-gotchas.md for the four runtime gotchas (defaults not injected on the property-level path, pagination key drift, empty() on string IDs, direct vs indirect invocation strictness).

7. Error-code vocabulary

Cross-reference ../wp-abilities-api/references/error-code-vocabulary.md. Inspect each callback's WP_Error returns; non-vocabulary codes → WARN.

Verification

The run produces a structured markdown report at the user-specified path:

---
Last updated: <YYYY-MM-DD HH:MM>
---

# <Plugin> Abilities Verification — <Static|Runtime> Mode

## Status: <PASS|WARN|FAIL>

## Audit doc validation (if provided)

## Static inventory

## Annotation correctness
| Ability | Claim | Result | Evidence |
|---|---|---|---|

## Permission gates

## Schema lints

## Error-code vocabulary

Every ability is OK, WARN, or FAIL. A single FAIL → top-line FAIL; WARNs without FAILs → WARN; otherwise PASS.

Failure modes / debugging

• Env not reachable (runtime) — env-up failed or Docker isn't

running. Re-run wp-project-triage, then fix the env. Don't fall back silently to static without noting it in the report.

• No abilities in source — return a clear "nothing to verify"

report.

• Audit schema mismatch — point at

references/audit-schema-validation.md; don't auto-fix the audit.

• False positive on readonly-writes — see the // verify-ignore

mechanism in references/annotation-correctness.md. Document why each suppression is legitimate.

• Runtime enumeration smaller than static — registration hook

isn't firing. Check init hook timing, activation state, autoloader order.

Escalation

• Recurring legitimate pattern that trips the adversarial check across

multiple plugins → propose adding it to the suppression guidance in annotation-correctness.md. Don't broaden the candidate-pattern list speculatively.

• Audit-schema validator rejects a legitimate audit → the canonical

schema in ../wp-abilities-audit/references/audit-schema.md has evolved. Update references/audit-schema-validation.md to match.

Out of scope

Token-budget measurement is a separate verification axis — an annotation-clean, schema-clean, runtime-passing ability set can still be unshippable if its tools/list form burns through an agent's context budget. That axis is tracked separately. Do not aggregate manual or external measurement into this skill's PASS / FAIL verdict.

Annotation Correctness

The adversarial core of this skill: verify what the annotation claims by reading the callback. A readonly: true ability that actually writes is a security and UX disaster, and unit tests don't catch it because the mock looks just like the real writer.

Why this matters

Agents plan actions on the basis of the annotations they introspect. If an ability is annotated readonly: true, an orchestrator will confidently invoke it in a dry-run, speculative exploration, or multi-agent fan-out without thinking twice — because readonly means "can't break anything".

A readonly: true ability that actually writes is therefore:

1. A security hazard — agents will invoke it in contexts where side effects are forbidden. 2. A UX disaster — the agent's mental model of what happened diverges silently from reality. 3. Undetectable at the annotation layer — the annotation says readonly: true; nothing in the registration forces it to be true.

Unit tests won't catch this class of bug because the mock the test constructs looks just like the real writer. What catches it is reading the execute callback body and comparing what it does against what the annotation says it does.

What each annotation promises

readonly: true prohibits durable writes to user or business state. Read-through cache writes (e.g. set_transient) and observability timestamps (e.g. last_read_at) are acceptable when explicitly annotated with verify-ignore — see the "Suppressing legitimate exceptions" section below. The static check treats unannotated writes as FAILs; annotated ones pass with the reason recorded as evidence.

These overlap but are not redundant: readonly is the strictest; destructive: false is weaker (updates that don't destroy are OK); idempotent is orthogonal (a POST that writes the same row twice is both "writes" and "idempotent").

The Abilities REST run controller operationalizes annotations into HTTP method routing (readonly: true → GET, destructive && idempotent → DELETE, otherwise POST — see WP_REST_Abilities_V1_Run_Controller::validate_request_method()). That mapping is the load-bearing semantic; verify checks that each callback's behavior is consistent with how the routing will treat it.

How to verify

For each ability, locate the execute_callback body (see static-enumeration.md step 4), then:

1. Read the callback end-to-end. Form a model of what it actually does. Don't rely on pattern-matching alone. 2. Compare to the claim. A readonly: true callback that writes anywhere — the database via $wpdb, options / post / user / term / comment writes, filesystem, cron schedules, or non-GET HTTP/REST delegates — FAILs readonly. A destructive: false callback that deletes, refunds, voids, cancels, or trashes FAILs destructive. An idempotent: true callback whose environmental effect accumulates per call (counters, append-only logs, per-call cron schedules) FAILs idempotent. 3. Record evidence. Cite file + line of the offending pattern so a reviewer can jump straight to it.

Use grep or ripgrep to surface candidates. Common writes worth looking for:

$wpdb->update / insert / delete / replace
update_option / add_option / delete_option
wp_insert_post / wp_update_post / wp_delete_post
update_post_meta / update_user_meta / update_term_meta
->save / ->delete / ->set_status / ->add_*
wp_remote_post / wp_remote_delete
file_put_contents / wp_upload_bits / unlink / rename
wp_schedule_event / wp_schedule_single_event

Treat the list as a starting set, not a checklist. Plugin vocabularies vary — domain-specific verbs (->markAsPaid, ->commit, ->refund) and framework patterns (Doctrine ->persist, queue ->dispatch) won't appear above. Once you've grepped for candidates, read the callback to confirm whether each hit is actually a write and whether it contradicts the annotation in context.

Known blind spots

Static reading + grep can't reach every write. A static-mode PASS means "no obvious-shape violations," not "verified write-free."

For high-stakes plugins, run runtime mode (see runtime-harness.md) before landing — it catches some blind spots via twin-invocation diff and live state inspection.

Suppressing legitimate exceptions

When a pattern that looks like a write is semantically a read (e.g. populating a read-through cache via set_transient, updating a last_read_at timestamp for tracking, diagnostic logging), suppress with an inline comment on the offending line:

// verify-ignore: readonly -- writes to read-through cache; semantically a read.
set_transient( $cache_key, $data, HOUR_IN_SECONDS );

Format: // verify-ignore: <annotation> -- <reason>. Legal annotation names: readonly, destructive, idempotent, all. Narrower is better than all.

Runtime check complement

For idempotent: true abilities, runtime mode adds a heuristic: invoke twice with the same input and compare. See runtime-harness.md Check 6. Differing returns are a signal to inspect, not a verdict — under core's definition, the question is whether the environment changed, not whether the return value matches. A response that embeds a per-call timestamp / nonce / random ID is fine; a response that reflects a counter that grew between calls is not.

Report format

Each finding gets one row in the run's "Annotation correctness" table:

| Ability | Claim | Result | Evidence |
|---|---|---|---|
| myplugin/get-things | readonly=true | OK | callback reads only |
| myplugin/get-things-with-counts | readonly=true | FAIL | `src/Abilities/Things.php:142`: `$wpdb->update( $table, ... )` |
| myplugin/submit-thing | destructive=false | OK | no destructive patterns |
| myplugin/submit-thing | idempotent=false | OK | check only applies when idempotent=true; false annotation acknowledged |

The evidence column MUST cite file + line so a reviewer can jump straight to the issue.

Audit Schema Validation

How wp-abilities-verify validates an audit document produced by wp-abilities-audit. The canonical schema (field tables, types, invariants, known limitations) lives in ../../wp-abilities-audit/references/audit-schema.md — this reference covers only the validation procedure: how to extract the YAML, what checks to run in what order, and how to report results.

If a field type or shape question is not answered here, look in the canonical schema. Do NOT duplicate field tables in this file — the canonical is the single source of truth.

Why verify owns the validator

Verify fails fast on a malformed audit so the rest of its procedure can assume well-formed input. Audit produces; verify validates the production. Co-locating the validator with verify keeps the "validate audit" step in the same procedure as "validate registered abilities" and lets a single run produce one consolidated report.

Step 1 — extract the YAML

The audit doc is a markdown file with a single fenced `yaml block containing the structured fields:

# Scan for the ```yaml fence and capture until the closing ``` fence.
awk '/^```yaml$/{f=1;next} /^```$/{f=0} f' <audit-doc.md> > /tmp/audit.yaml

If the audit has multiple YAML blocks (it shouldn't, but defensively), take the first one with proposed_abilities as a top-level key.

Parse with any YAML library — js-yaml from Node, yaml (Python), or yq from the command line. None of the canonical fields require non-standard YAML features (no anchors, no aliases), so a plain yaml.load is sufficient.

Step 2 — validate against the canonical schema

Apply the field-shape rules defined in ../../wp-abilities-audit/references/audit-schema.md. Specifically:

1. Every required top-level field is present and non-empty (see "Top-level fields" in the canonical). 2. capability_gate matches one of the legal shapes (single string, {read, write} object, or — with WARN per the canonical's "Known limitations" — the legacy slash-separated string). 3. Every entry in proposed_abilities has every required per-ability field with the right type (see "proposed_abilities" in the canonical). 4. Each ability's annotations block has all three booleans (readonly, destructive, idempotent) as actual booleans — string "true" / "false" is FAIL (indicates a quoting bug). 5. Each ability's backing is either an object with the canonical fields or null; null is WARN, not FAIL (it's intentional gap output).

Missing required field → FAIL. Wrong type → FAIL. Legacy capability_gate slash-string → WARN.

Step 3 — whole-audit invariants

Run these after per-field validation passes:

Exactly 0 or 1 abilities with reference_ability: true

Count abilities where reference_ability is true. More than 1 → FAIL (the schema permits at most one reference; multiple are ambiguous for implementers picking a starting point).

const refCount = audit.proposed_abilities.filter(a => a.reference_ability === true).length;
if (refCount > 1) fail("multiple abilities claim reference_ability: true");

Every backing: null ability appears in surfaced_gaps

Per the canonical's "Known limitations": a null backing is intentional gap output and MUST be paired with a matching surfaced_gaps entry.

const gapNames = new Set((audit.surfaced_gaps || []).map(g => g.name));
for (const ability of audit.proposed_abilities) {
  if (ability.backing === null && !gapNames.has(ability.name)) {
    fail(`ability ${ability.name} has backing: null but is missing from surfaced_gaps`);
  }
}

excluded_from_mvp and surfaced_gaps may be empty

Both are optional; empty arrays are legal. Missing entirely → WARN (schema expects them, even if empty).

Step 4 — emit the report section

Each check goes into the "Audit doc validation" section of the run's final report:

## Audit doc validation

| Check | Result | Detail |
|---|---|---|
| Top-level required fields | OK | All 7 required fields present |
| `capability_gate` shape | OK | string (single-cap) |
| Per-ability fields | WARN | 1 ability has `backing: null` (intentional) |
| `reference_ability` uniqueness | OK | 1 ability marked |
| `surfaced_gaps` consistency | OK | all `backing: null` entries present |

A single FAIL in this section makes the whole run FAIL; verify cannot meaningfully continue without a trustworthy audit. WARN entries don't block the rest of the procedure.

The procedure is manual-but-deterministic: follow the steps above in order, emit the report section, and fail fast on any missing required field. A future contribution may add a deterministic CLI helper that extracts the YAML fence and applies the rules end-to-end; until that exists, the steps above are the contract.

Escalation

If the validator rejects an audit that's actually well-formed, the canonical schema in ../../wp-abilities-audit/references/audit-schema.md has evolved. Update this file's procedure to match (likely adding a new invariant or relaxing a field rule). Don't loosen the validation in isolation — the canonical schema is the contract; this file is the enforcer.

Permission Roundtrip

Verify that the registered permission_callback on every ability actually gates on a real capability — statically (by source inspection) and, in runtime mode, by exercising the gate against unauthenticated, subscriber, and admin contexts.

Background — what permission_callback actually receives

When an ability is invoked, the registered permission_callback is called through WP_Ability::check_permissions( $input ), which dispatches to WP_Ability::invoke_callback( $callback, $input ) (both defined in class-wp-ability.php in WordPress core).

invoke_callback's contract:

• If the ability declares a non-empty input_schema, the callback is

invoked with one positional argument: the validated $input value (whatever the schema's root type produced — array, string, integer, boolean, etc.).

• If input_schema is empty or absent, the callback is invoked **with

no arguments**.

In particular, the callback never receives a `WP_REST_Request`, even when the ability is reached via the REST bridge. The bridge unwraps the request, runs schema validation, and passes the validated value down. Permission callbacks built around WP_REST_Request patterns (e.g. $request->get_method()) cannot work as-is when copied from a REST controller — flag any such usage as a static FAIL.

Static check — classify each callback's shape

Read each ability's permission_callback body and classify:

Record per ability: (shape, resolved_cap). Any Shape B-bad or Shape E → static FAIL. Shapes C and F → WARN. Shapes A, B, D → OK.

Runtime check

Exercise the gate against three user contexts using WP_Ability::check_permissions( $input ).

check_permissions() accepts an optional input value and passes it through to the registered permission_callback. Shape A callbacks (return current_user_can('cap')) don't read it. Shape B callbacks that branch on $input (the smell the static check flags) need a representative value to exercise the real gate; otherwise they receive null and the roundtrip result is misleading. The snippet below passes array() — the minimal safe input for object-typed schemas. For abilities with a non-object root schema, substitute a representative value of the declared root type.

<env-cli> wp --user=admin eval '
$ability = wp_get_ability( "<plugin>/<ability-name>" );
if ( ! $ability ) {
    echo "ability not registered" . PHP_EOL;
    exit( 1 );
}

$input   = array(); // representative input; substitute for non-object root schemas.
$results = array();

// Unauthenticated.
wp_set_current_user( 0 );
$results["anon"] = $ability->check_permissions( $input );

// Subscriber (create a fresh user).
$sub_login = "verify_sub_" . time();
$sub_id    = wp_create_user( $sub_login, "x", $sub_login . "@example.com" );
if ( ! is_wp_error( $sub_id ) ) {
    $sub_user = get_user_by( "id", $sub_id );
    $sub_user->set_role( "subscriber" );
    wp_set_current_user( $sub_id );
    $results["subscriber"] = $ability->check_permissions( $input );
}

// Admin.
wp_set_current_user( 1 );
$results["admin"] = $ability->check_permissions( $input );

foreach ( $results as $context => $result ) {
    if ( true === $result ) {
        $printable = "true";
    } elseif ( is_wp_error( $result ) ) {
        $printable = "WP_Error(" . $result->get_error_code() . ")";
    } else {
        $printable = var_export( $result, true );
    }
    echo $context . "=" . $printable . PHP_EOL;
}

// Cleanup the test subscriber so repeated harness runs don't accumulate users
// on shared dev environments. Already running as admin (line above), so the
// caller has the delete_users capability. On multisite, wp_delete_user() only
// removes the user from the current site's membership — wpmu_delete_user() in
// wp-admin/includes/ms.php is the network-wide delete.
if ( isset( $sub_id ) && ! is_wp_error( $sub_id ) ) {
    if ( is_multisite() ) {
        require_once ABSPATH . "wp-admin/includes/ms.php";
        wpmu_delete_user( $sub_id );
    } else {
        require_once ABSPATH . "wp-admin/includes/user.php";
        wp_delete_user( $sub_id );
    }
}
'

Notes on interpretation:

• check_permissions() returns bool|WP_Error. Treat true as

allowed; treat false or any WP_Error as denied.

• A WP_Error with code ability_invalid_permission_callback means

the registration didn't supply a valid callable — hard FAIL.

• A WP_Error with code ability_callback_exception means the

callback threw — hard FAIL; capture the underlying message.

Expected for a standard (non-public) ability:

anon=false
subscriber=false
admin=true

Expected for a deliberate public ability (Shape C):

anon=true
subscriber=true
admin=true

Any deviation → FAIL. Common causes: cap reference an admin doesn't hold, callback bug, or permission too permissive (Shape E or F when it should have been Shape A).

Audit cross-check

If an audit doc was provided, the audit's capability_gate (or each ability's permission.resolves_to) declares what the gate should be. Compare:

• Audit and registration resolve to the same cap → OK.
• Audit and registration disagree → FAIL. Either the audit is wrong or

the registration drifted.

• Audit declares a compound {read, write} gate, registration uses

Shape B with both caps → OK.

• Audit declares a compound gate, registration uses Shape A (single

cap) → FAIL. Write paths would inherit the read gate (or vice versa), under- or over-authorizing.

See ../../wp-abilities-audit/references/capability-gate-tracing.md for the tracing mechanics; this skill re-derives the same trace and diffs.

Output format

## Permission gates

| Ability | Shape | Resolved cap(s) | anon | subscriber | admin | Audit match |
|---|---|---|---|---|---|---|
| <ability> | A | manage_options | false | false | true | OK |
| <ability> | B | edit_posts (read), delete_posts (destructive) | false | false | true | OK |
| <ability> | C | __return_true (public) | true | true | true | WARN |
| <ability> | E | (literal true) | true | true | true | FAIL |

Static-only mode caveats

Without runtime mode, only the source-inspection columns are populated:

Roundtrip columns are omitted rather than guessed. Flag in the section header: Permission gates (static inspection only).

Runtime Harness

The runtime-mode procedure. Everything static-mode does, plus six canonical checks that need a live wp_get_abilities() call.

Static mode catches structural problems (annotation lies, schema lint failures, audit mismatches). Runtime mode catches the class of bug that only surfaces against a booted WordPress: missing constructor arguments, bootstrap-ordering issues, schema-validator paths, capability-roundtrip failures, and idempotency violations at the response-byte level.

Harness rule: stop on first fatal

If any step below produces a PHP fatal, STOP. Later steps won't produce meaningful output. Capture the failure, escalate to the plugin's implementer to fix, then re-run from step 1.

A WP_Error return is acceptable — any <plugin>_* or upstream-prefixed error means the execute callback handled the error path gracefully. Only PHP fatals block.

Step 0 — identify the env-up command

Read the plugin's AGENTS.md for the canonical env bring-up command. Do NOT assume npm run wp-env start works for every plugin. Common patterns seen in real plugin trees:

• npm run wp-env start — projects using @wordpress/env with a checked-in

.wp-env.json.

• npx wp-env start — projects using @wordpress/env without a custom

npm script wrapper.

• composer install && composer test-php --setup-only — package-local

test bootstraps that don't run a full WordPress install.

• docker-compose up -d — plugin-specific dev Docker stacks.

Plugin families with their own dev tooling will have their own bring-up command in AGENTS.md; follow it as documented.

If AGENTS.md doesn't document it, ask the user rather than guessing. Record the env-up command + the corresponding wp-cli invocation (e.g. npx wp-env run cli wp) and use them uniformly for the rest of the harness.

In this file, <env-cli> is shorthand for whatever wp-cli invocation the plugin uses.

Step 1 — bring up the env and sanity-check

<env-up-command>
<env-cli> wp core version
<env-cli> wp plugin list --status=active
<env-cli> wp eval 'var_export( function_exists( "wp_get_abilities" ) );'

Confirm:

• WordPress version >= 6.9 (Abilities API available in core).
• The plugin being verified is active.
• wp_get_abilities exists (true if WP >= 6.9, else the Abilities API

feature plugin/package must be active).

Any "no" answer halts the harness.

Check 1 — ability names match source-expected list

Enumerate runtime abilities and diff against the static inventory from static-enumeration.md:

<env-cli> wp --user=admin eval '
$names = array_filter(
    array_keys( (array) wp_get_abilities() ),
    function ( $n ) {
        return strpos( $n, "<plugin-slug>/" ) === 0;
    }
);
sort( $names );
echo "count=" . count( $names ) . PHP_EOL;
echo implode( PHP_EOL, $names ) . PHP_EOL;
'

Compare against the static inventory:

• Source contains ability, runtime missing → FAIL. Registration hook

isn't firing; check init hook timing and plugin activation.

• Runtime contains ability, source missing → WARN. Dynamic registration

path the enumerator couldn't follow. Document but don't block.

• Counts match → OK.

Check 2 — annotations read back as declared

<env-cli> wp --user=admin eval '
$names = array_filter(
    array_keys( (array) wp_get_abilities() ),
    function ( $n ) {
        return strpos( $n, "<plugin-slug>/" ) === 0;
    }
);
sort( $names );
foreach ( $names as $name ) {
    $a = wp_get_ability( $name );
    $m = $a->get_meta();
    printf(
        "%s | readonly=%s | destructive=%s | idempotent=%s | category=%s" . PHP_EOL,
        $name,
        var_export( $m["annotations"]["readonly"], true ),
        var_export( $m["annotations"]["destructive"], true ),
        var_export( $m["annotations"]["idempotent"], true ),
        $a->get_category()
    );
}
'

Cross-reference each annotation against the audit's declared value (if an audit was provided) AND against the static inventory's declared value. Mismatch on either axis → FAIL.

This check complements the static adversarial check from annotation-correctness.md: static checks the callback's actual behavior; runtime checks what the registration hook resolved to at boot time. Both must agree for the annotations to be trustworthy.

Check 3 — each read ability's execute() behaves as the contract claims

Two verification levels. The smoke level catches bootstrap and gross-error regressions; the high-confidence level is the one that actually exercises the ability against the data shape it will see in production. Run both when the audit doc provides seed_data_needs; run the smoke level alone when it doesn't.

Level 1 — smoke execution (synthetic inputs)

Confirm each read returns OK or a vocabulary WP_Error. Catches PHP fatals, un-bootstrapped services, registration failures.

<env-cli> wp --user=admin eval '
$reads = array(
    "<plugin-slug>/<read-ability-1>",
    "<plugin-slug>/<read-ability-2>",
    // ...
);
foreach ( $reads as $name ) {
    $r = wp_get_ability( $name )->execute();
    echo $name . ": " . ( is_wp_error( $r ) ? "WP_Error(" . $r->get_error_code() . ")" : "OK" ) . PHP_EOL;
}
'

Acceptable outcomes:

• OK — the ability returned without error.
• WP_Error(<plugin>_not_initialized) — bootstrap guard fired (e.g.

un-bootstrapped service). Happy error path.

• WP_Error(<plugin>_<resource>_data_unavailable) — transient backend

error. Acceptable.

• WP_Error(<upstream_code>) — upstream third-party error bubbled

through. Document, don't block.

Unacceptable:

• PHP fatal → stop the harness.
• WP_Error with a non-vocabulary code → WARN. Cross-reference

../../wp-abilities-api/references/error-code-vocabulary.md.

Abilities with required input get invoked separately with a synthetic-but-plausible value:

<env-cli> wp --user=admin eval '
$r = wp_get_ability( "<plugin>/<ability-with-required-input>" )
    ->execute( array( "<field>" => "<synthetic-id>" ) );
echo "<ability>: " . ( is_wp_error( $r ) ? "WP_Error(" . $r->get_error_code() . ")" : "OK" ) . PHP_EOL;
'

A synthetic ID on a fresh install typically triggers <plugin>_<resource>_data_unavailable or an upstream-equivalent code. Both are acceptable for Level 1 — they mean "ability dispatched cleanly, the backing reported no data," which is the smoke signal.

Level 2 — high-confidence verification (representative seeded data)

Synthetic inputs catch fatals and gross errors. They do NOT catch wrong IDs returning the wrong record, cached sentinel values being served instead of fresh data, permission gates appearing correct against a synthetic ID but not against a real one, filtered labels diverging from the unfiltered claim, missing capabilities surfacing only when a real record is in scope, or curated output drift (a new field added to the controller that the ability inherits and now leaks). These are the failure modes that matter and they only fall out when the ability runs against the data shape it will see in production.

For each ability whose audit entry declares a non-null seed_data_needs, seed the environment per that field, then call the ability with representative inputs (a real id, a real slug — not a synthetic placeholder), and assert the output shape AND the privacy contract:

<env-cli> wp --user=admin eval '
// Seed once at the top of the harness run, per the audit doc:
// e.g. wp post create, wp user create, wp option update, factory helpers.

$ability = wp_get_ability( "<plugin>/<read-ability>" );
$result  = $ability->execute( array( "<field>" => "<real-id-from-the-seeded-data>" ) );

if ( is_wp_error( $result ) ) {
    echo "FAIL: " . $result->get_error_code() . PHP_EOL;
    exit;
}

// Assert the documented output shape — keys present, types correct.
$expected_keys = array( "id", "label", "status" );  // from the ability schema or audit return_type
foreach ( $expected_keys as $k ) {
    if ( ! array_key_exists( $k, (array) $result ) ) {
        echo "FAIL: missing output key " . $k . PHP_EOL;
        exit;
    }
}

// Assert the privacy contract: sensitive fields the contract does NOT
// promise must not appear (e.g. full PAN, full bank account, raw token,
// internal-only debug fields).
$forbidden_keys = array( "full_pan", "card_number", "bank_account_number", "iban" );
foreach ( $forbidden_keys as $k ) {
    if ( array_key_exists( $k, (array) $result ) ) {
        echo "FAIL: privacy leak — " . $k . " present in output." . PHP_EOL;
        exit;
    }
}

echo "PASS: shape OK, privacy OK." . PHP_EOL;
'

Adapt $expected_keys and $forbidden_keys to each ability. The audit doc's return_type field hints at the shape; the privacy keys depend on the plugin's domain. For payments-family plugins, full PANs / bank numbers / raw tokens are the canonical forbidden set; other families substitute appropriately. The plugin's in-tree contract tests (the overlay's test-the-public-contract.md for WooCommerce extensions) are the durable home for these assertions on every CI run — this Level 2 check is the one-off harness run that produces the PR artifact.

When the audit doc declares seed_data_needs: null, Level 2 is not yet runnable: the auditor has not identified the seed shape. The harness reports LEVEL 2: pending (seed_data_needs is null — ask the implementer) and proceeds. When seed_data_needs is a string, the harness operator seeds per that description before running the representative-input block above.

Check 4 — each write ability's missing-input returns ability_invalid_input or <plugin>_missing_<field>

<env-cli> wp --user=admin eval '
$a = wp_get_ability( "<plugin>/<write-ability>" );

$r1 = $a->execute( array() );
echo "missing: " . ( is_wp_error( $r1 ) ? "WP_Error(" . $r1->get_error_code() . ")" : "UNEXPECTED_OK" ) . PHP_EOL;

$r2 = $a->execute( array( "<required_field>" => 123 ) );
echo "non-string: " . ( is_wp_error( $r2 ) ? "WP_Error(" . $r2->get_error_code() . ")" : "UNEXPECTED_OK" ) . PHP_EOL;
'

Acceptable codes, per ../../wp-abilities-api/references/error-code-vocabulary.md:

• ability_invalid_input — the Abilities API's schema validator fired

first (normal REST-bridge path; emitted by WP_Ability::validate_input() in core).

• <plugin>_missing_<field> — the execute callback's own guard fired

(direct-invocation path).

• <plugin>_invalid_<field> — same, for the wrong-type case.

UNEXPECTED_OK → FAIL. Validation is missing; the ability accepted no-input and proceeded to do something it shouldn't have.

Check 5 — permission gate denies subscriber, allows admin

<env-cli> wp --user=admin eval '
$ability = wp_get_ability( "<plugin>/<any-ability>" );
$input   = array(); // representative input; substitute for non-object root schemas.

// Admin path.
wp_set_current_user( 1 );
$admin_result = $ability->check_permissions( $input );

// Subscriber path.
$sub_login = "verify_sub_" . time();
$sub_id    = wp_create_user( $sub_login, "x", $sub_login . "@example.com" );
if ( ! is_wp_error( $sub_id ) ) {
    $user = get_user_by( "id", $sub_id );
    $user->set_role( "subscriber" );
    wp_set_current_user( $sub_id );
    $sub_result = $ability->check_permissions( $input );
} else {
    $sub_result = $sub_id; // surface the create_user error in the report
}

foreach ( array( "admin" => $admin_result, "subscriber" => $sub_result ) as $context => $r ) {
    if ( true === $r ) {
        $printable = "true";
    } elseif ( is_wp_error( $r ) ) {
        $printable = "WP_Error(" . $r->get_error_code() . ")";
    } else {
        $printable = var_export( $r, true );
    }
    echo $context . ": " . $printable . PHP_EOL;
}

// Cleanup the test subscriber so repeated harness runs don't accumulate users
// on shared dev environments. Switch back to admin first since we last ran as
// the subscriber and that role doesn't hold delete_users. On multisite,
// wp_delete_user() only removes the user from the current site's membership —
// wpmu_delete_user() in wp-admin/includes/ms.php is the network-wide delete.
if ( isset( $sub_id ) && ! is_wp_error( $sub_id ) ) {
    wp_set_current_user( 1 );
    if ( is_multisite() ) {
        require_once ABSPATH . "wp-admin/includes/ms.php";
        wpmu_delete_user( $sub_id );
    } else {
        require_once ABSPATH . "wp-admin/includes/user.php";
        wp_delete_user( $sub_id );
    }
}
'

Expected:

admin: true
subscriber: false

Any inversion is a bug in the registered permission_callback. See permission-roundtrip.md for the deeper cross-checks (audit's declared capability → registered callback → resolved current_user_can(...)).

Public abilities (deliberately ungated) expect subscriber: true AND admin: true. Verify that the ability's permission_callback is '__return_true' in source before accepting this outcome.

check_permissions() returns bool|WP_Error per WP_Ability::check_permissions() in WordPress core. A WP_Error with code ability_invalid_permission_callback means the registration didn't supply a valid callable — a hard FAIL. A WP_Error with code ability_callback_exception means the callback threw — also a hard FAIL.

Check 6 — twin-invocation heuristic for idempotent abilities

Only apply this to abilities annotated idempotent: true whose execute() returned without error in Check 3. Per annotation-correctness.md step "Runtime check complement", this is a heuristic: idempotent in core means "no additional effect on the environment" (per the idempotent annotation's docblock in class-wp-ability.php), not "byte-identical return values."

<env-cli> wp --user=admin eval '
$a  = wp_get_ability( "<plugin>/<idempotent-ability>" );
$r1 = $a->execute();
$r2 = $a->execute();

if ( is_wp_error( $r1 ) || is_wp_error( $r2 ) ) {
    echo "skipped: one or both invocations returned WP_Error" . PHP_EOL;
    if ( is_wp_error( $r1 ) ) { echo "r1=" . $r1->get_error_code() . PHP_EOL; }
    if ( is_wp_error( $r2 ) ) { echo "r2=" . $r2->get_error_code() . PHP_EOL; }
} else {
    $h1 = md5( serialize( $r1 ) );
    $h2 = md5( serialize( $r2 ) );
    echo "match=" . var_export( $h1 === $h2, true ) . PHP_EOL;
    echo "h1=" . $h1 . PHP_EOL;
    echo "h2=" . $h2 . PHP_EOL;
}
'

Interpretation:

• match=true → cheap PASS. Same input produced the same response, and

any environmental writes (write abilities) were the same on both calls.

• match=false → inspect what changed before deciding:
• Response embeds a per-call timestamp, nonce, or random ID →

environment unchanged. Still idempotent under core's reading. Optionally remove the field if the agent doesn't need it; the annotation stays idempotent: true.

• Response reflects a counter or sequence that grew between calls →

real environmental change. FAIL: drop the idempotent: true annotation or fix the underlying write to be input-determined.

For ambiguous cases (response varies but no obvious counter), supplement with a state diff: snapshot a representative table or option before call 1, snapshot after call 2, diff. If state changed by more than the input writes would explain, the ability is non-idempotent.

Output format

The runtime harness writes a dedicated section in the run report:

## Runtime harness

**Env:** wp-env (Docker), WordPress 6.9, <plugin> <version>
**Captured:** <YYYY-MM-DD HH:MM>

### Check 1 — enumeration

count=7 (expected 7 from static inventory)
<sorted list>

### Check 2 — annotations

| Ability | readonly | destructive | idempotent | Matches source? |
|---|---|---|---|---|

### Check 3 — read execution

| Ability | Result |
|---|---|

### Check 4 — write input validation

| Ability | Missing-input code | Wrong-type code |
|---|---|---|

### Check 5 — permission gate

| Ability | admin | subscriber | Expected |
|---|---|---|---|

### Check 6 — idempotency

| Ability | match | h1 | h2 |
|---|---|---|---|

### Notes / surprises

<Anything unexpected that didn't block.>

When the harness catches a bug

Observed pattern:

1. Harness surfaces a PHP fatal (e.g. ArgumentCountError: Too few arguments to function <controller>::__construct). 2. Implementer fixes the bug in a focused commit. 3. Harness re-runs; write the post-fix output as the headline status. 4. Preserve the pre-fix trace in the report under a "Pre-fix status" section so reviewers can see what verify caught.

This is the highest-leverage signal the runtime harness produces: bootstrap-ordering and missing-dependency bugs that pass static review because the source declares the right shape — they only manifest when the registration runs against a booted WordPress.

Schema Lints

Static lints against an ability's input_schema. Schema hygiene is about agent legibility: orchestrating agents read the schema to figure out how to call the ability. A schema that's hard to parse, ambiguous, or misleading wastes turns even when the ability itself works.

These lints are six small principles. Apply them by reading the schema, not by mechanically grepping — most plugins use enough formatting variety that grep recipes drift.

Lint 1 — additionalProperties: false for object schemas

For top-level 'type' => 'object' schemas, declare 'additionalProperties' => false unless you deliberately accept extras. Without this, an agent passing a typo (par_page instead of per_page) gets accepted silently and falls through to the backing, which ignores the unknown key.

• additionalProperties: false declared → OK.
• additionalProperties: true declared → WARN, unless the schema is

for genuinely free-form metadata (payment custom fields, form free-text); document the reason inline.

• Not declared on an object schema → WARN.
• Non-object root (string with enum, integer, etc.) → N/A. The lint

applies only to objects.

Lint 2 — every required field has a non-empty description

For each entry in required, the matching properties entry must declare a non-empty description. Required fields are where agents most need guidance; an opaque required key forces the agent to guess from the field name alone. Empty / missing → FAIL.

Optional-field descriptions are nice-to-have — absence is WARN.

Lint 3 — enums are non-empty

'enum' => [] accepts no values, rejecting every input. Almost always a bug. → FAIL.

A single-value enum ('enum' => [ 'pending' ]) is legal but unusual; WARN and prompt for review — often a copy-paste that lost the other values.

Lint 4 — no $ref

Agents read the schema via REST introspection. A $ref forces the agent to follow a reference to see the field shape — wastes a turn and often breaks because the referenced schema isn't in the same document. Inline the shape instead.

Any '$ref' in the schema → FAIL.

Lint 5 — defaults are statically constant

Each 'default' value must evaluate to the same shape on every call:

• Scalar literals — true, false, integer, float, quoted string,

null → OK.

• Empty or all-literal arrays — [], array(), [ 'a', 'b' ] → OK.
• Literal cast to an empty object — (object) array(), (object) []

→ OK. This is the recommended top-level default for zero-arg-allowed abilities; see ../../wp-abilities-api/references/input-schema-gotchas.md §4.

• new stdClass() with no arguments → OK.
• A function call (gmdate('c'), wp_generate_uuid4(), time()),

variable reference, or other computed expression → FAIL.

The principle: defaults that vary per call are both non-deterministic and surprising to agents that expect defaults to be static.

Lint 6 — reference_ability: true implies no required inputs

If an audit doc is provided and an ability has reference_ability: true, its input_schema.required array must be empty or absent. The reference ability is the smallest, safest bootstrap call an implementer lands first; it must work with execute([]). Required inputs on the reference ability → FAIL.

(No audit provided → this lint is skipped — no reference ability is declared.)

Cross-reference: gotchas 1-3 (callback hardening) and gotcha 4 (structural default)

Static lints catch shape; the four runtime gotchas in ../../wp-abilities-api/references/input-schema-gotchas.md split into two kinds.

Gotchas 1-3 need defensive code in the execute callback — array_key_exists instead of isset-only for property defaults, pagination key translation, ID validation that accepts "0". These are runtime behaviors the callback itself must handle; static schema lints can't enforce them.

Gotcha 4 — the direct vs indirect invocation strictness — is what motivates the (object) array() top-level default that Lint 5 explicitly accepts. This one IS structural and Lint 5 carries the enforcement.

Output format

## Schema lints

| Ability | Lint | Result | Detail |
|---|---|---|---|
| <ability> | additionalProperties (object schemas) | WARN | not declared on object schema |
| <ability> | required-field descriptions | OK | 3/3 required fields documented |
| <ability> | enum non-empty | OK | no enums |
| <ability> | no $ref | OK | inline |
| <ability> | static defaults | FAIL | `created_at` uses `gmdate('c')` |
| <ability> | reference_ability implies no required | N/A | not reference ability |

A FAIL on any lint flips that ability to FAIL in the run summary. WARNs surface but don't block.

Static Enumeration

Enumerate a plugin's abilities from source, with no running environment. Static enumeration is necessarily best-effort — PHP's dynamism (variable indirection, runtime-conditional registration) means a complete inventory only comes from a live wp_get_abilities() call. When static and runtime inventories diverge, trust runtime; static drives the diff so the reviewer knows where to look.

Typical registration shape

wp_register_ability(
    '<plugin-slug>/<ability-name>',
    array(
        'label'               => __( '...', '<text-domain>' ),
        'description'         => __( '...', '<text-domain>' ),
        'category'            => '<category-slug>',
        'input_schema'        => array( /* ... */ ),
        'execute_callback'    => array( self::class, 'execute_my_ability' ),
        'permission_callback' => array( self::class, 'check_permission' ),
        'meta'                => array(
            'annotations' => array(
                'readonly'    => true,
                'destructive' => false,
                'idempotent'  => true,
            ),
            'show_in_rest' => true,
        ),
    )
);

Step 1 — find every registration call

grep -rn --include='*.php' 'wp_register_ability\s*(' <plugin-root>/

Zero hits → return a clear "no abilities registered" report per SKILL.md "Failure modes." Don't fabricate an empty inventory.

Step 2 — extract each ability name

The first argument is the ability name — usually a literal string. Real-world formatting splits the call across lines (the example above is itself multi-line), so single-line regexes miss common cases. Use a multi-line tool:

# ripgrep with --multiline + PCRE2 captures the name regardless of line break:
rg --multiline --pcre2 --type=php -n \
   "wp_register_ability\s*\(\s*['\"]([^'\"]+)['\"]" <plugin-root>/

# Fallbacks: pcregrep -M, perl -0777.

If the first argument is a variable or constant (wp_register_ability( $name, ... ), wp_register_ability( MyPlugin\NAME, ... )), trace it: a recent assignment in the same function or a class constant usually resolves; a name computed in a loop won't, in which case flag the limitation and recommend runtime mode for authoritative enumeration.

Step 3 — extract the annotation block

Annotations live at meta.annotations.{readonly,destructive,idempotent}. For each registration, read the array literal forward until the matching close. Common shapes:

• Multi-line literal (most common).
• Short-form one-liner.
• Helper method ('annotations' => self::annotations_for_read()) —

resolve the helper if it returns a literal; otherwise mark <unresolved> and run the adversarial check against the callback alone.

• Class constant ('annotations' => self::READONLY_ANNOTATIONS) —

resolve the constant.

Record ability_name → declared_annotations.

Step 4 — follow execute_callback to its body

execute_callback is one of:

'execute_callback' => array( self::class, 'execute_get_things' ),
'execute_callback' => array( My_Class::class, 'execute_get_things' ),
'execute_callback' => array( $this, 'execute_get_things' ),
'execute_callback' => 'my_plugin_execute_get_things',  // top-level function
'execute_callback' => function ( $input ) { /* ... */ },  // closure

Resolve the reference to its source location: file + start line + end line. The annotation-correctness, schema-lint, and permission checks all operate on that byte range.

Limits of static enumeration

Cases where the inventory is incomplete or ambiguous:

• Variable-indirected names (foreach over a slug list).
• Variable-indirected annotations (built from config).
• Conditional registration (if ( feature_enabled() )).
• Variable-indirected callbacks (array( $this, $callback_name )).

Record each in the report's "Static enumeration limitations" section and recommend a runtime-mode rerun for the authoritative inventory.

Output format

## Static inventory

Found <N> ability registrations across <M> files:

| Ability | Source file | Registration line | Callback file | Callback lines |
|---|---|---|---|---|
| myplugin/get-foo | src/Abilities.php | 42 | src/Abilities.php | 102-134 |
| myplugin/submit-bar | src/Abilities.php | 68 | src/Services/Bar.php | 58-91 |

### Limitations

- <ability>: annotations built dynamically in `annotations_for_read()`;
  recommend runtime mode for annotation cross-check.