Dependency Verification

A package name the model produced is a claim, not a fact. The

registry is the fact. Verify before you install.

Dependency Verification

Overview

Code-generating language models recommend packages that do not exist at a measured rate of 5.2% (commercial models) to 21.7% (open models) across 576,000 samples (Spracklen et al. 2024, arXiv 2406.10279). Worse, 58% of hallucinated names recur across reruns, so an attacker can predict them, register the empty name, and ship malware. This is "slopsquatting." A proof-of-concept package (huggingface-cli) drew over 30,000 downloads after being registered against a commonly hallucinated name. Package hallucination is also inversely correlated with coding-benchmark score, so a better model does not make this go away.

The defense is cheap: confirm the name exists in its registry before installing or recommending it. This skill defines that check and is enforced by the guard_package_hallucination.py PreToolUse hook.

When to Use

Apply before any of these:

• Running pip install, uv add, npm install, pnpm add,

yarn add, cargo add, poetry add, or pdm add.

• Writing a dependency into pyproject.toml, requirements.txt,

package.json, or Cargo.toml.

• Recommending a package to the user in prose.

The Two Signals

A package fails verification on either signal:

1. Nonexistent: the name is absent from its registry. This is a likely hallucination. Do not install it. Search for the correct name or confirm the package was renamed or removed. 2. Typosquat / slopsquat: the name is one or two edits from a popular package (for example reqeusts versus requests). This is either a typo or a deliberate impersonation. Confirm the exact name you intend before proceeding.

A name that is unknown to the bundled popular-package set but present in the registry passes. A name that cannot be checked because the registry is unreachable is reported as unverified, never blocked: the guard does not fail closed on a network error.

A Confidence Signal, Not a Gate

Registry existence is the pass/fail check. Real-world usage is a separate, softer signal that builds confidence on top of it. Once a package clears the two signals above, cross-checking that it is actually used by other projects raises your confidence that the name is the established one rather than a freshly-registered impostor that happens to exist.

Useful confidence signals, none of them blocking:

• GitHub dependents or code search hits for the exact package name

(an established package is imported across many repositories).

• Repository stars, recent commit activity, and release cadence on

the package's source.

• Download counts and publish history on the registry (a popular

name registered yesterday is a red flag; a modest name with years of releases is reassuring).

Treat low usage as a prompt to look closer, never as a reason to reject on its own. New, niche, internal, and private packages are legitimately low-usage, so a missing GitHub footprint must not block an install the registry already confirmed. Use this signal to build confidence and to disambiguate between two similarly-named packages, not to gate.

Verification Procedure

1. Extract the exact package names the command or manifest edit would fetch (strip version specifiers and flags). 2. For each name, confirm existence in the registry. See registry-checks.md for the per-ecosystem endpoints and the offline-degradation rule. 3. For any name close to a popular package, restate the intended name and confirm it is the one you meant. 4. Capture the verification as evidence (the registry URL and the HTTP status) when the install lands in a PR, per imbue:proof-of-work.

Relationship to the Guard Hook

The guard_package_hallucination.py hook runs this check automatically on every Bash install command. Shadow mode (warn only) is the default; set VOW_SHADOW_MODE=0 to block typosquat and nonexistent installs. Disable the network lookup with IMBUE_PKG_REGISTRY_CHECK=0 to rely on the offline typosquat signal alone. The hook is a backstop, not a substitute: verify deliberately when you add a dependency rather than waiting for the gate.

Related Skills

• imbue:proof-of-work: capture the registry check as evidence.
• leyline:supply-chain-advisory: broader dependency supply-chain

auditing (lockfile drift, artifact integrity, bad versions).

Exit Criteria

• [ ] Every package in the install command or manifest edit is

confirmed present in its registry, or the install is abandoned.

• [ ] Any name within two edits of a popular package is restated

and confirmed before install.

• [ ] A name that could not be verified online is reported as

unverified and not silently installed.

• [ ] For PR-bound installs, the registry check is captured as

proof-of-work evidence.

Registry Checks

Per-ecosystem endpoints for confirming a package exists, and the rules the guard hook applies.

Endpoints

A manual check from the shell:

# PyPI: prints the status code; 200 exists, 404 does not.
curl -s -o /dev/null -w '%{http_code}\n' https://pypi.org/pypi/requests/json

Decision Rules

The hook classifies each install target into one of three states:

• exists (registry returns 200): pass.
• nonexistent (registry returns 404): likely hallucination.

Block when VOW_SHADOW_MODE=0, warn otherwise.

• unverified (timeout, rate limit, offline, non-404 error):

warn only, never block. The guard does not fail closed on a network problem.

Known-popular packages short-circuit before any network call, so a common install such as pip install requests never makes a request to the registry. Only names absent from the bundled popular set are looked up.

Timeout and Offline Behavior

The network lookup uses a 1.5 second per-package timeout. Set IMBUE_PKG_REGISTRY_CHECK=0 to skip network lookups entirely and rely on the offline typosquat signal (edit distance against the known-popular set). This is the right setting in sandboxed or air-gapped environments where the registry is unreachable.

Typosquat Distance

A name not in the known-popular set but within Levenshtein distance 1 or 2 of a popular package is flagged as a typosquat suspect. The distance threshold balances catching real typos (reqeusts, numpi) against false positives on legitimately short names. The known-popular set is the most-impersonated packages per ecosystem and is meant to be extended as new impersonation targets emerge.