
Memory Quality Auditor
- 50 installs
- 36 repo stars
- Updated July 14, 2026
- oimiragieo/agent-studio
Helps with security tasks.
About
memory-quality-auditor is a Claude Code skill for security. It helps solo builders move faster with AI-assisted development.
- memory-quality-auditor
- Security
- AI-coding skill
Memory Quality Auditor by the numbers
- 50 all-time installs (skills.sh)
- Ranked #1,325 of 2,203 Security skills by installs in the Skillselion catalog
- Data as of Aug 4, 2026 (Skillselion catalog sync)
npx skills add https://github.com/oimiragieo/agent-studio --skill memory-quality-auditorAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 50 |
|---|---|
| repo stars | ★ 36 |
| Last updated | July 14, 2026 |
| Repository | oimiragieo/agent-studio ↗ |
What it does
Helps with security tasks.
Files
Memory Quality Auditor
Audit the memory system as a unified retrieval layer (STM/MTM/LTM files + index + spawn citation outcomes).
Scope
- Retrieval drift signals
- stale memory ratio
- evidence injection coverage
- citation usage/groundedness continuity
Workflow
1. Read memory artifacts and latest eval reports. 2. Compute quality metrics and threshold status. 3. Emit remediation backlog with TDD checks. 4. Record findings in memory and optional evolution recommendation.
Iron Laws
1. ALWAYS establish a baseline metric snapshot before auditing — drift is only meaningful relative to a prior measurement; auditing without a baseline produces absolute numbers that cannot identify regression. 2. NEVER close a memory finding without re-running the affected retrieval query — closing without verification creates false improvement metrics and masks persistent degradation. 3. ALWAYS include citation-groundedness checks in every audit run — uncited memory injections are the primary source of hallucination in agent spawns; skipping this check leaves the highest-risk failure mode undetected. 4. NEVER audit only the STM tier — degradation often originates in MTM/LTM promotion corruption; all three tiers must be sampled in every full audit cycle. 5. ALWAYS emit TDD-ready remediation items with a failing-test condition and expected metric threshold — vague findings ("memory quality is low") cannot be actioned by any agent.
Anti-Patterns
| Anti-Pattern | Why It Fails | Correct Approach |
|---|---|---|
| Auditing without a baseline | Cannot distinguish regression from steady-state; all findings are ambiguous | Snapshot current metrics at session start; compute delta against the previous run |
| Closing findings without re-check | Produces false-positive resolution; degradation persists silently behind green metrics | Re-run the specific retrieval query after each remediation; close only on confirmed green metric |
| Skipping citation groundedness | Citation failures are the leading cause of agent hallucination; missing this check omits the highest-severity defect class | Include citation_coverage and grounded_ratio metrics in every audit report |
| Full-mode audit on every spawn | Full audit is expensive; running it unconditionally inflates cost and slows workflows | Use --mode summary for routine checks; reserve --mode full for scheduled or triggered audits |
| Auditing STM only | MTM/LTM corruption is invisible in STM-only scans; stale LTM entries contaminate future sessions | Sample all three tiers: STM (current session), MTM (last 10 sessions), LTM (permanent summaries) |
Memory Protocol (MANDATORY)
Before starting: Read .claude/context/memory/learnings.md
After completing:
- New pattern →
.claude/context/memory/learnings.md - Issue found →
.claude/context/memory/issues.md - Decision made →
.claude/context/memory/decisions.md
ASSUME INTERRUPTION: If it's not in memory, it didn't happen.
Invoke the memory-quality-auditor skill and follow it exactly as presented to you
'use strict';
/**
* Post-execute hook for memory-quality-auditor
* Auto-generated by enterprise-bundle-scaffolder
*
* Records metrics after skill execution.
*/
function postExecute(_context) {
// Record execution metrics
return { ok: true, skill: 'memory-quality-auditor' };
}
module.exports = { postExecute };
'use strict';
/**
* Pre-execute hook for memory-quality-auditor
* Auto-generated by enterprise-bundle-scaffolder
*
* Validates inputs before skill execution.
*/
function preExecute(context) {
// Validate skill invocation context
if (!context || typeof context !== 'object') {
return { allow: true, message: 'memory-quality-auditor: no context to validate' };
}
return { allow: true };
}
module.exports = { preExecute };
memory-quality-auditor Research Requirements
Generated: 2026-02-28
Skill Description
Audit memory retrieval quality (drift, staleness, citation-groundedness) and produce remediation backlog.
Research Areas
- Current best practices for memory-quality-auditor
- Industry standards and tooling
- Integration patterns
Source References
- To be populated by skill-updater research phase
memory-quality-auditor Rules
Purpose
Audit memory retrieval quality (drift, staleness, citation-groundedness) and produce remediation backlog.
Best Practices
- Follow established patterns
- Validate inputs at boundaries
Integration Points
See SKILL.md for complete documentation.
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "memory-quality-auditorInput",
"description": "Input schema for Audit memory retrieval quality (drift, staleness, citation-groundedness) and produce remediation backlog.",
"type": "object",
"additionalProperties": true,
"properties": {
"target": {
"type": "string",
"description": "Target file or path for the skill to operate on"
},
"options": {
"type": "object",
"description": "Additional options for skill execution",
"additionalProperties": true
}
}
}
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "memory-quality-auditorOutput",
"type": "object",
"additionalProperties": true,
"properties": {
"ok": {
"type": "boolean"
},
"summary": {
"type": "string"
}
}
}
#!/usr/bin/env node
'use strict';
const fs = require('node:fs');
const path = require('node:path');
function findProjectRoot() {
let dir = __dirname;
while (dir !== path.parse(dir).root) {
if (fs.existsSync(path.join(dir, '.claude'))) return dir;
if (path.basename(dir) === '.claude') return path.dirname(dir);
dir = path.dirname(dir);
}
return process.cwd();
}
const PROJECT_ROOT = findProjectRoot();
function parseArgs(argv) {
const options = {};
for (let i = 0; i < argv.length; i++) {
const arg = argv[i];
if (!arg.startsWith('--')) continue;
const key = arg.slice(2);
const next = argv[i + 1];
const hasValue = next && !next.startsWith('--');
options[key] = hasValue ? argv[++i] : true;
}
return options;
}
function safeReadJson(filePath, fallback = null) {
if (!fs.existsSync(filePath)) return fallback;
try {
return JSON.parse(fs.readFileSync(filePath, 'utf8'));
} catch {
return fallback;
}
}
function computeAudit() {
const evalPath = path.join(
PROJECT_ROOT,
'.claude',
'context',
'runtime',
'evals',
'subagent-memory-rag-live-latest.json'
);
const evalReport = safeReadJson(evalPath, {});
const patternsPath = path.join(PROJECT_ROOT, '.claude', 'context', 'memory', 'patterns.json');
const gotchasPath = path.join(PROJECT_ROOT, '.claude', 'context', 'memory', 'gotchas.json');
const accessStatsPath = path.join(
PROJECT_ROOT,
'.claude',
'context',
'memory',
'access-stats.json'
);
const patterns = safeReadJson(patternsPath, []) || [];
const gotchas = safeReadJson(gotchasPath, []) || [];
const access = safeReadJson(accessStatsPath, {}) || {};
const summary = evalReport.summary || {};
const metrics = {
memory_entries: patterns.length + gotchas.length,
access_events: Number(access.total || access.count || 0),
evidence_injection_rate: Number(summary.evidence_injection_rate || 0),
citation_use_rate: Number(summary.citation_use_rate || 0),
groundedness_rate: Number(summary.groundedness_rate || 0),
stale_ratio_estimate: patterns.length + gotchas.length === 0 ? 1 : 0,
};
const thresholds = {
min_evidence_injection_rate: 0.8,
min_citation_use_rate: 0.5,
min_groundedness_rate: 0.6,
max_stale_ratio_estimate: 0.3,
};
const failed = [];
if (metrics.evidence_injection_rate < thresholds.min_evidence_injection_rate)
failed.push('evidence_injection_rate');
if (metrics.citation_use_rate < thresholds.min_citation_use_rate)
failed.push('citation_use_rate');
if (metrics.groundedness_rate < thresholds.min_groundedness_rate)
failed.push('groundedness_rate');
if (metrics.stale_ratio_estimate > thresholds.max_stale_ratio_estimate)
failed.push('stale_ratio_estimate');
return {
ok: true,
metrics,
thresholds,
status: failed.length === 0 ? 'healthy' : 'degraded',
failed,
remediation: failed.map(item => ({
id: `remediate-${item}`,
action:
item === 'stale_ratio_estimate'
? 'refresh memory entries with context-compressor and remove stale items'
: `add tests/guards to improve ${item}`,
})),
};
}
function main(input = null) {
const options = input || parseArgs(process.argv.slice(2));
if (options.help) {
return {
ok: true,
usage: 'node .claude/skills/memory-quality-auditor/scripts/main.cjs [--mode summary|full]',
};
}
const result = computeAudit();
if ((options.mode || 'summary') === 'summary') {
return {
ok: result.ok,
status: result.status,
failed: result.failed,
metrics: result.metrics,
};
}
return result;
}
if (require.main === module) {
const result = main();
if (result.usage) {
console.log(result.usage);
process.exit(0);
}
console.log(JSON.stringify(result, null, 2));
process.exit(result.ok ? 0 : 1);
}
module.exports = { parseArgs, computeAudit, main };
memory-quality-auditor Implementation Template
Goal
- Define target outcome and acceptance criteria.
TDD
1. Red 2. Green 3. Refactor
Verification
- lint
- format
- targeted tests