Ai Memory Developer

AI Memory Developer

When to Use

• Building persistent memory for copilots, agents, or conversational AI
• Designing memory APIs (read/write/consolidate/forget)
• Choosing between vector stores, graph databases, or structured DBs for memory
• Implementing memory write/read policies and ACLs
• Debugging wrong, stale, or hallucinated memories
• Tuning what the model should remember across sessions (episodic vs semantic)
• Planning GDPR deletion paths and privacy retention for stored memories
• Evaluating memory quality (recall, precision, isolation)

When NOT to Use

• General RAG document search or indexing pipelines → ai-engineer
• Context window packing, token budgets, or compression → ai-context-engineer
• AI team operations, release governance, or SLOs → ai-lead-ops
• Org-wide token cost improvement roadmaps → ai-token-improvement-plan-engineer

Related skills

Core Workflows

1. Memory model design

Classify memory types:

Design decisions:

1. What may be written automatically vs requires user confirmation? 2. Per-tenant isolation and ACL on every read/write 3. TTL and deletion (GDPR erase path) 4. Conflict resolution when new fact contradicts old

See `references/memory_architecture.md` for patterns and anti-patterns.

2. Write path (ingestion to memory)

observe → extract candidates → score importance → dedupe → persist → index

Checklist:

• [ ] Extract only durable facts, not transient chit-chat
• [ ] Attach provenance (message ID, timestamp, source)
• [ ] Dedupe with embedding similarity + entity linking
• [ ] Never store secrets, raw payment data, or full medical records unless required and approved

See `references/write_consolidation.md` for extraction prompts and consolidation jobs.

3. Read path (retrieval for generation)

1. Build query from current user message + session summary 2. Retrieve top-k memories with metadata filters (user_id, tenant_id) 3. Rerank; drop below relevance threshold 4. Inject into context in structured block (see ai-context-engineer) 5. Cite memory IDs in logs for debugging

See `references/read_retrieval.md` for ranking and injection formats.

4. Forgetting and maintenance

Run nightly consolidation: merge episodic notes into semantic summaries.

See `references/write_consolidation.md` for consolidation algorithms.

5. Evaluation

See `references/memory_eval.md` for datasets and regression harness.

When to load references

• Architecture and stores → references/memory_architecture.md
• Write and consolidation → references/write_consolidation.md
• Read and ranking → references/read_retrieval.md
• Evaluation → references/memory_eval.md

Memory architecture

Table of contents

1. Patterns 2. Store selection 3. Anti-patterns

Patterns

Store selection

Always filter by tenant_id and user_id at query time.

Anti-patterns

• Storing entire chat logs as memory without extraction
• Global shared memory across users
• No deletion path for compliance
• Writing model hallucinations as facts without verification

Memory evaluation

Table of contents

1. Scenario tests 2. Regression harness

Scenario tests

Regression harness

Version memory extraction prompt and embedding model; run harness on CI with frozen transcripts.

Report: precision@k, recall@k, isolation failures.

Read and retrieval

Table of contents

1. Query construction 2. Injection format 3. Ranking

Query construction

Combine: latest user message + session summary + active entity IDs.

Injection format

<user_memories>
- [mem_12] Prefers metric units (source: 2024-01-10)
- [mem_45] Project codename: Apollo (source: 2024-02-01)
</user_memories>

Instruct model: use memories only when relevant; do not invent new memories.

Ranking

1. Vector similarity 2. Metadata filter (tenant, user, not expired) 3. Cross-encoder rerank top 20 → top 5 4. Drop if score < threshold

Write and consolidation

Table of contents

1. Extraction 2. Importance scoring 3. Consolidation job

Extraction

Prompt model or smaller model to output JSON:

{ "facts": [{ "text": "...", "confidence": 0.9, "category": "preference" }] }

Reject low confidence and duplicate entities.

Importance scoring

Boost: explicit user preferences, repeated mentions, task-critical constraints.

Penalize: greetings, one-off questions, stale time-bound facts.

Consolidation job

Nightly per user:

1. Load episodic memories from last 7 days 2. Summarize into semantic profile deltas 3. Mark episodes consolidated; retain audit link 4. Prune superseded facts