Knowledge Base Manager

Knowledge Base Manager

Build and maintain high-quality knowledge bases for AI systems and human consumption.

Core Principle

Knowledge Base = Structured Information + Quality Curation + Accessibility

A knowledge base is not just a data dump—it's curated, validated, versioned information designed to answer questions and enable reasoning.

---

When to Use Knowledge Bases

Use Knowledge Bases When:

• ✅ Need to answer factual questions consistently
• ✅ Information changes frequently and needs version control
• ✅ Multiple sources need to be unified and reconciled
• ✅ Provenance and citation tracking is critical
• ✅ Building AI systems that need grounded, verifiable information
• ✅ Organizational knowledge needs to be preserved and searchable
• ✅ Complex domain with interconnected concepts

Don't Use Knowledge Bases When:

• ❌ Static documentation is sufficient (use docs + search)
• ❌ No one will maintain/update it (knowledge rot guaranteed)
• ❌ Simple FAQ covers all questions (<50 items)
• ❌ Information doesn't change (static site faster/cheaper)
• ❌ Team lacks resources for curation

---

Knowledge Base Types: Decision Framework

1. Document-Based Knowledge Base (RAG)

What it is: Collection of documents, chunked and embedded for semantic search

Best for:

• Technical documentation
• Support articles, FAQs
• Policy documents
• Research papers
• Blog content
• User manuals

Strengths:

• Easy to add new documents
• Preserves full context
• Natural for text-heavy content

Weaknesses:

• Hard to query relationships ("Who works where?")
• Duplicate information across documents
• Difficult to keep facts consistent

Use: rag-implementer skill + vector-database-mcp

---

2. Entity-Based Knowledge Base (Knowledge Graph)

What it is: Network of entities (people, places, things) connected by relationships

Best for:

• Organizational charts
• Product catalogs with relationships
• Social networks
• Recommendation systems
• Fraud detection
• Supply chain tracking

Strengths:

• Excellent for "how are X and Y related?" queries
• Consistent facts (one source of truth)
• Powerful traversal ("friends of friends")

Weaknesses:

• Upfront modeling required (ontology design)
• Harder to add unstructured information
• Learning curve for graph queries

Use: knowledge-graph-builder skill + graph-database-mcp

---

3. Hybrid Knowledge Base (RAG + Graph)

What it is: Documents for unstructured knowledge + Graph for structured entities/relationships

Best for:

• Enterprise knowledge management
• Research with citations and relationships
• Medical systems (documents + patient/drug relationships)
• Legal systems (cases + precedents + entities)
• E-commerce (products + specs + relationships)

Strengths:

• Best of both worlds
• Flexible for different knowledge types
• Rich querying capabilities

Weaknesses:

• Most complex to build and maintain
• Requires expertise in both RAG and graphs
• Higher infrastructure costs

Use: Both rag-implementer + knowledge-graph-builder skills

---

Decision Tree: Which KB Type?

What kind of knowledge do you have?

├─ Mostly unstructured text (docs, articles, content)?
│  └─ Document-Based KB (RAG)
│     Use: rag-implementer skill
│
├─ Mostly structured entities with relationships?
│  └─ Entity-Based KB (Graph)
│     Use: knowledge-graph-builder skill
│
└─ Mix of both?
   └─ Hybrid KB (RAG + Graph)
      Use: Both skills + This skill for integration

---

6-Phase Knowledge Base Implementation

Phase 1: Knowledge Audit & Architecture

Goal: Understand what knowledge exists and how to structure it

Actions:

1. Inventory existing knowledge sources

• Internal: databases, documents, wikis, Slack, emails
• External: public data, APIs, third-party sources
• Tribal: SME interviews, recorded conversations

2. Classify knowledge types

• Factual: Verifiable facts ("Product X costs $50")
• Procedural: How-to knowledge ("How to deploy")
• Conceptual: Definitions and explanations
• Relationship: Connections between entities

3. Choose KB architecture

• Document-based? Entity-based? Hybrid?
• Decision: Use framework above

4. Define knowledge schema

• For documents: metadata fields (source, date, author, category)
• For entities: ontology (entity types, relationship types, properties)

Validation:

• [ ] All knowledge sources inventoried and prioritized
• [ ] KB architecture chosen and justified
• [ ] Schema defined and validated with users
• [ ] Success metrics established

---

Phase 2: Knowledge Curation & Ingestion

Goal: Transform raw information into high-quality knowledge

Actions:

1. Extract knowledge from sources

• Automated: scraping, API ingestion, file parsing
• Manual: expert input, annotation, validation

2. Clean and normalize

• Remove duplicates
• Standardize formats
• Fix inconsistencies
• Enrich with metadata

3. Structure knowledge

• For documents: chunk intelligently (semantic boundaries)
• For entities: extract entities, relationships, properties

4. Add provenance

• Source URL or reference
• Last updated timestamp
• Author/contributor
• Confidence score (if applicable)

Curation Best Practices:

• Single Source of Truth: One canonical answer per question
• Deduplication: Merge similar knowledge entries
• Conflict Resolution: When sources disagree, establish priority rules
• Metadata Richness: More metadata = better filtering and search

Validation:

• [ ] Knowledge extracted and structured
• [ ] Quality metrics above threshold (accuracy >95%)
• [ ] Provenance tracked for all entries
• [ ] Sample queries return relevant results

---

Phase 3: Storage & Retrieval Setup

Goal: Implement technical infrastructure for knowledge access

Architecture Patterns:

For Document-Based KB:

// Vector database for semantic search
interface DocumentKB {
  store: 'Pinecone' | 'Weaviate' | 'pgvector'
  chunks: {
    content: string
    embedding: number[]
    metadata: {
      source: string
      title: string
      updated_at: string
      category: string
    }
  }[]
}

For Entity-Based KB:

// Graph database for relationship queries
interface EntityKB {
  store: 'Neo4j' | 'ArangoDB'
  nodes: {
    id: string
    type: 'Person' | 'Organization' | 'Product' | 'Concept'
    properties: Record<string, any>
  }[]
  relationships: {
    from: string
    to: string
    type: string
    properties: Record<string, any>
  }[]
}

For Hybrid KB:

// Both vector DB + graph DB
interface HybridKB {
  vectorDB: DocumentKB
  graphDB: EntityKB
  linker: {
    // Links documents to entities mentioned in them
    linkDocumentToEntities(docId: string): string[]
    // Links entities to documents that mention them
    linkEntityToDocuments(entityId: string): string[]
  }
}

Actions:

1. Choose database(s)

• Document: Pinecone, Weaviate, pgvector
• Entity: Neo4j, ArangoDB
• Hybrid: Both + linking layer

2. Implement search/query layer

• Vector similarity search (for documents)
• Graph traversal (for entities)
• Hybrid queries (combining both)

3. Add caching and optimization

• Cache frequent queries
• Optimize for common access patterns

Validation:

• [ ] Database deployed and accessible
• [ ] Search/query functionality working
• [ ] Performance meets requirements (<100ms for most queries)

---

Phase 4: Quality Control & Validation

Goal: Ensure knowledge base accuracy and reliability

Quality Metrics:

1. Accuracy: % of correct answers to test questions 2. Coverage: % of user questions answerable 3. Freshness: Average age of knowledge 4. Consistency: % of conflicts/contradictions 5. Source Quality: % from authoritative sources

Validation Strategies:

1. Test Question Sets Create 100+ test questions with known correct answers:

interface TestQuestion {
  question: string
  expected_answer: string
  category: string
  difficulty: 'easy' | 'medium' | 'hard'
}

2. Human Review

• Sample random knowledge entries
• Subject matter expert validation
• User feedback loops

3. Automated Checks

• Duplicate Detection: Find near-identical entries
• Conflict Detection: Find contradictory facts
• Staleness Detection: Flag outdated information
• Citation Validation: Verify sources still exist

4. Continuous Monitoring

interface KBHealthMetrics {
  accuracy_score: number // 0-100
  coverage_score: number // % questions answered
  freshness_score: number // avg days since update
  consistency_score: number // % no conflicts
  user_satisfaction: number // feedback rating
}

Actions:

1. Run test question validation (target: >90% accuracy) 2. Conduct human review (sample 10% of entries) 3. Fix detected issues (duplicates, conflicts, staleness) 4. Establish monitoring dashboards

Validation:

• [ ] Accuracy >90% on test questions
• [ ] Coverage >80% of user questions
• [ ] <5% conflicting information
• [ ] Monitoring dashboard operational

---

Phase 5: Versioning & Evolution

Goal: Track knowledge changes over time and enable rollback

Why Versioning Matters:

• Knowledge changes (facts update, policies change)
• Need audit trail (who changed what when)
• Rollback capability (undo bad updates)
• Historical queries ("What was policy on X in 2023?")

Versioning Strategies:

1. Snapshot Versioning

interface KnowledgeEntry {
  id: string
  content: string
  version: number
  created_at: string
  updated_at: string
  updated_by: string
  changelog: string
  previous_version?: string // ID of prior version
}

2. Event Sourcing

interface KnowledgeEvent {
  event_id: string
  entity_id: string
  event_type: 'created' | 'updated' | 'deleted'
  timestamp: string
  changes: {
    field: string
    old_value: any
    new_value: any
  }[]
  author: string
}

3. Git-Style Versioning

• Treat knowledge like code
• Commit-based changes
• Branch for experimental knowledge
• Merge when validated

Actions:

1. Implement version tracking 2. Add changelog for all updates 3. Create rollback mechanism 4. Build version comparison tools

Validation:

• [ ] All changes tracked with versions
• [ ] Rollback tested and working
• [ ] Historical queries supported
• [ ] Audit trail complete

---

Phase 6: Maintenance & Governance

Goal: Keep knowledge base healthy long-term

Maintenance Tasks:

Daily:

• Monitor for errors and failures
• Review user feedback
• Address urgent corrections

Weekly:

• Review new content submissions
• Update time-sensitive knowledge
• Run automated quality checks

Monthly:

• Audit knowledge freshness
• Review and resolve conflicts
• Analyze usage patterns
• Update stale content

Quarterly:

• Comprehensive quality audit
• Schema/ontology review
• Performance optimization
• User satisfaction survey

Governance Framework:

1. Roles & Responsibilities

• Knowledge Owners: Domain experts responsible for content
• Curators: Review and approve changes
• Contributors: Submit new knowledge
• Consumers: Use knowledge and provide feedback

2. Change Process

Submit → Review → Approve → Publish → Monitor

3. Quality Standards

• Minimum source quality requirements
• Citation requirements
• Update frequency requirements
• Conflict resolution process

Actions:

1. Establish maintenance schedule 2. Assign roles and responsibilities 3. Create governance documentation 4. Train team on processes

Validation:

• [ ] Maintenance schedule in place
• [ ] Governance documented and communicated
• [ ] Team trained on processes
• [ ] Quality trending upward

---

Knowledge Base Anti-Patterns

❌ Anti-Pattern 1: Data Dump Without Curation

Problem: Ingesting everything without quality filtering

Impact: Low signal-to-noise ratio, poor search results, user frustration

Solution: Curate before ingesting. Quality > Quantity

---

❌ Anti-Pattern 2: No Version Control

Problem: Knowledge changes but no history tracked

Impact: Can't audit changes, can't rollback errors, no accountability

Solution: Implement versioning from Phase 5

---

❌ Anti-Pattern 3: Stale Knowledge

Problem: Knowledge base outdated but no one knows

Impact: AI systems hallucinate using old facts, users get wrong answers

Solution: Freshness monitoring + scheduled updates

---

❌ Anti-Pattern 4: Duplicate Information

Problem: Same fact in multiple places, becomes inconsistent

Impact: Conflicting answers, confused users

Solution: Deduplication + single source of truth

---

❌ Anti-Pattern 5: No Provenance

Problem: Knowledge without source citations

Impact: Can't verify accuracy, can't trace errors

Solution: Always track source + timestamp + author

---

Integration with Other Skills

With rag-implementer

• Use for document-based portion of hybrid KB
• Follow RAG implementation phases
• Integrate vector search with KB queries

With knowledge-graph-builder

• Use for entity-based portion of hybrid KB
• Follow graph design patterns
• Integrate graph traversal with KB queries

With data-engineer

• For ETL pipelines (extract, transform, load knowledge)
• For data quality monitoring
• For performance optimization

With quality-auditor

• For automated quality checks
• For testing and validation
• For continuous monitoring

With technical-writer

• For knowledge documentation
• For user guides on KB usage
• For governance documentation

---

Tools & Technologies

Document-Based KB Stack

• Vector DB: Pinecone, Weaviate, pgvector
• Embeddings: OpenAI, Cohere, custom
• Search: Semantic + keyword hybrid

Entity-Based KB Stack

• Graph DB: Neo4j, ArangoDB
• Query: Cypher, AQL
• Visualization: Neo4j Bloom, Gephi

Curation Tools

• Deduplication: Custom algorithms, fuzzy matching
• Conflict Detection: Rule-based, ML-based
• Validation: Test question sets, human review

Monitoring

• Metrics: Custom dashboard (Grafana)
• Logging: Structured logging of queries/updates
• Alerts: Freshness, accuracy, error rate alerts

---

Success Metrics

Knowledge Quality

• Accuracy: >90% on test questions
• Coverage: >80% of user questions answered
• Freshness: <30 days average age
• Consistency: <5% conflicting information

User Satisfaction

• Relevance: >85% query results rated relevant
• Usefulness: >80% users find KB valuable
• Speed: <100ms median query time

Operational Health

• Uptime: >99.9%
• Update frequency: Weekly minimum
• Team engagement: Regular contributions

---

Common Pitfalls & Solutions

Pitfall 1: "Build it and they will come"

Problem: No user validation, KB doesn't meet needs

Solution: Start with user research, validate continuously

Pitfall 2: Perfectionism

Problem: Waiting to launch until KB is "perfect"

Solution: Launch with 80% coverage, iterate based on usage

Pitfall 3: Over-engineering

Problem: Building complex hybrid system when simple docs would work

Solution: Start simple, add complexity only when needed

Pitfall 4: Maintenance neglect

Problem: Build once, never update

Solution: Establish maintenance schedule from day 1

---

Quick Start Checklist

Before you start:

• [ ] Read this entire skill
• [ ] Review rag-implementer if using document KB
• [ ] Review knowledge-graph-builder if using entity KB
• [ ] Have clear use case and success metrics

Phase 1 - Architecture (Week 1):

• [ ] Inventory knowledge sources
• [ ] Choose KB type (document/entity/hybrid)
• [ ] Define schema/ontology
• [ ] Set up infrastructure

Phase 2 - Initial Build (Week 2-3):

• [ ] Ingest and curate initial knowledge
• [ ] Implement search/query functionality
• [ ] Create test question set
• [ ] Validate with users

Phase 3 - Iterate (Ongoing):

• [ ] Add more knowledge based on usage
• [ ] Monitor quality metrics
• [ ] Fix issues as discovered
• [ ] Establish maintenance cadence

---

Related Resources

• Skills: rag-implementer, knowledge-graph-builder, data-engineer, quality-auditor
• MCPs: vector-database-mcp, graph-database-mcp, knowledge-base-mcp, semantic-search-mcp
• Patterns: STANDARDS/architecture-patterns/rag-pattern.md, knowledge-base-pattern.md (coming soon)
• Integrations: INTEGRATIONS/pinecone/, INTEGRATIONS/graph-databases/neo4j/

---

Further Reading

• The Knowledge Graph Cookbook
• Building Knowledge Bases with LLMs
• RAG: Retrieval-Augmented Generation
• Knowledge Management Best Practices

---

Remember: A knowledge base is only as good as its curation. Invest in quality from day 1, establish maintenance processes, and iterate based on user feedback. The goal is not to have all knowledge—it's to have the _right_ knowledge, well-organized, and easily accessible.

Knowledge Base Manager Skill

Version: 1.0.0 Category: AI & Data Difficulty: Intermediate Est. Time: 2-4 weeks for full implementation

What This Skill Does

Provides comprehensive guidance for designing, building, and maintaining knowledge bases that power AI systems and human information needs. Bridges the gap between document-based knowledge (RAG) and entity-based knowledge (graphs), offering a unified approach to knowledge management.

When to Use This Skill

Use this skill when you need to:

• Build knowledge-intensive AI applications
• Manage organizational knowledge systematically
• Integrate multiple knowledge sources
• Ensure knowledge quality and freshness
• Choose between RAG, knowledge graphs, or hybrid approaches
• Implement knowledge versioning and governance

What You'll Learn

• Knowledge Architecture: Document-based, entity-based, and hybrid approaches
• Curation Best Practices: How to maintain high-quality knowledge
• Storage & Retrieval: Technical patterns for different KB types
• Quality Control: Testing, validation, and monitoring strategies
• Versioning: Track knowledge evolution over time
• Governance: Long-term maintenance and team processes

Key Concepts

Three KB Types

1. Document-Based (RAG)

• Best for: Documentation, articles, policies
• Tools: Vector databases, embeddings
• Skills: rag-implementer

2. Entity-Based (Knowledge Graphs)

• Best for: Entities + relationships, org charts, catalogs
• Tools: Graph databases (Neo4j)
• Skills: knowledge-graph-builder

3. Hybrid (RAG + Graphs)

• Best for: Complex systems needing both
• Tools: Vector DB + Graph DB + linking layer
• Skills: Both rag-implementer + knowledge-graph-builder

6-Phase Implementation

1. Knowledge Audit & Architecture - Understand and structure 2. Curation & Ingestion - Transform raw info to quality knowledge 3. Storage & Retrieval - Implement technical infrastructure 4. Quality Control - Validate accuracy and coverage 5. Versioning & Evolution - Track changes over time 6. Maintenance & Governance - Keep healthy long-term

Quick Decision Framework

What kind of knowledge?

Unstructured text?     → Document-Based KB (RAG)
Structured entities?   → Entity-Based KB (Graph)
Both?                  → Hybrid KB

Prerequisites

Required Knowledge:

• Basic understanding of databases
• Familiarity with search concepts
• Understanding of data quality principles

Recommended Skills to Review First:

• rag-implementer - If using document-based KB
• knowledge-graph-builder - If using entity-based KB
• data-engineer - For ETL and data pipelines

Quick Start

1. Read the full SKILL.md to understand all phases 2. Inventory your knowledge sources (databases, docs, APIs) 3. Choose KB type using decision framework 4. Start with Phase 1 (Architecture) from SKILL.md 5. Iterate based on user feedback

Common Use Cases

Use Case 1: Company Documentation Search

• Type: Document-Based KB (RAG)
• Time: 2-3 weeks
• Skills: rag-implementer, knowledge-base-manager
• Tools: Pinecone + OpenAI embeddings

Use Case 2: Product Catalog with Relationships

• Type: Entity-Based KB (Graph)
• Time: 3-4 weeks
• Skills: knowledge-graph-builder, knowledge-base-manager
• Tools: Neo4j + custom ontology

Use Case 3: Enterprise Knowledge Hub

• Type: Hybrid KB
• Time: 6-8 weeks
• Skills: rag-implementer, knowledge-graph-builder, knowledge-base-manager
• Tools: Pinecone + Neo4j + linking layer

Anti-Patterns to Avoid

❌ Data dump without curation - Quality > Quantity ❌ No version control - Can't audit or rollback ❌ Stale knowledge - Monitor freshness ❌ Duplicate information - Single source of truth ❌ No provenance - Always track sources

Success Metrics

• Accuracy: >90% on test questions
• Coverage: >80% of user questions answered
• Freshness: <30 days average age
• User Satisfaction: >85% relevance rating

Integration with Other Skills

• rag-implementer - Document-based portion
• knowledge-graph-builder - Entity-based portion
• data-engineer - ETL pipelines and optimization
• quality-auditor - Testing and validation
• technical-writer - Documentation and guides

Tools & MCPs

Required MCPs

• knowledge-base-mcp - CRUD operations for knowledge entries (coming soon)
• vector-database-mcp - For document-based KB
• graph-database-mcp - For entity-based KB
• semantic-search-mcp - For hybrid search

Recommended Integrations

• Pinecone - Vector database (INTEGRATIONS/pinecone/)
• Neo4j - Graph database (INTEGRATIONS/graph-databases/neo4j/)
• OpenAI - Embeddings (INTEGRATIONS/openai/)

Resources

In This Repository

• Skill: SKILLS/knowledge-base-manager/SKILL.md (this file's full version)
• Pattern: STANDARDS/architecture-patterns/knowledge-base-pattern.md (coming soon)
• Related Skills: rag-implementer, knowledge-graph-builder, data-engineer

External Resources

• The Knowledge Graph Cookbook
• Building Knowledge Bases with LLMs
• RAG: Retrieval-Augmented Generation

Troubleshooting

Problem: Don't know which KB type to choose

Solution: Follow decision tree in SKILL.md. Start simple (document-based), add complexity only when needed.

Problem: Knowledge quality is poor

Solution: Implement Phase 4 (Quality Control). Create test question sets, run validation, fix issues.

Problem: Knowledge becomes stale

Solution: Implement Phase 6 (Maintenance). Set up automated freshness monitoring and scheduled updates.

Problem: Users can't find information

Solution: Check coverage metrics. Add missing knowledge, improve metadata, optimize search/query.

Next Steps

1. Read the complete SKILL.md file 2. Complete Phase 1: Knowledge Audit & Architecture 3. Choose your KB type based on requirements 4. Follow the 6-phase implementation framework 5. Establish maintenance processes from day 1

Support

For questions or issues:

• Review the comprehensive SKILL.md guide
• Check related skills: rag-implementer, knowledge-graph-builder
• Refer to architecture patterns (when available)
• Use knowledge-base-mcp for CRUD operations (coming soon)

---

Remember: A great knowledge base is curated, versioned, and maintained. Invest in quality from the start, and your knowledge will compound in value over time.