
Temple Generator
- 170 installs
- 339 repo stars
- Updated August 4, 2026
- glebis/claude-skills
Generate procedural temple layouts, assets, or scene descriptions for game worlds, creative demos, or narrative content pipelines inside Claude Code.
About
Temple-generator from glebis/claude-skills procedurally produces temple structures, motifs, and scene-ready descriptions for games, creative prototypes, and narrative content workflows inside Claude Code.
- Procedural temple layouts
- Theme and motif variation
- Scene description export
- Game-world content seeding
- Rapid visual concept iteration
Temple Generator by the numbers
- 170 all-time installs (skills.sh)
- Ranked #99 of 247 Game Development skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/glebis/claude-skills --skill temple-generatorAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 170 |
|---|---|
| repo stars | ★ 339 |
| Last updated | August 4, 2026 |
| Repository | glebis/claude-skills ↗ |
What it does
Generate procedural temple layouts, assets, or scene descriptions for game worlds, creative demos, or narrative content pipelines inside Claude Code.
Files
Temple Generator
Generate a 3D interactive knowledge visualization from any Obsidian vault. The output is a single HTML file (Three.js) with concentric entity rings, audio, discovery mechanics, and multi-scale semantic zoom.
When to Use
- User wants to visualize any Obsidian vault as a 3D knowledge map
- User wants to compare two vaults/document sets visually
- User wants to regenerate the temple from scratch with fresh vault analysis
Architecture
Two-part system: 1. Generation pipeline (this skill): discovers structure, names it, scores confidence, exports a scene package 2. Runtime renderer (template): handles navigation, transitions, audio, discovery
Pre-generate meaning. Runtime-render experience.
Workflow
Step 1: Scan the Vault
Run python3 ~/.claude/skills/temple-generator/scripts/extract_entities.py <vault_path>.
This produces vault-scan.json with:
- Files: path, title, tags, outgoing links, backlink counts, word count, folder, frontmatter
- Graph: adjacency list with bidirectional link counts
- Centrality: degree centrality per node
- Clusters: detected groups of tightly linked notes
Step 2: Read the Scan + Sample Notes
1. Read vault-scan.json 2. Read the top ~20 nodes by centrality (first 100 lines each) 3. Read references/classification-guide.md for entity type heuristics 4. Read 3-5 representative notes to calibrate the vault's "voice" (formal/informal, domain jargon, language)
Step 3: Classify Entities
Using references/classification-guide.md, assign each significant node to an entity type. Maintain two vocabularies:
- canonical: neutral labels for portability (
anxiety-management,fermentation-process) - poetic: mythic/art labels for the installation (
The Ferment Gate,The Cortisol Throne)
Target counts per type (adjust for vault size):
| Type | Small vault (< 100) | Medium (100-500) | Large (500+) |
|---|---|---|---|
| Gods | 2-3 | 3-5 | 5-7 |
| Demigods | 3-7 | 5-12 | 8-15 |
| Tensions | 2-4 | 3-7 | 5-9 |
| Narratives | 2-5 | 5-10 | 8-12 |
| Blind spots | 1-3 | 3-5 | 4-7 |
| Spirits | 1-3 | 3-5 | 3-5 |
| Research | 5-15 | 10-25 | 15-30 |
| Values | 2-5 | 3-8 | 5-10 |
| Trails | 2-5 | 3-8 | 5-10 |
| Questions | 3-6 | 5-10 | 8-12 |
| Depths | 2-5 | 5-10 | 8-15 |
| Crystals | 1-3 | 2-5 | 3-6 |
Step 4: Build Abstraction Levels
Levels are confidence-gated — only include a level if the vault supports it.
Level 0 — Entities (always exists): individual nodes with positions, connections, descriptions.
Level 1 — Domains (requires >= 3 meaningful clusters): groups of related entities. Each domain has:
canonical+poeticname- member entity keys
- centroid position (weighted average of member positions)
- representative exemplar (most central member)
- description (1-2 sentences in vault voice)
- confidence score (0-1)
Level 2 — Axes (requires >= 2 interpretable opposing pairs): fundamental tensions. Each axis has:
- two poles with names and descriptions
- member domains per pole
- axis description
- confidence score
Level 3 — Comparison (requires two vaults + sufficient alignment): shared/unique analysis.
Read references/merge-algorithm.md for dual-graph logic.
Step 5: Generate Scene Package
Follow the schema in references/entity-schema.md to produce temple-data.json.
Include:
entities: all classified nodeslevels: abstraction layers with zoom thresholdsmappings: entity → domain → axis crosswalkscomparison: (if dual-graph) shared/unique/alignment dataaudio: motif hints per type and levelstyle: poetic vocabulary, intro text, color palette, layer definitionsconfidence: per-abstraction and per-alignment scores
Step 6: Generate HTML
1. Copy ~/.claude/skills/temple-generator/assets/temple-template.html to the output location 2. If --inline flag: embed the JSON data as const TEMPLE_DATA = {...}; inside the HTML 3. Otherwise: place temple-data.json alongside the HTML
Step 7: Report
Show the user:
- Entity counts by type
- Abstraction levels generated (with confidence scores)
- Top 5 gods/central entities
- Detected tensions
- If dual-graph: overlap percentage and shared domains
Dual-Graph Mode
When --compare vault_path_2 is provided:
1. Scan both vaults independently (Step 1) 2. Classify entities for each vault (Steps 2-3) 3. Run merge algorithm from references/merge-algorithm.md 4. Generate merged scene package with source attribution 5. Template renders shared scaffold with divergence offsets
Quality Guidelines
- Skip trivial notes (daily todos, admin logs, empty stubs)
- Prefer nodes that reveal the vault's actual concerns, not its filing system
- Write in the vault's own voice, calibrated from sample notes
- If a level lacks confidence, omit it rather than fabricating structure
- Each abstraction level must be backed by membership weights, exemplars, and provenance
- "The abstraction hierarchy should be semantic, not just geometric"
Audio Guidance for Template
The template's audio system should respect hierarchical continuity across zoom levels:
- L0 (close): localized, identity-rich — entity whispers and textures
- L1 (medium): regional harmonic beds, cluster pulses
- L2 (far): sparse drones, tension-based tonal movement
- L3 (comparison): stereo/dialogic between two vault voices
Zoom should feel like changing resolution, not changing universes. Motifs relate across scales.
{
"name": "temple-generator",
"description": "Generate a 3D interactive knowledge map (Inner Temple) from any Obsidian vault or document set. Supports multi-scale abs",
"author": {
"name": "Gleb Kalinin"
},
"repository": "https://github.com/glebis/claude-skills",
"license": "MIT"
}Entity Classification Guide
How to map arbitrary vault content into temple entity types. The framework is universal; the content is domain-specific.
Classification Principles
1. Algorithm discovers, AI names. Use centrality metrics and cluster analysis to identify candidates. Then read the actual notes to decide what they mean. 2. Two vocabularies. Every entity gets a canonical label (neutral, portable) and a poetic label (mythic, evocative). The canonical name is for debugging; the poetic name is for the temple. 3. Semantic, not structural. A note's entity type comes from its role in the knowledge system, not its filename or folder. A note called "README" could be a god if everything links to it.
Entity Type Heuristics
Gods (3-7)
What they are: The foundational concepts the vault orbits around. Remove one and the graph fragments. Detection: Highest degree centrality. Most backlinks. Often appear in frontmatter tags of many other notes. May have their own MoC or hub note. Examples across domains:
- Pharmacogenomics vault: DRD2, FKBP5, Sertraline
- Cooking vault: Fermentation, Heat, Fat
- Software vault: Architecture, Testing, Deployment
- Philosophy vault: Consciousness, Ethics, Language
Naming: Gods get imposing names. "The Serotonin Gate", "The Ferment Throne", "The Test Oracle."
Demigods (5-15)
What they are: High-importance concepts that serve the gods. Specific mechanisms, techniques, or frameworks. Detection: High centrality but fewer cross-domain links than gods. Often connected to 1-2 gods strongly. Examples: COMT (serves neurochem god), Maillard Reaction (serves Heat god), CI/CD (serves Deployment god). Naming: Less imposing than gods but still mythic. "The Methylation Courier", "The Browning Ritual."
Tensions (3-9)
What they are: Pairs of concepts that pull in opposite directions. The vault's unresolved dialectics. Detection: Look for:
- Notes that mention both sides of a trade-off
- Clusters that rarely link to each other despite being near in centrality
- Explicit "vs" or "or" patterns in titles/content
- Contradictions between notes (one says X is good, another says X is harmful)
Examples: Building ↔ Calm, Tradition ↔ Innovation, Speed ↔ Quality, Solo ↔ Connection Naming: Use the ↔ symbol. Both poles get names. "The Architect's Restlessness ↔ The Monk's Silence."
Narratives (5-12)
What they are: Stories the vault tells about itself. Recurring themes across multiple notes. Self-referential patterns. Detection: Look for:
- Phrases/metaphors that appear across 3+ unrelated notes
- Self-referential observations ("I always...", "the pattern is...")
- Historical accounts that inform current thinking
- Cluster themes that span domains
Examples: "Hardware vs Willpower" (genetics as destiny), "The French Training" (formative experience shaping taste), "The Partnership Breakup" (trust injury shaping decisions) Naming: Quote-like or story-like. "The Story of the Slow Receptor."
Blind Spots (3-7)
What they are: Things the vault talks ABOUT but doesn't ACT on. Contradictions between stated values and actual practice. Absent presences. Detection: Look for:
- Notes with many outgoing links but few backlinks (written about, not integrated)
- Goals/values that appear in planning notes but never in action notes
- Topics the vault circles around but never directly addresses
- Advice given to others that isn't followed personally
Examples: "Teaching as Avoidance of Building", "Never tries baking" (in a cooking vault), "Writes about testing but has no tests" Naming: Honest and slightly uncomfortable. "The Unfollowed Advice."
Spirits (3-5)
What they are: Recurring behavioral patterns that cross domain boundaries. Not a topic but a mode of operating. Detection: Look for:
- Patterns visible across 3+ different contexts/folders
- Behavioral verbs that recur ("collecting", "planning", "demonstrating")
- Meta-observations about one's own process
Examples: "The Architect Demon" (over-planning), "The Recipe Collector" (gathering without doing), "The Demo Compulsion" (needing to show) Naming: Personified. "The _____ Demon/Spirit/Ghost."
Crystals (2-6)
What they are: Beliefs or assumptions stated as facts. High resistance to change. Often foundational but potentially limiting. Detection: Declarative statements that underpin multiple notes. Assumptions that go unquestioned. Core identity claims. Examples: "The Partnership Breakup" (solidified trauma-story), "Authentic only if handmade", "I'm not a real developer" Naming: Mineral/gem metaphors or declarative quotes. "The Handmade Crystal."
Research (10-30)
What they are: Domain concepts that are referenced and explored but not central organizing principles. Detection: Medium centrality. Well-linked within their cluster but not cross-domain. Often have source citations. Examples: Affordance, Polyvagal Theory, Koji, Tempering, Middleware pattern Naming: Keep closer to canonical names but add one evocative modifier. "The Polyvagal Bridge", "The Koji Bloom."
Values (3-10)
What they are: Prescriptive notes — rules, protocols, operating principles. Things the vault says "do this" about. Detection: Imperative language. Protocol/checklist format. Often referenced in daily notes. Examples: "Move First", "Season Early", "Test Before Ship", "Breathe Before Respond" Naming: Imperative voice. Keep short and commanding.
Trails (3-10)
What they are: Active projects, ongoing threads, journeys in progress. Detection: Notes with dates, progress markers, status fields. Often in a Trails/ or Projects/ folder. Have temporal dimension. Examples: "Claude Code Lab", "Sourdough Journey", "Migration to Rust" Naming: Journey metaphors. "The Lab Trail", "The Sourdough Path."
Questions (5-12)
What they are: Open questions the vault hasn't answered. Edges of knowledge. Detection: Notes ending in ?. Tags like #open-question. Notes with links to many topics but no conclusion. Empty or stub notes on important topics. Examples: "Who is Gleb without Claude?", "Can I make real ramen?", "What comes after microservices?" Naming: Keep as questions. May shorten or sharpen.
Depths (5-15)
What they are: Notes that connect surface concepts to deeper patterns. The "why beneath the what." Detection: Notes that reference psychology, identity, meaning, motivation. Notes that explain why a pattern exists, not just what it is. Examples: IFS parts, shadow work, food memories, "why I code" Naming: Depth metaphors. "The Root Beneath the Root."
Whispers (5-6, one per layer)
What they are: Ambient text displayed at different depth levels. Sets the mood for each vertical layer. Detection: Not derived from notes. Written by Claude to capture the layer's feeling. Format: Array of short phrases. Example: ["The genome speaks in probabilities", "Every receptor is a door half-open"]
Secrets (10-20)
What they are: Non-obvious connections between two entities that reward discovery. Detection: Surprising links between entities in different domains/types. Counter-intuitive relationships. Format: {trigger: ['key1', 'key2'], insight: 'Why this connection matters', status: 'hidden'}
Fields (3-5)
What they are: Atmospheric zones that group related entities spatially. Visual clusters with ambient effects. Detection: Derived from Level 1 domains. Each field covers a spatial region of the temple. Format: {key, name, center: [x,y,z], radius, color: [r,g,b], desc}
Classification Workflow
1. Sort nodes by degree centrality (descending) 2. Assign top 3-7 as god candidates → read their content to confirm 3. Next tier (up to ~15) as demigod candidates → confirm by reading 4. Scan for opposing pairs in high-centrality nodes → tensions 5. Look for cross-domain recurring patterns → spirits, narratives 6. Check for stated-but-not-practiced patterns → blind spots 7. Read cluster themes → domains (Level 1) 8. Fill remaining types from scan data 9. Write whispers and secrets last (these require understanding the whole)
Voice Calibration
Before writing any poetic names or descriptions: 1. Read 5 representative notes from the vault 2. Note: language (English/Russian/mixed?), tone (academic/casual/poetic?), person (first/third?), jargon level 3. Match temple text to this voice. A casual vault gets casual temple text. A formal vault gets formal text.
Temple Data JSON Schema
The contract between the generation pipeline and the runtime renderer.
Top-Level Structure
{
"version": 2,
"mode": "single" | "merged",
"config": { ... },
"entities": { ... },
"levels": [ ... ],
"mappings": { ... },
"secrets": [ ... ],
"whispers": { ... },
"fields": [ ... ],
"confidence": { ... },
"sources": [ ... ] // only in merged mode
}Config
{
"config": {
"title": "The Inner Temple",
"intro": {
"heading": "ENTER THE TEMPLE",
"body": "A living map of [vault description]. Click to begin."
},
"palette": {
"primary": "#e85d04",
"accent": "#ffd60a",
"bg": "#030108",
"text": "#f8f8f2"
},
"layers": [
{
"name": "Surface",
"y": 12, "h": 10,
"color": "#e85d04", "opacity": 0.05,
"whisperKey": "surface"
}
],
"voice": "first-person-casual" | "third-person-formal" | "mixed",
"language": "en" | "ru" | "mixed"
}
}Entities
Organized by type. Each entity has both canonical and poetic identities.
{
"entities": {
"gods": [
{
"key": "drd2",
"canonical": "DRD2 - Dopamine D2 Receptor",
"poetic": "The Reward Gate",
"type": "god",
"desc": "Short poetic description (1-2 sentences)",
"lore": "Longer mythological narrative (2-4 sentences)",
"domain": "neurochemistry",
"connects": ["comt", "dat1", "reward-deficiency"],
"source": "single" | "a" | "b" | "shared",
"vaultNote": "DRD2 - Dopamine D2 Receptor.md",
"centrality": 0.89,
"position": { "ring": 0, "angle": 0.0, "y": 10 }
}
],
"demigods": [ ... ],
"tensions": [
{
"key": "building-vs-calm",
"canonical": "Building vs Calm",
"poetic": "Scaffolding ↔ Stillness",
"type": "tension",
"poles": [
{ "name": "The Architect's Drive", "entities": ["trail-lab", "motka"] },
{ "name": "The Monk's Silence", "entities": ["calm-life", "mindfulness"] }
],
"desc": "The pull between productive creation and nervous system rest.",
"connects": ["gad", "exercise"],
"position": { "ring": 3, "angle": 1.2, "y": -2 }
}
],
"narratives": [ ... ],
"blindSpots": [ ... ],
"spirits": [ ... ],
"crystals": [ ... ],
"research": [ ... ],
"values": [ ... ],
"trails": [ ... ],
"questions": [ ... ],
"depths": [ ... ],
"growth": []
}
}Levels (Abstraction Hierarchy)
{
"levels": [
{
"id": 0,
"name": "Entities",
"zoomThreshold": 0,
"description": "Individual knowledge nodes",
"always": true
},
{
"id": 1,
"name": "Domains",
"zoomThreshold": 80,
"description": "Clusters of related entities",
"confidence": 0.85,
"clusters": [
{
"key": "neurochemistry",
"canonical": "Neurochemistry",
"poetic": "The Chemical Garden",
"members": ["drd2", "comt", "fkbp5", "sertraline"],
"exemplar": "drd2",
"centroid": { "x": 5.2, "y": 8.1, "z": -3.4 },
"color": [0.9, 0.4, 0.1],
"desc": "The genetic and pharmacological machinery beneath mood and motivation."
}
]
},
{
"id": 2,
"name": "Axes",
"zoomThreshold": 200,
"description": "Fundamental tensions and force fields",
"confidence": 0.72,
"axes": [
{
"key": "building-calm",
"poleA": { "name": "Building", "domains": ["neurochemistry", "tech"], "position": { "x": -20, "y": 0, "z": 0 } },
"poleB": { "name": "Calm", "domains": ["embodied", "therapy"], "position": { "x": 20, "y": 0, "z": 0 } },
"desc": "The vault's central tension: productive creation vs. nervous system rest."
}
]
},
{
"id": 3,
"name": "Comparison",
"zoomThreshold": 500,
"description": "Two-vault overlap and divergence",
"confidence": 0.65,
"overlap": {
"score": 0.34,
"sharedEntities": 18,
"sharedDomains": 3,
"sharedAxes": 1
}
}
]
}Mappings (Cross-Level Crosswalks)
{
"mappings": {
"entityToDomain": {
"drd2": "neurochemistry",
"skateboarding": "embodied-practice"
},
"domainToAxis": {
"neurochemistry": { "axis": "building-calm", "pole": "A", "weight": 0.7 },
"embodied-practice": { "axis": "building-calm", "pole": "B", "weight": 0.8 }
}
}
}Secrets
{
"secrets": [
{
"trigger": ["drd2", "skateboarding"],
"insight": "The same receptor that makes rewards feel faint also makes vestibular stimulation therapeutic.",
"status": "hidden"
}
]
}Whispers (Per-Layer Ambient Text)
{
"whispers": {
"surface": ["The genome speaks in probabilities", "Every receptor is a door half-open"],
"middle": ["Tension is not a problem to solve", "The body remembers what the mind forgets"],
"deep": ["What you avoid shapes you more than what you pursue"]
}
}Fields (Atmospheric Zones)
{
"fields": [
{
"key": "avoidance-field",
"name": "The Avoidance Field",
"center": [0, -5, 0],
"radius": 15,
"color": [0.3, 0.1, 0.4],
"desc": "Where blind spots and safety behaviours cluster."
}
]
}Confidence
{
"confidence": {
"overall": 0.78,
"perLevel": { "0": 1.0, "1": 0.85, "2": 0.72 },
"lowConfidenceEntities": ["crystal-3", "narrative-7"],
"notes": "Level 2 axes are interpretive — vault has few explicit opposing pairs."
}
}Merged Mode Additional Fields
When mode: "merged":
{
"sources": [
{ "name": "Gleb's Vault", "color": "#4ecdc4", "entityCount": 142 },
{ "name": "Partner Vault", "color": "#ff6b6b", "entityCount": 89 }
]
}Each entity gains "source": "a" | "b" | "shared" and optionally "sourceA_key" / "sourceB_key" for semantic matches.
Dual-Graph Merge Algorithm
How to build a common map between two vaults.
Overview
The merged temple uses a shared scaffold with divergence offsets (not split-hemisphere, not pure overlay).
- Shared domains/axes occupy canonical center positions
- Vault A entities appear as color-coded satellites biased left
- Vault B entities appear as color-coded satellites biased right
- Overlap entities sit at center, scaled larger
- Divergence leans outward
Merge Pipeline
1. Independent Classification
Run the full classification pipeline (Steps 1-4 of SKILL.md) for each vault independently. This produces two scene packages with their own entities, levels, and mappings.
2. Entity Matching
Three match types:
Exact match: Same canonical key exists in both vaults (e.g., both have anxiety-management).
- Confidence: 1.0
- These become shared entities immediately.
Semantic match: Different keys, same concept. Claude identifies these by reading entity descriptions and connections.
- Example: Vault A has
stress-response, Vault B hascortisol-regulation - Confidence: 0.5-0.9 (Claude assigns based on semantic overlap)
- Present matches to user for confirmation before generating the merged temple.
Unique: Only exists in one vault.
- Source:
"a"or"b" - These become divergence entities.
3. Domain Alignment
After entity matching, align Level 1 domains:
1. For each domain in Vault A, count how many of its members have matches in Vault B 2. If > 50% members match, find the Vault B domain with the most matched members → these are aligned domains 3. Aligned domains merge into a single shared domain with members from both vaults 4. Unaligned domains remain vault-specific
4. Axis Reconciliation (Level 2)
Compare tension axes across vaults:
- If both vaults have a tension with matching poles → shared axis
- If poles partially overlap → note the divergence in axis description
- If entirely different axes → vault-specific axes
5. Layout Generation
Shared entities: Position at the canonical center. Scale 1.2x normal. Dual-glow effect (both vault colors).
Vault A unique: Offset by -8 on X axis from their natural ring position. Tinted with Vault A color.
Vault B unique: Offset by +8 on X axis. Tinted with Vault B color.
Connections:
- Within-vault connections: colored by vault (A = blue tint, B = green tint)
- Cross-vault connections (through shared entities): gold/amber
6. Merged Scene Package
Additional fields in temple-data.json:
{
"mode": "merged",
"sources": [
{ "name": "Vault A Name", "color": "#4ecdc4", "entityCount": 45 },
{ "name": "Vault B Name", "color": "#ff6b6b", "entityCount": 62 }
],
"overlap": {
"score": 0.34,
"sharedEntities": 18,
"sharedDomains": 3,
"sharedAxes": 1
},
"entities": [
{ "key": "...", "source": "shared", "sourceA_key": "...", "sourceB_key": "...", ... },
{ "key": "...", "source": "a", ... },
{ "key": "...", "source": "b", ... }
]
}7. Template Behavior in Merged Mode
The template detects mode: "merged" and activates:
- Split-color rendering (left/right have different ambient tones)
- Toggle button: All / Vault A only / Vault B only / Shared only
- Shared nodes get dual-glow effect
- HUD displays overlap score
- Audio: stereo/dialogic treatment — Vault A motifs in left channel, Vault B in right, shared entities in center
Confidence Thresholds
- Don't generate Level 3 if overlap score < 0.1 (vaults have almost nothing in common)
- Flag low-confidence matches (< 0.6) for user review
- Minimum shared entities: 5 — below this, merged view adds noise not insight
User Confirmation
Before generating the merged temple, present to the user: 1. List of exact matches (auto-approved) 2. List of semantic matches with confidence scores (needs approval) 3. Proposed domain alignments 4. Overall overlap score
Only proceed after user confirms the match list.
#!/usr/bin/env python3
"""
Vault Scanner for Temple Generator.
Scans an Obsidian vault, extracts files/links/tags, builds a link graph,
computes degree centrality, detects clusters, and outputs vault-scan.json.
Usage: python3 extract_entities.py <vault_path> [--output vault-scan.json]
"""
import json
import os
import re
import sys
from collections import defaultdict
from pathlib import Path
SKIP_DIRS = {
'.obsidian', '.git', '.claude', '.trash', 'node_modules',
'.DS_Store', 'Templates', 'Attachments', 'assets'
}
SKIP_EXTENSIONS = {'.png', '.jpg', '.jpeg', '.gif', '.svg', '.pdf', '.mp3',
'.mp4', '.wav', '.ogg', '.zip', '.html', '.css', '.js',
'.json', '.py', '.pkl', '.lock'}
def parse_frontmatter(lines):
"""Extract YAML frontmatter as a dict of strings."""
fm = {}
if not lines or lines[0].strip() != '---':
return fm
for line in lines[1:]:
if line.strip() == '---':
break
if ':' in line:
key, _, val = line.partition(':')
fm[key.strip()] = val.strip().strip("'\"")
return fm
def extract_wikilinks(text):
"""Extract [[wikilink]] targets, ignoring display text after |."""
return [m.split('|')[0].strip() for m in re.findall(r'\[\[([^\]]+)\]\]', text)]
def extract_tags(text):
"""Extract #tags from text (not inside code blocks)."""
return re.findall(r'(?<!\S)#([a-zA-Z][\w/-]*)', text)
def scan_vault(vault_path):
"""Walk the vault and collect file metadata."""
vault = Path(vault_path).resolve()
files = {}
all_links = defaultdict(list) # source -> [targets]
for root, dirs, filenames in os.walk(vault):
# Skip hidden/system dirs
dirs[:] = [d for d in dirs if d not in SKIP_DIRS and not d.startswith('.')]
rel_root = Path(root).relative_to(vault)
for fname in filenames:
fpath = Path(root) / fname
if fpath.suffix.lower() in SKIP_EXTENSIONS:
continue
if fpath.suffix.lower() != '.md':
continue
rel_path = str(rel_root / fname)
try:
text = fpath.read_text(encoding='utf-8', errors='replace')
except Exception:
continue
lines = text.split('\n')
fm = parse_frontmatter(lines)
links = extract_wikilinks(text)
tags = extract_tags(text)
word_count = len(text.split())
# Title: frontmatter title > first H1 > filename
title = fm.get('title', '')
if not title:
for line in lines:
if line.startswith('# ') and not line.startswith('## '):
title = line[2:].strip()
break
if not title:
title = fpath.stem
# Clean title of quotes
title = title.strip('"\'')
folder = str(rel_root) if str(rel_root) != '.' else ''
stem = fpath.stem
files[stem] = {
'path': rel_path,
'title': title,
'stem': stem,
'folder': folder,
'tags': tags[:20], # cap tags
'links': links[:50], # cap links
'wordCount': word_count,
'frontmatter': {k: v for k, v in list(fm.items())[:10]},
}
all_links[stem] = links
return files, all_links
def build_graph(files, all_links):
"""Build adjacency list and compute backlinks."""
# Normalize link targets to stems
stem_set = set(files.keys())
# Also build a lowercase lookup for fuzzy matching
lower_to_stem = {}
for s in stem_set:
lower_to_stem[s.lower()] = s
edges = []
backlinks = defaultdict(int)
for source, targets in all_links.items():
seen = set()
for t in targets:
# Resolve target to a known stem
t_clean = t.split('#')[0].split('|')[0].strip()
resolved = None
if t_clean in stem_set:
resolved = t_clean
elif t_clean.lower() in lower_to_stem:
resolved = lower_to_stem[t_clean.lower()]
if resolved and resolved != source and resolved not in seen:
edges.append([source, resolved])
backlinks[resolved] += 1
seen.add(resolved)
# Add backlink counts to files
for stem, count in backlinks.items():
if stem in files:
files[stem]['backlinks'] = count
# Ensure all files have backlinks field
for stem in files:
if 'backlinks' not in files[stem]:
files[stem]['backlinks'] = 0
return edges
def compute_centrality(files, edges):
"""Compute degree centrality (in + out links)."""
degree = defaultdict(int)
for a, b in edges:
degree[a] += 1
degree[b] += 1
max_degree = max(degree.values()) if degree else 1
centrality = {}
for stem in files:
d = degree.get(stem, 0)
centrality[stem] = round(d / max_degree, 4)
files[stem]['centrality'] = centrality[stem]
files[stem]['degree'] = d
return centrality
def detect_clusters(files, edges, min_cluster_size=3):
"""Simple connected-component clustering with folder hints."""
# Build adjacency for connected components
adj = defaultdict(set)
for a, b in edges:
adj[a].add(b)
adj[b].add(a)
visited = set()
clusters = []
def bfs(start):
queue = [start]
component = set()
while queue:
node = queue.pop(0)
if node in visited:
continue
visited.add(node)
component.add(node)
for neighbor in adj[node]:
if neighbor not in visited:
queue.append(neighbor)
return component
for stem in files:
if stem not in visited:
component = bfs(stem)
if len(component) >= min_cluster_size:
clusters.append(sorted(component))
# Sort clusters by size (largest first)
clusters.sort(key=len, reverse=True)
# Also add folder-based clusters
folder_clusters = defaultdict(list)
for stem, info in files.items():
if info['folder']:
folder_clusters[info['folder']].append(stem)
folder_groups = []
for folder, members in folder_clusters.items():
if len(members) >= min_cluster_size:
folder_groups.append({
'type': 'folder',
'name': folder,
'members': sorted(members)
})
return {
'connected': [{'type': 'connected', 'members': c} for c in clusters[:20]],
'folders': folder_groups[:15]
}
def main():
if len(sys.argv) < 2:
print("Usage: python3 extract_entities.py <vault_path> [--output file.json]")
sys.exit(1)
vault_path = sys.argv[1]
output_path = 'vault-scan.json'
if '--output' in sys.argv:
idx = sys.argv.index('--output')
if idx + 1 < len(sys.argv):
output_path = sys.argv[idx + 1]
if not os.path.isdir(vault_path):
print(f"Error: {vault_path} is not a directory")
sys.exit(1)
print(f"Scanning vault: {vault_path}")
files, all_links = scan_vault(vault_path)
print(f" Found {len(files)} markdown files")
edges = build_graph(files, all_links)
print(f" Found {len(edges)} link edges")
centrality = compute_centrality(files, edges)
# Top nodes by centrality
top_nodes = sorted(centrality.items(), key=lambda x: x[1], reverse=True)[:30]
print(f" Top 10 by centrality:")
for stem, c in top_nodes[:10]:
title = files[stem]['title']
degree = files[stem]['degree']
print(f" {c:.3f} ({degree:3d} links) {title}")
clusters = detect_clusters(files, edges)
print(f" Found {len(clusters['connected'])} connected clusters, {len(clusters['folders'])} folder clusters")
# Build output
# Sort files by centrality for the output
sorted_files = sorted(files.values(), key=lambda f: f.get('centrality', 0), reverse=True)
result = {
'vaultPath': str(Path(vault_path).resolve()),
'totalFiles': len(files),
'totalEdges': len(edges),
'files': sorted_files[:200], # cap at 200 most central
'edges': edges[:500], # cap edges
'clusters': clusters,
'topNodes': [{'stem': s, 'centrality': c, 'title': files[s]['title']} for s, c in top_nodes],
}
with open(output_path, 'w', encoding='utf-8') as f:
json.dump(result, f, indent=2, ensure_ascii=False)
print(f"\nOutput written to: {output_path}")
print(f" {len(sorted_files[:200])} files, {len(edges[:500])} edges")
if __name__ == '__main__':
main()