Wiki Ingest

wiki-ingest

Process a source document: one literature note plus fan-out updates to existing notes. Two-pass shape (extract, then format + apply). You are the LLM.

Bundled files

This skill ships note-schema.md and wiki-maintain.mjs in its own directory. Determine that directory once — written <SKILL_DIR> below — and substitute its real absolute path into every Read and node call (never hardcode a username, .claude, or drive letter). <SKILL_DIR> is the folder this SKILL.md was loaded from; if you don't already know it, find it with Glob **/wiki-ingest/wiki-maintain.mjs (skills live under the host CLI's skills directory, e.g. ~/.claude/skills/ for Claude Code).

Then read <SKILL_DIR>\note-schema.md first — vault resolution, language rules, frontmatter, body skeleton, slug rule, and invariants. Everything below assumes it.

Steps

1. Resolve the vault path and output language. The vault is the directory where the CLI was started (the current working directory) unless --vault overrides it — see note-schema.md. Parse --lang, -p/--persona, -s/--structure, --vault; the remainder is the file argument. Normalize it: strip a leading @ (the file-reference marker CLIs like Claude Code prepend) and any surrounding quotes.

If the argument is an http(s) URL (matches ^https?://), fetch it instead of resolving a file:

• Fetch the URL with your web-fetch tool and reduce it to clean markdown — the main

readable content only, dropping nav/ads/boilerplate. For a YouTube URL (youtube.com/youtu.be/m.youtube.com), capture the video transcript rather than the page chrome.

• One fetch attempt only. If the fetch returns readable article prose, use it. If

it returns no usable readable content — a JS-rendered SPA, a login/paywall, raw serialized app state (React/JSON blobs), or near-empty text — stop and report that the URL could not be reduced to readable text, and tell the user to save the content to a local file and ingest that instead. Do not write scraper scripts, re-parse, or re-read fetched dumps in a loop — there is no second extraction path.

• Compute a slug from the page title (the same slug rule used for notes) and write the

markdown to <vault>\sources\<slug>.md with this frontmatter, then a blank line, then the content:

     ---
     title: <page title>
     url: <the url>
     type: <web | video-transcript>
     fetched: <YYYY-MM-DD>
     ---

• Set sourceTitle = the page title, sourceFile = <slug>.md (the file you just

wrote), and use the fetched markdown as the source content. Skip the file-resolution rules below and continue at step 2.

Otherwise resolve it to a real file:

• Bare filename — no / or \ (e.g. paper.md) → <vault>\sources\<name>.
• Otherwise — a path (e.g. from @C:\docs\paper.md, @sources/paper.md, or

./drafts/x.md) → treat as a literal path (relative to cwd or absolute); if it doesn't exist there, also try <vault>\sources\<name>.

Error if nothing resolves. Set sourceTitle = the resolved file's basename without extension.

Pin the source inside the vault. A note can only link to a source that lives in sources/. If the resolved file is not already inside <vault>\sources\, copy it there now — slugify its basename (same slug rule) and keep the original extension: Copy-Item "<resolved path>" "<vault>\sources\<slug><ext>". Set sourceFile = the basename with extension of the file in sources/ (e.g. paper.pdf, My-Notes.md).

2. Gather context. List notes/ basenames (existing-notes list) and read the domains taxonomy from wiki-config.json.

If -p/--persona <name> or -s/--structure <name> was given, read <vault>\templates\personas\<name>.md / <vault>\templates\structures\<name>.md. Error if a named file doesn't exist (Persona not found: <name> (looked in <vault>\templates\personas\<name>.md), same wording for structures). These are user-maintained, vault-local templates — empty templates/personas/ and templates/structures/ dirs already exist in every vault.

3. Pass 1 — Extract. Read the whole source with your Read tool:

• PDF (.pdf): Read extracts the text directly. If the PDF has more than 20

pages, read it in page-range chunks (pages: "1-20", "21-40", ...) and concatenate the results before extracting.

• Image (.png, .jpg, etc.): Read renders the image for you to view —

transcribe its visible content (text, diagrams, charts, labels) and use that transcription as the source content.

• Everything else: Read returns the file's text content directly.

Then produce, conceptually:

   {
     "summary": "thorough summary of the source's key facts, arguments, insights",
     "updates": [
       { "note": "<exact existing-note title, copied VERBATIM>", "addition": "<one focused paragraph of genuinely new information for that note>" }
     ]
   }

• Every note MUST be a title copied verbatim from the existing-notes list — never

translate it, never invent one. If no existing note needs updating, updates is empty.

• Each addition is one focused paragraph of genuinely new information (in the

resolved language).

• If a persona template was loaded, let its text shape the voice/framing of

summary. If a structure template was loaded, treat its text as a checklist of aspects the source may cover — extract information about them where the source addresses them, so they aren't neglected in summary. Both are pass-1 only: pass 2 below just reshapes whatever summary contains.

4. Pass 2 — Literature note. Format summary into the note schema as a literature note: type: literature. Set source: to a quoted wikilink to sourceFile (the source in sources/) — keep the extension for non-markdown files so Obsidian can resolve them, drop only a .md extension: source: "[[paper.pdf]]" for a PDF, source: "[[My-Notes]]" for My-Notes.md. Assign domain/topic against the taxonomy. Link only to existing notes. Compute the slug from the note title (fall back to sourceTitle) and write notes/<slug>.md.

5. Apply updates. For each { note, addition }:

• Compute the target path with the slug rule: notes/<slug-of-note>.md.
• If the file does not exist, skip it (report it as skipped — never create it).
• Else read it, capture its ## Human Insight body, integrate the addition

naturally into the most appropriate existing section (Source Facts, Synthesis, or Connections), preserving all existing content, bump updated: to today, restore the Human Insight body verbatim, and write it back.

• Append ^[<source-file-basename-without-extension>] to every bullet you add to

Source Facts — the citation marker tying the fact to the file in sources/ — and copy every existing ^[...] marker verbatim (see note-schema.md invariants).

6. Regenerate derived files:

   node "<SKILL_DIR>\wiki-maintain.mjs" "<vaultPath>" --op ingest --title "<sourceTitle>"

7. Report: the literature note path, the count of notes updated, and the list of any skipped (not-found) targets.

Wiki note schema & conventions

Shared reference for the note-writing skills (wiki-ask, wiki-rewrite, wiki-ingest). You are the LLM here — there is no external API. You do the generation yourself, then write files and run the maintenance helper.

Resolving the vault

The vault is the directory where the CLI was started — the current working directory. If the user runs Claude Code / Gemini CLI in D:\wiki, the vault is D:\wiki.

1. If an explicit --vault <path> argument is given, use it. 2. Otherwise, use the current working directory.

A vault contains: sources/, notes/, moc/, meta/, templates/personas/, templates/structures/, wiki-config.json, and WIKI.md. A fresh, empty directory is a valid vault — on the first run the maintenance helper scaffolds everything automatically: it creates the six subdirectories, writes a default wiki-config.json ({ "language": "zh", "domains": {} }), and generates a default WIKI.md from the bundled WIKI.template.md. This scaffolding is idempotent — existing files are never overwritten — so you don't need to create these yourself.

Output language

Resolve in this order: 1. --lang zh|en flag. 2. language field in wiki-config.json. 3. $env:WIKI_LANG. 4. Default zh (Simplified Chinese).

• zh — write all prose and frontmatter values (title, domain, topic, tags) in

Simplified Chinese, but keep widely-used technical terms and proper nouns in their original English form — do not translate them (AI, LLM, Prompt, Token, Docker, API, GPU, Transformer, product and company names, etc.).

• en — write everything in English.

Regardless of language, keep these structural tokens EXACTLY in English (the maintenance helper and Obsidian parse them by exact match):

• Section headings: ## Source Facts, ## Synthesis, ## Connections,

## Speculation, ## Open Questions, ## Human Insight

• Typed-link keywords: extends::, contradicts::, requires::, examples::, related::
• YAML keys: title:, type:, source:, domain:, topic:, tags:, aliases:, created:, updated:

Frontmatter

Output raw YAML between --- delimiters. No `\`\`\`yaml` fence, no code fence around the whole note.

---
title: <specific, unique noun phrase — Title Case, 3–7 words; distinctive enough to
  stand alone in an index. Avoid generic one-word labels ("Gemini", "Attention").
  Name the precise concept ("Scaled Dot-Product Attention"). For zh the value may be Chinese.>
type: <atomic | literature>
source: <a quoted wikilink to this note's source file in sources/, e.g. "[[My-Answer]]";
  KEEP the extension for non-markdown files so Obsidian can resolve them — "[[paper.pdf]]",
  NOT "[[paper]]" (which would resolve to paper.md). Empty when the note has no source file.>
domain: <closest match from the taxonomy, or a new concise domain>
topic: <closest match from the taxonomy, or a new concise topic>
tags: [tag-a, tag-b, tag-c]   # 3–6 tags, each a SINGLE token with NO spaces,
                              # kebab-case, no quotes (Obsidian rejects spaces in tags)
aliases: [<0–3 alternative names other notes might link by — the title's counterpart
  in the other language (English name for a Chinese title, or vice versa) and a
  widely-used abbreviation, when they exist; spaces allowed; [] when none>]
created: <YYYY-MM-DD today>
updated: <YYYY-MM-DD today>
---

Body skeleton (use these sections, in this order)

## Source Facts
Only what sources or established knowledge directly states. No interpretation here.
Present as a structured bulleted list — one discrete fact per bullet — and group
related bullets under bold sub-labels or short sub-headings when they cluster
naturally. Not a prose paragraph. Include inline citations as (Source: title) where
applicable.

## Synthesis
Cross-source interpretation — what the facts add up to. Clearly LLM-generated
inference, not source statements.

## Connections
Typed links ONLY. Relationship types:
  extends:: [[note]]       — this note builds on another
  contradicts:: [[note]]   — these claims conflict
  requires:: [[note]]      — this concept depends on another
  examples:: [[note]]      — concrete instances of this concept
  related:: [[note]]       — loose association
Use only the types that genuinely apply. Multiple links of one type are fine.
Every link must earn its place. Atomic notes: aim for 2–4 links. Literature notes:
up to 8 is reasonable.

## Speculation
Unverified but interesting inferences. Clearly marked as not established.

## Open Questions
What this note does not resolve. Gaps worth investigating.

## Human Insight
Leave this section completely empty (heading only). Reserved for the human author.

Filename (slug)

slug = <domain>-<topic>-<title> with every run of characters not in [a-zA-Z0-9一-鿿] replaced by -, then trim leading/trailing -. Alphanumerics and CJK ideographs are preserved, so Chinese domains/topics/titles produce Chinese filenames. Save the note to notes/<slug>.md. If the note has no domain or topic, fall back to the title alone. Naming is ultimately code-owned: wiki-maintain.mjs re-derives this name on every run and renames any note that drifts, rewriting inbound links — so just write your best name here and let the maintenance script normalize it.

Invariants (these hold regardless of the model behind this skill)

• Human Insight is sacred. Never write content under ## Human Insight. New notes:

leave it empty. When rewriting/updating an existing note, copy its existing Human Insight body back verbatim.

• No dead links. In ## Connections, only [[link]] to notes that **already

exist in `notes/` — matched by filename or** by a name in a note's aliases:. If none genuinely apply, leave Connections empty. Never invent a link to a note that does not exist.

• Citation markers are preserved verbatim. A Source Facts bullet may end with a

^[<source-name>] marker tying the fact to a file in sources/. When rewriting or updating a note, copy every existing marker unchanged with its bullet. When ingesting, append ^[<source-file-basename>] to each bullet you add to Source Facts.

• No code fences around the note or its frontmatter. Output clean Markdown only.

After writing — regenerate derived files

Run the bundled helper (next to this skill's SKILL.md) once at the end:

node "<this skill folder>\wiki-maintain.mjs" "<vaultPath>" --op <ask|rewrite|ingest> --title "<noteTitle>"

It rebuilds moc/*.md, meta/index.md, the wiki-config.json taxonomy, the WIKI.md domains block, and appends meta/log.md. Never hand-edit those files.

Wiki Schema

This document governs how the wiki is structured and maintained. The LLM reads it at the start of every session. You and the LLM co-evolve it over time.

This file was generated automatically on first use of the wiki skills. Edit it freely — only the auto-generated Domains block below is overwritten.

---

Philosophy

You source and ask questions. The LLM writes and maintains.

• Sources (sources/) — raw inputs, immutable. Never modified after ingestion.
• Notes (notes/) — LLM-generated pages. The LLM owns this layer entirely.
• MOCs (moc/) — per-domain navigation indexes. Auto-generated, never hand-edited.
• Meta (meta/) — index.md (catalog) and log.md (chronological record).

---

Note Structure

Every note follows this skeleton, in order:

## Source Facts
Only what sources or established knowledge directly states.
Inline citations as (Source: title) where applicable. No interpretation.

## Synthesis
Cross-source interpretation. What the facts add up to.
Clearly LLM-generated inference, not source statements.

## Connections
Typed links only:
  extends:: [[note]]       — this note builds on another
  contradicts:: [[note]]   — these claims conflict
  requires:: [[note]]      — this concept depends on another
  examples:: [[note]]      — concrete instances of this concept
  related:: [[note]]       — loose association

Use only the types that genuinely apply — not all types are required.
Multiple links of the same type are fine.
Every link should earn its place: if removing it costs nothing, drop it.
Atomic notes: aim for 2–4 links. Literature notes: up to 8 is reasonable.

## Speculation
Unverified but interesting inferences. Clearly marked as not established.

## Open Questions
What this note does not resolve. Gaps worth investigating.

## Human Insight
[Reserved for the human author. Never written to or modified by the LLM.]

---

Frontmatter Schema

Frontmatter is plain YAML between --- delimiters — no code fences inside:

---
title: concise noun-phrase title
type: atomic | literature
source: filename or title of the source document (literature notes only; empty for atomic)
domain: top-level domain (e.g. ai, engineering, language)
topic: sub-field within domain (e.g. llm, databases, grammar)
tags: [keyword-1, keyword-2, keyword-3]
created: YYYY-MM-DD
updated: YYYY-MM-DD
---

type:

• atomic — one concept, permanently valuable. Generated by ask.
• literature — tied to a specific source (paper, article, book). Generated by ingest.

source: the filename or title of the source document this note was derived from. Set automatically by ingest. Empty for atomic notes.

tags: 3–6 tags. Each tag MUST be a single token with no spaces — Obsidian rejects spaces in tags. Join multi-word concepts with hyphens (kebab-case), e.g. prompt-caching, kv-cache. Do not wrap tags in quotes. Technical terms stay in their original form (e.g. llm, docker).

---

Output Language

The output language is set by language in wiki-config.json (zh for Simplified Chinese, en for English).

• Prose and frontmatter values (title, domain, topic, tags) are written in the configured language.
• Technical terms and proper nouns stay in their original English form — never translate them (AI, LLM, Prompt, Token, Docker, API, GPU, Transformer, product and company names, …).
• Structural tokens stay in English, always, regardless of language. These are parsed by exact match when MOCs and the index are regenerated, so localizing them would break the wiki:
• Section headings: ## Source Facts, ## Synthesis, ## Connections, ## Speculation, ## Open Questions, ## Human Insight
• Typed-link keywords: extends::, contradicts::, requires::, examples::, related::
• YAML frontmatter keys: title:, type:, source:, domain:, topic:, tags:, created:, updated:

---

Taxonomy

Domains and topics are stored in wiki-config.json. The LLM uses existing entries when they fit and adds new ones when they don't. Edit wiki-config.json directly to rename or consolidate.

---

Domains

The current domains in the vault, regenerated from note frontmatter after every mutating command. Each links to its MOC.

<!-- domains:start (auto-generated — do not edit) --> _No domains yet._ <!-- domains:end -->

---

Operations

ask — user asks a question. LLM answers naturally (Layer 1), then formats into a note (Layer 2). Saves to notes/, updates moc/ and meta/index.md.

rewrite — user provides an existing file. LLM reformats it into the note schema. ## Human Insight is preserved exactly.

ingest — user drops a source into sources/. LLM reads it, writes a literature note, updates relevant existing notes, appends to meta/log.md.

lint — LLM scans the wiki for contradictions, stale claims, orphan notes, missing links, and concepts without pages. Report saved to meta/lint-YYYY-MM-DD.md.

---

Log Format

Entries in meta/log.md use this prefix for grep-ability:

## [YYYY-MM-DD] operation | Title

---

Conventions

• Note filenames: derived from the title by collapsing every run of characters outside [a-zA-Z0-9 + CJK ideographs ] to a single -, then trimming leading/trailing -. Alphanumerics and Chinese characters are preserved, so Chinese titles produce Chinese filenames (e.g. LLM-Prompt-Caching-KV缓存复用机制.md). English titles end up kebab-case.
• All notes live flat in notes/ — no subfolders
• Body links use [[note-title]] (Obsidian wikilink format)
• MOC links use [[slug|Title]] — slug is the safe filename, title is human-readable display text
• ## Human Insight is never overwritten by the LLM
• Sources in sources/ are never modified after being added