
Knowledge Harvester
- 10 installs
- 30 repo stars
- Updated July 7, 2026
- jeffallan/writing-with-agents
Captures research artifacts and finished writing outputs back into a vault or knowledge base with consistent frontmatter, citations, and cross-links.
About
Formats and files research sources, synthesized connections, domain maps, and produced articles back into a knowledge base, closing the research feedback loop. A writer uses it at the end of a project to preserve and link material for future work.
- Captures four artifact types with consistent structure, frontmatter, and linking
- Links produced articles back to the source notes that informed them
Knowledge Harvester by the numbers
- 10 all-time installs (skills.sh)
- +1 installs in the week ending Aug 2, 2026 (Skillselion tracking)
- Ranked #1,137 of 1,879 Documentation skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/jeffallan/writing-with-agents --skill knowledge-harvesterAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 10 |
|---|---|
| repo stars | ★ 30 |
| Last updated | July 7, 2026 |
| Repository | jeffallan/writing-with-agents ↗ |
What it does
Captures research artifacts and finished writing outputs back into a vault or knowledge base with consistent frontmatter, citations, and cross-links.
Files
Role Definition
The Knowledge Harvester captures research artifacts and writing outputs back into the user's vault or knowledge base, creating a feedback loop where each content creation cycle enriches the source material for future work.
Lead: AI formats artifacts with consistent structure, frontmatter, and linking. Support: Human approves what gets captured, where it goes, and how it connects to existing notes.
This skill handles four categories of artifacts: research sources discovered during gap-filling, synthesized connections identified during content creation, domain maps (whirlybirds) as persistent reference artifacts, and links from produced articles back to the source notes that informed them.
The Knowledge Harvester does not generate new research or content. It captures, formats, and files what was produced during the research-intake, content-strategist, and Flowers cycle phases. Every artifact it writes was already validated by the human during those upstream phases.
When to Use This Skill
- After completing a writing project and wanting to preserve research for future use
- After gap-filling research during research-intake that should be saved permanently
- When a produced article should link back to its source notes in the vault
- When domain whirlybirds should be saved as persistent reference artifacts
- When synthesized connections or insights emerged during writing that should be captured
- When enriching a knowledge base with citation metadata from content creation
- At the end of a content strategy cycle to close the feedback loop
- When consolidating scattered research notes into structured vault entries
- When a content cluster is complete and cross-references between articles need permanent capture
- When domain expertise accumulated over multiple articles should be formalized as reference material
Core Workflow
1. Identify artifacts to capture -- Review the outputs from the completed writing phases. Catalog what should be captured: research sources from gap-filling, connections discovered during content creation, domain whirlybirds, and article-to-source links. Present the catalog to the human for confirmation. Do not assume everything should be captured -- the human decides what has lasting value.
2. Format each artifact as structured markdown -- Apply consistent frontmatter, heading structure, and linking conventions to each artifact. Every artifact gets YAML frontmatter with type, domain, date, tags, and relationship metadata. See references/vault-format.md for the full format specification.
3. Present formatted artifacts to human for approval -- Use AskUserQuestion to show each formatted artifact (or a summary for large batches) and ask the human to approve, modify, or reject. Do not write to the vault without explicit approval. The human may adjust titles, tags, connections, or decide to skip specific artifacts.
4. Write approved artifacts to vault -- Save each approved artifact to the vault path established during the research-intake session setup. Match the existing vault's file naming and folder conventions. If no convention exists, use lowercase hyphenated filenames in the root vault directory.
5. Confirm capture with summary -- After all approved artifacts are written, present a summary of what was saved and where. Include a count of new notes, updated connections, and the enrichment this cycle added to the vault. State the feedback loop status: the vault is now richer for the next content creation cycle.
Reference Guide
| Topic | Reference | Load When |
|---|---|---|
| Artifact types, the feedback loop, what to capture | references/harvest-process.md | Identifying artifacts, understanding the feedback loop |
| Frontmatter templates, tagging, linking, Obsidian compatibility | references/vault-format.md | Formatting any artifact for vault capture |
Constraints
MUST DO:
- Present all artifacts to the human for approval before writing to the vault
- Apply consistent frontmatter to every captured artifact
- Match the existing vault's naming and folder conventions
- Include relationship metadata linking artifacts to source notes and produced articles
- Tag each artifact with the domain and content cycle that produced it
- Confirm the vault path before writing any files (use the path from research-intake session setup)
- Provide a capture summary after all artifacts are written
MUST NOT DO:
- Write to the vault without human approval
- Modify existing vault files unless explicitly asked to update them
- Invent metadata or connections that were not established during upstream phases
- Skip frontmatter or tagging -- every artifact gets structured metadata
- Assume a vault folder structure -- discover it from what exists or ask the human
- Generate new research or content -- this skill captures, it does not create
- Overwrite existing vault notes without explicit human instruction to update
- Create duplicate entries for artifacts that already exist in the vault -- check before writing
- Capture artifacts without relationship metadata -- every note must link to its origin phase and related artifacts
- Strip or simplify frontmatter to save time -- incomplete metadata breaks future indexing passes
- Capture speculative connections that were not validated during upstream phases
- Batch large artifact sets without giving the human an opportunity to review individual items
Output Templates
Capture Catalog (presented before writing):
## Artifacts Ready for Capture
| # | Artifact | Type | Proposed Filename | Source Phase |
|---|----------|------|-------------------|-------------|
| 1 | [Source title] | Research source | [filename.md] | Research-intake |
| 2 | [Connection description] | Synthesis note | [filename.md] | Content creation |
| 3 | [Domain whirlybird] | Domain map | [filename.md] | Content-strategist |
| 4 | [Article backlink] | Article link | [filename.md] | Flowers cycle |
Vault path: [confirmed path]
Approve all, select by number, or modify?Capture Summary (delivered after writing):
## Knowledge Harvest Complete
**Vault path:** [path]
**Artifacts captured:** [count]
**New research sources:** [count]
**Synthesis notes:** [count]
**Domain maps saved:** [count]
**Article backlinks:** [count]
The vault is now enriched with material from the [domain] content cycle.
Next content creation cycle in this domain will benefit from [count] new source notes and [count] documented connections.Knowledge Reference
The feedback loop is the core design principle of this skill. Without harvest, each content creation cycle starts from scratch. With harvest, each cycle inherits the research, connections, and domain maps from previous cycles. Over time, the vault becomes a compounding knowledge asset where later articles benefit from the accumulated research of earlier ones.
The four artifact types serve distinct purposes in the feedback loop. Research sources are raw material discovered during gap-filling -- they feed directly into future research-intake passes. Synthesized connections are insights that emerged during content creation but were not present in any single source -- they represent original thinking worth preserving. Domain maps (whirlybirds saved as persistent references) provide spatial overviews of a knowledge domain that inform future content strategy decisions. Article backlinks connect the produced article to its source notes, creating bidirectional traceability between published content and the research that informed it.
Frontmatter consistency across all captured artifacts enables programmatic discovery. When every artifact carries structured YAML metadata with type, domain, date, tags, and relationship fields, future research-intake passes can index the vault efficiently. Inconsistent metadata forces manual discovery and defeats the purpose of structured capture. The frontmatter templates in the reference files enforce this consistency at the point of creation rather than relying on retroactive cleanup.
The human approval gate before vault writes serves two purposes. First, it prevents low-value artifacts from cluttering the knowledge base. Not every research finding or connection justifies permanent storage. The human filters for lasting value. Second, it gives the human an opportunity to adjust titles, tags, and connections before the artifact enters the vault's link graph. Adjustments at write time are trivial. Corrections after the artifact has been linked to by other notes are disruptive.
Knowledge Harvest Process
This reference details what artifacts to capture, the feedback loop concept, and the step-by-step process for identifying, cataloging, and presenting artifacts for vault capture.
---
The Feedback Loop
The knowledge harvester exists to close a loop that most content creation workflows leave open:
Vault --> Research Intake --> Content Creation --> Knowledge Harvest --> Vault (enriched)Without the harvest step, research gathered during content creation evaporates. Sources are found, used, and forgotten. Connections are synthesized, applied to one article, and lost. Domain maps are built, used for planning, and discarded.
With the harvest step, every content creation cycle makes the vault denser and more valuable. The next time you write about this domain or an adjacent one, the research corpus is richer, the connections are documented, and the sources are cataloged with metadata.
Compound Returns
Each cycle through the loop produces compound returns:
- Cycle 1: Vault has scattered notes. Research-intake builds a knowledge map. Content is produced. Harvest captures 10 sources, 5 connections, 1 domain map.
- Cycle 2: Vault now has structured sources. Research-intake finds them immediately. Gap analysis is faster because existing coverage is clear. Harvest captures 8 new sources, 12 connections (including cross-cycle links).
- Cycle 3: Vault is dense enough that some articles require minimal new research. The knowledge map reveals new angles from accumulated connections. Each cycle builds on the last.
The value is not in any single artifact but in the accumulation over multiple cycles.
---
What to Capture
Four categories of artifacts emerge from a content creation cycle. Not every cycle produces all four, and the human decides which artifacts from each category have lasting value.
1. Research Sources
Sources discovered during the research-intake gap-filling process. These are external references that informed the content.
What to capture for each source:
- Title, author, URL, and access date
- Key claims extracted from the source
- Which articles the source informed
- How the source connects to other vault notes
- Relevance assessment: how central was this source to the content?
When to capture: After gap-filling research is validated during research-intake, or after the Flowers cycle reveals that a source was particularly valuable.
When NOT to capture: If the source was consulted but did not meaningfully inform the content. Not every search result deserves a vault note. Capture sources that you would want to find again.
2. Synthesized Connections
Insights that emerged during content creation that were not present in any single source. These are the cross-domain links, contradictions, patterns, and novel framings that the writing process surfaced.
What to capture for each connection:
- The connection itself (what relates to what, and how)
- Which sources or vault notes are connected
- Which article surfaced this connection
- Whether the connection is confirmed (supported by evidence) or hypothesized (identified during writing but not yet verified)
When to capture: After the Architect phase (which identifies connections during triage) or after the Judge phase (which may surface connections during detection passes).
When NOT to capture: If the connection is trivial or obvious within the domain. Capture connections that would be valuable to rediscover.
3. Domain Maps
Whirlybirds (Mermaid mindmaps) created during the content-strategist phase. These are spatial representations of the knowledge domain that have value as persistent reference artifacts.
What to capture for each domain map:
- The Mermaid mindmap diagram
- The center of gravity and what it means
- Which articles were derived from which branches
- Date created and whether the map is current or superseded
When to capture: After the content-strategist phase completes and production begins. The domain whirlybird is a snapshot of the domain understanding at a point in time.
When NOT to capture: If the domain whirlybird was a rough draft that was immediately superseded by a better version. Capture the version the human selected and used for planning.
4. Article Backlinks
Links from produced articles back to the source notes and research that informed them. These backlinks create bidirectional navigation: from a vault note, you can see which articles it informed; from an article, you can trace back to its sources.
What to capture for each backlink:
- The produced article title and location
- Which vault notes or sources informed it
- Which branch of the domain whirlybird it came from
- Publication date and status
When to capture: After an article completes the Flowers cycle (Judge phase complete, or quality-rubric passes).
When NOT to capture: If the article was abandoned or the content was merged into another piece. Capture backlinks for completed, published, or publication-ready content.
---
Artifact Identification Process
After a content creation cycle completes (or at any point the human requests a harvest), walk through the following steps to identify what to capture:
Step 1: Review Research-Intake Outputs
- Scan the knowledge map for sources cataloged during intake
- Check gap-fill notes for sources discovered during research
- Identify any connections that were documented in the knowledge map
Step 2: Review Content-Strategist Outputs
- Locate the selected domain whirlybird
- Note which articles were planned and which branches they map to
- Check for strategic decisions documented during topology selection
Step 3: Review Flowers Cycle Outputs
- For each completed article, identify the sources it drew from
- Check the Architect blueprint for connections identified during triage
- Check the Judge report for insights surfaced during detection passes
- Note any new research conducted during the Madman phase
Step 4: Build the Artifact Catalog
Compile all identified artifacts into a catalog table:
| # | Artifact | Type | Source Phase | Vault Value |
|---|----------|------|-------------|-------------|
| 1 | [description] | Research source | Research-intake | [why this is worth keeping] |
| 2 | [description] | Synthesis note | Architect | [why this is worth keeping] |
| 3 | [description] | Domain map | Content-strategist | [why this is worth keeping] |
| 4 | [description] | Article backlink | Judge | [why this is worth keeping] |Step 5: Present Catalog to Human
Use AskUserQuestion to present the catalog and ask the human to approve, modify, or reject each artifact. The human may:
- Approve all artifacts
- Select specific artifacts by number
- Modify proposed filenames, tags, or connections
- Add artifacts the AI missed
- Reject artifacts that do not have lasting value
Do not write any files until the human responds.
---
Batch Processing
When the artifact count is large (more than 10), present a summary rather than individual previews:
I identified [N] artifacts to capture:
- [X] research sources
- [Y] synthesis notes
- [Z] domain maps
- [W] article backlinks
Would you like to:
A) Review each artifact individually before capture
B) Review a summary of each category, then approve by category
C) Approve all and I will write them with standard formatting
D) Select specific categories to capture (skip others)For option A, present artifacts one at a time. For option B, show one example from each category plus a list of titles. For option C, proceed with capture using standard formatting from vault-format.md.
Vault Format Reference
This reference specifies the frontmatter templates, tagging conventions, linking conventions, and Obsidian compatibility requirements for all artifacts captured by the knowledge-harvester skill.
---
Frontmatter Templates
Every artifact written to the vault includes YAML frontmatter. The frontmatter structure varies by artifact type.
Research Source
---
type: research-source
domain: [domain topic from knowledge map]
date_captured: [ISO 8601 date, e.g., 2026-02-08]
source_author: [author name, or "unknown"]
source_url: [URL, or "N/A" for offline sources]
source_date: [publication date if known, or "undated"]
related_articles:
- [title of produced article that used this source]
tags:
- [topic tag 1]
- [topic tag 2]
- [topic tag 3]
---
# [Source Title]
**Author:** [author name]
**URL:** [url]
**Published:** [date]
**Accessed:** [date this source was consulted]
## Key Claims
- [claim 1]
- [claim 2]
- [claim 3]
## Relevance
[1-2 sentences on why this source matters to the domain and how it was used]
## Used In
- [[article-title]] -- [how this source informed the article]
## Connection To
- [[related-vault-note-1]] -- [nature of the connection]
- [[related-vault-note-2]] -- [nature of the connection]Synthesis Note
---
type: synthesis
domain: [domain topic]
date_captured: [ISO 8601 date]
confidence: [confirmed / hypothesized]
surfaced_during: [phase name -- e.g., Architect, Judge]
related_articles:
- [article title]
tags:
- [topic tag 1]
- [topic tag 2]
---
# [Connection or Insight Title]
## The Connection
[Clear statement of what relates to what, and how]
## Supporting Evidence
- [evidence point 1, with source]
- [evidence point 2, with source]
## Implications
[What this connection means for the domain or for future content]
## Source Notes
- [[source-note-1]] -- [role in this connection]
- [[source-note-2]] -- [role in this connection]
## Surfaced During
[Which article and which phase revealed this connection]Domain Map
---
type: domain-map
domain: [domain topic]
date_captured: [ISO 8601 date]
map_status: [current / superseded]
articles_derived:
- [article title 1]
- [article title 2]
tags:
- [domain tag]
- domain-map
---
# Domain Map: [Domain Name]
## Center of Gravity
[What the center represents and why it was chosen]
## Whirlybird
\```mermaid
mindmap
root((Domain Center))
Branch One
Feather 1a
Feather 1b
Branch Two
Feather 2a
Feather 2b
Branch Three
Feather 3a
Feather 3b
\```
## Branch Descriptions
- **Branch One:** [what this branch covers]
- **Branch Two:** [what this branch covers]
- **Branch Three:** [what this branch covers]
## Articles Derived from This Map
- [[article-1]] -- from Branch [N]
- [[article-2]] -- from Branch [N]
## Notes
[Any context about this map: why this center was chosen, what alternatives were considered, what gaps remain]Article Backlink
---
type: article-backlink
domain: [domain topic]
date_captured: [ISO 8601 date]
article_status: [draft / published]
publication_date: [date if published, or "unpublished"]
whirlybird_source: [branch label from domain whirlybird]
tags:
- [topic tag 1]
- [topic tag 2]
- article-backlink
---
# Article: [Article Title]
**Status:** [draft / published]
**Published:** [date or "not yet published"]
**Word count:** [approximate]
**Type:** [pillar / cluster / targeted / standalone]
## Source Material
This article drew from the following vault notes:
- [[source-note-1]] -- [how it was used]
- [[source-note-2]] -- [how it was used]
- [[source-note-3]] -- [how it was used]
## Domain Map Reference
Derived from: [[domain-map-note]], Branch [N]: [branch label]
## Key Arguments
- [Main argument 1 from the article]
- [Main argument 2 from the article]
## Cross-Links
- Links to: [[related-article-1]] -- [relationship]
- Links from: [[related-article-2]] -- [relationship]---
Citation Metadata Requirements
Every research source artifact must include sufficient metadata to relocate the source:
| Field | Required | Notes |
|---|---|---|
| Title | Yes | The title of the source document, article, or page |
| Author | Yes (use "unknown" if unavailable) | Person or organization |
| URL | Yes (use "N/A" for offline) | Full URL, not shortened |
| Source date | Yes (use "undated" if unavailable) | When the source was published or last updated |
| Access date | Yes | When the source was consulted during research |
For sources without URLs (books, interviews, personal communications), include enough identifying information to locate the source: ISBN, publication name, interview date, or other identifiers.
---
Tagging Conventions
Tags serve two purposes: topic discovery and artifact type filtering.
Topic Tags
- Use lowercase with hyphens:
api-security,content-strategy,rate-limiting - Use the same tag vocabulary as the knowledge map topics where possible
- Limit to 3-5 topic tags per artifact -- enough to find it, not so many that tags lose meaning
- Prefer specific tags over broad ones:
oauth-2-pkceoverauthentication
Type Tags
Every artifact automatically gets a type tag matching its frontmatter type:
research-sourcesynthesisdomain-maparticle-backlink
These type tags enable filtering all artifacts of a given kind.
Domain Tags
If the vault covers multiple domains, add a domain tag to every artifact:
domain/api-securitydomain/content-marketing
Use a consistent prefix so domain tags are visually distinct from topic tags.
---
Linking Conventions
Obsidian Wikilinks
Use Obsidian-style wikilinks for all internal vault references:
- [[note-title]] -- description of the link
- [[folder/note-title]] -- if the vault uses folder structure
- [[note-title|Display Text]] -- when the note title differs from desired display textLink Directionality
Every link should be bidirectional where possible:
- If Note A links to Note B, Note B should link back to Note A
- Research sources link to the articles they informed; article backlinks link to the sources
- Synthesis notes link to the source notes involved; source notes do not need to link back (to avoid cluttering source notes with every synthesis that references them)
External Links
For URLs to external content, use standard markdown links:
[Source Title](https://example.com/article)Do not use wikilinks for external URLs. Wikilinks are for vault-internal navigation only.
---
Obsidian Compatibility
All vault artifacts must be compatible with Obsidian's rendering and linking:
1. YAML frontmatter must be valid YAML between --- delimiters at the top of the file 2. Wikilinks use [[double brackets]] syntax 3. Tags in frontmatter use the tags: array format (not inline #tag in frontmatter) 4. Inline tags may be used in the body text as #tag-name where helpful for quick filtering 5. Mermaid diagrams render natively in Obsidian -- no special plugin required 6. Filenames should avoid special characters: use letters, numbers, hyphens, and spaces only 7. Folder paths in wikilinks use forward slashes: [[folder/note-title]]
Frontmatter Gotchas
- Dates must be quoted if they contain only numbers:
date_captured: "2026-02-08"or use ISO format which YAML parses as a string in most contexts - Lists in frontmatter use the indented dash format, not inline arrays
- Boolean values (
true/false) should not be quoted - Strings containing colons must be quoted:
source_url: "https://example.com"
---
File Naming
Convention Priority
1. Match the existing vault convention first. If the vault uses Title Case Names.md, follow that. If it uses lowercase-hyphenated.md, follow that. If it uses a date prefix like 2026-02-08-note-title.md, follow that.
2. If no convention is detectable, default to lowercase hyphenated: source-title-shortened.md
3. Never mix conventions within a single harvest batch. All artifacts from one harvest use the same naming convention.
Naming Patterns by Type
| Type | Pattern | Example |
|---|---|---|
| Research source | [short-source-title].md | api-security-owasp-top-10.md |
| Synthesis note | synthesis-[short-description].md | synthesis-rate-limiting-trust-connection.md |
| Domain map | domain-map-[domain].md | domain-map-api-security.md |
| Article backlink | article-[short-title].md | article-complete-guide-api-security.md |