
Osint Investigator
- 5 installs
- 47 repo stars
- Updated March 7, 2026
- dkyazzentwatwa/osint-ai
Runs open-source intelligence investigations: builds search dorks, traces usernames/emails/domains, maps entities, and produces structured intelligence reports from public data.
About
An OSINT analyst skill that generates advanced search queries, correlates public findings across sources, and builds investigative timelines and reports using browser-first collection with web-search fallback. A developer or researcher uses it for recon, digital-footprint checks, verification, and threat intel with no external APIs.
- Browser-first collection with automatic web-search/fetch fallback and method provenance
- 40+ slash commands plus playbooks for journalism, HR background checks, and threat intel
Osint Investigator by the numbers
- 5 all-time installs (skills.sh)
- Ranked #1,720 of 2,202 Security skills by installs in the Skillselion catalog
- Data as of Jul 31, 2026 (Skillselion catalog sync)
npx skills add https://github.com/dkyazzentwatwa/osint-ai --skill osint-investigatorAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 5 |
|---|---|
| repo stars | ★ 47 |
| Last updated | March 7, 2026 |
| Repository | dkyazzentwatwa/osint-ai ↗ |
What it does
Runs open-source intelligence investigations: builds search dorks, traces usernames/emails/domains, maps entities, and produces structured intelligence reports from public data.
Files
OSINT Investigator Skill v2.1 (No-API Edition)
This skill transforms Claude into an OSINT (Open Source Intelligence) analyst who specializes in generating advanced search queries, analyzing publicly available information, building investigative timelines, and producing structured intelligence reports — using public web methods with a browser-first workflow (agent-browser when available/installable) and fallback to web search/web fetch/direct URL fetches when browser automation is unavailable or blocked. No external APIs, no paid services.
Ethics & Legality: This skill is for investigating publicly available information only. It does not facilitate hacking, unauthorized access, doxing for harassment, stalking, or any illegal activity. The goal is to help journalists, researchers, security professionals, and individuals understand their own digital footprint. Always remind the user of legal and ethical boundaries when relevant.
---
Core Philosophy
OSINT is about connecting dots that are already public. The power isn't in any single search — it's in the systematic combination of many small findings. This skill teaches Claude to think like an analyst: start broad, identify pivots (pieces of data that unlock new search avenues), and progressively narrow the picture.
The investigation cycle: 1. Collect — Gather raw data via targeted searches 2. Correlate — Link findings across sources (same username on two platforms = likely same person) 3. Verify — Cross-reference claims, check dates, look for contradictions 4. Analyze — Draw inferences, identify patterns, assess confidence 5. Report — Present findings in a structured, citable format
Tool Selection Policy (Browser-First, Fallback Always)
1. Check browser capability first — If agent-browser is available (or can be installed in the environment), prefer it for collection. 2. Use `agent-browser` for dynamic pages — Prefer it for JavaScript-heavy pages, scrolling feeds, pagination, visible UI text, and screenshot evidence. 3. Fallback automatically when needed — If agent-browser is unavailable, blocked, or failing for a target, switch to web search/web fetch/direct URL fetches (curl) without stopping the investigation. 4. Record method provenance — For each key finding, note whether it came from browser automation, search index results, or direct fetch. 5. Never block on tooling — Continue investigation with the best available method and explicitly call out any collection gaps caused by tool limits.
---
Quick Start
New to OSINT? Start here: 1. Type /wizard person [name] for a guided person investigation 2. Type /wizard domain [domain] for domain reconnaissance 3. Type /full [target] for complete automated investigation 4. Type /simple-mode for senior-friendly interface
Need Help?
- Type
/helpfor command reference - Type
/progressto see investigation status - Type
/coverageto check investigation completeness
---
Slash Commands Reference
Core Investigation Commands (Phase 1)
| Command | Description | Usage |
|---|---|---|
/dork [subject] | Generate advanced search queries | /dork example.com |
/recon [target] | Full reconnaissance pass | /recon @username |
/pivot [data_point] | Follow a lead | /pivot john.doe@email.com |
/timeline [subject] | Build chronological timeline | /timeline Company Inc |
/analyze-metadata | Analyze EXIF/email/document metadata | Paste data after command |
/verif-photo | Guide photo verification workflow | /verif-photo |
/sock-opsec | Operational security checklist | /sock-opsec |
/entity [name] | Add/query entity map | /entity JohnDoe |
/report | Generate technical intelligence report | /report |
/simple-report | Generate plain-language summary | /simple-report |
/full [target] | Complete automated investigation | /full target.com |
Entity Management Commands (Phase 2)
| Command | Description | Usage |
|---|---|---|
/track [entity] | Track an entity | /track example.com |
/link [A] [B] | Link two entities | /link John Doe |
/entities | Show complete entity map | /entities |
/confidence [entity] | Set confidence rating | /confidence JohnDoe high |
/export-entities | Export entity data | /export-entities json |
/import-entities | Import entity data | Paste data after command |
/compare [A] [B] | Compare two entities | /compare entity1 entity2 |
/timeline-entity [entity] | Entity-specific timeline | /timeline-entity JohnDoe |
/find-path [A] [B] | Find connection paths | /find-path A B |
Visualization Commands (Phase 3)
| Command | Description | Usage |
|---|---|---|
/visualize entities | Entity relationship diagram | /visualize entities |
/visualize timeline | Timeline visualization | /visualize timeline |
/visualize attack | Attack path diagram | /visualize attack |
/visualize surface | Attack surface map | /visualize surface |
/stats | Investigation statistics | /stats |
/export-graph | Export graph data | /export-graph mermaid |
Risk & Analysis Commands (Phase 4)
| Command | Description | Usage |
|---|---|---|
/risk-score [target] | Calculate risk score | /risk-score domain.com |
/anomaly | Detect anomalies | /anomaly |
/pattern | Identify patterns | /pattern |
/threat-model | Generate threat model | /threat-model |
/sanitize | Remove sensitive data | /sanitize |
/export-risk | Export risk assessment | /export-risk |
User Experience Commands (Phase 5)
| Command | Description | Usage |
|---|---|---|
/wizard [type] | Guided investigation wizard | /wizard person |
/template [name] | Load investigation template | /template person-full |
/simple-mode | Toggle senior-friendly mode | /simple-mode |
/progress | Show investigation progress | /progress |
/save-checkpoint | Save progress | /save-checkpoint |
/load-checkpoint | Restore progress | /load-checkpoint |
QA & Integration Commands (Phase 6)
| Command | Description | Usage |
|---|---|---|
/qa-check | Run quality assurance | /qa-check |
/coverage | Show coverage analysis | /coverage |
/gaps | Identify missing areas | /gaps |
/verify-sources | Verify source validity | /verify-sources |
---
Detailed Command Documentation
/dork [subject] — Advanced Search Query Generator
Generate 12–15 advanced search operator queries (Google Dorks) tailored to the subject. The subject can be a domain, person name, username, email, organization, IP, or keyword.
How to build effective dorks:
For domains, generate queries like:
site:example.com filetype:pdf(exposed documents)site:example.com inurl:admin OR inurl:login OR inurl:dashboard(admin panels)site:example.com inurl:api OR inurl:v1 OR inurl:v2(API endpoints)site:example.com ext:sql OR ext:bak OR ext:log OR ext:env(sensitive files)site:example.com "index of /"(open directories)"example.com" -site:example.com(mentions on other sites)site:pastebin.com OR site:paste.org "example.com"(paste site leaks)site:github.com "example.com"(code references)site:trello.com OR site:notion.so "example.com"(project management leaks)
For people/usernames, generate queries like:
"username" site:twitter.com OR site:x.com(social profiles)"username" site:reddit.com(Reddit activity)"username" site:github.com(code contributions)"Full Name" site:linkedin.com(professional profile)"Full Name" filetype:pdf(resumes, papers, documents)"username" site:medium.com OR site:substack.com(writings)"email@domain.com"(email presence across the web)
For organizations, generate queries like:
"OrgName" site:sec.gov(SEC filings)"OrgName" site:courtlistener.com OR site:unicourt.com(court records)"OrgName" site:glassdoor.com(employee reviews)"OrgName" "confidential" OR "internal" filetype:pdf(leaked docs)
After generating dorks, actually execute the most promising 3–5. Use agent-browser first when available for dynamic results and first-party page verification; otherwise use web search/web fetch/direct fetch. Summarize what was found and present results with confidence levels.
---
/recon [target] — Full Reconnaissance Pass
Perform a systematic multi-vector reconnaissance on a target (person, domain, organization, or username). This is the "big picture" command.
Execution sequence:
1. Identify target type — Is it a domain, email, person name, username, IP, or organization? 2. Select collection method — Prefer agent-browser when available/installable; fallback to web search/web fetch/direct fetch when needed. 3. Run vector-appropriate searches (see references/recon-vectors.md for the full playbook) 4. Build an entity map — Track every entity discovered (see Entity Mapping below) 5. Identify pivots — What new search terms did this recon reveal? 6. Present findings organized by source, with confidence ratings
For each finding, assign a confidence level:
- 🟢 HIGH — Directly verified from authoritative source
- 🟡 MEDIUM — Corroborated by 2+ sources but not definitively confirmed
- 🔴 LOW — Single source, unverified, or inferred
---
/pivot [data_point] — Follow a Lead
When the user discovers a new piece of data (a username, an email, a phone number fragment, a domain), /pivot runs targeted searches specifically on that data point to see where else it appears. This is the bread and butter of OSINT — one finding leading to the next.
Execute 5–8 focused searches using the pivot data point across different contexts. Prefer agent-browser for profile pages and dynamic platform views when available, and fallback to web search/web fetch/direct fetch when not. Then report back what connected.
---
/timeline [subject] — Build a Chronological Timeline
Search for dated references to the subject and construct a chronological timeline of events. Look for:
- Earliest online presence (account creation dates, first posts)
- Domain registration dates (via web search for WHOIS info)
- News mentions with dates
- Social media post timestamps
- Job changes (LinkedIn, press releases)
- Legal filings with dates
Present as a clean chronological list with sources cited.
Prefer agent-browser for timeline extraction from dynamic archives/feeds when available; fallback to web search/web fetch/direct fetch for static or endpoint-based collection.
---
/analyze-metadata
Prompt the user to paste EXIF data, email headers, HTTP headers, or document metadata. Then perform a forensic breakdown:
For EXIF data: Extract GPS coordinates, camera model, software used, timestamps, and modification history. Flag discrepancies (e.g., EXIF date doesn't match file name date).
For email headers: Trace the full routing path, identify originating IP, check SPF/DKIM/DMARC alignment, flag suspicious relays.
For HTTP headers: Identify server technology, CMS, CDN, security headers present/missing.
For document metadata: Author names, organization fields, creation/modification software, revision counts, embedded file paths.
---
/verif-photo — Visual Verification Workflow
Guide the user through a 5-step photo verification process. Claude cannot perform vision analysis through this skill, so the workflow is guided/assisted:
1. Provenance Check — Where was this image first published? Search for the image URL, filename, or associated caption across the web. 2. Shadow & Lighting Analysis — Ask the user to describe shadow directions and lengths. Cross-reference with expected sun position for the claimed location/time (search for sun angle calculators and historical weather). 3. Landmark & Signage Identification — Ask the user to describe any visible landmarks, street signs, license plates, store names. Search for these to geolocate. 4. Weather Corroboration — If a date/location is claimed, search for historical weather data. Does it match what's visible in the image? 5. Reverse Image Guidance — Direct the user to perform a reverse image search (Google Images, TinEye, Yandex Images) and report back what they find. Suggest cropping strategies for better results.
---
/sock-opsec — Operational Security Checklist
Provide a phase-appropriate OPSEC checklist for the current investigation. This helps researchers maintain anonymity. Topics covered:
- Browser isolation (separate browser profiles, VPN considerations)
- Account separation (don't use personal accounts for research)
- Search hygiene (clearing cookies, using incognito/private modes)
- Note-taking security (where to store investigation notes safely)
- Digital trail awareness (what traces does your research leave?)
- Platform-specific risks (some platforms notify users of profile views)
Tailor the checklist to what the user is currently investigating.
---
/entity [name_or_handle] — Add to Entity Map
Manually add an entity to the running knowledge graph. Also used to query what's known about a specific entity.
Usage:
/entity JohnDoe— View or add entity "JohnDoe"/entity example.com— View or add domain
Entity Types Tracked:
- Person
- Username/Handle
- Email Address
- Domain
- IP Address
- Organization
- Phone Number
- Location
- Asset
- Event
---
/report — Generate Intelligence Summary (INTSUM)
Compile all findings from the current conversation into a structured report. Read references/report-template.md for the exact format. The report should include:
- Executive Summary
- Subject Profile
- Key Findings (with confidence ratings)
- Entity Relationship Map (text-based)
- Timeline of Events
- Source List
- Gaps & Recommended Next Steps
- Analyst Notes & Caveats
Generate this as a downloadable markdown file.
---
/simple-report — Generate Plain-Language Summary
Create an easy-to-understand report at an 8th-grade reading level (ages 13-14). This report translates complex intelligence findings into plain English for non-technical audiences, clients, or stakeholders who need actionable insights without jargon.
When to use:
- Explaining findings to clients or management
- Sharing results with non-technical team members
- Creating public-facing summaries
- When the user asks for "simple" or "easy" explanations
Writing guidelines:
- Use short sentences (15-20 words max)
- Avoid technical jargon (translate terms like "reconnaissance" to "research")
- Use analogies and relatable comparisons
- Break complex ideas into bullet points
- Define necessary technical terms in plain English
- Use active voice
- Include "What This Means" and "What To Do" sections
Structure:
PLAIN-LANGUAGE SUMMARY
THE BOTTOM LINE (2-3 sentences max)
[Simple explanation of the most important finding]
WHAT WE FOUND
[Easy-to-understand breakdown of key discoveries]
WHAT THIS MEANS FOR YOU
[Why it matters in practical terms]
WHAT YOU SHOULD DO NEXT
[Clear, actionable recommendations]
SIMPLE EXPLANATIONS
[Definitions of any technical terms used]Generate this as a separate markdown file from the technical /report.
---
/full [target] — Comprehensive Investigation
Run a complete, automated investigation using ALL available tools in sequence. This command performs a thorough, multi-layered analysis of the target by executing the full investigation cycle automatically.
Execution sequence:
1. Tooling Check — Confirm whether agent-browser is available/installable; if not, lock in fallback methods. 2. Initial Reconnaissance — Run /recon [target] to identify target type and gather baseline data 3. Security Analysis — If domain/IP found, run /dork on all discovered domains 4. Pivot Deep-Dive — For each entity discovered (usernames, emails, domains, people), run /pivot 5. Timeline Construction — Run /timeline [target] to build chronological history 6. Entity Mapping — Compile complete entity relationship map 7. Dual Reporting — Generate both technical /report AND plain-language /simple-report
What it produces:
- Complete entity map with all discovered connections
- Security assessment (if domains involved)
- Chronological timeline
- Technical intelligence report (INTSUM)
- Plain-language summary report
- Recommended next steps prioritized by impact
When to use:
- Starting a new investigation and want everything at once
- Due diligence research
- Comprehensive background checks
- When you don't know what you don't know
Duration: This runs multiple searches sequentially. Expect 3-5 minutes for completion.
---
/track [entity] — Track Entity
Add an entity to the active tracking system. Tracked entities are monitored across the investigation and included in all reports and visualizations.
Usage:
/track John Doe
/track example.com
/track johndoe@email.comTracks:
- Entity metadata
- First/last seen timestamps
- Confidence history
- Source references
- Related connections
---
/link [entity_a] [entity_b] — Link Entities
Create a relationship between two tracked entities.
Usage:
/link "John Doe" "example.com" owns
/link johndoe johndoe123 aliasRelationship Types:
- owns (domain, email, asset)
- uses (username, platform)
- works_at (employment)
- associated_with (general association)
- family (family relationship)
- communicated_with (contact)
---
/entities — Show Entity Map
Display the complete entity relationship map with all tracked entities and their connections.
Output includes:
- Entity list with types
- Relationship graph
- Confidence levels
- Source summary
- Entity statistics
---
/confidence [entity] — Set Confidence Rating
Assign or view confidence rating for an entity.
Usage:
/confidence johndoe high
/confidence example.com mediumRatings:
- high (90-100%) — Authoritative source confirmed
- medium (60-89%) — Corroborated but not definitive
- low (30-59%) — Single source or circumstantial
- speculative (<30%) — Analytical inference
---
/visualize [type] — Generate Visualizations
Create visual representations of investigation data.
Types:
/visualize entities— Entity relationship diagram (Mermaid)/visualize timeline— Timeline chart/visualize attack— Attack path diagram (for security investigations)/visualize surface— Attack surface map
Output: Mermaid-compatible markdown that renders in most modern markdown viewers.
---
/risk-score [target] — Calculate Risk Score
Calculate a comprehensive risk score for a target based on discovered indicators.
Risk Factors:
- Digital exposure (public data availability)
- Security posture (for domains)
- Threat indicators
- Privacy gaps
- Behavioral patterns
Output:
- Numerical score (0-100)
- Risk level (Critical/High/Medium/Low)
- Contributing factors
- Mitigation recommendations
---
/wizard [type] — Investigation Wizard
Guided step-by-step investigation for specific target types.
Available Wizards:
/wizard person [name]— Person investigation/wizard domain [domain]— Domain reconnaissance/wizard email [email]— Email investigation/wizard quick [target]— Rapid investigation
Each wizard asks clarifying questions and guides through the complete process.
---
/qa-check — Quality Assurance Check
Run comprehensive quality analysis on the current investigation.
Checks:
- Source quality and diversity
- Verification levels
- Citation completeness
- Bias indicators
- Redundancy issues
Output: Quality score (0-100) with prioritized improvement recommendations.
---
/coverage — Investigation Coverage
Show investigation coverage matrix identifying what's been checked and what gaps remain.
Categories Analyzed:
- Identity
- Digital Presence
- Professional
- Financial
- Legal
- Technical
- Geographic
- Associates
- Historical
- Media
Output: Coverage percentage per category with gap recommendations.
---
/gaps — Identify Missing Areas
List specific investigation gaps prioritized by impact on conclusions.
Output:
- Critical gaps (could change findings)
- High-priority gaps (should be addressed)
- Medium gaps (improve confidence)
- Low gaps (nice to have)
---
/verify-sources — Verify Sources
Check if cited sources are still accessible and valid.
Checks:
- URL accessibility (200 OK)
- Content changes since citation
- Archive availability
- Broken link alternatives
---
Passive Mode (Always Active)
Whenever a name, email, domain, username, IP address, phone number, or organization is mentioned in conversation — even outside of a slash command — Claude should:
1. Recognize the entity type automatically 2. Suggest 2–3 specific next steps the user could take (e.g., "That email domain is a custom domain — might be worth running /dork on it" or "That username format is distinctive — want me to /pivot on it across platforms?") 3. Add it to the internal entity map being tracked for this conversation
This passive awareness is what makes the skill feel like working with an actual analyst rather than just a search tool.
---
Entity Mapping
Throughout the conversation, maintain a running knowledge graph of discovered entities. Track:
| Field | Description |
|---|---|
| Entity | The name, handle, domain, email, IP, etc. |
| Type | person, username, email, domain, IP, organization, phone |
| First seen | Where/when this entity first appeared in the investigation |
| Connections | Links to other entities (e.g., "username123 → john.doe@example.com") |
| Confidence | How confident are we in each connection? |
| Notes | Any analyst observations |
When the user asks for the entity map (or when generating a /report), present this as a readable table or text-based graph showing relationships.
---
Confidence Rating System
Every claim in every response should have an inline confidence marker:
- 🟢 HIGH — Verified from authoritative or primary source (official website, government database result, direct platform profile)
- 🟡 MEDIUM — Multiple corroborating sources or strong circumstantial evidence
- 🔴 LOW — Single source, inference, or unverified lead
- ⚪ SPECULATIVE — Analyst hypothesis based on pattern, not direct evidence. Always clearly label.
Never present speculation as fact. When making inferences, explicitly state: "This is an inference based on [X] and [Y], not a confirmed finding."
---
Professional Playbooks
Available specialized workflows for different professions:
Journalist Source Verification
playbooks/journalist-source-verification.md
- Source verification workflow
- Anonymous source handling
- Document authentication
- Fact-checking procedures
- Legal considerations
- Source protection measures
HR Background Check
playbooks/hr-background-check.md
- Employment verification
- Credential checking
- Social media screening
- Reference verification
- Compliance guidelines
- Decision framework
Cyber Threat Intelligence
playbooks/cyber-threat-intel.md
- Threat actor profiling
- IOC identification
- Attack pattern analysis
- Attribution methodology
- Intelligence reporting
- Sharing guidelines
Private Investigator
playbooks/private-investigator.md
- Subject locating
- Asset discovery
- Relationship mapping
- Surveillance preparation
- Legal boundaries
- Report requirements
---
Tool Integrations
Maltego Export
integrations/maltego-export.md
- GraphML export format
- Entity type mapping
- Relationship definitions
- Import instructions
Obsidian Setup
integrations/obsidian-setup.md
- Vault folder structure
- Note templates
- Link syntax conventions
- Graph view optimization
Notion Schema
integrations/notion-schema.md
- Database schemas
- Property definitions
- View configurations
- Automation suggestions
---
Search Strategy Guide
When performing any OSINT search, follow this hierarchy:
1. Choose collection method first — Prefer agent-browser when available/installable; fallback to web search/web fetch/direct fetch if unavailable or blocked. 2. Start specific, then broaden — Try exact-match queries first ("john.doe@example.com"), then loosen (john doe example.com) 3. Vary search engines — Different engines index different content. If Google doesn't find it, suggest Bing or DuckDuckGo formulations 4. Use temporal operators — Add date ranges to find historical or recent content 5. Check secondary sources — Cached pages, archive.org references, paste sites, code repositories 6. Cross-platform correlation — Same username on multiple platforms is a strong signal 7. Look for metadata — Domain registration info, document properties, image data
For each search, log:
- What was searched
- What was found (or not found — negative results are informative)
- What new pivots were identified
---
Reference Files
Read these files when performing specific investigation types:
references/recon-vectors.md— Detailed playbooks for each target type (domain, person, email, username, IP, organization). Read this before running/recon.references/report-template.md— The exact template for/reportoutput. Read this before generating a report.references/dork-library.md— Extended library of Google Dork patterns organized by category. Read this before running/dork.references/timeline-guide.md— Timeline construction methodology and formatting.references/metadata-forensics.md— Detailed metadata analysis procedures.references/opsec-handbook.md— Comprehensive operational security guidance.
---
QA & Quality Assurance
qa/coverage-analysis.md— Investigation coverage matrix and gap identificationqa/quality-metrics.md— Quality scoring methodology and assurance proceduresqa/testing-checklist.md— Comprehensive testing validation checklist
---
Important Reminders
- All information gathered must be publicly available. Do not attempt to access private accounts, bypass authentication, or access restricted data.
- Correlation is not causation. Two accounts with the same username might be different people. Always caveat.
- People have a right to privacy. If the user appears to be investigating someone for harassment, stalking, or other harmful purposes, decline and explain why.
- This is research, not surveillance. Frame all outputs as research findings, not targeting packages.
- Always cite sources. Every finding should trace back to a URL or search query.
- Prefer browser automation when possible. Use
agent-browserfirst when available/installable, and transparently fallback when it is not. - Negative results matter. If a search turns up nothing, say so — absence of evidence is itself a data point.
- Maintain quality standards. Run
/qa-checkbefore finalizing reports. - Document coverage gaps. Use
/coverageto ensure comprehensive investigation. - Verify before trusting. Use
/verify-sourcesto ensure cited sources remain valid.
---
Version Information
Current Version: 2.1 Release Date: 2026 Previous Version: 2.0
See CHANGELOG.md for version history and feature additions.
---
Support & Documentation
- Advanced User Guide:
advanced-user-guide.md— Power user features and automation - Troubleshooting:
troubleshooting.md— Common issues and solutions - Testing Checklist:
qa/testing-checklist.md— Validation procedures
For additional help, use /help [command] for command-specific documentation.
# Auto detect text files and perform LF normalization
* text=auto
.DS_Store
*.log
*.tmp
*.swp
*.swo
*~
.env
.env.local
secrets/
credentials/
output/
reports/*.pdf
reports/*.docx
.vscode/
.idea/
__pycache__/
*.pyc
node_modules/
dist/
build/
Contradiction Detection System
Overview
The Contradiction Detection System automatically identifies conflicting information within an investigation, flags inconsistencies, and provides rules for confidence adjustment based on resolution outcomes.
Types of Contradictions
1. Value Contradictions
When the same entity attribute has different values across sources.
Example:
Entity: John Doe
Source A: Date of Birth = 1980-01-15
Source B: Date of Birth = 1985-03-222. Temporal Contradictions
When timeline events cannot coexist chronologically.
Example:
Event A: Started job at Company X in 2018
Event B: Graduated university in 2019
Event C: Claims 5 years experience at Company X in 20203. Existence Contradictions
When one source claims something exists and another claims it doesn't.
Example:
Source A: Domain example.com registered to John Doe
Source B: John Doe claims no domain ownership4. Relationship Contradictions
When entity relationships conflict.
Example:
Relationship A: Person A EMPLOYS Person B
Relationship B: Person B owns Company that COMPETES_WITH Person A's company5. Location Contradictions
When an entity cannot be in multiple places simultaneously.
Example:
Post A: User posts from New York at 2024-01-15 14:00 UTC
Post B: User posts from Tokyo at 2024-01-15 14:30 UTCContradiction Detection Algorithm
function detectContradictions(investigation) {
const contradictions = [];
// Check each entity for internal contradictions
investigation.entities.forEach(entity => {
const entityContradictions = checkEntityContradictions(entity);
contradictions.push(...entityContradictions);
});
// Check for cross-entity contradictions
const crossContradictions = checkCrossEntityContradictions(investigation.entities);
contradictions.push(...crossContradictions);
// Check relationship contradictions
const relationshipContradictions = checkRelationshipContradictions(
investigation.entities,
investigation.relationships
);
contradictions.push(...relationshipContradictions);
// Check temporal contradictions
const temporalContradictions = checkTemporalContradictions(investigation);
contradictions.push(...temporalContradictions);
return contradictions.map(c => ({
...c,
id: generateUUID(),
detected_at: now(),
status: "OPEN"
}));
}
function checkEntityContradictions(entity) {
const contradictions = [];
const evidenceByAttribute = {};
// Group evidence by attribute
entity.evidence.forEach(ev => {
const attr = extractAttribute(ev.content);
if (attr) {
if (!evidenceByAttribute[attr.name]) {
evidenceByAttribute[attr.name] = [];
}
evidenceByAttribute[attr.name].push({
evidence: ev,
value: attr.value
});
}
});
// Check for conflicting values
Object.entries(evidenceByAttribute).forEach(([attrName, items]) => {
const uniqueValues = [...new Set(items.map(i => normalizeValue(i.value)))];
if (uniqueValues.length > 1) {
contradictions.push({
type: "VALUE_CONTRADICTION",
severity: calculateSeverity(items),
entity_id: entity.id,
entity_type: entity.type,
attribute: attrName,
conflicting_values: uniqueValues.map(v => ({
value: v,
sources: items
.filter(i => normalizeValue(i.value) === v)
.map(i => ({
evidence_id: i.evidence.id,
source: i.evidence.source,
reliability: i.evidence.source_reliability
}))
})),
recommendation: generateRecommendation(items)
});
}
});
return contradictions;
}
function checkCrossEntityContradictions(entities) {
const contradictions = [];
// Check for duplicate entities with conflicting info
for (let i = 0; i < entities.length; i++) {
for (let j = i + 1; j < entities.length; j++) {
const e1 = entities[i];
const e2 = entities[j];
// Check if entities might be the same
if (mightBeSameEntity(e1, e2)) {
const conflicts = findAttributeConflicts(e1, e2);
if (conflicts.length > 0) {
contradictions.push({
type: "ENTITY_IDENTITY_CONTRADICTION",
severity: "HIGH",
entity_ids: [e1.id, e2.id],
entities: [
{ id: e1.id, value: e1.value, type: e1.type },
{ id: e2.id, value: e2.value, type: e2.type }
],
conflicting_attributes: conflicts,
recommendation: "Investigate if these are the same entity or resolve attribute conflicts"
});
}
}
}
}
return contradictions;
}
function checkRelationshipContradictions(entities, relationships) {
const contradictions = [];
// Check for mutually exclusive relationships
const mutualExclusives = {
"PARENT_OF": ["CHILD_OF", "SIBLING_OF"],
"EMPLOYMENT": ["COMPETES_WITH"],
"OWNERSHIP": ["REGISTRATION"] // Person can't own and register as different entities
};
relationships.forEach((rel, idx) => {
const conflictingTypes = mutualExclusives[rel.type] || [];
const conflicts = relationships.filter((other, otherIdx) => {
if (idx === otherIdx) return false;
return conflictingTypes.includes(other.type) &&
((rel.source_id === other.source_id && rel.target_id === other.target_id) ||
(rel.source_id === other.target_id && rel.target_id === other.source_id));
});
if (conflicts.length > 0) {
contradictions.push({
type: "RELATIONSHIP_CONTRADICTION",
severity: "HIGH",
relationship_id: rel.id,
relationship_type: rel.type,
conflicting_relationships: conflicts.map(c => c.id),
entities_involved: [rel.source_id, rel.target_id],
recommendation: "Verify relationship accuracy - relationships are mutually exclusive"
});
}
});
return contradictions;
}
function checkTemporalContradictions(investigation) {
const contradictions = [];
const events = extractTemporalEvents(investigation);
// Sort events by claimed time
events.sort((a, b) => new Date(a.timestamp) - new Date(b.timestamp));
// Check for impossible sequences
for (let i = 0; i < events.length; i++) {
for (let j = i + 1; j < events.length; j++) {
const e1 = events[i];
const e2 = events[j];
// Check if events involve the same entity
if (e1.entity_id === e2.entity_id) {
const impossible = isImpossibleSequence(e1, e2);
if (impossible) {
contradictions.push({
type: "TEMPORAL_CONTRADICTION",
severity: "HIGH",
entity_id: e1.entity_id,
event_1: {
description: e1.description,
claimed_time: e1.timestamp,
evidence_id: e1.evidence_id
},
event_2: {
description: e2.description,
claimed_time: e2.timestamp,
evidence_id: e2.evidence_id
},
reason: impossible.reason,
recommendation: "Verify timestamps - at least one event is incorrectly dated"
});
}
}
}
}
return contradictions;
}
function mightBeSameEntity(e1, e2) {
// Check for matching aliases
if (e1.aliases.some(a => e2.aliases.includes(a) || e2.value === a)) {
return true;
}
// Check for similar values
const similarity = calculateSimilarity(e1.value, e2.value);
if (similarity > 0.8) return true;
// Check for shared attributes
const sharedAttrs = findSharedAttributes(e1, e2);
if (sharedAttrs.length > 0) return true;
return false;
}
function findAttributeConflicts(e1, e2) {
const conflicts = [];
// Compare all attributes
Object.entries(e1.attributes).forEach(([key, value1]) => {
const value2 = e2.attributes[key];
if (value2 && normalizeValue(value1) !== normalizeValue(value2)) {
conflicts.push({
attribute: key,
entity_1_value: value1,
entity_2_value: value2
});
}
});
return conflicts;
}
function calculateSeverity(conflictingItems) {
// Severity based on source reliability differences
const reliabilities = conflictingItems.map(i => i.evidence.source_reliability);
const hasReliableSource = reliabilities.some(r => ['A', 'B'].includes(r));
const hasUnreliableSource = reliabilities.some(r => ['D', 'E'].includes(r));
if (hasReliableSource && hasUnreliableSource) {
return "LOW"; // Easy to resolve - trust reliable source
} else if (reliabilities.every(r => r === reliabilities[0])) {
return "HIGH"; // Same reliability sources disagree
}
return "MEDIUM";
}
function generateRecommendation(items) {
const reliabilities = items.map(i => i.evidence.source_reliability);
const bestReliability = Math.min(...reliabilities.map(r =>
({A:1, B:2, C:3, D:4, E:5, F:3})[r]
));
const bestSource = items.find(i =>
({A:1, B:2, C:3, D:4, E:5, F:3})[i.evidence.source_reliability] === bestReliability
);
return `Prefer value from ${bestSource.evidence.source} (reliability: ${bestSource.evidence.source_reliability})`;
}Contradiction Resolution Framework
const resolutionStrategies = {
// Strategy 1: Prefer higher reliability source
preferReliability: (contradiction) => {
const { conflicting_values } = contradiction;
// Score each value's sources
const scored = conflicting_values.map(cv => ({
value: cv.value,
score: cv.sources.reduce((sum, s) => {
const weights = { A: 1.0, B: 0.8, C: 0.6, D: 0.4, E: 0.2, F: 0.5 };
return sum + (weights[s.reliability] || 0.5);
}, 0)
}));
const best = scored.sort((a, b) => b.score - a.score)[0];
return {
resolution: "PREFER_RELIABILITY",
selected_value: best.value,
confidence_adjustment: 0.8,
reasoning: `Selected based on higher cumulative source reliability (score: ${best.score.toFixed(2)})`
};
},
// Strategy 2: Prefer more recent evidence
preferRecency: (contradiction, investigation) => {
const { conflicting_values } = contradiction;
const withDates = conflicting_values.map(cv => ({
value: cv.value,
latest: Math.max(...cv.sources.map(s =>
new Date(getEvidenceTimestamp(s.evidence_id, investigation)).getTime()
))
}));
const mostRecent = withDates.sort((a, b) => b.latest - a.latest)[0];
return {
resolution: "PREFER_RECENCY",
selected_value: mostRecent.value,
confidence_adjustment: 0.7,
reasoning: "Selected most recent evidence as more likely to be current"
};
},
// Strategy 3: Require additional verification
requireVerification: (contradiction) => {
return {
resolution: "REQUIRES_VERIFICATION",
selected_value: null,
confidence_adjustment: 0.0,
reasoning: "Conflicting sources have equal reliability - requires third-party verification",
action_required: "Find additional corroborating source"
};
},
// Strategy 4: Both values valid (different contexts)
acceptBoth: (contradiction, context) => {
return {
resolution: "ACCEPT_BOTH",
selected_value: null,
confidence_adjustment: 1.0,
reasoning: "Values may be valid in different contexts (e.g., maiden name vs married name)",
note: "Store both values with context annotations"
};
},
// Strategy 5: Flag as uncertain
flagUncertain: (contradiction) => {
return {
resolution: "FLAG_UNCERTAIN",
selected_value: null,
confidence_adjustment: 0.0,
reasoning: "Unable to resolve - mark entity confidence as DISPUTED",
action_required: "Manual review required"
};
}
};
function resolveContradiction(contradiction, strategy = "auto") {
let resolution;
if (strategy === "auto") {
// Auto-select strategy based on contradiction type
switch (contradiction.type) {
case "VALUE_CONTRADICTION":
resolution = resolutionStrategies.preferReliability(contradiction);
break;
case "TEMPORAL_CONTRADICTION":
resolution = resolutionStrategies.flagUncertain(contradiction);
break;
case "ENTITY_IDENTITY_CONTRADICTION":
resolution = resolutionStrategies.requireVerification(contradiction);
break;
default:
resolution = resolutionStrategies.flagUncertain(contradiction);
}
} else {
resolution = resolutionStrategies[strategy](contradiction);
}
return {
...contradiction,
resolution: resolution,
resolved_at: now(),
status: resolution.resolution === "REQUIRES_VERIFICATION" ? "PENDING" : "RESOLVED"
};
}Confidence Adjustment Rules
const confidenceAdjustmentRules = {
// When contradiction is resolved in favor of an entity
resolvedFavorably: (entity, resolution) => {
const adjustments = {
PREFER_RELIABILITY: 0.0, // No penalty - used reliable source
PREFER_RECENCY: -0.1, // Small penalty for recency-based selection
ACCEPT_BOTH: 0.0, // No penalty - both may be valid
REQUIRES_VERIFICATION: -0.3, // Significant penalty - unverified
FLAG_UNCERTAIN: -0.5 // Major penalty - unresolved
};
return adjustments[resolution.resolution] || -0.2;
},
// When entity has unresolved contradictions
unresolvedContradiction: (entity, contradictions) => {
const severityWeights = { HIGH: 0.4, MEDIUM: 0.2, LOW: 0.1 };
const totalPenalty = contradictions.reduce((sum, c) => {
return sum + (severityWeights[c.severity] || 0.2);
}, 0);
return -Math.min(totalPenalty, 0.8); // Cap at 0.8 reduction
},
// When contradiction is found and resolved during evidence addition
evidenceContradictionResolved: (originalConfidence, resolution) => {
switch (resolution.resolution) {
case "PREFER_RELIABILITY":
return originalConfidence; // Unchanged
case "PREFER_RECENCY":
return originalConfidence * 0.9;
case "REQUIRES_VERIFICATION":
return originalConfidence * 0.5;
case "FLAG_UNCERTAIN":
return "DISPUTED";
default:
return originalConfidence * 0.8;
}
}
};
function adjustConfidence(entity, contradictions) {
let currentConfidence = entity.confidence;
let confidenceScore = confidenceLevelToScore(currentConfidence);
// Separate resolved and unresolved
const resolved = contradictions.filter(c => c.status === "RESOLVED");
const unresolved = contradictions.filter(c => c.status === "OPEN" || c.status === "PENDING");
// Apply resolved adjustments
resolved.forEach(c => {
const adjustment = confidenceAdjustmentRules.resolvedFavorably(entity, c.resolution);
confidenceScore += adjustment;
});
// Apply unresolved penalties
if (unresolved.length > 0) {
const penalty = confidenceAdjustmentRules.unresolvedContradiction(entity, unresolved);
confidenceScore += penalty;
}
// Ensure within bounds
confidenceScore = Math.max(0, Math.min(1.5, confidenceScore));
// Convert back to level
return scoreToConfidenceLevel(confidenceScore);
}
function confidenceLevelToScore(level) {
const scores = {
"CERTAIN": 1.2,
"HIGH": 0.9,
"MEDIUM": 0.6,
"LOW": 0.3,
"UNCERTAIN": 0.1,
"DISPUTED": 0.0
};
return scores[level] || 0.5;
}
function scoreToConfidenceLevel(score) {
if (score >= 1.0) return "CERTAIN";
if (score >= 0.7) return "HIGH";
if (score >= 0.4) return "MEDIUM";
if (score >= 0.2) return "LOW";
if (score > 0) return "UNCERTAIN";
return "DISPUTED";
}Example Contradictions and Resolutions
Example 1: Simple Value Contradiction
// Contradiction Detected:
{
type: "VALUE_CONTRADICTION",
entity_id: "person-001",
attribute: "date_of_birth",
conflicting_values: [
{
value: "1980-01-15",
sources: [
{ source: "Public records", reliability: "A" }
]
},
{
value: "1985-03-22",
sources: [
{ source: "Social media profile", reliability: "C" }
]
}
],
severity: "LOW"
}
// Resolution:
{
resolution: "PREFER_RELIABILITY",
selected_value: "1980-01-15",
confidence_adjustment: 0.8,
reasoning: "Public records (reliability A) preferred over social media (reliability C)"
}
// Result: Entity confidence remains HIGHExample 2: Unresolvable Contradiction
// Contradiction Detected:
{
type: "VALUE_CONTRADICTION",
entity_id: "company-001",
attribute: "registration_number",
conflicting_values: [
{
value: "12345678",
sources: [
{ source: "Registry A", reliability: "A" }
]
},
{
value: "87654321",
sources: [
{ source: "Registry B", reliability: "A" }
]
}
],
severity: "HIGH"
}
// Resolution:
{
resolution: "REQUIRES_VERIFICATION",
selected_value: null,
confidence_adjustment: 0.0,
reasoning: "Both sources have equal reliability (A) - cannot auto-resolve",
action_required: "Contact company directly or verify against original incorporation documents"
}
// Result: Entity confidence reduced to DISPUTED until resolvedExample 3: Temporal Contradiction
// Contradiction Detected:
{
type: "TEMPORAL_CONTRADICTION",
entity_id: "person-002",
event_1: {
description: "Started at Company X",
claimed_time: "2018-06-01",
evidence_id: "ev-010"
},
event_2: {
description: "Graduated from University",
claimed_time: "2019-05-15",
evidence_id: "ev-011"
},
reason: "Claims 5 years experience at Company X in 2020 post, but graduated in 2019"
}
// Resolution Options:
// Option A: Start date is incorrect (maybe internship)
// Option B: Graduation date is incorrect (maybe earlier graduation)
// Option C: Experience claim is exaggerated
// Selected Resolution:
{
resolution: "FLAG_UNCERTAIN",
selected_value: null,
confidence_adjustment: -0.5,
reasoning: "Timeline inconsistency requires manual investigation",
notes: "Possible explanations: (1) Started as intern before graduation, (2) Previous graduation not recorded, (3) Inflated experience claim"
}
// Result: Entity marked with temporal flag, confidence reduced to MEDIUMExample 4: Entity Identity Contradiction
// Contradiction Detected:
{
type: "ENTITY_IDENTITY_CONTRADICTION",
entities: [
{ id: "email-001", value: "john.doe@example.com", type: "EMAIL" },
{ id: "email-002", value: "john.doe@company.com", type: "EMAIL" }
],
conflicting_attributes: [
{
attribute: "associated_person",
entity_1_value: "John Doe (Developer)",
entity_2_value: "John Doe (Marketing)"
}
]
}
// Resolution:
{
resolution: "ACCEPT_BOTH",
selected_value: null,
confidence_adjustment: 0.0,
reasoning: "Same person with multiple emails for different roles/employers",
note: "Create relationship between emails: ALIAS_OF or ASSOCIATED_WITH"
}
// Result: Both entities kept, relationship added, confidence unchangedExample 5: Location Contradiction
// Contradiction Detected:
{
type: "LOCATION_CONTRADICTION",
entity_id: "person-003",
location_1: {
location: "New York, NY",
timestamp: "2024-01-15T14:00:00Z",
source: "Twitter post with geotag"
},
location_2: {
location: "Tokyo, Japan",
timestamp: "2024-01-15T14:30:00Z",
source: "Instagram post with geotag"
},
impossibility: "Cannot travel NYC to Tokyo in 30 minutes"
}
// Resolution:
{
resolution: "PREFER_RELIABILITY",
selected_value: "New York, NY",
confidence_adjustment: 0.0,
reasoning: "Twitter geotag (device GPS) considered more reliable than Instagram location tag (user-selected)",
note: "Tokyo post may be: (1) Scheduled post, (2) VPN/proxy location, (3) Previously taken photo"
}
// Result: NYC location accepted, Tokyo flagged as potentially inaccurateCommand Interface
// Detect contradictions in current investigation
/detect-contradictions [--entity <id>] [--severity <level>]
// Show all detected contradictions
/list-contradictions [--status <open|resolved|pending>]
// Show details of specific contradiction
/show-contradiction <id>
// Resolve a contradiction
/resolve-contradiction <id> --strategy <strategy> [--value <selected>]
// Available strategies:
// - prefer-reliability: Choose value from most reliable source(s)
// - prefer-recency: Choose most recent evidence
// - accept-both: Keep both values (valid in different contexts)
// - flag-uncertain: Mark as disputed pending verification
// - custom: Manual resolution with explanation
// Examples:
/resolve-contradiction cnt-001 --strategy prefer-reliability
/resolve-contradiction cnt-002 --strategy custom --value "1980-01-15" --reason "Birth certificate found"
// Ignore a contradiction (mark as false positive)
/ignore-contradiction <id> --reason <explanation>
// Recheck after new evidence
/recheck-contradictions
// Export contradictions report
/export-contradictions [filename]Contradiction Report Format
function generateContradictionReport(investigation) {
const contradictions = detectContradictions(investigation);
return {
generated_at: now(),
investigation_id: investigation.id,
summary: {
total_contradictions: contradictions.length,
by_type: countBy(contradictions, 'type'),
by_severity: countBy(contradictions, 'severity'),
by_status: countBy(contradictions, 'status'),
open_count: contradictions.filter(c => c.status === "OPEN").length
},
open_contradictions: contradictions.filter(c => c.status === "OPEN"),
recently_resolved: contradictions.filter(c =>
c.status === "RESOLVED" &&
hoursSince(c.resolved_at) < 24
),
recommendations: generateContradictionRecommendations(contradictions)
};
}
function generateContradictionRecommendations(contradictions) {
const open = contradictions.filter(c => c.status === "OPEN");
const recs = [];
if (open.length === 0) {
recs.push("No open contradictions - investigation consistency is good");
} else {
const highSeverity = open.filter(c => c.severity === "HIGH");
if (highSeverity.length > 0) {
recs.push(`Address ${highSeverity.length} high-severity contradictions before finalizing report`);
}
const unresolvable = open.filter(c => c.severity === "HIGH" &&
c.conflicting_values?.every(v => v.sources.every(s => s.reliability === 'A'))
);
if (unresolvable.length > 0) {
recs.push(`${unresolvable.length} contradictions have conflicting A-grade sources - requires primary source verification`);
}
}
return recs;
}Visual Output Example
Contradiction Report for: Phishing Domain Investigation
═══════════════════════════════════════════════════════════════
Summary:
Total Contradictions: 5
By Severity: HIGH: 1 | MEDIUM: 2 | LOW: 2
By Status: OPEN: 3 | RESOLVED: 2 | PENDING: 0
OPEN CONTRADICTIONS:
─────────────────────────────────────────────────────────────────
[CRITICAL] #cnt-001 - Value Contradiction (HIGH)
Entity: person-001 (John Doe)
Attribute: Date of Birth
Conflict:
Value A: 1980-01-15 (Source: Public Records - Reliability: A)
Value B: 1985-03-22 (Source: Social Media - Reliability: C)
Recommendation: Prefer 1980-01-15 from Public Records
Action: /resolve-contradiction cnt-001 --strategy prefer-reliability
[WARNING] #cnt-003 - Temporal Contradiction (HIGH)
Entity: person-002
Issue: Timeline impossibility detected
Event: Started at Company X (2018-06-01)
Event: Graduated University (2019-05-15)
Claim: 5 years experience in 2020
Recommendation: Manual review required - possible explanations:
- Started as intern before graduation
- Earlier graduation not recorded
- Inflated experience claim
Action: /show-contradiction cnt-003 for details
─────────────────────────────────────────────────────────────────
RECOMMENDATIONS:
• Address 1 high-severity contradiction before finalizing report
• 1 contradiction has conflicting A-grade sources - requires verification
• Consider reviewing temporal consistency of all person entities
Run /detect-contradictions --entity person-001 to recheck after resolutionEntity Manager Specification
Overview
The Entity Manager provides comprehensive CRUD operations for OSINT investigation entities, with deduplication, cross-referencing, and relationship mapping capabilities.
Core Operations
1. Entity CRUD
Create Entity
// Command Interface
/add-entity <type> <value> [options]
// Options:
--confidence <level> # CERTAIN, HIGH, MEDIUM, LOW, UNCERTAIN
--source <description> # Evidence source
--alias <name> # Add alias
--attribute <k>=<v> # Type-specific attribute
--note <text> # Entity notes
// Examples:
/add-entity EMAIL john@example.com --confidence HIGH --source "WHOIS lookup"
/add-entity PERSON "John Doe" --alias "JD" --attribute dob=1980-01-01Internal Function:
function createEntity(type, value, options) {
// 1. Normalize value based on type
const normalizedValue = normalizeEntityValue(type, value);
// 2. Check for duplicates
const duplicate = findDuplicate(type, normalizedValue, options.aliases);
if (duplicate) {
return mergeOrCreateNew(duplicate, normalizedValue, options);
}
// 3. Create entity with UUID
const entity = {
id: generateUUID(),
type: type,
value: normalizedValue,
display_name: options.display_name || normalizedValue,
aliases: options.aliases || [],
attributes: options.attributes || {},
confidence: options.confidence || "MEDIUM",
evidence: [createEvidence(options)],
status: "ACTIVE",
first_seen: now(),
last_seen: now(),
created_at: now(),
modified_at: now(),
version: 1,
source_entities: options.source_entities || [],
tags: options.tags || []
};
// 4. Add to investigation
investigation.entities.push(entity);
// 5. Auto-create relationships if source_entities provided
if (options.source_entities) {
options.source_entities.forEach(sourceId => {
createRelationship(sourceId, entity.id, "REFERENCES", {
evidence: entity.evidence
});
});
}
// 6. Log audit entry
auditLog("ENTITY_CREATE", { entity_id: entity.id });
return entity;
}Read Entity
// Command Interface
/show-entity <id_or_value>
/find-entities [filters]
// Filters:
--type <type> # Filter by entity type
--confidence <level> # Filter by confidence
--tag <tag> # Filter by tag
--status <status> # Filter by status
--connected-to <id> # Find entities connected to specific entity
--has-evidence # Only entities with evidence
--no-evidence # Only entities without evidence
// Examples:
/show-entity john@example.com
/find-entities --type DOMAIN --confidence HIGH
/find-entities --connected-to <uuid> --type EMAILInternal Function:
function getEntity(idOrValue) {
// Try UUID first
let entity = investigation.entities.find(e => e.id === idOrValue);
// Try value match
if (!entity) {
entity = investigation.entities.find(e =>
e.value === idOrValue ||
e.aliases.includes(idOrValue)
);
}
// Fuzzy match if still not found
if (!entity) {
entity = fuzzyFindEntity(idOrValue);
}
return entity;
}
function findEntities(filters) {
return investigation.entities.filter(entity => {
if (filters.type && entity.type !== filters.type) return false;
if (filters.confidence && entity.confidence !== filters.confidence) return false;
if (filters.tag && !entity.tags.includes(filters.tag)) return false;
if (filters.status && entity.status !== filters.status) return false;
if (filters.connected_to && !isConnected(entity.id, filters.connected_to)) return false;
if (filters.has_evidence && entity.evidence.length === 0) return false;
if (filters.no_evidence && entity.evidence.length > 0) return false;
return true;
});
}Update Entity
// Command Interface
/update-entity <id_or_value> [changes]
// Change Options:
--value <new_value> # Update primary value
--add-alias <alias> # Add alias
--remove-alias <alias> # Remove alias
--attribute <k>=<v> # Set attribute
--remove-attribute <k> # Remove attribute
--confidence <level> # Update confidence
--status <status> # Update status
--note <text> # Update notes
--add-tag <tag> # Add tag
--remove-tag <tag> # Remove tag
// Examples:
/update-entity john@example.com --confidence CERTAIN --add-alias "john.doe"
/update-entity <uuid> --status VERIFIEDInternal Function:
function updateEntity(idOrValue, changes) {
const entity = getEntity(idOrValue);
if (!entity) throw new Error(`Entity not found: ${idOrValue}`);
// Store previous version
const previousVersion = {
version: entity.version,
modified_at: entity.modified_at,
snapshot: deepClone(entity)
};
entity.previous_versions = entity.previous_versions || [];
entity.previous_versions.push(previousVersion);
// Track changes
const changeLog = [];
// Apply changes
if (changes.value) {
changeLog.push({ field: "value", old: entity.value, new: changes.value });
entity.value = normalizeEntityValue(entity.type, changes.value);
}
if (changes.add_alias) {
if (!entity.aliases.includes(changes.add_alias)) {
entity.aliases.push(changes.add_alias);
changeLog.push({ field: "aliases", action: "add", value: changes.add_alias });
}
}
if (changes.remove_alias) {
entity.aliases = entity.aliases.filter(a => a !== changes.remove_alias);
changeLog.push({ field: "aliases", action: "remove", value: changes.remove_alias });
}
if (changes.attribute) {
const [key, value] = changes.attribute.split('=');
const oldValue = entity.attributes[key];
entity.attributes[key] = value;
changeLog.push({ field: `attributes.${key}`, old: oldValue, new: value });
}
if (changes.confidence) {
changeLog.push({ field: "confidence", old: entity.confidence, new: changes.confidence });
entity.confidence = changes.confidence;
}
if (changes.status) {
changeLog.push({ field: "status", old: entity.status, new: changes.status });
entity.status = changes.status;
}
// Update metadata
entity.modified_at = now();
entity.version++;
// Add change entry to previous version
previousVersion.changes = changeLog;
auditLog("ENTITY_UPDATE", { entity_id: entity.id, changes: changeLog });
return entity;
}Delete Entity
// Command Interface
/remove-entity <id_or_value> [--cascade]
// Examples:
/remove-entity john@example.com
/remove-entity <uuid> --cascade # Also remove related relationshipsInternal Function:
function deleteEntity(idOrValue, cascade = false) {
const entity = getEntity(idOrValue);
if (!entity) throw new Error(`Entity not found: ${idOrValue}`);
// Soft delete - mark as REMOVED
entity.status = "REMOVED";
entity.modified_at = now();
// Handle relationships
if (cascade) {
// Remove all relationships involving this entity
investigation.relationships = investigation.relationships.filter(r => {
if (r.source_id === entity.id || r.target_id === entity.id) {
auditLog("RELATIONSHIP_DELETE", { relationship_id: r.id, reason: "entity_cascade" });
return false;
}
return true;
});
} else {
// Mark relationships as involving removed entity
investigation.relationships
.filter(r => r.source_id === entity.id || r.target_id === entity.id)
.forEach(r => r.verified = false);
}
auditLog("ENTITY_DELETE", { entity_id: entity.id, cascade });
return { success: true, entity_id: entity.id };
}2. Deduplication Logic
function findDuplicate(type, value, aliases = []) {
const candidates = investigation.entities.filter(e =>
e.type === type && e.status !== "REMOVED"
);
for (const candidate of candidates) {
const matchScore = calculateMatchScore(candidate, type, value, aliases);
if (matchScore >= 0.95) {
return { entity: candidate, score: matchScore, action: "MERGE" };
} else if (matchScore >= 0.80) {
return { entity: candidate, score: matchScore, action: "REVIEW" };
}
}
return null;
}
function calculateMatchScore(candidate, type, value, aliases) {
let score = 0;
// Exact value match
if (normalizeValue(candidate.value) === normalizeValue(value)) {
score += 1.0;
}
// Alias match
const allAliases = [...candidate.aliases, ...aliases];
const normalizedAliases = allAliases.map(normalizeValue);
const normalizedValue = normalizeValue(value);
if (normalizedAliases.includes(normalizedValue)) {
score += 0.9;
}
// Type-specific matching
switch (type) {
case "EMAIL":
// Email local part match
const [local1, domain1] = candidate.value.split('@');
const [local2, domain2] = value.split('@');
if (local1 === local2 && domain1 !== domain2) {
score += 0.5; // Same username, different domain
}
break;
case "DOMAIN":
// Subdomain match
if (value.endsWith(candidate.value) || candidate.value.endsWith(value)) {
score += 0.7;
}
break;
case "PERSON":
// Name similarity
const nameScore = calculateNameSimilarity(candidate.value, value);
score += nameScore * 0.8;
break;
case "USERNAME":
// Case-insensitive match
if (candidate.value.toLowerCase() === value.toLowerCase()) {
score += 1.0;
}
break;
}
return Math.min(score, 1.0);
}
function mergeOrCreateNew(duplicateCheck, value, options) {
const { entity, score, action } = duplicateCheck;
if (action === "MERGE") {
// Add as alias if different
if (entity.value !== value && !entity.aliases.includes(value)) {
entity.aliases.push(value);
entity.modified_at = now();
auditLog("ENTITY_MERGE_ALIAS", { entity_id: entity.id, alias: value });
}
return entity;
}
if (action === "REVIEW") {
// Prompt user or flag for review
return {
requires_review: true,
existing: entity,
proposed: { value, options },
match_score: score
};
}
return null;
}3. Cross-Reference Engine
function crossReferenceEntity(entityId) {
const entity = getEntity(entityId);
const findings = {
direct_matches: [],
potential_matches: [],
related_entities: [],
inconsistencies: []
};
// Check all entities for relationships
investigation.entities.forEach(other => {
if (other.id === entity.id) return;
if (other.status === "REMOVED") return;
// Check for shared attributes
const sharedAttrs = findSharedAttributes(entity, other);
if (sharedAttrs.length > 0) {
findings.related_entities.push({
entity: other,
connection_type: "SHARED_ATTRIBUTE",
details: sharedAttrs
});
}
// Check for value similarities
const similarity = calculateSimilarity(entity.value, other.value);
if (similarity > 0.8) {
findings.potential_matches.push({
entity: other,
similarity
});
}
// Check for contradictions
const contradictions = findContradictions(entity, other);
if (contradictions.length > 0) {
findings.inconsistencies.push(...contradictions);
}
});
return findings;
}
function findSharedAttributes(entity1, entity2) {
const shared = [];
for (const [key, value1] of Object.entries(entity1.attributes)) {
if (entity2.attributes[key] === value1) {
shared.push({ attribute: key, value: value1 });
}
}
// Check for shared aliases
const sharedAliases = entity1.aliases.filter(a =>
entity2.aliases.includes(a) || entity2.value === a
);
if (sharedAliases.length > 0) {
shared.push({ attribute: "alias", values: sharedAliases });
}
return shared;
}
// Command Interface
/xref <id_or_value>
/xref john@example.com --depth 24. Relationship Mapping
function createRelationship(sourceId, targetId, type, options = {}) {
const source = getEntity(sourceId);
const target = getEntity(targetId);
if (!source || !target) {
throw new Error("Source or target entity not found");
}
// Check for existing relationship
const existing = investigation.relationships.find(r =>
r.source_id === source.id &&
r.target_id === target.id &&
r.type === type
);
if (existing) {
// Add evidence to existing relationship
if (options.evidence) {
existing.evidence.push(...options.evidence);
existing.modified_at = now();
}
return existing;
}
const relationship = {
id: generateUUID(),
source_id: source.id,
target_id: target.id,
type: type,
direction: options.direction || "DIRECTED",
confidence: options.confidence || "MEDIUM",
evidence: options.evidence || [],
attributes: options.attributes || {},
created_at: now(),
modified_at: now(),
verified: options.verified || false
};
investigation.relationships.push(relationship);
auditLog("RELATIONSHIP_CREATE", { relationship_id: relationship.id });
return relationship;
}
function getRelatedEntities(entityId, options = {}) {
const relationships = investigation.relationships.filter(r =>
r.source_id === entityId || r.target_id === entityId
);
return relationships.map(r => {
const isSource = r.source_id === entityId;
const otherId = isSource ? r.target_id : r.source_id;
const other = getEntity(otherId);
return {
entity: other,
relationship: r,
direction: isSource ? "OUTGOING" : "INCOMING",
connection_type: r.type
};
}).filter(r => options.type ? r.relationship.type === options.type : true);
}
// Command Interface
/add-relationship <source> <target> <type> [options]
/show-relationships <id_or_value>
/find-path <from> <to> [--max-hops <n>]
// Examples:
/add-relationship john@example.com example.com REGISTRATION --confidence HIGH
/show-relationships example.com --type HOSTING
/find-path john@example.com 192.168.1.1 --max-hops 35. Entity Queries
// Graph traversal query
function findPath(fromId, toId, maxHops = 5) {
const start = getEntity(fromId);
const end = getEntity(toId);
if (!start || !end) return null;
// BFS for shortest path
const queue = [{ entity: start, path: [], visited: new Set() }];
while (queue.length > 0) {
const { entity, path, visited } = queue.shift();
if (entity.id === end.id) {
return path;
}
if (path.length >= maxHops) continue;
if (visited.has(entity.id)) continue;
visited.add(entity.id);
const related = getRelatedEntities(entity.id);
for (const rel of related) {
if (!visited.has(rel.entity.id)) {
queue.push({
entity: rel.entity,
path: [...path, { from: entity, relationship: rel.relationship, to: rel.entity }],
visited: new Set(visited)
});
}
}
}
return null; // No path found
}
// Complex query builder
function queryEntities(querySpec) {
let results = [...investigation.entities];
// Apply filters
if (querySpec.types) {
results = results.filter(e => querySpec.types.includes(e.type));
}
if (querySpec.confidence_min) {
const levels = ["UNCERTAIN", "LOW", "MEDIUM", "HIGH", "CERTAIN"];
const minIndex = levels.indexOf(querySpec.confidence_min);
results = results.filter(e => levels.indexOf(e.confidence) >= minIndex);
}
if (querySpec.tags) {
results = results.filter(e =>
querySpec.tags.every(tag => e.tags.includes(tag))
);
}
if (querySpec.attributes) {
results = results.filter(e => {
return Object.entries(querySpec.attributes).every(([key, value]) => {
return e.attributes[key] === value;
});
});
}
if (querySpec.connected_to) {
results = results.filter(e => isConnected(e.id, querySpec.connected_to));
}
if (querySpec.has_evidence_type) {
results = results.filter(e =>
e.evidence.some(ev => querySpec.has_evidence_type.includes(ev.type))
);
}
// Sort
if (querySpec.sort_by) {
results.sort((a, b) => {
const aVal = getNestedValue(a, querySpec.sort_by);
const bVal = getNestedValue(b, querySpec.sort_by);
return querySpec.sort_order === "desc" ? bVal - aVal : aVal - bVal;
});
}
// Pagination
if (querySpec.limit) {
const offset = querySpec.offset || 0;
results = results.slice(offset, offset + querySpec.limit);
}
return results;
}
// Command Interface
/query-entities [query_spec]
// Examples:
/query-entities --type DOMAIN --confidence-min HIGH
/query-entities --tag suspicious --has-evidence-type DIRECT
/query-entities --connected-to <uuid> --sort-by modified_at --sort-order descNormalization Functions
const normalizationRules = {
EMAIL: (value) => value.toLowerCase().trim(),
DOMAIN: (value) => value.toLowerCase().trim().replace(/^www\./, ''),
USERNAME: (value) => value.toLowerCase().trim(),
IP_ADDRESS: (value) => {
// Normalize IPv6, validate IPv4
return value.trim();
},
PHONE: (value) => {
// Remove non-numeric, standardize format
return value.replace(/\D/g, '');
},
PERSON: (value) => {
// Title case, standardize spacing
return value.trim().replace(/\s+/g, ' ')
.split(' ')
.map(w => w.charAt(0).toUpperCase() + w.slice(1).toLowerCase())
.join(' ');
},
URL: (value) => {
try {
const url = new URL(value);
return url.toString().toLowerCase();
} catch {
return value.toLowerCase().trim();
}
}
};
function normalizeEntityValue(type, value) {
const rule = normalizationRules[type];
return rule ? rule(value) : value.trim();
}Type-Specific Attributes
PERSON Attributes
full_name: Complete namedob: Date of birth (YYYY-MM-DD)nationality: Country codeoccupation: Job title/rolelocation: Current locationgender: M/F/NBpob: Place of birth
DOMAIN Attributes
registrar: Domain registrarcreation_date: Registration dateexpiration_date: Expiry datenameservers: Array of NS recordsstatus: Array of domain status codesdns_records: Object with record types
EMAIL Attributes
domain: Extracted domainlocal_part: Username portionprovider: Email service providerdisposable: Boolean - is disposable emailbreached: Boolean - found in breaches
IP_ADDRESS Attributes
version: 4 or 6asn: Autonomous System Numberisp: Internet Service Providerlocation: Geo-location objectreverse_dns: PTR recordports: Array of open ports
USERNAME Attributes
platforms: Array of platforms where foundvariations: Array of similar usernamesavatar_url: Profile image URLbio: Profile descriptioncreated_at: Account creation date
ORGANIZATION Attributes
legal_name: Registered business nameregistration_number: Business IDincorporation_date: Date foundedjurisdiction: Registration country/stateindustry: Business sectoremployees: Number of employeesrevenue: Annual revenue
PHONE Attributes
country_code: Country calling codenational_format: Formatted numbercarrier: Mobile carriertype: MOBILE/LANDLINE/VOIPvalid: Boolean - validation statuswhatsapp: Boolean - WhatsApp availability
LOCATION Attributes
address: Street addresscity: City nameregion: State/Provincecountry: Country codepostal_code: ZIP/Postal codecoordinates: Lat/lng objecttimezone: Time zone
Output Formats
function formatEntityOutput(entity, format = "summary") {
switch (format) {
case "summary":
return `[${entity.type}] ${entity.display_name} (${entity.confidence})`;
case "detailed":
return `
Entity: ${entity.display_name}
Type: ${entity.type}
ID: ${entity.id}
Value: ${entity.value}
Confidence: ${entity.confidence}
Status: ${entity.status}
Aliases: ${entity.aliases.join(', ') || 'None'}
Attributes:
${formatAttributes(entity.attributes)}
Evidence Count: ${entity.evidence.length}
Created: ${entity.created_at}
Modified: ${entity.modified_at}
Version: ${entity.version}
Notes: ${entity.notes || 'None'}
`.trim();
case "json":
return JSON.stringify(entity, null, 2);
case "table":
// Return table row format
return {
Type: entity.type,
Value: entity.value.substring(0, 50),
Confidence: entity.confidence,
Evidence: entity.evidence.length,
Modified: entity.modified_at
};
}
}
function formatRelationshipOutput(rel, includeEntities = false) {
const source = getEntity(rel.source_id);
const target = getEntity(rel.target_id);
let output = `${source.display_name} --[${rel.type}]--> ${target.display_name} (${rel.confidence})`;
if (includeEntities) {
output += `\n Source: ${formatEntityOutput(source, "summary")}`;
output += `\n Target: ${formatEntityOutput(target, "summary")}`;
output += `\n Evidence: ${rel.evidence.length} items`;
}
return output;
}Command Summary
| Command | Description |
|---|---|
/add-entity | Create new entity |
/show-entity | Display entity details |
/update-entity | Modify existing entity |
/remove-entity | Remove entity (soft delete) |
/find-entities | Search/filter entities |
/add-relationship | Create entity relationship |
/show-relationships | List entity relationships |
/find-path | Find connection path between entities |
/xref | Cross-reference entity |
/query-entities | Advanced entity query |
/merge-entities | Merge duplicate entities |
/split-entity | Split multi-value entity |
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "osint://core/entity-schema.json",
"title": "OSINT Investigation Entity Schema v2.0",
"description": "Canonical JSON Schema for OSINT investigation entities, relationships, evidence, and audit history.",
"$comment": "Documentation source of truth: repository-root content under core/, examples/, and ux/ is canonical. Mirrored docs/ content is derivative and must not introduce schema changes independently.",
"type": "object",
"required": [
"version",
"investigation_id",
"created_at",
"entities",
"relationships"
],
"additionalProperties": false,
"properties": {
"version": {
"type": "string",
"const": "2.0.0",
"description": "Schema version."
},
"investigation_id": {
"$ref": "#/definitions/uuid",
"description": "Unique investigation identifier."
},
"name": {
"type": "string",
"maxLength": 256,
"description": "Investigation name or title."
},
"description": {
"type": "string",
"maxLength": 4000,
"description": "Investigation summary and objectives."
},
"classification": {
"type": "string",
"enum": [
"OPEN",
"CONFIDENTIAL",
"SENSITIVE",
"RESTRICTED"
],
"default": "OPEN",
"description": "Investigation classification level."
},
"created_at": {
"$ref": "#/definitions/timestamp"
},
"modified_at": {
"$ref": "#/definitions/timestamp"
},
"entities": {
"type": "array",
"items": {
"$ref": "#/definitions/entity"
}
},
"relationships": {
"type": "array",
"items": {
"$ref": "#/definitions/relationship"
}
},
"evidence_chains": {
"type": "array",
"items": {
"$ref": "#/definitions/evidenceChain"
}
},
"audit_log": {
"type": "array",
"items": {
"$ref": "#/definitions/auditEntry"
}
},
"session_metadata": {
"$ref": "#/definitions/sessionMetadata"
},
"tags": {
"type": "array",
"items": {
"type": "string"
},
"uniqueItems": true
},
"documentation": {
"$ref": "#/definitions/documentationMetadata"
}
},
"definitions": {
"confidenceLevel": {
"type": "string",
"enum": [
"CERTAIN",
"HIGH",
"MEDIUM",
"LOW",
"UNCERTAIN",
"DISPUTED"
],
"description": "Confidence in the accuracy of the data."
},
"evidenceType": {
"type": "string",
"enum": [
"DIRECT",
"INFERRED",
"CORROBORATED",
"CONFLICTING",
"HEARSAY"
],
"description": "Type of evidence supporting this data."
},
"sourceReliability": {
"type": "string",
"enum": [
"A",
"B",
"C",
"D",
"E",
"F"
],
"description": "A=Completely reliable, B=Usually reliable, C=Fairly reliable, D=Not usually reliable, E=Unreliable, F=Cannot be judged."
},
"timestamp": {
"type": "string",
"format": "date-time",
"description": "ISO 8601 timestamp."
},
"uuid": {
"type": "string",
"pattern": "^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$",
"description": "UUID identifier."
},
"documentationMetadata": {
"type": "object",
"description": "Machine-readable guidance for contributors about canonical documentation ownership.",
"additionalProperties": false,
"required": [
"source_of_truth",
"canonical_paths",
"mirrored_paths",
"notes"
],
"properties": {
"source_of_truth": {
"type": "string",
"const": "repository-root"
},
"canonical_paths": {
"type": "array",
"items": {
"type": "string"
},
"contains": {
"const": "core/entity-schema.json"
}
},
"mirrored_paths": {
"type": "array",
"items": {
"type": "string"
},
"description": "Derivative documentation trees that may summarize, reference, or mirror canonical root content."
},
"notes": {
"type": "string",
"minLength": 1
}
}
},
"sessionMetadata": {
"type": "object",
"additionalProperties": false,
"properties": {
"search_count": {
"type": "integer",
"minimum": 0
},
"entity_add_count": {
"type": "integer",
"minimum": 0
},
"relationship_add_count": {
"type": "integer",
"minimum": 0
},
"duration_seconds": {
"type": "integer",
"minimum": 0
},
"auto_save_enabled": {
"type": "boolean"
},
"last_auto_save": {
"$ref": "#/definitions/timestamp"
}
}
},
"entity": {
"type": "object",
"required": [
"id",
"type",
"value",
"created_at"
],
"additionalProperties": false,
"properties": {
"id": {
"$ref": "#/definitions/uuid"
},
"type": {
"type": "string",
"enum": [
"PERSON",
"ORGANIZATION",
"DOMAIN",
"EMAIL",
"USERNAME",
"IP_ADDRESS",
"IP_RANGE",
"PHONE",
"LOCATION",
"DEVICE",
"DOCUMENT",
"IMAGE",
"EVENT",
"URL",
"CRYPTO_ADDRESS",
"ASN",
"TECHNOLOGY",
"CUSTOM"
]
},
"value": {
"type": "string",
"description": "Primary identifier value."
},
"display_name": {
"type": "string",
"description": "Human-readable name."
},
"aliases": {
"type": "array",
"items": {
"type": "string"
},
"description": "Alternative names or identifiers."
},
"attributes": {
"type": "object",
"description": "Type-specific attributes.",
"additionalProperties": true
},
"confidence": {
"$ref": "#/definitions/confidenceLevel"
},
"evidence": {
"type": "array",
"items": {
"$ref": "#/definitions/evidence"
}
},
"source_entities": {
"type": "array",
"items": {
"$ref": "#/definitions/uuid"
},
"description": "Entities that led to discovering this one."
},
"status": {
"type": "string",
"enum": [
"ACTIVE",
"VERIFIED",
"DISPUTED",
"DEPRECATED",
"REMOVED"
],
"default": "ACTIVE"
},
"notes": {
"type": "string",
"maxLength": 10000
},
"first_seen": {
"$ref": "#/definitions/timestamp"
},
"last_seen": {
"$ref": "#/definitions/timestamp"
},
"created_at": {
"$ref": "#/definitions/timestamp"
},
"modified_at": {
"$ref": "#/definitions/timestamp"
},
"version": {
"type": "integer",
"minimum": 1,
"description": "Entity revision number."
},
"previous_versions": {
"type": "array",
"items": {
"$ref": "#/definitions/entityVersion"
}
},
"tags": {
"type": "array",
"items": {
"type": "string"
}
}
}
},
"entityVersion": {
"type": "object",
"additionalProperties": false,
"properties": {
"version": {
"type": "integer"
},
"modified_at": {
"$ref": "#/definitions/timestamp"
},
"changes": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"properties": {
"field": {
"type": "string"
},
"old_value": {},
"new_value": {}
}
}
}
}
},
"relationship": {
"type": "object",
"required": [
"id",
"source_id",
"target_id",
"type",
"created_at"
],
"additionalProperties": false,
"properties": {
"id": {
"$ref": "#/definitions/uuid"
},
"source_id": {
"$ref": "#/definitions/uuid",
"description": "Source entity ID."
},
"target_id": {
"$ref": "#/definitions/uuid",
"description": "Target entity ID."
},
"type": {
"type": "string",
"enum": [
"OWNERSHIP",
"EMPLOYMENT",
"CONTACT",
"REGISTRATION",
"HOSTING",
"REFERENCES",
"CONTAINS",
"AUTHENTICATES",
"COMMUNICATES_WITH",
"LOCATED_AT",
"ASSOCIATED_WITH",
"ALIAS_OF",
"PARENT_OF",
"CHILD_OF",
"SIBLING_OF",
"CONFLICTS_WITH",
"CUSTOM"
]
},
"direction": {
"type": "string",
"enum": [
"DIRECTED",
"BIDIRECTIONAL"
],
"default": "DIRECTED"
},
"confidence": {
"$ref": "#/definitions/confidenceLevel"
},
"evidence": {
"type": "array",
"items": {
"$ref": "#/definitions/evidence"
}
},
"attributes": {
"type": "object",
"description": "Relationship-specific attributes.",
"additionalProperties": true
},
"created_at": {
"$ref": "#/definitions/timestamp"
},
"modified_at": {
"$ref": "#/definitions/timestamp"
},
"verified": {
"type": "boolean",
"default": false
}
}
},
"evidence": {
"type": "object",
"required": [
"id",
"type",
"source",
"captured_at"
],
"additionalProperties": false,
"properties": {
"id": {
"$ref": "#/definitions/uuid"
},
"type": {
"$ref": "#/definitions/evidenceType"
},
"source": {
"type": "string",
"description": "Source description."
},
"source_url": {
"type": "string",
"format": "uri"
},
"source_reliability": {
"$ref": "#/definitions/sourceReliability"
},
"content": {
"type": "string",
"description": "Evidence content or excerpt."
},
"content_hash": {
"type": "string",
"pattern": "^[a-f0-9]{64}$",
"description": "SHA-256 hash of content for integrity."
},
"captured_at": {
"$ref": "#/definitions/timestamp"
},
"archived_copy": {
"type": "string",
"description": "Path to archived snapshot."
},
"corroborated_by": {
"type": "array",
"items": {
"$ref": "#/definitions/uuid"
},
"description": "Other evidence IDs that corroborate this item."
}
}
},
"evidenceChain": {
"type": "object",
"description": "Tracks how a conclusion was reached.",
"additionalProperties": false,
"properties": {
"id": {
"$ref": "#/definitions/uuid"
},
"conclusion_entity_id": {
"$ref": "#/definitions/uuid"
},
"steps": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"properties": {
"step_number": {
"type": "integer"
},
"entity_id": {
"$ref": "#/definitions/uuid"
},
"relationship_id": {
"$ref": "#/definitions/uuid"
},
"reasoning": {
"type": "string"
},
"evidence_id": {
"$ref": "#/definitions/uuid"
}
}
}
},
"created_at": {
"$ref": "#/definitions/timestamp"
}
}
},
"auditEntry": {
"type": "object",
"additionalProperties": false,
"properties": {
"id": {
"$ref": "#/definitions/uuid"
},
"timestamp": {
"$ref": "#/definitions/timestamp"
},
"action": {
"type": "string",
"enum": [
"ENTITY_CREATE",
"ENTITY_UPDATE",
"ENTITY_DELETE",
"RELATIONSHIP_CREATE",
"RELATIONSHIP_UPDATE",
"RELATIONSHIP_DELETE",
"EVIDENCE_ADD",
"SEARCH_EXECUTE",
"SESSION_SAVE",
"SESSION_LOAD",
"EXPORT",
"IMPORT",
"MERGE"
]
},
"entity_id": {
"$ref": "#/definitions/uuid"
},
"relationship_id": {
"$ref": "#/definitions/uuid"
},
"details": {
"type": "object",
"additionalProperties": true
}
}
}
},
"default": {
"version": "2.0.0",
"entities": [],
"relationships": [],
"evidence_chains": [],
"audit_log": [],
"tags": [],
"documentation": {
"source_of_truth": "repository-root",
"canonical_paths": [
"core/entity-schema.json",
"core/",
"examples/",
"ux/"
],
"mirrored_paths": [
"docs/core/",
"docs/examples/",
"docs/ux/"
],
"notes": "Update canonical repository-root content first. Mirrors under docs/ may summarize or copy root content, but must stay derivative."
}
}
}
Evidence System Specification
Overview
The Evidence System provides a structured framework for tracking, verifying, and weighting information gathered during OSINT investigations. Every piece of data must have associated evidence with clear provenance and confidence assessment.
Evidence Types
[DIRECT] Evidence
Direct evidence comes from firsthand observation or official records. This is the strongest evidence type.
Characteristics:
- Primary source material
- Directly observed or recorded
- Minimal interpretation required
- High confidence when source is reliable
Examples:
- WHOIS registration records
- SSL certificate data
- Official company registration documents
- Direct screenshots of a profile
- Raw API responses
- Email headers
Confidence Weight: 1.0 (baseline)
{
"id": "ev-001",
"type": "DIRECT",
"source": "WHOIS lookup for example.com",
"source_url": "https://whois.icann.org/...",
"source_reliability": "A",
"content": "Registrar: NameCheap, Inc. | Created: 2020-01-15",
"captured_at": "2024-01-15T10:30:00Z"
}[INFERRED] Evidence
Inferred evidence is derived through logical deduction from direct evidence. Requires explicit reasoning chain.
Characteristics:
- Derived from direct evidence
- Requires logical inference
- Documented reasoning required
- Confidence depends on inference strength
Examples:
- Inferring email from domain (admin@domain.com)
- Deducing location from IP geolocation
- Inferring real name from username patterns
- Deducing employment from LinkedIn connections
Confidence Weight: 0.6 - 0.8 (depending on inference strength)
{
"id": "ev-002",
"type": "INFERRED",
"source": "Logical deduction from domain",
"source_reliability": "N/A",
"content": "Email admin@example.com inferred from domain ownership",
"captured_at": "2024-01-15T10:35:00Z",
"inference_basis": ["ev-001"],
"reasoning": "Standard admin email pattern for owned domains"
}[CORROBORATED] Evidence
Corroborated evidence is confirmed by multiple independent sources.
Characteristics:
- Multiple independent sources
- Sources don't share common origin
- Consistent information across sources
- Higher confidence than single source
Examples:
- Email found on multiple social platforms
- Phone number in both WHOIS and business directory
- Name matching across LinkedIn and company registry
- Location confirmed by IP and social posts
Confidence Weight: 1.2 (multiplier applied to base)
{
"id": "ev-003",
"type": "CORROBORATED",
"source": "Cross-platform verification",
"source_reliability": "A",
"content": "john@example.com found on LinkedIn and GitHub profiles",
"captured_at": "2024-01-15T11:00:00Z",
"corroborated_by": ["ev-004", "ev-005"],
"corroboration_sources": [
{ "source": "LinkedIn profile", "evidence_id": "ev-004" },
{ "source": "GitHub profile", "evidence_id": "ev-005" }
]
}[CONFLICTING] Evidence
Conflicting evidence contradicts other known information and requires resolution.
Characteristics:
- Directly contradicts existing data
- Requires investigation
- Reduces confidence in disputed claims
- Must be resolved or documented as contradiction
Examples:
- Different registration dates in different WHOIS sources
- Name spelled differently across sources
- Location claims that can't coexist
- Age discrepancies
Confidence Weight: 0.0 (flags for review)
{
"id": "ev-006",
"type": "CONFLICTING",
"source": "Secondary WHOIS provider",
"source_reliability": "B",
"content": "Domain created 2019-12-01 (conflicts with 2020-01-15)",
"captured_at": "2024-01-15T11:30:00Z",
"conflicts_with": ["ev-001"],
"conflict_resolution": "PENDING",
"notes": "Requires verification against registry directly"
}[HEARSAY] Evidence
Hearsay evidence is secondhand information from unverified sources.
Characteristics:
- Not directly observed
- Passed through intermediaries
- Lower reliability
- Requires verification
Examples:
- Information from forum posts
- Claims in blog comments
- Unverified social media mentions
- Third-party reports without primary sources
Confidence Weight: 0.3
{
"id": "ev-007",
"type": "HEARSAY",
"source": "Forum post on reddit.com/r/example",
"source_reliability": "E",
"content": "User claims John Doe works at Example Corp",
"captured_at": "2024-01-15T12:00:00Z",
"verification_status": "UNVERIFIED",
"notes": "Requires confirmation from official source"
}Source Reliability Scale
| Grade | Description | Examples |
|---|---|---|
| A | Completely Reliable | Official registries, primary sources, direct observation |
| B | Usually Reliable | Established news, corporate websites, verified social media |
| C | Fairly Reliable | Blogs with track record, industry publications |
| D | Not Usually Reliable | Anonymous forums, unverified claims |
| E | Unreliable | Known disinformation sources, highly biased sources |
| F | Cannot Be Judged | Insufficient information to assess |
Evidence Strength Matrix
Calculate evidence strength by combining evidence type and source reliability:
const evidenceTypeWeights = {
DIRECT: 1.0,
INFERRED: 0.6,
CORROBORATED: 1.2,
CONFLICTING: 0.0,
HEARSAY: 0.3
};
const sourceReliabilityWeights = {
A: 1.0,
B: 0.8,
C: 0.6,
D: 0.4,
E: 0.2,
F: 0.5 // Unknown reliability = moderate weight
};
function calculateEvidenceStrength(evidence) {
const typeWeight = evidenceTypeWeights[evidence.type];
const reliabilityWeight = sourceReliabilityWeights[evidence.source_reliability];
// Base calculation
let strength = typeWeight * reliabilityWeight;
// Adjustments
if (evidence.type === "CORROBORATED" && evidence.corroborated_by) {
// Bonus for multiple corroborating sources
const corroborationBonus = Math.min(
evidence.corroborated_by.length * 0.1,
0.3
);
strength += corroborationBonus;
}
if (evidence.content_hash) {
// Bonus for content integrity verification
strength += 0.05;
}
if (evidence.archived_copy) {
// Bonus for archived snapshot
strength += 0.05;
}
return Math.min(strength, 1.5); // Cap at 1.5
}Evidence Chain Tracking
Document how conclusions are reached through linked evidence:
function createEvidenceChain(conclusionEntityId, steps) {
const chain = {
id: generateUUID(),
conclusion_entity_id: conclusionEntityId,
steps: steps.map((step, index) => ({
step_number: index + 1,
entity_id: step.entityId,
relationship_id: step.relationshipId,
evidence_id: step.evidenceId,
reasoning: step.reasoning,
confidence_at_step: calculateStepConfidence(steps.slice(0, index + 1))
})),
overall_confidence: calculateChainConfidence(steps),
created_at: now()
};
return chain;
}
function calculateStepConfidence(stepsSoFar) {
// Confidence compounds with each step
return stepsSoFar.reduce((acc, step) => {
const evidence = getEvidence(step.evidenceId);
const stepStrength = calculateEvidenceStrength(evidence);
return acc * stepStrength;
}, 1.0);
}
// Example: Chain showing how we determined a person's employer
const employmentChain = createEvidenceChain("person-001", [
{
entityId: "email-001",
relationshipId: "rel-001",
evidenceId: "ev-003",
reasoning: "Email found on LinkedIn profile, corroborated by GitHub"
},
{
entityId: "linkedin-001",
relationshipId: "rel-002",
evidenceId: "ev-008",
reasoning: "LinkedIn profile lists current employment at Example Corp"
},
{
entityId: "employer-001",
evidenceId: "ev-009",
reasoning: "Company website lists employee in team directory"
}
]);Uncertainty Quantification
Express confidence levels with explicit uncertainty bounds:
function quantifyUncertainty(entity) {
const evidenceStrengths = entity.evidence.map(calculateEvidenceStrength);
if (evidenceStrengths.length === 0) {
return {
confidence: "UNCERTAIN",
confidence_score: 0.0,
uncertainty_range: [0.0, 0.1],
basis: "No evidence"
};
}
// Strongest single evidence
const maxStrength = Math.max(...evidenceStrengths);
// Combined strength (with diminishing returns)
const combinedStrength = evidenceStrengths.reduce((acc, strength) => {
return acc + (strength * (1 - acc) * 0.5);
}, 0);
// Calculate uncertainty range
const variance = calculateVariance(evidenceStrengths);
const uncertaintyMargin = Math.sqrt(variance);
const confidenceScore = Math.max(maxStrength, combinedStrength);
const lowerBound = Math.max(0, confidenceScore - uncertaintyMargin);
const upperBound = Math.min(1.5, confidenceScore + uncertaintyMargin);
// Map to confidence level
const confidence = scoreToConfidenceLevel(confidenceScore);
return {
confidence: confidence,
confidence_score: confidenceScore,
uncertainty_range: [lowerBound, upperBound],
evidence_count: evidenceStrengths.length,
strongest_evidence: maxStrength,
combined_strength: combinedStrength,
variance: variance,
basis: generateConfidenceBasis(entity.evidence)
};
}
function scoreToConfidenceLevel(score) {
if (score >= 1.2) return "CERTAIN";
if (score >= 0.8) return "HIGH";
if (score >= 0.5) return "MEDIUM";
if (score >= 0.3) return "LOW";
return "UNCERTAIN";
}
function generateConfidenceBasis(evidence) {
const types = [...new Set(evidence.map(e => e.type))];
const directCount = evidence.filter(e => e.type === "DIRECT").length;
const corroboratedCount = evidence.filter(e => e.type === "CORROBORATED").length;
let basis = `Based on ${evidence.length} evidence item(s)`;
if (directCount > 0) basis += `, including ${directCount} direct`;
if (corroboratedCount > 0) basis += `, ${corroboratedCount} corroborated`;
return basis;
}Evidence Verification Workflow
const verificationWorkflow = {
// Step 1: Capture evidence
capture: (source, content, options) => {
const evidence = {
id: generateUUID(),
type: options.type || "DIRECT",
source: source,
source_url: options.url,
source_reliability: assessReliability(source),
content: content,
content_hash: hashContent(content),
captured_at: now(),
archived_copy: options.archive ? archiveContent(source, content) : null
};
return evidence;
},
// Step 2: Assess initial reliability
assessReliability: (source) => {
// Check against known reliable sources
if (isOfficialRegistry(source)) return "A";
if (isEstablishedPublication(source)) return "B";
if (isKnownBlog(source)) return "C";
if (isAnonymousForum(source)) return "D";
if (isKnownUnreliable(source)) return "E";
return "F";
},
// Step 3: Cross-reference with existing evidence
crossReference: (newEvidence, existingEvidence) => {
const matches = [];
const conflicts = [];
existingEvidence.forEach(existing => {
const similarity = compareEvidence(newEvidence, existing);
if (similarity > 0.9) {
matches.push({ evidence: existing, similarity });
} else if (similarity < 0.5 && sameSubject(newEvidence, existing)) {
conflicts.push({ evidence: existing, similarity });
}
});
// If corroborating matches found, upgrade type
if (matches.length > 0 && newEvidence.type === "DIRECT") {
newEvidence.type = "CORROBORATED";
newEvidence.corroborated_by = matches.map(m => m.evidence.id);
}
// If conflicts found, flag for review
if (conflicts.length > 0) {
newEvidence.type = "CONFLICTING";
newEvidence.conflicts_with = conflicts.map(c => c.evidence.id);
newEvidence.conflict_resolution = "PENDING";
}
return { evidence: newEvidence, matches, conflicts };
},
// Step 4: Calculate final strength
finalize: (evidence) => {
evidence.strength = calculateEvidenceStrength(evidence);
evidence.verified = evidence.strength > 0.7;
return evidence;
}
};Evidence Archiving
Preserve evidence at capture time to prevent loss or tampering:
function archiveEvidence(sourceUrl, content) {
const timestamp = now();
const sanitizedName = sanitizeFilename(sourceUrl);
const archivePath = `./archives/${sanitizedName}_${timestamp}.html`;
// If URL, attempt to save full page
if (sourceUrl.startsWith('http')) {
const archived = archiveWebPage(sourceUrl, archivePath);
return archived ? archivePath : null;
}
// Otherwise, save content to file
writeFile(archivePath, content);
return archivePath;
}
function verifyArchiveIntegrity(archivePath, originalHash) {
const archivedContent = readFile(archivePath);
const archivedHash = hashContent(archivedContent);
return archivedHash === originalHash;
}Command Interface
// Add evidence to entity
/add-evidence <entity_id> <type> --source <desc> --content <data>
// Examples:
/add-evidence domain-001 DIRECT \\
--source "WHOIS lookup" \\
--content "Registrar: NameCheap" \\
--url "https://whois.example.com" \\
--archive
/add-evidence person-001 INFERRED \\
--source "Email pattern analysis" \\
--content "john.doe@example.com" \\
--basis entity-002 \\
--reasoning "Standard corporate email format"
// Show evidence for entity
/show-evidence <entity_id> [--details]
// Verify evidence integrity
/verify-evidence <evidence_id>
// Archive evidence
/archive-evidence <evidence_id> [url]
// Show evidence chain
/show-chain <entity_id>
// Compare evidence
/compare-evidence <id1> <id2>
// Flag conflicting evidence
/flag-conflict <evidence_id> --conflicts-with <other_id>
// Resolve conflict
/resolve-conflict <evidence_id> --resolution <verdict> [--notes <text>]Evidence Report Generation
function generateEvidenceReport(entity) {
const report = {
entity_id: entity.id,
entity_value: entity.value,
overall_confidence: quantifyUncertainty(entity),
evidence_summary: {
total: entity.evidence.length,
by_type: countBy(entity.evidence, 'type'),
by_reliability: countBy(entity.evidence, 'source_reliability')
},
evidence_details: entity.evidence.map(ev => ({
id: ev.id,
type: ev.type,
source: ev.source,
reliability: ev.source_reliability,
strength: calculateEvidenceStrength(ev),
captured: ev.captured_at,
verified: ev.verified || false,
content_preview: ev.content.substring(0, 100) + "..."
})),
chains: getEvidenceChains(entity.id),
recommendations: generateRecommendations(entity)
};
return report;
}
function generateRecommendations(entity) {
const recs = [];
const analysis = quantifyUncertainty(entity);
if (analysis.confidence_score < 0.5) {
recs.push("Collect additional direct evidence to improve confidence");
}
if (!entity.evidence.some(e => e.type === "DIRECT")) {
recs.push("No direct evidence found - seek primary sources");
}
if (entity.evidence.some(e => e.type === "CONFLICTING")) {
recs.push("Resolve conflicting evidence before relying on this entity");
}
const lowReliabilityCount = entity.evidence.filter(
e => ['D', 'E', 'F'].includes(e.source_reliability)
).length;
if (lowReliabilityCount > entity.evidence.length / 2) {
recs.push("Majority of sources have low reliability - seek better sources");
}
return recs;
}Evidence Strength Visualization
Evidence Strength for: john@example.com
═══════════════════════════════════════════════════════════════
Overall Confidence: HIGH (0.85)
Uncertainty Range: [0.72, 0.98]
Evidence Items (4 total):
┌─────────────────────────────────────────────────────────────┐
│ [CORROBORATED] Email on social profiles │
│ Source: LinkedIn + GitHub (Reliability: A) │
│ Strength: ████████████████████░░░░░ 1.25 │
│ Corroborated by: 2 sources │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ [DIRECT] WHOIS record │
│ Source: ICANN WHOIS (Reliability: A) │
│ Strength: ████████████████░░░░░░░░░ 1.00 │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ [DIRECT] Company directory │
│ Source: Example Corp website (Reliability: B) │
│ Strength: ████████████░░░░░░░░░░░░░ 0.80 │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ [INFERRED] Pattern match │
│ Source: Email format analysis (Reliability: N/A) │
│ Strength: ███████░░░░░░░░░░░░░░░░░░ 0.60 │
│ ⚠️ Inference: Standard corporate format │
└─────────────────────────────────────────────────────────────┘
Recommendations:
• Evidence base is strong - confidence is well-supported
• Consider archiving corroborating sources for preservationOSINT Investigator — Advanced User Guide
Overview
This guide is for power users who want to maximize the OSINT Investigator skill's capabilities through advanced workflows, automation, and custom configurations.
---
Advanced Command Combinations
The "Full Spectrum" Investigation
For comprehensive due diligence:
/full target.com → /coverage → /qa-check → /report + /simple-reportWhen to use: Starting new investigations, comprehensive background checks Time required: 10-15 minutes Output: Complete analysis with quality validation
The "Entity Deep Dive"
For mapping complex relationships:
/track entity1 → /pivot entity1 → /track [new findings] → /link entity1 entity2 → /visualize entities → /find-path entity1 entity3When to use: Mapping networks, finding hidden connections Time required: 5-10 minutes per entity Output: Complete relationship graph with connection paths
The "Threat Hunter" Workflow
For security investigations:
/dork domain.com → /recon domain.com → /visualize attack → /risk-score domain.com → /anomaly → /threat-modelWhen to use: Security assessments, threat hunting Time required: 15-20 minutes Output: Security assessment with risk scoring
The "Timeline Reconstruction"
For chronological investigations:
/timeline subject → /timeline-entity entity1 → /timeline-entity entity2 → /compare entity1 entity2 → /visualize timelineWhen to use: Event reconstruction, alibi verification Time required: 10-15 minutes Output: Synchronized timeline across multiple entities
The "Source Verification Chain"
For ensuring quality:
/qa-check → /verify-sources → /coverage → /gaps → [address gaps] → /qa-checkWhen to use: Before finalizing reports, quality milestones Time required: 10 minutes + gap remediation time Output: Quality-validated investigation
---
Automation Strategies
Batch Entity Processing
When investigating multiple related entities:
1. Create entity list
2. /track entity1
3. /track entity2
4. /track entity3
5. /entities # Review all tracked
6. FOR each entity:
- /pivot [entity]
- /timeline-entity [entity]
7. /link [related entities]
8. /visualize entitiesTemplate Sequences
Create custom investigation sequences:
Quick Triage (5 minutes):
/recon [target] → /risk-score [target] → /simple-reportDeep Investigation (30 minutes):
/wizard [type] [target] → /full [target] → /coverage → /qa-check → /report + /simple-reportSecurity Assessment (20 minutes):
/dork [domain] → /recon [domain] → /visualize attack → /risk-score [domain] → /threat-model → /export-riskCheckpoint Automation
For long investigations, automate checkpoints:
# After each major phase
/save-checkpoint "phase-name"
# Naming convention
/save-checkpoint "initial-recon-complete"
/save-checkpoint "entity-mapping-50pct"
/save-checkpoint "pre-report-review"---
Custom Template Creation
Creating Investigation Templates
Step 1: Define Template Structure
# Custom Template: [Name]
## Trigger: /template [name]
## Steps:
1. /recon {{target}}
2. IF domain detected: /dork {{target}}
3. /track [entities found]
4. FOR each username: /pivot [username]
5. /timeline {{target}}
6. /coverage
7. IF coverage < 60%: Prompt for additional vectors
8. /qa-check
9. IF quality < 75%: Suggest improvements
10. /report
11. /simple-reportStep 2: Add to Template System
Add template metadata:
---
template_name: corporate-due-diligence
description: "Deep investigation for M&A due diligence"
target_types: [person, organization]
estimated_time: "20-30 minutes"
outputs: [report, simple-report, risk-assessment]
---Step 3: Usage
/template corporate-due-diligence target-company.comTemplate Variables
Available variables for custom templates:
| Variable | Description | Example |
|---|---|---|
{{target}} | Investigation target | example.com |
{{date}} | Current date | 2024-01-15 |
{{entities}} | Discovered entities | [list] |
{{confidence}} | Current confidence | 75% |
{{coverage}} | Coverage percentage | 60% |
---
Batch Operations
Multi-Target Investigation
When investigating multiple targets simultaneously:
Approach 1: Sequential Processing
FOR target IN [target1, target2, target3]:
/track {{target}}
/recon {{target}}
/timeline {{target}}
END
/compare target1 target2 target3
/link [common entities]
/visualize entitiesApproach 2: Parallel Tracking
/track target1 as T1
/track target2 as T2
/track target3 as T3
# Process common pivots
/pivot [shared entity]
/link T1 T2 [common finding]
/link T2 T3 [common finding]Bulk Entity Import
For importing external entity lists:
Format:
{
"entities": [
{"type": "person", "name": "John Doe", "confidence": "medium"},
{"type": "domain", "name": "example.com", "confidence": "high"}
],
"relationships": [
{"from": "John Doe", "to": "example.com", "type": "owns"}
]
}Import:
/import-entities [paste JSON]
/entities # Verify import---
Integration Workflows
Obsidian Integration Workflow
Real-time Sync Approach:
1. Create Obsidian vault using templates from integrations/obsidian-setup.md
2. During investigation:
- Export entities: /export-entities markdown
- Paste into Obsidian 02-Entity-Database/
3. Create investigation note from template
4. Link entities to investigation
5. Use Obsidian graph view for analysis
6. Export final report from ObsidianBatch Export Approach:
/full target.com
/export-entities markdown
# Import all files to Obsidian at once
# Use Obsidian templates for structureNotion Integration Workflow
Structured Database Approach:
1. Set up Notion databases using integrations/notion-schema.md
2. During investigation:
- /track [entity]
- Copy entity data to Notion People/Domains database
3. Link to Investigation database entry
4. Use Notion relations to build connections
5. Export reports from NotionMaltego Integration Workflow
Visual Analysis Approach:
/full target.com
/export-graph graphml
# Import to Maltego
# Run transforms on imported entities
# Export enhanced graph back to investigation---
Advanced Entity Analysis
Multi-Hop Path Analysis
Finding indirect connections:
/find-path entityA entityB
# If no direct path found:
/track intermediate entities
/link entityA intermediate1
/link intermediate1 intermediate2
/link intermediate2 entityB
/find-path entityA entityBConfidence Propagation
When confidence in one entity affects others:
IF entityA.confidence = "high"
AND entityA linked to entityB
THEN entityB.confidence = at least "medium"
/confidence entityA high
/link entityA entityB
/confidence entityB medium # Derived from AEntity Clustering
Group related entities:
/track cluster1-entity1
/track cluster1-entity2
/track cluster1-entity3
# Create cluster relationship
/link cluster1-entity1 cluster1-entity2 associates
/link cluster1-entity1 cluster1-entity3 associates
# Visualize cluster
/visualize entities---
Quality Assurance Automation
Pre-Report Checklist
Automated before generating reports:
/qa-check
IF quality_score < 75:
/coverage
/gaps
# Address critical gaps
/qa-check # Re-verify
/verify-sourcesContinuous Quality Monitoring
During investigation:
# After every 5 entities discovered
/qa-check
IF new_entities > 5:
/coverage
# Before each checkpoint
/qa-check
/save-checkpoint "[name]-quality-[score]"---
Performance Optimization
Large Investigation Management
For investigations with 50+ entities:
1. Split by Phase:
# Phase 1: Core entities only
/track [core entities only]
/recon target
/coverage
# Phase 2: Expand to associates
/pivot [core entities]
/track [associates]
/coverage
# Phase 3: Full network
/pivot [associates]2. Filtered Visualizations:
/visualize entities # May be too large
# Instead:
/track subset-entity1
/track subset-entity2
/visualize entities # Limited to tracked subset3. Regular Checkpoints:
/save-checkpoint after each phase
/load-checkpoint if performance degradesQuery Optimization
Efficient search patterns:
# Instead of multiple pivots:
/pivot entity1
/pivot entity2
/pivot entity3
# Use batch approach:
/track entity1
/track entity2
/track entity3
/entities # Review all, identify common patterns---
Pattern Recognition
Automated Pattern Detection
/anomaly # Detect outliers
/pattern # Identify commonalities
# When pattern found:
/track pattern-entities
/link [pattern entities] "shared-pattern-[type]"Temporal Pattern Analysis
/timeline subject
# Look for:
# - Regular intervals (scheduled activity)
# - Clusters (events grouped in time)
# - Gaps (missing activity periods)
# - Correlations (events across entities)---
Risk Analysis Deep Dive
Custom Risk Scoring
Adjust risk calculation weights:
/risk-score target --weight-exposure=1.5 --weight-security=2.0
# Increases weight of security factorsComparative Risk Analysis
/risk-score target1
/risk-score target2
/compare target1 target2
# Analyze differences in risk profilesRisk Trending
# Initial assessment
/risk-score target
/save-checkpoint "initial-risk-[score]"
# After new findings
/risk-score target
# Compare with previous score---
Collaboration Workflows
Team Investigation Setup
1. Establish Common Framework:
# All team members use:
/template [team-template]
/simple-mode off2. Entity Synchronization:
/export-entities json
# Share with team
/import-entities [team entities]3. Progress Coordination:
/progress
# Share progress percentage
# Coordinate on gap areasClient Reporting Workflow
# Technical team:
/full target.com
/qa-check
# For technical stakeholders:
/report
# For executive stakeholders:
/simple-report
# For security teams:
/risk-score target.com
/export-risk---
Troubleshooting Advanced Issues
Entity Map Corruption
Symptoms: Duplicate entities, broken links
Recovery:
/entities # Identify duplicates
# Manually consolidate
/sanitize # Remove invalid data
/rebuild-entities # If availableCoverage Gaps Persist
When `/gaps` shows unresolved gaps:
/gaps
# For each critical gap:
# 1. Document attempt
# 2. Note why gap exists (privacy, unavailable, etc.)
# 3. Assess impact on conclusions
# 4. Proceed with caveatsQuality Score Stagnation
When `/qa-check` score won't improve:
/qa-check
# Review specific issues
# Prioritize by:
# 1. Impact on findings
# 2. Time required
# 3. Information availability
# Accept 70+ for time-sensitive reports---
Best Practices for Power Users
1. Master the `/full` command — It's the fastest path to comprehensive results 2. Use checkpoints religiously — Save before major operations 3. Export early and often — Don't lose work to system issues 4. Leverage visualizations — They reveal patterns text misses 5. Automate quality checks — Run /qa-check at milestones 6. Document everything — Future you will thank present you 7. Use templates — Consistency improves efficiency 8. Learn the shortcuts — Command combinations save time 9. Know when to stop — Chasing 100% coverage has diminishing returns 10. Share your workflows — Help the community improve
---
Command Reference Quick Sheet
Investigation Lifecycle
Start: /wizard [type] [target] OR /full [target]
Track: /track [entity] → /link [A] [B]
Analyze: /anomaly → /pattern → /risk-score
Validate: /coverage → /qa-check → /verify-sources
Report: /report → /simple-report → /export-riskEntity Management
Add: /track [entity]
Connect: /link [A] [B]
View: /entities
Find: /find-path [A] [B]
Export: /export-entities [format]Visualization
Entities: /visualize entities
Timeline: /visualize timeline
Attack: /visualize attack
Surface: /visualize surface
Export: /export-graph [format]Quality Assurance
Check: /qa-check
Coverage: /coverage
Gaps: /gaps
Sources: /verify-sources---
Additional Resources
- SKILL.md — Complete command reference
- CHANGELOG.md — Version history and features
- Troubleshooting.md — Common issues and solutions
- Professional Playbooks — Domain-specific workflows
- Integration Guides — Tool setup instructions
Contributing to OSINT Investigator
Thank you for your interest in contributing! This document provides guidelines for contributing to the OSINT Investigator skill.
How to Contribute
Reporting Issues
1. Check if the issue already exists 2. Provide clear reproduction steps 3. Include example inputs/outputs 4. Note your environment (agent type, version)
Suggesting Features
1. Open a discussion first for major changes 2. Explain the use case 3. Describe expected behavior
Adding Dork Patterns
Dork patterns are in references/dork-library.md:
### Platform Name
- `site:platform.com "target"` - Description of what this finds
- `site:platform.com inurl:profile "target"` - Another patternRequirements:
- Test the dork actually works
- Include a brief description
- Categorize by target type (person, domain, org)
Adding Reconnaissance Vectors
Vectors are in references/recon-vectors.md:
Follow the existing structure: 1. Target type identification 2. Search sequence 3. Expected findings 4. Pivot opportunities
Improving Documentation
- Keep language clear and concise
- Include examples
- Update table of contents if adding sections
Code Style
For SKILL.md updates
- Use consistent heading levels
- Include confidence rating examples
- Add source citations where relevant
For Report Templates
- Maintain professional tone
- Include all required sections
- Ensure markdown renders correctly
Pull Request Process
1. Fork the repository 2. Create a feature branch (git checkout -b feature/amazing-feature) 3. Commit your changes (git commit -m 'Add amazing feature') 4. Push to the branch (git push origin feature/amazing-feature) 5. Open a Pull Request
PR Requirements
- Clear description of changes
- Reference any related issues
- Test that skill still loads correctly
- Update documentation if needed
Development Setup
To test changes:
1. Copy modified files to your agent's skill directory 2. Restart or reload your agent 3. Test commands in a fresh conversation 4. Verify no syntax errors in markdown
Areas Needing Help
- [ ] More platform-specific dork patterns
- [ ] Additional reconnaissance vectors for:
- Cryptocurrency addresses
- Vehicle registration lookups
- Property records
- [ ] Translation to other languages
- [ ] Video tutorial scripts
- [ ] Example investigation walkthroughs
Questions?
- Open a Discussion for general questions
- Comment on existing Issues/PRs
- Tag maintainers for urgent issues
Code of Conduct
- Be respectful and constructive
- Focus on the skill, not the person
- Welcome newcomers
- Assume good intent
Thank you for contributing to open-source OSINT!