Literature Review

Scientific Literature Review Skill

Overview

This skill guides comprehensive, systematic literature reviews that combine automated search capabilities across multiple academic databases with iterative analysis and synthesis into well-structured reports. It transforms the literature review process from a manual, time-consuming task into an efficient, systematic, and reproducible research methodology.

When to Use This Skill

Use this skill when you need to:

• Comprehensive Knowledge Synthesis: Gather and integrate knowledge across multiple research domains or methodologies
• Research Gap Identification: Identify underexplored areas, conflicting findings, or methodological limitations
• Methodology Tracking: Understand the evolution and current state of research methodologies and techniques
• Evidence Evaluation: Assess the quality, reliability, and significance of research findings
• Literature-Supported Writing: Write evidence-based sections for papers, grants, dissertations, or technical reports
• Research Foundation: Build a solid foundation for new research by understanding prior work
• Systematic Assessment: Conduct systematic reviews, meta-analyses, or structured literature analyses
• Knowledge Documentation: Create comprehensive, citable references for complex research domains

CRITICAL: Citation Verification Requirements

Every citation included in a literature review MUST be verified before inclusion. Hallucinated or fabricated citations undermine scientific integrity and are unacceptable.

Mandatory Verification Protocol

Before including ANY citation in the review, you MUST complete one of these verification steps:

1. CrossRef API Verification (Preferred for DOIs)

• Query https://api.crossref.org/works/{doi} via WebFetch
• Confirm the paper exists and metadata matches (title, authors, year)
• Extract the verified DOI for inclusion

2. WebSearch Verification (For papers without known DOIs)

• Search for the exact paper title in quotes
• Verify the paper appears in search results with matching authors/year
• Obtain the actual URL or DOI from search results

3. Database Search Verification (For specific databases)

• Search PubMed, arXiv, or domain-specific databases
• Confirm the paper exists with matching metadata
• Record the database identifier (PMID, arXiv ID, etc.)

4. User-Provided References (Still require verification)

• Even if the user provides a reference, verify it exists
• Confirm the DOI resolves or URL is accessible
• Report any discrepancies to the user

Verification Workflow

For each paper you intend to cite:

1. SEARCH: Find the paper via WebSearch or database query
2. VERIFY: Confirm paper exists (check DOI via CrossRef or URL via WebFetch)
3. EXTRACT: Record verified metadata (title, authors, journal, year, DOI/URL)
4. CITE: Only include the citation after verification succeeds
5. FLAG: If verification fails, DO NOT include the citation

What NOT to Do (Forbidden Practices)

NEVER do any of the following:

1. Never fabricate DOIs

• Do NOT create DOIs by pattern-matching (e.g., guessing 10.1038/ + random numbers)
• Do NOT assume a DOI format based on publisher patterns
• If you don't have a verified DOI, don't include one

2. Never include unverified citations

• Do NOT cite papers you haven't confirmed exist
• Do NOT assume a paper exists because it "sounds plausible"
• Do NOT include citations based on memory or training data alone

3. Never fabricate author names

• Do NOT guess author names or initials
• Do NOT create plausible-sounding author combinations
• Only use author names from verified sources

4. Never guess bibliographic details

• Do NOT invent page numbers, volume numbers, or issue numbers
• Do NOT fabricate journal names or conference proceedings
• Do NOT estimate publication years

5. Never cite papers you cannot verify

• If CrossRef returns 404, the DOI doesn't exist - don't use it
• If WebSearch finds no matching paper, it may not exist - don't cite it
• If you cannot access verification, explicitly state "citation unverified"

Acceptable Citation Sources

Citations may ONLY come from:

Handling Verification Failures

When verification fails:

1. Paper not found: Do NOT include the citation. Instead, note: "Unable to verify reference for [topic]. Further manual verification needed."

2. Partial match: If metadata partially matches but differs (different year, slightly different title), report the discrepancy and use ONLY the verified information.

3. Access restricted: If you cannot verify due to paywalls or access limits, explicitly mark as "[Verification pending - paywall]" and recommend manual verification.

4. Conflicting information: If multiple sources give conflicting metadata, use the DOI-registered metadata from CrossRef as authoritative.

Citation Format After Verification

Only after verification, format citations as:

Author(s). "Title." Journal, Volume(Issue), Pages (Year). DOI or URL

Example (verified):
Smith, J., & Jones, M. "Deep Learning for Medical Imaging." Nature Medicine, 29(3), 456-467 (2023). https://doi.org/10.1038/s41591-023-02345-6

If DOI is unavailable but URL is verified:

Smith, J., & Jones, M. "Deep Learning for Medical Imaging." Nature Medicine (2023). https://www.nature.com/articles/s41591-023-02345-6

Verification Documentation

For transparency, maintain a verification log:

| Citation | Verification Method | Status | Notes |
|----------|---------------------|--------|-------|
| Smith et al. 2023 | CrossRef API | Verified | DOI: 10.1038/xxx |
| Jones 2022 | WebSearch | Verified | URL confirmed |
| Brown et al. 2021 | CrossRef API | FAILED | DOI not found - excluded |
| Lee 2020 | PubMed | Verified | PMID: 12345678 |

Integration with Citation-Verifier Skill

After completing the literature review, use the citation-verifier skill to perform a final audit of all citations in the document. This provides an independent verification pass to catch any errors.

Literature Review Types

1. Narrative Literature Reviews

• Purpose: Provide comprehensive overview of a research topic
• Scope: Broad, subjective assessment of literature
• Search Strategy: Exploratory, iterative search across multiple angles
• Report Structure: Thematic organization by research concepts
• Best For: Introducing new research areas, providing context for research questions
• Timeline: 4-12 weeks for comprehensive review

2. Systematic Literature Reviews

• Purpose: Answer specific research questions using explicit methodology
• Scope: Comprehensive, predefined search with inclusion/exclusion criteria
• Search Strategy: Exhaustive search of specified databases
• Report Structure: PRISMA-compliant with search strategy documentation
• Best For: Evidence synthesis, clinical decision-making, policy development
• Timeline: 3-12 months for comprehensive review

3. Meta-Analyses

• Purpose: Quantitative synthesis of comparable study results
• Scope: Systematic review with statistical analysis
• Search Strategy: Identical to systematic reviews
• Report Structure: Statistical analysis with forest plots and funnel plots
• Best For: Combining results from multiple studies to determine overall effect
• Timeline: 6-18 months

4. Scoping Reviews

• Purpose: Map research landscape, identify key concepts and gaps
• Scope: Broader than systematic reviews, flexible inclusion criteria
• Search Strategy: Iterative, including grey literature
• Report Structure: Overview of research landscape with gap identification
• Best For: Emerging research areas, protocol development
• Timeline: 2-6 months

Systematic Literature Review Workflow

Phase 1: Planning and Question Formulation

Step 1: Define Research Question

Transform broad research interests into specific, answerable questions using PICO/PEO framework:

• Population/Problem: Who/what is the focus? (e.g., patients, methodologies, organisms)
• Intervention/Indicator: What is being studied? (e.g., treatment, technique, technology)
• Comparison: What alternative is compared? (optional, but recommended)
• Outcome: What results/impacts matter? (e.g., effectiveness, efficiency, validity)
• Experience (for qualitative): What are participants' perspectives?

Example PICO Questions:

• "What is the effectiveness of deep learning methods compared to traditional machine learning for medical image analysis in cancer detection?"
• "What methodologies are used to assess uncertainty in computational chemistry predictions?"
• "How do research groups address reproducibility in high-throughput screening studies?"

Step 2: Establish Scope and Criteria

Define Inclusion/Exclusion Criteria:

• Document type (peer-reviewed journals, preprints, grey literature)
• Time period (publication date range)
• Language (English-only or other languages)
• Study design (specific methodologies, approaches)
• Population/scope (specific organisms, systems, technologies)
• Quality thresholds (minimum standards for inclusion)

Example Inclusion Criteria:

• Peer-reviewed research articles published 2015-2025
• English language publications
• Empirical studies with quantitative or qualitative methodology
• Focus on computational methods in materials science
• Articles with >10 citations or published in high-impact journals (alternative: no citation threshold for very recent work)

Step 3: Plan Search Strategy

Select Appropriate Databases:

• PubMed/MEDLINE: Biomedical and life sciences literature
• arXiv: Preprints in physics, mathematics, computer science, statistics
• Web of Science: Multidisciplinary citation index with impact metrics
• Scopus: Large multidisciplinary abstract and citation database
• IEEE Xplore: Engineering and computer science literature
• ACM Digital Library: Computer science and information technology
• Springer Link: Multidisciplinary academic publisher
• ProQuest: Dissertations, theses, and comprehensive database
• Google Scholar: Broad academic search (verify findings in other databases)
• Domain-Specific Repositories: ArXiv (physics), bioRxiv (biology), chemRxiv (chemistry), SSRN (social sciences)

Formulate Search Queries:

• Use Boolean operators (AND, OR, NOT)
• Include synonyms and related terms
• Use wildcards and truncation (* for variations)
• Apply field-specific search syntax for each database
• Test query effectiveness before scaling

Example Boolean Queries:

• "machine learning" AND ("medical imaging" OR "diagnosis") AND ("cancer" OR "oncology")
• ("deep learning" OR "neural network") NOT "toy dataset"
• "materials discovery" AND (high-throughput OR computational) AND (screening OR optimization)

Step 4: Prepare Documentation System

Create a searchable, organized system for tracking:

• Search strategies and queries used
• Databases searched and date ranges
• Number of results per search
• Selection decisions and reasoning
• Paper characteristics (methodology, quality, relevance)
• Extracted data and findings
• Progress tracking and milestones

Recommended tools and formats:

• Spreadsheet/database: Track all papers with metadata
• Reference manager: Store full citations and PDFs (Zotero, Mendeley, EndNote)
• Note-taking system: Detailed notes on each paper
• Search log: Document all searches performed

Phase 2: Initial Broad Search

Step 5: Execute Broad Searches

Search across selected databases: 1. Start with primary query formulation 2. Execute search in each database 3. Limit results (usually by date range, document type) 4. Export results with complete metadata 5. Document search parameters and result counts

Expected workflow:

• Initial searches typically return 100-5,000+ results depending on topic breadth
• For broad topics, expect 500-5,000 results
• For narrow topics, expect 50-500 results
• If results exceed 10,000, refine search strategy

Documentation:

Search #1: "machine learning" AND "medical imaging"
Database: PubMed
Date: 2025-01-15
Results: 2,847 papers
Time period: 2015-2025
Filters: English language, human studies excluded at this stage
Export: PubMed format with abstracts
Notes: High number of results requires refinement

Step 6: Initial Screening

Level 1: Title and Abstract Screening

Review titles and abstracts to identify potentially relevant papers:

Inclusion criteria:

• Research question directly related
• Appropriate study type/design
• Relevant population/scope
• Outcome measures align with review objectives

Exclusion criteria:

• Clearly outside research scope
• Wrong study type (e.g., looking for empirical research but paper is opinion/editorial)
• Duplicate publications
• Conference abstracts without full papers (unless specified in protocol)

Process:

• Read title first (quick elimination of obviously irrelevant papers)
• Read abstract to verify relevance
• When uncertain, include rather than exclude (error on side of inclusion)
• Use systematic approach: every paper evaluated by same criteria
• Document decisions: Include, Exclude, or Uncertain

Expected results:

• Broad searches: typically 10-30% of papers pass initial screening
• Refined searches: typically 30-60% pass
• If <5% pass, search strategy may be too narrow or inclusion criteria too strict

Tracking:

Database: PubMed
Initial results: 2,847
Title screening: 1,200 eliminated (obviously off-topic)
Abstract screening: 847 eliminated (wrong study type, unclear relevance)
Potentially relevant: 800 papers
Pass rate: 28%

Phase 3: Iterative Search Refinement

Step 7: Concept and Keyword Extraction

From the initially relevant papers (Phase 2), identify:

New keywords and concepts:

• Terminology variations used by different research groups
• Specific methodologies mentioned repeatedly
• Technical terms and specialized vocabulary
• Author names appearing frequently
• Key research institutions/groups
• Specific journal names publishing in this area

Analysis method:

• Read abstracts and introductions of included papers
• Note any keywords not in original search strategy
• Identify methodological approaches (e.g., specific algorithms, experimental designs)
• Document emerging themes and subtopics

Example extraction:

Original search: "machine learning" + "medical imaging"
Extracted concepts:
- Specific methodologies: convolutional neural networks, attention mechanisms, transformer models
- Medical applications: radiotherapy planning, pathology analysis, diagnostic support
- Technical terms: semi-supervised learning, transfer learning, domain adaptation
- Related areas: data augmentation, interpretability, fairness in AI
- Key authors: [list of frequently appearing researchers]
- Important journals: IEEE TMI, Medical Image Analysis, Nature Medicine

Step 8: Citation Mining and Author Tracking

Follow reference trails: 1. Review reference lists of included papers 2. Identify papers cited multiple times (signal importance) 3. Retrieve and evaluate highly-cited references 4. Look for foundational/seminal papers in field

Author tracking: 1. Identify key researchers with multiple papers in review 2. Search for recent publications by these authors 3. Review their recent works not captured in initial searches 4. Check co-authors for related research

Citation tracking (forward citation searching): 1. Use Web of Science, Scopus, or Google Scholar 2. Find papers that cite key included papers 3. Evaluate citing papers for relevance 4. Identify recent developments building on earlier work

Process:

Key paper: "Deep Learning for Medical Image Analysis" (Smith et al., 2020)
Citation count: 1,247 (as of Jan 2025)
Recent citing papers (2023-2025): 342
Evaluate: Random sample or all, depending on review scope
References cited: 198 papers
New papers identified: 23 additional relevant papers

Step 9: Targeted Methodology-Based Searches

Based on identified methodologies, conduct focused searches:

Examples:

• Original: "machine learning" + "medical imaging"
• Refined searches:
• "convolutional neural networks" + "radiology"
• "attention mechanisms" + "medical images"
• "transfer learning" + "diagnostic imaging"
• "federated learning" + "clinical data"

Process: 1. Take each identified methodology 2. Combine with population/domain from original question 3. Execute new searches in selected databases 4. Screen results using same inclusion/exclusion criteria 5. Integrate new papers into literature base

Step 10: Gap Identification and Targeted Search

Identify underexplored areas:

Questions to investigate:

• "What specific applications are missing?"
• "Are there contradictory findings in any area?"
• "What methodological approaches haven't been compared?"
• "What populations/domains are underrepresented?"
• "What time periods show publication gaps?"

Targeted searches:

• Searches designed to find papers on identified gaps
• May use different search strategies (e.g., seeking negative results, null findings)
• Searches for specific applications or populations
• Temporal searches (recent developments)

Convergence assessment:

• Monitor if each new search finds mostly previously identified papers
• Track percentage of truly new papers with each search iteration
• Indicate saturation when <10-15% of results are new

Step 11: Temporal Analysis

Track research evolution: 1. Group papers by publication year 2. Identify trends in methodology adoption 3. Note shifts in research focus or populations studied 4. Identify recently emerging topics

Visualization:

2015: 15 papers (foundational period)
2016: 23 papers (growth phase)
2017: 45 papers (expansion)
2018: 89 papers (mainstream adoption)
2019: 156 papers (rapid growth)
2020: 289 papers (explosion due to...)
2021: 312 papers (continued growth)
2022: 298 papers (plateau)
2023: 276 papers (slight decline in pure methodology papers)
2024: 214 papers (applications and variants increasing)
2025: 127 papers (partial year, trend toward...)

Phase 4: Full-Text Review and Data Extraction

Step 12: Full-Text Screening

Level 2: Detailed Full-Text Review

For all papers passing abstract screening:

1. Obtain full text (use university library, ResearchGate, contact authors) 2. Read complete paper carefully 3. Apply detailed inclusion/exclusion criteria 4. Extract structured data 5. Assess quality

Detailed screening process:

• Read introduction for research context and questions
• Review methodology for study design and quality
• Examine results for outcome measures
• Assess conclusions for validity and scope
• Document reasons for exclusion if not included

Tracking results:

Papers for full-text review: 800
Full text obtained: 775 (97%)
Unable to obtain: 25 (contact authors or wait)
After full-text screening: 312 included
Excluded (with reasons):
  - Wrong study design: 156
  - No relevant outcomes: 189
  - No original data: 78
  - Data quality concerns: 40
  - Duplicate/updated version: 22

Step 13: Structured Data Extraction

Create standardized extraction form for consistent data collection:

Bibliographic information:

• Authors, year, journal, volume/issue, pages, DOI
• Publication type (original research, review, methodological)
• Journal impact factor (if relevant)

Study characteristics:

• Study design/methodology
• Population/scope (sample size, characteristics, organisms/systems studied)
• Intervention/methodology tested
• Comparison groups (if applicable)
• Study duration and setting

Results and findings:

• Primary outcomes reported
• Quantitative results (effect sizes, p-values, confidence intervals)
• Qualitative findings (themes, patterns)
• Statistical analysis methods used
• Quality metrics and limitations acknowledged

Quality assessment:

• Design quality (randomization, blinding, sample size calculation)
• Data quality (missing data, response rates, measurement validity)
• Bias assessment (selection bias, performance bias, reporting bias)
• Generalizability/applicability
• Reproducibility assessment

Thematic coding:

• Primary research theme(s)
• Secondary themes
• Methodological approach(es)
• Key contributions
• Relevance to review objectives

Extraction format:

---
Authors: Smith, J., Jones, M.
Year: 2023
Title: [Full title]
Journal: Nature Medicine
Impact Factor: 73.5
DOI: 10.1038/xxxxx

Study Design: Randomized controlled trial
Population: Adult patients with diagnosis [X], n=2,847
Intervention: Treatment A (n=1,424)
Comparison: Standard treatment (n=1,423)
Primary Outcome: Reduction in symptom severity
Results: 45% reduction (95% CI: 38-52%) vs 18% (95% CI: 12-24%), p<0.001
Effect Size: Cohen's d = 1.2 (large effect)

Quality: High (GRADE rating: A)
Key Limitations: Single-center study, limited demographic diversity
Reproducibility: Good (methods detailed, data availability statement: Yes)

Themes: Treatment efficacy, Clinical outcomes, Comparative effectiveness
Innovation Level: Incremental (application of known methodology to new population)

---

Phase 5: Synthesis and Analysis

Step 14: Thematic Organization

Group papers by key themes and organize findings:

Organizational approaches:

1. By Methodology:

• Group papers using similar approaches
• Compare methodological strengths/limitations
• Track methodology evolution over time

2. By Research Question/Aspect:

• Organize around components of research question (PICO elements)
• Compare how different studies address each component
• Integrate findings to answer overall question

3. By Chronological Development:

• Show how ideas evolved
• Track technological/methodological improvements
• Highlight paradigm shifts

4. By Research Group/Tradition:

• Organize by major research groups or schools of thought
• Compare different theoretical frameworks
• Note collaborations and conflicts

5. By Population/Application Domain:

• Different populations studied
• Different application contexts
• Geographic or demographic patterns

Example thematic map:

Main Theme: Applications of Deep Learning in Medical Imaging

Subtopic 1: Cancer Diagnosis
├─ Breast cancer detection
│  ├─ Mammography analysis (45 papers)
│  ├─ Ultrasound (12 papers)
│  └─ MRI analysis (8 papers)
├─ Lung cancer detection (34 papers)
├─ Colorectal cancer (18 papers)
└─ Other cancers (22 papers)

Subtopic 2: Treatment Planning
├─ Radiotherapy planning (28 papers)
├─ Surgical guidance (15 papers)
└─ Patient monitoring (8 papers)

Subtopic 3: Risk Stratification
├─ Prognostic models (31 papers)
└─ Predictive biomarkers (12 papers)

Step 15: Quality Assessment and Bias Detection

Quality assessment methodology:

For each research domain, use established quality assessment tools:

• Quantitative studies: GRADE, Cochrane Risk of Bias, ROBINS-I
• Qualitative studies: CASP Qualitative Checklist, STROBE-Q
• Methodological studies: STROBE, reporting checklists
• Model/Algorithm papers: Methodological rigor assessment

Bias detection:

1. Publication bias

• Papers showing positive results more likely to be published
• Searches for unpublished/negative studies
• Statistical tests: funnel plots, Egger's test
• Gray literature search

2. Selection bias

• Populations studied may not be representative
• Note demographic patterns in papers
• Identify missing populations/applications
• Track over/underrepresented areas

3. Methodological bias

• Study design limitations
• Sample size and power considerations
• Measurement validity
• Control for confounders

4. Reporting bias

• Selective outcome reporting
• Favorable vs. unfavorable results emphasis
• Statistical significance bias (p-hacking)
• Effect size reporting consistency

Quality summary:

Quality Assessment Results (80 papers):
High quality (GRADE: A): 12 papers (15%)
Good quality (GRADE: B): 28 papers (35%)
Fair quality (GRADE: C): 32 papers (40%)
Poor quality (GRADE: D): 8 papers (10%)

Bias Assessment:
- Publication bias: Moderate (positive results overrepresented)
- Selection bias: Low (diverse populations generally well-represented)
- Methodological bias: Moderate (sample size, blinding concerns)
- Reporting bias: Moderate (selective outcome reporting in 40% of papers)

Key limitation: Predominantly English-language journals (may miss non-English research)

Step 16: Evidence Synthesis

Create integrated summary of evidence:

Types of synthesis:

1. Narrative synthesis

• Written summary of findings organized thematically
• Discussion of agreement/disagreement between studies
• Integration of quantitative and qualitative findings
• Assessment of strength of evidence

2. Meta-analysis (if appropriate)

• Statistical combination of comparable results
• Analysis by subgroup (methodology, population, etc.)
• Assessment of heterogeneity
• Sensitivity analysis

3. Framework synthesis

• Organize findings using conceptual framework
• Map concepts and relationships
• Show how different findings interconnect
• Identify patterns and principles

4. Critical interpretive synthesis

• Deep interpretation of findings
• Identify underlying assumptions and interpretations
• Reconcile contradictory findings
• Build new understanding through integration

Synthesis questions to address:

• What do we know (consensus findings)?
• What do we NOT know (gaps)?
• What conflicts exist (controversial or contradictory findings)?
• What are implications for practice/future research?
• What quality/strength of evidence supports conclusions?
• What remains uncertain or debated?

Evidence summary table example:

| Topic | Consensus Finding | Evidence Strength | Conflicting Evidence | Notes |
|-------|------------------|-------------------|---------------------|-------|
| Effectiveness | 70-85% success rate | Strong (24 RCTs) | Effectiveness varies by subtype (40-90%) | Context-dependent |
| Mechanism | Works via pathway X | Moderate (12 studies) | Some evidence for pathway Y (3 studies) | Need more mechanistic work |
| Best practice | Method A superior | Moderate (8 trials) | Method B equivalent in 2 trials | Population and context matter |
| Safety | Well-tolerated overall | Strong (1,200+ patients) | 5-10% adverse events in vulnerable populations | More data needed on elderly |

Step 17: Identify Gaps and Future Directions

Research gaps identified through review:

1. Knowledge gaps

• Questions not yet addressed by research
• Populations/contexts not yet studied
• Outcomes not yet measured
• Mechanisms not yet understood

2. Methodological gaps

• Inadequate study designs for certain questions
• Lack of comparative studies
• Gaps in measurement approaches
• Need for larger/longer studies

3. Application gaps

• Research not translated to practice
• Implementation challenges not addressed
• Real-world effectiveness not studied
• Access/equity issues underexplored

4. Conflict/uncertainty areas

• Contradictory findings needing resolution
• Debated interpretations
• Emerging evidence not yet synthesized

Gap identification process:

• List what questions remain unanswered
• Note populations not well-represented in literature
• Identify methodological approaches not yet compared
• Document inconsistent findings
• Highlight emerging/underexplored topics

Future research recommendations:

Priority gaps identified:

1. HIGH PRIORITY - Unaddressed populations
   Gap: Limited research on [specific population]
   Current evidence: [brief summary]
   Recommended study: Prospective cohort study in [population]
   Rationale: [explanation]

2. HIGH PRIORITY - Methodological comparison
   Gap: No direct comparison of Method A vs. Method B
   Current evidence: [separate evidence for each]
   Recommended study: Randomized comparison trial
   Rationale: [explanation]

3. MEDIUM PRIORITY - Mechanism clarification
   Gap: Exact mechanism of action unclear
   Current evidence: [partial evidence]
   Recommended study: Mechanistic studies using [approach]
   Rationale: [explanation]

4. EMERGING AREA - Novel applications
   Gap: New applications identified but not yet studied
   Current evidence: [anecdotal/preliminary evidence]
   Recommended study: Feasibility study
   Rationale: [explanation]

Phase 6: Report Writing and Organization

Step 18: Report Structure and Organization

For Narrative Literature Review:

1. Executive Summary (1 page)

• Research scope and objectives
• Key findings and consensus
• Major gaps and recommendations

2. Introduction (2-3 pages)

• Problem statement and significance
• Research questions/objectives
• Scope and relevance

3. Methods (1-2 pages)

• Search strategy and databases
• Inclusion/exclusion criteria
• Data extraction and analysis approach
• Study quality assessment method

4. Results (variable, typically 5-10 pages)

• Literature search and selection process (with flow diagram)
• Characteristics of included studies
• Study quality and bias assessment
• Findings organized thematically

5. Thematic Sections (variable, typically 8-20 pages)

• Current state of knowledge on each theme
• Key methodologies and approaches
• Major findings and areas of consensus
• Conflicting results and areas of debate
• Recent developments and emerging trends

6. Synthesis and Discussion (3-5 pages)

• Integration of findings across themes
• Identification of patterns, trends, and paradigm shifts
• Assessment of research quality and reliability
• Critical analysis of methodological approaches
• Implications for theory, practice, and future research

7. Gaps and Future Directions (1-2 pages)

• Identified research gaps
• Methodological limitations
• Suggested research priorities
• Barriers to translation/implementation (if applicable)

8. Conclusions (1 page)

• Summary of key insights
• Implications for research and practice
• Recommendations
• Final synthesis statement

9. References (formatted bibliography)

• Complete citation information
• Organized by theme (optional)
• Hyperlinks to DOIs (if digital format)

For Systematic Literature Review:

Follow PRISMA 2020 guidelines with additional elements:

• Inclusion/exclusion criteria fully documented
• Detailed search strategy for every database
• PRISMA flow diagram
• Meta-analysis (if appropriate)
• Risk of bias assessment for each study
• Certainty of evidence assessment (GRADE)
• Extended appendices with tables of study characteristics

Step 19: Citation Management

Citation format selection:

Choose appropriate format for your discipline:

• APA: Social sciences, psychology, education
• Chicago/Notes-Bibliography: History, humanities
• IEEE: Engineering, computer science
• Nature: Natural sciences, biology
• Science: Scientific research
• MLA: Literature, humanities
• OSCOLA: Law

In-text citation approaches:

• Numbered system: (1), (2), (3)...
• Author-date system: (Smith, 2023)
• Footnote system: Superscript numbers with notes

Reference list organization:

• Alphabetical (typical)
• By thematic section
• By methodological approach
• By publication date

Citation accuracy checklist:

• All cited papers listed in references
• Reference information complete and accurate
• Formatting consistent throughout
• URLs and DOIs functional
• Access dates included (for websites, if required)

Step 20: Report Quality Assurance

Logical flow and coherence:

• Each section connects to research question
• Transitions between sections smooth
• Overall narrative logical and clear
• Conclusions follow from evidence

Balanced perspective:

• All major viewpoints represented
• Contradictory findings acknowledged
• Limitations transparently discussed
• No over-emphasis of favored interpretations

Appropriate detail level:

• Key studies discussed in detail
• Minor studies appropriately summarized
• Methods described at level appropriate for audience
• Results clearly explained without unnecessary statistics

Clear distinctions:

• Established facts vs. interpretations clearly marked
• Author's analysis vs. findings from reviewed literature clear
• Consensus vs. minority views distinguished
• Evidence strength indicated

Phase 7: Output and Dissemination

Step 21: Multiple Output Formats

Markdown format (for collaborative editing, version control)

# Literature Review: [Title]

## Executive Summary
...

## Introduction
...

## Methods
...

## Results
...

LaTeX format (for academic submission, journal publication)

\documentclass{article}
\usepackage{natbib}

\title{Literature Review: ...}
\author{...}

\begin{document}

\section{Executive Summary}
...

\end{document}

Word document (for institutional requirements, collaborative editing)

• Export from markdown/LaTeX
• Format with institutional templates
• Track changes for feedback

HTML for web publication

• Interactive tables of contents
• Hyperlinked references
• Searchable content
• Figure galleries

Step 22: Citation Data Management

Export formats:

• BibTeX (.bib): For LaTeX documents
• RIS (.ris): For reference managers
• CSL JSON: For citation processing
• CSV: For data analysis and spreadsheets

Citation database integration:

• Zotero: Open-source, browser integration
• Mendeley: Commercial, cloud-based
• EndNote: Enterprise solution
• RefWorks: Cloud-based institutional solution

Search history documentation:

• All databases searched
• Queries used
• Date ranges
• Number of results
• Refinements made
• Rationale for changes

Best Practices for Literature Reviews

Search Strategy Best Practices

1. Start broad, then refine

• Initial searches cast wide net
• Analyze results to identify concepts
• Refine searches based on learning
• Converge toward saturation

2. Document everything

• Record every search executed
• Note parameters and dates
• Keep search logs
• Enable reproducibility and auditing

3. Use multiple strategies

• Boolean combinations
• Citation mining
• Author searching
• Keyword and phrase variations

4. Cross-database verification

• Important papers should appear in multiple databases
• Compare top papers across databases
• Use cross-database differences to identify missed areas

5. Combine automated and manual searching

• Automated searches for breadth
• Manual browsing of key journals
• Citation tracking for depth
• Expert consultation for validation

Content Analysis Best Practices

1. Structured extraction

• Use standardized forms
• Consistency checks
• Double-checking of key papers
• Inter-rater reliability assessment (if team effort)

2. Quality assessment

• Use published, validated tools
• Assess all papers using same criteria
• Document quality concerns
• Weight evidence by quality

3. Bias awareness

• Acknowledge reviewer biases
• Explicit inclusion/exclusion criteria
• Sensitivity analyses
• Transparent decision-making

4. Contextual understanding

• Understand publication context
• Note journal prestige and impact
• Recognize field-specific practices
• Account for disciplinary differences

5. Accuracy and verification

• Double-check data extraction
• Verify direct quotes
• Confirm effect sizes and statistical values
• Cross-reference key claims

Synthesis Best Practices

1. Systematic organization

• Use consistent frameworks
• Organize thematically for narrative clarity
• Show relationships between studies
• Map evidence gaps visually

2. Avoid cherry-picking

• Include all relevant papers, not just supporting ones
• Acknowledge contradictions
• Give weight to quality of evidence
• Report effect size ranges

3. Integrate multiple types of evidence

• Combine quantitative and qualitative findings
• Synthesize across methodological approaches
• Build comprehensive understanding
• Identify convergent and divergent findings

4. Make evidence strength explicit

• Use GRADE or similar system
• Distinguish high-quality from preliminary evidence
• Acknowledge uncertainty
• Indicate confidence in conclusions

5. Address conflicting evidence

• Describe conflicting findings fairly
• Investigate sources of disagreement
• Propose explanations for conflicts
• Note when evidence is inconclusive

Common Mistakes to Avoid

1. Inadequate search strategy

• Too narrow, missing relevant papers
• Too broad, overwhelming results
• Missing important databases or grey literature
• Insufficient iterative refinement

2. Insufficient documentation

• Unclear how papers were selected
• Search strategy not reproducible
• Selection decisions not justified
• Preventing others from assessing quality

3. Quality assessment omission

• Treating all papers as equally valid
• Over-reliance on poor-quality studies
• Missing bias assessment
• Unweighted evidence synthesis

4. Shallow analysis

• Mere summarization instead of synthesis
• Isolated findings not integrated
• Contradictions not addressed
• Limited critical evaluation

5. Inadequate citation

• Incomplete bibliographic information
• Inconsistent formatting
• Incorrect attributions
• Missing DOIs or URLs (for web sources)

6. Bias toward recent literature

• Seminal foundational papers missed
• Publication bias not addressed
• Overweight to recent trends
• Missing historical context

7. Ignoring methodological variation

• Papers using different methodologies treated as directly comparable
• Methodology-specific findings not noted
• Invalid meta-analyses (combining incompatible studies)
• Missed insights from methodological diversity

Tools and Resources

Search and Management Tools

Reference Management:

• Zotero (zotero.org) - Free, open-source
• Mendeley (mendeley.com) - Free basic version
• RefWorks - Institutional access
• Papers (papersapp.com) - PDF-focused

Search Tools:

• PubMed Central: pubmed.ncbi.nlm.nih.gov
• arXiv: arxiv.org
• Web of Science: webofscience.com
• Scopus: scopus.com
• Google Scholar: scholar.google.com

Citation Tools:

• CrossRef (crossref.org) - DOI lookup and verification
• Unpaywall (unpaywall.org) - Open access article locator

Data Extraction and Organization

Spreadsheets and Databases:

• Excel: Simple, widely available
• Google Sheets: Collaborative, cloud-based
• Airtable: Database functionality, templates
• Covidence: Systematic review platform
• DistillerSR: Systematic review software

Synthesis and Visualization

Concept Mapping:

• CmapTools: Free concept mapping
• MindMeister: Collaborative mind mapping
• VosViewer: Citation network visualization
• Gephi: Network analysis and visualization

Statistical Analysis:

• R: Advanced analysis and visualization
• Python: Data analysis and synthesis
• STATA: Statistical analysis
• RevMan: Systematic review meta-analysis

Integration with Other Skills

This skill integrates well with:

• scientific-writing: For writing the literature review sections of papers
• eln: For documenting literature review process in electronic lab notebook
• planning: For project planning and tracking literature review progress
• troubleshooting: For debugging searches and refining strategy when facing challenges
• phd-qualifier: For comprehensive literature reviews needed for exams

Response Patterns

When conducting literature reviews, I will:

1. Clarify scope and research questions - Ensure clear, specific research questions using PICO/PEO framework 2. Propose search strategy - Recommend appropriate databases and search queries 3. Guide iterative refinement - Suggest ways to refine searches based on initial results 4. Suggest organizational frameworks - Propose thematic organization structures 5. Help with synthesis - Guide integration of findings and identification of gaps 6. Provide quality assessment - Apply relevant quality assessment tools 7. Support report writing - Help structure and write literature review sections 8. Document comprehensively - Ensure all decisions and searches are documented

I will ask clarifying questions about:

• Research domain and field
• Intended scope (comprehensive vs. focused)
• Time constraints and available resources
• Target audience for the review
• Specific research question(s)
• Preferred output format
• Publication requirements (PRISMA compliance, specific journal guidelines, etc.)

Limitations and Considerations

1. Time and resources: Comprehensive literature reviews require significant time investment (weeks to months) 2. Database access: Some databases require institutional access or subscriptions 3. Full-text availability: Not all papers are freely available online 4. Language barriers: This skill focuses on English-language resources 5. Currency: Review methodology follows published guidelines; emerging trends in literature review methodology may not be fully captured 6. Subjective elements: Some aspects of literature review (theme identification, synthesis) involve interpretive judgment 7. Tool limitations: External tools (databases, search engines) have inherent limitations and biases

---

This comprehensive literature review skill provides a systematic, reproducible approach to synthesizing knowledge across research domains while maintaining rigor, transparency, and critical evaluation of evidence.

Citation Styles Guide

Comprehensive reference for formatting citations in common academic styles.

---

APA Style (7th Edition)

Most commonly used in social sciences, psychology, education, and many other fields.

Key Characteristics

• Author-date system (citations in parentheses in text)
• References list in alphabetical order at end
• Hanging indent for reference list entries
• Title case for journal articles, sentence case for books

APA In-Text Citations

Single Author

(Smith, 2023) or Smith (2023) noted that...

Multiple Authors

• 2 authors: (Smith & Jones, 2023) or Smith and Jones (2023)
• 3-5 authors first citation: (Smith, Jones, & Brown, 2023)
• 3-5 authors subsequent: (Smith et al., 2023)
• 6+ authors: (Smith et al., 2023) [use et al. from first citation]

Multiple Works

(Smith, 2023; Jones, 2022; Brown & Williams, 2024)

Multiple Works Same Author

(Smith, 2022, 2023) [chronological order]

Direct Quote

"Direct quote" (Smith, 2023, p. 45) [include page number]

APA Reference List Formats

Journal Article

Smith, J., & Jones, M. (2023). Title of article with lowercase except proper nouns.
    Journal Name in Title Case, 45(3), 123-145.
    https://doi.org/10.xxxx/xxxxx

Key points:

• Authors: Last name, first initial(s)
• Year in parentheses
• Title: Only first word and proper nouns capitalized
• Journal title: All major words capitalized (title case)
• Volume (issue), pages, DOI

Book

Smith, J. (2023). Title of book: Subtitle in title case. Publisher.
    https://doi.org/10.xxxx/xxxxx

Book Chapter

Smith, J. (2023). Chapter title: Lowercase except proper nouns. In M. Editor
    & R. CoEditor (Eds.), Title of book in title case (pp. 123-145).
    Publisher. https://doi.org/10.xxxx/xxxxx

Web Page

Smith, J. (2023, January 15). Title of web page.
    Retrieved from https://www.website.com/path

Or with access date if no publication date:

Smith, J. (n.d.). Title of web page. Retrieved from https://www.website.com
    [Accessed January 20, 2025]

Dissertation/Thesis

Smith, J. (2023). Title of dissertation [Doctoral dissertation, University Name].
    ProQuest Dissertations and Theses Global. https://doi.org/10.xxxx/xxxxx

Conference Paper

Smith, J. (2023, July). Title of conference paper. In Proceedings of the [Conference
    Name] (pp. 45-56). Conference Publishers.

---

Chicago Style (Author-Date System)

Common in history, humanities, and some social sciences.

Two Chicago Systems

Notes-Bibliography System (for humanities)

Uses footnotes/endnotes + bibliography

Author-Date System (for sciences/social sciences)

Similar to APA with some differences

Chicago Author-Date In-Text Citations

Basic Format

(Smith 2023) or Smith (2023)

Multiple Authors

(Smith and Jones 2023) [2 authors - use "and"]
(Smith, Jones, and Brown 2023) [3 authors - use "and" before last]
(Smith et al. 2023) [4+ authors - use et al.]

Direct Quote

(Smith 2023, 45) [page number after comma]

Chicago Reference List (Bibliography)

Journal Article

Smith, James, and Maria Jones. "Title of Article." Journal Name 45, no. 3 (2023):
    123-145. https://doi.org/10.xxxx/xxxxx

Key differences from APA:

• Author names: First name first
• Article title: In quotation marks
• Journal: All major words capitalized
• Issue: "no. 3" instead of "(3)"
• Page numbers after colon

Book

Smith, James. Title of Book: Subtitle in Title Case. Publisher, 2023.
    https://doi.org/10.xxxx/xxxxx

Book Chapter

Smith, James. "Chapter Title." In Title of Book: Subtitle in Title Case, edited by
    Maria Editor and Robert CoEditor, 123-145. Publisher, 2023.

Website

Smith, James. "Title of Web Page." Accessed January 15, 2025. https://www.website.com

---

IEEE Style

Standard in engineering and computer science.

Key Characteristics

• Numbered system [1], [2], [3]...
• References numbered in order of appearance
• No hanging indent
• Initials then surname for authors
• Abbreviated journal titles

IEEE In-Text Citations

Basic Format

The authors [1] demonstrated that...
[1] shows effectiveness of the approach.
As noted in [1, 2, 4], the methodology...

IEEE Reference List Formats

Journal Article

[1] J. Smith and M. Jones, "Title of article," Journal Abbreviation, vol. 45,
    no. 3, pp. 123–145, 2023, doi: 10.xxxx/xxxxx.

Book

[2] J. Smith, Title of Book: Subtitle. Publisher, 2023, doi: 10.xxxx/xxxxx.

Book Chapter

[3] J. Smith, "Chapter title," in Title of Book: Subtitle, M. Editor and
    R. CoEditor, Eds. Publisher, 2023, pp. 123–145.

Conference Paper

[4] J. Smith, "Title of paper," in Proceedings of [Conference Name], July 2023,
    pp. 45–56, doi: 10.xxxx/xxxxx.

Website

[5] J. Smith. (2023). Title of web page. [Online]. Available: https://www.website.com
    [Accessed: Jan. 15, 2025].

---

Nature/Science Style

For natural sciences publications.

Key Characteristics

• Numbered system [1], [2], [3]...
• Vancouver-style formatting
• References in order of appearance
• Specific abbreviations for journal names

In-Text Citations

Same as IEEE - superscript numbers [1]²³

Reference List Formats

Journal Article

Smith, J. & Jones, M. Title of article. Journal Name 45, 123–145 (2023).

Book

Smith, J. Title of Book: Subtitle. Publisher (2023).

Book Chapter

Smith, J. in Title of Book: Subtitle (eds Editor, M. & CoEditor, R.) 123–145
(Publisher, 2023).

Online Publication (Early View/Preprint)

Smith, J. & Jones, M. Title of article. Journal Name https://doi.org/10.xxxx/xxxxx
(2023).

---

MLA Style

Common in literature, humanities, and languages.

Key Characteristics

• Author-page system
• Works Cited page (reversed alphabetical)
• Title case for titles
• Parenthetical citations

MLA In-Text Citations

Basic Format

(Smith 45) [author and page number]
Smith argues that... (45).

Multiple Authors

(Smith and Jones 45)
(Smith, Jones, and Brown 45)

MLA Works Cited Format

Journal Article

Smith, James, and Maria Jones. "Title of Article." Journal Name, vol. 45,
    no. 3, 2023, pp. 123-145. DOI: 10.xxxx/xxxxx.

Book

Smith, James. Title of Book: Subtitle. Publisher, 2023.

Website

Smith, James. "Title of Web Page." Website Name, 2023, https://www.website.com.
    Accessed 15 Jan. 2025.

---

Harvard Style

Widely used in UK universities, similar to APA.

Key Characteristics

• Author-date system
• Similar to APA but with some formatting differences
• References in alphabetical order

Harvard In-Text Citations

(Smith, 2023) or Smith (2023)

Harvard Reference List

Journal Article

Smith, J. and Jones, M., 2023. Title of article. Journal Name, 45(3), pp.123-145.
doi: 10.xxxx/xxxxx

Key differences from APA:

• Uses commas instead of periods in some places
• Includes ", pp." before page numbers
• Author initials: J. with period

Book

Smith, J., 2023. Title of book: Subtitle. Publisher.

---

Quick Comparison Table

---

DOI and URL Formatting

DOI (Digital Object Identifier)

Format: https://doi.org/10.xxxx/xxxxx

In citations:

• APA: https://doi.org/10.xxxx/xxxxx
• Chicago: https://doi.org/10.xxxx/xxxxx
• IEEE: doi: 10.xxxx/xxxxx
• Nature: Include DOI (varies by publication)

Example:

DOI: 10.1038/nature12373
Full URL: https://doi.org/10.1038/nature12373

URLs

General best practices:

• Include full URL
• For works with DOI, prefer DOI over URL
• Retrieve/access dates usually not needed (unless content changes frequently)
• Remove "http://" or "https://" in some styles

---

Common Citation Mistakes to Avoid

---

Using Reference Management Software

Recommended Tools

Free:

• Zotero (zotero.org) - Open source, excellent
• Mendeley (mendeley.com) - Free tier available
• Google Scholar (scholar.google.com) - Limited formatting

Commercial:

• EndNote - Industry standard, expensive
• RefWorks - Institutional licensing common

Features to Use

• Auto-format citations in your chosen style
• Import from databases with one click
• Generate reference lists automatically
• Sync across devices
• Organize by theme or project

Workflow

1. Find article online 2. Click Zotero/Mendeley button to import 3. Extract metadata (usually automatic) 4. Organize in collection/folder 5. Generate formatted bibliography 6. Insert citations in your document

---

Citing Different Source Types

Preprints (arXiv, bioRxiv, etc.)

APA:

Smith, J., & Jones, M. (2023). Title of preprint. arXiv.
    https://arxiv.org/abs/2301.12345

IEEE:

[1] J. Smith and M. Jones, "Title of preprint," arXiv:2301.12345, 2023.

Conference Presentations (without published proceedings)

APA:

Smith, J. (2023, July). Title of presentation. Paper presented at the
    [Conference Name], [City, State/Country].

Datasets

APA:

Smith, J. (2023). Title of dataset [Data set]. Publisher or Repository.
    https://doi.org/10.xxxx/xxxxx

Software/Code

APA:

Smith, J. (2023). Software name (Version X.X) [Computer software].
    https://github.com/username/repo

---

Citation Verification Checklist

For each citation in your paper:

• [ ] Author names spelled correctly
• [ ] Year of publication accurate
• [ ] Page numbers included (for direct quotes)
• [ ] Citation appears in reference list
• [ ] Reference list entry complete
• [ ] Format consistent with style guide
• [ ] DOI included (if available)
• [ ] Title capitalization correct
• [ ] Journal/Publisher name accurate
• [ ] Volume and issue numbers correct (if applicable)

---

Resources

Official Style Guides

• APA: Publication Manual of the American Psychological Association
• Chicago: Chicago Manual of Style
• IEEE: IEEE Standard for Information Technology
• MLA: MLA Handbook
• Nature: Nature Publishing Group Editorial Policies

Online References

• Purdue OWL: https://owl.purdue.edu/ (All major styles)
• Citefast: https://www.citefast.com/ (Citation formatter)
• BibTeX Builder: https://www.bibtex.com/ (For LaTeX)
• Citation Machine: https://www.citationmachine.net/ (Various styles)

---

Use these guides to ensure accurate, consistent citation formatting throughout your literature review.

Literature Matrix Template

Use this template to organize and compare studies in your review. This spreadsheet format makes it easy to identify patterns and synthesis themes.

Literature Matrix: [Review Topic]

Template Guide

Column Definitions

Author(s), Year: Full citation format. Link to DOI if possible.

Study Design:

• RCT = Randomized controlled trial
• Quasi-RCT = Quasi-randomized trial
• Cohort = Prospective or retrospective cohort
• Case-control = Case-control design
• Case series = Multiple case reports
• Cross-sectional = Observational snapshot
• Qualitative = Interview/observation-based

N: Total sample size (or number of qualitative participants/interviews)

Population:

• Age range
• Disease/condition characteristics
• Key demographic characteristics
• Setting (hospital, community, virtual, etc.)
• Geographic location if relevant

Intervention:

• Specific treatment/method tested
• Duration/intensity/dose
• Format (individual/group/online/etc.)
• Key characteristics

Comparison:

• What intervention(s) compared against
• If none: "None (pre-post)" or "None (descriptive)"
• Alternative treatments or standard care

Primary Outcome:

• Main outcome measure
• How measured/assessed
• Timepoint if relevant

Effect Size:

• Quantitative result with CI
• Statistical significance (p-value)
• Direction of effect (beneficial/harmful/null)
• Format: Effect (95% CI), p-value

Quality:

• GRADE level: A (high), B (good), C (fair), D (poor)
• Or: Low/Some/High risk of bias
• Brief note of main quality concern

Key Findings:

• 1-2 sentence summary of main results
• Notable patterns or surprises
• Limitations acknowledged by authors

Themes:

• Primary research theme(s)
• Secondary themes if relevant
• Methodological approach if notable

Extended Matrix Version

For larger reviews or more detailed analysis, expand columns:

Additional Columns

Baseline Comparability: Yes/No/Partial - Were groups similar at baseline?

Main Results: More detailed results/effect sizes

Secondary Outcomes: Other measured outcomes and results

Adherence Rate: Dropout/loss to follow-up percentage

Publication Bias Risk: Low/Some/High risk based on study characteristics

Overall Risk of Bias: Summary assessment across all bias domains

GRADE: Overall quality rating with rationale

Notes: Important contextual details, conflicts of interest, funding source, unique contributions

Thematic Organization Matrix

Alternative organization: Group papers by research themes

Example: Deep Learning in Medical Imaging

Data Summary Table

By Study Quality

Temporal Trend Table

By Publication Year

Geographic/Demographic Coverage Table

Distribution of Studies

Creating Your Literature Matrix

Step 1: Set Up Spreadsheet

• Use Excel, Google Sheets, or Airtable
• Start with basic template (7-10 columns)
• Expand as needed for your review

Step 2: Populate from Reading

• Read each paper's abstract and introduction
• Extract key information into matrix
• Continue updating as you read full paper

Step 3: Sort and Filter

• Sort by quality to identify best evidence
• Filter by outcome to compare results
• Group by theme for synthesis

Step 4: Analyze Patterns

• Look for effect size patterns
• Identify methodological variations
• Note populations underrepresented
• Spot conflicting findings

Step 5: Use for Writing

• Reference matrix while writing synthesis
• Matrix serves as fact-check for statements
• Use for creating summary tables in report

Tips for Effective Matrix

1. Keep it organized: Consistent formatting, complete entries 2. Use consistent terminology: Same labels across rows 3. Include quality assessment: Weight evidence by quality 4. Add hyperlinks: Link to DOI or PDF for easy reference 5. Use conditional formatting: Color-code by quality, effect size, theme 6. Update regularly: Maintain as you add/remove papers 7. Create submatrices: Separate matrices for different themes/outcomes 8. Document decisions: Notes section for inclusion/exclusion rationale

Example: Using Matrix for Synthesis

Identifying Consensus

Matrix review shows:
- 35/40 studies (87%) report positive effect
- Effect size range: d = 0.3-1.2
- Quality doesn't predict effect size (Grade A and D show similar effects)
- Trend over time: increasing effect sizes in recent studies

Interpretation:
- Strong consensus for effectiveness
- Consistent across methodologies
- Improving methodology over time suggests real effect

Identifying Gaps

Matrix review shows:
- Geographic: 80% of studies from USA/Europe
- Population: Only 3 studies in [specific population]
- Outcome: Primary outcome well-covered (38/40)
           but safety outcomes limited (12/40)
- Timeline: Follow-up mostly <6 months (28/40)

Identified gaps:
- Need research in underrepresented regions/populations
- Need longer-term follow-up studies
- Need comprehensive safety assessment

Identifying Heterogeneity

Matrix analysis reveals:
- By methodology: Traditional ML (d=0.4) vs. DL (d=0.8)
  → Different methods produce different results
- By population: Mild cases (d=0.3) vs. Severe cases (d=0.9)
  → Effect size depends on baseline severity
- By outcome: Objective measures (d=0.7) vs. Self-report (d=0.5)
  → Measurement type affects effect size

Explanation for heterogeneity:
- Methodological differences are substantial
- Study populations not comparable
- Different outcome measures capture different constructs

---

For data extraction guidance, see references/data-extraction.md. For synthesis approaches, see references/synthesis-methods.md.

Literature Review Search Log Template

Document all searches performed to ensure reproducibility and completeness.

Review Information

---

Individual Search Records

Search #1: [Date - Database]

Search #2: [Date - Database]

Search #3: [Citation Tracking]

Search #4: [Author Tracking]

Search #5: [Targeted Methodological Search]

---

Search Strategy Summary

Databases Searched

1. ☐ PubMed 2. ☐ arXiv 3. ☐ Web of Science 4. ☐ Scopus 5. ☐ IEEE Xplore 6. ☐ ACM Digital Library 7. ☐ Google Scholar 8. ☐ [Other: _____________]

Search Methods Used

1. ☐ Controlled vocabulary (MeSH terms, etc.) 2. ☐ Boolean operators (AND, OR, NOT) 3. ☐ Citation tracking 4. ☐ Author/researcher tracking 5. ☐ Hand searching key journals 6. ☐ Grey literature search (theses, conference papers) 7. ☐ Contact with experts/authors

Search Refinement Decisions

---

Final Search Summary

Search Completion Status

Overall Search Results

Search Convergence Assessment

Convergence Status: ☐ Not converged ☐ Approaching convergence ☐ Converged

Evidence for convergence:

• Most recent searches finding [X]% new relevant papers
• New searches predominantly returning previously identified papers
• Theoretical saturation reached in [themes]
• Diminishing returns evident after search [#]

Plan:

• ☐ Continue comprehensive searching
• ☐ Conduct targeted searches for remaining gaps
• ☐ Proceed to data extraction phase
• ☐ Return to searching if gaps identified in analysis phase

---

Reproducibility Checklist

• ☐ All databases and search dates documented
• ☐ Exact search queries recorded word-for-word
• ☐ Search parameters (date ranges, filters) noted
• ☐ Number of results and pass rates documented
• ☐ All search decisions and rationale explained
• ☐ New concepts extracted and documented
• ☐ Follow-up searches based on findings recorded
• ☐ Key papers and citations tracked
• ☐ Authors and institutions tracked
• ☐ Final search strategy documented for reproducibility

---

Narrative Summary

Search Process Overview

[Provide 2-3 paragraph summary of overall search strategy and approach]

Key Findings from Search

[Describe main concepts identified, methodological trends, research evolution]

Challenges and Solutions

[Describe any search challenges and how they were addressed]

Rationale for Search Boundaries

[Explain decisions about scope, databases, time periods, languages]

Confidence in Search Completeness

Rate confidence in having found most/all relevant literature:

Confidence level: ☐ Very high ☐ High ☐ Moderate ☐ Low

Rationale:

• [Reason for confidence level]
• [Any potential gaps in coverage]
• [Limitations of search strategy]

---

This search log will be included as supplementary material with final review report to ensure transparency and reproducibility.

Literature Review Synthesis Outline Template

Use this template to organize and structure your synthesis of findings.

---

Review Information

Title: [Your review title]

Research Question(s): [Primary question and subquestions]

Review Type: ☐ Narrative ☐ Systematic ☐ Scoping ☐ Meta-analysis

Number of Papers Included: [Total N]

Date of Synthesis: [Date]

---

Executive Summary

Brief Research Context

[1-2 sentences explaining why this review is important]

Research Question

[Clear statement of what this review addresses]

Key Findings (2-3 bullet points)

• Finding 1: [Summary with support - e.g., "70% of studies show X (n=35 studies)"]
• Finding 2: [Summary with support]
• Finding 3: [Summary with support]

Research Gaps Identified

• Gap 1: [What is not known]
• Gap 2: [What is not known]
• Gap 3: [What is not known]

Recommendations

• Recommendation 1: [For practice/research]
• Recommendation 2: [For practice/research]

---

Thematic Organization

Main Themes Identified

From the [N] papers included, the following [X] main themes emerged:

1. Theme 1: [Main Concept] (n=[papers], [%] of review) 2. Theme 2: [Main Concept] (n=[papers], [%] of review) 3. Theme 3: [Main Concept] (n=[papers], [%] of review) 4. Theme 4: [Main Concept] (n=[papers], [%] of review)

---

Detailed Synthesis by Theme

THEME 1: [Main Concept]

Definition

[Clear definition of this theme and what it encompasses]

Subtopics

• Subtopic 1.1: [Specific aspect]
• Subtopic 1.2: [Specific aspect]
• Subtopic 1.3: [Specific aspect]

Key Findings

1.1 [Subtopic 1] (n=[papers])

[2-3 sentences summarizing findings on this subtopic, with supporting evidence]

Example structure:

• Consensus finding: [What most papers agree on - "12 of 15 papers (80%) reported..."]
• Effect size: [If quantitative - "Effect sizes ranged from Cohen's d = 0.3 to 0.9"]
• Consistency: [Are findings consistent? "Results consistent across methodologies, populations, and time periods"]
• Notable studies: [Name key exemplary papers if particularly important]

Citation 1 (Smith et al., 2023): [Key finding with effect size]
Citation 2 (Jones & Brown, 2022): [Key finding supporting/contradicting]
Citation 3 (Williams et al., 2023): [Additional supporting evidence]

1.2 [Subtopic 2] (n=[papers])

[Similar structure as above]

1.3 [Subtopic 3] (n=[papers])

[Similar structure as above]

Conflicting Evidence

[If present, describe contradictory findings]

Sources of conflict:

• Difference 1: [Explain why findings differ]
• Difference 2: [Explain why findings differ]

Resolution: [How should conflicting evidence be interpreted?]

Quality Assessment for This Theme

• High-quality evidence: n=[X] papers
• Good-quality evidence: n=[X] papers
• Fair-quality evidence: n=[X] papers
• Poor-quality evidence: n=[X] papers

Overall confidence in findings: ☐ High ☐ Moderate ☐ Low

---

THEME 2: [Main Concept]

Definition

[Clear definition]

Subtopics

• Subtopic 2.1: [Specific aspect]
• Subtopic 2.2: [Specific aspect]

Key Findings

2.1 [Subtopic 1] (n=[papers])

[Findings and evidence]

2.2 [Subtopic 2] (n=[papers])

[Findings and evidence]

Conflicting Evidence

[If present]

Quality Assessment for This Theme

[Quality summary]

---

THEME 3: [Main Concept]

[Same structure as Themes 1-2]

---

THEME 4: [Main Concept]

[Same structure as Themes 1-2]

---

Cross-Theme Analysis

Relationships Between Themes

How themes connect:

1. Connection 1: [Theme X relates to Theme Y through...] 2. Connection 2: [Theme A influences Theme B by...] 3. Connection 3: [Integration of themes...]

Integrated model/framework:

[If applicable, describe how themes fit together into overall understanding]

Theme 1 ──→ Theme 2 ──→ Theme 3
   ↓              ↓
   └─────→ Theme 4 ←─────┘

[Mechanism pathway diagram explaining how themes relate]

Methodological Patterns

Study design influences on outcomes:

• RCTs (n=[X]): Results show [pattern]
• Observational (n=[X]): Results show [pattern]
• Qualitative (n=[X]): Results show [pattern]

Interpretation: [Do design differences explain outcome differences?]

Population characteristics influencing outcomes:

• [Population 1] (n=[X]): Results show [effect]
• [Population 2] (n=[X]): Results show [effect]

Interpretation: [How do populations differ in outcomes?]

Temporal trends:

• Pre-2015 (n=[X]): [Research focus/methods]
• 2015-2020 (n=[X]): [Research focus/methods]
• 2020+ (n=[X]): [Research focus/methods]

Interpretation: [How has research evolved?]

---

Heterogeneity Analysis

Sources of Variation in Findings

Design heterogeneity:

• Different study designs produced [similar/different] results
• [Specific comparison]

Population heterogeneity:

• Effects varied by [characteristic]
• [Specific variation noted]

Methodological heterogeneity:

• Different approaches showed [range of results]
• [Specific patterns]

Geographic/contextual heterogeneity:

• [Geographic variation]
• [Contextual differences]

Is heterogeneity problematic?

• ☐ No (results homogeneous, safe to pool/integrate)
• ☐ Partially (some variation but explained by identifiable factors)
• ☐ Yes (substantial heterogeneity limits integration)

Explanation: [Why heterogeneity exists and implications]

---

Bias Assessment

Publication Bias

[Is there evidence that positive results are overrepresented?]

Assessment:

• Evidence: [Funnel plot asymmetry, positive result prevalence, etc.]
• Conclusion: ☐ No bias ☐ Some bias ☐ Substantial bias
• Impact: [How might bias affect conclusions?]

Selection Bias

[Are included papers systematically different from excluded papers?]

Assessment:

• Geographic bias: [Geographic regions underrepresented?]
• Population bias: [Specific populations underrepresented?]
• Language bias: [Non-English literature missing?]
• Conclusion: ☐ No bias ☐ Some bias ☐ Substantial bias

Other Biases

[Author bias, methodological bias, reporting bias, etc.]

---

Research Gaps and Future Directions

Identified Gaps

Knowledge gaps

Gap 1: [Question not yet answered]

• Current evidence: [What we know]
• Missing evidence: [What's unknown]
• Research recommendation: [What's needed]
• Rationale: [Why important]

Gap 2: [Question not yet answered]

• Current evidence: [What we know]
• Missing evidence: [What's unknown]
• Research recommendation: [What's needed]
• Rationale: [Why important]

Gap 3: [Question not yet answered] [Same structure]

Methodological gaps

Gap 1: [Methodological limitation]

• Current approach: [How it's been studied]
• Needed approach: [How it should be studied]
• Benefit: [What would be gained]

Population gaps

Gap 1: [Underrepresented population]

• Current coverage: [What's been studied]
• Missing coverage: [What's not studied]
• Importance: [Why this population matters]

Application/implementation gaps

Gap 1: [Translation to practice/application]

• Current status: [Extent of implementation]
• Barriers: [Why not implemented]
• Solutions: [How to improve]

Future Research Priorities

Priority 1 (Highest priority)

Gap addressed: [Gap #] Recommended study type: [RCT, observational, qualitative, implementation, etc.] Study characteristics:

• Population: [Who to study]
• Intervention: [What to test]
• Outcomes: [What to measure]
• Duration: [How long]
• Sample size**: [Estimated N needed]

Rationale: [Why this is most important] Timeline: [Suggested timeline]

Priority 2 (High priority)

[Same structure]

Priority 3 (Important)

[Same structure]

---

Strength of Evidence Summary

Evidence Graded by Outcome

---

Implications

Implications for Practice

1. [Implication 1 with evidence support] 2. [Implication 2 with evidence support] 3. [Implication 3 with evidence support]

Implications for Research

1. [Research implication 1] 2. [Research implication 2] 3. [Research implication 3]

Implications for Policy

1. [Policy implication 1] 2. [Policy implication 2]

Implications for Theory/Understanding

1. [Theoretical implication 1] 2. [Theoretical implication 2]

---

Overall Assessment

Summary of Findings

[2-3 paragraph summary integrating all themes and evidence]

Strength of Evidence

Overall: ☐ High ☐ Moderate ☐ Low ☐ Very Low

Rationale: [Explain confidence in findings based on evidence quality, consistency, directness, precision]

Confidence in Conclusions

Overall: ☐ Very confident ☐ Fairly confident ☐ Somewhat confident ☐ Not confident

Sources of uncertainty: 1. [Source of uncertainty] 2. [Source of uncertainty] 3. [Source of uncertainty]

Critical Limitations

1. [Limitation and impact] 2. [Limitation and impact] 3. [Limitation and impact]

---

Recommendations

For Research (Priority Order)

1. Priority 1: [What research should be done] - [Rationale] 2. Priority 2: [What research should be done] - [Rationale] 3. Priority 3: [What research should be done] - [Rationale]

For Practice

1. Recommendation 1: [What practitioners should do] - Based on [evidence] 2. Recommendation 2: [What practitioners should do] - Based on [evidence]

For Policy

1. Recommendation 1: [Policy change recommended] - Based on [evidence] 2. Recommendation 2: [Policy change recommended] - Based on [evidence]

---

Next Steps in Review Writing

• [ ] Finalize theme organization and naming
• [ ] Complete detailed synthesis for all themes
• [ ] Write introduction contextualizing findings
• [ ] Write methods section documenting search/selection process
• [ ] Conduct quality check (balanced, coherent, well-supported)
• [ ] Develop recommendations section
• [ ] Write discussion of implications
• [ ] Draft conclusion synthesizing key messages
• [ ] Format citations and references
• [ ] Final review and editing

---

Use this outline to structure your written literature review. Each section can be expanded into full paragraphs with citations when writing the final report.

Literature Review Quick Reference Guide

PICO/PEO Question Template

Research Question: [Start with "What is..." or "How does..."]

Population (P): [Who/what is the focus?]
  Example: Adult patients with [condition]
  Example: Specific organisms, systems, or methodologies

Intervention (I): [What is being studied?]
  Example: Treatment or therapy
  Example: Technique, technology, or methodology

Comparison (C): [What alternative?] (optional)
  Example: Standard treatment, alternative approach, or no intervention

Outcome (O): [What results matter?]
  Example: Effectiveness, efficiency, safety, validity

Experience (E): [For qualitative research - what are perspectives?] (optional)
  Example: Patient experiences, researcher perspectives

Final PICO Question:
"In [Population], what is the effectiveness of [Intervention]
compared to [Comparison] for [Outcome]?"

Database Quick Selection

Boolean Search Operators

Basic Operators

AND       - All terms must appear
           "deep learning" AND "medical imaging"

OR        - Any term can appear
           "neural network" OR "deep learning"

NOT       - Exclude terms
           "cancer" NOT "pediatric"

( )       - Group concepts
           ("deep learning" OR "neural network") AND "diagnosis"

Field Operators (database-specific)

Title search:     TITLE:("deep learning")
Abstract search:  ABSTRACT:("medical imaging")
Author search:    AUTHOR:(Smith)
Publication year: YEAR:(2020-2025)
Journal name:     JOURNAL:("Nature Medicine")

Wildcards and Truncation

*         - Replace multiple characters
           "learn*" matches: learning, learner, learned

?         - Replace single character
           "neur?l" matches: neural, neuroal

""        - Exact phrase
           "deep learning" (in quotes for exact match)

Search Query Templates by Domain

Medical/Clinical Research

Basic template:
[Condition] AND ([Treatment] OR [Intervention])
AND ([Outcome] OR [Measurement])

Example:
("depression" OR "major depressive disorder") AND
("cognitive behavioral therapy" OR "CBT") AND
("remission" OR "symptom reduction")

Advanced template:
([Specific condition]) AND ([Intervention type]) AND
([Outcome measure]) NOT ([Exclusion criteria])

Example:
("type 2 diabetes") AND ("metformin") AND
("hemoglobin A1c" OR "glycemic control")
NOT ("gestational diabetes")

Computer Science/Technology

Basic template:
([Technology] OR [Method]) AND ([Application] OR [Task])
AND ([Metric] OR [Benchmark])

Example:
("transformer model*" OR "attention mechanism*") AND
("natural language processing" OR "NLP") AND
("accuracy" OR "BLEU score")

Advanced template:
([Core technology]) AND ([Application domain]) AND
([Performance aspect]) AND ([Year range]) NOT ([Excluded approach])

Example:
("neural architecture search" OR "NAS") AND
("computer vision" OR "image classification") AND
("energy efficiency" OR "latency" OR "inference time")
2018:2025 NOT ("toy dataset*")

Materials Science

Basic template:
[Property] AND ([Material] OR [Composition]) AND
([Method] OR [Technique])

Example:
("thermal conductivity") AND
("graphene" OR "carbon nanotubes") AND
("molecular dynamics" OR "first principles")

Advanced template:
([Property of interest]) AND ([Material class]) AND
([Computational/experimental method]) AND
([Application]) NOT ([Excluded condition])

Example:
("band gap") AND
("perovskite*" OR "halide perovskite*") AND
("density functional theory" OR "DFT") AND
("solar cell*" OR "photovoltaic*")
NOT ("lead-free" only for finding lead-containing)

Social Sciences

Basic template:
([Concept]) AND ([Population]) AND ([Context]) AND
([Method])

Example:
("social media") AND ("adolescent*" OR "teenager*") AND
("mental health" OR "depression" OR "anxiety") AND
("qualitative" OR "interview*")

Advanced template:
([Core concept]) AND ([Study population]) AND
([Outcome/impact]) NOT ([Excluded population/approach])

Example:
("remote work" OR "telework") AND
("employee*" OR "worker*") AND
("productivity" OR "burnout" OR "job satisfaction")
NOT ("COVID-19" if wanting non-pandemic data)

Inclusion/Exclusion Criteria Template

INCLUSION CRITERIA
✓ Study design: [Specify types]
  Example: Randomized controlled trials, cohort studies

✓ Population: [Define target population]
  Example: Adults (18-75 years), diagnosed with condition X

✓ Intervention: [What must be studied]
  Example: [Specific treatment, technique, or methodology]

✓ Outcomes: [Must measure these]
  Example: Primary outcome: [Y], Secondary: [Z]

✓ Publication: [Type and language]
  Example: Peer-reviewed English-language publications

✓ Time period: [Publication date range]
  Example: Published 2015-2025

✓ Setting: [Where study conducted]
  Example: Clinical settings, hospital-based, ambulatory care

EXCLUSION CRITERIA
✗ Wrong population: [Examples]
✗ Wrong intervention: [Examples]
✗ Wrong outcomes: [Examples]
✗ Wrong study design: [Examples]
✗ Review articles: [If seeking original research]
✗ Non-English publications: [If language constraint]
✗ Before [date]: [If time-limited review]
✗ Conference abstracts only: [No full papers]

Search Documentation Template

SEARCH LOG ENTRY

Search #: [Number]
Database: [PubMed/arXiv/etc]
Search Date: [YYYY-MM-DD]
Time Range: [E.g., 2015-2025]

Search Query:
[Exact query as entered]

Results: [Number of papers returned]
Fields Searched: [Title/Abstract/Full text/etc]
Filters Applied: [Language, publication type, etc]

New Relevant Papers: [Number found]
Pass Rate: [Percentage of results relevant]

Refinements for Next Search:
[Notes for improving future searches]

New Concepts Identified:
[Any new keywords/concepts to explore]

Citation Format Quick Reference

APA (Author-Date)

Journal article:
Smith, J., & Jones, M. (2023). Title of article. Journal Name, 45(3), 123-145.
https://doi.org/10.xxxx/xxxxx

Book chapter:
Johnson, R. (2022). Chapter title. In M. Editor (Ed.), Book title (pp. 45-67).
Publisher. https://doi.org/10.xxxx/xxxxx

In-text citation:
(Smith & Jones, 2023) or Smith and Jones (2023) showed...

Multiple works:
(Smith & Jones, 2023; Johnson, 2022)

Chicago (Notes-Bibliography)

Footnote (first reference):
1. Jennifer Smith and Maria Jones, "Title of Article," Journal Name 45,
   no. 3 (2023): 123-145, https://doi.org/10.xxxx/xxxxx.

Footnote (subsequent reference):
2. Smith and Jones, "Title of Article," 130.

Bibliography entry:
Smith, Jennifer, and Maria Jones. "Title of Article." Journal Name 45,
no. 3 (2023): 123-145. https://doi.org/10.xxxx/xxxxx.

IEEE (Numbered)

Journal article:
[1] J. Smith and M. Jones, "Title of article," Journal Name, vol. 45,
    no. 3, pp. 123–145, 2023, doi: 10.xxxx/xxxxx.

In-text citation:
[1] or The authors in [1] showed...

References list (numbered in order of appearance):
[1] first cited paper
[2] second cited paper
etc.

Nature/Science (Vancouver-style)

Journal article:
Smith, J., Jones, M. Title of article. Journal Name 45, 123–145 (2023).

Online publication:
Smith, J., Jones, M. Title of article. Journal Name https://doi.org/10.xxxx/xxxxx (2023).

In-text citations:
as superscript numbers¹²³

References in order of appearance

Data Extraction Template

---
Citation: [Full formatted reference]
DOI: [Digital object identifier]
Link: [URL to paper]

BIBLIOGRAPHIC INFO
Authors: [Full list]
Year: [Publication year]
Journal/Venue: [Publication outlet]
Impact Factor: [If applicable]
Type: [Original research/Review/Methods/etc]

STUDY CHARACTERISTICS
Design: [RCT/Cohort/Case-control/Observational/etc]
Population: [N=X, characteristics]
Setting: [Where conducted]
Duration: [Study timeframe]
Intervention: [What was tested]
Comparison: [Comparison group/control]

KEY FINDINGS
Primary outcome: [Measure and result]
Effect size: [If quantitative]
Statistical significance: [p-value, CI]
Secondary findings: [Other important results]

QUALITY ASSESSMENT
GRADE rating: [A/B/C/D]
Risk of bias: [High/Medium/Low]
Key concerns: [Specific quality issues]
Strengths: [What study does well]
Limitations: [Acknowledged and others]

RELEVANCE & CODING
Primary theme: [Main research topic]
Secondary themes: [Related areas]
Methodology: [Key methodological approach]
Novelty: [Foundational/Incremental/Novel]
Relevance to review: [High/Medium/Low]

NOTES
[Any additional important information]
---

Thematic Organization Quick Template

THEME: [Main theme name]
Papers: [Number of papers in theme]

Definition: [Clear definition of this theme]

Subtopics:
1. [Subtopic 1] - [X papers]
   Key finding: [Summary statement]
   Examples: [Exemplary papers]

2. [Subtopic 2] - [X papers]
   Key finding: [Summary statement]
   Examples: [Exemplary papers]

Key Consensus Finding:
[What most papers agree on]

Areas of Disagreement:
[Conflicting findings]

Research Evolution:
[How this topic has changed over time]

Gaps in this theme:
[What needs more research]

Quality Assessment Quick Scores

GRADE Assessment

A - High quality: Very unlikely biased estimate of true effect
B - Good quality: Unlikely biased estimate of true effect
C - Fair quality: May be biased but likely addresses question
D - Poor quality: Very likely biased; limited confidence

Risk of Bias Quick Check (Quantitative)

✓ Selection bias: Random assignment or adequate matching
✓ Performance bias: Blinding of participants/providers
✓ Detection bias: Blinding of outcome assessors
✓ Attrition bias: <10-15% dropout, balanced loss to follow-up
✓ Reporting bias: All planned outcomes reported
✓ Conflict of interest: No undisclosed conflicts

Each criterion: YES/NO/UNCLEAR

Qualitative Study Quick Assessment

Clear research aims? YES / NO / UNCLEAR
Appropriate methodology? YES / NO / UNCLEAR
Rigorous data collection? YES / NO / UNCLEAR
Clear analysis process? YES / NO / UNCLEAR
Reflexivity addressed? YES / NO / UNCLEAR
Data presented clearly? YES / NO / UNCLEAR
Limitations acknowledged? YES / NO / UNCLEAR

→ Rate overall: High / Medium / Low quality

Search Refinement Decision Tree

Do you have?

Fewer than 50 papers after screening?
├─ YES → Broaden search
│        • Add synonyms: "term" OR "synonym"
│        • Reduce specificity: Remove restrictive terms
│        • Expand date range
│        • Include grey literature
│        • Try different databases
│        └─ Rerun search
└─ NO → Continue

More than 5,000 papers?
├─ YES → Narrow search
│        • Add specificity: Add more AND operators
│        • Combine similar terms with OR
│        • Add publication type filters
│        • Reduce date range
│        └─ Rerun search
└─ NO → Continue

Are results relevant?
├─ ~70-80% relevant → GOOD, proceed to full-text screening
├─ 30-70% relevant → MODERATE, may refine but proceed
├─ <30% relevant → POOR, refine search before screening
└─ >90% relevant → VERY NARROW, consider broadening

Common Search Mistakes & Fixes

Report Structure Checklist

Required Sections

• [ ] Executive summary
• [ ] Introduction (problem, objectives, scope)
• [ ] Methods (search, inclusion/exclusion, quality assessment)
• [ ] Results (selection flowchart, characteristics, quality)
• [ ] Findings (organized thematically)
• [ ] Discussion (synthesis, gaps, implications)
• [ ] Conclusions
• [ ] References (formatted, complete, verified)

For Systematic Reviews (PRISMA)

• [ ] PRISMA 2020 checklist completed
• [ ] PRISMA flow diagram included
• [ ] Risk of bias assessment table
• [ ] Evidence quality summary (GRADE)
• [ ] Search strategy fully documented
• [ ] Detailed inclusion/exclusion criteria

For All Reviews

• [ ] All citations complete and accurate
• [ ] DOIs included where available
• [ ] Tables of study characteristics
• [ ] Clear distinction: fact vs. interpretation
• [ ] Conflicts of interest statement
• [ ] Funding source disclosure

Tools & Links

Search Tools

• PubMed: pubmed.ncbi.nlm.nih.gov
• arXiv: arxiv.org
• Google Scholar: scholar.google.com
• Web of Science: webofscience.com
• Scopus: scopus.com
• IEEE Xplore: ieeexplore.ieee.org
• ACM DL: dl.acm.org

Reference Managers

• Zotero: zotero.org (free)
• Mendeley: mendeley.com (free tier)
• EndNote: endnote.com (paid)
• RefWorks: refworks.com (institutional)

Quality Assessment Tools

• GRADE: gradeworkinggroup.org
• Cochrane Risk of Bias: training.cochrane.org
• CASP Checklists: casp-uk.net
• ROBINS-I: riskofbiasinhewvi.herokuapp.com

Citation Tools

• CrossRef: crossref.org
• DOI.org: doi.org
• Unpaywall: unpaywall.org (OA finder)

Key Metrics Quick Reference

Research Impact

• Citation count: Number of times paper cited
• h-index: Author impact (h papers with ≥h citations)
• Journal Impact Factor: Average citations per article
• Altmetric: Broader impact (mentions, downloads)

Study Quality (Effect Sizes)

• Cohen's d: 0.2=small, 0.5=medium, 0.8=large
• r: 0.1=small, 0.3=medium, 0.5=large
• Odds Ratio: 1.0=no effect, 1.5+=small effect
• Relative Risk: 1.0=no effect, 1.5+=moderate effect

Convergence Indicators (When to Stop Searching)

Stop searching when:

• [ ] Each new search finds <10-15% truly new papers
• [ ] Same papers appearing across multiple searches
• [ ] New searches mostly cited papers already in review
• [ ] Limited new conceptual themes identified
• [ ] Theoretical saturation reached (concepts repeating)
• [ ] 2-3 consecutive searches yield <5% new relevant papers
• [ ] Budget/timeline constraints reached

Continue searching if:

• [ ] Finding >20% new relevant papers in recent searches
• [ ] New research areas continuously identified
• [ ] Important methodological gaps still emerging
• [ ] Time and resources permit

---

For detailed information, see SKILL.md. For examples and workflows, see README.md.

Scientific Literature Review Skill

A comprehensive skill for conducting systematic, evidence-based literature reviews that integrates multi-source searches, iterative analysis, and synthesis into well-structured, citable reports.

Overview

This skill transforms the literature review process from a time-consuming, manual task into a systematic, reproducible, and efficient research methodology. Whether you're writing a research paper, grant proposal, dissertation, or conducting a systematic review, this skill provides proven strategies, tools, and workflows.

What This Skill Does

Core Capabilities

• Multi-source search: Integrate searches across PubMed, arXiv, Web of Science, Scopus, IEEE Xplore, ACM Digital Library, and web searches
• Iterative refinement: Systematically refine searches through concept extraction, citation mining, and gap identification
• Content analysis: Extract and organize bibliographic information, methodology, findings, and quality assessments
• Thematic synthesis: Organize findings by theme, research question, methodology, or chronological development
• Quality assessment: Evaluate research quality using established tools (GRADE, ROBINS-I, CASP)
• Bias detection: Identify publication bias, selection bias, and methodological limitations
• Citation management: Format citations in APA, Chicago, IEEE, Nature, Science, and other styles
• Report generation: Create comprehensive, well-structured literature review reports
• Gap identification: Identify research gaps, contradictions, and future research directions

Review Types Supported

1. Narrative Literature Reviews: Comprehensive overviews of research topics 2. Systematic Literature Reviews: Exhaustive, methodology-driven reviews following PRISMA guidelines 3. Scoping Reviews: Landscape mapping with flexible inclusion criteria 4. Meta-Analyses: Quantitative synthesis of comparable studies

When to Use This Skill

Use this skill when you need to:

Academic Writing

• Write literature review chapters for dissertations or theses
• Develop comprehensive background sections for research papers
• Create introduction sections that synthesize prior work
• Support manuscript submissions with evidence synthesis

Research Planning

• Identify gaps before starting research projects
• Understand the current state of knowledge in a domain
• Track methodological developments and trends
• Evaluate different research approaches

Grant Proposals

• Write compelling, evidence-based background sections
• Demonstrate comprehensive knowledge of field
• Show gap in current knowledge
• Justify research approach and innovation

Policy and Practice

• Support evidence-based decision-making
• Synthesize research for policy documents
• Create clinical practice guidelines
• Develop technology assessments

Academic Evaluation

• Prepare for comprehensive exams (PhD qualifiers)
• Write literature review papers
• Create research overviews for promotion/tenure
• Document expertise in research area

Quick Start Example

Scenario: Systematic Review of Deep Learning in Medical Imaging

Step 1: Define Research Question

Research Question: What is the effectiveness and safety of deep learning models
compared to traditional machine learning and human experts for diagnostic accuracy
in medical imaging?

PICO Framework:
- Population: Patients undergoing medical imaging (any modality)
- Intervention: Deep learning-based diagnostic systems
- Comparison: Traditional ML, human experts, standard of care
- Outcome: Diagnostic accuracy, sensitivity, specificity, clinical outcomes

Step 2: Plan Search Strategy

Selected Databases:
- PubMed (biomedical literature)
- arXiv (ML methodology preprints)
- IEEE Xplore (engineering approaches)
- Web of Science (multidisciplinary)

Primary Search Query:
"deep learning" AND ("medical imaging" OR "radiology" OR "diagnosis")
AND ("accuracy" OR "performance" OR "sensitivity" OR "specificity")

Time frame: 2015-2025 (deep learning era)
Language: English
Study types: Empirical research (exclude opinion/editorial)

Step 3: Execute and Track Initial Search

Search Results:
- PubMed: 1,247 papers
- IEEE Xplore: 893 papers
- arXiv: 2,156 papers
- Web of Science: 3,421 papers
Total (after deduplication): 4,789 papers

First screening (titles): Remove obviously irrelevant
After title screening: 1,248 papers (26% pass rate)

Abstract screening: Apply inclusion criteria
After abstract screening: 387 papers (8% of original)

Step 4: Iterative Refinement

Analysis of initial 387 papers:
- Common methodologies identified:
  * Convolutional neural networks (245 papers, 63%)
  * Attention mechanisms (87 papers, 22%)
  * Vision Transformers (23 papers, 6%)

- Underrepresented areas:
  * Pediatric imaging (34 papers)
  * Rare diseases (12 papers)
  * Low-income country settings (8 papers)

New targeted searches:
1. "attention mechanisms" + "medical imaging"
   Result: 45 new papers (23 relevant)

2. "pediatric" + "deep learning" + "diagnosis"
   Result: 78 papers (12 new relevant papers)

3. "Vision Transformer" + "radiology"
   Result: 34 papers (8 new relevant papers)

Convergence check: New searches finding 20-30% new papers
→ Continue iterating

Step 5: Full-Text Review and Data Extraction

After iterative searches: 423 papers for full-text review
Obtained: 401 papers (95%)
Unable to obtain: 22 papers

After quality assessment and detailed inclusion/exclusion:
Final included papers: 178 papers

Quality distribution:
- High quality (GRADE: A): 34 papers (19%)
- Good quality (GRADE: B): 78 papers (44%)
- Fair quality (GRADE: C): 56 papers (31%)
- Poor quality (GRADE: D): 10 papers (6%)

Step 6: Thematic Organization

Main themes identified:

1. CANCER DIAGNOSIS (89 papers)
   - Breast cancer: 34 papers
     * Mammography (18)
     * Ultrasound (8)
     * MRI (8)
   - Lung cancer: 32 papers
   - Colorectal cancer: 15 papers
   - Other: 8 papers

2. TREATMENT PLANNING (45 papers)
   - Radiotherapy: 28 papers
   - Surgical guidance: 12 papers
   - Monitoring: 5 papers

3. RISK STRATIFICATION (32 papers)
   - Prognostic models: 20 papers
   - Biomarker prediction: 12 papers

4. EMERGING APPLICATIONS (12 papers)
   - Rare diseases
   - Low-resource settings

Step 7: Synthesis and Findings

Key findings:
✓ Deep learning shows superior performance vs. traditional ML
  (87 papers show improvement, effect size: moderate to large)

✓ Comparable to human experts in specific tasks
  (32 papers; 12 show superiority, 15 show equivalence, 5 show inferiority)

✓ Generalization challenges
  (45 papers report overfitting/poor generalization to new data)

? Safety and regulatory concerns
  (Only 23 papers address regulatory/safety aspects)

⚠ Publication bias detected
  - Positive results more common (87% show improvement)
  - Negative/null results underrepresented
  - High-impact journals biased toward novel methods

Step 8: Identify Gaps

Major research gaps:

1. POPULATION GAPS
   Gap: Limited research on pediatric patients
   Current: 34 papers (8% of literature)
   Needed: More diverse age groups

2. METHODOLOGY GAPS
   Gap: No direct comparison of DL vs. expert radiologists
   Current: Separate studies for each
   Recommendation: Prospective comparative studies

3. GENERALIZATION GAPS
   Gap: Limited evidence on model transfer to new institutions
   Current: Single-center studies dominant
   Recommendation: Multi-center validation studies

4. APPLICATION GAPS
   Gap: Limited real-world implementation studies
   Current: Lab/pilot studies dominant
   Recommendation: Implementation science approaches

5. EQUITY GAPS
   Gap: Underrepresentation of low-income settings
   Current: 8 papers on non-US, non-Europe settings
   Recommendation: Targeted research in diverse settings

Step 9: Write Comprehensive Report

Report sections:
1. Executive Summary (1 page)
   - 178 relevant papers synthesized
   - Deep learning superior to traditional ML
   - Significant generalization challenges
   - Major gaps in pediatric and low-resource settings

2. Introduction (2 pages)
   - Historical context
   - Clinical motivation
   - Research question and objectives

3. Methods (1.5 pages)
   - Search strategy documented
   - Inclusion/exclusion criteria
   - Quality assessment approach

4. Results (4 pages)
   - Selection flowchart
   - Paper characteristics
   - Quality assessment summary
   - Bias assessment

5. Thematic Analysis (12 pages)
   - Cancer diagnosis (3 pages)
   - Treatment planning (2 pages)
   - Risk stratification (2 pages)
   - Emerging applications (2 pages)
   - Methodological trends (3 pages)

6. Synthesis and Discussion (4 pages)
   - Integration of findings
   - Quality and bias assessment
   - Implications for practice
   - Implications for research

7. Research Gaps and Future Directions (2 pages)
   - Identified gaps
   - Priority research needs
   - Implementation challenges

8. Conclusions (1 page)
   - Summary of key findings
   - Clinical implications
   - Research implications

Total: ~30 pages with comprehensive citations

Key Workflow Steps

1. Planning Phase

• Define clear research question (PICO format)
• Establish inclusion/exclusion criteria
• Select appropriate databases
• Develop search strategy
• Create documentation system

2. Initial Search Phase

• Execute searches across databases
• Track all search parameters
• Initial title/abstract screening
• Document pass rates

3. Iterative Refinement Phase

• Concept extraction from initial results
• Citation mining (reference lists + forward citation tracking)
• Author tracking (identify key researchers)
• Targeted searches for identified gaps
• Monitor for saturation (diminishing returns)

4. Full-Text Phase

• Obtain full texts
• Detailed inclusion/exclusion assessment
• Quality assessment using validated tools
• Structured data extraction

5. Analysis Phase

• Thematic organization
• Quality assessment and bias detection
• Evidence synthesis
• Gap identification
• Conflict identification

6. Writing Phase

• Outline structure
• Write sections with citations
• Integrate findings across themes
• Discuss gaps and implications
• Final quality review

7. Output Phase

• Format for target audience
• Generate citations (BibTeX, RIS, etc.)
• Verify all references
• Final formatting and review

Tools Recommended by Reviewtype

For All Literature Reviews

Essential:

• Reference manager: Zotero (free, open-source)
• Spreadsheet/database: Excel, Google Sheets, or Airtable
• Writing tool: Word, Google Docs, or Markdown + LaTeX

Highly Recommended:

• Citation formatter: CrossRef or DOI lookup
• Note-taking: OneNote, Notion, or Obsidian

For Systematic Reviews

Required:

• PRISMA checklist for compliance
• Systematic review platform: Covidence or DistillerSR
• Quality assessment tools: GRADE, ROBINS-I, CASP

Recommended:

• Statistical software: R, Python, or STATA (for meta-analysis)
• Network visualization: VosViewer (for citation networks)
• Forest plot software: RevMan (for meta-analysis)

For Narrative Reviews

Recommended:

• Concept mapping: CmapTools or MindMeister
• Visualization: Tableau or ggplot2
• Markdown editor: VS Code with Markdown extension

Integration with Your Research Workflow

This skill integrates with:

scientific-writing skill

Use together for:

• Writing literature review sections of manuscripts
• Formatting citations correctly
• Organizing section content
• Revising and improving prose

eln (Electronic Lab Notebook) skill

Use together for:

• Documenting literature review process
• Recording search decisions
• Tracking paper annotations
• Storing extracted data
• Creating research timeline

planning skill

Use together for:

• Setting literature review milestones
• Tracking progress toward completion
• Managing team assignments (for collaborative reviews)
• Scheduling database searches

phd-qualifier skill

Use together for:

• Preparing for comprehensive exams
• Writing exam literature reviews
• Demonstrating comprehensive knowledge
• Creating study guides

Output Examples

Example 1: Literature Review Section for Research Paper

## Background

The application of machine learning to [domain] has grown substantially
over the past decade. We conducted a systematic review of [X] papers
published between 2015-2025 to synthesize current knowledge and identify
research gaps.

### Current State of Knowledge

[Thematic summary of 1-2 sentences for each major finding]

### Methodological Approaches

[Summary of common approaches, with citations to exemplary papers]

### Open Questions and Limitations

[Discussion of conflicting findings, limitations, and gaps]

Example 2: Standalone Literature Review Report

Structure: 20-40 pages with:

• Comprehensive searchable reference list
• Thematic sections with detailed discussion
• Evidence synthesis tables
• Quality and bias assessment
• Gap identification with recommendations

Example 3: Grant Proposal Background

Project Background

[1-2 paragraph overview of field]

Current State of Knowledge

[Discussion of relevant research, with strategic citations to
show comprehensive knowledge]

Research Gap

[Clear articulation of what is NOT known, based on systematic review]

Research Innovation

[How proposed work addresses identified gap]

Best Practices from This Skill

1. Document everything: Every search, decision, and finding recorded 2. Iterate systematically: Use initial results to refine strategy 3. Quality over quantity: One high-quality paper worth more than many poor-quality ones 4. Synthesize, don't summarize: Integrate findings, don't just list papers 5. Address contradictions: Acknowledge and explain conflicting findings 6. Identify gaps: Translate review into research agenda 7. Maintain reproducibility: Others should be able to replicate your search strategy 8. Cross-verify: Important findings should appear in multiple papers/databases 9. Time-bound appropriately: Comprehensive reviews take time; plan accordingly 10. Use teams where possible: Multiple reviewers improve quality and reduce bias

Common Challenges and Solutions

Challenge 1: Too Many Results

Problem: Broad search returns 10,000+ papers Solutions:

• Narrow inclusion criteria
• Add specificity to search terms
• Use database filters (date, study type, etc.)
• Consider focused review instead of comprehensive

Challenge 2: Insufficient Results

Problem: Narrow search returns <50 papers Solutions:

• Broaden search terms and synonyms
• Include related concepts
• Search for grey literature
• Consider scoping review instead of systematic

Challenge 3: Contradictory Findings

Problem: Papers reach opposite conclusions Solutions:

• Compare methodologies (differences may explain contradictions)
• Assess quality (higher-quality studies may be more reliable)
• Look for study moderators (different populations, contexts)
• Note as area of uncertainty

Challenge 4: Biased Literature

Problem: Predominantly positive results, few null/negative findings Solutions:

• Search for negative studies explicitly
• Check grey literature (conferences, theses)
• Use funnel plots to detect bias
• Note publication bias in discussion
• Weight high-quality null findings appropriately

Challenge 5: Limited Full-Text Access

Problem: Many papers behind paywalls Solutions:

• Use institutional library access
• Use legal repositories (PubMed Central, arXiv, institutional repos)
• Contact authors directly for reprints
• Use Unpaywall to find legal OA versions
• Note limitation if significant access barriers exist

Getting Started

For Your First Literature Review:

1. Choose review type: Narrative, systematic, or scoping? 2. Define your question: Use PICO/PEO framework 3. Select databases: 2-3 primary, 1-2 supplementary 4. Set up tracking: Spreadsheet for papers and decisions 5. Start searching: Execute searches with full documentation 6. Screen iteratively: Title → Abstract → Full text 7. Synthesize thematically: Organize by theme, not by paper 8. Write sections: Guided by SKILL.md structure 9. Verify citations: All references complete and accurate 10. Quality review: Ensure logical flow and balanced perspective

Time Expectations

• Narrative review (focused): 4-8 weeks
• Narrative review (comprehensive): 8-16 weeks
• Systematic review (small scope): 3-6 months
• Systematic review (large scope): 6-12+ months
• Meta-analysis: 6-18 months

These estimates assume 50-100 papers for narrative review, 100-300+ for systematic reviews.

Conclusion

This skill provides a systematic, reproducible approach to literature reviews that:

• ✓ Saves time through efficient searching and organization
• ✓ Improves quality through structured analysis and synthesis
• ✓ Enhances credibility through transparent, documented methodology
• ✓ Creates comprehensive, citable reports with proper attribution
• ✓ Identifies actionable research gaps and future directions

Whether you're writing a paper, preparing for exams, planning research, or supporting policy decisions, this skill provides the methodology and guidance for conducting literature reviews that advance knowledge in your field.

---

For comprehensive details, see SKILL.md. For quick reference, see QUICK_REFERENCE.md.