Scientific Critical Thinking

# Common Biases in Scientific Research

## Cognitive Biases Affecting Researchers

### 1. Confirmation Bias
**Description:** Tendency to search for, interpret, and recall information that confirms preexisting beliefs.

**Manifestations:**
- Designing studies that can only support the hypothesis
- Interpreting ambiguous results as supportive
- Remembering hits and forgetting misses
- Selectively citing literature that agrees

**Mitigation:**
- Preregister hypotheses and analysis plans
- Actively seek disconfirming evidence
- Use blinded data analysis
- Consider alternative hypotheses

### 2. Hindsight Bias (I-Knew-It-All-Along Effect)
**Description:** After an event, people perceive it as having been more predictable than it actually was.

**Manifestations:**
- HARKing (Hypothesizing After Results are Known)
- Claiming predictions that weren't made
- Underestimating surprise at results

**Mitigation:**
- Document predictions before data collection
- Preregister studies
- Distinguish exploratory from confirmatory analyses

### 3. Publication Bias (File Drawer Problem)
**Description:** Positive/significant results are more likely to be published than negative/null results.

**Manifestations:**
- Literature appears to support effects that don't exist
- Overestimation of effect sizes
- Inability to estimate true effects from published literature

**Mitigation:**
- Publish null results
- Use preregistration and registered reports
- Conduct systematic reviews with grey literature
- Check for funnel plot asymmetry in meta-analyses

### 4. Anchoring Bias
**Description:** Over-reliance on the first piece of information encountered.

**Manifestations:**
- Initial hypotheses unduly influence interpretation
- First studies in a field set expectations
- Pilot data biases main study interpretation

**Mitigation:**
- Consider multiple initial hypotheses
- Evaluate evidence independently
- Use structured decision-making

### 5. Availability Heuristic
**Description:** Overestimating likelihood of events based on how easily examples come to mind.

**Manifestations:**
- Overemphasizing recent or dramatic findings
- Neglecting base rates
- Anecdotal evidence overshadowing statistics

**Mitigation:**
- Consult systematic reviews, not memorable papers
- Consider base rates explicitly
- Use statistical thinking, not intuition

### 6. Bandwagon Effect
**Description:** Adopting beliefs because many others hold them.

**Manifestations:**
- Following research trends without critical evaluation
- Citing widely-cited papers without reading
- Accepting "textbook knowledge" uncritically

**Mitigation:**
- Evaluate evidence independently
- Read original sources
- Question assumptions

### 7. Belief Perseverance
**Description:** Maintaining beliefs even after evidence disproving them.

**Manifestations:**
- Defending theories despite contradictory evidence
- Finding ad hoc explanations for discrepant results
- Dismissing replication failures

**Mitigation:**
- Explicitly consider what evidence would change your mind
- Update beliefs based on evidence
- Distinguish between theories and ego

### 8. Outcome Bias
**Description:** Judging decisions based on outcomes rather than the quality of the decision at the time.

**Manifestations:**
- Valuing lucky guesses over sound methodology
- Dismissing good studies with null results
- Rewarding sensational findings over rigorous methods

**Mitigation:**
- Evaluate methodology independently of results
- Value rigor and transparency
- Recognize role of chance

## Experimental and Methodological Biases

### 9. Selection Bias
**Description:** Systematic differences between those selected for study and those not selected.

**Types:**
- **Sampling bias:** Non-random sample
- **Attrition bias:** Systematic dropout
- **Volunteer bias:** Self-selected participants differ
- **Berkson's bias:** Hospital patients differ from general population
- **Survivorship bias:** Only examining "survivors"

**Detection:**
- Compare characteris