Scientific Critical Thinking
Stress-test hypotheses, study designs, and published claims with a structured catalog of research biases and mitigation tactics while you build or write.
Overview
Scientific Critical Thinking is a journey-wide agent skill that flags cognitive and publication biases in research—usable whenever a solo builder needs to sanity-check evidence before committing.
Install
npx skills add https://github.com/k-dense-ai/scientific-agent-skills --skill scientific-critical-thinkingWhat is this skill?
- Documented cognitive biases affecting researchers including confirmation bias and hindsight bias
- Publication bias (file drawer) patterns and literature distortion risks
- Mitigations: preregistration, blinded analysis, registered reports, grey literature in reviews
- HARKing and exploratory vs confirmatory analysis separation
- Actionable researcher-facing bias descriptions with manifestation checklists
- Structured bias entries with manifestations and mitigation bullets (e.g. confirmation, hindsight, publication bias)
Adoption & trust: 670 installs on skills.sh; 27.6k GitHub stars; 2/3 security scanners passed (skills.sh audits).
What problem does it solve?
You are making product or science decisions from studies and metrics but cannot name which biases might be skewing your conclusions.
Who is it for?
Founders, data-curious engineers, and agent builders who read papers, run experiments, or write research-backed content and want repeatable skepticism.
Skip if: Pure codegen tasks with no evidentiary claims, or teams that need legal/medical compliance sign-off without human experts.
When should I use this skill?
When analyzing scientific claims, designing studies, interpreting ambiguous results, or preparing literature-backed product decisions.
What do I get? / Deliverables
You get clearer threat models for your claims, named biases with mitigations, and a sharper split between exploratory and confirmatory work.
- Bias identification mapped to your scenario
- Recommended mitigations (preregistration, blinding, null-result visibility)
Recommended Skills
Journey fit
Useful at every journey phase - explore requirements and options before committing to a direction.
Where it fits
Map publication bias before treating a hot arXiv result as your roadmap anchor.
Preregister metrics and guard against HARKing before running a landing-page experiment.
Review an internal benchmark write-up for confirmation bias and selective citation.
Challenge a dashboards narrative that ignores null weeks and survivorship in cohorts.
How it compares
Methodology and bias reference for agents—not a statistical computing library or automated peer-review submission tool.
Common Questions / FAQ
Who is scientific-critical-thinking for?
Solo builders and small teams who interpret experiments, academic literature, or internal metrics and want structured bias awareness inside the agent workflow.
When should I use scientific-critical-thinking?
In Validate when scoping studies; in Idea when surveying literature; in Ship during review of eval results; in Grow when judging analytics narratives; whenever claims outrun evidence.
Is scientific-critical-thinking safe to install?
It is informational procedural content with no elevated permissions described; confirm trust via the Security Audits panel on this Prism page like any third-party skill package.
SKILL.md
READMESKILL.md - Scientific Critical Thinking
# Common Biases in Scientific Research ## Cognitive Biases Affecting Researchers ### 1. Confirmation Bias **Description:** Tendency to search for, interpret, and recall information that confirms preexisting beliefs. **Manifestations:** - Designing studies that can only support the hypothesis - Interpreting ambiguous results as supportive - Remembering hits and forgetting misses - Selectively citing literature that agrees **Mitigation:** - Preregister hypotheses and analysis plans - Actively seek disconfirming evidence - Use blinded data analysis - Consider alternative hypotheses ### 2. Hindsight Bias (I-Knew-It-All-Along Effect) **Description:** After an event, people perceive it as having been more predictable than it actually was. **Manifestations:** - HARKing (Hypothesizing After Results are Known) - Claiming predictions that weren't made - Underestimating surprise at results **Mitigation:** - Document predictions before data collection - Preregister studies - Distinguish exploratory from confirmatory analyses ### 3. Publication Bias (File Drawer Problem) **Description:** Positive/significant results are more likely to be published than negative/null results. **Manifestations:** - Literature appears to support effects that don't exist - Overestimation of effect sizes - Inability to estimate true effects from published literature **Mitigation:** - Publish null results - Use preregistration and registered reports - Conduct systematic reviews with grey literature - Check for funnel plot asymmetry in meta-analyses ### 4. Anchoring Bias **Description:** Over-reliance on the first piece of information encountered. **Manifestations:** - Initial hypotheses unduly influence interpretation - First studies in a field set expectations - Pilot data biases main study interpretation **Mitigation:** - Consider multiple initial hypotheses - Evaluate evidence independently - Use structured decision-making ### 5. Availability Heuristic **Description:** Overestimating likelihood of events based on how easily examples come to mind. **Manifestations:** - Overemphasizing recent or dramatic findings - Neglecting base rates - Anecdotal evidence overshadowing statistics **Mitigation:** - Consult systematic reviews, not memorable papers - Consider base rates explicitly - Use statistical thinking, not intuition ### 6. Bandwagon Effect **Description:** Adopting beliefs because many others hold them. **Manifestations:** - Following research trends without critical evaluation - Citing widely-cited papers without reading - Accepting "textbook knowledge" uncritically **Mitigation:** - Evaluate evidence independently - Read original sources - Question assumptions ### 7. Belief Perseverance **Description:** Maintaining beliefs even after evidence disproving them. **Manifestations:** - Defending theories despite contradictory evidence - Finding ad hoc explanations for discrepant results - Dismissing replication failures **Mitigation:** - Explicitly consider what evidence would change your mind - Update beliefs based on evidence - Distinguish between theories and ego ### 8. Outcome Bias **Description:** Judging decisions based on outcomes rather than the quality of the decision at the time. **Manifestations:** - Valuing lucky guesses over sound methodology - Dismissing good studies with null results - Rewarding sensational findings over rigorous methods **Mitigation:** - Evaluate methodology independently of results - Value rigor and transparency - Recognize role of chance ## Experimental and Methodological Biases ### 9. Selection Bias **Description:** Systematic differences between those selected for study and those not selected. **Types:** - **Sampling bias:** Non-random sample - **Attrition bias:** Systematic dropout - **Volunteer bias:** Self-selected participants differ - **Berkson's bias:** Hospital patients differ from general population - **Survivorship bias:** Only examining "survivors" **Detection:** - Compare characteris