Statistical Audit
- 1 installs
- Updated May 22, 2026
- bytetiff/math-research-lab-for-codex
statistical-audit is a Claude skill that assesses empirical evidence strength using confidence intervals, effect sizes, and bootstrap or permutation tests.
About
This skill audits the strength of empirical evidence for research claims. It computes confidence intervals, effect sizes, and bootstrap or permutation tests, checks sensitivity to seeds and subsets, and separates exploratory correlations from confirmatory testing. A researcher uses it to quantify uncertainty on tabular data before stating conclusions.
- Quantifies uncertainty and distinguishes exploratory correlations from confirmatory claims
- Computes confidence intervals, effect sizes, and bootstrap or permutation tests, and checks seed sensitivity
- Bundles scripts: bootstrap_ci.py, effect_size.py, permutation_test.py, seed_sensitivity.py
Statistical Audit by the numbers
- 1 all-time installs (skills.sh)
- Ranked #1,803 of 2,064 Data Science & ML skills by installs in the Skillselion catalog
- Data as of Jul 8, 2026 (Skillselion catalog sync)
statistical-audit capabilities & compatibility
- Capabilities
- data analysis
- Use cases
- data analysis · research
- Pricing
- Free
What statistical-audit says it does
Use to assess empirical evidence strength, uncertainty, confidence intervals, effect sizes, permutation tests, bootstrap tests, and seed sensitivity.
Quantify uncertainty and distinguish exploratory correlations from confirmatory claims.
Do not treat statistical association as causation.
npx skills add https://github.com/bytetiff/math-research-lab-for-codex --skill statistical-auditAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 1 |
|---|---|
| Last updated | May 22, 2026 |
| Repository | bytetiff/math-research-lab-for-codex ↗ |
What it does
Audit empirical evidence with confidence intervals, effect sizes, and bootstrap/permutation tests on tabular data.
Who is it for?
quantifying uncertainty and validating research claims on tabular data
Skip if: claiming causation from association or ignoring small replication counts
When should I use this skill?
assessing uncertainty, effect sizes, or whether a claim is exploratory or confirmatory
What you get
Produces uncertainty, confidence intervals, effect sizes, and sensitivity findings that qualify claim strength.
- uncertainty estimate
- confidence interval
- effect size
By the numbers
- 6-step required workflow
- 4 bundled analysis scripts
Files
Purpose
Quantify uncertainty and distinguish exploratory correlations from confirmatory claims.
When to use
Use to assess empirical evidence strength, uncertainty, confidence intervals, effect sizes, permutation tests, bootstrap tests, and seed sensitivity.
Inputs to locate or request
- CSV or tabular data.
- Metric definition, sample unit, replication structure, grouping, seeds, and protocol scope.
Required workflow
1. Identify the experimental unit and metric definition. 2. Check replication structure and estimate uncertainty. 3. Compute confidence intervals, effect size, bootstrap or permutation tests where appropriate. 4. Check sensitivity to seeds, orders, subsets, or configurations. 5. Distinguish exploratory correlation from confirmatory testing. 6. Use scripts/bootstrap_ci.py, scripts/permutation_test.py, scripts/effect_size.py, and scripts/seed_sensitivity.py.
Required outputs
- Metric and sample unit.
- Number of observations.
- Uncertainty.
- Confidence interval.
- Effect size.
- Sensitivity findings.
- Inference limits.
- Conclusion strength.
Uncertainty policy
- Separate empirical fact, metric result, mechanism, theory, implementation detail, speculation, and candidate concept.
- Missing evidence may lower confidence but is not contradictory evidence.
- Contradictory evidence may weaken or reject only the specific claim it conflicts with.
- A script result is a measurement or check, not a proof of mechanism or novelty.
What not to do
- Do not treat statistical association as causation.
- Do not ignore small replication count warnings.
- Do not collapse exploratory and confirmatory analyses.
Scripts
scripts/bootstrap_ci.pyscripts/effect_size.pyscripts/permutation_test.pyscripts/seed_sensitivity.py
References
- Read the local
references/files for this skill and the plugin-levelreferences/policy files before producing final judgments.
Statistical Audit Protocol
Identify the unit, metric, replication, uncertainty, confidence interval, effect size, sensitivity, and whether the test is exploratory or confirmatory.
from __future__ import annotations
from pathlib import Path
import sys
PLUGIN_ROOT = Path(__file__).resolve().parents[3]
sys.path.insert(0, str(PLUGIN_ROOT / "src"))
from math_research_lab.script_cli import main as _main
def main(argv: list[str] | None = None) -> int:
'''Run this Math Research Lab command line utility.'''
return _main('bootstrap_ci', argv)
if __name__ == "__main__":
raise SystemExit(main())
from __future__ import annotations
from pathlib import Path
import sys
PLUGIN_ROOT = Path(__file__).resolve().parents[3]
sys.path.insert(0, str(PLUGIN_ROOT / "src"))
from math_research_lab.script_cli import main as _main
def main(argv: list[str] | None = None) -> int:
'''Run this Math Research Lab command line utility.'''
return _main('effect_size', argv)
if __name__ == "__main__":
raise SystemExit(main())
from __future__ import annotations
from pathlib import Path
import sys
PLUGIN_ROOT = Path(__file__).resolve().parents[3]
sys.path.insert(0, str(PLUGIN_ROOT / "src"))
from math_research_lab.script_cli import main as _main
def main(argv: list[str] | None = None) -> int:
'''Run this Math Research Lab command line utility.'''
return _main('permutation_test', argv)
if __name__ == "__main__":
raise SystemExit(main())
from __future__ import annotations
from pathlib import Path
import sys
PLUGIN_ROOT = Path(__file__).resolve().parents[3]
sys.path.insert(0, str(PLUGIN_ROOT / "src"))
from math_research_lab.script_cli import main as _main
def main(argv: list[str] | None = None) -> int:
'''Run this Math Research Lab command line utility.'''
return _main('seed_sensitivity', argv)
if __name__ == "__main__":
raise SystemExit(main())
Related skills
Microsoft FoundryDeploy, evaluate, and continuously improve Microsoft Foundry agents from a single agent interface.490k1.3k
Ai Research ReproductionOrchestrate trustworthy, auditable reproduction of deep learning repositories directly from their READMEs.176k511Run TrainSafely execute selected deep learning training commands with standardized evidence capture.176k511Explore RunSafely run isolated exploratory experiments with clear recording and conservative selection before committing changes.176k511Paper Context ResolverFetch precise reproduction-critical details like dataset splits, preprocessing steps, or evaluation protocols from the original academic paper when the repo README leav141k511Repo Intake And PlanScan unfamiliar AI research repositories and receive a minimal, trustworthy reproduction target before investing significant time.140k511FAQ
What data does it need?
CSV or tabular data plus the metric definition, sample unit, replication structure, grouping, and seeds.
Does it prove causation?
No. It explicitly warns not to treat statistical association as causation.