Ads Test
Design paid-ad A/B tests with a clear hypothesis, sample size, duration, and platform setup for Meta, Google, or LinkedIn.
Overview
ads-test is an agent skill for the Grow phase that designs paid-ad A/B experiments with hypotheses, sample sizes, duration estimates, and Meta, Google, or LinkedIn setup guidance.
Install
npx skills add https://github.com/agricidaniel/claude-ads --skill ads-testWhat is this skill?
- Five-step process: test intent, hypothesis, sample size/duration, platform setup, success criteria
- IF / THEN / BECAUSE hypothesis framework with quality checklist
- Statistical significance, sample size, and test duration estimators
- Platform guides: Meta Experiments, Google Experiments, LinkedIn A/B
- Single-variable isolation emphasized in hypothesis quality checklist
- 5-step experiment process from intent through measurement plan
- Hypothesis quality checklist includes single-variable isolation
Adoption & trust: 568 installs on skills.sh; 5.8k GitHub stars; 3/3 security scanners passed (skills.sh audits).
What problem does it solve?
You are changing ad creative and budgets without a isolated hypothesis, so you cannot tell if CTR moved because of the video or the audience.
Who is it for?
Founders self-serve paid social/search who need disciplined experiment design without a dedicated growth analyst.
Skip if: Organic-only channels with no paid experiments, or teams that already run centralized experimentation platforms with legal/stat review baked in.
When should I use this skill?
User says A/B test, split test, experiment design, test hypothesis, statistical significance, sample size, or test duration.
What do I get? / Deliverables
You get a documented hypothesis, sample-size and duration estimates, platform-specific setup steps, and success criteria before spend goes live.
- Structured IF-THEN-BECAUSE hypothesis
- Sample size and duration estimates
- Platform-specific experiment setup and success metrics plan
Recommended Skills
Journey fit
Grow → analytics is the canonical shelf because experiment design, significance, and measurement plans are analytics work—not first-time launch distribution alone. Structured hypotheses, sample-size math, and success criteria belong with growth measurement, not generic Build integrations.
How it compares
Experiment design and stats scaffolding for ads—not a campaign creative generator or account API integration.
Common Questions / FAQ
Who is ads-test for?
Solo and indie operators managing Meta, Google, or LinkedIn paid campaigns who want structured A/B tests and clear readout criteria.
When should I use ads-test?
Use it when you say A/B test, split test, experiment design, test hypothesis, statistical significance, sample size, or test duration while planning or optimizing paid ads in Grow.
Is ads-test safe to install?
It provides planning guidance only; review the Security Audits panel on this Prism page and never paste ad account secrets into untrusted skill flows.
SKILL.md
READMESKILL.md - Ads Test
# A/B Test Design & Experiment Planning <!-- Created: 2026-04-13 | v1.5 --> <!-- Source: OpenClaudia/openclaudia-skills (ab-test-setup concept) --> ## Process 1. Understand what the user wants to test (creative, audience, bidding, landing page) 2. Build structured hypothesis using the framework below 3. Calculate required sample size and estimated duration 4. Recommend platform-specific test setup 5. Define success criteria and measurement plan ## Hypothesis Framework Every test must start with a structured hypothesis: ``` IF we [change/action] THEN [metric] will [increase/decrease] by [estimated %] BECAUSE [reasoning based on data or insight] Example: IF we replace polished product shots with UGC creator videos THEN Meta CTR will increase by 25-40% BECAUSE Andromeda prioritizes diverse creative formats and UGC consistently outperforms polished in 2025-2026 benchmarks ``` ### Hypothesis Quality Checklist - [ ] Single variable being tested (isolate the change) - [ ] Specific metric defined (not "performance") - [ ] Estimated effect size stated (needed for sample size calculation) - [ ] Timeframe defined - [ ] Success/failure criteria clear before launch ## Statistical Significance Calculator ``` Required Sample Size (per variant): n = (Z_alpha + Z_beta)^2 × 2 × p × (1-p) / MDE^2 Where: - Z_alpha = 1.96 (for 95% confidence) - Z_beta = 0.84 (for 80% power) - p = baseline conversion rate - MDE = minimum detectable effect (relative %) Simplified lookup: ``` | Baseline CVR | 5% MDE | 10% MDE | 20% MDE | 30% MDE | |-------------|---------|---------|---------|---------| | 1% | 612,000 | 153,000 | 38,300 | 17,000 | | 2% | 302,400 | 75,600 | 18,900 | 8,400 | | 5% | 116,800 | 29,200 | 7,300 | 3,200 | | 10% | 55,200 | 13,800 | 3,450 | 1,530 | | 20% | 24,600 | 6,150 | 1,540 | 680 | *Per variant, 95% confidence, 80% power* ## Test Duration Estimator ``` Duration = Required Sample Size / Daily Traffic per Variant Minimum duration: 7 days (capture weekly patterns) Maximum recommended: 28 days (avoid seasonal drift) Learning phase: Google 7-14 days, Meta 3-7 days, LinkedIn 7-14 days Inputs needed: - Daily impressions or clicks - Number of variants (2 = A/B, 3+ = multivariate) - Baseline conversion rate - Minimum detectable effect desired ``` ### Duration Quick Estimates | Daily Clicks | 2% CVR, 20% MDE | 5% CVR, 20% MDE | 10% CVR, 20% MDE | |-------------|-----------------|-----------------|-----------------| | 100 | 189 days | 73 days | 35 days | | 500 | 38 days | 15 days | 7 days | | 1,000 | 19 days | 7 days | 4 days* | | 5,000 | 4 days* | 2 days* | 1 day* | *Minimum 7 days recommended regardless of sample sufficiency ## Platform-Specific Test Setup ### Meta Experiments - Use Ads Manager > Experiments tab (not manual ad set duplication) - Automatic audience splitting ensures no overlap - Supported test types: A/B (creative, audience, placement), Holdout, Brand Survey - Meta's Incremental Attribution (April 2025) provides AI-powered holdout testing for measuring real causal impact - Budget: split evenly across variants; minimum $100/day per variant recommended - Duration: 7-14 days typical; Meta auto-determines winner at 95% confidence ### Google Experiments - Campaign Experiments (custom experiments) or Ad Variations - Create experiment from existing campaign > select experiment type - Traffic split: 50/50 recommended for fastest results - Supported: bidding strategy, ad copy, landing page, audien