Ab Test Designer
- 338 installs
- 43 repo stars
- Updated March 7, 2026
- pmprompt/claude-plugin-product-management
ab-test-designer is a product management skill that designs statistically sound A/B experiments with hypotheses, metrics, sample sizes, and guardrails for developers planning feature validation.
About
ab-test-designer is a product management skill from pmprompt/claude-plugin-product-management for engineers and PMs planning feature experiments before variant code ships. The skill designs robust A/B tests with clear hypotheses, success metrics, sample-size estimates, guardrail metrics, and rollout plans using an industry PM framework. Developers reach for ab-test-designer when testing a new feature, validating a conversion hypothesis, or optimizing funnels and need experiment specs engineers can implement confidently. Input includes product context, optional research data, and the change under test via an argument hint for the feature or variation. The output is a structured experiment plan that reduces underpowered tests and missing guardrails. Skip it when experiments already have signed-off statistical design, for qualitative-only discovery, or when no measurable metric exists for the change.
- Hypothesis and metric framing
- Sample size and duration estimates
- Guardrail metric selection
- Variant allocation plans
- Pre-registration checklists
Ab Test Designer by the numbers
- 338 all-time installs (skills.sh)
- +13 installs in the week ending Aug 4, 2026 (Skillselion tracking)
- Ranked #842 of 3,282 Productivity & Planning skills by installs in the Skillselion catalog
- Data as of Aug 4, 2026 (Skillselion catalog sync)
npx skills add https://github.com/pmprompt/claude-plugin-product-management --skill ab-test-designerAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 338 |
|---|---|
| repo stars | ★ 43 |
| Last updated | March 7, 2026 |
| Repository | pmprompt/claude-plugin-product-management ↗ |
How do you design a statistically sound A/B test?
Design statistically sound A/B experiments with clear hypotheses, success metrics, sample-size estimates, guardrails, and rollout plans before committing engineering to variant builds.
Who is it for?
Engineers and PMs scoping feature experiments who need statistically grounded A/B plans before variant implementation starts.
Skip if: Teams with approved experiment designs, qualitative-only research, or changes lacking measurable success metrics.
When should I use this skill?
A developer or PM asks to design an A/B test, validate a hypothesis, or plan conversion experiments with metrics and sample sizes.
What you get
An experiment spec with hypothesis, success metrics, sample-size estimate, guardrails, and rollout plan.
Files
Domain Context
This skill implements a proven product management framework. The approach combines best practices from industry leaders and is designed for practical application in day-to-day PM work.
Input Requirements
- Context about your product, feature, or problem
- Relevant data, research, or constraints (recommended but optional)
- Clear articulation of what you're trying to achieve
A/B Test Designer
When to Use
- Testing a new feature or design variation
- Validating a hypothesis before full rollout
- Optimizing conversion rates or key metrics
- Choosing between multiple design approaches
- Need to make a data-driven decision on a change
What This Skill Does
Helps you design rigorous A/B tests with clear hypotheses, success metrics, sample size calculations, and analysis plans.
Instructions
Help me design an A/B test for [feature/change]. Include:
1. Hypothesis
- Current situation and metrics
- Proposed change
- Expected impact and why
2. Test Design
- Primary success metric
- Secondary metrics
- Sample size needed
- Test duration
- User segments to include/exclude
3. Variants
- Control (A): current experience
- Variant (B): new experience
- Any additional variants (C, D, etc.)
4. Risks and Controls
- Potential negative impacts
- Guardrail metrics
- When to stop the test early
5. Analysis Plan
- Statistical significance threshold
- How to handle edge cases
- Decision criteria
Feature context: [Add context about the change you want to test]
Best Practices
- Start with a clear, falsifiable hypothesis
- Choose one primary metric to avoid multiple comparison issues
- Calculate sample size upfront based on expected effect size
- Run tests for full weekly cycles to account for day-of-week effects
- Set a minimum test duration (usually 1-2 weeks)
- Define success criteria before running the test
- Monitor guardrail metrics (revenue, errors, performance)
Example
Input: Testing new onboarding flow vs current 3-step process Output: Hypothesis (new 1-step flow will increase co...
Related skills
Grill MeHave an agent relentlessly pressure-test their plan or design until every decision branch is resolved and shared understanding is reached.691k196kGrill With DocsPressure-test architecture decisions against their live codebase and living documentation before writing production code.585k196k
Lark CalendarLet their coding agents read, create, and manage events inside Lark (Feishu) Calendar directly from natural language instructions.493kLark ImLet their coding agent send instant notifications, post updates, and receive commands directly inside Lark (Feishu) workspaces.493kLark ContactLet their AI coding agents read and write contacts, send messages, and manage user data inside Lark (Feishu).493kLark AttendanceAutomatically fetch and report daily attendance data from Lark (Feishu) into their own tools or agents.493kFAQ
What does ab-test-designer output?
ab-test-designer outputs a structured A/B experiment plan with hypothesis, success metrics, sample-size estimates, guardrail metrics, and a rollout plan ready for engineering implementation.
When should engineers invoke ab-test-designer?
Engineers should invoke ab-test-designer before building feature variants, when testing a new feature, validating a hypothesis, or optimizing conversion with measurable outcomes.