Design Of Experiments

Design of Experiments

Table of Contents

• Workflow
• Common Patterns
• Guardrails
• Quick Reference

Workflow

Copy this checklist and track your progress:

Design of Experiments Progress:
- [ ] Step 1: Define objectives and constraints
- [ ] Step 2: Identify factors, levels, and responses
- [ ] Step 3: Choose experimental design
- [ ] Step 4: Plan execution details
- [ ] Step 5: Create experiment plan document
- [ ] Step 6: Validate quality

Step 1: Define objectives and constraints

Clarify the experiment goal (screening vs optimization), response metric(s), experimental budget (max runs), time/cost constraints, and success criteria. See Common Patterns for typical objectives.

Step 2: Identify factors, levels, and responses

List all candidate factors (controllable inputs), specify levels for each factor (low/high or discrete values), categorize factors (control vs noise), and define response variables (measurable outputs). For screening many factors (8+), see resources/methodology.md for Plackett-Burman and fractional factorial approaches.

Step 3: Choose experimental design

Based on objective and constraints:

• For screening 5+ factors with limited runs → Use resources/methodology.md for fractional factorial or Plackett-Burman
• For optimizing 2-5 factors → Use resources/template.md for full or fractional factorial
• For response surface mapping → Use resources/methodology.md for central composite or Box-Behnken
• For robust design against noise → Use resources/methodology.md for parameter vs noise factor arrays

Step 4: Plan execution details

Specify randomization order (eliminate time trends), blocking strategy (control nuisance variables), replication plan (estimate error), sample size justification (power analysis), and measurement protocols. See Guardrails for critical requirements.

Step 5: Create experiment plan document

Create design-of-experiments.md with sections: objective, factors table, design matrix (run order with factor settings), response variables, execution protocol, and analysis plan. Use resources/template.md for structure.

Step 6: Validate quality

Self-assess using resources/evaluators/rubric_design_of_experiments.json. Check: objective clarity, factor completeness, design appropriateness, randomization plan, measurement protocol, statistical power, analysis plan, and deliverable quality. Minimum standard: Average score ≥ 3.5 before delivering.

Common Patterns

Pattern 1: Screening (many factors → vital few)

• Context: 10-30 candidate factors, limited budget, want to identify 3-5 critical factors
• Approach: Plackett-Burman or fractional factorial (Resolution III/IV)
• Output: Pareto chart of effect sizes, shortlist for follow-up optimization
• Example: Software performance tuning with 15 configuration parameters

Pattern 2: Optimization (find best settings)

• Context: 2-5 factors already identified as important, want to find optimal levels
• Approach: Full factorial (2^k) or fractional factorial + steepest ascent
• Output: Main effects plot, interaction plots, recommended settings
• Example: Manufacturing process with temperature, pressure, time factors

Pattern 3: Response Surface (map the landscape)

• Context: Need to understand curvature, find maximum/minimum, quantify tradeoffs
• Approach: Central Composite Design (CCD) or Box-Behnken
• Output: Response surface equation, contour plots, optimal region
• Example: Chemical formulation with ingredient ratios

Pattern 4: Robust Design (work despite noise)

• Context: Product/process must perform well despite uncontrollable variation
• Approach: Taguchi inner-outer array (control × noise factors)
• Output: Settings that minimize sensitivity to noise factors
• Example: Consumer product that must work across temperature/humidity ranges

Pattern 5: Sequential Experimentation (learn then refine)

• Context: High uncertainty, want to learn iteratively with minimal waste
• Approach: Screening → Steepest ascent → Response surface → Confirmation
• Output: Progressively refined understanding and settings
• Example: New product development with unknown factor relationships

Guardrails

Design requirements:

1. Randomize run order: Eliminates time-order bias and confounding with lurking variables. Use random number generator, not "convenient" sequences.

2. Replicate center points: For designs with continuous factors, replicate center point runs (3-5 times) to estimate pure error and detect curvature.

3. Preserve critical interactions: In fractional factorials, avoid confounding important 2-way interactions with main effects. Choose Resolution IV or higher if interactions matter.

4. Check design balance: Ensure orthogonality (factors are uncorrelated in design matrix). Correlation > 0.3 reduces precision and interpretability.

5. Define response precisely: Use objective, quantitative, repeatable measurements. Avoid subjective scoring unless calibrated with multiple raters.

6. Justify sample size: Run power analysis to ensure design can detect meaningful effect sizes with acceptable Type II error risk (beta at most 0.20).

7. Document assumptions: State expected effect magnitudes, interaction assumptions, noise variance estimates. Design validity depends on these.

8. Plan for analysis before running: Specify statistical tests, significance level (alpha), effect size metrics before data collection to prevent p-hacking.

Common pitfalls:

• ❌ One-factor-at-a-time (OFAT): Misses interactions, requires more runs than factorial designs
• ❌ Ignoring blocking: If runs span days/batches/operators, block accordingly or confound results with time trends
• ❌ Too many levels: Use 2-3 levels initially. More levels increase runs exponentially.
• ❌ Unmeasured factors: If an important factor isn't controlled/measured, it becomes noise
• ❌ Changing protocols mid-experiment: Breaks design structure. If necessary, restart or analyze separately.

Quick Reference

Key resources:

• [resources/template.md](resources/template.md): Quick-start templates for common designs (factorial, screening, response surface)
• [resources/methodology.md](resources/methodology.md): Advanced techniques (optimal designs, Taguchi, mixture experiments, sequential strategies)
• [resources/evaluators/rubric_design_of_experiments.json](resources/evaluators/rubric_design_of_experiments.json): Quality criteria for experiment plans

Typical workflow time:

• Simple factorial (2-4 factors): 15-30 minutes
• Screening design (8+ factors): 30-45 minutes
• Response surface design: 45-60 minutes
• Robust design (Taguchi): 60-90 minutes

When to escalate:

• User needs mixture experiments (factors must sum to 100%)
• Split-plot designs required (hard-to-change factors)
• Optimal designs for irregular constraints
• Bayesian adaptive designs

→ Use resources/methodology.md for these advanced cases

Inputs required:

• Process/System: What you're experimenting on
• Factors: List of controllable inputs with candidate levels
• Responses: Measurable outputs (KPIs, metrics)
• Constraints: Budget (max runs), time, resources
• Objective: Screening, optimization, response surface, or robust design

Outputs produced:

• design-of-experiments.md: Complete experiment plan with design matrix, randomization, protocols, analysis approach

Design of Experiments - Advanced Methodology

Workflow

Copy this checklist for advanced DOE cases:

Advanced DOE Progress:
- [ ] Step 1: Assess complexity and choose advanced technique
- [ ] Step 2: Design experiment using specialized method
- [ ] Step 3: Plan execution with advanced considerations
- [ ] Step 4: Analyze with appropriate statistical methods
- [ ] Step 5: Iterate or confirm findings

Step 1: Assess complexity

Identify which advanced technique applies: screening 8+ factors, response surface with curvature, robust design, mixture constraints, hard-to-change factors, or irregular factor space. See technique selection criteria below.

Step 2: Design experiment

Apply specialized design method from sections: Screening Designs, Response Surface Methodology, Taguchi Methods, Optimal Designs, Mixture Experiments, or Split-Plot Designs.

Step 3: Plan execution

Address advanced considerations: blocking for nuisance variables, replication for variance estimation, center points for curvature detection, and sequential strategies. See Sequential Experimentation for iterative approaches.

Step 4: Analyze

Use appropriate analysis for design type: effect estimation, ANOVA, regression modeling, response surface equations, contour plots, and residual diagnostics. See Analysis Techniques.

Step 5: Iterate or confirm

Based on findings, run confirmation experiments, refine factor ranges, add center/axial points for RSM, or screen additional factors. See Sequential Experimentation.

---

1. Screening Designs

When to use: 8-30 candidate factors, limited experimental budget, goal is to identify 3-5 vital factors for follow-up optimization.

Plackett-Burman Designs

Structure: Orthogonal designs with runs = multiple of 4. Screens k factors in k+1 runs.

Standard designs:

• 12 runs → screen up to 11 factors
• 16 runs → screen up to 15 factors
• 20 runs → screen up to 19 factors
• 24 runs → screen up to 23 factors

Example: 12-run Plackett-Burman generator matrix:

Run 1: + + - + + + - - - + -
Subsequent runs: Cycle previous row left, last run is all minus

Analysis: Fit linear model Y = β₀ + β₁X₁ + β₂X₂ + ... + βₖXₖ. Rank factors by |βᵢ|. Select top 3-5 for optimization. Pareto chart: cumulative % variance explained.

Limitation: Main effects confounded with 2-way interactions. Only valid if interactions negligible (sparsity-of-effects principle).

Fractional Factorial Screening

When to use: 5-8 factors, need to estimate some 2-way interactions, Resolution IV or V required.

Common designs:

• 2⁵⁻¹ (Resolution V): 16 runs, 5 factors. Main effects and 2-way interactions clear. Generator: I = ABCDE.
• 2⁶⁻² (Resolution IV): 16 runs, 6 factors. Main effects clear, 2-way confounded with 2-way. Generators: I = ABCE, I = BCDF.
• 2⁷⁻³ (Resolution IV): 16 runs, 7 factors. Generators: I = ABCD, I = ABEF, I = ACEG.

Confounding analysis: Use defining relation to determine alias structure. Example for 2⁵⁻¹ with I = ABCDE:

• A aliased with BCDE
• AB aliased with CDE
• ABC aliased with DE

Fold-over technique: If screening reveals ambiguous confounding, run fold-over design (flip all signs) to de-alias. 16 runs + 16 fold-over = 32 runs = full 2⁵ factorial.

---

2. Response Surface Methodology

When to use: 2-5 factors already identified as important, need to find optimum, expect curvature (quadratic relationship).

Central Composite Design (CCD)

Structure: Factorial points + axial points + center points

Components:

• Factorial points: 2^k corner points (±1 for all factors)
• Axial points: 2k points on axes (±α, 0, 0, ...) where α determines rotatability
• Center points: 3-5 replicates at origin (0, 0, ..., 0)

Total runs: 2^k + 2k + nc (nc = number of center points)

Example: CCD for 3 factors (8 + 6 + 5 = 19 runs):

Rotatability: Choose α = (2^k)^(1/4) for equal prediction variance at equal distance from center. For 3 factors: α = 1.682.

Model: Fit quadratic: Y = β₀ + Σβᵢxᵢ + Σβᵢᵢxᵢ² + Σβᵢⱼxᵢxⱼ

Analysis: Canonical analysis to find stationary point (maximum, minimum, or saddle). Ridge analysis if optimum outside design region.

Box-Behnken Design

Structure: 3-level design that avoids extreme corners (all factors at ±1 simultaneously).

Advantages: Fewer runs than CCD, safer when extreme combinations may damage equipment or produce out-of-spec product.

Example: Box-Behnken for 3 factors (12 + 3 = 15 runs):

Model: Same quadratic as CCD.

Trade-off: Slightly less efficient than CCD for prediction, but avoids extreme points.

---

3. Taguchi Methods (Robust Parameter Design)

When to use: Product/process must perform well despite uncontrollable variation (noise factors). Goal: Find control factor settings that minimize sensitivity to noise.

Inner-Outer Array Structure

Inner array: Control factors (factors you can set in production) Outer array: Noise factors (environmental conditions, material variation, user variation)

Approach: Cross inner array with outer array. Each inner array run is repeated at all outer array conditions.

Example: L₈ inner × L₄ outer (8 control combinations × 4 noise conditions = 32 runs):

Inner array (control factors A, B, C):

Outer array (noise factors N₁, N₂):

Data collection: For each inner run, measure response Y at all 4 noise conditions. Calculate mean (Ȳ) and variance (s²) or signal-to-noise ratio (SNR).

Signal-to-Noise Ratios:

• Larger-is-better: SNR = -10 log₁₀(Σ(1/Y²)/n)
• Smaller-is-better: SNR = -10 log₁₀(ΣY²/n)
• Target-is-best: SNR = 10 log₁₀(Ȳ²/s²)

Analysis: Choose control factor settings that maximize SNR (robust to noise) while achieving target mean.

Two-step optimization: 1. Maximize SNR to reduce variability 2. Adjust mean to target using control factors that don't affect SNR

---

4. Optimal Designs

When to use: Irregular factor space (constraints, categorical factors, unequal ranges), custom run budget, or standard designs don't fit.

D-Optimal Designs

Criterion: Minimize determinant of (X'X)⁻¹ (maximize information, minimize variance of coefficient estimates).

Algorithm: Computer-generated. Start with candidate set of all feasible runs, select subset that maximizes |X'X|.

Use cases:

• Mixture experiments with additional process variables
• Constrained factor spaces (e.g., temperature + pressure can't both be high)
• Irregular grids (e.g., existing data points + new runs)
• Unequal factor ranges

Software: Use R (AlgDesign package), JMP, Design-Expert, or Python (pyDOE).

A-Optimal and G-Optimal

A-optimal: Minimize average variance of predictions (trace of (X'X)⁻¹). G-optimal: Minimize maximum variance across design space (minimax criterion).

Choice: D-optimal for parameter estimation, G-optimal for prediction across entire space, A-optimal for average prediction quality.

---

5. Mixture Experiments

When to use: Factors are proportions that must sum to 100% (e.g., chemical formulations, blend compositions, budget allocations).

Simplex-Lattice Designs

Constraints: x₁ + x₂ + ... + xₖ = 1, all xᵢ ≥ 0

{q,k} designs: q = number of levels for each component, k = number of components

Example: {2,3} simplex-lattice (3 components at 0%, 50%, 100%):

Model: Scheffé canonical polynomials. Linear: Y = β₁x₁ + β₂x₂ + β₃x₃. Quadratic: Y = Σβᵢxᵢ + Σβᵢⱼxᵢxⱼ.

Simplex-Centroid Designs

Structure: Pure components + binary blends + ternary blends + overall centroid.

Example: 3-component simplex-centroid (7 runs):

Constraints: Add lower/upper bounds if components have minimum/maximum limits. Use D-optimal design for constrained mixture space.

---

6. Split-Plot Designs

When to use: Some factors are hard to change (e.g., temperature requires hours to stabilize), others are easy to change. Randomizing all factors fully is impractical or expensive.

Structure

Whole-plot factors: Hard to change (temperature, batch, supplier) Subplot factors: Easy to change (concentration, time, operator)

Design: Randomize whole-plot factors at top level, randomize subplot factors within each whole-plot level.

Example: 2² split-plot (Temperature = whole-plot, Time = subplot, 2 replicates):

Analysis: Mixed model with whole-plot error and subplot error terms. Whole-plot factors tested with lower precision (fewer degrees of freedom).

Trade-off: Allows practical execution when full randomization impossible, but reduces statistical power for hard-to-change factors.

---

7. Sequential Experimentation

Philosophy: Learn iteratively, adapt design based on results. Minimize total runs while maximizing information.

Stage 1: Screening

Objective: Reduce 10-20 candidates to 3-5 critical factors. Design: Plackett-Burman or 2^(k-p) fractional factorial (Resolution III-IV). Runs: 12-20. Output: Ranked factor list, effect sizes with uncertainty.

Stage 2: Steepest Ascent/Descent

Objective: Move quickly toward optimal region using main effects from screening. Method: Calculate path of steepest ascent (gradient = effect estimates). Run experiments along this path until response stops improving. Example: If screening finds temp effect = +10, pressure effect = +5, move in direction (temp: +2, pressure: +1).

Stage 3: Factorial Optimization

Objective: Explore region around best settings from steepest ascent, estimate interactions. Design: 2^k full factorial or Resolution V fractional factorial with 3-5 factors. Runs: 16-32. Output: Optimal settings, interaction effects, linear model.

Stage 4: Response Surface Refinement

Objective: Fit curvature, find true optimum. Design: CCD or Box-Behnken centered at best settings from Stage 3. Runs: 15-20. Output: Quadratic model, stationary point (optimum), contour plots.

Stage 5: Confirmation

Objective: Validate predicted optimum. Design: 3-5 replication runs at predicted optimal settings. Output: Confidence interval for response at optimum. If prediction interval contains observed mean, model validated.

Total runs example: Screening (16) + Steepest ascent (4) + Factorial (16) + RSM (15) + Confirmation (3) = 54 runs. Compare to one-shot full factorial for 10 factors = 1024 runs.

---

8. Analysis Techniques

Effect Estimation

Factorial designs: Estimate main effect of factor A as: Effect(A) = (Ȳ₊ - Ȳ₋) where Ȳ₊ = mean response when A is high, Ȳ₋ = mean when A is low.

Interaction effect: Effect(AB) = [(Ȳ₊₊ + Ȳ₋₋) - (Ȳ₊₋ + Ȳ₋₊)] / 2

Standard error: SE(effect) = 2σ/√n, where σ estimated from replicates or center points.

ANOVA

Purpose: Test statistical significance of effects. Null hypothesis: Effect = 0. Test statistic: F = MS(effect) / MS(error), compare to F-distribution. Significance: p < 0.05 (or chosen α level) → reject H₀, effect is significant.

Regression Modeling

Linear model: Y = β₀ + β₁x₁ + β₂x₂ + β₁₂x₁x₂ + ε Quadratic model (RSM): Y = β₀ + Σβᵢxᵢ + Σβᵢᵢxᵢ² + Σβᵢⱼxᵢxⱼ + ε

Fit: Least squares (minimize Σ(Yᵢ - Ŷᵢ)²). Assessment: R², adjusted R², RMSE, residual plots.

Residual Diagnostics

Check assumptions: 1. Normal probability plot: Residuals should fall on straight line. Non-normality indicates transformation needed. 2. Residuals vs fitted: Random scatter around zero. Funnel shape indicates non-constant variance (transform Y). 3. Residuals vs run order: Random. Trend indicates time drift, lack of randomization. 4. Residuals vs factors: Random. Pattern indicates missing interaction or curvature.

Transformations: Log(Y) for multiplicative effects, √Y for count data, 1/Y for rate data, Box-Cox for data-driven choice.

Optimization

Contour plots: Visualize response surface, identify optimal region, assess tradeoffs. Desirability functions: Multi-response optimization. Convert each response to 0-1 scale (0 = unacceptable, 1 = ideal). Maximize geometric mean of desirabilities. Canonical analysis: Find stationary point (∂Y/∂xᵢ = 0), classify as maximum, minimum, or saddle point based on eigenvalues of Hessian matrix.

---

9. Sample Size and Power Analysis

Before designing experiment, determine required runs:

Power: Probability of detecting true effect if it exists (1 - β). Standard: power ≥ 0.80.

Effect size (δ): Minimum meaningful difference. Example: "Must detect 10% yield improvement."

Noise (σ): Process variability. Estimate from historical data, pilot runs, or engineering judgment.

Formula for factorial designs: n ≥ 2(Zα/2 + Zβ)²σ² / δ² per cell.

Example: Detect δ = 5 units, σ = 3 units, α = 0.05, power = 0.80.

• n ≥ 2(1.96 + 0.84)²(3²) / 5² = 2(7.84)(9) / 25 ≈ 6 replicates per factor level combination.

For screening: Use effect sparsity assumption. If testing 10 factors, expect 2-3 active. Size design to detect large effects (1-2σ).

Software: Use G*Power, R (pwr package), JMP, or online calculators.

---

10. Common Pitfalls and Solutions

Pitfall 1: Ignoring confounding in screening designs

• Problem: Plackett-Burman confounds main effects with 2-way interactions. If interactions exist, main effect estimates are biased.
• Solution: Use only when sparsity-of-effects applies (most interactions negligible). Follow up ambiguous results with Resolution IV/V design or fold-over.

Pitfall 2: Extrapolating beyond design region

• Problem: Response surface models are local approximations. Predicting outside tested factor ranges is unreliable.
• Solution: Expand design if optimum appears outside current region. Run steepest ascent, then new RSM centered on improved region.

Pitfall 3: Inadequate replication

• Problem: Without replicates, cannot estimate pure error or test lack-of-fit.
• Solution: Always include 3-5 center point replicates. For critical experiments, replicate entire design (2-3 times).

Pitfall 4: Changing protocols mid-experiment

• Problem: Breaks orthogonality, confounds design structure with time.
• Solution: Complete design as planned. If protocol change necessary, analyze before/after separately or treat as blocking factor.

Pitfall 5: Treating categorical factors as continuous

• Problem: Assigning arbitrary numeric codes (-1, 0, +1) to unordered categories (e.g., Supplier A/B/C) implies ordering that doesn't exist.
• Solution: Use indicator variables (dummy coding) or separate experiments for each category level.

Design of Experiments - Template

Workflow

Copy this checklist and track your progress:

DOE Template Progress:
- [ ] Step 1: Define experiment objective
- [ ] Step 2: List factors and levels
- [ ] Step 3: Select design type
- [ ] Step 4: Generate design matrix
- [ ] Step 5: Randomize and document protocol
- [ ] Step 6: Finalize experiment plan

Step 1: Define experiment objective

Specify what you're trying to learn (screening, optimization, response surface, robust design), primary response metric(s), and success criteria. See Objective Definition for examples.

Step 2: List factors and levels

Identify all factors (controllable inputs), specify levels for each (2-3 initially), distinguish control vs noise factors, and define measurable responses. See Factor Table Template for structure.

Step 3: Select design type

Based on objective:

• 2-5 factors, want all combinations → Full Factorial
• 5+ factors, limited runs → Fractional Factorial
• Screening 8+ factors → Plackett-Burman

Step 4: Generate design matrix

Create run-by-run table with factor settings for each experimental run. See Design Matrix Examples for format.

Step 5: Randomize and document protocol

Randomize run order, specify blocking if needed, detail measurement procedures, and plan replication strategy. See Execution Details for guidance.

Step 6: Finalize experiment plan

Create complete design-of-experiments.md document using Document Structure Template. Self-check with quality criteria in Quality Checklist.

---

Document Structure Template

Use this structure for the final design-of-experiments.md file:

# Design of Experiments: [Experiment Name]

## 1. Objective

**Goal**: [Screening | Optimization | Response Surface | Robust Design]

**Context**: [1-2 sentences describing the system/process being studied]

**Success Criteria**: [What constitutes a successful experiment? Measurable outcomes.]

**Constraints**:
- Budget: [Maximum number of runs allowed]
- Time: [Deadline or duration per run]
- Resources: [Equipment, personnel, materials]

## 2. Factors and Levels

| Factor | Type | Low Level (-1) | High Level (+1) | Center (0) | Units | Rationale |
|--------|------|----------------|-----------------|------------|-------|-----------|
| A: [Name] | Control | [value] | [value] | [value] | [units] | [Why this factor?] |
| B: [Name] | Control | [value] | [value] | [value] | [units] | [Why this factor?] |
| C: [Name] | Noise | [value] | [value] | - | [units] | [Uncontrollable variation] |

**Factor Selection Rationale**: [Why these factors? Any excluded? Assumptions?]

## 3. Response Variables

| Response | Description | Measurement Method | Target | Units |
|----------|-------------|-------------------|---------|-------|
| Y1: [Name] | [What it measures] | [How measured] | [Maximize/Minimize/Target value] | [units] |
| Y2: [Name] | [What it measures] | [How measured] | [Maximize/Minimize/Target value] | [units] |

**Response Selection Rationale**: [Why these responses? Any tradeoffs?]

## 4. Experimental Design

**Design Type**: [Full Factorial 2^k | Fractional Factorial 2^(k-p) | Plackett-Burman | Central Composite | Box-Behnken]

**Resolution**: [For fractional factorials: III, IV, or V]

**Runs**:
- Design points: [number]
- Center points: [number of replicates at center]
- Total runs: [design + center]

**Design Rationale**: [Why this design? What can/can't it detect?]

## 5. Design Matrix

| Run | Order | Block | A | B | C | Y1 | Y2 | Notes |
|-----|-------|-------|---|---|---|----|----|-------|
| 1 | 5 | 1 | -1 | -1 | -1 | | | |
| 2 | 12 | 1 | +1 | -1 | -1 | | | |
| 3 | 3 | 1 | -1 | +1 | -1 | | | |
| 4 | 8 | 1 | +1 | +1 | -1 | | | |
| 5 | 1 | 2 | -1 | -1 | +1 | | | |
| ... | ... | ... | ... | ... | ... | | | |

**Randomization**: Run order randomized using [method]. Original design point order preserved in "Run" column.

**Blocking**: [If used] Runs blocked by [day/batch/operator/etc.] to control for [nuisance variable].

## 6. Execution Protocol

**Preparation**:
- [ ] [Equipment setup/calibration steps]
- [ ] [Material preparation]
- [ ] [Personnel training]

**Run Procedure**:
1. [Step-by-step protocol for each run]
2. [Factor settings to apply]
3. [Wait/equilibration time]
4. [Response measurement procedure]
5. [Recording method]

**Quality Controls**:
- [Measurement calibration checks]
- [Process stability verification]
- [Outlier detection procedure]

**Timeline**: [Start date, duration per run, expected completion]

## 7. Analysis Plan

**Primary Analysis**:
- Calculate main effects for factors A, B, C
- Calculate 2-way interaction effects (AB, AC, BC)
- Fit linear model: Y = β0 + β1·A + β2·B + β3·C + β12·AB + ...
- ANOVA to test significance (α = 0.05)
- Residual diagnostics (normality, constant variance, independence)

**Graphical Analysis**:
- Main effects plot
- Interaction plot
- Pareto chart of standardized effects
- Residual plots (normal probability, vs fitted, vs order)

**Decision Criteria**:
- Effects significant at p < 0.05 are considered important
- Interaction present if p(interaction) < 0.05
- Optimal settings chosen to [maximize/minimize] Y1 while [constraint on Y2]

**Follow-up**:
- If curvature detected → Run [response surface design]
- If additional factors identified → Run [screening design]
- Confirmation runs: [Number] at predicted optimum settings

## 8. Assumptions and Limitations

**Assumptions**:
- [Linear relationship between factors and response]
- [No strong higher-order interactions]
- [Homogeneous variance across factor space]
- [Errors are independent and normally distributed]
- [Process is stable during experiment]

**Limitations**:
- [Design resolution limits – e.g., 2-way interactions confounded]
- [Factor range restrictions]
- [Measurement precision limits]
- [External validity – generalization beyond tested region]

**Risks**:
- [What could invalidate results?]
- [Mitigation strategies]

## 9. Expected Outcomes

**If screening design**:
- Pareto chart identifying 3-5 critical factors from [N] candidates
- Effect size estimates with confidence intervals
- Shortlist for follow-up optimization experiment

**If optimization design**:
- Optimal factor settings: A = [value], B = [value], C = [value]
- Predicted response at optimum: Y1 = [value] ± [CI]
- Interaction insights: [Which factors interact? How?]

**If response surface**:
- Response surface equation: Y = [polynomial model]
- Contour/surface plots showing optimal region
- Sensitivity analysis showing robustness

**Deliverables**:
- This experiment plan document
- Completed design matrix with results (after execution)
- Analysis report with plots and recommendations

---

Objective Definition

Screening: Screen 12 software config parameters to identify 3-5 affecting API response time. Success: Reduce candidates 60%+. Constraint: Max 16 runs.

Optimization: Optimize injection molding (temp, pressure, time) to minimize defect rate while cycle time < 45s. Success: < 2% defects (currently 8%). Constraint: Max 20 runs, 2 days.

Response Surface: Map yield vs temperature/pH, find maximum, model curvature. Success: R² > 0.90, optimal region. Constraint: Max 15 runs.

---

Factor Table Template

Type definitions:

• Control: Factors you can set deliberately in the experiment
• Noise: Factors that vary but can't be controlled (for robust design)
• Held constant: Factors fixed at one level (not in design)

Level selection guidance:

• 2 levels: Start here for screening/optimization. Detects linear effects and interactions.
• 3 levels: Add center point to detect curvature. Required for response surface designs.
• Categorical: Use coded values (-1, +1) for categories (e.g., Supplier A = -1, Supplier B = +1)

---

Full Factorial Designs

When to use: 2-5 factors, want to estimate all main effects and interactions, budget allows 2^k runs.

Design structure: Test all combinations of factor levels.

Example: 2³ factorial (3 factors, 2 levels each = 8 runs)

Advantages:

• Estimates all main effects and 2-way/3-way interactions
• No confounding
• Maximum precision for given number of factors

Limitations:

• Runs grow exponentially: 2³ = 8, 2⁴ = 16, 2⁵ = 32, 2⁶ = 64
• Inefficient for screening (wastes runs on unimportant factors)

Add center points: Replicate 3-5 runs at center (0, 0, 0) to detect curvature and estimate pure error.

---

Fractional Factorial Designs

When to use: 5+ factors, limited budget, willing to sacrifice some interaction information.

Design structure: Test a fraction (1/2, 1/4, 1/8) of full factorial, deliberately confounding higher-order interactions.

Example: 2⁵⁻¹ design (5 factors, 16 runs instead of 32)

Resolution IV: Main effects clear, 2-way interactions confounded with each other.

Generator: E = ABCD (defining relation: I = ABCDE)

Confounding structure:

• A confounded with BCDE
• AB confounded with CDE
• ABC confounded with DE

Resolution levels:

• Resolution III: Main effects confounded with 2-way interactions. Use for screening only.
• Resolution IV: Main effects clear, 2-way confounded with 2-way. Good for screening + some optimization.
• Resolution V: Main effects and 2-way clear, 2-way confounded with 3-way. Preferred for optimization.

Choosing fraction: Use standard designs (tables available) or design software to ensure desired resolution.

---

Plackett-Burman Screening

When to use: Screen 8-15 factors with minimal runs, only care about main effects.

Design structure: Orthogonal design with runs = next multiple of 4 above number of factors.

Example: 12-run Plackett-Burman for up to 11 factors

Advantages:

• Very efficient: Screen 11 factors in 12 runs (vs 2048 for full factorial)
• Main effects estimated independently

Limitations:

• 2-way interactions completely confounded with main effects
• Only use when interactions unlikely or unimportant
• Cannot estimate interactions

Use case: Early-stage screening to reduce 15 candidates to 4-5 for follow-up factorial design.

---

Design Matrix Examples

Format: Each row = one run. Columns: Run (design point #), Order (randomized sequence), Block (if used), Factors (coded -1/0/+1 or actual values), Responses (blank until execution), Notes (observations).

---

Execution Details

Randomization: Eliminates bias from time trends/drift. Method: (1) List runs, (2) Assign random numbers, (3) Sort by random number = execution order, (4) Document both orders. Exception: Don't randomize hard-to-change factors (use split-plot design, see methodology.md).

Blocking: Use when runs span days/batches/operators. Method: Divide into 2-4 balanced blocks, randomize within each, analyze with block as factor. Example: 16 runs over 2 days → 2 blocks of 8.

Replication: True replication (repeat entire run), repeated measures (multiple measurements per run), or center points (3-5 replicates at center for pure error). Guidance: Always include 3-5 center points for continuous factors.

---

Quality Checklist

Before finalizing the experiment plan, verify:

Objective & Scope:

• [ ] Goal clearly stated (screening | optimization | response surface | robust)
• [ ] Success criteria are measurable and realistic
• [ ] Constraints documented (runs, time, cost)

Factors:

• [ ] All important factors included
• [ ] Levels span meaningful range (not too narrow, not outside feasible region)
• [ ] Factor types identified (control vs noise)
• [ ] Rationale for each factor documented

Responses:

• [ ] Responses are objective and quantitative
• [ ] Measurement method specified and validated
• [ ] Target direction clear (maximize | minimize | hit target)

Design:

• [ ] Design type appropriate for objective and budget
• [ ] Design resolution adequate (e.g., Resolution IV+ if interactions matter)
• [ ] Run count justified (power analysis or practical limit)
• [ ] Design matrix correct (orthogonal, balanced)

Execution:

• [ ] Randomization method specified
• [ ] Blocking used if runs span nuisance variable levels
• [ ] Replication plan documented (center points, full replicates)
• [ ] Protocol detailed enough for independent execution
• [ ] Timeline realistic

Analysis:

• [ ] Analysis plan specified before data collection
• [ ] Significance level (α) stated
• [ ] Decision criteria clear
• [ ] Residual diagnostics planned
• [ ] Follow-up strategy identified

Assumptions & Risks:

• [ ] Key assumptions stated explicitly
• [ ] Limitations acknowledged (resolution, range, measurement)
• [ ] Risks identified with mitigation plans