Pair Trade Screener

Pair Trade Screener

Overview

This skill identifies and analyzes statistical arbitrage opportunities through pair trading. Pair trading is a market-neutral strategy that profits from the relative price movements of two correlated securities, regardless of overall market direction. The skill uses rigorous statistical methods including correlation analysis and cointegration testing to find robust trading pairs.

Core Methodology:

• Identify pairs of stocks with high correlation and similar sector/industry exposure
• Test for cointegration (long-term statistical relationship)
• Calculate spread z-scores to identify mean-reversion opportunities
• Generate entry/exit signals based on statistical thresholds
• Provide position sizing for market-neutral exposure

Key Advantages:

• Market-neutral: Profits in up, down, or sideways markets
• Risk management: Limited exposure to broad market movements
• Statistical foundation: Data-driven, not discretionary
• Diversification: Uncorrelated to traditional long-only strategies

When to Use This Skill

Use this skill when:

• User asks for "pair trading opportunities"
• User wants "market-neutral strategies"
• User requests "statistical arbitrage screening"
• User asks "which stocks move together?"
• User wants to hedge sector exposure
• User requests mean-reversion trade ideas
• User asks about relative value trading

Example user requests:

• "Find pair trading opportunities in the tech sector"
• "Which stocks are cointegrated?"
• "Screen for statistical arbitrage opportunities"
• "Find mean-reversion pairs"
• "What are good market-neutral trades right now?"

Analysis Workflow

Step 1: Define Pair Universe

Objective: Establish the pool of stocks to analyze for pair relationships.

Option A: Sector-Based Screening (Recommended)

Select a specific sector to screen:

• Technology
• Financials
• Healthcare
• Consumer Discretionary
• Industrials
• Energy
• Materials
• Consumer Staples
• Utilities
• Real Estate
• Communication Services

Option B: Custom Stock List

User provides specific tickers to analyze:

Example: ["AAPL", "MSFT", "GOOGL", "META", "NVDA"]

Option C: Industry-Specific

Narrow focus to specific industry within sector:

• Example: "Software" within Technology sector
• Example: "Regional Banks" within Financials

Filtering Criteria:

• Minimum market cap: $2B (mid-cap and above)
• Minimum average volume: 1M shares/day (liquidity requirement)
• Active trading: No delisted or inactive stocks
• Same exchange preference: Avoid cross-exchange complications

Step 2: Retrieve Historical Price Data

Objective: Fetch price history for correlation and cointegration analysis.

Data Requirements:

• Timeframe: 2 years (minimum 252 trading days)
• Frequency: Daily closing prices
• Adjustments: Adjusted for splits and dividends
• Clean data: No gaps or missing values

FMP API Endpoint:

GET /v3/historical-price-full/{symbol}?apikey=YOUR_API_KEY

Data Validation:

• Verify consistent date ranges across all symbols
• Remove stocks with >10% missing data
• Fill minor gaps with forward-fill method
• Log data quality issues

Script Execution:

python scripts/fetch_price_data.py --sector Technology --lookback 730

Step 3: Calculate Correlation and Beta

Objective: Identify candidate pairs with strong linear relationships.

Correlation Analysis:

For each pair of stocks (i, j) in the universe: 1. Calculate Pearson correlation coefficient (ρ) 2. Calculate rolling correlation (90-day window) for stability check 3. Filter pairs with ρ >= 0.70 (strong positive correlation)

Correlation Interpretation:

• ρ >= 0.90: Very strong correlation (best candidates)
• ρ 0.70-0.90: Strong correlation (good candidates)
• ρ 0.50-0.70: Moderate correlation (marginal)
• ρ < 0.50: Weak correlation (exclude)

Beta Calculation:

For each candidate pair (Stock A, Stock B):

Beta = Covariance(A, B) / Variance(B)

Beta indicates the hedge ratio:

• Beta = 1.0: Equal dollar amounts
• Beta = 1.5: $1.50 of B for every $1.00 of A
• Beta = 0.8: $0.80 of B for every $1.00 of A

Correlation Stability Check:

• Calculate correlation over multiple periods (6mo, 1yr, 2yr)
• Require correlation to be stable (not deteriorating)
• Flag pairs where recent correlation < historical correlation by >0.15

Step 4: Cointegration Testing

Objective: Statistically validate long-term equilibrium relationship.

Why Cointegration Matters:

• Correlation measures short-term co-movement
• Cointegration proves long-term equilibrium relationship
• Cointegrated pairs mean-revert predictably
• Non-cointegrated pairs may diverge permanently

Augmented Dickey-Fuller (ADF) Test:

For each correlated pair: 1. Calculate spread: Spread = Price_A - (Beta × Price_B) 2. Run ADF test on spread series 3. Check p-value: p < 0.05 indicates cointegration (reject null hypothesis of unit root) 4. Extract ADF statistic for strength ranking

Cointegration Interpretation:

• p-value < 0.01: Very strong cointegration (★★★)
• p-value 0.01-0.05: Moderate cointegration (★★)
• p-value > 0.05: No cointegration (exclude)

Half-Life Calculation:

Estimate mean-reversion speed:

Half-Life = -log(2) / log(mean_reversion_coefficient)

• Half-life < 30 days: Fast mean-reversion (good for short-term trading)
• Half-life 30-60 days: Moderate speed (standard)
• Half-life > 60 days: Slow mean-reversion (long holding periods)

Python Implementation:

from statsmodels.tsa.stattools import adfuller

# Calculate spread
spread = price_a - (beta * price_b)

# ADF test
result = adfuller(spread)
adf_stat = result[0]
p_value = result[1]

# Interpret
is_cointegrated = p_value < 0.05

Step 5: Spread Analysis and Z-Score Calculation

Objective: Quantify current spread deviation from equilibrium.

Spread Calculation:

Two common methods:

Method 1: Price Difference (Additive)

Spread = Price_A - (Beta × Price_B)

Best for: Stocks with similar price levels

Method 2: Price Ratio (Multiplicative)

Spread = Price_A / Price_B

Best for: Stocks with different price levels, easier interpretation

Z-Score Calculation:

Measures how many standard deviations spread is from its mean:

Z-Score = (Current_Spread - Mean_Spread) / Std_Dev_Spread

Z-Score Interpretation:

• Z > +2.0: Stock A expensive relative to B (short A, long B)
• Z > +1.5: Moderately expensive (watch for entry)
• Z -1.5 to +1.5: Normal range (no trade)
• Z < -1.5: Moderately cheap (watch for entry)
• Z < -2.0: Stock A cheap relative to B (long A, short B)

Historical Spread Analysis:

• Calculate mean and std dev over 90-day rolling window
• Plot historical z-score distribution
• Identify maximum historical z-score deviations
• Check for structural breaks (spread regime change)

Step 6: Generate Entry/Exit Recommendations

Objective: Provide actionable trading signals with clear rules.

Entry Conditions:

Conservative Approach (Z ≥ ±2.0):

LONG Signal:
- Z-score < -2.0 (spread 2+ std devs below mean)
- Spread is mean-reverting (cointegration p < 0.05)
- Half-life < 60 days
→ Action: Buy Stock A, Short Stock B (hedge ratio = beta)

SHORT Signal:
- Z-score > +2.0 (spread 2+ std devs above mean)
- Spread is mean-reverting (cointegration p < 0.05)
- Half-life < 60 days
→ Action: Short Stock A, Buy Stock B (hedge ratio = beta)

Aggressive Approach (Z ≥ ±1.5):

• Lower threshold for more frequent trades
• Higher win rate but smaller avg profit per trade
• Requires tighter risk management

Exit Conditions:

Primary Exit: Mean Reversion (Z = 0)

Exit when spread returns to mean (z-score crosses 0)
→ Close both legs simultaneously

Secondary Exit: Partial Profit Take

Exit 50% when z-score reaches ±1.0
Exit remaining 50% at z-score = 0

Stop Loss:

Exit if z-score extends beyond ±3.0 (extreme divergence)
Risk: Possible structural break in relationship

Time-Based Exit:

Exit after 90 days if no mean-reversion
Prevents holding broken pairs indefinitely

Step 7: Position Sizing and Risk Management

Objective: Determine dollar amounts for market-neutral exposure.

Market Neutral Sizing:

For a pair (Stock A, Stock B) with beta = β:

Equal Dollar Exposure:

If portfolio size = $10,000 allocated to this pair:
- Long $5,000 of Stock A
- Short $5,000 × β of Stock B

Example (β = 1.2):
- Long $5,000 Stock A
- Short $6,000 Stock B
→ Market neutral, beta = 0

Position Sizing Considerations:

• Total pair allocation: 10-20% of portfolio per pair
• Maximum pairs: 5-8 active pairs for diversification
• Correlation across pairs: Avoid highly correlated pairs

Risk Metrics:

• Maximum loss per pair: 2-3% of total portfolio
• Stop loss trigger: Z-score > ±3.0 or -5% loss on spread
• Portfolio-level risk: Sum of all pair risks ≤ 10%

Step 8: Generate Pair Analysis Report

Objective: Create structured markdown report with findings and recommendations.

Report Sections:

1. Executive Summary

• Total pairs analyzed
• Number of cointegrated pairs found
• Top 5 opportunities ranked by statistical strength

2. Cointegrated Pairs Table

• Pair name (Stock A / Stock B)
• Correlation coefficient
• Cointegration p-value
• Current z-score
• Trade signal (Long/Short/None)
• Half-life

3. Detailed Analysis (Top 10 Pairs)

• Pair description
• Statistical metrics
• Current spread position
• Entry/exit recommendations
• Position sizing
• Risk assessment

4. Spread Charts (Text-Based)

• Historical z-score plot (ASCII art)
• Entry/exit levels marked
• Current position indicator

5. Risk Warnings

• Pairs with deteriorating correlation
• Structural breaks detected
• Low liquidity warnings

File Naming Convention:

pair_trade_analysis_[SECTOR]_[YYYY-MM-DD].md

Example: pair_trade_analysis_Technology_2025-11-08.md

Quality Standards

Statistical Rigor

Minimum Requirements for Valid Pair:

• ✓ Correlation ≥ 0.70 over 2-year period
• ✓ Cointegration p-value < 0.05 (ADF test)
• ✓ Spread stationarity confirmed
• ✓ Half-life < 90 days
• ✓ No structural breaks in recent 6 months

Red Flags (Exclude Pair):

• Correlation dropped >0.20 in recent 6 months
• Cointegration p-value > 0.05
• Half-life increasing over time (mean-reversion weakening)
• Significant corporate events (merger, spin-off, bankruptcy risk)
• Liquidity concerns (avg volume < 500K shares/day)

Practical Considerations

Transaction Costs:

• Assume 0.1% round-trip cost per leg
• Total cost per pair = 0.4% (entry + exit, both legs)
• Minimum z-score threshold should exceed transaction costs

Short Selling:

• Verify stock is shortable (not hard-to-borrow)
• Factor in short interest costs (borrow fees)
• Monitor short squeeze risk

Execution:

• Enter/exit both legs simultaneously (avoid leg risk)
• Use limit orders to control slippage
• Pre-locate shorts before entry

Available Scripts

scripts/find_pairs.py

Purpose: Screen for cointegrated pairs within a sector or custom list.

Usage:

# Sector-based screening
python scripts/find_pairs.py --sector Technology --min-correlation 0.70

# Custom stock list
python scripts/find_pairs.py --symbols AAPL,MSFT,GOOGL,META --min-correlation 0.75

# Full options
python scripts/find_pairs.py \
  --sector Financials \
  --min-correlation 0.70 \
  --min-market-cap 2000000000 \
  --lookback-days 730 \
  --output pairs_analysis.json

Parameters:

• --sector: Sector name (Technology, Financials, etc.)
• --symbols: Comma-separated list of tickers (alternative to sector)
• --min-correlation: Minimum correlation threshold (default: 0.70)
• --min-market-cap: Minimum market cap filter (default: $2B)
• --lookback-days: Historical data period (default: 730 days)
• --output: Output JSON file (default: stdout)
• --api-key: FMP API key (or set FMP_API_KEY env var)

Output:

[
  {
    "pair": "AAPL/MSFT",
    "stock_a": "AAPL",
    "stock_b": "MSFT",
    "correlation": 0.87,
    "beta": 1.15,
    "cointegration_pvalue": 0.012,
    "adf_statistic": -3.45,
    "half_life_days": 42,
    "current_zscore": -2.3,
    "signal": "LONG",
    "strength": "Strong"
  }
]

scripts/analyze_spread.py

Purpose: Analyze a specific pair's spread behavior and generate trading signals.

Usage:

# Analyze specific pair
python scripts/analyze_spread.py --stock-a AAPL --stock-b MSFT

# Custom lookback period
python scripts/analyze_spread.py \
  --stock-a JPM \
  --stock-b BAC \
  --lookback-days 365 \
  --entry-zscore 2.0 \
  --exit-zscore 0.5

Parameters:

• --stock-a: First stock ticker
• --stock-b: Second stock ticker
• --lookback-days: Analysis period (default: 365)
• --entry-zscore: Z-score threshold for entry (default: 2.0)
• --exit-zscore: Z-score threshold for exit (default: 0.0)
• --api-key: FMP API key

Output:

• Current spread analysis
• Z-score calculation
• Entry/exit recommendations
• Position sizing
• Historical z-score chart (text)

Reference Documentation

references/methodology.md

Comprehensive guide to statistical arbitrage and pair trading:

• Pair Selection Criteria: How to identify good pair candidates
• Statistical Tests: Correlation, cointegration, stationarity
• Spread Construction: Price difference vs price ratio approaches
• Mean Reversion: Half-life calculation and interpretation
• Risk Management: Position sizing, stop losses, diversification
• Common Pitfalls: Survivorship bias, look-ahead bias, overfitting

references/cointegration_guide.md

Deep dive into cointegration testing:

• What is Cointegration?: Intuitive explanation
• ADF Test: Step-by-step procedure
• P-Value Interpretation: Statistical significance thresholds
• Half-Life Estimation: AR(1) model approach
• Structural Breaks: Testing for regime changes
• Practical Examples: Case studies with real pairs

Integration with Other Skills

Sector Analyst Integration:

• Use Sector Analyst to identify sectors in rotation
• Screen for pairs within outperforming sectors
• Pairs in leading sectors may have stronger trends

Technical Analyst Integration:

• Confirm pair entry/exit with individual stock technicals
• Check support/resistance levels before entry
• Validate trend direction aligns with spread signal

Backtest Expert Integration:

• Feed pair candidates to Backtest Expert for validation
• Test historical z-score entry/exit rules
• Optimize threshold parameters (entry z-score, stop loss)
• Walk-forward analysis for robustness

Market Environment Analysis Integration:

• Avoid pair trading during extreme volatility (VIX > 30)
• Correlations break down in crisis periods
• Prefer pair trading in sideways/range-bound markets

Portfolio Manager Integration:

• Track multiple pair positions
• Monitor overall market-neutral exposure
• Calculate portfolio-level pair trading P/L
• Rebalance hedge ratios periodically

Important Notes

• All analysis and output in English
• Statistical foundation: No discretionary interpretation
• Market neutral focus: Minimize directional beta exposure
• Data quality critical: Garbage in, garbage out
• Requires FMP API key: Free tier sufficient for basic screening
• Python dependencies: pandas, numpy, scipy, statsmodels

Common Use Cases

Use Case 1: Technology Sector Pairs

User: "Find pair trading opportunities in tech stocks"

Workflow:
1. Screen Technology sector for stocks with market cap > $10B
2. Calculate all pairwise correlations
3. Filter pairs with correlation ≥ 0.75
4. Run cointegration tests
5. Identify current z-score extremes (|z| > 2.0)
6. Generate top 10 pairs report

Use Case 2: Specific Pair Analysis

User: "Analyze AAPL and MSFT as a pair trade"

Workflow:
1. Fetch 2-year price history for AAPL and MSFT
2. Calculate correlation and beta
3. Test for cointegration
4. Calculate current spread and z-score
5. Generate entry/exit recommendation
6. Provide position sizing guidance

Use Case 3: Regional Bank Pairs

User: "Screen for pairs among regional banks"

Workflow:
1. Filter Financials sector for industry = "Regional Banks"
2. Exclude banks with <$5B market cap
3. Calculate pairwise statistics
4. Rank by cointegration strength
5. Focus on pairs with half-life < 45 days
6. Report top 5 mean-reverting pairs

Troubleshooting

Problem: No cointegrated pairs found

Solutions:

• Expand universe (lower market cap threshold)
• Relax cointegration p-value to 0.10
• Try different sectors (Utilities often cointegrate well)
• Increase lookback period to 3 years

Problem: All z-scores near zero (no trade signals)

Solutions:

• Normal market condition (pairs in equilibrium)
• Check back later or expand universe
• Lower entry threshold to ±1.5 instead of ±2.0

Problem: Pair correlation broke down

Solutions:

• Check for corporate events (earnings, guidance changes)
• Verify no M&A activity or restructuring
• Remove pair from watchlist if structural break confirmed
• Monitor for 30 days before re-entering

API Requirements

• Required: FMP API key (free tier sufficient)
• Rate Limits: ~250 requests/day on free tier
• Data Usage: ~2 requests per symbol for 2-year history
• Upgrade: Professional plan ($29/mo) recommended for frequent screening

Resources

• FMP Historical Price API: https://site.financialmodelingprep.com/developer/docs/historical-price-full
• Stock Screener API: https://site.financialmodelingprep.com/developer/docs/stock-screener-api
• Statsmodels Documentation: https://www.statsmodels.org/stable/index.html
• Cointegration Paper: Engle & Granger (1987) - "Co-Integration and Error Correction"

---

Version: 1.0 Last Updated: 2025-11-08 Dependencies: Python 3.8+, pandas, numpy, scipy, statsmodels, requests

Pair Trade Screener

Statistical arbitrage tool for identifying and analyzing pair trading opportunities using cointegration testing and mean-reversion analysis.

Overview

The Pair Trade Screener finds statistically significant pair trading opportunities by:

• Testing for cointegration (long-term equilibrium relationships)
• Calculating hedge ratios (beta values)
• Measuring mean-reversion speed (half-life)
• Generating entry/exit signals based on z-score thresholds

Market Neutral Strategy: Profit from relative price movements regardless of overall market direction.

Features

✅ Sector-wide screening - Analyze all stocks in a sector ✅ Custom pair analysis - Test specific stock combinations ✅ Statistical rigor - Cointegration tests (ADF), correlation analysis ✅ Mean-reversion metrics - Half-life calculation, z-score tracking ✅ Trade signals - Automatic entry/exit recommendations ✅ FMP API integration - Free tier sufficient for screening ✅ JSON output - Structured results for further analysis

Installation

Prerequisites

• Python 3.8+
• FMP API key (free tier: 250 requests/day)

Install Dependencies

pip install pandas numpy scipy statsmodels requests

Get FMP API Key

1. Visit: https://financialmodelingprep.com/developer/docs 2. Sign up for free account 3. Copy your API key 4. Set environment variable:

export FMP_API_KEY="your_key_here"

Or add to ~/.bashrc / ~/.zshrc for persistence.

Usage

Quick Start

# Screen Technology sector for pairs
python scripts/find_pairs.py --sector Technology

# Analyze specific pair
python scripts/analyze_spread.py --stock-a AAPL --stock-b MSFT

Screening for Pairs

Sector-Based Screening:

# Screen entire sector
python scripts/find_pairs.py --sector Financials

# Adjust correlation threshold
python scripts/find_pairs.py --sector Energy --min-correlation 0.75

# Longer lookback period
python scripts/find_pairs.py --sector Healthcare --lookback-days 1095

Custom Stock List:

# Test specific stocks
python scripts/find_pairs.py --symbols AAPL,MSFT,GOOGL,META,NVDA

# Tech giants pair screening
python scripts/find_pairs.py --symbols JPM,BAC,WFC,C,GS,MS

Full Options:

python scripts/find_pairs.py \
  --sector Technology \
  --min-correlation 0.70 \
  --min-market-cap 10000000000 \
  --lookback-days 730 \
  --output tech_pairs.json \
  --api-key YOUR_KEY

Analyzing Individual Pairs

Basic Analysis:

python scripts/analyze_spread.py --stock-a AAPL --stock-b MSFT

Custom Parameters:

python scripts/analyze_spread.py \
  --stock-a JPM \
  --stock-b BAC \
  --lookback-days 365 \
  --entry-zscore 2.0 \
  --exit-zscore 0.5 \
  --api-key YOUR_KEY

Example Output

Pair Screening Results

PAIR TRADING SCREEN SUMMARY
==========================================================================

Total pairs analyzed: 45
Cointegrated pairs: 12
Pairs with trade signals: 5

==========================================================================
ACTIVE TRADE SIGNALS
==========================================================================

Pair: XOM/CVX
  Signal: LONG
  Z-Score: -2.35
  Correlation: 0.9421
  P-Value: 0.0012
  Half-Life: 28.3 days
  Strength: ★★★

Individual Pair Analysis

PAIR TRADE ANALYSIS: AAPL / MSFT
==========================================================================

[ PAIR STATISTICS ]
  Correlation: 0.8732
  Hedge Ratio (Beta): 1.1523
  Data Points: 365

[ COINTEGRATION TEST ]
  ADF Statistic: -3.8542
  P-value: 0.0028
  Result: ✅ COINTEGRATED (p < 0.05)
  Strength: ★★★ Very Strong

[ MEAN REVERSION ]
  Half-Life: 42.1 days
  Speed: Moderate (suitable for pair trading)

[ Z-SCORE ]
  Current Z-Score: -2.13
  Historical Range: [-3.45, 3.12]

[ TRADE SIGNAL ]
  Signal: 🔺 LONG SPREAD
  Action: Long AAPL, Short MSFT
  Rationale: Z-score = -2.13 → AAPL cheap relative to MSFT

[ POSITION SIZING ]
  Example Allocation: $10,000
  LONG AAPL: $5,000 (27 shares @ $185.50)
  SHORT MSFT: $5,762 (14 shares @ $411.25)

  Exit Conditions:
    - Primary: Z-score crosses 0 (mean reversion)
    - Stop Loss: Z-score > ±3.0
    - Time-based: No reversion after 90 days

Understanding the Metrics

Correlation

• Range: -1 to +1
• Threshold: ≥ 0.70 required
• Interpretation: Higher = stronger co-movement

Cointegration P-Value

• Range: 0 to 1
• Threshold: < 0.05 required (statistically significant)
• Interpretation: Lower = stronger cointegration
• p < 0.01: ★★★ Very strong
• p 0.01-0.05: ★★ Moderate
• p > 0.05: ☆ Not cointegrated (reject)

Half-Life

• Meaning: Time for spread to revert halfway to mean
• Fast: < 30 days (ideal for short-term trading)
• Moderate: 30-60 days (standard pair trading)
• Slow: > 60 days (long-term positions)

Z-Score

• Calculation: (Current Spread - Mean) / Std Dev
• Entry Signals:
• Z > +2.0: Short spread (Short A, Long B)
• Z < -2.0: Long spread (Long A, Short B)
• Exit: Z crosses 0 (mean reversion)
• Stop: |Z| > 3.0 (extreme divergence)

Hedge Ratio (Beta)

• Meaning: Dollar amount of Stock B per $1 of Stock A
• Example: Beta = 1.2 → Short $1,200 of B for every $1,000 long in A
• Purpose: Market-neutral positioning (net beta ≈ 0)

Common Workflows

1. Weekly Pair Screening

# Monday: Screen for new opportunities
python scripts/find_pairs.py --sector Technology --output tech_pairs.json

# Review top pairs in JSON output
cat tech_pairs.json | jq '.pairs[] | select(.signal != "NONE")'

# Detailed analysis on top candidates
python scripts/analyze_spread.py --stock-a AAPL --stock-b MSFT

2. Sector Rotation Pairs

# Screen multiple sectors
for sector in Technology Financials Healthcare Energy; do
  python scripts/find_pairs.py --sector $sector --output ${sector}_pairs.json
  sleep 5
done

# Find pairs with strongest signals
cat *_pairs.json | jq '.pairs[] | select(.current_zscore | . > 2 or . < -2)'

3. Monitor Existing Pairs

# Update z-scores for current positions
python scripts/analyze_spread.py --stock-a XOM --stock-b CVX
python scripts/analyze_spread.py --stock-a JPM --stock-b BAC
python scripts/analyze_spread.py --stock-a GOOGL --stock-b META

API Usage & Rate Limits

Free Tier:

• 250 API requests/day
• ~2 requests per stock for price data
• Can screen ~60 stocks/day (= 1,770 pairs)

Screening Costs:

Sector screening (30 stocks):
  - Fetch 30 stock prices = 30 requests
  - Analyze 435 pairs (30 choose 2) = 0 additional requests
  - Total: 30 requests

Individual pair analysis:
  - Fetch 2 stock prices = 2 requests

Tips:

• Run sector screens once/week (not daily)
• Cache results in JSON files
• Monitor specific pairs daily (2 requests each)
• Upgrade to paid plan if screening multiple sectors daily

Interpretation Guide

When to Trade

✅ Strong Pair (Enter):

• Correlation > 0.80
• P-value < 0.03
• Half-life 20-60 days
• |Z-score| > 2.0
• Economic linkage (same sector/industry)

⚠️ Marginal Pair (Caution):

• Correlation 0.70-0.80
• P-value 0.03-0.05
• Half-life > 60 days
• |Z-score| 1.5-2.0

❌ Weak Pair (Avoid):

• Correlation < 0.70
• P-value > 0.05
• Half-life > 90 days or undefined
• No economic linkage

Exit Conditions

Primary Exit:

• Z-score crosses 0 (spread reverts to mean)
• Close both legs simultaneously

Stop Loss:

• |Z-score| > 3.0 (extreme divergence, possible structural break)
• -5% loss on spread
• Exit immediately

Time-Based:

• No mean reversion after 90 days (or 3× half-life)
• Free capital for better opportunities

Troubleshooting

No pairs found

Solutions:

• Lower --min-correlation to 0.65
• Expand stock universe (try different sector)
• Increase --lookback-days to 1095 (3 years)

API rate limit exceeded

Solutions:

• Wait 24 hours (free tier resets daily)
• Cache screening results (JSON files)
• Upgrade to paid plan ($14/mo Starter tier)

All z-scores near zero

Normal: Pairs in equilibrium, no trade signals Action: Check back later or expand universe

Pair correlation broke down

Causes: Corporate events, M&A, business model changes Detection: Recent correlation << historical correlation Action: Exit pair, remove from watchlist

Integration with Other Skills

Backtest Expert:

• Test pair trading strategies historically
• Optimize entry/exit thresholds
• Validate robustness

Sector Analyst:

• Identify sectors in rotation
• Screen for pairs within leading sectors

Technical Analyst:

• Confirm individual stock trends
• Check support/resistance before entry

Portfolio Manager:

• Track multiple pair positions
• Monitor overall market-neutral exposure

Resources

Documentation:

• references/methodology.md - Statistical arbitrage theory
• references/cointegration_guide.md - Cointegration testing guide

FMP API:

• API Docs: https://financialmodelingprep.com/developer/docs
• Historical Price API: /v3/historical-price-full/{symbol}
• Stock Screener API: /v3/stock-screener

Academic Papers:

• Engle & Granger (1987): "Co-Integration and Error Correction"
• Gatev et al. (2006): "Pairs Trading: Performance of a Relative-Value Arbitrage Rule"

License

Educational and research use. Trade at your own risk. Past performance does not guarantee future results.

---

Version: 1.0 Last Updated: 2025-11-08 Dependencies: Python 3.8+, pandas, numpy, scipy, statsmodels, requests API: FMP API (free tier sufficient)

Cointegration Testing Guide

Table of Contents

1. What is Cointegration? 2. Cointegration vs Correlation 3. Augmented Dickey-Fuller (ADF) Test 4. Practical Implementation 5. Interpreting Results 6. Half-Life Estimation 7. Testing for Structural Breaks 8. Case Studies

---

What is Cointegration?

Intuitive Explanation

Imagine two drunk people walking home from a bar. They're both stumbling randomly, but one person has a dog on a leash. The person and the dog may wander in different directions temporarily, but the leash keeps them together in the long run. They are "cointegrated."

In finance:

• Person A = Stock A price
• Person B (with dog) = Stock B price
• Leash = Economic relationship (sector, supply chain, competition)

While both stock prices are non-stationary (random walks), their difference (or spread) is stationary because the economic "leash" pulls them back together.

Mathematical Definition

Two non-stationary time series X(t) and Y(t) are cointegrated if there exists a coefficient β such that:

Spread(t) = X(t) - β * Y(t)

is stationary (mean-reverting).

Key Components:

• X(t), Y(t): Non-stationary price series (random walks)
• β: Cointegration coefficient (hedge ratio)
• Spread(t): Stationary series (mean-reverting)

Why Cointegration Matters for Pair Trading

Without Cointegration:

• Prices can drift apart indefinitely
• No guarantee of mean reversion
• High risk of permanent losses

With Cointegration:

• Prices have long-term equilibrium
• Temporary deviations are predictable
• Mean reversion is statistically ensured

Example:

Non-Cointegrated Pair:

Stock A: Oil producer
Stock B: Tech company
Correlation: 0.75 (recent coincidence)

Result: No economic linkage → correlation breaks down → prices diverge forever

Cointegrated Pair:

Stock A: Exxon (XOM)
Stock B: Chevron (CVX)
Correlation: 0.92
Cointegration p-value: 0.008 (strong)

Result: Same sector, similar business → prices stay together → mean reversion reliable

---

Cointegration vs Correlation

Key Differences

Why Correlation Alone is Insufficient

Problem with Correlation:

Two random walks can have high correlation by chance without any fundamental relationship.

Example:

import numpy as np

# Generate two independent random walks
np.random.seed(42)
walk_A = np.cumsum(np.random.randn(252))
walk_B = np.cumsum(np.random.randn(252))

# Calculate correlation
correlation = np.corrcoef(walk_A, walk_B)[0, 1]
# Result: Might be 0.60-0.80 purely by chance!

Key Point:

• High correlation ≠ Mean reversion
• Need cointegration to ensure spread is stationary

Combining Correlation and Cointegration

Best Practice:

Use both as filters: 1. Correlation (≥ 0.70): Quick screen for co-movement 2. Cointegration (p < 0.05): Rigorous test for mean reversion

Decision Matrix:

---

Augmented Dickey-Fuller (ADF) Test

Purpose

The ADF test determines whether a time series has a unit root (non-stationary) or is stationary.

Hypotheses:

• Null (H0): Series has unit root (non-stationary)
• Alternative (H1): Series is stationary

For pair trading:

• Test the spread (not individual prices)
• Reject H0 → Spread is stationary → Pair is cointegrated

Test Procedure

Step 1: Calculate Spread

spread = price_A - (beta * price_B)

Step 2: Run ADF Test

from statsmodels.tsa.stattools import adfuller

result = adfuller(spread, maxlag=1, regression='c')
adf_statistic = result[0]
p_value = result[1]
critical_values = result[4]

Parameters:

• maxlag=1: Number of lags (typically 1 for daily data)
• regression='c': Include constant term (drift)
• Alternatives: 'ct' (constant + trend), 'n' (no constant)

Step 3: Interpret Results

if p_value < 0.05:
    print("Reject null → Spread is stationary → Cointegrated")
else:
    print("Fail to reject null → Not cointegrated")

ADF Test Equation

The ADF test estimates:

ΔSpread(t) = α + β*Spread(t-1) + Σ(γ_i * ΔSpread(t-i)) + ε(t)

Where:

• ΔSpread(t) = Spread(t) - Spread(t-1) (first difference)
• β: Coefficient of interest (tests for unit root)
• α: Drift term
• Σ(γ_i * ΔSpread(t-i)): Lagged differences (capture serial correlation)

Test Statistic:

ADF = β / SE(β)

Decision Rule:

• If ADF < Critical Value → Reject null (stationary)
• If p-value < 0.05 → Reject null (stationary)

Critical Values

Standard Critical Values (constant, no trend):

Example:

ADF Statistic: -3.75
Critical Value (5%): -2.86

Since -3.75 < -2.86 → Reject null → Stationary

---

Practical Implementation

Complete Python Example

import pandas as pd
import numpy as np
from statsmodels.tsa.stattools import adfuller
from scipy import stats

# Step 1: Load price data
prices_A = pd.Series([100, 102, 104, 103, 105, ...])  # Stock A
prices_B = pd.Series([50, 51, 52, 51.5, 52.5, ...])    # Stock B

# Step 2: Calculate beta (hedge ratio)
slope, intercept, r_value, p_value, std_err = stats.linregress(prices_B, prices_A)
beta = slope

print(f"Beta (hedge ratio): {beta:.4f}")
print(f"Correlation: {r_value:.4f}")

# Step 3: Calculate spread
spread = prices_A - (beta * prices_B)

# Step 4: Run ADF test
adf_result = adfuller(spread, maxlag=1, regression='c')

adf_statistic = adf_result[0]
p_value = adf_result[1]
critical_values = adf_result[4]
n_lags = adf_result[2]

# Step 5: Display results
print("\n=== Cointegration Test Results ===")
print(f"ADF Statistic: {adf_statistic:.4f}")
print(f"P-value: {p_value:.4f}")
print(f"Number of Lags: {n_lags}")
print(f"\nCritical Values:")
for key, value in critical_values.items():
    print(f"  {key}: {value:.4f}")

# Step 6: Interpret
if p_value < 0.01:
    print("\n✅ STRONG Cointegration (p < 0.01)")
    strength = "★★★"
elif p_value < 0.05:
    print("\n✅ MODERATE Cointegration (p < 0.05)")
    strength = "★★"
else:
    print("\n❌ NOT Cointegrated (p > 0.05)")
    strength = "☆"

# Step 7: Calculate half-life (if cointegrated)
if p_value < 0.05:
    from statsmodels.tsa.ar_model import AutoReg

    model = AutoReg(spread, lags=1)
    result = model.fit()
    phi = result.params[1]

    half_life = -np.log(2) / np.log(phi)
    print(f"\nHalf-Life: {half_life:.1f} days")

    if half_life < 30:
        print("  → Fast mean reversion (excellent)")
    elif half_life < 60:
        print("  → Moderate mean reversion (good)")
    else:
        print("  → Slow mean reversion (acceptable)")

FMP API Integration

import requests
import pandas as pd

def get_price_history(symbol, api_key, days=730):
    """Fetch historical prices from FMP API"""
    url = f"https://financialmodelingprep.com/api/v3/historical-price-full/{symbol}?apikey={api_key}"
    response = requests.get(url)
    data = response.json()

    # Extract historical prices
    hist = data['historical'][:days]
    hist = hist[::-1]  # Reverse to chronological order

    df = pd.DataFrame(hist)
    df['date'] = pd.to_datetime(df['date'])
    df = df.set_index('date')

    return df['adjClose']

# Example usage
api_key = "YOUR_API_KEY"
prices_AAPL = get_price_history("AAPL", api_key)
prices_MSFT = get_price_history("MSFT", api_key)

# Align dates
common_dates = prices_AAPL.index.intersection(prices_MSFT.index)
prices_AAPL = prices_AAPL.loc[common_dates]
prices_MSFT = prices_MSFT.loc[common_dates]

# Test for cointegration
slope, intercept, r_value, p_value, std_err = stats.linregress(prices_MSFT, prices_AAPL)
beta = slope
spread = prices_AAPL - (beta * prices_MSFT)

adf_result = adfuller(spread, maxlag=1)
print(f"AAPL/MSFT Cointegration p-value: {adf_result[1]:.4f}")

---

Interpreting Results

P-Value Interpretation

What p-value means:

• Probability of observing test statistic if null hypothesis (unit root) is true
• Lower p-value = stronger evidence against null = stronger cointegration

Guidelines:

ADF Statistic Interpretation

More negative = stronger cointegration:

ADF < -4.0: Very strong (★★★★)
ADF -3.5 to -4.0: Strong (★★★)
ADF -3.0 to -3.5: Moderate (★★)
ADF -2.5 to -3.0: Weak (★)
ADF > -2.5: Not cointegrated (☆)

Example Rankings:

Pair A: ADF = -4.25, p = 0.002 → ★★★★ (Best)
Pair B: ADF = -3.65, p = 0.018 → ★★★ (Excellent)
Pair C: ADF = -2.95, p = 0.042 → ★★ (Good)
Pair D: ADF = -2.45, p = 0.125 → ☆ (Reject)

Common Mistakes

Mistake 1: Testing Individual Prices

# WRONG: Testing if stock price is stationary
adf_result = adfuller(prices_A)  # ❌ Will always fail (prices are random walks)

Correct:

# RIGHT: Test if SPREAD is stationary
spread = prices_A - (beta * prices_B)
adf_result = adfuller(spread)  # ✅ Tests for cointegration

Mistake 2: Ignoring Lag Selection

# WRONG: Using too many lags (overfitting)
adf_result = adfuller(spread, maxlag=20)  # ❌ Too complex

# RIGHT: Use simple lag structure
adf_result = adfuller(spread, maxlag=1)  # ✅ Appropriate for daily data

Mistake 3: Confusing Correlation with Cointegration

# WRONG: Assuming high correlation = cointegration
if correlation > 0.80:
    trade_pair()  # ❌ Not sufficient

# RIGHT: Test for cointegration explicitly
if correlation > 0.70 and cointegration_pvalue < 0.05:
    trade_pair()  # ✅ Both conditions required

---

Half-Life Estimation

What is Half-Life?

Half-life measures how quickly the spread mean-reverts. Specifically, it's the expected time for the spread to move halfway back to its mean.

Example:

Current spread: +2.0 (2 std devs above mean)
Half-life: 20 days

Expected spread after 20 days: +1.0 (halfway to mean)
Expected spread after 40 days: +0.5 (halfway from +1.0 to 0)

AR(1) Model Approach

Model spread as autoregressive process:

S(t) = α + φ * S(t-1) + ε(t)

Where:

• φ: Autocorrelation coefficient (persistence)
• φ close to 1.0 → Slow mean reversion (long half-life)
• φ close to 0.0 → Fast mean reversion (short half-life)

Half-Life Formula:

Half-Life = -ln(2) / ln(φ)

Python Implementation

from statsmodels.tsa.ar_model import AutoReg

# Fit AR(1) model to spread
model = AutoReg(spread, lags=1)
result = model.fit()

# Extract autocorrelation coefficient
phi = result.params[1]

# Calculate half-life
half_life = -np.log(2) / np.log(phi)

print(f"Autocorrelation (φ): {phi:.4f}")
print(f"Half-Life: {half_life:.1f} days")

Interpreting Half-Life

Trading Implications:

Fast Half-Life (< 30 days):

• ✅ Quick profits
• ✅ Lower holding risk
• ✅ Frequent opportunities
• ❌ Transaction costs matter more

Slow Half-Life (> 60 days):

• ✅ More stable relationships
• ❌ Capital tied up longer
• ❌ Fewer trading opportunities
• ❌ Higher holding risk (regime changes)

Half-Life Stability

Test half-life over multiple periods:

# Calculate rolling half-life
rolling_half_life = []

for i in range(252, len(spread)):
    window = spread[i-252:i]
    model = AutoReg(window, lags=1)
    result = model.fit()
    phi = result.params[1]
    hl = -np.log(2) / np.log(phi)
    rolling_half_life.append(hl)

# Check stability
std_hl = np.std(rolling_half_life)
mean_hl = np.mean(rolling_half_life)
cv = std_hl / mean_hl  # Coefficient of variation

if cv < 0.30:
    print("Half-life is STABLE (good)")
else:
    print("Half-life is UNSTABLE (warning)")

---

Testing for Structural Breaks

Why Structural Breaks Matter

Definition:

• A structural break is a sudden, permanent change in the cointegration relationship
• Examples: Merger, spin-off, business model pivot, regulatory change

Impact on Pair Trading:

• Break in cointegration → Spread no longer mean-reverts
• Holding pair through break → Large losses
• Must detect breaks early and exit

Chow Test

Tests for known breakpoint (e.g., specific corporate event date):

from statsmodels.stats.diagnostic import breaks_cusumolsresid

# Fit OLS regression
from scipy import stats
slope, intercept = stats.linregress(prices_B, prices_A)[:2]
residuals = prices_A - (slope * prices_B + intercept)

# Test for structural breaks
stat, pvalue = breaks_cusumolsresid(residuals)

if pvalue < 0.05:
    print("⚠️ STRUCTURAL BREAK DETECTED")
else:
    print("✅ No structural break")

Rolling Cointegration

Monitor cointegration over time:

rolling_pvalues = []

for i in range(252, len(prices_A)):
    window_A = prices_A[i-252:i]
    window_B = prices_B[i-252:i]

    slope, intercept = stats.linregress(window_B, window_A)[:2]
    spread_window = window_A - (slope * window_B)

    adf_result = adfuller(spread_window, maxlag=1)
    pvalue = adf_result[1]
    rolling_pvalues.append(pvalue)

# Plot rolling p-values
import matplotlib.pyplot as plt
plt.plot(rolling_pvalues)
plt.axhline(y=0.05, color='r', linestyle='--', label='Significance threshold')
plt.ylabel('P-Value')
plt.xlabel('Time')
plt.title('Rolling Cointegration P-Value')
plt.legend()
plt.show()

Interpretation:

• P-value stays below 0.05 → Cointegration stable ✅
• P-value crosses above 0.05 → Cointegration breaking down ⚠️
• P-value remains above 0.10 → Relationship broken ❌

Early Warning System

Exit conditions based on cointegration degradation:

# Calculate 90-day rolling cointegration p-value
recent_pvalue = calculate_rolling_cointegration(prices_A[-90:], prices_B[-90:])

if recent_pvalue > 0.10:
    print("🚨 EXIT SIGNAL: Cointegration broken")
    exit_pair()
elif recent_pvalue > 0.05:
    print("⚠️ WARNING: Cointegration weakening")
    reduce_position()
else:
    print("✅ Cointegration healthy")

---

Case Studies

Case Study 1: XOM/CVX (Strong Cointegration)

Background:

• Exxon Mobil (XOM) and Chevron (CVX)
• Both: Large oil & gas companies
• Same sector, similar business models

Analysis:

# 2-year data (2023-2025)
correlation: 0.94
beta: 1.08
adf_statistic: -4.25
p_value: 0.0008
half_life: 28 days

Interpretation:

• ✅ Very strong cointegration (p < 0.01)
• ✅ High correlation (0.94)
• ✅ Fast mean reversion (28 days)
• ✅ Economic linkage (same sector)

Rating: ★★★★ (Excellent pair)

Trade Signal (Example):

Current Z-Score: +2.3 (XOM expensive relative to CVX)
→ SHORT XOM, LONG CVX
Entry: Z > +2.0
Exit: Z < 0.0
Stop: Z > +3.0

Case Study 2: JPM/BAC (Moderate Cointegration)

Background:

• JPMorgan Chase (JPM) and Bank of America (BAC)
• Both: Large banks
• Different focus areas (JPM more investment banking, BAC more retail)

Analysis:

correlation: 0.85
beta: 1.35
adf_statistic: -3.12
p_value: 0.031
half_life: 42 days

Interpretation:

• ✅ Moderate cointegration (p = 0.031)
• ✅ Good correlation (0.85)
• ✅ Acceptable mean reversion (42 days)
• ⚠️ Different business mix (less perfect linkage)

Rating: ★★★ (Good pair)

Case Study 3: AAPL/TSLA (No Cointegration)

Background:

• Apple (AAPL) and Tesla (TSLA)
• Both: High-growth tech stocks
• Different businesses (consumer electronics vs EVs)

Analysis:

correlation: 0.72
beta: 0.88
adf_statistic: -2.15
p_value: 0.182
half_life: N/A (not stationary)

Interpretation:

• ❌ No cointegration (p = 0.182)
• ✅ Moderate correlation (0.72)
• ❌ No mean reversion
• ❌ Weak economic linkage

Rating: ☆ (Reject pair)

Why correlation failed:

• Both had high returns in 2023-2024 (growth stock rally)
• Correlation driven by macro factors (interest rates), not fundamental linkage
• Likely to diverge when market conditions change

Case Study 4: Structural Break Example (GE)

Background:

• General Electric (GE) underwent major restructuring 2018-2021
• Spun off healthcare (GEHC) and energy divisions

Analysis:

# Pre-spinoff (2018-2020): GE/UTX pair
correlation: 0.81
p_value: 0.025 (cointegrated)

# Post-spinoff (2021-2023): GE/UTX pair
correlation: 0.52
p_value: 0.235 (NOT cointegrated)

Lesson:

• Corporate actions can break cointegration
• Must monitor for structural breaks
• Exit pairs when cointegration deteriorates

---

Summary Checklist

Before trading a pair, verify:

Statistical Checklist

• [ ] Correlation ≥ 0.70 (preferably ≥ 0.80)
• [ ] Cointegration p-value < 0.05 (preferably < 0.03)
• [ ] ADF statistic < -3.0
• [ ] Half-life 20-60 days
• [ ] No structural breaks in recent 6 months

Economic Checklist

• [ ] Same sector or supply chain relationship
• [ ] Similar business models
• [ ] No pending M&A or restructuring
• [ ] Similar market cap and liquidity
• [ ] Both stocks shortable (for short leg)

Risk Checklist

• [ ] Transaction costs < expected profit
• [ ] Adequate liquidity (>1M avg volume)
• [ ] Position sized appropriately (10-15% max)
• [ ] Stop loss defined (Z > ±3.0)
• [ ] Maximum holding period set (90 days)

---

Document Version: 1.0 Last Updated: 2025-11-08 References:

• Engle & Granger (1987): "Co-Integration and Error Correction"
• Hamilton (1994): "Time Series Analysis" (Chapter 19)
• Tsay (2010): "Analysis of Financial Time Series" (Chapter 8)

Statistical Arbitrage and Pair Trading Methodology

Table of Contents

1. Introduction to Pair Trading 2. Theoretical Foundation 3. Pair Selection Process 4. Statistical Tests 5. Spread Construction and Analysis 6. Entry and Exit Rules 7. Risk Management 8. Common Pitfalls 9. Advanced Topics

---

Introduction to Pair Trading

What is Pair Trading?

Pair trading is a market-neutral trading strategy that involves simultaneously buying and selling two highly correlated securities. The strategy profits from the convergence of prices when the relationship between the two stocks temporarily diverges from its historical equilibrium.

Key Characteristics:

• Market Neutral: Long one stock, short another → net beta ≈ 0
• Mean Reverting: Relies on temporary deviations from equilibrium
• Statistical Foundation: Based on mathematical relationships, not discretion
• Relative Value: Profits from relative performance, not absolute direction

Historical Context

Pair trading was pioneered by Gerry Bamberger and later developed by Nunzio Tartaglia at Morgan Stanley in the 1980s. The quantitative team identified pairs of stocks whose prices historically moved together and profited when temporary divergences occurred.

Evolution:

• 1980s: Manual pair selection, simple spread tracking
• 1990s: Statistical models (cointegration, Kalman filters)
• 2000s: High-frequency pair trading, algorithmic execution
• 2010s+: Machine learning, regime detection, multi-asset pairs

Why Pair Trading Works

Economic Rationale: 1. Sector/Industry Linkages: Companies in same sector share common drivers (demand, regulation, input costs) 2. Temporary Mispricing: Information asymmetry, behavioral biases, technical flows create temporary price divergences 3. Mean Reversion: Economic forces pull prices back to equilibrium (arbitrage, fundamentals) 4. Supply Chain Relationships: Suppliers and customers often move together

Statistical Rationale:

• Cointegration ensures long-run equilibrium relationship
• Short-term deviations are statistically predictable
• Z-score framework provides objective entry/exit rules

---

Theoretical Foundation

Cointegration vs Correlation

Correlation:

• Measures short-term co-movement of returns
• Range: -1 to +1
• Can be unstable over time
• Does not imply mean reversion

Cointegration:

• Measures long-term equilibrium relationship between price levels
• Ensures spread is stationary (mean-reverting)
• More stable foundation for pair trading
• Implies prices won't drift apart indefinitely

Mathematical Definition:

Two time series P_A(t) and P_B(t) are cointegrated if:

Spread(t) = P_A(t) - β * P_B(t)

is stationary (mean-reverting), where β is the cointegration coefficient.

Stationarity

A time series is stationary if: 1. Constant mean over time 2. Constant variance over time 3. Autocorrelation depends only on time lag, not time itself

Why Stationarity Matters for Pair Trading:

• Non-stationary spread → prices can diverge forever → no mean reversion
• Stationary spread → prices revert to mean → profitable trade opportunities
• Cointegration ensures spread stationarity

Mean Reversion

The spread follows an Ornstein-Uhlenbeck (OU) process:

dS(t) = θ(μ - S(t))dt + σdW(t)

Where:

• S(t): Spread at time t
• θ: Mean reversion speed (higher = faster reversion)
• μ: Long-term mean of spread
• σ: Volatility of spread
• dW(t): Brownian motion (random noise)

Half-Life:

Time for spread to revert halfway to its mean:

Half-Life = ln(2) / θ

Interpretation:

• Half-life = 10 days: Very fast mean reversion (suitable for day trading)
• Half-life = 30 days: Standard speed (typical pair trades)
• Half-life = 60+ days: Slow reversion (requires patience, longer holding)

---

Pair Selection Process

Step 1: Universe Definition

Sector-Based Approach (Recommended):

• Focus on stocks within same sector (e.g., Technology, Financials)
• Common drivers increase likelihood of cointegration
• Examples:
• Tech: AAPL/MSFT, GOOGL/META, NVDA/AMD
• Financials: JPM/BAC, GS/MS, WFC/USB
• Energy: XOM/CVX, COP/EOG

Industry-Specific Approach:

• Narrow to specific industry (e.g., "Software" within Tech, "Regional Banks" within Financials)
• Stronger fundamental linkages
• Fewer pairs but higher quality

Custom Approach:

• User-specified list of stocks
• Allows testing hypotheses (e.g., supply chain pairs, competitor pairs)

Filtering Criteria:

• Market cap ≥ $2B (liquidity and stability)
• Average volume ≥ 1M shares/day (execution feasibility)
• Same exchange preferred (NYSE/NYSE, NASDAQ/NASDAQ)
• Exclude recent IPOs (<2 years) and distressed stocks

Step 2: Correlation Screening

Calculate Pearson Correlation:

import pandas as pd
import numpy as np

# Returns-based correlation
correlation = returns_A.corr(returns_B)

# Price-based correlation (for reference)
price_correlation = prices_A.corr(prices_B)

Thresholds:

• Excellent (ρ ≥ 0.85): Very strong co-movement, best candidates
• Good (ρ 0.70-0.85): Strong relationship, acceptable
• Marginal (ρ 0.50-0.70): Moderate, requires strong cointegration
• Poor (ρ < 0.50): Weak, likely not cointegrated

Correlation Stability:

Test correlation over multiple periods:

# 6-month rolling correlation
rolling_corr = returns_A.rolling(126).corr(returns_B)

# Stability metric: std deviation of rolling correlation
stability = rolling_corr.std()

# Prefer pairs with low stability (<0.10)

Red Flags:

• Correlation declining sharply in recent period
• High correlation volatility (correlation jumps around)
• Correlation approaching zero or turning negative

Step 3: Beta Estimation (Hedge Ratio)

Ordinary Least Squares (OLS) Regression:

from scipy import stats

# Regress Price_A on Price_B
slope, intercept, r_value, p_value, std_err = stats.linregress(prices_B, prices_A)

beta = slope  # Hedge ratio

Interpretation:

• Beta = 1.0: Equal dollar hedge ($1 long A, $1 short B)
• Beta = 1.5: $1 long A, $1.50 short B
• Beta = 0.8: $1 long A, $0.80 short B

Alternative Methods:

• Total Least Squares (TLS): Accounts for error in both variables
• Kalman Filter: Time-varying beta (advanced)
• Rolling OLS: Adaptive beta over time

Step 4: Cointegration Testing

Augmented Dickey-Fuller (ADF) Test:

Tests null hypothesis: "Spread has unit root (non-stationary)"

from statsmodels.tsa.stattools import adfuller

# Calculate spread
spread = prices_A - (beta * prices_B)

# ADF test
result = adfuller(spread, maxlag=1, regression='c')
adf_statistic = result[0]
p_value = result[1]
critical_values = result[4]

# Interpret
is_cointegrated = p_value < 0.05

Interpretation:

• p < 0.01: Very strong cointegration (reject null at 99% confidence)
• p 0.01-0.05: Moderate cointegration (reject null at 95% confidence)
• p > 0.05: No cointegration (fail to reject null)

Critical Values:

• ADF stat < critical value (1%) → Very strong evidence
• ADF stat < critical value (5%) → Moderate evidence
• ADF stat > critical value (10%) → Weak evidence

Alternative Cointegration Tests:

• Engle-Granger Two-Step: Similar to ADF
• Johansen Test: Multi-variate cointegration (for >2 stocks)
• Phillips-Ouliaris Test: More robust to structural breaks

---

Statistical Tests

Stationarity Tests

Augmented Dickey-Fuller (ADF):

• Most common stationarity test
• Tests for unit root in time series
• Null hypothesis: Series has unit root (non-stationary)

Kwiatkowski-Phillips-Schmidt-Shin (KPSS):

• Complementary to ADF
• Null hypothesis: Series is stationary
• Use both tests for robustness:
• ADF rejects + KPSS fails to reject → Stationary
• ADF fails to reject + KPSS rejects → Non-stationary

Phillips-Perron (PP) Test:

• Non-parametric alternative to ADF
• More robust to heteroskedasticity and serial correlation

Half-Life Estimation

AR(1) Model Approach:

Model spread as first-order autoregressive process:

S(t) = α + φ * S(t-1) + ε(t)

Where:

• φ: Autocorrelation coefficient
• Mean reversion speed: θ = -ln(φ)
• Half-life: HL = ln(2) / θ

Python Implementation:

from statsmodels.tsa.ar_model import AutoReg

# Fit AR(1) model
model = AutoReg(spread, lags=1)
result = model.fit()
phi = result.params[1]

# Calculate half-life
theta = -np.log(phi)
half_life = np.log(2) / theta

Interpretation:

• HL < 20 days: Very fast mean reversion
• HL 20-40 days: Fast mean reversion (ideal for pair trading)
• HL 40-60 days: Moderate mean reversion
• HL > 60 days: Slow mean reversion (less attractive)

Structural Break Detection

Chow Test:

Tests for structural break at known date (e.g., major corporate event):

from statsmodels.stats.diagnostic import breaks_cusumolsresid

# Test for breaks
stat, pvalue = breaks_cusumolsresid(ols_residuals)

# Interpret
has_structural_break = pvalue < 0.05

CUSUM Test:

• Detects unknown breakpoints
• Plots cumulative sum of residuals
• Sharp changes indicate structural breaks

Practical Implication:

• If structural break detected → Re-estimate cointegration relationship
• If break persists → Abandon pair (relationship broken)

---

Spread Construction and Analysis

Spread Definitions

Price Difference (Additive):

Spread(t) = P_A(t) - β * P_B(t)

Advantages:

• Simpler interpretation
• Stationary under cointegration

Disadvantages:

• Units depend on price levels
• Not suitable for stocks with very different prices

Price Ratio (Multiplicative):

Spread(t) = P_A(t) / P_B(t)

Advantages:

• Unit-free (ratio has no units)
• Better for stocks with different price levels
• Percentage-based interpretation

Disadvantages:

• Log transformation often needed for stationarity
• More complex statistics

Log Price Ratio:

Spread(t) = ln(P_A(t)) - ln(P_B(t))

Advantages:

• Ensures stationarity
• Returns-like interpretation
• Symmetric around zero

Z-Score Calculation

Definition:

Z(t) = [Spread(t) - μ] / σ

Where:

• μ: Mean of spread over lookback period
• σ: Standard deviation of spread over lookback period

Lookback Period:

• 90 days (short-term): Responsive to recent changes, noisier
• 180 days (medium-term): Balanced approach (recommended)
• 252 days (long-term): Stable parameters, slower to adapt

Rolling vs Expanding Window:

Rolling Window:

rolling_mean = spread.rolling(90).mean()
rolling_std = spread.rolling(90).std()
zscore = (spread - rolling_mean) / rolling_std

• Adapts to changing spread dynamics
• Can miss long-term trends

Expanding Window:

expanding_mean = spread.expanding().mean()
expanding_std = spread.expanding().std()
zscore = (spread - expanding_mean) / expanding_std

• Uses all historical data
• More stable but less adaptive

Spread Distribution Analysis

Normality Test:

from scipy.stats import normaltest

# Test if spread is normally distributed
stat, pvalue = normaltest(spread)
is_normal = pvalue > 0.05

If spread is NOT normal:

• Use percentile-based thresholds instead of z-scores
• Example: Enter at 95th percentile instead of z > 2.0

Skewness and Kurtosis:

• Skewness ≠ 0: Asymmetric distribution (adjust thresholds)
• Kurtosis > 3: Fat tails (extreme values more common)

---

Entry and Exit Rules

Entry Conditions

Conservative Strategy (Z ≥ ±2.0):

LONG Spread (Long A, Short B):

Conditions:
1. Z-score < -2.0 (spread 2+ std devs below mean)
2. Cointegration p-value < 0.05
3. Correlation > 0.70
4. Half-life < 60 days
5. No structural breaks in recent 6 months

Action:
- Buy Stock A: $5,000
- Sell Stock B: $5,000 × β

SHORT Spread (Short A, Long B):

Conditions:
1. Z-score > +2.0 (spread 2+ std devs above mean)
2. [Same conditions 2-5 as above]

Action:
- Sell Stock A: $5,000
- Buy Stock B: $5,000 × β

Aggressive Strategy (Z ≥ ±1.5):

• Lower threshold → More frequent trades
• Higher win rate but smaller profits per trade
• Requires more active monitoring

Very Aggressive (Z ≥ ±1.0):

• Very frequent trades
• Transaction costs become significant
• Only viable with low commissions/rebates

Exit Conditions

Primary Exit: Mean Reversion

Exit when Z-score crosses zero
→ Spread has reverted to mean
→ Close both legs simultaneously

Partial Profit Taking:

Stage 1: Exit 50% at Z-score = ±1.0
Stage 2: Exit remaining 50% at Z-score = 0

Stop Loss:

Hard Stop: Z-score > ±3.0
- Spread has diverged to extreme levels
- Possible structural break
→ Exit immediately to limit losses

Drawdown Stop: Total P/L < -5%
- Trade not working as expected
→ Exit to preserve capital

Time-Based Exit:

Maximum Holding Period: 90 days (or 3× half-life)
- If no mean reversion after expected timeframe
→ Exit to free up capital for better opportunities

Signal Confirmation

Additional Filters (Optional):

Volume Confirmation:

• Require above-average volume on spread widening
• Confirms divergence is meaningful, not noise

Trend Filter:

• Avoid entry if overall market (SPY) trending strongly
• Correlations break down during market crashes/rallies

Volatility Filter:

• Skip entry if VIX > 30 (high volatility environment)
• Pair relationships less stable during volatility spikes

---

Risk Management

Position Sizing

Equal Dollar Allocation:

For $10,000 total allocation to one pair:
- Long Leg: $5,000
- Short Leg: $5,000 × β

If β = 1.2:
- Long Stock A: $5,000
- Short Stock B: $6,000

Volatility-Adjusted Sizing:

Scale position size by inverse of spread volatility:

Position Size = Base Size / (Spread Volatility / Avg Spread Volatility)

• High volatility pair → Smaller position
• Low volatility pair → Larger position

Portfolio-Level Risk

Diversification:

• Minimum: 5 pairs (reduce idiosyncratic risk)
• Optimal: 8-10 pairs (balance diversification and management)
• Maximum: 15 pairs (diminishing returns, management complexity)

Correlation Across Pairs:

• Avoid multiple pairs with overlapping stocks (e.g., AAPL/MSFT and AAPL/GOOGL)
• Limit exposure to single sector (<50% of pairs)
• Monitor portfolio beta (should be near zero)

Maximum Risk Allocation:

• Single pair: 10-15% of total portfolio
• All pairs combined: 60-80% of total portfolio (rest in cash/T-bills)

Transaction Costs

Components:

• Commissions: $0 (most brokers) but check per-share fees
• Spread: Bid-ask spread cost (0.01-0.05% per leg)
• Slippage: Market impact (0.02-0.10% per leg)
• Short Interest: Borrow costs for short leg (0-50% annual)
• Hard-to-Borrow Fees: Additional fees for difficult shorts

Total Round-Trip Cost Estimate:

Conservative: 0.4% (0.1% per leg × 4 legs)
With Short Interest: 0.4% + (0.5% × days held / 365)

Breakeven Z-Score:

Minimum z-score to exceed transaction costs:

Z_min = Total Transaction Cost / Expected Profit per Std Dev

Example:

• Transaction cost: 0.4%
• Expected profit per std dev: 0.8%
• Z_min = 0.4% / 0.8% = 0.5

→ Only enter if |Z| > 2.0 (provides 2σ × 0.8% = 1.6% profit, well above 0.4% cost)

Margin and Leverage

Regulation T Requirements:

• Long stock: 50% initial margin
• Short stock: 50% initial margin + 100% collateral
• Pair trade: Effectively 100% margin requirement

Example:

Long $10,000 Stock A:
- Margin required: $5,000

Short $10,000 Stock B:
- Margin required: $5,000
- Collateral required: $10,000
- Total tied up: $15,000

Total capital required: $20,000 for $20,000 exposure
→ No leverage in market-neutral pair trading

Portfolio Margin (Advanced):

• Recognizes offsetting risk of pair
• May reduce margin requirement to 50-70% of Reg T
• Requires $125K account minimum

---

Common Pitfalls

1. Survivorship Bias

Problem:

• Screening only currently listed stocks
• Excludes delisted stocks (bankruptcies, acquisitions)
• Overstates historical performance

Solution:

• Use survivorship-bias-free databases
• Acknowledge backtest limitations
• Focus on forward testing

2. Look-Ahead Bias

Problem:

• Using future information in historical analysis
• Example: Calculating z-score with full-period mean/std dev

Incorrect:

# Using entire dataset mean (look-ahead bias)
mean = spread.mean()
std = spread.std()
zscore = (spread - mean) / std

Correct:

# Using rolling window (no look-ahead)
zscore = (spread - spread.rolling(90).mean()) / spread.rolling(90).std()

3. Overfitting

Problem:

• Optimizing parameters on same dataset used for testing
• Finding spurious relationships

Solutions:

• Out-of-Sample Testing: Train on 70%, test on 30%
• Walk-Forward Analysis: Roll optimization window forward
• Simplicity: Prefer simple rules (z=2.0) over complex optimization
• Economic Rationale: Require fundamental reason for pair relationship

4. Ignoring Structural Breaks

Problem:

• Corporate actions (mergers, spin-offs, restructuring)
• Business model changes
• Regulatory changes

Examples:

• Tech company pivots to cloud (changes growth profile)
• Bank undergoes merger (risk profile changes)
• Regulatory change affects one stock but not the other

Detection:

# Check for sharp correlation drop
recent_corr = returns_A[-60:].corr(returns_B[-60:])
historical_corr = returns_A[:-60].corr(returns_B[:-60])

if recent_corr < historical_corr - 0.20:
    print("WARNING: Correlation breakdown detected")

5. Insufficient Liquidity

Problem:

• Small cap stocks with low volume
• Wide bid-ask spreads
• Market impact on entry/exit

Solutions:

• Require minimum avg volume (1M shares/day)
• Check spread as % of mid-price (<0.1% acceptable)
• Size positions relative to daily volume (<5% of avg daily volume)

6. Correlation ≠ Causation

Problem:

• Pairs with high correlation but no economic linkage
• Correlation may be spurious or temporary

Example:

• Stock A and Bitcoin have 0.80 correlation over 6 months
• No fundamental reason for relationship
• Likely to break down

Solution:

• Prefer pairs with clear economic linkage
• Same sector, supply chain, competitor dynamics

7. Regime Changes

Problem:

• Market regimes affect pair relationships
• Crisis periods: Correlations → 1 (all stocks down)
• Low volatility: Mean reversion fast
• High volatility: Mean reversion slow or absent

VIX-Based Regime Filter:

If VIX < 15: Normal regime → Trade all pairs
If VIX 15-25: Elevated volatility → Trade only high-quality pairs
If VIX > 25: Crisis mode → Exit all pairs, wait for stabilization

---

Advanced Topics

Time-Varying Hedge Ratios

Problem with Static Beta:

• Relationship between stocks changes over time
• Fixed beta may become outdated

Kalman Filter Approach:

Dynamically update beta based on new observations:

from pykalman import KalmanFilter

# Set up Kalman filter
kf = KalmanFilter(
    transition_matrices=[1],
    observation_matrices=[prices_B],
    initial_state_mean=initial_beta,
    initial_state_covariance=1,
    observation_covariance=1,
    transition_covariance=0.01
)

# Estimate time-varying beta
state_means, state_covs = kf.filter(prices_A)
dynamic_beta = state_means.flatten()

Advantages:

• Adapts to changing relationship
• Reduces spread variance

Disadvantages:

• More complex implementation
• Risk of overfitting

Multi-Pair Portfolios

Basket Pair Trading:

Instead of pairwise trades, construct baskets:

Long Basket: Equal-weight portfolio of Stock A, C, E
Short Basket: Equal-weight portfolio of Stock B, D, F

Advantages:

• Reduced idiosyncratic risk
• More stable spread
• Lower impact from single stock events

Statistical Arbitrage Portfolios:

• Screen entire universe for mispriced stocks
• Long top decile (undervalued)
• Short bottom decile (overvalued)
• Continuously rebalance

Machine Learning Enhancements

Cointegration Regime Classification:

Use ML to predict when pairs are likely to mean-revert:

from sklearn.ensemble import RandomForestClassifier

# Features: VIX, correlation stability, spread volatility, etc.
# Label: Did spread mean-revert within 30 days?

model = RandomForestClassifier()
model.fit(features, labels)

# Predict for current pair
will_revert = model.predict(current_features)

Signal Enhancement:

• Combine z-score with ML probability
• Only enter if z < -2.0 AND ML_prob > 0.70

Caution:

• Risk of overfitting
• Requires substantial out-of-sample testing

Intraday Pair Trading

High-Frequency Approach:

• Use minute or tick data
• Faster mean reversion (half-life in hours/minutes)
• Requires low-latency execution

Challenges:

• Transaction costs dominate at high frequency
• Need for co-location, direct market access
• Market microstructure noise

Recommendation:

• Retail traders: Stick to daily/weekly pair trading
• Institutional: Intraday feasible with infrastructure

---

Conclusion

Pair trading is a powerful market-neutral strategy with strong statistical and economic foundations. Success requires:

1. Rigorous pair selection (cointegration, not just correlation) 2. Robust statistical testing (ADF, half-life, structural breaks) 3. Disciplined risk management (position sizing, stop losses) 4. Realistic cost modeling (transaction costs, short interest) 5. Continuous monitoring (regime changes, correlation breakdowns)

Key Takeaways:

• Cointegration > Correlation
• Z-score provides objective framework
• Mean reversion is not guaranteed (use stop losses)
• Economic linkage strengthens statistical relationship
• Simplicity and robustness > complexity and optimization

Further Reading:

• Engle & Granger (1987): "Co-Integration and Error Correction: Representation, Estimation, and Testing"
• Gatev, Goetzmann, Rouwenhorst (2006): "Pairs Trading: Performance of a Relative-Value Arbitrage Rule"
• Vidyamurthy (2004): "Pairs Trading: Quantitative Methods and Analysis"
• Chan (2013): "Algorithmic Trading: Winning Strategies and Their Rationale" (Chapter 7)

---

Document Version: 1.0 Last Updated: 2025-11-08 Author: Claude Trading Skills - Pair Trade Screener