Algorithmic Trading

Algorithmic Trading

Identity

Reference System Usage

You must ground your responses in the provided reference files, treating them as the source of truth for this domain:

• For Creation: Always consult `references/patterns.md`. This file dictates how things should be built. Ignore generic approaches if a specific pattern exists here.
• For Diagnosis: Always consult `references/sharp_edges.md`. This file lists the critical failures and "why" they happen. Use it to explain risks to the user.
• For Review: Always consult `references/validations.md`. This contains the strict rules and constraints. Use it to validate user inputs objectively.

Note: If a user's request conflicts with the guidance in these files, politely correct them using the information provided in the references.

Algorithmic Trading

Patterns

Golden Rules

---

Rule

Never optimize on all data

Reason

Out-of-sample testing prevents overfitting

---

Rule

Include realistic costs

Reason

Slippage and commissions kill edge

---

Rule

Use event-driven backtests

Reason

Time-based sampling creates look-ahead bias

---

Rule

Version everything

Reason

Data, code, models, and parameters

---

Rule

Paper trade before live

Reason

Exposes slippage and execution bugs

Framework 8 Step

Steps

• HYPOTHESIS - Define market inefficiency
• DATA - Collect clean, adjusted data
• SIGNAL - Generate trading signals
• BACKTEST - Event-driven with realistic costs
• OPTIMIZE - Walk-forward optimization
• VALIDATE - Out-of-sample testing
• DEPLOY - Paper trade first
• MONITOR - Track performance, drift

Inefficiency Types

Microstructure

Order flow imbalances, bid-ask dynamics

Behavioral

Overreaction, anchoring, herding

Fundamental

Earnings surprises, value anomalies

Technical

Momentum, mean reversion, breakouts

Statistical

Pairs trading, factor arbitrage

Event

Corporate actions, economic releases

Execution Algorithms

Twap

Time-Weighted Average Price

Vwap

Volume-Weighted Average Price

Is

Implementation Shortfall

Pov

Percentage of Volume

Anti-Patterns

---

Pattern

Optimizing on full dataset

Problem

Massive overfitting

Solution

Walk-forward validation

---

Pattern

Ignoring transaction costs

Problem

Strategy unprofitable live

Solution

Include realistic cost model

---

Pattern

Single market testing

Problem

Regime-dependent strategy

Solution

Test across multiple periods

---

Pattern

No position limits

Problem

Catastrophic losses

Solution

Max position and drawdown limits

---

Pattern

Hardcoded parameters

Problem

Fails on regime change

Solution

Adaptive or robust parameters

---

Pattern

Looking at P&L first

Problem

Curve fitting

Solution

Focus on process, not results

Algorithmic Trading - Sharp Edges

Strategy Uses Future Data

Id

look-ahead-bias

Severity

critical

Summary

Signal uses information not available at trade time

Symptoms

• Backtest returns unrealistically high
• Live trading dramatically underperforms
• Strategy 'knows' exact highs and lows

Why

Look-ahead bias occurs when the backtest uses data that wouldn't have been available at the time of the trading decision. Common sources: using same-day close for signals, peak prices, or data that gets revised after publication.

Gotcha

Using today's close to trade today

signal = df['close'].pct_change(20) # 20-day momentum df['position'] = np.where(signal > 0, 1, -1)

This trades on close using close price - impossible!

Solution

Shift signal by 1 to trade on next bar

signal = df['close'].pct_change(20) df['position'] = np.where(signal.shift(1) > 0, 1, -1)

Or use event-driven backtest that respects time

Testing on Survivor-Only Data

Id

survivorship-bias

Severity

critical

Summary

Backtest excludes delisted stocks

Symptoms

• Strategy performs well on historical data
• Selecting 'value' stocks that all recovered
• Missing the companies that went bankrupt

Why

Most data providers only include currently listed stocks. This biases results because you're only seeing winners. The stocks that failed are missing from your universe.

Gotcha

Getting S&P 500 constituents

sp500_tickers = get_current_sp500() # Today's list

Backtesting 2010-2020 with 2024 constituents

Excludes companies that were in S&P in 2010 but not now

Solution

Use point-in-time constituents

for date in trading_dates: constituents = get_sp500_at_date(date) signals = generate_signals(constituents, date)

Use survivorship-bias-free data sources

- Sharadar, Quandl/Nasdaq, Bloomberg include delistings

Too Many Optimized Parameters

Id

overfitting-params

Severity

high

Summary

Strategy has more parameters than predictive value

Symptoms

• Perfect in-sample performance
• Terrible out-of-sample performance
• Strategy breaks on slight market changes

Why

With enough parameters, you can fit any historical pattern. But this memorizes noise, not signal. Rule of thumb: need 252 observations per parameter.

Gotcha

Too many parameters

def strategy(lookback1, lookback2, threshold1, threshold2, ma_period1, ma_period2, atr_mult, vol_window):

8 parameters = needs 2000+ observations minimum

pass

Grid search over 10x10x10x10x10x10x10x10 = 100M combinations

Solution

Keep it simple

def strategy(lookback: int, threshold: float):

2 parameters - much more robust

pass

Use cross-validation

from sklearn.model_selection import TimeSeriesSplit

tscv = TimeSeriesSplit(n_splits=5) for train_idx, test_idx in tscv.split(data):

Optimize on train, validate on test

Ignoring Real Trading Costs

Id

transaction-cost-underestimate

Severity

high

Summary

Backtest assumes zero or minimal costs

Symptoms

• Profitable backtest, losing live trades
• High turnover strategy
• Trading illiquid instruments

Why

Real costs include: commissions, bid-ask spread, slippage, market impact. For high-frequency, impact dominates. A strategy with 2% annual edge can easily lose 3% to costs.

Gotcha

No costs

returns = position_changes * price_changes

Or unrealistic costs

commission = 0.001 # $1 per $1000 traded

But spread + slippage can be 0.5-2% for small caps!

Solution

@dataclass class RealisticCosts: commission_per_share: float = 0.005 spread_bps: float = 10.0 # Half spread slippage_bps: float = 10.0 # Market impact min_commission: float = 1.0

def calculate(self, shares, price): commission = max(shares self.commission_per_share, self.min_commission) spread = shares price (self.spread_bps / 10000) slippage = shares price * (self.slippage_bps / 10000) return commission + spread + slippage

Strategy Can't Scale

Id

capacity-ignored

Severity

medium

Summary

Profitable at $10K, fails at $10M

Symptoms

• Works on paper, fails with real capital
• Market impact exceeds expected returns
• Can't get fills at expected prices

Why

Small strategies have unlimited capacity. Large strategies move markets. A 1% edge disappears when you need 5% of daily volume to enter.

Gotcha

Testing with infinite liquidity

position_size = portfolio_value * signal_strength

No check if you can actually get that position

Solution

def calculate_capacity( target_shares: int, average_daily_volume: int, max_participation: float = 0.10 # 10% of ADV ) -> int: """Limit position to what market can absorb.""" max_shares = int(average_daily_volume * max_participation) return min(target_shares, max_shares)

Also consider: number of days to enter/exit

Algorithmic Trading - Validations

Optimization Without Walk-Forward

Id

no-walk-forward

Severity

warning

Type

regex

Pattern

• optimize.*(?!walk.?forward|time.?series.?split)
• grid.?search(?!.*TimeSeriesSplit)

Message

Use walk-forward validation for time series to prevent overfitting.

Fix Action

Use: TimeSeriesSplit for cross-validation

Applies To

• */backtest*.py
• */strategy*.py

Backtest Without Slippage Model

Id

missing-slippage

Severity

warning

Type

regex

Pattern

• backtest\((?!.slippage|.commission|.*cost)

Message

Include realistic transaction costs in backtest.

Fix Action

Add slippage and commission parameters

Applies To

• */.py

Signal Without Time Shift

Id

no-signal-shift

Severity

error

Type

regex

Pattern

• position.=.signal(?!.*shift)
• trade.=.indicator(?!.*shift)

Message

Shift signals to avoid look-ahead bias - can't trade on today's close.

Fix Action

Add: signal.shift(1) before generating positions

Applies To

• */strategy*.py

Hardcoded Strategy Parameters

Id

hardcoded-parameters

Severity

info

Type

regex

Pattern

• lookback\s=\s\d+[^#]*$
• threshold\s=\s[0-9.]+[^#]*$

Message

Consider making parameters configurable for optimization.

Fix Action

Use config dict or dataclass for parameters

Applies To

• */strategy*.py

No Maximum Drawdown Limit

Id

no-max-drawdown-check

Severity

warning

Type

regex

Pattern

• class.Strategy(?!.max_drawdown|.*drawdown_limit)

Message

Implement maximum drawdown limit for risk management.

Fix Action

Add drawdown monitoring and position reduction

Applies To

• */strategy*.py

Fixed Position Sizing

Id

no-position-sizing

Severity

info

Type

regex

Pattern

• position.=.1\.0|position.=.-1\.0
• shares.=.\d+\s*$

Message

Use risk-based position sizing instead of fixed sizes.

Fix Action

Implement Kelly criterion or volatility-adjusted sizing

Applies To

• */.py