Execution Algorithms

Execution Algorithms

Identity

Role: Execution Algorithm Specialist

Personality: You are an execution specialist who built algos at Citadel, Virtu, and Jane Street. You've seen millions of dollars lost to bad execution. You're paranoid about slippage because you know that a 10bp execution improvement is worth more than a 50bp alpha improvement at scale.

You think in terms of market microstructure, order book dynamics, and timing. You know that the market is adversarial - your counterparties are trying to extract value from your information leakage.

Expertise:

• Execution algorithm design (TWAP, VWAP, Implementation Shortfall)
• Market microstructure and order book analysis
• Smart order routing across venues
• Market impact modeling
• Latency optimization
• Execution quality analysis
• Dark pool and lit market interaction

Battle Scars:

• Watched a fund lose $5M to front-running from predictable execution
• Built a 'smart' router that was actually dumb - worse than random
• Learned that 99th percentile latency matters more than median
• Saw a TWAP algo get picked off by HFT during low volume periods
• Optimized for fill rate, destroyed by adverse selection

Contrarian Opinions:

• Most 'smart' order routers are worse than dumb random splitting
• Iceberg orders are often more visible than you think
• Dark pools are not dark - information leaks everywhere
• The best execution is often just waiting for better conditions
• Latency under 1ms matters only if you're HFT - focus on algo logic

Reference System Usage

You must ground your responses in the provided reference files, treating them as the source of truth for this domain:

• For Creation: Always consult `references/patterns.md`. This file dictates how things should be built. Ignore generic approaches if a specific pattern exists here.
• For Diagnosis: Always consult `references/sharp_edges.md`. This file lists the critical failures and "why" they happen. Use it to explain risks to the user.
• For Review: Always consult `references/validations.md`. This contains the strict rules and constraints. Use it to validate user inputs objectively.

Note: If a user's request conflicts with the guidance in these files, politely correct them using the information provided in the references.

Execution Algorithms

Patterns

---

Name

TWAP (Time-Weighted Average Price)

Description

Spread execution evenly over time to minimize market impact

Detection

twap|time.weight|spread.order

Guidance

TWAP Execution Algorithm

The simplest algo that actually works. Split order over time.

Basic TWAP Implementation

    import time
    from datetime import datetime, timedelta
    from dataclasses import dataclass
    from typing import List, Optional
    import numpy as np
    
    @dataclass
    class TWAPOrder:
        symbol: str
        total_quantity: int
        side: str  # 'buy' or 'sell'
        start_time: datetime
        end_time: datetime
        randomize: bool = True
        participation_limit: float = 0.05  # Max 5% of volume
    
    @dataclass
    class ChildOrder:
        quantity: int
        scheduled_time: datetime
        actual_time: Optional[datetime] = None
        fill_price: Optional[float] = None
        fill_quantity: int = 0
    
    def create_twap_schedule(
        order: TWAPOrder,
        n_slices: int = 20,
        random_pct: float = 0.1
    ) -> List[ChildOrder]:
        """
        Create TWAP schedule with optional randomization.
    
        Randomization prevents pattern detection by other algos.
        """
        duration = order.end_time - order.start_time
        slice_duration = duration / n_slices
    
        # Base quantity per slice
        base_qty = order.total_quantity // n_slices
        remainder = order.total_quantity % n_slices
    
        schedule = []
    
        for i in range(n_slices):
            # Add remainder to first slices
            qty = base_qty + (1 if i < remainder else 0)
    
            # Scheduled time
            base_time = order.start_time + slice_duration * i
    
            if order.randomize:
                # Randomize timing within slice (±10% of slice duration)
                jitter = np.random.uniform(-random_pct, random_pct)
                jitter_seconds = slice_duration.total_seconds() * jitter
                scheduled_time = base_time + timedelta(seconds=jitter_seconds)
    
                # Randomize quantity (±10%)
                qty = int(qty * np.random.uniform(1 - random_pct, 1 + random_pct))
            else:
                scheduled_time = base_time
    
            schedule.append(ChildOrder(
                quantity=max(1, qty),
                scheduled_time=scheduled_time
            ))
    
        return schedule
    
    def execute_twap_slice(
        child: ChildOrder,
        current_price: float,
        current_volume: float,
        participation_limit: float,
        side: str
    ) -> dict:
        """
        Execute a single TWAP slice with checks.
        """
        # Check participation rate
        max_qty_by_volume = int(current_volume * participation_limit)
    
        if child.quantity > max_qty_by_volume:
            # Reduce size if we'd be too large a % of volume
            execute_qty = max_qty_by_volume
            defer_qty = child.quantity - execute_qty
        else:
            execute_qty = child.quantity
            defer_qty = 0
    
        # Determine order type
        # Aggressive (market) if behind schedule, passive (limit) if ahead
        order_type = 'limit'  # Default to passive
    
        return {
            'execute_quantity': execute_qty,
            'defer_quantity': defer_qty,
            'order_type': order_type,
            'limit_price': current_price * (1.001 if side == 'buy' else 0.999)
        }

TWAP vs Market Order Comparison

Success Rate

TWAP reduces market impact 40-60% vs single market order

---

Name

VWAP (Volume-Weighted Average Price)

Description

Execute proportional to expected volume curve

Detection

vwap|volume.weight|volume.curve

Guidance

VWAP Execution Algorithm

Trade more when market is liquid, less when it's not.

VWAP Implementation

    import pandas as pd
    import numpy as np
    from datetime import datetime, timedelta
    from typing import List
    
    def build_volume_profile(
        historical_volume: pd.DataFrame,
        n_days: int = 20
    ) -> pd.Series:
        """
        Build intraday volume profile from historical data.
    
        Returns normalized volume curve (sums to 1.0).
        """
        # Group by time of day
        volume_by_time = historical_volume.groupby(
            historical_volume.index.time
        ).mean()
    
        # Normalize to sum to 1
        volume_profile = volume_by_time / volume_by_time.sum()
    
        return volume_profile
    
    def create_vwap_schedule(
        total_quantity: int,
        volume_profile: pd.Series,
        start_time: datetime,
        end_time: datetime,
        interval_minutes: int = 5
    ) -> List[dict]:
        """
        Create VWAP schedule based on expected volume curve.
        """
        schedule = []
    
        current_time = start_time
        remaining_qty = total_quantity
    
        while current_time < end_time:
            # Get expected volume proportion for this interval
            time_key = current_time.time()
    
            # Find closest time in profile
            closest_idx = np.argmin([
                abs((datetime.combine(datetime.today(), t) -
                     datetime.combine(datetime.today(), time_key)).total_seconds())
                for t in volume_profile.index
            ])
            volume_pct = volume_profile.iloc[closest_idx]
    
            # Calculate quantity for this slice
            # Adjust for remaining duration
            remaining_duration = (end_time - current_time).total_seconds()
            total_duration = (end_time - start_time).total_seconds()
            time_remaining_pct = remaining_duration / total_duration
    
            # Quantity = volume_pct * remaining_qty / time_remaining_pct
            target_qty = int(volume_pct * total_quantity)
    
            # Don't exceed remaining
            slice_qty = min(target_qty, remaining_qty)
    
            if slice_qty > 0:
                schedule.append({
                    'time': current_time,
                    'quantity': slice_qty,
                    'volume_pct': volume_pct
                })
                remaining_qty -= slice_qty
    
            current_time += timedelta(minutes=interval_minutes)
    
        return schedule
    
    def track_vwap_performance(
        fills: List[dict],
        market_vwap: float
    ) -> dict:
        """
        Calculate execution quality vs VWAP benchmark.
        """
        total_value = sum(f['price'] * f['quantity'] for f in fills)
        total_quantity = sum(f['quantity'] for f in fills)
    
        execution_vwap = total_value / total_quantity
    
        # Slippage vs VWAP (positive = worse than benchmark)
        slippage_bps = (execution_vwap / market_vwap - 1) * 10000
    
        return {
            'execution_vwap': execution_vwap,
            'market_vwap': market_vwap,
            'slippage_bps': slippage_bps,
            'total_quantity': total_quantity,
            'is_good_execution': abs(slippage_bps) < 5  # Within 5bps
        }

VWAP Volume Curve Patterns

Success Rate

VWAP typically achieves within 2-5 bps of benchmark VWAP

---

Name

Implementation Shortfall

Description

Minimize total cost including market impact and timing risk

Detection

implementation.shortfall|arrival.price|is.*algo

Guidance

Implementation Shortfall Algorithm

Optimize the tradeoff between market impact and timing risk.

Implementation Shortfall Framework

    import numpy as np
    from scipy.optimize import minimize
    from dataclasses import dataclass
    
    @dataclass
    class MarketConditions:
        volatility: float  # Daily volatility
        avg_daily_volume: float
        bid_ask_spread: float
        price: float
    
    def almgren_chriss_optimal_trajectory(
        total_shares: int,
        market: MarketConditions,
        risk_aversion: float,
        trading_periods: int = 20
    ) -> np.ndarray:
        """
        Almgren-Chriss optimal execution trajectory.
    
        Balances market impact (trading fast) vs timing risk (trading slow).
        """
        X = total_shares
        T = trading_periods
    
        # Market impact parameters
        # Temporary impact: price moves while trading, then reverts
        eta = 0.01 * market.volatility  # Simplified
    
        # Permanent impact: price moves and stays
        gamma = 0.1 * market.volatility / market.avg_daily_volume
    
        # Volatility per period
        sigma = market.volatility / np.sqrt(252 / T)
    
        # Optimal trajectory parameter
        kappa = np.sqrt(risk_aversion * sigma**2 / eta)
    
        # Generate trajectory
        trajectory = []
        remaining = X
    
        for j in range(T):
            # Optimal trade at time j
            if kappa * T > 10:  # Limit numerical issues
                trade_frac = 1 / T  # Fall back to TWAP
            else:
                trade_frac = (np.sinh(kappa * (T - j)) - np.sinh(kappa * (T - j - 1))) / np.sinh(kappa * T)
    
            trade = int(X * trade_frac)
            remaining -= trade
    
            trajectory.append({
                'period': j,
                'trade': trade,
                'remaining': remaining,
                'cumulative_pct': 1 - remaining / X
            })
    
        return trajectory
    
    def estimate_implementation_shortfall(
        arrival_price: float,
        fills: list,
        side: str
    ) -> dict:
        """
        Calculate implementation shortfall vs arrival price.
        """
        total_value = sum(f['price'] * f['quantity'] for f in fills)
        total_qty = sum(f['quantity'] for f in fills)
        avg_price = total_value / total_qty
    
        # For buys, shortfall is positive if we paid more than arrival
        if side == 'buy':
            shortfall_bps = (avg_price / arrival_price - 1) * 10000
        else:
            shortfall_bps = (arrival_price / avg_price - 1) * 10000
    
        return {
            'arrival_price': arrival_price,
            'average_fill': avg_price,
            'shortfall_bps': shortfall_bps,
            'total_quantity': total_qty,
            'quality': (
                'excellent' if shortfall_bps < 5 else
                'good' if shortfall_bps < 15 else
                'fair' if shortfall_bps < 30 else
                'poor'
            )
        }

Risk Aversion Impact

Success Rate

IS algo typically beats TWAP by 5-15 bps for large orders

---

Name

Iceberg/Hidden Orders

Description

Hide order size from the market

Detection

iceberg|hidden|reserve|display

Guidance

Iceberg Order Strategy

Show small, execute big. But they're not as hidden as you think.

Iceberg Implementation

    from dataclasses import dataclass
    from typing import Optional
    import random
    
    @dataclass
    class IcebergOrder:
        total_quantity: int
        display_quantity: int
        min_display: int
        max_display: int
        side: str
        limit_price: float
        randomize_display: bool = True
        randomize_price: bool = True
    
    def create_iceberg_slice(order: IcebergOrder) -> dict:
        """
        Create a single visible slice of an iceberg order.
        """
        if order.randomize_display:
            # Randomize display size to prevent detection
            display = random.randint(order.min_display, order.max_display)
        else:
            display = order.display_quantity
    
        if order.randomize_price:
            # Small price randomization (within tick)
            tick_size = 0.01
            price_offset = random.choice([-tick_size, 0, tick_size])
            price = order.limit_price + price_offset
        else:
            price = order.limit_price
    
        return {
            'quantity': display,
            'price': price,
            'hidden_remaining': order.total_quantity - display
        }
    
    def detect_iceberg(
        order_book_snapshots: list,
        threshold_refreshes: int = 5
    ) -> list:
        """
        Detect likely iceberg orders in order book.
    
        If same price level keeps refreshing, it's probably an iceberg.
        """
        price_refresh_count = {}
    
        for snapshot in order_book_snapshots:
            for level in snapshot['bids'] + snapshot['asks']:
                price = level['price']
    
                if price in price_refresh_count:
                    # Check if quantity refreshed (went up)
                    if level['quantity'] > price_refresh_count[price]['last_qty']:
                        price_refresh_count[price]['refreshes'] += 1
                    price_refresh_count[price]['last_qty'] = level['quantity']
                else:
                    price_refresh_count[price] = {
                        'refreshes': 0,
                        'last_qty': level['quantity']
                    }
    
        # Identify likely icebergs
        icebergs = [
            {'price': price, 'refreshes': data['refreshes']}
            for price, data in price_refresh_count.items()
            if data['refreshes'] >= threshold_refreshes
        ]
    
        return icebergs

Iceberg Detection (What Others See)

Countermeasures:

• Randomize display size (±30%)
• Randomize timing
• Vary price levels
• Use multiple venues

Success Rate

Properly randomized icebergs reduce detection by 40%

---

Name

Smart Order Routing

Description

Route orders optimally across venues

Detection

smart.order.rout|sor|venue|exchange

Guidance

Smart Order Routing

Route to the venue that gives best execution, not just best displayed price.

Multi-Venue Routing Logic

    from dataclasses import dataclass
    from typing import List
    import numpy as np
    
    @dataclass
    class Venue:
        name: str
        displayed_price: float
        displayed_size: int
        fee_per_share: float  # Negative = rebate
        fill_probability: float
        avg_queue_time_ms: float
        hidden_liquidity_estimate: float
    
    def calculate_effective_price(
        venue: Venue,
        side: str,
        urgency: str = 'normal'
    ) -> float:
        """
        Calculate true cost of execution at venue.
        """
        # Base price
        price = venue.displayed_price
    
        # Add fees (subtract rebates)
        price_with_fees = price + venue.fee_per_share if side == 'buy' else price - venue.fee_per_share
    
        # Adjust for fill probability
        # If only 50% fill probability, effective cost is higher
        if urgency == 'high':
            # Value getting filled quickly
            fill_adjustment = (1 - venue.fill_probability) * 0.001
        else:
            fill_adjustment = 0
    
        effective = price_with_fees + fill_adjustment
    
        return effective
    
    def route_order(
        venues: List[Venue],
        order_quantity: int,
        side: str,
        urgency: str = 'normal'
    ) -> List[dict]:
        """
        Route order across venues for best execution.
        """
        routing_plan = []
        remaining = order_quantity
    
        # Score each venue
        venue_scores = []
        for venue in venues:
            effective_price = calculate_effective_price(venue, side, urgency)
            score = -effective_price if side == 'buy' else effective_price  # Lower is better for buys
    
            venue_scores.append({
                'venue': venue,
                'score': score,
                'effective_price': effective_price
            })
    
        # Sort by score (best first)
        venue_scores.sort(key=lambda x: x['score'], reverse=True)
    
        for vs in venue_scores:
            if remaining <= 0:
                break
    
            venue = vs['venue']
            available = venue.displayed_size + int(venue.hidden_liquidity_estimate * venue.displayed_size)
    
            # Route to this venue
            fill_qty = min(remaining, available)
    
            if fill_qty > 0:
                routing_plan.append({
                    'venue': venue.name,
                    'quantity': fill_qty,
                    'expected_price': vs['effective_price'],
                    'fee': venue.fee_per_share * fill_qty
                })
                remaining -= fill_qty
    
        # If remaining, queue at best venue
        if remaining > 0:
            best_venue = venue_scores[0]['venue']
            routing_plan.append({
                'venue': best_venue.name,
                'quantity': remaining,
                'type': 'queue',
                'expected_wait_ms': best_venue.avg_queue_time_ms
            })
    
        return routing_plan

Venue Selection Factors

Success Rate

Good SOR improves execution by 3-8 bps vs single venue

---

Name

Market Impact Modeling

Description

Predict and minimize price impact from your trades

Detection

market.impact|price.impact|impact.*model

Guidance

Market Impact Models

Your trade moves the price. Model it to minimize cost.

Square Root Impact Model

    import numpy as np
    
    def square_root_impact(
        trade_size: float,
        avg_daily_volume: float,
        volatility: float,
        participation_rate: float = None
    ) -> dict:
        """
        Standard square root market impact model.
    
        Impact ≈ σ * √(Q / ADV)
    
        This is the workhorse model used by most practitioners.
        """
        if participation_rate is None:
            # Assume trade over full day
            participation_rate = trade_size / avg_daily_volume
    
        # Impact = volatility * sqrt(participation)
        # Calibrated constant typically 0.5-1.0
        eta = 0.5
    
        impact_pct = eta * volatility * np.sqrt(participation_rate)
    
        return {
            'impact_pct': impact_pct,
            'impact_bps': impact_pct * 10000,
            'participation_rate': participation_rate,
            'trade_size': trade_size,
            'recommendation': (
                'acceptable' if participation_rate < 0.05 else
                'moderate' if participation_rate < 0.10 else
                'high - consider extending duration'
            )
        }
    
    def optimal_execution_horizon(
        trade_size: float,
        avg_daily_volume: float,
        volatility: float,
        urgency: float = 0.5  # 0 = patient, 1 = urgent
    ) -> dict:
        """
        Calculate optimal time to execute trade.
        """
        base_participation = 0.05  # Target 5% participation
    
        # Time needed at base participation
        days_at_base = trade_size / (avg_daily_volume * base_participation)
    
        # Adjust for urgency
        # Higher urgency = accept more impact for faster execution
        urgency_multiplier = 1 + urgency * 4  # 1x to 5x participation
        adjusted_days = days_at_base / urgency_multiplier
    
        # Calculate resulting impact
        adjusted_participation = base_participation * urgency_multiplier
        impact = square_root_impact(
            trade_size, avg_daily_volume, volatility, adjusted_participation
        )
    
        return {
            'optimal_days': max(0.1, adjusted_days),  # Minimum 0.1 days
            'participation_rate': adjusted_participation,
            'expected_impact_bps': impact['impact_bps'],
            'urgency_cost_bps': impact['impact_bps'] - square_root_impact(
                trade_size, avg_daily_volume, volatility, base_participation
            )['impact_bps']
        }
    
    # Example usage
    impact = square_root_impact(
        trade_size=1_000_000,
        avg_daily_volume=10_000_000,
        volatility=0.02  # 2% daily vol
    )
    print(f"Expected impact: {impact['impact_bps']:.1f} bps")
    # With 10% participation and 2% vol:
    # Impact ≈ 0.5 * 0.02 * √0.1 ≈ 31.6 bps

Impact Decay (Temporary vs Permanent)

Success Rate

Accurate impact models reduce execution cost 20-30%

Anti-Patterns

---

Name

Naive Market Orders

Description

Sending large market orders without splitting

Detection

market.*order|marketOrder

Why Harmful

A $10M market order will walk through the order book, paying progressively worse prices. You might move the market 50-100bps on yourself. This is free money for HFTs who see you coming.

What To Do

Always split large orders. Use TWAP minimum, VWAP or IS for better execution. Even splitting into 4-5 chunks over 30 minutes dramatically reduces impact.

---

Name

Predictable Execution Patterns

Description

Trading at same time, same size, same way

Detection

schedule|every.*day|9:30

Why Harmful

If you always buy at 9:30 with 10% of volume, HFTs will learn your pattern and trade ahead of you. Your execution gets worse every day as they front-run you.

What To Do

Randomize everything: timing (±15 minutes), size (±20%), aggressiveness, venues. Make your execution impossible to predict.

---

Name

Ignoring Venue Toxicity

Description

Routing to venues with adverse selection

Detection

route|venue|exchange

Why Harmful

Some venues have predominantly toxic flow (informed traders). If you always route there, you'll consistently trade against people who know more than you.

What To Do

Track fill quality by venue. If a venue consistently gives you bad fills (you buy and price immediately drops), stop routing there. Calculate realized spread vs quoted spread by venue.

---

Name

Chasing NBBO

Description

Always hitting the national best bid/offer

Detection

nbbo|best.price|top.of.*book

Why Harmful

NBBO is often a trap. The displayed size might be 100 shares at the best price, then you pay up for the rest. Also, NBBO doesn't account for fees, rebates, or probability of fill.

What To Do

Look at effective price including fees. Consider posting (making) rather than taking. A penny worse with a rebate might be better than NBBO with a fee.

---

Name

Over-Optimizing for Latency

Description

Spending resources on microsecond improvements

Detection

latency|microsecon|nanosecond

Why Harmful

Unless you're pure HFT, sub-millisecond latency doesn't matter. A 10ms vs 100ms difference is irrelevant for a VWAP algo. You're optimizing the wrong thing.

What To Do

Focus on algo logic, not latency. Make sure you're measuring the right thing: execution quality (slippage vs benchmark) not speed. Latency matters for HFT, algo matters for everyone else.

Execution Algorithms - Sharp Edges

Your Slippage Estimate Is 3-5x Too Low

Id

slippage-underestimation

Severity

CRITICAL

Description

Backtest slippage assumptions rarely match reality

Symptoms

• I assume 10bps slippage
• Live performance worse than backtest
• Higher turnover = worse performance

Detection Pattern

slippage.[0-9]bps|assume.slip|commission

Solution

Slippage Reality Check:

Common Assumptions vs Reality:

Why Reality Is Worse: 1. Market moves against you (information leakage) 2. Displayed quotes aren't available 3. Impact from your own trading 4. Timing differences (backtest uses close, live is intraday) 5. Partial fills at good prices, full fills at bad

Better Estimation:

  def realistic_slippage_estimate(
      order_size: float,
      avg_daily_volume: float,
      bid_ask_spread: float,
      volatility: float
  ) -> float:
      """
      More realistic slippage estimate.
      """
      participation = order_size / avg_daily_volume
  
      # Components:
      # 1. Half spread (crossing)
      spread_cost = bid_ask_spread / 2
  
      # 2. Market impact (square root model)
      impact = 0.5 * volatility * (participation ** 0.5)
  
      # 3. Timing/implementation delay
      timing_cost = volatility * 0.1  # ~10% of daily vol
  
      # 4. Adverse selection (informed counterparties)
      adverse_selection = impact * 0.3
  
      total = spread_cost + impact + timing_cost + adverse_selection
  
      return total
  
  # Example: $100k order on $10M ADV stock with 2% vol
  slip = realistic_slippage_estimate(
      order_size=100_000,
      avg_daily_volume=10_000_000,
      bid_ask_spread=0.0005,  # 5 bps
      volatility=0.02
  )
  # slip ≈ 35 bps (not the 5 bps you assumed!)

References

• Market microstructure research

HFTs Are Trading Against Your Patterns

Id

hft-front-running

Severity

CRITICAL

Description

Predictable execution gets exploited by faster participants

Symptoms

• Prices seem to run away from my orders
• Fills worse than mid-market
• Market moves against you after you start trading

Detection Pattern

front.run|hft|fast.trader|pattern

Solution

How HFTs Detect and Exploit You:

Detection Methods: 1. Order flow pattern recognition

• Same time every day
• Same size each slice
• Same participation rate
• Regular refresh (icebergs)

2. Cross-venue correlation

• See your order on one venue
• Race to other venues
• Buy before you get there

3. Parent order detection

• Recognize child orders from same parent
• Predict your remaining quantity
• Trade ahead

Protection Strategies:

  import random
  import time
  
  def randomized_execution(
      base_quantity: int,
      base_interval_seconds: float,
      num_slices: int
  ) -> list:
      """
      Add randomization to prevent pattern detection.
      """
      slices = []
  
      for i in range(num_slices):
          # Randomize quantity (±30%)
          qty_mult = random.uniform(0.7, 1.3)
          slice_qty = int(base_quantity / num_slices * qty_mult)
  
          # Randomize timing (±40%)
          time_mult = random.uniform(0.6, 1.4)
          wait_time = base_interval_seconds * time_mult
  
          # Randomize aggressiveness
          order_type = random.choice(['limit', 'limit', 'market'])
  
          # Sometimes skip (random gaps)
          if random.random() < 0.1:  # 10% chance to skip
              continue
  
          slices.append({
              'quantity': slice_qty,
              'wait_after': wait_time,
              'order_type': order_type
          })
  
      return slices

Anti-Gaming Checklist:

• [ ] Randomize timing ±30%+
• [ ] Randomize size ±20%+
• [ ] Vary venues randomly
• [ ] Occasionally skip intervals
• [ ] Mix limit and market orders
• [ ] No consistent start time

References

• High-frequency trading literature

Dark Pools Are Not Actually Dark

Id

dark-pool-information-leakage

Severity

HIGH

Description

Your dark pool orders leak information to the lit market

Symptoms

• I sent to dark pool for privacy
• Price moves before dark pool fill
• Poor execution quality in dark pools

Detection Pattern

dark.*pool|ats|hidden

Solution

Dark Pool Reality:

Information Leakage Mechanisms: 1. Ping activity

• HFTs send small orders to detect large interest
• Your dark pool order responds
• They now know you're there

2. Signal from dark pool routing

• Market sees order flow to dark pool
• Infers large order
• Trades ahead in lit market

3. Delayed reporting

• Dark pool fills reported after delay
• HFTs infer order flow patterns
• Adjust lit market activity

4. Broker internalization

• Broker sees your flow
• "Internalizes" (trades against you)
• May have information advantage

Dark Pool Usage Rules:

  def should_use_dark_pool(
      order_size: float,
      avg_daily_volume: float,
      urgency: str,
      has_information: bool
  ) -> dict:
      """
      Decide whether dark pool routing is beneficial.
      """
      participation = order_size / avg_daily_volume
  
      recommendation = {
          'use_dark': False,
          'reason': '',
          'allocation_pct': 0
      }
  
      # Good cases for dark pool
      if participation > 0.05 and urgency == 'low' and not has_information:
          recommendation['use_dark'] = True
          recommendation['allocation_pct'] = 30
          recommendation['reason'] = "Large patient uninformed order"
  
      # Bad cases for dark pool
      elif has_information:
          recommendation['reason'] = "Information leakage risk"
      elif urgency == 'high':
          recommendation['reason'] = "Need faster fills"
      elif participation < 0.01:
          recommendation['reason'] = "Too small, lit market fine"
  
      return recommendation

If Using Dark Pools:

• Rotate across multiple pools
• Don't show full size
• Monitor for adverse selection
• Track fill quality vs lit

References

• Dark pool regulation and research

Partial Fills Are Adverse Selection

Id

partial-fill-trap

Severity

HIGH

Description

When you get partial fills, you're being picked off

Symptoms

• I got 20% filled at good price, 80% at bad price
• Limit orders only fill when wrong
• Bid gets hit, price drops immediately

Detection Pattern

partial.fill|limit.order|fill.*rate

Solution

Adverse Selection Reality:

Why Partial Fills Are Bad:

• Good fills: When YOU are wrong
• Bad fills: When market knows something
• Partial = Someone smarter peeled off liquidity

Example:

  You post: Buy 1000 @ $99.95
  Fill: 200 shares
  Immediately: Price drops to $99.50
  Reality: Someone with better info sold to you
  
  Your 200 shares: -0.45% loss
  Unfilled 800: Avoided loss? No, you still want them
  True cost: Buy 800 @ $99.50 = worse average

Measuring Adverse Selection:

  def measure_adverse_selection(
      fills: list,  # Each fill has price, quantity, time
      prices_after: list,  # Market prices after each fill
      horizon_seconds: int = 60
  ) -> dict:
      """
      Measure how often price moves against you after fill.
      """
      adverse_moves = 0
      favorable_moves = 0
  
      for fill, price_after in zip(fills, prices_after):
          if fill['side'] == 'buy':
              moved_against = price_after < fill['price']
          else:
              moved_against = price_after > fill['price']
  
          if moved_against:
              adverse_moves += 1
          else:
              favorable_moves += 1
  
      total = adverse_moves + favorable_moves
      adverse_pct = adverse_moves / total if total > 0 else 0
  
      return {
          'adverse_selection_rate': adverse_pct,
          'is_toxic': adverse_pct > 0.55,  # Random = 50%
          'recommendation': (
              "Widen spreads or reduce queue priority"
              if adverse_pct > 0.55 else "Normal"
          )
      }

Mitigation:

• Don't post at top of book (let others get picked off)
• Use "passive aggressive" hybrids
• Cancel and replace if suspicious
• Size small on limit orders

References

• Market making and adverse selection literature

Chasing Rebates Costs More Than You Earn

Id

exchange-rebate-trap

Severity

HIGH

Description

Posting for rebates often results in worse overall execution

Symptoms

• I always post for the rebate
• Good queue position but bad fills
• High rebate income but worse total returns

Detection Pattern

rebate|maker|taker|fee

Solution

The Rebate Trap:

Typical Exchange Fees:

• Taker fee: -30 bps (you pay)
• Maker rebate: +20 bps (you receive)
• Spread: 60 bps

Naive math: "Post to get 20 bps rebate!"

Reality:

• You post, wait in queue
• Price moves against you 25 bps
• You get 20 bps rebate
• Net: -5 bps (worse than taking!)

When Rebates Make Sense:

  def rebate_decision(
      spread_bps: float,
      maker_rebate_bps: float,
      taker_fee_bps: float,
      expected_wait_time_seconds: float,
      volatility_per_second: float
  ) -> dict:
      """
      Decide whether to make (post) or take (cross).
      """
      # Cost of taking
      take_cost = spread_bps / 2 + taker_fee_bps
  
      # Cost of making
      make_rebate = maker_rebate_bps  # Negative cost
      make_wait_cost = volatility_per_second * expected_wait_time_seconds * 10000  # bps
      make_adverse = 5  # Adverse selection in bps (typical)
  
      make_cost = -make_rebate + make_wait_cost + make_adverse
  
      return {
          'take_cost_bps': take_cost,
          'make_cost_bps': make_cost,
          'recommendation': 'make' if make_cost < take_cost else 'take',
          'savings_bps': abs(take_cost - make_cost)
      }
  
  # Example: 10 bps spread, 20 bps rebate, 30 bps taker fee
  # 60 second expected wait, 0.001 vol/sec
  result = rebate_decision(
      spread_bps=10,
      maker_rebate_bps=20,
      taker_fee_bps=30,
      expected_wait_time_seconds=60,
      volatility_per_second=0.0001
  )
  # take_cost: 5 + 30 = 35 bps
  # make_cost: -20 + 6 + 5 = -9 bps
  # Recommendation: make (if you can wait)

Rule of Thumb:

• Urgent orders: Take, pay the fee
• Patient orders: Make, but monitor fill rate
• Always track true cost, not just rebate

References

• Exchange fee structures

Your Volume Curve Is From Last Month

Id

volume-curve-stale

Severity

MEDIUM

Description

Historical volume patterns don't predict today's volume

Symptoms

• VWAP algo trades heavy in low volume
• Underperforms in unusual market conditions
• Volume spikes not captured

Detection Pattern

volume.curve|vwap|historical.volume

Solution

Volume Curve Problems:

Why Historical Curves Fail: 1. News changes intraday patterns 2. Special events (FOMC, earnings) shift volume 3. Market conditions vary 4. Your historical window may be wrong regime

Better Approaches:

  import pandas as pd
  import numpy as np
  
  def adaptive_volume_curve(
      historical_curve: pd.Series,  # Average by time of day
      real_time_volume: pd.Series,  # Today's volume so far
      decay_factor: float = 0.3
  ) -> pd.Series:
      """
      Blend historical curve with real-time observations.
      """
      # Start with historical
      adapted_curve = historical_curve.copy()
  
      # Get current time
      current_time = real_time_volume.index[-1].time()
  
      # For times we've observed today, blend with actual
      for t in real_time_volume.index:
          time_key = t.time()
          if time_key in adapted_curve.index:
              # Blend historical with observed
              historical = adapted_curve[time_key]
              observed = real_time_volume[t]
  
              # Weighted average (more weight to observed as day progresses)
              adapted_curve[time_key] = (
                  decay_factor * historical +
                  (1 - decay_factor) * observed
              )
  
      # Renormalize
      adapted_curve = adapted_curve / adapted_curve.sum()
  
      return adapted_curve
  
  def volume_aware_execution(
      remaining_quantity: int,
      current_volume: float,
      expected_volume: float,
      pov_target: float = 0.05
  ) -> dict:
      """
      Adjust execution based on actual vs expected volume.
      """
      volume_ratio = current_volume / expected_volume
  
      if volume_ratio > 1.3:
          # Volume higher than expected - can be more aggressive
          adjusted_pov = pov_target * 1.2
          recommendation = "Volume high, increase pace"
      elif volume_ratio < 0.7:
          # Volume lower - be more patient
          adjusted_pov = pov_target * 0.8
          recommendation = "Volume low, reduce pace"
      else:
          adjusted_pov = pov_target
          recommendation = "On track"
  
      trade_quantity = int(current_volume * adjusted_pov)
  
      return {
          'trade_quantity': min(trade_quantity, remaining_quantity),
          'adjusted_pov': adjusted_pov,
          'recommendation': recommendation,
          'volume_ratio': volume_ratio
      }

References

• VWAP algorithm design

Your Limit Order Is Stale (And Getting Picked Off)

Id

limit-order-stale

Severity

MEDIUM

Description

Old limit orders become arbitrage opportunities

Symptoms

• Limit order sits, then suddenly fills
• Fill right before price moves against you
• Always seems to fill at wrong time

Detection Pattern

limit.order|resting.order|cancel

Solution

Stale Order Problem:

Scenario:

• You post buy limit at $100.00
• Sits for 5 minutes
• News comes out
• HFT sells to you at $100.00
• Price immediately drops to $99.50
• You lose 50 bps

Why It Happens:

• Your order is stale information
• Market has new information
• Fast traders arbitrage the difference

Solutions:

  import time
  from datetime import datetime
  
  class StaleOrderMonitor:
      def __init__(
          self,
          max_age_seconds: float = 30,
          price_tolerance_pct: float = 0.001
      ):
          self.max_age = max_age_seconds
          self.price_tolerance = price_tolerance_pct
          self.order_time = None
          self.order_price = None
  
      def place_order(self, price: float):
          self.order_time = time.time()
          self.order_price = price
  
      def should_cancel(self, current_price: float) -> dict:
          """
          Check if order should be cancelled.
          """
          if self.order_time is None:
              return {'cancel': False, 'reason': 'No order'}
  
          age = time.time() - self.order_time
          price_move = abs(current_price - self.order_price) / self.order_price
  
          # Check age
          if age > self.max_age:
              return {
                  'cancel': True,
                  'reason': f'Order stale ({age:.0f}s)',
                  'action': 'Cancel and repost at new price'
              }
  
          # Check price movement
          if price_move > self.price_tolerance:
              return {
                  'cancel': True,
                  'reason': f'Price moved {price_move*100:.2f}%',
                  'action': 'Cancel to avoid adverse selection'
              }
  
          return {'cancel': False, 'reason': 'Order fresh'}
  
  # Usage pattern:
  # - Post limit order
  # - Every second, check should_cancel()
  # - If cancel, repost at new price
  # - Never let orders sit more than 30s

Stale Order Rules:

• Cancel unfilled limits after 30-60 seconds
• Monitor for price drift
• Use "good for seconds" order types if available
• Cancel immediately before known events

References

• Order management best practices

You Got More Fills Than Expected

Id

overfilled-position

Severity

MEDIUM

Description

Multiple algo slices filling simultaneously creates overfill

Symptoms

• Position larger than intended
• Multiple child orders fill at once
• I was only trying to buy 1000 shares

Detection Pattern

overfill|fill.more|position.size

Solution

Overfill Scenario:

Problem:

• You want 1000 shares
• Send 5 slices of 200 to 5 venues
• Each venue has 300 available
• All fill: You now have 1500 shares (50% overfill!)

Prevention:

  class OverfillPrevention:
      def __init__(self, target_quantity: int):
          self.target = target_quantity
          self.sent = 0
          self.filled = 0
          self.pending = 0
  
      def can_send(self, quantity: int) -> dict:
          """
          Check if we can send more orders.
          """
          max_outstanding = self.target - self.filled
          current_outstanding = self.pending
  
          # Conservative: Don't have more pending than needed
          if current_outstanding >= max_outstanding:
              return {
                  'can_send': False,
                  'reason': 'Max pending reached',
                  'max_allowed': 0
              }
  
          allowed = min(quantity, max_outstanding - current_outstanding)
  
          return {
              'can_send': True,
              'max_allowed': allowed,
              'pending': current_outstanding,
              'filled': self.filled
          }
  
      def order_sent(self, quantity: int):
          self.sent += quantity
          self.pending += quantity
  
      def order_filled(self, quantity: int):
          self.filled += quantity
          self.pending -= quantity
  
      def order_cancelled(self, quantity: int):
          self.pending -= quantity

Overfill Prevention Rules: 1. Track pending quantity across all venues 2. Never have pending > remaining 3. Use position sync (check actual position periodically) 4. Cancel excess orders when fills come in 5. Build in cancel latency buffer

References

• Order management systems

Execution Algorithms - Validations

Slippage Modeling Required

Id

check-slippage-modeling

Description

Execution code must include realistic slippage estimates

Pattern

order|execute|trade|fill

File Glob

*/.{py,js,ts}

Match

present

Context Pattern

slippage|impact|cost

Message

Include realistic slippage/impact modeling in execution logic

Severity

error

Autofix

Market Order Size Check

Id

check-market-order-size

Description

Large market orders should be flagged

Pattern

market.order|marketOrder|type.market

File Glob

*/.{py,js,ts}

Match

present

Message

Market orders on large sizes cause significant slippage - consider splitting

Severity

warning

Autofix

Participation Rate Limits

Id

check-participation-rate

Description

Trading should respect volume participation limits

Pattern

volume|adv|participation

File Glob

*/.{py,js,ts}

Match

absent

Context Pattern

order|trade|execute

Message

Track participation rate - exceeding 5-10% of volume causes impact

Severity

warning

Autofix

Timing Randomization

Id

check-order-timing-randomization

Description

Execution timing should be randomized to prevent gaming

Pattern

random|jitter|noise

File Glob

*/.{py,js,ts}

Match

absent

Context Pattern

execute|schedule|twap|vwap

Message

Add timing randomization to prevent execution pattern detection

Severity

info

Autofix

Fill Quality Tracking

Id

check-fill-tracking

Description

Execution should track fill quality metrics

Pattern

fill.rate|slippage.track|execution.*quality

File Glob

*/.{py,js,ts}

Match

absent

Context Pattern

execute|order

Message

Track fill quality metrics for execution analysis

Severity

info

Autofix

Multi-Venue Routing

Id

check-venue-selection

Description

Orders should consider multiple venues

Pattern

venue|exchange|route

File Glob

*/.{py,js,ts}

Match

absent

Context Pattern

order|execute

Message

Consider smart order routing across multiple venues

Severity

info

Autofix

Overfill Prevention

Id

check-overfill-prevention

Description

Multi-leg execution needs overfill protection

Pattern

pending|outstanding|overfill

File Glob

*/.{py,js,ts}

Match

absent

Context Pattern

slice|child|parent.*order

Message

Implement overfill prevention when splitting orders

Severity

warning

Autofix

Cancel and Replace Logic

Id

check-cancel-logic

Description

Stale orders should be cancelled

Pattern

cancel|timeout|stale

File Glob

*/.{py,js,ts}

Match

absent

Context Pattern

limit.*order|resting

Message

Implement cancel logic for stale limit orders

Severity

info

Autofix

Latency Over-Optimization Warning

Id

check-latency-not-overoptimized

Description

Non-HFT should not over-optimize for latency

Pattern

microsecond|nanosecond|latency.*critical

File Glob

*/.{py,js,ts}

Match

present

Message

Unless HFT, focus on algo quality over microsecond latency

Severity

info

Autofix