Cost Optimizer

Cost Optimizer

Tracks cumulative LLM costs across DAG execution and makes real-time decisions to stay within budget: downgrade models, skip optional nodes, or stop early.

---

When to Use

✅ Use for:

• Setting and enforcing cost budgets for DAG executions
• Real-time cost monitoring during execution
• Deciding when to downgrade model tiers mid-execution
• Identifying which nodes are most expensive and why
• Post-execution cost analysis and optimization recommendations

❌ NOT for:

• Choosing which model to use per node (use llm-router)
• Provider pricing comparisons (static data, not a skill)
• Billing or invoicing features

---

Budget Enforcement Process

flowchart TD
  N[Node about to execute] --> C[Check: spent + estimated_node_cost vs budget]
  C --> S{Within budget?}
  S -->|Yes, >20% remaining| E[Execute at planned model tier]
  S -->|Yes, <20% remaining| W[Execute but downgrade to Tier 1 if possible]
  S -->|No| D{Node optional?}
  D -->|Yes| SK[Skip node]
  D -->|No| H{Human gate available?}
  H -->|Yes| A[Ask human: continue over budget?]
  H -->|No| ST[Stop execution, return partial results]

Budget Tiers

Cost Estimation Per Node

Before each node executes, estimate its cost:

estimated_cost = (avg_input_tokens × input_price + avg_output_tokens × output_price)

Use historical averages for this skill + model combination. If no history, use defaults:

• Tier 1 (Haiku): ~800 input + 400 output = ~$0.001
• Tier 2 (Sonnet): ~2000 input + 1000 output = ~$0.012
• Tier 3 (Opus): ~3000 input + 1500 output = ~$0.16

Post-Execution Report

cost_report:
  total_budget: 0.50
  total_spent: 0.37
  budget_remaining: 0.13
  nodes_executed: 8
  nodes_skipped: 1
  nodes_downgraded: 2
  model_breakdown:
    haiku: { calls: 4, cost: 0.004 }
    sonnet: { calls: 3, cost: 0.036 }
    opus: { calls: 1, cost: 0.33 }
  savings_recommendations:
    - "Node 'deep-analysis' used Opus ($0.33) but downstream accepted on first try. Try Sonnet next time — potential saving: $0.32"
    - "Nodes 'validate-a' and 'validate-b' are sequential but independent. Parallelize to reduce wall-clock time."

---

Anti-Patterns

No Budget at All

Wrong: Running DAGs without any cost tracking until the API bill arrives. Right: Every DAG execution has a budget, even if generous. Track spend per node.

Aggressive Downgrading

Wrong: Downgrading Opus nodes to Haiku at 50% budget remaining, causing quality failures that trigger expensive retries. Right: Only downgrade when the alternative is stopping execution. Retries cost more than the original model tier.

Ignoring Retries in Cost

Wrong: Budgeting for one attempt per node. Right: Budget for avg_retries × cost_per_attempt. A node with 3 retries on Sonnet costs $0.036, not $0.012.