
Speculative Pipeline
Overlap multi-agent pipeline stages with staggered starts and convergence loops so long agent workflows finish in less wall-clock time.
Install
npx skills add https://github.com/juliusbrussee/cavekit --skill speculative-pipelineWhat is this skill?
- Leader stage starts first; followers begin after a configurable delay on partial upstream output
- Convergence loops let followers re-read stabilizing upstream artifacts each iteration
- Targets dramatic wall-clock cuts vs strict sequential waiting for perfect upstream completion
- Documented example: ~12-hour sequential 3-stage pipeline vs ~7 hours with staggering
- Triggers include speculative-pipeline, staggered pipeline, and overlap pipeline stages
Adoption & trust: 15 installs on skills.sh; 1k GitHub stars; 3/3 security scanners passed (skills.sh audits).
Recommended Skills
Journey fit
First appears when planning and executing multi-step agent pipelines during product build, though the same timing strategy applies to ship prep and ops automation chains. Staggered leader/follower stages and delay configuration are planning and execution mechanics for agent PM, not a single integration or test task.
Common Questions / FAQ
Is Speculative Pipeline safe to install?
skills.sh reports 3 of 3 security scanners passed. Review the Security Audits panel on this page before installing in production.
SKILL.md
READMESKILL.md - Speculative Pipeline
# Speculative-pipeline Strategy Run pipeline stages with staggered timing instead of sequentially. The leader starts first; followers start after a configurable delay and build from whatever upstream output exists at that point. Combined with convergence loops, followers self-correct as upstream artifacts arrive and stabilize. ## Core Principle > **Start downstream work early with partial upstream output. Convergence loops correct the > errors introduced by working from incomplete input.** The insight is that waiting for perfect upstream output is wasteful. A follower working from 80% of the upstream artifacts will produce output that is ~60-70% correct on the first pass. But with convergence loops running, the follower re-reads the upstream artifacts on each iteration and corrects course. By the time the leader finishes, the follower is already most of the way done. --- ## The Pattern ### Sequential (Traditional) ``` Stage 1: Specs ████████████████████ (5 hours) Stage 2: Plans ████████████████ (4 hours) Stage 3: Implement ████████ (3 hours) ───────────────────────────────────────────── Total: 12 hours ``` ### Speculative-pipeline (Staggered) ``` Stage 1: Specs ████████████████████ (5 hours) Stage 2: Plans ████████████████ (4 hours, started 1.5h after Stage 1) Stage 3: Implement ████████████ (3 hours, started 3h after Stage 1) ───────────────────────────────────────────── Total: ~7 hours ``` ### Why It Works 1. **Stage 2 starts after a 1.5-hour offset.** By then, Stage 1 has produced a meaningful set of partial specs. 2. **Stage 2 generates plans from whatever specs are available.** Some plans will be built on incomplete information. 3. **Stage 1 keeps refining specs.** When Stage 2 loops back for its next pass, it picks up the newly completed specs and adjusts its plans accordingly. 4. **Stage 3 starts after a 3-hour offset.** By then, both specs and plans exist in draft form. 5. **All stages self-correct through iteration.** Each pass re-reads the latest upstream artifacts. Mistakes caused by working from partial input are washed out on subsequent passes. The key mechanism is **convergence** -- the iterative loop that re-reads inputs each pass. Without convergence loops, speculative-pipeline would produce garbage. With them, early errors wash out over iterations. --- ## Example: 3-Stage Pipeline ### Directory Structure ``` context/ ├── specs/ # Stage 1 output: implementation-agnostic specs ├── plans/ # Stage 2 output: framework-specific plans ├── impl/ # Stage 3 output: implementation tracking └── prompts/ ├── 001-generate-specs.md # Stage 1 prompt ├── 002-generate-plans.md # Stage 2 prompt └── 003-implement.md # Stage 3 prompt ``` ### Terminal Commands Open three terminal windows (or use tmux panes): ```bash # Terminal 1: Specs from reference materials (leader -- starts immediately) {LOOP_TOOL} context/prompts/001-generate-specs.md -n 5 -t 2h # Terminal 2: Plans fr