
Swarm
- 4.3k installs
- 1.1k repo stars
- Updated July 30, 2026
- langchain-ai/langchain-skills
swarm is a LangChain skill that fans out independent items to parallel subagent or model dispatches and merges structured results back into a table handle.
About
swarm is a LangChain agent skill for processing many independent items in parallel through a table handle workflow. create builds one row per file, glob match, or pre-parsed task record; run dispatches an instruction template with required responseSchema across rows and returns completed, failed, skipped, and failures counts. Omit subagentType for cheap direct model classification; set subagentType when rows need tools, file access, or multi-step reasoning. Sources include glob or filePaths for one-file-one-row work, or tasks arrays parsed inside eval from JSONL, CSV, or chunked readFile loops for large files. Aggregation uses rows with plain JavaScript filters and counts without spawning extra subagents. Chaining passes updates tables in place; filter supports equals, notEquals, in, exists, and and/or combinations for retries on failed rows. batchSize controls auto-batching capped at ten dispatches by default with optional per-row functions clamped between one and fifty. Technical notes require importing @/skills/swarm only in eval blocks that call it, cap console output around five kilobytes, and never write directly to .swarm directories.
- create plus run table workflow with one row per independent unit of work.
- responseSchema required; schema properties become row columns for structured outputs.
- subagentType optional: omit for direct model calls, set for tool-using agentic loops.
- Supports glob, filePaths, and parsed tasks sources with chunked readFile for large files.
- Retry failed rows with filter exists false and aggregate via rows without extra subagents.
Swarm by the numbers
- 4,334 all-time installs (skills.sh)
- +368 installs in the week ending Aug 5, 2026 (Skillselion tracking)
- Ranked #181 of 16,546 AI & Agent Building skills by installs in the Skillselion catalog
- Security screen: LOW risk (skills.sh audit)
- Data as of Aug 5, 2026 (Skillselion catalog sync)
swarm capabilities & compatibility
- Capabilities
- table creation from glob, files, or task records · parallel instruction dispatch with json schema o · auto batching and per row batchsize control · row filtering and chained multi pass runs · javascript aggregation via rows api
- Works with
- openai · anthropic
- Use cases
- orchestration · data analysis · code review
What swarm says it does
One row = one unit of work — swarm handles batching automatically.
npx skills add https://github.com/langchain-ai/langchain-skills --skill swarmAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 4.3k |
|---|---|
| repo stars | ★ 1.1k |
| Security audit | 2 / 3 scanners passed |
| Last updated | July 30, 2026 |
| Repository | langchain-ai/langchain-skills ↗ |
How do I classify, extract, or review hundreds of files or records in parallel with structured outputs and retry only failed rows?
Fan out independent work items across parallel subagent dispatches with structured JSON schema results and table-based aggregation.
Who is it for?
LangChain agents using @langchain/quickjs PTC with swarm_task that need parallel per-file or per-record processing and structured JSON results.
Skip if: Skip when work is a single item, requires sequential dependencies between rows, or lacks the quickjs swarm_task PTC tool.
When should I use this skill?
User needs to batch classify files, extract labels from JSONL records, review many TypeScript files, or retry only rows missing output columns.
What you get
A table with per-row schema columns plus run statistics for completed, failed, skipped rows and optional JS aggregation summaries.
- multi-agent workflow results
- batched processing output
By the numbers
- MAX_BATCH_SIZE is 50 rows per auto-batch
- createBatches allows a smaller final batch when totals are uneven
Files
Swarm
Process many independent items in parallel. create builds a table handle; run fans work out across rows and merges results back. One row = one unit of work — swarm handles batching automatically.
Flow
1. Create. Build a table from a source — files, a glob pattern, or pre-parsed records. One row per item. Returns a handle. 2. Run. Dispatch an instruction template across rows. Results are merged back into the table. Returns { completed, failed, skipped, failures }. 3. Aggregate. Use rows() and plain JS to count, filter, or summarize. Do not spawn additional subagents for aggregation. 4. Retry. Re-run with filter: { column: "<col>", exists: false } to reprocess only failed rows.
Choosing a source
`glob` / `filePaths` — one file = one row. Use when each file is an independent unit of work. Each row gets { id, file }; the subagent reads the file itself via the {file} placeholder.
`tasks` — pass pre-built records directly. Use when the data lives inside a file (JSONL, CSV, JSON array). Read and parse the file first inside eval, then pass the records. One record = one row — do not group multiple items into a single row.
For small files (under ~500 lines), parse and create in one block:
const { create } = await import("@/skills/swarm");
const raw = await tools.readFile({ file_path: "/data.jsonl" });
const records = raw.trim().split("\n").map(l => JSON.parse(l));
const table = await create({ tasks: records });
console.log(table);For large files, read in chunks of 500 lines to avoid truncation:
const { create } = await import("@/skills/swarm");
let records = [];
let offset = 0;
while (true) {
const chunk = await tools.readFile({ file_path: "/data.txt", offset, limit: 500 });
const lines = chunk.split("\n").filter(l => l.trim());
for (const l of lines) { records.push({ id: `r${records.length}`, text: l }); }
if (lines.length < 500) break;
offset += 500;
}
const table = await create({ tasks: records });
console.log(table);When the file is too large to parse and dispatch in one eval call, split across two blocks. Only the block that calls swarm functions needs the import:
// eval 1: parse only — no swarm import needed
const raw = await tools.readFile({ file_path: "/data.jsonl" });
globalThis.records = raw.trim().split("\n").map(l => JSON.parse(l));
console.log(`Parsed ${globalThis.records.length} records`);// eval 2: create and dispatch
const { create, run } = await import("@/skills/swarm");
const table = await create({ tasks: globalThis.records });
const result = await run(table.id, {
instruction: "Classify {text}",
responseSchema: {
type: "object",
properties: { label: { type: "string" } },
required: ["label"],
},
});
console.log(result);Passing filePaths: ["/data.jsonl"] would produce a table with one row pointing at the file — not one row per record inside it.
When to use subagentType
Omit subagentType for classification, extraction, labeling, and any task where a single model call with structured output is sufficient. This is the default and is significantly cheaper and faster — each dispatch is a direct model call, no tools, no iteration.
Set subagentType when the task requires tools, file access, or multi-step reasoning. Each dispatch runs a full agentic loop with the named subagent.
// Direct model call — classification, no tools needed
await run(table.id, {
instruction: "Classify {text}",
responseSchema: { type: "object", properties: { label: { type: "string" } }, required: ["label"] },
});
// Subagent — needs to read files and reason over multiple steps
await run(table.id, {
subagentType: "reviewer",
instruction: "Review {file} for security issues.",
responseSchema: { type: "object", properties: { finding: { type: "string" } }, required: ["finding"] },
});Instruction + context
instruction is a per-item template with {column} placeholders. Placeholders are resolved by the framework — your column names appear in prompts as references to the values listed alongside, never as raw template syntax. Subagents do the work — do not process items yourself in JS and write the results into rows.
context is free-form prose prepended to every subagent prompt. Use it for shared background: domain terms, classification rules, examples, etc.
const { create, run } = await import("@/skills/swarm");
const table = await create({ glob: "src/**/*.ts" });
const r = await run(table.id, {
subagentType: "reviewer",
instruction: "Review {file} for security issues. List findings or write 'no issues'.",
context: "TypeScript Express backend using Prisma ORM. Focus on injection, auth bypass, path traversal.",
responseSchema: {
type: "object",
properties: { review: { type: "string" } },
required: ["review"],
},
});
console.log(r);
// → { completed: 45, failed: 2, skipped: 0, failures: [...] }Structured output
responseSchema is required. Schema properties become top-level columns on each row and constrain what subagents can return.
const { run } = await import("@/skills/swarm");
await run(table.id, {
instruction: "Classify: {text}",
responseSchema: {
type: "object",
properties: {
sentiment: { type: "string", enum: ["positive", "negative", "neutral"] },
},
required: ["sentiment"],
},
});
// Row after: { id: "r1", text: "...", sentiment: "positive" }Batching
By default, swarm auto-batches to keep total dispatches under 10. For small tables (≤10 rows) each row gets its own subagent call. For larger tables, rows are grouped automatically.
Set batchSize to control grouping:
- Number — uniform batch size for all rows.
batchSize: 1forces per-row
dispatch; batchSize: 20 groups in twenties.
- Function —
(row, rowCount) => number. Returns the desired batch size
for each row. Rows with the same batch size are grouped together, then chunked. Allows mixed dispatch where some rows go solo and others batch.
const { create, run } = await import("@/skills/swarm");
const table = await create({ tasks: items });
// Complex items get individual attention; simple ones batch together
await run(table.id, {
instruction: "Analyze {text}",
responseSchema: {
type: "object",
properties: { analysis: { type: "string" } },
required: ["analysis"],
},
batchSize: (row) => (row.token_count > 1000 ? 1 : 10),
});Batch sizes are clamped to [1, 50] after evaluation.
Aggregation
After run(), use rows() and plain JS — no additional subagents needed.
const { rows } = await import("@/skills/swarm");
const data = await rows(table.id, { columns: ["sentiment"] });
const counts = {};
data.forEach(r => { counts[r.sentiment] = (counts[r.sentiment] || 0) + 1 });
console.log(counts);
// → { positive: 120, negative: 45, neutral: 35 }Chaining passes
run updates the table in place — chain calls to accumulate columns.
const { create, run } = await import("@/skills/swarm");
const table = await create({ tasks: interviews });
await run(table.id, {
instruction: "Classify sentiment of {text}",
responseSchema: {
type: "object",
properties: { sentiment: { type: "string", enum: ["positive", "negative", "neutral"] } },
required: ["sentiment"],
},
});
await run(table.id, {
filter: { column: "sentiment", equals: "negative" },
instruction: "Summarize why {text} had negative sentiment.",
responseSchema: {
type: "object",
properties: { summary: { type: "string" } },
required: ["summary"],
},
});Action-only tasks
When subagents perform actions (write a file, apply a fix) rather than return data, use a simple schema with a status or marker field. The exists: false filter still works for retries.
const { create, run } = await import("@/skills/swarm");
const fixedSchema = {
type: "object",
properties: { fixed: { type: "string" } },
required: ["fixed"],
};
const table = await create({ glob: "src/**/*.ts" });
await run(table.id, {
subagentType: "fixer",
instruction: "Add missing JSDoc to all exported functions in {file}.",
responseSchema: fixedSchema,
});
// retry any that failed
await run(table.id, {
subagentType: "fixer",
instruction: "Add missing JSDoc to all exported functions in {file}.",
responseSchema: fixedSchema,
filter: { column: "fixed", exists: false },
});Filtering
{ column: "status", equals: "done" }
{ column: "status", notEquals: "done" }
{ column: "category", in: ["A", "B"] }
{ column: "result", exists: false } // not yet processed
{ and: [filter1, filter2] }
{ or: [filter1, filter2] }Technical notes
- Only import `@/skills/swarm` in blocks where you call swarm functions.
Data preparation (reading files, parsing, storing in globalThis) does not need the import. Destructure only what you use: { create }, { run }, { create, run }, etc.
- Console output is capped at ~5 KB. Never log raw file contents —
log only counts and short samples.
- **
readFileinsideevalreturns raw content — no line-number
prefixes.** Request at most 500 lines per call. For files with more than 500 lines, loop with incrementing offset.
- When building a table from a file, read it inside `eval`. Data read
inside the sandbox stays there; it never enters the agent's context window.
- Never write to `.swarm/` directly. Always use
create(). - Everything the subagent needs must be in `instruction` + `context`.
Subagents can't see the agent's context.
- Row ids must be unique.
create()rejects sources that produce
duplicate ids. For tasks, that's a caller-side responsibility; for glob / filePaths, ids are auto-disambiguated by parent directory.
- Unknown columns fail fast. If
instructionreferences{foo}and
no matched row provides foo, run() throws before any subagent is dispatched.
API Reference
create(source)
Create a table. Returns a handle { id, count, columns }.
| Source | Description |
|---|---|
{ glob: "src/**/*.ts" } or { glob: ["src/**/*.ts", "lib/**/*.ts"] } | Match files by one or more patterns. Columns: id, file |
{ filePaths: ["a.ts", "b.ts"] } | Explicit file list. Columns: id, file |
{ tasks: [{ id: "t1", text: "..." }] } | Custom rows. Each must have id |
run(tableId, options)
Dispatch work across rows. Returns { completed, failed, skipped, failures }.
| Option | Default | Description |
|---|---|---|
instruction | (required) | Template with {column} placeholders |
responseSchema | (required) | JSON Schema (type: "object") — properties become row columns |
context | — | Prose prepended to every subagent prompt |
filter | — | Only dispatch matching rows |
subagentType | — | Name of subagent to dispatch to. When set, runs a full agentic loop. When omitted, runs a direct model call |
batchSize | auto | Number or (row, rowCount) => number. Auto caps dispatches at 10; 1 = per-row; function = per-row sizing |
concurrency | 10 | Max concurrent subagent dispatches (clamped to 1–10) |
rows(tableId, options?)
Retrieve rows. Use for inspection and JS-based aggregation.
| Option | Description |
|---|---|
filter | Only return matching rows |
columns | Project to specific columns |
limit | Max rows returned |
import { extractPlaceholders } from "./interpolate.js";
import type { BatchFn } from "./types.js";
import { readColumn } from "./utils.js";
/**
* Maximum rows per batch when auto-batching.
*/
export const MAX_BATCH_SIZE = 50;
/**
* Group an array of items into batches of a given size.
*
* The last batch may be smaller than `batchSize` if the total count
* is not evenly divisible.
*
* @param items - Array of items to batch.
* @param batchSize - Maximum number of items per batch.
* @returns Array of batches (each batch is an array of items).
*/
export function createBatches<T>(items: T[], batchSize: number): T[][] {
const batches: T[][] = [];
for (let i = 0; i < items.length; i += batchSize) {
batches.push(items.slice(i, i + batchSize));
}
return batches;
}
/**
* Clamp a batch size to [1, MAX_BATCH_SIZE].
*/
function clampBatchSize(n: number): number {
return Math.max(1, Math.min(Math.round(n), MAX_BATCH_SIZE));
}
/**
* Resolve batch sizes and group rows into dispatch-ready batches.
*
* Handles all three modes:
* - **Auto** (`batchSize` undefined): computes a uniform size from row
* count and `maxSubagents` to stay within the concurrency budget.
* - **Uniform** (`batchSize` is a number): all rows use that size.
* - **Per-row** (`batchSize` is a function): evaluates per row, groups
* rows sharing the same batch size, then chunks each group.
*
* Every batch size is clamped to [1, MAX_BATCH_SIZE].
*
* @param rows - Matched rows to dispatch.
* @param batchSize - Batch strategy: undefined (auto), number, or function.
* @param maxSubagents - Concurrency cap used for auto-batch calculation.
* @returns Array of row batches, each ready for dispatch as a single task.
*/
export function resolveBatchGroups(
rows: Record<string, unknown>[],
maxSubagents: number,
batchSize?: number | BatchFn,
): Record<string, unknown>[][] {
if (rows.length === 0) {
return [];
}
if (batchSize === undefined) {
// Auto: keep total dispatches under maxSubagents
const auto =
rows.length > maxSubagents
? Math.min(Math.ceil(rows.length / maxSubagents), MAX_BATCH_SIZE)
: 1;
return createBatches(rows, auto);
}
if (typeof batchSize === "number") {
return createBatches(rows, clampBatchSize(batchSize));
}
const groups = new Map<number, Record<string, unknown>[]>();
for (const row of rows) {
const size = clampBatchSize(batchSize(row, rows.length));
let group = groups.get(size);
if (!group) {
group = [];
groups.set(size, group);
}
group.push(row);
}
const batches: Record<string, unknown>[][] = [];
for (const [size, group] of groups) {
for (const batch of createBatches(group, size)) {
batches.push(batch);
}
}
return batches;
}
/**
* Wrap a per-item JSON Schema into a batch-level response schema.
*
* Produces a schema of the form:
* ```json
* { "results": [{ "id": "...", ...itemProps }] }
* ```
*
* The item schema's properties are merged with an `id` field so each
* batch entry can be matched back to its row.
*
* @param itemSchema - Per-item JSON Schema.
* @returns Batch-level JSON Schema wrapping items in a `results` array.
*/
export function wrapSchema(
itemSchema: Record<string, unknown>,
count?: number,
): Record<string, unknown> {
const props = (itemSchema.properties as Record<string, unknown>) ?? {};
const req = (itemSchema.required as string[]) ?? [];
const itemProperties: Record<string, unknown> = {
id: { type: "string" },
...props,
};
const itemRequired: string[] = ["id", ...req];
const resultsArray: Record<string, unknown> = {
type: "array",
items: {
type: "object",
additionalProperties: false,
properties: itemProperties,
required: itemRequired,
},
};
if (count != null) {
resultsArray.minItems = count;
resultsArray.maxItems = count;
}
return {
type: "object",
additionalProperties: false,
properties: {
results: resultsArray,
},
required: ["results"],
};
}
/**
* Format a single column value for inclusion in a batch prompt.
*
* Strings are inserted verbatim; numbers/booleans are stringified;
* objects/arrays are JSON-serialized; `undefined` and `null` become
* the empty string so the row still renders with its id.
*/
function formatValue(value: unknown): string {
if (value === undefined || value === null) {
return "";
}
if (typeof value === "string") {
return value;
}
if (typeof value === "number" || typeof value === "boolean") {
return String(value);
}
return JSON.stringify(value);
}
/**
* Rewrite `{col}` placeholders in the author's instruction to
* `` `col` `` (backtick-quoted column name, no braces). The model
* sees a column name as a name, never as template syntax.
*/
function renderTaskBlock(instruction: string): string {
return instruction.replace(
/\{([^}]+)\}/g,
(_m, raw) => `\`${String(raw).trim()}\``,
);
}
/**
* Render the items section.
*
* - 0 placeholders → `[id]` per row (no values, degenerate).
* - 1 placeholder → `[id] <value>` per row (flat).
* - 2+ placeholders → labeled block:
* [id]
* col1: <value>
* col2: <value>
*/
function renderItemsBlock(
rows: Array<Record<string, unknown>>,
placeholders: string[],
): string {
const lines: string[] = [];
for (const row of rows) {
const id = String(row.id);
if (placeholders.length === 0) {
lines.push(`[${id}]`);
continue;
}
if (placeholders.length === 1) {
const value = readColumn(row, placeholders[0]);
lines.push(`[${id}] ${formatValue(value)}`);
continue;
}
lines.push(`[${id}]`);
for (const col of placeholders) {
const value = readColumn(row, col);
lines.push(` ${col}: ${formatValue(value)}`);
}
}
return lines.join("\n");
}
/**
* Build a single prompt for a batch of rows.
*
* The instruction is rewritten to drop template-syntax braces — every
* `{col}` becomes `` `col` `` so the model sees column names as names,
* not as slots it must fill in. Items are rendered as either a flat
* list (single-column case) or a labeled per-column block, so the
* binding from row id to column value is structural and explicit.
*
* @param instruction - Instruction template with `{column}` placeholders.
* @param rows - Array of row objects to include in the batch.
* @param context - Optional context prose prepended to the prompt.
* @returns A single prompt string covering all rows in the batch.
*/
export function buildBatchPrompt(
instruction: string,
rows: Array<Record<string, unknown>>,
context?: string,
): string {
const placeholders = extractPlaceholders(instruction);
const taskBlock = renderTaskBlock(instruction);
const itemsBlock = renderItemsBlock(rows, placeholders);
const parts: string[] = [];
if (context) {
parts.push(context);
parts.push("");
}
parts.push("# Task");
parts.push(taskBlock);
parts.push("");
parts.push(`# Items (${rows.length})`);
if (placeholders.length === 1) {
parts.push(`Each item below is the value of \`${placeholders[0]}\`.`);
parts.push("");
} else if (placeholders.length > 1) {
const cols = placeholders.map((p) => `\`${p}\``).join(", ");
parts.push(`Each item below provides ${cols}.`);
parts.push("");
}
parts.push(itemsBlock);
parts.push("");
parts.push(
`Return a JSON object with a 'results' array of exactly ${rows.length} ` +
"entries, each including the item's 'id' exactly as shown above.",
);
return parts.join("\n");
}
/**
* Unpack a batch response string into per-row results.
*
* Parses the JSON response expecting `{ results: [{ id, ...fields }] }`.
* Maps each item's `id` to its remaining fields. IDs present in
* `expectedIds` but absent from the response are returned in `missing`.
*
* @param response - Raw JSON string from the subagent.
* @param expectedIds - List of row IDs the batch was supposed to cover.
* @returns Map of ID → result fields, plus a list of IDs missing from
* the response.
*/
export function unpackBatchResults(
response: string,
expectedIds: string[],
): { results: Map<string, unknown>; missing: string[] } {
const resultsMap = new Map<string, unknown>();
const missing: string[] = [];
try {
const parsed = JSON.parse(response);
const items: Array<Record<string, unknown>> = parsed?.results ?? [];
for (const item of items) {
if (item && typeof item.id === "string") {
const { id, ...fields } = item;
resultsMap.set(id, fields);
}
}
} catch {
// Parse failure — all IDs are missing
}
for (const id of expectedIds) {
if (!resultsMap.has(id)) {
missing.push(id);
}
}
return { results: resultsMap, missing };
}
import type { FailureGroup, TaskResult, TaskSpec } from "./types.js";
/**
* PTC tool declaration for swarm subagent dispatch.
*
* At runtime in QuickJS, `tools` is an ambient global injected by the
* PTC layer. The swarm skill uses `swarm_task` (a PTC-only tool) rather
* than the general `task` tool, so that response_schema and mode support
* are decoupled from the main agent's task tool.
*
* For vitest, set up `globalThis.tools` in `beforeEach`.
*/
declare const tools: {
swarmTask?: (args: {
description: string;
subagent_type?: string;
response_schema?: Record<string, unknown>;
mode?: "agent" | "invoke";
}) => Promise<string>;
};
/**
* Column names that must not be overwritten by structured output merging.
*/
const RESERVED_COLUMNS = new Set(["id", "file"]);
/**
* Call the PTC `swarm_task` tool.
*
* @internal Exported for testing — not part of the public API.
* @param args - Task arguments forwarded to the swarm task tool.
* @returns The subagent's response as a string.
* @throws Error if the `swarm_task` PTC tool is not configured.
*/
export async function callTask(args: {
description: string;
subagent_type?: string;
response_schema?: Record<string, unknown>;
mode?: "agent" | "invoke";
}): Promise<string> {
if (typeof tools.swarmTask !== "function") {
throw new Error(
"Swarm requires a 'swarm_task' tool in the PTC configuration.",
);
}
return tools.swarmTask(args);
}
/**
* Dispatch an array of task specs to subagents with bounded concurrency.
*
* Spawns up to `concurrency` workers that pull from the task queue.
* Each worker calls the task function and records the result (or error)
* at the same index as the input spec, preserving order.
*
* @param tasks - Task specs to dispatch.
* @param options - Dispatch options (currently just `concurrency`).
* @returns Results in the same order as the input tasks.
*/
export async function dispatch(
tasks: TaskSpec[],
options: { concurrency: number },
): Promise<TaskResult[]> {
const results = new Array<TaskResult>(tasks.length);
let idx = 0;
async function worker(): Promise<void> {
while (idx < tasks.length) {
const i = idx++;
const spec = tasks[i];
try {
const output = await callTask({
description: spec.prompt,
...(spec.subagentType != null && {
subagent_type: spec.subagentType,
}),
...(spec.responseSchema != null && {
response_schema: spec.responseSchema,
}),
...(spec.mode != null && { mode: spec.mode }),
});
results[i] = {
id: spec.id,
status: "completed",
result: String(output),
};
} catch (err: unknown) {
const msg =
err != null && typeof (err as Error).message === "string"
? (err as Error).message
: String(err);
results[i] = { id: spec.id, status: "failed", error: msg };
}
}
}
const workers: Promise<void>[] = [];
for (let w = 0; w < Math.min(options.concurrency, tasks.length); w++) {
workers.push(worker());
}
await Promise.all(workers);
return results;
}
/**
* Group failed task results by error message.
*
* Produces deduplicated failure groups sorted by count descending,
* each containing the shared error message, the count of affected
* rows, and the full list of affected row IDs.
*
* @param results - Array of task results (may include completed results).
* @returns Deduplicated failure groups, sorted by count descending.
*/
export function deduplicateFailures(results: TaskResult[]): FailureGroup[] {
const groups = new Map<string, string[]>();
for (const r of results) {
if (r.status !== "failed" || !r.error) {
continue;
}
const ids = groups.get(r.error);
if (ids) {
ids.push(r.id);
} else {
groups.set(r.error, [r.id]);
}
}
const out: FailureGroup[] = [];
for (const [error, ids] of groups) {
out.push({ error, count: ids.length, ids });
}
out.sort((a, b) => b.count - a.count);
return out;
}
/**
* Merge a subagent result into a table row.
*
* Each property of the parsed structured output is spread onto the
* row as a top-level column — except reserved columns (`id`, `file`)
* which are never overwritten.
*
* @param row - The table row to update (mutated in place).
* @param value - The subagent's parsed structured output.
*/
export function mergeResult(
row: Record<string, unknown>,
value: Record<string, unknown>,
): void {
for (const [k, v] of Object.entries(value)) {
if (!RESERVED_COLUMNS.has(k)) {
row[k] = v;
}
}
}
import type { SwarmFilter } from "./types.js";
import { readColumn } from "./utils.js";
/**
* Compare two values for deep equality.
*
* Handles primitives via `===` and objects/arrays via JSON
* serialization. `null` and `undefined` only equal themselves.
*
* @param a - First value.
* @param b - Second value.
* @returns `true` if the values are deeply equal.
*/
function deepEquals(a: unknown, b: unknown): boolean {
if (a === b) {
return true;
}
if (a == null || b == null) {
return false;
}
return JSON.stringify(a) === JSON.stringify(b);
}
/**
* Evaluate a filter clause against a single table row.
*
* Supports leaf predicates (`equals`, `notEquals`, `in`, `exists`)
* and recursive combinators (`and`, `or`). Column paths support
* dot notation for nested access.
*
* @param filter - The filter clause to evaluate.
* @param row - The table row to test against.
* @returns `true` if the row matches the filter.
*/
export function evaluateFilter(
filter: SwarmFilter,
row: Record<string, unknown>,
): boolean {
if (filter == null || typeof filter !== "object") {
throw new Error(
`evaluateFilter: expected a filter object, got ${JSON.stringify(filter)}`,
);
}
if ("and" in filter) {
return filter.and.every((f) => evaluateFilter(f, row));
}
if ("or" in filter) {
return filter.or.some((f) => evaluateFilter(f, row));
}
const value = readColumn(row, filter.column);
if ("equals" in filter) {
return deepEquals(value, filter.equals);
}
if ("notEquals" in filter) {
return !deepEquals(value, filter.notEquals);
}
if ("in" in filter) {
return filter.in.some((item) => deepEquals(value, item));
}
if ("exists" in filter) {
return filter.exists ? value != null : value == null;
}
return false;
}
import { createTable, loadTable, saveTable } from "./table.js";
import { interpolate, extractPlaceholders } from "./interpolate.js";
import { readColumn } from "./utils.js";
import { evaluateFilter } from "./filter.js";
import { dispatch, deduplicateFailures, mergeResult } from "./executor.js";
import {
resolveBatchGroups,
wrapSchema,
buildBatchPrompt,
unpackBatchResults,
} from "./batching.js";
import type {
CreateSource,
SwarmHandle,
RunOptions,
RunResult,
RowsOptions,
TaskSpec,
TaskResult,
} from "./types.js";
/**
* Maximum concurrent subagent dispatches per `run()` call.
*
* When matched rows exceed this and no explicit `batchSize` is set,
* auto-batching groups rows to stay within this concurrency budget.
*/
const MAX_SUBAGENTS = 10;
/**
* A dispatch unit is a single task for the executor. It tracks
* whether it covers one row (single) or multiple (batch) so the
* merge step knows how to unpack the result.
*/
interface DispatchUnit {
/**
* The task to dispatch to the executor.
*/
task: TaskSpec;
/**
* Row IDs covered by this task. Single: length 1. Batch: length > 1.
*/
rowIds: string[];
}
/**
* Build dispatch units from pre-grouped batches.
*
* Single-row batches produce interpolated per-row prompts with the
* user's responseSchema. Multi-row batches produce batch prompts
* with a wrapped schema.
*/
function buildDispatchUnits(
batches: Record<string, unknown>[][],
opts: {
instruction: string;
context?: string;
subagentType?: string;
responseSchema: Record<string, unknown>;
mode: "agent" | "invoke";
},
): { units: DispatchUnit[]; errors: TaskResult[] } {
const units: DispatchUnit[] = [];
const errors: TaskResult[] = [];
let batchIndex = 0;
for (const batch of batches) {
if (batch.length === 1) {
// Single-row dispatch: interpolate instruction, use schema directly
const row = batch[0];
const rowId = String(row.id);
try {
let prompt = interpolate(opts.instruction, row);
if (opts.context) {
prompt = `${opts.context}\n\n${prompt}`;
}
units.push({
task: {
id: rowId,
prompt,
subagentType: opts.subagentType,
responseSchema: opts.responseSchema,
mode: opts.mode,
},
rowIds: [rowId],
});
} catch (err) {
errors.push({
id: rowId,
status: "failed",
error: (err as Error).message,
});
}
} else {
// Multi-row batch: build batch prompt, wrap schema
const rowIds = batch.map((r) => String(r.id));
units.push({
task: {
id: `batch_${batchIndex}`,
prompt: buildBatchPrompt(opts.instruction, batch, opts.context),
subagentType: opts.subagentType,
responseSchema: wrapSchema(opts.responseSchema, batch.length),
mode: opts.mode,
},
rowIds,
});
batchIndex++;
}
}
return { units, errors };
}
/**
* Normalize dispatch results into per-row results.
*
* Single-row units pass through directly. Batch units are unpacked
* into one result per row — missing rows become failures.
*/
function unpackDispatchResults(
units: DispatchUnit[],
results: TaskResult[],
): TaskResult[] {
const rowResults: TaskResult[] = [];
for (let idx = 0; idx < units.length; idx++) {
const unit = units[idx];
const result = results[idx];
if (unit.rowIds.length === 1) {
rowResults.push(result);
continue;
}
if (result.status === "failed") {
for (const rowId of unit.rowIds) {
rowResults.push({ id: rowId, status: "failed", error: result.error });
}
continue;
}
const { results: unpacked } = unpackBatchResults(
result.result ?? "",
unit.rowIds,
);
for (const rowId of unit.rowIds) {
const value = unpacked.get(rowId);
if (value !== undefined) {
rowResults.push({
id: rowId,
status: "completed",
result: typeof value === "string" ? value : JSON.stringify(value),
});
} else {
rowResults.push({
id: rowId,
status: "failed",
error: "Missing from batch response",
});
}
}
}
return rowResults;
}
/**
* Parse and merge per-row results into table rows.
*
* Each completed result is JSON-parsed and spread onto the
* corresponding row via `mergeResult`.
*/
function mergeRowResults(
rowResults: TaskResult[],
rowById: Map<string, Record<string, unknown>>,
): { completed: number; failed: number } {
let completed = 0;
let failed = 0;
for (const result of rowResults) {
const row = rowById.get(result.id);
if (!row) {
failed++;
continue;
}
if (result.status === "completed" && result.result != null) {
try {
mergeResult(row, JSON.parse(result.result));
completed++;
} catch {
failed++;
}
} else {
failed++;
}
}
return { completed, failed };
}
/**
* Verify every `{column}` reference in `instruction` resolves on at
* least one matched row. Throws with a list of unresolved paths.
*/
function validatePlaceholders(
instruction: string,
rows: Record<string, unknown>[],
): void {
const placeholders = extractPlaceholders(instruction);
if (placeholders.length === 0) {
return;
}
const unresolved = placeholders.filter(
(p) => !rows.some((r) => readColumn(r, p) !== undefined),
);
if (unresolved.length > 0) {
throw new Error(
`instruction references unknown column(s): ${unresolved.join(", ")}`,
);
}
}
/**
* Create a table from a source specification and persist it to the backend.
*
* Thin wrapper around `createTable` — validates the source, builds rows,
* runs eviction if necessary, and persists the table as JSONL.
*
* @param source - Exactly one of `glob`, `filePaths`, or `tasks`.
* @returns A lightweight handle with the table's ID, row count, and columns.
*/
export async function create(source: CreateSource): Promise<SwarmHandle> {
return createTable(source);
}
/**
* Dispatch work across table rows and update the table in place.
*
* Loads the table, partitions rows by filter, interpolates the
* instruction template per-row (or builds batch prompts), dispatches
* to subagents via `tools.swarm_task()`, merges results into rows,
* and persists the updated table.
*
* @param handle - A table handle or object with an `id` field.
* @param options - Dispatch configuration (instruction, filter, schema, etc.).
* @returns A summary with completion counts and deduplicated failure groups.
*/
export async function run(
tableId: string,
options: RunOptions,
): Promise<RunResult> {
const allRows = await loadTable(tableId);
const {
instruction,
context,
filter,
subagentType,
responseSchema,
batchSize,
concurrency,
} = options;
const mode = subagentType != null ? "agent" : "invoke";
const effectiveConcurrency = Math.max(
1,
Math.min(concurrency ?? MAX_SUBAGENTS, MAX_SUBAGENTS),
);
// -----------------------------------------------------------------------
// 1. Partition rows into matched (dispatched) and skipped (filtered out)
// -----------------------------------------------------------------------
const matched: Record<string, unknown>[] = [];
let skippedCount = 0;
for (const row of allRows) {
if (!filter || evaluateFilter(filter, row)) {
matched.push(row);
} else {
skippedCount++;
}
}
if (matched.length === 0) {
return {
completed: 0,
failed: 0,
skipped: allRows.length,
failures: [],
};
}
validatePlaceholders(instruction, matched);
// -----------------------------------------------------------------------
// 2. Resolve batches and build dispatch units
// -----------------------------------------------------------------------
const batches = resolveBatchGroups(matched, effectiveConcurrency, batchSize);
const { units, errors: interpolationErrors } = buildDispatchUnits(batches, {
instruction,
context,
subagentType,
responseSchema,
mode,
});
// -----------------------------------------------------------------------
// 3. Dispatch
// -----------------------------------------------------------------------
const dispatchResults = await dispatch(
units.map((u) => u.task),
{ concurrency: effectiveConcurrency },
);
// -----------------------------------------------------------------------
// 4. Unpack and merge results into rows
// -----------------------------------------------------------------------
const rowById = new Map<string, Record<string, unknown>>();
for (const row of matched) {
rowById.set(String(row.id), row);
}
const rowResults = unpackDispatchResults(units, dispatchResults);
const { completed, failed: mergeFailed } = mergeRowResults(
rowResults,
rowById,
);
const failed = mergeFailed + interpolationErrors.length;
const allRowResults = [...interpolationErrors, ...rowResults];
// -----------------------------------------------------------------------
// 5. Persist and return summary
// -----------------------------------------------------------------------
await saveTable(tableId, allRows);
return {
completed,
failed,
skipped: skippedCount,
failures: deduplicateFailures(allRowResults),
};
}
/**
* Retrieve rows from a table, optionally filtered and projected.
*
* Loads the table and applies filter, column projection, and row
* limiting in that order. Use for inspection and JS-based aggregation
* — the heavy data stays in the sandbox and only the computed result
* (via `console.log`) goes back to the agent's context.
*
* @param handle - A table handle or object with an `id` field.
* @param options - Optional filtering, projection, and limiting.
* @returns Array of row objects matching the criteria.
*/
export async function rows(
tableId: string,
options?: RowsOptions,
): Promise<Record<string, unknown>[]> {
let result = await loadTable(tableId);
if (options?.filter) {
const f = options.filter;
result = result.filter((row) => evaluateFilter(f, row));
}
if (options?.columns) {
const cols = options.columns;
result = result.map((row) => {
const projected: Record<string, unknown> = {};
for (const col of cols) {
if (col in row) projected[col] = row[col];
}
return projected;
});
}
if (options?.limit != null && options.limit >= 0) {
result = result.slice(0, options.limit);
}
return result;
}
import { readColumn } from "./utils.js";
/**
* Replace `{column}` placeholders in a template string with values
* from a table row.
*
* Placeholders use curly braces and support dot-paths for nested
* access (e.g. `{meta.score}`). String values are inserted verbatim,
* numbers and booleans are stringified, and objects/arrays are
* JSON-serialized.
*
* Unlike fail-fast interpolation, this collects ALL missing columns
* and throws a single error listing every unresolvable placeholder.
*
* @param template - The instruction template (e.g. `"Review {file} for issues"`).
* @param row - The table row providing column values.
* @returns The interpolated string with all placeholders resolved.
* @throws Error listing all missing column paths.
*/
export function interpolate(
template: string,
row: Record<string, unknown>,
): string {
const missing: string[] = [];
const result = template.replace(/\{([^}]+)\}/g, (_match, rawPath) => {
const path = rawPath.trim();
const value = readColumn(row, path);
if (value === undefined) {
missing.push(path);
return `{${path}}`;
}
if (typeof value === "string") {
return value;
}
if (typeof value === "number" || typeof value === "boolean") {
return String(value);
}
return JSON.stringify(value);
});
if (missing.length > 0) {
throw new Error(
`Interpolation failed: missing columns: ${missing.join(", ")}`,
);
}
return result;
}
/**
* Extract the unique column paths referenced by `{column}` placeholders
* in a template string.
*
* Preserves first-seen order. Dot-paths are returned as-is (e.g.
* `"meta.score"`). Whitespace inside braces is trimmed, matching
* `interpolate`'s behavior.
*
* @param template - Instruction template (e.g. `"Review {file}: {note}"`).
* @returns Array of unique placeholder paths in first-seen order.
*/
export function extractPlaceholders(template: string): string[] {
const seen = new Set<string>();
const ordered: string[] = [];
const re = /\{([^}]+)\}/g;
let m: RegExpExecArray | null = re.exec(template);
while (m !== null) {
const path = m[1].trim();
if (path.length > 0 && !seen.has(path)) {
seen.add(path);
ordered.push(path);
}
m = re.exec(template);
}
return ordered;
}
import type { CreateSource, SwarmHandle } from "./types.js";
/**
* PTC tool declarations for file operations.
*
* At runtime in QuickJS, `tools` is an ambient global injected by the
* PTC layer. For vitest, set up `globalThis.tools` in `beforeEach`.
*/
declare const tools: {
glob?: (args: { pattern: string }) => Promise<string>;
readFile?: (args: { file_path: string }) => Promise<string>;
writeFile?: (args: { file_path: string; content: string }) => Promise<string>;
editFile?: (args: {
file_path: string;
old_string: string;
new_string: string;
}) => Promise<string>;
};
/**
* Session ID injected by the QuickJS middleware as a global.
* Scopes table files to the current conversation thread.
*/
declare const __sessionId__: string | undefined;
/**
* Sanitize a session ID for use as a directory name component.
* Replaces any character that isn't alphanumeric, hyphen, or underscore
* with an underscore, and caps length to prevent excessively long paths.
*/
function sanitizeSessionId(id: string): string {
return id.replace(/[^a-zA-Z0-9_-]/g, "_").slice(0, 64);
}
/**
* Directory prefix for all table JSONL files, scoped to the session.
*/
function getTableDir(): string {
const id = typeof __sessionId__ !== "undefined" ? __sessionId__ : "default";
return `/tmp/.swarm/${sanitizeSessionId(id)}`;
}
/**
* Maximum number of tables before oldest are evicted.
*/
const MAX_TABLES = 5;
/**
* A table's rows and backend file path, cached in memory to avoid
* redundant PTC reads within the same session.
*/
interface CachedTable {
/**
* The table's row data. Mutated in place during `run()`.
*/
rows: Record<string, unknown>[];
/**
* Backend file path (e.g. `".swarm/003-t_a1b2c3.jsonl"`).
*/
path: string;
/**
* The JSONL content from the most recent successful write.
* Used as `old_string` when falling back to editFile for overwrites.
*/
lastWritten: string;
}
/**
* In-memory table cache keyed by table ID.
*/
const cache = new Map<string, CachedTable>();
/**
* Monotonic counter for table file sequence numbers.
*/
let sequenceCounter = 0;
/**
* Reset all module-level state for testing.
*
* Clears the in-memory cache and resets the sequence counter.
*/
export function _resetForTesting(): void {
cache.clear();
sequenceCounter = 0;
}
/**
* Generate a random 6-hex-char table ID prefixed with `t_`.
*
* @returns A string like `"t_a1b2c3"`.
*/
export function generateId(): string {
const hex = Math.floor(Math.random() * 0xffffff)
.toString(16)
.padStart(6, "0");
return `t_${hex}`;
}
/**
* Build the backend file path for a table.
*
* @param sequence - Zero-padded monotonic sequence number.
* @param id - Table ID (e.g. `"t_a1b2c3"`).
* @returns Path like `".swarm/003-t_a1b2c3.jsonl"`.
*/
export function tablePath(sequence: number, id: string): string {
const padded = String(sequence).padStart(3, "0");
return `${getTableDir()}/${padded}-${id}.jsonl`;
}
/**
* Serialize an array of row objects to JSONL format.
* One JSON object per line, no trailing newline.
*
* @param rows - Array of row objects to serialize.
* @returns JSONL string.
*/
export function serializeJsonl(rows: Record<string, unknown>[]): string {
return rows.map((r) => JSON.stringify(r)).join("\n");
}
/**
* Parse a JSONL string into an array of row objects.
* Validates that each line parses to a non-null, non-array object.
*
* @param content - Raw JSONL content from the backend.
* @returns Array of parsed row objects.
* @throws Error with line number if any line is malformed.
*/
export function parseJsonl(content: string): Record<string, unknown>[] {
if (!content.trim()) {
return [];
}
const parseLine = (line: string, idx: number): Record<string, unknown> => {
try {
const parsed = JSON.parse(line);
if (
typeof parsed !== "object" ||
parsed === null ||
Array.isArray(parsed)
) {
throw new Error(`expected object`);
}
return parsed as Record<string, unknown>;
} catch (e) {
throw new Error(
`JSONL parse error at line ${idx + 1}: ${(e as Error).message}`,
{ cause: e },
);
}
};
return content
.split("\n")
.filter((line) => line.trim() !== "")
.map(parseLine);
}
/**
* Extract a table ID from a `.swarm/NNN-t_XXXXXX.jsonl` filename.
*
* @param filePath - Full path to a table JSONL file.
* @returns The table ID (e.g. `"t_a1b2c3"`), or `undefined` if the
* filename doesn't match the expected pattern.
*/
export function extractIdFromPath(filePath: string): string | undefined {
const filename = filePath.split("/").pop() || "";
const match = filename.match(/^\d+-(t_[a-f0-9]+)\.jsonl$/);
return match ? match[1] : undefined;
}
/**
* Extract the sequence number from a `.swarm/NNN-t_XXXXXX.jsonl` filename.
*
* @param filePath - Full path to a table JSONL file.
* @returns The sequence number, or `0` if the filename doesn't match.
*/
export function extractSeqFromPath(filePath: string): number {
const filename = filePath.split("/").pop() || "";
const match = filename.match(/^(\d+)-/);
return match ? parseInt(match[1], 10) : 0;
}
/**
* Build `{ id, file }` rows from a list of file paths.
*
* Uses the basename (last path segment) as the row ID. When multiple
* paths share the same basename, disambiguates by prepending the
* parent directory name (e.g. `"routes-index.ts"` vs `"handlers-index.ts"`).
*
* @param paths - List of file paths.
* @returns Array of `{ id, file }` row objects.
*/
export function pathsToRows(
paths: string[],
): Array<{ id: string; file: string }> {
const basenames = paths.map((p) => {
const parts = p.split("/");
return parts[parts.length - 1] || p;
});
const counts = new Map<string, number>();
for (const basename of basenames) {
counts.set(basename, (counts.get(basename) ?? 0) + 1);
}
return paths.map((filePath, idx) => {
let id = basenames[idx];
if ((counts.get(id) ?? 0) > 1) {
const parts = filePath.split("/");
if (parts.length >= 2) {
id = `${parts[parts.length - 2]}-${id}`;
}
}
return { id, file: filePath };
});
}
/**
* Find duplicate `id` values in a row array.
*
* @param rows - Row objects to scan.
* @returns Array of duplicate ids, in first-seen order, deduplicated.
*/
function findDuplicateIds(rows: Record<string, unknown>[]): string[] {
const seen = new Set<string>();
const dupes = new Set<string>();
for (const row of rows) {
const id = String(row.id);
if (seen.has(id)) {
dupes.add(id);
} else {
seen.add(id);
}
}
return [...dupes];
}
/**
* Resolve a glob pattern to a list of file paths via the PTC `glob` tool.
*
* Handles both `string[]` and `{ path: string }[]` return formats
* from different glob tool implementations.
*
* @param pattern - Glob pattern to resolve.
* @returns Array of matching file paths.
* @throws Error if the `glob` PTC tool is not configured.
*
* @internal
*/
export async function globFiles(pattern: string): Promise<string[]> {
if (typeof tools.glob !== "function") {
throw new Error(`Swarm requires a 'glob' tool in the PTC configuration`);
}
const raw = await tools.glob({ pattern });
const parsed = JSON.parse(raw);
if (!Array.isArray(parsed)) {
return [];
}
const paths: string[] = [];
for (const item of parsed) {
if (typeof item === "string") {
paths.push(item);
} else if (item && typeof item.path === "string") {
paths.push(item.path);
}
}
return paths;
}
/**
* Read a file's content from the backend via the PTC `readFile` tool.
*
* @param path - Backend file path.
* @returns The file content as a string.
* @throws Error if the `readFile` PTC tool is not configured.
*
* @internal
*/
export async function readFile(path: string): Promise<string> {
if (typeof tools.readFile !== "function") {
throw new Error(
`Swarm requires a 'readFile' tool in the PTC configuration`,
);
}
return tools.readFile({ file_path: path });
}
/**
* Write string content to a backend file via the PTC `writeFile` tool.
*
* If the file already exists, falls back to `editFile` for a full
* replacement — the backend's `write` rejects overwrites by design.
* When `previousContent` is provided it is used as the `old_string`
* for the edit, avoiding an unreliable round-trip through readFile.
*
* @param path - Backend file path. Created if it doesn't exist.
* @param content - String content to write.
* @param previousContent - The last-known content of the file, used
* as `old_string` for the editFile fallback.
* @throws Error if the `writeFile` PTC tool is not configured.
*
* @internal
*/
export async function writeFile(
path: string,
content: string,
previousContent?: string,
): Promise<void> {
if (typeof tools.writeFile !== "function") {
throw new Error(
`Swarm requires a 'writeFile' tool in the PTC configuration`,
);
}
const result = await tools.writeFile({ file_path: path, content });
if (typeof result === "string" && result.includes("already exists")) {
if (typeof tools.editFile !== "function") {
throw new Error(
"Swarm requires an 'edit_file' PTC tool to update existing tables",
);
}
if (previousContent == null) {
throw new Error(
`Cannot overwrite ${path}: file already exists and no previous content available`,
);
}
await tools.editFile({
file_path: path,
old_string: previousContent,
new_string: content,
});
}
}
/**
* List all table JSONL files in the `.swarm/` directory, sorted by
* filename (which encodes creation order via the sequence prefix).
*
* @returns Sorted array of file paths, or empty array on failure.
*/
async function listTableFiles(): Promise<string[]> {
try {
const files = await globFiles(`${getTableDir()}/*.jsonl`);
return files.sort();
} catch {
return [];
}
}
/**
* Evict the oldest tables when the count meets or exceeds `MAX_TABLES`.
*
* Clears evicted entries from the in-memory cache and overwrites
* backend files with empty content (no delete_file tool available).
* Empty files are treated as evicted by `loadTable`.
*/
async function evict(): Promise<void> {
const files = await listTableFiles();
if (files.length < MAX_TABLES) {
return;
}
const toEvict = files.slice(0, files.length - MAX_TABLES + 1);
for (const filePath of toEvict) {
const id = extractIdFromPath(filePath);
const prev = id ? cache.get(id)?.lastWritten : undefined;
if (id) {
cache.delete(id);
}
try {
await writeFile(filePath, "", prev);
} catch {
// Best-effort eviction — non-fatal if overwrite fails
}
}
}
/**
* Determine the next sequence number for a new table file.
*
* Reads existing files on the backend to avoid sequence collisions
* across runs (same thread, new session). The counter only advances
* forward — it never reuses a sequence number.
*
* @returns The next available sequence number.
*/
async function nextSequence(): Promise<number> {
const files = await listTableFiles();
if (files.length > 0) {
const lastSequence = extractSeqFromPath(files[files.length - 1]);
if (lastSequence >= sequenceCounter) {
sequenceCounter = lastSequence + 1;
}
}
return sequenceCounter++;
}
/**
* Resolve one or more glob patterns into a deduplicated, sorted list
* of file paths.
*
* @param pattern - A single glob string or array of glob strings.
* @returns Sorted, deduplicated array of matching file paths.
* @throws Error if no files match any of the provided patterns.
*/
async function resolveGlob(pattern: string | string[]): Promise<string[]> {
const patterns = Array.isArray(pattern) ? pattern : [pattern];
const allPaths: string[] = [];
for (const p of patterns) {
const paths = await globFiles(p);
allPaths.push(...paths);
}
const unique = [...new Set(allPaths)].sort();
if (unique.length === 0) {
throw new Error(`No files matched pattern: ${JSON.stringify(pattern)}`);
}
return unique;
}
/**
* Create a table from a source spec.
*
* Validates the source, builds rows, runs eviction if the table count
* is at capacity, persists the new table to the backend as JSONL, and
* returns a lightweight handle.
*
* @param source - Exactly one of `glob`, `filePaths`, or `tasks`.
* @returns A handle with the table's ID, row count, and column names.
* @throws Error if the source is invalid, empty, or missing required PTC tools.
*/
export async function createTable(source: CreateSource): Promise<SwarmHandle> {
const sourceCount = [source.glob, source.filePaths, source.tasks].filter(
(s) => s != null,
).length;
if (sourceCount === 0) {
throw new Error(
"create() requires exactly one source: glob, filePaths, or tasks",
);
}
if (sourceCount > 1) {
throw new Error("create() accepts only one source type at a time");
}
let rows: Record<string, unknown>[];
if (source.glob != null) {
const paths = await resolveGlob(source.glob);
rows = pathsToRows(paths);
} else if (source.filePaths != null) {
if (source.filePaths.length === 0) {
throw new Error("filePaths array is empty");
}
rows = pathsToRows(source.filePaths);
} else {
const tasks = source.tasks ?? [];
if (tasks.length === 0) {
throw new Error("tasks array is empty");
}
for (let idx = 0; idx < tasks.length; idx++) {
if (typeof tasks[idx].id !== "string") {
throw new Error(`tasks[${idx}] is missing string 'id' field`);
}
}
rows = tasks;
}
const dupes = findDuplicateIds(rows);
if (dupes.length > 0) {
throw new Error(`create() received duplicate row ids: ${dupes.join(", ")}`);
}
await evict();
const id = generateId();
const seq = await nextSequence();
const path = tablePath(seq, id);
const content = serializeJsonl(rows);
await writeFile(path, content);
cache.set(id, { rows, path, lastWritten: content });
return {
id,
count: rows.length,
columns: Object.keys(rows[0] ?? {}),
};
}
/**
* Load a table's rows by ID.
*
* Checks the in-memory cache first. On a cache miss (e.g. cross-run
* resume), globs the backend to locate the JSONL file, reads and
* parses it, and populates the cache.
*
* @param id - The table ID from a `SwarmHandle`.
* @returns The table's row array (by reference — mutations are visible).
* @throws Error if the table is not found (evicted or never created).
*/
export async function loadTable(
id: string,
): Promise<Record<string, unknown>[]> {
const cached = cache.get(id);
if (cached) {
return cached.rows;
}
const files = await listTableFiles();
const match = files.find((f) => f.endsWith(`-${id}.jsonl`));
if (!match) {
throw new Error(`Table "${id}" not found. It may have been evicted`);
}
const content = await readFile(match);
if (!content.trim()) {
throw new Error(`Table "${id}" not found. It may have been evicted`);
}
const rows = parseJsonl(content);
cache.set(id, { rows, path: match, lastWritten: serializeJsonl(rows) });
return rows;
}
/**
* Persist a table's current rows to the backend.
*
* Updates both the in-memory cache and the backend JSONL file.
* The table must have been previously loaded via `loadTable` so
* that its backend file path is known.
*
* @param id - The table ID from a `SwarmHandle`.
* @param rows - The updated row array to persist.
* @throws Error if the table has not been loaded into cache.
*/
export async function saveTable(
id: string,
rows: Record<string, unknown>[],
): Promise<void> {
const cached = cache.get(id);
if (!cached) {
throw new Error(`Table "${id}" is not loaded - call loadTable first`);
}
cached.rows = rows;
const content = serializeJsonl(rows);
await writeFile(cached.path, content, cached.lastWritten);
cached.lastWritten = content;
}
/**
* Lightweight handle returned by `create()`.
*
* Contains only metadata — actual row data stays on the backend.
* The agent uses this handle to reference the table in subsequent
* `run()` and `rows()` calls. Handles are stable across evals.
*/
export interface SwarmHandle {
/**
* Unique table identifier (e.g. `"t_a1b2c3"`).
*/
id: string;
/**
* Number of rows in the table at creation time.
*/
count: number;
/**
* Column names present in the first row (e.g. `["id", "file"]`).
*/
columns: string[];
}
/**
* Source specification for `create()`.
*
* Exactly one of `glob`, `filePaths`, or `tasks` must be set.
* Providing zero or more than one source throws an error.
*/
export interface CreateSource {
/**
* Glob pattern(s) to match files. Each match becomes a row with
* `{ id: <basename>, file: <full path> }` columns. Requires a
* `glob` tool in the PTC configuration.
*/
glob?: string | string[];
/**
* Explicit list of file paths. Same row structure as `glob`
* (`{ id, file }`) but skips pattern resolution.
*/
filePaths?: string[];
/**
* Custom row data. Each object must include a string `id` field.
* All other fields become table columns.
*/
tasks?: Array<Record<string, unknown>>;
}
/**
* Per-row batch size function.
*
* Returns the desired batch size for a given row. Rows that return
* the same batch size are grouped together, then chunked into
* batches of that size.
*/
export type BatchFn = (
row: Record<string, unknown>,
rowCount: number,
) => number;
/**
* Options for `run()`.
*
* Controls how rows are selected, how instructions are templated,
* and how subagent dispatch is configured.
*/
export interface RunOptions {
/**
* Instruction template with `{column}` placeholders that are
* interpolated per-row (e.g. `"Review {file} for security issues"`).
*/
instruction: string;
/**
* Context prose prepended to every subagent prompt. Use for shared
* background that applies to all rows (e.g. project description).
*/
context?: string;
/**
* Filter clause to select a subset of rows. Rows that don't match
* are skipped (counted in `RunResult.skipped`).
*/
filter?: SwarmFilter;
/**
* Name of the subagent type to dispatch to. When set, each dispatch
* runs a full agentic loop with tools. When omitted, each dispatch
* is a direct model call with structured output (no tools, no iteration).
*/
subagentType?: string;
/**
* JSON Schema (type: "object") for structured output. Each property
* in the schema becomes a top-level column on the row.
*/
responseSchema: Record<string, unknown>;
/**
* Controls how rows are grouped into subagent calls.
*
* - **Number**: uniform batch size for all rows.
* - **Function**: called per-row, returns desired batch size. Rows with
* the same batch size are grouped together, then chunked.
*
* Batch sizes are clamped to [1, MAX_BATCH_SIZE] after evaluation.
*
* @default auto-batch based on table size to cap total dispatches.
*/
batchSize?: number | BatchFn;
/**
* Maximum concurrent subagent dispatches. Clamped to [1, MAX_SUBAGENTS].
* Defaults to MAX_SUBAGENTS (10) when omitted.
*/
concurrency?: number;
}
/**
* Summary returned by `run()`.
*
* Contains counts and deduplicated failure groups. The agent uses
* this to decide whether to retry, inspect, or proceed.
*/
export interface RunResult {
/**
* Number of rows where the subagent succeeded and a result was merged.
*/
completed: number;
/**
* Number of rows where the subagent failed or interpolation failed.
*/
failed: number;
/**
* Number of rows excluded by the filter (not dispatched).
*/
skipped: number;
/**
* Failures grouped by error message, sorted by count descending.
*/
failures: FailureGroup[];
}
/**
* A group of rows that failed with the same error message.
*
* Deduplication keeps the failure list compact even when hundreds of
* rows hit the same error (e.g. rate limiting).
*/
export interface FailureGroup {
/**
* The error message shared by all rows in this group.
*/
error: string;
/**
* Number of rows that hit this error.
*/
count: number;
/**
* IDs of all rows that hit this error.
*/
ids: string[];
}
/**
* Options for `rows()`.
*
* Controls filtering, column projection, and row limiting when
* retrieving table data for inspection or aggregation.
*/
export interface RowsOptions {
/**
* Filter clause — only rows matching the filter are returned.
*/
filter?: SwarmFilter;
/**
* Project to specific columns. Omit to return all columns.
*/
columns?: string[];
/**
* Maximum number of rows to return. Omit for no limit.
*/
limit?: number;
}
/**
* Filter clause for selecting rows. Can be a leaf predicate or a
* combinator (`and`/`or`) composing multiple clauses.
*
* Leaf predicates operate on a single column (supports dot-paths
* for nested access, e.g. `"meta.score"`).
*/
export type SwarmFilter =
| {
/**
* Column path to compare.
*/
column: string;
/**
* Row matches if column value deeply equals this value.
*/
equals: unknown;
}
| {
/**
* Column path to compare.
*/
column: string;
/**
* Row matches if column value does NOT deeply equal this value.
*/
notEquals: unknown;
}
| {
/** Column path to compare. */
column: string;
/**
* Row matches if column value deeply equals any item in this array.
*/
in: unknown[];
}
| {
/**
* Column path to compare.
*/
column: string;
/**
* When true, matches non-null/non-undefined. When false, matches null/undefined.
*/
exists: boolean;
}
| {
/**
* All sub-filters must match for the row to match.
*/
and: SwarmFilter[];
}
| {
/**
* At least one sub-filter must match for the row to match.
*/
or: SwarmFilter[];
};
/**
* A single dispatch unit for the executor.
*
* Represents one subagent call — either a single row's interpolated
* prompt or a batch prompt covering multiple rows.
*/
export interface TaskSpec {
/**
* Row ID (single dispatch) or batch ID (batched dispatch).
*/
id: string;
/**
* Fully interpolated prompt to send to the subagent.
*/
prompt: string;
/**
* Name of the subagent type to dispatch to. When omitted, the
* dispatch is a direct model call (invoke mode).
*/
subagentType?: string;
/**
* Optional JSON Schema to constrain the subagent's response.
*/
responseSchema?: Record<string, unknown>;
/**
* Dispatch mode for this task.
*
* - `"agent"` — Full agentic loop with tools and middleware.
* - `"invoke"` — Direct model call, no tools or iteration.
*
* @default "agent"
*/
mode?: "agent" | "invoke";
}
/**
* Result of a single subagent dispatch.
*
* Returned by the executor in the same order as the input `TaskSpec[]`.
*/
export interface TaskResult {
/**
* Row ID or batch ID that this result corresponds to.
*/
id: string;
/**
* Whether the dispatch succeeded or failed.
*/
status: "completed" | "failed";
/**
* The subagent's response string (present when `status` is `"completed"`).
*/
result?: string;
/**
* Error message (present when `status` is `"failed"`).
*/
error?: string;
}
/**
* Read a value from a row by dot-separated column path.
*
* Traverses nested objects segment by segment (e.g. `"meta.score"`
* reads `row.meta.score`). Returns `undefined` if any intermediate
* segment is missing or not an object.
*
* @param row - The table row to read from.
* @param path - Dot-separated column path (e.g. `"file"` or `"meta.score"`).
* @returns The resolved value, or `undefined` if the path is invalid.
*/
export function readColumn(
row: Record<string, unknown>,
path: string,
): unknown {
const segments = path.split(".");
let current = row;
for (let idx = 0; idx < segments.length - 1; idx++) {
const next = current[segments[idx]];
if (next == null || typeof next !== "object" || Array.isArray(next)) {
return undefined;
}
current = next as Record<string, unknown>;
}
return current[segments[segments.length - 1]];
}
Related skills
How it compares
Choose swarm when tasks need delegated sub-agents and batching; use a single LangChain agent for straightforward one-shot generation.
FAQ
When should swarm omit subagentType?
Omit it for classification or extraction where a single structured model call is enough; it is cheaper and faster than a full agent loop.
How should large JSONL files become table rows?
Read and parse inside eval, optionally in five-hundred-line chunks, then pass records to create tasks rather than filePaths with one row per file.
How do I retry only failed swarm rows?
Re-run with filter column set to the output field and exists false so only unprocessed rows dispatch again.
Is Swarm safe to install?
skills.sh reports 2 of 3 security scanners passed. Review the Security Audits panel on this page before installing in production.