
Changeset Validation
- 113 installs
- 3.5k repo stars
- Updated August 4, 2026
- openai/openai-agents-js
openai-agents-js changeset validation using LLM judgment against git diffs.
About
openai-agents-js changeset validation skill. Uses LLM judgment to verify that changeset files in .changeset/ accurately describe the git diff for modified packages. Runs when packages/ or .changeset/ are modified or when verifying release notes before publish. Compares declared version bump type and changelog entries against actual code changes including uncommitted local work, catching mismatches between changeset prose and implementation before merge.
- LLM judgment validates changesets against git diffs
- Includes uncommitted local changes in diff comparison
- Triggers when packages/ or .changeset/ are modified
- Catches version bump type mismatches before publish
- Verifies changelog entries match actual implementation
Changeset Validation by the numbers
- 113 all-time installs (skills.sh)
- +4 installs in the week ending Jul 26, 2026 (Skillselion tracking)
- Ranked #106 of 248 Release Management skills by installs in the Skillselion catalog
- Data as of Aug 4, 2026 (Skillselion catalog sync)
changeset-validation capabilities & compatibility
- Capabilities
- validate changeset · compare git diff · verify bump type
- Use cases
- code review · ci cd
What changeset-validation says it does
Validate changesets in openai-agents-js using LLM judgment against git diffs (including uncommitted local changes).
npx skills add https://github.com/openai/openai-agents-js --skill changeset-validationAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 113 |
|---|---|
| repo stars | ★ 3.5k |
| Last updated | August 4, 2026 |
| Repository | openai/openai-agents-js ↗ |
Does this changeset accurately describe the code changes?
Validate changesets in openai-agents-js using LLM judgment against git diffs including uncommitted local changes.
Who is it for?
Maintainers verifying changesets before openai-agents-js releases.
Skip if: Changes with no package or changeset impact.
When should I use this skill?
packages/ or .changeset/ modified or user asks to verify changeset accuracy.
What you get
Changeset validated or corrected to match git diff including local uncommitted changes.
Files
Changeset Validation
Overview
This skill validates whether changesets correctly reflect package changes and follow the repository rules. It relies on the shared prompt in references/validation-prompt.md so local Codex reviews and GitHub Actions share the same logic. Experimental or preview-only feature additions that are explicitly labeled as such in the diff may remain a patch bump when they do not change existing behavior. Major bumps are only allowed after the first major release; before that, do not use major bumps for feature-level changes.
Quick start
Local (Codex-driven):
1. Run:
pnpm changeset:validate-prompt2. Apply the rules from references/validation-prompt.md to the generated prompt. 3. Respond with a JSON verdict containing ok/errors/warnings/required_bump (English-only strings).
CI (Codex Action):
1. Run:
pnpm changeset:validate-prompt -- --ci --output .github/codex/prompts/changeset-validation.generated.md2. Use openai/codex-action with the generated prompt and JSON schema to get a structured verdict.
Workflow
1. Generate the prompt context via pnpm changeset:validate-prompt. 2. Apply the rules in references/validation-prompt.md to judge correctness. 3. Provide a clear verdict and required bump (patch/minor/major/none). 4. If the changeset needs edits, update it and re-run the validation.
Branch-level changeset rule (do not skip)
- If any
.changeset/*.mdfile is already present in the current branch diff, treat it as the active changeset. Do not add a new changeset file. - Instead, update the existing changeset file by rewriting its summary to reflect the primary change relative to `main`. Do not try to capture every incremental update.
- Keep the summary at one line; replace the line as needed rather than appending. If you must include multiple points, compress them into a single line.
Shared source of truth
- Keep the prompt file as the single source of validation rules.
- Keep the script lightweight: it should only gather context and emit the prompt.
Resources
references/validation-prompt.md
interface:
display_name: "Changeset Validation"
short_description: "Validate changesets against the current branch diff"
default_prompt: "Use $changeset-validation to validate the current branch changeset against the diff and report any required edits or bump changes."
Use $changeset-validation skill. Use $changeset-validation. It should be located at .agents/skills/changeset-validation under the root directory of this repository.
You are validating changesets for the openai-agents-js monorepo.
Return JSON only. The output must be a JSON object with:
- ok: boolean
- errors: string[] (polite, actionable English sentences)
- warnings: string[] (polite, actionable English sentences)
- required_bump: "patch" | "minor" | "major" | "none"
Always use English in errors and warnings, regardless of the conversation language.
Rules to enforce:
1. Allowed package names are fixed and must be used exactly as listed. 2. Use the changed packages list + diff to judge whether changeset packages are correct and reflect the actual changes. 3. If any package under packages/ changed, at least one changeset must exist and must include every changed package. Only count changeset files listed in CHANGESET_FILES (i.e., added/modified in the current branch diff). 4. If CHANGESET_FILES lists one or more changeset files for the current branch, do not require creating additional new changeset files for incremental branch changes; instead require updating the existing changeset(s) so they cover all changed packages and reflect the latest diff. 5. If no packages changed, changesets are optional; if present, they still must be consistent with the diff. 6. Each changeset summary must be 1-2 non-empty lines. 7. If the PR body contains GitHub issue references like #123 and a changeset exists, the changeset summary should include those references. 8. required_bump is the minimum semver bump required by the diff. Default bump is patch. Judge breaking changes against the latest release tag, not against unreleased branch-only churn or post-tag changes already on main. Rewriting or removing APIs introduced after the latest release tag is not by itself a breaking change. Require minor only when there is a breaking change, dropped support, or a behaviorally significant feature that changes existing workflows or expectations (not simply additive APIs or types). Do not require minor solely because new APIs, options, or types were added; additive features can stay patch when they do not change existing behavior. If you are unsure between patch and minor, prefer patch. Major is allowed only after the first major release and only for changes that warrant a major release (breaking changes, dropped support, or significant behavior shifts). Before the first major release, do not use major bumps for feature-level changes. Exception: if the new feature is explicitly labeled experimental/preview in the diff (e.g., module name, docs, comments, or exports) and does not change existing behavior, a patch bump is acceptable. 9. If an existing changeset uses a bump larger than required_bump, treat that as an error by default and ask to lower it to the minimum required bump. 10. Exception: if PR labels include changeset:minor-approved, an intentional minor bump is allowed for an otherwise patch-level change. In that case, do not ask to lower a minor changeset to patch, but still keep required_bump set to the minimum required bump. 11. If changeset:minor-approved is present and a changeset exists, the summary should make the feature scope clear rather than using a vague summary. 12. required_bump must be "none" when there are no package changes. 13. If unknown package directories are changed, treat it as an error.
If changeset entries include packages that do not appear changed, add a warning (unless the diff indicates a valid reason). If the changeset summary is too vague or clearly unrelated to the diff, add an error. When a changeset file exists in CHANGESET_FILES but is missing packages or no longer reflects the diff, the error should instruct updating the existing changeset file(s), not creating a new one. Do not treat changeset files outside CHANGESET_FILES as satisfying the requirement.
Context: Allowed packages: {{ALLOWED_PACKAGES}}
Unknown package directories: {{UNKNOWN_PACKAGE_DIRS}}
Changed packages: {{CHANGED_PACKAGES}}
Changed files: {{CHANGED_FILES}}
Changeset files:
{{CHANGESET_FILES}}
Notes:
- Only the content inside each fenced
mdblock under "Changeset files" is the actual changeset file content. - The changeset file list includes only files present in the head commit; deleted or renamed source paths may be omitted.
PR body (CI only; otherwise this may be "(not provided)"): {{PR_BODY}}
PR labels (CI only; otherwise this may be "(not provided)"): {{PR_LABELS}}
Package diff (truncated if large): {{PACKAGE_DIFF}}
#!/usr/bin/env node
import fs from 'fs';
import { execFileSync } from 'child_process';
const { fetch, console, process } = globalThis;
function printUsage() {
console.log('Usage: pnpm changeset:assign-milestone -- <path-to-json>');
}
function readJson(filePath) {
const contents = fs.readFileSync(filePath, 'utf8');
return JSON.parse(contents);
}
function parseMilestoneTitle(title) {
const match = title.match(/^(\d+)\.(\d+)\.x$/);
if (!match) return null;
return { major: Number(match[1]), minor: Number(match[2]), title };
}
const bumpRanks = {
none: 0,
patch: 1,
minor: 2,
major: 3,
};
function maxBump(left, right) {
return bumpRanks[right] > bumpRanks[left] ? right : left;
}
function bumpFromChangesetLine(line) {
const match = line.match(
/^\s*['"]?[^'":]+['"]?\s*:\s*(patch|minor|major)\s*$/,
);
return match?.[1];
}
function changesetBumpFromContent(content) {
const parts = content.split('---');
if (parts.length < 3) {
return 'none';
}
let bump = 'none';
for (const line of parts[1].split(/\r?\n/)) {
const lineBump = bumpFromChangesetLine(line);
if (lineBump) {
bump = maxBump(bump, lineBump);
}
}
return bump;
}
function changedChangesetFiles(baseSha, headSha) {
const diff = execFileSync(
'git',
['diff', '--name-only', `${baseSha}...${headSha}`, '--', '.changeset'],
{ encoding: 'utf8' },
).trim();
return diff
? diff
.split(/\r?\n/)
.filter((file) => file.endsWith('.md') && !file.endsWith('README.md'))
: [];
}
function changesetBumpForEvent(event) {
const baseSha = event?.pull_request?.base?.sha;
const headSha = event?.pull_request?.head?.sha;
if (!baseSha || !headSha) {
return 'none';
}
let bump = 'none';
for (const file of changedChangesetFiles(baseSha, headSha)) {
let content;
try {
content = execFileSync('git', ['show', `${headSha}:${file}`], {
encoding: 'utf8',
});
} catch (_error) {
continue;
}
bump = maxBump(bump, changesetBumpFromContent(content));
}
return bump;
}
async function listOpenMilestones(owner, repo, token) {
if (process.env.CHANGESET_ASSIGN_MILESTONE_MILESTONES_JSON) {
return JSON.parse(process.env.CHANGESET_ASSIGN_MILESTONE_MILESTONES_JSON);
}
const milestonesResponse = await fetch(
`https://api.github.com/repos/${owner}/${repo}/milestones?state=open&per_page=100`,
{
headers: {
Accept: 'application/vnd.github+json',
Authorization: `Bearer ${token}`,
},
},
);
if (!milestonesResponse.ok) {
console.warn(
`Milestone assignment skipped (failed to list milestones: ${milestonesResponse.status}).`,
);
return null;
}
return milestonesResponse.json();
}
async function assignMilestone(releaseBump) {
if (releaseBump === 'none') {
console.log('Milestone assignment skipped (no package changes).');
return;
}
const token = process.env.GITHUB_TOKEN;
if (!token) {
console.warn('Milestone assignment skipped (missing GITHUB_TOKEN).');
return;
}
const eventPath = process.env.GITHUB_EVENT_PATH;
if (!eventPath) {
console.warn('Milestone assignment skipped (missing GITHUB_EVENT_PATH).');
return;
}
let event;
try {
event = JSON.parse(fs.readFileSync(eventPath, 'utf8'));
} catch (_error) {
console.warn(
'Milestone assignment skipped (failed to read event payload).',
);
return;
}
const owner = event?.repository?.owner?.login;
const repo = event?.repository?.name;
const prNumber = event?.pull_request?.number;
if (!owner || !repo || !prNumber) {
console.warn(
'Milestone assignment skipped (missing repository or PR info).',
);
return;
}
const milestones = await listOpenMilestones(owner, repo, token);
if (!milestones) {
return;
}
const parsed = milestones
.map((milestone) => ({
milestone,
parsed: parseMilestoneTitle(milestone.title),
}))
.filter((entry) => entry.parsed)
.sort((a, b) => {
if (a.parsed.major !== b.parsed.major)
return a.parsed.major - b.parsed.major;
return a.parsed.minor - b.parsed.minor;
});
if (parsed.length === 0) {
console.warn(
'Milestone assignment skipped (no open milestones matching X.Y.x).',
);
return;
}
const majors = Array.from(
new Set(parsed.map((entry) => entry.parsed.major)),
).sort((a, b) => a - b);
const currentMajor = majors[0];
const nextMajor = majors[1];
const currentMajorEntries = parsed.filter(
(entry) => entry.parsed.major === currentMajor,
);
const patchTarget = currentMajorEntries[0];
const minorTarget = currentMajorEntries[1] ?? patchTarget;
let majorTarget;
if (nextMajor !== undefined) {
const nextMajorEntries = parsed.filter(
(entry) => entry.parsed.major === nextMajor,
);
majorTarget = nextMajorEntries[0];
}
let targetEntry;
if (releaseBump === 'major') {
targetEntry = majorTarget;
} else if (releaseBump === 'minor') {
targetEntry = minorTarget;
} else {
targetEntry = patchTarget;
}
if (!targetEntry) {
console.warn(
'Milestone assignment skipped (not enough open milestones for selection).',
);
return;
}
if (process.env.CHANGESET_ASSIGN_MILESTONE_DRY_RUN === '1') {
console.log(`Milestone would be set to ${targetEntry.milestone.title}.`);
return;
}
const updateResponse = await fetch(
`https://api.github.com/repos/${owner}/${repo}/issues/${prNumber}`,
{
method: 'PATCH',
headers: {
Accept: 'application/vnd.github+json',
Authorization: `Bearer ${token}`,
},
body: JSON.stringify({ milestone: targetEntry.milestone.number }),
},
);
if (!updateResponse.ok) {
console.warn(
`Milestone assignment skipped (failed to update PR milestone: ${updateResponse.status}).`,
);
return;
}
console.log(`Milestone set to ${targetEntry.milestone.title}.`);
}
function main() {
const inputPath = process.argv
.slice(2)
.filter((arg) => arg !== '--')
.find(Boolean);
if (!inputPath) {
printUsage();
console.warn('Milestone assignment skipped (missing input path).');
return;
}
let data;
try {
data = readJson(inputPath);
} catch (_error) {
console.warn(
`Milestone assignment skipped (failed to read JSON from ${inputPath}).`,
);
return;
}
const requiredBump = data?.required_bump;
if (!requiredBump) {
console.warn('Milestone assignment skipped (missing required_bump).');
return;
}
let releaseBump = requiredBump;
if (process.env.GITHUB_EVENT_PATH) {
try {
const event = JSON.parse(
fs.readFileSync(process.env.GITHUB_EVENT_PATH, 'utf8'),
);
releaseBump = maxBump(requiredBump, changesetBumpForEvent(event));
} catch (_error) {
releaseBump = requiredBump;
}
}
assignMilestone(releaseBump).catch((error) => {
console.warn(`Milestone assignment skipped: ${error.message}`);
});
}
main();
#!/usr/bin/env node
import fs from 'fs';
import path from 'path';
import { execSync, spawnSync } from 'child_process';
const { console, process } = globalThis;
const EXEC_MAX_BUFFER = Number(
process.env.CHANGESET_MAX_BUFFER_BYTES || 50 * 1024 * 1024,
);
const ALLOWED_PACKAGES = [
'@openai/agents',
'@openai/agents-core',
'@openai/agents-extensions',
'@openai/agents-openai',
'@openai/agents-realtime',
];
const MAX_DIFF_CHARS = Number(process.env.CHANGESET_MAX_DIFF_CHARS || 12000);
const PROMPT_PATH = path.join(
'.agents',
'skills',
'changeset-validation',
'references',
'validation-prompt.md',
);
function printUsage() {
console.log(`changeset-prompt
Usage:
pnpm changeset:validate-prompt -- [--base <ref>] [--head <ref>] [--ci] [--output <path>]
Options:
--base <ref> Base ref or SHA (default: origin/main if available, else main).
--head <ref> Head ref or SHA (default: HEAD).
--ci Use CI context (PR body, no working tree diffs).
--output <path> Write the generated prompt to a file instead of stdout.
--help Show this help text.
`);
}
function run(cmd, options = {}) {
return execSync(cmd, {
encoding: 'utf8',
stdio: ['ignore', 'pipe', 'pipe'],
maxBuffer: EXEC_MAX_BUFFER,
...options,
}).trim();
}
function runOptional(cmd) {
try {
return run(cmd);
} catch (_error) {
return '';
}
}
function parseArgs(argv) {
const options = {
base: null,
head: null,
ci: process.env.GITHUB_ACTIONS === 'true',
output: null,
};
for (let i = 0; i < argv.length; i += 1) {
const arg = argv[i];
if (arg === '--help') {
options.help = true;
continue;
}
if (arg === '--base') {
options.base = argv[i + 1];
i += 1;
continue;
}
if (arg === '--head') {
options.head = argv[i + 1];
i += 1;
continue;
}
if (arg === '--ci') {
options.ci = true;
continue;
}
if (arg === '--output') {
options.output = argv[i + 1];
i += 1;
continue;
}
}
return options;
}
function parseNameStatus(text) {
if (!text) return [];
const lines = text.split(/\r?\n/).filter(Boolean);
const entries = [];
for (const line of lines) {
const parts = line.split('\t');
const status = parts[0];
if (!status) continue;
if (status.startsWith('R') || status.startsWith('C')) {
if (parts[1]) entries.push({ path: parts[1], status });
if (parts[2]) entries.push({ path: parts[2], status });
} else if (parts[1]) {
entries.push({ path: parts[1], status });
}
}
return entries;
}
function isChangesetFile(filePath) {
if (!filePath.startsWith('.changeset/')) return false;
if (!filePath.endsWith('.md')) return false;
return path.basename(filePath) !== 'README.md';
}
function readFileSafe(filePath) {
try {
return fs.readFileSync(filePath, 'utf8');
} catch (_error) {
return null;
}
}
function truncateText(text, maxChars) {
if (text.length <= maxChars) return text;
return `${text.slice(0, maxChars)}\n... (truncated ${text.length - maxChars} chars)`;
}
function getDiffNoIndex(filePath) {
const result = spawnSync(
'git',
['diff', '--no-index', '--', '/dev/null', filePath],
{
encoding: 'utf8',
},
);
return result.stdout.trim();
}
function readFileFromGit(ref, filePath) {
const result = spawnSync('git', ['show', `${ref}:${filePath}`], {
encoding: 'utf8',
});
if (result.status !== 0) return null;
return result.stdout;
}
function readFileAtRefOrWorktree(ref, filePath, preferWorkingTree = false) {
if (preferWorkingTree) {
const contents = readFileSafe(filePath);
if (contents !== null) return contents;
}
return readFileFromGit(ref, filePath);
}
function parseJsonSafe(contents) {
if (!contents) return null;
try {
return JSON.parse(contents);
} catch (_error) {
return null;
}
}
function cloneJsonValue(value) {
return JSON.parse(JSON.stringify(value));
}
function stripReleaseOnlyPackageJsonFields(packageJson) {
if (!packageJson || typeof packageJson !== 'object') return packageJson;
const normalized = cloneJsonValue(packageJson);
delete normalized.version;
return normalized;
}
function isReleaseManagedPackageFile(filePath, dir) {
return (
filePath === `packages/${dir}/CHANGELOG.md` ||
filePath === `packages/${dir}/package.json` ||
filePath === `packages/${dir}/src/metadata.ts`
);
}
function hasMeaningfulPackageJsonChanges({
dir,
baseSha,
headSha,
includeWorkingTree,
}) {
const filePath = `packages/${dir}/package.json`;
const basePackageJson = parseJsonSafe(readFileFromGit(baseSha, filePath));
const headPackageJson = parseJsonSafe(
readFileAtRefOrWorktree(headSha, filePath, includeWorkingTree),
);
if (!basePackageJson || !headPackageJson) {
return true;
}
return (
JSON.stringify(stripReleaseOnlyPackageJsonFields(basePackageJson)) !==
JSON.stringify(stripReleaseOnlyPackageJsonFields(headPackageJson))
);
}
function collectRelevantPackageDirs({
changedPackageDirs,
packageFilesByDir,
baseSha,
headSha,
includeWorkingTree,
}) {
const relevantDirs = new Set();
for (const dir of changedPackageDirs) {
const files = packageFilesByDir.get(dir) || [];
const onlyReleaseManagedFiles =
files.length > 0 &&
files.every((filePath) => isReleaseManagedPackageFile(filePath, dir));
const meaningfulPackageJsonChanges = files.includes(
`packages/${dir}/package.json`,
)
? hasMeaningfulPackageJsonChanges({
dir,
baseSha,
headSha,
includeWorkingTree,
})
: false;
if (onlyReleaseManagedFiles && !meaningfulPackageJsonChanges) {
continue;
}
relevantDirs.add(dir);
}
return relevantDirs;
}
function readEventPayload() {
const eventPath = process.env.GITHUB_EVENT_PATH;
if (!eventPath) return null;
const contents = readFileSafe(eventPath);
if (!contents) return null;
try {
return JSON.parse(contents);
} catch (_error) {
return null;
}
}
function renderPrompt(template, data) {
return template
.replaceAll('{{ALLOWED_PACKAGES}}', data.allowedPackages)
.replaceAll('{{CHANGED_PACKAGES}}', data.changedPackages)
.replaceAll('{{CHANGED_FILES}}', data.changedFiles)
.replaceAll('{{CHANGESET_FILES}}', data.changesetFiles)
.replaceAll('{{PR_BODY}}', data.prBody)
.replaceAll('{{PR_LABELS}}', data.prLabels)
.replaceAll('{{PACKAGE_DIFF}}', data.packageDiff)
.replaceAll('{{UNKNOWN_PACKAGE_DIRS}}', data.unknownPackageDirs);
}
function formatChangesetFiles(entries) {
if (entries.length === 0) return '(none)';
return entries
.map((entry) => {
const header = `File: ${entry.path} (${entry.status || 'unknown'})`;
const content = entry.content ? entry.content.trimEnd() : '(missing)';
return `${header}\n\n\`\`\`md\n${content}\n\`\`\``;
})
.join('\n\n');
}
function writeOutputFile(outputPath, prompt) {
const dir = path.dirname(outputPath);
fs.mkdirSync(dir, { recursive: true });
fs.writeFileSync(outputPath, `${prompt.trim()}\n`, 'utf8');
console.log(`Wrote prompt to ${outputPath}.`);
}
async function main() {
const options = parseArgs(process.argv.slice(2));
if (options.help) {
printUsage();
process.exit(0);
}
const repoRoot = run('git rev-parse --show-toplevel');
process.chdir(repoRoot);
const eventPayload = readEventPayload();
const eventBaseSha = eventPayload?.pull_request?.base?.sha;
const eventHeadSha = eventPayload?.pull_request?.head?.sha;
const baseRef =
options.base ||
eventBaseSha ||
(runOptional('git rev-parse --verify origin/main')
? 'origin/main'
: 'main');
const headRef = options.head || eventHeadSha || 'HEAD';
let baseSha;
let headSha;
try {
headSha = run(`git rev-parse ${headRef}`);
baseSha = run(`git merge-base ${baseRef} ${headRef}`);
} catch (error) {
console.error(`Failed to resolve git refs: ${error.message}`);
process.exit(1);
}
const includeWorkingTree = !options.ci;
const changes = new Map();
const committedDiff = runOptional(
`git diff --name-status ${baseSha} ${headSha}`,
);
for (const entry of parseNameStatus(committedDiff)) {
changes.set(entry.path, entry.status);
}
if (includeWorkingTree) {
const staged = runOptional('git diff --name-status --cached');
const unstaged = runOptional('git diff --name-status');
for (const entry of parseNameStatus(staged)) {
changes.set(entry.path, entry.status);
}
for (const entry of parseNameStatus(unstaged)) {
changes.set(entry.path, entry.status);
}
const untracked = runOptional('git ls-files --others --exclude-standard');
for (const line of untracked.split(/\r?\n/).filter(Boolean)) {
changes.set(line, 'A');
}
}
const packageDirs = fs
.readdirSync('packages', { withFileTypes: true })
.filter((dirent) => dirent.isDirectory())
.map((dirent) => dirent.name);
const packageNameByDir = new Map();
for (const dir of packageDirs) {
const packageJsonPath = path.join('packages', dir, 'package.json');
const contents = readFileSafe(packageJsonPath);
if (!contents) continue;
try {
const parsed = JSON.parse(contents);
if (parsed?.name) {
packageNameByDir.set(dir, parsed.name);
}
} catch (_error) {
console.error(`Failed to parse ${packageJsonPath}.`);
process.exit(1);
}
}
const changedPackageDirs = new Set();
const packageFilesByDir = new Map();
const unknownPackageDirs = new Set();
for (const filePath of changes.keys()) {
if (!filePath.startsWith('packages/')) continue;
const parts = filePath.split('/');
const dir = parts[1];
if (!dir) continue;
changedPackageDirs.add(dir);
const files = packageFilesByDir.get(dir) || [];
files.push(filePath);
packageFilesByDir.set(dir, files);
const packageName = packageNameByDir.get(dir);
if (!packageName) {
unknownPackageDirs.add(dir);
}
}
const relevantPackageDirs = collectRelevantPackageDirs({
changedPackageDirs,
packageFilesByDir,
baseSha,
headSha,
includeWorkingTree,
});
const changedPackages = new Set(
[...relevantPackageDirs]
.map((dir) => packageNameByDir.get(dir))
.filter(Boolean),
);
const changesetPaths = [...changes.keys()].filter((filePath) =>
isChangesetFile(filePath),
);
const changesetEntries = changesetPaths
.map((filePath) => {
const status = changes.get(filePath) || 'unknown';
if (status.startsWith('D')) return null;
const content =
readFileSafe(filePath) || readFileFromGit(headSha, filePath);
if (!content) {
if (status.startsWith('R') || status.startsWith('C')) return null;
}
return { path: filePath, status, content };
})
.filter(Boolean);
const diffSections = [];
const relevantPackagePaths = [...relevantPackageDirs].map(
(dir) => `packages/${dir}`,
);
const committedPackageDiff =
relevantPackagePaths.length > 0
? runOptional(
`git diff ${baseSha} ${headSha} -- ${relevantPackagePaths.join(' ')}`,
)
: '';
if (committedPackageDiff) {
diffSections.push(`Committed diff (packages):\n${committedPackageDiff}`);
}
if (includeWorkingTree) {
const stagedPackageDiff =
relevantPackagePaths.length > 0
? runOptional(`git diff --cached -- ${relevantPackagePaths.join(' ')}`)
: '';
if (stagedPackageDiff && relevantPackageDirs.size > 0) {
diffSections.push(`Staged diff (packages):\n${stagedPackageDiff}`);
}
const unstagedPackageDiff =
relevantPackagePaths.length > 0
? runOptional(`git diff -- ${relevantPackagePaths.join(' ')}`)
: '';
if (unstagedPackageDiff && relevantPackageDirs.size > 0) {
diffSections.push(`Unstaged diff (packages):\n${unstagedPackageDiff}`);
}
const untrackedPackageDiffs = [];
for (const filePath of changes.keys()) {
if (!filePath.startsWith('packages/')) continue;
const dir = filePath.split('/')[1];
if (!dir || !relevantPackageDirs.has(dir)) continue;
if (!fs.existsSync(filePath)) continue;
if (
!runOptional(`git ls-files --others --exclude-standard -- ${filePath}`)
)
continue;
const diff = getDiffNoIndex(filePath);
if (diff) untrackedPackageDiffs.push(diff);
}
if (untrackedPackageDiffs.length > 0) {
diffSections.push(
`Untracked diff (packages):\n${untrackedPackageDiffs.join('\n')}`,
);
}
}
const packageDiff = truncateText(
diffSections.join('\n\n') || '(no package diff provided)',
MAX_DIFF_CHARS,
);
const prBody = options.ci
? eventPayload?.pull_request?.body || '(none)'
: '(not provided)';
const prLabels = options.ci
? eventPayload?.pull_request?.labels
?.map((label) =>
typeof label === 'string' ? label : label?.name || '',
)
.filter(Boolean)
.sort()
.join(', ') || '(none)'
: '(not provided)';
const promptTemplate = readFileSafe(PROMPT_PATH);
if (!promptTemplate) {
console.error(`Prompt template not found at ${PROMPT_PATH}.`);
process.exit(1);
}
const prompt = renderPrompt(promptTemplate, {
allowedPackages: ALLOWED_PACKAGES.join(', '),
changedPackages: [...changedPackages].sort().join(', ') || '(none)',
changedFiles:
[...changes.keys()]
.filter((filePath) => {
if (!filePath.startsWith('packages/')) return true;
const dir = filePath.split('/')[1];
return Boolean(dir && relevantPackageDirs.has(dir));
})
.sort()
.join('\n') || '(none)',
changesetFiles: formatChangesetFiles(changesetEntries),
prBody,
prLabels,
packageDiff,
unknownPackageDirs: [...unknownPackageDirs].sort().join(', ') || '(none)',
});
if (options.output) {
writeOutputFile(options.output, prompt);
return;
}
console.log(prompt.trim());
}
main().catch((error) => {
console.error(`changeset-prompt failed: ${error.message}`);
process.exit(1);
});
#!/usr/bin/env node
import fs from 'fs';
const { console, process } = globalThis;
function printUsage() {
console.log('Usage: pnpm changeset:validate-result -- <path-to-json>');
}
function readJson(filePath) {
const contents = fs.readFileSync(filePath, 'utf8');
return JSON.parse(contents);
}
function validateShape(data) {
if (typeof data?.ok !== 'boolean') return 'Missing ok boolean.';
if (!Array.isArray(data?.errors)) return 'Missing errors array.';
if (!Array.isArray(data?.warnings)) return 'Missing warnings array.';
if (!['patch', 'minor', 'major', 'none'].includes(data?.required_bump)) {
return 'Missing required_bump with value patch/minor/major/none.';
}
return null;
}
function printWarnings(warnings) {
if (warnings.length === 0) return;
console.warn('\nWarnings:');
for (const warning of warnings) {
console.warn(`- ${warning}`);
}
}
function main() {
const inputPath = process.argv
.slice(2)
.filter((arg) => arg !== '--')
.find(Boolean);
if (!inputPath) {
printUsage();
process.exit(1);
}
let data;
try {
data = readJson(inputPath);
} catch (_error) {
console.error(`Failed to read JSON from ${inputPath}.`);
process.exit(1);
}
const shapeError = validateShape(data);
if (shapeError) {
console.error(`changeset-validation failed: ${shapeError}`);
process.exit(1);
}
if (!data.ok) {
console.error('changeset-validation failed.');
for (const message of data.errors) {
console.error(`- ${message}`);
}
printWarnings(data.warnings);
process.exit(1);
}
console.log('changeset-validation passed.');
printWarnings(data.warnings);
}
main();
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR=$(git rev-parse --show-toplevel)
SKILL_DIR="$ROOT_DIR/.agents/skills/changeset-validation"
node "$SKILL_DIR/scripts/changeset-prompt.mjs" --output "$SKILL_DIR/tmp/prompt.md" >/dev/null
TMP_DIR=$(mktemp -d)
PASS_JSON="$TMP_DIR/pass.json"
FAIL_JSON="$TMP_DIR/fail.json"
SCHEMA_JSON="$TMP_DIR/schema.json"
PATCH_JSON="$TMP_DIR/patch.json"
cat > "$PASS_JSON" <<'JSON'
{"ok":true,"errors":[],"warnings":[],"required_bump":"none"}
JSON
cat > "$FAIL_JSON" <<'JSON'
{"ok":false,"errors":["Missing changeset."],"warnings":[],"required_bump":"patch"}
JSON
cat > "$SCHEMA_JSON" <<'JSON'
{"ok":true}
JSON
cat > "$PATCH_JSON" <<'JSON'
{"ok":true,"errors":[],"warnings":[],"required_bump":"patch"}
JSON
run_expect() {
local expected=$1
local label=$2
shift 2
local output
set +e
output=$("$@" 2>&1)
local status=$?
set -e
if [ "$status" -ne "$expected" ]; then
echo "FAIL: $label"
echo "$output"
exit 1
fi
if [ "$expected" -eq 0 ]; then
echo "OK: $label"
else
echo "OK (expected failure): $label"
fi
}
run_expect 0 "valid JSON passes" node "$SKILL_DIR/scripts/changeset-validation-result.mjs" "$PASS_JSON"
run_expect 1 "invalid JSON fails" node "$SKILL_DIR/scripts/changeset-validation-result.mjs" "$FAIL_JSON"
run_expect 1 "schema errors fail" node "$SKILL_DIR/scripts/changeset-validation-result.mjs" "$SCHEMA_JSON"
run_expect 0 "milestone assignment skips without token" node "$SKILL_DIR/scripts/changeset-assign-milestone.mjs" "$PASS_JSON"
run_expect 0 "milestone assignment handles fail case" node "$SKILL_DIR/scripts/changeset-assign-milestone.mjs" "$FAIL_JSON"
MILESTONE_REPO="$TMP_DIR/milestone-repo"
mkdir -p "$MILESTONE_REPO"
git -C "$MILESTONE_REPO" init >/dev/null
git -C "$MILESTONE_REPO" config user.email test@example.com
git -C "$MILESTONE_REPO" config user.name "Test User"
mkdir -p "$MILESTONE_REPO/.changeset"
git -C "$MILESTONE_REPO" commit --allow-empty -m "base" >/dev/null
BASE_SHA=$(git -C "$MILESTONE_REPO" rev-parse HEAD)
cat > "$MILESTONE_REPO/.changeset/minor.md" <<'MD'
---
'@openai/agents-core': minor
---
feat: add test feature
MD
git -C "$MILESTONE_REPO" add .changeset/minor.md
git -C "$MILESTONE_REPO" commit -m "head" >/dev/null
HEAD_SHA=$(git -C "$MILESTONE_REPO" rev-parse HEAD)
cat > "$TMP_DIR/event.json" <<JSON
{
"repository": {
"owner": { "login": "openai" },
"name": "openai-agents-js"
},
"pull_request": {
"number": 1210,
"base": { "sha": "$BASE_SHA" },
"head": { "sha": "$HEAD_SHA" }
}
}
JSON
output=$(
cd "$MILESTONE_REPO"
GITHUB_TOKEN=dummy \
GITHUB_EVENT_PATH="$TMP_DIR/event.json" \
CHANGESET_ASSIGN_MILESTONE_DRY_RUN=1 \
CHANGESET_ASSIGN_MILESTONE_MILESTONES_JSON='[{"title":"0.8.x","number":6},{"title":"0.9.x","number":7}]' \
node "$SKILL_DIR/scripts/changeset-assign-milestone.mjs" "$PATCH_JSON"
)
if [ "$output" != "Milestone would be set to 0.9.x." ]; then
echo "FAIL: milestone assignment uses changeset release bump"
echo "$output"
exit 1
fi
echo "OK: milestone assignment uses changeset release bump"
STALE_REPO="$TMP_DIR/stale-repo"
mkdir -p "$STALE_REPO"
git -C "$STALE_REPO" init >/dev/null
git -C "$STALE_REPO" config user.email test@example.com
git -C "$STALE_REPO" config user.name "Test User"
mkdir -p "$STALE_REPO/.changeset"
cat > "$STALE_REPO/.changeset/stale.md" <<'MD'
---
'@openai/agents-core': minor
---
feat: old branch-local changeset
MD
git -C "$STALE_REPO" add .changeset/stale.md
git -C "$STALE_REPO" commit -m "common" >/dev/null
COMMON_SHA=$(git -C "$STALE_REPO" rev-parse HEAD)
git -C "$STALE_REPO" checkout -b pr-head "$COMMON_SHA" >/dev/null 2>&1
HEAD_SHA=$(git -C "$STALE_REPO" rev-parse HEAD)
git -C "$STALE_REPO" checkout -b current-base "$COMMON_SHA" >/dev/null 2>&1
git -C "$STALE_REPO" rm .changeset/stale.md >/dev/null
git -C "$STALE_REPO" commit -m "remove stale changeset on base" >/dev/null
BASE_SHA=$(git -C "$STALE_REPO" rev-parse HEAD)
cat > "$TMP_DIR/stale-event.json" <<JSON
{
"repository": {
"owner": { "login": "openai" },
"name": "openai-agents-js"
},
"pull_request": {
"number": 1210,
"base": { "sha": "$BASE_SHA" },
"head": { "sha": "$HEAD_SHA" }
}
}
JSON
output=$(
cd "$STALE_REPO"
GITHUB_TOKEN=dummy \
GITHUB_EVENT_PATH="$TMP_DIR/stale-event.json" \
CHANGESET_ASSIGN_MILESTONE_DRY_RUN=1 \
CHANGESET_ASSIGN_MILESTONE_MILESTONES_JSON='[{"title":"0.8.x","number":6},{"title":"0.9.x","number":7}]' \
node "$SKILL_DIR/scripts/changeset-assign-milestone.mjs" "$PATCH_JSON"
)
if [ "$output" != "Milestone would be set to 0.8.x." ]; then
echo "FAIL: milestone assignment ignores base-only stale changesets"
echo "$output"
exit 1
fi
echo "OK: milestone assignment ignores base-only stale changesets"
rm -rf "$TMP_DIR"
rm -rf "$SKILL_DIR/tmp"
echo "changeset-validation fixture checks passed."
Related skills
FAQ
What does validation compare?
Changeset files against git diff including uncommitted local changes.
When should I run this?
When packages/ or .changeset/ are modified or before release publish.
How is judgment applied?
LLM evaluates whether changelog entries and bump type match the actual diff.