Systematic Qa
- 1 installs
- 95 repo stars
- Updated June 28, 2026
- pedronauck/kodebase-go
Runs full-project QA by discovering the repo verification contract, exercising core workflows end-to-end, and fixing root-cause regressions.
About
Discovers and runs build, lint, test, and startup commands, then drives core workflows end-to-end as a real user before rerunning the full gate. A developer uses it to validate a branch, release candidate, or risky commit.
- Discovers the repo verification contract and runs build/lint/test/start
- Exercises core workflows end-to-end like a real user
Systematic Qa by the numbers
- 1 all-time installs (skills.sh)
- Ranked #1,750 of 2,153 Testing & QA skills by installs in the Skillselion catalog
- Data as of Aug 4, 2026 (Skillselion catalog sync)
npx skills add https://github.com/pedronauck/kodebase-go --skill systematic-qaAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 1 |
|---|---|
| repo stars | ★ 95 |
| Last updated | June 28, 2026 |
| Repository | pedronauck/kodebase-go ↗ |
What it does
Runs full-project QA by discovering the repo verification contract, exercising core workflows end-to-end, and fixing root-cause regressions.
Files
Systematic Project QA
Procedures
Step 1: Discover the Repository QA Contract
1. Read root instructions, repository docs, and CI/build files before running commands. 2. Execute python3 scripts/discover-project-contract.py --root . to surface candidate install, verify, build, test, lint, and start commands. 3. Prefer repository-defined umbrella commands such as make verify, just verify, or CI entrypoints over language-default commands. 4. Read references/project-signals.md when command ownership is ambiguous or when multiple ecosystems are present. 5. Identify the changed surface and the regression-critical surface before choosing scenarios. 6. Choose a QA artifact location using repository conventions. If the repository has no QA artifact convention, store scratch artifacts under /tmp/codex-qa-<slug>.
Step 2: Define the QA Scope
1. Build a short execution matrix covering baseline verification, changed workflows, and unchanged business-critical workflows. 2. Read references/checklist.md and ensure every required category has a planned validation. 3. Prefer public entry points such as CLI commands, HTTP endpoints, browser flows, worker jobs, and documented setup commands over internal test helpers. 4. Create the smallest realistic fixture or fake project needed to exercise the workflow when the repository does not already include one. 5. Treat mocks as a local unit-test boundary only. Do not use mocks or stubs as final proof that a user flow works.
Step 3: Establish the Baseline
1. Install dependencies with the repository-preferred command before testing runtime flows. 2. Run the canonical verification gate once before scenario testing to establish baseline health. 3. If the baseline fails, read the first failing output carefully and determine whether it is pre-existing or introduced by current work before moving on. 4. Start services in the closest supported production-like mode and confirm readiness through observable signals such as health checks, startup logs, or successful handshakes.
Step 4: Execute User-Like Flows
1. Drive workflows through the same interfaces a real operator or user would use. 2. Capture the exact command, input, and observable result for each scenario. 3. Validate changed features first, then validate at least one regression-critical flow outside the changed surface. 4. Exercise live integrations when credentials and local prerequisites exist. When they do not, validate every reachable local boundary and record the blocked live step explicitly. 5. Re-run the scenario from a clean state when the first attempt leaves the environment ambiguous.
Step 5: Diagnose and Fix Regressions
1. Reproduce each failure consistently before proposing a fix. 2. Activate companion debugging and test-hygiene skills when available, especially root-cause debugging and anti-workaround guidance. 3. Add or update the narrowest regression test that proves the bug when the repository supports automated coverage for that surface. 4. Fix production code or real configuration at the source of the failure. Do not weaken tests to match broken behavior. 5. Re-run the narrow reproduction, the impacted scenario, and the baseline gate after each fix. 6. Use assets/issue-template.md when the user wants persisted issue files or when the repository already has a QA issue convention.
Step 6: Verify the Final State
1. Re-run the full repository verification gate from scratch after the last code change. 2. Re-run the most important user-like scenarios after the full gate passes. 3. Summarize the evidence using assets/verification-report-template.md. 4. Report blocked scenarios, missing credentials, or environment gaps with the exact command or prerequisite that stopped execution. 5. Do not claim completion without fresh verification evidence from the current state of the repository.
Error Handling
- If command discovery returns multiple plausible gates, prefer the broadest repository-defined command and explain the tie-breaker.
- If no canonical verify command exists, read
references/project-signals.md, choose the broadest safe install, lint, test, and build commands for the detected ecosystem, and state that assumption explicitly. - If a required live dependency is unavailable, validate every local boundary that does not require the missing dependency and report the blocked live validation separately.
- If a workflow requires data or services absent from the repository, create the smallest realistic fixture outside the main source tree unless the repository has its own fixture convention.
- If a failure appears unrelated to the requested change, prove that with a clean reproduction before excluding it from the QA scope.
Issue <num>: <short-title>
Summary
<Describe the observable failure in one short paragraph.>
Reproduction
<exact command or sequence>Observed before the fix:
- <observable result>
Expected
<Describe the correct behavior.>
Root cause
<Describe the actual source of the failure, not the symptom.>
Fix
<Describe the production change that fixed the root cause.>
Verification
- <narrow reproduction rerun>
- <broader regression or full gate rerun>
VERIFICATION REPORT ------------------- Claim: <what is being claimed> Command: <full verification command> Executed: <timestamp or relative time> Exit code: <0 or non-zero> Output summary: <key pass/fail lines, counts, build result> Warnings: <none or list> Errors: <none or list> Verdict: PASS or FAIL
Systematic Project QA Checklist
Mark every item as complete before claiming the QA pass is done.
Contract Discovery
- [ ] Root instructions and repository docs were read
- [ ] The canonical verify gate was identified or an explicit fallback was chosen
- [ ] The changed surface and regression-critical surface were identified
Baseline
- [ ] Dependencies were installed with the repository-preferred command
- [ ] The baseline verification gate was run before scenario testing
- [ ] Any pre-existing failures were isolated with evidence
User-Like Validation
- [ ] Changed workflows were exercised through public interfaces
- [ ] At least one unchanged regression-critical workflow was exercised
- [ ] Runtime readiness was confirmed with observable signals
- [ ] Fixtures or fake projects were realistic and minimal
Regression Handling
- [ ] Every failure was reproduced before fixing
- [ ] Root cause was identified before implementation
- [ ] Regression coverage was added or updated when the repository supported it
- [ ] The narrow repro and impacted flows were rerun after each fix
Final Verification
- [ ] The full verification gate was rerun after the last code change
- [ ] The most important user-like flows were rerun after the final gate
- [ ] A verification report was produced from fresh evidence
- [ ] Blocked scenarios or missing prerequisites were disclosed explicitly
Project Signal Guide
Use this guide when repository instructions do not already define the canonical QA contract.
Priority Order
1. Root instructions such as AGENTS.md, CLAUDE.md, or repository-specific agent docs 2. Dedicated umbrella commands in Makefile, Justfile, task runners, or CI wrapper scripts 3. CI workflows under .github/workflows/ 4. Ecosystem-native manifests such as package.json, go.mod, pyproject.toml, or Cargo.toml 5. Language-default commands as a last resort
Common Signals
Makefile or Justfile
Treat verify, check, ci, test, lint, build, start, run, and dev as high-confidence targets.
package.json
Prefer explicit scripts in this order:
1. verify, check, ci 2. test, test:ci, test:e2e, test:integration 3. lint, typecheck 4. build 5. start, dev, serve, preview
Go modules
If no umbrella command exists, treat go test ./..., go build ./..., and repository formatting/lint commands as the minimum baseline. Prefer repository wrappers over direct Go commands when both exist.
Python projects
Look for pytest, tox, nox, ruff, mypy, python -m build, and any scripts declared in pyproject.toml.
Rust projects
Treat cargo test, cargo build, cargo fmt --check, and cargo clippy --all-targets --all-features -- -D warnings as strong defaults when the repository does not define wrappers.
Mixed Repositories
When multiple ecosystems exist, identify the product entrypoint first. Do not assume every manifest is part of the same runtime surface.
Scenario Selection Rules
Always cover:
1. A baseline verification gate 2. The workflows directly touched by the change 3. At least one adjacent regression-critical workflow 4. Startup or readiness if the change can affect bootstrapping 5. A realistic fixture path if the feature consumes external projects, repos, files, or APIs
Evidence Rules
Capture exact commands, inputs, outputs, and artifact paths. Prefer observable outcomes over interpretation.
#!/usr/bin/env python3
import argparse
import json
import re
from pathlib import Path
try:
import tomllib
except ModuleNotFoundError: # pragma: no cover
tomllib = None
MAKEFILE_TARGETS = {
"install": ["install", "deps", "setup", "bootstrap"],
"verify": ["verify", "check", "ci"],
"lint": ["lint", "fmt", "format"],
"test": ["test", "unit", "integration", "e2e"],
"build": ["build", "compile"],
"start": ["start", "run", "dev", "serve"],
}
PACKAGE_JSON_TARGETS = {
"install": [],
"verify": ["verify", "check", "ci"],
"lint": ["lint", "lint:ci", "typecheck", "format:check"],
"test": ["test", "test:ci", "test:unit", "test:integration", "test:e2e"],
"build": ["build"],
"start": ["start", "dev", "serve", "preview"],
}
def read_text(path: Path) -> str:
return path.read_text(encoding="utf-8")
def add_command(result: dict, category: str, command: str) -> None:
commands = result["commands"][category]
if command not in commands:
commands.append(command)
def add_signal(result: dict, signal: str) -> None:
if signal not in result["signals"]:
result["signals"].append(signal)
def parse_makefile(path: Path, runner: str, result: dict) -> None:
add_signal(result, path.name)
targets = []
for line in read_text(path).splitlines():
match = re.match(r"^([A-Za-z0-9_.-]+):(?:\s|$)", line)
if not match:
continue
target = match.group(1)
if target.startswith("."):
continue
targets.append(target)
for category, preferred in MAKEFILE_TARGETS.items():
for target in preferred:
if target in targets:
add_command(result, category, f"{runner} {target}")
def parse_package_json(path: Path, result: dict) -> None:
add_signal(result, path.name)
payload = json.loads(read_text(path))
scripts = payload.get("scripts", {})
if not isinstance(scripts, dict):
return
if (path.parent / "package-lock.json").exists():
add_command(result, "install", "npm ci")
elif (path.parent / "pnpm-lock.yaml").exists():
add_command(result, "install", "pnpm install --frozen-lockfile")
elif (path.parent / "yarn.lock").exists():
add_command(result, "install", "yarn install --frozen-lockfile")
else:
add_command(result, "install", "npm install")
for category, preferred in PACKAGE_JSON_TARGETS.items():
for target in preferred:
if target not in scripts:
continue
if target == "test":
add_command(result, category, "npm test")
elif target == "start":
add_command(result, category, "npm start")
else:
add_command(result, category, f"npm run {target}")
def parse_go_mod(path: Path, result: dict) -> None:
add_signal(result, path.name)
add_command(result, "install", "go mod download")
add_command(result, "test", "go test ./...")
add_command(result, "build", "go build ./...")
def parse_cargo_toml(path: Path, result: dict) -> None:
add_signal(result, path.name)
add_command(result, "install", "cargo fetch")
add_command(result, "verify", "cargo test && cargo build")
add_command(result, "lint", "cargo fmt --check")
add_command(result, "lint", "cargo clippy --all-targets --all-features -- -D warnings")
add_command(result, "test", "cargo test")
add_command(result, "build", "cargo build")
def parse_pyproject(path: Path, result: dict) -> None:
add_signal(result, path.name)
data = {}
if tomllib is not None:
data = tomllib.loads(read_text(path))
if (path.parent / "poetry.lock").exists():
add_command(result, "install", "poetry install")
elif (path.parent / "uv.lock").exists():
add_command(result, "install", "uv sync")
elif (path.parent / "requirements.txt").exists():
add_command(result, "install", "python3 -m pip install -r requirements.txt")
tool = data.get("tool", {}) if isinstance(data, dict) else {}
if "pytest" in tool or "pytest.ini_options" in tool.get("pytest", {}):
add_command(result, "test", "pytest")
else:
add_command(result, "test", "pytest")
if "ruff" in tool:
add_command(result, "lint", "ruff check .")
if "black" in tool:
add_command(result, "lint", "black --check .")
if "mypy" in tool:
add_command(result, "lint", "mypy .")
if "build-system" in data:
add_command(result, "build", "python3 -m build")
def collect_ci_signal(root: Path, result: dict) -> None:
workflows = root / ".github" / "workflows"
if not workflows.exists():
return
files = sorted(p.name for p in workflows.iterdir() if p.is_file())
if files:
add_signal(result, ".github/workflows")
def build_result(root: Path) -> dict:
result = {
"root": str(root.resolve()),
"signals": [],
"commands": {
"install": [],
"verify": [],
"lint": [],
"test": [],
"build": [],
"start": [],
},
"notes": [
"Prefer repository-defined umbrella commands over ecosystem defaults.",
"Treat every discovered command as a candidate until repository instructions or CI confirm ownership.",
],
}
if (root / "Makefile").exists():
parse_makefile(root / "Makefile", "make", result)
if (root / "Justfile").exists():
parse_makefile(root / "Justfile", "just", result)
if (root / "package.json").exists():
parse_package_json(root / "package.json", result)
if (root / "go.mod").exists():
parse_go_mod(root / "go.mod", result)
if (root / "Cargo.toml").exists():
parse_cargo_toml(root / "Cargo.toml", result)
if (root / "pyproject.toml").exists():
parse_pyproject(root / "pyproject.toml", result)
collect_ci_signal(root, result)
return result
def main() -> None:
parser = argparse.ArgumentParser(description="Discover candidate QA commands for a repository.")
parser.add_argument("--root", default=".", help="Repository root to inspect.")
args = parser.parse_args()
root = Path(args.root).resolve()
result = build_result(root)
print(json.dumps(result, indent=2, sort_keys=True))
if __name__ == "__main__":
main()
Related skills
TddFollow test-driven development with a strict red-green-refactor loop when creating reliable features or fixing bugs.544k196k
Test Driven DevelopmentEnforce writing failing tests before any production implementation code.181k264kQaRun conversational QA sessions that turn user-reported bugs into well-written, domain-aware GitHub issues without manual ticket writing.179k196kMigrate To ShoehornAutomatically update TypeScript test files that rely on unsafe `as` type assertions by replacing them with type-safe partial objects from @total-typescript/shoehorn.164k196k
Webapp TestingVerify frontend behavior, debug UI issues, capture screenshots, and inspect logs of a running local web application using Playwright.123k165k
Playwright CliRun browser automation, generate element snapshots, inspect DOM attributes, and execute Playwright tests from the terminal.102k12.3k