Genotoxic

Genotoxic

Combines mutation testing and necessist (test statement removal) with code graph analysis to triage findings into actionable categories: false positives, missing unit tests, and fuzzing targets.

When to Use

• After mutation testing reveals survived mutants that need triage
• Identifying where unit tests would have the highest impact
• Finding functions that need fuzz harnesses instead of unit tests
• Prioritizing test improvements using data flow context
• Filtering out harmless mutants from actionable ones
• Finding unnecessary test statements that indicate weak assertions (necessist)

When NOT to Use

• Codebase has no existing test suite (write tests first)
• Pure documentation or configuration changes
• Single-file scripts with trivial logic

Prerequisites

• trailmark installed — if uv run trailmark fails, run:

  uv pip install trailmark

DO NOT fall back to "manual verification" or "manual analysis" as a substitute for running trailmark. Install it first. If installation fails, report the error instead of switching to manual analysis.

• A mutation testing framework for the target language — if the framework

command fails (not found, not installed), install it using the instructions in references/mutation-frameworks.md. DO NOT fall back to "manual mutation analysis" or skip mutation testing. Install the framework first. If installation fails, report the error instead of switching to manual mutation analysis.

• necessist (optional, recommended) — if the target language is

supported (Go, Rust, Solidity/Foundry, TypeScript/Hardhat, TypeScript/Vitest, Rust/Anchor), install with cargo install necessist. See references/mutation-frameworks.md for details.

• An existing test suite that passes
• macOS environment: Run ulimit -n 1024 before any mull-runner

invocation. macOS Tahoe (26+) sets unlimited file descriptors by default, which crashes Mull's subprocess spawning. See references/mutation-frameworks.md for details.

---

Rationalizations to Reject

---

Quick Start

# 1. Build the code graph
uv run trailmark analyze --language auto --summary {targetDir}

# 2. Run mutation testing (language-dependent)
# Python:
uv run mutmut run --paths-to-mutate {targetDir}/src
uv run mutmut results

# 2b. Run necessist (if language supported)
necessist

# 3. Analyze results with this skill's workflow (Phase 3)

---

Workflow Overview

Phase 1: Graph Build      → Parse codebase with trailmark
      ↓
Phase 2: Mutation Run     → Execute mutation testing framework
Phase 2b: Necessist Run   → Remove test statements (optional, parallel)
      ↓
Phase 3: Triage           → Classify findings using graph data
      ↓
Output: Categorized Report
  ├── Corroborated         (both tools flag same function — highest value)
  ├── False Positives      (harmless, skip)
  ├── Missing Tests        (write unit tests)
  └── Fuzzing Targets      (set up fuzz harnesses)

---

Decision Tree

├─ Need to set up mutation testing for a language?
│  └─ Read: references/mutation-frameworks.md
│
├─ Need to set up necessist or find weak test statements?
│  └─ Read: references/mutation-frameworks.md (Necessist section)
│
├─ Need to understand the triage criteria in depth?
│  └─ Read: references/triage-methodology.md
│
├─ Need to understand how graph data informs triage?
│  └─ Read: references/graph-analysis.md
│
└─ Already have results + graph? Use Phase 3 below.

---

Phase 1: Build Code Graph and Run Pre-Analysis

Parse the target codebase with trailmark and run pre-analysis before mutation testing. Pre-analysis computes blast radius, entry points, privilege boundaries, and taint propagation, which Phase 3 uses for triage.

uv run trailmark analyze --language auto --summary {targetDir}

Use the QueryEngine API to build the graph and run pre-analysis: 1. QueryEngine.from_directory("{targetDir}", language="auto") 2. Call engine.preanalysis() — mandatory before triage 3. Export with engine.to_json() for cross-referencing with mutation results

If auto-detection is wrong for the target, rerun with an explicit language or comma-separated list such as python,rust.

See references/graph-analysis.md for the full API: node mapping, reachability queries, blast radius, and pre-analysis subgraph lookups.

---

Phase 2: Run Mutation Testing

Select and run the appropriate framework. See references/mutation-frameworks.md for language-specific setup.

Capture survived mutants. Each framework reports differently, but extract these fields per mutant:

Filter to survived mutants only for Phase 3.

---

Phase 2b: Run Necessist (Optional)

If the target language is supported (Go, Rust, Solidity/Foundry, TypeScript/Hardhat, TypeScript/Vitest, Rust/Anchor), run necessist to find unnecessary test statements. This runs independently of Phase 2 and can execute in parallel.

# Auto-detect framework
necessist

# Or target specific test files
necessist tests/test_parser.rs

# Export results
necessist --dump

Filter to findings where the test passed after removal. See references/mutation-frameworks.md for framework-specific configuration and the normalized record format.

Map each removal to a production function using the algorithm in references/graph-analysis.md.

---

Phase 3: Triage Findings

For each survived mutant and each necessist removal, determine its triage bucket using graph data. Necessist removals must first be mapped to a production function (see references/graph-analysis.md).

Quick Classification (Mutation Testing)

Quick Classification (Necessist)

When both mutation testing and necessist flag the same production function, mark as corroborated — highest confidence finding.

For detailed criteria, see references/triage-methodology.md.

Graph Queries for Triage

For each mutant, map it to its containing graph node and use pre-analysis subgraphs (tainted, high_blast_radius, privilege_boundary) from Phase 1 to classify it. The classification logic checks: no callers → false positive, privilege boundary → fuzzing, high CC + tainted → fuzzing, high blast radius → fuzzing, otherwise → missing tests.

See references/graph-analysis.md for the batch_triage implementation and node mapping functions.

---

Output Format

Generate a markdown report:

# Genotoxic Triage Report

## Summary
- Total survived mutants: N
- Total necessist removals: N
- Corroborated findings: N
- False positives: N (N%)
- Missing test coverage: N (N%)
- Fuzzing targets: N (N%)

## Corroborated Findings
| File | Line | Function | Mutation Signal | Necessist Signal | Action |
|------|------|----------|----------------|------------------|--------|

## False Positives
| File | Line | Mutation | Reason | Source |
|------|------|----------|--------|--------|

## Missing Test Coverage
| File | Line | Function | CC | Callers | Suggested Test | Source |
|------|------|----------|----|---------|----------------|--------|

## Fuzzing Targets
| File | Line | Function | CC | Entrypoint Path | Blast Radius | Source |
|------|------|----------|----|-----------------|--------------|--------|

The Source column is mutation, necessist, or corroborated.

Write the report to GENOTOXIC_REPORT.md in the working directory.

---

Quality Checklist

Before delivering:

• [ ] Trailmark graph built for target language
• [ ] Mutation framework ran to completion
• [ ] Necessist ran (if language supported) or noted as not applicable
• [ ] All survived mutants triaged (none unclassified)
• [ ] All necessist removals triaged (if applicable)
• [ ] Corroborated findings identified (if both tools ran)
• [ ] False positives have clear justifications
• [ ] Missing test items include suggested test type
• [ ] Fuzzing targets include entrypoint paths and blast radius
• [ ] Report file written to GENOTOXIC_REPORT.md
• [ ] User notified with summary statistics

---

Integration

trailmark skill:

• Phase 1: Build code graph, query complexity and entrypoints
• Phase 3: Caller analysis, reachability, blast radius

property-based-testing skill:

• Missing test coverage items involving boundary conditions
• Roundtrip/idempotence properties for serialization mutants

testing-handbook-skills (fuzzing):

• Fuzzing target items: use harness-writing, cargo-fuzz, atheris

---

Supporting Documentation

• [references/mutation-frameworks.md](references/mutation-frameworks.md) -

Language-specific framework setup, output parsing, and necessist configuration

• [references/triage-methodology.md](references/triage-methodology.md) -

Detailed triage criteria, edge cases, and worked examples for both mutation testing and necessist

• [references/graph-analysis.md](references/graph-analysis.md) -

Graph query patterns, test-to-production mapping, and result merging

---

First-time users: Start with Phase 1 (graph build), then run mutations, then use the Quick Classification table in Phase 3.

Experienced users: Jump to Phase 3 and use the Decision Tree to load specific reference material.

Graph Analysis for Mutant Triage

How to use trailmark's code graph data to contextualize survived mutants and assign them to the correct triage bucket.

Contents

• Mapping mutants to graph nodes
• Reachability analysis
• Blast radius calculation
• Complexity correlation
• Annotation-driven triage
• Batch triage workflow
• Mapping necessist removals to graph nodes
• Merging mutation and necessist results

---

Mapping Mutants to Graph Nodes

Each survived mutant has a file_path and line number. Map it to the containing function in the trailmark graph:

def find_containing_node(nodes: dict, file_path: str, line: int):
    """Find the graph node that contains a given source line."""
    candidates = []
    for node_id, node in nodes.items():
        loc = node.get("location", {})
        if not loc:
            continue
        if loc["file_path"] != file_path:
            continue
        if loc["start_line"] <= line <= loc["end_line"]:
            candidates.append((node_id, node))

    if not candidates:
        return None

    # Prefer the most specific (smallest range) containing node
    candidates.sort(
        key=lambda x: (
            x[1]["location"]["end_line"]
            - x[1]["location"]["start_line"]
        )
    )
    return candidates[0][1]

Why smallest range? A line inside a method is also inside its containing class. The method node is the more useful context for triage.

---

Reachability Analysis

Determine whether a mutated function is reachable from untrusted input.

From Entrypoints

def is_entrypoint_reachable(engine, node_id: str) -> bool:
    """Check if any entrypoint can reach this node."""
    return bool(engine.entrypoint_paths_to(node_id))

Entrypoint Path Details

For fuzzing targets, include the specific entrypoint paths in the report:

def entrypoint_paths(engine, node_id: str) -> list[dict]:
    """Get all entrypoint paths to this node with metadata."""
    surface_by_id = {
        ep["node_id"]: ep for ep in engine.attack_surface()
    }
    results = []
    for path in engine.entrypoint_paths_to(node_id):
        ep = surface_by_id.get(path[0], {})
        results.append({
            "entrypoint": path[0],
            "trust_level": ep.get("trust_level"),
            "kind": ep.get("kind"),
            "path": path,
            "hops": len(path),
        })
    return results

Trust Level Weighting

Not all entrypoints are equally dangerous:

Higher-weight entrypoints push mutants toward the fuzzing bucket.

---

Blast Radius Calculation

Blast radius measures how many other functions depend on the mutated function. Higher blast radius means a bug has wider impact.

Direct Callers

def blast_radius(engine, node_id: str) -> dict:
    """Calculate blast radius for a node."""
    callers = engine.callers_of(node_id)
    callees = engine.callees_of(node_id)

    return {
        "direct_callers": len(callers),
        "direct_callees": len(callees),
        "caller_ids": [c["id"] for c in callers],
    }

Transitive Impact

For critical functions, calculate transitive callers (all functions that eventually call this one):

def transitive_context(engine, node_id: str) -> dict:
    """Calculate transitive caller and entrypoint context."""
    ancestors = [
        node for node in engine.ancestors_of(node_id)
        if node["kind"] in {"function", "method"}
    ]
    paths = engine.entrypoint_paths_to(node_id)
    return {
        "transitive_callers": len(ancestors),
        "entrypoint_paths": len(paths),
        "entrypoint_reachable": bool(paths),
    }

Blast Radius Classification

---

Complexity Correlation

Cross-reference survived mutants with complexity data to distinguish "simple enough to unit test" from "complex enough to fuzz."

Per-Function Complexity

def complexity_context(engine, node_id: str) -> dict:
    """Get complexity context for triage decision."""
    hotspots = engine.complexity_hotspots(threshold=1)
    for h in hotspots:
        if h["id"] == node_id:
            return {
                "cyclomatic_complexity": h["cyclomatic_complexity"],
                "is_hotspot": h["cyclomatic_complexity"] >= 10,
            }
    return {"cyclomatic_complexity": 0, "is_hotspot": False}

Decision Matrix

---

Annotation-Driven Triage

Use trailmark annotations to record triage decisions and refine classification over time.

Recording Decisions

from trailmark.models import AnnotationKind

# Mark a function as triaged
engine.annotate(
    node_id,
    AnnotationKind.ASSUMPTION,
    "genotoxic: false_positive (equivalent mutant in logging)",
    source="llm",
)

# Mark a fuzzing target with rationale
engine.annotate(
    node_id,
    AnnotationKind.ASSUMPTION,
    "genotoxic: fuzzing_target (CC=14, entrypoint-reachable via /api/parse)",
    source="llm",
)

Querying Previous Triage

# Check if a function was previously triaged
annotations = engine.annotations_of(node_id)
genotoxic_annotations = [
    a for a in annotations
    if a["description"].startswith("genotoxic:")
]

This enables incremental triage across multiple mutation testing runs.

---

Batch Triage Workflow

For large codebases with many survived mutants, process in batch:

import json

def batch_triage(engine, survived_mutants: list[dict]) -> dict:
    """Classify all survived mutants."""
    graph_json = json.loads(engine.to_json())
    nodes = graph_json["nodes"]

    results = {
        "false_positives": [],
        "missing_tests": [],
        "fuzzing_targets": [],
    }

    for mutant in survived_mutants:
        node = find_containing_node(
            nodes, mutant["file_path"], mutant["line"]
        )
        if not node:
            results["false_positives"].append({
                **mutant,
                "reason": "no containing function in graph",
            })
            continue

        node_id = node["id"]
        callers = engine.callers_of(node_id)
        cc = node.get("cyclomatic_complexity", 0) or 0

        # Dead code
        if not callers:
            results["false_positives"].append({
                **mutant,
                "reason": "no callers (dead code)",
                "node_id": node_id,
            })
            continue

        reachable = is_entrypoint_reachable(engine, node_id)

        # Fuzzing criteria
        if (cc > 10 and reachable) or (len(callers) > 10 and cc > 5):
            ep_paths = entrypoint_paths(engine, node_id)
            results["fuzzing_targets"].append({
                **mutant,
                "node_id": node_id,
                "cyclomatic_complexity": cc,
                "caller_count": len(callers),
                "entrypoint_paths": ep_paths,
                "blast_radius": blast_radius(engine, node_id),
            })
            continue

        # Default: missing tests
        results["missing_tests"].append({
            **mutant,
            "node_id": node_id,
            "cyclomatic_complexity": cc,
            "caller_count": len(callers),
            "entrypoint_reachable": reachable,
        })

    return results

Performance Considerations

• Path queries are expensive. Cache paths_between results when

checking multiple mutants against the same entrypoints.

• Process by function, not by mutant. Multiple mutants in the same

function share the same graph context. Group mutants by containing function first, query graph once per function.

• Use `complexity_hotspots` as a prefilter. Functions with CC < 5

are almost never fuzzing targets. Skip reachability analysis for them unless caller count is very high.

---

Mapping Necessist Removals to Graph Nodes

Necessist findings reference test code locations, but triage requires the production function that the removed statement exercises. Extract the called function name from the removed statement and match it against graph nodes.

import re


def map_removal_to_production_node(
    nodes: dict,
    removed_statement: str,
    test_file_path: str,
) -> dict | None:
    """Map a necessist removal to the production function it exercises."""
    # Extract function/method name from the removed statement.
    # Handles: obj.method(args), function(args), obj.method!(args)
    match = re.search(
        r"(?:(\w+)\.)?(\w+!?)\s*\(", removed_statement
    )
    if not match:
        return None

    func_name = match.group(2)

    # Search graph nodes for matching function name
    candidates = [
        (nid, n) for nid, n in nodes.items()
        if n.get("name") == func_name
        and "test" not in n.get("location", {})
            .get("file_path", "").lower()
    ]

    if len(candidates) == 1:
        return candidates[0][1]

    # Disambiguate: prefer node in the production module
    # that mirrors the test file path
    prod_path = infer_production_path(test_file_path)
    for nid, n in candidates:
        if n.get("location", {}).get("file_path") == prod_path:
            return n

    # Fall back to first non-test candidate
    return candidates[0][1] if candidates else None


def infer_production_path(test_file_path: str) -> str:
    """Heuristic: map test file to likely production file.

    tests/test_parser.py  → src/parser.py
    test/parser_test.go   → parser.go
    tests/Parser.test.ts  → src/Parser.ts
    """
    path = test_file_path
    # Strip test directory prefixes
    path = re.sub(r"^tests?/", "src/", path)
    # Strip test_ prefix or _test / .test suffix
    path = re.sub(r"test_(\w+)", r"\1", path)
    path = re.sub(r"(\w+)_test\.", r"\1.", path)
    path = re.sub(r"(\w+)\.test\.", r"\1.", path)
    return path

When mapping fails: If no production node matches, classify the removal as a false positive with reason "unmappable to production code." This is conservative — the removal may still be meaningful, but without graph context triage cannot assign a confident bucket.

---

Merging Mutation and Necessist Results

When both mutation testing and necessist produce findings for the same production function, this is a corroborated finding: the function has both uncaught production mutations and unnecessary test statements. Corroborated findings are highest confidence.

def merge_results(
    mutation_results: dict,
    necessist_results: dict,
) -> dict:
    """Merge mutation and necessist triage results.

    Identifies corroborated findings where both tools flag
    the same production function.
    """
    merged = {
        "corroborated": [],
        "false_positives": (
            mutation_results["false_positives"]
            + necessist_results["false_positives"]
        ),
        "missing_tests": [],
        "fuzzing_targets": [],
    }

    # Index necessist findings by production node_id
    necessist_by_node = {}
    for item in (
        necessist_results["missing_tests"]
        + necessist_results["fuzzing_targets"]
    ):
        nid = item.get("node_id")
        if nid:
            necessist_by_node.setdefault(nid, []).append(item)

    # Check mutation findings for corroboration
    for bucket in ("missing_tests", "fuzzing_targets"):
        for item in mutation_results[bucket]:
            nid = item.get("node_id")
            if nid and nid in necessist_by_node:
                merged["corroborated"].append({
                    "node_id": nid,
                    "mutation": item,
                    "necessist": necessist_by_node.pop(nid),
                })
            else:
                merged[bucket].append(item)

    # Add remaining non-corroborated necessist findings
    for items in necessist_by_node.values():
        for item in items:
            bucket = (
                "fuzzing_targets"
                if item in necessist_results["fuzzing_targets"]
                else "missing_tests"
            )
            merged[bucket].append(item)

    return merged

Corroborated findings should appear in a dedicated report section before the individual buckets, since they represent the highest-value action items.

Mutation Testing Frameworks

Language-specific setup, execution, and output parsing for mutation testing.

Contents

• Language detection
• Framework reference table
• Per-language setup and commands
• Parsing survived mutants
• Necessist (test statement removal)

---

Installation Policy

Every mutation testing framework listed below MUST be installed before proceeding. If a framework command is not found or fails to install:

1. Try the primary install method for the platform 2. Try the alternative install methods listed in the language section 3. If all methods fail, report the error to the user — do NOT fall back to "manual mutation analysis", "manual verification", or any other substitute that skips running the tool

Manual analysis is not a replacement for mutation testing. Mutation testing tools systematically apply hundreds or thousands of mutations that manual review cannot replicate. Skipping installation and doing manual analysis produces false confidence with minimal actual coverage.

---

Language Detection

Use file extensions to determine the target language, then select the appropriate mutation framework:

---

Python: pytest-gremlins (preferred) or mutmut

pytest-gremlins

Faster alternative to mutmut. Uses mutation switching (no file I/O or module reloads), coverage-guided test selection, and parallel execution. Requires Python 3.11+.

Install:

uv add --dev pytest-gremlins

Run:

uv run pytest --gremlins

No configuration needed — it integrates directly with pytest.

Parse survived mutants: pytest-gremlins reports survived gremlins in its test output. Each entry includes the file, line, mutation type, and original/replacement values.

mutmut

Install:

uv add --dev mutmut

Configure in pyproject.toml:

[tool.mutmut]
paths_to_mutate = "src/"
tests_dir = "tests/"
runner = "python -m pytest -x -q"

Run:

uv run mutmut run
uv run mutmut results

Parse survived mutants:

# List survived mutant IDs
uv run mutmut results | grep "Survived"

# Show specific mutant
uv run mutmut show <id>

# Export all results as JSON (mutmut 3.x+)
uv run mutmut junitxml > mutmut-results.xml

Extract from results output: Each survived mutant line contains the file path, line number, and mutation description. Parse with:

uv run mutmut results 2>&1 | grep "Survived" | \
  sed 's/.*Survived: //'

macOS note: If using rustworkx or other Rust extensions, set:

export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

---

JavaScript/TypeScript: Stryker

Install:

pnpm add -D @stryker-mutator/core
pnpm dlx stryker init

Configure stryker.config.json:

{
  "mutate": ["src/**/*.ts", "!src/**/*.test.ts"],
  "testRunner": "vitest",
  "reporters": ["json", "clear-text"],
  "jsonReporter": { "fileName": "stryker-report.json" }
}

Run:

pnpm dlx stryker run

Parse survived mutants:

# JSON report at reports/mutation/stryker-report.json
# Filter survived:
cat reports/mutation/stryker-report.json | \
  jq '.files | to_entries[] | .value.mutants[] | select(.status == "Survived")'

Output fields: mutatorName, replacement, location.start.line, location.start.column, fileName.

---

Rust: cargo-mutants

Install:

cargo install cargo-mutants

Run:

cargo mutants --json

Parse survived mutants:

# Results in mutants.out/outcomes.json
cat mutants.out/outcomes.json | \
  jq '.[] | select(.outcome == "survived")'

Output fields: scenario.function, scenario.file, scenario.line, scenario.replacement, outcome.

Filtering by module:

cargo mutants --file src/parser.rs --json

---

Go: gremlins (preferred) or go-mutesting

gremlins

Actively maintained mutation testing tool for Go. Works best on small-to-medium Go modules (microservices, libraries).

Install:

# macOS
brew tap go-gremlins/tap && brew install gremlins

# Any platform with Go
go install github.com/go-gremlins/gremlins/cmd/gremlins@latest

Run:

gremlins unleash .

Parse results: gremlins reports survived mutants to stdout with file path, line number, and mutation type.

go-mutesting

Install:

go install github.com/zimmski/go-mutesting/cmd/go-mutesting@latest

Run:

go-mutesting ./...

Parse results: go-mutesting prints survived mutants to stdout. Each line contains the file, line number, and mutation operator.

Alternative: native fuzzing (Go 1.18+)

go test -fuzz=FuzzTarget -fuzztime=60s ./pkg/...

---

Java: PITest

Configure in pom.xml:

<plugin>
  <groupId>org.pitest</groupId>
  <artifactId>pitest-maven</artifactId>
  <configuration>
    <targetClasses>com.example.*</targetClasses>
    <outputFormats>XML,CSV</outputFormats>
  </configuration>
</plugin>

Run:

mvn org.pitest:pitest-maven:mutationCoverage

Parse survived mutants:

# Results in target/pit-reports/mutations.xml
# Filter SURVIVED status
grep 'status="SURVIVED"' target/pit-reports/*/mutations.xml

Output fields: mutatedClass, mutatedMethod, lineNumber, mutator, status.

---

C/C++: Mull

Mull is an LLVM-based mutation testing tool for C and C++. It works as a compiler plugin — it instruments the compiled test binary with mutations, then selectively activates them during test execution.

Mull requires a specific LLVM version. Check the Mull releases page for the LLVM version supported by the latest release. The project must compile with the matching Clang version.

Install

Mull is distributed as prebuilt binaries on GitHub Releases. Each binary targets a specific LLVM version — you must match the Mull binary's LLVM version to the Clang version installed on the system.

Step 1: Determine your Clang/LLVM version:

clang --version
# Look for the major version number (e.g., 19, 20)

If Clang is not installed, install it first. On macOS, use brew install llvm@<version>. On Ubuntu, use sudo apt-get install clang-<version>.

Step 2: Download the matching Mull binary.

Go to the Mull releases page and download the asset matching your LLVM version, platform, and architecture. Asset naming convention:

Mull-<LLVM_MAJOR>-<MULL_VERSION>-LLVM-<LLVM_FULL>-<OS>-<ARCH>.<ext>

Examples (Mull 0.29.0):

Step 3: Install.

macOS:

# 1. Install the matching LLVM/Clang version via Homebrew
#    Check Mull releases for which LLVM versions are available
brew install llvm@18  # or llvm@19, llvm@20

# 2. Download the matching Mull binary
gh release download --repo mull-project/mull \
  --pattern 'Mull-18-*-macOS-aarch64-*.zip'  # match LLVM version
unzip Mull-18-*.zip

# 3. Install binaries to a known location
sudo mkdir -p /usr/local/bin /usr/local/lib
sudo cp usr/local/bin/mull-runner-* /usr/local/bin/
sudo cp usr/local/bin/mull-reporter-* /usr/local/bin/
sudo cp usr/local/lib/mull-ir-frontend-* /usr/local/lib/

# 4. Verify
mull-runner-18 --version
/opt/homebrew/opt/llvm@18/bin/clang --version

Important macOS notes:

• The Mull binary's LLVM version must exactly match the installed

Clang. Using brew install llvm@18 with Mull-19-* will not work.

• Use the Homebrew Clang, not Apple's system Clang (which is a

different LLVM version and lacks plugin support).

• Set ulimit -n 1024 before running mull-runner (see Environment

Setup section below).

Ubuntu/Debian:

# Option A: Cloudsmith APT repository
curl -1sLf \
  'https://dl.cloudsmith.io/public/mull-project/mull-stable/setup.deb.sh' \
  | sudo -E bash
sudo apt-get update
sudo apt-get install mull-19  # match your LLVM version

# Option B: Direct .deb from GitHub
gh release download --repo mull-project/mull \
  --pattern 'Mull-19-*-ubuntu-amd64-24.04.deb'
sudo dpkg -i Mull-19-*.deb

RHEL/Fedora:

# Option A: Cloudsmith RPM repository
curl -1sLf \
  'https://dl.cloudsmith.io/public/mull-project/mull-stable/setup.rpm.sh' \
  | sudo -E bash
sudo dnf install mull-20  # match your LLVM version

# Option B: Direct .rpm from GitHub
gh release download --repo mull-project/mull \
  --pattern 'Mull-20-*-rhel-amd64-*.rpm'
sudo rpm -i Mull-20-*.rpm

Verify installation:

mull-runner --version

If mull-runner is not found after installation, check that the install prefix is on $PATH. DO NOT fall back to "manual mutation analysis" — fix the installation or report the error.

Configure and Build

Mull requires the project to be compiled with Clang and the Mull compiler plugin. The plugin injects mutations at the LLVM IR level.

Key build requirements:

• Use the same Clang version that matches your Mull release
• Pass -fpass-plugin=<path-to-mull-ir-frontend> to the compiler
• Use -g -O0 (debug info required, no optimization)
• Disable assembly (--disable-asm) — Mull can only mutate

LLVM IR, not hand-written assembly

• Disable hardening flags that interfere: --disable-ssp --disable-pie

Find the plugin path:

# The plugin is typically installed alongside mull-runner:
# Linux:  /usr/lib/mull-ir-frontend-<N>  (or mull-ir-frontend.so)
# macOS:  <install-prefix>/lib/mull-ir-frontend-<N>
# Use `find` or `locate` if unsure:
find /usr/local /opt/homebrew /tmp -name "mull-ir-frontend*" 2>/dev/null

Simple projects:

MULL_PLUGIN=$(find /usr/local /opt/homebrew -name "mull-ir-frontend*" 2>/dev/null | head -1)
clang -fpass-plugin=$MULL_PLUGIN -g -O0 \
  -o test_binary test_main.c src/*.c

Autotools projects (configure/make):

MULL_PLUGIN=$(find /usr/local /opt/homebrew -name "mull-ir-frontend*" 2>/dev/null | head -1)
LLVM_BIN=$(dirname $(which clang))  # or /opt/homebrew/opt/llvm@18/bin

CC=$LLVM_BIN/clang \
CFLAGS="-fpass-plugin=$MULL_PLUGIN -g -grecord-command-line -O0" \
./configure --disable-shared --enable-static --disable-asm \
  --disable-ssp --disable-pie
make clean && make -j$(nproc)

CMake projects:

set(CMAKE_C_COMPILER clang)
set(CMAKE_CXX_COMPILER clang++)
set(MULL_PLUGIN_PATH "" CACHE STRING "Path to Mull plugin")
if(MULL_PLUGIN_PATH)
  add_compile_options(-fpass-plugin=${MULL_PLUGIN_PATH} -g -O0)
endif()

MULL_PLUGIN=$(find /usr/local /opt/homebrew -name "mull-ir-frontend*" 2>/dev/null | head -1)
cmake -B build -DMULL_PLUGIN_PATH=$MULL_PLUGIN
cmake --build build

Run

# Set FD limit (required on macOS, see Environment Setup)
ulimit -n 1024

# Run with GoogleTest binary
mull-runner --allow-surviving --no-output --timeout=5000 \
  --reporters=Elements --report-dir=mull-report ./build/tests

# Run with custom test command
mull-runner --test-program=ctest ./build/tests

# Generate report
mull-runner --report-dir=mull-report ./build/tests

Recommended flags:

• --allow-surviving — don't treat survived mutants as errors
• --no-output — suppress stdout/stderr from mutant runs
• --timeout=5000 — 5 second timeout per mutant (adjust based on

baseline test runtime; use 1000ms for tests completing in <100ms)

• --reporters=Elements — JSON output in Mutation Testing Elements

format (machine-parseable for triage)

• --report-dir=DIR — write JSON reports to this directory
• --report-name=NAME — control output filename (useful when

running multiple test binaries)

• --workers=N — parallelism for mutant execution (defaults to

CPU count)

Parse survived mutants

Mull outputs results to stdout and optionally to report files. Each survived mutant includes the file path, line number, and mutation type.

# JSON report (if --report-dir used)
cat mull-report/mutation-testing-report.json | \
  jq '.files | to_entries[] | .value.mutants[] |
    select(.status == "Survived")'

Environment Setup (Required)

Before running `mull-runner`, always set a bounded file descriptor limit. On macOS (especially Tahoe / macOS 26+), the default ulimit -n is unlimited, which causes Mull's subprocess library (reproc) to fail with EINVAL when it tries to close inherited file descriptors in the forked child process. The fix:

# REQUIRED before any mull-runner invocation
ulimit -n 1024

Add this to your Mull runner scripts or shell session. Without it, you will see:

[error] Cannot run executable: Invalid argument

Root cause: reproc calls getrlimit(RLIMIT_NOFILE) to determine the max FD to close. When the soft limit is RLIM_INFINITY, reproc computes max_fd = INT_MAX, which exceeds its internal MAX_FD_LIMIT (1048576) safety check, causing the child to exit with EMFILE.

Troubleshooting

---

C#: Stryker.NET

Install:

dotnet tool install -g dotnet-stryker

Run:

dotnet stryker --reporter json

Parse survived mutants:

cat StrykerOutput/*/reports/mutation-report.json | \
  jq '.files | to_entries[] | .value.mutants[] | select(.status == "Survived")'

---

Ruby: mutant

Install:

gem install mutant

Run:

bundle exec mutant run --include lib --require mylib 'MyLib*'

Parse results: mutant outputs surviving mutations to stdout with file paths, line numbers, and mutation descriptions.

---

PHP: Infection

Install:

composer require --dev infection/infection

Run:

vendor/bin/infection --show-mutations --min-msi=0

Parse survived mutants:

# JSON log at infection-log.json
cat infection-log.json | jq '.survived[]'

---

Circom: circomvent

circomvent is Trail of Bits' mutation testing framework for Circom ZK circuits. It applies circuit-specific mutations (constraint removal, signal swaps, operator changes) and checks whether the test suite detects each mutation.

Install:

# Clone and install from source
git clone https://github.com/trailofbits/circomvent
cd circomvent
# Follow install instructions in the repo README

Run:

circomvent --help  # Check available commands and options

Parse survived mutants: circomvent reports survived mutants with the circuit file path, line number, and mutation type. Normalize to the universal mutant record format for Phase 3 triage.

Note: circomvent is an internal Trail of Bits tool. Check the repository README for the latest installation and usage instructions.

---

Cairo: cairo-mutants

cairo-mutants is Trail of Bits' mutation testing framework for Cairo smart contracts (StarkNet). It applies Cairo-specific mutations and verifies test detection.

Install:

# Clone and install from source
git clone https://github.com/trailofbits/cairo-mutants
cd cairo-mutants
# Follow install instructions in the repo README

Run:

cairo-mutants --help  # Check available commands and options

Parse survived mutants: cairo-mutants reports survived mutants with the file path, line number, and mutation type. Normalize to the universal mutant record format for Phase 3 triage.

Note: cairo-mutants is an internal Trail of Bits tool. Check the repository README for the latest installation and usage instructions.

---

Haskell: MuCheck or Hedgehog

MuCheck is the primary mutation testing tool for Haskell. For projects without MuCheck support, property-based testing with Hedgehog or QuickCheck serves as a mutation-resistant alternative.

MuCheck

Install:

cabal install MuCheck

Run:

mucheck -t "cabal test" src/MyModule.hs

MuCheck applies standard mutation operators (negate guards, swap operators, replace patterns) to the target module and runs the test suite against each mutant.

Parse survived mutants: MuCheck prints results to stdout. Each survived mutant includes the file path, line number, and mutation description (e.g., "Negated guard on line 42").

Limitations: MuCheck requires the project to build with cabal and has limited support for large multi-module projects. For Stack-based projects, wrap the test command: mucheck -t "stack test" src/Module.hs.

Alternative: property-based testing as mutation proxy

For projects where MuCheck is impractical, strong property-based tests provide equivalent mutation resistance. Properties that assert invariants over all inputs catch most mutations that MuCheck would surface.

Hedgehog (preferred):

cabal install hedgehog

Write properties in test/ that cover arithmetic, branching, and boundary behavior. A comprehensive property suite catches the same classes of defects as mutation testing.

QuickCheck:

cabal install QuickCheck

QuickCheck properties work similarly. Use forAll with custom generators to target the input domain of each function under test.

---

Solidity: slither-mutate

slither-mutate is Slither's built-in mutation testing tool for Solidity smart contracts. It applies 15 Solidity-aware mutation operators to production code, runs the project's test suite against each mutant, and saves survived mutants as diffs. Based on arxiv.org/abs/2006.11597.

Install

slither-mutate ships with Slither. Install Slither to get it:

# From PyPI
uv tool install slither-analyzer

# From source (editable, for development)
uv tool install -e /path/to/slither

Verify:

slither-mutate --help

Mutation Operators

slither-mutate applies mutations in severity order. High-severity operators (RR, CR) run first. If a high-severity mutant survives on a line, lower-severity operators skip that line (unless --comprehensive is set).

Run

# Foundry project
slither-mutate . --test-cmd "forge test" --compile-force-framework foundry

# Hardhat project
slither-mutate . --test-cmd "npx hardhat test" --compile-force-framework hardhat

# Single contract file
slither-mutate src/Vault.sol --test-cmd "forge test"

# Scope to specific contracts
slither-mutate . --test-cmd "forge test" --contract-names "Vault,Router"

# Scope to specific functions by selector or signature
slither-mutate . --test-cmd "forge test" \
  --contract-names Vault \
  --target-functions "deposit(uint256),withdraw(uint256,address)"

# Run all operators even when severe mutants survive
slither-mutate . --test-cmd "forge test" --comprehensive

# Ignore library/interface directories
slither-mutate . --test-cmd "forge test" --ignore-dirs "lib,interfaces"

# Custom timeout (default: 2x baseline test runtime)
slither-mutate . --test-cmd "forge test" --timeout 120

# Verbose mode (log each mutant's status)
slither-mutate . --test-cmd "forge test" -v

Output Structure

Results are saved to mutation_campaign/ (override with --output-dir):

mutation_campaign/
├── patches_files.txt          # Unified diffs of all uncaught mutants
└── <ContractName>/
    ├── <ContractName>_RR_0.sol   # Survived mutant: require removal #0
    ├── <ContractName>_CR_0.sol   # Survived mutant: comment replacement #0
    ├── <ContractName>_AOR_0.sol  # Survived mutant: arithmetic op #0
    └── ...

The filename encodes the operator and sequence number: <Contract>_<OPERATOR>_<N>.sol.

Parse Survived Mutants

slither-mutate does not produce structured JSON output directly. Parse the patches_files.txt diff file to extract survived mutants:

# Extract file paths and line numbers from unified diffs
grep -E '^\+\+\+ |^@@ ' mutation_campaign/patches_files.txt

Each diff block in patches_files.txt represents one uncaught mutant. Extract:

• File path from the +++ b/<path> line
• Line number from the @@ -N,M +N,M @@ hunk header
• Mutation type from the mutant filename in the output directory

To normalize for Phase 3, map each diff to the universal mutant record:

# List all survived mutant files with their operators
ls mutation_campaign/*/*.sol | \
  sed 's/.*\///' | \
  sed 's/\(.*\)_\([A-Z]*\)_\([0-9]*\)\.sol/\2 \3/'

Mapping to Universal Record Format

For each survived mutant file, construct the normalized record:

{
  "file_path": "src/Vault.sol",
  "line": 87,
  "mutation_type": "RR",
  "original": "require(amount > 0, \"zero amount\");",
  "replacement": "/* require removed */",
  "function_name": "deposit",
  "status": "survived"
}

Map mutation_type to the operator table above. Extract line from the diff hunk header. Map function_name by matching the line against trailmark graph nodes or by diffing the mutant .sol file against the original.

Severity Cascade and Triage Integration

The severity ordering directly informs genotoxic triage:

• RR survived (require removal) → high-confidence Missing Tests or

Fuzzing Target. A missing require guard that tests don't catch is a real coverage gap.

• CR survived (code deletion) → function body or branch is untested.

Classify as Missing Tests if low CC, Fuzzing Target if high CC or entrypoint-reachable.

• Tweak survived (AOR, ROR, LIR, etc.) → boundary or arithmetic behavior

is untested. Good candidates for property-based tests.

• Mutant doesn't compile → skip (slither-mutate already filters these).

Complementary Use with Necessist

For Foundry projects, run both slither-mutate (production code mutations) and necessist with --framework foundry (test statement removal). When both tools flag the same function, mark as corroborated in the triage report.

# Production mutations
slither-mutate . --test-cmd "forge test" --comprehensive -v

# Test statement removal (parallel)
necessist --framework foundry

Troubleshooting

---

Universal Mutant Record Format

Regardless of framework, normalize each survived mutant to this schema before feeding into Phase 3 triage:

{
  "file_path": "src/parser.py",
  "line": 42,
  "mutation_type": "arithmetic_operator",
  "original": "+",
  "replacement": "-",
  "function_name": "parse_header",
  "status": "survived"
}

Map the containing function name by matching file_path:line against trailmark graph nodes using their location.start_line and location.end_line ranges.

---

Necessist: Test Statement Removal

Necessist complements mutation testing by removing statements and method calls from test code and re-running the tests. If a test still passes after a statement is removed, that statement may be unnecessary — indicating weak assertions or missing coverage.

Mutation testing mutates production code to check if tests detect changes. Necessist mutates test code to check if each test statement is actually needed. Run both when the language supports it.

Supported Frameworks

Necessist auto-detects the framework from project files. Use --framework to override when auto-detection fails.

Install

cargo install necessist

Run

# Auto-detect framework, run on all test files
necessist

# Explicit framework selection
necessist --framework foundry

# Target specific test files
necessist tests/test_parser.rs tests/test_validator.rs

# Set timeout per test (default 60s, 0 = no timeout)
necessist --timeout 120

# Resume a previous run (results stored in SQLite)
necessist --resume

Parse Results

Necessist stores results in a SQLite database by default. Use --dump to export:

necessist --dump

Each result line contains the test file, line number, the removed statement, and whether the test passed or failed after removal. Filter to passed after removal entries — these are the findings to triage.

Configuration

Create necessist.toml in the project root (necessist --default-config generates a template):

ignored_functions = ["println", "eprintln", "dbg"]
ignored_methods = ["clone", "to_string", "unwrap"]
ignored_macros = ["debug_assert", "trace"]

• ignored_functions — Skip removals of these function calls
• ignored_methods — Skip removals of these method calls
• ignored_macros — Skip removals of these macro invocations

For Foundry projects, consider ignoring common cheatcodes that are setup-only (e.g., vm.label, vm.deal for labeling/funding).

Normalized Necessist Record Format

Normalize each finding before feeding into Phase 3 triage:

{
  "test_file_path": "tests/test_parser.rs",
  "test_line": 42,
  "removed_statement": "parser.validate(&input)",
  "test_function": "test_parse_header",
  "status": "passed_after_removal",
  "source": "necessist"
}

The source field distinguishes necessist findings from mutation testing results during triage and reporting. Map the removed statement to a production function using the graph analysis algorithm.

Triage Methodology

Detailed criteria for classifying survived mutants into actionable buckets.

Contents

• False positive detection
• Missing test coverage identification
• Fuzzing target selection
• Edge cases and ambiguous mutants
• Worked examples
• Necessist removal triage

---

False Positive Detection

A mutant is a false positive when killing it would not improve code quality or catch real bugs. Classify as false positive when ANY of these conditions hold:

Dead Code

The mutated function has zero callers in the trailmark graph.

callers = engine.callers_of(node_id)
if not callers:
    # Dead code. The mutant is unreachable in production.
    # Action: flag function for removal, not testing.

Subtlety: A function with no direct callers may still be reachable via dynamic dispatch (reflection, callbacks, decorators). Check edge confidence: if all edges to the function are uncertain, investigate before dismissing.

Test-Only Code

The function is only called from test files.

callers = engine.callers_of(node_id)
prod_callers = [
    c for c in callers
    if "test" not in c["location"]["file_path"].lower()
]
if not prod_callers:
    # Only tests call this. Mutant in test infrastructure.

Equivalent Mutants

The mutation produces identical behavior. Common patterns:

Detection strategy: Check if the mutation is in a logging call, display string, comment-adjacent code, or assertion message. These are cosmetic and do not affect program behavior.

Redundant Checks

The mutation weakens a condition, but another check in the same call path enforces the same constraint:

# Caller validates x > 0 before calling this function.
# Mutating this function's own x > 0 check is redundant.
callers = engine.callers_of(node_id)
# Inspect caller source for equivalent preconditions.

Use trailmark annotations to track this:

from trailmark.models import AnnotationKind

engine.annotate(
    node_id,
    AnnotationKind.PRECONDITION,
    "x > 0 enforced by all callers",
    source="llm",
)

---

Missing Test Coverage

A mutant indicates missing test coverage when:

1. The function is reachable in production (has callers) 2. The mutated behavior should be caught by tests 3. A unit test is the appropriate testing strategy

Criteria

Suggested Test Types

Map the mutation type to a test strategy:

When to Prefer Property-Based Testing

If the survived mutant involves:

• Serialization/deserialization (roundtrip property)
• Idempotent operations (applying twice = applying once)
• Ordering invariants (sorted output)
• Numeric ranges or bounds

Use the property-based-testing skill for guidance.

---

Fuzzing Target Selection

A survived mutant is a fuzzing target when unit testing alone is insufficient due to complexity, input space, or exposure to untrusted data.

Criteria

Prioritization

Combine signals for priority assignment:

CRITICAL: Entrypoint reachable + CC > 15 + parser/validator
HIGH:     Entrypoint reachable + CC > 10
HIGH:     CC > 10 + caller count > 20
MEDIUM:   CC > 10 OR (entrypoint reachable + caller count > 10)
LOW:      Moderate complexity, not entrypoint reachable

Framework Selection

Based on target language, recommend the appropriate fuzzer:

---

Edge Cases and Ambiguous Mutants

Some mutants don't cleanly fit one bucket. Resolution rules:

Mutant in Validation Code

If the mutant weakens input validation:

• Entrypoint reachable? → Fuzzing target (attacker can exploit)
• Internal only? → Missing test (regression risk)

Mutant in Error Path

If the mutant changes error handling behavior:

• Error path tested? → Check if test expects specific error
• Error path untested? → Missing test (negative test case)
• Error in parser? → Fuzzing target (malformed input testing)

Mutant Straddles Complexity Threshold

CC is near the threshold (8-12 range):

• Has entrypoint path? → Fuzzing target (exposure wins)
• No entrypoint path? → Missing test (unit test is feasible)

Tie-Breaking Rule

When signals conflict, prefer the higher-assurance category:

Fuzzing Target > Missing Test > False Positive

A function that could be unit tested but is also entrypoint-reachable and complex should be fuzzed. Fuzzing subsumes the unit test goal while providing broader coverage.

---

Worked Example

Scenario: Python web application, mutmut reports 47 survived mutants.

Graph context (from trailmark):

• 312 nodes, 1,847 edges
• 8 entrypoints (Flask route handlers)
• 14 functions with CC > 10

Triage results:

Key decisions:

• parse_query_params (CC=14, entrypoint-reachable via /search) → Fuzzing
• format_error_response (CC=3, 2 callers, string formatting) → False positive (cosmetic)
• validate_email (CC=6, 4 callers, no entrypoint path) → Missing test (boundary cases)
• build_sql_filter (CC=12, entrypoint-reachable via /api/filter) → Fuzzing (injection risk)

---

Necessist Removal Triage

Necessist findings differ from mutation testing: they identify test statements whose removal doesn't cause test failure. Triage maps each removal to a production function using the graph analysis algorithm and then classifies it.

False Positive Detection (Necessist)

Classify a necessist removal as false positive when:

Missing Test Coverage (Necessist)

A removal indicates missing coverage when the test should fail but doesn't — meaning the test has weak or missing assertions:

Fuzzing Target Selection (Necessist)

After mapping to a production function, apply the same graph-based criteria as mutation testing:

• CC > 10 and entrypoint reachable → Fuzzing Target
• High blast radius and CC > 5 → Fuzzing Target
• On a privilege boundary → Fuzzing Target

The reasoning is identical: if a production function is complex, exposed, and its test coverage is demonstrably weak (necessist proved a test statement was unnecessary), fuzzing is the appropriate response.

Edge Cases (Necessist)

Async/await removals: Removing an await may cause a test to pass because the assertion runs before the async operation completes. This is a genuine test weakness (race condition in test), not a false positive. Classify as missing test coverage — the test needs to properly await and assert.

Macro expansions (Foundry/Anchor): Cheatcodes like vm.prank, vm.expectRevert, vm.warp are setup-critical. If removing one causes the test to still pass, the test likely doesn't exercise the behavior the cheatcode was supposed to enable. Classify as missing test coverage unless the cheatcode is purely cosmetic (vm.label).

Chained method calls: foo.bar().baz() — necessist may remove the entire chain. Map to the outermost call (foo.bar) for triage. If the chain involves multiple production functions, triage against the one with highest blast radius.

Solidity `assert` vs `require`: Removing a require check in a test helper is different from removing an assert in a test body. require removals in test helpers are usually false positives (guard conditions). assert removals in test bodies are missing coverage.

Worked Example: Foundry Project

Scenario: Foundry DeFi lending protocol, necessist reports 31 removals that passed.

Graph context (from trailmark):

• 89 nodes, 412 edges
• 5 entrypoints (external functions)
• 6 functions with CC > 10

Triage results:

Key decisions:

• calculateInterest (CC=11, reachable via borrow()) → Fuzzing — test removed assertApproxEqRel and still passed, meaning the interest calculation has untested edge cases in a complex, exposed function
• vm.label(address(pool), "pool") → False positive — cosmetic labeling for trace output
• assertEq(token.balanceOf(user), expectedBalance) removed and test passes → Missing test — the balance check was the only assertion verifying the transfer succeeded