Create Test Project

Skill: Create Test Project

Build a minimal compiled test project whose annotated samples reproduce the flow a rule or approximation is checked against. The compiled model is the deliverable; its sources sit alongside it

Inputs

From the caller; if omitted, fall back to the default. Ask only when a required input is missing and has no sensible default

• What to test <spec> — a rule's requirements, or the package's methods to exercise
• Project root <project-root> — the real sources the requirements point into. Default: current directory
• Tracking file <tracking-file> — the rule or approximation file this test serves. Default: .opentaint/tracking/rules/lib/<name>.yaml or .opentaint/tracking/approximations/<name>.yaml
• Test project <test-project> — sources. Default: .opentaint/test-projects/<name> (a rule project holds a sinks/ and/or sources/ sub-project under it)
• Compiled output <test-compiled> — the model. Default: .opentaint/test-compiled/<name> (one model per sub-project: <name>/sinks, <name>/sources)
• Dependencies — exact Maven coordinates the samples need; default: the dependencies list in <tracking-file>; with no tracking file, derive them from the project's build.gradle/pom.xml

<name> is the package (<package-kebab>) for a rule, or the dataflow approximation unit (<package-kebab>-dataflow, e.g. reactor-core-publisher-dataflow) for an approximation; the two never share a folder

Workflow

1. Init the project

Pick the scaffold by shape, then pass each coordinate from the tracking file's dependencies as a --dependency:

• a rule → test rule init — scaffolds a sinks/ and a sources/ sub-project under <test-project>, each with Taint.java (the generic source()/sink()) and the generic marker lib rules in its test-rules/. Pass --sinks-only / --sources-only for a package with only one side, so you get a single sub-project
• a dataflow approximation → test approximation init (Gradle build + the test-util jar, plus Taint.java and the fixed approximation-rule.yaml the harness applies)

# rule test projects — both sides (this package has new sinks and new sources)
opentaint test rule init <test-project> \
  --dependency "org.springframework:spring-webflux:6.1.0"
# sink-only package
opentaint test rule init <test-project> --sinks-only \
  --dependency "org.mybatis:mybatis:3.5.13"

# dataflow approximation test project
opentaint test approximation init <test-project> \
  --dependency "io.projectreactor:reactor-core:3.8.5"

2. Read the real signatures, then write samples

The requirements name sources and sinks. For each new source and new sink, read its real method signature from the package jar in .opentaint/project/dependencies (with javap) — the pattern matches on that, so a sample built on the wrong signature compiles but verifies nothing. The flow is minimal, not the app's real path, and the counterpart is always the generic Taint marker (so types always fit — never a real source/sink):

• a sink sample (in the sinks/ sub-project): assign test.Taint.source() to a local of the sink argument's type, then pass it in — String t = test.Taint.source(); pkg.theSink(t); (the generic source() infers the type, no cast)
• a source sample (in the sources/ sub-project): call the new source, then pass its value into test.Taint.sink(...) — var v = pkg.theSource(); test.Taint.sink(v); (sink takes Object, so any type fits)

Write Java samples under <test-project>/<sinks|sources>/src/main/java/test/, each annotated with its expected verdict — @PositiveRuleSample (must flag) or @NegativeRuleSample (must not). value/id point at that sub-project's test join, which create-rule writes: value = "java/security/<name>-sinks.yaml", id = "<name>-sinks" for sink samples, <name>-sources for source samples (<name> = the package-kebab). value is the rule path relative to the test-rules root, id the short id — not the full --rule-id used by opentaint scan. One expected verdict per sample

Load and follow references/rule.md (for a rule) or references/approximation.md (for a dataflow approximation)

3. Compile

Compile each project to its own model — a rule's sinks/ and sources/ sub-projects separately; an approximation's single project once:

# rule
opentaint compile <test-project>/sinks   -o <test-compiled>/sinks
opentaint compile <test-project>/sources -o <test-compiled>/sources
# approximation
opentaint compile <test-project> -o <test-compiled>

A clean compile is the deliverable. If one won't build, fix that project's samples or dependencies before handing off

Output

• The compiled model(s) (<test-compiled>, per sub-project for a rule) plus their sources (<test-project>); report the paths and the exact compile command(s) used
• The tracking file's test_project stage marked done (see Tracking)

Tracking

In <tracking-file>, set only the test-project stage (in_progress while building, done once it compiles):

stages:
  test_project: done

Do not touch other stages or fields

Gotchas

• One expected verdict per sample
• One unit per <name> folder — never write into another unit's project, so concurrent agents don't race
• The scaffold (test rule init / test approximation init) defaults to Java 8 — bump source/targetCompatibility when the samples use a library needing Java 17/21 (Spring 7, spring-data 4, Lucene 10, Jackson 3). Set release on the running JDK; a Gradle toolchain{} block fails here (only JDK 21 is locatable, with no download repo)
• A positive must route the marker source() into the sink — a sink whose only untrusted input is a bare method parameter with no in-sample source (e.g. getValue(Expression e)) can't be satisfied by any taint-flow join; feed the parameter from test.Taint.source() or the sample is unprovable
• For library-method behavior the requirements don't pin down (does it sanitize? propagate taint?), read the dependency or its docs rather than guessing

Dataflow approximation test project

How it tests

opentaint test approximation run applies one fixed source → sink rule automatically — you do not author or pass a rule. That rule matches a fixed pair, test.Taint.source() and test.Taint.sink(...), provided by the Taint helper scaffolded into the project. Your samples route taint from Taint.source() through the method being approximated into Taint.sink(...). Granularity is per sample (className#methodName), so the one fixed rule covers every sample — a broken approximation only flips its own sample

opentaint test approximation init <dir> scaffolds the Gradle build, Taint.java, and the approximation-rule.yaml reference — you add only the samples (under src/main/java/test/)

Positive sample

Put samples under src/main/java/test/, each a public method annotated with the fixed rule. A positive sends Taint.source() through the approximated method into Taint.sink(...); it stays a falseNegative until the approximation propagates the taint, then flips to success. One positive per method being approximated

package test;

import org.opentaint.sast.test.util.PositiveRuleSample;

import java.util.HashMap;
import java.util.Map;

public class ApproximationSamples {

    @PositiveRuleSample(value = "approximation-rule.yaml", id = "approximation-rule")
    public void taintReachesSink() {
        String tainted = Taint.source();
        Map<String, String> cache = new HashMap<>();
        String routed = cache.computeIfAbsent(tainted, k -> k);   // the approximated method
        Taint.sink(routed);
    }
}

Negative sample — only for shared state

Add a @NegativeRuleSample only when the method holds state that taint must not cross — a container, cache, registry, or builder where you store under one key/field and read from another. Write a negative that stores tainted data under one variable and reads a different one; with a correct model the read stays clean, so the sample must not fire. For plain propagation (argument → result, or a value through a callback) the positive alone proves the model — skip the negative

    @NegativeRuleSample(value = "approximation-rule.yaml", id = "approximation-rule")
    public void taintDoesNotCrossKeys() {
        Map<String, String> cache = new HashMap<>();
        cache.put("k1", Taint.source());   // taint stored under one key
        Taint.sink(cache.get("k2"));       // a different key — must stay clean
    }

A negative that fires (falsePositive in test-result.json) means the model is over-broad — it taints a read it shouldn't. Narrow the approximation until the negative stays non-firing while the positive still passes

Notes

• value/id always reference the fixed rule: approximation-rule.yaml / approximation-rule. test approximation run applies its own bundled copy, so the project's approximation-rule.yaml is only a reference — what matters is that samples call test.Taint.source() / test.Taint.sink(...)
• the sample's receiver type fixes the dropped method's fully-qualified name, and the approximation must @Approximate that exact class — so mirror the real call's receiver type. An interface-typed receiver (Map<String,String> m, e.g. a method parameter) drops java.util.Map#computeIfAbsent; a concrete Map<String,String> cache = new HashMap<>() drops java.util.HashMap#computeIfAbsent. The new HashMap<>() form above is just one case — match whichever the real flow uses

Rule test project

Samples

The fixed counterpart is always the generic Taint marker (scaffolded by test rule init), never a real source/sink — so types fit cast-free and the sample only exercises the rule under test.

• @PositiveRuleSample — a minimal flow that must flag, with real sink/source signatures and no extra hops:
• sink under test → <Type> t = test.Taint.source(); pkg.theSink(t); — declare the local as the sink argument's type; the generic source() infers it, no cast
• source under test → var v = pkg.theSource(); test.Taint.sink(v); — sink takes Object, so any type fits

One positive per new sink (in sinks/) and per new source (in sources/); value/id point at that sub-project's test join (<name>-sinks / <name>-sources, <name> = the package-kebab)

• @NegativeRuleSample — the safe (sanitized or parameterized) variant of the same, which must not flag. Keep it realistic, not stripped to constants

package test;

import org.opentaint.sast.test.util.PositiveRuleSample;
import org.opentaint.sast.test.util.NegativeRuleSample;
import java.sql.Connection;
import java.sql.Statement;

// sinks/ sub-project — a SQL sink fed by the generic marker source
public class SqlSinkTest {
    private Connection db;

    @PositiveRuleSample(value = "java/security/jdbc-sinks.yaml", id = "jdbc-sinks")
    public void vulnerable() throws Exception {
        String input = test.Taint.source();          // generic marker: infers String, no cast
        Statement stmt = db.createStatement();
        stmt.executeQuery("SELECT * FROM users WHERE id = " + input);
    }

    @NegativeRuleSample(value = "java/security/jdbc-sinks.yaml", id = "jdbc-sinks")
    public void safe() throws Exception {
        String input = test.Taint.source();
        var pstmt = db.prepareStatement("SELECT * FROM users WHERE id = ?");
        pstmt.setString(1, input);
        pstmt.executeQuery();
    }
}

Spring-entry flows

If the flow only fires through a Spring entry point (controller → bean → sink), a plain method sample will be a falseNegative. Use the multi-module Spring layout — read spring-multimodule.md and follow it

Spring multi-module test projects

Load this when a plain method-level sample returns falseNegative because the flow only fires through a Spring entry point (controller → bean → sink). Some rules only trigger inside a full Spring MVC entry-point graph — a @PositiveRuleSample on a bare method won't trigger them, because the tainted data must flow from a discovered @Controller.

For these rules, create one dedicated Gradle sub-project per sample. Each sub-project is a complete, minimal Spring application containing exactly one @PositiveRuleSample or @NegativeRuleSample. Split positive and negative cases into separate sub-projects, e.g. xss-spring-test-positive and xss-spring-test-negative.

How detection works

TestProjectAnalyzer computes a testSetName per module as module.moduleSourceRoot.relativeTo(project.sourceRoot), with / replaced by - (see core/src/main/kotlin/org/opentaint/jvm/sast/project/TestProjectAnalyzer.kt). If the name starts with spring-app-tests, the module is treated as a Spring test set:

• All sample annotations in the module are collected as usual
• Each sample is wrapped in a SpringTestSample that uses the Spring dispatcher method as the analysis entry point instead of the annotated method itself
• Taint therefore originates from real @Controller request parameters and must reach the annotated sink method through normal Spring wiring

Consequence: the annotated method is only a marker for which rule to run and the expected verdict. The actual vulnerable/safe flow must be reachable from a controller in the same module. Keep each module to a single annotation so the verdict is unambiguous.

Project layout

Multi-module Gradle build where every spring-app-tests/<name> directory is its own sub-project:

<test-project>/
├── settings.gradle.kts
├── build.gradle.kts
└── spring-app-tests/
    ├── xss-spring-test-positive/
    │   ├── build.gradle.kts
    │   └── src/main/java/test/
    │       ├── VulnerableController.java    // @Controller with the tainted flow
    │       └── VulnerableSink.java          // carries the single @PositiveRuleSample
    └── xss-spring-test-negative/
        ├── build.gradle.kts
        └── src/main/java/test/
            ├── SafeController.java
            └── SafeSink.java                // carries the single @NegativeRuleSample

settings.gradle.kts should auto-discover every spring-app-tests/*/build.gradle.kts so adding a case only needs a new directory. See rules/test/settings.gradle.kts in the OpenTaint repo for a reference implementation.

Required dependencies

Each Spring sub-project needs at least:

• compileOnly on opentaint-sast-test-util (the sample annotations)
• org.springframework:spring-webmvc and spring-context (so @Controller is recognized)
• Any libraries the sample itself uses (servlet-api, JDBC, etc.)

Compile

opentaint compile <test-project> -o <test-compiled>

Each spring-app-tests/<name> sub-project becomes an independent test set and appears as its own entry in test-result.json.

Common pitfalls

• No @Controller in the module → TestProjectAnalyzer logs No spring entry point found and the sample is analyzed without Spring context, usually a false negative. Always include a controller that reaches the sink
• More than one annotation per module → results become ambiguous; keep it to one sample per sub-project
• Module path not starting with spring-app-tests → isSpringAppTestSet() returns false and the sample runs as a regular method-level test, so Spring flows won't trigger