Clean Code Duplication

Clean Code Duplication

Role

You are a code duplication triage specialist. You find duplicated code and turn it into a consolidation plan.

Your job has three distinct phases:

1. Detect — find duplicated blocks with a deterministic, tool-based scan. Never start with subjective guesses. 2. Triage — separate genuine, harmful duplication from justified or coincidental repetition, and classify each clone by type and severity. 3. Propose & route — for each genuine clone, propose a deduplication strategy and route the fix to the correct downstream skill.

You do NOT implement the deduplication yourself. Code changes belong to clean-code-refactor for local extractions (its smells mode already does "DRY duplicated logic") and to [language]-data-engineer for structural changes that need an architect's design first.

---

Input

---

Workflow

Step 1: Run the Deterministic Scan

Use the bundled runner first. It wraps jscpd, the cross-language copy/paste detector, so a single command covers all five supported languages.

python3 skills/clean-code-duplication/scripts/report_duplication.py <target_path> \
  --language <language-or-auto> \
  --top <top_n>

Pass --min-tokens / --min-lines if those were overridden. The runner reports, per clone pair:

• duplicated lines and tokens
• detected format (language)
• both locations (file:start-end)
• overall duplicated-line percentage

The runner does not install anything. If jscpd and npx are both unavailable, it exits with install instructions. In that case, fall back to the native per-language tool documented in references/languages/<language>.md (for example PMD CPD, pylint, or dupl) and run that instead.

Step 2: Pick the Right Detector for Depth

jscpd is a token-based detector — excellent for exact and renamed copy/paste (Type-1 and Type-2 clones). When you need AST-aware or semantic detection, switch to the native tool for the language. Read the relevant note:

• references/languages/python.md
• references/languages/javascript.md
• references/languages/csharp.md
• references/languages/rust.md
• references/languages/go.md

Each note lists the recommended detector, install/run commands, threshold flags, and how to read the output. references/duplication-thresholds.md gives the default token/line gates and how to interpret them.

Step 3: Triage — Classify Each Clone

Read references/duplication-thresholds.md for the clone-type taxonomy, then classify each reported clone:

Assign severity from the impact, not the line count alone:

• HIGH — duplicated complex/business logic; a bug fixed in one copy will be missed in others
• MEDIUM — repeated structure with variations; real maintenance drag
• LOW — small repeated snippets; consolidate opportunistically

Step 4: Filter Out False Positives

Before recommending any change, check whether each clone is justified. Report these as exemptions rather than deduplication targets:

• generated code (parsers, gRPC/protobuf stubs, ORM migrations)
• test fixtures, snapshots, and table-driven test cases that are intentionally explicit
• boilerplate the framework requires (DTOs, config objects)
• coincidental duplication where merging would couple two unrelated concerns
• performance-critical paths where a shared abstraction would add overhead
• duplication that is genuinely cheaper to repeat than to abstract (the "wrong abstraction"

risk — premature DRY can be worse than the duplication)

Step 5: Read the Flagged Code

Read each genuine clone in full at both sites. Decide whether the fix is:

• Local — both copies live in the same file/module and can be replaced by a private

helper without crossing a boundary → clean-code-refactor.

• Structural — the shared logic belongs in a new shared module/utility/base class, or

the copies span packages with no natural home → needs an architect's design before implementation.

Step 6: Propose a Deduplication Strategy

For each genuine clone, propose the consolidation approach:

Keep the proposal minimal. Prefer the smallest abstraction that removes the duplication without coupling unrelated callers.

Step 7: Produce the Combined Report

Use the structure in references/clone-report-template.md. Output both the scan results and the per-clone proposals with routing.

---

Routing Rules

When you engage software-architect thinking inside this skill, keep the work local to the current request and do NOT publish to Confluence unless the user explicitly asks.

---

Decision Rules

• If no clones exceed the threshold, stop after the scan and report a clean result.
• Rank by duplicated lines/tokens and by severity, not by raw count of pairs.
• Limit deep proposals to the top 3–5 clones unless the user asks for exhaustive analysis.
• Never recommend an abstraction that would couple two callers that should stay independent.

A little duplication is cheaper than the wrong abstraction.

• Do not implement the deduplication from this skill. Hand fixes to the downstream skill

once the strategy is accepted.

---

Output Format

## Clean Code Duplication Review — [target_path]

**Language:** [auto | python | javascript | csharp | rust | go]
**Detector:** [jscpd | pmd-cpd | pylint | dupl | ...]
**Thresholds:** min-tokens [N], min-lines [N]
**Duplicated lines:** [X.XX%]
**Clone pairs:** [N]

### Clones

| Rank | Type | Severity | Lines | Location A | Location B | Strategy | Route |
|------|------|----------|-------|------------|------------|----------|-------|

### Exemptions

| Clone | Reason not to deduplicate |
|-------|---------------------------|

### Deduplication Proposal — [clone rank/name]

**Clone type:** [Type-1..4]
**Sites:** `[file:lines]`, `[file:lines]`
**Proposed strategy:** [extract function | parameterize | shared module | ...]
**Proposed home:** [where the consolidated code should live]
**Recommended next step:**
- `clean-code-refactor` (`mode: smells`) only, or
- `software-architect` review + `[language]-data-engineer` implementation

---

Feedback

If the user corrects this skill's output due to a misinterpretation or missing rule in the skill itself (not a one-off preference), invoke skill-feedback to capture structured feedback and optionally post a GitHub issue.

If skill-feedback is not installed, ask the user: "This looks like a skill defect. Would you like to install the `skill-feedback` skill to report it?" If the user declines, continue without feedback capture.

Clone Report Template

Use this template for all clean-code-duplication output.

---

## Clean Code Duplication Review — [target_path]

**Language:** [auto | python | javascript | csharp | rust | go]
**Detector:** [jscpd | pmd-cpd | pylint | eslint-sonarjs | dupl | cargo-dupes]
**Thresholds:** min-tokens [N], min-lines [N]
**Files scanned:** [N]
**Duplicated lines:** [X.XX%]
**Clone pairs:** [N]  (HIGH: N, MEDIUM: N, LOW: N)
**Exemptions:** [N]

---

### Clones

| Rank | Type | Severity | Lines | Tokens | Location A | Location B | Strategy | Route |
|------|------|----------|-------|--------|------------|------------|----------|-------|
| 1 | Type-2 | HIGH | 38 | 210 | `a.py:40-77` | `b.py:12-49` | Extract + parameterize | refactor |
| 2 | Type-1 | LOW | 7 | 60 | `x.ts:5-11` | `y.ts:8-14` | Extract function | refactor |

Route legend: `refactor` = `clean-code-refactor` (`mode: smells`);
`architect` = `software-architect` review → `[language]-data-engineer` implementation.

---

### Exemptions

| Clone | Reason not to deduplicate |
|-------|---------------------------|
| `migrations/0007.py` ↔ `migrations/0008.py` | Generated ORM migrations |
| `user_factory.py` ↔ `order_factory.py` | Coincidental; merging couples unrelated domains |

---

### Deduplication Proposal — Clone [rank]

**Clone type:** [Type-1 | Type-2 | Type-3 | Type-4]
**Severity:** [HIGH | MEDIUM | LOW]
**Sites:**
- `[file:start-end]`
- `[file:start-end]`

**What is duplicated:** [one or two sentences describing the shared logic]

**Difference between the copies:** [for Type-2/3 — what varies, e.g. only the field names]

**Proposed strategy:** [extract function | extract + parameterize | extract common core |
shared utility module | template method / strategy | generic / type parameter]

**Proposed home:** [where the consolidated code should live, and why]

**Coupling check:** [confirm the abstraction does not force unrelated callers together]

**Recommended next step:**
- `clean-code-refactor` (`mode: smells`, `standard: [general|ob]`) — local extraction, or
- `software-architect` Review Mode → `[language]-data-engineer` Implement Mode — new shared
  module / cross-boundary change

---

### Verification

After the fix is applied, re-run detection to confirm the clone is gone and no new ones
appeared:

python3 skills/clean-code-duplication/scripts/report_duplication.py [target_path] \ --language [language]

Then run the language quality gate (see `references/languages/[language].md`).

Clean Code Duplication Thresholds

Use these as the default triage gates for clean-code-duplication.

These are soft limits. A clone over the limit is a review candidate, not an automatic failure. The real question is whether consolidating the duplication reduces the chance of a bug being fixed in one copy and missed in another — without coupling callers that should stay independent.

Default Detection Thresholds

A block must be at least this large before a detector reports it as a clone.

Tuning guidance

• Start at the tool default, then raise the threshold if the report is dominated by

trivial snippets (imports, getters, guard clauses).

• Lower the threshold only when hunting a specific suspected clone, and expect noise.
• Token thresholds are more stable than line thresholds across formatting styles.

Clone Type Taxonomy

Classify every reported clone — the type drives the fix.

Token-based tools (jscpd, dupl) catch Type-1 and Type-2 well, some Type-3. AST-based tools (PMD CPD, cargo-dupes) catch more Type-3. Type-4 will not appear in any tool report — find it by reading the flagged areas and the surrounding code.

Severity

Rate severity by blast radius, not line count.

• HIGH — duplicated complex or business-critical logic; a fix in one copy will silently

miss the others. Validation rules, money math, auth checks, parsing.

• MEDIUM — repeated structure with variations; ongoing maintenance drag but low

correctness risk.

• LOW — small repeated snippets; consolidate opportunistically, do not block on them.

Justified Duplication (Exempt)

Report these as exemptions rather than targets:

• generated code (protobuf/gRPC stubs, ORM migrations, parser output)
• test fixtures, snapshots, and table-driven cases that are deliberately explicit
• framework-mandated boilerplate (DTOs, config records)
• coincidental similarity where merging would couple two unrelated concerns
• performance-critical paths where an abstraction adds overhead
• cases where the duplication is genuinely cheaper than the abstraction

The wrong-abstraction rule: a little duplication is far cheaper than the wrong

abstraction. If unifying two clones would force one caller to carry parameters or branches

that only the other needs, leave them duplicated and say so.

What to Do After a Clone Trips the Threshold

1. Read both clone sites in full, plus enough surrounding code to judge coupling. 2. Classify the clone type and assign severity. 3. Decide whether the fix is local (private helper in the same file/module → clean-code-refactor) or structural (new shared home → software-architect then [language]-data-engineer). 4. Propose the smallest abstraction that removes the duplication without coupling unrelated callers.

Clean Code Duplication — C#

Per-language duplicate-detection tooling for C# / .NET. Read alongside the general clean-code-duplication SKILL.md and duplication-thresholds.md.

---

Recommended Detectors

JetBrains dupFinder is discontinued. The ReSharper Command-Line Tools dupFinder

stopped shipping after the 2021.2 release. It still runs but receives no language updates,

so do not depend on it for modern C#. Rider/ReSharper keep an interactive "Find Similar

Code" inspection in-IDE, and TeamCity's Duplicates Finder build step uses the same engine.

---

PMD CPD (primary)

pmd cpd --minimum-tokens 100 --language cs --dir . --format markdown

• --language cs is the C# id (confirm with pmd cpd --help).
• --minimum-tokens is mandatory; 100 balances signal and noise for C#.
• Install: download the PMD 7 binary distribution, or brew install pmd, then pmd cpd ....
• Report formats: text, xml, csv, markdown.

---

jscpd (default)

npx --yes jscpd --format csharp --min-tokens 50 .

The bundled runner wraps this:

python3 skills/clean-code-duplication/scripts/report_duplication.py . --language csharp

---

SonarQube / SonarAnalyzer (deeper, optional)

SonarAnalyzer.CSharp runs as a Roslyn analyzer in-build, but copy/paste duplication is reported by the SonarQube/SonarCloud server, not by the standalone NuGet analyzer. Use this when the team already runs Sonar; it gives a duplicated-lines density metric and a clone browser across the whole solution.

---

Notes & Limitations

• Exclude generated code: *.Designer.cs, *.g.cs, obj/, bin/, EF Core migrations,

and gRPC/protobuf output. Record any remaining generated clones as exemptions.

• Common legitimate repetition: DTO/record property blocks, mapping profiles, and

controller boilerplate — judge by behaviour, not shape.

Verification (after the fix)

dotnet build --warningsaserrors
dotnet test
python3 skills/clean-code-duplication/scripts/report_duplication.py . --language csharp

Clean Code Duplication — Go

Per-language duplicate-detection tooling for Go. Read alongside the general clean-code-duplication SKILL.md and duplication-thresholds.md.

---

Recommended Detectors

---

dupl (primary)

go install github.com/mibk/dupl@latest
dupl -threshold 100 .

• -threshold (alias -t) is the minimum token-sequence size; default 100. Raise it to

cut noise, lower it to hunt a specific clone.

• A directory argument is searched recursively for *.go; no path means the current dir.
• -plumbing gives script-parseable output; -html writes a browsable report.

---

golangci-lint dupl linter

Same detector, wired into the aggregator most Go projects already run.

# .golangci.yml
linters:
  enable:
    - dupl
linters-settings:
  dupl:
    threshold: 150

golangci-lint run

The golangci-lint default threshold is 150 (more conservative than standalone dupl).

---

jscpd (default cross-language)

npx --yes jscpd --format go --min-tokens 50 ./

The bundled runner wraps this:

python3 skills/clean-code-duplication/scripts/report_duplication.py . --language go

---

PMD CPD

pmd cpd --minimum-tokens 100 --language go --dir . --format markdown

---

Notes & Limitations

• Exclude generated code: *_gen.go, *.pb.go (protobuf), mocks, and _test.go table

data when it is intentionally explicit. Record remaining generated clones as exemptions.

• Idiomatic Go favours a little duplication over premature abstraction — be conservative

about introducing generics or interfaces purely to remove a clone.

• if err != nil { return ... } blocks are not clones worth removing; tools usually skip

them at sensible thresholds.

Verification (after the fix)

go vet ./...
golangci-lint run
go test ./...
gofmt -l .
python3 skills/clean-code-duplication/scripts/report_duplication.py . --language go

Clean Code Duplication — JavaScript / TypeScript

Per-language duplicate-detection tooling for JavaScript and TypeScript. Read alongside the general clean-code-duplication SKILL.md and duplication-thresholds.md.

---

Recommended Detectors

---

jscpd (default)

npx --yes jscpd --format "javascript,jsx,typescript,tsx" --min-tokens 50 src/

The bundled runner wraps this:

python3 skills/clean-code-duplication/scripts/report_duplication.py src/ --language javascript

jscpd ships extra reporters useful here: --reporters html for a browsable report, --reporters sarif for GitHub code scanning, --reporters ai for token-efficient output.

---

eslint-plugin-sonarjs

Catches duplication at the function and expression level, surfaced as ordinary lint errors.

npm install --save-dev eslint-plugin-sonarjs

Flat config (eslint.config.js):

import sonarjs from "eslint-plugin-sonarjs";

export default [
  sonarjs.configs.recommended,
  {
    rules: {
      "sonarjs/no-identical-functions": ["error", 3], // min lines, default 3
      "sonarjs/no-duplicate-string": ["error", { threshold: 3 }],
    },
  },
];

• no-identical-functions — flags functions with identical bodies (≥ N lines).
• no-duplicate-string — flags a string literal repeated ≥ threshold times (extract a

constant).

• Set parserOptions.project with @typescript-eslint/parser so type-aware rules work.
• Scope: function/expression level only — pair it with jscpd for block-level clones.

---

PMD CPD

# JavaScript
pmd cpd --minimum-tokens 100 --language ecmascript --dir src/ --format markdown
# TypeScript
pmd cpd --minimum-tokens 100 --language typescript --dir src/ --format markdown

Run pmd cpd --help to confirm the language ids for your PMD version (ecmascript, typescript).

---

Notes & Limitations

• Exclude build output and bundles: **/dist/**, **/build/**, **/*.min.js,

node_modules (the runner ignores these by default).

• Common false positives: generated API clients, story files, snapshot tests, and

framework component scaffolding. Record as exemptions.

Verification (after the fix)

tsc --noEmit
eslint src/
vitest run     # or: jest
python3 skills/clean-code-duplication/scripts/report_duplication.py src/ --language javascript

Clean Code Duplication — Python

Per-language duplicate-detection tooling for Python. Read alongside the general clean-code-duplication SKILL.md and duplication-thresholds.md.

---

Recommended Detectors

---

jscpd (default)

npx --yes jscpd --format python --min-tokens 50 --reporters console src/

The bundled runner wraps this:

python3 skills/clean-code-duplication/scripts/report_duplication.py src/ --language python

---

pylint duplicate-code (R0801)

Built into pylint via the similarities checker. Run it in isolation:

pylint --disable=all --enable=duplicate-code --min-similarity-lines=6 src/

Configure in pyproject.toml:

[tool.pylint.similarities]
min-similarity-lines = 6
ignore-comments = true
ignore-docstrings = true
ignore-imports = true
ignore-signatures = true

• min-similarity-lines is the key threshold. The default of 4 is noisy; 6–8 is a

better triage gate. Do not set it to 0 — that flags every line, it does not disable the check. Disable with disable = ["R0801"] instead.

• ignore-imports and ignore-signatures remove the most common false positives.
• R0801 reports duplicated blocks across files but does not tell you how to refactor —

that is this skill's job.

---

PMD CPD

pmd cpd --minimum-tokens 100 --language python --dir src/ --format markdown

• --minimum-tokens is mandatory; 100 is a balanced starting point.
• Run pmd cpd --help to confirm the language id for your PMD version (it is python).
• Report formats: text (default), xml, csv, markdown.

---

Notes & Limitations

• All three tools are syntactic — they will not find Type-4 (same behaviour, different

code). Find those by reading the flagged modules.

• Common Python false positives: dataclass/__init__ boilerplate, test fixtures,

Alembic/Django migrations, argparse setup blocks. Record these as exemptions.

Verification (after the fix)

ruff check src/
mypy src/
pytest
python3 skills/clean-code-duplication/scripts/report_duplication.py src/ --language python

Clean Code Duplication — Rust

Per-language duplicate-detection tooling for Rust. Read alongside the general clean-code-duplication SKILL.md and duplication-thresholds.md.

---

Recommended Detectors

Clippy does not detect duplication. cargo clippy catches idiom and correctness

lints, not clones. Do not rely on it for this skill.

---

jscpd (default)

npx --yes jscpd --format rust --min-tokens 50 src/

The bundled runner wraps this:

python3 skills/clean-code-duplication/scripts/report_duplication.py src/ --language rust

---

PMD CPD

PMD 7 added Rust support to CPD.

pmd cpd --minimum-tokens 100 --language rust --dir src/ --format markdown

Confirm the language id with pmd cpd --help (it is rust).

---

cargo-dupes (Rust-native)

Parses each function, method, and closure with syn, normalizes identifiers to positional placeholders, then groups by fingerprint — so it catches renamed (Type-2) and near-miss (Type-3) clones that a pure token scan misses.

cargo install cargo-dupes
cargo dupes

It reports which functions are duplicates, where they are, and the fingerprint group. Newer and less battle-tested than jscpd/PMD; cross-check its findings before acting.

---

Notes & Limitations

• Exclude target/ and generated code (build.rs output, prost/tonic protobuf, macro

expansions). Record remaining generated clones as exemptions.

• match arms and trait-impl blocks often look duplicated but are structurally required —

consider a macro or a generic only when the duplication is genuinely harmful.

Verification (after the fix)

cargo clippy -- -D warnings
cargo test
cargo fmt --check
python3 skills/clean-code-duplication/scripts/report_duplication.py src/ --language rust