duplicate-code-detector

分类: 数据与AI | 上传者: pvillega | 下载: 0 | 版本: v1.0（最新）

当用户提及重复代码、复制粘贴、代码克隆、重复代码、跨文件的相似代码、重叠实现、代码质量审计、代码异味、涉及重复或重构准备的技术债务时使用。还适用于跨语言重复检查、CI 重复警告和代码库冗余扫描。不要触发实施重构（使用增量重构）、修复错误、编写测试、添加功能或代码审查。 --- # 重复代码检测器 jscpd 对每个文件对执行令牌级别比较，捕获手动检查不可见的重复项。该技能运行该分析并将输出转化为分类的、优先考虑的重构计划。 **所需的后续技能：** 使用“增量重构”来实现检测后的重构。如果用户说“重构技术债务”或“清理此代码库”，请首先从这里开始寻找目标。 ```dot digraph detector_flow { "检查项目配置" -> "jscpd 可用吗？" [形状=钻石]； “jscpd 可用吗？” ->“运行jscpd”[label=“yes”]; “jscpd 可用吗？” ->“Grep 后备”[label="no"]; “运行 jscpd”->“提取并限制结果 (jq)”; “Grep后备”->“提取并限制结果（jq）”； “提取并限制结果（jq）”->“分类（精确/接近/结构）”； “分类（精确/接近/结构）”->“按影响分数排名”； “按影响分数排名”->“当前发现→增量重构”； } ``` --- ## 快速参考 |克隆类型|描述 |重构模式| |---|---|---| | **准确** |逐字节相同 |提取函数（无参数）| | **附近** |名称/文字/次要表达式不同 |参数化（使用参数提取不同部分）| | **结构** |相同的算法模式，不同的实现 |模板方法/策略| **排名：** `impact_score = dup licated_lines x 个实例`。决胜局：精确>接近>结构。 --- ## 工作流程 ### 步骤 1：检查环境和配置 ```bash ls .jscpd.json .jscpdrc .jscpdrc.json 2>/dev/

更新日志: Source: GitHub https://github.com/pvillega/claude-templates

目录结构

当前层级: tree/main/plugins/ct/skills/duplicate-code-detector/

📁 evals/
- 📄 evals.json 3.8 KB
📄 SKILL.md 8.7 KB

SKILL.md

---
name: duplicate-code-detector
description: >
  Use when the user mentions duplicated code, copy-paste, code clones, repeated code, similar code across files, overlapping implementations, code quality audit, code smells, technical debt involving duplication, or refactoring preparation. Also for cross-language duplication checks, CI duplication warnings, and codebase redundancy scans. DO NOT TRIGGER for implementing refactoring (use incremental-refactoring), fixing bugs, writing tests, adding features, or code review.
---

# Duplicate Code Detector

jscpd performs token-level comparison across every file pair, catching duplicates invisible to manual review. This skill runs that analysis and turns output into a classified, prioritized refactoring plan.

**REQUIRED FOLLOW-UP SKILL:** Use `incremental-refactoring` to implement the refactoring after detection.

If the user says "refactor technical debt" or "clean up this codebase", start here to find targets first.

```dot
digraph detection_flow {
  "Check project config" -> "jscpd available?" [shape=diamond];
  "jscpd available?" -> "Run jscpd" [label="yes"];
  "jscpd available?" -> "Grep fallback" [label="no"];
  "Run jscpd" -> "Extract & cap results (jq)";
  "Grep fallback" -> "Extract & cap results (jq)";
  "Extract & cap results (jq)" -> "Classify (exact/near/structural)";
  "Classify (exact/near/structural)" -> "Rank by impact score";
  "Rank by impact score" -> "Present findings → incremental-refactoring";
}
```

---

## Quick Reference

| Clone Type | Description | Refactoring Pattern |
|---|---|---|
| **Exact** | Byte-for-byte identical | Extract Function (no params) |
| **Near** | Differs in names/literals/minor expressions | Parameterize (extract with args for varying parts) |
| **Structural** | Same algorithm pattern, different implementations | Template Method / Strategy |

**Ranking:** `impact_score = duplicated_lines x instances`. Tiebreaker: exact > near > structural.

---

## Workflow

### Step 1: Check environment and config

```bash
ls .jscpd.json .jscpdrc .jscpdrc.json 2>/dev/null  # Check project config
which jscpd || npm install -g jscpd                  # Check/install jscpd
```

If a config file exists, **respect it** — run jscpd with no overriding flags so the project's thresholds and ignore patterns take effect.

If jscpd/npm unavailable, use the **Grep Fallback** section below. Tell the user: grep finds exact duplicates only.

### Step 2: Run jscpd

**With project config** (minimal flags, let config drive):
```bash
jscpd --reporters json --output /tmp/jscpd-report --gitignore /path/to/code
```

**Without project config** (sensible defaults):
```bash
jscpd --min-lines 10 --min-tokens 50 \
  --ignore "node_modules,dist,build,vendor,.git,__pycache__,*.min.js,coverage,tmp,generated" \
  --reporters json --output /tmp/jscpd-report --gitignore /path/to/code
```

**Always use:** `--gitignore` (respect .gitignore), `--output /tmp/jscpd-report` (avoid flooding stdout), `--reporters json` (not console for large codebases).

Tuning: `--min-lines` 5 (small) to 15-20 (large/verbose). `--format "javascript,typescript"` to scope languages.

### Step 3: Extract and cap results

**Token budget management** — use `jq` to avoid loading raw JSON into context:

```bash
# Summary metrics
jq '{percentage: .statistics.total.percentage, duplicatedLines: .statistics.total.duplicatedLines, clones: (.clones | length)}' /tmp/jscpd-report/jscpd-report.json

# All groups above threshold, sorted by impact, fragments capped at 500 chars
# Default: include groups with impact >= 20 (e.g., 10 lines x 2 instances)
# Adjust threshold based on codebase size — lower for small, higher for large
jq '[.clones | group_by(.fragment) | map({fragment: .[0].fragment[0:500], lines: (.[0].duplicationA.end.line - .[0].duplicationA.start.line), instances: length, impact: ((.[0].duplicationA.end.line - .[0].duplicationA.start.line) * length), files: [.[] | .duplicationA.sourceId, .duplicationB.sourceId] | unique}) | sort_by(-.impact) | map(select(.impact >= 20))]' /tmp/jscpd-report/jscpd-report.json
```

**Filtering:** Include all groups above an impact threshold (default: impact >= 20) rather than a hard top-N cap. This surfaces all meaningful duplicates in one pass. Fragments truncated to 500 chars. If the result set is still too large for context, raise the threshold or batch into pages of 10. Present summary metrics first.

**Inline exclusions:** If expected duplicates are missing, check for `jscpd:ignore-start` / `jscpd:ignore-end` markers. Mention these to the user.

### Step 4: Classify and analyze

Dispatch subagents in parallel for all groups above the impact threshold:

```
Analyze this duplicate group:
- Source A: <file_a> lines <X-Y>
- Source B: <file_b> lines <M-N>

1. Classify: Exact / Near / Structural (see Quick Reference)
2. Match refactoring pattern to classification
3. Note differences between instances
4. Estimate impact: lines saved, files touched
5. Flag risks: same-looking code with different side effects, or coincidental structural similarity (should NOT be unified)
```

### Step 5: Generate TDD refactoring plan

For each priority item:

```markdown
## Priority 1: [Name] (X lines, Y instances)
**Type:** Exact / Near / Structural
**Pattern:** Extract Function / Parameterize / Template Method
**Impact score:** X | **Files:** list

1. Write test capturing current behavior of one instance
2. Extract shared code with parameters for variation points
3. Replace each instance, run tests after each
4. Remove dead code
```

### Step 6: Present findings

```
## Duplicate Code Analysis
**Metrics:** X% duplication (Y lines, Z clone groups)
**Config:** [project .jscpd.json / defaults] | **Scanned:** [path] (respecting .gitignore)

**Top priorities:**
1. [Name] — Exact, X lines, Y instances (impact: Z) → Extract Function
2. [Name] — Near, X lines, Y instances (impact: Z) → Parameterize

Want to start refactoring Priority 1? (I'll hand off to incremental-refactoring)
```

---

## Grep Fallback (when jscpd unavailable)

**Tell the user upfront:** grep finds exact text duplicates only — no near or structural detection.

Use **ripgrep (`rg`)** if available (respects `.gitignore` by default, handles nested `.gitignore` files). Use `--type` to scope languages (e.g., `--type py --type go`).

**With ripgrep:**
```bash
# Find most-repeated non-trivial lines, with file locations
rg -n --no-heading --type py --type go '.' /path/to/code \
  | awk -F: '{line=$3; for(i=4;i<=NF;i++) line=line":"$i; if(length(line)>60) print line}' \
  | sort | uniq -c | sort -rn | head -20
# Then locate which files contain the top hit:
rg -n --no-heading 'exact duplicate line text here' /path/to/code
```

**Without ripgrep (grep -rn fallback):**
```bash
grep -rn --include='*.py' --include='*.go' \
  --exclude-dir=node_modules --exclude-dir=vendor --exclude-dir=dist --exclude-dir=coverage --exclude-dir=.git \
  '.' /path/to/code \
  | awk -F: '{line=$3; for(i=4;i<=NF;i++) line=line":"$i; if(length(line)>60) print line}' \
  | sort | uniq -c | sort -rn | head -20
```

**Processing results:**
- Cap to **top 20 lines**, investigate top 5-10 as block candidates
- For each candidate line, run a second search to find **which files** contain it
- Group adjacent repeated lines into blocks — if lines N, N+1, N+2 all appear duplicated in the same file pair, that's one multi-line block, not 3 separate duplicates
- Read only 10 lines context around each block candidate
- Classify all as **Exact**. Flag "possible Near" if lines differ by 1-2 tokens
- Impact formula still applies. Add to presentation:
```
**Detection:** grep fallback (exact only) | **Recommendation:** Install jscpd for full detection
```

---

## Common Mistakes

| Mistake | Fix |
|---|---|
| Overriding project `.jscpd.json` with hardcoded flags | Check for config first, use minimal flags if present |
| Loading full jscpd JSON into context | Use `jq` to filter by impact threshold and truncate fragments |
| Hard-capping to top N groups | Use impact threshold instead — surfaces all meaningful duplicates in one pass |
| Skipping `--gitignore` flag | Always pass it — without it, generated/coverage dirs get scanned |
| Treating all duplicates the same | Classify (exact/near/structural) — each needs different refactoring |
| Unifying coincidentally similar code | Structural clones with different domains should often stay separate |

## Before finishing

1. jscpd ran (or fallback used) with metrics extracted
2. Project config respected if present
3. `.gitignore` respected via `--gitignore`
4. Results filtered by impact threshold (not hard-capped), fragments truncated to 500 chars
5. Each duplicate classified with matching refactoring pattern
6. Priorities ranked by impact score
7. User told about `jscpd:ignore` markers if relevant

登录后下载/点赞/收藏 ❤ 13 | ★ 0

duplicate-code-detector

目录结构

SKILL.md

举报内容

提示