paper-reviewer

Category: Testing & Security | Uploader: Ingar30 | Downloads: 0 | Version: v1.0（Latest）
Use this skill when the task is to review an academic paper PDF from the repo input folder, run specialized auditors, and compile a final report.
Changelog: Source: GitHub https://github.com/Ingar30/reviewer
Directory Structure

Current level: tree/main/.agents/skills/paper-reviewer/
📁 agents/
- 📄 openai.yaml 156 B
📄 SKILL.md 6.6 KB
SKILL.md

---
name: paper-reviewer
description: Use this skill when the task is to review an academic paper PDF from the repo input folder, run specialized auditors, and compile a final report.
---

# Paper Reviewer Skill

This skill runs a reproducible multi-agent paper-review workflow for academic PDFs.

## Use this skill when
- the input is an academic paper PDF
- the task is review, auditing, or verification
- the user wants a structured report
- the task involves literature claims, references, numeric checks, or internal cross-references

## Do not use this skill when
- the user wants only a summary
- the user wants only proofreading
- the user wants only a rewrite
- parsed artifacts already exist and the request is unrelated to the review pipeline

## Default input convention
- If the user names a bare PDF filename, first look for it under `inputs/`.
- If the user gives a repo-relative path, use it.
- If the user gives an absolute path, use it as provided.

## Workflow
For fresh runs, use `scripts/review_paper.py` as the primary entry point.

The pipeline stages are:
1. Resolve the input PDF path.
2. Derive `paper_id` from the filename stem unless explicitly provided.
3. Preprocess the PDF into `work/<paper_id>/parsed/`.
4. Render run-specific prompts into `work/<paper_id>/prompts/`.
5. Launch preflight reviewers from `config/reviewers.json`.
6. Validate preflight JSON and stop on blocking parser-quality failures.
7. If `--parser-repair plan` is enabled and parser-quality preflight reports high- or medium-severity parser artifacts, run `scripts/run_parser_repair_agent.py` and write `work/<paper_id>/repair/parser_repair_notes.md`.
8. In dynamic mode, run the reviewer selector and write `work/<paper_id>/selection/reviewer_selection.json`.
9. Write the active run roster to `work/<paper_id>/selection/selected_reviewers.json`.
10. Rerender prompts using the selected reviewer roster and parser repair notes when present.
11. Launch mandatory review-stage reviewers and selected optional reviewers.
12. Validate each reviewer JSON output under `work/<paper_id>/reviews/`.
13. Normalize and deduplicate reviewer outputs into `work/<paper_id>/editor/normalized_bundle.json`.
14. Build editor input at `work/<paper_id>/editor/editor_input.md`.
15. Run the editor to write `outputs/<paper_id>/report.md`.
16. Smoke-check the final report with `scripts/check_final_report.py --bundle work/<paper_id>/editor/normalized_bundle.json`.

Use `--reviewer-selection static` only when all enabled review-stage reviewers should run.
Use `--parser-repair plan` only when parser-quality issues should be converted into a reviewer-facing repair overlay before substantive reviewers run.

## Editor-only refresh
If parsed artifacts, reviewer JSON files, `work/<paper_id>/selection/selected_reviewers.json`, and `work/<paper_id>/editor/normalized_bundle.json` already exist, rerun only the editor when the change is limited to editor prompt/report presentation:
1. Rerender prompts with `scripts/render_prompts.py`, passing `--reviewers-config work/<paper_id>/selection/selected_reviewers.json`.
2. Rebuild editor input with `scripts/build_editor_input.py`, passing the same selected reviewer config.
3. Run the editor with `codex exec --output-last-message outputs/<paper_id>/report.md -`.
4. Smoke-check with `scripts/check_final_report.py --input outputs/<paper_id>/report.md --bundle work/<paper_id>/editor/normalized_bundle.json`.

Do not use editor-only refresh when reviewer evidence, parser artifacts, reviewer selection, or normalized findings need to change.

Use editor-only refresh to test narrowly scoped editor prompt changes against the same evidence bundle before changing the full workflow. This is especially useful for checking whether report emphasis improved without changing reviewer evidence, such as when adjusting how parser/preprocessing caveats are surfaced in prose.

## Critical rules
- Internal reviewers return JSON only.
- The editor is the only component that emits final markdown.
- Literature and reference verification require web search when enabled.
- Never guess missing evidence; use `cannot_verify`.
- Preserve exact source locations whenever possible.
- If parsed artifacts are poor, fix preprocessing before trusting reviewer outputs.
- Treat parser-quality preflight warnings as reportable caveats; treat high-confidence blocking parser findings as a reason to stop before substantive review.
- Treat parser repair notes as routing guidance to existing safer artifacts, not as evidence that OCR, tables, figures, or page ordering were actually regenerated.
- Keep final-report traceability in the traceability appendix. Do not reintroduce repeated traceability footers in the body.
- Literature and novelty critiques must be grounded in concrete studies or marked `cannot_verify`; do not assert lack of novelty from vague prior-work impressions.
- If the final report cites external studies, registry records, web pages, or other external evidence, include the external-sources appendix using only source details already present in reviewer evidence.
- Do not bury parser/preprocessing issues when they materially distort auditability of a central formula, table, figure, citation target, or quantitative claim. Keep them in the parser-caveats section, but mention them explicitly in prose and treat them as revision-priority material when the auditability risk is substantial.
- Treat `scripts/check_final_report.py` as a structure and traceability smoke check, not as independent verification that external sources are real or current.
- Treat institutional context, power/multiple testing, design/randomization, and economic magnitude reviewers as narrow optional pilots. They should run only when the selected paper has strong cues for those risks.
- If `codex exec --output-last-message` writes only a short acknowledgement for the editor, rely on the wrapper's recovery from the editor transcript and then rerun the final report checker.

## Output conventions
- Parsed artifacts: `work/<paper_id>/parsed/`
- Parser repair overlays: `work/<paper_id>/repair/`
- Reviewer selection: `work/<paper_id>/selection/`
- Reviewer outputs: `work/<paper_id>/reviews/`
- Final report: `outputs/<paper_id>/report.md`

The expected final report shape is synthesis-first: executive summary, prose review configuration, roughly 3 to 8 high-confidence highest-priority findings when supported by the evidence, suggested revision priorities, additional findings, domain-specific sections, grammar appendix when needed, external-sources appendix when external evidence is cited, parser-caveat prose that keeps central auditability failures visible, and traceability map appendix.
Comments 0
Latest | Hot
Please login before commenting.
Loading comments...
paper-reviewer

Directory Structure

SKILL.md

Report

Notice
