t3-hardware-scoring

Category: Tools & Productivity | Uploader: ToyWorks | Downloads: 0 | Version: v1.0（Latest）
Changelog: Source: GitHub https://github.com/ToyWorks/agent-skills
Directory Structure

Current level: skills/t3-hardware-scoring/
📁 references/
- 📄 defluff-guide.md 4.9 KB
- 📄 eagle-eye-validator.md 4.8 KB
- 📄 organize-guide.md 2.1 KB
- 📄 peer-review-guide.md 8.2 KB
- 📄 report-schema.md 4.2 KB
- 📄 t3-classification.md 8.8 KB
- 📄 tool-auditor-template.md 3.8 KB
- 📄 tool-auditor.md 12.8 KB
- 📄 toy-auditor-template.md 3.8 KB
- 📄 toy-auditor.md 13.5 KB
- 📄 trash-auditor.md 15.1 KB
- 📄 trash-red-flags.md 7.4 KB
📁 scripts/
- 📄 crawl_product_info.py 7.7 KB
- 📄 run_t3_audit.sh 1.1 KB
- 📄 synthesize_results.py 14.3 KB
📄 SKILL.md 12.2 KB
SKILL.md

---
name: t3-hardware-scoring
description: >-
  T3 Hardware Audit System. Brand Blinding, Triple-Auditor (Tool/Toy/Trash) scoring,
  Eagle Eye Validation, and Final Judge synthesis for objective hardware classification.
  Triggers: product links, T3 audit, hardware evaluation
dependency:
  python:
    - requests==2.32.5
    - beautifulsoup4==4.12.3
metadata:
  version: "2.1"
---

# MantaBase T3 Hardware Audit System — v2.1

## Objectives
- Crawl product information → Brand Blind → Triple-Auditor independent scoring → Eagle Eye Validation → Final Judge synthesis
- Output: objective Tool / Toy / Trash classification with Eagle Eye safety veto
- Core principles: evidence-first, parallel audits, zero hallucination, transparent math

## Architecture Overview (v2.1)

```
Main Agent                             Subagents
───────────────────────────────────    ──────────────────────────────
Step 1: Data Collection                
Step 2: Organize + Brand Blind         
  ↓ write 02-brand-blinded.md          
                                       Step 3: ×3 parallel (Tool/Toy/Trash)
                                         · read file paths, print JSON to stdout
                                         · main agent writes 03-*.json files
  ↓ receive JSON, write files           
  ↓ auto-build auditor_reports.json    
                                       Step 4: ×1 Eagle Eye Validator (conditional)
                                         · AI subagent, not Python script
                                         · outputs adjustments[] diff
  ↓ apply diffs, rebuild JSON           
Step 5: synthesize_results.py           
Step 6: 99-audit-report.md             
```

**Key v2.1 changes:**
- Step 4 replaces 3-way Peer Review with single AI-driven Eagle Eye Validator
- Subagents print JSON to stdout; main agent writes all files
- New Eagle Eye pattern: Architectural Implausibility
- Mandatory auto-rebuild of auditor_reports.json from 03-*.json files
- Token guardrails on verbatim_evidence and prompt size

---

## Data Collection Strategy (Step 1-2)

### Primary: crawl_product_info.py
```bash
python3 scripts/crawl_product_info.py --url https://example.com/product --pretty
```
Uses Jina AI Reader first, then direct HTTP fallback.
Exit code 2 = content insufficient → trigger Exa fallback.

### Fallback: Exa MCP (use when crawl fails or is insufficient)
```bash
mcporter call 'exa.web_search_exa(query: "{product name} specs review", numResults: 5)'
mcporter call 'exa.company_research_exa(companyName: "{brand}", numResults: 3)'
```
Exa is the **preferred fallback** — always available, no API key required.
Collect at least **3 independent sources** before proceeding.

### Additional sources (check in order)
1. Official product page (specs, pricing, marketing claims)
2. Third-party reviews (TechAdvisor, The Verge, Tom's Guide, TrustedReviews)
3. Community discussion (Reddit, forums) for real-user pain points
4. Investigation/news coverage (privacy issues, lawsuits, controversies)

### Minimum data threshold
Proceed only when you have:
- [ ] At least one official source with **tech specs + price**
- [ ] At least one **third-party review** with hands-on findings
- [ ] Marketing claims verbatim (needed for Eagle Eye Honest check)

### Pre-launch data flag
If the product meets ALL of the following, append `(Pre-launch)` to the case_id and skip Step 4:
- Zero independent hands-on reviews
- ≥3 critical specs undisclosed (battery, weight, connectivity, regulatory)
- Pre-order only / not shipping

---

## Procedure

### Step 1: Collect Raw Data
- Run `crawl_product_info.py` on official product URL
- If exit code 2, run Exa MCP queries
- Save raw extracts to `01-level0-extracts.md` with source tags [S1], [S2], etc.

### Step 2: Organize + Brand Blind (combined step)
- Read [references/organize-guide.md](references/organize-guide.md)
- Read [references/defluff-guide.md](references/defluff-guide.md)
- Merge all sources into one structured document
- Apply Brand Blinding inline (replace brand names with [BRAND], [PRODUCT], [FEATURE])
- **Preserve ALL adjectives and marketing claims verbatim** — needed by downstream auditors
- Brand-blinded text should be **≤ 1200 words** — deduplicate aggressively but keep all unique claims
- Save to `02-brand-blinded.md`

### Step 3: Triple Auditor Scoring (independent, parallel)

Each Auditor sees **only the Brand-Blinded text** from Step 2.

**Subagent execution rules (mandatory):**
- Pass **file path** to `02-brand-blinded.md` — do NOT embed full text in prompt
- Each subagent reads the file, reads their rubric file, and **prints JSON to stdout**
- **Main agent writes files** — subagents never call `write` (prevents timeout file loss)
- `verbatim_evidence` limits:
  - score 0–1: max **1** quote (or `[]` for score 0)
  - score 2–3: max **2** quotes
- Recommended timeouts: Tool=150s, Toy=150s, Trash=180s

#### 🟢 Tool Auditor
- Reads [references/tool-auditor.md](references/tool-auditor.md)
- Template: [references/tool-auditor-template.md](references/tool-auditor-template.md)
- Score 11 items (0-3 scale), max 33 points
- Output `litmus_gate: "Yes"/"No"` at top level

#### 🟡 Toy Auditor
- Reads [references/toy-auditor.md](references/toy-auditor.md)
- Template: [references/toy-auditor-template.md](references/toy-auditor-template.md)
- Score 11 items (0-3 scale), max 33 points
- Output `litmus_gate: "Yes"/"No"` at top level

#### 🔴 Trash Auditor
- Reads [references/trash-auditor.md](references/trash-auditor.md)
- Reads [references/trash-red-flags.md](references/trash-red-flags.md) (**required** for Eagle Eye)
- Score 14 items (0-3 scale), max 42 points
- Eagle Eye triggers → auto-set score to 3 for flagged items
- Output `litmus_gate: "Yes"/"No"` and `critical_issues: [...]` at top level

**Key principles:**
- Each auditor works from Brand-Blinded text only
- No cross-talk between auditors during scoring
- Evidence FIRST, then score — no score without a verbatim quote
- If a feature isn't in the text, it scores 0

### Step 3.5: Auto-rebuild auditor_reports.json (mandatory)

After all three auditors return, the main agent MUST:

1. **Read** each `03-tool-auditor.json`, `03-toy-auditor.json`, `03-trash-auditor.json`
2. **Rebuild** `auditor_reports.json` programmatically:
   ```python
   merged = {
     "tool":  { "total_score": tool["total_score"], "litmus_gate": tool["litmus_gate"], "extract_for_report": tool["extract_for_report"] },
     "toy":   { "total_score": toy["total_score"],  "litmus_gate": toy["litmus_gate"],  "extract_for_report": toy["extract_for_report"] },
     "trash": { "total_score": trash["total_score"], "litmus_gate": trash["litmus_gate"], "critical_issues": trash["critical_issues"], "extract_for_report": trash["extract_for_report"] }
   }
   ```
3. **Sanity check** before proceeding:
   - `merged.trash.total_score` == sum of all 14 items in `03-trash-auditor.json`
   - `len(merged.trash.critical_issues)` == count of Eagle Eye triggers in `03-trash-auditor.json`
   - If mismatch: re-read the source file (never hand-edit scores)

**🚨 Never manually write auditor_reports.json from memory or summaries — always rebuild from 03-*.json files.**

### Step 4: Eagle Eye Validator (conditional, AI-driven)

Read [references/eagle-eye-validator.md](references/eagle-eye-validator.md)

**Trigger conditions** — run Step 4 if ANY is true:
- Trash report `critical_issues` is non-empty
- Any auditor has any item scored **3**
- Composite score is in Gray Zone (-10 to +10)

Otherwise: skip directly to Step 5.

**Execution:**
- Spawn **1 subagent** (timeout: 120s)
- Subagent reads `02-brand-blinded.md` + only the flagged items from `03-*.json`
- Performs three checks:
  1. **Eagle Eye trigger validity** — do both conflicting quotes exist verbatim?
  2. **Architectural Plausibility** — are on-device/local claims technically feasible for the form factor? (NEW in v2.1)
  3. **Score=3 evidence check** — does evidence actually support the 3-level rubric?
- Outputs `adjustments[]` diff (not a full re-score)
- Main agent applies diffs to `auditor_reports.json` and re-saves

### Step 5: Final Judge Synthesis

```bash
python3 scripts/synthesize_results.py --input auditor_reports.json --text
```

Or manually per [references/t3-classification.md](references/t3-classification.md):
1. Normalize scores: Tool=(raw/33)×100, Toy=(raw/33)×100, Trash=(raw/42)×100
2. Composite = max(NormTool, NormToy) - NormTrash
3. Check primary conditions (need 2+ of 4 conditions per category)
4. Check secondary conditions
5. Apply Eagle Eye Veto (if Trash `critical_issues` non-empty → force Trash into label)

### Step 6: Generate Audit Report

**Output file**: `99-audit-report.md`
**Directory**: `tmp/reports/t3-{YYYY-MM-DD}-{slug}/`

Follow YAML schema in [references/report-schema.md](references/report-schema.md).

---

## Output Format for Messaging Surfaces (WhatsApp / Telegram)

When delivering results in a chat interface, **do not paste the full file**.
Use this condensed format instead:

```
🏷️ T3 Audit — {Product Name}
Price: ${price} | Date: {YYYY-MM-DD}

📊 Scores
🟢 Tool:  {raw}/{33} → {norm}/100
🟡 Toy:   {raw}/{33} → {norm}/100
🔴 Trash: {raw}/{42} → {norm}/100
Composite: {composite:+.1f}

🏆 Classification: {final_label}
Confidence: {confidence}

{If Eagle Eye:}
🚨 Eagle Eye: {trigger name}
"{verbatim quote A}"
vs.
"{verbatim quote B}"

🟢 Tool highlights (3 bullets)
🟡 Toy highlights (3 bullets)
🔴 Trash issues (3 bullets)

💬 One-line verdict
```

Rules for messaging output:
- No markdown tables (WhatsApp doesn't render them)
- Use **bold** for emphasis, bullet lists for details
- Keep total message under 600 words
- If Eagle Eye triggered, always show the conflicting quotes

---

## Important Constraints

### 🚨 Information Isolation (critical)
- Brand-Blinded text is the ONLY input for auditors
- Auditors must never see original brand names, marketing copy, or each other's reports during scoring
- Final Judge reads only the three auditor JSON outputs, not original product text

### 📊 Objective data completeness
- Collect specs, performance, reliability, market, and cost data
- Brand Blinding must **retain** all specs and numbers
- Every score must link to extracted verbatim evidence

### Eagle Eye enforcement
- Trash Auditor MUST check every item against [references/trash-red-flags.md](references/trash-red-flags.md)
- Eagle Eye trigger → automatic score 3 → populate `critical_issues[]`
- Eagle Eye Validator (Step 4) re-checks triggers including **Architectural Plausibility**
- Final Judge must respect Eagle Eye Veto regardless of composite math

### synthesize_results.py input format
The script expects a JSON file like:
```json
{
  "tool":  { "total_score": <int 0-33>, "litmus_gate": "Yes|No", ... },
  "toy":   { "total_score": <int 0-33>, "litmus_gate": "Yes|No", ... },
  "trash": { "total_score": <int 0-42>, "litmus_gate": "Yes|No",
             "critical_issues": ["<eagle eye trigger text>", ...], ... }
}
```

---

## Resource Index

### Scripts
- `scripts/crawl_product_info.py` — Multi-strategy product crawler (Jina → direct HTTP fallback)
- `scripts/synthesize_results.py` — Final Judge math (normalization + Eagle Eye + Litmus Gate)

### References (v2.1)
| File | Used In | Notes |
|------|---------|-------|
| [organize-guide.md](references/organize-guide.md) | Step 2 | |
| [defluff-guide.md](references/defluff-guide.md) | Step 2 | |
| [tool-auditor.md](references/tool-auditor.md) | Step 3 | |
| [tool-auditor-template.md](references/tool-auditor-template.md) | Step 3 | |
| [toy-auditor.md](references/toy-auditor.md) | Step 3 | |
| [toy-auditor-template.md](references/toy-auditor-template.md) | Step 3 | |
| [trash-auditor.md](references/trash-auditor.md) | Step 3 | |
| [trash-red-flags.md](references/trash-red-flags.md) | Step 3 (Eagle Eye) | v2.1: added Architectural Implausibility |
| [eagle-eye-validator.md](references/eagle-eye-validator.md) | Step 4 | v2.1: AI-driven, replaces peer review |
| [t3-classification.md](references/t3-classification.md) | Step 5 | |
| [report-schema.md](references/report-schema.md) | Step 6 | |
| [peer-review-guide.md](references/peer-review-guide.md) | ~~Step 4~~ | **Deprecated in v2.1** — kept for reference only |
Comments 0
Latest | Hot
Please login before commenting.
No comments yet. Be the first one!
t3-hardware-scoring

Directory Structure

SKILL.md

Report

Notice
