model-matchmaker

Category: 数据与AI | Uploader: coyvalyss1 | Downloads: 0 | Version: v1.0（Latest）
This skill helps you personalize your Model Matchmaker classifier based on your actual usage patterns. After you've collected 50+ recommendations, this skill analyzes when you disagreed with the advisor and tunes your local classifier to match your preferences.
Changelog: Source: GitHub https://github.com/coyvalyss1/model-matchmaker
Directory Structure

Current level: skills/optimize-classifier/
📄 SKILL.md 6.7 KB
SKILL.md

# Optimize My Classifier

This skill helps you personalize your Model Matchmaker classifier based on your actual usage patterns. After you've collected 50+ recommendations, this skill analyzes when you disagreed with the advisor and tunes your local classifier to match your preferences.

## What This Skill Does

1. **Reads** your local Model Matchmaker logs (`~/.cursor/hooks/model-matchmaker.ndjson`)
2. **Analyzes** when you overrode the advisor's recommendations
3. **Finds patterns** in the prompts where you disagreed (common words, task types)
4. **Suggests** keyword additions to your `model-advisor.sh` file
5. **Updates** your classifier (with your approval) so future recommendations match your preferences

**Privacy:** Everything happens locally. No data leaves your machine. You review and approve every change.

## Instructions for the AI

You are helping the user optimize their Model Matchmaker classifier based on their personal override patterns. Follow these steps:

### Step 1: Read and Validate Log File

Read the NDJSON log file at `~/.cursor/hooks/model-matchmaker.ndjson`.

Check if there's enough data:
- Need at least 50 `recommendation` events total
- Need at least 5 `OVERRIDE` events to find patterns
- If not enough data, tell the user: "You need more usage data. Come back after 50+ prompts with at least a few overrides."

### Step 2: Analyze Override Patterns

Filter to `action: "OVERRIDE"` events and group by model direction:

**Group A: User preferred Opus over recommended Haiku/Sonnet**
- These are prompts where the classifier said "use a cheaper model" but you said "no, I need Opus"
- Extract common words from `prompt_snippet` fields in this group

**Group B: User preferred Sonnet over recommended Opus**
- These are prompts where the classifier said "use Opus" but you said "no, Sonnet is fine"
- Extract common words from `prompt_snippet` fields in this group

**Group C: User preferred Sonnet over recommended Haiku**
- Extract common words from `prompt_snippet` fields in this group

**Group D: User preferred Haiku over recommended Sonnet/Opus**
- Extract common words from `prompt_snippet` fields in this group

### Step 3: Extract Keyword Candidates

For each group, find words that appear in 3+ override prompts (minimum frequency threshold).

**Exclude common stop words:**
- the, a, an, is, are, was, were, be, been, being, have, has, had, do, does, did
- in, on, at, to, for, of, with, from, by, about, as, into, through, during
- this, that, these, those, I, you, we, they, it, he, she, me, my, your

**Focus on action verbs and technical terms:**
- Examples: debug, investigate, refactor, optimize, analyze, build, create, fix, update, configure

### Step 4: Generate Proposed Changes

For each keyword group, map to the correct section of `model-advisor.sh`:

**For Group A keywords (user prefers Opus):**
→ Add to `opus_keywords` list around line 46

**For Group B/C keywords (user prefers Sonnet):**
→ Add to `sonnet_patterns` list around line 63

**For Group D keywords (user prefers Haiku):**
→ Add to `haiku_patterns` list around line 53

### Step 5: Calculate Impact

For each proposed keyword:
- Count how many past overrides would become correct recommendations if this keyword were added
- Show confidence: "3 overrides → 'debug' should trigger Opus"

### Step 6: Present Recommendations

Output in this format:

```markdown
# Classifier Optimization Report

## Your Usage Summary

- Total recommendations: N
- Overrides: M (X.X%)
- Ready for optimization: [Yes/No - need 5+ overrides]

## Patterns Found

### You prefer Opus for:

**Keyword: "debug"**
- Frequency: 5 overrides
- Current behavior: Recommends Sonnet
- Proposed: Add "debug" to opus_keywords
- Impact: 5 past overrides would become correct recommendations

**Keyword: "investigate"**
- Frequency: 3 overrides
- Current behavior: Recommends Sonnet
- Proposed: Add "investigate" to opus_keywords
- Impact: 3 past overrides would become correct recommendations

[Repeat for other keywords]

### You prefer Sonnet over Opus for:

[Same structure]

### You prefer Haiku for:

[Same structure]

## Proposed Changes to ~/.cursor/hooks/model-advisor.sh

**Add to opus_keywords (line 46):**
```python
opus_keywords = [
    "architect", "architecture", "evaluate", "tradeoff", "trade-off",
    "strategy", "strategic", "compare approaches", "why does", "deep dive",
    "redesign", "across the codebase", "investor", "multi-system",
    "complex refactor", "analyze", "analysis", "plan mode", "rethink",
    "high-stakes", "critical decision",
    # NEW - Added based on your override patterns:
    "debug", "investigate"  # 8 overrides support this
]
```

**Add to sonnet_patterns (line 63):**
[Show specific regex additions if any]

## Next Steps

Would you like me to apply these changes to your classifier?

1. **Yes** - I'll update ~/.cursor/hooks/model-advisor.sh with these keywords
2. **Some of them** - Tell me which keywords to add
3. **No** - Just show me the analysis, don't change anything

If you approve, I'll:
1. Read your current model-advisor.sh
2. Add the new keywords to the appropriate lists
3. Write the updated file back
4. Confirm the changes

You can test immediately by restarting Cursor or starting a new composer session.
```

### Step 7: Apply Changes (If User Approves)

If user says "yes" or approves specific keywords:

1. Read `~/.cursor/hooks/model-advisor.sh`
2. Locate the appropriate keyword list (opus_keywords, sonnet_patterns, or haiku_patterns)
3. Add the new keywords to the list with a comment explaining they were auto-added
4. Write the file back using the StrReplace tool
5. Confirm: "Updated! Your classifier now recommends [model] for prompts containing [keywords]. Changes take effect in your next Cursor session."

### Important Notes

- Only suggest keywords with 3+ override occurrences (confidence threshold)
- Don't add generic words that could cause false positives ("the", "make", "update")
- Show the user exactly what will change before changing it
- If the user's overrides are inconsistent (sometimes want Opus, sometimes want Sonnet for the same keyword), note that and ask for clarification

---

## How to Use This Skill

As a user, invoke this skill by saying something like:

- "Optimize my Model Matchmaker classifier"
- "Tune my classifier based on my overrides"
- "Analyze my Model Matchmaker usage and improve it"
- "Update my classifier to match my preferences"

The AI will analyze your logs, find patterns, and propose keyword additions. You review and approve before any changes are made.

## When to Run This

- After your first 50-100 prompts (to establish baseline patterns)
- Monthly or quarterly as your workflow evolves
- After a major project shift (switching from backend to frontend work, for example)
- Whenever you notice you're overriding the advisor frequently
Comments 0
Latest | Hot
Please login before commenting.
No comments yet. Be the first one!
model-matchmaker

Directory Structure

SKILL.md

Report

Notice
