sinain-hud

Category: Data & AI | Uploader: anthillnet | Downloads: 0 | Version: v1.0（Latest）

Two-way communication with sinain-core on Geravant's Mac:

Changelog: Source: GitHub https://github.com/anthillnet/sinain-hud

Directory Structure

Current level: sinain-hud-plugin/

📁 sinain-knowledge/
- 📁 adapters/
  - 📁 generic/
    
    📄 adapter.ts 3.2 KB
  - 📁 openclaw/
    
    📄 adapter.ts 8.1 KB
  - 📄 interface.ts 2.6 KB
- 📁 curation/
  - 📄 engine.ts 17.2 KB
  - 📄 resilience.ts 13.3 KB
- 📁 data/
  - 📄 git-store.ts 12.2 KB
  - 📄 schema.ts 2.4 KB
  - 📄 snapshot.ts 7.6 KB
  - 📄 store.ts 17.2 KB
- 📁 deploy/
  - 📄 cli.ts 7.7 KB
  - 📄 manifest.ts 2.0 KB
- 📁 protocol/
  - 📁 bindings/
    
    📄 generic.md 116 B
    
    📄 openclaw.md 144 B
  - 📄 heartbeat.md 2.0 KB
  - 📄 renderer.ts 1.7 KB
  - 📄 skill.md 13.7 KB
📁 sinain-memory/
- 📁 eval/
  - 📁 judges/
    
    📄 __init__.py 0 B
    
    📄 base_judge.py 1.9 KB
    
    📄 curation_judge.py 1.6 KB
    
    📄 insight_judge.py 1.8 KB
    
    📄 mining_judge.py 1.6 KB
    
    📄 signal_judge.py 1.5 KB
  - 📄 __init__.py 0 B
  - 📄 assertions.py 12.5 KB
  - 📄 schemas.py 8.8 KB
- 📁 memory/
  - 📁 playbook-archive/
    
    📄 sinain-playbook-2026-03-01-1418.md 544 B
  - 📁 playbook-logs/
    
    📄 2026-03-01.jsonl 593 B
  - 📄 2026-03-01.md 425 B
  - 📄 sinain-playbook.md 979 B
- 📁 tests/
  - 📄 __init__.py 0 B
  - 📄 conftest.py 6.8 KB
  - 📄 test_curator_helpers.py 3.4 KB
  - 📄 test_embedder.py 6.9 KB
  - 📄 test_extract_json.py 4.3 KB
  - 📄 test_feedback_computation.py 5.0 KB
  - 📄 test_miner_helpers.py 2.7 KB
  - 📄 test_module_management.py 18.7 KB
  - 📄 test_parsers.py 3.5 KB
  - 📄 test_tick_evaluator.py 17.7 KB
  - 📄 test_triple_extractor.py 10.1 KB
  - 📄 test_triple_ingest.py 6.6 KB
  - 📄 test_triple_migrate.py 5.1 KB
  - 📄 test_triplestore.py 8.9 KB
- 📄 common.py 13.6 KB
- 📄 demo_knowledge_transfer.sh 3.0 KB
- 📄 embedder.py 8.9 KB
- 📄 eval_delta.py 3.9 KB
- 📄 eval_reporter.py 24.5 KB
- 📄 feedback_analyzer.py 7.1 KB
- 📄 git_backup.sh 490 B
- 📄 insight_synthesizer.py 6.7 KB
- 📄 memory-config.json 1.3 KB
- 📄 memory_miner.py 6.2 KB
- 📄 module_manager.py 25.0 KB
- 📄 playbook_curator.py 8.6 KB
- 📄 requirements.txt 58 B
- 📄 signal_analyzer.py 5.1 KB
- 📄 test_local.py 15.2 KB
- 📄 tick_evaluator.py 14.0 KB
- 📄 triple_extractor.py 16.0 KB
- 📄 triple_ingest.py 11.4 KB
- 📄 triple_migrate.py 9.7 KB
- 📄 triple_query.py 6.3 KB
- 📄 triplestore.py 17.6 KB
📄 HEARTBEAT.md 2.0 KB
📄 index.ts 39.6 KB
📄 install.js 17.2 KB
📄 openclaw.plugin.json 2.0 KB
📄 package.json 591 B
📄 README.md 7.7 KB
📄 SKILL.md 13.7 KB

SKILL.md

# SinainHUD Skill

## What
Two-way communication with sinain-core on Geravant's Mac:
1. **Receive escalations** — respond to context, errors, and questions from sinain-core
2. **Delegate tasks** — spawn subagents for research and background work
3. **Read transcripts** — access sinain session history for context

---

## Architecture

```
sinain-core (Mac)                    OpenClaw Server
     │                                     │
     ├── Escalates ──────────────────────→ │ (message with inline context)
     │   [digest, OCR, audio, errors]      │
     │                                     ├── Main agent processes
     │                                     │
     │ ←───────────────────────────────────┤ Response via escalation reply
     │   (displayed on HUD overlay)        │
     │                                     │
     └── Session transcripts ────────────→ sessions_history (queryable)
```

**Key insight:** There is no separate relay server. sinain-core escalates directly to the OpenClaw server. Responses flow back through the escalation response mechanism and are displayed on the Mac's HUD overlay.

---

## Part 1: Handling Escalations

sinain-core monitors screen and audio on the Mac, then escalates to OpenClaw when it needs help or has context to share.

### Message Format

Escalation messages carry **inline context**:
- **Digest** — summarized screen/audio activity
- **OCR** — text extracted from screen
- **Audio transcripts** — recent speech-to-text
- **Errors** — detected errors from IDEs, terminals, logs

**Example incoming escalation:**
```
[sinain-hud live context — tick #42]

## Digest
User is editing analyzer.ts in IntelliJ IDEA. TypeScript error visible on line 45.

## OCR
TypeError: Cannot read property 'map' of undefined
  at processItems (analyzer.ts:45:12)

## Audio
"Why isn't this working... I already checked for null"
```

### Response Guidelines

- **Respond directly** — your reply goes back to the HUD overlay
- **Be detailed** — 5-10 sentences for escalation responses, drawing on the digest/OCR/audio context provided
- **Address errors first** — prioritize fixing what's broken
- **Reference the context** — use the digest/OCR/audio to be specific
- **Never NO_REPLY** — always respond to escalations

**Example response:**
"The error suggests `ctx.items` is undefined before the `.map()` call. Add a guard: `ctx.items?.map(...)` or check the data source that populates `ctx.items`."

---

## Part 2: Spawn-Task Requests

Messages with `[sinain-core:spawn-task]` request a background subagent for longer work.

### How to Handle

1. Extract the task description
2. Call `sessions_spawn` with the task
3. **Include spawn result in your response** (sinain-core tracks this)

**Example incoming:**
```
[sinain-core:spawn-task] (label: "Team standup")

Please spawn a subagent to handle this task:

Clean up and summarize this recording transcript:

[00:00] Hey everyone, let's start the standup...
[00:15] I finished the authentication PR yesterday...
```

**Your response (must include JSON for tracking):**
```
Spawning subagent for 'Team standup'...
spawn_result: {"status":"accepted","childSessionKey":"agent:main:subagent:xxxxx","runId":"xxxxx"}
```

### sessions_spawn Usage

```javascript
sessions_spawn({
  task: "Clean up and summarize this recording transcript:\n\n[00:00] Hey everyone...",
  label: "standup-summary",  // optional, for tracking
  cleanup: "delete"          // auto-cleanup when done
})
```

**Note:** Subagent results are delivered via Telegram (the main channel), not directly to the HUD. The user sees completion notifications in their normal message flow.

---

## Part 3: Reading Sinain Session History

Use `sessions_history` to read past sinain session transcripts and context.

### When to Use
- Understanding what the user was working on earlier
- Finding context from past escalations
- Reviewing audio transcripts from earlier in the day
- Checking what errors were reported and whether they were resolved

### Usage

```javascript
// Get recent sinain session history
sessions_history({
  sessionKey: "agent:main:sinain",  // full session key (required)
  limit: 20,                        // number of messages (optional, default varies)
  includeTools: false                // include tool calls in output (optional, default false)
})
```

**Parameters:**
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `sessionKey` | string | **yes** | The session to query. For sinain: `"agent:main:sinain"` |
| `limit` | number | no | Max messages to return |
| `includeTools` | boolean | no | Whether to include tool call messages |

**Important:** There is no `since` or `session` parameter. Always use the full `sessionKey`.

### What You Get Back
- Timestamped messages with inline context (digest, OCR, audio, errors)
- Your previous responses to escalations
- Spawn-task requests and their outcomes

---

## Output Routing

| Content Type | Destination | Why |
|--------------|-------------|-----|
| Escalation responses | **HUD** (via escalation reply) | Immediate, contextual help |
| Subagent completions | **Telegram** | Persistent notification |
| Urgent alerts | **Telegram** | Needs acknowledgment |
| Proactive research results | **Telegram** | Non-blocking, async delivery |

**Rule:** Your direct response to an escalation goes to the HUD. Everything else flows through normal Telegram messaging.

---

## Part 4: Learning from Feedback Signals

sinain-core tracks what happens **after** each escalation and computes feedback signals. These are sent periodically as `[sinain-core:feedback-summary]` messages.

### Feedback Signal Format

Each escalation gets scored on these signals:

| Signal | What It Measures | Good | Bad |
|--------|-----------------|------|-----|
| `errorCleared` | Error disappeared from screen after your response | `true` | `false` |
| `noReEscalation` | Same issue didn't re-escalate within 5 minutes | `true` | `false` |
| `dwellTimeMs` | Time user spent on app after response | > 60s | < 10s |
| `quickAppSwitch` | User switched apps within 10s of your response | `false` | `true` |
| `compositeScore` | Weighted combo of above signals | 0.5 to 1.0 | -1.0 to 0 |

### Reading a Feedback Summary

```
[sinain-core:feedback-summary]

Escalations: 12 | Avg score: 0.45 | Avg latency: 3200ms
Top tags: coding (8), error (5), app:intellij (7)

Recent (last 5):
  ✓ 0.80 [coding, error] — error cleared, no re-escalation
  ✓ 0.60 [coding] — no re-escalation, good dwell
  ✗ -0.20 [error, app:intellij] — error persisted, re-escalated
  ✓ 0.50 [coding] — no re-escalation
  ✓ 0.30 [general] — quick app switch but no re-escalation
```

### How to Use Feedback

**Positive patterns (score > 0.3):**
- Your approach is working — keep it up
- Note what tags/contexts correlate with high scores

**Negative patterns (score < 0):**
- `errorCleared: false` → your fixes aren't working. Try different approaches:
  - Ask for more context instead of guessing
  - Suggest debugging steps rather than direct fixes
  - Check if the error is a red herring (build artifact, stale cache)
- `quickAppSwitch: true` → user found your response unhelpful. Adjust:
  - Be more concise — user may be scanning quickly
  - Lead with the actionable fix, not the explanation
  - Match the urgency — if they're debugging, don't philosophize
- High re-escalation rate → same issues keep coming back:
  - Your fixes are superficial — address root cause
  - The issue might be environmental (config, dependency)
  - Consider suggesting a different tool or approach entirely

### Anti-Patterns to Watch

1. **Always suggesting restarts** — if `errorCleared` is often false after restart suggestions, stop suggesting restarts
2. **Too verbose** — if `quickAppSwitch` is consistently true, shorten your responses dramatically
3. **Missing the real error** — if the same error type keeps re-escalating, you're probably fixing symptoms not causes
4. **Ignoring audio context** — if user mentions something in audio that your response doesn't address, the feedback will be negative

### Inline Feedback in Escalations

Escalation messages may include a `## Recent Feedback` section at the bottom:

```
## Recent Feedback (last 5 escalations)
Score: 0.45 avg | Errors cleared: 3/5 | Re-escalated: 1/5
Recent: ✓ 0.8 (coding, error) | ✓ 0.6 (coding) | ✗ -0.2 (error, intellij) | ✓ 0.5 (coding) | ✓ 0.3 (general)
```

Use this inline context to calibrate your response — if recent scores are low, try a different approach than what you've been doing.

---

## Part 5: Proactive Research from Escalations

When processing escalation ticks, watch for opportunities to **spawn research subagents** proactively. This turns passive observation into active assistance.

### When to Spawn Research

| Signal in Escalation | Research to Spawn |
|---------------------|-------------------|
| User researching new tech/library | `sessions_spawn`: "Deep-dive on [tech]. Quickstart, gotchas, best practices, alternatives." |
| Environment-related error (config, deps, infra) | `sessions_spawn`: "Root cause analysis for [error]. Check known issues, changelogs, migration guides." |
| User asked a question in audio transcript | `sessions_spawn`: "Find answer to: [question]. Summarize with sources." |
| User reading docs/tutorials on a topic | `sessions_spawn`: "Compile key insights on [topic]. What matters most, common mistakes." |

### Rules

1. **Respond to the escalation first** — always send your HUD response before spawning
2. **Max 1 spawn per escalation** — don't flood with subagents
3. **Don't duplicate heartbeat research** — check recent chat history for overlapping spawns
4. **Non-blocking** — `sessions_spawn` returns immediately; the subagent works in the background
5. **Only spawn when the value is clear** — if the user's activity is routine (normal coding, switching apps), don't spawn

### Example Flow

```
Incoming escalation:
  Digest: User browsing Bun documentation. Switched from Node.js project.
  OCR: "Bun.serve() API reference — HTTP server"
  Audio: "I wonder if Bun handles websockets differently..."

Your response (to HUD):
  "Bun handles WebSockets natively via Bun.serve() with a `websocket` handler —
  no separate ws package needed. The API is: server.upgrade(req) in the fetch handler,
  then define open/message/close callbacks in the websocket config."

Then spawn (non-blocking):
  sessions_spawn({
    task: "Deep-dive: Bun WebSocket API vs Node.js ws package. Key differences, migration path, performance characteristics, gotchas. Include code examples.",
    label: "bun-websocket-research",
    cleanup: "delete"
  })
```

The subagent result will arrive via Telegram later, giving the user a thorough reference.

---

## Part 6: Self-Improving Heartbeat

The heartbeat (`HEARTBEAT.md`) runs every 15 minutes and implements a **self-improving feedback loop** — it doesn't just observe and act, it learns from accumulated experience.

### Three-Phase Cycle

| Phase | Purpose | Key Action |
|-------|---------|------------|
| **1. Git Backup** | Persist learning | Commit `memory/` (playbook, archive, logs) and push |
| **2. Observe & Act** | Reactive | Scan recent history, spawn subagents or send suggestions |
| **3. Reflect & Curate** | Self-improving | Update playbook from feedback signals, log decisions, synthesize output |

### Playbook — `memory/sinain-playbook.md`

The heartbeat maintains a curated playbook of what works and what doesn't, updated each tick:
- **Successful patterns** — approaches with high `compositeScore` (>0.3) from feedback signals
- **Failed patterns** — approaches that didn't work, with reasons
- **User preferences** — recurring topics, preferred tools, work rhythms
- Capped at 50 lines; stale entries are pruned, repeated patterns promoted from "observed" to "established"

### Decision Logs — `memory/playbook-logs/YYYY-MM-DD.jsonl`

Every heartbeat tick appends a structured JSON line capturing: what was observed, what actions were considered (and why some were skipped), playbook changes, and the output produced. This enables offline evaluation of the self-improvement process.

### Playbook Archive — `memory/playbook-archive/`

Before each mutation, the current playbook is archived with a timestamp. The archive forms a versioned dataset showing how the agent's strategy evolves over time.

### Output Format

Each heartbeat produces a Telegram message with two components:
- **Suggestion** — practical, actionable recommendation grounded in playbook patterns
- **Insight** — surprising, non-obvious connection from accumulated observations

Both must reference concrete data. Generic advice and repeated outputs are prohibited.

### Setup Requirements

The `openclaw-config-patch.json` documents the required server configuration:
- `agents.defaults.heartbeat.every: "15m"` — 15-minute heartbeat cadence
- `agents.defaults.sandbox.sessionToolsVisibility: "all"` — cross-session history access

---

## Quick Reference

| Task | Method |
|------|--------|
| Respond to live context | Reply directly (goes to HUD) |
| Spawn background work | `sessions_spawn({task: "...", cleanup: "delete"})` |
| Read past context (light) | `sessions_history({sessionKey: "agent:main:sinain", limit: 50, includeTools: false})` |
| Read past context (detailed) | `sessions_history({sessionKey: "agent:main:sinain", limit: 10, includeTools: true})` — only when signals found |
| Proactive research | Respond first, then `sessions_spawn` with research task (max 1 per escalation) |
| Process feedback | Read `[sinain-core:feedback-summary]`, adjust strategy |
| Self-improving loop | Heartbeat curates `memory/sinain-playbook.md` every 15m (see HEARTBEAT.md) |

Comments 0

Latest | Hot

Please login before commenting.

No comments yet. Be the first one!

sinain-hud

Directory Structure

SKILL.md

Report

Notice