browser-automation

分类: 开发与编程 | 上传者: shantanugoel | 下载: 0 | 版本: v1.0（最新）
Control a headless Chrome browser via Pinchtab's REST API
更新日志: Source: GitHub https://github.com/shantanugoel/oxydra
目录结构

当前层级: config/skills/BrowserAutomation/
📁 references/
- 📄 pinchtab-api.md 9.9 KB
📄 SKILL.md 6.5 KB
SKILL.md

---
name: browser-automation
description: Control a headless Chrome browser via Pinchtab's REST API
activation: auto
requires:
  - shell_exec
env:
  - PINCHTAB_URL
priority: 50
---

## Browser Automation (Pinchtab)

Use the `browser` tool for common operations. It handles tab locking,
cleanup, and accessibility tree wait times automatically.

### Quick Start

1. **Navigate** (auto-acquires tab, waits 3s, returns snapshot):
   `browser(action="navigate", url="https://example.com")`

2. **Act** on a ref from the snapshot:
   `browser(action="act", kind="click", ref="e5", tabId="<from step 1>")`

3. **Read text** (~800 tokens vs ~10K for full snapshot):
   `browser(action="text", tabId="...")`

4. **Diff snapshot** (only changes, ~90% fewer tokens):
   `browser(action="snapshot", tabId="...", diff=true)`

5. **List tabs** (see lock status):
   `browser(action="tabs")`

### Core Loop

1. **Navigate** → opens URL, auto-acquires + locks a tab, returns snapshot
2. **Act** → click/type/fill/press using refs from the snapshot
3. **Snapshot** → use `diff=true` to see only changes (~90% fewer tokens)
4. Repeat 2–3 until done

### Tab Management

- The tool acquires and locks a tab automatically on each call
- Pass `tabId` from a previous result to reuse the same tab
- The tool unlocks the tab when the operation completes (even on failure)
- **20-tab system limit** — the tool handles capacity automatically
- Locks auto-expire after timeout; if your tab is stolen, the tool
  re-acquires from available tabs

### Navigate vs Click — Critical Rule

**`navigate` is for going to URLs. `click` is for interactions with no URL.**

When following a link, **never click it** — extract the URL and navigate directly.
Clicking triggers hover states, preview popovers, and JavaScript event handlers that
often intercept navigation (search result previews, HackerNews/Reddit card expansions,
SPAs with custom routing, sticky overlays). Direct navigation bypasses all of this.

```
# WRONG — triggers Google preview, HN popover, etc.
browser(action="act", kind="click", ref="e5", tabId="...")

# RIGHT — extract href from snapshot or evaluate, then navigate
browser(action="evaluate", expression="document.querySelector('a.result').href", tabId="...")
browser(action="navigate", url="<extracted url>")
```

If the snapshot already shows the URL on a link ref, navigate to it directly without
touching the DOM at all. Use `click` only when there is no URL involved: buttons,
form submissions, toggles, checkboxes, dropdowns, and in-page UI controls.

### Action Kinds

For `action="act"`, use these `kind` values:
- `click` — click element by ref (add `waitNav=true` if it triggers navigation)
- `type` — type text into element (set `text`)
- `press` — press a key (set `key`, e.g. "Enter")
- `fill` — set value directly (set `text` and optionally `selector`)
- `hover` — hover over element (triggers dropdowns/tooltips)
- `scroll` — scroll to element by ref
- `select` — select dropdown option (set `value`)
- `focus` — focus an element

### Key Endpoints

All endpoints use flat paths. Multi-tab targeting uses `?tabId=ID` query parameter or `"tabId"` in POST body.

| Endpoint | Method | Purpose |
|---|---|---|
| `/navigate` | POST | Navigate URL → `{tabId, url, title}` |
| `/tabs` | GET | List all tabs → `{tabs: [{id, title, url, type}]}` |
| `/tab` | POST | `{"action":"new","url":"..."}` or `{"action":"close","tabId":"..."}` |
| `/snapshot` | GET | Accessibility tree. Params: `tabId`, `filter=interactive`, `diff=true`, `maxTokens=2000`, `format=compact` |
| `/text` | GET | Readable text. Params: `tabId`, `mode=raw` |
| `/action` | POST | `{"kind":"click\|type\|fill\|press\|hover\|scroll\|select\|focus", "ref":"e5", "tabId":"..."}` |
| `/actions` | POST | Batch: `{"actions":[...], "stopOnError":true, "tabId":"..."}` |
| `/screenshot` | GET | Binary PNG. Params: `tabId`, `raw=true` → save with `curl -o /shared/file.png` |
| `/pdf` | GET | `?tabId=...&output=file&path=/shared/.pinchtab/page.pdf` |
| `/evaluate` | POST | Run JS: `{"expression":"document.title", "tabId":"..."}` |
| `/cookies` | GET/POST | Get/set cookies. Param: `tabId` |
| `/download` | GET | Download file: `?url=...&output=file&path=/shared/.pinchtab/file.ext` |
| `/upload` | POST | Upload: `{"selector":"input[type=file]","paths":["/shared/file.jpg"],"tabId":"..."}` |
| `/health` | GET | Health check → `{status, tabs}` |

### Best Practices

- Always use `maxTokens=2000` on snapshots (full trees can exceed 10K tokens)
- Use `filter=interactive` to see only clickable/input elements
- Use `diff=true` after actions for ~90% token savings
- Use `/text` for reading content (~800 tokens/page)
- Use `format=compact` for most token-efficient snapshots
- Batch interactions with `POST /actions` for fewer round-trips

### File Operations (use shell_exec + curl)

Save screenshots:
  `curl "{{PINCHTAB_URL}}/screenshot?tabId=$TAB&raw=true" \
    -H "Authorization: Bearer $BRIDGE_TOKEN" -o /shared/screenshot.png`

Export PDF:
  `curl "{{PINCHTAB_URL}}/pdf?tabId=$TAB&output=file&path=/shared/.pinchtab/page.pdf" \
    -H "Authorization: Bearer $BRIDGE_TOKEN" && \
    cp /shared/.pinchtab/page.pdf /shared/page.pdf`

Download file:
  `curl "{{PINCHTAB_URL}}/download?url=URL&output=file&path=/shared/.pinchtab/f.ext" \
    -H "Authorization: Bearer $BRIDGE_TOKEN" && \
    cp /shared/.pinchtab/f.ext /shared/f.ext`

Upload file:
  `curl -X POST "{{PINCHTAB_URL}}/upload" \
    -H "Authorization: Bearer $BRIDGE_TOKEN" \
    -H 'Content-Type: application/json' \
    -d '{"selector":"input[type=file]","paths":["/shared/photo.jpg"],"tabId":"ID"}'`

Evaluate JS:
  `curl -X POST "{{PINCHTAB_URL}}/evaluate" \
    -H "Authorization: Bearer $BRIDGE_TOKEN" \
    -H 'Content-Type: application/json' \
    -d '{"expression":"document.title","tabId":"ID"}'`

Set cookies:
  `curl -X POST "{{PINCHTAB_URL}}/cookies" \
    -H "Authorization: Bearer $BRIDGE_TOKEN" \
    -H 'Content-Type: application/json' \
    -d '{"url":"https://example.com","cookies":[{"name":"s","value":"v"}]}'`

Stealth/fingerprint:
  `curl "{{PINCHTAB_URL}}/stealth/status" -H "Authorization: Bearer $BRIDGE_TOKEN"`
  `curl -X POST "{{PINCHTAB_URL}}/fingerprint/rotate" \
    -H "Authorization: Bearer $BRIDGE_TOKEN" \
    -H 'Content-Type: application/json' -d '{"os":"windows"}'`

After saving files to /shared/, use `send_media` to deliver to the user.

### If Blocked

If you encounter CAPTCHAs, 2FA, or login walls, call `request_human_assistance`
with a clear description of the blocker.

Full API reference:
`cat /shared/.oxydra/skills/BrowserAutomation/references/pinchtab-api.md`
登录后下载/点赞/收藏 ❤ 37 | ★ 0
browser-automation

目录结构

SKILL.md

举报内容

提示
