apitap

分类: 文档与知识管理 | 上传者: n1byn1kt | 下载: 0 | 版本: v1.0（最新）

> No docs, no SDK, no browser. Just data.

更新日志: Source: GitHub https://github.com/n1byn1kt/apitap

目录结构

当前层级: 根目录

📁 .github/
- 📁 workflows/
  - 📄 ci.yml 388 B
- 📄 CODE_OF_CONDUCT.md 926 B
- 📄 copilot-instructions.md 2.7 KB
- 📄 SECURITY.md 1.8 KB
📁 docs/
- 📁 plans/
  - 📄 2026-02-28-mcp-bugfixes-design.md 4.7 KB
  - 📄 2026-02-28-mcp-bugfixes.md 16.5 KB
  - 📄 2026-03-02-agent-browser-bridge-design.md 12.2 KB
  - 📄 2026-03-02-agent-browser-bridge.md 42.3 KB
  - 📄 2026-03-07-passive-index-design.md 13.1 KB
  - 📄 2026-03-07-passive-index-impl-plan.md 72.7 KB
- 📄 demo.gif 299.3 KB
📁 extension/
- 📁 src/
  - 📄 background.ts 27.8 KB
  - 📄 consent.ts 1.0 KB
  - 📄 domain-utils.ts 620 B
  - 📄 index-store.ts 2.5 KB
  - 📄 lifecycle.ts 1.3 KB
  - 📄 multi-domain.ts 915 B
  - 📄 observer.ts 4.3 KB
  - 📄 popup.ts 8.3 KB
  - 📄 promotion.ts 583 B
  - 📄 security.ts 3.2 KB
  - 📄 sensitive-paths.ts 848 B
  - 📄 shim.ts 671 B
  - 📄 types.ts 2.2 KB
- 📄 .gitignore 20 B
- 📄 esbuild.config.mjs 402 B
- 📄 manifest.json 650 B
- 📄 package-lock.json 14.1 KB
- 📄 package.json 277 B
- 📄 popup.css 2.3 KB
- 📄 popup.html 1.9 KB
- 📄 SPIKE-RESULTS.md 4.6 KB
- 📄 tsconfig.json 390 B
📁 scripts/
- 📄 release.sh 2.5 KB
📁 src/
- 📁 auth/
  - 📄 crypto.ts 4.3 KB
  - 📄 handoff.ts 11.9 KB
  - 📄 manager.ts 7.9 KB
  - 📄 oauth-refresh.ts 4.7 KB
  - 📄 refresh.ts 8.7 KB
- 📁 bridge/
  - 📄 client.ts 1.8 KB
- 📁 capture/
  - 📄 anti-bot.ts 1.7 KB
  - 📄 blocklist.ts 1.6 KB
  - 📄 body-diff.ts 3.0 KB
  - 📄 body-variables.ts 4.9 KB
  - 📄 browser.ts 3.2 KB
  - 📄 domain.ts 1.0 KB
  - 📄 entropy.ts 3.3 KB
  - 📄 filter.ts 1.3 KB
  - 📄 graphql.ts 2.8 KB
  - 📄 idle.ts 1.2 KB
  - 📄 monitor.ts 8.9 KB
  - 📄 oauth-detector.ts 4.3 KB
  - 📄 pagination.ts 1.4 KB
  - 📄 parameterize.ts 2.0 KB
  - 📄 scrubber.ts 2.4 KB
  - 📄 session.ts 17.1 KB
  - 📄 token-detector.ts 2.2 KB
  - 📄 verifier.ts 5.8 KB
- 📁 contract/
  - 📄 diff.ts 2.2 KB
  - 📄 schema.ts 1.2 KB
- 📁 discovery/
  - 📄 auth.ts 3.1 KB
  - 📄 fetch.ts 2.9 KB
  - 📄 frameworks.ts 7.4 KB
  - 📄 index.ts 7.7 KB
  - 📄 openapi.ts 6.5 KB
  - 📄 probes.ts 2.2 KB
- 📁 extension/
  - 📄 install.ts 2.3 KB
- 📁 index/
  - 📄 reader.ts 1.7 KB
- 📁 inspect/
  - 📄 report.ts 7.6 KB
- 📁 orchestration/
  - 📄 browse.ts 11.3 KB
  - 📄 cache.ts 785 B
- 📁 read/
  - 📁 decoders/
    
    📄 deepwiki.ts 5.8 KB
    
    📄 grokipedia.ts 7.9 KB
    
    📄 hackernews.ts 5.2 KB
    
    📄 index.ts 704 B
    
    📄 reddit.ts 6.9 KB
    
    📄 twitter.ts 6.3 KB
    
    📄 wikipedia.ts 2.1 KB
    
    📄 youtube.ts 2.1 KB
  - 📄 extract.ts 11.7 KB
  - 📄 index.ts 2.4 KB
  - 📄 peek.ts 3.9 KB
  - 📄 types.ts 904 B
- 📁 replay/
  - 📄 engine.ts 23.3 KB
  - 📄 truncate.ts 3.4 KB
- 📁 skill/
  - 📄 generator.ts 20.0 KB
  - 📄 importer.ts 2.7 KB
  - 📄 search.ts 2.1 KB
  - 📄 signing.ts 1.7 KB
  - 📄 ssrf.ts 10.5 KB
  - 📄 store.ts 4.2 KB
  - 📄 validate.ts 4.5 KB
- 📁 stats/
  - 📄 report.ts 6.2 KB
- 📄 cli.ts 42.5 KB
- 📄 index.ts 2.0 KB
- 📄 mcp.ts 25.0 KB
- 📄 native-host.ts 11.6 KB
- 📄 plugin.ts 6.2 KB
- 📄 serve.ts 5.2 KB
- 📄 types.ts 6.8 KB
📁 test/
- 📁 auth/
  - 📄 crypto.test.ts 3.0 KB
  - 📄 handoff.test.ts 4.1 KB
  - 📄 key-separation.test.ts 2.9 KB
  - 📄 manager.test.ts 10.2 KB
  - 📄 oauth-refresh.test.ts 11.9 KB
  - 📄 refresh-dispatcher.test.ts 6.9 KB
  - 📄 refresh.test.ts 2.6 KB
- 📁 bridge/
  - 📄 client.test.ts 2.6 KB
- 📁 capture/
  - 📄 anti-bot.test.ts 3.3 KB
  - 📄 blocklist.test.ts 1.1 KB
  - 📄 body-diff.test.ts 4.3 KB
  - 📄 body-variables.test.ts 7.2 KB
  - 📄 browser.test.ts 1.7 KB
  - 📄 domain.test.ts 1.4 KB
  - 📄 entropy.test.ts 7.1 KB
  - 📄 filter.test.ts 4.1 KB
  - 📄 graphql.test.ts 3.5 KB
  - 📄 idle.test.ts 2.3 KB
  - 📄 monitor-body.test.ts 977 B
  - 📄 monitor-session-injection.test.ts 2.4 KB
  - 📄 oauth-detector.test.ts 12.5 KB
  - 📄 pagination.test.ts 2.3 KB
  - 📄 parameterize.test.ts 3.3 KB
  - 📄 scrubber.test.ts 2.6 KB
  - 📄 session-injection.test.ts 3.2 KB
  - 📄 session-navigate-ssrf.test.ts 3.7 KB
  - 📄 session.test.ts 11.9 KB
  - 📄 token-detector.test.ts 3.0 KB
  - 📄 verifier.test.ts 10.0 KB
- 📁 cli/
  - 📄 audit.test.ts 4.0 KB
  - 📄 auth-cli.test.ts 3.5 KB
  - 📄 browse.test.ts 3.1 KB
  - 📄 discover-index.test.ts 1.4 KB
  - 📄 forget.test.ts 4.3 KB
  - 📄 replay-body.test.ts 1.5 KB
  - 📄 search-cli.test.ts 3.9 KB
- 📁 contract/
  - 📄 diff.test.ts 4.2 KB
  - 📄 schema.test.ts 2.8 KB
- 📁 discovery/
  - 📄 auth.test.ts 3.5 KB
  - 📄 discovery.test.ts 7.6 KB
  - 📄 fetch.test.ts 4.2 KB
  - 📄 frameworks.test.ts 7.0 KB
  - 📄 openapi.test.ts 7.9 KB
  - 📄 probes.test.ts 4.1 KB
- 📁 e2e/
  - 📁 helpers/
    
    📄 mock-oauth-server.ts 4.2 KB
  - 📄 anti-bot.test.ts 1.8 KB
  - 📄 auth-refresh.test.ts 10.0 KB
  - 📄 bridge-e2e.test.ts 8.3 KB
  - 📄 capture-replay.test.ts 5.3 KB
  - 📄 graphql-capture-replay.test.ts 5.4 KB
  - 📄 oauth-auth0.test.ts 5.2 KB
  - 📄 oauth-capture-live.test.ts 8.3 KB
  - 📄 oauth-e2e.test.ts 11.3 KB
  - 📄 oauth-firebase.test.ts 6.8 KB
  - 📄 oauth-supabase.test.ts 5.3 KB
  - 📄 search-replay.test.ts 5.7 KB
- 📁 extension/
  - 📄 consent.test.ts 1.8 KB
  - 📄 domain-tracking.test.ts 1.0 KB
  - 📄 index-store.test.ts 5.4 KB
  - 📄 index-types.test.ts 1.8 KB
  - 📄 install.test.ts 2.7 KB
  - 📄 lifecycle.test.ts 2.9 KB
  - 📄 multi-domain.test.ts 1.5 KB
  - 📄 observer.test.ts 5.4 KB
  - 📄 promotion.test.ts 1.7 KB
  - 📄 security.test.ts 4.0 KB
  - 📄 sensitive-paths.test.ts 1.6 KB
- 📁 index/
  - 📄 reader.test.ts 2.7 KB
- 📁 inspect/
  - 📄 report.test.ts 7.0 KB
- 📁 mcp/
  - 📄 batch.test.ts 4.8 KB
  - 📄 browse.test.ts 4.7 KB
  - 📄 capture-mcp.test.ts 11.5 KB
  - 📄 mcp.test.ts 12.3 KB
- 📁 orchestration/
  - 📄 browse.test.ts 11.4 KB
  - 📄 cache.test.ts 2.5 KB
- 📁 plugin/
  - 📄 plugin.test.ts 7.8 KB
- 📁 read/
  - 📁 decoders/
    
    📄 deepwiki.test.ts 10.6 KB
    
    📄 grokipedia.test.ts 13.0 KB
    
    📄 hackernews.test.ts 6.5 KB
    
    📄 reddit-third-party.test.ts 1.5 KB
    
    📄 reddit.test.ts 5.4 KB
    
    📄 twitter.test.ts 13.6 KB
    
    📄 wikipedia.test.ts 4.6 KB
    
    📄 youtube.test.ts 4.6 KB
  - 📄 extract.test.ts 9.5 KB
  - 📄 peek.test.ts 6.0 KB
  - 📄 read.test.ts 4.1 KB
- 📁 replay/
  - 📄 batch.test.ts 7.3 KB
  - 📄 engine.test.ts 40.7 KB
  - 📄 truncate.test.ts 5.0 KB
- 📁 security/
  - 📄 audit-fixes.test.ts 9.9 KB
  - 📄 credential-leak.test.ts 19.1 KB
  - 📄 dns-rebinding.test.ts 3.6 KB
  - 📄 header-injection.test.ts 6.9 KB
  - 📄 path-traversal.test.ts 1.8 KB
  - 📄 redirect-ssrf.test.ts 4.4 KB
  - 📄 session-navigation.test.ts 3.1 KB
  - 📄 ssrf-oauth.test.ts 5.6 KB
  - 📄 ssrf-replay.test.ts 2.4 KB
  - 📄 timeout.test.ts 1.8 KB
- 📁 serve/
  - 📄 serve.test.ts 7.0 KB
- 📁 skill/
  - 📄 generator.test.ts 25.0 KB
  - 📄 importer.test.ts 4.0 KB
  - 📄 search.test.ts 4.5 KB
  - 📄 signing-enforcement.test.ts 3.5 KB
  - 📄 signing.test.ts 2.2 KB
  - 📄 ssrf.test.ts 5.1 KB
  - 📄 store-verify.test.ts 3.4 KB
  - 📄 store.test.ts 3.4 KB
  - 📄 validate.test.ts 5.4 KB
- 📁 smoke/
  - 📄 capture-mcp-smoke.ts 4.8 KB
- 📁 stats/
  - 📄 report.test.ts 6.2 KB
- 📁 types/
  - 📄 types.test.ts 3.1 KB
- 📄 native-host-index.test.ts 2.7 KB
- 📄 native-host.test.ts 8.2 KB
- 📄 types.test.ts 1.6 KB
📄 .gitignore 971 B
📄 .traction-baseline 498 B
📄 CLAUDE.md 5.4 KB
📄 CONTRIBUTING.md 1.8 KB
📄 launch-baseline.md 318 B
📄 launch-traction.md 1.7 KB
📄 LICENSE 3.1 KB
📄 package-lock.json 57.9 KB
📄 package.json 1.6 KB
📄 README.md 17.0 KB
📄 ROADMAP.md 2.2 KB
📄 SECURITY.md 1.8 KB
📄 SKILL.md 9.3 KB
📄 TRACTION.md 684 B
📄 tsconfig.json 352 B

SKILL.md

# ApiTap — The MCP Server That Turns Any Website Into an API

> No docs, no SDK, no browser. Just data.

## What It Does

ApiTap gives AI agents cheap access to web data through three layers:

1. **Read** — Decode any URL into structured text without a browser (side-channel APIs, og: tags, HTML extraction). 0-10K tokens vs 50-200K for browser automation.
2. **Replay** — Call captured API endpoints directly. 1-5K tokens per call.
3. **Capture** — Record API traffic from a headless browser session, generating reusable skill files.

## MCP Tools (12)

### Tier 0: Triage (free)

#### `apitap_peek`
Zero-cost URL triage. HTTP HEAD only — checks accessibility, bot protection, framework detection.
```
apitap_peek(url: string) → PeekResult
```
**Use when:** You want to know if a site is accessible before spending tokens. Check bot protection, detect frameworks.

**Returns:** `{ status, accessible, server, framework, botProtection, signals[], recommendation }`

`recommendation` is one of: `read` | `capture` | `auth_required` | `blocked`

**Example:**
```
apitap_peek("https://www.zillow.com") → { status: 200, recommendation: "read" }
apitap_peek("https://www.doordash.com") → { status: 403, botProtection: "cloudflare", recommendation: "blocked" }
```

### Tier 1: Read (0-10K tokens, no browser)

#### `apitap_read`
Extract content from any URL without a browser. Uses side-channel APIs for known sites and HTML extraction for everything else.
```
apitap_read(url: string, maxBytes?: number) → ReadResult
```
**Use when:** You need page content, article text, post data, or listing info. Always try this before capture.

**Returns:** `{ title, author, description, content (markdown), links[], images[], metadata: { source, type, publishedAt }, cost: { tokens } }`

**Site-specific decoders (free, structured):**
| Site | Side Channel | What You Get |
|------|-------------|-------------|
| Reddit | `.json` suffix | Posts, scores, comments, authors — full structured data |
| YouTube | oembed API | Title, author, channel, thumbnail |
| Wikipedia | REST API | Article summary, structured, with edit dates |
| Hacker News | Firebase API | Stories, scores, comments, real-time |
| Grokipedia | xAI public API | Full articles with citations, search, 6M+ articles |
| Twitter/X | fxtwitter API | Full tweets, articles, engagement, quotes, media |
| Everything else | og: tags + HTML extraction | Title, content as markdown, links, images |

**Examples:**
```
# Reddit — full subreddit listing, ~500 tokens
apitap_read("https://www.reddit.com/r/technology")

# Reddit post with comments
apitap_read("https://www.reddit.com/r/wallstreetbets/comments/abc123/some-post")

# YouTube — 36 tokens
apitap_read("https://www.youtube.com/watch?v=dQw4w9WgXcQ")

# Wikipedia — 116 tokens
apitap_read("https://en.wikipedia.org/wiki/Artificial_intelligence")

# Grokipedia — full article with citations, 6M+ articles
apitap_read("https://grokipedia.com/wiki/SpaceX")

# Grokipedia — search across 6M articles
apitap_read("https://grokipedia.com/search?q=artificial+intelligence")

# Grokipedia — site stats and recent activity
apitap_read("https://grokipedia.com/")

# Twitter/X — full tweet with engagement, articles, quotes
apitap_read("https://x.com/elonmusk/status/123456789")

# Twitter/X article (long-form post) — full text extracted
apitap_read("https://twitter.com/writer/status/987654321")

# Any article/blog/news — generic extraction
apitap_read("https://example.com/blog/some-article")

# Zillow listing (bypasses PerimeterX via og: tags)
apitap_read("https://www.zillow.com/homedetails/123-Main-St/12345_zpid/")
```

### Tier 2: Replay (1-5K tokens, needs skill file)

#### `apitap_search`
Find available skill files by domain or keyword.
```
apitap_search(query: string) → { found, results[] }
```
**Use when:** Looking for captured API endpoints. Search by domain name or topic.

#### `apitap_replay`
Call a captured API endpoint directly — no browser needed.
```
apitap_replay(domain: string, endpointId: string, endpointParams?: object, maxBytes?: number) → ReplayResult
```
**Use when:** A skill file exists for this domain. This is the cheapest way to get structured API data.

**Returns:** `{ status, data (JSON), domain, endpointId, tier, fromCache }`

**Example:**
```
# Get live stock quote (Robinhood, no auth needed)
apitap_replay("api.robinhood.com", "get-marketdata-quotes", { symbols: "TSLA,MSFT" })

# Get NBA scores (ESPN)
apitap_replay("site.api.espn.com", "get-apis-personalized-v2-scoreboard-header")

# Get crypto trending (CoinMarketCap)
apitap_replay("api.coinmarketcap.com", "get-data-api-v3-unified-trending-top-boost-listing")
```

#### `apitap_replay_batch`
Replay multiple endpoints in one call.
```
apitap_replay_batch(requests: Array<{ domain, endpointId, endpointParams? }>, maxBytes?: number)
```

### Tier 3: Capture (15-20K tokens, uses browser)

#### `apitap_capture`
Launch a headless browser to capture API traffic from a website.
```
apitap_capture(url: string, duration?: number) → { sessionId }
```
**Use when:** No skill file exists and `apitap_read` doesn't give you the data you need. This is expensive but creates a skill file for future free replays.

#### `apitap_capture_interact`
Send browser commands during an active capture session.
```
apitap_capture_interact(sessionId: string, action: string, ...) → result
```
Actions: `click`, `type`, `navigate`, `snapshot`, `scroll`, `wait`

#### `apitap_capture_finish`
End capture session, generate skill file, verify endpoints.
```
apitap_capture_finish(sessionId: string) → { skillFile, endpoints[] }
```

### Auto-Router

#### `apitap_browse`
Automatic escalation: cache → skill file → discover → **read** → capture_needed.
```
apitap_browse(url: string, query?: string, maxBytes?: number) → result
```
**Use when:** You don't know which tier to use. This tries the cheapest option first and escalates automatically.

### Inspection

#### `apitap_inspect`
Get details about a skill file's endpoints.
```
apitap_inspect(domain: string) → { endpoints[], metadata }
```

#### `apitap_stats`
Usage statistics across all skill files.
```
apitap_stats() → { domains, endpoints, tiers }
```

## Decision Tree

```
Need web data?
│
├─ Know the domain? → apitap_search → found? → apitap_replay (cheapest)
│
├─ Unknown URL → apitap_peek first (free)
│   ├─ recommendation: "blocked" → STOP, tell user
│   ├─ recommendation: "read" → apitap_read (no browser)
│   ├─ recommendation: "capture" → apitap_capture (browser)
│   └─ recommendation: "auth_required" → needs human login
│
├─ Just need article/post content → apitap_read directly
│
└─ Need structured API data → apitap_capture → creates skill file → future replays free
```

## Key Patterns

### Instagram profile data (login wall bypass)
Instagram blocks all normal scraping (Googlebot UA, oembed, noembed). But the **mobile API** works:
```bash
curl -s 'https://i.instagram.com/api/v1/users/web_profile_info/?params={"user_name":"TARGET_USERNAME"}' \
  -H 'User-Agent: Instagram 275.0.0.27.98 Android (33/13; 420dpi; 1080x2400; samsung; SM-G991B; o1s; exynos2100)' \
  -H 'X-IG-App-ID: 936619743392459'
```
**Returns:** Full profile JSON — bio, follower/following counts, post count, contact info (email, phone), category, highlights, recent posts with captions/engagement.

**When to use:** Need Instagram profile data, follower counts, contact info, or recent post summaries. Works without auth.

**Limitations:** Only public profiles. Rate-limited if abused. Does NOT return full post feeds — just recent edge.

### Morning news scan
```
# Scan multiple subreddits
for sub in ["technology", "wallstreetbets", "privacy"]:
    apitap_read(f"https://www.reddit.com/r/{sub}")
```

### Stock research
```
# Live quote via captured API
apitap_replay("api.robinhood.com", "get-marketdata-quotes", { symbols: "TSLA" })

# Company fundamentals
apitap_replay("api.robinhood.com", "get-fundamentals", { symbol: "TSLA" })
```

### Research any topic (dual knowledge base)
```
# 1. Read Wikipedia summary (established knowledge)
apitap_read("https://en.wikipedia.org/wiki/Topic")

# 2. Read Grokipedia article (AI-curated, with citations)
apitap_read("https://grokipedia.com/wiki/Topic")

# 3. Check Reddit discussion (community sentiment)
apitap_read("https://www.reddit.com/r/relevant_sub")

# 4. Read a linked article
apitap_read("https://news-site.com/article")
```

### Check before committing
```
# Peek first — is it worth reading?
result = apitap_peek("https://some-site.com")
if result.recommendation == "read":
    apitap_read("https://some-site.com")
elif result.recommendation == "blocked":
    # Don't waste tokens
    pass
```

## Token Economics

| Method | Cost per page | Notes |
|--------|-------------|-------|
| Browser automation | 50-200K tokens | Full DOM serialization |
| apitap_read | 0-10K tokens | No browser, side channels |
| apitap_replay | 1-5K tokens | Direct API call, needs skill file |
| apitap_peek | ~0 tokens | HEAD request only |

## CLI Usage

All MCP tools are also available as CLI commands:
```bash
apitap peek <url> [--json]
apitap read <url> [--json] [--max-bytes <n>]
apitap search <query> [--json]
apitap replay <domain> <endpointId> [--params '{}'] [--json]
apitap capture <url> [--duration <sec>] [--json]
apitap inspect <domain> [--json]
apitap stats [--json]
```

Every command supports `--json` for machine-readable output.

登录后下载/点赞/收藏 ❤ 73 | ★ 0

apitap

目录结构

SKILL.md

举报内容

提示