twag

Category: Data & AI | Uploader: clifton | Downloads: 0 | Version: v1.0（Latest）

Twitter/X aggregator for market-relevant signals with LLM-powered scoring. Fetches, processes, and curates tweets for market analysis.

Changelog: Source: GitHub https://github.com/clifton/twag

Directory Structure

Current level: Root

📁 .claude/
- 📄 settings.local.json 208 B
📁 .githooks/
- 📄 pre-commit 467 B
📁 .github/
- 📁 workflows/
  - 📄 auto-release.yml 1.7 KB
  - 📄 ci.yml 1.9 KB
  - 📄 release.yml 1.2 KB
- 📄 release.yml 817 B
📁 scripts/
- 📄 benchmark_parallelism.py 7.1 KB
- 📄 cron-runner.sh 4.3 KB
- 📄 migrate.sh 2.7 KB
- 📄 test_models.py 4.3 KB
📁 tests/
- 📁 fixtures/
  - 📄 tweet_2019488673935552978.json 74.5 KB
- 📄 __init__.py 22 B
- 📄 conftest.py 179 B
- 📄 test_article_processing.py 9.9 KB
- 📄 test_article_sections.py 2.4 KB
- 📄 test_article_visuals.py 3.1 KB
- 📄 test_cli_analyze_status.py 10.8 KB
- 📄 test_cli_fetch_status.py 3.2 KB
- 📄 test_cli_process_status.py 3.4 KB
- 📄 test_cli_search.py 11.8 KB
- 📄 test_db_dump_restore.py 7.8 KB
- 📄 test_db_retweet_backfill.py 4.3 KB
- 📄 test_fetcher.py 29.7 KB
- 📄 test_link_utils.py 5.8 KB
- 📄 test_processor.py 11.4 KB
- 📄 test_processor_parallelization.py 15.1 KB
- 📄 test_renderer_article_sections.py 4.5 KB
- 📄 test_renderer_links.py 4.3 KB
- 📄 test_tables.py 6.5 KB
- 📄 test_web_tweets_api.py 14.5 KB
📁 tmp/
- 📄 .gitkeep 0 B
📁 twag/
- 📁 cli/
  - 📄 __init__.py 1.7 KB
  - 📄 _console.py 285 B
  - 📄 _helpers.py 843 B
  - 📄 _progress.py 2.4 KB
  - 📄 accounts.py 4.6 KB
  - 📄 analyze.py 7.2 KB
  - 📄 config_cmd.py 1.2 KB
  - 📄 db_cmd.py 4.3 KB
  - 📄 digest.py 988 B
  - 📄 fetch.py 7.9 KB
  - 📄 init_cmd.py 7.3 KB
  - 📄 narratives.py 865 B
  - 📄 process.py 6.5 KB
  - 📄 search.py 9.8 KB
  - 📄 stats.py 3.5 KB
  - 📄 web.py 3.2 KB
- 📁 db/
  - 📄 __init__.py 3.7 KB
  - 📄 accounts.py 4.3 KB
  - 📄 connection.py 8.0 KB
  - 📄 context_commands.py 3.3 KB
  - 📄 maintenance.py 6.5 KB
  - 📄 narratives.py 2.0 KB
  - 📄 prompts.py 7.5 KB
  - 📄 reactions.py 3.7 KB
  - 📄 schema.py 7.0 KB
  - 📄 search.py 14.9 KB
  - 📄 time_utils.py 4.0 KB
  - 📄 tweets.py 23.3 KB
- 📁 fetcher/
  - 📄 __init__.py 503 B
  - 📄 bird_cli.py 9.8 KB
  - 📄 extractors.py 17.9 KB
- 📁 models/
  - 📄 __init__.py 1.6 KB
  - 📄 api.py 2.6 KB
  - 📄 config.py 2.4 KB
  - 📄 db_models.py 2.7 KB
  - 📄 links.py 744 B
  - 📄 media.py 738 B
  - 📄 scoring.py 1.9 KB
  - 📄 tweet.py 1.2 KB
- 📁 processor/
  - 📄 __init__.py 2.0 KB
  - 📄 dependencies.py 15.3 KB
  - 📄 pipeline.py 14.0 KB
  - 📄 storage.py 7.3 KB
  - 📄 triage.py 30.5 KB
- 📁 scorer/
  - 📄 __init__.py 943 B
  - 📄 llm_client.py 7.1 KB
  - 📄 prompts.py 5.2 KB
  - 📄 scoring.py 12.4 KB
- 📁 web/
  - 📁 frontend/
    
    📁 src/
    
    📁 api/
    
    📄 client.ts 1.3 KB
    
    📄 context.ts 1.4 KB
    
    📄 prompts.ts 1.3 KB
    
    📄 reactions.ts 846 B
    
    📄 tweets.ts 600 B
    
    📄 types.ts 5.3 KB
    
    📁 components/
    
    📁 context/
    
    📄 CommandForm.tsx 4.9 KB
    
    📄 CommandList.tsx 3.4 KB
    
    📄 CommandTest.tsx 4.9 KB
    
    📄 ContextPage.tsx 2.0 KB
    
    📁 feed/
    
    📄 articleVisuals.ts 3.6 KB
    
    📄 FeedPage.tsx 3.2 KB
    
    📄 QuoteBlock.tsx 1.2 KB
    
    📄 ReactionModal.tsx 1.9 KB
    
    📄 ScoreBadge.tsx 1.4 KB
    
    📄 TweetActions.tsx 4.2 KB
    
    📄 TweetCard.tsx 13.8 KB
    
    📄 TweetContent.tsx 3.1 KB
    
    📄 TweetMedia.tsx 5.4 KB
    
    📁 layout/
    
    📄 AppShell.tsx 1.8 KB
    
    📄 FilterBar.tsx 5.4 KB
    
    📁 prompts/
    
    📄 PromptEditor.tsx 3.2 KB
    
    📄 PromptHistory.tsx 2.4 KB
    
    📄 PromptsPage.tsx 3.4 KB
    
    📄 TuningSuggestion.tsx 3.1 KB
    
    📁 ui/
    
    📄 badge.tsx 1005 B
    
    📄 button.tsx 1.8 KB
    
    📄 dialog.tsx 3.1 KB
    
    📄 input.tsx 654 B
    
    📄 select.tsx 3.2 KB
    
    📄 skeleton.tsx 264 B
    
    📄 switch.tsx 1.0 KB
    
    📄 textarea.tsx 660 B
    
    📄 toaster.tsx 1.8 KB
    
    📄 tooltip.tsx 980 B
    
    📁 hooks/
    
    📄 use-categories.ts 405 B
    
    📄 use-context-commands.ts 1.8 KB
    
    📄 use-filter-state.ts 2.1 KB
    
    📄 use-prompts.ts 1.7 KB
    
    📄 use-reactions.ts 838 B
    
    📄 use-tweets.ts 508 B
    
    📁 lib/
    
    📄 utils.ts 2.0 KB
    
    📁 styles/
    
    📄 globals.css 1.4 KB
    
    📄 App.tsx 680 B
    
    📄 main.tsx 618 B
    
    📄 biome.json 1.0 KB
    
    📄 index.html 663 B
    
    📄 package-lock.json 141.0 KB
    
    📄 package.json 1.3 KB
    
    📄 tsconfig.json 614 B
    
    📄 tsconfig.tsbuildinfo 1.5 KB
    
    📄 vite.config.ts 578 B
  - 📁 routes/
    
    📄 __init__.py 146 B
    
    📄 context.py 11.8 KB
    
    📄 prompts.py 8.0 KB
    
    📄 reactions.py 5.4 KB
    
    📄 tweets.py 19.6 KB
  - 📁 templates/
    
    📄 .gitkeep 0 B
  - 📄 __init__.py 85 B
  - 📄 app.py 2.4 KB
  - 📄 tweet_utils.py 2.1 KB
- 📄 __init__.py 207 B
- 📄 article_sections.py 2.4 KB
- 📄 article_visuals.py 5.1 KB
- 📄 auth.py 1.5 KB
- 📄 config.py 4.8 KB
- 📄 link_utils.py 11.6 KB
- 📄 media.py 2.6 KB
- 📄 notifier.py 4.2 KB
- 📄 py.typed 0 B
- 📄 renderer.py 12.6 KB
- 📄 tables.py 926 B
📄 .gitignore 791 B
📄 AGENTS.md 103 B
📄 CLAUDE.md 4.7 KB
📄 CONTRIBUTING.md 1.2 KB
📄 INSTALL.md 4.9 KB
📄 LICENSE 1.0 KB
📄 pyproject.toml 3.6 KB
📄 README.md 11.7 KB
📄 SKILL.md 8.0 KB
📄 SUGGESTED_CRON_SCHEDULE.md 8.0 KB
📄 TELEGRAM_DIGEST_FORMAT.md 4.4 KB
📄 uv.lock 332.7 KB

SKILL.md

---
name: twag
description: Twitter/X aggregator for market-relevant signals with LLM-powered scoring. Fetches, processes, and curates tweets for market analysis.
read_when:
  - Processing Twitter/X feed for market signals
  - Searching for market-relevant tweets
  - Generating daily tweet digests
  - Managing followed Twitter accounts
  - Finding tweets about specific tickers or topics
homepage: https://github.com/clifton/twag
metadata:
  openclaw:
    emoji: "📊"
    requires:
      bins: ["twag", "bird"]
      env: ["GEMINI_API_KEY", "AUTH_TOKEN", "CT0"]
    install:
      - id: pip
        kind: pip
        package: twag
        bins: ["twag"]
        label: "Install twag (pip)"
      - id: bird-npm
        kind: node
        package: "@steipete/bird"
        bins: ["bird"]
        label: "Install bird CLI (npm)"
      - id: bird-brew
        kind: brew
        formula: "steipete/tap/bird"
        bins: ["bird"]
        label: "Install bird CLI (brew)"
        os: ["darwin"]
allowed-tools: Bash(twag:*), Bash(bird:*)
---

# twag — Twitter/X Market Signal Aggregator

## Installation

### Step 1: Install bird CLI (Twitter access)

bird is required for Twitter/X API access. Install via npm:

```bash
npm install -g @steipete/bird
```

Or on macOS via Homebrew:

```bash
brew install steipete/tap/bird
```

Verify: `bird --version`

### Step 2: Configure Twitter auth

bird uses cookie-based auth. The user must provide two cookies from their browser:

- `AUTH_TOKEN` — The `auth_token` cookie from x.com
- `CT0` — The `ct0` cookie from x.com

Set as environment variables:

```bash
export AUTH_TOKEN="..."
export CT0="..."
```

Or add to `~/.env` (twag sources this automatically).

Verify auth: `bird whoami`

### Step 3: Install twag

```bash
pip install twag
```

Or from source:

```bash
git clone https://github.com/clifton/twag.git
cd twag && pip install -e .
```

### Step 4: Initialize and verify

```bash
twag init
twag doctor
```

### Step 5: Set LLM API key

```bash
export GEMINI_API_KEY="..."  # Required
export ANTHROPIC_API_KEY="..."  # Optional, for enrichment
```

## Quick Reference

```bash
# Full pipeline
twag fetch && twag process && twag digest

# Browse high-signal tweets
twag search --today -s 7

# Rich JSON for digest/agent consumption
twag search --time 2h -s 6 -f json -n 50

# Check stats
twag stats --today
```

## Common Workflows

### Morning market check

```bash
twag fetch && twag process
twag search --today -s 7
twag digest --stdout
```

### Track specific topic

```bash
twag search "fed rate" -c fed_policy --today
twag search "powell" -a NickTimiraos --time 7d
```

### Monitor ticker

```bash
twag search "earnings" --ticker NVDA --today
twag search "AAPL" -s 6 --time 7d
```

### Analyze single tweet

```bash
twag analyze https://x.com/user/status/123456789
twag analyze 123456789 --reprocess  # Force re-analyze
```

## Commands

### Search (most common)

```bash
# Browse mode (no query) — uses rich FeedTweet data
twag search --today -s 7             # High-signal tweets since market close
twag search --time 2h -s 6 -f json -n 50  # Rich JSON for digests
twag search -c fed_policy --time 7d  # Browse by category

# Full-text search mode
twag search "query"                  # FTS5 search
twag search "query" -c fed_policy    # Filter by category
twag search "query" -a handle        # Filter by author
twag search "query" --ticker AAPL    # Filter by ticker
twag search "query" --today          # Since last market close
twag search "query" --time 7d        # Last N days
twag search "query" -s 7             # Min score threshold
twag search "query" --format full    # Digest-style output
twag search "query" --format json    # JSON output
```

**Query syntax:**
- Simple: `inflation fed` (matches both)
- Phrase: `"rate hike"` (exact)
- Boolean: `inflation AND fed`, `fed NOT fomc`
- Prefix: `infla*` (wildcard)

### Fetch & Process

```bash
twag fetch                    # Home + tier-1 + bookmarks
twag fetch --no-tier1         # Home only
twag fetch -u @handle         # Specific user
twag fetch --source search -q "query"  # Search tweets

twag process                  # Score unprocessed
twag process -n 100           # Limit batch
twag process --dry-run        # Preview
twag process --no-notify      # Skip alerts
```

### Digest

```bash
twag digest                   # Today's digest (saves to file)
twag digest --stdout          # Output to terminal
twag digest -d 2026-01-29     # Specific date
twag digest --min-score 6     # Custom threshold
```

### Account Management

```bash
twag accounts list            # All accounts
twag accounts list -t 1       # Tier-1 only
twag accounts add @handle     # Add account
twag accounts add @handle -t 1  # Add as tier-1
twag accounts promote @handle # Promote to tier-1
twag accounts demote @handle  # Demote to tier-2
twag accounts mute @handle    # Mute account
twag accounts boost @handle --amount 10
twag accounts decay           # Apply daily decay
twag accounts import          # Import from following.txt
```

### Stats & Maintenance

```bash
twag stats                    # All-time
twag stats --today            # Today
twag prune --days 14          # Delete old tweets
twag export --days 7          # Export recent
```

### Database

```bash
twag db path                  # Show location
twag db shell                 # SQLite shell
twag db init                  # Initialize/reset database
twag db rebuild-fts           # Rebuild search index
twag db dump                  # Backup
twag db dump --stdout         # Backup to stdout
twag db restore backup.sql    # Restore
twag db restore backup.sql --force  # Restore without confirmation
```

### Web UI

```bash
twag web                      # Start (localhost:5173)
twag web --host 127.0.0.1     # Localhost only
twag web --port 8080          # Custom port
```

### Config

```bash
twag config show              # Show config
twag config path              # Show path
twag config set key value     # Update setting
```

## Scoring Tiers

| Score | Level | Behavior |
|-------|-------|----------|
| 8-10 | Alert | Telegram alert |
| 7 | High signal | Enriched, in digests |
| 5-6 | Market relevant | In digests |
| 3-4 | News/context | Searchable |
| 0-2 | Noise | Filtered out |

## Categories

`fed_policy`, `inflation`, `job_market`, `macro_data`, `earnings`, `equities`, `rates_fx`, `credit`, `banks`, `consumer_spending`, `capex`, `commodities`, `energy`, `metals_mining`, `geopolitical`, `sanctions`, `tech_business`, `ai_advancement`, `crypto`, `noise`

## Environment Variables

| Variable | Required | Purpose |
|----------|----------|---------|
| `AUTH_TOKEN` | Yes | Twitter auth cookie |
| `CT0` | Yes | Twitter ct0 cookie |
| `GEMINI_API_KEY` | Yes | LLM triage |
| `ANTHROPIC_API_KEY` | No | Enrichment |
| `TELEGRAM_BOT_TOKEN` | No | Alerts |
| `TELEGRAM_CHAT_ID` | No | Alert destination |

## Telegram Digest Format

When sending digests to Telegram, follow [{baseDir}/TELEGRAM_DIGEST_FORMAT.md]({baseDir}/TELEGRAM_DIGEST_FORMAT.md):

- Use `twag search --time Xh -s 6 -f json -n 50` for structured input
- Group tweets by theme (don't list chronologically)
- Use `**BOLD CAPS**` for section headers (no markdown `###`)
- Use `•` for bullet points
- Citations: `[📊](url)` when `has_media: true`, `[🔗](url)` otherwise
- Condense multiple tweets on same topic into bullets
- Extract key facts and numbers
- Use `delivery.mode: "direct"` in cron jobs to preserve `[🔗](url)` and `[📊](url)` links

## Automation

See [{baseDir}/SUGGESTED_CRON_SCHEDULE.md]({baseDir}/SUGGESTED_CRON_SCHEDULE.md) for:

- systemd/launchd timers for data collection (every 15 min)
- OpenClaw cron jobs for Telegram digest delivery

## Troubleshooting

### bird not found

```bash
npm install -g @steipete/bird
```

### Auth errors (401)

```bash
# Check cookies are set
bird whoami

# If expired, get fresh cookies from x.com
```

### Query ID errors (404)

```bash
bird query-ids --fresh
```

### Database issues

```bash
twag db rebuild-fts
```

## Links

- **twag:** https://github.com/clifton/twag
- **bird:** https://github.com/steipete/bird
- **OpenClaw:** https://github.com/openclaw/openclaw

Comments 0

Latest | Hot

Please login before commenting.

No comments yet. Be the first one!

twag

Directory Structure

SKILL.md

Report

Notice