Browser Automation

Category: Databases & Storage | Uploader: spamsch | Downloads: 0 | Version: v1.0（Latest）

Automate web interactions for bookings, form submissions, and purchases using ARIA-based Safari control.

Changelog: Source: GitHub https://github.com/spamsch/son-of-simon

Directory Structure

Current level: skills/browser_automation/

📄 SKILL.md 3.8 KB

SKILL.md

---
id: browser_automation
name: Browser Automation
description: Automate web interactions for bookings, form submissions, and purchases using ARIA-based Safari control.
apps:
  - Safari
tasks:
  - browser_navigate
  - browser_snapshot
  - browser_click
  - browser_type
  - browser_select
  - browser_scroll_to
  - browser_get_text
  - browser_execute_js
  - browser_screenshot
  - browser_close_tab
  - browser_dismiss_cookies
  - browser_wait
  - browser_press_key
  - browser_visual_snapshot
  - analyze_screenshot
essential_tasks: []
keywords: ["browser", "webseite", "formular", "buchen", "buchung"]
examples:
  - "Book a table at Restaurant XYZ for Friday at 7pm"
  - "Make a reservation for 2 people tomorrow night"
  - "Buy tickets for the concert next week"
  - "Fill out the contact form on this page"
  - "Submit the inquiry form with my details"
safe_defaults:
  timeout: 20
  max_elements: 200
confirm_before_write:
  - submit form
  - complete purchase
  - confirm booking
requires_permissions:
  - Automation:Safari
  - Allow JavaScript from Apple Events
---

## Behavior Notes

### When to Use Browser Automation
Use browser_* tools for interactive web tasks:
- Making reservations or bookings
- Filling out and submitting forms
- Completing purchases
- Navigating multi-step workflows

For simple lookups (reading content, searching), use web_search and web_fetch instead.

### The ARIA Workflow
Browser automation uses semantic element references (e1, e2, etc.):

1. **Navigate**: `browser_navigate` to load the page
2. **Snapshot**: `browser_snapshot` to see interactive elements with refs
3. **Interact**: Use refs with `browser_click`, `browser_type`, `browser_select`
4. **Verify**: Take another snapshot to confirm changes

### Making Online Bookings/Reservations

When asked to book, reserve, or purchase something online, ALWAYS try to complete it.

**First, verify required info**: date, exact time, party size, name, contact. Ask if anything is missing or vague.

**Then proceed with the booking:**

1. **Find the website**: Use web_search to find the official site
2. **Quick recon**: Use web_fetch to quickly scan the page for booking options
3. **Switch to browser**: Use browser_navigate to go to the booking page
4. **Take a snapshot**: Use browser_snapshot to see elements with refs (e1, e2, etc.)
5. **Fill forms**: Use browser_type to enter details (name, date, time, party size, etc.)
6. **Click buttons**: Use browser_click to select options, proceed, submit
7. **Verify completion**: Take another snapshot to confirm the booking went through

### Key Insights

- **web_fetch first**: Great for quickly checking what's available
- **browser_* to interact**: Must continue with browser tools to fill forms and click buttons
- **Don't stop early**: The user asked to BOOK, not just find contact info

### Being Persistent

If one approach doesn't work, try alternatives:
- Look for "Reservierung", "Buchen", "Book", "Reserve" buttons/links
- Try contact forms if no booking system exists
- Fill out and submit inquiry forms on the user's behalf
- As a last resort, draft an email for the user to send

### Handling Common Issues

- **Cookie banners**: Use `browser_dismiss_cookies` to clear consent popups
- **Overlays/popups**: Use `browser_press_key` with "escape" to dismiss
- **Dynamic content**: Use `browser_wait` after clicks for AJAX to complete
- **Can't find element**: Try `browser_scroll_to` or take a new snapshot
- **Visual debugging**: Use `browser_visual_snapshot` + `analyze_screenshot` for complex pages

### Form Filling Best Practices

- Use `browser_type` with `clear=true` (default) to replace existing values
- For dropdowns, use `browser_select` with the option value
- Use `browser_type` with `submit=true` to press Enter after typing
- Always snapshot after major interactions to verify state

Comments 0

Latest | Hot

Please login before commenting.

No comments yet. Be the first one!

Browser Automation

Directory Structure

SKILL.md

Report

Notice