feat(skills): Add hierarchical structure with references and templates (#157)
* feat(skills): Add hierarchical structure with references and templates Adds modular documentation and executable templates to the agent-browser skill for better AI agent consumption and progressive disclosure. ## Added ### References (deep-dive documentation) - `references/snapshot-refs.md` - Ref lifecycle, invalidation, troubleshooting - `references/session-management.md` - Parallel sessions, state persistence - `references/authentication.md` - Login flows, OAuth, 2FA patterns - `references/video-recording.md` - Recording for debugging/docs - `references/proxy-support.md` - Proxy configuration, geo-testing ### Templates (ready-to-use workflows) - `templates/form-automation.sh` - Form filling with validation - `templates/authenticated-session.sh` - Login once, reuse state - `templates/capture-workflow.sh` - Content extraction with screenshots ## Modified - `SKILL.md` - Added reference tables linking to new documentation ## Benefits - Progressive disclosure: Load overview first, deep dives on demand - Reduced context: Smaller chunks for better LLM token efficiency - Ready workflows: Copy-paste templates for common patterns * fix(templates): Make authenticated-session.sh runnable out-of-box Addresses review feedback: login actions were commented but verification wasn't, causing script to fail when run as-is. New approach: - DISCOVERY MODE runs first (shows form structure) - LOGIN FLOW section is fully commented as a unit - User runs once to see refs, then customizes ┌─────────────────────────────────────────────────────────────┐ │ LOGIN FORM STRUCTURE │ ├─────────────────────────────────────────────────────────────┤ │ @e1 [input type="email"] │ │ @e2 [input type="password"] │ │ @e3 [button] "Sign In" │ └─────────────────────────────────────────────────────────────┘
This commit is contained in:
Yonatan
@@ -316,3 +316,32 @@ agent-browser trace stop trace.zip # Stop and save trace
|
||||
agent-browser record start ./debug.webm # Record video from current page
|
||||
agent-browser record stop # Save recording
|
||||
```
|
||||
|
||||
## Deep-dive documentation
|
||||
|
||||
For detailed patterns and best practices, see:
|
||||
|
||||
| Reference | Description |
|
||||
|-----------|-------------|
|
||||
| [references/snapshot-refs.md](references/snapshot-refs.md) | Ref lifecycle, invalidation rules, troubleshooting |
|
||||
| [references/session-management.md](references/session-management.md) | Parallel sessions, state persistence, concurrent scraping |
|
||||
| [references/authentication.md](references/authentication.md) | Login flows, OAuth, 2FA handling, state reuse |
|
||||
| [references/video-recording.md](references/video-recording.md) | Recording workflows for debugging and documentation |
|
||||
| [references/proxy-support.md](references/proxy-support.md) | Proxy configuration, geo-testing, rotating proxies |
|
||||
|
||||
## Ready-to-use templates
|
||||
|
||||
Executable workflow scripts for common patterns:
|
||||
|
||||
| Template | Description |
|
||||
|----------|-------------|
|
||||
| [templates/form-automation.sh](templates/form-automation.sh) | Form filling with validation |
|
||||
| [templates/authenticated-session.sh](templates/authenticated-session.sh) | Login once, reuse state |
|
||||
| [templates/capture-workflow.sh](templates/capture-workflow.sh) | Content extraction with screenshots |
|
||||
|
||||
Usage:
|
||||
```bash
|
||||
./templates/form-automation.sh https://example.com/form
|
||||
./templates/authenticated-session.sh https://app.example.com/login
|
||||
./templates/capture-workflow.sh https://example.com ./output
|
||||
```
|
||||
|
||||
@@ -0,0 +1,188 @@
|
||||
# Authentication Patterns
|
||||
|
||||
Patterns for handling login flows, session persistence, and authenticated browsing.
|
||||
|
||||
## Basic Login Flow
|
||||
|
||||
```bash
|
||||
# Navigate to login page
|
||||
agent-browser open https://app.example.com/login
|
||||
agent-browser wait --load networkidle
|
||||
|
||||
# Get form elements
|
||||
agent-browser snapshot -i
|
||||
# Output: @e1 [input type="email"], @e2 [input type="password"], @e3 [button] "Sign In"
|
||||
|
||||
# Fill credentials
|
||||
agent-browser fill @e1 "user@example.com"
|
||||
agent-browser fill @e2 "password123"
|
||||
|
||||
# Submit
|
||||
agent-browser click @e3
|
||||
agent-browser wait --load networkidle
|
||||
|
||||
# Verify login succeeded
|
||||
agent-browser get url # Should be dashboard, not login
|
||||
```
|
||||
|
||||
## Saving Authentication State
|
||||
|
||||
After logging in, save state for reuse:
|
||||
|
||||
```bash
|
||||
# Login first (see above)
|
||||
agent-browser open https://app.example.com/login
|
||||
agent-browser snapshot -i
|
||||
agent-browser fill @e1 "user@example.com"
|
||||
agent-browser fill @e2 "password123"
|
||||
agent-browser click @e3
|
||||
agent-browser wait --url "**/dashboard"
|
||||
|
||||
# Save authenticated state
|
||||
agent-browser state save ./auth-state.json
|
||||
```
|
||||
|
||||
## Restoring Authentication
|
||||
|
||||
Skip login by loading saved state:
|
||||
|
||||
```bash
|
||||
# Load saved auth state
|
||||
agent-browser state load ./auth-state.json
|
||||
|
||||
# Navigate directly to protected page
|
||||
agent-browser open https://app.example.com/dashboard
|
||||
|
||||
# Verify authenticated
|
||||
agent-browser snapshot -i
|
||||
```
|
||||
|
||||
## OAuth / SSO Flows
|
||||
|
||||
For OAuth redirects:
|
||||
|
||||
```bash
|
||||
# Start OAuth flow
|
||||
agent-browser open https://app.example.com/auth/google
|
||||
|
||||
# Handle redirects automatically
|
||||
agent-browser wait --url "**/accounts.google.com**"
|
||||
agent-browser snapshot -i
|
||||
|
||||
# Fill Google credentials
|
||||
agent-browser fill @e1 "user@gmail.com"
|
||||
agent-browser click @e2 # Next button
|
||||
agent-browser wait 2000
|
||||
agent-browser snapshot -i
|
||||
agent-browser fill @e3 "password"
|
||||
agent-browser click @e4 # Sign in
|
||||
|
||||
# Wait for redirect back
|
||||
agent-browser wait --url "**/app.example.com**"
|
||||
agent-browser state save ./oauth-state.json
|
||||
```
|
||||
|
||||
## Two-Factor Authentication
|
||||
|
||||
Handle 2FA with manual intervention:
|
||||
|
||||
```bash
|
||||
# Login with credentials
|
||||
agent-browser open https://app.example.com/login --headed # Show browser
|
||||
agent-browser snapshot -i
|
||||
agent-browser fill @e1 "user@example.com"
|
||||
agent-browser fill @e2 "password123"
|
||||
agent-browser click @e3
|
||||
|
||||
# Wait for user to complete 2FA manually
|
||||
echo "Complete 2FA in the browser window..."
|
||||
agent-browser wait --url "**/dashboard" --timeout 120000
|
||||
|
||||
# Save state after 2FA
|
||||
agent-browser state save ./2fa-state.json
|
||||
```
|
||||
|
||||
## HTTP Basic Auth
|
||||
|
||||
For sites using HTTP Basic Authentication:
|
||||
|
||||
```bash
|
||||
# Set credentials before navigation
|
||||
agent-browser set credentials username password
|
||||
|
||||
# Navigate to protected resource
|
||||
agent-browser open https://protected.example.com/api
|
||||
```
|
||||
|
||||
## Cookie-Based Auth
|
||||
|
||||
Manually set authentication cookies:
|
||||
|
||||
```bash
|
||||
# Set auth cookie
|
||||
agent-browser cookies set session_token "abc123xyz"
|
||||
|
||||
# Navigate to protected page
|
||||
agent-browser open https://app.example.com/dashboard
|
||||
```
|
||||
|
||||
## Token Refresh Handling
|
||||
|
||||
For sessions with expiring tokens:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Wrapper that handles token refresh
|
||||
|
||||
STATE_FILE="./auth-state.json"
|
||||
|
||||
# Try loading existing state
|
||||
if [[ -f "$STATE_FILE" ]]; then
|
||||
agent-browser state load "$STATE_FILE"
|
||||
agent-browser open https://app.example.com/dashboard
|
||||
|
||||
# Check if session is still valid
|
||||
URL=$(agent-browser get url)
|
||||
if [[ "$URL" == *"/login"* ]]; then
|
||||
echo "Session expired, re-authenticating..."
|
||||
# Perform fresh login
|
||||
agent-browser snapshot -i
|
||||
agent-browser fill @e1 "$USERNAME"
|
||||
agent-browser fill @e2 "$PASSWORD"
|
||||
agent-browser click @e3
|
||||
agent-browser wait --url "**/dashboard"
|
||||
agent-browser state save "$STATE_FILE"
|
||||
fi
|
||||
else
|
||||
# First-time login
|
||||
agent-browser open https://app.example.com/login
|
||||
# ... login flow ...
|
||||
fi
|
||||
```
|
||||
|
||||
## Security Best Practices
|
||||
|
||||
1. **Never commit state files** - They contain session tokens
|
||||
```bash
|
||||
echo "*.auth-state.json" >> .gitignore
|
||||
```
|
||||
|
||||
2. **Use environment variables for credentials**
|
||||
```bash
|
||||
agent-browser fill @e1 "$APP_USERNAME"
|
||||
agent-browser fill @e2 "$APP_PASSWORD"
|
||||
```
|
||||
|
||||
3. **Clean up after automation**
|
||||
```bash
|
||||
agent-browser cookies clear
|
||||
rm -f ./auth-state.json
|
||||
```
|
||||
|
||||
4. **Use short-lived sessions for CI/CD**
|
||||
```bash
|
||||
# Don't persist state in CI
|
||||
agent-browser open https://app.example.com/login
|
||||
# ... login and perform actions ...
|
||||
agent-browser close # Session ends, nothing persisted
|
||||
```
|
||||
@@ -0,0 +1,175 @@
|
||||
# Proxy Support
|
||||
|
||||
Configure proxy servers for browser automation, useful for geo-testing, rate limiting avoidance, and corporate environments.
|
||||
|
||||
## Basic Proxy Configuration
|
||||
|
||||
Set proxy via environment variable before starting:
|
||||
|
||||
```bash
|
||||
# HTTP proxy
|
||||
export HTTP_PROXY="http://proxy.example.com:8080"
|
||||
agent-browser open https://example.com
|
||||
|
||||
# HTTPS proxy
|
||||
export HTTPS_PROXY="https://proxy.example.com:8080"
|
||||
agent-browser open https://example.com
|
||||
|
||||
# Both
|
||||
export HTTP_PROXY="http://proxy.example.com:8080"
|
||||
export HTTPS_PROXY="http://proxy.example.com:8080"
|
||||
agent-browser open https://example.com
|
||||
```
|
||||
|
||||
## Authenticated Proxy
|
||||
|
||||
For proxies requiring authentication:
|
||||
|
||||
```bash
|
||||
# Include credentials in URL
|
||||
export HTTP_PROXY="http://username:password@proxy.example.com:8080"
|
||||
agent-browser open https://example.com
|
||||
```
|
||||
|
||||
## SOCKS Proxy
|
||||
|
||||
```bash
|
||||
# SOCKS5 proxy
|
||||
export ALL_PROXY="socks5://proxy.example.com:1080"
|
||||
agent-browser open https://example.com
|
||||
|
||||
# SOCKS5 with auth
|
||||
export ALL_PROXY="socks5://user:pass@proxy.example.com:1080"
|
||||
agent-browser open https://example.com
|
||||
```
|
||||
|
||||
## Proxy Bypass
|
||||
|
||||
Skip proxy for specific domains:
|
||||
|
||||
```bash
|
||||
# Bypass proxy for local addresses
|
||||
export NO_PROXY="localhost,127.0.0.1,.internal.company.com"
|
||||
agent-browser open https://internal.company.com # Direct connection
|
||||
agent-browser open https://external.com # Via proxy
|
||||
```
|
||||
|
||||
## Common Use Cases
|
||||
|
||||
### Geo-Location Testing
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Test site from different regions using geo-located proxies
|
||||
|
||||
PROXIES=(
|
||||
"http://us-proxy.example.com:8080"
|
||||
"http://eu-proxy.example.com:8080"
|
||||
"http://asia-proxy.example.com:8080"
|
||||
)
|
||||
|
||||
for proxy in "${PROXIES[@]}"; do
|
||||
export HTTP_PROXY="$proxy"
|
||||
export HTTPS_PROXY="$proxy"
|
||||
|
||||
region=$(echo "$proxy" | grep -oP '^\w+-\w+')
|
||||
echo "Testing from: $region"
|
||||
|
||||
agent-browser --session "$region" open https://example.com
|
||||
agent-browser --session "$region" screenshot "./screenshots/$region.png"
|
||||
agent-browser --session "$region" close
|
||||
done
|
||||
```
|
||||
|
||||
### Rotating Proxies for Scraping
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Rotate through proxy list to avoid rate limiting
|
||||
|
||||
PROXY_LIST=(
|
||||
"http://proxy1.example.com:8080"
|
||||
"http://proxy2.example.com:8080"
|
||||
"http://proxy3.example.com:8080"
|
||||
)
|
||||
|
||||
URLS=(
|
||||
"https://site.com/page1"
|
||||
"https://site.com/page2"
|
||||
"https://site.com/page3"
|
||||
)
|
||||
|
||||
for i in "${!URLS[@]}"; do
|
||||
proxy_index=$((i % ${#PROXY_LIST[@]}))
|
||||
export HTTP_PROXY="${PROXY_LIST[$proxy_index]}"
|
||||
export HTTPS_PROXY="${PROXY_LIST[$proxy_index]}"
|
||||
|
||||
agent-browser open "${URLS[$i]}"
|
||||
agent-browser get text body > "output-$i.txt"
|
||||
agent-browser close
|
||||
|
||||
sleep 1 # Polite delay
|
||||
done
|
||||
```
|
||||
|
||||
### Corporate Network Access
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Access internal sites via corporate proxy
|
||||
|
||||
export HTTP_PROXY="http://corpproxy.company.com:8080"
|
||||
export HTTPS_PROXY="http://corpproxy.company.com:8080"
|
||||
export NO_PROXY="localhost,127.0.0.1,.company.com"
|
||||
|
||||
# External sites go through proxy
|
||||
agent-browser open https://external-vendor.com
|
||||
|
||||
# Internal sites bypass proxy
|
||||
agent-browser open https://intranet.company.com
|
||||
```
|
||||
|
||||
## Verifying Proxy Connection
|
||||
|
||||
```bash
|
||||
# Check your apparent IP
|
||||
agent-browser open https://httpbin.org/ip
|
||||
agent-browser get text body
|
||||
# Should show proxy's IP, not your real IP
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Proxy Connection Failed
|
||||
|
||||
```bash
|
||||
# Test proxy connectivity first
|
||||
curl -x http://proxy.example.com:8080 https://httpbin.org/ip
|
||||
|
||||
# Check if proxy requires auth
|
||||
export HTTP_PROXY="http://user:pass@proxy.example.com:8080"
|
||||
```
|
||||
|
||||
### SSL/TLS Errors Through Proxy
|
||||
|
||||
Some proxies perform SSL inspection. If you encounter certificate errors:
|
||||
|
||||
```bash
|
||||
# For testing only - not recommended for production
|
||||
agent-browser open https://example.com --ignore-https-errors
|
||||
```
|
||||
|
||||
### Slow Performance
|
||||
|
||||
```bash
|
||||
# Use proxy only when necessary
|
||||
export NO_PROXY="*.cdn.com,*.static.com" # Direct CDN access
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use environment variables** - Don't hardcode proxy credentials
|
||||
2. **Set NO_PROXY appropriately** - Avoid routing local traffic through proxy
|
||||
3. **Test proxy before automation** - Verify connectivity with simple requests
|
||||
4. **Handle proxy failures gracefully** - Implement retry logic for unstable proxies
|
||||
5. **Rotate proxies for large scraping jobs** - Distribute load and avoid bans
|
||||
@@ -0,0 +1,181 @@
|
||||
# Session Management
|
||||
|
||||
Run multiple isolated browser sessions concurrently with state persistence.
|
||||
|
||||
## Named Sessions
|
||||
|
||||
Use `--session` flag to isolate browser contexts:
|
||||
|
||||
```bash
|
||||
# Session 1: Authentication flow
|
||||
agent-browser --session auth open https://app.example.com/login
|
||||
|
||||
# Session 2: Public browsing (separate cookies, storage)
|
||||
agent-browser --session public open https://example.com
|
||||
|
||||
# Commands are isolated by session
|
||||
agent-browser --session auth fill @e1 "user@example.com"
|
||||
agent-browser --session public get text body
|
||||
```
|
||||
|
||||
## Session Isolation Properties
|
||||
|
||||
Each session has independent:
|
||||
- Cookies
|
||||
- LocalStorage / SessionStorage
|
||||
- IndexedDB
|
||||
- Cache
|
||||
- Browsing history
|
||||
- Open tabs
|
||||
|
||||
## Session State Persistence
|
||||
|
||||
### Save Session State
|
||||
|
||||
```bash
|
||||
# Save cookies, storage, and auth state
|
||||
agent-browser state save /path/to/auth-state.json
|
||||
```
|
||||
|
||||
### Load Session State
|
||||
|
||||
```bash
|
||||
# Restore saved state
|
||||
agent-browser state load /path/to/auth-state.json
|
||||
|
||||
# Continue with authenticated session
|
||||
agent-browser open https://app.example.com/dashboard
|
||||
```
|
||||
|
||||
### State File Contents
|
||||
|
||||
```json
|
||||
{
|
||||
"cookies": [...],
|
||||
"localStorage": {...},
|
||||
"sessionStorage": {...},
|
||||
"origins": [...]
|
||||
}
|
||||
```
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Authenticated Session Reuse
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Save login state once, reuse many times
|
||||
|
||||
STATE_FILE="/tmp/auth-state.json"
|
||||
|
||||
# Check if we have saved state
|
||||
if [[ -f "$STATE_FILE" ]]; then
|
||||
agent-browser state load "$STATE_FILE"
|
||||
agent-browser open https://app.example.com/dashboard
|
||||
else
|
||||
# Perform login
|
||||
agent-browser open https://app.example.com/login
|
||||
agent-browser snapshot -i
|
||||
agent-browser fill @e1 "$USERNAME"
|
||||
agent-browser fill @e2 "$PASSWORD"
|
||||
agent-browser click @e3
|
||||
agent-browser wait --load networkidle
|
||||
|
||||
# Save for future use
|
||||
agent-browser state save "$STATE_FILE"
|
||||
fi
|
||||
```
|
||||
|
||||
### Concurrent Scraping
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Scrape multiple sites concurrently
|
||||
|
||||
# Start all sessions
|
||||
agent-browser --session site1 open https://site1.com &
|
||||
agent-browser --session site2 open https://site2.com &
|
||||
agent-browser --session site3 open https://site3.com &
|
||||
wait
|
||||
|
||||
# Extract from each
|
||||
agent-browser --session site1 get text body > site1.txt
|
||||
agent-browser --session site2 get text body > site2.txt
|
||||
agent-browser --session site3 get text body > site3.txt
|
||||
|
||||
# Cleanup
|
||||
agent-browser --session site1 close
|
||||
agent-browser --session site2 close
|
||||
agent-browser --session site3 close
|
||||
```
|
||||
|
||||
### A/B Testing Sessions
|
||||
|
||||
```bash
|
||||
# Test different user experiences
|
||||
agent-browser --session variant-a open "https://app.com?variant=a"
|
||||
agent-browser --session variant-b open "https://app.com?variant=b"
|
||||
|
||||
# Compare
|
||||
agent-browser --session variant-a screenshot /tmp/variant-a.png
|
||||
agent-browser --session variant-b screenshot /tmp/variant-b.png
|
||||
```
|
||||
|
||||
## Default Session
|
||||
|
||||
When `--session` is omitted, commands use the default session:
|
||||
|
||||
```bash
|
||||
# These use the same default session
|
||||
agent-browser open https://example.com
|
||||
agent-browser snapshot -i
|
||||
agent-browser close # Closes default session
|
||||
```
|
||||
|
||||
## Session Cleanup
|
||||
|
||||
```bash
|
||||
# Close specific session
|
||||
agent-browser --session auth close
|
||||
|
||||
# List active sessions
|
||||
agent-browser session list
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Name Sessions Semantically
|
||||
|
||||
```bash
|
||||
# GOOD: Clear purpose
|
||||
agent-browser --session github-auth open https://github.com
|
||||
agent-browser --session docs-scrape open https://docs.example.com
|
||||
|
||||
# AVOID: Generic names
|
||||
agent-browser --session s1 open https://github.com
|
||||
```
|
||||
|
||||
### 2. Always Clean Up
|
||||
|
||||
```bash
|
||||
# Close sessions when done
|
||||
agent-browser --session auth close
|
||||
agent-browser --session scrape close
|
||||
```
|
||||
|
||||
### 3. Handle State Files Securely
|
||||
|
||||
```bash
|
||||
# Don't commit state files (contain auth tokens!)
|
||||
echo "*.auth-state.json" >> .gitignore
|
||||
|
||||
# Delete after use
|
||||
rm /tmp/auth-state.json
|
||||
```
|
||||
|
||||
### 4. Timeout Long Sessions
|
||||
|
||||
```bash
|
||||
# Set timeout for automated scripts
|
||||
timeout 60 agent-browser --session long-task get text body
|
||||
```
|
||||
@@ -0,0 +1,186 @@
|
||||
# Snapshot + Refs Workflow
|
||||
|
||||
The core innovation of agent-browser: compact element references that reduce context usage dramatically for AI agents.
|
||||
|
||||
## How It Works
|
||||
|
||||
### The Problem
|
||||
Traditional browser automation sends full DOM to AI agents:
|
||||
```
|
||||
Full DOM/HTML sent → AI parses → Generates CSS selector → Executes action
|
||||
~3000-5000 tokens per interaction
|
||||
```
|
||||
|
||||
### The Solution
|
||||
agent-browser uses compact snapshots with refs:
|
||||
```
|
||||
Compact snapshot → @refs assigned → Direct ref interaction
|
||||
~200-400 tokens per interaction
|
||||
```
|
||||
|
||||
## The Snapshot Command
|
||||
|
||||
```bash
|
||||
# Basic snapshot (shows page structure)
|
||||
agent-browser snapshot
|
||||
|
||||
# Interactive snapshot (-i flag) - RECOMMENDED
|
||||
agent-browser snapshot -i
|
||||
```
|
||||
|
||||
### Snapshot Output Format
|
||||
|
||||
```
|
||||
Page: Example Site - Home
|
||||
URL: https://example.com
|
||||
|
||||
@e1 [header]
|
||||
@e2 [nav]
|
||||
@e3 [a] "Home"
|
||||
@e4 [a] "Products"
|
||||
@e5 [a] "About"
|
||||
@e6 [button] "Sign In"
|
||||
|
||||
@e7 [main]
|
||||
@e8 [h1] "Welcome"
|
||||
@e9 [form]
|
||||
@e10 [input type="email"] placeholder="Email"
|
||||
@e11 [input type="password"] placeholder="Password"
|
||||
@e12 [button type="submit"] "Log In"
|
||||
|
||||
@e13 [footer]
|
||||
@e14 [a] "Privacy Policy"
|
||||
```
|
||||
|
||||
## Using Refs
|
||||
|
||||
Once you have refs, interact directly:
|
||||
|
||||
```bash
|
||||
# Click the "Sign In" button
|
||||
agent-browser click @e6
|
||||
|
||||
# Fill email input
|
||||
agent-browser fill @e10 "user@example.com"
|
||||
|
||||
# Fill password
|
||||
agent-browser fill @e11 "password123"
|
||||
|
||||
# Submit the form
|
||||
agent-browser click @e12
|
||||
```
|
||||
|
||||
## Ref Lifecycle
|
||||
|
||||
**IMPORTANT**: Refs are invalidated when the page changes!
|
||||
|
||||
```bash
|
||||
# Get initial snapshot
|
||||
agent-browser snapshot -i
|
||||
# @e1 [button] "Next"
|
||||
|
||||
# Click triggers page change
|
||||
agent-browser click @e1
|
||||
|
||||
# MUST re-snapshot to get new refs!
|
||||
agent-browser snapshot -i
|
||||
# @e1 [h1] "Page 2" ← Different element now!
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Always Snapshot Before Interacting
|
||||
|
||||
```bash
|
||||
# CORRECT
|
||||
agent-browser open https://example.com
|
||||
agent-browser snapshot -i # Get refs first
|
||||
agent-browser click @e1 # Use ref
|
||||
|
||||
# WRONG
|
||||
agent-browser open https://example.com
|
||||
agent-browser click @e1 # Ref doesn't exist yet!
|
||||
```
|
||||
|
||||
### 2. Re-Snapshot After Navigation
|
||||
|
||||
```bash
|
||||
agent-browser click @e5 # Navigates to new page
|
||||
agent-browser snapshot -i # Get new refs
|
||||
agent-browser click @e1 # Use new refs
|
||||
```
|
||||
|
||||
### 3. Re-Snapshot After Dynamic Changes
|
||||
|
||||
```bash
|
||||
agent-browser click @e1 # Opens dropdown
|
||||
agent-browser snapshot -i # See dropdown items
|
||||
agent-browser click @e7 # Select item
|
||||
```
|
||||
|
||||
### 4. Snapshot Specific Regions
|
||||
|
||||
For complex pages, snapshot specific areas:
|
||||
|
||||
```bash
|
||||
# Snapshot just the form
|
||||
agent-browser snapshot @e9
|
||||
```
|
||||
|
||||
## Ref Notation Details
|
||||
|
||||
```
|
||||
@e1 [tag type="value"] "text content" placeholder="hint"
|
||||
│ │ │ │ │
|
||||
│ │ │ │ └─ Additional attributes
|
||||
│ │ │ └─ Visible text
|
||||
│ │ └─ Key attributes shown
|
||||
│ └─ HTML tag name
|
||||
└─ Unique ref ID
|
||||
```
|
||||
|
||||
### Common Patterns
|
||||
|
||||
```
|
||||
@e1 [button] "Submit" # Button with text
|
||||
@e2 [input type="email"] # Email input
|
||||
@e3 [input type="password"] # Password input
|
||||
@e4 [a href="/page"] "Link Text" # Anchor link
|
||||
@e5 [select] # Dropdown
|
||||
@e6 [textarea] placeholder="Message" # Text area
|
||||
@e7 [div class="modal"] # Container (when relevant)
|
||||
@e8 [img alt="Logo"] # Image
|
||||
@e9 [checkbox] checked # Checked checkbox
|
||||
@e10 [radio] selected # Selected radio
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Ref not found" Error
|
||||
|
||||
```bash
|
||||
# Ref may have changed - re-snapshot
|
||||
agent-browser snapshot -i
|
||||
```
|
||||
|
||||
### Element Not Visible in Snapshot
|
||||
|
||||
```bash
|
||||
# Scroll to reveal element
|
||||
agent-browser scroll --bottom
|
||||
agent-browser snapshot -i
|
||||
|
||||
# Or wait for dynamic content
|
||||
agent-browser wait 1000
|
||||
agent-browser snapshot -i
|
||||
```
|
||||
|
||||
### Too Many Elements
|
||||
|
||||
```bash
|
||||
# Snapshot specific container
|
||||
agent-browser snapshot @e5
|
||||
|
||||
# Or use get text for content-only extraction
|
||||
agent-browser get text @e5
|
||||
```
|
||||
@@ -0,0 +1,162 @@
|
||||
# Video Recording
|
||||
|
||||
Capture browser automation sessions as video for debugging, documentation, or verification.
|
||||
|
||||
## Basic Recording
|
||||
|
||||
```bash
|
||||
# Start recording
|
||||
agent-browser record start ./demo.webm
|
||||
|
||||
# Perform actions
|
||||
agent-browser open https://example.com
|
||||
agent-browser snapshot -i
|
||||
agent-browser click @e1
|
||||
agent-browser fill @e2 "test input"
|
||||
|
||||
# Stop and save
|
||||
agent-browser record stop
|
||||
```
|
||||
|
||||
## Recording Commands
|
||||
|
||||
```bash
|
||||
# Start recording to file
|
||||
agent-browser record start ./output.webm
|
||||
|
||||
# Stop current recording
|
||||
agent-browser record stop
|
||||
|
||||
# Restart with new file (stops current + starts new)
|
||||
agent-browser record restart ./take2.webm
|
||||
```
|
||||
|
||||
## Use Cases
|
||||
|
||||
### Debugging Failed Automation
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Record automation for debugging
|
||||
|
||||
agent-browser record start ./debug-$(date +%Y%m%d-%H%M%S).webm
|
||||
|
||||
# Run your automation
|
||||
agent-browser open https://app.example.com
|
||||
agent-browser snapshot -i
|
||||
agent-browser click @e1 || {
|
||||
echo "Click failed - check recording"
|
||||
agent-browser record stop
|
||||
exit 1
|
||||
}
|
||||
|
||||
agent-browser record stop
|
||||
```
|
||||
|
||||
### Documentation Generation
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Record workflow for documentation
|
||||
|
||||
agent-browser record start ./docs/how-to-login.webm
|
||||
|
||||
agent-browser open https://app.example.com/login
|
||||
agent-browser wait 1000 # Pause for visibility
|
||||
|
||||
agent-browser snapshot -i
|
||||
agent-browser fill @e1 "demo@example.com"
|
||||
agent-browser wait 500
|
||||
|
||||
agent-browser fill @e2 "password"
|
||||
agent-browser wait 500
|
||||
|
||||
agent-browser click @e3
|
||||
agent-browser wait --load networkidle
|
||||
agent-browser wait 1000 # Show result
|
||||
|
||||
agent-browser record stop
|
||||
```
|
||||
|
||||
### CI/CD Test Evidence
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Record E2E test runs for CI artifacts
|
||||
|
||||
TEST_NAME="${1:-e2e-test}"
|
||||
RECORDING_DIR="./test-recordings"
|
||||
mkdir -p "$RECORDING_DIR"
|
||||
|
||||
agent-browser record start "$RECORDING_DIR/$TEST_NAME-$(date +%s).webm"
|
||||
|
||||
# Run test
|
||||
if run_e2e_test; then
|
||||
echo "Test passed"
|
||||
else
|
||||
echo "Test failed - recording saved"
|
||||
fi
|
||||
|
||||
agent-browser record stop
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Add Pauses for Clarity
|
||||
|
||||
```bash
|
||||
# Slow down for human viewing
|
||||
agent-browser click @e1
|
||||
agent-browser wait 500 # Let viewer see result
|
||||
```
|
||||
|
||||
### 2. Use Descriptive Filenames
|
||||
|
||||
```bash
|
||||
# Include context in filename
|
||||
agent-browser record start ./recordings/login-flow-2024-01-15.webm
|
||||
agent-browser record start ./recordings/checkout-test-run-42.webm
|
||||
```
|
||||
|
||||
### 3. Handle Recording in Error Cases
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
cleanup() {
|
||||
agent-browser record stop 2>/dev/null || true
|
||||
agent-browser close 2>/dev/null || true
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
agent-browser record start ./automation.webm
|
||||
# ... automation steps ...
|
||||
```
|
||||
|
||||
### 4. Combine with Screenshots
|
||||
|
||||
```bash
|
||||
# Record video AND capture key frames
|
||||
agent-browser record start ./flow.webm
|
||||
|
||||
agent-browser open https://example.com
|
||||
agent-browser screenshot ./screenshots/step1-homepage.png
|
||||
|
||||
agent-browser click @e1
|
||||
agent-browser screenshot ./screenshots/step2-after-click.png
|
||||
|
||||
agent-browser record stop
|
||||
```
|
||||
|
||||
## Output Format
|
||||
|
||||
- Default format: WebM (VP8/VP9 codec)
|
||||
- Compatible with all modern browsers and video players
|
||||
- Compressed but high quality
|
||||
|
||||
## Limitations
|
||||
|
||||
- Recording adds slight overhead to automation
|
||||
- Large recordings can consume significant disk space
|
||||
- Some headless environments may have codec limitations
|
||||
+91
@@ -0,0 +1,91 @@
|
||||
#!/bin/bash
|
||||
# Template: Authenticated Session Workflow
|
||||
# Login once, save state, reuse for subsequent runs
|
||||
#
|
||||
# Usage:
|
||||
# ./authenticated-session.sh <login-url> [state-file]
|
||||
#
|
||||
# Setup:
|
||||
# 1. Run once to see your form structure
|
||||
# 2. Note the @refs for your fields
|
||||
# 3. Uncomment LOGIN FLOW section and update refs
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
LOGIN_URL="${1:?Usage: $0 <login-url> [state-file]}"
|
||||
STATE_FILE="${2:-./auth-state.json}"
|
||||
|
||||
echo "Authentication workflow for: $LOGIN_URL"
|
||||
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
# SAVED STATE: Skip login if we have valid saved state
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
if [[ -f "$STATE_FILE" ]]; then
|
||||
echo "Loading saved authentication state..."
|
||||
agent-browser state load "$STATE_FILE"
|
||||
agent-browser open "$LOGIN_URL"
|
||||
agent-browser wait --load networkidle
|
||||
|
||||
CURRENT_URL=$(agent-browser get url)
|
||||
if [[ "$CURRENT_URL" != *"login"* ]] && [[ "$CURRENT_URL" != *"signin"* ]]; then
|
||||
echo "Session restored successfully!"
|
||||
agent-browser snapshot -i
|
||||
exit 0
|
||||
fi
|
||||
echo "Session expired, performing fresh login..."
|
||||
rm -f "$STATE_FILE"
|
||||
fi
|
||||
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
# DISCOVERY MODE: Show form structure (remove after setup)
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
echo "Opening login page..."
|
||||
agent-browser open "$LOGIN_URL"
|
||||
agent-browser wait --load networkidle
|
||||
|
||||
echo ""
|
||||
echo "┌─────────────────────────────────────────────────────────┐"
|
||||
echo "│ LOGIN FORM STRUCTURE │"
|
||||
echo "├─────────────────────────────────────────────────────────┤"
|
||||
agent-browser snapshot -i
|
||||
echo "└─────────────────────────────────────────────────────────┘"
|
||||
echo ""
|
||||
echo "Next steps:"
|
||||
echo " 1. Note refs: @e? = username, @e? = password, @e? = submit"
|
||||
echo " 2. Uncomment LOGIN FLOW section below"
|
||||
echo " 3. Replace @e1, @e2, @e3 with your refs"
|
||||
echo " 4. Delete this DISCOVERY MODE section"
|
||||
echo ""
|
||||
agent-browser close
|
||||
exit 0
|
||||
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
# LOGIN FLOW: Uncomment and customize after discovery
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
# : "${APP_USERNAME:?Set APP_USERNAME environment variable}"
|
||||
# : "${APP_PASSWORD:?Set APP_PASSWORD environment variable}"
|
||||
#
|
||||
# agent-browser open "$LOGIN_URL"
|
||||
# agent-browser wait --load networkidle
|
||||
# agent-browser snapshot -i
|
||||
#
|
||||
# # Fill credentials (update refs to match your form)
|
||||
# agent-browser fill @e1 "$APP_USERNAME"
|
||||
# agent-browser fill @e2 "$APP_PASSWORD"
|
||||
# agent-browser click @e3
|
||||
# agent-browser wait --load networkidle
|
||||
#
|
||||
# # Verify login succeeded
|
||||
# FINAL_URL=$(agent-browser get url)
|
||||
# if [[ "$FINAL_URL" == *"login"* ]] || [[ "$FINAL_URL" == *"signin"* ]]; then
|
||||
# echo "ERROR: Login failed - still on login page"
|
||||
# agent-browser screenshot /tmp/login-failed.png
|
||||
# agent-browser close
|
||||
# exit 1
|
||||
# fi
|
||||
#
|
||||
# # Save state for future runs
|
||||
# echo "Saving authentication state to: $STATE_FILE"
|
||||
# agent-browser state save "$STATE_FILE"
|
||||
# echo "Login successful!"
|
||||
# agent-browser snapshot -i
|
||||
+68
@@ -0,0 +1,68 @@
|
||||
#!/bin/bash
|
||||
# Template: Content Capture Workflow
|
||||
# Extract content from web pages with optional authentication
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
TARGET_URL="${1:?Usage: $0 <url> [output-dir]}"
|
||||
OUTPUT_DIR="${2:-.}"
|
||||
|
||||
echo "Capturing content from: $TARGET_URL"
|
||||
mkdir -p "$OUTPUT_DIR"
|
||||
|
||||
# Optional: Load authentication state if needed
|
||||
# if [[ -f "./auth-state.json" ]]; then
|
||||
# agent-browser state load "./auth-state.json"
|
||||
# fi
|
||||
|
||||
# Navigate to target page
|
||||
agent-browser open "$TARGET_URL"
|
||||
agent-browser wait --load networkidle
|
||||
|
||||
# Get page metadata
|
||||
echo "Page title: $(agent-browser get title)"
|
||||
echo "Page URL: $(agent-browser get url)"
|
||||
|
||||
# Capture full page screenshot
|
||||
agent-browser screenshot --full "$OUTPUT_DIR/page-full.png"
|
||||
echo "Screenshot saved: $OUTPUT_DIR/page-full.png"
|
||||
|
||||
# Get page structure
|
||||
agent-browser snapshot -i > "$OUTPUT_DIR/page-structure.txt"
|
||||
echo "Structure saved: $OUTPUT_DIR/page-structure.txt"
|
||||
|
||||
# Extract main content
|
||||
# Adjust selector based on target site structure
|
||||
# agent-browser get text @e1 > "$OUTPUT_DIR/main-content.txt"
|
||||
|
||||
# Extract specific elements (uncomment as needed)
|
||||
# agent-browser get text "article" > "$OUTPUT_DIR/article.txt"
|
||||
# agent-browser get text "main" > "$OUTPUT_DIR/main.txt"
|
||||
# agent-browser get text ".content" > "$OUTPUT_DIR/content.txt"
|
||||
|
||||
# Get full page text
|
||||
agent-browser get text body > "$OUTPUT_DIR/page-text.txt"
|
||||
echo "Text content saved: $OUTPUT_DIR/page-text.txt"
|
||||
|
||||
# Optional: Save as PDF
|
||||
agent-browser pdf "$OUTPUT_DIR/page.pdf"
|
||||
echo "PDF saved: $OUTPUT_DIR/page.pdf"
|
||||
|
||||
# Optional: Capture with scrolling for infinite scroll pages
|
||||
# scroll_and_capture() {
|
||||
# local count=0
|
||||
# while [[ $count -lt 5 ]]; do
|
||||
# agent-browser scroll down 1000
|
||||
# agent-browser wait 1000
|
||||
# ((count++))
|
||||
# done
|
||||
# agent-browser screenshot --full "$OUTPUT_DIR/page-scrolled.png"
|
||||
# }
|
||||
# scroll_and_capture
|
||||
|
||||
# Cleanup
|
||||
agent-browser close
|
||||
|
||||
echo ""
|
||||
echo "Capture complete! Files saved to: $OUTPUT_DIR"
|
||||
ls -la "$OUTPUT_DIR"
|
||||
+64
@@ -0,0 +1,64 @@
|
||||
#!/bin/bash
|
||||
# Template: Form Automation Workflow
|
||||
# Fills and submits web forms with validation
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
FORM_URL="${1:?Usage: $0 <form-url>}"
|
||||
|
||||
echo "Automating form at: $FORM_URL"
|
||||
|
||||
# Navigate to form page
|
||||
agent-browser open "$FORM_URL"
|
||||
agent-browser wait --load networkidle
|
||||
|
||||
# Get interactive snapshot to identify form fields
|
||||
echo "Analyzing form structure..."
|
||||
agent-browser snapshot -i
|
||||
|
||||
# Example: Fill common form fields
|
||||
# Uncomment and modify refs based on snapshot output
|
||||
|
||||
# Text inputs
|
||||
# agent-browser fill @e1 "John Doe" # Name field
|
||||
# agent-browser fill @e2 "user@example.com" # Email field
|
||||
# agent-browser fill @e3 "+1-555-123-4567" # Phone field
|
||||
|
||||
# Password fields
|
||||
# agent-browser fill @e4 "SecureP@ssw0rd!"
|
||||
|
||||
# Dropdowns
|
||||
# agent-browser select @e5 "Option Value"
|
||||
|
||||
# Checkboxes
|
||||
# agent-browser check @e6 # Check
|
||||
# agent-browser uncheck @e7 # Uncheck
|
||||
|
||||
# Radio buttons
|
||||
# agent-browser click @e8 # Select radio option
|
||||
|
||||
# Text areas
|
||||
# agent-browser fill @e9 "Multi-line text content here"
|
||||
|
||||
# File uploads
|
||||
# agent-browser upload @e10 /path/to/file.pdf
|
||||
|
||||
# Submit form
|
||||
# agent-browser click @e11 # Submit button
|
||||
|
||||
# Wait for response
|
||||
# agent-browser wait --load networkidle
|
||||
# agent-browser wait --url "**/success" # Or wait for redirect
|
||||
|
||||
# Verify submission
|
||||
echo "Form submission result:"
|
||||
agent-browser get url
|
||||
agent-browser snapshot -i
|
||||
|
||||
# Take screenshot of result
|
||||
agent-browser screenshot /tmp/form-result.png
|
||||
|
||||
# Cleanup
|
||||
agent-browser close
|
||||
|
||||
echo "Form automation complete"
|
||||
Reference in New Issue
Block a user