* feat(skills): Add hierarchical structure with references and templates Adds modular documentation and executable templates to the agent-browser skill for better AI agent consumption and progressive disclosure. ## Added ### References (deep-dive documentation) - `references/snapshot-refs.md` - Ref lifecycle, invalidation, troubleshooting - `references/session-management.md` - Parallel sessions, state persistence - `references/authentication.md` - Login flows, OAuth, 2FA patterns - `references/video-recording.md` - Recording for debugging/docs - `references/proxy-support.md` - Proxy configuration, geo-testing ### Templates (ready-to-use workflows) - `templates/form-automation.sh` - Form filling with validation - `templates/authenticated-session.sh` - Login once, reuse state - `templates/capture-workflow.sh` - Content extraction with screenshots ## Modified - `SKILL.md` - Added reference tables linking to new documentation ## Benefits - Progressive disclosure: Load overview first, deep dives on demand - Reduced context: Smaller chunks for better LLM token efficiency - Ready workflows: Copy-paste templates for common patterns * fix(templates): Make authenticated-session.sh runnable out-of-box Addresses review feedback: login actions were commented but verification wasn't, causing script to fail when run as-is. New approach: - DISCOVERY MODE runs first (shows form structure) - LOGIN FLOW section is fully commented as a unit - User runs once to see refs, then customizes ┌─────────────────────────────────────────────────────────────┐ │ LOGIN FORM STRUCTURE │ ├─────────────────────────────────────────────────────────────┤ │ @e1 [input type="email"] │ │ @e2 [input type="password"] │ │ @e3 [button] "Sign In" │ └─────────────────────────────────────────────────────────────┘
176 lines
4.2 KiB
Markdown
176 lines
4.2 KiB
Markdown
# Proxy Support
|
|
|
|
Configure proxy servers for browser automation, useful for geo-testing, rate limiting avoidance, and corporate environments.
|
|
|
|
## Basic Proxy Configuration
|
|
|
|
Set proxy via environment variable before starting:
|
|
|
|
```bash
|
|
# HTTP proxy
|
|
export HTTP_PROXY="http://proxy.example.com:8080"
|
|
agent-browser open https://example.com
|
|
|
|
# HTTPS proxy
|
|
export HTTPS_PROXY="https://proxy.example.com:8080"
|
|
agent-browser open https://example.com
|
|
|
|
# Both
|
|
export HTTP_PROXY="http://proxy.example.com:8080"
|
|
export HTTPS_PROXY="http://proxy.example.com:8080"
|
|
agent-browser open https://example.com
|
|
```
|
|
|
|
## Authenticated Proxy
|
|
|
|
For proxies requiring authentication:
|
|
|
|
```bash
|
|
# Include credentials in URL
|
|
export HTTP_PROXY="http://username:password@proxy.example.com:8080"
|
|
agent-browser open https://example.com
|
|
```
|
|
|
|
## SOCKS Proxy
|
|
|
|
```bash
|
|
# SOCKS5 proxy
|
|
export ALL_PROXY="socks5://proxy.example.com:1080"
|
|
agent-browser open https://example.com
|
|
|
|
# SOCKS5 with auth
|
|
export ALL_PROXY="socks5://user:pass@proxy.example.com:1080"
|
|
agent-browser open https://example.com
|
|
```
|
|
|
|
## Proxy Bypass
|
|
|
|
Skip proxy for specific domains:
|
|
|
|
```bash
|
|
# Bypass proxy for local addresses
|
|
export NO_PROXY="localhost,127.0.0.1,.internal.company.com"
|
|
agent-browser open https://internal.company.com # Direct connection
|
|
agent-browser open https://external.com # Via proxy
|
|
```
|
|
|
|
## Common Use Cases
|
|
|
|
### Geo-Location Testing
|
|
|
|
```bash
|
|
#!/bin/bash
|
|
# Test site from different regions using geo-located proxies
|
|
|
|
PROXIES=(
|
|
"http://us-proxy.example.com:8080"
|
|
"http://eu-proxy.example.com:8080"
|
|
"http://asia-proxy.example.com:8080"
|
|
)
|
|
|
|
for proxy in "${PROXIES[@]}"; do
|
|
export HTTP_PROXY="$proxy"
|
|
export HTTPS_PROXY="$proxy"
|
|
|
|
region=$(echo "$proxy" | grep -oP '^\w+-\w+')
|
|
echo "Testing from: $region"
|
|
|
|
agent-browser --session "$region" open https://example.com
|
|
agent-browser --session "$region" screenshot "./screenshots/$region.png"
|
|
agent-browser --session "$region" close
|
|
done
|
|
```
|
|
|
|
### Rotating Proxies for Scraping
|
|
|
|
```bash
|
|
#!/bin/bash
|
|
# Rotate through proxy list to avoid rate limiting
|
|
|
|
PROXY_LIST=(
|
|
"http://proxy1.example.com:8080"
|
|
"http://proxy2.example.com:8080"
|
|
"http://proxy3.example.com:8080"
|
|
)
|
|
|
|
URLS=(
|
|
"https://site.com/page1"
|
|
"https://site.com/page2"
|
|
"https://site.com/page3"
|
|
)
|
|
|
|
for i in "${!URLS[@]}"; do
|
|
proxy_index=$((i % ${#PROXY_LIST[@]}))
|
|
export HTTP_PROXY="${PROXY_LIST[$proxy_index]}"
|
|
export HTTPS_PROXY="${PROXY_LIST[$proxy_index]}"
|
|
|
|
agent-browser open "${URLS[$i]}"
|
|
agent-browser get text body > "output-$i.txt"
|
|
agent-browser close
|
|
|
|
sleep 1 # Polite delay
|
|
done
|
|
```
|
|
|
|
### Corporate Network Access
|
|
|
|
```bash
|
|
#!/bin/bash
|
|
# Access internal sites via corporate proxy
|
|
|
|
export HTTP_PROXY="http://corpproxy.company.com:8080"
|
|
export HTTPS_PROXY="http://corpproxy.company.com:8080"
|
|
export NO_PROXY="localhost,127.0.0.1,.company.com"
|
|
|
|
# External sites go through proxy
|
|
agent-browser open https://external-vendor.com
|
|
|
|
# Internal sites bypass proxy
|
|
agent-browser open https://intranet.company.com
|
|
```
|
|
|
|
## Verifying Proxy Connection
|
|
|
|
```bash
|
|
# Check your apparent IP
|
|
agent-browser open https://httpbin.org/ip
|
|
agent-browser get text body
|
|
# Should show proxy's IP, not your real IP
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### Proxy Connection Failed
|
|
|
|
```bash
|
|
# Test proxy connectivity first
|
|
curl -x http://proxy.example.com:8080 https://httpbin.org/ip
|
|
|
|
# Check if proxy requires auth
|
|
export HTTP_PROXY="http://user:pass@proxy.example.com:8080"
|
|
```
|
|
|
|
### SSL/TLS Errors Through Proxy
|
|
|
|
Some proxies perform SSL inspection. If you encounter certificate errors:
|
|
|
|
```bash
|
|
# For testing only - not recommended for production
|
|
agent-browser open https://example.com --ignore-https-errors
|
|
```
|
|
|
|
### Slow Performance
|
|
|
|
```bash
|
|
# Use proxy only when necessary
|
|
export NO_PROXY="*.cdn.com,*.static.com" # Direct CDN access
|
|
```
|
|
|
|
## Best Practices
|
|
|
|
1. **Use environment variables** - Don't hardcode proxy credentials
|
|
2. **Set NO_PROXY appropriately** - Avoid routing local traffic through proxy
|
|
3. **Test proxy before automation** - Verify connectivity with simple requests
|
|
4. **Handle proxy failures gracefully** - Implement retry logic for unstable proxies
|
|
5. **Rotate proxies for large scraping jobs** - Distribute load and avoid bans
|