feat(skills): Add hierarchical structure with references and templates (#157)
* feat(skills): Add hierarchical structure with references and templates Adds modular documentation and executable templates to the agent-browser skill for better AI agent consumption and progressive disclosure. ## Added ### References (deep-dive documentation) - `references/snapshot-refs.md` - Ref lifecycle, invalidation, troubleshooting - `references/session-management.md` - Parallel sessions, state persistence - `references/authentication.md` - Login flows, OAuth, 2FA patterns - `references/video-recording.md` - Recording for debugging/docs - `references/proxy-support.md` - Proxy configuration, geo-testing ### Templates (ready-to-use workflows) - `templates/form-automation.sh` - Form filling with validation - `templates/authenticated-session.sh` - Login once, reuse state - `templates/capture-workflow.sh` - Content extraction with screenshots ## Modified - `SKILL.md` - Added reference tables linking to new documentation ## Benefits - Progressive disclosure: Load overview first, deep dives on demand - Reduced context: Smaller chunks for better LLM token efficiency - Ready workflows: Copy-paste templates for common patterns * fix(templates): Make authenticated-session.sh runnable out-of-box Addresses review feedback: login actions were commented but verification wasn't, causing script to fail when run as-is. New approach: - DISCOVERY MODE runs first (shows form structure) - LOGIN FLOW section is fully commented as a unit - User runs once to see refs, then customizes ┌─────────────────────────────────────────────────────────────┐ │ LOGIN FORM STRUCTURE │ ├─────────────────────────────────────────────────────────────┤ │ @e1 [input type="email"] │ │ @e2 [input type="password"] │ │ @e3 [button] "Sign In" │ └─────────────────────────────────────────────────────────────┘
This commit is contained in:
@@ -0,0 +1,175 @@
|
||||
# Proxy Support
|
||||
|
||||
Configure proxy servers for browser automation, useful for geo-testing, rate limiting avoidance, and corporate environments.
|
||||
|
||||
## Basic Proxy Configuration
|
||||
|
||||
Set proxy via environment variable before starting:
|
||||
|
||||
```bash
|
||||
# HTTP proxy
|
||||
export HTTP_PROXY="http://proxy.example.com:8080"
|
||||
agent-browser open https://example.com
|
||||
|
||||
# HTTPS proxy
|
||||
export HTTPS_PROXY="https://proxy.example.com:8080"
|
||||
agent-browser open https://example.com
|
||||
|
||||
# Both
|
||||
export HTTP_PROXY="http://proxy.example.com:8080"
|
||||
export HTTPS_PROXY="http://proxy.example.com:8080"
|
||||
agent-browser open https://example.com
|
||||
```
|
||||
|
||||
## Authenticated Proxy
|
||||
|
||||
For proxies requiring authentication:
|
||||
|
||||
```bash
|
||||
# Include credentials in URL
|
||||
export HTTP_PROXY="http://username:password@proxy.example.com:8080"
|
||||
agent-browser open https://example.com
|
||||
```
|
||||
|
||||
## SOCKS Proxy
|
||||
|
||||
```bash
|
||||
# SOCKS5 proxy
|
||||
export ALL_PROXY="socks5://proxy.example.com:1080"
|
||||
agent-browser open https://example.com
|
||||
|
||||
# SOCKS5 with auth
|
||||
export ALL_PROXY="socks5://user:pass@proxy.example.com:1080"
|
||||
agent-browser open https://example.com
|
||||
```
|
||||
|
||||
## Proxy Bypass
|
||||
|
||||
Skip proxy for specific domains:
|
||||
|
||||
```bash
|
||||
# Bypass proxy for local addresses
|
||||
export NO_PROXY="localhost,127.0.0.1,.internal.company.com"
|
||||
agent-browser open https://internal.company.com # Direct connection
|
||||
agent-browser open https://external.com # Via proxy
|
||||
```
|
||||
|
||||
## Common Use Cases
|
||||
|
||||
### Geo-Location Testing
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Test site from different regions using geo-located proxies
|
||||
|
||||
PROXIES=(
|
||||
"http://us-proxy.example.com:8080"
|
||||
"http://eu-proxy.example.com:8080"
|
||||
"http://asia-proxy.example.com:8080"
|
||||
)
|
||||
|
||||
for proxy in "${PROXIES[@]}"; do
|
||||
export HTTP_PROXY="$proxy"
|
||||
export HTTPS_PROXY="$proxy"
|
||||
|
||||
region=$(echo "$proxy" | grep -oP '^\w+-\w+')
|
||||
echo "Testing from: $region"
|
||||
|
||||
agent-browser --session "$region" open https://example.com
|
||||
agent-browser --session "$region" screenshot "./screenshots/$region.png"
|
||||
agent-browser --session "$region" close
|
||||
done
|
||||
```
|
||||
|
||||
### Rotating Proxies for Scraping
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Rotate through proxy list to avoid rate limiting
|
||||
|
||||
PROXY_LIST=(
|
||||
"http://proxy1.example.com:8080"
|
||||
"http://proxy2.example.com:8080"
|
||||
"http://proxy3.example.com:8080"
|
||||
)
|
||||
|
||||
URLS=(
|
||||
"https://site.com/page1"
|
||||
"https://site.com/page2"
|
||||
"https://site.com/page3"
|
||||
)
|
||||
|
||||
for i in "${!URLS[@]}"; do
|
||||
proxy_index=$((i % ${#PROXY_LIST[@]}))
|
||||
export HTTP_PROXY="${PROXY_LIST[$proxy_index]}"
|
||||
export HTTPS_PROXY="${PROXY_LIST[$proxy_index]}"
|
||||
|
||||
agent-browser open "${URLS[$i]}"
|
||||
agent-browser get text body > "output-$i.txt"
|
||||
agent-browser close
|
||||
|
||||
sleep 1 # Polite delay
|
||||
done
|
||||
```
|
||||
|
||||
### Corporate Network Access
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Access internal sites via corporate proxy
|
||||
|
||||
export HTTP_PROXY="http://corpproxy.company.com:8080"
|
||||
export HTTPS_PROXY="http://corpproxy.company.com:8080"
|
||||
export NO_PROXY="localhost,127.0.0.1,.company.com"
|
||||
|
||||
# External sites go through proxy
|
||||
agent-browser open https://external-vendor.com
|
||||
|
||||
# Internal sites bypass proxy
|
||||
agent-browser open https://intranet.company.com
|
||||
```
|
||||
|
||||
## Verifying Proxy Connection
|
||||
|
||||
```bash
|
||||
# Check your apparent IP
|
||||
agent-browser open https://httpbin.org/ip
|
||||
agent-browser get text body
|
||||
# Should show proxy's IP, not your real IP
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Proxy Connection Failed
|
||||
|
||||
```bash
|
||||
# Test proxy connectivity first
|
||||
curl -x http://proxy.example.com:8080 https://httpbin.org/ip
|
||||
|
||||
# Check if proxy requires auth
|
||||
export HTTP_PROXY="http://user:pass@proxy.example.com:8080"
|
||||
```
|
||||
|
||||
### SSL/TLS Errors Through Proxy
|
||||
|
||||
Some proxies perform SSL inspection. If you encounter certificate errors:
|
||||
|
||||
```bash
|
||||
# For testing only - not recommended for production
|
||||
agent-browser open https://example.com --ignore-https-errors
|
||||
```
|
||||
|
||||
### Slow Performance
|
||||
|
||||
```bash
|
||||
# Use proxy only when necessary
|
||||
export NO_PROXY="*.cdn.com,*.static.com" # Direct CDN access
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use environment variables** - Don't hardcode proxy credentials
|
||||
2. **Set NO_PROXY appropriately** - Avoid routing local traffic through proxy
|
||||
3. **Test proxy before automation** - Verify connectivity with simple requests
|
||||
4. **Handle proxy failures gracefully** - Implement retry logic for unstable proxies
|
||||
5. **Rotate proxies for large scraping jobs** - Distribute load and avoid bans
|
||||
Reference in New Issue
Block a user