Chris Tate
4cc6ca40b7
Before this change, the main skill served by the CLI (`agent-browser
skills get agent-browser`) was a ~40-line discovery stub whose content
was essentially "run `agent-browser skills get <name>` before doing
anything." Agents already inside the CLI got no signal from it — the
content they needed to actually use the tool lived only in the `--full`
references.
Split the two jobs apart:
- **`skill-data/core/`** (new) — the runtime usage guide. 420-line
`SKILL.md` covering the snapshot-and-ref loop, common workflows
(login, extract, screenshot, multi-tab, sessions, iframes, dialogs),
waiting strategies, element selection strategies, troubleshooting,
and when to load a specialized skill. Supplementary `references/` and
`templates/` (moved from `skills/agent-browser/`) provide the full
command reference under `--full`.
- **`skills/agent-browser/SKILL.md`** — still the discovery stub that
`npx skills add` installs, now marked `hidden: true` so it stays out
of `skills list` inside the CLI. Body is a clean pointer to
`agent-browser skills get core` and the specialized skills.
The `hidden: true` frontmatter flag is a new, general mechanism: skills
marked hidden are omitted from `skills list` and `skills get --all` but
can still be fetched by explicit name. This keeps the stub reachable
for anyone who installed via `npx skills add` without polluting the
CLI-side skill listing.
## Behavior
```
$ agent-browser skills list
agentcore Run agent-browser on AWS Bedrock AgentCore cloud browsers...
core Core agent-browser usage guide. Read this before running...
dogfood Systematically explore and test a web application...
electron Automate Electron desktop apps (VS Code, Slack, Discord...)
slack Interact with Slack workspaces using browser automation...
vercel-sandbox Run agent-browser + Chrome inside Vercel Sandbox microVMs...
$ agent-browser skills get core # the actual usage guide
# ~420 lines of workflows, patterns, troubleshooting
$ agent-browser skills get agent-browser # still works if called explicitly
# the thin stub, now pointing at `core`
```
External `npx skills add vercel-labs/agent-browser` behavior is
unchanged: it finds and installs the thin `agent-browser` stub, which
tells the agent to run `agent-browser skills get core` for real
content. Version drift protection is preserved — the stub is the only
thing that gets copied; the real content is always runtime-fetched.
## Updated
- `cli/src/skills.rs` — `SkillInfo.hidden: bool`, parsed from
frontmatter; `run_list` and `run_get --all` filter it. 3 new unit
tests for the frontmatter parser.
- `cli/src/output.rs` — top-level `--help` and `skills` subcommand help
reference `skills get core` / `skills get core --full`.
- `AGENTS.md` — "update these files for user-facing features" now
points at `skill-data/core/` instead of the stub, with a note that
the stub is not the right place for feature content.
- `README.md`, `docs/src/app/skills/page.mdx` — describe the new
split and `skills get core --full` as the recommended entry point.
- `evals/cases/{command-usage,skill-selection}.ts` — expect
`skills get core` in agent output instead of `skills get
agent-browser`. Eval lib still reads `skills/agent-browser/SKILL.md`
(simulating what an agent sees after `npx skills add`).
All 11 skills unit tests pass. `cargo clippy -- -D warnings` and
`cargo fmt --check` clean. Verified end-to-end: `skills list` shows
`core` + specialized (no stub), `skills get core` returns the new
content, `skills get agent-browser` still returns the stub on explicit
request.
5.3 KiB
5.3 KiB
Snapshot and Refs
Compact element references that reduce context usage dramatically for AI agents.
Related: commands.md for full command reference, SKILL.md for quick start.
Contents
- How Refs Work
- Snapshot Command
- Using Refs
- Ref Lifecycle
- Best Practices
- Ref Notation Details
- Troubleshooting
How Refs Work
Traditional approach:
Full DOM/HTML → AI parses → CSS selector → Action (~3000-5000 tokens)
agent-browser approach:
Compact snapshot → @refs assigned → Direct interaction (~200-400 tokens)
The Snapshot Command
# Basic snapshot (shows page structure)
agent-browser snapshot
# Interactive snapshot (-i flag) - RECOMMENDED
agent-browser snapshot -i
Snapshot Output Format
Page: Example Site - Home
URL: https://example.com
@e1 [header]
@e2 [nav]
@e3 [a] "Home"
@e4 [a] "Products"
@e5 [a] "About"
@e6 [button] "Sign In"
@e7 [main]
@e8 [h1] "Welcome"
@e9 [form]
@e10 [input type="email"] placeholder="Email"
@e11 [input type="password"] placeholder="Password"
@e12 [button type="submit"] "Log In"
@e13 [footer]
@e14 [a] "Privacy Policy"
Using Refs
Once you have refs, interact directly:
# Click the "Sign In" button
agent-browser click @e6
# Fill email input
agent-browser fill @e10 "user@example.com"
# Fill password
agent-browser fill @e11 "password123"
# Submit the form
agent-browser click @e12
Ref Lifecycle
IMPORTANT: Refs are invalidated when the page changes!
# Get initial snapshot
agent-browser snapshot -i
# @e1 [button] "Next"
# Click triggers page change
agent-browser click @e1
# MUST re-snapshot to get new refs!
agent-browser snapshot -i
# @e1 [h1] "Page 2" ← Different element now!
Best Practices
1. Always Snapshot Before Interacting
# CORRECT
agent-browser open https://example.com
agent-browser snapshot -i # Get refs first
agent-browser click @e1 # Use ref
# WRONG
agent-browser open https://example.com
agent-browser click @e1 # Ref doesn't exist yet!
2. Re-Snapshot After Navigation
agent-browser click @e5 # Navigates to new page
agent-browser snapshot -i # Get new refs
agent-browser click @e1 # Use new refs
3. Re-Snapshot After Dynamic Changes
agent-browser click @e1 # Opens dropdown
agent-browser snapshot -i # See dropdown items
agent-browser click @e7 # Select item
4. Snapshot Specific Regions
For complex pages, snapshot specific areas:
# Snapshot just the form
agent-browser snapshot @e9
Ref Notation Details
@e1 [tag type="value"] "text content" placeholder="hint"
│ │ │ │ │
│ │ │ │ └─ Additional attributes
│ │ │ └─ Visible text
│ │ └─ Key attributes shown
│ └─ HTML tag name
└─ Unique ref ID
Common Patterns
@e1 [button] "Submit" # Button with text
@e2 [input type="email"] # Email input
@e3 [input type="password"] # Password input
@e4 [a href="/page"] "Link Text" # Anchor link
@e5 [select] # Dropdown
@e6 [textarea] placeholder="Message" # Text area
@e7 [div class="modal"] # Container (when relevant)
@e8 [img alt="Logo"] # Image
@e9 [checkbox] checked # Checked checkbox
@e10 [radio] selected # Selected radio
Iframes
Snapshots automatically detect and inline iframe content. When the main-frame snapshot runs, each Iframe node is resolved and its child accessibility tree is included directly beneath it in the output. Refs assigned to elements inside iframes carry frame context, so interactions like click, fill, and type work without manually switching frames.
agent-browser snapshot -i
# @e1 [heading] "Checkout"
# @e2 [Iframe] "payment-frame"
# @e3 [input] "Card number"
# @e4 [input] "Expiry"
# @e5 [button] "Pay"
# @e6 [button] "Cancel"
# Interact with iframe elements directly using their refs
agent-browser fill @e3 "4111111111111111"
agent-browser fill @e4 "12/28"
agent-browser click @e5
Key details:
- Only one level of iframe nesting is expanded (iframes within iframes are not recursed)
- Cross-origin iframes that block accessibility tree access are silently skipped
- Empty iframes or iframes with no interactive content are omitted from the output
- To scope a snapshot to a single iframe, use
frame @refthensnapshot -i
Troubleshooting
"Ref not found" Error
# Ref may have changed - re-snapshot
agent-browser snapshot -i
Element Not Visible in Snapshot
# Scroll down to reveal element
agent-browser scroll down 1000
agent-browser snapshot -i
# Or wait for dynamic content
agent-browser wait 1000
agent-browser snapshot -i
Too Many Elements
# Snapshot specific container
agent-browser snapshot @e5
# Or use get text for content-only extraction
agent-browser get text @e5