Chris Tate
585d93a02b
* fix(tabs): preserve refs across --tab peek and cover outer-tab-closed path Follow-up to #1249 so `--tab <id>` is actually useful for agents: - Save and restore the outer tab's `ref_map`, `iframe_sessions`, and `active_frame_id` across a scoped command instead of clearing them. `snapshot` → `--tab N <cmd>` → `click @e1` now keeps the outer tab's refs intact. Scoped commands still see a clean slate so outer refs can't resolve against the scoped tab's DOM. - Close the coverage gap the Vercel review bot flagged on #1249: the previous `e2e_tab_scoped_command_handles_outer_tab_closed` test used `tab_close`, which is in the scoped-dispatch exclusion list, so it never exercised the restore-skip branch it claimed to test. Renamed to `e2e_tab_close_with_tab_id_closes_active_tab` with an honest docstring, and added `e2e_tab_scoped_command_outer_tab_closed_mid_dispatch` that actually hits the branch via `window.opener.close()` on a script-opened intermediate tab. - Add `e2e_tab_scoped_command_isolates_refs_from_outer_tab` pinning that outer refs don't bleed into the scoped tab's DOM resolution. - Rewrite `e2e_tab_scoped_command_clears_state_on_switch` as `e2e_tab_scoped_command_preserves_outer_tab_state`, verifying the restored @e1 still clicks end-to-end. - Update the 52 `--help` entries for `--tab <id>` to describe peek / restore semantics instead of a vague "Target specific tab ID". - Update README, docs site, config schema, and the agent-facing skills reference with working examples (refs survive the peek) and a "when to use \`--tab <id>\` vs \`tab <id>\`" guide so agents pick the right flag for their workflow. * fix(tabs): use t<N> prefix for tab ids, add --label for named tabs Follow-on to the tab work in #1249 and the prior commit, redesigning the tab handle surface before release since nothing ships these features yet. ## Why Incrementing integer tab ids (`1`, `2`, `3`) look indistinguishable from positional indices in command output, LLM-generated scripts, and docs. In the common single-agent case where position and id coincide, readers have no visual cue for which mental model they're using. Positional indices silently shift when unrelated tabs open/close, so misreading a handle as an index is a correctness hazard. ## Changes **Tab ids are now `t1`, `t2`, `t3` (strings).** Bare integer `tabId` values are rejected with a teaching message rather than silently accepted. The `t` prefix matches the `@e1` element-ref convention and makes ids unmistakably non-positional at a glance. **Labels.** Tabs can be created with a user-assigned label (e.g. `docs`, `app`) via `tab new --label <name> [url]`. Labels are interchangeable with `t<N>` ids everywhere a tab ref is accepted. They're never auto-generated, never rewritten on navigation, and must be unique within a session. **Dashboard fix.** `packages/dashboard/src/types.ts` declared `TabInfo.index: number` but the daemon has been sending `tabId` (not `index`) since #892, making `tab.index` `undefined` and breaking the dashboard's close/switch buttons silently. Updated the TS types and usages to consume `tabId` (string) and optional `label`, restoring the dashboard's tab interactions. ## Surface - `cli/src/native/browser.rs`: `TabRef::parse` / `format_tab_id` / `is_valid_label` / `PageInfo.label` / `BrowserManager::resolve_tab_ref` / `BrowserManager::has_label`. `tab_new` gains an optional label argument with duplicate rejection. All JSON responses use the string form and include the label. - `cli/src/native/actions.rs`: scoped-command pre-dispatch and `handle_tab_{switch,close,new}` parse string refs and resolve to stable ids. - `cli/src/{flags,commands,main,output}.rs`: `--tab` / config `tab` are `String`; `tab` subcommand accepts `t<N>` or a label and supports `tab new --label <name> [url]`. All 52 `--help` entries updated. - `agent-browser.schema.json`: `tab` property type is now `string` with a pattern matching `t<N>` or label form. - `packages/dashboard`: `TabInfo.tabId: string` / `label?: string | null`; `closeTabAtom`/`switchTabAtom` take `tabRef: string`; component props updated. - Docs: README, docs site (`commands/` and `configuration/`), and the agent-facing skills reference rewritten with the new examples. ## Tests - Added `TabRef::parse` / `format_tab_id` / `is_valid_label` unit tests pinning the bare-integer rejection, the teaching error, label rules, and round-tripping. - Added `test_tab_switch_by_id` / `_by_label` / `test_tab_new_with_label` / `_with_label_and_url` / `_with_url_then_label` in `commands.rs`; rewrote `test_tab_unknown_subcommand_errors` since labels make `tab select` a legitimate ref. - Added `e2e_tab_new_with_label_can_be_switched_and_peeked`, `e2e_tab_new_with_duplicate_label_errors`, `e2e_tab_scoped_command_rejects_bare_integer`. - Migrated every existing tab e2e test (and one unit test) from integer `tabId` to the string form. `cargo fmt`, `cargo clippy -- -D warnings`, all 30 non-ignored tab unit tests, all 13 tab e2e tests, and `tsc --noEmit` on the dashboard all pass. * refactor(tabs): drop --tab scoped peek flag; keep t<N> ids and labels After fleshing out `--tab <id|label>` in the previous commits (scoped pre/post-dispatch save/restore, ref preservation, outer-tab-closed edge case, full e2e coverage), the machinery-to-value ratio makes the feature hard to justify. Nixing it now while nothing has shipped. ## Why - Every new daemon feature touching per-tab state has to reason about scoped-dispatch interleaving. `ScopedRestore`, pre/post-dispatch hooks, and the exclusion list add ongoing maintenance tax. - Three separate PRs (#892, #1249, and this one pre-nix) were needed to reach "works correctly." That's a smell. - `tab <id|label>` switch + labels already cover the legible multi-tab workflow case. - `--tab` vs `tab <id>` have opposite lifecycle semantics but look identical, teaching every agent two things where one would do. - "Non-disruptive peek" isn't actually race-free: the daemon does swap active tab during execution, so a concurrent client between pre- and post-dispatch sees the scoped tab as active. - Ref-based interaction with scoped tabs never worked ergonomically — refs are per-tab, so `--tab N click @e1` requires `@e1` to already be on tab N, which means a prior switch, which negates the peek. - Adding a feature back is easy; removing shipped API is hard. If per-tab caching (`HashMap<tab_id, RefMap>`) lands later, `--tab` can be reintroduced essentially for free. That's the right time. ## Removed - `--tab <id|label>` global flag (`cli/src/flags.rs`, `cli/src/main.rs`, all 52 `--help` entries in `cli/src/output.rs`). - `tab` property in `agent-browser.schema.json` and the config-options row in `docs/src/app/configuration/page.mdx`. - `ScopedRestore` struct, pre/post-dispatch save/restore in `execute_command` (`cli/src/native/actions.rs`). - `impl Default for RefMap` in `cli/src/native/element.rs` (only added for `mem::take` in the scoped machinery). - `e2e_tab_global_targeting`, `_snapshot`, `_snapshot_non_contiguous`, `e2e_tab_scoped_command_preserves_outer_tab_state`, `_isolates_refs_from_outer_tab`, `_restores_active_tab`, `_outer_tab_closed_mid_dispatch`. 590 lines. - The "When to use `--tab` vs `tab <id|label>`" sections in README, docs site, and skills reference. ## Kept - Stable tab ids (`t1`, `t2`, `t3`) with bare-integer rejection. - User-assigned labels (`tab new --label docs [url]`), with duplicate rejection and interchangeable use everywhere a tab ref is accepted. - `BrowserManager::{active_tab_id, has_tab_id, resolve_tab_ref, has_label}` accessors (still used by the remaining tab handlers). - `TabRef::parse`, `format_tab_id`, `is_valid_label` and their unit tests. - Dashboard TS fix (`TabInfo.tabId` + `label`). - `e2e_tab_close_with_tab_id_closes_active_tab` (renamed docstring to drop the gone exclusion-list reference). - `e2e_tab_new_with_label_can_be_switched_and_closed` (rewrite of the previous `_and_peeked` test — now exercises only switch and close). - `e2e_tab_switch_rejects_bare_integer` (rewrite targeting the `tab_switch` daemon handler rather than the removed scoped path). net: -900 lines across 12 files. `cargo fmt`, `cargo clippy -D warnings`, all 25 non-ignored tab unit tests, all 6 tab e2e tests, and `tsc --noEmit` on the dashboard all pass.
323 lines
13 KiB
Markdown
323 lines
13 KiB
Markdown
# Command Reference
|
|
|
|
Complete reference for all agent-browser commands. For quick start and common patterns, see SKILL.md.
|
|
|
|
## Navigation
|
|
|
|
```bash
|
|
agent-browser open <url> # Navigate to URL (aliases: goto, navigate)
|
|
# Supports: https://, http://, file://, about:, data://
|
|
# Auto-prepends https:// if no protocol given
|
|
agent-browser back # Go back
|
|
agent-browser forward # Go forward
|
|
agent-browser reload # Reload page
|
|
agent-browser close # Close browser (aliases: quit, exit)
|
|
agent-browser connect 9222 # Connect to browser via CDP port
|
|
```
|
|
|
|
## Snapshot (page analysis)
|
|
|
|
```bash
|
|
agent-browser snapshot # Full accessibility tree
|
|
agent-browser snapshot -i # Interactive elements only (recommended)
|
|
agent-browser snapshot -c # Compact output
|
|
agent-browser snapshot -d 3 # Limit depth to 3
|
|
agent-browser snapshot -s "#main" # Scope to CSS selector
|
|
```
|
|
|
|
## Interactions (use @refs from snapshot)
|
|
|
|
```bash
|
|
agent-browser click @e1 # Click
|
|
agent-browser click @e1 --new-tab # Click and open in new tab
|
|
agent-browser dblclick @e1 # Double-click
|
|
agent-browser focus @e1 # Focus element
|
|
agent-browser fill @e2 "text" # Clear and type
|
|
agent-browser type @e2 "text" # Type without clearing
|
|
agent-browser press Enter # Press key (alias: key)
|
|
agent-browser press Control+a # Key combination
|
|
agent-browser keydown Shift # Hold key down
|
|
agent-browser keyup Shift # Release key
|
|
agent-browser hover @e1 # Hover
|
|
agent-browser check @e1 # Check checkbox
|
|
agent-browser uncheck @e1 # Uncheck checkbox
|
|
agent-browser select @e1 "value" # Select dropdown option
|
|
agent-browser select @e1 "a" "b" # Select multiple options
|
|
agent-browser scroll down 500 # Scroll page (default: down 300px)
|
|
agent-browser scrollintoview @e1 # Scroll element into view (alias: scrollinto)
|
|
agent-browser drag @e1 @e2 # Drag and drop
|
|
agent-browser upload @e1 file.pdf # Upload files
|
|
```
|
|
|
|
## Get Information
|
|
|
|
```bash
|
|
agent-browser get text @e1 # Get element text
|
|
agent-browser get html @e1 # Get innerHTML
|
|
agent-browser get value @e1 # Get input value
|
|
agent-browser get attr @e1 href # Get attribute
|
|
agent-browser get title # Get page title
|
|
agent-browser get url # Get current URL
|
|
agent-browser get cdp-url # Get CDP WebSocket URL
|
|
agent-browser get count ".item" # Count matching elements
|
|
agent-browser get box @e1 # Get bounding box
|
|
agent-browser get styles @e1 # Get computed styles (font, color, bg, etc.)
|
|
```
|
|
|
|
## Check State
|
|
|
|
```bash
|
|
agent-browser is visible @e1 # Check if visible
|
|
agent-browser is enabled @e1 # Check if enabled
|
|
agent-browser is checked @e1 # Check if checked
|
|
```
|
|
|
|
## Screenshots and PDF
|
|
|
|
```bash
|
|
agent-browser screenshot # Save to temporary directory
|
|
agent-browser screenshot path.png # Save to specific path
|
|
agent-browser screenshot --full # Full page
|
|
agent-browser pdf output.pdf # Save as PDF
|
|
```
|
|
|
|
## Video Recording
|
|
|
|
```bash
|
|
agent-browser record start ./demo.webm # Start recording
|
|
agent-browser click @e1 # Perform actions
|
|
agent-browser record stop # Stop and save video
|
|
agent-browser record restart ./take2.webm # Stop current + start new
|
|
```
|
|
|
|
## Wait
|
|
|
|
```bash
|
|
agent-browser wait @e1 # Wait for element
|
|
agent-browser wait 2000 # Wait milliseconds
|
|
agent-browser wait --text "Success" # Wait for text (or -t)
|
|
agent-browser wait --url "**/dashboard" # Wait for URL pattern (or -u)
|
|
agent-browser wait --load networkidle # Wait for network idle (or -l)
|
|
agent-browser wait --fn "window.ready" # Wait for JS condition (or -f)
|
|
```
|
|
|
|
## Mouse Control
|
|
|
|
```bash
|
|
agent-browser mouse move 100 200 # Move mouse
|
|
agent-browser mouse down left # Press button
|
|
agent-browser mouse up left # Release button
|
|
agent-browser mouse wheel 100 # Scroll wheel
|
|
```
|
|
|
|
## Semantic Locators (alternative to refs)
|
|
|
|
```bash
|
|
agent-browser find role button click --name "Submit"
|
|
agent-browser find text "Sign In" click
|
|
agent-browser find text "Sign In" click --exact # Exact match only
|
|
agent-browser find label "Email" fill "user@test.com"
|
|
agent-browser find placeholder "Search" type "query"
|
|
agent-browser find alt "Logo" click
|
|
agent-browser find title "Close" click
|
|
agent-browser find testid "submit-btn" click
|
|
agent-browser find first ".item" click
|
|
agent-browser find last ".item" click
|
|
agent-browser find nth 2 "a" hover
|
|
```
|
|
|
|
## Browser Settings
|
|
|
|
```bash
|
|
agent-browser set viewport 1920 1080 # Set viewport size
|
|
agent-browser set viewport 1920 1080 2 # 2x retina (same CSS size, higher res screenshots)
|
|
agent-browser set device "iPhone 14" # Emulate device
|
|
agent-browser set geo 37.7749 -122.4194 # Set geolocation (alias: geolocation)
|
|
agent-browser set offline on # Toggle offline mode
|
|
agent-browser set headers '{"X-Key":"v"}' # Extra HTTP headers
|
|
agent-browser set credentials user pass # HTTP basic auth (alias: auth)
|
|
agent-browser set media dark # Emulate color scheme
|
|
agent-browser set media light reduced-motion # Light mode + reduced motion
|
|
```
|
|
|
|
## Cookies and Storage
|
|
|
|
```bash
|
|
agent-browser cookies # Get all cookies
|
|
agent-browser cookies set name value # Set cookie
|
|
agent-browser cookies clear # Clear cookies
|
|
agent-browser storage local # Get all localStorage
|
|
agent-browser storage local key # Get specific key
|
|
agent-browser storage local set k v # Set value
|
|
agent-browser storage local clear # Clear all
|
|
```
|
|
|
|
## Network
|
|
|
|
```bash
|
|
agent-browser network route <url> # Intercept requests
|
|
agent-browser network route <url> --abort # Block requests
|
|
agent-browser network route <url> --body '{}' # Mock response
|
|
agent-browser network unroute [url] # Remove routes
|
|
agent-browser network requests # View tracked requests
|
|
agent-browser network requests --filter api # Filter requests
|
|
```
|
|
|
|
## Tabs and Windows
|
|
|
|
```bash
|
|
agent-browser tab # List tabs with tabId and label
|
|
agent-browser tab new [url] # New tab
|
|
agent-browser tab new --label docs [url] # New tab with a memorable label
|
|
agent-browser tab t2 # Switch to tab by id
|
|
agent-browser tab docs # Switch to tab by label
|
|
agent-browser tab close # Close current tab
|
|
agent-browser tab close t2 # Close tab by id
|
|
agent-browser tab close docs # Close tab by label
|
|
agent-browser window new # New window
|
|
```
|
|
|
|
Tab ids are stable strings of the form `t1`, `t2`, `t3`. They're never reused
|
|
within a session, so the same id keeps referring to the same tab across
|
|
commands. Positional integers are **not** accepted — `tab 2` errors with a
|
|
teaching message; use `t2`.
|
|
|
|
User-assigned labels (`docs`, `app`, `admin`) are interchangeable with ids
|
|
everywhere a tab ref is accepted. Labels are the agent-friendly way to write
|
|
multi-tab workflows:
|
|
|
|
```bash
|
|
agent-browser tab new --label docs https://docs.example.com
|
|
agent-browser tab new --label app https://app.example.com
|
|
agent-browser tab docs # switch to docs
|
|
agent-browser snapshot # populate refs for docs
|
|
agent-browser click @e1 # ref click on docs
|
|
agent-browser tab app # switch to app
|
|
agent-browser tab close docs # close by label
|
|
```
|
|
|
|
Labels are never auto-generated, never rewritten on navigation, and must be
|
|
unique within a session. To interact with another tab, switch to it first:
|
|
the daemon maintains a single active tab, so refs (`@eN`) belong to the tab
|
|
that was active when the snapshot ran.
|
|
|
|
## Frames
|
|
|
|
```bash
|
|
agent-browser frame "#iframe" # Switch to iframe by CSS selector
|
|
agent-browser frame @e3 # Switch to iframe by element ref
|
|
agent-browser frame main # Back to main frame
|
|
```
|
|
|
|
### Iframe support
|
|
|
|
Iframes are detected automatically during snapshots. When the main-frame snapshot runs, `Iframe` nodes are resolved and their content is inlined beneath the iframe element in the output (one level of nesting; iframes within iframes are not expanded).
|
|
|
|
```bash
|
|
agent-browser snapshot -i
|
|
# @e3 [Iframe] "payment-frame"
|
|
# @e4 [input] "Card number"
|
|
# @e5 [button] "Pay"
|
|
|
|
# Interact directly — refs inside iframes already work
|
|
agent-browser fill @e4 "4111111111111111"
|
|
agent-browser click @e5
|
|
|
|
# Or switch frame context for scoped snapshots
|
|
agent-browser frame @e3 # Switch using element ref
|
|
agent-browser snapshot -i # Snapshot scoped to that iframe
|
|
agent-browser frame main # Return to main frame
|
|
```
|
|
|
|
The `frame` command accepts:
|
|
- **Element refs** — `frame @e3` resolves the ref to an iframe element
|
|
- **CSS selectors** — `frame "#payment-iframe"` finds the iframe by selector
|
|
- **Frame name/URL** — matches against the browser's frame tree
|
|
|
|
## Dialogs
|
|
|
|
By default, `alert` and `beforeunload` dialogs are automatically accepted so they never block the agent. `confirm` and `prompt` dialogs still require explicit handling. Use `--no-auto-dialog` to disable this behavior.
|
|
|
|
```bash
|
|
agent-browser dialog accept [text] # Accept dialog
|
|
agent-browser dialog dismiss # Dismiss dialog
|
|
agent-browser dialog status # Check if a dialog is currently open
|
|
```
|
|
|
|
## JavaScript
|
|
|
|
```bash
|
|
agent-browser eval "document.title" # Simple expressions only
|
|
agent-browser eval -b "<base64>" # Any JavaScript (base64 encoded)
|
|
agent-browser eval --stdin # Read script from stdin
|
|
```
|
|
|
|
Use `-b`/`--base64` or `--stdin` for reliable execution. Shell escaping with nested quotes and special characters is error-prone.
|
|
|
|
```bash
|
|
# Base64 encode your script, then:
|
|
agent-browser eval -b "ZG9jdW1lbnQucXVlcnlTZWxlY3RvcignW3NyYyo9Il9uZXh0Il0nKQ=="
|
|
|
|
# Or use stdin with heredoc for multiline scripts:
|
|
cat <<'EOF' | agent-browser eval --stdin
|
|
const links = document.querySelectorAll('a');
|
|
Array.from(links).map(a => a.href);
|
|
EOF
|
|
```
|
|
|
|
## State Management
|
|
|
|
```bash
|
|
agent-browser state save auth.json # Save cookies, storage, auth state
|
|
agent-browser state load auth.json # Restore saved state
|
|
```
|
|
|
|
## Global Options
|
|
|
|
```bash
|
|
agent-browser --session <name> ... # Isolated browser session
|
|
agent-browser --json ... # JSON output for parsing
|
|
agent-browser --headed ... # Show browser window (not headless)
|
|
agent-browser --full ... # Full page screenshot (-f)
|
|
agent-browser --cdp <port> ... # Connect via Chrome DevTools Protocol
|
|
agent-browser -p <provider> ... # Cloud browser provider (--provider)
|
|
agent-browser --proxy <url> ... # Use proxy server
|
|
agent-browser --proxy-bypass <hosts> # Hosts to bypass proxy
|
|
agent-browser --headers <json> ... # HTTP headers scoped to URL's origin
|
|
agent-browser --executable-path <p> # Custom browser executable
|
|
agent-browser --extension <path> ... # Load browser extension (repeatable)
|
|
agent-browser --ignore-https-errors # Ignore SSL certificate errors
|
|
agent-browser --help # Show help (-h)
|
|
agent-browser --version # Show version (-V)
|
|
agent-browser <command> --help # Show detailed help for a command
|
|
```
|
|
|
|
## Debugging
|
|
|
|
```bash
|
|
agent-browser --headed open example.com # Show browser window
|
|
agent-browser --cdp 9222 snapshot # Connect via CDP port
|
|
agent-browser connect 9222 # Alternative: connect command
|
|
agent-browser console # View console messages
|
|
agent-browser console --clear # Clear console
|
|
agent-browser errors # View page errors
|
|
agent-browser errors --clear # Clear errors
|
|
agent-browser highlight @e1 # Highlight element
|
|
agent-browser inspect # Open Chrome DevTools for this session
|
|
agent-browser trace start # Start recording trace
|
|
agent-browser trace stop trace.zip # Stop and save trace
|
|
agent-browser profiler start # Start Chrome DevTools profiling
|
|
agent-browser profiler stop trace.json # Stop and save profile
|
|
```
|
|
|
|
## Environment Variables
|
|
|
|
```bash
|
|
AGENT_BROWSER_SESSION="mysession" # Default session name
|
|
AGENT_BROWSER_EXECUTABLE_PATH="/path/chrome" # Custom browser path
|
|
AGENT_BROWSER_EXTENSIONS="/ext1,/ext2" # Comma-separated extension paths
|
|
AGENT_BROWSER_PROVIDER="browserbase" # Cloud browser provider
|
|
AGENT_BROWSER_STREAM_PORT="9223" # Override WebSocket streaming port (default: OS-assigned)
|
|
AGENT_BROWSER_HOME="/path/to/agent-browser" # Custom install location
|
|
```
|