When Chrome crashes (e.g. SIGTRAP from CHECK() assertion), the daemon
now:
1. Reaps the zombie immediately via a SIGCHLD handler in the event loop
that calls waitpid(-1, WNOHANG)
2. Detects the crash instantly on the next command via a non-blocking
try_wait() check (has_process_exited), avoiding the 3-second CDP
timeout that is_connection_alive() would incur
3. Auto-relaunches Chrome transparently for the caller
Fixes#1017
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
2026-03-25 08:04:43 -07:00
Chris Tateandgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix: handle proxy authentication via CDP Fetch.authRequired
Chrome's --proxy-server flag does not support credentials embedded in
the URL. When a proxy requires authentication, Chrome receives a 407
from the proxy but has no way to respond with credentials, resulting
in net::ERR_INVALID_AUTH_CREDENTIALS.
Fix by:
1. Parsing credentials from the proxy URL (already done by parse_proxy)
2. Storing them in DaemonState.proxy_credentials
3. Enabling Fetch.enable with handleAuthRequests: true
4. Responding to Fetch.authRequired events with Fetch.continueWithAuth
5. Passing only the server URL (without credentials) to --proxy-server
6. Forwarding credentials to the daemon via dedicated env vars
Also adds fallback to standard proxy env vars (HTTP_PROXY, HTTPS_PROXY,
ALL_PROXY, NO_PROXY) when AGENT_BROWSER_PROXY is not set.
Fixes#990
* refactor: use typed struct for parse_proxy, fix double Fetch.enable and username-only auth
- Replace serde_json::Value return from parse_proxy with a typed ParsedProxy struct
- Fix double Fetch.enable call when both proxy auth and domain filter are active
(the second call could overwrite handleAuthRequests from the first)
- Allow username-only proxy auth (some proxies don't require a password)
- Handle empty username/password in parse_proxy as None instead of Some("")
- Use install_domain_filter_fetch in auto_launch for consistency
- Update unit tests to use typed struct fields
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
The `--with-deps` flag installed font rendering libraries (libfreetype6,
libfontconfig1) but no actual font files, causing CJK characters and
emoji to render as invisible/tofu on headless Linux systems.
Add font file packages for all three supported package managers:
- apt: fonts-noto-color-emoji, fonts-noto-cjk, fonts-freefont-ttf
- dnf: google-noto-cjk-fonts, google-noto-emoji-color-fonts, liberation-fonts
- yum: google-noto-cjk-fonts, liberation-fonts
Closes#1001
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Fixes#992
When a JavaScript dialog (alert/confirm/prompt) blocks the page, agents
had no way to detect it — all commands just timed out with generic errors.
- Add `dialog status` command to check for pending dialogs
- Track dialog state via CDP Page.javascriptDialogOpening/Closed events
- Auto-inject `warning` field into all command responses when a dialog is
pending, so agents can distinguish dialog-blocked timeouts from other issues
- Document dialog commands in SKILL.md (was missing entirely), README.md,
docs site, and --help output
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Chrome's Browser.downloadWillBegin and Browser.downloadProgress events
may arrive without a sessionId or with a different sessionId than the
page session used to configure the download behavior. The previous code
required an exact session match, silently dropping these events and
causing the 30-second timeout -- which manifests as an endless download
loop when callers retry.
Changes:
- Accept Browser-domain download events regardless of sessionId while
still matching Page-domain events by session to avoid cross-tab issues
- Add a brief retry loop (up to 1s) for the GUID file to appear on disk
after Chrome signals completion, handling filesystem flush races
- Return a proper error instead of silently succeeding when the
GUID-named file cannot be found
- Apply the same sessionId fix to handle_waitfordownload and add
Browser.downloadProgress support (previously only checked
Page.downloadProgress)
Fixes#989
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
When --cdp is given an HTTP/HTTPS URL (e.g. http://host:5095?mode=Hello),
resolve_cdp_url extracts host and port for CDP discovery but discards the
query string. The discovered WebSocket URL therefore never includes the
user's original query parameters, breaking relay servers that depend on
them.
Thread the original query string through discover_cdp_url and append it
to the final WebSocket URL after host/port rewriting. WebSocket URLs
(ws://, wss://) are already passed through unchanged and are unaffected.
Fixes#977
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
The `press` command was not parsing modifier+key chords (e.g.
`Control+a`, `Shift+Enter`). It sent the raw string as a single key
name, so CDP never applied the modifier — and the `text` field caused
the literal character to be inserted instead of triggering the shortcut.
Two changes:
1. `actions.rs` — add `parse_key_chord()` to split inputs like
`Control+Shift+a` into the base key (`a`) and a CDP modifier
bitmask (Alt=1, Ctrl=2, Meta=4, Shift=8), then call
`press_key_with_modifiers` instead of `press_key`.
2. `interaction.rs` — suppress the `text` field in `keyDown`/`keyUp`
events when Control or Meta modifiers are active, so the browser
treats them as command chords rather than text input.
Includes unit tests for the chord parser covering plain keys, single
modifiers, multi-modifier combos, modifier aliases, and edge cases.
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* Fix download command to properly handle absolute paths and click elements
The download command was not working correctly - it would return "done" but not actually download files to the specified path. The command was only setting download behavior without clicking the element or waiting for completion.
**Changes made:**
- Modified `handle_download` to take a `selector` parameter and click the download element
- Added proper absolute path resolution and directory creation
- Implemented CDP event listening to wait for download completion with 30s timeout
- Added file renaming logic to handle Chrome's GUID-based temporary filenames
- Changed response format to return the actual download path
- Fixed function signature to use `&mut DaemonState` for state modifications
**Implementation details:**
- Uses `Browser.downloadWillBegin` and `Browser.downloadProgress` CDP events to track downloads
- Falls back to finding the most recently modified file if GUID capture fails
- Creates parent directories automatically if they don't exist
- Handles both absolute and relative path inputs
Fixes#965
* Address review feedback: harden download path handling
- Canonicalize download directory to prevent path traversal attacks
- Remove dangerous fallback that renamed the most-recently-modified file
in the directory (could silently rename unrelated files)
- Extract timeout to a named constant (DOWNLOAD_TIMEOUT)
* Fix download event loop: handle canceled state and Page.downloadWillBegin
- Detect "canceled" download state and return an error immediately instead
of spinning until the 30s timeout.
- Also capture the download GUID from the deprecated Page.downloadWillBegin
event for older Chrome compatibility, matching the existing
Page.downloadProgress fallback.
- Consolidate duplicated session/event checks with a shared is_this_session
variable and use match for cleaner state handling.
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
This PR fixes an issue where pressing the Enter key via the `press Enter` command had no effect on web pages, even though the command executed successfully without errors.
## Problem
The Chrome DevTools Protocol `Input.dispatchKeyEvent` requires a `text` field in `keyDown` events for certain keys to trigger their default browser actions. Without this field, keys like Enter don't actually submit forms or perform their expected actions.
## Changes
- Added new `key_text()` function that returns the appropriate text value for special keys:
- Enter returns `"\r"` (carriage return)
- Tab returns `"\t"` (tab character)
- Space returns `" "` (space character)
- Single printable characters return themselves
- Non-printable keys (Escape, Arrow keys, etc.) return `None`
- Modified `press_key_with_modifiers()` to populate both `text` and `unmodified_text` fields in the CDP keyDown event
- Added comprehensive unit tests for the new functionality
## Implementation Details
The fix ensures that when pressing Enter on a filled search box, the form actually submits as expected, matching the behavior of manually pressing the Enter key.
Fixes#966
Previously, the `--auto-connect` flag would send connection commands to the daemon on every CLI invocation, even when the daemon was already running and maintaining an active connection. This caused Chrome to repeatedly prompt for remote debugging permissions during long-running tasks.
This fix adds checks for `daemon_result.already_running` to skip sending launch commands when the daemon is already active and holding connections. The changes apply to:
- Auto-connect flow: Skip when daemon already running since it holds the connection from previous launch
- CDP connections: Skip sending commands but validate input eagerly for immediate error feedback
- Cloud provider connections: Skip when daemon already maintains the provider connection
This preserves the existing connection reuse logic in the daemon while preventing redundant connection attempts from the CLI side.
Fixes#962
* Improve upgrade command installation method detection robustness
The upgrade command was failing to detect installation method for users who installed via pnpm, yarn, or bun, showing "Could not detect installation method" errors.
## Changes Made
- **Added support for additional package managers**: Extended detection to include pnpm, yarn, and bun alongside existing npm, Homebrew, and Cargo support
- **Implemented install-time marker system**: Modified `postinstall.js` to write a `.install-method` marker file during installation, providing reliable detection that doesn't depend on fragile path heuristics
- **Enhanced path-based fallback detection**: Improved executable path analysis to better identify installation locations for all supported package managers
- **Added command probing**: Implemented fallback checks that query package managers directly to verify global installations
The detection now follows this robust hierarchy:
1. Read install-time marker file (most reliable)
2. Analyze executable path patterns
3. Probe package managers via subprocess calls
This ensures users can successfully upgrade regardless of their chosen package manager.
Fixes#954
* Add .install-method to .gitignore and note Yarn v2+ limitation
- Prevent bin/.install-method marker file from being accidentally
committed during development
- Add comment clarifying that yarn global upgrade path only works
with Yarn Classic (v1), not Yarn Berry (v2+)
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
## Summary
Fixed an issue where custom viewport settings were ignored in WebSocket streaming, causing status messages to always report hardcoded dimensions (1280x720) instead of the actual viewport size.
## Changes Made
- **Added viewport tracking to StreamServer**: New `viewport_width` and `viewport_height` fields to store current dimensions
- **Updated viewport synchronization**: Modified `handle_viewport()` and `handle_device()` to update the stream server when viewport changes
- **Fixed status message broadcasting**: Replaced hardcoded dimensions with actual viewport values in `broadcast_status()` calls
- **Improved screencast defaults**: Changed screencast to use stored viewport dimensions as defaults instead of hardcoded 1280x720
- **Added viewport getter methods**: New `set_viewport()` and `viewport()` methods on StreamServer for dimension management
This ensures that WebSocket clients receive consistent viewport dimensions across all message types (status and frame messages), matching the actual browser viewport settings.
Fixes#950
The v0.20.0 Rust rewrite dropped two keepalive mechanisms that the
Node.js/Playwright daemon provided, causing CDP connections to remote
Browserless instances to silently die between commands when traversing
multi-hop proxy topologies (Istio Envoy, OpenResty, etc.).
Restore parity with v0.19.0 and improve on it:
1. TCP SO_KEEPALIVE (v0.19.0 parity): Playwright's WebSocketTransport
used HTTP agents with keepAlive: true, which set SO_KEEPALIVE on the
underlying TCP socket. Restored via socket2::SockRef on the
tokio_tungstenite stream before splitting.
2. WebSocket Ping frames (improvement): Send Ping frames every 30s on
idle connections. This goes beyond v0.19.0 because L7 proxies (Envoy,
nginx, OpenResty) can see WebSocket pings but not TCP keepalive
probes, making this effective through application-layer proxy hops.
The keepalive task is coordinated with the reader loop via a watch
channel and stops automatically when the connection closes.
Fixes#934
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* Fix Chrome headless launch failures by adding --disable-gpu flag
Fixes silent Chrome crashes in headless mode when GPU drivers are unavailable or restricted (common in VMs, containers, and cloud environments).
## Changes Made
- **Auto-add `--disable-gpu` flag**: Automatically includes `--disable-gpu` when launching Chrome in headless mode to prevent GPU initialization crashes
- **Improved error reporting**: Enhanced error messages to include Chrome's exit code when it crashes before writing DevToolsActivePort
- **Better user guidance**: Added helpful hints in error messages suggesting `--no-sandbox` and `--disable-gpu` flags for troubleshooting
- **Updated tests**: Added test coverage for the new `--disable-gpu` flag behavior
## Implementation Details
The `--disable-gpu` flag is only added in headless mode (when `options.headless && !has_extensions`), preserving GPU acceleration for non-headless usage. The error handling now captures Chrome's exit code and provides actionable debugging information when Chrome fails silently.
Fixes#914
* Use --enable-unsafe-swiftshader instead of --disable-gpu for Playwright parity
--disable-gpu disables all GPU acceleration and breaks WebGL on Chrome 130+.
Playwright uses --enable-unsafe-swiftshader to enable CPU-based software
rendering via SwiftShader, which prevents GPU-driver crashes while preserving
WebGL support. This matches the behavior from v0.19 (Playwright-based daemon).
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Adds a new `agent-browser upgrade` command that automatically detects the installation method (npm, Homebrew, or Cargo) and runs the appropriate update command.
**Changes:**
- Added new `upgrade.rs` module with upgrade logic
- Updated `main.rs` to handle the `upgrade` command
- Added upgrade help text in `output.rs`
- Updated README.md and documentation with upgrade instructions
- Updated SKILL.md to mention the upgrade command
**Implementation details:**
- Fetches latest version from npm registry to show version diff
- Auto-detects installation method by checking Homebrew, Cargo paths, and npm global packages
- Provides fallback instructions if installation method cannot be determined
- Uses existing color module for consistent styled output
- Gracefully handles network failures and continues with upgrade
Fixes#895
* fix: prevent system package removal during Ubuntu dependency install
This PR fixes a critical issue where installing agent-browser dependencies on Ubuntu 24.04+ could trigger removal of hundreds of system packages due to library naming conflicts during the 64-bit time_t transition.
**Problem:**
On Ubuntu 24.04+, many core libraries were renamed with a "t64" suffix as part of the 64-bit time_t transition. The installer was trying to install old library names, causing apt to propose removing 400+ essential system packages to resolve conflicts.
**Changes:**
- Added comprehensive t64 variant detection for all apt dependencies
- Implemented pre-install simulation check to detect potential package removals
- Added safety abort mechanism when removals are detected (>0 packages)
- Enhanced error messaging to explain the issue and provide manual installation guidance
- Improved logging and user feedback during the installation process
**Implementation Details:**
- Uses `apt-get install --simulate` to safely check for conflicts before proceeding
- Checks for t64 variants first, falls back to original names if t64 versions don't exist
- Provides detailed output showing which packages would be removed
- Maintains compatibility with older Ubuntu versions that don't have t64 packages
Fixes#881
* refactor: extract duplicate install status handling into helper
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* Add iframe support for CLI interactions and snapshots
This PR adds comprehensive iframe support to the agent browser CLI, allowing users to interact with elements inside iframes seamlessly.
## Problem
Users couldn't interact with elements inside iframes via the command line. The existing `frame` command was non-functional as it set `active_frame_id` but no other code read this value.
## Changes Made
### Enhanced Frame Context Tracking
- Added `frame_id` field to `RefEntry` to track which frame each element reference belongs to
- Updated `RefMap::add` and related methods to accept and store frame context
- Modified element resolution functions to use frame context from ref entries
### Improved Frame Command
- Fixed the existing `frame` command to actually work by threading `active_frame_id` through snapshot operations
- Added support for iframe element references (e.g., `frame @e2`) in addition to CSS selectors
- Enhanced frame detection to work with both named frames and iframe elements
### Updated Snapshot Behavior
- Modified `take_snapshot` to accept optional frame context parameter
- Updated all snapshot call sites to pass appropriate frame context
- Maintained backward compatibility while enabling frame-scoped operations
### Element Resolution Updates
- Updated `resolve_element_center` and `resolve_element_object_id` to use frame context from ref entries
- Modified `find_node_id_by_role_name` to support frame-specific element lookup
- Ensured all interaction functions work correctly within iframe contexts
## Implementation Details
- Frame context is now properly propagated through the entire element interaction pipeline
- The `frame` command can accept both CSS selectors and element references
- All existing functionality remains intact while adding iframe capabilities
- Added `Iframe` to interactive roles for better element discovery
Fixes#863
* docs: add iframe support documentation
Document the new iframe capabilities across all documentation surfaces:
- Auto-inlining of iframe content in snapshots
- Direct interaction with iframe element refs
- frame command support for element refs (@e3)
- Scoped snapshots via frame switching
* fix: pass active frame context to diff snapshots and fix nameless iframe lookup
- handle_diff_snapshot now respects active_frame_id instead of always
passing None, so diff snapshots work correctly inside iframes
- Nameless/id-less iframes now fall back to src URL (or null) instead of
the literal string 'frame' which never matched any frame in the tree
* fix: resolve iframe frame ID via DOM.describeNode and reduce code duplication
- handle_frame: Use DOM.describeNode + contentDocument.frameId to resolve
iframe frame IDs directly, fixing failures for nameless iframes that
lack name/id/src attributes
- element.rs: Deduplicate add() by delegating to add_with_frame()
- snapshot.rs: Guard against out-of-bounds insert_str when iframe marker
is on the last line without a trailing newline
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* refactor: make --full/-f a command-level flag instead of global
Move --full/-f from global flags (parsed in flags.rs) to command-level
parsing in commands.rs, scoped to the three commands that actually use
it: `screenshot`, `diff screenshot`, and `diff url`.
This frees up `-f` for other commands (e.g. `--follow` on
`console`/`errors`, see #867) and better reflects that full-page
capture is not a global concern.
Changes:
- Remove `full` from Flags struct, Config struct, and global flag parsing
- Remove `--full`/`-f` from clean_args global boolean flags list
- Parse `--full`/`-f` inline in `screenshot` command handler
- Accept `-f` shorthand in `diff screenshot` and `diff url` (previously
only `--full` was accepted at command level)
- Remove fallback from global `flags.full` in diff subcommands
- Update tests to pass --full as a command argument rather than a global flag
Fixes#876
* fix: remove stale AGENT_BROWSER_FULL env var from help and add -f shorthand tests
- Remove AGENT_BROWSER_FULL from help text in output.rs since the env
var is no longer read after moving --full to command-level parsing
- Add test_screenshot_full_page_shorthand to verify screenshot -f works
- Add test_diff_screenshot_command_full_flag_shorthand to verify
diff screenshot -f works
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Generic elements (e.g. <div>) with cursor:pointer/onclick have empty ARIA
names because their text lives in StaticText children, which get filtered
in interactive mode. Fall back to the JS-collected textContent so the text
appears on the rendered tree line.
Fixes e2e_snapshot_cursor_many_elements CI failure from #855.
The `wait --load networkidle` command was incorrectly returning immediately when pages were served from cache, causing subsequent commands to fail. This happened because the network idle logic would return instantly when no network requests were pending, without observing any idle period.
## Changes Made
- Extract network idle polling logic into a separate `poll_network_idle` function for better testability
- Fix the timeout handling to start a 500ms idle timer when no requests are pending, instead of returning immediately
- Add comprehensive unit tests covering the regression case and normal network request flows
- Ensure the function always observes at least 500ms of network inactivity before resolving, even for cached pages
## Key Fix
The critical change is in the timeout branch: when no CDP events arrive within 600ms, we now start the idle timer if no requests are pending, rather than returning `Ok(())` immediately. This prevents false-positive idle detection for pages that load entirely from cache.
Fixes#846
* fix: resolve snapshot -C and screenshot --annotate hang over WSS (#841)
Root cause: sequential CDP round-trips per element in
find_cursor_interactive_elements() and collect_annotations() caused
timeouts over high-latency WSS connections (~200ms × 200+ elements
exceeds the 30s CDP timeout).
Fix:
- snapshot -C: Replace per-element CDP calls with a single JS eval
that detects cursor:pointer/onclick/tabindex elements in-browser,
then batch-resolve via DOM.querySelectorAll + concurrent
DOM.describeNode calls using join_all
- screenshot --annotate: Replace sequential DOM.resolveNode +
getRect calls with concurrent join_all, matching v0.19.0's
Promise.all() pattern
Behavioral parity with v0.19.0 (Node.js/Playwright):
- cursor:pointer detection via getComputedStyle
- Inherited cursor:pointer dedup (skip children of pointer parents)
- interactiveTags and interactive ARIA roles exclusion
- Role differentiation: clickable vs focusable
- Text dedup against ARIA tree ref names and quoted strings
- Edge case: -i -C shows cursor elements even when ARIA tree is empty
Tests:
- 5 unit tests for build_dedup_set() helper
- 3 e2e regression tests: cursor-interactive detection, annotation
scaling to 50 elements, cursor scaling to 100 elements
* fix: add hidden/aria-hidden filtering, contentEditable support, and cleanup robustness
- Restore hidden/aria-hidden element filtering in cursor-interactive JS
(was present in old code, dropped during rewrite)
- Add contentEditable detection with 'editable' role and hint
- Replace fire-and-forget cleanup with warning on failure
- Simplify build_dedup_set to use ref_map only (eliminates fragile
tree-text quote parsing; ref_map already has all ref-bearing names)
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: use correct Windows virtual-key codes for punctuation in type command
The `type` command was dropping punctuation characters like `.`, `'`, and
`#` because `char_to_key_info()` used raw ASCII codes as the
`windowsVirtualKeyCode` in CDP `Input.dispatchKeyEvent` calls. For
punctuation the ASCII value collides with unrelated VK codes — most
critically '.' (ASCII 46) equals VK_DELETE (0x2E), causing Chrome to
interpret periods as Delete key presses.
Changes:
- Add `punctuation_key_info()` with correct VK_OEM_* codes matching
Playwright's USKeyboardLayout (e.g. Period=190, Slash=191, Semicolon=186)
- Fall back to `Input.insertText` for characters without a US keyboard
mapping (emoji, CJK, etc.), matching Playwright's `keyboard.type()`
- Update e2e test to use `type` instead of `fill` workaround for email
- Add unit tests verifying VK code parity with Playwright's layout
Fixes#833
* style: fix rustfmt formatting for InsertTextParams
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
The v0.20.0 migration from Playwright to the native Rust daemon introduced
two regressions in checkbox/radio handling:
1. `is_element_checked` only read `this.checked`, which is undefined on
non-input elements. Material Design and ARIA controls use wrapper divs
with `role="checkbox"` and `aria-checked`, or hide the native input
off-screen inside a label. The function now mirrors Playwright's
`getChecked()` with follow-label retargeting: native `.checked`,
`aria-checked` for ARIA roles, `label.control` traversal, and nested
input lookup.
2. `check`/`uncheck` accepted the coordinate-based CDP click result
without verifying the state actually changed. When the AX tree's
`backendDOMNodeId` points to a hidden off-screen input (common in
Material Design), `Input.dispatchMouseEvent` hits nothing. The actions
now re-check state after clicking and fall back to a JS `.click()` on
the resolved input — matching Playwright's `_setChecked` verify step.
Adds e2e regression test covering Material Design (hidden input + ripple
overlay), ARIA-only, and native checkbox patterns.
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: restore WebSocket streaming in native daemon
The v0.20.0 Rust rewrite broke WebSocket streaming — connections opened
but received zero messages before closing. Multiple issues contributed:
1. StreamServer was dropped immediately after creation in daemon.rs,
closing the broadcast channel and killing all WS connections.
2. Screencast frames were only processed during command polling
(drain_cdp_events) instead of in real-time, unlike the 0.19.0
TypeScript cdp.on('Page.screencastFrame') callback.
3. Auto-start/stop screencast on WS client connect/disconnect was
missing from the Rust implementation.
4. Screencast CDP commands used the wrong session ID (daemon session
name instead of the CDP page session from Target.attachToTarget).
5. Broadcast channel Lagged errors killed WS connections instead of
being handled gracefully.
The fix adds a background CDP event loop in StreamServer that subscribes
to Chrome events and broadcasts screencast frames in real-time, properly
tracks the CDP page session ID, restores auto-screencast lifecycle, and
keeps the StreamServer alive in DaemonState.
Fixes#820
* fix: use actual CDP session ID for input dispatch in stream WebSocket
Pass the real cdp_session_id (from Target.attachToTarget) through to
handle_ws_client instead of an empty string. Previously, input commands
(mouse, keyboard, touch) were sent with `"sessionId": ""` which Chrome
silently rejects. Now the correct page session ID is read at dispatch
time, and when no session ID is set yet (before browser launch),
the field is omitted entirely via `None` so Chrome uses browser-level
dispatch.
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: snapshot --selector scopes to the matched element subtree
The native Rust daemon accepted the --selector flag but never used it —
the full accessibility tree was always returned regardless of the
selector. This restores the 0.19.0 behaviour where snapshot --selector
returns only the subtree rooted at the matched CSS selector.
The implementation resolves the selector via Runtime.evaluate, fetches
the full DOM subtree with DOM.describeNode(depth: -1) to collect all
descendant backendNodeIds, then filters the AX tree to render only the
nodes whose backendDOMNodeId falls within that set. This correctly
handles elements like <body> that don't map to a direct AX node.
Also fixes handle_snapshot reading "depth" instead of "maxDepth" from
the command JSON, which caused --depth to be silently ignored.
Fixes#822
* style: run cargo fmt on snapshot.rs
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* Improve postinstall message to detect existing Chrome installations
Previously, the npm postinstall script always recommended running `agent-browser install` to download Chrome for Testing, even when users already had a working Chrome installation on their system.
This change adds Chrome detection logic to the postinstall script that mirrors the runtime behavior:
- Checks for system Chrome installations on macOS, Linux, and Windows
- Shows a success message when Chrome is found, indicating it will be used automatically
- Only shows the `agent-browser install` warning when no Chrome is detected
- Provides platform-specific guidance (Linux `--with-deps` flag, `--executable-path` alternative)
The detection logic matches the existing Rust `find_chrome()` implementation to ensure consistency between postinstall messaging and runtime behavior.
Fixes#814
* Mention --cdp, --provider, --engine as alternatives in postinstall message
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: gracefully fall back to role/name lookup when backend_node_id is stale
When the DOM changes between snapshot and click (common with SPAs and
dynamic UIs), the stored backend_node_id becomes invalid. Previously,
DOM.getBoxModel and DOM.resolveNode failures propagated as hard errors,
bypassing the role/name fallback path entirely. Now these failures are
caught and the code falls through to a JS-based element lookup.
Also adds resolve_object_id_by_role_name so that resolve_element_object_id
has a fallback for ref-based lookups (previously it had none), and
improves the role matching JS to correctly map implicit ARIA roles
(e.g. <input type="submit"> → "button", <a href> → "link").
Closes#805
* test: add e2e regression test for stale ref click fallback (#805)
Verifies that clicking a ref whose backend_node_id has become stale
(because the DOM was replaced by JavaScript) falls back to role/name
lookup instead of failing with "Could not compute box model".
* fix: use accessibility tree for stale ref fallback instead of JS heuristic
Replace the hand-rolled JS role/name matching (getImplicitRole,
getAccessibleName) with a re-query of Accessibility.getFullAXTree —
the same data source that built the ref map during snapshot. This
guarantees role/name matching is identical to what was stored,
preventing silent wrong-element clicks from name computation
divergence (e.g. aria-labelledby, <label for>, alt text).
Matches v0.19.0 (Playwright) behavior where getByRole always
re-queried the live accessibility tree.
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Replace all `eprintln!` calls in daemon-context code with
`let _ = writeln!(std::io::stderr(), ...)` so that broken pipe errors
on stderr are silently ignored instead of panicking.
The CLI client spawns the daemon with piped stderr to capture startup
errors, then drops the pipe handle once the daemon is ready. Any
subsequent `eprintln!` in the daemon panics because Rust's `eprintln!`
macro internally unwraps the write result. This caused the reported
"failed printing to stderr: Broken pipe (os error 32)" panic during
Chrome launch on Linux.
Closes#799
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
The global-install job only uses npm (npm pack, npm install -g) but had
pnpm setup with cache enabled. On Windows, the pnpm store directory
doesn't exist since pnpm is never used, causing the Post Setup Node.js
cache step to fail with a path validation error.
The CDP event broadcast channel (capacity 256) can overflow on slow CI
runners when Chrome emits many events during navigation. Previously,
RecvError::Lagged was treated the same as RecvError::Closed, causing
spurious "Event stream closed" errors even though Chrome was still
running. Now all 5 event-receiving loops correctly continue on Lagged
instead of breaking.
Chrome uses /dev/shm for shared memory, which is typically limited to
64MB on CI runners and containers. When Chrome exhausts this, it crashes
mid-session with "Event stream closed" errors. Auto-detect CI/container
environments and pass --disable-dev-shm-usage to use /tmp instead.
The CDP WebSocket client had three issues causing snapshot to hang
indefinitely when connected to remote browsers via WSS:
1. Binary WebSocket frames were silently dropped — remote CDP proxies
(Browserless, Browserbase, etc.) may send large responses like
Accessibility.getFullAXTree as Binary frames instead of Text frames.
2. Default tungstenite size limits (16 MiB frame / 64 MiB message)
could be exceeded by large accessibility tree responses, causing the
WebSocket connection to error out and the reader task to die.
3. When the reader task died, pending commands waited for the full
30-second timeout instead of failing immediately.
Fixes#788
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Chrome occasionally crashes during startup on CI runners before
printing the DevTools URL, causing random e2e test failures across
different tests each run. Retry the launch with a 500ms delay to
handle these transient crashes.
- Restore the `refs` dictionary in `--json` snapshot output, matching the documented API contract
- The `refs` field was silently dropped during the Node.js to Rust rewrite (v0.20), causing consumers parsing `data.refs` for programmatic element interaction to receive no structured ref data
Fixes#785
Regenerate pnpm-lock.yaml to match the cleaned-up package.json (only
@changesets/cli remains). Add CI environment detection to
should_disable_sandbox() so Chrome launches with --no-sandbox on GitHub
Actions runners where AppArmor blocks unprivileged user namespaces.
This PR fixes CI build failures by addressing code formatting and linting issues that were causing the builds to fail.
**Changes made:**
1. **Rust formatting fixes in `cli/src/commands.rs`:**
- Removed unnecessary multi-line formatting for clipboard operations
- Applied consistent single-line formatting for return statements
- Fixed line length and formatting for the `test_wait_text_with_timeout` test function
2. **TypeScript fixes in `src/actions.ts`:**
- Fixed `waitForFunction` usage in the `handleWait` function by replacing the function parameter approach with a string-based implementation
- Properly escaped the text parameter using `JSON.stringify` to prevent potential injection issues
These changes ensure the code passes linting checks (clippy for Rust, ESLint for TypeScript) and formatting validation (rustfmt, prettier) that are enforced in the CI pipeline.
Fixes#751
* feat: add screenshot output config, clipboard CLI commands, and fix wait --text native path
## Summary
- Add `--screenshot-dir`, `--screenshot-quality`, and `--screenshot-format` CLI flags (with corresponding `AGENT_BROWSER_SCREENSHOT_DIR`, `AGENT_BROWSER_SCREENSHOT_QUALITY`, `AGENT_BROWSER_SCREENSHOT_FORMAT` env vars) so users can configure where and how screenshots are saved without specifying a full path every time
- Add `clipboard read`, `clipboard write <text>`, `clipboard copy`, and `clipboard paste` CLI commands, exposing the existing protocol-level clipboard handlers that were previously only accessible via JSON-RPC
- Fix `wait --text` in native mode: the CLI was emitting `selector: "text=..."` (a Playwright-style locator) which native's `querySelector` can't handle. Now emits a `text` field that correctly hits the native `wait_for_text` polling path
- Add native clipboard `copy` and `paste` support via CDP `Input.dispatchKeyEvent`, and a `write` operation to the Node.js handler
* fix: resolve CI failures in Rust formatting and TypeScript typecheck
Use string-based page.evaluate for clipboard writeText to avoid
referencing `navigator` in Node.js compilation context. Run cargo fmt
to fix formatting in commands.rs and screenshot.rs.
* fix: clipboard write captures full multi-word text
Use rest[1..].join(" ") instead of rest.get(1) so unquoted multi-word
input like `clipboard write hello world` sends the full string rather
than silently dropping everything after the first word.
* improvements
* fixes
* improvements
* improvements
Fix issue where Chrome extensions specified in the `extensions` field of `config.json` were not being loaded when launching the browser.
## Problem
Extensions configured via the `extensions` field in `config.json` were not being passed to the Chrome browser launch command, causing them to be ignored.
## Changes
- Added `!flags.extensions.is_empty()` to the launch trigger condition to ensure browser launch is triggered when extensions are configured
- Added extensions to the launch command JSON payload so they are properly passed to the browser
Fixes#726
* feat: add browserless provider integration to native browser implementation
This PR adds support for the Browserless provider to the native browser implementation, expanding the available remote browser providers from 3 to 4.
## Changes Made
- **Added `connect_browserless()` function**: Implements session creation with Browserless API using environment variables for configuration
- **Updated provider routing**: Added "browserless" case to the main provider switch statement
- **Added session cleanup**: Implemented proper session termination using the stop URL returned by Browserless
- **Updated documentation**: Modified comments and error messages to include Browserless in the supported provider list
- **Environment variable support**: Added support for configurable Browserless settings including API key, URL, browser type, TTL, and stealth mode
## Implementation Details
- Uses standard Browserless session API with POST to create sessions and DELETE to terminate
- Supports both chromium and chrome browser types with validation
- Includes proper error handling for API failures and missing configuration
- Stores the stop URL as session_id for cleanup purposes
- Follows the existing provider pattern for consistency
Fixes#744
* fix: URL-encode API key in browserless session request
Use reqwest's .query() method instead of string-formatting the token
directly into the URL, matching the Node.js implementation's use of
encodeURIComponent. Prevents malformed URLs if the API key contains
special characters.
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* Fix HTML retrieval by using browser.getLocator() for selector operations
This PR fixes an issue where HTML content retrieval was not working properly when using selectors.
**Problem:**
The `get html` command and other selector-based operations were failing because they were using `page.locator()` directly instead of the browser manager's locator method.
**Changes:**
- Updated `handleContent()` to use `browser.getLocator()` instead of `page.locator()` for HTML retrieval with selectors
- Applied the same fix to other affected functions: `handleCount()`, `handleBoundingBox()`, `handleInnerText()`, `handleInnerHtml()`, and `handleSetValue()`
- Ensures consistent locator handling across all selector-based operations
**Implementation Details:**
The fix replaces direct `page.locator(command.selector)` calls with `browser.getLocator(command.selector)` to ensure proper element selection and interaction through the browser manager's abstraction layer.
Fixes#735
* Fix remaining page.locator() calls to use browser.getLocator()
Apply the same fix to all remaining functions that were using
page.locator(command.selector) directly instead of going through
browser.getLocator(): handleWheel, handleHighlight, handleClear,
handleSelectAll, handleDispatch, handleNth, handleMultiSelect,
and handleDiffScreenshot.
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Resolves CI slowdown issues caused by limited availability of Windows containers with 8 cores by switching to the standard Windows runner image.
## Changes Made
- Updated `rust-cross` job to use `windows-latest` instead of `windows-latest-8-cores`
- Updated `windows-integration` job to use `windows-latest` instead of `windows-latest-8-cores`
- Updated `global-install` job matrix to use `windows-latest` instead of `windows-latest-8-cores`
This change trades some performance for better availability and faster CI queue times, as the standard Windows runners have much better availability than the 8-core variant.
Fixes#741
Updates the skills documentation to include the missing `vercel-sandbox` skill that was missing from both the available skills list and installation commands.
## Changes
- Added `vercel-sandbox` skill to the Available Skills list with description
- Added installation command for `vercel-sandbox` skill
- Added dedicated section for `vercel-sandbox` with key features and usage details
- Removed duplicate paragraph in the electron section
The `vercel-sandbox` skill enables running agent-browser + headless Chrome inside ephemeral Vercel Sandbox microVMs with features like snapshot startup, persistent workflows, and automatic OIDC authentication.
Fixes#712
* next.js guide
* better
* shadcn
* fixes
* fix: correct screenshot test assertion to check path instead of base64
The daemon returns { path: savePath } for screenshot commands, not base64.
* fix: cross-platform Chrome detection and gitignore hardening
- Replace hardcoded macOS Chrome path with findLocalChrome() that
searches common paths on macOS, Linux, and WSL, with a clear error
message when no Chrome is found.
- Add .env and .env*.local to .gitignore to prevent accidental
secret commits.
* fix: correct Vercel deploy button repo URL to vercel-labs/agent-browser
* clean up
* demo
* next page
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* Fix clippy warnings across CLI codebase
Fixes#653
* Fix remaining items_after_test_module clippy warnings
Move functions defined after `mod tests` blocks to before the test
modules in recording.rs and webdriver/client.rs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Fix Chrome extensions not loading by forcing headed mode when extensions present
Fixes#640
* Restore wait_or_kill() and add tests for headless+extensions logic
Restore the ChromeProcess::wait_or_kill() method that was accidentally
removed. It is still referenced by BrowserProcess in browser.rs and is
needed for graceful shutdown / cookie persistence (PR #650).
Add unit tests verifying --headless=new is omitted when extensions are
present.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix window-size leak in headed+extensions mode and remove unused channel option
- Skip --window-size=1280,720 when extensions force headed mode (native)
- Remove unexplained channel: 'chromium' from extensions launch path (TS)
- Add window-size assertion to existing extension test
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
When the daemon process crashes during startup (e.g., missing
Playwright), stderr was discarded via Stdio::null(), so users only
saw "Daemon failed to start (port: ...)" with no diagnostic info.
Now captures daemon stderr via Stdio::piped() and detects early process
exit with try_wait() during the startup polling loop. If the daemon
crashes, the actual error from stderr is shown to the user.
Also forwards --debug flag to the daemon process as AGENT_BROWSER_DEBUG
so debug logging works end-to-end.
Closes#56
Fixes#600
Three improvements to `--native` Chrome launching:
- `find_chrome()` now falls back to Playwright's browser cache (`~/.cache/ms-playwright/`) when no system Chrome is found
- Auto-detect containers/VMs (root, Docker, Podman, cgroups) and inject `--no-sandbox`
- Chrome stderr is now captured and included in launch error messages, with a hint when sandbox errors are detected
