Generic elements (e.g. <div>) with cursor:pointer/onclick have empty ARIA
names because their text lives in StaticText children, which get filtered
in interactive mode. Fall back to the JS-collected textContent so the text
appears on the rendered tree line.
Fixes e2e_snapshot_cursor_many_elements CI failure from #855.
* fix: Windows auto-connect profiling (#835)
Fix three interrelated bugs causing `--auto-connect` to fail on Windows,
plus a UX issue where auto-connect hijacked existing tabs:
1. Stale DevToolsActivePort — add TCP port liveness check before returning
M144+ WebSocket URL; remove stale files when port is dead.
2. Missing Windows error codes — add os error 10061 (WSAECONNREFUSED) and
10054 (WSAECONNRESET) to is_transient_error() so daemon startup races
are retried on Windows.
3. --auto-connect not propagated to daemon — add auto_connect to
DaemonOptions, set AGENT_BROWSER_AUTO_CONNECT env var via
apply_daemon_env(), and guard the headed launch block so it doesn't
send a second launch that overrides the auto-connect.
4. Auto-connect opens a fresh tab — after connecting to an existing
Chrome, create a new about:blank tab and bring it to front so
navigations don't hijack the user's existing tabs.
Made-with: Cursor
* fix: address review feedback — cargo fmt, shared helper, Windows tests
- Run cargo fmt on is_port_reachable() formatting
- Extract duplicated auto-connect-with-fresh-tab logic into
connect_auto_with_fresh_tab() helper used by both handle_launch()
and auto_launch()
- Add unit tests for Windows WSAECONNREFUSED (os error 10061) and
WSAECONNRESET (os error 10054) in is_transient_error()
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: propagate --cdp flag to daemon via AGENT_BROWSER_CDP env var
The --cdp flag was not being passed to the daemon process as an environment
variable, causing auto-reconnection to fail. The daemon's auto_launch()
function checks for AGENT_BROWSER_CDP, but this was never set when spawning
the daemon.
This commit adds:
- cdp field to DaemonOptions struct
- AGENT_BROWSER_CDP env var setting in apply_daemon_env()
- flags.cdp propagation in main.rs
This ensures reliable CDP connection recovery when using --cdp with external
browsers like Lightpanda, Electron apps, or remote Chrome instances.
Fixes reconnection issues with --cdp flag after connection drops.
* chore: remove changeset
---------
Co-authored-by: Jake Shore <jakeshore@Jakes-Mac-mini.local>
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* feat: embed cursor-interactive elements into snapshot tree
* optimize format
* fix: address review feedback for e2e_snapshot_cursor_interactive unitest
---------
Co-authored-by: 羲洋 <lipengyang.lpy@alibaba-inc.com>
* feat: add --idle-timeout CLI flag for daemon auto-shutdown
Add user-friendly --idle-timeout flag that converts time strings
to milliseconds. Supports formats like '10s', '3m', '1h', or raw ms.
This addresses a common need for ephemeral/CI environments where
daemon processes can be orphaned if not explicitly closed, leading
to resource consumption from zombie chrome-headless-shell processes.
Co-authored-by: Hermes (via claude-sonnet-4-20250520) <agent@hermes.ai>
* fix: address idle-timeout review feedback
* fix: normalize idle-timeout parsing
---------
Co-authored-by: Merlin <merlin@rbeckner.com>
Co-authored-by: Hermes (via claude-sonnet-4-20250520) <agent@hermes.ai>
* fix: support remote host in CDP discovery (#851)
`discover_cdp_url` now accepts a host parameter instead of hardcoding
127.0.0.1, allowing `connect "http://<remote-ip>:<port>"` to query the
correct remote `/json/version` endpoint. The returned webSocketDebuggerUrl
is rewritten to match the requested host and port, since Chrome always
reports 127.0.0.1 regardless of the interface it was reached through.
* style: apply cargo fmt
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: unify discover_cdp_url and discover_cdp_url_with_request_timeout
Merge the two discovery functions into discover_cdp_url(host, port) and
discover_cdp_url_with_timeout(host, port, timeout), eliminating duplicated
logic.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: merge discover_cdp_url into single function with optional timeout
Replace discover_cdp_url + discover_cdp_url_with_timeout with a single
discover_cdp_url(host, port, timeout) where timeout is Option<Duration>.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: replace Option<Duration> with separate discover_cdp_url_with_timeout
Split back into two functions for cleaner call sites:
- discover_cdp_url(host, port) for default timeout
- discover_cdp_url_with_timeout(host, port, timeout) for custom timeout
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: bracket IPv6 addresses in CDP discovery HTTP URL
Extract bracket_ipv6 helper and apply it in fetch_cdp_info to produce
valid URLs like http://[::1]:9222/json/version instead of malformed
http://::1:9222/json/version.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The `wait --load networkidle` command was incorrectly returning immediately when pages were served from cache, causing subsequent commands to fail. This happened because the network idle logic would return instantly when no network requests were pending, without observing any idle period.
## Changes Made
- Extract network idle polling logic into a separate `poll_network_idle` function for better testability
- Fix the timeout handling to start a 500ms idle timer when no requests are pending, instead of returning immediately
- Add comprehensive unit tests covering the regression case and normal network request flows
- Ensure the function always observes at least 500ms of network inactivity before resolving, even for cached pages
## Key Fix
The critical change is in the timeout branch: when no CDP events arrive within 600ms, we now start the idle timer if no requests are pending, rather than returning `Ok(())` immediately. This prevents false-positive idle detection for pages that load entirely from cache.
Fixes#846
* fix: resolve snapshot -C and screenshot --annotate hang over WSS (#841)
Root cause: sequential CDP round-trips per element in
find_cursor_interactive_elements() and collect_annotations() caused
timeouts over high-latency WSS connections (~200ms × 200+ elements
exceeds the 30s CDP timeout).
Fix:
- snapshot -C: Replace per-element CDP calls with a single JS eval
that detects cursor:pointer/onclick/tabindex elements in-browser,
then batch-resolve via DOM.querySelectorAll + concurrent
DOM.describeNode calls using join_all
- screenshot --annotate: Replace sequential DOM.resolveNode +
getRect calls with concurrent join_all, matching v0.19.0's
Promise.all() pattern
Behavioral parity with v0.19.0 (Node.js/Playwright):
- cursor:pointer detection via getComputedStyle
- Inherited cursor:pointer dedup (skip children of pointer parents)
- interactiveTags and interactive ARIA roles exclusion
- Role differentiation: clickable vs focusable
- Text dedup against ARIA tree ref names and quoted strings
- Edge case: -i -C shows cursor elements even when ARIA tree is empty
Tests:
- 5 unit tests for build_dedup_set() helper
- 3 e2e regression tests: cursor-interactive detection, annotation
scaling to 50 elements, cursor scaling to 100 elements
* fix: add hidden/aria-hidden filtering, contentEditable support, and cleanup robustness
- Restore hidden/aria-hidden element filtering in cursor-interactive JS
(was present in old code, dropped during rewrite)
- Add contentEditable detection with 'editable' role and hint
- Replace fire-and-forget cleanup with warning on failure
- Simplify build_dedup_set to use ref_map only (eliminates fragile
tree-text quote parsing; ref_map already has all ref-bearing names)
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: use correct Windows virtual-key codes for punctuation in type command
The `type` command was dropping punctuation characters like `.`, `'`, and
`#` because `char_to_key_info()` used raw ASCII codes as the
`windowsVirtualKeyCode` in CDP `Input.dispatchKeyEvent` calls. For
punctuation the ASCII value collides with unrelated VK codes — most
critically '.' (ASCII 46) equals VK_DELETE (0x2E), causing Chrome to
interpret periods as Delete key presses.
Changes:
- Add `punctuation_key_info()` with correct VK_OEM_* codes matching
Playwright's USKeyboardLayout (e.g. Period=190, Slash=191, Semicolon=186)
- Fall back to `Input.insertText` for characters without a US keyboard
mapping (emoji, CJK, etc.), matching Playwright's `keyboard.type()`
- Update e2e test to use `type` instead of `fill` workaround for email
- Add unit tests verifying VK code parity with Playwright's layout
Fixes#833
* style: fix rustfmt formatting for InsertTextParams
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
The v0.20.0 migration from Playwright to the native Rust daemon introduced
two regressions in checkbox/radio handling:
1. `is_element_checked` only read `this.checked`, which is undefined on
non-input elements. Material Design and ARIA controls use wrapper divs
with `role="checkbox"` and `aria-checked`, or hide the native input
off-screen inside a label. The function now mirrors Playwright's
`getChecked()` with follow-label retargeting: native `.checked`,
`aria-checked` for ARIA roles, `label.control` traversal, and nested
input lookup.
2. `check`/`uncheck` accepted the coordinate-based CDP click result
without verifying the state actually changed. When the AX tree's
`backendDOMNodeId` points to a hidden off-screen input (common in
Material Design), `Input.dispatchMouseEvent` hits nothing. The actions
now re-check state after clicking and fall back to a JS `.click()` on
the resolved input — matching Playwright's `_setChecked` verify step.
Adds e2e regression test covering Material Design (hidden input + ripple
overlay), ARIA-only, and native checkbox patterns.
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
When using --auto-connect, discover_and_attach_targets() was selecting
Chrome internal pages (chrome://, chrome-extension://, devtools://) as
the active target. Follow-up commands like `get url` and `snapshot`
would then return data from targets like chrome://omnibox-popup.top-chrome/
instead of the actual application tab.
Add is_internal_chrome_target() filter to exclude internal Chrome targets
from the discovery results. If no user-facing targets remain after
filtering, the existing "create a new tab" fallback handles it.
Fixes#813
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
* fix: restore WebSocket streaming in native daemon
The v0.20.0 Rust rewrite broke WebSocket streaming — connections opened
but received zero messages before closing. Multiple issues contributed:
1. StreamServer was dropped immediately after creation in daemon.rs,
closing the broadcast channel and killing all WS connections.
2. Screencast frames were only processed during command polling
(drain_cdp_events) instead of in real-time, unlike the 0.19.0
TypeScript cdp.on('Page.screencastFrame') callback.
3. Auto-start/stop screencast on WS client connect/disconnect was
missing from the Rust implementation.
4. Screencast CDP commands used the wrong session ID (daemon session
name instead of the CDP page session from Target.attachToTarget).
5. Broadcast channel Lagged errors killed WS connections instead of
being handled gracefully.
The fix adds a background CDP event loop in StreamServer that subscribes
to Chrome events and broadcasts screencast frames in real-time, properly
tracks the CDP page session ID, restores auto-screencast lifecycle, and
keeps the StreamServer alive in DaemonState.
Fixes#820
* fix: use actual CDP session ID for input dispatch in stream WebSocket
Pass the real cdp_session_id (from Target.attachToTarget) through to
handle_ws_client instead of an empty string. Previously, input commands
(mouse, keyboard, touch) were sent with `"sessionId": ""` which Chrome
silently rejects. Now the correct page session ID is read at dispatch
time, and when no session ID is set yet (before browser launch),
the field is omitted entirely via `None` so Chrome uses browser-level
dispatch.
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: snapshot --selector scopes to the matched element subtree
The native Rust daemon accepted the --selector flag but never used it —
the full accessibility tree was always returned regardless of the
selector. This restores the 0.19.0 behaviour where snapshot --selector
returns only the subtree rooted at the matched CSS selector.
The implementation resolves the selector via Runtime.evaluate, fetches
the full DOM subtree with DOM.describeNode(depth: -1) to collect all
descendant backendNodeIds, then filters the AX tree to render only the
nodes whose backendDOMNodeId falls within that set. This correctly
handles elements like <body> that don't map to a direct AX node.
Also fixes handle_snapshot reading "depth" instead of "maxDepth" from
the command JSON, which caused --depth to be silently ignored.
Fixes#822
* style: run cargo fmt on snapshot.rs
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: add appium: vendor prefix to iOS capabilities for Appium v3
Appium v3 enforces the W3C WebDriver spec strictly, requiring
non-standard capabilities to use vendor prefixes. The iOS provider
was sending capabilities like `automationName`, `noReset`, `deviceName`,
`platformVersion`, and `udid` without the required `appium:` prefix,
causing session creation to fail with InvalidArgumentError.
This change prefixes all non-standard capabilities with `appium:` while
leaving standard W3C capabilities (`platformName`, `browserName`)
unprefixed. Backwards-compatible with Appium v2, which accepts both
formats.
Fixes#629
* fix: extract build_ios_capabilities for testable production code path
Addresses review feedback: removes unused `mut manager` warning and
validates the actual capability-building logic instead of reconstructing
JSON inline.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove obsolete BrowserManager TypeScript API references from README
The TypeScript src/ was removed in commit 8e43469 (full native #754),
but README still referenced the non-existent BrowserManager API.
Replace Lambda example with CLI-based handler and remove Programmatic API section.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: replace screenshot polling with screencast-based piped ffmpeg recording
Recording previously used Page.captureScreenshot polling at 10fps,
which was CPU-heavy and produced inconsistent results. Now uses
Page.startScreencast with throttled acks (35ms interval) to receive
frames event-driven from Chrome, and pipes JPEG data directly to
ffmpeg stdin in real-time instead of saving temp files.
- Spawn ffmpeg at recording start with piped stdin (image2pipe)
- Background task receives screencast frames, interpolates gaps by
repeating the last frame based on timestamps, targets 25fps
- Ack throttling controls Chrome's frame push rate
- Fix: current frame was never written after the first one
- Fix: frame count was read before task finished padding
- Remove tokio-util dependency (replaced CancellationToken with oneshot)
- Add tokio "process" feature for async child process stdin pipe
- Extract start/stop_recording_task helpers on DaemonState
- Add tests for restart, ffmpeg codec selection, and stop without task
* fmt
* chore
* fix: switch WebM codec from VP9 to VP8 for correct framerate and browser
compatibility
VP9 realtime encoder ignored input framerate, producing 10fps output
instead of 25fps. This caused inconsistent playback in browsers.
VP8 respects -framerate 25 and has wider browser playback support.
* fmt
* fix: add kill_on_drop to ffmpeg process to prevent zombie on task panic
* fix: switch from screencast to screenshot polling for reliable recording duration
Screencast only pushes frames on visual changes, producing short videos
on static pages. Screenshot polling captures at a fixed 10fps interval
regardless of page activity, guaranteeing duration matches wall-clock time.
ffmpeg piped stdin architecture is preserved — no temp files.
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Brave Browser is Chromium-based and uses the same DevToolsActivePort
mechanism. Add its user-data-dir paths to get_chrome_user_data_dirs()
and its executable paths to find_chrome() on all three platforms
(macOS, Linux, Windows).
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Improve postinstall message to detect existing Chrome installations
Previously, the npm postinstall script always recommended running `agent-browser install` to download Chrome for Testing, even when users already had a working Chrome installation on their system.
This change adds Chrome detection logic to the postinstall script that mirrors the runtime behavior:
- Checks for system Chrome installations on macOS, Linux, and Windows
- Shows a success message when Chrome is found, indicating it will be used automatically
- Only shows the `agent-browser install` warning when no Chrome is detected
- Provides platform-specific guidance (Linux `--with-deps` flag, `--executable-path` alternative)
The detection logic matches the existing Rust `find_chrome()` implementation to ensure consistency between postinstall messaging and runtime behavior.
Fixes#814
* Mention --cdp, --provider, --engine as alternatives in postinstall message
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: gracefully fall back to role/name lookup when backend_node_id is stale
When the DOM changes between snapshot and click (common with SPAs and
dynamic UIs), the stored backend_node_id becomes invalid. Previously,
DOM.getBoxModel and DOM.resolveNode failures propagated as hard errors,
bypassing the role/name fallback path entirely. Now these failures are
caught and the code falls through to a JS-based element lookup.
Also adds resolve_object_id_by_role_name so that resolve_element_object_id
has a fallback for ref-based lookups (previously it had none), and
improves the role matching JS to correctly map implicit ARIA roles
(e.g. <input type="submit"> → "button", <a href> → "link").
Closes#805
* test: add e2e regression test for stale ref click fallback (#805)
Verifies that clicking a ref whose backend_node_id has become stale
(because the DOM was replaced by JavaScript) falls back to role/name
lookup instead of failing with "Could not compute box model".
* fix: use accessibility tree for stale ref fallback instead of JS heuristic
Replace the hand-rolled JS role/name matching (getImplicitRole,
getAccessibleName) with a re-query of Accessibility.getFullAXTree —
the same data source that built the ref map during snapshot. This
guarantees role/name matching is identical to what was stored,
preventing silent wrong-element clicks from name computation
divergence (e.g. aria-labelledby, <label for>, alt text).
Matches v0.19.0 (Playwright) behavior where getByRole always
re-queried the live accessibility tree.
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Replace all `eprintln!` calls in daemon-context code with
`let _ = writeln!(std::io::stderr(), ...)` so that broken pipe errors
on stderr are silently ignored instead of panicking.
The CLI client spawns the daemon with piped stderr to capture startup
errors, then drops the pipe handle once the daemon is ready. Any
subsequent `eprintln!` in the daemon panics because Rust's `eprintln!`
macro internally unwraps the write result. This caused the reported
"failed printing to stderr: Broken pipe (os error 32)" panic during
Chrome launch on Linux.
Closes#799
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
The global-install job only uses npm (npm pack, npm install -g) but had
pnpm setup with cache enabled. On Windows, the pnpm store directory
doesn't exist since pnpm is never used, causing the Post Setup Node.js
cache step to fail with a path validation error.
The CDP event broadcast channel (capacity 256) can overflow on slow CI
runners when Chrome emits many events during navigation. Previously,
RecvError::Lagged was treated the same as RecvError::Closed, causing
spurious "Event stream closed" errors even though Chrome was still
running. Now all 5 event-receiving loops correctly continue on Lagged
instead of breaking.
Chrome uses /dev/shm for shared memory, which is typically limited to
64MB on CI runners and containers. When Chrome exhausts this, it crashes
mid-session with "Event stream closed" errors. Auto-detect CI/container
environments and pass --disable-dev-shm-usage to use /tmp instead.
The CDP WebSocket client had three issues causing snapshot to hang
indefinitely when connected to remote browsers via WSS:
1. Binary WebSocket frames were silently dropped — remote CDP proxies
(Browserless, Browserbase, etc.) may send large responses like
Accessibility.getFullAXTree as Binary frames instead of Text frames.
2. Default tungstenite size limits (16 MiB frame / 64 MiB message)
could be exceeded by large accessibility tree responses, causing the
WebSocket connection to error out and the reader task to die.
3. When the reader task died, pending commands waited for the full
30-second timeout instead of failing immediately.
Fixes#788
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Chrome occasionally crashes during startup on CI runners before
printing the DevTools URL, causing random e2e test failures across
different tests each run. Retry the launch with a 500ms delay to
handle these transient crashes.
