- Windows: match "actively refused it" error message in
download_bytes_connection_refused test (os error 10061)
- E2E relaunch: use userAgent instead of extensions to trigger
relaunch, since extensions force headed mode which requires a
display server unavailable in CI
- E2E auth_login SPA: use addEventListener instead of inline
onsubmit for more reliable form submission prevention
* fix: support accessibility tree refs in upload command (#1107)
The upload command only accepted CSS selectors while click/fill supported
accessibility tree refs (e.g. e1, @e1, ref=e1). This resolves the API
inconsistency by reusing resolve_element_object_id for all selector types.
* style: apply cargo fmt
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
* v0.24.1
* fix: e2e test failures on CI
- e2e_relaunch_on_options_change: use headless for all launches;
the third launch only changes extensions, which is sufficient to
trigger the relaunch hash mismatch without needing an X display
- e2e_auth_login flake: reduce SPA render delay from 1200ms to 800ms
to add headroom within the 5s preferred selector window on slower
CI runners
* fix: relaunch browser when launch options change (#993)
When the daemon already held a running browser, handle_launch only
checked connection type and liveness to decide reuse. Config changes
like adding extensions to config.json were silently ignored.
Store a hash of the relaunch-relevant LaunchOptions fields and compare
on each launch command. If the hash differs the browser is closed and
relaunched with the new options.
* fmt
* fix
* fix
* fmt
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: include buttons bitmask in drag mouseMoved events
The drag handler was omitting the `buttons` field from every
`mouseMoved` event dispatched during the move phase. Without it the
browser sees `event.buttons === 0`, meaning no button is held, so
`dragstart`/`dragover`/`drop` never fire and the drop target never
receives the element.
Fix:
- Add `"buttons": 1` (left-button mask) to each `mouseMoved` sent
while the button is held.
- Add `"buttons": 1` to `mousePressed` and `"buttons": 0` to
`mouseReleased`, consistent with how `dispatch_click` handles the
same fields in interaction.rs.
- Correct the parity-test fixture for `drag`, which was supplying a
`selector` key instead of the `source` key that `handle_drag` reads.
- Add an e2e test (`e2e_drag_action_sends_buttons_during_move`) that
drives the high-level `drag` action against the existing
`html5_drag_probe` fixture and asserts that `mousemove` events carry
`buttons == 1` and that `dragstart` fires on the source element.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* style: fix rustfmt formatting in e2e drag test
---------
Co-authored-by: wangjingjing <wangjingjing.99@bytedance.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Chrome returns loader_id: None for same-document navigations (e.g., hash
routing in SPAs). In these cases, Page.loadEventFired never fires, causing
wait_for_lifecycle to hang forever.
The fix checks nav_result.loader_id.is_some() before waiting for lifecycle
events. Also added regression test e2e_navigate_same_url_twice_should_not_hang.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: detect externally opened tabs in --cdp mode (#1037)
Tabs opened outside of agent-browser (e.g. by the user or another CDP
client) were invisible to `tab list` because:
1. `Target.targetCreated` with chrome://newtab/ was filtered by
`is_internal_chrome_target`, and the subsequent `targetInfoChanged`
with the real URL could not update a target that was never tracked.
2. The background drain loop only ran when `request_tracking ||
har_recording` was active, so target events between commands were
silently dropped from the broadcast channel.
Fix: promote untracked targets in `targetInfoChanged` to new targets,
run the background drain unconditionally (guarded by browser presence),
and extract `apply_drained_events` to share target lifecycle processing
(attach, domain filter, iframe sessions) between execute_command and
the background drain.
* refactor: clean up HashSet import and remove call-site duplication
- Import HashSet alongside HashMap instead of using fully-qualified path
- Replace duplicated drain+apply sequence in execute_command with
drain_cdp_events_background call
* style: apply cargo fmt
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
The Rust rewrite of save_state only captured cookies and localStorage
for the current page's origin, silently dropping cross-domain data
(e.g. SSO/CAS auth cookies). This was a regression from the JS version.
Cookies: replace Network.getCookies with Network.getAllCookies to
return cookies from all domains the browser has visited.
localStorage: track visited origins in BrowserManager during navigation,
then collect their localStorage via a temporary CDP target with Fetch
interception (serves blank HTML to avoid real network requests).
Co-authored-by: hyunjinee <leehj0110@kakao.com>
* fix: add ref for cursor-interactive content roles
* fix: format
* feat: always include cursor-interactive elements in snapshot, -C is deprecated
* feat: process StaticText aggregation and deduplication
* update test
* clean up
* fix: escape text of elements in snapshot
* fix: redundant slicing
* fix: cargo fmt
* feat: deduplicate redundant StaticText
---------
Co-authored-by: 羲洋 <lipengyang.lpy@alibaba-inc.com>
Navigate with load, then wait for username/password/submit selectors using the default action timeout. This avoids networkidle hangs on pages with continuous background requests.
* fix: restore origin-scoped --headers persistence across commands
In the v0.20 Rust rewrite, headers passed via --headers on open were
only applied to that single navigation via Network.setExtraHTTPHeaders,
which did not persist them for subsequent commands. In v0.19
(Playwright-based), these headers persisted for all subsequent
same-origin requests.
This restores the v0.19 behavior using CDP Fetch interception:
- A background task processes Fetch.requestPaused events in real-time,
injecting origin-scoped headers into matching requests and continuing
non-matching requests unmodified. This avoids the deadlock that occurs
when Fetch interception pauses requests during Page.navigate or
Runtime.evaluate (which block waiting for completion).
- The same background task also handles domain filtering and route
interception, replacing the previous drain-between-commands approach
that couldn't process events during navigation or script evaluation.
Fixes:
- --headers persist for same-origin navigations without re-passing flag
- --headers persist for in-page fetch/XHR to the same origin
- --headers do not leak to cross-origin navigations or sub-resources
- `set headers` (global) is unaffected and stacks with --headers
- Domain filter Fetch interception no longer deadlocks during navigation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: apply cargo fmt formatting
* revert inaccurate comment change
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* feat: embed cursor-interactive elements into snapshot tree
* optimize format
* fix: address review feedback for e2e_snapshot_cursor_interactive unitest
---------
Co-authored-by: 羲洋 <lipengyang.lpy@alibaba-inc.com>
* fix: resolve snapshot -C and screenshot --annotate hang over WSS (#841)
Root cause: sequential CDP round-trips per element in
find_cursor_interactive_elements() and collect_annotations() caused
timeouts over high-latency WSS connections (~200ms × 200+ elements
exceeds the 30s CDP timeout).
Fix:
- snapshot -C: Replace per-element CDP calls with a single JS eval
that detects cursor:pointer/onclick/tabindex elements in-browser,
then batch-resolve via DOM.querySelectorAll + concurrent
DOM.describeNode calls using join_all
- screenshot --annotate: Replace sequential DOM.resolveNode +
getRect calls with concurrent join_all, matching v0.19.0's
Promise.all() pattern
Behavioral parity with v0.19.0 (Node.js/Playwright):
- cursor:pointer detection via getComputedStyle
- Inherited cursor:pointer dedup (skip children of pointer parents)
- interactiveTags and interactive ARIA roles exclusion
- Role differentiation: clickable vs focusable
- Text dedup against ARIA tree ref names and quoted strings
- Edge case: -i -C shows cursor elements even when ARIA tree is empty
Tests:
- 5 unit tests for build_dedup_set() helper
- 3 e2e regression tests: cursor-interactive detection, annotation
scaling to 50 elements, cursor scaling to 100 elements
* fix: add hidden/aria-hidden filtering, contentEditable support, and cleanup robustness
- Restore hidden/aria-hidden element filtering in cursor-interactive JS
(was present in old code, dropped during rewrite)
- Add contentEditable detection with 'editable' role and hint
- Replace fire-and-forget cleanup with warning on failure
- Simplify build_dedup_set to use ref_map only (eliminates fragile
tree-text quote parsing; ref_map already has all ref-bearing names)
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: use correct Windows virtual-key codes for punctuation in type command
The `type` command was dropping punctuation characters like `.`, `'`, and
`#` because `char_to_key_info()` used raw ASCII codes as the
`windowsVirtualKeyCode` in CDP `Input.dispatchKeyEvent` calls. For
punctuation the ASCII value collides with unrelated VK codes — most
critically '.' (ASCII 46) equals VK_DELETE (0x2E), causing Chrome to
interpret periods as Delete key presses.
Changes:
- Add `punctuation_key_info()` with correct VK_OEM_* codes matching
Playwright's USKeyboardLayout (e.g. Period=190, Slash=191, Semicolon=186)
- Fall back to `Input.insertText` for characters without a US keyboard
mapping (emoji, CJK, etc.), matching Playwright's `keyboard.type()`
- Update e2e test to use `type` instead of `fill` workaround for email
- Add unit tests verifying VK code parity with Playwright's layout
Fixes#833
* style: fix rustfmt formatting for InsertTextParams
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
The v0.20.0 migration from Playwright to the native Rust daemon introduced
two regressions in checkbox/radio handling:
1. `is_element_checked` only read `this.checked`, which is undefined on
non-input elements. Material Design and ARIA controls use wrapper divs
with `role="checkbox"` and `aria-checked`, or hide the native input
off-screen inside a label. The function now mirrors Playwright's
`getChecked()` with follow-label retargeting: native `.checked`,
`aria-checked` for ARIA roles, `label.control` traversal, and nested
input lookup.
2. `check`/`uncheck` accepted the coordinate-based CDP click result
without verifying the state actually changed. When the AX tree's
`backendDOMNodeId` points to a hidden off-screen input (common in
Material Design), `Input.dispatchMouseEvent` hits nothing. The actions
now re-check state after clicking and fall back to a JS `.click()` on
the resolved input — matching Playwright's `_setChecked` verify step.
Adds e2e regression test covering Material Design (hidden input + ripple
overlay), ARIA-only, and native checkbox patterns.
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: gracefully fall back to role/name lookup when backend_node_id is stale
When the DOM changes between snapshot and click (common with SPAs and
dynamic UIs), the stored backend_node_id becomes invalid. Previously,
DOM.getBoxModel and DOM.resolveNode failures propagated as hard errors,
bypassing the role/name fallback path entirely. Now these failures are
caught and the code falls through to a JS-based element lookup.
Also adds resolve_object_id_by_role_name so that resolve_element_object_id
has a fallback for ref-based lookups (previously it had none), and
improves the role matching JS to correctly map implicit ARIA roles
(e.g. <input type="submit"> → "button", <a href> → "link").
Closes#805
* test: add e2e regression test for stale ref click fallback (#805)
Verifies that clicking a ref whose backend_node_id has become stale
(because the DOM was replaced by JavaScript) falls back to role/name
lookup instead of failing with "Could not compute box model".
* fix: use accessibility tree for stale ref fallback instead of JS heuristic
Replace the hand-rolled JS role/name matching (getImplicitRole,
getAccessibleName) with a re-query of Accessibility.getFullAXTree —
the same data source that built the ref map during snapshot. This
guarantees role/name matching is identical to what was stored,
preventing silent wrong-element clicks from name computation
divergence (e.g. aria-labelledby, <label for>, alt text).
Matches v0.19.0 (Playwright) behavior where getByRole always
re-queried the live accessibility tree.
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Chris Tate