* fix: use custom viewport dimensions in streaming frame metadata
CDP's Page.screencastFrame metadata returns physical device dimensions
instead of the emulated viewport, causing frame messages to report
incorrect deviceWidth/deviceHeight when a custom viewport is set.
Use the viewport dimensions captured at screencast start instead of
the CDP metadata values, since the screencast image is already captured
at the configured viewport size.
Closes#1031
* fix: resize browser content area on viewport change for correct
screencast dimensions
Emulation.setDeviceMetricsOverride only changes the CSS viewport, but
screencast captures the actual browser content area. This caused frame
images to have incorrect dimensions (e.g., 1000x451 instead of
1000x1000)
when a custom viewport was set.
- Call Browser.setContentsSize after setDeviceMetricsOverride so the
content area matches the emulated viewport
- Restart active screencast when viewport dimensions change so
maxWidth/maxHeight parameters are updated
- Skip redundant screencast restarts when dimensions are unchanged
- Extend E2E test to verify actual JPEG image dimensions, not just
metadata
* fix: pass viewport dimensions to --window-size at launch and log setContentsSize failures
- Add viewport_size to LaunchOptions so --window-size matches the
configured viewport from the start, reducing reliance on the
experimental Browser.setContentsSize CDP call at runtime
- Log Browser.setContentsSize failures instead of silently ignoring
them with let _ =
* fix: remove duplicate viewport change detection block (dead code from merge)
* fix: use log::debug! instead of eprintln! for setContentsSize failure
* revert: use eprintln! instead of log crate for setContentsSize failure
The daemon's stderr pipe is closed after startup, so log crate
subscribers cannot output during normal operation. eprintln! is
visible during startup and in tests, matching the existing convention.
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: promote hidden radio/checkbox inputs in snapshot refs (#1024)
When a <label> wraps a display:none <input type="radio">, Chrome
excludes the input from the accessibility tree entirely. The label
appears as role="LabelText" with an empty name, making it impossible
for AI agents to identify radio buttons via data.refs.
Detect hidden radio/checkbox inputs during cursor-interactive scanning
and promote their parent LabelText/generic nodes to the correct role
with proper name and checked state.
- Add HiddenInputKind enum to validate input types at parse boundary
- Extend cursor-interactive JS to detect hidden inputs inside elements
- Extract promote_hidden_inputs() for testable role promotion logic
- Add unit tests for promotion, name preservation, and skip conditions
* style: apply cargo fmt
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
* fix: support accessibility tree refs in upload command (#1107)
The upload command only accepted CSS selectors while click/fill supported
accessibility tree refs (e.g. e1, @e1, ref=e1). This resolves the API
inconsistency by reusing resolve_element_object_id for all selector types.
* style: apply cargo fmt
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
* fix: rewrite getByRole to use CDP accessibility tree instead of CSS selectors
The old `handle_getbyrole` generated `querySelectorAll('[role="link"], link')`
which matched `<link>` stylesheet elements instead of `<a>` anchor tags.
This happened because ARIA role names were used directly as CSS tag selectors,
and several roles differ from their HTML element names (e.g. link → a,
heading → h1-h6, textbox → input/textarea).
The fix replaces the JS-based DOM query with the CDP `Accessibility.getFullAXTree`
API, where the browser engine correctly computes implicit ARIA roles per the
WAI-ARIA / HTML-AAM spec. This is the same approach already used by `snapshot.rs`
and `element.rs` in this codebase.
Changes:
- Rewrite `handle_getbyrole` to query the browser's accessibility tree via CDP
- Add `find_ax_node_by_role` helper for AX tree traversal with role/name/exact matching
- Use `DOM.resolveNode` + `Runtime.callFunctionOn` to bridge AX node → DOM marker
- Add iframe support via `resolve_ax_session` (missing in old implementation)
- Fix cleanup to use correct CDP session (old code used default session, breaking iframe cleanup)
- Export `extract_ax_string` as `pub(super)` for reuse
- Add 4 regression tests for `find_ax_node_by_role`
Fixes#1123
* style: apply cargo fmt
* chore: remove redundant comments
* refactor: replace marker attribute with temporary ref for element resolution
Eliminates 3 CDP round-trips (DOM.resolveNode, Runtime.callFunctionOn,
Runtime.evaluate cleanup) by registering a temporary ref in the ref_map.
execute_subaction resolves the element via backendNodeId directly.
No more DOM pollution with marker attributes.
* fix: ref counter collision, ref_map leak, and stale fallback name
- Increment next_ref_num after inserting temp ref to prevent id collision
- Remove temp ref after execute_subaction to prevent unbounded ref_map growth
- Return actual AX name from find_ax_node_by_role for accurate fallback resolution
- Add RefMap::remove method
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
* fix: idle timeout not respected on Unix/macOS (#1101)
The idle sleep future was recreated inside the select loop on every
iteration. Because the drain interval ticks every 500 ms the future
was dropped and replaced before it could reach its deadline, so the
daemon never shut down.
Move the pinned Sleep future outside the loop so it survives drain
ticks and only resets on actual command receipt (reset_rx). Apply the
same fix to the Windows path where accept events caused an identical
timer reset.
* style: apply cargo fmt
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
* fix: relaunch browser when launch options change (#993)
When the daemon already held a running browser, handle_launch only
checked connection type and liveness to decide reuse. Config changes
like adding extensions to config.json were silently ignored.
Store a hash of the relaunch-relevant LaunchOptions fields and compare
on each launch command. If the hash differs the browser is closed and
relaunched with the new options.
* fmt
* fix
* fix
* fmt
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Closes#1039
- Add `preview` field to `RemoteObject` to capture CDP object previews
- Implement `format_console_arg` using preview data (value → preview → description)
- Store raw CDP args in `ConsoleEntry` and include in JSON output
- Skip typed `ConsoleApiCalledEvent` deserialization in favor of direct param extraction
- Unify console arg formatting between daemon (actions.rs) and stream (stream.rs)
Before: `console.log({userId: "abc", count: 42})` → `"Object"`
After: `console.log({userId: "abc", count: 42})` → `{userId: "abc", count: 42}`
JSON output now includes raw `args` array for programmatic access by AI agents.
Co-authored-by: hyunjinee <leehj0110@kakao.com>
* fix: detect externally opened tabs in --cdp mode (#1037)
Tabs opened outside of agent-browser (e.g. by the user or another CDP
client) were invisible to `tab list` because:
1. `Target.targetCreated` with chrome://newtab/ was filtered by
`is_internal_chrome_target`, and the subsequent `targetInfoChanged`
with the real URL could not update a target that was never tracked.
2. The background drain loop only ran when `request_tracking ||
har_recording` was active, so target events between commands were
silently dropped from the broadcast channel.
Fix: promote untracked targets in `targetInfoChanged` to new targets,
run the background drain unconditionally (guarded by browser presence),
and extract `apply_drained_events` to share target lifecycle processing
(attach, domain filter, iframe sessions) between execute_command and
the background drain.
* refactor: clean up HashSet import and remove call-site duplication
- Import HashSet alongside HashMap instead of using fully-qualified path
- Replace duplicated drain+apply sequence in execute_command with
drain_cdp_events_background call
* style: apply cargo fmt
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
The Rust rewrite of save_state only captured cookies and localStorage
for the current page's origin, silently dropping cross-domain data
(e.g. SSO/CAS auth cookies). This was a regression from the JS version.
Cookies: replace Network.getCookies with Network.getAllCookies to
return cookies from all domains the browser has visited.
localStorage: track visited origins in BrowserManager during navigation,
then collect their localStorage via a temporary CDP target with Fetch
interception (serves blank HTML to avoid real network requests).
Co-authored-by: hyunjinee <leehj0110@kakao.com>
* fix: prevent state commands from starting daemon without session_name
(#677)
State management commands (state_list, state_show, state_clear,
state_clean, state_rename) are pure file operations that don't need a
running daemon. Previously, these commands would trigger daemon
startup
via ensure_daemon(), and if AGENT_BROWSER_SESSION_NAME was exported
after the first command (e.g. `state clear --all`), the daemon would
start without session_name. Subsequent open/close commands would
reuse
that daemon, causing close to skip state persistence entirely.
Fix: execute state management commands locally in the CLI process
before
ensure_daemon() is called. This is done via a new
dispatch_state_command() function in state.rs that centralizes the
command routing, used by both the CLI (local path) and the daemon
(batch/IPC path).
Also:
- Add OutputOptions::from_flags() helper to deduplicate construction
- Add unit tests for dispatch_state_command routing and error
handling
* style: fix fmt and clippy warnings
- Remove redundant closure in dispatch_state_command (clippy::redundant_closure)
- Remove needless borrow in run_batch (clippy::needless_borrow)
- Fix trailing blank lines (rustfmt)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: remove dead code and unused variables in actions.rs
Remove `daemon_state_from_env` (superseded by `DaemonState::new`) and
`resolve_semantic_locator` (superseded by `handle_semantic_locator`),
both of which had zero call sites. Also remove unused `_session_id`
bindings in `handle_find` and `handle_multiselect`, and a no-op
`let _ = sid` in `handle_getbyrole`.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: simplify `_sid` to `_` per review feedback
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: use `is_ok()` instead of `if let Ok(_)` for idiomatic Rust
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: support xpath= selector prefix in element resolution
Resolves#907. When a selector starts with "xpath=", use
document.evaluate() instead of document.querySelector() so that
XPath expressions like "xpath=//button" work correctly.
* test: replace overlapping test with edge case tests
Replace test_build_selector_js_xpath_strips_prefix (which overlapped
with the xpath test) with two edge case tests: empty xpath and
selector starting with "xpath" without "=" delimiter.
* fix: support xpath= selector in resolve_element_object_id
Apply the same xpath= handling to resolve_element_object_id, which is
used by type, fill, focus, hover, check, select, screenshot, drag,
and all other selector-based commands beyond basic click.
* refactor: extract build_find_element_js to deduplicate xpath/css logic
The xpath= vs querySelector branching was duplicated in both
build_selector_js and resolve_element_object_id. Extract the shared
logic into build_find_element_js and reuse it in both places.
* fix: support xpath= selector in get_element_count
Use ORDERED_NODE_SNAPSHOT_TYPE with snapshotLength for XPath counting,
matching the querySelectorAll().length behavior for CSS selectors.
* refactor: rename find to find_expr for clarity
* refactor: extract build_count_elements_js and add regression tests
Extract element counting JS generation into build_count_elements_js
helper (matching the pattern of build_find_element_js) and add tests
for both CSS and XPath counting paths.
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
* feat: fall back to ws://host:port/devtools/browser when HTTP discovery
fails
Chrome 136+ with UI-based remote debugging (chrome://inspect) exposes
CDP over WebSocket but does not serve /json/version or /json/list HTTP
endpoints. Add a third fallback in discover_cdp_url_with_timeout() that
connects directly to ws://host:port/devtools/browser and verifies the
endpoint with Browser.getVersion.
Fixes#870
* chore
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
* fix: support remote host in CDP discovery (#851)
`discover_cdp_url` now accepts a host parameter instead of hardcoding
127.0.0.1, allowing `connect "http://<remote-ip>:<port>"` to query the
correct remote `/json/version` endpoint. The returned webSocketDebuggerUrl
is rewritten to match the requested host and port, since Chrome always
reports 127.0.0.1 regardless of the interface it was reached through.
* style: apply cargo fmt
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: unify discover_cdp_url and discover_cdp_url_with_request_timeout
Merge the two discovery functions into discover_cdp_url(host, port) and
discover_cdp_url_with_timeout(host, port, timeout), eliminating duplicated
logic.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: merge discover_cdp_url into single function with optional timeout
Replace discover_cdp_url + discover_cdp_url_with_timeout with a single
discover_cdp_url(host, port, timeout) where timeout is Option<Duration>.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: replace Option<Duration> with separate discover_cdp_url_with_timeout
Split back into two functions for cleaner call sites:
- discover_cdp_url(host, port) for default timeout
- discover_cdp_url_with_timeout(host, port, timeout) for custom timeout
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: bracket IPv6 addresses in CDP discovery HTTP URL
Extract bracket_ipv6 helper and apply it in fetch_cdp_info to produce
valid URLs like http://[::1]:9222/json/version instead of malformed
http://::1:9222/json/version.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove obsolete BrowserManager TypeScript API references from README
The TypeScript src/ was removed in commit 8e43469 (full native #754),
but README still referenced the non-existent BrowserManager API.
Replace Lambda example with CLI-based handler and remove Programmatic API section.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: replace screenshot polling with screencast-based piped ffmpeg recording
Recording previously used Page.captureScreenshot polling at 10fps,
which was CPU-heavy and produced inconsistent results. Now uses
Page.startScreencast with throttled acks (35ms interval) to receive
frames event-driven from Chrome, and pipes JPEG data directly to
ffmpeg stdin in real-time instead of saving temp files.
- Spawn ffmpeg at recording start with piped stdin (image2pipe)
- Background task receives screencast frames, interpolates gaps by
repeating the last frame based on timestamps, targets 25fps
- Ack throttling controls Chrome's frame push rate
- Fix: current frame was never written after the first one
- Fix: frame count was read before task finished padding
- Remove tokio-util dependency (replaced CancellationToken with oneshot)
- Add tokio "process" feature for async child process stdin pipe
- Extract start/stop_recording_task helpers on DaemonState
- Add tests for restart, ffmpeg codec selection, and stop without task
* fmt
* chore
* fix: switch WebM codec from VP9 to VP8 for correct framerate and browser
compatibility
VP9 realtime encoder ignored input framerate, producing 10fps output
instead of 25fps. This caused inconsistent playback in browsers.
VP8 respects -framerate 25 and has wider browser playback support.
* fmt
* fix: add kill_on_drop to ffmpeg process to prevent zombie on task panic
* fix: switch from screencast to screenshot polling for reliable recording duration
Screencast only pushes frames on visual changes, producing short videos
on static pages. Screenshot polling captures at a fixed 10fps interval
regardless of page activity, guaranteeing duration matches wall-clock time.
ffmpeg piped stdin architecture is preserved — no temp files.
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
* fix: use VP9 codec for webm recording output
The recording command hardcoded libx264 (H.264) which is incompatible
with the WebM container format. WebM only supports VP8/VP9/AV1 codecs,
causing ffmpeg to fail when users specify a .webm output file.
Select codec based on output file extension: libvpx-vp9 for .webm,
libx264 for other formats.
Fixes#778
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: use CRF mode for VP9 webm encoding
Switch from bitrate target (-b:v 2M) to constant quality mode (-crf 30),
which is the standard approach for screen recording (used by Puppeteer
and recommended by ffmpeg VP9 guide). CRF adapts bitrate to scene
complexity for more consistent quality.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add -b:v 0 for true constant quality VP9 encoding
Without -b:v 0, libvpx-vp9 uses its default bitrate target alongside
-crf, resulting in constrained quality mode instead of true constant
quality mode.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: pad video dimensions to even numbers for h264 compatibility
libx264 requires width and height to be divisible by 2, but CDP
screencast can capture frames with odd dimensions (e.g. 1280x577).
Add pad filter to ensure even dimensions for all codecs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The comment said "Ignore SIGPIPE" but the code actually resets SIGPIPE
to SIG_DFL (default behavior = process termination), not SIG_IGN (ignore).
Updated the comment to accurately describe what the code does and why.
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: isolate getEncryptionKey tests from local filesystem
Tests for getEncryptionKey() failed on machines where
~/.agent-browser/.encryption-key existed, because the file-based
fallback was not mocked out. Mock node:fs to isolate both env var
and key file paths, and add missing tests for the file fallback.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: clean up fs mock naming in encryption tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: sanitize lone Unicode surrogates in snapshot and response serialization (#635)
Pages with emoji/special characters can contain lone surrogates (e.g. \uD800
without a matching \uDC00-\uDFFF), causing serde_json to fail with
"unexpected end of hex escape" when parsing the JSON response.
- Add sanitizeSurrogates() to replace lone surrogates with U+FFFD in
ariaSnapshot output
- Add sanitizeJsonSurrogates() safety net in serializeResponse for other
response fields (page.title, page.content, etc.)
- Add tests for both sanitization paths
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: remove trivial "no surrogates unchanged" test
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* ci: retry flaky Rust test
* refactor: remove unnecessary sanitizeSurrogates from snapshot.ts
Chromium's ariaSnapshot() converts lone surrogates to literal text
(e.g. the 6-char string "\ud800"), not actual surrogate code points.
The real fix is sanitizeJsonSurrogates() in protocol.ts which handles
eval and other response paths where actual surrogates appear.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: use toWellFormed() instead of regex for lone surrogate sanitization
Upgrade tsconfig target/lib from ES2022 to ES2024 and replace the
manual regex-based surrogate sanitization with String.prototype.toWellFormed().
This is simpler, more readable, and relies on the standard API.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: remove trivial no-surrogate test
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Use 127.0.0.1 instead of localhost when constructing CDP URL from port
number, since Chrome only binds to IPv4. This prevents connection
failures on systems like Ubuntu 24.04 where localhost resolves to ::1.
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Add `cargo clippy -- -D warnings` step to the Rust CI job so that
clippy warnings fail the build. Also fix the one new lint
(`unnecessary_map_or`) introduced in the current stable clippy.
Fixes#672
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: resolve unnamed element refs matching multiple elements (#500)
When a page has one unnamed button among several named buttons,
clicking its ref fails with "matched N elements" because the
locator `getByRole('button')` matches all buttons on the page.
Normalize unnamed interactive elements to `name: ""` so the
selector becomes `getByRole('button', { name: "", exact: true })`
which matches only buttons with empty accessible names.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: remove dead code branch in buildSelector
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: make RefMap.name required string, remove dead code branches
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
