* fix: relaunch browser when launch options change (#993)
When the daemon already held a running browser, handle_launch only
checked connection type and liveness to decide reuse. Config changes
like adding extensions to config.json were silently ignored.
Store a hash of the relaunch-relevant LaunchOptions fields and compare
on each launch command. If the hash differs the browser is closed and
relaunched with the new options.
* fmt
* fix
* fix
* fmt
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
When a non-launch command (e.g. open, snapshot) triggers auto_launch()
before the explicit launch command is processed, auto_launch() now checks
AGENT_BROWSER_PROVIDER and connects via the provider API instead of
always falling back to a local Chrome instance.
Also redirects daemon stderr to /dev/null when not in debug mode to
prevent crashes from broken pipe after the CLI drops the piped stderr
handle. Cloud providers may write to stderr during connection setup.
Fixes#1125
Related: #979
The CDP event broadcast buffer (256 events) was too small for pages with
many concurrent API requests, causing silent event drops. Modern SPAs
routinely fire 100+ API calls during page load, generating 300+ CDP
network events that would overflow the buffer between drain cycles.
Changes:
- Increase CDP broadcast buffer from 256 to 4096 (event channel) and
512 to 4096 (raw channel)
- Reduce background drain interval from 500ms to 100ms
- Handle Network.loadingFailed events in HAR recording
- Enable Network.enable on cross-origin iframe sessions during HAR
recording and request tracking
- Allow Network events from iframe sessions through the session filter
- Log a warning when buffer overflow occurs instead of silently dropping
Fixes#1128
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* feat: add AWS Bedrock AgentCore browser provider (native Rust)
- Add agentcore provider with SigV4 authentication
- AWS SDK deps are optional behind 'agentcore' feature flag
- Build with: cargo build --features agentcore
- Supports AGENTCORE_REGION, AGENTCORE_PROFILE_ID, AGENTCORE_BROWSER_ID env vars
- Returns session ID and Live View URL in launch response
- Add connect_cdp_with_headers for signed WebSocket connections
* test: add unit tests for AgentCore provider
* refactor: use lightweight manual SigV4 signing instead of AWS SDK
- Replace aws-sigv4/aws-config with manual HMAC-SHA256 signing
- Removes ~60s compile time and significant binary size
- Credentials read from AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY env vars
- Supports AWS_SESSION_TOKEN for temporary credentials
* fix: correct AgentCore API endpoints
- Host: bedrock-agentcore.{region}.amazonaws.com
- Start session: PUT /browsers/{id}/sessions/start
- Stop session: PUT /browsers/{id}/sessions/stop
- Add urlencoding for browser ID in path
- Add AWS_DEFAULT_REGION fallback
* fix: use profileConfiguration.profileIdentifier for AgentCore profile
The AWS Bedrock AgentCore API expects profile configuration in the format:
{
"profileConfiguration": {
"profileIdentifier": "<profile-id>"
}
}
Not the flat "profileId" field that was previously used.
* feat: support AWS credential provider chain via AWS CLI
- Try env vars first (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
- Fall back to 'aws configure export-credentials --format env'
- Honor AWS_PROFILE environment variable
- Works with SSO, IAM roles, credential files, etc.
---------
Co-authored-by: Chris Tate <chris@ctate.dev>
* fix: include buttons bitmask in drag mouseMoved events
The drag handler was omitting the `buttons` field from every
`mouseMoved` event dispatched during the move phase. Without it the
browser sees `event.buttons === 0`, meaning no button is held, so
`dragstart`/`dragover`/`drop` never fire and the drop target never
receives the element.
Fix:
- Add `"buttons": 1` (left-button mask) to each `mouseMoved` sent
while the button is held.
- Add `"buttons": 1` to `mousePressed` and `"buttons": 0` to
`mouseReleased`, consistent with how `dispatch_click` handles the
same fields in interaction.rs.
- Correct the parity-test fixture for `drag`, which was supplying a
`selector` key instead of the `source` key that `handle_drag` reads.
- Add an e2e test (`e2e_drag_action_sends_buttons_during_move`) that
drives the high-level `drag` action against the existing
`html5_drag_probe` fixture and asserts that `mousemove` events carry
`buttons == 1` and that `dragstart` fires on the source element.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* style: fix rustfmt formatting in e2e drag test
---------
Co-authored-by: wangjingjing <wangjingjing.99@bytedance.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Add provider icons and session creation from the dashboard UI.
Sessions can now be created with cloud providers (Browserbase,
Browserless, Browser Use, Kernel) in addition to local engines.
CLI changes:
- Track provider via .provider files alongside .engine files
- Add WaitUntil::None variant to skip lifecycle event waits for providers
- Auto-set waitUntil=none when --provider is used with navigate
- Fix Browser Use: use direct WSS connection (wss://connect.browser-use.com)
- Add connect_cdp_direct for providers with page-level CDP proxies
- Fix resolve_cdp_url to convert https:// provider URLs to wss://
- Treat empty CDP session_id as None (omit from protocol messages)
- Fix Browserbase: send explicit JSON body + Content-Type header
- Increase CDP connect timeout to 25s for remote providers
- Clean up .provider files on session close
Dashboard changes:
- Show provider or engine icon per session in sidebar
- New session dialog with unified engine/provider selector grid
- Async session creation with loading state and error display
- Kill zombie daemons on provider connection failure
- Parse CLI JSON error output for user-friendly messages
- Default new session URL to https://agent-browser.dev
Closes#1039
- Add `preview` field to `RemoteObject` to capture CDP object previews
- Implement `format_console_arg` using preview data (value → preview → description)
- Store raw CDP args in `ConsoleEntry` and include in JSON output
- Skip typed `ConsoleApiCalledEvent` deserialization in favor of direct param extraction
- Unify console arg formatting between daemon (actions.rs) and stream (stream.rs)
Before: `console.log({userId: "abc", count: 42})` → `"Object"`
After: `console.log({userId: "abc", count: 42})` → `{userId: "abc", count: 42}`
JSON output now includes raw `args` array for programmatic access by AI agents.
Co-authored-by: hyunjinee <leehj0110@kakao.com>
* Add auto-dismissal for alert and beforeunload dialogs
This PR adds automatic handling of JavaScript dialogs to prevent the agent from blocking indefinitely when `alert()` or `beforeunload` dialogs appear on web pages.
## Summary
Previously, when a website displayed native browser confirmation dialogs (like alerts or "Are you sure you want to leave?" prompts), agent-browser would hang waiting for manual intervention. This is a common issue since many websites use these dialogs for notifications or navigation warnings.
## Changes Made
- **Auto-dismiss functionality**: Added a background task that automatically accepts `alert` and `beforeunload` dialogs while leaving `confirm` and `prompt` dialogs for explicit handling
- **New flag**: Added `--no-auto-dialog` flag to disable automatic handling when needed
- **Environment variable**: Added `AGENT_BROWSER_NO_AUTO_DIALOG` for configuration
- **Documentation**: Updated README and docs with usage examples and configuration details
- **Tests**: Added comprehensive test coverage for flag parsing and dialog handling logic
## Implementation Details
- Only `alert` (notification-only) and `beforeunload` (navigation warning) dialogs are auto-handled for safety
- `confirm` and `prompt` dialogs still require explicit `dialog accept/dismiss` commands to ensure agents make deliberate choices for destructive actions
- The feature is enabled by default since these dialog types rarely require user decision-making
- Uses Chrome DevTools Protocol's `Page.handleJavaScriptDialog` for reliable dialog dismissal
Fixes#1070
* Log dialog type and message before auto-dismissal
Without this, auto-dismissed alert/beforeunload dialogs are silently
swallowed and the agent has no way to see what the dialog said. Adding
an eprintln before the CDP call makes the dismissal visible in stderr
for debugging.
* Log dialog dismissal errors instead of silently discarding them
- Remove premature "accepted" from log message since it fires before
the CDP command executes
- Replace `let _ =` with `if let Err(e)` to log failures when
Page.handleJavaScriptDialog fails
- Apply rustfmt to auto-dialog tests
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: detect externally opened tabs in --cdp mode (#1037)
Tabs opened outside of agent-browser (e.g. by the user or another CDP
client) were invisible to `tab list` because:
1. `Target.targetCreated` with chrome://newtab/ was filtered by
`is_internal_chrome_target`, and the subsequent `targetInfoChanged`
with the real URL could not update a target that was never tracked.
2. The background drain loop only ran when `request_tracking ||
har_recording` was active, so target events between commands were
silently dropped from the broadcast channel.
Fix: promote untracked targets in `targetInfoChanged` to new targets,
run the background drain unconditionally (guarded by browser presence),
and extract `apply_drained_events` to share target lifecycle processing
(attach, domain filter, iframe sessions) between execute_command and
the background drain.
* refactor: clean up HashSet import and remove call-site duplication
- Import HashSet alongside HashMap instead of using fully-qualified path
- Replace duplicated drain+apply sequence in execute_command with
drain_cdp_events_background call
* style: apply cargo fmt
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
The Rust rewrite of save_state only captured cookies and localStorage
for the current page's origin, silently dropping cross-domain data
(e.g. SSO/CAS auth cookies). This was a regression from the JS version.
Cookies: replace Network.getCookies with Network.getAllCookies to
return cookies from all domains the browser has visited.
localStorage: track visited origins in BrowserManager during navigation,
then collect their localStorage via a temporary CDP target with Fetch
interception (serves blank HTML to avoid real network requests).
Co-authored-by: hyunjinee <leehj0110@kakao.com>
* dashboard
* fix: re-apply download behavior on recording context (#1019)
* fix: re-apply download behavior on recording context
record start creates a new browser context via Target.createBrowserContext.
Browser.setDownloadBehavior called at launch only applies to the default
context, so downloads in the recording context are silently dropped.
Fix:
1. Store download_path on BrowserManager (from LaunchOptions)
2. After creating the recording context, call Browser.setDownloadBehavior
with the new browserContextId
This ensures downloads work during recording.
Fixes#1018
* fix: add download_path to third BrowserManager constructor (auto_connect_cdp)
* fix: reap zombie Chrome process and fast-detect crash for auto-restart (#1023)
When Chrome crashes (e.g. SIGTRAP from CHECK() assertion), the daemon
now:
1. Reaps the zombie immediately via a SIGCHLD handler in the event loop
that calls waitpid(-1, WNOHANG)
2. Detects the crash instantly on the next command via a non-blocking
try_wait() check (has_process_exited), avoiding the 3-second CDP
timeout that is_connection_alive() would incur
3. Auto-relaunches Chrome transparently for the caller
Fixes#1017
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: route keyboard type through text input (#1014)
* fix: handle --clear flag in console command (#1015)
The console and errors commands parsed --clear from CLI args but the
action handlers silently ignored the flag. The handlers did not accept
the cmd parameter so they had no way to read the clear field.
Changes:
- Add clear_console() method to EventTracker in network.rs
- Update handle_console to accept cmd, read the clear field, and clear
the buffer when --clear is passed (returns {cleared: true})
- Update call site in execute_command to pass cmd
Co-authored-by: xuyongliang <yongliang.xyl@alibaba-inc.com>
* chore: patch release - ### Bug Fixes
- **Re-apply download behavior on r... (#1025)
* Add runtime stream enable/disable/status commands (#951)
* Add runtime stream management commands
* Run rustfmt and satisfy clippy
* Fix stream disable cleanup semantics
* Format stream disable regression tests
* fix: retain radio/checkbox elements in compact snapshot tree (#1008)
compact_tree() checked for "[ref=" to identify lines worth keeping, but
radio and checkbox elements render as e.g. [checked=false, ref=e1] where
the "[" opens before "checked=", not "ref=". Dropping the leading bracket
so the check is just "ref=" fixes the match for all elements with refs.
Fixes#1006
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* chore: version packages (#1027)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fixes
* dashboard
* fixes
* remove observe
* fmt
* fixes
* fixes
* jotai
* fmt
* upload dashboard
---------
Co-authored-by: Stefan Smiljkovic <stefan@vanila.io>
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Co-authored-by: zhanba <c5e1856@gmail.com>
Co-authored-by: xuyongliang <478439790@qq.com>
Co-authored-by: xuyongliang <yongliang.xyl@alibaba-inc.com>
Co-authored-by: Thomas Kosiewski <thoma471@googlemail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
The console and errors commands parsed --clear from CLI args but the
action handlers silently ignored the flag. The handlers did not accept
the cmd parameter so they had no way to read the clear field.
Changes:
- Add clear_console() method to EventTracker in network.rs
- Update handle_console to accept cmd, read the clear field, and clear
the buffer when --clear is passed (returns {cleared: true})
- Update call site in execute_command to pass cmd
Co-authored-by: xuyongliang <yongliang.xyl@alibaba-inc.com>
When Chrome crashes (e.g. SIGTRAP from CHECK() assertion), the daemon
now:
1. Reaps the zombie immediately via a SIGCHLD handler in the event loop
that calls waitpid(-1, WNOHANG)
2. Detects the crash instantly on the next command via a non-blocking
try_wait() check (has_process_exited), avoiding the 3-second CDP
timeout that is_connection_alive() would incur
3. Auto-relaunches Chrome transparently for the caller
Fixes#1017
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: re-apply download behavior on recording context
record start creates a new browser context via Target.createBrowserContext.
Browser.setDownloadBehavior called at launch only applies to the default
context, so downloads in the recording context are silently dropped.
Fix:
1. Store download_path on BrowserManager (from LaunchOptions)
2. After creating the recording context, call Browser.setDownloadBehavior
with the new browserContextId
This ensures downloads work during recording.
Fixes#1018
* fix: add download_path to third BrowserManager constructor (auto_connect_cdp)
* fix: handle proxy authentication via CDP Fetch.authRequired
Chrome's --proxy-server flag does not support credentials embedded in
the URL. When a proxy requires authentication, Chrome receives a 407
from the proxy but has no way to respond with credentials, resulting
in net::ERR_INVALID_AUTH_CREDENTIALS.
Fix by:
1. Parsing credentials from the proxy URL (already done by parse_proxy)
2. Storing them in DaemonState.proxy_credentials
3. Enabling Fetch.enable with handleAuthRequests: true
4. Responding to Fetch.authRequired events with Fetch.continueWithAuth
5. Passing only the server URL (without credentials) to --proxy-server
6. Forwarding credentials to the daemon via dedicated env vars
Also adds fallback to standard proxy env vars (HTTP_PROXY, HTTPS_PROXY,
ALL_PROXY, NO_PROXY) when AGENT_BROWSER_PROXY is not set.
Fixes#990
* refactor: use typed struct for parse_proxy, fix double Fetch.enable and username-only auth
- Replace serde_json::Value return from parse_proxy with a typed ParsedProxy struct
- Fix double Fetch.enable call when both proxy auth and domain filter are active
(the second call could overwrite handleAuthRequests from the first)
- Allow username-only proxy auth (some proxies don't require a password)
- Handle empty username/password in parse_proxy as None instead of Some("")
- Use install_domain_filter_fetch in auto_launch for consistency
- Update unit tests to use typed struct fields
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Fixes#992
When a JavaScript dialog (alert/confirm/prompt) blocks the page, agents
had no way to detect it — all commands just timed out with generic errors.
- Add `dialog status` command to check for pending dialogs
- Track dialog state via CDP Page.javascriptDialogOpening/Closed events
- Auto-inject `warning` field into all command responses when a dialog is
pending, so agents can distinguish dialog-blocked timeouts from other issues
- Document dialog commands in SKILL.md (was missing entirely), README.md,
docs site, and --help output
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Chrome's Browser.downloadWillBegin and Browser.downloadProgress events
may arrive without a sessionId or with a different sessionId than the
page session used to configure the download behavior. The previous code
required an exact session match, silently dropping these events and
causing the 30-second timeout -- which manifests as an endless download
loop when callers retry.
Changes:
- Accept Browser-domain download events regardless of sessionId while
still matching Page-domain events by session to avoid cross-tab issues
- Add a brief retry loop (up to 1s) for the GUID file to appear on disk
after Chrome signals completion, handling filesystem flush races
- Return a proper error instead of silently succeeding when the
GUID-named file cannot be found
- Apply the same sessionId fix to handle_waitfordownload and add
Browser.downloadProgress support (previously only checked
Page.downloadProgress)
Fixes#989
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
The `press` command was not parsing modifier+key chords (e.g.
`Control+a`, `Shift+Enter`). It sent the raw string as a single key
name, so CDP never applied the modifier — and the `text` field caused
the literal character to be inserted instead of triggering the shortcut.
Two changes:
1. `actions.rs` — add `parse_key_chord()` to split inputs like
`Control+Shift+a` into the base key (`a`) and a CDP modifier
bitmask (Alt=1, Ctrl=2, Meta=4, Shift=8), then call
`press_key_with_modifiers` instead of `press_key`.
2. `interaction.rs` — suppress the `text` field in `keyDown`/`keyUp`
events when Control or Meta modifiers are active, so the browser
treats them as command chords rather than text input.
Includes unit tests for the chord parser covering plain keys, single
modifiers, multi-modifier combos, modifier aliases, and edge cases.
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* Fix download command to properly handle absolute paths and click elements
The download command was not working correctly - it would return "done" but not actually download files to the specified path. The command was only setting download behavior without clicking the element or waiting for completion.
**Changes made:**
- Modified `handle_download` to take a `selector` parameter and click the download element
- Added proper absolute path resolution and directory creation
- Implemented CDP event listening to wait for download completion with 30s timeout
- Added file renaming logic to handle Chrome's GUID-based temporary filenames
- Changed response format to return the actual download path
- Fixed function signature to use `&mut DaemonState` for state modifications
**Implementation details:**
- Uses `Browser.downloadWillBegin` and `Browser.downloadProgress` CDP events to track downloads
- Falls back to finding the most recently modified file if GUID capture fails
- Creates parent directories automatically if they don't exist
- Handles both absolute and relative path inputs
Fixes#965
* Address review feedback: harden download path handling
- Canonicalize download directory to prevent path traversal attacks
- Remove dangerous fallback that renamed the most-recently-modified file
in the directory (could silently rename unrelated files)
- Extract timeout to a named constant (DOWNLOAD_TIMEOUT)
* Fix download event loop: handle canceled state and Page.downloadWillBegin
- Detect "canceled" download state and return an error immediately instead
of spinning until the 30s timeout.
- Also capture the download GUID from the deprecated Page.downloadWillBegin
event for older Chrome compatibility, matching the existing
Page.downloadProgress fallback.
- Consolidate duplicated session/event checks with a shared is_this_session
variable and use match for cleaner state handling.
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* feat: add network request detail and filtering for request tracking
- Add `network request <requestId>` command to view full request/response
details including response body via CDP Network.getResponseBody
- Add --type, --method, --status filter flags to `network requests`
- --type: comma-separated resource types (xhr,fetch,document)
- --method: filter by HTTP method
- --status: supports exact (200), class (2xx), range (400-499)
- Extend TrackedRequest with request_id, post_data, status,
response_headers, mime_type fields
- Update Network.responseReceived handler to also populate
tracked_requests (previously only updated HAR entries)
- Add tests for parse commands and matches_status_filter
- Update README, SKILL.md, docs, and help text
Closes#932
* fix: show request ID and status in network requests output
* fix: add ref for cursor-interactive content roles
* fix: format
* feat: always include cursor-interactive elements in snapshot, -C is deprecated
* feat: process StaticText aggregation and deduplication
* update test
* clean up
* fix: escape text of elements in snapshot
* fix: redundant slicing
* fix: cargo fmt
* feat: deduplicate redundant StaticText
---------
Co-authored-by: 羲洋 <lipengyang.lpy@alibaba-inc.com>
* fix: prevent state commands from starting daemon without session_name
(#677)
State management commands (state_list, state_show, state_clear,
state_clean, state_rename) are pure file operations that don't need a
running daemon. Previously, these commands would trigger daemon
startup
via ensure_daemon(), and if AGENT_BROWSER_SESSION_NAME was exported
after the first command (e.g. `state clear --all`), the daemon would
start without session_name. Subsequent open/close commands would
reuse
that daemon, causing close to skip state persistence entirely.
Fix: execute state management commands locally in the CLI process
before
ensure_daemon() is called. This is done via a new
dispatch_state_command() function in state.rs that centralizes the
command routing, used by both the CLI (local path) and the daemon
(batch/IPC path).
Also:
- Add OutputOptions::from_flags() helper to deduplicate construction
- Add unit tests for dispatch_state_command routing and error
handling
* style: fix fmt and clippy warnings
- Remove redundant closure in dispatch_state_command (clippy::redundant_closure)
- Remove needless borrow in run_batch (clippy::needless_borrow)
- Fix trailing blank lines (rustfmt)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: remove dead code and unused variables in actions.rs
Remove `daemon_state_from_env` (superseded by `DaemonState::new`) and
`resolve_semantic_locator` (superseded by `handle_semantic_locator`),
both of which had zero call sites. Also remove unused `_session_id`
bindings in `handle_find` and `handle_multiselect`, and a no-op
`let _ = sid` in `handle_getbyrole`.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: simplify `_sid` to `_` per review feedback
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: use `is_ok()` instead of `if let Ok(_)` for idiomatic Rust
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
Fixed an issue where custom viewport settings were ignored in WebSocket streaming, causing status messages to always report hardcoded dimensions (1280x720) instead of the actual viewport size.
## Changes Made
- **Added viewport tracking to StreamServer**: New `viewport_width` and `viewport_height` fields to store current dimensions
- **Updated viewport synchronization**: Modified `handle_viewport()` and `handle_device()` to update the stream server when viewport changes
- **Fixed status message broadcasting**: Replaced hardcoded dimensions with actual viewport values in `broadcast_status()` calls
- **Improved screencast defaults**: Changed screencast to use stored viewport dimensions as defaults instead of hardcoded 1280x720
- **Added viewport getter methods**: New `set_viewport()` and `viewport()` methods on StreamServer for dimension management
This ensures that WebSocket clients receive consistent viewport dimensions across all message types (status and frame messages), matching the actual browser viewport settings.
Fixes#950
Navigate with load, then wait for username/password/submit selectors using the default action timeout. This avoids networkidle hangs on pages with continuous background requests.
* fix: restore origin-scoped --headers persistence across commands
In the v0.20 Rust rewrite, headers passed via --headers on open were
only applied to that single navigation via Network.setExtraHTTPHeaders,
which did not persist them for subsequent commands. In v0.19
(Playwright-based), these headers persisted for all subsequent
same-origin requests.
This restores the v0.19 behavior using CDP Fetch interception:
- A background task processes Fetch.requestPaused events in real-time,
injecting origin-scoped headers into matching requests and continuing
non-matching requests unmodified. This avoids the deadlock that occurs
when Fetch interception pauses requests during Page.navigate or
Runtime.evaluate (which block waiting for completion).
- The same background task also handles domain filtering and route
interception, replacing the previous drain-between-commands approach
that couldn't process events during navigation or script evaluation.
Fixes:
- --headers persist for same-origin navigations without re-passing flag
- --headers persist for in-page fetch/XHR to the same origin
- --headers do not leak to cross-origin navigations or sub-resources
- `set headers` (global) is unaffected and stacks with --headers
- Domain filter Fetch interception no longer deadlocks during navigation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: apply cargo fmt formatting
* revert inaccurate comment change
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* feat: enhance HAR entries with timings, cookies, postData, and protocol normalisation
Extends #874's HarEntry struct and CDP handlers with richer capture:
- wall_time (f64) replaces pre-formatted started_date_time, preserving
sub-second precision from CDP wallTime for accurate RFC 3339 output
- request_headers / response_headers changed from Value to Vec<(String,String)>
for typed access without re-parsing JSON objects
- post_data captured from Network.requestWillBeSent for HAR postData
- cdp_timing and loading_finished_timestamp captured from responseReceived
and loadingFinished respectively, enabling har_compute_timings to produce
accurate blocked/dns/connect/ssl/send/wait/receive phases
- har_cdp_protocol_to_http_version normalises CDP protocol strings
(h2 -> HTTP/2.0, h3 -> HTTP/3.0, etc.)
- Request cookies parsed from Cookie header; response cookies from Set-Cookie
with ';'-first split to correctly strip Path/HttpOnly attributes before '='
- har_wall_time_to_rfc3339 replaces har_started_date_time, using the time
crate directly on the f64 epoch value
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: apply cargo fmt
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Add iframe support for CLI interactions and snapshots
This PR adds comprehensive iframe support to the agent browser CLI, allowing users to interact with elements inside iframes seamlessly.
## Problem
Users couldn't interact with elements inside iframes via the command line. The existing `frame` command was non-functional as it set `active_frame_id` but no other code read this value.
## Changes Made
### Enhanced Frame Context Tracking
- Added `frame_id` field to `RefEntry` to track which frame each element reference belongs to
- Updated `RefMap::add` and related methods to accept and store frame context
- Modified element resolution functions to use frame context from ref entries
### Improved Frame Command
- Fixed the existing `frame` command to actually work by threading `active_frame_id` through snapshot operations
- Added support for iframe element references (e.g., `frame @e2`) in addition to CSS selectors
- Enhanced frame detection to work with both named frames and iframe elements
### Updated Snapshot Behavior
- Modified `take_snapshot` to accept optional frame context parameter
- Updated all snapshot call sites to pass appropriate frame context
- Maintained backward compatibility while enabling frame-scoped operations
### Element Resolution Updates
- Updated `resolve_element_center` and `resolve_element_object_id` to use frame context from ref entries
- Modified `find_node_id_by_role_name` to support frame-specific element lookup
- Ensured all interaction functions work correctly within iframe contexts
## Implementation Details
- Frame context is now properly propagated through the entire element interaction pipeline
- The `frame` command can accept both CSS selectors and element references
- All existing functionality remains intact while adding iframe capabilities
- Added `Iframe` to interactive roles for better element discovery
Fixes#863
* docs: add iframe support documentation
Document the new iframe capabilities across all documentation surfaces:
- Auto-inlining of iframe content in snapshots
- Direct interaction with iframe element refs
- frame command support for element refs (@e3)
- Scoped snapshots via frame switching
* fix: pass active frame context to diff snapshots and fix nameless iframe lookup
- handle_diff_snapshot now respects active_frame_id instead of always
passing None, so diff snapshots work correctly inside iframes
- Nameless/id-less iframes now fall back to src URL (or null) instead of
the literal string 'frame' which never matched any frame in the tree
* fix: resolve iframe frame ID via DOM.describeNode and reduce code duplication
- handle_frame: Use DOM.describeNode + contentDocument.frameId to resolve
iframe frame IDs directly, fixing failures for nameless iframes that
lack name/id/src attributes
- element.rs: Deduplicate add() by delegating to add_with_frame()
- snapshot.rs: Guard against out-of-bounds insert_str when iframe marker
is on the last line without a trailing newline
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Expose HAR recording as a CLI subcommand under the existing `network`
command so users can capture and export network traffic without a
separate tool or opening the browser twice.
- Parse `network har start` and `network har stop [path]` in commands.rs
- Enrich HarEntry with request/response headers, timestamps, status text,
resource type, HTTP version, and body sizes from CDP events
- Produce HAR 1.2 output with creator/browser metadata, query strings,
and proper header arrays compatible with Chrome DevTools HAR viewer
- Auto-generate output path under ~/.agent-browser/tmp/har/ when omitted
- Add har_stop to skip_launch list so export works without a live browser
- Update help text, README, docs site, SKILL.md, and security policy docs
- Add unit tests for parsing, HAR entry serialization, and stop behavior
* fix: Windows auto-connect profiling (#835)
Fix three interrelated bugs causing `--auto-connect` to fail on Windows,
plus a UX issue where auto-connect hijacked existing tabs:
1. Stale DevToolsActivePort — add TCP port liveness check before returning
M144+ WebSocket URL; remove stale files when port is dead.
2. Missing Windows error codes — add os error 10061 (WSAECONNREFUSED) and
10054 (WSAECONNRESET) to is_transient_error() so daemon startup races
are retried on Windows.
3. --auto-connect not propagated to daemon — add auto_connect to
DaemonOptions, set AGENT_BROWSER_AUTO_CONNECT env var via
apply_daemon_env(), and guard the headed launch block so it doesn't
send a second launch that overrides the auto-connect.
4. Auto-connect opens a fresh tab — after connecting to an existing
Chrome, create a new about:blank tab and bring it to front so
navigations don't hijack the user's existing tabs.
Made-with: Cursor
* fix: address review feedback — cargo fmt, shared helper, Windows tests
- Run cargo fmt on is_port_reachable() formatting
- Extract duplicated auto-connect-with-fresh-tab logic into
connect_auto_with_fresh_tab() helper used by both handle_launch()
and auto_launch()
- Add unit tests for Windows WSAECONNREFUSED (os error 10061) and
WSAECONNRESET (os error 10054) in is_transient_error()
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* feat: embed cursor-interactive elements into snapshot tree
* optimize format
* fix: address review feedback for e2e_snapshot_cursor_interactive unitest
---------
Co-authored-by: 羲洋 <lipengyang.lpy@alibaba-inc.com>
* fix: restore WebSocket streaming in native daemon
The v0.20.0 Rust rewrite broke WebSocket streaming — connections opened
but received zero messages before closing. Multiple issues contributed:
1. StreamServer was dropped immediately after creation in daemon.rs,
closing the broadcast channel and killing all WS connections.
2. Screencast frames were only processed during command polling
(drain_cdp_events) instead of in real-time, unlike the 0.19.0
TypeScript cdp.on('Page.screencastFrame') callback.
3. Auto-start/stop screencast on WS client connect/disconnect was
missing from the Rust implementation.
4. Screencast CDP commands used the wrong session ID (daemon session
name instead of the CDP page session from Target.attachToTarget).
5. Broadcast channel Lagged errors killed WS connections instead of
being handled gracefully.
The fix adds a background CDP event loop in StreamServer that subscribes
to Chrome events and broadcasts screencast frames in real-time, properly
tracks the CDP page session ID, restores auto-screencast lifecycle, and
keeps the StreamServer alive in DaemonState.
Fixes#820
* fix: use actual CDP session ID for input dispatch in stream WebSocket
Pass the real cdp_session_id (from Target.attachToTarget) through to
handle_ws_client instead of an empty string. Previously, input commands
(mouse, keyboard, touch) were sent with `"sessionId": ""` which Chrome
silently rejects. Now the correct page session ID is read at dispatch
time, and when no session ID is set yet (before browser launch),
the field is omitted entirely via `None` so Chrome uses browser-level
dispatch.
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: snapshot --selector scopes to the matched element subtree
The native Rust daemon accepted the --selector flag but never used it —
the full accessibility tree was always returned regardless of the
selector. This restores the 0.19.0 behaviour where snapshot --selector
returns only the subtree rooted at the matched CSS selector.
The implementation resolves the selector via Runtime.evaluate, fetches
the full DOM subtree with DOM.describeNode(depth: -1) to collect all
descendant backendNodeIds, then filters the AX tree to render only the
nodes whose backendDOMNodeId falls within that set. This correctly
handles elements like <body> that don't map to a direct AX node.
Also fixes handle_snapshot reading "depth" instead of "maxDepth" from
the command JSON, which caused --depth to be silently ignored.
Fixes#822
* style: run cargo fmt on snapshot.rs
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: replace screenshot polling with screencast-based piped ffmpeg recording
Recording previously used Page.captureScreenshot polling at 10fps,
which was CPU-heavy and produced inconsistent results. Now uses
Page.startScreencast with throttled acks (35ms interval) to receive
frames event-driven from Chrome, and pipes JPEG data directly to
ffmpeg stdin in real-time instead of saving temp files.
- Spawn ffmpeg at recording start with piped stdin (image2pipe)
- Background task receives screencast frames, interpolates gaps by
repeating the last frame based on timestamps, targets 25fps
- Ack throttling controls Chrome's frame push rate
- Fix: current frame was never written after the first one
- Fix: frame count was read before task finished padding
- Remove tokio-util dependency (replaced CancellationToken with oneshot)
- Add tokio "process" feature for async child process stdin pipe
- Extract start/stop_recording_task helpers on DaemonState
- Add tests for restart, ffmpeg codec selection, and stop without task
* fmt
* chore
* fix: switch WebM codec from VP9 to VP8 for correct framerate and browser
compatibility
VP9 realtime encoder ignored input framerate, producing 10fps output
instead of 25fps. This caused inconsistent playback in browsers.
VP8 respects -framerate 25 and has wider browser playback support.
* fmt
* fix: add kill_on_drop to ffmpeg process to prevent zombie on task panic
* fix: switch from screencast to screenshot polling for reliable recording duration
Screencast only pushes frames on visual changes, producing short videos
on static pages. Screenshot polling captures at a fixed 10fps interval
regardless of page activity, guaranteeing duration matches wall-clock time.
ffmpeg piped stdin architecture is preserved — no temp files.
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Replace all `eprintln!` calls in daemon-context code with
`let _ = writeln!(std::io::stderr(), ...)` so that broken pipe errors
on stderr are silently ignored instead of panicking.
The CLI client spawns the daemon with piped stderr to capture startup
errors, then drops the pipe handle once the daemon is ready. Any
subsequent `eprintln!` in the daemon panics because Rust's `eprintln!`
macro internally unwraps the write result. This caused the reported
"failed printing to stderr: Broken pipe (os error 32)" panic during
Chrome launch on Linux.
Closes#799
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
The CDP event broadcast channel (capacity 256) can overflow on slow CI
runners when Chrome emits many events during navigation. Previously,
RecvError::Lagged was treated the same as RecvError::Closed, causing
spurious "Event stream closed" errors even though Chrome was still
running. Now all 5 event-receiving loops correctly continue on Lagged
instead of breaking.
- Restore the `refs` dictionary in `--json` snapshot output, matching the documented API contract
- The `refs` field was silently dropped during the Node.js to Rust rewrite (v0.20), causing consumers parsing `data.refs` for programmatic element interaction to receive no structured ref data
Fixes#785
The test was reading AGENT_BROWSER_HEADED without holding ENV_MUTEX,
causing a race with test_launch_options_from_env_headed_flag when
tests run in parallel.
* feat: add screenshot output config, clipboard CLI commands, and fix wait --text native path
## Summary
- Add `--screenshot-dir`, `--screenshot-quality`, and `--screenshot-format` CLI flags (with corresponding `AGENT_BROWSER_SCREENSHOT_DIR`, `AGENT_BROWSER_SCREENSHOT_QUALITY`, `AGENT_BROWSER_SCREENSHOT_FORMAT` env vars) so users can configure where and how screenshots are saved without specifying a full path every time
- Add `clipboard read`, `clipboard write <text>`, `clipboard copy`, and `clipboard paste` CLI commands, exposing the existing protocol-level clipboard handlers that were previously only accessible via JSON-RPC
- Fix `wait --text` in native mode: the CLI was emitting `selector: "text=..."` (a Playwright-style locator) which native's `querySelector` can't handle. Now emits a `text` field that correctly hits the native `wait_for_text` polling path
- Add native clipboard `copy` and `paste` support via CDP `Input.dispatchKeyEvent`, and a `write` operation to the Node.js handler
* fix: resolve CI failures in Rust formatting and TypeScript typecheck
Use string-based page.evaluate for clipboard writeText to avoid
referencing `navigator` in Node.js compilation context. Run cargo fmt
to fix formatting in commands.rs and screenshot.rs.
* fix: clipboard write captures full multi-word text
Use rest[1..].join(" ") instead of rest.get(1) so unquoted multi-word
input like `clipboard write hello world` sends the full string rather
than silently dropping everything after the first word.
* improvements
* fixes
* improvements
* improvements
