* fix: load storage state at launch when --state / AGENT_BROWSER_STATE is set
The `--state` flag and `AGENT_BROWSER_STATE` env var were documented as
restoring saved browser state (cookies + localStorage) at launch, but
`load_state()` was never called after the browser started. The feature
has been broken since it was introduced.
Adds `try_load_storage_state()` and calls it from every early-return
path in `auto_launch()` (lazy launch triggered by commands like
`navigate`) and from `handle_launch()` (explicit `launch` command).
Also adds 4 e2e tests covering all state-persistence paths:
- Explicit launch with `storageState` field
- Auto-launch via `AGENT_BROWSER_STATE` env var
- Session-name auto-restore via `try_auto_restore_state`
- Explicit `state_load` command (baseline sanity check)
Fixes#1164.
* style: apply cargo fmt to e2e_tests.rs
Reformats a single long format\! call to satisfy CI's rustfmt check.
No behavior change.
* fix: call try_load_storage_state in all handle_launch branches
The CDP URL, CDP port, auto-connect, and provider early-return branches
were skipping storage state loading because try_load_storage_state was
only called in the normal BrowserManager::launch() path at the bottom
of handle_launch().
Also compute storage_state_owned once and reuse it across all branches
rather than borrowing storage_state (a &str tied to cmd) in a helper
that needs an owned Option<String>.
* Fix storage state reload on reused launches
* Fix storage-state launch errors
* Fix storage state replay ordering
* Align storage-state errors across launch paths
* Fix storageState launch cleanup
* fix: inherit viewport dimensions in recording context
When `record start` creates a new browser context, it now re-applies the
current viewport settings (from `set viewport` or `set device`) so the
recording resolution matches what the user configured instead of falling
back to the default 1280×720.
Closes#1207
* style: apply cargo fmt to e2e test
* chore: remove obvious comments
* chore: remove obvious comments from e2e test
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
* fix: use custom viewport dimensions in streaming frame metadata
CDP's Page.screencastFrame metadata returns physical device dimensions
instead of the emulated viewport, causing frame messages to report
incorrect deviceWidth/deviceHeight when a custom viewport is set.
Use the viewport dimensions captured at screencast start instead of
the CDP metadata values, since the screencast image is already captured
at the configured viewport size.
Closes#1031
* fix: resize browser content area on viewport change for correct
screencast dimensions
Emulation.setDeviceMetricsOverride only changes the CSS viewport, but
screencast captures the actual browser content area. This caused frame
images to have incorrect dimensions (e.g., 1000x451 instead of
1000x1000)
when a custom viewport was set.
- Call Browser.setContentsSize after setDeviceMetricsOverride so the
content area matches the emulated viewport
- Restart active screencast when viewport dimensions change so
maxWidth/maxHeight parameters are updated
- Skip redundant screencast restarts when dimensions are unchanged
- Extend E2E test to verify actual JPEG image dimensions, not just
metadata
* fix: pass viewport dimensions to --window-size at launch and log setContentsSize failures
- Add viewport_size to LaunchOptions so --window-size matches the
configured viewport from the start, reducing reliance on the
experimental Browser.setContentsSize CDP call at runtime
- Log Browser.setContentsSize failures instead of silently ignoring
them with let _ =
* fix: remove duplicate viewport change detection block (dead code from merge)
* fix: use log::debug! instead of eprintln! for setContentsSize failure
* revert: use eprintln! instead of log crate for setContentsSize failure
The daemon's stderr pipe is closed after startup, so log crate
subscribers cannot output during normal operation. eprintln! is
visible during startup and in tests, matching the existing convention.
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Security.setIgnoreCertificateErrors is session-scoped, so creating a new
BrowserContext for recording (Target.createBrowserContext) starts with the
default certificate validation enabled, ignoring the launch-time flag.
Store ignore_https_errors in BrowserManager alongside download_path, and
re-apply Security.setIgnoreCertificateErrors to the new session after
recording context creation — matching the existing pattern for download
behavior re-application.
Fixes#1172
Co-authored-by: wangjingjing <wangjingjing.99@bytedance.com>
* fix: support accessibility tree refs in upload command (#1107)
The upload command only accepted CSS selectors while click/fill supported
accessibility tree refs (e.g. e1, @e1, ref=e1). This resolves the API
inconsistency by reusing resolve_element_object_id for all selector types.
* style: apply cargo fmt
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
* fix(cli): honor AGENT_BROWSER_DEFAULT_TIMEOUT env var for wait commands
The `AGENT_BROWSER_DEFAULT_TIMEOUT` environment variable was being ignored by CLI wait commands, causing them to use hardcoded 30-second timeouts instead of the configured default.
## Changes Made
- **Centralized timeout injection**: Modified `parse_command()` to automatically inject `flags.default_timeout` into any wait-family command that doesn't already have an explicit `--timeout` flag
- **Environment variable parsing**: Added `default_timeout` field to `Flags` struct that reads from `AGENT_BROWSER_DEFAULT_TIMEOUT` env var
- **Daemon propagation**: Updated daemon spawning to pass through the default timeout via environment variables
- **Unified timeout handling**: Added `timeout_ms()` helper method in `DaemonState` that all wait handlers now use instead of scattered `unwrap_or()` calls
- **Comprehensive test coverage**: Added 10 regression tests covering all wait command variants and edge cases
## Implementation Details
The fix uses a two-stage approach:
1. CLI parses the env var and injects timeout values into command JSON for any `wait*` action
2. Daemon reads the env var and provides a centralized fallback via `timeout_ms()` helper
This ensures new wait variants automatically inherit the default timeout without requiring per-variant wiring.
Fixes#1147
* fix: preserve 30s default timeout for backward compatibility
The default_timeout_ms fallback was set to 25_000ms, which silently
changes the existing 30_000ms behavior for users who haven't set
AGENT_BROWSER_DEFAULT_TIMEOUT. Restore the original 30s default.
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: rewrite getByRole to use CDP accessibility tree instead of CSS selectors
The old `handle_getbyrole` generated `querySelectorAll('[role="link"], link')`
which matched `<link>` stylesheet elements instead of `<a>` anchor tags.
This happened because ARIA role names were used directly as CSS tag selectors,
and several roles differ from their HTML element names (e.g. link → a,
heading → h1-h6, textbox → input/textarea).
The fix replaces the JS-based DOM query with the CDP `Accessibility.getFullAXTree`
API, where the browser engine correctly computes implicit ARIA roles per the
WAI-ARIA / HTML-AAM spec. This is the same approach already used by `snapshot.rs`
and `element.rs` in this codebase.
Changes:
- Rewrite `handle_getbyrole` to query the browser's accessibility tree via CDP
- Add `find_ax_node_by_role` helper for AX tree traversal with role/name/exact matching
- Use `DOM.resolveNode` + `Runtime.callFunctionOn` to bridge AX node → DOM marker
- Add iframe support via `resolve_ax_session` (missing in old implementation)
- Fix cleanup to use correct CDP session (old code used default session, breaking iframe cleanup)
- Export `extract_ax_string` as `pub(super)` for reuse
- Add 4 regression tests for `find_ax_node_by_role`
Fixes#1123
* style: apply cargo fmt
* chore: remove redundant comments
* refactor: replace marker attribute with temporary ref for element resolution
Eliminates 3 CDP round-trips (DOM.resolveNode, Runtime.callFunctionOn,
Runtime.evaluate cleanup) by registering a temporary ref in the ref_map.
execute_subaction resolves the element via backendNodeId directly.
No more DOM pollution with marker attributes.
* fix: ref counter collision, ref_map leak, and stale fallback name
- Increment next_ref_num after inserting temp ref to prevent id collision
- Remove temp ref after execute_subaction to prevent unbounded ref_map growth
- Return actual AX name from find_ax_node_by_role for accurate fallback resolution
- Add RefMap::remove method
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
* feat(chrome): add Chrome profile name resolution and copy for --profile flag
When --profile receives a name without path separators (e.g., "Default"),
it now resolves the name against installed Chrome profiles, copies the
profile to a temp directory (excluding large cache dirs), and launches
Chrome with the copied profile to reuse login state.
Key changes:
- Add profile resolution: is_chrome_profile_name, find_chrome_user_data_dir,
list_chrome_profiles, resolve_chrome_profile (3-tier matching)
- Add copy_chrome_profile with best-effort copy and exclusion list
- Wire preprocessing into launch_chrome before retry loop
- Add use_real_keychain field to LaunchOptions for conditional keychain flags
- Make --password-store=basic and --use-mock-keychain conditional
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(cli): add `profiles` command to list available Chrome profiles
Adds `agent-browser profiles` command that reads Chrome's Local State
file to list available profiles with directory names and display names.
Supports --json output. Added help text in print_command_help and
print_help.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add Chrome profile reuse documentation across all locations
Update all 5 documentation locations per AGENTS.md:
- output.rs: updated --profile help text and examples
- README.md: added Chrome Profile Reuse section, updated options table
- SKILL.md: added profile reuse as Option 2
- docs/src/app/sessions/page.mdx: added Chrome profile reuse section
- chrome.rs: added doc comments to get_chrome_user_data_dirs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: fix formatting and clippy warning in chrome.rs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: simplify profile resolution and launch integration
- Only clone LaunchOptions when profile name requires resolution
(avoids unnecessary allocation on every Chrome launch)
- Remove redundant is_file() check before copy of Local State
(copy() handles missing files naturally)
- Extract format_profile_list() to deduplicate error formatting
- Remove unnecessary section comments in tests
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(tests): use RAII TempDir guard for test cleanup
Replace manual remove_dir_all calls with a TempDir struct that
auto-cleans on drop, preventing temp dir leaks on test panics.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
When connecting to a real, already-running browser (Chrome 144+) via CDP,
targets may be paused waiting for the debugger after attach. Without an
explicit Runtime.runIfWaitingForDebugger call, page-level commands hang
indefinitely even though the WebSocket connection is live.
Add Runtime.runIfWaitingForDebugger after Runtime.enable in all target
attachment paths: enable_domains (covers initial attach, tab_new,
tab_switch), enable_domains_direct (provider proxies), and the iframe
auto-attach handler. The call is placed before Network.enable to avoid
the documented deadlock when Network.enable precedes the resume. It is
a no-op for targets that are not paused.
Fixes#1130
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: relaunch browser when launch options change (#993)
When the daemon already held a running browser, handle_launch only
checked connection type and liveness to decide reuse. Config changes
like adding extensions to config.json were silently ignored.
Store a hash of the relaunch-relevant LaunchOptions fields and compare
on each launch command. If the hash differs the browser is closed and
relaunched with the new options.
* fmt
* fix
* fix
* fmt
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
When a non-launch command (e.g. open, snapshot) triggers auto_launch()
before the explicit launch command is processed, auto_launch() now checks
AGENT_BROWSER_PROVIDER and connects via the provider API instead of
always falling back to a local Chrome instance.
Also redirects daemon stderr to /dev/null when not in debug mode to
prevent crashes from broken pipe after the CLI drops the piped stderr
handle. Cloud providers may write to stderr during connection setup.
Fixes#1125
Related: #979
The CDP event broadcast buffer (256 events) was too small for pages with
many concurrent API requests, causing silent event drops. Modern SPAs
routinely fire 100+ API calls during page load, generating 300+ CDP
network events that would overflow the buffer between drain cycles.
Changes:
- Increase CDP broadcast buffer from 256 to 4096 (event channel) and
512 to 4096 (raw channel)
- Reduce background drain interval from 500ms to 100ms
- Handle Network.loadingFailed events in HAR recording
- Enable Network.enable on cross-origin iframe sessions during HAR
recording and request tracking
- Allow Network events from iframe sessions through the session filter
- Log a warning when buffer overflow occurs instead of silently dropping
Fixes#1128
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* feat: add AWS Bedrock AgentCore browser provider (native Rust)
- Add agentcore provider with SigV4 authentication
- AWS SDK deps are optional behind 'agentcore' feature flag
- Build with: cargo build --features agentcore
- Supports AGENTCORE_REGION, AGENTCORE_PROFILE_ID, AGENTCORE_BROWSER_ID env vars
- Returns session ID and Live View URL in launch response
- Add connect_cdp_with_headers for signed WebSocket connections
* test: add unit tests for AgentCore provider
* refactor: use lightweight manual SigV4 signing instead of AWS SDK
- Replace aws-sigv4/aws-config with manual HMAC-SHA256 signing
- Removes ~60s compile time and significant binary size
- Credentials read from AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY env vars
- Supports AWS_SESSION_TOKEN for temporary credentials
* fix: correct AgentCore API endpoints
- Host: bedrock-agentcore.{region}.amazonaws.com
- Start session: PUT /browsers/{id}/sessions/start
- Stop session: PUT /browsers/{id}/sessions/stop
- Add urlencoding for browser ID in path
- Add AWS_DEFAULT_REGION fallback
* fix: use profileConfiguration.profileIdentifier for AgentCore profile
The AWS Bedrock AgentCore API expects profile configuration in the format:
{
"profileConfiguration": {
"profileIdentifier": "<profile-id>"
}
}
Not the flat "profileId" field that was previously used.
* feat: support AWS credential provider chain via AWS CLI
- Try env vars first (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
- Fall back to 'aws configure export-credentials --format env'
- Honor AWS_PROFILE environment variable
- Works with SSO, IAM roles, credential files, etc.
---------
Co-authored-by: Chris Tate <chris@ctate.dev>
* fix: include buttons bitmask in drag mouseMoved events
The drag handler was omitting the `buttons` field from every
`mouseMoved` event dispatched during the move phase. Without it the
browser sees `event.buttons === 0`, meaning no button is held, so
`dragstart`/`dragover`/`drop` never fire and the drop target never
receives the element.
Fix:
- Add `"buttons": 1` (left-button mask) to each `mouseMoved` sent
while the button is held.
- Add `"buttons": 1` to `mousePressed` and `"buttons": 0` to
`mouseReleased`, consistent with how `dispatch_click` handles the
same fields in interaction.rs.
- Correct the parity-test fixture for `drag`, which was supplying a
`selector` key instead of the `source` key that `handle_drag` reads.
- Add an e2e test (`e2e_drag_action_sends_buttons_during_move`) that
drives the high-level `drag` action against the existing
`html5_drag_probe` fixture and asserts that `mousemove` events carry
`buttons == 1` and that `dragstart` fires on the source element.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* style: fix rustfmt formatting in e2e drag test
---------
Co-authored-by: wangjingjing <wangjingjing.99@bytedance.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Add provider icons and session creation from the dashboard UI.
Sessions can now be created with cloud providers (Browserbase,
Browserless, Browser Use, Kernel) in addition to local engines.
CLI changes:
- Track provider via .provider files alongside .engine files
- Add WaitUntil::None variant to skip lifecycle event waits for providers
- Auto-set waitUntil=none when --provider is used with navigate
- Fix Browser Use: use direct WSS connection (wss://connect.browser-use.com)
- Add connect_cdp_direct for providers with page-level CDP proxies
- Fix resolve_cdp_url to convert https:// provider URLs to wss://
- Treat empty CDP session_id as None (omit from protocol messages)
- Fix Browserbase: send explicit JSON body + Content-Type header
- Increase CDP connect timeout to 25s for remote providers
- Clean up .provider files on session close
Dashboard changes:
- Show provider or engine icon per session in sidebar
- New session dialog with unified engine/provider selector grid
- Async session creation with loading state and error display
- Kill zombie daemons on provider connection failure
- Parse CLI JSON error output for user-friendly messages
- Default new session URL to https://agent-browser.dev
Closes#1039
- Add `preview` field to `RemoteObject` to capture CDP object previews
- Implement `format_console_arg` using preview data (value → preview → description)
- Store raw CDP args in `ConsoleEntry` and include in JSON output
- Skip typed `ConsoleApiCalledEvent` deserialization in favor of direct param extraction
- Unify console arg formatting between daemon (actions.rs) and stream (stream.rs)
Before: `console.log({userId: "abc", count: 42})` → `"Object"`
After: `console.log({userId: "abc", count: 42})` → `{userId: "abc", count: 42}`
JSON output now includes raw `args` array for programmatic access by AI agents.
Co-authored-by: hyunjinee <leehj0110@kakao.com>
* Add auto-dismissal for alert and beforeunload dialogs
This PR adds automatic handling of JavaScript dialogs to prevent the agent from blocking indefinitely when `alert()` or `beforeunload` dialogs appear on web pages.
## Summary
Previously, when a website displayed native browser confirmation dialogs (like alerts or "Are you sure you want to leave?" prompts), agent-browser would hang waiting for manual intervention. This is a common issue since many websites use these dialogs for notifications or navigation warnings.
## Changes Made
- **Auto-dismiss functionality**: Added a background task that automatically accepts `alert` and `beforeunload` dialogs while leaving `confirm` and `prompt` dialogs for explicit handling
- **New flag**: Added `--no-auto-dialog` flag to disable automatic handling when needed
- **Environment variable**: Added `AGENT_BROWSER_NO_AUTO_DIALOG` for configuration
- **Documentation**: Updated README and docs with usage examples and configuration details
- **Tests**: Added comprehensive test coverage for flag parsing and dialog handling logic
## Implementation Details
- Only `alert` (notification-only) and `beforeunload` (navigation warning) dialogs are auto-handled for safety
- `confirm` and `prompt` dialogs still require explicit `dialog accept/dismiss` commands to ensure agents make deliberate choices for destructive actions
- The feature is enabled by default since these dialog types rarely require user decision-making
- Uses Chrome DevTools Protocol's `Page.handleJavaScriptDialog` for reliable dialog dismissal
Fixes#1070
* Log dialog type and message before auto-dismissal
Without this, auto-dismissed alert/beforeunload dialogs are silently
swallowed and the agent has no way to see what the dialog said. Adding
an eprintln before the CDP call makes the dismissal visible in stderr
for debugging.
* Log dialog dismissal errors instead of silently discarding them
- Remove premature "accepted" from log message since it fires before
the CDP command executes
- Replace `let _ =` with `if let Err(e)` to log failures when
Page.handleJavaScriptDialog fails
- Apply rustfmt to auto-dialog tests
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: detect externally opened tabs in --cdp mode (#1037)
Tabs opened outside of agent-browser (e.g. by the user or another CDP
client) were invisible to `tab list` because:
1. `Target.targetCreated` with chrome://newtab/ was filtered by
`is_internal_chrome_target`, and the subsequent `targetInfoChanged`
with the real URL could not update a target that was never tracked.
2. The background drain loop only ran when `request_tracking ||
har_recording` was active, so target events between commands were
silently dropped from the broadcast channel.
Fix: promote untracked targets in `targetInfoChanged` to new targets,
run the background drain unconditionally (guarded by browser presence),
and extract `apply_drained_events` to share target lifecycle processing
(attach, domain filter, iframe sessions) between execute_command and
the background drain.
* refactor: clean up HashSet import and remove call-site duplication
- Import HashSet alongside HashMap instead of using fully-qualified path
- Replace duplicated drain+apply sequence in execute_command with
drain_cdp_events_background call
* style: apply cargo fmt
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
The Rust rewrite of save_state only captured cookies and localStorage
for the current page's origin, silently dropping cross-domain data
(e.g. SSO/CAS auth cookies). This was a regression from the JS version.
Cookies: replace Network.getCookies with Network.getAllCookies to
return cookies from all domains the browser has visited.
localStorage: track visited origins in BrowserManager during navigation,
then collect their localStorage via a temporary CDP target with Fetch
interception (serves blank HTML to avoid real network requests).
Co-authored-by: hyunjinee <leehj0110@kakao.com>
* dashboard
* fix: re-apply download behavior on recording context (#1019)
* fix: re-apply download behavior on recording context
record start creates a new browser context via Target.createBrowserContext.
Browser.setDownloadBehavior called at launch only applies to the default
context, so downloads in the recording context are silently dropped.
Fix:
1. Store download_path on BrowserManager (from LaunchOptions)
2. After creating the recording context, call Browser.setDownloadBehavior
with the new browserContextId
This ensures downloads work during recording.
Fixes#1018
* fix: add download_path to third BrowserManager constructor (auto_connect_cdp)
* fix: reap zombie Chrome process and fast-detect crash for auto-restart (#1023)
When Chrome crashes (e.g. SIGTRAP from CHECK() assertion), the daemon
now:
1. Reaps the zombie immediately via a SIGCHLD handler in the event loop
that calls waitpid(-1, WNOHANG)
2. Detects the crash instantly on the next command via a non-blocking
try_wait() check (has_process_exited), avoiding the 3-second CDP
timeout that is_connection_alive() would incur
3. Auto-relaunches Chrome transparently for the caller
Fixes#1017
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: route keyboard type through text input (#1014)
* fix: handle --clear flag in console command (#1015)
The console and errors commands parsed --clear from CLI args but the
action handlers silently ignored the flag. The handlers did not accept
the cmd parameter so they had no way to read the clear field.
Changes:
- Add clear_console() method to EventTracker in network.rs
- Update handle_console to accept cmd, read the clear field, and clear
the buffer when --clear is passed (returns {cleared: true})
- Update call site in execute_command to pass cmd
Co-authored-by: xuyongliang <yongliang.xyl@alibaba-inc.com>
* chore: patch release - ### Bug Fixes
- **Re-apply download behavior on r... (#1025)
* Add runtime stream enable/disable/status commands (#951)
* Add runtime stream management commands
* Run rustfmt and satisfy clippy
* Fix stream disable cleanup semantics
* Format stream disable regression tests
* fix: retain radio/checkbox elements in compact snapshot tree (#1008)
compact_tree() checked for "[ref=" to identify lines worth keeping, but
radio and checkbox elements render as e.g. [checked=false, ref=e1] where
the "[" opens before "checked=", not "ref=". Dropping the leading bracket
so the check is just "ref=" fixes the match for all elements with refs.
Fixes#1006
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* chore: version packages (#1027)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fixes
* dashboard
* fixes
* remove observe
* fmt
* fixes
* fixes
* jotai
* fmt
* upload dashboard
---------
Co-authored-by: Stefan Smiljkovic <stefan@vanila.io>
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Co-authored-by: zhanba <c5e1856@gmail.com>
Co-authored-by: xuyongliang <478439790@qq.com>
Co-authored-by: xuyongliang <yongliang.xyl@alibaba-inc.com>
Co-authored-by: Thomas Kosiewski <thoma471@googlemail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
The console and errors commands parsed --clear from CLI args but the
action handlers silently ignored the flag. The handlers did not accept
the cmd parameter so they had no way to read the clear field.
Changes:
- Add clear_console() method to EventTracker in network.rs
- Update handle_console to accept cmd, read the clear field, and clear
the buffer when --clear is passed (returns {cleared: true})
- Update call site in execute_command to pass cmd
Co-authored-by: xuyongliang <yongliang.xyl@alibaba-inc.com>
When Chrome crashes (e.g. SIGTRAP from CHECK() assertion), the daemon
now:
1. Reaps the zombie immediately via a SIGCHLD handler in the event loop
that calls waitpid(-1, WNOHANG)
2. Detects the crash instantly on the next command via a non-blocking
try_wait() check (has_process_exited), avoiding the 3-second CDP
timeout that is_connection_alive() would incur
3. Auto-relaunches Chrome transparently for the caller
Fixes#1017
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: re-apply download behavior on recording context
record start creates a new browser context via Target.createBrowserContext.
Browser.setDownloadBehavior called at launch only applies to the default
context, so downloads in the recording context are silently dropped.
Fix:
1. Store download_path on BrowserManager (from LaunchOptions)
2. After creating the recording context, call Browser.setDownloadBehavior
with the new browserContextId
This ensures downloads work during recording.
Fixes#1018
* fix: add download_path to third BrowserManager constructor (auto_connect_cdp)
* fix: handle proxy authentication via CDP Fetch.authRequired
Chrome's --proxy-server flag does not support credentials embedded in
the URL. When a proxy requires authentication, Chrome receives a 407
from the proxy but has no way to respond with credentials, resulting
in net::ERR_INVALID_AUTH_CREDENTIALS.
Fix by:
1. Parsing credentials from the proxy URL (already done by parse_proxy)
2. Storing them in DaemonState.proxy_credentials
3. Enabling Fetch.enable with handleAuthRequests: true
4. Responding to Fetch.authRequired events with Fetch.continueWithAuth
5. Passing only the server URL (without credentials) to --proxy-server
6. Forwarding credentials to the daemon via dedicated env vars
Also adds fallback to standard proxy env vars (HTTP_PROXY, HTTPS_PROXY,
ALL_PROXY, NO_PROXY) when AGENT_BROWSER_PROXY is not set.
Fixes#990
* refactor: use typed struct for parse_proxy, fix double Fetch.enable and username-only auth
- Replace serde_json::Value return from parse_proxy with a typed ParsedProxy struct
- Fix double Fetch.enable call when both proxy auth and domain filter are active
(the second call could overwrite handleAuthRequests from the first)
- Allow username-only proxy auth (some proxies don't require a password)
- Handle empty username/password in parse_proxy as None instead of Some("")
- Use install_domain_filter_fetch in auto_launch for consistency
- Update unit tests to use typed struct fields
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Fixes#992
When a JavaScript dialog (alert/confirm/prompt) blocks the page, agents
had no way to detect it — all commands just timed out with generic errors.
- Add `dialog status` command to check for pending dialogs
- Track dialog state via CDP Page.javascriptDialogOpening/Closed events
- Auto-inject `warning` field into all command responses when a dialog is
pending, so agents can distinguish dialog-blocked timeouts from other issues
- Document dialog commands in SKILL.md (was missing entirely), README.md,
docs site, and --help output
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Chrome's Browser.downloadWillBegin and Browser.downloadProgress events
may arrive without a sessionId or with a different sessionId than the
page session used to configure the download behavior. The previous code
required an exact session match, silently dropping these events and
causing the 30-second timeout -- which manifests as an endless download
loop when callers retry.
Changes:
- Accept Browser-domain download events regardless of sessionId while
still matching Page-domain events by session to avoid cross-tab issues
- Add a brief retry loop (up to 1s) for the GUID file to appear on disk
after Chrome signals completion, handling filesystem flush races
- Return a proper error instead of silently succeeding when the
GUID-named file cannot be found
- Apply the same sessionId fix to handle_waitfordownload and add
Browser.downloadProgress support (previously only checked
Page.downloadProgress)
Fixes#989
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
The `press` command was not parsing modifier+key chords (e.g.
`Control+a`, `Shift+Enter`). It sent the raw string as a single key
name, so CDP never applied the modifier — and the `text` field caused
the literal character to be inserted instead of triggering the shortcut.
Two changes:
1. `actions.rs` — add `parse_key_chord()` to split inputs like
`Control+Shift+a` into the base key (`a`) and a CDP modifier
bitmask (Alt=1, Ctrl=2, Meta=4, Shift=8), then call
`press_key_with_modifiers` instead of `press_key`.
2. `interaction.rs` — suppress the `text` field in `keyDown`/`keyUp`
events when Control or Meta modifiers are active, so the browser
treats them as command chords rather than text input.
Includes unit tests for the chord parser covering plain keys, single
modifiers, multi-modifier combos, modifier aliases, and edge cases.
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* Fix download command to properly handle absolute paths and click elements
The download command was not working correctly - it would return "done" but not actually download files to the specified path. The command was only setting download behavior without clicking the element or waiting for completion.
**Changes made:**
- Modified `handle_download` to take a `selector` parameter and click the download element
- Added proper absolute path resolution and directory creation
- Implemented CDP event listening to wait for download completion with 30s timeout
- Added file renaming logic to handle Chrome's GUID-based temporary filenames
- Changed response format to return the actual download path
- Fixed function signature to use `&mut DaemonState` for state modifications
**Implementation details:**
- Uses `Browser.downloadWillBegin` and `Browser.downloadProgress` CDP events to track downloads
- Falls back to finding the most recently modified file if GUID capture fails
- Creates parent directories automatically if they don't exist
- Handles both absolute and relative path inputs
Fixes#965
* Address review feedback: harden download path handling
- Canonicalize download directory to prevent path traversal attacks
- Remove dangerous fallback that renamed the most-recently-modified file
in the directory (could silently rename unrelated files)
- Extract timeout to a named constant (DOWNLOAD_TIMEOUT)
* Fix download event loop: handle canceled state and Page.downloadWillBegin
- Detect "canceled" download state and return an error immediately instead
of spinning until the 30s timeout.
- Also capture the download GUID from the deprecated Page.downloadWillBegin
event for older Chrome compatibility, matching the existing
Page.downloadProgress fallback.
- Consolidate duplicated session/event checks with a shared is_this_session
variable and use match for cleaner state handling.
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* feat: add network request detail and filtering for request tracking
- Add `network request <requestId>` command to view full request/response
details including response body via CDP Network.getResponseBody
- Add --type, --method, --status filter flags to `network requests`
- --type: comma-separated resource types (xhr,fetch,document)
- --method: filter by HTTP method
- --status: supports exact (200), class (2xx), range (400-499)
- Extend TrackedRequest with request_id, post_data, status,
response_headers, mime_type fields
- Update Network.responseReceived handler to also populate
tracked_requests (previously only updated HAR entries)
- Add tests for parse commands and matches_status_filter
- Update README, SKILL.md, docs, and help text
Closes#932
* fix: show request ID and status in network requests output
* fix: add ref for cursor-interactive content roles
* fix: format
* feat: always include cursor-interactive elements in snapshot, -C is deprecated
* feat: process StaticText aggregation and deduplication
* update test
* clean up
* fix: escape text of elements in snapshot
* fix: redundant slicing
* fix: cargo fmt
* feat: deduplicate redundant StaticText
---------
Co-authored-by: 羲洋 <lipengyang.lpy@alibaba-inc.com>
* fix: prevent state commands from starting daemon without session_name
(#677)
State management commands (state_list, state_show, state_clear,
state_clean, state_rename) are pure file operations that don't need a
running daemon. Previously, these commands would trigger daemon
startup
via ensure_daemon(), and if AGENT_BROWSER_SESSION_NAME was exported
after the first command (e.g. `state clear --all`), the daemon would
start without session_name. Subsequent open/close commands would
reuse
that daemon, causing close to skip state persistence entirely.
Fix: execute state management commands locally in the CLI process
before
ensure_daemon() is called. This is done via a new
dispatch_state_command() function in state.rs that centralizes the
command routing, used by both the CLI (local path) and the daemon
(batch/IPC path).
Also:
- Add OutputOptions::from_flags() helper to deduplicate construction
- Add unit tests for dispatch_state_command routing and error
handling
* style: fix fmt and clippy warnings
- Remove redundant closure in dispatch_state_command (clippy::redundant_closure)
- Remove needless borrow in run_batch (clippy::needless_borrow)
- Fix trailing blank lines (rustfmt)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: remove dead code and unused variables in actions.rs
Remove `daemon_state_from_env` (superseded by `DaemonState::new`) and
`resolve_semantic_locator` (superseded by `handle_semantic_locator`),
both of which had zero call sites. Also remove unused `_session_id`
bindings in `handle_find` and `handle_multiselect`, and a no-op
`let _ = sid` in `handle_getbyrole`.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: simplify `_sid` to `_` per review feedback
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: use `is_ok()` instead of `if let Ok(_)` for idiomatic Rust
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
Fixed an issue where custom viewport settings were ignored in WebSocket streaming, causing status messages to always report hardcoded dimensions (1280x720) instead of the actual viewport size.
## Changes Made
- **Added viewport tracking to StreamServer**: New `viewport_width` and `viewport_height` fields to store current dimensions
- **Updated viewport synchronization**: Modified `handle_viewport()` and `handle_device()` to update the stream server when viewport changes
- **Fixed status message broadcasting**: Replaced hardcoded dimensions with actual viewport values in `broadcast_status()` calls
- **Improved screencast defaults**: Changed screencast to use stored viewport dimensions as defaults instead of hardcoded 1280x720
- **Added viewport getter methods**: New `set_viewport()` and `viewport()` methods on StreamServer for dimension management
This ensures that WebSocket clients receive consistent viewport dimensions across all message types (status and frame messages), matching the actual browser viewport settings.
Fixes#950
Navigate with load, then wait for username/password/submit selectors using the default action timeout. This avoids networkidle hangs on pages with continuous background requests.
* fix: restore origin-scoped --headers persistence across commands
In the v0.20 Rust rewrite, headers passed via --headers on open were
only applied to that single navigation via Network.setExtraHTTPHeaders,
which did not persist them for subsequent commands. In v0.19
(Playwright-based), these headers persisted for all subsequent
same-origin requests.
This restores the v0.19 behavior using CDP Fetch interception:
- A background task processes Fetch.requestPaused events in real-time,
injecting origin-scoped headers into matching requests and continuing
non-matching requests unmodified. This avoids the deadlock that occurs
when Fetch interception pauses requests during Page.navigate or
Runtime.evaluate (which block waiting for completion).
- The same background task also handles domain filtering and route
interception, replacing the previous drain-between-commands approach
that couldn't process events during navigation or script evaluation.
Fixes:
- --headers persist for same-origin navigations without re-passing flag
- --headers persist for in-page fetch/XHR to the same origin
- --headers do not leak to cross-origin navigations or sub-resources
- `set headers` (global) is unaffected and stacks with --headers
- Domain filter Fetch interception no longer deadlocks during navigation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: apply cargo fmt formatting
* revert inaccurate comment change
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* feat: enhance HAR entries with timings, cookies, postData, and protocol normalisation
Extends #874's HarEntry struct and CDP handlers with richer capture:
- wall_time (f64) replaces pre-formatted started_date_time, preserving
sub-second precision from CDP wallTime for accurate RFC 3339 output
- request_headers / response_headers changed from Value to Vec<(String,String)>
for typed access without re-parsing JSON objects
- post_data captured from Network.requestWillBeSent for HAR postData
- cdp_timing and loading_finished_timestamp captured from responseReceived
and loadingFinished respectively, enabling har_compute_timings to produce
accurate blocked/dns/connect/ssl/send/wait/receive phases
- har_cdp_protocol_to_http_version normalises CDP protocol strings
(h2 -> HTTP/2.0, h3 -> HTTP/3.0, etc.)
- Request cookies parsed from Cookie header; response cookies from Set-Cookie
with ';'-first split to correctly strip Path/HttpOnly attributes before '='
- har_wall_time_to_rfc3339 replaces har_started_date_time, using the time
crate directly on the f64 epoch value
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: apply cargo fmt
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Add iframe support for CLI interactions and snapshots
This PR adds comprehensive iframe support to the agent browser CLI, allowing users to interact with elements inside iframes seamlessly.
## Problem
Users couldn't interact with elements inside iframes via the command line. The existing `frame` command was non-functional as it set `active_frame_id` but no other code read this value.
## Changes Made
### Enhanced Frame Context Tracking
- Added `frame_id` field to `RefEntry` to track which frame each element reference belongs to
- Updated `RefMap::add` and related methods to accept and store frame context
- Modified element resolution functions to use frame context from ref entries
### Improved Frame Command
- Fixed the existing `frame` command to actually work by threading `active_frame_id` through snapshot operations
- Added support for iframe element references (e.g., `frame @e2`) in addition to CSS selectors
- Enhanced frame detection to work with both named frames and iframe elements
### Updated Snapshot Behavior
- Modified `take_snapshot` to accept optional frame context parameter
- Updated all snapshot call sites to pass appropriate frame context
- Maintained backward compatibility while enabling frame-scoped operations
### Element Resolution Updates
- Updated `resolve_element_center` and `resolve_element_object_id` to use frame context from ref entries
- Modified `find_node_id_by_role_name` to support frame-specific element lookup
- Ensured all interaction functions work correctly within iframe contexts
## Implementation Details
- Frame context is now properly propagated through the entire element interaction pipeline
- The `frame` command can accept both CSS selectors and element references
- All existing functionality remains intact while adding iframe capabilities
- Added `Iframe` to interactive roles for better element discovery
Fixes#863
* docs: add iframe support documentation
Document the new iframe capabilities across all documentation surfaces:
- Auto-inlining of iframe content in snapshots
- Direct interaction with iframe element refs
- frame command support for element refs (@e3)
- Scoped snapshots via frame switching
* fix: pass active frame context to diff snapshots and fix nameless iframe lookup
- handle_diff_snapshot now respects active_frame_id instead of always
passing None, so diff snapshots work correctly inside iframes
- Nameless/id-less iframes now fall back to src URL (or null) instead of
the literal string 'frame' which never matched any frame in the tree
* fix: resolve iframe frame ID via DOM.describeNode and reduce code duplication
- handle_frame: Use DOM.describeNode + contentDocument.frameId to resolve
iframe frame IDs directly, fixing failures for nameless iframes that
lack name/id/src attributes
- element.rs: Deduplicate add() by delegating to add_with_frame()
- snapshot.rs: Guard against out-of-bounds insert_str when iframe marker
is on the last line without a trailing newline
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
Expose HAR recording as a CLI subcommand under the existing `network`
command so users can capture and export network traffic without a
separate tool or opening the browser twice.
- Parse `network har start` and `network har stop [path]` in commands.rs
- Enrich HarEntry with request/response headers, timestamps, status text,
resource type, HTTP version, and body sizes from CDP events
- Produce HAR 1.2 output with creator/browser metadata, query strings,
and proper header arrays compatible with Chrome DevTools HAR viewer
- Auto-generate output path under ~/.agent-browser/tmp/har/ when omitted
- Add har_stop to skip_launch list so export works without a live browser
- Update help text, README, docs site, SKILL.md, and security policy docs
- Add unit tests for parsing, HAR entry serialization, and stop behavior
* fix: Windows auto-connect profiling (#835)
Fix three interrelated bugs causing `--auto-connect` to fail on Windows,
plus a UX issue where auto-connect hijacked existing tabs:
1. Stale DevToolsActivePort — add TCP port liveness check before returning
M144+ WebSocket URL; remove stale files when port is dead.
2. Missing Windows error codes — add os error 10061 (WSAECONNREFUSED) and
10054 (WSAECONNRESET) to is_transient_error() so daemon startup races
are retried on Windows.
3. --auto-connect not propagated to daemon — add auto_connect to
DaemonOptions, set AGENT_BROWSER_AUTO_CONNECT env var via
apply_daemon_env(), and guard the headed launch block so it doesn't
send a second launch that overrides the auto-connect.
4. Auto-connect opens a fresh tab — after connecting to an existing
Chrome, create a new about:blank tab and bring it to front so
navigations don't hijack the user's existing tabs.
Made-with: Cursor
* fix: address review feedback — cargo fmt, shared helper, Windows tests
- Run cargo fmt on is_port_reachable() formatting
- Extract duplicated auto-connect-with-fresh-tab logic into
connect_auto_with_fresh_tab() helper used by both handle_launch()
and auto_launch()
- Add unit tests for Windows WSAECONNREFUSED (os error 10061) and
WSAECONNRESET (os error 10054) in is_transient_error()
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* feat: embed cursor-interactive elements into snapshot tree
* optimize format
* fix: address review feedback for e2e_snapshot_cursor_interactive unitest
---------
Co-authored-by: 羲洋 <lipengyang.lpy@alibaba-inc.com>
* fix: restore WebSocket streaming in native daemon
The v0.20.0 Rust rewrite broke WebSocket streaming — connections opened
but received zero messages before closing. Multiple issues contributed:
1. StreamServer was dropped immediately after creation in daemon.rs,
closing the broadcast channel and killing all WS connections.
2. Screencast frames were only processed during command polling
(drain_cdp_events) instead of in real-time, unlike the 0.19.0
TypeScript cdp.on('Page.screencastFrame') callback.
3. Auto-start/stop screencast on WS client connect/disconnect was
missing from the Rust implementation.
4. Screencast CDP commands used the wrong session ID (daemon session
name instead of the CDP page session from Target.attachToTarget).
5. Broadcast channel Lagged errors killed WS connections instead of
being handled gracefully.
The fix adds a background CDP event loop in StreamServer that subscribes
to Chrome events and broadcasts screencast frames in real-time, properly
tracks the CDP page session ID, restores auto-screencast lifecycle, and
keeps the StreamServer alive in DaemonState.
Fixes#820
* fix: use actual CDP session ID for input dispatch in stream WebSocket
Pass the real cdp_session_id (from Target.attachToTarget) through to
handle_ws_client instead of an empty string. Previously, input commands
(mouse, keyboard, touch) were sent with `"sessionId": ""` which Chrome
silently rejects. Now the correct page session ID is read at dispatch
time, and when no session ID is set yet (before browser launch),
the field is omitted entirely via `None` so Chrome uses browser-level
dispatch.
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: snapshot --selector scopes to the matched element subtree
The native Rust daemon accepted the --selector flag but never used it —
the full accessibility tree was always returned regardless of the
selector. This restores the 0.19.0 behaviour where snapshot --selector
returns only the subtree rooted at the matched CSS selector.
The implementation resolves the selector via Runtime.evaluate, fetches
the full DOM subtree with DOM.describeNode(depth: -1) to collect all
descendant backendNodeIds, then filters the AX tree to render only the
nodes whose backendDOMNodeId falls within that set. This correctly
handles elements like <body> that don't map to a direct AX node.
Also fixes handle_snapshot reading "depth" instead of "maxDepth" from
the command JSON, which caused --depth to be silently ignored.
Fixes#822
* style: run cargo fmt on snapshot.rs
---------
Co-authored-by: ctate <366502+ctate@users.noreply.github.com>
* fix: replace screenshot polling with screencast-based piped ffmpeg recording
Recording previously used Page.captureScreenshot polling at 10fps,
which was CPU-heavy and produced inconsistent results. Now uses
Page.startScreencast with throttled acks (35ms interval) to receive
frames event-driven from Chrome, and pipes JPEG data directly to
ffmpeg stdin in real-time instead of saving temp files.
- Spawn ffmpeg at recording start with piped stdin (image2pipe)
- Background task receives screencast frames, interpolates gaps by
repeating the last frame based on timestamps, targets 25fps
- Ack throttling controls Chrome's frame push rate
- Fix: current frame was never written after the first one
- Fix: frame count was read before task finished padding
- Remove tokio-util dependency (replaced CancellationToken with oneshot)
- Add tokio "process" feature for async child process stdin pipe
- Extract start/stop_recording_task helpers on DaemonState
- Add tests for restart, ffmpeg codec selection, and stop without task
* fmt
* chore
* fix: switch WebM codec from VP9 to VP8 for correct framerate and browser
compatibility
VP9 realtime encoder ignored input framerate, producing 10fps output
instead of 25fps. This caused inconsistent playback in browsers.
VP8 respects -framerate 25 and has wider browser playback support.
* fmt
* fix: add kill_on_drop to ffmpeg process to prevent zombie on task panic
* fix: switch from screencast to screenshot polling for reliable recording duration
Screencast only pushes frames on visual changes, producing short videos
on static pages. Screenshot polling captures at a fixed 10fps interval
regardless of page activity, guaranteeing duration matches wall-clock time.
ffmpeg piped stdin architecture is preserved — no temp files.
---------
Co-authored-by: hyunjinee <leehj0110@kakao.com>
Tom Dale