Compare commits

...
Author SHA1 Message Date
leeguooooo 0db6604105 chore(release): bump to 0.27.0-fork.4 — ref identity guard 2026-05-09 03:26:48 +09:00
leeguooooo 007fd1b27f fix(refs): verify identity before using cached backendNodeId
Closes the "click @e20 hits the sibling element" bug. Real-world
example: snapshot shows @e20=[button "Add post"] next to
@e17=[button "Post all"]. By the time you click @e20, React has
re-rendered — and React often re-uses the same <button> DOM node
across renders, just updating its accessible name. The cached
backendNodeId still resolves to a real, well-positioned node, so
the click lands cleanly. It just lands on what is now the "Post all"
button, silently submitting the entire thread instead of adding a
draft row.

Before every ref-based interaction (click / fill / type / hover /
select / drag — anything routing through resolve_element_center or
resolve_element_object_id), call Accessibility.getPartialAXTree for
the cached backendNodeId and check role + name still match the
snapshot entry. On mismatch, abort with an error that names both
labels:

  Ref @e20 no longer matches its snapshot. Was [button "Add post"],
  now [button "Post all"].
  ...Take a fresh snapshot, then re-target.

If the node is gone (CDP fails / no AX node), we silently fall
through to the existing "find by role+name" recovery path, so this
guard never makes a working flow worse.

Adds one CDP roundtrip per ref interaction (~5–20ms). Disable with
AGENT_BROWSER_VERIFY_REF=0 if you control the page lifecycle and
need the latency back.
2026-05-09 03:26:20 +09:00
leeguooooo 3d1132af90 chore(release): bump to 0.27.0-fork.3 — click paint-settle + wait --gone 2026-05-09 02:45:45 +09:00
leeguooooo 90ba44cd38 feat(wait): add --gone / --hidden flags so users can fail fast on closed UIs
Pairs with the click paint-settle fix: even with that, a thread builder
that clicks "Add post" can race a misbehaving handler that closes the
parent modal instead of mounting the next textbox. To make that case
observable instead of silently corrupting the next inserttext, you can
now write:

  click @add-post
  wait .modal --gone --timeout 2000   # asserts modal stays mounted
  inserttext "tweet 3"

If the modal vanished, `wait --gone` succeeds — flip the assertion to
`wait .modal` (default visible) to fail-fast on disappearance.

Implementation just sets `state: "detached"` (or "hidden") on the wait
command — daemon-side `wait_for_selector` already supported these
states; only the CLI parser was missing the user-facing flag.

Also accepts `--detached` as alias for `--gone` to match the daemon's
internal vocabulary.
2026-05-09 02:45:34 +09:00
leeguooooo 52f8ead0f2 fix(click): wait for paint to settle so SPA renders complete before next command
Closes a real-world race that broke X multi-tweet thread composition
(and similar SPA flows): clicking "Add post" returned immediately,
inserttext fired before React had committed the new textarea, the
keystroke landed on the dialog wrapper, and X interpreted the stray
input as a request to dismiss the modal.

After mouseReleased we now wait for two requestAnimationFrame ticks
plus a microtask boundary (~33ms at 60fps, bounded). That's enough
for React/Vue/Svelte to commit any state update scheduled by the
click handler. Errors during the wait are swallowed — a click never
fails because of post-processing.

Opt out for perf-sensitive scripts that don't drive SPA UIs:
  AGENT_BROWSER_CLICK_WAIT_STABLE=0
2026-05-09 02:45:21 +09:00
leeguooooo ffa5bd63f6 chore(release): bump to 0.27.0-fork.2 — find error UX + URL preservation 2026-05-09 01:41:25 +09:00
leeguooooo 926f08203c chore: regenerate pnpm-lock.yaml after dashboard removal
The previous lockfile had ~11k lines of transitive deps for
packages/dashboard which we deleted in 86c4cff. Re-running pnpm install
shrinks it to ~24 lines (just husky for git hooks).
2026-05-09 01:41:12 +09:00
leeguooooo 2b1a3c308a feat(daemon): preserve URL across version-mismatch restart
Before: after `npm i -g` upgrade, the next agent-browser command would
detect daemon version mismatch, kill the old daemon, spawn a fresh one,
and connect to a brand-new about:blank tab. The user's previous
navigation state was silently lost — `get url` returned about:blank
even though the user's Chrome was still on the same page.

Now: before killing the old daemon, the CLI synchronously asks it for
its current URL via the existing socket. If non-empty and not
about:blank, it's persisted to a `.restore-url` sidecar in the socket
dir. After the new daemon spawns and auto-connects, it reads the
sidecar (read-and-delete), navigates the fresh tab to the saved URL,
and prints `⚠ Restored previous URL: <url>`.

Manual `agent-browser close` does NOT write the sidecar, so a clean
shutdown won't trigger surprise navigation. The sidecar is consumed on
read regardless of whether navigation succeeded, so a stale entry
can't haunt later auto-launches.
2026-05-09 01:41:07 +09:00
leeguooooo 6c556e519d feat(parse): friendly error when find has --flag where action verb expected
Before, `agent-browser find role button --name Submit` errored at the
daemon side with the cryptic `Unknown subaction: --name`. Now it errors
at parse time with the offending flag echoed back, the list of valid
actions (click, fill, check, hover, text), and a "Did you mean" hint
showing where to put the action verb.

Backwards compat: `find role button` (no flags, no action) still
defaults to click — only `--xxx` in action position errors.
2026-05-09 01:40:56 +09:00
leeguooooo 9e48b0757c fix(package): drop ./ prefix from bin entries
npm 10+ strips bin paths starting with ./ as invalid, leaving the
package with no executable entries (so `npm i -g` doesn't put any
binary on PATH). Match the upstream form `bin/agent-browser.js`.
2026-05-09 00:52:36 +09:00
9 changed files with 379 additions and 11159 deletions
+1 -1
View File
@@ -45,7 +45,7 @@ dependencies = [
[[package]]
name = "agent-browser-stealth"
version = "0.27.0-fork.1"
version = "0.27.0-fork.4"
dependencies = [
"aes-gcm",
"async-trait",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "agent-browser-stealth"
version = "0.27.0-fork.1"
version = "0.27.0-fork.4"
edition = "2021"
description = "Fast browser automation CLI for AI agents"
license = "Apache-2.0"
+154 -4
View File
@@ -614,17 +614,44 @@ fn parse_command_inner(args: &[String], flags: &Flags) -> Result<Value, ParseErr
return Ok(cmd);
}
// --gone / --hidden: wait for an element to leave the DOM or
// become invisible. Useful after a click that's supposed to
// close a dialog, so the next command fails fast instead of
// racing into a half-rendered UI.
let state_override = if rest.iter().any(|&s| s == "--gone" || s == "--detached") {
Some("detached")
} else if rest.iter().any(|&s| s == "--hidden") {
Some("hidden")
} else {
None
};
// Default: selector or timeout
if let Some(arg) = rest.first() {
// First non-flag positional is selector or numeric timeout
let positional = rest.iter().find(|&&s| !s.starts_with("--"));
let timeout_ms = rest
.iter()
.position(|&s| s == "--timeout")
.and_then(|idx| rest.get(idx + 1))
.and_then(|s| s.parse::<u64>().ok());
if let Some(arg) = positional {
if let Ok(timeout) = arg.parse::<u64>() {
Ok(json!({ "id": id, "action": "wait", "timeout": timeout }))
} else {
Ok(json!({ "id": id, "action": "wait", "selector": arg }))
let mut cmd = json!({ "id": id, "action": "wait", "selector": arg });
if let Some(state) = state_override {
cmd["state"] = json!(state);
}
if let Some(t) = timeout_ms {
cmd["timeout"] = json!(t);
}
Ok(cmd)
}
} else {
Err(ParseError::MissingArguments {
context: "wait".to_string(),
usage: "wait <selector|ms|--url|--load|--fn|--text>",
usage: "wait <selector|ms> [--gone|--hidden] [--timeout ms]",
})
}
}
@@ -2180,7 +2207,33 @@ fn parse_find(rest: &[&str], id: &str) -> Result<Value, ParseError> {
_ => "find <locator> <value> [action] [text]",
},
})?;
let subaction = rest.get(2).unwrap_or(&"click");
let raw_subaction = rest.get(2).copied();
if let Some(s) = raw_subaction {
if s.starts_with("--") {
return Err(ParseError::InvalidValue {
message: format!(
"Missing action verb for `find {locator}` (got `{flag}` where action was expected).\n\
Valid actions: click, fill, check, hover, text\n\
Did you mean: agent-browser find {locator} <value> click {flag} ...?",
locator = locator,
flag = s,
),
usage: match *locator {
"role" => "find role <role> <action> [--name <name>] [--exact]",
"text" => "find text <text> <action> [--exact]",
"label" => "find label <label> <action> [text] [--exact]",
"placeholder" => "find placeholder <text> <action> [text] [--exact]",
"alt" => "find alt <text> <action> [--exact]",
"title" => "find title <text> <action> [--exact]",
"testid" => "find testid <id> <action> [text]",
"first" => "find first <selector> <action> [text]",
"last" => "find last <selector> <action> [text]",
_ => "find <locator> <value> <action> [text]",
},
});
}
}
let subaction = raw_subaction.unwrap_or("click");
let mut name: Option<&str> = None;
let mut exact = false;
let mut fill_parts: Vec<&str> = Vec::new();
@@ -5105,4 +5158,101 @@ mod tests {
let cmd = parse_command(&args("batch"), &default_flags()).unwrap();
assert!(cmd.get("commands").is_none());
}
// === parse_find: friendly error when action verb is missing ===
#[test]
fn test_find_role_missing_action_verb_with_name_flag() {
let err = parse_command(
&args("find role button --name Submit"),
&default_flags(),
)
.unwrap_err();
let msg = err.format();
assert!(
msg.contains("Missing action verb"),
"expected 'Missing action verb' in error, got: {msg}"
);
assert!(
msg.contains("Did you mean"),
"expected 'Did you mean' suggestion, got: {msg}"
);
assert!(
msg.contains("--name"),
"error should echo the offending flag back, got: {msg}"
);
}
#[test]
fn test_find_testid_missing_action_verb_with_exact_flag() {
let err = parse_command(
&args("find testid foo --exact"),
&default_flags(),
)
.unwrap_err();
assert!(err.format().contains("Missing action verb"));
}
#[test]
fn test_find_role_with_action_still_works() {
let cmd = parse_command(
&args("find role button click --name Submit"),
&default_flags(),
)
.unwrap();
assert_eq!(cmd["action"], "getbyrole");
assert_eq!(cmd["subaction"], "click");
assert_eq!(cmd["name"], "Submit");
}
#[test]
fn test_find_role_default_subaction_click_when_no_action() {
// Backwards compat: `find role button` (no flags, no action) keeps
// defaulting to click — only `--xxx` in action position errors.
let cmd = parse_command(&args("find role button"), &default_flags()).unwrap();
assert_eq!(cmd["subaction"], "click");
}
// === wait --gone / --hidden ===
#[test]
fn test_wait_selector_default_visible() {
let cmd = parse_command(&args("wait .toast"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "wait");
assert_eq!(cmd["selector"], ".toast");
assert!(cmd.get("state").is_none(), "default state stays implicit");
}
#[test]
fn test_wait_selector_gone_sets_detached_state() {
let cmd = parse_command(&args("wait .toast --gone"), &default_flags()).unwrap();
assert_eq!(cmd["selector"], ".toast");
assert_eq!(cmd["state"], "detached");
}
#[test]
fn test_wait_selector_hidden_sets_hidden_state() {
let cmd = parse_command(&args("wait .toast --hidden"), &default_flags()).unwrap();
assert_eq!(cmd["state"], "hidden");
}
#[test]
fn test_wait_gone_with_timeout() {
let cmd = parse_command(
&args("wait .modal --gone --timeout 2000"),
&default_flags(),
)
.unwrap();
assert_eq!(cmd["selector"], ".modal");
assert_eq!(cmd["state"], "detached");
assert_eq!(cmd["timeout"], 2000);
}
#[test]
fn test_wait_numeric_timeout_still_works() {
// `wait 500` keeps meaning "sleep 500ms", not "wait for selector 500"
let cmd = parse_command(&args("wait 500"), &default_flags()).unwrap();
assert_eq!(cmd["timeout"], 500);
assert!(cmd.get("selector").is_none());
}
}
+41
View File
@@ -127,6 +127,15 @@ fn get_version_path(session: &str) -> PathBuf {
get_socket_dir().join(format!("{}.version", session))
}
/// Path to the sidecar file that records the URL the previous daemon was on,
/// used to restore navigation after a version-mismatch restart. Only written
/// when the version-mismatch branch fires; cleared after the new daemon
/// reads it. Manual `close` does not write this file, so a clean shutdown
/// won't trigger surprise navigation.
pub fn get_restore_url_path(session: &str) -> PathBuf {
get_socket_dir().join(format!("{}.restore-url", session))
}
/// Clean up stale socket and PID files for a session
pub fn cleanup_stale_files(session: &str) {
let pid_path = get_pid_path(session);
@@ -135,6 +144,10 @@ pub fn cleanup_stale_files(session: &str) {
let _ = fs::remove_file(&version_path);
let stream_path = get_socket_dir().join(format!("{}.stream", session));
let _ = fs::remove_file(&stream_path);
// Note: the .restore-url sidecar is intentionally NOT removed here —
// it lives across the brief window between killing the old daemon
// and the new daemon reading it back. The new daemon deletes it after
// restoring (see actions::auto_launch).
#[cfg(unix)]
{
@@ -527,6 +540,24 @@ fn daemon_version_matches(session: &str) -> bool {
}
}
/// One-shot socket query for the running daemon's current URL.
/// Returns None on any kind of failure — caller must treat as best-effort.
fn query_current_url(session: &str) -> Option<String> {
let cmd = serde_json::json!({
"id": format!("restore-url-probe-{}", std::process::id()),
"action": "url",
});
let resp = send_command_once(&cmd, session).ok()?;
if !resp.success {
return None;
}
resp.data
.as_ref()
.and_then(|d| d.get("url"))
.and_then(|v| v.as_str())
.map(|s| s.to_string())
}
/// Kill a running daemon by reading its PID file and sending a kill signal.
fn kill_stale_daemon(session: &str) {
// Remove the socket first so no new connections reach the old daemon
@@ -592,6 +623,16 @@ pub fn ensure_daemon(session: &str, opts: &DaemonOptions) -> Result<DaemonResult
"{} Daemon version mismatch detected, restarting...",
crate::color::warning_indicator()
);
// Best-effort: ask the old daemon for its current URL so the
// new daemon can restore navigation after auto-connect. If the
// query fails (already shutting down, no browser, etc.) we
// silently skip — the user just sees about:blank as before.
if let Some(url) = query_current_url(session) {
if !url.is_empty() && url != "about:blank" {
let path = get_restore_url_path(session);
let _ = fs::write(&path, &url);
}
}
kill_stale_daemon(session);
// Fall through to spawn a new daemon below
} else {
+45 -1
View File
@@ -9,7 +9,7 @@ use std::sync::Arc;
use time::{format_description::well_known::Rfc3339, OffsetDateTime};
use tokio::sync::{broadcast, oneshot, RwLock};
use crate::connection::get_socket_dir;
use crate::connection::{get_restore_url_path, get_socket_dir};
use super::auth;
use super::browser::{should_track_target, BrowserManager, WaitUntil};
@@ -1552,6 +1552,7 @@ async fn auto_launch(state: &mut DaemonState) -> Result<(), String> {
try_auto_restore_state(state).await;
try_load_storage_state(state, &storage_state_path).await;
apply_stealth_to_browser(state).await;
try_restore_navigation(state).await;
return Ok(());
}
@@ -1573,6 +1574,7 @@ async fn auto_launch(state: &mut DaemonState) -> Result<(), String> {
try_auto_restore_state(state).await;
try_load_storage_state(state, &storage_state_path).await;
apply_stealth_to_browser(state).await;
try_restore_navigation(state).await;
return Ok(());
}
Err(_e) => {
@@ -1660,6 +1662,7 @@ async fn auto_launch(state: &mut DaemonState) -> Result<(), String> {
try_load_storage_state(state, &storage_state_path).await;
// Apply stealth anti-detection patches after browser is ready
apply_stealth_to_browser(state).await;
try_restore_navigation(state).await;
Ok(())
}
@@ -1770,6 +1773,47 @@ async fn apply_stealth_to_browser(state: &DaemonState) {
}
}
/// If the previous daemon left a `.restore-url` sidecar (because it was killed
/// by a version-mismatch restart), navigate the freshly-connected browser to
/// that URL so `agent-browser get url` after `npm i -g` upgrade still reports
/// the page the user was on. Read-and-delete: the file is removed regardless
/// of whether navigation succeeds, so a stale sidecar can't haunt later
/// auto-launches.
async fn try_restore_navigation(state: &mut DaemonState) {
let path = get_restore_url_path(&state.session_id);
let url = match fs::read_to_string(&path) {
Ok(s) => s.trim().to_string(),
Err(_) => return,
};
let _ = fs::remove_file(&path);
if url.is_empty() {
return;
}
let Some(mgr) = state.browser.as_mut() else {
return;
};
state.ref_map.clear();
state.iframe_sessions.clear();
state.active_frame_id = None;
match mgr.navigate(&url, super::browser::WaitUntil::Load).await {
Ok(_) => {
eprintln!(
"{} Restored previous URL: {}",
crate::color::warning_indicator(),
url
);
}
Err(e) => {
eprintln!(
"{} Could not restore previous URL ({}): {}",
crate::color::warning_indicator(),
url,
e
);
}
}
}
fn launch_options_from_env() -> LaunchOptions {
let headed = env::var("AGENT_BROWSER_HEADED")
.map(|v| v == "1" || v == "true")
+93
View File
@@ -163,6 +163,30 @@ pub async fn resolve_element_center(
// Try cached backend_node_id first (fast path)
if let Some(backend_node_id) = entry.backend_node_id {
// Identity check: React often re-uses the same DOM node when
// re-rendering — backendNodeId stays the same but accessibleName
// / role changes. Without this verification, `click @e20` (saved
// when the button said "Add post") happily clicks the *same*
// node that now says "Post all", silently submitting the thread.
//
// Set AGENT_BROWSER_VERIFY_REF=0 to skip (saves one CDP
// roundtrip per ref-based interaction; only safe if you know
// the page is static between snapshot and click).
if std::env::var("AGENT_BROWSER_VERIFY_REF").as_deref() != Ok("0") {
if let Err(e) = verify_ref_identity(
client,
effective_session_id,
backend_node_id,
&ref_id,
&entry.role,
&entry.name,
)
.await
{
return Err(e);
}
}
let result: Result<DomGetBoxModelResult, String> = client
.send_command_typed(
"DOM.getBoxModel",
@@ -230,6 +254,24 @@ pub async fn resolve_element_object_id(
// Try cached backend_node_id first (fast path)
if let Some(backend_node_id) = entry.backend_node_id {
// Same identity guard as resolve_element_center — see that
// function for why React DOM-node-reuse breaks ref-based
// interactions if we skip this.
if std::env::var("AGENT_BROWSER_VERIFY_REF").as_deref() != Ok("0") {
if let Err(e) = verify_ref_identity(
client,
effective_session_id,
backend_node_id,
&ref_id,
&entry.role,
&entry.name,
)
.await
{
return Err(e);
}
}
let result: Result<DomResolveNodeResult, String> = client
.send_command_typed(
"DOM.resolveNode",
@@ -333,6 +375,57 @@ fn resolve_frame_session<'a>(
.unwrap_or(session_id)
}
/// Verify that the cached backendNodeId still has the same accessible role
/// and name it had when the snapshot ran. Catches the case where React (or
/// any reconciler) reused the DOM node for a different component instance
/// — same physical node, different semantics.
///
/// On mismatch, returns an actionable error naming both the snapshot label
/// and the current label so the agent can re-snapshot intelligently.
/// On any CDP failure (e.g. node deleted), returns Ok(()) so the caller's
/// existing fallback (`find_node_id_by_role_name`) takes over.
async fn verify_ref_identity(
client: &CdpClient,
session_id: &str,
backend_node_id: i64,
ref_id: &str,
expected_role: &str,
expected_name: &str,
) -> Result<(), String> {
let params = serde_json::json!({
"backendNodeId": backend_node_id,
"fetchRelatives": false,
});
let resp: Result<GetFullAXTreeResult, String> = client
.send_command_typed("Accessibility.getPartialAXTree", &params, Some(session_id))
.await;
let Ok(tree) = resp else {
// Node likely gone; let the box-model call fail and trigger fallback.
return Ok(());
};
// Find the AXNode for our backendNodeId. fetchRelatives=false still
// returns ancestors; the target node has the matching backendNodeId.
let Some(node) = tree
.nodes
.iter()
.find(|n| n.backend_d_o_m_node_id == Some(backend_node_id))
else {
return Ok(());
};
let actual_role = extract_ax_string(&node.role);
let actual_name = extract_ax_string(&node.name);
if actual_role == expected_role && actual_name == expected_name {
return Ok(());
}
Err(format!(
"Ref {} no longer matches its snapshot. Was [{} \"{}\"], now [{} \"{}\"].\n\
The DOM mutated between snapshot and interaction (typical with React/Vue \
reusing nodes during re-render). Take a fresh snapshot, then re-target.\n\
To bypass this guard set AGENT_BROWSER_VERIFY_REF=0.",
ref_id, expected_role, expected_name, actual_role, actual_name,
))
}
/// Re-query the accessibility tree to find a node matching role+name+nth,
/// returning its fresh backendDOMNodeId. This uses the same data source
/// (Accessibility.getFullAXTree) that built the ref map during snapshot,
+33
View File
@@ -884,6 +884,38 @@ pub async fn tap_touch(
Ok(())
}
/// After a click is dispatched, give the page two animation frames + a
/// microtask boundary to let React/Vue/Svelte commit any state update
/// scheduled by the click handler. Without this wait, follow-up commands
/// (e.g. `inserttext` against the textbox the click was supposed to mount)
/// race the renderer and can land on stale or wrong elements.
///
/// The wait is bounded to ~33ms in the common case (two RAFs at 60fps) and
/// returns immediately on any error — never an exception path.
///
/// Set `AGENT_BROWSER_CLICK_WAIT_STABLE=0` to disable for perf-sensitive
/// scripts that don't drive SPA UIs.
async fn wait_for_paint_settled(client: &CdpClient, session_id: &str) {
if std::env::var("AGENT_BROWSER_CLICK_WAIT_STABLE").as_deref() == Ok("0") {
return;
}
let script = "new Promise(resolve => \
requestAnimationFrame(() => \
requestAnimationFrame(() => \
queueMicrotask(() => resolve(true)))))";
let _ = client
.send_command_typed::<_, Value>(
"Runtime.evaluate",
&EvaluateParams {
expression: script.to_string(),
return_by_value: Some(true),
await_promise: Some(true),
},
Some(session_id),
)
.await;
}
async fn dispatch_click(
client: &CdpClient,
session_id: &str,
@@ -955,6 +987,7 @@ async fn dispatch_click(
)
.await?;
wait_for_paint_settled(client, session_id).await;
Ok(())
}
+4 -4
View File
@@ -1,6 +1,6 @@
{
"name": "agent-browser-stealth",
"version": "0.27.0-fork.1",
"version": "0.27.0-fork.4",
"description": "Browser automation CLI for AI agents — stealth fork with anti-detection",
"type": "module",
"files": [
@@ -11,9 +11,9 @@
"extensions"
],
"bin": {
"agent-browser-stealth": "./bin/agent-browser.js",
"agent-browser": "./bin/agent-browser.js",
"abs": "./bin/agent-browser.js"
"agent-browser-stealth": "bin/agent-browser.js",
"agent-browser": "bin/agent-browser.js",
"abs": "bin/agent-browser.js"
},
"scripts": {
"prepare": "husky",
+7 -11148
View File
File diff suppressed because it is too large Load Diff