The srcdoc-iframe contentWindow Proxy returned native window methods
unbound, so iframe.contentWindow.getComputedStyle()/addEventListener()/
setTimeout() ran with the Proxy as `this` and threw "Illegal invocation"
on any page that uses a srcdoc iframe under --launch (FullLaunch). The
sibling matchMedia proxy already bound its methods; this one did not.
Wrap each function in an apply/construct trap that swaps the Proxy
receiver for the real window while passing .prototype/.name/.toString/
identity straight through (a plain .bind() drops .prototype and breaks
instanceof/constructors). Cached in a WeakMap for stable identity.
Verified before/after on a launched stealth browser: getComputedStyle,
addEventListener, setTimeout all OK; .prototype preserved.
Local stealth verification with no external detector: reports mode (connect vs
launch), live fingerprint probes (navigator.webdriver / window.chrome / plugins /
UA-headless) as pass/fail, and an audit of the active overrides for the path
(incl. the iframe-proxy state from #4). `--json` for a stable shape agents can
gate a sensitive flow on. Distinct from `doctor` (install/env health).
--launch mode scored ~20% stealth on CreepJS because the srcdoc-iframe
contentWindow Proxy trips `hasIframeProxy` — the proxy that hides automation is
itself a fingerprintable tell (violates this fork's own "native > JS lies" rule).
Add a config-driven opt-out (no detectable global): AGENT_BROWSER_DISABLE_IFRAME_PROXY=1
drops the patch via __abStealth.disableIframeProxy → the iframe IIFE early-returns
→ clean 0% CreepJS, trading the niche srcdoc-iframe masking. Default keeps current
behavior. README now documents the --launch 20% honestly and scopes the headline
0% to the extension-connect path. Verified: launch + srcdoc page intact with the
toggle; stealth tests green (config strip-prefix kept in sync).
- snapshot -c (compact) now always keeps lines with an interactive ARIA role
(button/link/textbox/combobox/option/…), not only `ref=`/`": "` lines — so a
clickable control can't vanish from compact output and leave the agent clicking
an empty ref (issue #2 P1). Additive: only ever keeps more. compact tests green.
- stale-ref error now leads with "take a fresh snapshot" and points to the `eval`
fallback for ref-churning SPAs, and demotes AGENT_BROWSER_VERIFY_REF=0 to a
flagged last resort instead of presenting it as the fix (issue #3 P1).
The session's active tab was a bare index into `pages`, which drifts when a
foreign/user/other-session tab is passively discovered, a tab closes, or the list
reorders — so `eval`/`screenshot`/`snapshot`/`click` could land on the wrong page.
With login state that's a safety bug (a fetch firing on the wrong origin), and it
made screenshot disagree with snapshot/eval.
Pin the intended tab by stable target_id (`active_target_id`), set on every
explicit open / tab new / tab switch / connect. `active_session_id` and
`active_target_id` resolve through it (falling back to the index only if the
pinned tab is gone), so all commands stick to the agent's tab regardless of
passive churn — and they all agree.
Verified (--cdp, multi-tab): a window.open foreign tab no longer drifts eval;
tab new / switch re-pin correctly.
The biggest manual-cost point in the dogfood reports: `select @ref` is a silent
no-op on non-native dropdowns, and click+wait+Enter on react-select/ARIA/portal
menus took ~20 turns of hand-written eval to get right.
New `pick <selector> --option "<text>"` does it atomically in one in-page async
routine: native <select> → set value + input/change; custom widget → focus +
open (pointer/mouse sequence), poll up to 2.5s for the option to render anywhere
(portals included), match by visible text, scroll it in, fire the full
pointer/mouse sequence. ERRORS loudly if the option never appears — no silent
success.
Verified headless: native <select> → "Gamma"; portal combobox → "欧洲"
(non-ASCII); missing option → explicit error. Documented in the skill.
- eval now prints `eval @ <url>` to stderr (stdout stays the raw value) so an
agent can catch tab drift — e.g. a logged-in fetch that hit the wrong origin —
before trusting the result. Mitigates the issue #2/#3 P0 safety concern. (eval
already returned the origin; the default output just never surfaced it.)
- `type --focused <text>`: type into the currently-focused element with no
selector, for custom widgets that move focus to a hidden input (issue #2 P3).
- AGENT_BROWSER_HUMANIZE set to an unrecognized value now warns once (like the
--humanize flag) instead of being silently ignored (Hermes #3).
A coordinate click resolved from a CSS selector (incl. the getByText/find path's
located node) skipped the occlusion check that @ref clicks already get, so an
overlay on top made the click land on the overlay while still reporting ✓ Done —
the worst failure mode for an agent (Hermes #1, issue #2/#3). Now: if the click
point doesn't hit the target (elementFromPoint isn't the element / a descendant /
an ancestor wrapper), dispatch through the DOM instead, which fires the real
handler. Best-effort probe (a flaky check never blocks the normal path); skipped
for strict CLICK_MODE=coord and non-left/multi-clicks.
Verified: occluded button click hits 0→1 (was silent ✓Done); normal click
unaffected.
Root cause behind Hermes #1 (CLICK_MODE=dom "does nothing") and #2 (--humanize
"does nothing"): both are env vars the daemon reads, but the daemon's env is
frozen at spawn — set them on a command to an already-running daemon and they
were silently ignored. (Confirmed: setting CLICK_MODE=dom at daemon spawn made
dom_click fire; setting it later did not.)
Fix: the client forwards AGENT_BROWSER_CLICK_MODE / AGENT_BROWSER_HUMANIZE in the
command envelope (_clickMode/_humanize); execute_command applies them per command
— mirrors CLICK_MODE into the process env (interaction::click reads it fresh) and
sets the humanize session level. Each command is authoritative.
Verified on an already-running daemon: CLICK_MODE=dom now fires dom_click
(hits 0→1); --humanize human typing applies.
When connected to the user's real Chrome, mgr.close() disconnected but never
closed the tabs the session opened — so every session (especially one that
failed before calling close, or a forgotten one) left its tabs piling up in the
user's browser. Idle-timeout and shutdown have the same exit path.
Track the target_ids this session creates via Target.createTarget in
`created_targets` (only ever our own tabs — never the user's existing tabs, which
the raw-CDP path attaches to, nor other sessions'). On close(), for the connected
path (not a launched browser, which Browser.close handles wholesale), close each
of those targets — the extension maps Target.closeTarget → chrome.tabs.remove.
Verified against a throwaway --cdp Chrome: open + 2 `tab new` → 3 pages; `close`
→ back to 1 (our 2 closed, the pre-existing tab untouched).
fork.40 errored after 5s ("reload the extension"), which still pushed the problem
onto the user. Extend the relay-reconnect wait to ~15s when the extension is
installed: enough for the MV3 service worker to wake and reconnect on its own
(onStartup after a Chrome restart, or the keepalive alarm). The loop re-checks
the relay file each iteration, so a mid-wait recovery is picked up instantly and
the full window is only spent when the extension is genuinely down. End users no
longer have to do anything when the relay blips.
Root cause of the recurring "Allow remote debugging?" dialog: when the ab-connect
relay was momentarily down (MV3 service worker drops the relay-url file across a
Chrome restart / idle wake), auto_connect_cdp silently fell through to the raw
:9222 DevToolsActivePort path — which pops Chrome 136+'s consent modal, the exact
thing the extension exists to avoid. Even a relay-aware build hit this if it
connected during the blip.
Fix: if the native-messaging host is installed (connect::host_installed() — the
durable signal that the user chose the extension path), auto_connect retries the
relay for ~5s while the SW reconnects, and then ERRORS with an actionable message
instead of attaching to a raw debug port. The raw :9222 path now runs only when
no extension is set up (where the dialog is expected). `--cdp <port>` still forces
the raw path explicitly.
Driving the user's real Chrome should not yank their view around. Now the agent
operates entirely in the background:
- New tabs are created with `background: true` (CreateTargetParams) so opening
one never foregrounds it (the ab-connect extension already used active:false;
this covers the raw-CDP path too).
- Dropped the two AUTO `Page.bringToFront` calls (auto-connect fresh tab, and the
internal active-page switch). The explicit `bringToFront` command is untouched —
surfacing a tab stays opt-in.
- enable_domains now sets `Emulation.setFocusEmulationEnabled(true)` so a
backgrounded agent tab still renders (screenshots work), isn't render-throttled,
and reports document.hasFocus()/visibilityState='visible' — which also removes
the "tab is hidden the whole session" bot tell.
Verified headless: hasFocus=true/visible while backgrounded; click + screenshot
still work. Default behaviour, no flag.
Completes the humanize suite:
- Clicks land on a jittered point inside the element's box (Fast/Human) instead
of its exact centre. `resolve_element_center` now also returns the element
width/height (box_model_dims); the CSS-selector path reports zero size → land
on centre (no jitter, no regression). Jitter is clamped to the inner box so the
click never misses.
- Wheel scrolls split into eased, jittered segments (humanize::scroll_segments,
unit-tested) instead of one instant jump.
- Drag follows the curved trajectory at Fast/Human (linear 10-step at Off).
Off is unchanged throughout. 9/9 unit tests; verified headless — jittered click
still lands (→ iana.org), segmented scroll moves the page.
- Typing: type_text_into_active_context now uses variable, human-like
inter-keystroke gaps from humanize::keystroke_delays when no explicit --delay
is given (Fast/Human); Off stays instant. Explicit --delay still wins.
- CLI: `--humanize off|fast|human` surfaces AGENT_BROWSER_HUMANIZE so the
session's daemon (a child that inherits this env) applies it, overriding the
adaptive detector. Invalid values warn and are ignored.
Verified headless: `--humanize human` + type lands "hello world" correctly.
Deferred: in-bbox landing jitter (helper ready, needs bbox threaded) + wheel/drag
easing.
After each navigation, probe the loaded page for known behavioural anti-bot
vendor fingerprints — cookies (_abck/Akamai, _px/PerimeterX, datadome,
reese84/Imperva, …), script URLs, and window globals — and escalate this
session to HumanizeLevel::Human when one is present, else fall back to the Off
baseline. So ordinary sites run at full speed (instant clicks) and only pages
actually guarded by behavioural detection pay for human-like motion.
`AGENT_BROWSER_HUMANIZE` still forces a fixed level and short-circuits the probe.
Best-effort: a failed probe leaves the level unchanged. Verified end-to-end
(headless --launch): a HUMANIZE=human click on example.com traverses the curved
trajectory and lands correctly (→ iana.org), identical outcome to Off.
Behavioural stealth: a click that teleports the cursor to an element's exact
centre with no approach path and zero press/release delay is a tell that
advanced anti-bot vendors (Akamai/PerimeterX/DataDome) flag, even though our CDP
events are isTrusted.
New `native::humanize` module — pure, unit-tested motion maths (cubic-Bézier
eased trajectories, in-bounds landing jitter, variable keystroke cadence, and an
anti-bot vendor detector) plus a small daemon-wide runtime (current level + last
cursor + per-action seed). `dispatch_click` now moves along a curved,
decelerating path from the last cursor position and dwells before releasing.
Three levels off|fast|human. Default is Off → byte-for-byte the old teleport, so
nothing changes until opted in. `AGENT_BROWSER_HUMANIZE=human` forces it now;
the adaptive per-navigation detector (set_detected_level) and type/wheel/drag
coverage land next. 8/8 unit tests; fmt + clippy clean.
e2e_save_state_cross_domain navigated to httpbin.org as "domain A", which is an
unreliable external service — when it was slow/unreachable in CI the page didn't
load on that origin, so its localStorage origin was missing from the saved state
and the test failed intermittently. Cookies/localStorage are set client-side via
CDP, so the page just needs to load reliably: use example.org (IANA-reserved,
like example.com) instead. Match full hostnames so the two example.* origins
don't alias. Verified locally: passes deterministically.
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
- invalid CSS selector now errors "Invalid selector '<sel>': <reason>" instead of
the misleading "Element not found" — the coordinate path (resolve_by_selector)
now also inspects exception_details, matching resolve_element_object_id.
- `wait --url ""` is rejected at parse time ("needs a non-empty pattern") rather
than silently matching any URL. Unit test added.
Not changed: verb-less `find role X` defaulting to a click. That default is a
deliberate, tested decision (test_find_role_default_subaction_click_when_no_action);
changing it to locate-and-report is a design choice left to the maintainer.
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
- wait --url: the arg parser never read `--timeout`, so a non-matching pattern
waited the large default and wedged the daemon. Parse it. Also: matching was a
literal substring (`includes`) so globs never matched — convert `**`/`*`/`?`
globs to an anchored regex. And `poll_until_true` now bounds each probe with a
timeout and tolerates transient navigation errors, so a hung `Runtime.evaluate`
can never block past the deadline (un-wedges the daemon).
- find role <role> [--name]: the query was `[role="X"], X`, which matches a
literal <X> tag / explicit attribute but NOT implicit-role elements — so
`find role link` (<a href>) and `find role heading` (<h1>) never matched. Add a
proper ARIA-role → implicit-element map and broaden accessible-name matching
(aria-label/title/alt/value/text).
- click on a syntactically-invalid selector returned `✓ Done`: querySelector
throws, and Runtime.evaluate returned the thrown DOMException as an objectId
that was clicked as if it were the element. Check exception_details → error.
- output: a title-less page now prints `✓ <url>` instead of an empty title line.
- docs(skill): tab refs are `t2`, not `2` (SKILL.md, electron).
Verified live (isolated launch): wait --url glob matches instantly; non-matching
honors --timeout (2s) and leaves the daemon responsive; find role link/heading
match; invalid selector errors. Unit tests added for the glob + role map + parse.
The fork's CI had never been green. Pre-existing failures:
- version-sync: check-version-sync.js read packages/dashboard/package.json,
which doesn't exist in this fork (workspace is just "."). Drop the dashboard
comparison; check package.json vs cli/Cargo.toml only.
- Dashboard job: `pnpm install --filter dashboard` for a non-existent package.
Remove the job.
- Format check: repo was never `cargo fmt`-clean. Ran cargo fmt (mechanical).
- Clippy -D warnings (newly enforced on Rust 1.94 stable): manual_contains in
commands.rs (.iter().any()->.contains()), question_mark in element.rs
(if-let-Err -> ?), result_large_err on the tungstenite handshake callback in
connect.rs (allow — the Result type is fixed by the accept_hdr_async contract).
- rust-cross: lightpanda::waits_for_ready_without_logs spawns a real process +
binds a socket with timing assumptions; flaky in CI. Marked #[ignore].
Also: skill docs note fork.30's relay-preferred auto-connect (plain
`agent-browser open` is dialog-free once the ab-connect extension is loaded) and
the extension's new "agent-browser-stealth" display name.
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
rebrowser's navigatorWebdriver probe checks Object.getOwnPropertyNames(navigator)
== [] (real Chrome keeps navigator members on Navigator.prototype). The launch-mode
stealth script defined language/languages/userAgentData/contacts as instance
own-properties, leaking them as an automation tell.
- add __abRedefineNavProto(name, getterImpl): redefines a navigator member on the
PROTOTYPE with a native-masked getter toString, then deletes any instance shadow
(mirrors the existing vendor patch). Falls back to instance only if proto is locked.
- convert language/languages/userAgentData to it; make the contacts block prototype-first.
After: Object.getOwnPropertyNames(navigator) == [], values intact, getters native,
rebrowser navigatorWebdriver 🟢, runtimeEnableLeak/pwInitScripts 🟢, sannysoft 0 fails.
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
- stealth(plugins): stop overwriting real native navigator.plugins in headed
mode (the JS fake had a non-native item(), broken uint32 wrap → incolumitas
overflowTest FAIL, and an anachronistic Native Client plugin). Leave native
plugins untouched when present; modernize the headless-escape fallback to the
real 5 PDF-viewer set with masked-native item()/namedItem().
- connect: auto_connect_cdp() now prefers the dialog-free ab-connect relay over
the raw :9222 CDP port, so Chrome 136+'s "Allow remote debugging?" consent
modal no longer fires when the extension relay is live. Gated by a bare-TCP
relay_is_live() probe (+3 unit tests).
- extension: rename ab-connect to "agent-browser-stealth" + new stealth icon set
(16/32/48/128).
- docs(README): hero/shield/fingerprint images, expanded detector results
(CreepJS 0% stealth, incolumitas all-OK, BrowserScan CDP-clean), and a
"Verify it yourself" section. .gitignore: allow assets/ + extension icons.
Headless Chrome is a bot-detection tell: creepjs scores ~33% headless even with
--headless=new, while a headed window with a real GPU scores 0%. Since this is a
stealth fork, headless is now forbidden — build_chrome_args ignores the headless
LaunchOption and never emits --headless/--enable-unsafe-swiftshader/forced
--window-size. The only escape is AGENT_BROWSER_ALLOW_HEADLESS=1 for genuinely
display-less servers (discouraged — forfeits stealth).
Verified locally: default launch (no env) is headed (webdriver=false,
platform=MacIntel, no --headless flag); creepjs headed = 0% headless vs 33%
headless. chrome.rs: 48 tests pass incl. forbids-headless + escape.
platform_string() feeds the CDP Emulation.setUserAgentOverride 'platform' field,
which sets the LEGACY navigator.platform. It was returning the UA-CH form
("macOS"/"Linux") — but real Chrome reports navigator.platform = "MacIntel" on
macOS and "Linux x86_64" on Linux. "macOS" contradicts the UA's "Intel Mac OS X"
and is a trivial bot-detection tell (platform vs UA mismatch). UA-CH
(navigator.userAgentData.platform via platform_hint) stays "macOS"/"Windows"/
"Linux" — that form is correct there.
Verified locally on bot.sannysoft.com (all rows green incl. navigator.platform=
MacIntel) + eval probes: webdriver false, no Headless in UA, real WebGL
(Apple M3 Metal, not SwiftShader), plugins/permissions consistent.
ROOT CAUSE of per-session command drift on the extension path: the daemon's
liveness check (`is_connection_alive` → `Browser.getVersion`) is a BROWSER-level
command. The relay only answered Target.* locally and forwarded the rest, so
Browser.getVersion went to the extension, which can only do per-tab
chrome.debugger → it errored → CdpClient saw TransportError → connection deemed
DEAD → the daemon closed + reconnected + re-ran discover_and_attach_targets on
EVERY command. Each re-discover rebuilds pages from the relay's minimal
targetInfo and resets active_page_index=0, so eval/get-title/screenshot drifted
to the first tab (about:blank / a foreign focused tab).
Reproduced locally (throwaway Chrome + Extensions.loadUnpacked + fork.24 nm-host):
trace showed discover_and_attach_targets running on every command (pages
before=0) and [ev] active_idx reset to 0.
Fix: relay answers Browser.getVersion locally with a stub version (like
getTargets), so the liveness probe succeeds → connection stays alive → no
reconnect/re-discover → the session's active tab is preserved. Pairs with
fork.24's add_background_page. relay.rs: 10 unit tests.
After connect+grouping worked, follow-up eval/get-title/screenshot drifted to a
foreign tab: on a shared browser, Target.targetCreated events for tabs the user
or OTHER agent sessions open stream in and are drained on every command. The
drain path routed them through add_page(), which sets active_page_index to the
new page — so the session's active tab silently jumped to a foreign tab and its
commands landed there.
Add BrowserManager::add_background_page() (push without touching active, dedup by
target_id) and use it in the event-drain path. Explicit opens (tab new, the
add-and-switch paths) keep using add_page() and still focus the new tab.
Closes the last gap in concurrent multi-agent: each session now drives its OWN
tab regardless of other sessions'/the user's tab activity.
After the connect fix, extension connect reached the relay but Target.getTargets
failed: 'missing field title'. The ab-connect relay builds targets from the
extension's synthesized Target.attachedToTarget; the re-announce path
(reannounceAttachedTabs) emits a minimal targetInfo {targetId,type,attached}
with no title/url, so strict deserialize of TargetInfo blew up the whole
getTargets response.
Make TargetInfo.title/url #[serde(default)] (empty) — tolerant of minimal CDP
targetInfo from the relay (and the occasional real-CDP omission). Titles
re-populate from Target.targetInfoChanged / page events after attach.
The nm-host fanned extension→client messages over a broadcast channel and
forwarded commands under the client's own id, so two sessions connected to one
relay collided: command replies went to every client and ids overlapped → the
2nd session's connect hung (EAGAIN after 30s×5) and responses cross-talked.
Now the relay demultiplexes:
- each forwarded command is re-keyed to a relay-global id mapped to (client,
original_id); the extension's reply routes back to ONLY that client with its
original id restored (relay.rs: pending map + ClientId)
- CDP events fan out to all clients (they ignore unknown sessions)
- nm-host keeps a client_id -> sender registry instead of a broadcast; clients
are unregistered + their pending dropped on disconnect
Unblocks concurrent multi-agent on one shared Chrome (each --session its own tab
group from fork.20). relay.rs: 9 unit tests incl. cross-client id isolation.
Shared browser, separate tab groups: when an agent drives the user's real Chrome
via ab-connect, every tab it opens lands in a Chrome tab group named after its
--session (stable color per name). Each agent's tabs stay visually separated from
other agents' and from the user's own (ungrouped) tabs. Visibility is NOT
restricted — all agents still see all tabs (per design).
- CreateTargetParams gains an optional non-CDP `agentGroup` hint (skip-if-none),
so a strict real-Chrome endpoint never receives it
- BrowserManager.agent_group(): Some(session) only when ws_url == the live
ab-connect relay URL (never on launched/direct CDP); DAEMON_SESSION set at
daemon start supplies the name; emitted at all createTarget sites (transient
storage target stays None)
- ab-connect: +tabGroups permission; Target.createTarget reads agentGroup and
groups the new tab (create/reuse by title, deterministic color), best-effort
- extension 0.3.0 -> 0.4.0; re-signed crx + zip (id unchanged)
Needs the v0.4.0 extension reloaded + a build with this change to take effect.
Pure, unit-tested core of the daemon-side relay that bridges the ab-connect
extension to the existing CdpClient. The extension exposes per-tab
chrome.debugger + synthesized Target events; CdpClient expects a browser-level
endpoint. So RelayState:
- answers Target.getTargets / attachToTarget / setDiscoverTargets LOCALLY from
targets learned via the extension's forwardCDPEvent(Target.attachedToTarget),
returning the extension's cb-tab-N sessionId (consumes those synth events
rather than double-forwarding them);
- forwards every other command as a forwardCDPCommand envelope (carrying
method/params/sessionId);
- maps forwardCDPCommand responses and forwardCDPEvent events back to raw CDP;
- validates the connect-handshake token; emits challenge/ping.
Keeps CdpClient and browser.rs unchanged. 8 unit tests; clippy clean. Still
inert — the tokio WS server + `connect` command wire it next.
Real-world dogfooding surfaced clicks that resolve a valid @ref but still miss:
- Scroll the target into view before computing click coordinates
(scrollIntoViewIfNeeded). Without it, an element below the fold — or revealed
after a scroll/popup — yields off-viewport coordinates and the click lands on
whatever occupies that screen point.
- Fall back to a DOM-dispatched `.click()` when the coordinate path fails (a
persistent floating layer failing the occlusion guard, or coordinates that
won't resolve). The DOM dispatch targets the intended element directly instead
of a screen point, so an overlay or portal can't divert it.
- AGENT_BROWSER_CLICK_MODE: "" (default: scroll + coordinate + DOM fallback),
"coord" (strict coordinate, hard-fail on occlusion), "dom" (always
element.click() — best for autocomplete/menu <li> that close on input blur).
Fallback is limited to left single-clicks (DOM .click() can't express
right/middle/double). Non-left/multi and "coord" mode keep the original error.
Docs: README knob table + skill commands.md gain CLICK_MODE, a click-reliability
note, and a "debug forms/hidden inputs with eval" section (snapshot doesn't show
hidden inputs — the fast path to bugs like a hidden point_choice=none).
6 click/interaction e2e green; full suite 760 passed.
- snapshot: make collect_fingerprints private (TreeNode is private, so a
pub(super) fn leaked a more-private type)
- adaptive: if-let instead of single-arm match in attr_score
- stealth: move timezone test module to end of file (items-after-test-module)
No behavior change. Pre-release cleanup.
Borrow Scrapling's adaptive element finding, adapted to this project's
in-session AX-ref model. When a saved @ref's node is gone (or its identity
no longer matches) and the role/name/nth re-query also fails, score the
current page's candidate elements against an AX fingerprint captured at
snapshot time and relocate to the best match.
- New `adaptive` module: pure, browser-free scoring (role, accessible name
via Levenshtein, AX properties, ancestor-role LCS, parent/sibling) plus
pick_best with a high absolute threshold (0.70) AND a clear margin (0.15)
over the runner-up — so ambiguous twins are refused rather than mis-clicked,
matching the existing "fail loudly over wrong click" posture.
- Fingerprint captured during the existing AX-tree snapshot walk — no extra
CDP round-trips. TreeNode is AX-only (no DOM tag/attrs), so we use AX role
as the type and a few discriminating AX properties (value/url/level/checked);
DOM id/class would have cost an N×describeNode storm per snapshot.
- Wired into both resolve_element_center and resolve_element_object_id: on a
verify-identity mismatch or a stale-node fallback miss, relocation is tried
before erroring. A confident match overrides the identity guard; otherwise
the original error is surfaced. Opt out with AGENT_BROWSER_ADAPTIVE_REF=0.
README documents the new tuning knobs. Adds 9 unit tests; full suite 760 passed.
Borrow anti-detection hardening from Scrapling/patchright, preferring native
CDP/Chrome overrides over JS lies:
- Runtime.enable is now opt-in via AGENT_BROWSER_CAPTURE_CONSOLE (default off).
It was called on every session INCLUDING CdpAttach (the user's real Chrome),
leaking the patchright/rebrowser "runtime" CDP signal and undermining the
"real browser, no lies" guarantee. Runtime.evaluate/callFunctionOn and
runIfWaitingForDebugger work without it; only console/error capture needs it.
The console/errors commands now return a hint when capture is disabled.
- Timezone alignment via native Emulation.setTimezoneOverride, opt-in with
AGENT_BROWSER_TIMEZONE=<IANA>|auto (FullLaunch only). Intl and Date both
follow with no JS artifact.
- WebRTC IP-leak handling via the --force-webrtc-ip-handling-policy Chrome
flag: auto disable_non_proxied_udp when a proxy is set (so the real IP can't
leak past the proxy); AGENT_BROWSER_BLOCK_WEBRTC=1 hides the local IP when
there is no proxy; =0 opts out.
- Opt-in canvas/audio fingerprint noise via AGENT_BROWSER_HIDE_CANVAS=1
(FullLaunch only). Session-stable seed so reads stay consistent within a
session while differing from the headless-stable hash.
Adds 5 unit tests; full suite 751 passed, 0 failed.
Addresses the footgun raised in issue #1 follow-up: plain `--launch` silently
uses a temporary EMPTY profile (no cookies/login), and the connect-failure
error even recommended it — trapping agents into thinking they reused the
logged-in browser when they didn't.
- `--profile auto`: resolves to the Chrome profile last used (from Local State
`profile.last_used`), falling back to "Default", then the first profile. So
`--launch --profile auto open <url>` reuses real login state without naming
the profile. (--profile <name>/Default already worked.)
- connect-failure error now recommends `--launch --profile auto` and states
plainly that bare `--launch` is a temporary EMPTY profile — no cookies/login.
- bare `--launch` (no --profile, not CI) now prints a warning to that effect.
- README: fix Setup (relaunch with --remote-debugging-port, not chrome://inspect)
and split Standalone mode into throwaway vs. keep-your-login (`--profile auto`).
Tests: resolve_chrome_profile("auto") prefers last_used, falls back to Default.
handle_launch's fresh-launch path (the path `--launch open <url>` takes) never
called apply_stealth_to_browser — only the launch FLAGS were applied (e.g.
--disable-blink-features=AutomationControlled, which is why navigator.webdriver
was already false). As a result the 32 JS fingerprint patches and the
Emulation.setUserAgentOverride HeadlessChrome→Chrome UA strip NEVER ran on a
launched browser: navigator.userAgent kept the HeadlessChrome marker (a
longstanding bug — identical on the prior prebuilt binary).
Add the apply_stealth_to_browser call after launch (the auto_launch path
already had it; only the explicit-launch path was missing it).
Verified, FullLaunch headless:
- navigator.webdriver === false, navigator.userAgent => Chrome/<v> (no Headless)
- new tabs and the initial page both clean
- bot.sannysoft.com: 0 failed / 31 passed
The webdriver patch deleted navigator.webdriver, leaving it `undefined`. Real
Chrome reports `false`, so `undefined` is itself a detection tell, and deleting
it also removes the native `false` that Emulation.setAutomationOverride sets.
Now we rely on setAutomationOverride for a native (undetectable) `false` and
only force `false` via a getter as a fallback when webdriver is still `true`
(older Chrome without that override) — never delete it. Verified: FullLaunch
headless now reports navigator.webdriver === false (boolean), consistently.
- output: a success response with no data payload now prints "Done" instead of
nothing (a silent exit 0 looked like a no-op).
- commands: add top-level aliases for `get` status reads — `url`, `cdp-url`
(and `cdp_url`), `title`, `html`, `text`, `value`, `count`, `box`, `styles`,
`attr` — so `agent-browser url` no longer errors "Unknown command".
- connect errors now explain the Chrome 136+ realities:
- connect-failure mentions the "Allow remote debugging?" consent dialog and
that remote debugging is a startup flag, not a setting.
- no-Chrome error tells the user to relaunch Chrome with
--remote-debugging-port (auto-connect then works).
- --cdp discovery failure explains Chrome 136+ dropped the HTTP discovery
endpoints and to use the default auto-connect instead.
Stealth coverage (the fork's core value was leaking on secondary surfaces):
- stealth scripts are registered per CDP session, so new tabs (`tab new`) and
cross-origin iframe sessions created after the initial page had NO patches.
Extract apply_stealth_via_mgr/apply_stealth_to_session and re-apply on
tab_new and on iframe attach. Fixes automation markers (and FullLaunch UA)
leaking in new tabs / cross-origin frames.
Resource hygiene (temp profiles filled the disk):
- ChromeProcess::drop already cleans the temp user-data-dir on normal exit, but
a hard kill (kill -9 / version-mismatch restart / crash) skips Drop and leaks
~50MB per session. Add cleanup_orphaned_chrome_profiles() on daemon startup
that sweeps agent-browser-chrome-* temp dirs NOT referenced by any live
process (so an in-use profile is never deleted).
auto-connect resolved the DevToolsActivePort URL by first opening a
verification WebSocket (verify_ws_endpoint: connect, Browser.getVersion,
close) and only then opening the real connection. On Chrome 136+ the
"Allow remote debugging?" consent is granted per-connection, so the user's
single Allow click was consumed by the throwaway probe and the real
connection (opened afterwards) asked again — surfacing as repeated prompts
or a hung command after the user had already clicked Allow.
resolve_cdp_from_active_port now gates the direct DevToolsActivePort URL on
a consent-free TCP liveness check (tcp_port_alive) instead of a WebSocket
probe, so the real connection is the single WebSocket the user consents to.
A bare TCP connect does not trigger the consent flow (that fires on the CDP
upgrade), and the real connect_async has no client-side timeout, so it waits
for the user to click Allow at their own pace. verify_ws_endpoint removed;
discovery-order tests updated, plus a guard test that resolution opens no
WebSocket.
Verified live: single prompt on a real Chrome attach, then open + eval +
scroll x2 + eval with zero re-prompts and no freeze.
The daemon re-validates the CDP connection before every browsing command via
is_connection_alive() (Browser.getVersion, 3s timeout). It treated any
timeout-or-error as "dead" and tore the connection down + reconnected.
For an externally-attached browser (the stealth fork's default — the user's
real Chrome), a timed-out probe is almost always Chrome being briefly busy or
showing the Chrome 136+ "Allow remote debugging?" consent modal, which blocks
CDP responses until the user clicks Allow. Tearing the already-consented
connection down forces a reconnect that re-pops the consent prompt — repeated
on every command this becomes an endless prompt loop, and the close +
multiple new /devtools/browser WS probes storm Chrome into a freeze.
Fix: distinguish the probe outcome.
- Responded -> alive
- TransportError -> dead (WS closed/reset; user closing Chrome lands here too,
so zombie-socket detection is preserved)
- TimedOut -> alive for an external attach (don't tear down a consented
connection on transient slowness); dead for a browser we
launched ourselves (a real hang worth reconnecting, and no
consent modal in play).
Extracted the verdict into a pure connection_alive_from_probe() with unit
tests covering all outcomes. No behavior change for locally-launched browsers.
Cherry-picks upstream agent-browser #1396. Adds a configurable
--hide-scrollbars flag (AGENT_BROWSER_HIDE_SCROLLBARS env, hideScrollbars
config key, default true) that appends Chrome's --hide-scrollbars launch arg
for headless (non-extension) launches so native scrollbars aren't painted into
screenshots. Plumbed through flags.rs, connection.rs, main.rs, native/actions.rs
and native/cdp/chrome.rs; help text in output.rs + skill-data.
Fork adaptation:
- the arg lands in the headless && !has_extensions block, separate from the
stealth base args — no interaction with anti-detection.
- dropped upstream docs/, agent-browser.schema.json and README hunks (removed
or rewritten in this fork).
Verified: cargo check --tests passes.
fork.7 caught the X mask-overlay race correctly but reported it to
the user verbatim — every transient overlay (modal backdrop, focus
ring, click-outside mask, sticky banner) became an error the user
had to wrap in their own retry loop. Most of these clear within a
frame or two on their own.
Now `verify_click_target` retries the elementFromPoint probe a few
times (default 3 × 200ms = 600ms total grace period) before failing.
Real-world overlays that blink in for a render cycle clear during
the first retry; persistent overlays still surface as errors with
the same actionable message — just qualified with "still occluded
after N retries / Mms" so the user knows we tried.
Tunable:
AGENT_BROWSER_OCCLUSION_RETRIES (default 3, 0 disables)
AGENT_BROWSER_OCCLUSION_RETRY_DELAY_MS (default 200)
DOM.resolveNode is called once outside the loop — backendNodeId is
stable across renders, only the element under (x, y) changes when
overlays flicker. Each probe is still capped at 500ms so a stuck
Runtime.callFunctionOn can't stall a click for longer than the user
expects.
Closes the "modal silently closes when clicking 'Add post' on a thread"
bug. Verified root cause via instrumented page-side click logger:
click @e31 (aria-label="Add post" at button (1034, 285))
→ mouse event dispatched to (1045, 296)
→ document.elementFromPoint(1045, 296) returned:
DIV[testid="mask"], bounds (0,0,1746x934)
→ X interpreted as "click outside modal" → close + nav to /home
The cached coordinates were correct. Between snapshot and click, X
laid a transient full-viewport mask over the modal (their own
"click-outside-to-close" overlay). stealth dispatched the click
without checking what was actually at that pixel — the overlay
intercepted it.
Fix: just before returning (x, y) from resolve_element_center for
ref-based interactions, run a Runtime.callFunctionOn against the
ref's resolved element with `function(x, y) { return this.contains(
document.elementFromPoint(x, y)) || that.contains(this) ? null :
{...occluder details...}; }`. If the element at the point isn't us
(or our descendant — clicking the SVG icon inside a button is fine
— or our ancestor), we fail with a specific message:
Ref @e31 is occluded by DIV[testid=mask] at the click point.
A transient overlay (modal backdrop, mask, sticky banner, etc.)
appeared between snapshot and click. Wait for it to clear or
re-snapshot, then retry.
So instead of silently submitting an entire thread or nuking the
user's modal, agent gets a parseable error and can wait + retry.
Tight 500ms timeout per CDP call (matching the verify_ref_identity
defensive guard from fork.6) so a stuck DOM.resolveNode can't
re-introduce the multi-minute hang we just fixed. On any timeout
or error in the guard itself, fall through and let the click
proceed — strictly no worse than the unguarded code path.
Disable with AGENT_BROWSER_VERIFY_CLICK_TARGET=0.
Reported: a single `click @ref` could hang 5+ minutes, with multiple
queued click invocations adding up to 7+ minutes — worst case 30s
timeout × 3 CDP calls × N parallel processes:
- verify_ref_identity (Accessibility.getPartialAXTree) → default 30s
- resolveNode / getBoxModel → default 30s
- wait_for_paint_settled (Runtime.evaluate awaitPromise) → default 30s
The latter two are best-effort defenses added in fork.3-5 to fix SPA
race / DOM-reuse bugs. They should never block a real click for
30s — the unguarded code path was always faster than the guarded
path-that-hangs.
- verify_ref_identity capped at 1s (skips check on timeout)
- wait_for_paint_settled capped at 500ms (skips wait on timeout)
Both skip-on-timeout intentionally: the worst case is the click
behaves like fork.2 (race-prone but fast), which is strictly better
than the user pkilling stuck processes.
Also rewrites the misleading "Chrome 144+ chrome://inspect tip" in
the auto-connect failure message — the toggle exposes target
discovery only, not the /json/version HTTP API the auto-connect
flow expects (verified by user: lsof shows :9222 listening but
curl /json/version returns 404).
Two changes that pair with each other:
1. connect_auto_with_fresh_tab now does a Runtime.evaluate "1"
round-trip after creating the fresh tab. This catches the zombie
CDP socket case (process alive, websocket dead) where every step
up to that point reports success but the next user command would
silently no-op against a dead session. Failing here lets the
caller surface a proper "CDP session unresponsive" error instead
of returning Ok and letting `agent-browser open URL` exit 0 with
a still-blank tab.
2. handle_wait now recognizes @ref selectors (e.g. `wait @e8 --gone`).
It polls resolve_element_object_id, which already runs the
verify_ref_identity check from 007fd1b — so:
- `wait @e8` succeeds while the original element is
still mounted with its snapshot role+name
- `wait @e8 --gone` succeeds when the ref's identity changes
(modal closed, button re-textified, etc.)
This gives users the "assert modal still open" primitive that
prior versions could only approximate with screenshots.
Closes the "click @e20 hits the sibling element" bug. Real-world
example: snapshot shows @e20=[button "Add post"] next to
@e17=[button "Post all"]. By the time you click @e20, React has
re-rendered — and React often re-uses the same <button> DOM node
across renders, just updating its accessible name. The cached
backendNodeId still resolves to a real, well-positioned node, so
the click lands cleanly. It just lands on what is now the "Post all"
button, silently submitting the entire thread instead of adding a
draft row.
Before every ref-based interaction (click / fill / type / hover /
select / drag — anything routing through resolve_element_center or
resolve_element_object_id), call Accessibility.getPartialAXTree for
the cached backendNodeId and check role + name still match the
snapshot entry. On mismatch, abort with an error that names both
labels:
Ref @e20 no longer matches its snapshot. Was [button "Add post"],
now [button "Post all"].
...Take a fresh snapshot, then re-target.
If the node is gone (CDP fails / no AX node), we silently fall
through to the existing "find by role+name" recovery path, so this
guard never makes a working flow worse.
Adds one CDP roundtrip per ref interaction (~5–20ms). Disable with
AGENT_BROWSER_VERIFY_REF=0 if you control the page lifecycle and
need the latency back.
Closes a real-world race that broke X multi-tweet thread composition
(and similar SPA flows): clicking "Add post" returned immediately,
inserttext fired before React had committed the new textarea, the
keystroke landed on the dialog wrapper, and X interpreted the stray
input as a request to dismiss the modal.
After mouseReleased we now wait for two requestAnimationFrame ticks
plus a microtask boundary (~33ms at 60fps, bounded). That's enough
for React/Vue/Svelte to commit any state update scheduled by the
click handler. Errors during the wait are swallowed — a click never
fails because of post-processing.
Opt out for perf-sensitive scripts that don't drive SPA UIs:
AGENT_BROWSER_CLICK_WAIT_STABLE=0
leeguooooo