Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
Agents finish a task and just stop (never calling `close`), so daemons used to
run forever, leaving their per-session scratch tabs + tab group in the user's
Chrome. Two cases now handled:
- Default idle timeout (10 min; AGENT_BROWSER_IDLE_TIMEOUT_MS overrides, 0
disables). On idle the daemon close()s the tabs IT created → the empty tab
group is auto-removed by Chrome. Timer resets on every command, so active
sessions are untouched. Adopted user tabs are never owned, so never closed.
- `keep` — leave the ACTIVE tab for the user: unown it (exempt from
close/idle) + ask the extension to ungroup it (ABExt.ungroupTab → 0.4.12) so
it becomes a normal tab. Scratch gets cleaned, deliverable tabs stay.
Also fix two clippy violations in the concurrently-landed #47 viewport code
(manual char comparison + iter().any→contains) that were failing main's CI.
ext 0.4.12: handle ABExt.ungroupTab (chrome.tabs.ungroup). 870 tests pass.
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
v1.5.24 (d5cd9cd) shipped a commands.rs caller of `clear_viewport` but not the
browser.rs method it lives in (a concurrent in-progress #47 viewport/resize edit
was only partly staged), so main didn't compile and the format check failed.
Commit the matching browser.rs method + via_relay() helper and run cargo fmt.
Full tree builds; 863 tests pass.
canvas — chrome-use can now read canvas/WebGL apps (Figma, games, maps, charts,
drawing tools) that expose no DOM/refs:
- canvas list: enumerate <canvas> (backing+CSS size, visibility, toDataUrl/tainted)
- canvas capture [selector] [path]: save rendered pixels to PNG — toDataURL
(full backing-store resolution), with a CDP screenshot fallback for WebGL
without preserveDrawingBuffer or cross-origin-tainted canvases. --screenshot
forces the screenshot path. Gets the RENDER, not hidden source data.
Verified live: captured Figma's canvas at full 2522x1904 via toDataURL.
site — fix map_args losing the adapter's declared arg order: serde sorts @meta
keys alphabetically, so a 2-arg adapter like {projectId, path} mapped positionals
to {path, projectId} (swapped). Now parses declaration order from the raw @meta
text (Adapter.arg_order) + regression test. Affects any multi-arg adapter.
skill — core skill now documents `adopt <url|targetId>` (read a pre-existing tab,
the explicit way through strict isolation) and `canvas list`/`canvas capture` in
the canvas/WebGL section.
863 tests pass.
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
New `chrome-use adopt <url-substring|targetId>`: drive a tab the user (or
another session) already has open, with ZERO new tabs. After group-scoped
isolation (#40) a session can't see foreign tabs, so adopt adds an explicit,
opt-in path:
- Relay (relay.rs): `ABRelay.getAllTargets` returns every attached target
UNSCOPED (ignores group scoping), so the agent can find a specific tab by URL
or targetId. +1 unit test.
- Daemon (browser.rs): `collect_all_targets` (unscoped, falls back to scoped on
older relays) + `adopt_existing_target` — matches by exact targetId or
case-insensitive URL substring, attaches it (the relay re-tags it into the
adopter's group, so isolation holds), pins it; never creates a tab. On no
match it errors AND lists the open tabs it can see, rather than launching.
discover_and_attach_targets honors AGENT_BROWSER_ADOPT at first connect, so no
about:blank is ever created.
- CLI (main.rs): `adopt` sets the env, forces a fresh daemon, and rewrites into
`connect <relay-url>` (like `extension connect`) so the daemon attaches to the
user's real Chrome before parse_command.
Extension (ab-connect 0.4.11): `reannounceAttachedTabs` now re-sends each tab's
url/title (it previously sent neither) so the relay's target list stays matchable
by URL after the MV3 service worker reconnects — otherwise reannounced tabs show
a blank url and `adopt <url>` can't find them. Repacked upload zip + crx.
Mechanism verified live (enumerated all 11 of the user's open tabs incl. the
target). 862 tests pass.
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
Move multi-agent isolation from blunt daemon-side filtering to relay-side
group scoping, so a session can adopt new tabs again (follow-popup, OAuth
results, cross-session adopt-by-targetId) without ever seeing the user's or
another agent's tabs.
Relay (relay.rs): track client->group (announced via new local ABRelay.setGroup,
or the first createTarget's agentGroup) and target->group (created tabs tagged
from the createTarget reply; an explicit attachToTarget tags the target into the
adopter's group = #21; a pop-up inherits its opener's group via openerTargetId).
Target.getTargets returns ONLY the requesting client's group; a client that never
announced a group (older daemon) gets the full list — fully backward-compatible.
+5 unit tests.
Daemon (browser.rs): announce_group() on connect sets relay_scoped. Adoption in
discover/resync/adopt_newly_opened is re-enabled ONLY when relay_scoped; without
it (launch / real CDP / older relay that didn't answer the announce) the daemon
keeps strict daemon-side isolation. So this can't regress the 125/125 isolation.
Extension (ab-connect 0.4.10): synthesized Target.attachedToTarget targetInfo now
carries openerTargetId (pop-ups inherit opener's group) and abGroup (the tab-group
title, so the relay re-attributes existing tabs after ITS own restart, since
createTarget tagging won't re-run). tabScopeHints().
Back-compat verified live: new daemon + OLD relay -> announce fails ->
relay_scoped=false -> strict fallback, open/eval/url all work. The new extension
(publish to CWS, strip manifest key) activates follow-popup; relay+daemon ship now.
861 tests pass.
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
Dogfooding backlog from this session's embedded-form/editor work.
#41 fill on rich editors: detect CodeMirror 5 / Monaco / ProseMirror /
contenteditable and set via their own API or execCommand('insertText') so
beforeinput/input fire (a raw .value/textContent write no-op'd juejin's
CodeMirror and skipped React composers). Response echoes the `engine` used.
`fill <sel> --file <path>` / `--stdin` set large multiline text without
shell-escaping. `get value` now reads CodeMirror/Monaco/contenteditable too.
#42 screenshot --max-width/--max-height/--scale, plus a default 2000px
longest-edge cap (AGENT_BROWSER_SCREENSHOT_MAX_EDGE; 0 disables) so retina
full-page shots fit an agent's image reader and --scale 0.5 makes screenshot
px line up with click px. Annotated shots are never downscaled.
#43 `box @ref` (already a top-level alias of `get box`) now also returns
centerX/centerY/inViewport in CSS px — feed straight into `click x y` when a
ref-click no-ops (e.g. a button in a cross-origin iframe).
#44 no code change needed — disabled elements already list as
`button "Save" [disabled, ref=eN]`; the reporter's missing button was
DOM-gated on validity. Added a skill note: `find text` can't reach into a
cross-origin iframe — target those by snapshot @ref.
#45 core skill now distinguishes screenshot-to-locate (discouraged) from
screenshot-to-capture a reusable image asset via `screenshot [--clip] <file>`
(encouraged), so agents stop over-reading the prohibition.
#40 (group-scoped relay) stays deferred — needs an ab-connect extension change.
Verified live: fill --file round-trips multiline+CJK+backticks; contenteditable
engine=contenteditable + get value reads it back; box gives centerX/centerY/
inViewport; screenshot of retina example.com → 2000px; disabled button shows
[disabled]. 856 tests pass.
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
Make `site` trigger itself so an agent doesn't have to know adapters exist.
Auto-sync: the pack refreshes on first use and on a TTL (default 7d), both in
the `site` command path (blocking, fast) and as a non-blocking background task
on daemon startup — so ~/.chrome-use/sites/.index.json is always populated with
zero added latency. Tune via AGENT_BROWSER_SITES_TTL_DAYS; disable with
AGENT_BROWSER_SITES_NO_AUTO_UPDATE=1. `update` now writes .last_update + a
domain→adapters .index.json (read-only adapters ordered first).
Auto-suggest: `open`/`navigate`/`snapshot` onto a domain with adapters attaches
`siteAdapters: {domain, commands}` to the response; the CLI prints a
`💡 site adapters for <domain>` hint (stderr) and the field rides along in --json.
SKILL.md tells the agent to prefer the listed `site <name>/<cmd>` over scraping.
This keeps the 'never auto-disrupt user tabs' guarantee — it suggests, the agent
decides; nothing auto-runs on navigation.
site.rs: needs_refresh/adapters_for_domain/write_domain_index + timestamp/index
in update(). daemon.rs: background bootstrap. actions.rs: with_site_hint on
navigate + snapshot. output.rs: hint render. Verified live: open github.com →
hint leads with read-only github/issues; --json carries siteAdapters.
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
Add `chrome-use site` — run community bb-sites adapters over chrome-use's
stealth transport. An adapter is a per-command JS function that calls a
site's own JSON API from inside your logged-in tab (your cookies, same-origin
fetch, the site's modules) and returns clean JSON — no clicking/scraping.
- site update fetch the upstream bb-sites pack into ~/.chrome-use/sites
- site list list installed adapters (name/cmd)
- site info show an adapter's @meta (args, domain, capabilities)
- site <name>/<cmd> [args] navigate to its domain (reuse tab if already there) + eval, return JSON
chrome-use ships zero adapter code; `site update` fetches epiral/bb-sites at
runtime (like a package manager). Adapters remain their authors' property.
cli/src/site.rs (load/parse/build_eval/list/update/map_args + tests), wired
via commands.rs (parse), actions.rs (handle_site), main.rs (CLI dispatch).
Docs in README, README.zh, skill-data/core. Verified live: github/issues
returned 30 real issues as JSON over the relay.
#5: the CLI never pointed users at the issue tracker. `--help` and `--version`
now print the issues URL (https://github.com/leeguooooo/chrome-use/issues), so
agents hitting a rough edge know where to report it (the skill already nudges).
#1: navigating to a `data:` URL over the extension relay fails with a cryptic
`net::ERR_ABORTED` on an about:blank tab (chrome.debugger/chrome.tabs can't drive
a top-frame data: navigation; it works fine under --launch). Detect that case and
explain it — use a real http(s)://file:// URL or --launch — instead of leaking
the raw code. (Surfaced while stress-testing 5 concurrent agents.)
After strict isolation (v1.5.16) a session's tracked set is only its OWN tabs, so
in `adopt_newly_opened` (run after every click to follow a pop-up) EVERY foreign
tab looks "new" relative to the session's `before` set and got adopted — a
click-heavy flow on a busy shared Chrome pulled other agents'/the user's tabs
(github, Lark, iphone-use) into the session mid-flow.
A pop-up the agent itself opened can't be told apart from a foreign tab over the
relay (the synthesized targetInfo carries no opener/window/group), so on the relay
`adopt_newly_opened` now adopts nothing: the agent drives only tabs it explicitly
created; pop-ups (OAuth/login windows) are the user's. Launched browsers (every
tab ours) still follow pop-ups. Verified live: a fresh click-heavy relay session
stays clean (only its own tabs), and 5 concurrent agents churning tabs show zero
cross-agent drift.
Several agents (and other tools opening tabs) share one real Chrome via the
relay. Previously every session adopted ALL tabs from Target.getTargets, so
another agent's tab churn polluted the list, dropped the tab being driven, and
drifted commands onto the wrong page (the W-8BEN tax tab vanished mid-flow when a
concurrent iphone-use agent opened tabs).
A tab group belongs to exactly one agent. On the relay a session now tracks and
drives ONLY the tabs it created (its own colored group) plus pop-ups its own
clicks open — it never adopts the user's or other agents' tabs:
- discover_and_attach (relay): create the session's own tab and pin it; do not
adopt any existing foreign tab.
- resync_targets (relay): never adopt unknown targets; never prune the session's
tabs on a single getTargets snapshot (multi-agent churn / cross-process-nav
gaps routinely omit live tabs) — prune only after RELAY_PRUNE_MISSES
consecutive absent snapshots (debounced), pinned active always protected.
- adopt_newly_opened: a tab that appears right after this session's action is a
pop-up we opened — record it as owned.
Launched browsers (every tab ours) keep adopting all tabs. Adds debounced_prune_ids
+ unit tests for the churn tolerance.
A DOM `.click()` is isTrusted:false, which security-sensitive embedded forms
reject — Google Payments' enabled `保存` button silently no-op'd, so a
cross-origin payment/checkout/KYC form could be read, scrolled, and typed into
but never submitted. A coordinate click can't help either: getBoxModel for a
sub-frame node returns frame-local coords that don't compose the iframe offset,
so it lands wrong (verified — the same-origin probe came back isTrusted:false
via the coordinate fallback).
Fix: click on an in-iframe ref now focuses the element in its own frame session
and dispatches a real Enter (Space for checkbox-like roles) on the page session.
Chrome routes the key to the focused element across frames (same mechanism as
`type --focused`), and Enter/Space on a focused button/link/checkbox fires a
trusted click. Non-activatable roles fall back to DOM .click().
Adds e2e_iframe_button_click_is_trusted (+ fixture): an in-iframe button records
event.isTrusted into its own text; the test asserts the ref-click delivers
isTrusted:true.
replMode and awaitPromise are mutually exclusive in Chrome — under replMode a
returned promise serialises to {} instead of being awaited, which broke every
fetch/async eval (e2e_domain_filter, e2e_headers, e2e_react_tree all regressed).
Enable replMode only for synchronous scripts that declare a top-level let/const
(the #38 case); promise-returning scripts keep awaitPromise — restoring the
pre-#38 await behaviour while still fixing the let-redeclaration collision.
The centered-wheel default no-op'd on some pages (headless e2e_hover_scroll_press
regressed). Restore window.scrollBy for plain page scroll; the coordinate wheel
stays opt-in via --at/--frame for cross-origin iframe content.
#38: `chrome-use eval` now runs with Runtime.evaluate replMode (like the DevTools
console) — top-level `let`/`const` no longer throw "already been declared" across
successive evals (independent `eval` steps in a `test` suite collided in the
page's shared lexical scope), and top-level await is allowed. Main-world and
completion-value semantics are unchanged.
#37: core skill gains a hard rule — snapshot-first, never screenshot+coordinates
to locate form fields/buttons; `snapshot -i` now pierces cross-origin iframes and
lists their elements by @ref; screenshots are for visual checks only, and a
full-page retina screenshot often exceeds an image reader's limits.
Follow-up to #36 — make the whole interaction surface reach cross-origin OOPIFs
and stop coordinate events drifting onto the user's foreground tab over the relay.
- hover/dblclick/drag now DOM-dispatch over the relay or into an iframe (like
click already did): a coordinate Input event isn't confined to the target tab
on a busy real Chrome and can't map an OOPIF element's box to a top-viewport
point. drag does an HTML5 DnD in the element's frame; cross-frame drag errors
loudly instead of drifting.
- snapshot recurses iframes to MAX_IFRAME_DEPTH (3) instead of one level, so refs
inside nested payment/checkout widgets get a frame_id and resolve into the
right frame.
- relay tab adoption merges several Target.getTargets snapshots — a single flaky
relay snapshot was dropping live tabs (a driven tab vanished after restart).
- `type --key-events` (alias --keys) sends real per-character keyDown/keyUp
instead of Input.insertText, so autocomplete/combobox widgets that ignore the
insertText input event fire (Google address postal lookup; commits Angular
reactive forms so Save enables).
- SKILL: hard "snapshot-first, never default to screenshot+coordinates" rule;
snapshot -i pierces cross-origin iframes since v1.5.12; cross-origin iframe
driving guidance (#37).
#35: `open` auto-reattaches when the bound relay tab is gone — drops the dead
page, opens a fresh tab in the session's group, and navigates it, instead of
only `tab new` recovering.
#36: scroll and click now reach content inside cross-origin OOPIFs:
- scroll dispatches a real wheel at a viewport point (default center, --at x,y,
or --frame n) so it scrolls the iframe under the pointer, which
window.scrollBy on the top document silently no-ops on.
- over the extension relay, clicks always use DOM-dispatch instead of
coordinate Input events — a coordinate event isn't confined to the target tab
on a busy real Chrome (it drifted onto the foreground tab) and an OOPIF
element's box can't be mapped to a top-viewport point.
#33: some sites prepend runs of ZWJ/word-joiner/invisible-times/BOM to
document.title (badging/anti-scrape); left in, they polluted 'tab list', broke
text matching, and wrecked column alignment. sanitize_title() now strips
zero-width/bidi-format chars at every title ingestion point + get_title().
#34: 'screenshot <selector>' (element capture) already worked but was
undocumented; added 'screenshot --clip x,y,w,h' for an explicit pixel region
(CDP captureScreenshot clip), documented both in --help. Verified live.
chrome.debugger forbids DOM.setFileInputFiles, so 'upload' used to hard-fail on
the relay and push users to a --launch/direct-CDP session. Now it falls back to
reconstructing the File entirely in the page (Playwright/Cypress-style: build a
File from the bytes, assign input.files = dataTransfer.files, fire input/change;
for drop/paste composers like X, dispatch synthetic paste+drop with the
DataTransfer). The bytes are streamed in <1 MiB base64 chunks because the relay
tunnels CDP through native messaging (1 MiB/message cap) — a whole image as one
arg closed the channel. Verified live over the relay: an 809 KB PNG lands intact
on a file input with change firing. No more direct-CDP needed for uploads.
Passing a Cloudflare challenge mints an HttpOnly cf_clearance cookie bound to
IP+UA. 'chrome-use cf-status' (aliases cf/cloudflare-status/clearance) reports
whether the active page is currently a CF challenge and whether a still-valid
cf_clearance exists (read via CDP — HttpOnly is invisible to document.cookie),
plus CF_VERIFIED_DEVICE trust, and a recommendation: proceed (already cleared,
don't re-solve) / solve (challenge up, no clearance) / reissue (clearance present
but page still blocks → IP/UA drifted). Lets an agent avoid re-solving what it
already cleared — the persistence optimization. Pure helpers unit-tested; live
-verified on a real cf_clearance.
Driving a busy real Chrome via the relay, a single Target.getTargets call
occasionally returns a different window's tabs ('tab list hops windows'). resync
pruned every tracked page absent from that snapshot — including the agent's
explicitly-adopted (pinned) tab — after which active-target resolution fell back
to active_page_index and eval/click/snapshot drifted onto a foreign tab
(about:blank / chrome-extension:// / the user's page), breaking any 3+ step flow.
prunable_target_ids() now protects the pinned active target from snapshot-based
pruning; a genuine close still arrives as Target.targetDestroyed (event drain) and
removes it properly. Unit-tested.
The consent modal only appears on a raw remote-debugging attach or a browser we
launched with a debug port — never on the ab-connect extension relay. Append one
line per connection to ~/.chrome-use/connect-mode.log (relay | raw-port-attach |
launched | remote-ws), flagging 'CONSENT-MODAL-RISK' when a raw-port/launch path
runs while the relay was available. Lets us tell a code regression from Chrome's
own extension-debugger consent UX when the modal reappears. Best-effort, never
fails a connection. Verified: normal 'open' logs mode=relay (consent-free).
Some injected UI (browser-extension debug panels, web components) renders into a
CLOSED shadow root that eval/innerText cannot read. --pierce walks the CDP DOM
tree (DOM.getDocument depth:-1 pierce:true), which includes closed shadow roots
and child documents, and collects text nodes (skipping script/style/etc).
Review-safe: rides the per-tab debugger session already attached, no new Chrome
permission and no ab-connect/extension change — so it works in extension-relay
mode without touching the published extension. Verified live: a closed-shadow
panel that main-world eval reports HIDDEN is read in full via --pierce.
First slice of #30 (read extension/injected-panel content). Deeper extension
introspection (background SW / chrome.storage) stays a launch-mode / raw-CDP
concern, deliberately NOT done by expanding ab-connect's debugger powers.
- chrome-use sessions: top-level alias for the daemon inventory (the skill
advertises sessions, so it's a natural guess that used to error).
- Unknown commands now suggest the nearest valid one (Levenshtein + prefix
match), staying silent when nothing is close (e.g. 'clik' -> click,
'sesions' -> sessions, 'xyzzy' -> no suggestion).
- tab <id>: probe the switched session and show a warning indicator instead of
a green check when it isn't responding yet, so a switch onto a re-attaching
(churned-tabId) session no longer reports false success. The #24 targetId
recovery self-heals within ~6s, hence a warning rather than a hard error.
On listing/marketplace pages (Yahoo Auctions, Rakuten, Mercari shops) the
seller's description lives in a child frame or under a related-items sidebar,
so 'get text body' returned only header/nav boilerplate.
- get text --all-frames: aggregate visible text across every reachable frame.
Same-process child frames are read via Page.createIsolatedWorld; OOPIFs via
their auto-attached debugger session (iframe_sessions). Each non-top frame is
labelled with a '----- frame [kind] url -----' separator.
- get text --main: readability-lite — prefer the densest <main>/<article>
region over the whole body, dropping global header/nav/footer chrome.
- frames: enumerate frames (kind + url + per-frame text length) so an agent can
see where a page's text actually lives and pick the right read.
Verified live: inline srcdoc frame text aggregated through --all-frames; Yahoo
Auctions <main> (2881 chars) extracted via --main, stripping the Yahoo header.
#25 — fill() didn't fire the events framework inputs / site autocomplete need:
it set value directly (bypassing React's value-tracker) and typed via
Input.insertText, so controlled components and input/change/blur listeners (e.g.
Mercari's postal-code → 都道府県 lookup) never ran though the value showed. fill
now emulates a real edit: focus, set through the element's prototype value setter
(React _valueTracker registers), then dispatch input → input → change → blur/
focusout. SELECT and contenteditable handled too. type <sel> <text> remains for
per-keystroke sites. Verified live: an input wired with input/change/blur fired
'IICB' from one fill.
#26 (ergonomics):
- 'close <tab>' now closes just that tab and prints 'Tab [tN] closed'; bare
'close' still closes the browser. Previously 'close t12' ran a browser close
and alarmingly printed 'Browser closed'.
- new 'chrome-use current': prints the active tab's stable handle (tabId + CDP
targetId + url/title), refreshed live — so an agent holds the targetId (which
survives cross-process nav) instead of re-deriving 'which tab is live' from
'tabs' every step. The deeper tab-id churn is the #21/#23 stable-targetId story.
Tests cover fill events (live), close tab-vs-browser parse, and current.
A click on a target=_blank link / window.open opened a new tab, but the active
tab stayed put, so the post-click snapshot showed the OLD page — looking exactly
like the click failed. On the relay the new tab is discovered only via getTargets
(the relay doesn't push target events to the daemon), so it went unsurfaced.
handle_click now snapshots tracked targets before the click and, after, runs a
lightweight BrowserManager::adopt_newly_opened (one getTargets, attaches only the
new target — far cheaper than a full resync) to detect a freshly-opened tab. It's
reported as openedTab {tabId,url,title} in the response (and a '→ opened new tab
[tN] <url>' hint in text mode). Default keeps focus on the current tab (so
multi-tab flows aren't hijacked, per #7/#8.1); 'click <sel> --follow' switches to
the new tab. Verified live: clicking a _blank link prints
'→ opened new tab [t13] https://example.org/'.
Completes the #24 friction items (B/C/D shipped in 770708b).
Three CLI gaps surfaced driving a Mercari signup→checkout flow:
- #24-B (correctness): a bare label like 'click 購入手続きへ' was fed straight to
document.querySelector as CSS and failed as an invalid selector, even though
snapshot listed the button by that exact name. build_find_element_js now tries
CSS first, then falls back to matching an interactive element by visible text
(exact then contains) — nested and non-ASCII labels resolve. 'text=<label>'
forces the text path. CSS still wins when it matches.
- #24-D: 'get text' with no selector now returns the whole page (body).
- #24-C: 'tab <ref> --activate' (alias --front) switches to the tab AND raises it
to the foreground — to surface a specific tab for the human.
Tests cover the text fallback / text= / xpath builder, body default, activate
flag. The core stale-sessionId-after-cross-process-nav bug is the #20/#23 class,
already fixed in ext 0.4.8 — needs that extension deployed.
Dogfooding by driving a canvas game surfaced that per-action shell round-trips
(keydown; sleep; keyup) are the slowest, lowest-fidelity way to drive anything
timed — each is a process spawn + relay round-trip with ~250ms jitter, so a
'0.8s hold' is anything but.
- 'press <key> --hold <ms>': keyDown, wait, keyUp all inside the daemon, so the
hold duration is precise and it's one round-trip. For games (hold-to-move/
charge) and any press-and-hold.
- Documented the real driving pattern in the core skill + --help: script a timed
sequence in ONE round-trip with 'batch "press d --hold 900" "press j" "wait 200"'
(batch sends each step to the running daemon; --hold/wait block in-daemon), and
prefer reading engine state via main-world 'eval' over guessing from pixels.
Parser test covers plain/held/missing-duration. Builds on the keydown/keyup full
descriptor fix.
`keydown`/`keyup` dispatched a minimal Input.dispatchKeyEvent carrying only
{key}, so games/handlers that read event.code ("KeyD", "ArrowRight") or
event.keyCode saw nothing — a held key set no movement flag and the player
barely moved (dogfood: Dead Cell). They now build the same descriptor `press`
uses (key + code + windows/nativeVirtualKeyCode + printable text on down) via a
shared interaction::dispatch_single_key. Verified live: holding a direction now
drives continuous movement (player ran into an enemy and took damage), where
before it nudged ~80px.
Dogfooding a canvas game over the extension relay surfaced three issues:
1. (serious) A fresh relay session's first `open` navigated one of the USER's
existing tabs instead of opening its own — in testing it replaced a
half-filled form with the target site. On connect the daemon passively
attaches to the user's tabs and pinned one as active; navigate() then drove
it. Now: on the relay (agent_group set), if the active tab isn't one this
session created, navigate() opens its own tab in the session's group first.
Off the relay (a browser we launched) reusing the active tab stays correct.
Pure helper active_index_is_owned() + regression tests.
2. (discoverability) `keydown <key>` / `keyup <key>` (hold-to-move, essential
for games/shortcuts) already existed as commands+daemon handlers but were
absent from --help and the skill, so they were undiscoverable. Documented in
--help, the core skill, and the canvas-app hint.
3. (UX) Canvas/WebGL pages expose almost no a11y tree, so `snapshot` is empty
and agents get stuck hunting refs. snapshot now detects a viewport-dominating
canvas with a sparse tree and prints a hint pointing at the screenshot +
coordinate-click + keydown/keyup path.
Verified live over the relay: `open` now lands the game in its own new tab with
the user's tabs (incl. the Rakuten recovery form) untouched; the canvas hint
fires on the game page; `close` cleans up only the session's own tab.
Multi-session over one relayed Chrome had a tab-identity fracture: each daemon
discovered targets ONCE at connect and assigned its own t<N> indices, so a tab
filled in session A was unreachable from session B — B saw a disjoint/blank set
and rebinding via 'open' piled up duplicate tabs. A stranded, still-filled tab
could not be finished from any other session.
- 'tab list' now re-syncs the live target set on every call: adopts tabs other
sessions opened (or that re-attached after a cross-process nav), drops gone
ones (clears phantom rows), and refreshes url/title from each live tab via
Target.getTargetInfo (the relay only stamps target_info on attach, so it goes
stale/blank after navigation — which made rows indistinguishable).
- 'tab list --full' now prints each tab's stable CDP targetId. Unlike t<N>
(per-session, reassigned each connect), targetId is stable across every session
on the relayed Chrome.
- 'tab <targetId>' adopts a specific pre-existing tab — including another
session's — WITHOUT reloading, so a half-filled form survives. handle_tab_switch
resyncs first, then resolves a raw targetId before falling back to t<N>/label.
- 'open <url> --reuse-tab' (alias --reuse) switches to an existing tab already on
that URL (matched by origin+path, ignoring volatile query/fragment) instead of
spawning a duplicate.
Verified live over the extension relay: a fresh session's 'tab list --full' lists
the user's real tabs with correct titles + full URLs + targetIds, and
'tab <targetId>' lands on and reads the exact stranded Rakuten account-recovery
form from the report. Unit tests cover URL normalization + --reuse-tab parsing;
full suite green. Docs: --help Tabs section + core skill multi-session guidance.
Two SPA-SSO debugging gaps:
- The core skill referenced `bringToFront` but the CLI parser never mapped it
(the daemon handler existed) → 'Unknown command'. Wire up
bringToFront / bring-to-front / bringtofront → the existing action.
- 'stale sessionId — re-open your target URL' recovery was impossible because
`tab list` truncates long URLs with '…', cutting client_id/state out of SSO
links. Add `tab list --full` (also `tab --full`) to print untruncated URLs;
SKILL.md documents the recovery (full URL + re-open the stable entry URL).
The stale-session itself auto-recovers via the stable per-tab relay session id
(#17, extension 0.4.4). Parse tests for both new forms; verified live.
When connected to the user's real Chrome via the extension relay, sequential
commands could land on the wrong tab: `get url` returned x.com/home, then with
no navigation in between `eval` executed against x.com/notifications — so it
read the wrong page and returned nothing.
Root cause: the session's anti-drift anchor is `active_target_id` (pinned by
stable target_id), documented to be set "on every explicit open". But `open`
runs through `navigate()`, which never pinned. On the relay path `open` reuses
an existing tab via `navigate` rather than `add_page` (the only "explicit" path
that pins), so `active_target_id` stayed `None` and the session rode the fragile
`active_page_index`. A later passive tab close/reorder (drained before every
command) then drifted `eval`/`get url`/`snapshot` onto a foreign tab.
Fix:
- `navigate()` now syncs the index to the resolved active page and pins it by
target_id after a successful navigation — restoring the "pin on explicit open"
invariant for the relay path.
- `ensure_page()` pins its freshly-created tab too (matches `add_page`).
- Extract the pin-vs-index resolution into a pure `resolve_active_index()` and
cover the invariant with unit tests (pin beats stale index; falls back when
the pin is gone; survives passive background-tab discovery).
cargo fmt + clippy -D warnings clean; full suite 816 passed.
When a tab's chrome.debugger session was torn down and re-established
(cross-process navigation, MV3 service-worker restart wiping the in-memory
maps, DevTools stealing the debugger), the extension minted a brand-new
monotonic `cb-tab-N` for the same tab. The daemon stays bound to the old id and
the relay consumes attach/detach events without telling it to rebind, so the
session was orphaned permanently → `stale sessionId / tab is gone`, and re-open
never recovered.
Derive the session id from the STABLE Chrome tabId (`cb-tab-<tabId>`) instead.
Any re-attach of the same tab now restores the SAME session the daemon already
holds, so eval/snapshot transparently follow the new page after a navigation.
Extension 0.4.3 → 0.4.4. Adds a relay unit test for the detach→reattach-same-
session recovery contract.
#16: handle_screenshot now returns a canonicalized ABSOLUTE path, so the
`✓ Screenshot saved to …` line is the same regardless of process cwd and the
agent can read the file without guessing the cwd.
#15: `session list` now reflects the extension-relay connection — when the relay
is up it shows the active session as `(relay/extension → live Chrome)` instead
of "No active sessions", and the --json output gains a `relay` bool. Stops agents
misjudging a live relay connection as down.
DOM.setFileInputFiles is forbidden by Chrome's chrome.debugger API, so upload
always fails over the extension relay with an opaque -32000 "Not allowed". Map
it to a clear message: file upload needs a --launch/direct-CDP session, and
point at the cookies export|set --curl workaround. Note the limit in the core
skill upload line too.
N parallel `open --profile <same>` (e.g. chatgpt-imagegen's web backend firing
3 image gens at once) collided on the profile-copy disk I/O and Chrome's profile
lock: every candidate burned its full ~30s launch timeout and ALL failed (0
success), because the loser instances hung without writing DevToolsActivePort.
ProfileLaunchLock takes a cross-process flock on a per-resolved-profile lock
file, held across the copy + launch until Chrome is up, so concurrent
same-profile launches queue instead of colliding — the storm becomes
all-succeed-serially instead of all-fail. The kernel releases the lock when the
holder exits, so a crash can't wedge the queue; acquisition is best-effort
(launch proceeds unlocked if it can't be taken). Uncontended single launches
are unaffected.
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
Standalone product rename across the whole repo (issue: project identity):
- Binary/package/repo/skill/docs: agent-browser[-stealth] → chrome-use
(single binary name `chrome-use`; old aliases agent-browser/abs dropped).
- Version: 0.27.0-fork.51 → 1.0.0 (drop the upstream-fork counter).
- Native-messaging host: com.agent_browser.connect → com.leeguoo.chrome_use
(CLI + ab-connect extension in lockstep — this is a breaking handshake change,
extension bumped 0.4.2 → 0.5.0, needs a Web Store republish).
- Config dir: ~/.agent-browser → ~/.chrome-use.
- README/zh: reframed from "stealth fork of agent-browser" to a standalone
product with a small `originally based on vercel-labs/agent-browser` credit.
- Kept AGENT_BROWSER_* env vars working (63 vars across the codebase; renaming
them would break every existing script/skill for no user-facing gain).
Build green, 802 unit tests pass, fmt + clippy clean. Upstream attribution to
vercel-labs/agent-browser preserved.
`open` waits for the `load` event by default. SPAs whose `load` never fires
(a long-pending XHR or a stuck sub-resource holds it open) made `open`
hard-fail after the lifecycle timeout — even though the DOM was ready and
eval/screenshot worked immediately right after.
- Graceful degradation: if the lifecycle event times out but document.readyState
is interactive/complete, navigate returns success carrying a `warning` in the
response (the CLI prints it to stderr; --json keeps the field) instead of
erroring. Only a still-loading document is a real failure.
- `open/goto/navigate` now accept `--wait-until <load|domcontentloaded|
networkidle|none>` so SPAs can return as soon as the DOM is parsed. The URL
parser skips the --wait-until value so it isn't mistaken for the URL.
- WaitUntil::as_str() for the warning label; output.rs surfaces response warnings.
Verified live: --wait-until domcontentloaded returns immediately on a page whose
load never fires; default load on the same page now succeeds at the timeout with
a clear stderr warning instead of failing. Adds parse tests for both arg orders
+ bogus value.
A bare --launch opens an isolated empty profile (no cookies/login/
extensions). A human watching the desktop sees a mystery Chrome window
under an unfamiliar profile and reads it as broken/suspicious.
- Seed the temp profile's Local State (profile.info_cache.Default.name,
the field Chrome's profile chip reads) + Default/Preferences with
'agent-browser (<session>)', so the window self-identifies which agent
session owns it.
- Rewrite the --launch warning to explain it's an isolated test profile and
point at the escape hatches: --profile auto / AGENT_BROWSER_PROFILE=auto
to reuse real Chrome, and --args "--load-extension=<dir>" for extensions.
- SKILL.md documents the same.
Adds a unit test for the profile-label writer.
Field report #8: in extension-relay sessions, reads (eval/screenshot/network)
could silently run against whatever tab drifted into focus, with no signal,
and network capture was intermittently empty.
- #8.1: screenshot and `network requests` now print `screenshot @ <url>` /
`network @ <url>` to stderr (mirrors the existing `eval @ <url>`), and the
responses carry `origin`. A read against the wrong/drifted tab — and the
"0 captured" vs "wrong page" ambiguity — is now obvious.
- #8.3: `network requests --clear` now enables Network capture immediately
instead of lazily on the next read, so requests fired between `--clear` and
the following read are tracked (fixes the "No requests captured" on first
try, works on retry" race). Extracted enable_request_tracking helper.
- #8.2: the daemon version-mismatch restart notice now spells out that
in-memory context (active tab, refs, captured requests) is reset and tells
the user to re-open the target URL if the next read looks blank/wrong.
Verified on a launched browser: coordinate clicks land, screenshot/network
stamps appear, and a fetch after --clear is captured on the first read.
Field-report ergonomics fixes so agents stop wasting a round on a wrong guess:
- Coordinate click is now first-class: `click <x> <y>`, `click <x>,<y>`,
and `click --coords <x>,<y>` dispatch a raw viewport-point click (no
element resolution), reusing the humanize trajectory + press dwell. Was
previously only reachable via eval(elementFromPoint(...).click()).
- Aliases: `tabs` (plural) → the `tab` subcommand tree; `get-text`/`get_text`
→ `get text <selector>`.
- `find <value> <action>` with a bare value (no locator keyword), e.g.
`find "I'm not a robot" click`, now errors with the corrected command
(`find text "I'm not a robot" click`) plus concrete examples, instead of
a bare "Valid options: role, text, ..." list.
Adds parse-layer regression tests for every form.
remove_page_by_target_id left active_target_id dangling when the pinned
page itself was removed, so resolved_active_index silently fell back to
active_page_index — which after a passive about:blank discovery can point
at a blank tab. That matches issue #7's intermittent symptom: `wait` then
eval/snapshot landing on about:blank in a --launch session.
Re-pin to the surviving active page after removing the pinned target so
the pin is never left pointing at a target that no longer exists. Adds
pure regression tests for the re-anchor invariant (BrowserManager needs a
live CDP client, so the method can't be unit-constructed directly).
The srcdoc-iframe contentWindow Proxy returned native window methods
unbound, so iframe.contentWindow.getComputedStyle()/addEventListener()/
setTimeout() ran with the Proxy as `this` and threw "Illegal invocation"
on any page that uses a srcdoc iframe under --launch (FullLaunch). The
sibling matchMedia proxy already bound its methods; this one did not.
Wrap each function in an apply/construct trap that swaps the Proxy
receiver for the real window while passing .prototype/.name/.toString/
identity straight through (a plain .bind() drops .prototype and breaks
instanceof/constructors). Cached in a WeakMap for stable identity.
Verified before/after on a launched stealth browser: getComputedStyle,
addEventListener, setTimeout all OK; .prototype preserved.
Local stealth verification with no external detector: reports mode (connect vs
launch), live fingerprint probes (navigator.webdriver / window.chrome / plugins /
UA-headless) as pass/fail, and an audit of the active overrides for the path
(incl. the iframe-proxy state from #4). `--json` for a stable shape agents can
gate a sensitive flow on. Distinct from `doctor` (install/env health).
--launch mode scored ~20% stealth on CreepJS because the srcdoc-iframe
contentWindow Proxy trips `hasIframeProxy` — the proxy that hides automation is
itself a fingerprintable tell (violates this fork's own "native > JS lies" rule).
Add a config-driven opt-out (no detectable global): AGENT_BROWSER_DISABLE_IFRAME_PROXY=1
drops the patch via __abStealth.disableIframeProxy → the iframe IIFE early-returns
→ clean 0% CreepJS, trading the niche srcdoc-iframe masking. Default keeps current
behavior. README now documents the --launch 20% honestly and scopes the headline
0% to the extension-connect path. Verified: launch + srcdoc page intact with the
toggle; stealth tests green (config strip-prefix kept in sync).
- snapshot -c (compact) now always keeps lines with an interactive ARIA role
(button/link/textbox/combobox/option/…), not only `ref=`/`": "` lines — so a
clickable control can't vanish from compact output and leave the agent clicking
an empty ref (issue #2 P1). Additive: only ever keeps more. compact tests green.
- stale-ref error now leads with "take a fresh snapshot" and points to the `eval`
fallback for ref-churning SPAs, and demotes AGENT_BROWSER_VERIFY_REF=0 to a
flagged last resort instead of presenting it as the fix (issue #3 P1).
leeguooooo