Compare commits

...
35 Commits
Author SHA1 Message Date
leeguooooo c85e3faa82 chore(release): 1.5.3 — get text defaults to cross-frame; #29 (sessions/did-you-mean/tab liveness); CI changelog fix
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
2026-06-15 13:42:58 +09:00
leeguooooo b25958946c feat(text): 'get text' (no selector) defaults to cross-frame whole-page read
So an agent never silently misses iframed content (listing descriptions etc.)
without having to know the --all-frames flag. Single-frame pages are unchanged
(identical to the old body read); multi-frame pages now include child frames —
a strict superset. Skill + help updated to make the default and 'frames'/--main
discoverable.
2026-06-15 13:42:48 +09:00
leeguooooo d4ff49caa8 ci(release): don't let an empty changelog section abort the release (bash -e)
The changelog step runs under 'bash -e'. section() returned non-zero when a
commit category was empty (grep no-match / empty [ -n ] test), aborting the
script before the closing heredoc delimiter — so any release whose range lacked
a whole category (e.g. 1.5.2: only feat, no fix) failed to attach binaries.
Add '|| true' + 'return 0' so section() always succeeds.
2026-06-15 13:42:48 +09:00
leeguooooo 4e949fffbf chore(release): 1.5.2 — sessions command + did-you-mean + honest tab-switch liveness (#29) 2026-06-15 13:34:26 +09:00
leeguooooo af46490812 feat(cli): sessions command + 'did you mean' suggestions + honest tab-switch liveness (#29)
- chrome-use sessions: top-level alias for the daemon inventory (the skill
  advertises sessions, so it's a natural guess that used to error).
- Unknown commands now suggest the nearest valid one (Levenshtein + prefix
  match), staying silent when nothing is close (e.g. 'clik' -> click,
  'sesions' -> sessions, 'xyzzy' -> no suggestion).
- tab <id>: probe the switched session and show a warning indicator instead of
  a green check when it isn't responding yet, so a switch onto a re-attaching
  (churned-tabId) session no longer reports false success. The #24 targetId
  recovery self-heals within ~6s, hence a warning rather than a hard error.
2026-06-15 13:25:16 +09:00
leeguooooo 57c52d6517 chore(release): 1.5.1 — frame-aware text extraction (get text --all-frames/--main, frames; #27) + ab-connect 0.4.9 targetId recovery (#24)
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
2026-06-15 13:15:47 +09:00
leeguooooo 2707ceb1c4 feat(text): frame-aware text extraction — get text --all-frames / --main + frames (#27)
On listing/marketplace pages (Yahoo Auctions, Rakuten, Mercari shops) the
seller's description lives in a child frame or under a related-items sidebar,
so 'get text body' returned only header/nav boilerplate.

- get text --all-frames: aggregate visible text across every reachable frame.
  Same-process child frames are read via Page.createIsolatedWorld; OOPIFs via
  their auto-attached debugger session (iframe_sessions). Each non-top frame is
  labelled with a '----- frame [kind] url -----' separator.
- get text --main: readability-lite — prefer the densest <main>/<article>
  region over the whole body, dropping global header/nav/footer chrome.
- frames: enumerate frames (kind + url + per-frame text length) so an agent can
  see where a page's text actually lives and pick the right read.

Verified live: inline srcdoc frame text aggregated through --all-frames; Yahoo
Auctions <main> (2881 chars) extracted via --main, stripping the Yahoo header.
2026-06-15 12:51:59 +09:00
leeguooooo f7a657ac46 ci(release): group changelog by type ( Features / 🐛 Fixes / 🔧 Other)
Release notes were a flat list of commit subjects — hard to tell at a glance what
was added vs fixed (recurring '看不出改了什么'). Group by conventional-commit type
so every future release auto-shows scannable Features/Fixes sections.
2026-06-15 12:17:38 +09:00
leeguooooo 4e295ce139 fix(ab-connect): recover a churned-tabId session by stable CDP targetId (0.4.9, #24)
Live-reproduced #24 on 0.4.8 driving the Mercari signin token-exchange hop
(login.jp.mercari.com): the cross-process nav gives the tab a NEW Chrome tabId
while the CDP targetId stays the same. So cb-tab-<oldTabId> can't be recovered —
recoverSessionTab parsed the old tabId, chrome.tabs.get(oldTabId) failed (gone),
and it gave up → permanent 'stale sessionId ... its tab is gone' until the page
settled ~6s later and something re-attached. current/tab <targetId>/daemon
restart all failed because the relay still mapped the targetId to the dead
session.

Fix: remember each session's targetId across detach (sessionTargets map). When
recoverSessionTab can't recover by the encoded tabId, fall back to the STABLE
targetId — chrome.debugger.getTargets() to find the tab now hosting that target,
attach it, and ALIAS the dead cb-tab-<oldTabId> session to the live tab so the
daemon's session id keeps resolving. Longer retry window (~6s) since this hop
takes seconds to settle. Builds on 0.4.6/0.4.8 reattach; covers the tabId-churn
case those missed.

Needs dogfood on the real Mercari flow (can't repro the tabId churn synthetically).
2026-06-15 12:08:32 +09:00
leeguooooo 3d82f11ff2 chore(release): 1.5.0 — text-selector click + get text→body + tab --activate + click --follow/openedTab (#24); fill fires input/change/blur (#25); close <tab> wording + chrome-use current (#26)
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
2026-06-15 11:32:30 +09:00
leeguooooo 33269adc1a fix(fill/tabs): dispatch real input/change/blur (#25); close <tab> wording + chrome-use current (#26)
#25 — fill() didn't fire the events framework inputs / site autocomplete need:
it set value directly (bypassing React's value-tracker) and typed via
Input.insertText, so controlled components and input/change/blur listeners (e.g.
Mercari's postal-code → 都道府県 lookup) never ran though the value showed. fill
now emulates a real edit: focus, set through the element's prototype value setter
(React _valueTracker registers), then dispatch input → input → change → blur/
focusout. SELECT and contenteditable handled too. type <sel> <text> remains for
per-keystroke sites. Verified live: an input wired with input/change/blur fired
'IICB' from one fill.

#26 (ergonomics):
- 'close <tab>' now closes just that tab and prints 'Tab [tN] closed'; bare
  'close' still closes the browser. Previously 'close t12' ran a browser close
  and alarmingly printed 'Browser closed'.
- new 'chrome-use current': prints the active tab's stable handle (tabId + CDP
  targetId + url/title), refreshed live — so an agent holds the targetId (which
  survives cross-process nav) instead of re-deriving 'which tab is live' from
  'tabs' every step. The deeper tab-id churn is the #21/#23 stable-targetId story.

Tests cover fill events (live), close tab-vs-browser parse, and current.
2026-06-15 11:26:34 +09:00
leeguooooo 9ab8753b48 feat(click): report (and optionally --follow) a tab opened by a click (#24-A)
A click on a target=_blank link / window.open opened a new tab, but the active
tab stayed put, so the post-click snapshot showed the OLD page — looking exactly
like the click failed. On the relay the new tab is discovered only via getTargets
(the relay doesn't push target events to the daemon), so it went unsurfaced.

handle_click now snapshots tracked targets before the click and, after, runs a
lightweight BrowserManager::adopt_newly_opened (one getTargets, attaches only the
new target — far cheaper than a full resync) to detect a freshly-opened tab. It's
reported as openedTab {tabId,url,title} in the response (and a '→ opened new tab
[tN] <url>' hint in text mode). Default keeps focus on the current tab (so
multi-tab flows aren't hijacked, per #7/#8.1); 'click <sel> --follow' switches to
the new tab. Verified live: clicking a _blank link prints
'→ opened new tab [t13] https://example.org/'.

Completes the #24 friction items (B/C/D shipped in 770708b).
2026-06-15 11:13:34 +09:00
leeguooooo 770708b8e6 fix(cli): text-selector click by visible label + get text→body + tab --activate (#24)
Three CLI gaps surfaced driving a Mercari signup→checkout flow:

- #24-B (correctness): a bare label like 'click 購入手続きへ' was fed straight to
  document.querySelector as CSS and failed as an invalid selector, even though
  snapshot listed the button by that exact name. build_find_element_js now tries
  CSS first, then falls back to matching an interactive element by visible text
  (exact then contains) — nested and non-ASCII labels resolve. 'text=<label>'
  forces the text path. CSS still wins when it matches.
- #24-D: 'get text' with no selector now returns the whole page (body).
- #24-C: 'tab <ref> --activate' (alias --front) switches to the tab AND raises it
  to the foreground — to surface a specific tab for the human.

Tests cover the text fallback / text= / xpath builder, body default, activate
flag. The core stale-sessionId-after-cross-process-nav bug is the #20/#23 class,
already fixed in ext 0.4.8 — needs that extension deployed.
2026-06-15 11:00:49 +09:00
leeguooooo 7c594820da docs(stream): document the bidirectional WS as the real-time driving path
Dogfooding (driving a canvas game) showed the slow, low-fidelity way — one
screenshot + one CLI call per action — when chrome-use already ships the right
tool: the session WebSocket is BIDIRECTIONAL. It streams ~60fps screencast frames
AND accepts input_keyboard/input_mouse/input_touch on the same socket, straight
to CDP Input.dispatch* — verified live over the extension relay (217 frames in
3.4s, ~64fps, and the input drove the game). But the inbound input protocol was
undocumented, so agents default to the CLI-per-action grind.

Document it in --help (stream) and the core skill: the frame + input message
schemas and the 'connect once, read frames, send timed input' loop, with a node
snippet. Reserve screenshots for one-off checks; use the WS for sustained
real-time control.
2026-06-14 00:54:30 +09:00
leeguooooo 81d18bbd2e feat(input): press --hold <ms> for precise timed key-holds + document timed-driving pattern
Dogfooding by driving a canvas game surfaced that per-action shell round-trips
(keydown; sleep; keyup) are the slowest, lowest-fidelity way to drive anything
timed — each is a process spawn + relay round-trip with ~250ms jitter, so a
'0.8s hold' is anything but.

- 'press <key> --hold <ms>': keyDown, wait, keyUp all inside the daemon, so the
  hold duration is precise and it's one round-trip. For games (hold-to-move/
  charge) and any press-and-hold.
- Documented the real driving pattern in the core skill + --help: script a timed
  sequence in ONE round-trip with 'batch "press d --hold 900" "press j" "wait 200"'
  (batch sends each step to the running daemon; --hold/wait block in-daemon), and
  prefer reading engine state via main-world 'eval' over guessing from pixels.

Parser test covers plain/held/missing-duration. Builds on the keydown/keyup full
descriptor fix.
2026-06-14 00:43:52 +09:00
leeguooooo d99a223d23 chore(release): 1.4.1 — hold-to-move (keydown/keyup full descriptor, #game) + expects ab-connect 0.4.8 (#23 reattach hardening)
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
2026-06-14 00:04:08 +09:00
leeguooooo 4e7e80a596 fix(ab-connect): bind to stable tabId as the primary key + retry on mid-flight detach (0.4.8, #23)
claude-in-chrome completes the Rakuten cart→購入手続き→checkout flow that
chrome-use 1.2.3 couldn't, because it binds to the browser-level tabId (survives
renderer-process swaps) rather than a CDP target/sessionId (torn down by the
cross-origin OAuth/SSO nav). chrome-use's relay is already keyed to the stable
tabId (cb-tab-<tabId>, #17) and sends commands by {tabId} — the gap was purely
that the extension treated the session→tab map as the source of truth and only
reactively re-attached after a failed lookup.

Make the tabId the PRIMARY resolution path: derive it straight from the session
id (tabIdFromSession), ensure-attach with short retries across the swap window
(recoverSessionTab now loops), and route every send through sendCdpToTab, which
on a detached-style error drops the stale handle, re-attaches the stable tab, and
retries once. So a cross-process nav never surfaces as a hard error — there's no
'session gone' window, matching claude-in-chrome. Builds on 0.4.5/0.4.6 reattach;
makes it primary + bulletproof rather than a fallback.
2026-06-14 00:02:22 +09:00
leeguooooo 9bf79a4242 fix(keyboard): keydown/keyup send full key descriptor so hold-to-move works
`keydown`/`keyup` dispatched a minimal Input.dispatchKeyEvent carrying only
{key}, so games/handlers that read event.code ("KeyD", "ArrowRight") or
event.keyCode saw nothing — a held key set no movement flag and the player
barely moved (dogfood: Dead Cell). They now build the same descriptor `press`
uses (key + code + windows/nativeVirtualKeyCode + printable text on down) via a
shared interaction::dispatch_single_key. Verified live: holding a direction now
drives continuous movement (player ran into an enemy and took damage), where
before it nudged ~80px.
2026-06-13 23:58:30 +09:00
leeguooooo 5b4ffdb2bb chore(release): 1.4.0 — no-hijack open + tab adopt-by-targetId + keydown/keyup docs + canvas hint + all-component version coherence (doctor)
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
2026-06-13 23:45:30 +09:00
leeguooooo 62e7229b47 feat(version): extension reports its version; doctor shows all-component coherence
The upgrade story spanned four parts (CLI, daemon, extension, skill) with no
single view and — worst — the extension was a total black box: nothing reported
which build was live, so a user could sit on a stale extension with zero signal.

- ext (0.4.7): on connect the extension sends a `hello` with
  chrome.runtime.getManifest().version; the native-messaging host records it to a
  `relay-ext-version` sidecar (next to relay-cdp-url, removed on exit).
- build.rs embeds the shipped extension version (AB_CONNECT_VERSION, read from the
  ext manifest at compile time) so the CLI knows what extension it expects.
- `chrome-use doctor` gains a Versions section: CLI (vs the cached latest from the
  background update check), extension (connected version vs the bundled expected —
  warns + tells you to reload it in Chrome if behind), and skill (bundled, version-
  locked; `skills add` copies may be stale). Daemon coherence was already covered.

So 'which of the four parts is on what version, and what needs upgrading' is now
one command. Verified: doctor warns on a simulated old extension and passes on a
current one; gracefully shows 'not connected / predates reporting' when the host
hasn't learned a version yet.
2026-06-13 23:41:16 +09:00
leeguooooo 23ab4ce68f fix(relay): don't hijack a user tab on open; surface keydown/keyup + canvas hint
Dogfooding a canvas game over the extension relay surfaced three issues:

1. (serious) A fresh relay session's first `open` navigated one of the USER's
   existing tabs instead of opening its own — in testing it replaced a
   half-filled form with the target site. On connect the daemon passively
   attaches to the user's tabs and pinned one as active; navigate() then drove
   it. Now: on the relay (agent_group set), if the active tab isn't one this
   session created, navigate() opens its own tab in the session's group first.
   Off the relay (a browser we launched) reusing the active tab stays correct.
   Pure helper active_index_is_owned() + regression tests.

2. (discoverability) `keydown <key>` / `keyup <key>` (hold-to-move, essential
   for games/shortcuts) already existed as commands+daemon handlers but were
   absent from --help and the skill, so they were undiscoverable. Documented in
   --help, the core skill, and the canvas-app hint.

3. (UX) Canvas/WebGL pages expose almost no a11y tree, so `snapshot` is empty
   and agents get stuck hunting refs. snapshot now detects a viewport-dominating
   canvas with a sparse tree and prints a hint pointing at the screenshot +
   coordinate-click + keydown/keyup path.

Verified live over the relay: `open` now lands the game in its own new tab with
the user's tabs (incl. the Rakuten recovery form) untouched; the canvas hint
fires on the game page; `close` cleans up only the session's own tab.
2026-06-13 23:27:17 +09:00
leeguooooo e272546b5c chore(release): 1.3.0 — daemon restart/status (#20.2) + live tab resync, adopt-by-targetId, open --reuse-tab (#21)
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
2026-06-13 17:16:49 +09:00
leeguooooo c7de19b099 feat(tabs): live tab resync + adopt-by-targetId + open --reuse-tab (#21)
Multi-session over one relayed Chrome had a tab-identity fracture: each daemon
discovered targets ONCE at connect and assigned its own t<N> indices, so a tab
filled in session A was unreachable from session B — B saw a disjoint/blank set
and rebinding via 'open' piled up duplicate tabs. A stranded, still-filled tab
could not be finished from any other session.

- 'tab list' now re-syncs the live target set on every call: adopts tabs other
  sessions opened (or that re-attached after a cross-process nav), drops gone
  ones (clears phantom rows), and refreshes url/title from each live tab via
  Target.getTargetInfo (the relay only stamps target_info on attach, so it goes
  stale/blank after navigation — which made rows indistinguishable).
- 'tab list --full' now prints each tab's stable CDP targetId. Unlike t<N>
  (per-session, reassigned each connect), targetId is stable across every session
  on the relayed Chrome.
- 'tab <targetId>' adopts a specific pre-existing tab — including another
  session's — WITHOUT reloading, so a half-filled form survives. handle_tab_switch
  resyncs first, then resolves a raw targetId before falling back to t<N>/label.
- 'open <url> --reuse-tab' (alias --reuse) switches to an existing tab already on
  that URL (matched by origin+path, ignoring volatile query/fragment) instead of
  spawning a duplicate.

Verified live over the extension relay: a fresh session's 'tab list --full' lists
the user's real tabs with correct titles + full URLs + targetIds, and
'tab <targetId>' lands on and reads the exact stranded Rakuten account-recovery
form from the report. Unit tests cover URL normalization + --reuse-tab parsing;
full suite green. Docs: --help Tabs section + core skill multi-session guidance.
2026-06-13 17:11:38 +09:00
leeguooooo 6b9de10c73 fix(ab-connect): transparently re-attach a stale cb-tab session before failing (0.4.6, #20.1)
When a tab navigates across render processes (e.g. an SSO redirect to another
origin like login.account.rakuten.com), the debugger handle detaches and the
session drops out of the relay maps, so the next command dead-ends with
'stale sessionId ... its tab is gone' — even open/navigate, which should always
be able to drive the tab. But cb-tab-<tabId> encodes the STABLE Chrome tabId
(#17), and the tab itself usually survives the nav.

So before throwing, recoverSessionTab() parses the tabId out of the session,
checks the tab still exists + is eligible, and re-attaches (attachTab re-mints
the identical cb-tab-<tabId> session, keeping the daemon's binding valid), then
the in-flight command retries against the recovered tab. Complements the 0.4.5
onDetach proactive re-attach: that heals on the detach event, this heals lazily
on the next command if the event was missed. Falls back to the original error
only when the tab is genuinely gone (closed/restricted).
2026-06-13 16:50:07 +09:00
leeguooooo 63e0dd5921 docs(skill): document single-session relay limit — no cross-session tab reads (#20.3) 2026-06-13 16:43:21 +09:00
leeguooooo c0ee65d0d8 feat(cli): add 'chrome-use daemon restart|status' to reset stuck session state
A mid-session 'chrome-use upgrade' (or a crashed worker) can leave per-session
daemons holding stale/cross-leaked tab handles, and the only fix was hunting
PIDs with pgrep/kill. Add a first-class command:

- 'daemon restart' kills every session daemon worker (SIGTERM→SIGKILL +
  sidecar cleanup) but leaves the Chrome-launched __nm-host bridge alone, so
  the extension relay stays up — the next command spins a fresh, clean daemon
  against the same live Chrome. Closes no tabs.
- 'daemon status' lists running session daemons (pid + version) and relay state.

Wires connection::restart_all_daemons(), skips the command in the update-notify
nag, documents it in --help and the core skill. Unit tests cover the empty case
and a live-session kill (spawns a real child, asserts it's reaped + sidecars
cleaned). Issue #20.
2026-06-13 16:42:53 +09:00
leeguooooo 7601919a04 fix(ab-connect): auto-reattach on cross-process detach (Rakuten SSO #19 follow-up)
v1.2.3's stable per-tab session id (#17) fixed sessionId STABILITY, but nothing
re-attached when an origin swaps the render process (e.g. the
login.account.rakuten.com SSO redirect — full-page nav + OOPIF). chrome.debugger
detached, the tab survived, but only onUpdated('complete') could re-attach — and
for that flow it didn't, so the session went permanently stale (even
open/navigate failed, retries didn't recover).

onDetach now proactively re-attaches the surviving tab (retry w/ backoff for the
swapped-in process to settle; skips user/DevTools-initiated detaches), so the
stable cb-tab-<tabId> session is restored and commands self-heal. Extension
0.4.4 → 0.4.5; needs a Web Store republish + dogfood on the Rakuten flow.
2026-06-13 16:14:15 +09:00
leeguooooo 345c0d62a2 ci(release): checkout repo in the release job so the changelog isn't empty
The changelog step lived in the separate `release` job (needs: build), which
had no checkout — so git ran with no repo ('fatal: not a git repository') and
the body came out empty. Add a fetch-depth:0 checkout to that job; drop the
now-pointless fetch-depth:0 from the build job.
2026-06-13 15:55:46 +09:00
leeguooooo 0644fb2d0b chore(release): 1.2.3 — bringToFront command + tab list --full untruncated URLs (#19)
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
2026-06-13 15:45:03 +09:00
leeguooooo 1ea6b1a2c5 fix(cli): add bringToFront command + tab list --full untruncated URLs (issue #19)
Two SPA-SSO debugging gaps:
- The core skill referenced `bringToFront` but the CLI parser never mapped it
  (the daemon handler existed) → 'Unknown command'. Wire up
  bringToFront / bring-to-front / bringtofront → the existing action.
- 'stale sessionId — re-open your target URL' recovery was impossible because
  `tab list` truncates long URLs with '…', cutting client_id/state out of SSO
  links. Add `tab list --full` (also `tab --full`) to print untruncated URLs;
  SKILL.md documents the recovery (full URL + re-open the stable entry URL).

The stale-session itself auto-recovers via the stable per-tab relay session id
(#17, extension 0.4.4). Parse tests for both new forms; verified live.
2026-06-13 15:44:25 +09:00
leeguooooo 7bb50d54b3 ci(release): fetch tags before building changelog (was empty)
v1.2.2's auto-changelog came out empty: in a detached-HEAD tag checkout the
tag refs git describe/git log need aren't reliably present even with
fetch-depth:0. Fetch them explicitly first.
2026-06-12 22:55:47 +09:00
leeguooooo e8864c96e2 chore(release): 1.2.2 — non-blocking 'update available' notice + release changelogs
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
- feat(cli): non-blocking update-available notice (stderr, once/day, opt-out) so users learn to upgrade
- ci(release): auto-generate changelog from commit log on every release
2026-06-12 22:47:03 +09:00
leeguooooo 8432f6cd69 feat(cli): non-blocking 'update available' notice so users know to upgrade
The CLI ships as a GitHub Release binary with a manual `chrome-use upgrade`,
but nothing told users a newer version existed — so releases didn't reach them.

Add a lightweight update check: each run reads a cached latest-version and, if
it's newer than the running binary, prints a one-line hint to STDERR (never
stdout, so --json stays clean): "⚠ chrome-use X.Y.Z is available — run
chrome-use upgrade". The cache is refreshed at most once a day by a DETACHED
`__update-check` worker (curl → GitHub latest release), so the current command
never waits on the network. Skipped for meta commands (upgrade/install/doctor/
__*/--version/--help), in CI, in daemon mode, and via
CHROME_USE_NO_UPDATE_CHECK / AGENT_BROWSER_NO_UPDATE_CHECK.

Verified: nag shows for a newer cached version, suppressed by the opt-out env +
on meta commands + when up-to-date; the detached worker writes the real latest
tag from the GitHub API.
2026-06-12 22:46:44 +09:00
leeguooooo a8089310a6 ci(release): build changelog from commit log (not PR-only notes)
GitHub's generate_release_notes only lists merged PRs — near-empty for this
commit-to-main repo, so releases still showed nothing. Render the
conventional-commit subjects since the previous tag instead, and full-clone
(fetch-depth:0) so the diff is available.
2026-06-12 18:18:11 +09:00
leeguooooo ab92d2590b ci(release): auto-generate release changelog (commits + merged PRs since last tag)
GitHub Releases had an empty body — you couldn't tell what changed between
versions. Add generate_release_notes:true so every release ships an
auto-generated changelog.
2026-06-12 18:15:18 +09:00
22 changed files with 2186 additions and 141 deletions
+55 -1
View File
@@ -116,6 +116,16 @@ jobs:
permissions:
contents: write
steps:
# The release job is separate from the build matrix and has no repo by
# default — check it out (full history + tags) so the changelog step has a
# git repo to diff. Without this, `git` failed with "not a git repository"
# and the changelog came out empty.
- name: Checkout
uses: actions/checkout@v6
with:
ref: ${{ github.event.inputs.tag || github.ref }}
fetch-depth: 0
- name: Download all artifacts
uses: actions/download-artifact@v8
with:
@@ -125,6 +135,47 @@ jobs:
- name: List assets
run: ls -la dist
# Build the changelog from conventional-commit subjects since the previous
# tag. GitHub's built-in generate_release_notes only lists merged PRs,
# which is near-empty for this commit-to-main workflow — so we render the
# commit log ourselves and every release shows what actually changed.
- name: Generate changelog
id: changelog
run: |
# fetch-depth:0 gets history, but the tag refs the changelog needs
# aren't always present in a detached-HEAD tag checkout — pull them in.
git fetch --tags --force --quiet origin 2>/dev/null || true
TAG="${{ github.event.inputs.tag || github.ref_name }}"
PREV="$(git describe --tags --abbrev=0 "${TAG}^" 2>/dev/null || true)"
RANGE="${TAG}"
[ -n "$PREV" ] && RANGE="${PREV}..${TAG}"
# Group commit subjects by conventional-commit type so the notes are
# scannable ("what's new / what's fixed") instead of a flat dev log.
LOG="$(git log "$RANGE" --no-merges --pretty='%s' | grep -v '^chore(release)' || true)"
# NOTE: the job runs under `bash -e`. grep returning 1 (no match) and
# the `[ -n "$body" ]` test returning 1 (empty section) must NOT abort
# the script — otherwise a release whose commit range lacks a whole
# category (e.g. only `feat`, no `fix`) dies before writing the closing
# heredoc delimiter and the whole release step fails. `|| true` +
# `return 0` keep section() always-succeeding.
section() { # $1=header $2=grep-pattern
local body; body="$(printf '%s\n' "$LOG" | grep -E "$2" | sed 's/^/- /' || true)"
[ -n "$body" ] && printf '\n### %s\n%s\n' "$1" "$body"
return 0
}
{
echo "notes<<__NOTES_EOF__"
echo "## What changed"
section "✨ Features" '^feat'
section "🐛 Fixes" '^fix'
section "🔧 Other" '^(perf|refactor|docs|build|ci|test|style|revert)'
if [ -n "$PREV" ]; then
echo ""
echo "**Full changelog**: https://github.com/${{ github.repository }}/compare/${PREV}...${TAG}"
fi
echo "__NOTES_EOF__"
} >> "$GITHUB_OUTPUT"
- name: Attach to release
uses: softprops/action-gh-release@v3
with:
@@ -133,5 +184,8 @@ jobs:
dist/*.tar.gz
dist/*.tar.gz.sha256
fail_on_unmatched_files: true
# keep existing release notes if the release was created beforehand
# The commit-based changelog so every release shows what changed. The
# first matrix job to run creates the release with these notes;
# append_body:false keeps later platform jobs from duplicating them.
body: ${{ steps.changelog.outputs.notes }}
append_body: false
+1 -1
View File
@@ -290,7 +290,7 @@ checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724"
[[package]]
name = "chrome-use"
version = "1.2.1"
version = "1.5.3"
dependencies = [
"aes",
"aes-gcm",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "chrome-use"
version = "1.2.1"
version = "1.5.3"
edition = "2021"
description = "Fast browser automation CLI for AI agents"
license = "Apache-2.0"
+18
View File
@@ -3,6 +3,23 @@ use std::env;
use std::fs;
use std::path::Path;
/// Embed the version of the `ab-connect` extension this CLI ships alongside, so
/// `doctor` can tell a connected extension "you're older than what this CLI
/// expects, update it." Read from the extension manifest at build time so it
/// stays in sync with whatever extension version is in the same checkout/release
/// (the ext is on its own 0.4.x line, separate from the CLI version). Falls back
/// to "unknown" if the manifest can't be read.
fn embed_extension_version() {
let manifest = Path::new("../extensions/ab-connect/manifest.json");
println!("cargo:rerun-if-changed=../extensions/ab-connect/manifest.json");
let version = fs::read_to_string(manifest)
.ok()
.and_then(|s| serde_json::from_str::<serde_json::Value>(&s).ok())
.and_then(|v| v.get("version").and_then(|x| x.as_str()).map(String::from))
.unwrap_or_else(|| "unknown".to_string());
println!("cargo:rustc-env=AB_CONNECT_VERSION={}", version);
}
/// Ensure `packages/dashboard/out/` exists so `rust-embed` doesn't fail during
/// Rust-only dev builds where the dashboard hasn't been built. The placeholder
/// `index.html` is only written when the directory is completely absent.
@@ -20,6 +37,7 @@ fn ensure_dashboard_dir() {
fn main() {
ensure_dashboard_dir();
embed_extension_version();
let protocol_dir = Path::new("cdp-protocol");
let out_dir = env::var("OUT_DIR").unwrap();
+354 -33
View File
@@ -30,12 +30,65 @@ pub enum ParseError {
InvalidSessionName { name: String },
}
/// Top-level commands an agent is likely to mistype, used for "did you mean"
/// suggestions on an unknown command (issue #29). Not exhaustive — just the
/// common verbs plus a few known wrong-guesses mapped to the real command.
const KNOWN_COMMANDS: &[&str] = &[
"open", "navigate", "click", "fill", "type", "press", "snapshot", "screenshot", "eval", "get",
"text", "html", "frames", "find", "wait", "scroll", "hover", "select", "check", "uncheck",
"tab", "tabs", "close", "back", "forward", "reload", "sessions", "status", "daemon", "doctor",
"upgrade", "connect", "cookies", "mouse", "keyboard", "stream", "frame", "profiles", "title",
"url", "is", "drag", "dialog", "upload",
];
/// Levenshtein distance, capped — small inputs only (command names).
fn edit_distance(a: &str, b: &str) -> usize {
let a: Vec<char> = a.chars().collect();
let b: Vec<char> = b.chars().collect();
let mut prev: Vec<usize> = (0..=b.len()).collect();
let mut curr = vec![0usize; b.len() + 1];
for (i, &ca) in a.iter().enumerate() {
curr[0] = i + 1;
for (j, &cb) in b.iter().enumerate() {
let cost = if ca == cb { 0 } else { 1 };
curr[j + 1] = (prev[j + 1] + 1).min(curr[j] + 1).min(prev[j] + cost);
}
std::mem::swap(&mut prev, &mut curr);
}
prev[b.len()]
}
/// Closest known command within a small edit distance, or a prefix/substring
/// match — `None` if nothing is close enough to suggest confidently.
fn nearest_command(input: &str) -> Option<String> {
let lower = input.to_lowercase();
// Exact prefix/substring hits first (e.g. "session" -> "sessions").
if let Some(c) = KNOWN_COMMANDS
.iter()
.find(|c| c.starts_with(&lower) || lower.starts_with(**c))
{
return Some(c.to_string());
}
// Tolerance scales with length: short words get distance 1, longer get 2.
let max_dist = if lower.len() <= 4 { 1 } else { 2 };
KNOWN_COMMANDS
.iter()
.map(|c| (*c, edit_distance(&lower, c)))
.filter(|(_, d)| *d <= max_dist)
.min_by_key(|(_, d)| *d)
.map(|(c, _)| c.to_string())
}
impl ParseError {
pub fn format(&self) -> String {
match self {
ParseError::UnknownCommand { command } => {
format!("Unknown command: {}", command)
}
ParseError::UnknownCommand { command } => match nearest_command(command) {
Some(suggestion) => format!(
"Unknown command: {}\nDid you mean: chrome-use {}?",
command, suggestion
),
None => format!("Unknown command: {}", command),
},
ParseError::UnknownSubcommand {
subcommand,
valid_options,
@@ -370,6 +423,12 @@ fn parse_command_inner(args: &[String], flags: &Flags) -> Result<Value, ParseErr
if flags.provider.is_some() {
nav_cmd["waitUntil"] = json!("none");
}
// `--reuse-tab`: adopt an existing tab already on this URL instead of
// navigating/spawning a new one (issue #21 — avoids duplicate tabs on
// rebind, preserves in-page state).
if rest.iter().any(|a| *a == "--reuse-tab" || *a == "--reuse") {
nav_cmd["reuseTab"] = json!(true);
}
// Explicit readiness override (issue #10): SPAs whose `load` event
// never fires (a long-lived XHR/websocket holds it open) hang out the
// load-event wait. `--wait-until domcontentloaded` returns as soon as
@@ -408,10 +467,19 @@ fn parse_command_inner(args: &[String], flags: &Flags) -> Result<Value, ParseErr
"back" => Ok(json!({ "id": id, "action": "back" })),
"forward" => Ok(json!({ "id": id, "action": "forward" })),
"reload" => Ok(json!({ "id": id, "action": "reload" })),
// Explicit opt-in to raise the active tab to the foreground (the core
// skill references it; the daemon handler existed but the CLI didn't map
// it — issue #19). Accept the documented camelCase + kebab/lowercase.
"bringToFront" | "bring-to-front" | "bringtofront" => {
Ok(json!({ "id": id, "action": "bringtofront" }))
}
// === Core Actions ===
"click" => {
let new_tab = rest.contains(&"--new-tab");
// `--follow`: if the click opens a new tab, switch the active tab to
// it (default reports the opened tab but stays put) (issue #24-A).
let follow = rest.contains(&"--follow");
// Coordinate click as a first-class form (issue #8.4): when the only
// handle is a pixel position, no element/selector is needed.
// click <x> <y> e.g. click 449 320
@@ -420,23 +488,27 @@ fn parse_command_inner(args: &[String], flags: &Flags) -> Result<Value, ParseErr
let coord_args: Vec<&str> = rest
.iter()
.copied()
.filter(|a| *a != "--new-tab" && *a != "--coords")
.filter(|a| !a.starts_with("--"))
.collect();
if let Some((x, y)) = parse_coords(&coord_args) {
return Ok(json!({ "id": id, "action": "click", "x": x, "y": y }));
}
let sel = rest
.iter()
.find(|arg| **arg != "--new-tab")
.find(|arg| !arg.starts_with("--"))
.ok_or_else(|| ParseError::MissingArguments {
context: "click".to_string(),
usage: "click <selector> | click <x> <y> | click --coords <x>,<y> [--new-tab]",
usage:
"click <selector> | click <x> <y> | click --coords <x>,<y> [--new-tab] [--follow]",
})?;
let mut cmd = json!({ "id": id, "action": "click", "selector": sel });
if new_tab {
Ok(json!({ "id": id, "action": "click", "selector": sel, "newTab": true }))
} else {
Ok(json!({ "id": id, "action": "click", "selector": sel }))
cmd["newTab"] = json!(true);
}
if follow {
cmd["follow"] = json!(true);
}
Ok(cmd)
}
"dblclick" => {
let sel = rest.first().ok_or_else(|| ParseError::MissingArguments {
@@ -571,11 +643,27 @@ fn parse_command_inner(args: &[String], flags: &Flags) -> Result<Value, ParseErr
// === Keyboard ===
"press" | "key" => {
let key = rest.first().ok_or_else(|| ParseError::MissingArguments {
context: "press".to_string(),
usage: "press <key>",
let key = rest.iter().find(|a| !a.starts_with("--")).ok_or_else(|| {
ParseError::MissingArguments {
context: "press".to_string(),
usage: "press <key> [--hold <ms>]",
}
})?;
Ok(json!({ "id": id, "action": "press", "key": key }))
let mut c = json!({ "id": id, "action": "press", "key": key });
// `--hold <ms>`: hold the key down for <ms> then release, timed inside
// the daemon (one round-trip, no shell-sleep jitter) — for games and
// hold-to-charge where keydown+sleep+keyup over 3 round-trips is too
// imprecise.
if let Some(i) = rest.iter().position(|a| *a == "--hold") {
let ms = rest.get(i + 1).and_then(|s| s.parse::<u64>().ok()).ok_or(
ParseError::MissingArguments {
context: "press --hold".to_string(),
usage: "press <key> --hold <ms>",
},
)?;
c["hold"] = json!(ms);
}
Ok(c)
}
"keydown" => {
let key = rest.first().ok_or_else(|| ParseError::MissingArguments {
@@ -991,7 +1079,22 @@ fn parse_command_inner(args: &[String], flags: &Flags) -> Result<Value, ParseErr
}
// === Close ===
"close" | "quit" | "exit" => Ok(json!({ "id": id, "action": "close" })),
"close" | "quit" | "exit" => {
// `close <tab>` closes only that tab (and the output says "Tab
// closed"); bare `close` closes the browser/session. `close --all` is
// intercepted earlier in the dispatcher. Previously `close t12` still
// ran a browser close and alarmingly printed "Browser closed" (#26).
if let Some(tab_ref) = rest.iter().find(|a| !a.starts_with("--")) {
Ok(json!({ "id": id, "action": "tab_close", "tabId": tab_ref }))
} else {
Ok(json!({ "id": id, "action": "close" }))
}
}
// The active tab's stable handle — `targetId` survives cross-process
// navigation and is reusable across sessions, so an agent can hold it
// instead of re-deriving "which tab is live" from `tabs` each step (#26).
"current" => Ok(json!({ "id": id, "action": "current" })),
// === Inspect ===
"inspect" => Ok(json!({ "id": id, "action": "inspect" })),
@@ -1246,6 +1349,11 @@ fn parse_command_inner(args: &[String], flags: &Flags) -> Result<Value, ParseErr
// === Get ===
"get" => parse_get(&rest, &id),
// List every frame the session can reach (top + same-process child
// frames + out-of-process iframes), with a text-length per frame so you
// can see where a listing's description actually lives (issue #27).
"frames" => Ok(json!({ "id": id, "action": "frames" })),
// Top-level shortcuts for `get <x>` status reads — users naturally type
// `chrome-use url` / `cdp-url` / `title` without the `get` prefix
// (and expect `cdp-url`/`cdp_url` to work interchangeably).
@@ -1496,7 +1604,12 @@ fn parse_command_inner(args: &[String], flags: &Flags) -> Result<Value, ParseErr
// `tabs` (plural) is a natural guess for the `tab` subcommand tree —
// alias it so `tabs` / `tabs list` / `tabs new` all work (issue #8.4).
"tab" | "tabs" => {
match rest.first().copied() {
// `--full` makes `tab list` emit untruncated URLs (needed to re-open
// a long SSO/redirect URL after a stale session — issue #19). Pick
// the subcommand as the first non-flag arg so the flag can appear
// anywhere (`tab --full`, `tab list --full`).
let full = rest.contains(&"--full");
match rest.iter().find(|a| !a.starts_with("--")).copied() {
Some("new") => {
// Accepted forms:
// tab new [url]
@@ -1528,7 +1641,13 @@ fn parse_command_inner(args: &[String], flags: &Flags) -> Result<Value, ParseErr
}
Ok(cmd)
}
Some("list") => Ok(json!({ "id": id, "action": "tab_list" })),
Some("list") => {
let mut cmd = json!({ "id": id, "action": "tab_list" });
if full {
cmd["full"] = json!(true);
}
Ok(cmd)
}
Some("close") => {
let mut cmd = json!({ "id": id, "action": "tab_close" });
if let Some(tab_ref) = rest.get(1) {
@@ -1536,12 +1655,23 @@ fn parse_command_inner(args: &[String], flags: &Flags) -> Result<Value, ParseErr
}
Ok(cmd)
}
Some(tab_ref) => Ok(json!({
"id": id,
"action": "tab_switch",
"tabId": tab_ref,
})),
None => Ok(json!({ "id": id, "action": "tab_list" })),
Some(tab_ref) => {
// `tab <ref> --activate` (alias `--front`) switches to the tab
// AND raises it to the foreground — for handing a specific tab
// to the human (SMS code, captcha) (issue #24-C).
let mut cmd = json!({ "id": id, "action": "tab_switch", "tabId": tab_ref });
if rest.iter().any(|a| *a == "--activate" || *a == "--front") {
cmd["activate"] = json!(true);
}
Ok(cmd)
}
None => {
let mut cmd = json!({ "id": id, "action": "tab_list" });
if full {
cmd["full"] = json!(true);
}
Ok(cmd)
}
}
}
@@ -2316,11 +2446,33 @@ fn parse_get(rest: &[&str], id: &str) -> Result<Value, ParseError> {
match rest.first().copied() {
Some("text") => {
let sel = rest.get(1).ok_or_else(|| ParseError::MissingArguments {
context: "get text".to_string(),
usage: "get text <selector>",
})?;
Ok(json!({ "id": id, "action": "gettext", "selector": sel }))
// `get text --all-frames` aggregates visible text across every
// frame, including out-of-process iframes invisible to the top
// document (issue #27). The selector is ignored in this mode.
let all_frames = rest[1..]
.iter()
.any(|a| matches!(*a, "--all-frames" | "--frames" | "-a"));
if all_frames {
return Ok(json!({ "id": id, "action": "gettext", "allFrames": true }));
}
// `get text --main` returns the main-content region (readability),
// skipping header/nav/footer/sidebar boilerplate (issue #27).
let main = rest[1..]
.iter()
.any(|a| matches!(*a, "--main" | "--readable" | "-m"));
if main {
return Ok(json!({ "id": id, "action": "gettext", "main": true }));
}
// `get text` with no selector reads the WHOLE PAGE and now defaults
// to cross-frame aggregation, so an agent gets a page's iframed
// content (listing descriptions etc.) without having to know about
// `--all-frames` (#27). On a single-frame page this is identical to
// the old body read; multi-frame pages get the child frames too —
// a strict superset. An explicit selector stays element-scoped.
match rest.iter().skip(1).find(|a| !a.starts_with("--")).copied() {
Some(sel) => Ok(json!({ "id": id, "action": "gettext", "selector": sel })),
None => Ok(json!({ "id": id, "action": "gettext", "allFrames": true })),
}
}
Some("html") => {
let sel = rest.get(1).ok_or_else(|| ParseError::MissingArguments {
@@ -3552,6 +3704,40 @@ mod tests {
assert_eq!(cmd["url"], "https://example.com");
}
#[test]
fn test_press_plain_and_hold() {
let cmd = parse_command(&args("press d"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "press");
assert_eq!(cmd["key"], "d");
assert!(cmd.get("hold").is_none());
let held = parse_command(&args("press d --hold 800"), &default_flags()).unwrap();
assert_eq!(held["key"], "d");
assert_eq!(held["hold"], 800);
// Missing/invalid duration is an error, not a silent no-hold.
assert!(parse_command(&args("press d --hold"), &default_flags()).is_err());
assert!(parse_command(&args("press d --hold abc"), &default_flags()).is_err());
}
#[test]
fn test_navigate_reuse_tab_flag() {
let cmd = parse_command(
&args("open https://example.com --reuse-tab"),
&default_flags(),
)
.unwrap();
assert_eq!(cmd["action"], "navigate");
assert_eq!(cmd["reuseTab"], true);
// Alias.
let cmd2 =
parse_command(&args("open https://example.com --reuse"), &default_flags()).unwrap();
assert_eq!(cmd2["reuseTab"], true);
// Absent by default.
let cmd3 = parse_command(&args("open https://example.com"), &default_flags()).unwrap();
assert!(cmd3.get("reuseTab").is_none());
}
#[test]
fn test_navigate_with_headers() {
let mut flags = default_flags();
@@ -3700,6 +3886,21 @@ mod tests {
assert!(cmd.get("x").is_none());
}
#[test]
fn test_click_follow_flag() {
// `--follow` sets the flag; the selector is still found even with the flag
// before it (issue #24-A).
let cmd = parse_command(&args("click @e5 --follow"), &default_flags()).unwrap();
assert_eq!(cmd["selector"], "@e5");
assert_eq!(cmd["follow"], true);
let cmd2 = parse_command(&args("click --follow @e5"), &default_flags()).unwrap();
assert_eq!(cmd2["selector"], "@e5");
assert_eq!(cmd2["follow"], true);
// Absent by default.
let plain = parse_command(&args("click @e5"), &default_flags()).unwrap();
assert!(plain.get("follow").is_none());
}
#[test]
fn test_tabs_alias_lists() {
assert_eq!(
@@ -3716,6 +3917,30 @@ mod tests {
);
}
#[test]
fn test_tab_list_full_flag() {
// issue #19: `--full` → untruncated URLs; works as `tab list --full`,
// `tab --full`, and `tabs --full`. Plain list has no `full`.
for inv in ["tab list --full", "tab --full", "tabs --full"] {
let cmd = parse_command(&args(inv), &default_flags()).unwrap();
assert_eq!(cmd["action"], "tab_list", "{inv}");
assert_eq!(cmd["full"], true, "{inv}");
}
let plain = parse_command(&args("tab list"), &default_flags()).unwrap();
assert_eq!(plain["action"], "tab_list");
assert!(plain.get("full").is_none());
}
#[test]
fn test_bring_to_front_aliases() {
// issue #19: the documented `bringToFront` (+ kebab/lowercase) maps to
// the existing daemon action.
for inv in ["bringToFront", "bring-to-front", "bringtofront"] {
let cmd = parse_command(&args(inv), &default_flags()).unwrap();
assert_eq!(cmd["action"], "bringtofront", "{inv}");
}
}
#[test]
fn test_get_text_hyphen_and_underscore_aliases() {
for verb in ["get-text", "get_text"] {
@@ -3884,6 +4109,28 @@ mod tests {
assert_eq!(cmd["tabId"], "docs");
}
#[test]
fn test_close_tab_vs_browser() {
// `close <tab>` closes that tab (says "Tab closed"); bare `close` closes
// the browser (#26).
let tab = parse_command(&args("close t12"), &default_flags()).unwrap();
assert_eq!(tab["action"], "tab_close");
assert_eq!(tab["tabId"], "t12");
let browser = parse_command(&args("close"), &default_flags()).unwrap();
assert_eq!(browser["action"], "close");
// `quit`/`exit` aliases still browser-close.
assert_eq!(
parse_command(&args("quit"), &default_flags()).unwrap()["action"],
"close"
);
}
#[test]
fn test_current_command() {
let cmd = parse_command(&args("current"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "current");
}
#[test]
fn test_tab_sends_string_tab_id() {
let cmd = parse_command(&args("tab t2"), &default_flags()).unwrap();
@@ -4583,12 +4830,86 @@ mod tests {
}
#[test]
fn test_get_text_missing_selector() {
let result = parse_command(&args("get text"), &default_flags());
assert!(result.is_err());
let err = result.unwrap_err();
assert!(matches!(err, ParseError::MissingArguments { .. }));
assert!(err.format().contains("get text"));
fn test_get_text_defaults_to_all_frames() {
// `get text` with no selector now reads the whole page across ALL frames
// by default (#27), so iframed content isn't silently missed. (Was: a
// top-frame `body` read, #24-D.)
let cmd = parse_command(&args("get text"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "gettext");
assert_eq!(cmd["allFrames"], true);
assert!(cmd.get("selector").is_none());
// An explicit selector still wins and stays element-scoped.
let cmd2 = parse_command(&args("get text h1"), &default_flags()).unwrap();
assert_eq!(cmd2["selector"], "h1");
assert!(cmd2.get("allFrames").is_none());
// `text` top-level shortcut behaves the same.
let cmd3 = parse_command(&args("text"), &default_flags()).unwrap();
assert_eq!(cmd3["allFrames"], true);
}
#[test]
fn test_get_text_all_frames() {
// `--all-frames` switches to whole-page, cross-frame aggregation and
// drops the selector (issue #27).
for variant in ["get text --all-frames", "get text --frames", "text -a"] {
let cmd = parse_command(&args(variant), &default_flags()).unwrap();
assert_eq!(cmd["action"], "gettext", "{variant}");
assert_eq!(cmd["allFrames"], true, "{variant}");
assert!(cmd.get("selector").is_none(), "{variant}");
}
// A flag mixed with a selector still triggers all-frames.
let cmd = parse_command(&args("get text body --all-frames"), &default_flags()).unwrap();
assert_eq!(cmd["allFrames"], true);
// Without the flag, a leading flag-like token is skipped for the selector.
let cmd = parse_command(&args("get text main"), &default_flags()).unwrap();
assert_eq!(cmd["selector"], "main");
assert!(cmd.get("allFrames").is_none());
}
#[test]
fn test_frames_command() {
let cmd = parse_command(&args("frames"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "frames");
}
#[test]
fn test_nearest_command_suggestions() {
assert_eq!(nearest_command("sesions").as_deref(), Some("sessions"));
assert_eq!(nearest_command("session").as_deref(), Some("sessions"));
assert_eq!(nearest_command("clik").as_deref(), Some("click"));
assert_eq!(nearest_command("screenshits").as_deref(), Some("screenshot"));
// Nonsense with no close match stays silent.
assert_eq!(nearest_command("xyzzy"), None);
// The unknown-command error embeds the suggestion.
let err = ParseError::UnknownCommand {
command: "sesions".to_string(),
};
assert!(err.format().contains("Did you mean: chrome-use sessions?"));
}
#[test]
fn test_get_text_main() {
for variant in ["get text --main", "get text --readable", "text -m"] {
let cmd = parse_command(&args(variant), &default_flags()).unwrap();
assert_eq!(cmd["action"], "gettext", "{variant}");
assert_eq!(cmd["main"], true, "{variant}");
assert!(cmd.get("selector").is_none(), "{variant}");
}
}
#[test]
fn test_tab_activate_flag() {
let plain = parse_command(&args("tab t3"), &default_flags()).unwrap();
assert_eq!(plain["action"], "tab_switch");
assert!(plain.get("activate").is_none());
let act = parse_command(&args("tab t3 --activate"), &default_flags()).unwrap();
assert_eq!(act["action"], "tab_switch");
assert_eq!(act["tabId"], "t3");
assert_eq!(act["activate"], true);
// `--front` alias.
let front = parse_command(&args("tab t3 --front"), &default_flags()).unwrap();
assert_eq!(front["activate"], true);
}
// === Protocol alignment tests ===
+32
View File
@@ -449,6 +449,28 @@ pub fn relay_url() -> Option<String> {
}
}
/// Sidecar recording the connected extension's version, written by the host when
/// it receives the extension's `hello` (sibling of `relay-cdp-url`). Lets
/// `doctor` surface which extension build is live without a CDP round-trip.
fn relay_ext_version_path() -> PathBuf {
relay_url_path().with_file_name("relay-ext-version")
}
/// Version of the connected `ab-connect` extension, if the host learned it from
/// the extension's `hello`. `None` when no extension has connected since the
/// host started, or the extension predates version reporting.
pub fn relay_ext_version() -> Option<String> {
let s = std::fs::read_to_string(relay_ext_version_path())
.ok()?
.trim()
.to_string();
if s.is_empty() {
None
} else {
Some(s)
}
}
/// Hidden `__nm-host` mode: launched by Chrome for the ab-connect extension.
///
/// Bridges the extension (native-messaging stdio, envelope protocol) to a local
@@ -570,6 +592,15 @@ async fn nm_host_main() {
Ok(v) => v,
Err(_) => continue,
};
// Extension version handshake: record it next to the relay URL so
// `doctor` can report which extension build is live (and whether it's
// behind). Best-effort; the message carries no CDP payload.
if v.get("method").and_then(|m| m.as_str()) == Some("hello") {
if let Some(ver) = v.get("version").and_then(|x| x.as_str()) {
let _ = std::fs::write(relay_ext_version_path(), ver);
}
continue;
}
let outs = {
let mut s = state.lock().await;
s.handle_ext_message(&v, "")
@@ -602,6 +633,7 @@ async fn nm_host_main() {
}
nm_log("[nm-host] stdin EOF — Chrome closed the port");
let _ = std::fs::remove_file(relay_url_path());
let _ = std::fs::remove_file(relay_ext_version_path());
}
#[allow(clippy::too_many_arguments)]
+65
View File
@@ -642,6 +642,22 @@ fn kill_stale_daemon(session: &str) {
cleanup_stale_files(session);
}
/// Kill every per-session daemon worker (SIGTERM→SIGKILL + sidecar cleanup),
/// leaving the Chrome-launched `__nm-host` native-messaging bridge alone — it's
/// not a tracked session daemon, so the extension relay stays up. Returns the
/// session names that were stopped. Powers `chrome-use daemon restart`, which
/// clears corrupted/cross-leaked daemon state (e.g. after a version-mismatch
/// restart) without the user resorting to `pgrep`/`kill` (issue #20).
pub fn restart_all_daemons() -> Vec<String> {
let inventory = walk_daemons();
let mut stopped = Vec::new();
for session in &inventory.sessions {
kill_stale_daemon(&session.name);
stopped.push(session.name.clone());
}
stopped
}
pub fn ensure_daemon(session: &str, opts: &DaemonOptions) -> Result<DaemonResult, String> {
// Socket connectivity is the sole liveness check — no PID check — so
// callers in a different PID namespace (e.g. unshare) can still reuse
@@ -1182,6 +1198,55 @@ mod tests {
let _ = fs::remove_dir(&dir);
}
#[test]
fn test_restart_all_daemons_empty_dir() {
let dir = std::env::temp_dir().join("ab-test-restart-empty");
let _ = fs::create_dir_all(&dir);
let _guard = EnvGuard::new(&["AGENT_BROWSER_SOCKET_DIR", "XDG_RUNTIME_DIR"]);
_guard.set("AGENT_BROWSER_SOCKET_DIR", dir.to_str().unwrap());
// No daemons registered → nothing to stop, and it must not blow up.
assert!(restart_all_daemons().is_empty());
let _ = fs::remove_dir(&dir);
}
#[cfg(unix)]
#[test]
fn test_restart_all_daemons_kills_live_session() {
let dir = std::env::temp_dir().join("ab-test-restart-live");
let _ = fs::create_dir_all(&dir);
let _guard = EnvGuard::new(&["AGENT_BROWSER_SOCKET_DIR", "XDG_RUNTIME_DIR"]);
_guard.set("AGENT_BROWSER_SOCKET_DIR", dir.to_str().unwrap());
// Spawn a real, killable child and register it as a session daemon.
let mut child = Command::new("sleep")
.arg("30")
.spawn()
.expect("spawn sleep");
let pid = child.id();
let _ = fs::write(dir.join("rktest.pid"), pid.to_string());
let _ = fs::write(get_socket_path("rktest"), b"");
let stopped = restart_all_daemons();
assert!(
stopped.contains(&"rktest".to_string()),
"stopped: {:?}",
stopped
);
// Reap the killed child first — until the parent waits, it lingers as a
// zombie that still answers `kill(pid, 0)`, so is_pid_alive would lie.
let _ = child.wait();
assert!(!is_pid_alive(pid));
// Sidecars are cleaned up.
assert!(!dir.join("rktest.pid").exists());
assert!(!get_socket_path("rktest").exists());
let _ = fs::remove_dir(&dir);
}
#[test]
fn test_cleanup_stale_files_removes_version() {
let dir = std::env::temp_dir().join("ab-test-cleanup-version");
+2
View File
@@ -18,6 +18,7 @@ mod launch;
mod network;
mod providers;
mod security;
mod versions;
use serde_json::{json, Value};
@@ -97,6 +98,7 @@ pub fn run_doctor(opts: DoctorOptions) -> i32 {
let mut fixed: Vec<String> = Vec::new();
environment::check(&mut checks);
versions::check(&mut checks);
chrome::check(&mut checks);
daemon::check(&mut checks);
config::check(&mut checks);
+89
View File
@@ -0,0 +1,89 @@
//! Version-coherence checks across all four moving parts: the CLI binary, the
//! per-session daemons (covered by `daemon.rs`), the connected `ab-connect`
//! extension, and the bundled skill. The extension was previously a black box —
//! nothing reported which build was live — so a user could sit on an old
//! extension with no signal. The extension now reports its version over the
//! relay (`hello`), the host records it, and this surfaces it in one place.
use super::{Check, Status};
use crate::{connect, upgrade};
pub(super) fn check(checks: &mut Vec<Check>) {
let category = "Versions";
let cli_version = env!("CARGO_PKG_VERSION");
// CLI — compare against the latest seen by the background update check.
match upgrade::cached_latest_version() {
Some(latest) if upgrade::version_is_newer(&latest, cli_version) => {
checks.push(
Check::new(
"versions.cli",
category,
Status::Warn,
format!("CLI {cli_version} (newer available: {latest})"),
)
.with_fix("chrome-use upgrade".to_string()),
);
}
_ => {
checks.push(Check::new(
"versions.cli",
category,
Status::Pass,
format!("CLI {cli_version}"),
));
}
}
// Extension — the build this CLI shipped alongside (embedded at compile time
// from the extension manifest) is what we expect to be running.
let expected_ext = env!("AB_CONNECT_VERSION");
match connect::relay_ext_version() {
Some(ext) if upgrade::version_is_newer(expected_ext, &ext) => {
checks.push(
Check::new(
"versions.extension",
category,
Status::Warn,
format!("extension {ext} is behind the bundled {expected_ext}"),
)
.with_fix(
"update ab-connect in Chrome: chrome://extensions \u{2192} reload \
(or wait for the Web Store auto-update)"
.to_string(),
),
);
}
Some(ext) => {
checks.push(Check::new(
"versions.extension",
category,
Status::Pass,
format!("extension {ext}"),
));
}
None => {
checks.push(Check::new(
"versions.extension",
category,
Status::Info,
format!(
"extension not connected (or it predates version reporting — \
expected {expected_ext})"
),
));
}
}
// Skill — ships inside the same release artifact as the binary, so it's
// version-locked here. Copies made elsewhere via `skills add` aren't.
checks.push(Check::new(
"versions.skill",
category,
Status::Info,
format!(
"skills bundled with this CLI ({cli_version}); copies made via `skills add` \
elsewhere may be stale re-run to refresh"
),
));
}
+115 -2
View File
@@ -29,8 +29,8 @@ use windows_sys::Win32::System::Threading::OpenProcess;
use commands::{gen_id, parse_command, ParseError};
use connection::{
cleanup_stale_files, ensure_daemon, get_socket_dir, is_pid_alive, send_command, walk_daemons,
DaemonOptions,
cleanup_stale_files, ensure_daemon, get_socket_dir, is_pid_alive, restart_all_daemons,
send_command, walk_daemons, DaemonOptions,
};
use flags::{clean_args, parse_flags, Flags};
use install::run_install;
@@ -320,6 +320,94 @@ fn run_session(args: &[String], session: &str, json_mode: bool) {
}
}
/// `chrome-use daemon <restart|status>` — manage the per-session daemon workers
/// without resorting to `pgrep`/`kill`. `restart` clears corrupted or
/// cross-leaked daemon state (e.g. after a mid-session `chrome-use upgrade`
/// where stale tab handles bleed across sessions, issue #20) by killing every
/// session worker. The Chrome-launched `__nm-host` native-messaging bridge is
/// NOT a tracked session daemon, so the extension relay survives a restart —
/// the next command spins up a fresh, clean daemon against the same live Chrome.
fn run_daemon(args: &[String], json_mode: bool) {
match args.get(1).map(|s| s.as_str()) {
Some("restart") => {
let stopped = restart_all_daemons();
let relay_up = connect::relay_url().is_some();
if json_mode {
print_json_value(json!({
"success": true,
"data": { "stopped": stopped, "count": stopped.len(), "relay": relay_up },
}));
} else if stopped.is_empty() {
println!("No session daemons running — nothing to restart.");
if relay_up {
println!(
"{}",
color::dim("Extension relay still up; next command starts a fresh daemon.")
);
}
} else {
for s in &stopped {
println!("{} Stopped daemon: {}", color::green(""), s);
}
println!(
"{}",
color::dim(if relay_up {
"Extension relay (__nm-host) left running; next command starts a fresh daemon."
} else {
"Next command starts a fresh daemon."
})
);
}
}
Some("status") | Some("list") => {
let inventory = walk_daemons();
let relay_up = connect::relay_url().is_some();
if json_mode {
let sessions: Vec<_> = inventory
.sessions
.iter()
.map(|s| json!({ "name": s.name, "pid": s.pid, "version": s.version }))
.collect();
print_json_value(json!({
"success": true,
"data": { "sessions": sessions, "relay": relay_up },
}));
} else if inventory.sessions.is_empty() {
println!("No session daemons running.");
if relay_up {
println!("{}", color::dim("Extension relay (__nm-host): up"));
}
} else {
println!("Session daemons:");
for s in &inventory.sessions {
let ver = s
.version
.as_deref()
.map(|v| format!(" {}", color::dim(&format!("(v{})", v))))
.unwrap_or_default();
println!(" {} pid {}{}", s.name, s.pid, ver);
}
if relay_up {
println!("{}", color::dim("Extension relay (__nm-host): up"));
}
}
}
other => {
eprintln!(
"{} usage: chrome-use daemon <restart|status>",
color::error_indicator()
);
if let Some(unknown) = other {
eprintln!(
"{}",
color::dim(&format!(" unknown subcommand: {}", unknown))
);
}
exit(2);
}
}
}
fn get_dashboard_pid_path() -> std::path::PathBuf {
get_socket_dir().join("dashboard.pid")
}
@@ -586,6 +674,17 @@ fn main() {
return;
}
// Hidden update-check worker, spawned detached by maybe_notify_update() to
// refresh the cached latest version without blocking a real command.
if env::args().nth(1).as_deref() == Some("__update-check") {
upgrade::run_update_check();
return;
}
// Non-blocking "update available" hint (stderr only; self-skips meta
// commands, daemon mode, CI, and the opt-out env vars).
upgrade::maybe_notify_update();
// Native daemon mode: when AGENT_BROWSER_DAEMON is set, run as the daemon process
if env::var("AGENT_BROWSER_DAEMON").is_ok() {
// Ignore SIGPIPE so the daemon isn't killed when the parent drops
@@ -788,6 +887,20 @@ fn main() {
return;
}
// Handle daemon management (doesn't talk to a daemon — it manages them).
if clean.first().map(|s| s.as_str()) == Some("daemon") {
run_daemon(&clean, flags.json);
return;
}
// `sessions` is a natural top-level guess for "list my sessions" (the skill
// advertises sessions as a feature) — route it to the daemon inventory the
// same way `daemon status` does (issue #29).
if clean.first().map(|s| s.as_str()) == Some("sessions") {
run_daemon(&["sessions".to_string(), "status".to_string()], flags.json);
return;
}
// Handle close --all: close all active sessions
if matches!(
clean.first().map(|s| s.as_str()),
+216 -24
View File
@@ -1332,6 +1332,7 @@ pub async fn execute_command(cmd: &Value, state: &mut DaemonState) -> Value {
"uncheck" => handle_uncheck(cmd, state).await,
"wait" => handle_wait(cmd, state).await,
"gettext" => handle_gettext(cmd, state).await,
"frames" => handle_frames(cmd, state).await,
"getattribute" => handle_getattribute(cmd, state).await,
"isvisible" => handle_isvisible(cmd, state).await,
"isenabled" => handle_isenabled(cmd, state).await,
@@ -1364,7 +1365,7 @@ pub async fn execute_command(cmd: &Value, state: &mut DaemonState) -> Value {
"recording_stop" => handle_recording_stop(state).await,
"recording_restart" => handle_recording_restart(cmd, state).await,
"pdf" => handle_pdf(cmd, state).await,
"tab_list" => handle_tab_list(state).await,
"tab_list" => handle_tab_list(cmd, state).await,
"tab_new" => handle_tab_new(cmd, state).await,
"tab_switch" => handle_tab_switch(cmd, state).await,
"tab_close" => handle_tab_close(cmd, state).await,
@@ -1395,6 +1396,7 @@ pub async fn execute_command(cmd: &Value, state: &mut DaemonState) -> Value {
"count" => handle_count(cmd, state).await,
"styles" => handle_styles(cmd, state).await,
"bringtofront" => handle_bringtofront(state).await,
"current" => handle_current(state).await,
"timezone" => handle_timezone(cmd, state).await,
"locale" => handle_locale(cmd, state).await,
"geolocation" => handle_geolocation(cmd, state).await,
@@ -2531,6 +2533,20 @@ async fn handle_navigate(cmd: &Value, state: &mut DaemonState) -> Result<Value,
state.ref_map.clear();
state.iframe_sessions.clear();
state.active_frame_id = None;
// `--reuse-tab`: if a tab already shows this URL (same origin+path), switch
// to it instead of navigating — preserves any in-page state and stops
// re-`open` from piling up duplicate tabs on rebind (issue #21).
if cmd
.get("reuseTab")
.and_then(|v| v.as_bool())
.unwrap_or(false)
{
if let Ok(Some(switched)) = mgr.reuse_tab_for_url(url).await {
return Ok(switched);
}
}
let result = mgr.navigate(url, wait_until).await?;
// Adaptive humanize: sample the freshly loaded page for known behavioural
// anti-bot vendors and escalate this session to Human if any are present.
@@ -2874,7 +2890,32 @@ async fn handle_snapshot(cmd: &Value, state: &mut DaemonState) -> Result<Value,
})
.collect();
Ok(json!({ "snapshot": tree, "origin": url, "refs": refs }))
let ref_count = refs.len();
let mut out = json!({ "snapshot": tree, "origin": url, "refs": refs });
// Canvas/WebGL apps (games, map/3D viewers, drawing tools) paint to a
// <canvas> and expose almost no accessibility tree, so `snapshot` comes back
// near-empty and agents get stuck looking for refs that will never exist
// (dogfood: the Dead Cell game). When the tree is sparse but a canvas
// dominates the viewport, tell them to switch to the screenshot-driven path.
if ref_count < 3 {
let canvas_js =
"(() => { const c = document.querySelector('canvas'); if (!c) return false; \
const r = c.getBoundingClientRect(); \
return r.width * r.height > innerWidth * innerHeight * 0.5; })()";
if let Ok(v) = mgr.evaluate(canvas_js, None).await {
if v.as_bool() == Some(true) {
out["note"] = json!(
"This page renders to a <canvas> (game / WebGL / editor) and exposes almost no \
accessibility tree refs won't help. Use `screenshot` to see it, coordinate \
`click <x> <y>` to interact, and `keydown`/`keyup`/`press` for keyboard \
(hold-to-move: `keydown d` `keyup d`)."
);
}
}
}
Ok(out)
}
/// Resolve a (possibly relative) saved-file path to an absolute one so the CLI
@@ -3077,6 +3118,15 @@ async fn handle_click(cmd: &Value, state: &mut DaemonState) -> Result<Value, Str
let button = cmd.get("button").and_then(|v| v.as_str()).unwrap_or("left");
let click_count = cmd.get("clickCount").and_then(|v| v.as_i64()).unwrap_or(1) as i32;
let follow = cmd.get("follow").and_then(|v| v.as_bool()).unwrap_or(false);
// Snapshot tracked targets so we can tell if this click opened a NEW tab
// (target=_blank link / window.open). On the relay the new tab is discovered
// passively and doesn't steal focus (#7/#8.1), so without surfacing it the
// post-click snapshot shows the OLD page and looks like the click failed
// (issue #24-A).
let before: std::collections::HashSet<String> =
mgr.pages_list().into_iter().map(|p| p.target_id).collect();
interaction::click(
&mgr.client,
@@ -3089,7 +3139,26 @@ async fn handle_click(cmd: &Value, state: &mut DaemonState) -> Result<Value, Str
)
.await?;
Ok(json!({ "clicked": selector }))
// Give a just-opened tab a moment to register, then look for it.
let mgr = state.browser.as_mut().ok_or("Browser not launched")?;
tokio::time::sleep(std::time::Duration::from_millis(150)).await;
let opened = mgr.adopt_newly_opened(&before).await;
let mut out = json!({ "clicked": selector });
if let Some(page) = opened {
let tab_id = super::browser::format_tab_id(page.tab_id);
out["openedTab"] = json!({ "tabId": tab_id, "url": page.url, "title": page.title });
// `--follow`: switch the active tab to the newly-opened one (default is
// to report it but stay put, so multi-tab flows aren't hijacked).
if follow {
state.ref_map.clear();
state.iframe_sessions.clear();
state.active_frame_id = None;
let _ = mgr.tab_switch_by_id(page.tab_id).await;
out["followed"] = json!(true);
}
}
Ok(out)
}
async fn handle_dblclick(cmd: &Value, state: &mut DaemonState) -> Result<Value, String> {
@@ -3295,6 +3364,16 @@ async fn handle_press(cmd: &Value, state: &mut DaemonState) -> Result<Value, Str
// Parse modifier+key chords like "Control+a", "Shift+Enter", "Control+Shift+a"
let (actual_key, modifiers) = parse_key_chord(key);
// `--hold <ms>`: keyDown, wait, keyUp — all inside the daemon so the hold
// duration is precise (no shell-sleep / round-trip jitter). For games
// (hold-to-move/charge) and any press-and-hold interaction.
if let Some(ms) = cmd.get("hold").and_then(|v| v.as_u64()) {
interaction::dispatch_single_key(&mgr.client, &session_id, &actual_key, "keyDown").await?;
tokio::time::sleep(std::time::Duration::from_millis(ms)).await;
interaction::dispatch_single_key(&mgr.client, &session_id, &actual_key, "keyUp").await?;
return Ok(json!({ "pressed": key, "heldMs": ms }));
}
interaction::press_key_with_modifiers(&mgr.client, &session_id, &actual_key, modifiers).await?;
Ok(json!({ "pressed": key }))
}
@@ -3516,6 +3595,47 @@ async fn handle_wait(cmd: &Value, state: &mut DaemonState) -> Result<Value, Stri
async fn handle_gettext(cmd: &Value, state: &mut DaemonState) -> Result<Value, String> {
let mgr = state.browser.as_ref().ok_or("Browser not launched")?;
let session_id = mgr.active_session_id()?.to_string();
// `get text --all-frames` aggregates visible text across every frame the
// session can reach — including out-of-process iframes that never show up
// in the top document (#27: Yahoo/Rakuten/Mercari listing descriptions).
if cmd.get("allFrames").and_then(|v| v.as_bool()) == Some(true) {
let frames = super::element::collect_all_frames_text(
&mgr.client,
&session_id,
&state.iframe_sessions,
)
.await?;
let mut combined = String::new();
let mut frame_count = 0usize;
for f in &frames {
let t = f.text.trim();
if t.is_empty() {
continue;
}
frame_count += 1;
if f.kind != "top" {
combined.push_str(&format!("\n\n----- frame [{}] {} -----\n", f.kind, f.url));
}
combined.push_str(t);
}
let url = mgr.get_url().await.unwrap_or_default();
return Ok(json!({
"text": combined,
"origin": url,
"frames": frame_count,
"allFrames": true,
}));
}
// `get text --main` returns the page's main-content region (readability-lite),
// skipping global header/nav/footer/sidebar boilerplate (#27).
if cmd.get("main").and_then(|v| v.as_bool()) == Some(true) {
let text = super::element::get_main_content_text(&mgr.client, &session_id).await?;
let url = mgr.get_url().await.unwrap_or_default();
return Ok(json!({ "text": text, "origin": url, "main": true }));
}
let selector = cmd
.get("selector")
.and_then(|v| v.as_str())
@@ -3533,6 +3653,32 @@ async fn handle_gettext(cmd: &Value, state: &mut DaemonState) -> Result<Value, S
Ok(json!({ "text": text, "origin": url }))
}
async fn handle_frames(_cmd: &Value, state: &mut DaemonState) -> Result<Value, String> {
let mgr = state.browser.as_ref().ok_or("Browser not launched")?;
let session_id = mgr.active_session_id()?.to_string();
let frames = super::element::collect_all_frames_text(
&mgr.client,
&session_id,
&state.iframe_sessions,
)
.await?;
let list: Vec<Value> = frames
.iter()
.enumerate()
.map(|(i, f)| {
json!({
"index": i,
"kind": f.kind,
"url": f.url,
"frameId": f.frame_id,
"textLen": f.text.trim().chars().count(),
})
})
.collect();
let url = mgr.get_url().await.unwrap_or_default();
Ok(json!({ "frames": list, "count": list.len(), "origin": url }))
}
async fn handle_getattribute(cmd: &Value, state: &mut DaemonState) -> Result<Value, String> {
let mgr = state.browser.as_ref().ok_or("Browser not launched")?;
let session_id = mgr.active_session_id()?.to_string();
@@ -4355,10 +4501,19 @@ async fn handle_keyboard(cmd: &Value, state: &DaemonState) -> Result<Value, Stri
// Phase 5 handlers
// ---------------------------------------------------------------------------
async fn handle_tab_list(state: &DaemonState) -> Result<Value, String> {
let mgr = state.browser.as_ref().ok_or("Browser not launched")?;
async fn handle_tab_list(cmd: &Value, state: &mut DaemonState) -> Result<Value, String> {
let mgr = state.browser.as_mut().ok_or("Browser not launched")?;
// Re-sync with the live browser so the list reflects tabs opened by other
// sessions or re-attached after a cross-process nav, and drops gone ones
// (issue #21). Best-effort: a stale list still beats erroring the command.
mgr.resync_targets().await.ok();
let tabs = mgr.tab_list();
Ok(json!({ "tabs": tabs }))
// Echo `full` so the formatter prints untruncated URLs (issue #19).
if cmd.get("full").and_then(|v| v.as_bool()).unwrap_or(false) {
Ok(json!({ "tabs": tabs, "full": true }))
} else {
Ok(json!({ "tabs": tabs }))
}
}
async fn handle_tab_new(cmd: &Value, state: &mut DaemonState) -> Result<Value, String> {
@@ -4389,13 +4544,50 @@ async fn handle_tab_switch(cmd: &Value, state: &mut DaemonState) -> Result<Value
let tab_ref_str = cmd
.get("tabId")
.and_then(|v| v.as_str())
.ok_or("Missing 'tabId' parameter (expected `t<N>` or a label)")?;
let tab_ref = super::browser::TabRef::parse(tab_ref_str)?;
let tab_id = mgr.resolve_tab_ref(&tab_ref)?;
.ok_or("Missing 'tabId' parameter (expected `t<N>`, a label, or a targetId)")?;
// Re-sync first so a tab opened by another session, or one that re-attached
// after a cross-process nav, is adoptable from here (issue #21).
mgr.resync_targets().await.ok();
// A CDP `targetId` (shown in `tab list`) is stable across sessions, so accept
// it directly for adopting a specific pre-existing tab — falling back to the
// per-session `t<N>` / label form.
let tab_id = match mgr.tab_id_for_target(tab_ref_str) {
Some(id) => id,
None => {
let tab_ref = super::browser::TabRef::parse(tab_ref_str)?;
mgr.resolve_tab_ref(&tab_ref)?
}
};
state.ref_map.clear();
state.iframe_sessions.clear();
state.active_frame_id = None;
let result = mgr.tab_switch_by_id(tab_id).await?;
let mut result = mgr.tab_switch_by_id(tab_id).await?;
// Liveness probe: confirm the new session actually answers before we report
// success, so `tab <id>` doesn't print a misleading ✓ for a session that's
// stale and will fail on the very next command (issue #29.3). On the churned
// -tabId case the ext-0.4.9 targetId recovery (#24) self-heals within ~6s, so
// we surface a warning rather than a hard error to avoid a false failure
// during that window.
if mgr.evaluate("1", None).await.is_err() {
if let Some(obj) = result.as_object_mut() {
obj.insert(
"warning".to_string(),
json!("switched tab is not responding yet (session re-attaching); retry the next command"),
);
}
}
// `--activate`: raise this tab to the foreground (the switch made it active;
// bring_to_front acts on the active tab) — for handing a specific tab to the
// human (issue #24-C). Best-effort; don't fail the switch if it can't.
if cmd
.get("activate")
.and_then(|v| v.as_bool())
.unwrap_or(false)
{
let _ = mgr.bring_to_front().await;
}
if let Some(ref server) = state.stream_server {
if let Ok(dims) = mgr
@@ -5225,6 +5417,18 @@ async fn handle_bringtofront(state: &DaemonState) -> Result<Value, String> {
Ok(json!({ "broughtToFront": true }))
}
async fn handle_current(state: &mut DaemonState) -> Result<Value, String> {
let mgr = state.browser.as_mut().ok_or("Browser not launched")?;
// Refresh so `current` reflects the live URL/title even after a cross-process
// nav (the relay's cached target_info can lag) (#26).
mgr.resync_targets().await.ok();
let mut info = mgr.active_page_info().ok_or("No active tab")?;
if let Some(obj) = info.as_object_mut() {
obj.insert("current".to_string(), json!(true));
}
Ok(info)
}
async fn handle_timezone(cmd: &Value, state: &DaemonState) -> Result<Value, String> {
let mgr = state.browser.as_ref().ok_or("Browser not launched")?;
let timezone = cmd
@@ -8726,13 +8930,7 @@ async fn handle_keydown(cmd: &Value, state: &DaemonState) -> Result<Value, Strin
.and_then(|v| v.as_str())
.ok_or("Missing 'key' parameter")?;
mgr.client
.send_command(
"Input.dispatchKeyEvent",
Some(json!({ "type": "keyDown", "key": key })),
Some(&session_id),
)
.await?;
interaction::dispatch_single_key(&mgr.client, &session_id, key, "keyDown").await?;
Ok(json!({ "keydown": key }))
}
@@ -8744,13 +8942,7 @@ async fn handle_keyup(cmd: &Value, state: &DaemonState) -> Result<Value, String>
.and_then(|v| v.as_str())
.ok_or("Missing 'key' parameter")?;
mgr.client
.send_command(
"Input.dispatchKeyEvent",
Some(json!({ "type": "keyUp", "key": key })),
Some(&session_id),
)
.await?;
interaction::dispatch_single_key(&mgr.client, &session_id, key, "keyUp").await?;
Ok(json!({ "keyup": key }))
}
+338 -1
View File
@@ -106,6 +106,18 @@ pub(crate) fn should_track_target(target: &TargetInfo) -> bool {
&& (target.url.is_empty() || !is_internal_chrome_target(&target.url))
}
/// Origin + path of a URL, dropping the query string and fragment, for
/// `--reuse-tab` matching. SPA/SSO URLs carry volatile `?client_id=…&state=…`
/// and `#/route` parts, so two opens of the "same" page rarely match
/// byte-for-byte; comparing origin+path lands the reuse on the right tab.
/// Returns the input unchanged if it doesn't parse as a URL.
fn normalize_url_for_match(url: &str) -> String {
match url::Url::parse(url) {
Ok(u) => format!("{}{}", u.origin().ascii_serialization(), u.path()),
Err(_) => url.to_string(),
}
}
fn update_page_target_info_in_pages(pages: &mut [PageInfo], target: &TargetInfo) -> bool {
if let Some(page) = pages.iter_mut().find(|p| p.target_id == target.target_id) {
page.url = target.url.clone();
@@ -154,6 +166,25 @@ fn resolve_active_index(
active_page_index
}
/// Whether the resolved active page is a tab the session created (its target_id
/// is in `created_targets`). Pure core of [`BrowserManager::active_is_session_owned`]
/// so the relay no-hijack rule is unit-testable without a live browser.
fn active_index_is_owned(
pages: &[PageInfo],
active_target_id: Option<&str>,
active_page_index: usize,
created_targets: &HashSet<String>,
) -> bool {
pages
.get(resolve_active_index(
pages,
active_target_id,
active_page_index,
))
.map(|p| created_targets.contains(&p.target_id))
.unwrap_or(false)
}
/// Converts common error messages into AI-friendly, actionable descriptions.
pub fn to_ai_friendly_error(error: &str) -> String {
let lower = error.to_lowercase();
@@ -787,6 +818,20 @@ impl BrowserManager {
)
}
/// Whether the resolved active page is a tab THIS session created (via
/// `Target.createTarget` — `tab new`, `ensure_page`, or the first `open`).
/// On the shared real browser a fresh session also passively attaches to the
/// user's existing tabs; those are NOT owned, and navigating one would
/// clobber the user's page. Used to gate `navigate` on the relay.
fn active_is_session_owned(&self) -> bool {
active_index_is_owned(
&self.pages,
self.active_target_id.as_deref(),
self.active_page_index,
&self.created_targets,
)
}
/// Pin the current active page by target_id so later commands stick to it.
/// Call after any explicit open / tab new / tab switch.
fn pin_active_target(&mut self) {
@@ -804,6 +849,18 @@ impl BrowserManager {
}
pub async fn navigate(&mut self, url: &str, wait_until: WaitUntil) -> Result<Value, String> {
// On the shared real browser (extension relay), a fresh session only
// passively attached to the user's existing tabs — it doesn't own any. The
// pre-fix code made one of those the active tab, so the first `open` then
// navigated (clobbered) the user's page: in dogfooding an `open` replaced a
// half-filled form with the target site. If the active tab isn't one we
// created, open our own tab in this session's group and navigate THAT, so
// the user's (and other sessions') tabs are never hijacked. Off the relay
// (a browser we launched) reusing the active tab is correct, so this is
// gated on `agent_group()`.
if self.agent_group().is_some() && !self.active_is_session_owned() {
self.tab_new(None, None).await?;
}
let session_id = self.active_session_id()?.to_string();
let mut lifecycle_rx = self.client.subscribe();
@@ -1184,22 +1241,245 @@ impl BrowserManager {
}
pub fn tab_list(&self) -> Vec<Value> {
let active = self.resolved_active_index();
self.pages
.iter()
.enumerate()
.map(|(i, p)| {
json!({
"tabId": format_tab_id(p.tab_id),
// Stable CDP target id. Unlike `t<N>` (per-session, reassigned
// each connect) this is the same handle across every session
// attached to the relayed Chrome, so it's how you adopt a
// specific pre-existing tab from another session (issue #21).
"targetId": p.target_id,
"label": p.label,
"title": p.title,
"url": p.url,
"type": p.target_type,
"active": i == self.active_page_index,
"active": i == active,
})
})
.collect()
}
/// The active tab's stable handle + current location, for `chrome-use
/// current` (#26). `targetId` survives cross-process navigation, so it's the
/// handle an agent should hold across a multi-step flow.
pub fn active_page_info(&self) -> Option<Value> {
let i = self.resolved_active_index();
self.pages.get(i).map(|p| {
json!({
"tabId": format_tab_id(p.tab_id),
"targetId": p.target_id,
"label": p.label,
"url": p.url,
"title": p.title,
})
})
}
/// Stable `tab_id` for a page identified by its CDP `targetId`, if tracked.
/// Lets callers adopt a tab by the cross-session-stable target id.
pub fn tab_id_for_target(&self, target_id: &str) -> Option<u32> {
self.pages
.iter()
.find(|p| p.target_id == target_id)
.map(|p| p.tab_id)
}
/// Re-pull the live target set and reconcile `self.pages`: adopt tabs that
/// appeared since connect (another session's tab, or one that just
/// re-attached after a cross-process nav), refresh url/title on known tabs,
/// and drop tabs that are gone (clearing phantom rows). Never steals focus —
/// the active tab is preserved, and re-pinned if it was pruned. Powers a live
/// `tab list` and adopt-by-targetId so a fresh session can reach a stranded,
/// still-filled tab without reloading it (issue #21).
/// Detect targets that appeared since the `before` set (e.g. a click that
/// opened a new tab via a `target=_blank` link or `window.open`), attach +
/// track each in the background, and return the first newly-opened page.
///
/// Lighter than [`resync_targets`] — one `getTargets` and work only on the
/// new targets, no whole-tab url/title refresh — so it's cheap enough to run
/// after every click. The new tab is added in the background (never steals
/// the active tab, per #7/#8.1); the caller surfaces it so the agent knows a
/// tab opened instead of seeing the old page (issue #24-A).
pub async fn adopt_newly_opened(&mut self, before: &HashSet<String>) -> Option<PageInfo> {
let result: GetTargetsResult = self
.client
.send_command_typed("Target.getTargets", &json!({}), None)
.await
.ok()?;
let live: Vec<TargetInfo> = result
.target_infos
.into_iter()
.filter(should_track_target)
.collect();
let mut opened: Option<PageInfo> = None;
for target in &live {
if before.contains(&target.target_id)
|| self.pages.iter().any(|p| p.target_id == target.target_id)
{
continue;
}
let attach: AttachToTargetResult = match self
.client
.send_command_typed(
"Target.attachToTarget",
&AttachToTargetParams {
target_id: target.target_id.clone(),
flatten: true,
},
None,
)
.await
{
Ok(r) => r,
Err(_) => continue,
};
let tab_id = self.assign_tab_id();
let page = PageInfo {
tab_id,
label: None,
target_id: target.target_id.clone(),
session_id: attach.session_id.clone(),
url: target.url.clone(),
title: target.title.clone(),
target_type: target.target_type.clone(),
};
self.add_background_page(page.clone());
let _ = self.enable_domains(&attach.session_id).await;
if opened.is_none() {
opened = Some(page);
}
}
opened
}
pub async fn resync_targets(&mut self) -> Result<(), String> {
self.client
.send_command_typed::<_, Value>(
"Target.setDiscoverTargets",
&SetDiscoverTargetsParams { discover: true },
None,
)
.await?;
let result: GetTargetsResult = self
.client
.send_command_typed("Target.getTargets", &json!({}), None)
.await?;
let live: Vec<TargetInfo> = result
.target_infos
.into_iter()
.filter(should_track_target)
.collect();
let live_ids: HashSet<String> = live.iter().map(|t| t.target_id.clone()).collect();
for target in &live {
if self.update_page_target_info(target) {
continue;
}
// A target this session hasn't tracked yet — attach and add it in the
// background so it's listable/adoptable without stealing the active tab.
let attach_result: AttachToTargetResult = match self
.client
.send_command_typed(
"Target.attachToTarget",
&AttachToTargetParams {
target_id: target.target_id.clone(),
flatten: true,
},
None,
)
.await
{
Ok(r) => r,
// The tab may have closed between getTargets and attach, or be a
// restricted page — skip it rather than failing the whole resync.
Err(_) => continue,
};
let tab_id = self.assign_tab_id();
self.add_background_page(PageInfo {
tab_id,
label: None,
target_id: target.target_id.clone(),
session_id: attach_result.session_id.clone(),
url: target.url.clone(),
title: target.title.clone(),
target_type: target.target_type.clone(),
});
let _ = self.enable_domains(&attach_result.session_id).await;
}
// Drop tabs that no longer exist so `tab list` doesn't show phantom rows.
let gone: Vec<String> = self
.pages
.iter()
.map(|p| p.target_id.clone())
.filter(|tid| !live_ids.contains(tid))
.collect();
for tid in gone {
self.remove_page_by_target_id(&tid);
}
// Refresh url/title from each live tab. The relay only stamps target_info
// on attach, so after a navigation its cached url/title go stale (or stay
// blank for a tab attached at about:blank) — which made `tab list` show
// blank rows you couldn't tell apart, defeating the point of listing them
// to pick a tab to adopt (issue #21). `Target.getTargetInfo` is a plain
// CDP read (no Runtime fingerprint), one cheap call per tab.
let sessions: Vec<(usize, String)> = self
.pages
.iter()
.enumerate()
.map(|(i, p)| (i, p.session_id.clone()))
.collect();
for (i, sid) in sessions {
if sid.is_empty() {
continue;
}
if let Ok(resp) = self
.client
.send_command("Target.getTargetInfo", None, Some(&sid))
.await
{
if let Some(ti) = resp.get("targetInfo") {
if let Some(page) = self.pages.get_mut(i) {
if let Some(u) = ti.get("url").and_then(|v| v.as_str()) {
if !u.is_empty() {
page.url = u.to_string();
}
}
if let Some(t) = ti.get("title").and_then(|v| v.as_str()) {
page.title = t.to_string();
}
}
}
}
}
Ok(())
}
/// If `--reuse-tab` and a tracked tab already shows `url`, switch to it
/// (without reloading, so any in-page state survives) and return its info.
/// Returns `None` when no tab matches and the caller should navigate/create.
/// Matches on exact URL or the same origin+path (ignoring query/fragment) so
/// a re-`open` of a stable entry URL lands on the existing tab instead of
/// piling up duplicates (issue #21).
pub async fn reuse_tab_for_url(&mut self, url: &str) -> Result<Option<Value>, String> {
self.resync_targets().await.ok();
let want = normalize_url_for_match(url);
let tab_id = self
.pages
.iter()
.find(|p| !want.is_empty() && (p.url == url || normalize_url_for_match(&p.url) == want))
.map(|p| p.tab_id);
match tab_id {
Some(id) => Ok(Some(self.tab_switch_by_id(id).await?)),
None => Ok(None),
}
}
/// Resolve a user-supplied `TabRef` (either `t<N>` or a label) to the
/// stable numeric `tab_id`. Returns a teaching error for unknown tabs.
pub fn resolve_tab_ref(&self, tab_ref: &TabRef) -> Result<u32, String> {
@@ -2217,6 +2497,34 @@ mod tests {
}
}
// --- issue #21: --reuse-tab URL matching ignores query/fragment ---
#[test]
fn normalize_url_match_strips_query_and_fragment() {
// Two opens of the "same" SSO page differ only in volatile query/hash —
// they must normalize equal so --reuse-tab lands on the existing tab.
let a = normalize_url_for_match(
"https://login.account.rakuten.com/sso/authorize?client_id=x&state=abc#/sign_in",
);
let b = normalize_url_for_match(
"https://login.account.rakuten.com/sso/authorize?client_id=y&state=zzz#/forgot",
);
assert_eq!(a, b);
assert_eq!(a, "https://login.account.rakuten.com/sso/authorize");
}
#[test]
fn normalize_url_match_distinguishes_different_paths() {
let cart = normalize_url_for_match("https://cart.step.rakuten.co.jp/cart");
let order = normalize_url_for_match("https://cart.step.rakuten.co.jp/order");
assert_ne!(cart, order);
}
#[test]
fn normalize_url_match_passes_through_unparseable() {
assert_eq!(normalize_url_for_match("not a url"), "not a url");
}
// --- issue #14: a pinned target must keep commands on the right tab ---
#[test]
@@ -2245,6 +2553,35 @@ mod tests {
assert_eq!(resolve_active_index(&pages, Some("CLOSED"), 1), 1);
}
// --- issue: `open` must not hijack a user's tab on the relay (dogfood) ---
#[test]
fn active_not_owned_when_only_user_tabs_discovered() {
// A fresh relay session passively attached to the user's tabs but created
// none — so navigate must NOT reuse the active tab (it'd clobber the
// user's page); it has to open its own first.
let pages = vec![page("USER_A"), page("USER_B")];
let created = HashSet::new();
assert!(!active_index_is_owned(&pages, Some("USER_A"), 0, &created));
}
#[test]
fn active_owned_when_session_created_the_tab() {
let pages = vec![page("USER_A"), page("OURS")];
let mut created = HashSet::new();
created.insert("OURS".to_string());
// Active pinned to the tab we created → safe to navigate it.
assert!(active_index_is_owned(&pages, Some("OURS"), 1, &created));
// But pinned to the user's tab → not owned, even though we own another.
assert!(!active_index_is_owned(&pages, Some("USER_A"), 0, &created));
}
#[test]
fn active_not_owned_when_no_pages() {
let created = HashSet::new();
assert!(!active_index_is_owned(&[], None, 0, &created));
}
#[test]
fn resolve_active_index_pin_survives_passive_background_tab() {
// A foreign tab ("Z") gets appended by passive discovery after we pinned
+249 -8
View File
@@ -796,16 +796,43 @@ pub(super) fn extract_ax_string(value: &Option<AXValue>) -> String {
/// Build a JS expression that finds a DOM element by CSS selector or XPath.
fn build_find_element_js(selector: &str) -> String {
if let Some(xpath) = selector.strip_prefix("xpath=") {
format!(
return format!(
"document.evaluate({}, document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue",
serde_json::to_string(xpath).unwrap_or_default()
)
} else {
format!(
"document.querySelector({})",
serde_json::to_string(selector).unwrap_or_default()
)
);
}
// Bare string (or explicit `text=`): try CSS first, then fall back to
// matching an interactive element by its VISIBLE TEXT. snapshot exposes
// buttons/links by their name, so `click "購入手続きへ"` should resolve by
// that label — previously it was fed straight to `querySelector` as CSS and
// failed as an invalid selector even though the button was right there
// (issue #24-B). CSS still wins when it matches, so existing selectors are
// unaffected; nested/non-ASCII labels now resolve too.
let text_only = selector.strip_prefix("text=");
let force_text = text_only.is_some();
let sel_json = serde_json::to_string(selector).unwrap_or_default();
let want_json = serde_json::to_string(text_only.unwrap_or(selector)).unwrap_or_default();
format!(
r#"(() => {{
const sel = {sel};
const css = {force_text} ? null : (() => {{ try {{ return document.querySelector(sel); }} catch (_e) {{ return null; }} }})();
if (css) return css;
const norm = s => (s == null ? '' : String(s)).replace(/\s+/g, ' ').trim();
const w = norm({want}); if (!w) return null;
const wl = w.toLowerCase();
const interactive = Array.from(document.querySelectorAll(
'button,a,[role=button],[role=link],[role=menuitem],[role=tab],[role=option],input[type=submit],input[type=button],input[type=reset],summary,label,[onclick]'));
const textOf = e => norm(e.innerText || e.textContent) || norm(e.value) ||
norm(e.getAttribute && e.getAttribute('aria-label')) || norm(e.getAttribute && e.getAttribute('title'));
let hit = interactive.find(e => textOf(e) === w) || interactive.find(e => textOf(e).toLowerCase().includes(wl));
if (hit) return hit;
const leaves = Array.from(document.querySelectorAll('*')).filter(e => !e.children.length);
return leaves.find(e => norm(e.textContent) === w) || leaves.find(e => norm(e.textContent).toLowerCase().includes(wl)) || null;
}})()"#,
sel = sel_json,
want = want_json,
force_text = force_text
)
}
/// Build a JS expression that counts matching DOM elements by CSS selector or XPath.
@@ -948,6 +975,200 @@ pub async fn get_element_text(
.unwrap_or_default())
}
/// Text content collected from a single frame of the page.
#[derive(Debug, Clone)]
pub struct FrameText {
pub frame_id: String,
pub url: String,
/// "top" | "inline" (same-process child frame) | "oopif" (out-of-process).
pub kind: &'static str,
pub text: String,
}
// The expression we run in every frame to read its visible text. innerText
// honors CSS visibility (skips display:none), textContent is the fallback.
const FRAME_INNERTEXT_JS: &str = "(function(){try{var b=document.body||document.documentElement;return b?(b.innerText||b.textContent||''):'';}catch(e){return '';}})()";
async fn eval_text_default(client: &CdpClient, session_id: &str) -> String {
let res = client
.send_command(
"Runtime.evaluate",
Some(serde_json::json!({
"expression": FRAME_INNERTEXT_JS,
"returnByValue": true,
})),
Some(session_id),
)
.await;
res.ok()
.and_then(|v| v.get("result").and_then(|r| r.get("value")).cloned())
.and_then(|v| v.as_str().map(|s| s.to_string()))
.unwrap_or_default()
}
// Same-process child frames share the top renderer but live in their own
// execution context. Page.createIsolatedWorld hands us a context id bound to
// that frame so Runtime.evaluate reads the child document, not the parent.
async fn eval_text_in_frame(client: &CdpClient, session_id: &str, frame_id: &str) -> String {
let ctx = client
.send_command(
"Page.createIsolatedWorld",
Some(serde_json::json!({ "frameId": frame_id, "worldName": "chrome_use_text" })),
Some(session_id),
)
.await
.ok()
.and_then(|v| v.get("executionContextId").and_then(|c| c.as_i64()));
let Some(ctx_id) = ctx else { return String::new() };
let res = client
.send_command(
"Runtime.evaluate",
Some(serde_json::json!({
"expression": FRAME_INNERTEXT_JS,
"returnByValue": true,
"contextId": ctx_id,
})),
Some(session_id),
)
.await;
res.ok()
.and_then(|v| v.get("result").and_then(|r| r.get("value")).cloned())
.and_then(|v| v.as_str().map(|s| s.to_string()))
.unwrap_or_default()
}
fn flatten_frame_tree(node: &Value, is_top: bool, out: &mut Vec<(String, String, bool)>) {
if let Some(frame) = node.get("frame") {
if let Some(id) = frame.get("id").and_then(|v| v.as_str()) {
let url = frame
.get("url")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string();
out.push((id.to_string(), url, is_top));
}
}
if let Some(children) = node.get("childFrames").and_then(|v| v.as_array()) {
for child in children {
flatten_frame_tree(child, false, out);
}
}
}
/// Collect visible text from every frame reachable in the active session,
/// including out-of-process iframes (which never appear in the top frame's
/// `Page.getFrameTree` and so are invisible to `document.body.innerText`).
///
/// Same-process child frames are read through `Page.createIsolatedWorld`;
/// OOPIFs are read through their own auto-attached debugger session
/// (`iframe_sessions`, keyed by frameId == targetId). This is the engine
/// behind `get text --all-frames` and `chrome-use frames` — the fix for
/// listing/marketplace pages whose description lives in a child frame (#27).
pub async fn collect_all_frames_text(
client: &CdpClient,
top_session: &str,
iframe_sessions: &HashMap<String, String>,
) -> Result<Vec<FrameText>, String> {
let mut out: Vec<FrameText> = Vec::new();
let mut seen: std::collections::HashSet<String> = std::collections::HashSet::new();
// 1. Top session: the top frame plus its same-process descendants. OOPIF
// frames that happen to surface here are skipped — they're read via
// their dedicated session in step 2 (cross-process isolated worlds fail).
let tree = client
.send_command_no_params("Page.getFrameTree", Some(top_session))
.await?;
let mut frames: Vec<(String, String, bool)> = Vec::new();
flatten_frame_tree(&tree["frameTree"], true, &mut frames);
for (fid, url, is_top) in frames {
if iframe_sessions.contains_key(&fid) {
continue;
}
if !seen.insert(fid.clone()) {
continue;
}
let (kind, text) = if is_top {
("top", eval_text_default(client, top_session).await)
} else {
("inline", eval_text_in_frame(client, top_session, &fid).await)
};
out.push(FrameText {
frame_id: fid,
url,
kind,
text,
});
}
// 2. Each out-of-process iframe, read through its own session.
for (fid, sid) in iframe_sessions {
if !seen.insert(fid.clone()) {
continue;
}
let url = client
.send_command_no_params("Page.getFrameTree", Some(sid))
.await
.ok()
.and_then(|t| {
t.get("frameTree")
.and_then(|ft| ft.get("frame"))
.and_then(|f| f.get("url"))
.and_then(|v| v.as_str())
.map(|s| s.to_string())
})
.unwrap_or_default();
let text = eval_text_default(client, sid).await;
out.push(FrameText {
frame_id: fid.clone(),
url,
kind: "oopif",
text,
});
}
Ok(out)
}
// Readability-lite: prefer the page's semantic main-content region over the
// whole body so global header/nav/footer chrome (and, on many listing pages,
// the "related items" sidebar) doesn't drown out the actual content. Runs on
// the live, rendered tree (innerText needs layout — a detached clone returns
// empty), so we pick the densest <main>/<article> region rather than cloning
// and stripping. Falls back to <body> when no substantial main region exists.
const MAIN_CONTENT_JS: &str = r#"(function(){
function txt(el){try{return (el.innerText||'').trim();}catch(e){return '';}}
var sels=['main','[role=main]','article','#main','#contents','#l-content'];
var best=null,bestLen=0;
for(var i=0;i<sels.length;i++){
var els=document.querySelectorAll(sels[i]);
for(var j=0;j<els.length;j++){var l=txt(els[j]).length;if(l>bestLen){bestLen=l;best=els[j];}}
}
if(best&&bestLen>200)return txt(best);
return txt(document.body);
})()"#;
/// Extract the page's main-content text (readability-lite), preferring a
/// semantic `<main>`/`<article>` region over the full body. Used by
/// `get text --main` to avoid header/nav/sidebar boilerplate (#27).
pub async fn get_main_content_text(client: &CdpClient, session_id: &str) -> Result<String, String> {
let res = client
.send_command(
"Runtime.evaluate",
Some(serde_json::json!({
"expression": MAIN_CONTENT_JS,
"returnByValue": true,
})),
Some(session_id),
)
.await?;
Ok(res
.get("result")
.and_then(|r| r.get("value"))
.and_then(|v| v.as_str())
.unwrap_or_default()
.to_string())
}
pub async fn get_element_attribute(
client: &CdpClient,
session_id: &str,
@@ -1452,10 +1673,30 @@ mod tests {
#[test]
fn test_build_selector_js_css() {
let js = build_selector_js("#submit-btn");
assert!(js.contains("document.querySelector(\"#submit-btn\")"));
// CSS is now tried via a `sel` variable, with a visible-text fallback
// appended (issue #24-B). It must still use querySelector (not xpath).
assert!(js.contains("const sel = \"#submit-btn\""));
assert!(js.contains("document.querySelector(sel)"));
assert!(!js.contains("document.evaluate"));
}
#[test]
fn test_build_find_element_js_text_fallback() {
// A bare label gets a text-matching fallback so `click "購入手続きへ"`
// resolves by visible text, not just CSS (issue #24-B).
let js = build_find_element_js("購入手続きへ");
assert!(js.contains("購入手続きへ"));
assert!(js.contains("interactive")); // the text-match branch
assert!(js.contains("textOf"));
// `text=` forces the text path (skips CSS).
let forced = build_find_element_js("text=Buy now");
assert!(forced.contains("true ? null")); // force_text => css skipped
// xpath is unchanged.
let xp = build_find_element_js("xpath=//button");
assert!(xp.contains("document.evaluate"));
assert!(!xp.contains("interactive"));
}
#[test]
fn test_build_selector_js_xpath() {
let js = build_selector_js("xpath=//button[@id='ok']");
+81 -32
View File
@@ -306,32 +306,50 @@ pub async fn fill(
)
.await?;
// Focus the element
client
.send_command_typed::<_, Value>(
"Runtime.callFunctionOn",
&CallFunctionOnParams {
function_declaration: "function() { this.focus(); }".to_string(),
object_id: Some(object_id.clone()),
arguments: None,
return_by_value: Some(true),
await_promise: Some(false),
},
Some(&effective_session_id),
)
.await?;
// Emulate a real edit so framework-controlled inputs (React/Vue) and
// site-side listeners actually see the change (issue #25): the old path set
// `this.value` directly and used Input.insertText, which left React's
// internal value-tracker out of sync and never fired change/blur — so
// dependent logic (e.g. Mercari's postal-code → 都道府県 autocomplete) never
// ran even though the value was visible. Set the value through the element's
// PROTOTYPE setter (which React's _valueTracker hooks), then dispatch
// input → change → blur/focusout. `type <sel> <text>` remains for sites that
// need per-keystroke events.
let fill_js = format!(
r#"function() {{
const el = this;
const v = {val};
try {{ el.focus(); }} catch (e) {{}}
const tag = el.tagName;
const fire = (type, ctor) => el.dispatchEvent(new (ctor || Event)(type, {{ bubbles: true }}));
if (tag === 'SELECT') {{
el.value = v; fire('input'); fire('change'); return true;
}}
if (el.isContentEditable) {{
el.textContent = v; fire('input', window.InputEvent || Event); fire('change');
try {{ el.blur(); }} catch (e) {{}} fire('focusout'); return true;
}}
const proto = tag === 'TEXTAREA' ? window.HTMLTextAreaElement.prototype
: window.HTMLInputElement.prototype;
const desc = Object.getOwnPropertyDescriptor(proto, 'value');
const set = desc && desc.set ? (x) => desc.set.call(el, x) : (x) => {{ el.value = x; }};
set(''); // reset the framework tracker
fire('input', window.InputEvent || Event);
set(v); // native setter → React/Vue registers
fire('input', window.InputEvent || Event);
fire('change');
try {{ el.blur(); }} catch (e) {{}}
fire('focusout'); // blur-triggered lookups/validation
return true;
}}"#,
val = serde_json::to_string(value).unwrap_or_default()
);
// Select all + delete to clear
client
.send_command_typed::<_, Value>(
"Runtime.callFunctionOn",
&CallFunctionOnParams {
function_declaration: r#"function() {
this.select && this.select();
this.value = '';
this.dispatchEvent(new Event('input', { bubbles: true }));
}"#
.to_string(),
function_declaration: fill_js,
object_id: Some(object_id),
arguments: None,
return_by_value: Some(true),
@@ -341,17 +359,6 @@ pub async fn fill(
)
.await?;
// Insert text (keyboard input dispatched at page level, use parent session_id)
client
.send_command_typed::<_, Value>(
"Input.insertText",
&InsertTextParams {
text: value.to_string(),
},
Some(session_id),
)
.await?;
Ok(())
}
@@ -557,6 +564,48 @@ pub async fn press_key_with_modifiers(
Ok(())
}
/// Dispatch a SINGLE key event (`keyDown` or `keyUp`) carrying the full key
/// descriptor — `key`, `code`, `windowsVirtualKeyCode`/`nativeVirtualKeyCode`,
/// and (on key-down) printable `text`. Powers the `keydown`/`keyup` commands.
///
/// The previous implementation sent only `{key}`, so games and shortcut handlers
/// that read `event.code` (e.g. `"KeyD"`, `"ArrowRight"`) or `event.keyCode` saw
/// nothing — a held key set no movement flag and did nothing (dogfood: holding a
/// direction in a canvas platformer barely nudged the player). Sending the same
/// descriptor `press` uses makes hold-to-move work regardless of which field the
/// page keys off.
pub async fn dispatch_single_key(
client: &CdpClient,
session_id: &str,
key: &str,
event_type: &str,
) -> Result<(), String> {
let (key_name, code, key_code) = named_key_info(key);
// Printable text is only meaningful on key-down; key-up never inserts.
let text = if event_type == "keyDown" {
key_text(&key_name)
} else {
None
};
client
.send_command_typed::<_, Value>(
"Input.dispatchKeyEvent",
&DispatchKeyEventParams {
event_type: event_type.to_string(),
key: Some(key_name),
code: Some(code),
text: text.clone(),
unmodified_text: text,
windows_virtual_key_code: Some(key_code),
native_virtual_key_code: Some(key_code),
modifiers: None,
},
Some(session_id),
)
.await?;
Ok(())
}
pub async fn scroll(
client: &CdpClient,
session_id: &str,
+140 -18
View File
@@ -186,6 +186,44 @@ pub fn print_response_with_opts(resp: &Response, action: Option<&str>, opts: &Ou
}
if let Some(data) = &resp.data {
// A click that opened a new tab: surface it so the agent doesn't read the
// unchanged old page as a failed click (issue #24-A).
if let Some(opened) = data.get("openedTab") {
let tid = opened.get("tabId").and_then(|v| v.as_str()).unwrap_or("?");
let url = opened.get("url").and_then(|v| v.as_str()).unwrap_or("");
let followed = data
.get("followed")
.and_then(|v| v.as_bool())
.unwrap_or(false);
let verb = if followed {
"switched to new tab"
} else {
"opened new tab"
};
eprintln!(
"{} {} [{}] {}",
color::cyan(""),
verb,
tid,
color::dim(url)
);
}
// `current`: the active tab's stable handle (#26).
if data
.get("current")
.and_then(|v| v.as_bool())
.unwrap_or(false)
{
let tid = data.get("tabId").and_then(|v| v.as_str()).unwrap_or("?");
let title = data.get("title").and_then(|v| v.as_str()).unwrap_or("");
let url = data.get("url").and_then(|v| v.as_str()).unwrap_or("");
let target = data.get("targetId").and_then(|v| v.as_str()).unwrap_or("");
println!("{} [{}] {} - {}", color::cyan(""), tid, title, url);
println!(" {}", color::dim(&format!("target: {}", target)));
return;
}
// Dialog status response
if action == Some("dialog") {
if let Some(has_dialog) = data.get("hasDialog").and_then(|v| v.as_bool()) {
@@ -297,6 +335,39 @@ pub fn print_response_with_opts(resp: &Response, action: Option<&str>, opts: &Ou
// Snapshot
if let Some(snapshot) = data.get("snapshot").and_then(|v| v.as_str()) {
print_with_boundaries(snapshot, origin, opts);
// Canvas-app hint: the tree was near-empty but the page paints to a
// <canvas>, so refs are a dead end — point at the screenshot path.
if let Some(note) = data.get("note").and_then(|v| v.as_str()) {
eprintln!("{}", color::dim(note));
}
return;
}
// Frame list (`chrome-use frames`)
if action == Some("frames") {
if let Some(list) = data.get("frames").and_then(|v| v.as_array()) {
let count = list.len();
println!(
"{}",
color::bold(&format!("{} frame{}", count, if count == 1 { "" } else { "s" }))
);
for f in list {
let idx = f.get("index").and_then(|v| v.as_i64()).unwrap_or(0);
let kind = f.get("kind").and_then(|v| v.as_str()).unwrap_or("?");
let url = f.get("url").and_then(|v| v.as_str()).unwrap_or("");
let len = f.get("textLen").and_then(|v| v.as_i64()).unwrap_or(0);
println!(
" [{}] {:<6} {} chars {}",
idx,
kind,
len,
color::dim(if url.is_empty() { "(about:blank)" } else { url })
);
}
eprintln!(
"{}",
color::dim("read everything with: chrome-use get text --all-frames")
);
}
return;
}
// Title
@@ -477,6 +548,9 @@ pub fn print_response_with_opts(resp: &Response, action: Option<&str>, opts: &Ou
}
// Tabs
if let Some(tabs) = data.get("tabs").and_then(|v| v.as_array()) {
// `tab list --full` prints untruncated URLs so a long SSO/redirect
// URL can actually be re-opened after a stale session (issue #19).
let full = data.get("full").and_then(|v| v.as_bool()).unwrap_or(false);
for tab in tabs {
let tab_id = tab.get("tabId").and_then(|v| v.as_str()).unwrap_or("?");
let tab_label = tab.get("label").and_then(|v| v.as_str());
@@ -491,8 +565,13 @@ pub fn print_response_with_opts(resp: &Response, action: Option<&str>, opts: &Ou
let title = title.as_str();
let url = tab.get("url").and_then(|v| v.as_str()).unwrap_or("");
// Truncate very long URLs (e.g. multi-KB JWT/OTP login links) so
// the list stays readable instead of flooding the terminal.
let url = truncate_middle(url, 120);
// the list stays readable instead of flooding the terminal
// unless `--full` was asked for (to re-open the exact URL).
let url = if full {
url.to_string()
} else {
truncate_middle(url, 120)
};
let active = tab.get("active").and_then(|v| v.as_bool()).unwrap_or(false);
let marker = if active {
color::cyan("")
@@ -504,25 +583,35 @@ pub fn print_response_with_opts(resp: &Response, action: Option<&str>, opts: &Ou
} else {
println!("{} [{}] {} - {}", marker, tab_id, title, url);
}
// `--full` also surfaces the stable cross-session CDP targetId so
// a stranded tab can be adopted from another session via
// `tab <targetId>` (issue #21).
if full {
if let Some(target_id) = tab.get("targetId").and_then(|v| v.as_str()) {
println!(" {}", color::dim(&format!("target: {}", target_id)));
}
}
}
return;
}
// Tab switch
if action == Some("tab_switch") {
if let Some(tab_id) = data.get("tabId").and_then(|v| v.as_str()) {
if let Some(url) = data.get("url").and_then(|v| v.as_str()) {
println!(
"{} Switched to tab [{}] ({})",
color::success_indicator(),
tab_id,
url
);
let warning = data.get("warning").and_then(|v| v.as_str());
// A non-responding session isn't a real success — show a warning
// indicator instead of the green ✓ (issue #29.3).
let indicator = if warning.is_some() {
color::warning_indicator()
} else {
println!(
"{} Switched to tab [{}]",
color::success_indicator(),
tab_id
);
color::success_indicator()
};
if let Some(url) = data.get("url").and_then(|v| v.as_str()) {
println!("{} Switched to tab [{}] ({})", indicator, tab_id, url);
} else {
println!("{} Switched to tab [{}]", indicator, tab_id);
}
if let Some(w) = warning {
eprintln!("{}", color::dim(w));
}
return;
}
@@ -1870,7 +1959,8 @@ Usage: chrome-use get <subcommand> [args]
Retrieves various types of information from elements or the page.
Subcommands:
text <selector> Get text content of element
text [selector] Element text; no selector = WHOLE PAGE, all frames
text --main Main-content text only (skip nav/header/sidebar)
html <selector> Get inner HTML of element
value <selector> Get value of input element
attr <selector> <name> Get attribute value
@@ -1886,7 +1976,10 @@ Global Options:
--session <name> Use specific session
Examples:
chrome-use get text @e1
chrome-use get text # whole page across ALL frames (default)
chrome-use get text @e1 # one element
chrome-use get text --main # main content, no nav/sidebar boilerplate
chrome-use frames # list frames + where the text lives
chrome-use get html "#content"
chrome-use get value "#email-input"
chrome-use get attr "#link" href
@@ -2757,6 +2850,20 @@ Notes:
- Streaming is always enabled. Set AGENT_BROWSER_STREAM_PORT to bind to a
specific port instead of the default OS-assigned port.
The WS is BIDIRECTIONAL the high-throughput way to drive a live/real-time page
(games, canvas apps) instead of one screenshot + one CLI call per action:
- Server -> client (JSON text frames):
{"type":"frame","data":"<base64 jpeg>"} live screencast (~60fps)
plus status / tabs messages.
- Client -> server (send JSON text):
{"type":"input_keyboard","eventType":"keyDown|keyUp","key":" ","code":"Space",
"windowsVirtualKeyCode":32}
{"type":"input_mouse","eventType":"mousePressed|mouseReleased|mouseMoved",
"x":640,"y":360,"button":"left","clickCount":1}
{"type":"input_touch","eventType":"touchStart|touchEnd","touchPoints":[...]}
Connect once and run a tight local loop: read frames, send timed input no
per-action process spawn, no round-trip. Works over the extension relay too.
Global Options:
--json Output as JSON
--session <name> Use specific session
@@ -3049,7 +3156,12 @@ Core Commands:
dblclick <sel> Double-click element
type <sel> <text> Type into element
fill <sel> <text> Clear and fill
press <key> Press key (Enter, Tab, Control+a)
press <key> [--hold <ms>] Press key (Enter, Tab, Control+a). --hold keeps it
down <ms> then releases precise (in-daemon), for
games/charge: `press d --hold 800`
keydown <key> Hold a key down (no auto-release) for games/shortcuts
keyup <key> Release a held key. Pair with keydown to hold-to-move:
`keydown d` `keyup d`
keyboard type <text> Type text with real keystrokes (no selector)
keyboard inserttext <text> Insert text without key events
hover <sel> Hover element
@@ -3077,6 +3189,7 @@ Navigation:
Get Info: chrome-use get <what> [selector]
text, html, value, attr <name>, title, url, count, box, styles, cdp-url
text (no selector = whole page, all frames), text --main, frames (list)
Check State: chrome-use is <what> <selector>
visible, enabled, checked
@@ -3108,7 +3221,12 @@ Storage:
storage <local|session> Manage web storage
Tabs:
tab [new|list|close|<n>] Manage tabs
tab [new|list|close|<ref>] Manage tabs (<ref> = t<N>, a label, or a CDP targetId)
tab list --full Full URLs + stable cross-session targetId per tab
tab <targetId> Adopt a specific tab (incl. another session's) by its
stable targetId, no reload preserves in-page state
open <url> --reuse-tab Reuse an existing tab on that URL instead of spawning
a duplicate (matches origin+path; preserves state)
Diff:
diff snapshot Compare current vs last snapshot
@@ -3170,6 +3288,10 @@ Confirmation:
Sessions:
session Show current session name
session list List active sessions
sessions List running session daemons (alias of daemon status)
daemon status List running session daemons (+ relay state)
daemon restart Kill all session daemons; keeps the extension relay
up. Clears stale/cross-leaked state after an upgrade.
Chat (AI):
chat <message> Send a natural language instruction (single-shot)
+161 -1
View File
@@ -1,5 +1,7 @@
use crate::color;
use std::process::{exit, Command};
use std::path::PathBuf;
use std::process::{exit, Command, Stdio};
use std::time::{SystemTime, UNIX_EPOCH};
const CURRENT_VERSION: &str = env!("CARGO_PKG_VERSION");
@@ -7,6 +9,164 @@ const CURRENT_VERSION: &str = env!("CARGO_PKG_VERSION");
/// upgrade path and the install path are identical (GitHub Release, no npm).
const INSTALL_URL: &str = "https://raw.githubusercontent.com/leeguooooo/chrome-use/main/install.sh";
/// GitHub API for the latest published release (used by the update check).
const LATEST_RELEASE_API: &str =
"https://api.github.com/repos/leeguooooo/chrome-use/releases/latest";
/// Re-check the latest version at most this often (seconds).
const UPDATE_CHECK_INTERVAL_SECS: u64 = 86_400; // once a day
fn now_secs() -> u64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0)
}
fn update_cache_path() -> PathBuf {
crate::connection::config_home().join("update-check.json")
}
fn write_update_cache(checked_at: u64, latest: &str) {
let path = update_cache_path();
if let Some(parent) = path.parent() {
let _ = std::fs::create_dir_all(parent);
}
let body = serde_json::json!({ "checked_at": checked_at, "latest": latest }).to_string();
let _ = std::fs::write(&path, body);
}
/// Parse a dotted version (`1.2.1`, `v1.2.1`, `1.2.1-fork.3`) into a comparable
/// `(major, minor, patch)`, ignoring any pre-release/build suffix.
fn parse_version(v: &str) -> Option<(u64, u64, u64)> {
let core = v.trim().trim_start_matches('v');
let core = core.split(['-', '+']).next().unwrap_or(core);
let mut parts = core.split('.');
let major = parts.next()?.parse().ok()?;
let minor = parts.next().unwrap_or("0").parse().ok()?;
let patch = parts.next().unwrap_or("0").parse().ok()?;
Some((major, minor, patch))
}
fn is_newer(latest: &str, current: &str) -> bool {
matches!((parse_version(latest), parse_version(current)), (Some(l), Some(c)) if l > c)
}
/// Public semver-ish comparison (`latest` strictly newer than `current`), so
/// `doctor` can flag a stale extension/CLI without re-implementing parsing.
pub fn version_is_newer(latest: &str, current: &str) -> bool {
is_newer(latest, current)
}
/// The latest CLI version recorded by the background update check, if any.
/// `doctor` uses it to show "a newer chrome-use is available" without a network
/// call (the `__update-check` worker refreshes the cache out of band).
pub fn cached_latest_version() -> Option<String> {
std::fs::read_to_string(update_cache_path())
.ok()
.and_then(|s| serde_json::from_str::<serde_json::Value>(&s).ok())
.and_then(|j| {
j.get("latest")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
})
.filter(|s| !s.is_empty())
}
/// Hidden `__update-check` subcommand: fetch the latest release tag and cache it.
/// Spawned detached by [`maybe_notify_update`] so the network call never blocks a
/// real command. Uses `curl` (no extra deps, matches `upgrade`).
pub fn run_update_check() {
let latest = Command::new("curl")
.args([
"-fsSL",
"--max-time",
"8",
"-H",
"User-Agent: chrome-use-update-check",
LATEST_RELEASE_API,
])
.output()
.ok()
.filter(|o| o.status.success())
.and_then(|o| serde_json::from_slice::<serde_json::Value>(&o.stdout).ok())
.and_then(|j| {
j.get("tag_name")
.and_then(|v| v.as_str())
.map(|s| s.trim_start_matches('v').to_string())
});
if let Some(latest) = latest {
write_update_cache(now_secs(), &latest);
}
}
/// Non-blocking "update available" notice. Called once per command run:
/// - prints a one-line hint to **stderr** (never stdout, so `--json` is clean)
/// when a cached release is newer than the running binary;
/// - refreshes the cached latest version at most once a day via a **detached**
/// background process, so the current command never waits on the network.
///
/// Skipped for meta commands (upgrade/install/doctor/`__*`/--version/--help),
/// in CI, in daemon mode, and when CHROME_USE_NO_UPDATE_CHECK /
/// AGENT_BROWSER_NO_UPDATE_CHECK is set.
pub fn maybe_notify_update() {
if std::env::var_os("CHROME_USE_NO_UPDATE_CHECK").is_some()
|| std::env::var_os("AGENT_BROWSER_NO_UPDATE_CHECK").is_some()
|| std::env::var_os("CI").is_some()
|| std::env::var_os("AGENT_BROWSER_DAEMON").is_some()
{
return;
}
let first = std::env::args().nth(1).unwrap_or_default();
if first.starts_with("__")
|| matches!(
first.as_str(),
"upgrade" | "install" | "doctor" | "dashboard" | "daemon"
)
{
return;
}
if std::env::args().any(|a| matches!(a.as_str(), "--version" | "-V" | "--help" | "-h")) {
return;
}
let (checked_at, latest) = std::fs::read_to_string(update_cache_path())
.ok()
.and_then(|s| serde_json::from_str::<serde_json::Value>(&s).ok())
.map(|j| {
(
j.get("checked_at").and_then(|v| v.as_u64()).unwrap_or(0),
j.get("latest")
.and_then(|v| v.as_str())
.unwrap_or_default()
.to_string(),
)
})
.unwrap_or((0, String::new()));
if is_newer(&latest, CURRENT_VERSION) {
eprintln!(
"{} chrome-use {latest} is available (you have {CURRENT_VERSION}) — run `chrome-use upgrade`",
color::warning_indicator()
);
}
// Refresh in the background at most once a day. Bump the timestamp first
// (keeping the last-known latest) so concurrent runs don't all spawn a
// checker, then fire a detached child that does the network fetch.
if now_secs().saturating_sub(checked_at) >= UPDATE_CHECK_INTERVAL_SECS {
write_update_cache(now_secs(), &latest);
if let Ok(exe) = std::env::current_exe() {
let _ = Command::new(exe)
.arg("__update-check")
.stdin(Stdio::null())
.stdout(Stdio::null())
.stderr(Stdio::null())
.spawn();
}
}
}
/// Upgrade to the latest GitHub Release.
///
/// The stealth fork ships as a prebuilt binary attached to a GitHub Release —
Binary file not shown.
Binary file not shown.
+152 -15
View File
@@ -30,6 +30,16 @@ const tabs = new Map()
const sessionToTab = new Map()
/** child (OOPIF/worker) sessionId -> tabId */
const childSessionToTab = new Map()
/** sessionId -> CDP targetId, kept ACROSS detach so a dead `cb-tab-<oldTabId>`
* session can be recovered by its stable targetId when the cross-process nav
* gave the tab a new Chrome tabId (issue #24). Capped to bound memory. */
const sessionTargets = new Map()
function rememberSessionTarget(sessionId, targetId) {
if (!sessionId || !targetId) return
sessionTargets.delete(sessionId)
sessionTargets.set(sessionId, targetId)
if (sessionTargets.size > 256) sessionTargets.delete(sessionTargets.keys().next().value)
}
/** tab-group name -> chrome tabGroups id (best-effort cache) */
const groupIdByName = new Map()
@@ -108,6 +118,12 @@ function connectHost() {
// reconnect. Keep chrome.debugger attached so reconnect is cheap.
for (const tabId of tabs.keys()) setBadge(tabId, 'connecting')
})
// Report our version so the host can tell the CLI/`doctor` which extension
// build is live (otherwise the extension version is a black box — the user
// can't tell they're on an old one). Best-effort; ignored by older hosts.
try {
postToHost({ method: 'hello', version: chrome.runtime.getManifest().version })
} catch {}
// Tell the daemon about everything we already have attached, then attach
// anything new.
reannounceAttachedTabs()
@@ -149,6 +165,90 @@ function tabForTarget(targetId) {
return null
}
// The STABLE Chrome tabId encoded in a `cb-tab-<tabId>` session id (#17), or
// null for any other session shape (child/iframe sessions). The tabId is the
// real source of truth: it survives the renderer-process swaps (cross-origin
// OAuth/SSO navs) that tear down the page's CDP target — which is why binding to
// it (like claude-in-chrome) rides through the hop that killed the old
// target/sessionId binding (issue #23).
function tabIdFromSession(sessionId) {
const m = /^cb-tab-(\d+)$/.exec(sessionId || '')
return m ? Number(m[1]) : null
}
// Ensure the debugger is attached to a `cb-tab-<tabId>` session's tab, re-attaching
// across the transient window of a process swap (with a couple of short retries).
// Returns the tabId on success, or null when the tab is genuinely gone
// (closed / restricted). (issues #20.1, #23)
async function recoverSessionTab(sessionId) {
const tabId = tabIdFromSession(sessionId)
// 1) Fast path: the encoded Chrome tabId still exists — re-attach it (covers
// the common renderer-process swap where the tabId is preserved, #23).
if (tabId != null) {
for (let i = 0; i < 3; i++) {
const tab = await chrome.tabs.get(tabId).catch(() => null)
if (!eligible(tab)) break // tabId is gone — fall through to targetId recovery
try {
await attachTab(tabId)
if (tabs.has(tabId)) return tabId
} catch {
// mid-swap: tab exists but isn't attachable yet — back off and retry.
}
await new Promise((r) => setTimeout(r, 120 + i * 150))
}
}
// 2) The Chrome tabId is gone, but the CDP targetId is STABLE across the nav.
// Some cross-process hops (Mercari's signin token exchange) give the tab a
// NEW tabId while keeping the same target, so `cb-tab-<oldTabId>` can't be
// recovered by tabId. Find the tab now hosting our remembered targetId via
// chrome.debugger.getTargets(), attach it, and ALIAS the dead session to it
// so the daemon's session id keeps resolving. Longer window: this hop can
// take several seconds to settle (issue #24).
const targetId = sessionTargets.get(sessionId)
if (targetId) {
for (let i = 0; i < 6; i++) {
const targets = await chrome.debugger.getTargets().catch(() => null)
const t = targets && targets.find((x) => x.id === targetId && x.tabId != null)
if (t && t.tabId != null) {
const tab = await chrome.tabs.get(t.tabId).catch(() => null)
if (eligible(tab)) {
try {
await attachTab(t.tabId)
if (tabs.has(t.tabId)) {
sessionToTab.set(sessionId, t.tabId) // alias dead session -> live tab
return t.tabId
}
} catch {
// not attachable yet — keep waiting for the swap to settle.
}
}
}
await new Promise((r) => setTimeout(r, 300 + i * 300))
}
}
return null
}
// Send a CDP command to a tab, riding a debugger detach that can happen between
// our attach check and the command itself (a renderer-process swap mid-flight).
// On a detached-style failure, drop the stale handle, re-attach the stable tab,
// and retry once — so a cross-process nav never surfaces as a hard error (#23).
async function sendCdpToTab(tabId, method, params) {
const dbg = { tabId }
try {
return await chrome.debugger.sendCommand(dbg, method, params)
} catch (e) {
const msg = String((e && e.message) || e)
if (!/detached|not attached|target.*(closed|gone)|no target|cannot access|frame.*detached/i.test(msg)) {
throw e
}
detachTab(tabId, false)
const ok = await recoverSessionTab(`cb-tab-${tabId}`)
if (!ok) throw e
return await chrome.debugger.sendCommand(dbg, method, params)
}
}
function anyConnectedTab() {
const it = tabs.keys().next()
return it.done ? null : it.value
@@ -207,13 +307,26 @@ async function handleForwardCdpCommand(msg) {
// Fail loudly instead so the agent sees an actionable error, not bad data.
let tabId
if (sessionId) {
tabId = tabForSession(sessionId)
if (!tabId) {
throw new Error(
`stale sessionId ${sessionId} for ${method}: its tab is gone (closed, ` +
`navigated across processes, or lost after an extension restart). ` +
`Re-attach by re-opening your target URL before retrying.`,
)
// The stable Chrome tabId encoded in `cb-tab-<tabId>` is the source of truth
// (it survives renderer-process swaps; the CDP target/sessionId does not).
// Resolve via it primarily — don't depend on a session→tab map entry that the
// detach handler may have cleared — and ensure the debugger is attached,
// re-attaching across a cross-process nav before failing (issues #20.1, #23).
// `tabForSession` still covers child/iframe sessions that aren't `cb-tab-*`.
tabId = tabIdFromSession(sessionId) ?? tabForSession(sessionId)
if (tabId == null) {
throw new Error(`unknown sessionId ${sessionId} for ${method}`)
}
if (!tabs.has(tabId)) {
const recovered = await recoverSessionTab(sessionId)
if (!recovered) {
throw new Error(
`stale sessionId ${sessionId} for ${method}: its tab is gone (closed, ` +
`navigated across processes, or lost after an extension restart). ` +
`Re-attach by re-opening your target URL before retrying.`,
)
}
tabId = recovered
}
} else if (typeof params?.targetId === 'string') {
tabId = tabForTarget(params.targetId)
@@ -223,18 +336,17 @@ async function handleForwardCdpCommand(msg) {
// applies to any attached tab.
tabId = anyConnectedTab()
}
if (!tabId) throw new Error(`no attached tab for ${method}`)
const dbg = { tabId }
if (tabId == null) throw new Error(`no attached tab for ${method}`)
// Re-enabling Runtime can leave a stale state; bounce it (matches upstream).
if (method === 'Runtime.enable') {
try {
await chrome.debugger.sendCommand(dbg, 'Runtime.disable')
await sendCdpToTab(tabId, 'Runtime.disable', undefined)
await new Promise((r) => setTimeout(r, 30))
} catch {}
return await chrome.debugger.sendCommand(dbg, 'Runtime.enable', params)
return await sendCdpToTab(tabId, 'Runtime.enable', params)
}
return await chrome.debugger.sendCommand(dbg, method, params)
return await sendCdpToTab(tabId, method, params)
}
// ---- attach / detach ------------------------------------------------------
@@ -273,6 +385,7 @@ async function attachTab(tabId) {
const entry = { sessionId, targetId }
tabs.set(tabId, entry)
sessionToTab.set(sessionId, tabId)
rememberSessionTarget(sessionId, targetId)
setBadge(tabId, port ? 'on' : 'connecting')
postToHost({
method: 'forwardCDPEvent',
@@ -355,9 +468,33 @@ chrome.debugger.onEvent.addListener((source, method, params) =>
}),
)
chrome.debugger.onDetach.addListener((source) =>
void whenReady(() => {
if (source.tabId) detachTab(source.tabId, true)
chrome.debugger.onDetach.addListener((source, reason) =>
void whenReady(async () => {
const tabId = source.tabId
if (!tabId) return
detachTab(tabId, true)
// A cross-process navigation (e.g. an SSO redirect like
// login.account.rakuten.com that swaps the render process / spawns OOPIFs)
// detaches the debugger, but the TAB survives. Without re-attaching, the
// session goes permanently stale and even open/navigate fails — exactly the
// #19 follow-up. So proactively re-attach (the stable `cb-tab-<tabId>`
// session id then restores the daemon's binding). Don't fight a detach the
// user or DevTools initiated.
if (reason === 'canceled_by_user' || reason === 'replaced_with_devtools') return
if (!port) return
// The swapped-in process needs a moment to settle; retry with backoff.
for (let i = 0; i < 6; i++) {
await new Promise((r) => setTimeout(r, 250 + i * 200))
if (tabs.has(tabId)) return // already re-attached (e.g. via onUpdated)
const tab = await chrome.tabs.get(tabId).catch(() => null)
if (!tab || !eligible(tab)) return // tab gone or now a restricted page
try {
await attachTab(tabId)
return
} catch (e) {
console.warn(`ab-connect: reattach attempt ${i + 1} for tab ${tabId} failed:`, e)
}
}
}),
)
+1 -1
View File
@@ -1,7 +1,7 @@
{
"manifest_version": 3,
"name": "chrome-use",
"version": "0.4.4",
"version": "0.4.9",
"description": "Let chrome-use drive your logged-in Chrome \u2014 install once, no token, no per-use confirmation.",
"key": "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA6vQIyscGIPYPZdSpPwPL0+0gxUROyRgCpmvCSDoc8XUm4qm97VbKnD9Ijc1lV22lNWZtE78gaRjt6BeSfuMgnBymnhLKjN1gU6AI5QUU0mrJyeHdWKvrKQR5FmsM2A7Xr1ykE2SiiS8zNUS3Y/6O5l+Nva7wrVy6E4a2dkBVQkOsu+DV+nEZvhIyuDY5D5SPXqNwUTWTaglwj5mjvHz36xSwCWlPmrtJ+ED0AUyrb2z4GIOmvk4kqtBVrh/UD058klLo4CkYOnIybB5aV6WYuwarfPY4bF/dLggPem+ewLNTUNBuwrxj/A4nUv0LJTuRO8rR7f8WR9qnRCY0Ic5saQIDAQAB",
"icons": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "chrome-use",
"version": "1.2.1",
"version": "1.5.3",
"description": "chrome-use — drive your real, logged-in Chrome from any AI agent, stealth by default",
"type": "module",
"packageManager": "pnpm@11.1.3",
+115 -2
View File
@@ -207,7 +207,10 @@ assigned fresh on every snapshot.
For unstructured reading (no refs needed):
```bash
chrome-use get text @e1 # visible text of an element
chrome-use get text # WHOLE PAGE — all frames by default (see below)
chrome-use get text @e1 # visible text of one element (or a CSS selector)
chrome-use get text --main # main content only — skip nav/header/sidebar
chrome-use frames # list every frame + where the text lives
chrome-use get html @e1 # innerHTML
chrome-use get attr @e1 href # any attribute
chrome-use get value @e1 # input value
@@ -216,6 +219,21 @@ chrome-use get url # current URL
chrome-use get count ".item" # count matching elements
```
**Whole-page text is cross-frame by default.** `chrome-use get text` with no
selector aggregates visible text across **every** frame — top document plus
same-process child frames plus cross-origin iframes — so you never silently miss
content that lives in an iframe (Yahoo Auctions / Rakuten / Mercari shop
descriptions, embedded checkout/spec frames). Each child frame is delimited with
a `----- frame [kind] url -----` marker. You do **not** need to remember a flag —
the default already reads all frames. (`--all-frames` is still accepted as an
explicit alias.)
So: when text looks missing or wrong, you don't have to guess — just
`chrome-use get text` reads everything. To **see** the structure (which frame
holds what), run `chrome-use frames`. To **cut boilerplate** (global nav/header/
footer, "related items" sidebars), use `chrome-use get text --main`. If content
is lazy-loaded, `scroll` it into view first, then read.
## Interacting
```bash
@@ -226,8 +244,11 @@ chrome-use hover @e1 # hover
chrome-use focus @e1 # focus (useful before keyboard input)
chrome-use fill @e2 "hello" # clear then type
chrome-use type @e2 " world" # type without clearing
chrome-use press Enter # press a key at current focus
chrome-use press Enter # press a key at current focus (down+up)
chrome-use press Control+a # key combination
chrome-use keydown d # HOLD a key down (no auto-release)
chrome-use keyup d # release it — pair them to hold-to-move
# in a game: `keydown d; sleep; keyup d`
chrome-use check @e3 # check checkbox
chrome-use uncheck @e3 # uncheck
chrome-use select @e4 "option-value" # native <select> only
@@ -296,6 +317,54 @@ chrome-use click --coords 449,320 # same, explicit flag
A bare-number argument is always a coordinate, never a selector.
### Canvas / WebGL apps (games, map & 3D viewers, drawing tools)
These paint everything to a `<canvas>` and expose **almost no accessibility
tree**, so `snapshot` comes back near-empty and refs are a dead end. `snapshot`
detects this and prints a one-line hint. Drive them the screenshot way:
```bash
chrome-use screenshot /tmp/s.png # SEE the state (your only read path —
# eval/get text return nothing useful)
chrome-use click 640 360 # interact by viewport coordinate
chrome-use press d --hold 800 # hold-to-move, precise (timed in-daemon —
# NOT keydown+shell-sleep+keyup, which
# adds ~250ms jitter per round-trip)
chrome-use press Space # discrete actions (jump/attack/confirm)
```
**Don't drive frame-by-frame with one CLI call per action** — that's the slowest,
lowest-fidelity way (each call is a process spawn + round-trip). Script a *timed
sequence in a single round-trip* with `batch` (it sends each step to the running
daemon; `press --hold` and `wait` block in-daemon, so timing is precise):
```bash
chrome-use batch "press d --hold 900" "press j" "press j" "wait 200" "press d --hold 500"
```
Also try reading real state instead of pixels: `eval` runs in the page's main
world, so for a framework/engine game you can often reach its globals (e.g. a
Phaser/PIXI/Three instance, a store, `window.__GAME__`) and read positions/score
directly — far better than guessing from a screenshot.
**For genuinely real-time driving, drop the CLI entirely and use the WebSocket.**
`chrome-use stream enable` opens a bidirectional WS (`stream status` prints the
`ws://127.0.0.1:<port>`). Connect once and you get a live ~60fps screencast AND
can send input on the same socket — no per-action process spawn, no round-trip,
works over the extension relay:
```js
// node (global WebSocket): live frames + locally-timed input
const ws = new WebSocket("ws://127.0.0.1:PORT")
ws.onmessage = e => { const m = JSON.parse(e.data); if (m.type==="frame") {/* base64 jpeg */} }
const k = (eventType,key,code,vk) => ws.send(JSON.stringify({type:"input_keyboard",eventType,key,code,windowsVirtualKeyCode:vk}))
k("keyDown"," ","Space",32); setTimeout(()=>k("keyUp"," ","Space",32), 80) // a jump
// also: {type:"input_mouse",eventType:"mousePressed",x,y,button:"left",clickCount:1}
```
This is the difference between watching a slideshow and playing the game. Reserve
screenshots for one-off checks; use the WS for any sustained real-time control.
## Waiting (read this)
Agents fail more often from bad waits than from bad selectors. Pick the
@@ -501,6 +570,43 @@ the same browser's existing targets, so a second session's first `open` can
navigate a sibling's tab. For concurrent agents on one real Chrome, use the
extension (each with a distinct `--session`), not raw `--cdp`.
Each session owns its own tab group and assigns its own `t<N>` indices (the same
physical tab is `t8` in one session, `t1` in another), so `t<N>` is **not** a
stable cross-session handle. To reach a *specific* tab from another session — e.g.
a tab that was filled in a session whose handle later died — use the **stable CDP
`targetId`**:
```bash
chrome-use tab list --full --session B # re-syncs live tabs; prints `target: <id>` per row
chrome-use tab <targetId> --session B # adopt that exact tab, NO reload (state preserved)
```
`tab list` re-discovers the live tab set on every call, so a fresh session sees
tabs other sessions opened (and re-attached ones), not just its own. Adopting by
`targetId` lands session B on the stranded tab without reloading it, so a
half-filled form survives. Still, the simplest recovery for a session whose own
tab died is to recover *that* session (reload / re-`open` / `daemon restart`).
To avoid piling up duplicate tabs when you re-`open` the same entry URL on
rebind, pass **`--reuse-tab`**: if a tab already shows that URL (matched by
origin+path), it switches to it instead of spawning a new one.
### Reset stuck daemon state
Each session runs a background daemon worker that holds the page handles. If a
session starts misbehaving — commands hit the wrong tab, refs/handles look stale,
or you upgraded `chrome-use` mid-session and old workers linger — restart the
daemons instead of hunting PIDs with `pgrep`/`kill`:
```bash
chrome-use daemon status # list running session daemons (+ relay state)
chrome-use daemon restart # kill every session daemon worker
```
`daemon restart` leaves the extension's native-messaging bridge (`__nm-host`)
alone, so the relay to your live Chrome stays up — the next command just spins up
a fresh, clean daemon against the same browser. It does **not** close any tabs.
### Mock network requests
```bash
@@ -610,6 +716,13 @@ forbids debugging). The session no longer has a live tab — re-run
replaces the old silent behaviour where the command ran on some *other*
tab and returned wrong data.
To recover, you need the tab's **exact** URL (query params and all — a long
SSO/redirect link breaks if truncated). `tab list` shortens long URLs with
`…`; use **`tab list --full`** to print them untruncated, then re-`open` the
right one. For multi-redirect SSO flows, re-open the **stable entry URL**
(not the mid-redirect one) and `wait` a few seconds for the SPA to settle
before snapshotting.
**Reads landing on the wrong page**
`eval`, `screenshot`, and `network requests` print the page they ran
against to stderr: `eval @ <url>`, `screenshot @ <url>`, `network @ <url>`.