fix(relay): reach cross-origin iframes; auto-reattach open (#35, #36)

#35: `open` auto-reattaches when the bound relay tab is gone — drops the dead
page, opens a fresh tab in the session's group, and navigates it, instead of
only `tab new` recovering.

#36: scroll and click now reach content inside cross-origin OOPIFs:
- scroll dispatches a real wheel at a viewport point (default center, --at x,y,
  or --frame n) so it scrolls the iframe under the pointer, which
  window.scrollBy on the top document silently no-ops on.
- over the extension relay, clicks always use DOM-dispatch instead of
  coordinate Input events — a coordinate event isn't confined to the target tab
  on a busy real Chrome (it drifted onto the foreground tab) and an OOPIF
  element's box can't be mapped to a top-viewport point.
This commit is contained in:
leeguooooo
2026-06-16 18:05:06 +09:00
parent cd47ec43d0
commit 6830df50ea
8 changed files with 372 additions and 18 deletions
+14 -1
View File
@@ -1735,12 +1735,23 @@ Usage: chrome-use scroll [direction] [amount] [options]
Scrolls the page or a specific element in the specified direction.
Without --selector, scroll dispatches a real (isTrusted) mouse wheel at a
viewport coordinate, so it scrolls whatever container is under the pointer —
including cross-origin iframes (Google Payments, Stripe, embedded checkout/KYC)
that plain page scroll can't reach.
Arguments:
direction up, down, left, right (default: down)
amount Pixels to scroll (default: 300)
Options:
-s, --selector <sel> CSS selector for a scrollable container
-s, --selector <sel> CSS selector for a scrollable container (same-origin)
--at <x,y> Dispatch the wheel at this viewport pixel (read it from a
screenshot) — precise way into a cross-origin iframe
--frame <n> Scroll the n-th frame from `chrome-use frames` (wheel at
that frame's center)
Without --selector/--at/--frame the wheel lands at the viewport center.
Global Options:
--json Output as JSON
@@ -1752,6 +1763,8 @@ Examples:
chrome-use scroll up 200
chrome-use scroll left 100
chrome-use scroll down 500 --selector "div.scroll-container"
chrome-use scroll down 700 --at 640,400 # wheel at a pixel over an iframe
chrome-use scroll down 700 --frame 2 # scroll frame 2 from `frames`
"##
}
"scrollintoview" | "scrollinto" => {