feat: rich-editor fill, box centers, screenshot downscale, disabled+docs (#41-#45)
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
Dogfooding backlog from this session's embedded-form/editor work. #41 fill on rich editors: detect CodeMirror 5 / Monaco / ProseMirror / contenteditable and set via their own API or execCommand('insertText') so beforeinput/input fire (a raw .value/textContent write no-op'd juejin's CodeMirror and skipped React composers). Response echoes the `engine` used. `fill <sel> --file <path>` / `--stdin` set large multiline text without shell-escaping. `get value` now reads CodeMirror/Monaco/contenteditable too. #42 screenshot --max-width/--max-height/--scale, plus a default 2000px longest-edge cap (AGENT_BROWSER_SCREENSHOT_MAX_EDGE; 0 disables) so retina full-page shots fit an agent's image reader and --scale 0.5 makes screenshot px line up with click px. Annotated shots are never downscaled. #43 `box @ref` (already a top-level alias of `get box`) now also returns centerX/centerY/inViewport in CSS px — feed straight into `click x y` when a ref-click no-ops (e.g. a button in a cross-origin iframe). #44 no code change needed — disabled elements already list as `button "Save" [disabled, ref=eN]`; the reporter's missing button was DOM-gated on validity. Added a skill note: `find text` can't reach into a cross-origin iframe — target those by snapshot @ref. #45 core skill now distinguishes screenshot-to-locate (discouraged) from screenshot-to-capture a reusable image asset via `screenshot [--clip] <file>` (encouraged), so agents stop over-reading the prohibition. #40 (group-scoped relay) stays deferred — needs an ab-connect extension change. Verified live: fill --file round-trips multiline+CJK+backticks; contenteditable engine=contenteditable + get value reads it back; box gives centerX/centerY/ inViewport; screenshot of retina example.com → 2000px; disabled button shows [disabled]. 856 tests pass.
This commit is contained in:
@@ -59,6 +59,18 @@ next ref interaction.
|
||||
> anyway.) Driving off pixels on the relay also risks a coordinate event drifting
|
||||
> onto the user's foreground tab — refs never do. See issue #37.
|
||||
|
||||
> **Two different intents — only one is discouraged.** The rule above is about
|
||||
> *screenshot-to-locate* (using a picture to find/hit an element) — that's the bug.
|
||||
> *screenshot-to-capture* — saving a region or element to a file as a **reusable
|
||||
> image asset** (maps, charts, og-images, visual-diff baselines, report figures) —
|
||||
> is fully supported and encouraged: `screenshot [selector] [--clip x,y,w,h] <file>`.
|
||||
> Capturing a rendered map region to a PNG for a blog post is the right tool, not a
|
||||
> smell. Screenshots are auto-downscaled to ≤2000px (longest edge) so they fit an
|
||||
> image reader and their pixels line up with `click x y`; override with
|
||||
> `--max-width`/`--max-height`/`--scale`. To click something you couldn't hit by
|
||||
> ref, `box @ref` gives the element's CSS-px box + `centerX/centerY` to feed
|
||||
> straight into `click <centerX> <centerY>` — no screenshot needed.
|
||||
|
||||
## Before you automate: pick the cheapest tool
|
||||
|
||||
Driving a browser is the heavy option. chrome-use earns its keep when you
|
||||
@@ -373,6 +385,12 @@ foreground, so prefer refs. For below-the-fold content in such a frame, scroll i
|
||||
with `scroll down N --at x,y` (a pixel over the frame) or `--frame n`. For a
|
||||
postal/autocomplete box inside the frame, `type @e "…" --key-events`.
|
||||
|
||||
> **Caveat: `find text "…"` can't reach into a cross-origin iframe** — it errors
|
||||
> "Element not found" even though `snapshot -i` lists those nodes and
|
||||
> `get text` reads them. Inside cross-origin iframes, target elements by their
|
||||
> **snapshot `@ref`**, not by `find`. (`box @ref` also works on iframe refs when
|
||||
> you need a coordinate fallback.)
|
||||
|
||||
### When refs don't work or you don't want to snapshot
|
||||
|
||||
Use semantic locators:
|
||||
|
||||
Reference in New Issue
Block a user