Compare commits

...
11 Commits
Author SHA1 Message Date
leeguooooo d81bc01645 feat(site): bb-sites adapters — turn any site into a structured-data CLI
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
Add `chrome-use site` — run community bb-sites adapters over chrome-use's
stealth transport. An adapter is a per-command JS function that calls a
site's own JSON API from inside your logged-in tab (your cookies, same-origin
fetch, the site's modules) and returns clean JSON — no clicking/scraping.

- site update   fetch the upstream bb-sites pack into ~/.chrome-use/sites
- site list     list installed adapters (name/cmd)
- site info     show an adapter's @meta (args, domain, capabilities)
- site <name>/<cmd> [args]  navigate to its domain (reuse tab if already there) + eval, return JSON

chrome-use ships zero adapter code; `site update` fetches epiral/bb-sites at
runtime (like a package manager). Adapters remain their authors' property.

cli/src/site.rs (load/parse/build_eval/list/update/map_args + tests), wired
via commands.rs (parse), actions.rs (handle_site), main.rs (CLI dispatch).
Docs in README, README.zh, skill-data/core. Verified live: github/issues
returned 30 real issues as JSON over the relay.
2026-06-17 16:02:40 +09:00
leeguooooo c667e0e704 docs(zh): bring Chinese README to parity with English
Translate the sections the zh README was missing: 为什么用扩展 (extension vs raw
debug port + comparison table), 命令名 (chrome-use/abs same binary), 自动化测试
(chrome-use test YAML suite), 自己验证 (verify-yourself detector list), 调参 (env
var table); fill out 反检测 (0% stealth explanation + --launch CreepJS caveat),
the install 'other ways' details, and the blog/X footer. All 12 top-level sections
now match EN.
2026-06-17 14:49:52 +09:00
leeguooooo 08cb8dbeb9 docs: link blog + X in README 2026-06-17 14:15:52 +09:00
leeguooooo fd2cdcde77 chore(release): 1.5.18 — issue-reporting guidance + clear data:-over-relay error
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
2026-06-17 14:10:07 +09:00
leeguooooo 8e001e3d88 feat(dx): guide users to file issues; clear error for data: URLs over the relay
#5: the CLI never pointed users at the issue tracker. `--help` and `--version`
now print the issues URL (https://github.com/leeguooooo/chrome-use/issues), so
agents hitting a rough edge know where to report it (the skill already nudges).

#1: navigating to a `data:` URL over the extension relay fails with a cryptic
`net::ERR_ABORTED` on an about:blank tab (chrome.debugger/chrome.tabs can't drive
a top-frame data: navigation; it works fine under --launch). Detect that case and
explain it — use a real http(s)://file:// URL or --launch — instead of leaking
the raw code. (Surfaced while stress-testing 5 concurrent agents.)
2026-06-17 14:10:04 +09:00
leeguooooo eb2bc343a0 chore(release): 1.5.17 — complete multi-agent isolation (no foreign tab/pop-up adoption on click)
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
2026-06-17 13:31:50 +09:00
leeguooooo d86c9c4be2 fix(relay): don't adopt foreign tabs/pop-ups on click — completes multi-agent isolation (#last hole)
After strict isolation (v1.5.16) a session's tracked set is only its OWN tabs, so
in `adopt_newly_opened` (run after every click to follow a pop-up) EVERY foreign
tab looks "new" relative to the session's `before` set and got adopted — a
click-heavy flow on a busy shared Chrome pulled other agents'/the user's tabs
(github, Lark, iphone-use) into the session mid-flow.

A pop-up the agent itself opened can't be told apart from a foreign tab over the
relay (the synthesized targetInfo carries no opener/window/group), so on the relay
`adopt_newly_opened` now adopts nothing: the agent drives only tabs it explicitly
created; pop-ups (OAuth/login windows) are the user's. Launched browsers (every
tab ours) still follow pop-ups. Verified live: a fresh click-heavy relay session
stays clean (only its own tabs), and 5 concurrent agents churning tabs show zero
cross-agent drift.
2026-06-17 13:31:49 +09:00
leeguooooo 32c25a6627 chore(release): 1.5.16 — strict multi-agent tab isolation on the relay
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
2026-06-17 11:50:10 +09:00
leeguooooo a8ce3dd3f8 fix(relay): strict multi-agent isolation — a session owns only its own tabs
Several agents (and other tools opening tabs) share one real Chrome via the
relay. Previously every session adopted ALL tabs from Target.getTargets, so
another agent's tab churn polluted the list, dropped the tab being driven, and
drifted commands onto the wrong page (the W-8BEN tax tab vanished mid-flow when a
concurrent iphone-use agent opened tabs).

A tab group belongs to exactly one agent. On the relay a session now tracks and
drives ONLY the tabs it created (its own colored group) plus pop-ups its own
clicks open — it never adopts the user's or other agents' tabs:

- discover_and_attach (relay): create the session's own tab and pin it; do not
  adopt any existing foreign tab.
- resync_targets (relay): never adopt unknown targets; never prune the session's
  tabs on a single getTargets snapshot (multi-agent churn / cross-process-nav
  gaps routinely omit live tabs) — prune only after RELAY_PRUNE_MISSES
  consecutive absent snapshots (debounced), pinned active always protected.
- adopt_newly_opened: a tab that appears right after this session's action is a
  pop-up we opened — record it as owned.

Launched browsers (every tab ours) keep adopting all tabs. Adds debounced_prune_ids
+ unit tests for the churn tolerance.
2026-06-17 11:50:09 +09:00
leeguooooo 997373fd57 chore(release): 1.5.15 — trusted activation for in-iframe buttons (#39)
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
2026-06-17 10:16:17 +09:00
leeguooooo 5a858af93f fix(iframe): trusted activation for in-iframe buttons — keyboard, not synthetic click (#39)
A DOM `.click()` is isTrusted:false, which security-sensitive embedded forms
reject — Google Payments' enabled `保存` button silently no-op'd, so a
cross-origin payment/checkout/KYC form could be read, scrolled, and typed into
but never submitted. A coordinate click can't help either: getBoxModel for a
sub-frame node returns frame-local coords that don't compose the iframe offset,
so it lands wrong (verified — the same-origin probe came back isTrusted:false
via the coordinate fallback).

Fix: click on an in-iframe ref now focuses the element in its own frame session
and dispatches a real Enter (Space for checkbox-like roles) on the page session.
Chrome routes the key to the focused element across frames (same mechanism as
`type --focused`), and Enter/Space on a focused button/link/checkbox fires a
trusted click. Non-activatable roles fall back to DOM .click().

Adds e2e_iframe_button_click_is_trusted (+ fixture): an in-iframe button records
event.isTrusted into its own text; the test asserts the ref-click delivers
isTrusted:true.
2026-06-17 10:16:15 +09:00
15 changed files with 979 additions and 41 deletions
+31
View File
@@ -205,6 +205,34 @@ chrome-use --launch --profile auto open https://x.com/home
In CI environments, standalone mode is used automatically. In CI environments, standalone mode is used automatically.
## Site adapters — turn a website into a structured-data CLI
Most "read GitHub issues" / "search Reddit" / "get my Bilibili feed" tasks don't
need clicking and screenshotting at all — the site already has a JSON API behind
its own login. A **site adapter** is a tiny JS function that calls that API *from
inside your logged-in tab* (your cookies, same-origin `fetch`, the site's own
modules) and returns clean JSON. The site can't tell it apart from you, because it
*is* you.
chrome-use ships none of these adapters — `site update` fetches the community
[**bb-sites**](https://github.com/epiral/bb-sites) pack at runtime (like a package
manager pulling a dependency), then runs them over chrome-use's stealth transport:
```bash
chrome-use site update # fetch the adapter pack (~145 commands)
chrome-use site list # github/issues, reddit/search, bilibili/feed, …
chrome-use site info github/issues # see an adapter's args + domain
# Run one — navigates to the site (reusing the tab if you're already there) and returns JSON
chrome-use site github/issues epiral/bb-browser --json
chrome-use site reddit/search "rust async" --json
chrome-use site bilibili/feed --json # works because it's your logged-in session
```
Positional args fill the adapter's declared args in order; `--key value` overrides
by name. Adapters are authored by the bb-sites community and remain their authors'
property — chrome-use just runs them.
## Automated testing (`chrome-use test`) ## Automated testing (`chrome-use test`)
Turn the repetitive "open it, click around, check it's right" work into a Turn the repetitive "open it, click around, check it's right" work into a
@@ -326,3 +354,6 @@ We deliberately **don't ship our own bot detector** — the strongest, most hone
## License ## License
Apache-2.0 Apache-2.0
---
> Built by **leeguooooo** — field notes on AI agents, reverse engineering & Cloudflare Workers at **[blog.misonote.com](https://blog.misonote.com)** · follow on **[X @leeguooooo](https://x.com/leeguooooo)**
+124 -1
View File
@@ -63,6 +63,25 @@ chrome-use 让**任意** agentClaude Code、Cursor、Codex、你自己的脚
每个 `--session` 拿到**自己的彩色标签组**,多个 agent 共用同一个真实浏览器、互不干扰,也不动你自己的标签页。 每个 `--session` 拿到**自己的彩色标签组**,多个 agent 共用同一个真实浏览器、互不干扰,也不动你自己的标签页。
## 为什么用扩展(而非裸调试端口)
其他本地工具走裸 `--remote-debugging-port`CDP)驱动 Chrome。从 **Chrome 136** 起,每次这样连接都会弹出一个阻塞式的 **"Allow remote debugging?"** 同意框 —— 而且端口得提前开好。我们的扩展改用原生消息:**装一次,之后零确认。**
| | **chrome-use**(本扩展) | web-access(裸 CDP 端口) | Claude in Chromechrome.debugger |
|---|---|---|---|
| 连接方式 | 原生消息 —— 无端口、无 token | `--remote-debugging-port` | `chrome.debugger` |
| **"Allow remote debugging?" 弹框** | **从不** ✅ | **每次连都弹** 🔴 | 无 |
| 复用你的真实登录 | 是 | 是 | 是 |
| `Runtime.enable`CDP)泄漏¹ | **默认关闭 → 干净** ✅ | 域已启用 | 不适用 |
| CreepJS 隐身分² | **0% stealth · 0% headless** ✅ | 真实 Chrome | 真实 Chrome |
| 每会话标签组 / 并发 agent | **支持** ✅ | 无 | 无 |
| 为 chrome-use CLI 打造 | 是 | 独立代理 | 单 app 助手 |
> ¹ 对 [rebrowser-bot-detector](https://bot-detector.rebrowser.net/) 实测:我们的中继报 `runtimeEnableLeak: 🟢 No leak`、`navigatorWebdriver: 🟢`。
> ² 对 [CreepJS](https://abrahamjuliot.github.io/creepjs/) 在「连接真实 Chrome」路径上实测 —— 见 [反检测](#反检测)。
>
> 同意框不是假想:裸端口工具**每次** attach 都会弹(Chrome 136+ 安全策略)。扩展路径从不弹。
## 安装 ## 安装
```bash ```bash
@@ -71,6 +90,15 @@ curl -fsSL https://raw.githubusercontent.com/leeguooooo/chrome-use/main/install.
从最新的 [GitHub Release](https://github.com/leeguooooo/chrome-use/releases) 下载对应平台的预编译二进制,安装 `chrome-use`(以及 `abs` 别名)。无需 npm,无需 token。 从最新的 [GitHub Release](https://github.com/leeguooooo/chrome-use/releases) 下载对应平台的预编译二进制,安装 `chrome-use`(以及 `abs` 别名)。无需 npm,无需 token。
<details>
<summary>其他安装方式</summary>
- **锁定版本:** `AGENT_BROWSER_VERSION=v0.27.0-fork.12 curl -fsSL https://raw.githubusercontent.com/leeguooooo/chrome-use/main/install.sh | sh`
- **自定义路径:** `AGENT_BROWSER_BIN_DIR=$HOME/bin curl -fsSL … | sh`
- **Windows** 从 [Releases 页](https://github.com/leeguooooo/chrome-use/releases) 下载 `chrome-use-win32-x64.tar.gz`,把 `chrome-use.exe` 放进 PATH。
- **npm(旧渠道):** `npm install -g chrome-use` —— 仍在发布,但 GitHub Releases 现在是主渠道。
</details>
### 安装 AI agent skills ### 安装 AI agent skills
```bash ```bash
@@ -79,6 +107,10 @@ npx skills add leeguooooo/chrome-use
`skills/chrome-use` 拉进当前项目,让你的 AI agent 拿到正确的用法和预授权的 bash 权限。 `skills/chrome-use` 拉进当前项目,让你的 AI agent 拿到正确的用法和预授权的 bash 权限。
## 命令名
`chrome-use``chrome-use``abs` 是**同一个二进制** —— `abs` 只是短别名。没有单独的「隐身可执行文件」;隐身是**运行时行为**(见下方 [反检测](#反检测)),根据你是连接真实 Chrome 还是 `--launch` 全新实例自动启用。
## 连接你的 Chrome ## 连接你的 Chrome
**推荐 —— 浏览器扩展(一键,无弹窗)。** 从 Chrome 应用商店安装 [**chrome-use** 扩展](https://chromewebstore.google.com/detail/chrome-use/knfcmbamhjmaonkfnjhldjedeobeafmk),再注册一次本地桥: **推荐 —— 浏览器扩展(一键,无弹窗)。** 从 Chrome 应用商店安装 [**chrome-use** 扩展](https://chromewebstore.google.com/detail/chrome-use/knfcmbamhjmaonkfnjhldjedeobeafmk),再注册一次本地桥:
@@ -128,8 +160,69 @@ chrome-use --launch open https://example.com
# 保留登录:用你真实的 Chrome profile 启动 # 保留登录:用你真实的 Chrome profile 启动
chrome-use --launch --profile auto open https://x.com/home chrome-use --launch --profile auto open https://x.com/home
# 或显式指定:--profile Default / --profile "Profile 1"
``` ```
## 站点适配器 —— 把一个网站变成「结构化数据 CLI」
大多数「读 GitHub issue」「搜 Reddit」「拉我的 B 站动态」这类任务,根本不需要点击 +
截图 —— 网站登录态背后本来就有 JSON 接口。**站点适配器**就是一小段 JS 函数,它在你
**已登录的标签页内**调用那个接口(用你的 cookie、同源 `fetch`、网站自己的模块),返回
干净的 JSON。网站分辨不出它和你的区别,因为它**就是你**。
chrome-use 本身不附带任何适配器 —— `site update` 会在运行时拉取社区的
[**bb-sites**](https://github.com/epiral/bb-sites) 适配器包(就像包管理器拉依赖),
然后在 chrome-use 的隐身通道上运行它们:
```bash
chrome-use site update # 拉取适配器包(约 145 条命令)
chrome-use site list # github/issues、reddit/search、bilibili/feed…
chrome-use site info github/issues # 查看某个适配器的参数 + 域名
# 运行一个 —— 会导航到对应站点(已在该站点则复用当前标签页)并返回 JSON
chrome-use site github/issues epiral/bb-browser --json
chrome-use site reddit/search "rust async" --json
chrome-use site bilibili/feed --json # 能用,因为走的是你的登录态
```
位置参数按适配器声明的参数顺序填入;`--key value` 按名覆盖。适配器由 bb-sites 社区编写、
版权归各自作者所有 —— chrome-use 只负责运行它们。
## 自动化测试(`chrome-use test`
把反复的「打开它、点一圈、看对不对」变成**可重跑的测试套件** —— 前端的单元测试。用 YAML 写用例;步骤复用 chrome-use 自己的命令,断言编译成一次检查:
```yaml
# smoke.yaml
suite: chatgpt smoke
setup:
- account: chatgpt/huayue # 注入一个 cookie-use 登录(可选)
cases:
- name: home loads logged in
steps:
- open: https://chatgpt.com/
- wait: { load: networkidle }
assert:
- url: { contains: chatgpt.com }
- visible: "#prompt-textarea"
```
```bash
chrome-use test smoke.yaml # 启动隔离浏览器,跑用例
chrome-use test smoke.yaml --session default # …或对你已连接的 Chrome 跑
```
```
suite: chatgpt smoke (session cu-test)
✓ home loads logged in 1.2s
✗ composer takes text 0.8s
assert text "#prompt-textarea" contains "hi" → got ""
↳ cu-test-artifacts/composer-takes-text.png
2 cases · 1 passed · 1 failed
```
任一用例失败时退出码非零(可直接丢进 CI),失败用例会存截图。断言:`url` · `visible` · `hidden` · `text` · `count` · `eval`。步骤:`open` · `click` · `fill` · `type` · `press` · `wait` · `scroll` · `eval`。完整指南:`chrome-use skills get test`。发现回归?加个用例 —— 用得越多,套件越值钱。
## 反检测 ## 反检测
连接你真实 Chrome 时,我们**零** JS 注入 —— 浏览器指纹完全是真的。指导原则是 **native CDP/Chrome 覆盖优先于 JS 谎言**:被重定义的 getter 本身可被检测,原生覆盖则不会。 连接你真实 Chrome 时,我们**零** JS 注入 —— 浏览器指纹完全是真的。指导原则是 **native CDP/Chrome 覆盖优先于 JS 谎言**:被重定义的 getter 本身可被检测,原生覆盖则不会。
@@ -146,7 +239,9 @@ chrome-use --launch --profile auto open https://x.com/home
| [rebrowser-bot-detector](https://bot-detector.rebrowser.net/) | `runtimeEnableLeak` 🟢 · `pwInitScripts` 🟢 | | [rebrowser-bot-detector](https://bot-detector.rebrowser.net/) | `runtimeEnableLeak` 🟢 · `pwInitScripts` 🟢 |
| [bot.sannysoft.com](https://bot.sannysoft.com) | 全绿 | | [bot.sannysoft.com](https://bot.sannysoft.com) | 全绿 |
`--launch` 独立模式下会改用一整套隐身补丁,同样过上述检测。 CreepJS 上的 `0% stealth` 是关键数字:因为连接路径**什么都不打补丁**,根本没有可供说谎检测器抓的 override。(读 `navigator.languages` 顺序或 IP 地理位置的面板可能给个软性的「navigator」/「location」标记 —— 那反映的是*你真实 Chrome* 的语言列表和网络,不是自动化破绽。)
`--launch` 独立模式(全新浏览器)会改用一整套隐身补丁,也能过上述检测 —— 唯一例外:CreepJS 报 **~20% stealth**,因为 srcdoc-iframe 的 `contentWindow` 补丁触发了它的 `hasIframeProxy` 探测(用来藏自动化的 proxy 本身成了破绽)。其余全干净(`0% headless`、sannysoft/browserscan 全绿、Cloudflare 通过)。设 **`AGENT_BROWSER_DISABLE_IFRAME_PROXY=1`** 去掉那个补丁即可拿到干净的 **0% stealth**(代价是放弃小众的 srcdoc-iframe 遮蔽)。**扩展连接路径**(你的真实 Chrome)零 JS 注入、不受影响 —— 它才是货真价实的 0% 路径。
### 类人输入(行为隐身) ### 类人输入(行为隐身)
@@ -167,6 +262,30 @@ chrome-use --launch --profile auto open https://x.com/home
操作你的真实 Chrome 不该打断你的工作。agent **全程在后台操作**:新标签后台打开(在自己的彩色会话标签组里),**从不强制把标签拽到前台**,并用 `Emulation.setFocusEmulationEnabled` 让每个 agent 标签照常渲染、`document.hasFocus()` / `visibilityState` 仍报 `visible`。于是截图正常、页面不被降频,"标签全程隐藏"也不会变成新的机器人信号。你在自己的标签里照常工作,agent 在旁边默默干活。(想置顶某个标签仍可显式调用命令。) 操作你的真实 Chrome 不该打断你的工作。agent **全程在后台操作**:新标签后台打开(在自己的彩色会话标签组里),**从不强制把标签拽到前台**,并用 `Emulation.setFocusEmulationEnabled` 让每个 agent 标签照常渲染、`document.hasFocus()` / `visibilityState` 仍报 `visible`。于是截图正常、页面不被降频,"标签全程隐藏"也不会变成新的机器人信号。你在自己的标签里照常工作,agent 在旁边默默干活。(想置顶某个标签仍可显式调用命令。)
### 自己验证
别光听我们说 —— 把你连接的 Chrome 指向最硬的公开检测器,自己对比:
- **[CreepJS](https://abrahamjuliot.github.io/creepjs/)** —— 最全面的指纹 / 说谎检测器
- **[bot.incolumitas.com](https://bot.incolumitas.com/)** —— 行为 + 指纹打分,方法公开
- **[BrowserScan](https://www.browserscan.net/bot-detection)** —— Webdriver / User-Agent / CDP / Navigator
- **[bot.sannysoft.com](https://bot.sannysoft.com)** —— 经典自动化特征清单
- **[pixelscan.net](https://pixelscan.net/)** · **[iphey.com](https://iphey.com/)** —— 一致性与身份
我们故意**不自带 bot 检测器** —— 最强、最诚实的基准,就是拿市面上最好的检测器去测你的真实浏览器。
### 调参(环境变量)
| 变量 | 默认 | 作用 |
|---|---|---|
| `AGENT_BROWSER_CAPTURE_CONSOLE` | 关 | 启用 `Runtime` 域,让 `console` / `errors` 捕获页面输出。关闭可保持最隐身的画像。 |
| `AGENT_BROWSER_HUMANIZE` | 关 | 类人输入动作:`off`(瞬时)、`fast`(轻量缓动轨迹)、`human`(全套曲线轨迹 + 落点抖动 + 击键节奏 + 缓动滚动/拖拽)。也可用 `--humanize`。默认 `off`;自适应检测器会把 Akamai/PerimeterX/DataDome 守护的页面自动升到 `human`。 |
| `AGENT_BROWSER_TIMEZONE` | 未设 | 仅 `--launch`。IANA id(如 `Asia/Tokyo`)原生设置时区(Intl + Date 跟随,无 JS 谎言)以匹配代理;`auto` 按 locale 推导。 |
| `AGENT_BROWSER_BLOCK_WEBRTC` | auto | 仅 `--launch`。设了代理时自动强制 WebRTC 走代理(不泄漏真实 IP)。`1` 无代理时也隐藏本地 IP;`0` 退出。 |
| `AGENT_BROWSER_HIDE_CANVAS` | 关 | 仅 `--launch`。加入会话稳定的 canvas/audio 指纹噪声。默认关(噪声本身就是一种「谎言」)。 |
| `AGENT_BROWSER_ADAPTIVE_REF` | 开 | 当保存的 `@ref` 移动且 role/name 重查失败时,按指纹相似度重定位(需高分 + 明显领先,否则明确报错)。`0` 关闭。 |
| `AGENT_BROWSER_CLICK_MODE` | _(auto)_ | 点击策略。默认先滚动入视、派发坐标点击,若被浮层遮挡则回退 DOM `.click()``dom` 始终用 `.click()`(适合 blur 即关的自动补全/菜单项);`coord` 严格只用坐标(遮挡时硬失败)。 |
## chrome-use 的独特之处 ## chrome-use 的独特之处
- **默认 auto-connect** —— `chrome-use open` 连你现有的 Chrome 而非启新的 - **默认 auto-connect** —— `chrome-use open` 连你现有的 Chrome 而非启新的
@@ -181,3 +300,7 @@ chrome-use --launch --profile auto open https://x.com/home
## License ## License
Apache-2.0 Apache-2.0
---
> 由 **leeguooooo** 打造 —— AI agent、逆向工程与 Cloudflare Workers 的实战笔记见 **[blog.misonote.com](https://blog.misonote.com)** · 关注 **[X @leeguooooo](https://x.com/leeguooooo)**
+1 -1
View File
@@ -290,7 +290,7 @@ checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724"
[[package]] [[package]]
name = "chrome-use" name = "chrome-use"
version = "1.5.14" version = "1.5.19"
dependencies = [ dependencies = [
"aes", "aes",
"aes-gcm", "aes-gcm",
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "chrome-use" name = "chrome-use"
version = "1.5.14" version = "1.5.19"
edition = "2021" edition = "2021"
description = "Fast browser automation CLI for AI agents" description = "Fast browser automation CLI for AI agents"
license = "Apache-2.0" license = "Apache-2.0"
+42
View File
@@ -78,6 +78,7 @@ const KNOWN_COMMANDS: &[&str] = &[
"drag", "drag",
"dialog", "dialog",
"upload", "upload",
"site",
]; ];
/// Levenshtein distance, capped — small inputs only (command names). /// Levenshtein distance, capped — small inputs only (command names).
@@ -1193,6 +1194,47 @@ fn parse_command_inner(args: &[String], flags: &Flags) -> Result<Value, ParseErr
Ok(json!({ "id": id, "action": "evaluate", "script": script })) Ok(json!({ "id": id, "action": "evaluate", "script": script }))
} }
"site" => {
// `site <name>/<command> [positional...] [--key value]`. The
// `update`/`list`/`info` subcommands are handled CLI-side (main.rs)
// and never reach here — by this point `rest[0]` is a `name/cmd`
// adapter spec. Load it, map the args onto the adapter's declared
// `args`, and emit a `site` action: the daemon navigates to the
// adapter's @meta.domain (reusing the tab if already there) and evals
// the adapter function in the site's own logged-in page.
let spec = rest.first().ok_or(ParseError::InvalidValue {
message: "site requires <name>/<command> (run `chrome-use site list`)".to_string(),
usage: "site <name>/<command> [args]",
})?;
let adapter =
crate::site::load_adapter(spec).map_err(|e| ParseError::InvalidValue {
message: e,
usage: "site <name>/<command> [args]",
})?;
let domain = adapter
.domain()
.ok_or(ParseError::InvalidValue {
message: format!("site: adapter `{spec}` @meta is missing a \"domain\""),
usage: "site <name>/<command>",
})?
.to_string();
// Split remaining args: `--key value` → named, everything else → positional.
let mut positional: Vec<String> = Vec::new();
let mut named: Vec<(String, String)> = Vec::new();
let mut it = rest[1..].iter();
while let Some(a) = it.next() {
if let Some(key) = a.strip_prefix("--") {
let val = it.next().map(|s| s.to_string()).unwrap_or_default();
named.push((key.to_string(), val));
} else {
positional.push(a.to_string());
}
}
let mapped = crate::site::map_args(&adapter, &positional, &named);
let script = crate::site::build_eval(&adapter, &mapped);
Ok(json!({ "id": id, "action": "site", "domain": domain, "script": script }))
}
// === Stealth self-check === // === Stealth self-check ===
"stealth" => { "stealth" => {
// `stealth [status]` — local stealth self-check: mode, live probes // `stealth [status]` — local stealth self-check: mode, live probes
+79
View File
@@ -10,6 +10,7 @@ mod flags;
mod install; mod install;
mod native; mod native;
mod output; mod output;
mod site;
mod skills; mod skills;
mod test_runner; mod test_runner;
#[cfg(test)] #[cfg(test)]
@@ -834,6 +835,84 @@ fn main() {
exit(test_runner::run_test(suite, &flags)); exit(test_runner::run_test(suite, &flags));
} }
// Handle `site`: site adapters — turn a website into a structured-data CLI by
// running a per-command JS adapter inside your logged-in tab. `update`/`list`/
// `info` are CLI-side (download/filesystem); `site <name>/<cmd> [args]` falls
// through to the daemon dispatch below (navigate to the adapter's domain + eval).
if clean.first().map(|s| s.as_str()) == Some("site") {
match clean.get(1).map(|s| s.as_str()) {
Some("update") => {
let rt = tokio::runtime::Runtime::new().expect("Failed to create tokio runtime");
match rt.block_on(site::update()) {
Ok(n) if flags.json => {
println!("{}", json!({ "success": true, "adapters": n }))
}
Ok(n) => println!(
"{} synced {} site adapters → ~/.chrome-use/sites (run `chrome-use site list`)",
color::success_indicator(),
n
),
Err(e) => {
eprintln!("{} {}", color::error_indicator(), e);
exit(1);
}
}
return;
}
Some("list") => {
match site::list_adapters() {
Ok(list) if flags.json => {
println!("{}", json!({ "success": true, "adapters": list }))
}
Ok(list) if list.is_empty() => {
println!("no site adapters installed — run `chrome-use site update`")
}
Ok(list) => {
for a in &list {
println!("{a}");
}
eprintln!(
"{}",
color::dim(&format!(
"{} adapters · run: chrome-use site <name>/<cmd> [args]",
list.len()
))
);
}
Err(e) => {
eprintln!("{} {}", color::error_indicator(), e);
exit(1);
}
}
return;
}
Some("info") => {
let spec = clean.get(2).cloned().unwrap_or_default();
match site::load_adapter(&spec) {
Ok(a) => println!(
"{}",
serde_json::to_string_pretty(&a.meta).unwrap_or_default()
),
Err(e) => {
eprintln!("{} {}", color::error_indicator(), e);
exit(1);
}
}
return;
}
// `site <name>/<cmd> [args]` → fall through to the daemon dispatch.
Some(spec) if spec.contains('/') => {}
_ => {
eprintln!(
"{} usage: chrome-use site <name>/<cmd> [args] | site update | site list | \
site info <name>/<cmd>",
color::error_indicator()
);
exit(2);
}
}
}
// Handle skills command (doesn't need daemon) // Handle skills command (doesn't need daemon)
if clean.first().map(|s| s.as_str()) == Some("skills") { if clean.first().map(|s| s.as_str()) == Some("skills") {
skills::run_skills(&clean, flags.json); skills::run_skills(&clean, flags.json);
+42
View File
@@ -1315,6 +1315,7 @@ pub async fn execute_command(cmd: &Value, state: &mut DaemonState) -> Value {
"title" => handle_title(state).await, "title" => handle_title(state).await,
"content" => handle_content(state).await, "content" => handle_content(state).await,
"evaluate" => handle_evaluate(cmd, state).await, "evaluate" => handle_evaluate(cmd, state).await,
"site" => handle_site(cmd, state).await,
"close" => handle_close(state).await, "close" => handle_close(state).await,
"stealth_status" => handle_stealth_status(state).await, "stealth_status" => handle_stealth_status(state).await,
"snapshot" => handle_snapshot(cmd, state).await, "snapshot" => handle_snapshot(cmd, state).await,
@@ -2698,6 +2699,47 @@ async fn handle_evaluate(cmd: &Value, state: &DaemonState) -> Result<Value, Stri
Ok(json!({ "result": result, "origin": url })) Ok(json!({ "result": result, "origin": url }))
} }
/// Run a site adapter: navigate to its `@meta.domain` (only if we're not already
/// there — the point is to run as you, in the page that's already open) and eval
/// the adapter function in the site's own logged-in page. The CLI/commands.rs has
/// already loaded the adapter and built the `script`; here we just place the page
/// and evaluate. Never disrupts the user's foreground tab — navigation happens on
/// the daemon's own tab (same as every other command on the relay).
async fn handle_site(cmd: &Value, state: &mut DaemonState) -> Result<Value, String> {
let domain = cmd
.get("domain")
.and_then(|v| v.as_str())
.ok_or("site: missing 'domain'")?
.to_string();
let script = cmd
.get("script")
.and_then(|v| v.as_str())
.ok_or("site: missing 'script'")?
.to_string();
let current = match state.browser.as_ref() {
Some(mgr) => mgr.get_url().await.unwrap_or_default(),
None => String::new(),
};
let on_domain = url::Url::parse(&current)
.ok()
.and_then(|u| u.host_str().map(|h| h.to_string()))
.map(|h| h == domain || h.ends_with(&format!(".{domain}")))
.unwrap_or(false);
if !on_domain {
let nav = json!({ "url": format!("https://{domain}/") });
handle_navigate(&nav, state).await?;
}
let eval_cmd = json!({ "script": script });
let out = handle_evaluate(&eval_cmd, state).await?;
Ok(json!({
"result": out.get("result").cloned().unwrap_or(Value::Null),
"origin": out.get("origin").cloned().unwrap_or(Value::Null),
"domain": domain,
}))
}
/// Local stealth self-check: reports the active mode, live fingerprint probes, /// Local stealth self-check: reports the active mode, live fingerprint probes,
/// and the list of applied overrides — so an agent (or human) can confirm /// and the list of applied overrides — so an agent (or human) can confirm
/// stealth is working without driving an external detector, and audit exactly /// stealth is working without driving an external detector, and audit exactly
+157 -26
View File
@@ -235,6 +235,43 @@ fn prunable_target_ids(
.collect() .collect()
} }
/// Consecutive missing `getTargets` snapshots before an owned relay tab is
/// pruned. >1 so a single churning/partial snapshot (other agents opening/closing
/// tabs) or a brief cross-process-nav gap can't drop the tab the agent is driving.
const RELAY_PRUNE_MISSES: u32 = 3;
/// Debounced prune for the relay: target ids to drop, mutating per-target miss
/// counters. A tab in `live_ids` resets to 0; an absent (non-pinned) tab
/// increments and is pruned only at `RELAY_PRUNE_MISSES`. Counters for
/// no-longer-tracked targets are forgotten. Pure, so the multi-agent churn
/// tolerance is unit-testable without a live browser.
fn debounced_prune_ids(
pages: &[PageInfo],
live_ids: &HashSet<String>,
pinned: Option<&str>,
misses: &mut HashMap<String, u32>,
) -> Vec<String> {
let tracked: HashSet<&str> = pages.iter().map(|p| p.target_id.as_str()).collect();
misses.retain(|tid, _| tracked.contains(tid.as_str()));
let mut prune = Vec::new();
for p in pages {
let tid = p.target_id.as_str();
if live_ids.contains(tid) {
misses.remove(tid);
continue;
}
if pinned == Some(tid) {
continue;
}
let c = misses.entry(p.target_id.clone()).or_insert(0);
*c += 1;
if *c >= RELAY_PRUNE_MISSES {
prune.push(p.target_id.clone());
}
}
prune
}
/// Whether the resolved active page is a tab the session created (its target_id /// Whether the resolved active page is a tab the session created (its target_id
/// is in `created_targets`). Pure core of [`BrowserManager::active_is_session_owned`] /// is in `created_targets`). Pure core of [`BrowserManager::active_is_session_owned`]
/// so the relay no-hijack rule is unit-testable without a live browser. /// so the relay no-hijack rule is unit-testable without a live browser.
@@ -464,6 +501,14 @@ pub struct BrowserManager {
/// the session's commands onto the wrong page — the wrong-origin-fetch hazard /// the session's commands onto the wrong page — the wrong-origin-fetch hazard
/// in the dogfood reports. Falls back to the index if the pinned tab is gone. /// in the dogfood reports. Falls back to the index if the pinned tab is gone.
active_target_id: Option<String>, active_target_id: Option<String>,
/// Per-target count of CONSECUTIVE `resync_targets` snapshots in which an
/// owned tab was missing from `Target.getTargets`. Over the relay a single
/// snapshot routinely omits live tabs (multi-agent churn, a cross-process nav
/// briefly dropping the target), so we must not prune on one miss — that lost
/// the tab the agent was driving. A tab is removed only after it's been absent
/// for `RELAY_PRUNE_MISSES` consecutive snapshots; any snapshot that includes
/// it resets the counter. Keyed by stable target_id.
relay_target_misses: HashMap<String, u32>,
next_tab_id: u32, next_tab_id: u32,
/// Whether to enable the CDP `Runtime` domain (console / error / exception capture). /// Whether to enable the CDP `Runtime` domain (console / error / exception capture).
/// OFF by default for stealth: a live `Runtime.enable` is a detectable CDP signal /// OFF by default for stealth: a live `Runtime.enable` is a detectable CDP signal
@@ -600,6 +645,7 @@ impl BrowserManager {
visited_origins: HashSet::new(), visited_origins: HashSet::new(),
created_targets: HashSet::new(), created_targets: HashSet::new(),
active_target_id: None, active_target_id: None,
relay_target_misses: HashMap::new(),
next_tab_id: 1, next_tab_id: 1,
capture_console: console_capture_enabled(), capture_console: console_capture_enabled(),
}; };
@@ -702,6 +748,7 @@ impl BrowserManager {
visited_origins: HashSet::new(), visited_origins: HashSet::new(),
created_targets: HashSet::new(), created_targets: HashSet::new(),
active_target_id: None, active_target_id: None,
relay_target_misses: HashMap::new(),
next_tab_id: 1, next_tab_id: 1,
capture_console: console_capture_enabled(), capture_console: console_capture_enabled(),
}; };
@@ -823,7 +870,19 @@ impl BrowserManager {
self.active_page_index = 0; self.active_page_index = 0;
self.pin_active_target(); self.pin_active_target();
self.enable_domains(&attach_result.session_id).await?; self.enable_domains(&attach_result.session_id).await?;
} else if self.agent_group().is_some() {
// STRICT MULTI-AGENT ISOLATION (relay / the user's real Chrome).
// `page_targets` here are the USER's and OTHER agents' tabs. A tab
// group belongs to exactly ONE agent, so this session must NOT adopt
// any of them — it tracks ONLY tabs it creates (its own colored group)
// plus popups it opens. Adopting foreign tabs is precisely what let
// another concurrent agent's tab churn drop the tab we were driving and
// drift eval/click onto the wrong page (multi-agent failure). Open our
// own dedicated background tab in the session's group and pin it; the
// user's / other agents' tabs stay invisible to us.
self.tab_new(None, None).await?;
} else { } else {
// A browser WE launched: every tab is ours, so adopt them all.
for target in &page_targets { for target in &page_targets {
let attach_result: AttachToTargetResult = self let attach_result: AttachToTargetResult = self
.client .client
@@ -849,24 +908,10 @@ impl BrowserManager {
target_type: target.target_type.clone(), target_type: target.target_type.clone(),
}); });
} }
self.active_page_index = 0;
if self.agent_group().is_some() { self.pin_active_target();
// Relay: the adopted tabs above are the USER's, in their real let session_id = self.pages[0].session_id.clone();
// Chrome. NEVER make one of them the agent's working tab — that is self.enable_domains(&session_id).await?;
// how commands drifted onto whatever page the user was viewing
// between steps (eval/click/get landed on the user's foreground
// tab; #35). Open our own dedicated background tab in the session's
// group and pin THAT as active. The user's tabs stay adopted (so
// `tab list` / explicit `tab switch` can reach them) but are never
// auto-selected — the agent only ever drives a tab it owns.
self.tab_new(None, None).await?;
} else {
// A browser we launched: every tab is ours, so the first is fine.
self.active_page_index = 0;
self.pin_active_target();
let session_id = self.pages[0].session_id.clone();
self.enable_domains(&session_id).await?;
}
} }
Ok(()) Ok(())
@@ -1071,6 +1116,17 @@ impl BrowserManager {
}; };
if let Some(ref error_text) = nav_result.error_text { if let Some(ref error_text) = nav_result.error_text {
// `data:` URLs abort over the extension relay: chrome.debugger /
// chrome.tabs can't drive a top-frame data: navigation, so it comes
// back net::ERR_ABORTED on an about:blank tab. Explain it instead of
// leaking the cryptic code (data: works fine under `--launch`).
if url.starts_with("data:") && error_text.contains("ERR_ABORTED") {
return Err(format!(
"Navigation failed: {error_text}. Chrome blocks top-frame `data:` URLs over \
the extension relay use a real http(s):// or file:// URL, or run with \
`--launch` (where data: URLs work)."
));
}
return Err(format!("Navigation failed: {}", error_text)); return Err(format!("Navigation failed: {}", error_text));
} }
@@ -1515,6 +1571,18 @@ impl BrowserManager {
/// the active tab, per #7/#8.1); the caller surfaces it so the agent knows a /// the active tab, per #7/#8.1); the caller surfaces it so the agent knows a
/// tab opened instead of seeing the old page (issue #24-A). /// tab opened instead of seeing the old page (issue #24-A).
pub async fn adopt_newly_opened(&mut self, before: &HashSet<String>) -> Option<PageInfo> { pub async fn adopt_newly_opened(&mut self, before: &HashSet<String>) -> Option<PageInfo> {
// STRICT MULTI-AGENT ISOLATION: on the relay this session's `before` set is
// only its OWN tabs, so EVERY foreign tab (the user's, other agents') looks
// "new" relative to it and would be adopted here — exactly the leak where a
// concurrent agent's tabs (github/Lark/iphone-use) showed up in this
// session mid-flow. A tab the agent itself opened (a pop-up) can't be
// distinguished from a foreign tab over the relay (no opener/window/group
// in the synthesized targetInfo), so don't adopt anything: the agent drives
// only tabs it explicitly created, and pop-ups (e.g. an OAuth/login window)
// are the user's. A launched browser (every tab ours) still follows pop-ups.
if self.agent_group().is_some() {
return None;
}
let result: GetTargetsResult = self let result: GetTargetsResult = self
.client .client
.send_command_typed("Target.getTargets", &json!({}), None) .send_command_typed("Target.getTargets", &json!({}), None)
@@ -1557,6 +1625,11 @@ impl BrowserManager {
title: sanitize_title(&target.title), title: sanitize_title(&target.title),
target_type: target.target_type.clone(), target_type: target.target_type.clone(),
}; };
// A tab that appeared right after THIS session's action (a click that
// opened a popup/new tab) is ours — record it as owned so it's tracked,
// protected from churn-pruning, and cleaned up on close, consistent with
// strict multi-agent isolation (we only ever own tabs we created/opened).
self.created_targets.insert(target.target_id.clone());
self.add_background_page(page.clone()); self.add_background_page(page.clone());
let _ = self.enable_domains(&attach.session_id).await; let _ = self.enable_domains(&attach.session_id).await;
if opened.is_none() { if opened.is_none() {
@@ -1584,13 +1657,19 @@ impl BrowserManager {
.filter(should_track_target) .filter(should_track_target)
.collect(); .collect();
let live_ids: HashSet<String> = live.iter().map(|t| t.target_id.clone()).collect(); let live_ids: HashSet<String> = live.iter().map(|t| t.target_id.clone()).collect();
let on_relay = self.agent_group().is_some();
for target in &live { for target in &live {
if self.update_page_target_info(target) { if self.update_page_target_info(target) {
continue; continue;
} }
// A target this session hasn't tracked yet — attach and add it in the // STRICT MULTI-AGENT ISOLATION: on the relay (the user's real Chrome,
// background so it's listable/adoptable without stealing the active tab. // shared with other agents), NEVER adopt a tab this session didn't
// create — it belongs to the user or another agent's group. Only a
// browser we launched (every tab ours) adopts unknown targets.
if on_relay {
continue;
}
let attach_result: AttachToTargetResult = match self let attach_result: AttachToTargetResult = match self
.client .client
.send_command_typed( .send_command_typed(
@@ -1621,12 +1700,26 @@ impl BrowserManager {
let _ = self.enable_domains(&attach_result.session_id).await; let _ = self.enable_domains(&attach_result.session_id).await;
} }
// Drop tabs that no longer exist so `tab list` doesn't show phantom rows — // Prune tabs that are gone. On a LAUNCHED browser a missing target really
// but never prune the explicitly-pinned active target on a transient // is closed, so prune immediately. On the RELAY a single `getTargets`
// getTargets snapshot (issue #31; see `prunable_target_ids`). // snapshot routinely omits live tabs (multi-agent churn, a brief
let gone = prunable_target_ids(&self.pages, &live_ids, self.active_target_id.as_deref()); // cross-process-nav gap) — dropping the tab we're driving on one bad
for tid in gone { // snapshot is the failure we're fixing — so prune only after the tab has
self.remove_page_by_target_id(&tid); // been absent for several CONSECUTIVE snapshots (debounced). The pinned
// active target is protected either way (issue #31).
let gone = if on_relay {
debounced_prune_ids(
&self.pages,
&live_ids,
self.active_target_id.as_deref(),
&mut self.relay_target_misses,
)
} else {
prunable_target_ids(&self.pages, &live_ids, self.active_target_id.as_deref())
};
for tid in &gone {
self.relay_target_misses.remove(tid);
self.remove_page_by_target_id(tid);
} }
// Refresh url/title from each live tab. The relay only stamps target_info // Refresh url/title from each live tab. The relay only stamps target_info
@@ -2536,6 +2629,7 @@ async fn initialize_lightpanda_manager(
visited_origins: HashSet::new(), visited_origins: HashSet::new(),
created_targets: HashSet::new(), created_targets: HashSet::new(),
active_target_id: None, active_target_id: None,
relay_target_misses: HashMap::new(),
next_tab_id: 1, next_tab_id: 1,
capture_console: console_capture_enabled(), capture_console: console_capture_enabled(),
}; };
@@ -2992,6 +3086,43 @@ mod tests {
); );
} }
#[test]
fn debounced_prune_tolerates_transient_churn() {
// Multi-agent churn: a single getTargets snapshot omits our owned tab "B"
// (another agent opened/closed tabs). It must NOT be pruned on one miss.
let pages = vec![page("A"), page("B")];
let mut misses = HashMap::new();
let empty: HashSet<String> = HashSet::new();
// Misses 1 and 2: B absent but under threshold → not pruned.
assert!(debounced_prune_ids(&pages, &empty, Some("A"), &mut misses).is_empty());
assert!(debounced_prune_ids(&pages, &empty, Some("A"), &mut misses).is_empty());
// Miss 3 (== RELAY_PRUNE_MISSES): genuinely gone → pruned.
assert_eq!(
debounced_prune_ids(&pages, &empty, Some("A"), &mut misses),
vec!["B".to_string()]
);
}
#[test]
fn debounced_prune_resets_on_reappearance_and_protects_pin() {
let pages = vec![page("A"), page("B")];
let mut misses = HashMap::new();
let empty: HashSet<String> = HashSet::new();
let mut live_b: HashSet<String> = HashSet::new();
live_b.insert("B".to_string());
// Two misses for B, then it reappears → counter resets, so it survives
// indefinitely under intermittent churn.
debounced_prune_ids(&pages, &empty, Some("A"), &mut misses);
debounced_prune_ids(&pages, &empty, Some("A"), &mut misses);
assert!(debounced_prune_ids(&pages, &live_b, Some("A"), &mut misses).is_empty());
assert!(debounced_prune_ids(&pages, &empty, Some("A"), &mut misses).is_empty()); // back to miss 1
// The pinned active "A" is never pruned no matter how many misses.
for _ in 0..5 {
let gone = debounced_prune_ids(&pages, &empty, Some("A"), &mut misses);
assert!(!gone.contains(&"A".to_string()));
}
}
#[test] #[test]
fn resolve_active_index_pin_survives_passive_background_tab() { fn resolve_active_index_pin_survives_passive_background_tab() {
// A foreign tab ("Z") gets appended by passive discovery after we pinned // A foreign tab ("Z") gets appended by passive discovery after we pinned
+71
View File
@@ -35,6 +35,7 @@ fn native_test_fixture_html(name: &str) -> &'static str {
"html5_drag_probe" => include_str!("test_fixtures/html5_drag_probe.html"), "html5_drag_probe" => include_str!("test_fixtures/html5_drag_probe.html"),
"pointer_capture_probe" => include_str!("test_fixtures/pointer_capture_probe.html"), "pointer_capture_probe" => include_str!("test_fixtures/pointer_capture_probe.html"),
"upload_probe" => include_str!("test_fixtures/upload_probe.html"), "upload_probe" => include_str!("test_fixtures/upload_probe.html"),
"iframe_button_probe" => include_str!("test_fixtures/iframe_button_probe.html"),
_ => panic!("Unknown native test fixture: {}", name), _ => panic!("Unknown native test fixture: {}", name),
} }
} }
@@ -573,6 +574,76 @@ async fn e2e_snapshot_and_click_ref() {
assert_success(&resp); assert_success(&resp);
} }
/// Clicking a button INSIDE an iframe by `@ref` must deliver a TRUSTED activation
/// (`event.isTrusted === true`), not a synthetic DOM `.click()`. Security-sensitive
/// embedded forms (Google Payments' `保存`) reject `isTrusted:false` clicks, so an
/// enabled submit button silently no-op'd (issue #39). The fix routes iframe-ref
/// clicks to a real `Input.dispatchMouseEvent` on the element's own frame session.
/// The fixture's iframe button writes `clicked:<isTrusted>` into its own text on
/// click, which the cross-frame snapshot reads back.
#[tokio::test]
#[ignore]
async fn e2e_iframe_button_click_is_trusted() {
let mut state = DaemonState::new();
let resp = execute_command(
&json!({ "id": "1", "action": "launch", "headless": true }),
&mut state,
)
.await;
assert_success(&resp);
let resp = execute_command(
&json!({ "id": "2", "action": "navigate", "url": native_test_fixture_url("iframe_button_probe") }),
&mut state,
)
.await;
assert_success(&resp);
// Snapshot (interactive) — the button lives in the iframe and must appear with
// a ref; that ref carries the frame_id so the click resolves into the frame.
let resp = execute_command(
&json!({ "id": "3", "action": "snapshot", "interactive": true }),
&mut state,
)
.await;
assert_success(&resp);
let snapshot = get_data(&resp)["snapshot"].as_str().unwrap_or("");
let ref_id = snapshot
.lines()
.find(|l| l.contains("button \"save\""))
.and_then(|l| l.split("ref=").nth(1))
.map(|r| r.trim_end_matches(']').trim())
.unwrap_or_else(|| panic!("iframe button not found in snapshot:\n{snapshot}"));
// Click it by ref.
let resp = execute_command(
&json!({ "id": "4", "action": "click", "selector": ref_id }),
&mut state,
)
.await;
assert_success(&resp);
tokio::time::sleep(tokio::time::Duration::from_millis(300)).await;
// The button rewrote its own text with the click's isTrusted flag; read it
// back across frames.
let resp = execute_command(
&json!({ "id": "5", "action": "snapshot", "interactive": true }),
&mut state,
)
.await;
assert_success(&resp);
let after = get_data(&resp)["snapshot"].as_str().unwrap_or("");
assert!(
after.contains("clicked:true"),
"iframe button click must be trusted (isTrusted:true); snapshot:\n{after}"
);
let resp = execute_command(&json!({ "id": "99", "action": "close" }), &mut state).await;
assert_success(&resp);
}
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// Screenshot // Screenshot
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
+90 -10
View File
@@ -56,19 +56,34 @@ pub async fn click(
.await; .await;
} }
// Over the extension relay we drive the user's real, in-use Chrome, where a // An element INSIDE an iframe needs a TRUSTED activation: a DOM `.click()` is
// coordinate `Input.dispatchMouseEvent` is NOT reliably confined to our target // `isTrusted:false`, which security-sensitive embedded forms reject — Google
// tab — it can be delivered to whatever tab is in the foreground, and an OOPIF // Payments' enabled `保存` button silently no-ops on a synthetic click (issue
// element's box can't be mapped to a top-viewport point at all. This twice // #39). A coordinate `Input.dispatchMouseEvent` can't help either: `getBoxModel`
// opened an unrelated tab on the user's busy Chrome (issues #31/#36). So on the // for a sub-frame node returns frame-local coordinates that don't compose the
// relay, never use coordinates for a normal left click: DOM-dispatch invokes // iframe's offset, so the click lands in the wrong place. The frame-agnostic
// the element's click in its own (frame) session, always hitting the right // trusted path is keyboard activation — focus the element in its own frame, then
// element in the right tab. Double/right clicks still need true pointer // dispatch a real Enter on the page session; Chrome routes the key to the
// semantics, and `coord` mode is an explicit opt-out. // focused element regardless of frame (same as `type --focused`), and Enter on a
// focused button/link fires a trusted `click`. `coord` mode opts out.
let in_iframe = ref_map.ref_is_in_iframe(selector_or_ref);
if mode != "coord" && button == "left" && click_count == 1 && in_iframe {
return dom_activate(
client,
session_id,
ref_map,
selector_or_ref,
iframe_sessions,
)
.await;
}
// On the relay (the user's real Chrome) a TOP-document coordinate click used to
// drift onto the foreground tab; that root cause is fixed (#5: the agent drives
// its own pinned tab), but DOM-dispatch stays the conservative default here.
if mode != "coord" if mode != "coord"
&& button == "left" && button == "left"
&& click_count == 1 && click_count == 1
&& prefer_dom_dispatch(ref_map, selector_or_ref) && crate::connect::relay_url().is_some()
{ {
return dom_click( return dom_click(
client, client,
@@ -270,6 +285,71 @@ async fn dom_click(
Ok(()) Ok(())
} }
/// Trusted activation of an element inside an iframe (issue #39). Focuses the
/// element in its own frame session, then dispatches a real Enter/Space on the
/// page session — Chrome routes the key to the focused element across frames, and
/// Enter/Space on a focused button/link/checkbox fires a `click` with
/// `isTrusted: true`, which security-sensitive embedded forms (Google Payments
/// `保存`) require. Non-activatable roles (a `div[onclick]`) can't be keyboard-
/// activated, so they fall back to a DOM `.click()`.
async fn dom_activate(
client: &CdpClient,
session_id: &str,
ref_map: &RefMap,
selector_or_ref: &str,
iframe_sessions: &HashMap<String, String>,
) -> Result<(), String> {
let role = parse_ref(selector_or_ref)
.and_then(|r| ref_map.get(&r).map(|e| e.role.clone()))
.unwrap_or_default();
// Space toggles checkbox-like controls; Enter activates buttons/links/menus.
let key = match role.as_str() {
"checkbox" | "radio" | "switch" | "option" | "menuitemcheckbox" | "menuitemradio" => {
Some("space")
}
"button" | "link" | "menuitem" | "tab" | "treeitem" => Some("enter"),
_ => None,
};
let Some(key) = key else {
// Not keyboard-activatable — best effort via DOM .click() (untrusted).
return dom_click(
client,
session_id,
ref_map,
selector_or_ref,
iframe_sessions,
)
.await;
};
let (object_id, effective_session_id) = resolve_element_object_id(
client,
session_id,
ref_map,
selector_or_ref,
iframe_sessions,
)
.await?;
// Focus the element in its OWN frame session so the keystroke lands on it.
client
.send_command_typed::<_, Value>(
"Runtime.callFunctionOn",
&CallFunctionOnParams {
function_declaration: "function() { this.focus(); }".to_string(),
object_id: Some(object_id),
arguments: None,
return_by_value: Some(true),
await_promise: Some(false),
},
Some(&effective_session_id),
)
.await?;
// Trusted key on the page session — routed to the focused (in-frame) element.
press_key(client, session_id, key).await?;
wait_for_paint_settled(client, &effective_session_id).await;
Ok(())
}
/// DOM-dispatch a double-click on the element in its own session (no coordinates) /// DOM-dispatch a double-click on the element in its own session (no coordinates)
/// — the relay/iframe-safe counterpart to a coordinate dblclick. Fires the full /// — the relay/iframe-safe counterpart to a coordinate dblclick. Fires the full
/// click,click,dblclick sequence so handlers bound to any of them respond. /// click,click,dblclick sequence so handlers bound to any of them respond.
@@ -0,0 +1,28 @@
<!doctype html>
<html>
<head>
<meta charset="utf-8" />
<title>iframe button probe</title>
</head>
<body>
<h1>iframe button probe</h1>
<iframe
id="frame"
width="320"
height="140"
srcdoc="
<!doctype html>
<html>
<body style='margin:24px'>
<button id='b' style='padding:24px;font-size:22px'>save</button>
<script>
document.getElementById('b').addEventListener('click', function (e) {
this.textContent = 'clicked:' + e.isTrusted;
});
</script>
</body>
</html>
"
></iframe>
</body>
</html>
+12
View File
@@ -3366,6 +3366,14 @@ Batch:
batch [--bail] ["cmd" ...] Execute multiple commands sequentially (args or stdin) batch [--bail] ["cmd" ...] Execute multiple commands sequentially (args or stdin)
--bail stops on first error (default: continue all) --bail stops on first error (default: continue all)
Site adapters: turn a website into a structured-data CLI (runs as you, in your tab)
site update Fetch the community adapter pack into ~/.chrome-use/sites
site list List installed adapters (name/cmd)
site info <name>/<cmd> Show an adapter's @meta (args, domain, capabilities)
site <name>/<cmd> [args] Run an adapter: navigate to its site + return JSON
e.g. site github/issues epiral/repo, site reddit/search rust
Positional args fill declared args in order; --key value overrides
Auth Vault: Auth Vault:
auth save <name> [opts] Save auth profile (--url, --username, --password/--password-stdin) auth save <name> [opts] Save auth profile (--url, --username, --password/--password-stdin)
auth login <name> Login using saved credentials (waits for form fields) auth login <name> Login using saved credentials (waits for form fields)
@@ -3577,6 +3585,9 @@ iOS Simulator (requires Xcode and Appium):
chrome-use -p ios device list # List simulators chrome-use -p ios device list # List simulators
chrome-use -p ios swipe up # Swipe gesture chrome-use -p ios swipe up # Swipe gesture
chrome-use -p ios tap @e1 # Touch element chrome-use -p ios tap @e1 # Touch element
Hit a bug or rough edge? A 30-second issue genuinely sharpens this tool:
https://github.com/leeguooooo/chrome-use/issues
"# "#
); );
} }
@@ -3659,6 +3670,7 @@ fn print_screenshot_diff(data: &serde_json::Map<String, serde_json::Value>) {
pub fn print_version() { pub fn print_version() {
println!("chrome-use {}", env!("CARGO_PKG_VERSION")); println!("chrome-use {}", env!("CARGO_PKG_VERSION"));
println!("report bugs / rough edges: https://github.com/leeguooooo/chrome-use/issues");
} }
#[cfg(test)] #[cfg(test)]
+267
View File
@@ -0,0 +1,267 @@
//! Site adapters: turn any website into a structured-data CLI by running a small
//! per-command JS adapter inside your real, logged-in browser tab (it reuses the
//! site's cookies / same-origin fetch / its own webpack modules — the site thinks
//! it's you, because it is).
//!
//! The adapter format is the community **bb-sites** convention
//! (<https://github.com/epiral/bb-sites>): one `.js` file per command, a
//! `/* @meta {...} */` JSON header (name, description, domain, args), then an
//! `async function(args){ ... return {...} }`. chrome-use ships none of those
//! adapters — `chrome-use site update` fetches the upstream repo at runtime into
//! `~/.chrome-use/sites` (like a package manager pulling a dependency), so the
//! adapters stay the property of their authors. Running an adapter navigates to
//! its `@meta.domain` and `eval`s the function in the site's own logged-in page.
use std::path::PathBuf;
use serde_json::Value;
const SITES_ZIP_URL: &str = "https://github.com/epiral/bb-sites/archive/refs/heads/main.zip";
/// `~/.chrome-use/sites` — where synced adapters live.
pub fn sites_dir() -> Option<PathBuf> {
dirs_home().map(|h| h.join(".chrome-use").join("sites"))
}
fn dirs_home() -> Option<PathBuf> {
std::env::var_os("HOME").map(PathBuf::from)
}
/// Parsed adapter: its `@meta` JSON and the raw `async function(args){...}` source.
pub struct Adapter {
pub meta: Value,
pub func_src: String,
}
impl Adapter {
pub fn domain(&self) -> Option<&str> {
self.meta.get("domain").and_then(|v| v.as_str())
}
}
/// Load `<sites>/<name>/<cmd>.js`, splitting the `/* @meta {...} */` header from
/// the function body. `spec` is `name/cmd`.
pub fn load_adapter(spec: &str) -> Result<Adapter, String> {
let (name, cmd) = spec
.split_once('/')
.ok_or_else(|| format!("site: expected <name>/<command>, got `{spec}`"))?;
if name.is_empty()
|| cmd.is_empty()
|| name.contains("..")
|| cmd.contains("..")
|| name.contains('/')
|| cmd.contains('/')
{
return Err(format!("site: invalid adapter spec `{spec}`"));
}
let dir = sites_dir().ok_or("site: cannot resolve home dir")?;
let path = dir.join(name).join(format!("{cmd}.js"));
if !path.exists() {
return Err(format!(
"site: adapter `{spec}` not found. Run `chrome-use site update` to sync adapters, \
or `chrome-use site list` to see what's installed."
));
}
let raw = std::fs::read_to_string(&path).map_err(|e| format!("site: read {spec}: {e}"))?;
parse_adapter(&raw, spec)
}
/// Split the `@meta` JSON block and the function source from an adapter file.
pub fn parse_adapter(raw: &str, spec: &str) -> Result<Adapter, String> {
let start = raw
.find("@meta")
.and_then(|i| raw[i..].find('{').map(|j| i + j))
.ok_or_else(|| format!("site: {spec} missing /* @meta {{...}} */ header"))?;
// Find the matching close brace for the @meta object (brace-count, string-aware).
let bytes = raw.as_bytes();
let mut depth = 0i32;
let mut in_str = false;
let mut esc = false;
let mut end = None;
for (k, &b) in bytes.iter().enumerate().skip(start) {
if in_str {
if esc {
esc = false;
} else if b == b'\\' {
esc = true;
} else if b == b'"' {
in_str = false;
}
continue;
}
match b {
b'"' => in_str = true,
b'{' => depth += 1,
b'}' => {
depth -= 1;
if depth == 0 {
end = Some(k + 1);
break;
}
}
_ => {}
}
}
let end = end.ok_or_else(|| format!("site: {spec} @meta header has no closing brace"))?;
let meta: Value = serde_json::from_str(&raw[start..end])
.map_err(|e| format!("site: {spec} @meta is not valid JSON: {e}"))?;
// The function is everything after the meta comment's closing `*/`.
let after = raw[end..].find("*/").map(|i| end + i + 2).unwrap_or(end);
let func_src = raw[after..].trim().to_string();
if func_src.is_empty() {
return Err(format!("site: {spec} has no function body after @meta"));
}
Ok(Adapter { meta, func_src })
}
/// Build the JS to eval: `(<adapter function>)(<args JSON>)`. The adapter's
/// `async function(args)` returns a promise; chrome-use's eval awaits it.
pub fn build_eval(adapter: &Adapter, args: &Value) -> String {
let args_json = serde_json::to_string(args).unwrap_or_else(|_| "{}".to_string());
format!("({})({})", adapter.func_src, args_json)
}
/// List installed adapters as `name/cmd` strings (sorted).
pub fn list_adapters() -> Result<Vec<String>, String> {
let dir = sites_dir().ok_or("site: cannot resolve home dir")?;
if !dir.exists() {
return Ok(Vec::new());
}
let mut out = Vec::new();
for site in std::fs::read_dir(&dir)
.map_err(|e| e.to_string())?
.flatten()
{
if !site.path().is_dir() {
continue;
}
let name = site.file_name().to_string_lossy().to_string();
for cmd in std::fs::read_dir(site.path())
.map_err(|e| e.to_string())?
.flatten()
{
let p = cmd.path();
if p.extension().and_then(|e| e.to_str()) == Some("js") {
if let Some(stem) = p.file_stem().and_then(|s| s.to_str()) {
out.push(format!("{name}/{stem}"));
}
}
}
}
out.sort();
Ok(out)
}
/// Download the bb-sites repo zip and extract its adapters into `~/.chrome-use/sites`.
pub async fn update() -> Result<usize, String> {
let dir = sites_dir().ok_or("site: cannot resolve home dir")?;
let client = reqwest::Client::builder()
.user_agent("chrome-use")
.build()
.map_err(|e| e.to_string())?;
let bytes = client
.get(SITES_ZIP_URL)
.send()
.await
.map_err(|e| format!("site update: download failed: {e}"))?
.error_for_status()
.map_err(|e| format!("site update: {e}"))?
.bytes()
.await
.map_err(|e| format!("site update: read body: {e}"))?;
let cursor = std::io::Cursor::new(bytes);
let mut zip = zip::ZipArchive::new(cursor).map_err(|e| format!("site update: bad zip: {e}"))?;
std::fs::create_dir_all(&dir).map_err(|e| e.to_string())?;
let mut count = 0usize;
for i in 0..zip.len() {
let mut f = zip.by_index(i).map_err(|e| e.to_string())?;
let Some(enclosed) = f.enclosed_name() else {
continue;
};
// Strip the top-level `bb-sites-main/` component from the archive path.
let rel: PathBuf = enclosed.components().skip(1).collect();
if rel.as_os_str().is_empty() {
continue;
}
let out = dir.join(&rel);
if f.is_dir() {
let _ = std::fs::create_dir_all(&out);
continue;
}
if let Some(parent) = out.parent() {
let _ = std::fs::create_dir_all(parent);
}
let mut buf = Vec::new();
std::io::copy(&mut f, &mut buf).map_err(|e| e.to_string())?;
std::fs::write(&out, &buf).map_err(|e| e.to_string())?;
if out.extension().and_then(|e| e.to_str()) == Some("js") {
count += 1;
}
}
Ok(count)
}
/// Map CLI args to the adapter's `args` object. Positional args fill the adapter's
/// declared `args` keys in order; `--key value` overrides by name. The adapter
/// validates required args itself.
pub fn map_args(adapter: &Adapter, positional: &[String], named: &[(String, String)]) -> Value {
let mut obj = serde_json::Map::new();
let keys: Vec<String> = adapter
.meta
.get("args")
.and_then(|a| a.as_object())
.map(|m| m.keys().cloned().collect())
.unwrap_or_default();
for (i, val) in positional.iter().enumerate() {
if let Some(k) = keys.get(i) {
obj.insert(k.clone(), Value::String(val.clone()));
}
}
for (k, v) in named {
obj.insert(k.clone(), Value::String(v.clone()));
}
Value::Object(obj)
}
#[cfg(test)]
mod tests {
use super::*;
const SAMPLE: &str = r#"/* @meta
{
"name": "github/issues",
"domain": "github.com",
"args": { "repo": {"required": true}, "state": {"required": false} }
}
*/
async function(args) { return { repo: args.repo }; }"#;
#[test]
fn parses_meta_and_function() {
let a = parse_adapter(SAMPLE, "github/issues").unwrap();
assert_eq!(a.domain(), Some("github.com"));
assert!(a.func_src.starts_with("async function(args)"));
}
#[test]
fn build_eval_wraps_and_passes_args() {
let a = parse_adapter(SAMPLE, "github/issues").unwrap();
let args = map_args(
&a,
&["owner/repo".into()],
&[("state".into(), "closed".into())],
);
let js = build_eval(&a, &args);
assert!(js.contains("async function(args)"));
assert!(js.contains("\"repo\":\"owner/repo\""));
assert!(js.contains("\"state\":\"closed\""));
}
#[test]
fn rejects_bad_spec() {
assert!(load_adapter("noslash").is_err());
assert!(load_adapter("../etc/passwd").is_err());
}
}
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "chrome-use", "name": "chrome-use",
"version": "1.5.14", "version": "1.5.19",
"description": "chrome-use — drive your real, logged-in Chrome from any AI agent, stealth by default", "description": "chrome-use — drive your real, logged-in Chrome from any AI agent, stealth by default",
"type": "module", "type": "module",
"packageManager": "pnpm@11.1.3", "packageManager": "pnpm@11.1.3",
+33 -1
View File
@@ -68,6 +68,7 @@ need a **real, logged-in browser** — not for reading text off a public page.
|---|---| |---|---|
| Discover what exists / find sources | `WebSearch` | | Discover what exists / find sources | `WebSearch` |
| Specific facts from a static or public page | `WebFetch` or `curl` (no browser) | | Specific facts from a static or public page | `WebFetch` or `curl` (no browser) |
| **Structured data from a known site** (GitHub issues, Reddit/HN search, Bilibili/Twitter feed, …) — esp. behind login | `chrome-use site <name>/<cmd>` (see below) — skip snapshot+click entirely |
| Login state, interaction, JS-rendered or anti-bot pages | **chrome-use** (this skill) | | Login state, interaction, JS-rendered or anti-bot pages | **chrome-use** (this skill) |
| A page the user saved before / an internal system | `chrome-use find-url <keywords>` (their bookmarks), then open it | | A page the user saved before / an internal system | `chrome-use find-url <keywords>` (their bookmarks), then open it |
| The user's **own already-open, logged-in** Chrome window | the **extension connect** flow (below) | | The user's **own already-open, logged-in** Chrome window | the **extension connect** flow (below) |
@@ -131,7 +132,17 @@ Each `--session` that connects gets its **own colored Chrome tab group** (named
after the session) and drives only its own tabs — multiple agents share the one after the session) and drives only its own tabs — multiple agents share the one
real browser without cross-talk, and the user's own tabs are never grouped. CDP real browser without cross-talk, and the user's own tabs are never grouped. CDP
drives the page without moving the user's mouse/keyboard, so it doesn't fight drives the page without moving the user's mouse/keyboard, so it doesn't fight
them for control. **Anti-detection ranking: this real logged-in Chrome (extension them for control.
**Strict multi-agent isolation.** A session over the relay tracks and drives
**only the tabs it created** (its own group). It does **not** adopt the user's
existing tabs, other agents' tabs, or pop-ups (e.g. an OAuth/login window — that's
the user's), so several agents (and other tools opening tabs) can work in the same
real Chrome concurrently without ever dropping or stealing each other's tabs —
another agent's tab churn can't make your bound tab vanish or drift your commands
onto the wrong page. Consequence: `tab list` shows only *your* session's tabs; to
drive a specific page, navigate to it in your own tab instead of expecting a
pre-existing or popped-up tab to appear in the list. **Anti-detection ranking: this real logged-in Chrome (extension
connect) > a headed launched browser > headless (forbidden).** A genuine human connect) > a headed launched browser > headless (forbidden).** A genuine human
browser has no headless/automation tells at all, so prefer it for anything browser has no headless/automation tells at all, so prefer it for anything
anti-bot-sensitive. anti-bot-sensitive.
@@ -186,6 +197,27 @@ chrome-use eval "[...document.forms[0].elements].filter(e=>!e.validity.valid).ma
chrome-use eval "document.querySelector('#stubborn').click()" # direct DOM click, bypasses overlays chrome-use eval "document.querySelector('#stubborn').click()" # direct DOM click, bypasses overlays
``` ```
## Site adapters — the cheapest path for "read structured data from site X"
Before you `open` + `snapshot` + click your way through GitHub/Reddit/Bilibili/etc.,
check whether a **site adapter** already exists. An adapter is a community-written JS
function that hits the site's own JSON API *from inside your logged-in tab* and returns
clean structured data — no clicking, no scraping, no screenshots. It's the same idea as
`eval`, packaged per-site.
```bash
chrome-use site update # one-time: fetch the adapter pack (~145 cmds)
chrome-use site list # what's installed (github/issues, reddit/search, …)
chrome-use site info github/issues # an adapter's args + which domain it runs on
chrome-use site github/issues owner/repo --json # run it → JSON (navigates there for you)
```
- Positional args fill the adapter's declared args **in order**; `--key value` overrides by name.
- It navigates to the adapter's domain (reusing the current tab if you're already on it), so
login-gated feeds (`bilibili/feed`, `twitter/...`) work because they run as *you*.
- If no adapter fits, fall back to the normal `snapshot`/`eval` loop. Adapters come from the
[bb-sites](https://github.com/epiral/bb-sites) community pack; chrome-use fetches & runs them.
## Quickstart ## Quickstart
```bash ```bash