Compare commits

...
Author SHA1 Message Date
leeguooooo 85f4635358 chore(release): 0.27.0-fork.19 — new extension id (Web Store signing key) + store-aware install
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
Transport (native messaging + extension connect) works today via Load unpacked.
Silent force-install is pending the Chrome Web Store listing going live (off-store
force-install is [BLOCKED] on unmanaged Chrome 149).
2026-06-09 19:20:45 +09:00
leeguooooo 726e9d4ea3 chore(store): add listing screenshot (force-add past png ignore) 2026-06-09 19:16:04 +09:00
leeguooooo ff8a340269 chore(store): add 1280x800 listing screenshot + bake in GitHub Pages privacy URL 2026-06-09 19:15:49 +09:00
leeguooooo c1fa237183 chore: add .nojekyll for GitHub Pages (serve privacy policy as-is) 2026-06-09 19:10:32 +09:00
leeguooooo f6b21461e9 feat(connect): pivot extension install to Chrome Web Store path
Verified on Chrome 149 (unmanaged macOS): a force-install policy pointing at a
SELF-HOSTED crx is tagged [BLOCKED] in chrome://policy ("Error, Warning") — Chrome
refuses off-Web-Store force-installs on non-cloud-managed browsers. So the
self-hosted-crx approach cannot work on consumer Chrome; the extension must ship
via the Chrome Web Store (same reason codex/claude do).

- UPDATE_URL -> Chrome Web Store update endpoint; add STORE_URL (one-click Add to
  Chrome) as the guaranteed path + headless fallback
- install instructions now offer: A) one-click store link, B) silent profile
  force-install (works once published), with Load-unpacked as the pre-publish stopgap
- build extensions/ab-connect.zip (CWS upload package; manifest "key" kept so the
  published id stays ciiljdlhdpfckdcfkphgmfalanpdejep)
- extensions/store/{SUBMISSION.html,privacy.html}: full listing copy, permission
  justifications (debugger is the review-sensitive one), privacy policy
- drop dead self-hosted extensions/updates.xml; pack-extension.sh now builds the zip

Not released yet — force-install only works after the store listing is Published.
2026-06-09 19:03:10 +09:00
leeguooooo e8ef57bf00 feat(connect): force-install ab-connect via Chrome config profile (no Load-unpacked GUI)
Chrome 149 killed every GUI-free way to load an *unpacked* extension into the
real profile: --load-extension removed in Chrome 142 (incl. the
--disable-features workaround), local-.crx external install blocked on macOS
since Chrome 44, remote-debugging-port killed in Chrome 136. So agents were
stuck automating the chrome://extensions Load-unpacked native file dialog —
unworkable.

`extension install` now writes a macOS configuration profile that force-installs
the signed .crx from a hosted update_url (ExtensionInstallForcelist policy). One
approval in System Settings (a single fixed Install button — cua-driver-friendly,
unlike a file dialog) → Chrome force-installs + auto-updates the extension on next
launch. No token, no per-use confirmation, and binary-install users no longer
need the extensions/ folder (crx is fetched from the URL).

- pin a stable signing key; new extension id ciiljdlhdpfckdcfkphgmfalanpdejep
- ship signed extensions/ab-connect.crx + extensions/updates.xml (raw GH host)
- scripts/pack-extension.sh re-signs with the stable key; .secrets/*.pem ignored
- uninstall removes the profile file + prints `profiles remove` hint
2026-06-09 18:27:51 +09:00
leeguooooo 9efcb56651 chore(release): bump to 0.27.0-fork.18 — extension connect (zero-token native-messaging control of real Chrome) + click reliability + eval-first/find-url/site-notes
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
2026-06-09 17:32:44 +09:00
leeguooooo 091a4ec02e docs(skills): teach agents the extension-connect flow + computer-use for setup
So an agent can operate the zero-confirmation real-Chrome feature itself:
- SKILL.md: tool matrix gains "the user's own already-open, logged-in window →
  extension connect", plus a short section pointing at the flow.
- commands.md: the one-time "Load unpacked" is a privileged GUI step the CLI
  can't do — call it out that the agent can perform it with a computer-use /
  GUI-automation tool (cua-driver), with the live gotchas (synthetic-keystroke
  tools like peekaboo don't reach Chrome; cua-driver does; the native file
  dialog may need the user to pick the folder).
2026-06-09 17:14:50 +09:00
leeguooooo 6c0f5cbaa1 feat(connect): attach existing tabs + extension connect one-command UX
Completes the zero-confirmation real-Chrome feature.

- Drive the user's EXISTING logged-in tabs (not just newly-created ones):
  extension attachTab now treats "already attached" (a lingering chrome.debugger
  binding after a service-worker restart) as success and announces the tab
  anyway, instead of skipping it. The nm-host also sends {method:"attachAll"}
  when an agent-browser CDP client connects, so the daemon doesn't race an empty
  target list.
- `agent-browser extension connect` auto-discovers the relay's CDP url
  (~/.agent-browser/relay-cdp-url) and attaches — no copying a ws URL. Rewrites
  into the normal `connect <url>` flow; `extension install/status/uninstall`
  unchanged.
- Skill docs: a "drive your real, logged-in Chrome (extension)" section.

Verified end-to-end: `extension connect` listed the user's real tabs (Lark,
LINUX DO, Rakuten, Discord) and read a logged-in Lark doc's title — zero token,
zero confirmation. Full suite 768 passed.
2026-06-09 17:11:59 +09:00
leeguooooo 0d72e0d889 feat(connect): bridge native-messaging host to a CDP endpoint — end-to-end works
The __nm-host now exposes a Chrome-compatible CDP WebSocket endpoint and bridges
it to the extension over native messaging via the relay translation core
(relay.rs): incoming raw CDP commands are answered locally for browser-level
Target discovery or forwarded to the extension as forwardCDPCommand; the
extension's forwardCDPEvent/results are relayed back as raw CDP.

Security without a token or user interaction: the ws URL carries an unguessable
guid and is written to ~/.agent-browser/relay-cdp-url (perms 600), so only this
user's agent-browser can drive the browser — mirroring how Chrome guards its own
remote-debugging URL.

Verified end-to-end on real Chrome: `agent-browser connect <relay-url>` then an
eval navigated a tab and read back "Example Domain | https://example.com/" —
abs → CDP → relay → native messaging → extension → chrome.debugger → real tab,
zero token, zero confirmation. Adds the tokio io-std feature for the host's
stdio.

Remaining polish: re-attach the user's EXISTING tabs after a service-worker
restart (currently attaches new tabs cleanly; existing ones need detach+reattach
since chrome.debugger may still be bound), and an `open --extension` UX that
reads relay-cdp-url so the URL isn't passed by hand.
2026-06-09 16:54:29 +09:00
leeguooooo 528de4230f feat(connect): native-messaging transport — zero-token connect to real Chrome
Optimal architecture (chosen over the WS+token copy): the ab-connect extension
talks to a local agent-browser native-messaging host. No localhost port, no
token — Chrome authenticates the extension to the host by id. This is the
codex/claude-style "install once, no per-use confirmation" model.

- extensions/ab-connect: rewritten transport WebSocket+token → native messaging
  (chrome.runtime.connectNative). Pinned the extension id via a manifest `key`
  (→ bdoiejojpjogcjojeladhioioijhgade) so the host manifest can authorize it.
  Kept the proven chrome.debugger attach + Target.attachedToTarget emulation;
  dropped WS/token/options. Rebranded to "agent-browser connect".
- cli connect.rs: `agent-browser extension install` writes the native-messaging
  host manifest (Chrome/Chromium/Edge/Brave) + a launcher; hidden `__nm-host`
  speaks the 4-byte-length native-messaging framing.

Validated end-to-end on real Chrome: Chrome spawned the host (origin matched the
pinned id) and the extension attached the user's real logged-in tabs, streaming
Target.attachedToTarget over native messaging — zero token, zero port.

Next: bridge the host to the daemon relay (relay.rs) + CdpClient so
`agent-browser click/eval/...` drives those tabs.
2026-06-09 16:39:38 +09:00
leeguooooo 7f672494c1 feat(connect): relay translation core (envelope <-> raw CDP + Target emulation)
Pure, unit-tested core of the daemon-side relay that bridges the ab-connect
extension to the existing CdpClient. The extension exposes per-tab
chrome.debugger + synthesized Target events; CdpClient expects a browser-level
endpoint. So RelayState:

- answers Target.getTargets / attachToTarget / setDiscoverTargets LOCALLY from
  targets learned via the extension's forwardCDPEvent(Target.attachedToTarget),
  returning the extension's cb-tab-N sessionId (consumes those synth events
  rather than double-forwarding them);
- forwards every other command as a forwardCDPCommand envelope (carrying
  method/params/sessionId);
- maps forwardCDPCommand responses and forwardCDPEvent events back to raw CDP;
- validates the connect-handshake token; emits challenge/ping.

Keeps CdpClient and browser.rs unchanged. 8 unit tests; clippy clean. Still
inert — the tokio WS server + `connect` command wire it next.
2026-06-09 14:59:54 +09:00
leeguooooo 8a8106ad75 feat(connect): vendor MV3 connect extension (adapted from openclaw-browser-relay)
First step toward zero-confirmation direct connect to the user's real Chrome:
Chrome 136 killed --remote-debugging-port on the default profile, so the only
sanctioned way to drive the user's live logged-in window is an extension using
chrome.debugger (same approach as Codex/Claude, whose extensions are closed).

Vendors the MIT-licensed openclaw-browser-relay extension into
extensions/ab-connect/, rebranded to "agent-browser connect" (NOTICE.md keeps
attribution). It already handles the hard parts: chrome.debugger auto-attach all
tabs, new-tab auto-attach, MV3 service-worker keepalive (alarms) + reconnect,
sessionId↔tab mapping, token auth, and a CDP-over-WebSocket envelope
(connect handshake / forwardCDPCommand / forwardCDPEvent / ping-pong).

Inert for now — not wired. Next: an abs-daemon relay that speaks this envelope
and bridges it to the existing CdpClient (raw CDP), then a `connect` command.
2026-06-09 14:54:03 +09:00
leeguooooo f9cc31d003 chore(release): bump to 0.27.0-fork.17 — eval-first skill + find-url (local bookmark search) + site-notes convention
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
2026-06-05 17:15:49 +09:00
leeguooooo a7a3f924b0 docs(skills): site-notes convention for remembering site quirks
Borrow web-access's site-experience persistence as an agent-workflow convention
(no CLI code): keep one markdown file per domain under
~/.agent-browser/site-patterns/<domain>.md. Read it before working a domain
(hints, not guarantees); update it after learning something durable — working
selectors, required hidden fields, anti-bot traps, login needs. Makes repeat
visits fast instead of re-solving the same page every run.
2026-06-05 16:55:47 +09:00
leeguooooo 7572c34229 feat(find-url): search local Chrome/Edge bookmarks by keyword
Borrow web-access's find-url: locate an internal system or a previously-saved
page that public search can't reach, without opening a browser.

- `agent-browser find-url <keywords> [--browser chrome|edge] [--profile X]
  [--limit N] [--json]` — local command, no daemon. All keywords must match a
  bookmark's name or url; results are most-recently-added first.
- Cross-platform Bookmarks JSON paths (macOS / Linux / Windows), zero new deps
  (serde_json). Skips javascript:/data: bookmarklets.
- Skill docs: "pick the cheapest tool" matrix now points at find-url, plus a
  commands.md section.

Bookmarks only for now — visited-history is a locked SQLite DB and would need a
SQLite dependency (deferred to avoid C-dep cross-compile risk in the release
pipeline).
2026-06-05 16:54:46 +09:00
leeguooooo 06f5f9e8f1 docs(skills): lead with eval-first + tool-choice matrix
Real dogfooding showed the skill pushed agents straight into the fragile
snapshot/@ref path. Reframe the core guidance toward how a developer actually
drives a real browser:

- "Pick the cheapest tool" matrix: WebSearch / WebFetch+curl for static, reach
  for agent-browser only when you need a real logged-in / interactive / dynamic
  browser. Plus: don't hand-build deep URLs — use links found by interacting.
- "Two ways to drive a page": structured (@ref/find) is convenient but lossy &
  fragile; eval-first (`eval "<js>"`) is the real DOM — read hidden inputs,
  Shadow DOM, form.elements/.validity, or el.click() directly. Drop to eval the
  moment the structured path fights you, instead of retrying it.
- Escalation ladder rewritten (refs → find → CSS → eval) and a note to retry a
  no-op click with AGENT_BROWSER_CLICK_MODE=dom.

Doc-only; closes the biggest part of the "abs feels worse than web-access" gap.
2026-06-05 16:44:29 +09:00
leeguooooo 5f50ca075c chore(release): bump to 0.27.0-fork.16 — click reliability (scroll-into-view + DOM fallback) + skill docs (console opt-in, CLICK_MODE, form/hidden-input eval)
Release binaries / Build macOS ARM64 (push) Has been cancelled
Release binaries / Build macOS x64 (push) Has been cancelled
Release binaries / Build Linux ARM64 (push) Has been cancelled
Release binaries / Build Linux musl ARM64 (push) Has been cancelled
Release binaries / Build Linux musl x64 (push) Has been cancelled
Release binaries / Build Linux x64 (push) Has been cancelled
Release binaries / Build Windows x64 (push) Has been cancelled
Release binaries / Attach binaries to GitHub Release (push) Has been cancelled
2026-06-05 11:00:57 +09:00
leeguooooo e7548c3eb5 fix(click): scroll into view + DOM-dispatch fallback for reliable clicks
Real-world dogfooding surfaced clicks that resolve a valid @ref but still miss:

- Scroll the target into view before computing click coordinates
  (scrollIntoViewIfNeeded). Without it, an element below the fold — or revealed
  after a scroll/popup — yields off-viewport coordinates and the click lands on
  whatever occupies that screen point.
- Fall back to a DOM-dispatched `.click()` when the coordinate path fails (a
  persistent floating layer failing the occlusion guard, or coordinates that
  won't resolve). The DOM dispatch targets the intended element directly instead
  of a screen point, so an overlay or portal can't divert it.
- AGENT_BROWSER_CLICK_MODE: "" (default: scroll + coordinate + DOM fallback),
  "coord" (strict coordinate, hard-fail on occlusion), "dom" (always
  element.click() — best for autocomplete/menu <li> that close on input blur).

Fallback is limited to left single-clicks (DOM .click() can't express
right/middle/double). Non-left/multi and "coord" mode keep the original error.

Docs: README knob table + skill commands.md gain CLICK_MODE, a click-reliability
note, and a "debug forms/hidden inputs with eval" section (snapshot doesn't show
hidden inputs — the fast path to bugs like a hidden point_choice=none).

6 click/interaction e2e green; full suite 760 passed.
2026-06-05 10:58:28 +09:00
leeguooooo b77a1e4568 docs(skills): document console-capture opt-in + stealth env knobs
console/errors capture is off by default in this fork (Runtime.enable is a
detectable CDP signal). Update the agent-facing skill docs so agents don't
treat empty console output as a bug:

- commands.md: new "Stealth / anti-detection knobs" env-var block
  (CAPTURE_CONSOLE, TIMEZONE, BLOCK_WEBRTC, HIDE_CANVAS, ADAPTIVE_REF) plus a
  heads-up note; annotate the console/errors lines.
- dogfood/slack SKILL.md: note that console/errors need
  AGENT_BROWSER_CAPTURE_CONSOLE=1.
2026-06-04 17:08:14 +09:00
26 changed files with 2221 additions and 14 deletions
+4
View File
@@ -67,3 +67,7 @@ docs/package-lock.json
# next
.next/
out/
# extension signing key (never commit) + local-only id record
.secrets/
*.pem
View File
+1
View File
@@ -139,6 +139,7 @@ When using `--launch` mode (standalone browser), a full suite of 32 stealth patc
| `AGENT_BROWSER_BLOCK_WEBRTC` | auto | `--launch` only. Auto-forces WebRTC through the proxy when one is set (no real-IP leak). `1` hides the local IP without a proxy; `0` opts out. |
| `AGENT_BROWSER_HIDE_CANVAS` | off | `--launch` only. Adds session-stable canvas/audio fingerprint noise. Off by default (noise is itself a "lie"). |
| `AGENT_BROWSER_ADAPTIVE_REF` | on | When a saved `@ref` moves and the role/name re-query fails, relocate it by fingerprint similarity (high score + clear margin required, else it fails loudly). `0` disables. |
| `AGENT_BROWSER_CLICK_MODE` | _(auto)_ | Click strategy. Default scrolls the target into view, dispatches a coordinate click, and falls back to a DOM `.click()` if a floating layer occludes the point. `dom` always uses `.click()` (best for autocomplete/menu items that close on blur); `coord` is strict coordinate-only (hard-fail on occlusion). |
## Differences from upstream
+1 -1
View File
@@ -45,7 +45,7 @@ dependencies = [
[[package]]
name = "agent-browser-stealth"
version = "0.27.0-fork.15"
version = "0.27.0-fork.19"
dependencies = [
"aes-gcm",
"async-trait",
+2 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "agent-browser-stealth"
version = "0.27.0-fork.15"
version = "0.27.0-fork.19"
edition = "2021"
description = "Fast browser automation CLI for AI agents"
license = "Apache-2.0"
@@ -22,7 +22,7 @@ dirs = "5.0"
include_dir = "0.7"
base64 = "0.22"
getrandom = "0.2"
tokio = { version = "1", features = ["rt-multi-thread", "macros", "net", "io-util", "time", "sync", "signal", "process"] }
tokio = { version = "1", features = ["rt-multi-thread", "macros", "net", "io-util", "io-std", "time", "sync", "signal", "process"] }
tokio-tungstenite = { version = "0.24", features = ["rustls-tls-webpki-roots"] }
futures-util = "0.3"
url = "2"
+571
View File
@@ -0,0 +1,571 @@
//! `agent-browser connect` — zero-confirmation control of the user's real,
//! logged-in Chrome via the `ab-connect` MV3 extension over Chrome **native
//! messaging** (no localhost port, no token; Chrome authenticates the extension
//! to this host by id).
//!
//! Two pieces live here:
//! - `run_connect` — `--install` writes the native-messaging host manifest (and
//! a tiny launcher) so Chrome will spawn us; with no flag it reports status.
//! - `run_nm_host` — the hidden `__nm-host` mode Chrome launches: it speaks the
//! native-messaging stdio framing (4-byte little-endian length + JSON).
//!
//! This step wires the transport end-to-end (Chrome ⇄ host). Bridging the host
//! to the daemon's relay + CdpClient is layered on next.
use std::io::Write;
use std::path::PathBuf;
/// Native-messaging host name; must match `HOST_NAME` in the extension and the
/// manifest filename.
pub const HOST_NAME: &str = "com.agent_browser.connect";
/// Stable id of the `ab-connect` extension, pinned by the `key` in its
/// manifest.json (and the signing key of the published `.crx`). Chrome only lets
/// that extension talk to this host, and the force-install policy references it.
pub const EXTENSION_ID: &str = "ciiljdlhdpfckdcfkphgmfalanpdejep";
/// Update URL the force-install policy points at. MUST be the Chrome Web Store
/// endpoint: Chrome 149 tags any **off-Web-Store** force-installed extension
/// `[BLOCKED]` on an unmanaged browser (verified on macOS — chrome://policy shows
/// `[BLOCKED]…` / "Error, Warning"). Self-hosting a `.crx` therefore does NOT
/// work on consumer Chrome; the extension must be published to the Web Store, and
/// then this policy force-installs it silently (Web Store extensions are allowed).
pub const UPDATE_URL: &str = "https://clients2.google.com/service/update2/crx";
/// Public Web Store listing — the guaranteed one-click "Add to Chrome" path,
/// and the fallback when the force-install profile can't be approved headlessly.
pub const STORE_URL: &str = "https://chromewebstore.google.com/detail/ciiljdlhdpfckdcfkphgmfalanpdejep";
/// Stable identifiers for the generated Chrome configuration profile, so a
/// re-install replaces (rather than duplicates) it in System Settings.
const PROFILE_ID: &str = "work.pwtk.agent-browser.ab-connect";
const PROFILE_UUID: &str = "A1B2C3D4-AB00-4CCE-9E10-AAAABBBBCCCC";
const PROFILE_PAYLOAD_UUID: &str = "A1B2C3D4-AB01-4CCE-9E10-DDDDEEEEFFFF";
/// `agent-browser extension <install|uninstall|status>` (local; no daemon).
/// `args` is the cleaned argv including the leading "extension".
pub fn run_connect(args: &[String], json: bool) {
let install = args.iter().any(|a| a == "--install" || a == "install");
let uninstall = args.iter().any(|a| a == "--uninstall" || a == "uninstall");
if uninstall {
let removed = remove_host_manifests();
let profile_removed = remove_force_install_profile();
if json {
report(json, true, &format!("removed {removed} native-host manifest(s)"));
} else {
println!("✓ removed {removed} native-host manifest(s).");
if profile_removed {
println!("✓ removed ~/.agent-browser/ab-connect.mobileconfig");
}
if cfg!(target_os = "macos") {
println!(
" To fully remove the extension, delete the \"agent-browser connect\" profile\n\
in System Settings → Profiles (or run: profiles remove -identifier {PROFILE_ID})."
);
}
}
return;
}
if install {
let no_open = args.iter().any(|a| a == "--no-open");
match install_native_host() {
Ok(paths) => {
let profile = install_force_install_profile(no_open);
if json {
println!(
"{}",
serde_json::to_string(&serde_json::json!({
"success": true,
"data": {
"installed": paths,
"extensionId": EXTENSION_ID,
"profile": profile.as_ref().ok().map(|p| p.display().to_string()),
"profileError": profile.as_ref().err(),
"updateUrl": UPDATE_URL,
}
}))
.unwrap_or_default()
);
} else {
println!("✓ native-messaging host installed:");
for p in &paths {
println!(" {p}");
}
match profile {
Ok(path) => {
println!("\n✓ Chrome force-install profile written:\n {}", path.display());
if cfg!(target_os = "macos") {
println!(
"\nGet the extension into Chrome (one-time). Either:\n\
A) One click: open {STORE_URL}\n and press \"Add to Chrome\".\n\
B) Silent: approve the profile, then restart Chrome —\n \
System Settings → General → Device Management → double-click\n \
\"agent-browser connect\" → Install. Chrome then force-installs +\n \
auto-updates it (no token, no per-use confirmation).\n\
Both need the extension published to the Web Store; until then use\n \
chrome://extensions → Developer mode → Load unpacked → extensions/ab-connect."
);
}
}
Err(e) => {
println!("\n! could not write the force-install profile: {e}");
println!(
" Fallback: load extensions/ab-connect via chrome://extensions →\n\
Developer mode → Load unpacked."
);
}
}
}
}
Err(e) => report(json, false, &format!("install failed: {e}")),
}
return;
}
// Status.
let manifest = host_manifest_path_for_chrome();
let installed = manifest.as_ref().map(|p| p.exists()).unwrap_or(false);
if json {
println!(
"{}",
serde_json::to_string(&serde_json::json!({
"success": true,
"data": {
"installed": installed,
"manifest": manifest.as_ref().map(|p| p.display().to_string()),
"extensionId": EXTENSION_ID,
}
}))
.unwrap_or_default()
);
} else if installed {
println!("✓ native-messaging host installed ({HOST_NAME}).");
println!(" Load the ab-connect extension and it connects automatically.");
} else {
println!("✗ not installed. Run: agent-browser connect --install");
}
}
/// Write the launcher script + native-messaging host manifest(s).
fn install_native_host() -> Result<Vec<String>, String> {
let home = dirs::home_dir().ok_or("no home dir")?;
let ab_dir = home.join(".agent-browser");
std::fs::create_dir_all(&ab_dir).map_err(|e| e.to_string())?;
// Chrome execs the manifest `path` directly with the calling extension's
// origin as argv[1]; a launcher lets us run the binary in __nm-host mode
// regardless of how/where agent-browser is installed.
let exe = std::env::current_exe().map_err(|e| e.to_string())?;
let launcher = ab_dir.join("nm-host.sh");
let script = format!(
"#!/bin/sh\n# agent-browser native-messaging host launcher (auto-generated)\nexec \"{}\" __nm-host \"$@\"\n",
exe.display()
);
std::fs::write(&launcher, script).map_err(|e| e.to_string())?;
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
let _ = std::fs::set_permissions(&launcher, std::fs::Permissions::from_mode(0o755));
}
let manifest = serde_json::json!({
"name": HOST_NAME,
"description": "agent-browser connect — native messaging host",
"path": launcher.display().to_string(),
"type": "stdio",
"allowed_origins": [format!("chrome-extension://{EXTENSION_ID}/")],
});
let body = serde_json::to_string_pretty(&manifest).map_err(|e| e.to_string())?;
let mut written = Vec::new();
for dir in native_messaging_dirs() {
if let Some(parent) = dir.parent() {
if !parent.exists() {
continue; // that browser isn't installed
}
}
std::fs::create_dir_all(&dir).map_err(|e| e.to_string())?;
let path = dir.join(format!("{HOST_NAME}.json"));
std::fs::write(&path, &body).map_err(|e| e.to_string())?;
written.push(path.display().to_string());
}
if written.is_empty() {
return Err("no Chrome/Chromium NativeMessagingHosts directory found".into());
}
Ok(written)
}
/// Write a Chrome configuration profile that force-installs `ab-connect` from
/// [`UPDATE_URL`], and (unless `no_open`) `open` it so the user approves it once
/// in System Settings. Returns the profile path. macOS only — elsewhere it
/// returns an error and the caller prints the manual fallback.
fn install_force_install_profile(no_open: bool) -> Result<PathBuf, String> {
if !cfg!(target_os = "macos") {
return Err("force-install profile is macOS-only; on Linux set Chrome's \
ExtensionInstallForcelist policy JSON, or Load unpacked from chrome://extensions"
.into());
}
let home = dirs::home_dir().ok_or("no home dir")?;
let ab_dir = home.join(".agent-browser");
std::fs::create_dir_all(&ab_dir).map_err(|e| e.to_string())?;
let path = ab_dir.join("ab-connect.mobileconfig");
std::fs::write(&path, force_install_mobileconfig()).map_err(|e| e.to_string())?;
if !no_open {
// `open` queues the profile in System Settings for one-time approval.
let _ = std::process::Command::new("open").arg(&path).status();
}
Ok(path)
}
/// The `.mobileconfig` payload: a user-scope Chrome policy that force-installs
/// the extension by id from our hosted update manifest. User scope installs
/// without admin — just a one-time approval click.
fn force_install_mobileconfig() -> String {
let forcelist = format!("{EXTENSION_ID};{UPDATE_URL}");
format!(
r#"<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>PayloadContent</key>
<array>
<dict>
<key>PayloadType</key><string>com.google.Chrome</string>
<key>PayloadVersion</key><integer>1</integer>
<key>PayloadIdentifier</key><string>{PROFILE_ID}.chrome</string>
<key>PayloadUUID</key><string>{PROFILE_PAYLOAD_UUID}</string>
<key>PayloadEnabled</key><true/>
<key>PayloadDisplayName</key><string>agent-browser connect (Chrome)</string>
<key>ExtensionInstallForcelist</key>
<array>
<string>{forcelist}</string>
</array>
</dict>
</array>
<key>PayloadType</key><string>Configuration</string>
<key>PayloadVersion</key><integer>1</integer>
<key>PayloadIdentifier</key><string>{PROFILE_ID}</string>
<key>PayloadUUID</key><string>{PROFILE_UUID}</string>
<key>PayloadDisplayName</key><string>agent-browser connect</string>
<key>PayloadDescription</key><string>Force-installs the agent-browser connect extension so agent-browser can drive your logged-in Chrome. No token, no per-use confirmation.</string>
<key>PayloadOrganization</key><string>agent-browser-stealth</string>
<key>PayloadScope</key><string>User</string>
<key>PayloadRemovalDisallowed</key><false/>
</dict>
</plist>
"#
)
}
/// Remove the generated `.mobileconfig` file (the profile itself is removed by
/// the user from System Settings, or via `profiles remove`).
fn remove_force_install_profile() -> bool {
dirs::home_dir()
.map(|h| h.join(".agent-browser").join("ab-connect.mobileconfig"))
.filter(|p| p.exists())
.map(|p| std::fs::remove_file(&p).is_ok())
.unwrap_or(false)
}
fn remove_host_manifests() -> usize {
let mut n = 0;
for dir in native_messaging_dirs() {
let path = dir.join(format!("{HOST_NAME}.json"));
if path.exists() && std::fs::remove_file(&path).is_ok() {
n += 1;
}
}
n
}
/// Per-OS NativeMessagingHosts directories for Chrome + Chromium-family browsers.
fn native_messaging_dirs() -> Vec<PathBuf> {
let mut dirs_out = Vec::new();
#[cfg(target_os = "macos")]
{
if let Some(app_support) = dirs::config_dir() {
for sub in [
"Google/Chrome",
"Google/Chrome Beta",
"Google/Chrome Canary",
"Chromium",
"Microsoft Edge",
"BraveSoftware/Brave-Browser",
] {
dirs_out.push(app_support.join(sub).join("NativeMessagingHosts"));
}
}
}
#[cfg(all(unix, not(target_os = "macos")))]
{
if let Some(config) = dirs::config_dir() {
for sub in ["google-chrome", "chromium", "microsoft-edge", "BraveSoftware/Brave-Browser"] {
dirs_out.push(config.join(sub).join("NativeMessagingHosts"));
}
}
}
dirs_out
}
fn host_manifest_path_for_chrome() -> Option<PathBuf> {
native_messaging_dirs()
.into_iter()
.map(|d| d.join(format!("{HOST_NAME}.json")))
.find(|p| p.exists())
.or_else(|| {
native_messaging_dirs()
.into_iter()
.next()
.map(|d| d.join(format!("{HOST_NAME}.json")))
})
}
fn report(json: bool, ok: bool, msg: &str) {
if json {
println!(
"{}",
serde_json::to_string(&serde_json::json!({ "success": ok, "error": if ok { serde_json::Value::Null } else { serde_json::json!(msg) }, "message": msg }))
.unwrap_or_default()
);
} else if ok {
println!("{msg}");
} else {
eprintln!("{msg}");
}
if !ok {
std::process::exit(1);
}
}
// ---- native messaging host (`__nm-host`) ----------------------------------
fn nm_log(line: &str) {
let path = dirs::home_dir()
.map(|h| h.join(".agent-browser").join("nm-host.log"))
.unwrap_or_else(|| PathBuf::from("/tmp/ab-nm-host.log"));
if let Some(p) = path.parent() {
let _ = std::fs::create_dir_all(p);
}
if let Ok(mut f) = std::fs::OpenOptions::new().create(true).append(true).open(&path) {
let _ = writeln!(f, "{line}");
}
}
fn random_guid() -> String {
let mut b = [0u8; 16];
let _ = getrandom::getrandom(&mut b);
b.iter().map(|x| format!("{x:02x}")).collect()
}
/// Where the daemon/CLI reads the relay's CDP WebSocket URL (perms 600).
fn relay_url_path() -> PathBuf {
dirs::home_dir()
.map(|h| h.join(".agent-browser").join("relay-cdp-url"))
.unwrap_or_else(|| PathBuf::from("/tmp/ab-relay-cdp-url"))
}
/// The live relay CDP WebSocket URL, if the native-messaging host is running
/// (it writes the file on connect and removes it on exit). Used by
/// `agent-browser extension connect` to attach without the user copying a URL.
pub fn relay_url() -> Option<String> {
let s = std::fs::read_to_string(relay_url_path()).ok()?;
let s = s.trim().to_string();
if s.starts_with("ws://") {
Some(s)
} else {
None
}
}
/// Hidden `__nm-host` mode: launched by Chrome for the ab-connect extension.
///
/// Bridges the extension (native-messaging stdio, envelope protocol) to a local
/// **CDP WebSocket endpoint** that agent-browser connects to like any Chrome.
/// `relay::RelayState` translates envelope ⇄ raw CDP and emulates browser-level
/// Target discovery. The ws URL carries an unguessable guid (written to a 600
/// file) so only this user's agent-browser — not arbitrary local processes —
/// can drive the browser. No token, no user interaction.
pub fn run_nm_host() {
let rt = match tokio::runtime::Builder::new_multi_thread().enable_all().build() {
Ok(rt) => rt,
Err(e) => {
nm_log(&format!("[nm-host] runtime build failed: {e}"));
return;
}
};
rt.block_on(nm_host_main());
}
async fn nm_host_main() {
use crate::native::relay::{RelayOut, RelayState};
use std::sync::Arc;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::sync::{broadcast, mpsc, Mutex};
nm_log(&format!(
"[nm-host] start argv={:?}",
std::env::args().skip(1).collect::<Vec<_>>()
));
let listener = match tokio::net::TcpListener::bind("127.0.0.1:0").await {
Ok(l) => l,
Err(e) => {
nm_log(&format!("[nm-host] bind failed: {e}"));
return;
}
};
let port = listener.local_addr().map(|a| a.port()).unwrap_or(0);
let guid = random_guid();
let url = format!("ws://127.0.0.1:{port}/{guid}");
let url_path = relay_url_path();
if let Some(p) = url_path.parent() {
let _ = std::fs::create_dir_all(p);
}
if std::fs::write(&url_path, &url).is_ok() {
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
let _ = std::fs::set_permissions(&url_path, std::fs::Permissions::from_mode(0o600));
}
}
nm_log(&format!("[nm-host] cdp endpoint {url}"));
let state = Arc::new(Mutex::new(RelayState::new()));
let (to_clients, _) = broadcast::channel::<String>(4096);
let (to_ext, mut to_ext_rx) = mpsc::channel::<Vec<u8>>(4096);
// Single writer to Chrome (extension) over stdout, native-messaging framed.
tokio::spawn(async move {
let mut out = tokio::io::stdout();
while let Some(frame) = to_ext_rx.recv().await {
let len = (frame.len() as u32).to_ne_bytes();
if out.write_all(&len).await.is_err() || out.write_all(&frame).await.is_err() {
break;
}
let _ = out.flush().await;
}
});
// Accept agent-browser CDP clients on the guid-scoped ws endpoint.
{
let state = state.clone();
let to_clients = to_clients.clone();
let to_ext = to_ext.clone();
let guid = guid.clone();
tokio::spawn(async move {
loop {
let (stream, _) = match listener.accept().await {
Ok(x) => x,
Err(_) => break,
};
let st = state.clone();
let rx = to_clients.subscribe();
let tx = to_ext.clone();
let g = guid.clone();
tokio::spawn(async move {
handle_cdp_client(stream, g, st, rx, tx).await;
});
}
});
}
// Extension → host frames.
let mut stdin = tokio::io::stdin();
loop {
let mut len_buf = [0u8; 4];
if stdin.read_exact(&mut len_buf).await.is_err() {
break;
}
let len = u32::from_ne_bytes(len_buf) as usize;
let mut buf = vec![0u8; len];
if stdin.read_exact(&mut buf).await.is_err() {
break;
}
let v: serde_json::Value = match serde_json::from_slice(&buf) {
Ok(v) => v,
Err(_) => continue,
};
let outs = {
let mut s = state.lock().await;
s.handle_ext_message(&v, "")
};
for o in outs {
match o {
RelayOut::ToClient(m) => {
let _ = to_clients.send(m.to_string());
}
RelayOut::ToExt(m) => {
let _ = to_ext.send(m.to_string().into_bytes()).await;
}
}
}
}
nm_log("[nm-host] stdin EOF — Chrome closed the port");
let _ = std::fs::remove_file(relay_url_path());
}
async fn handle_cdp_client(
stream: tokio::net::TcpStream,
guid: String,
state: std::sync::Arc<tokio::sync::Mutex<crate::native::relay::RelayState>>,
mut from_relay: tokio::sync::broadcast::Receiver<String>,
to_ext: tokio::sync::mpsc::Sender<Vec<u8>>,
) {
use crate::native::relay::ClientRoute;
use futures_util::{SinkExt, StreamExt};
use tokio::sync::broadcast::error::RecvError;
use tokio_tungstenite::tungstenite::Message;
let want_path = format!("/{guid}");
let cb = |req: &tokio_tungstenite::tungstenite::handshake::server::Request,
resp: tokio_tungstenite::tungstenite::handshake::server::Response| {
if req.uri().path() == want_path {
Ok(resp)
} else {
let mut reject = tokio_tungstenite::tungstenite::handshake::server::ErrorResponse::new(
Some("forbidden".to_string()),
);
*reject.status_mut() = tokio_tungstenite::tungstenite::http::StatusCode::FORBIDDEN;
Err(reject)
}
};
let ws = match tokio_tungstenite::accept_hdr_async(stream, cb).await {
Ok(ws) => ws,
Err(_) => return,
};
nm_log("[nm-host] cdp client connected");
// Ask the extension to (re)attach + announce every tab so this client
// discovers the user's existing tabs instead of racing an empty list.
let _ = to_ext.send(br#"{"method":"attachAll"}"#.to_vec()).await;
let (mut tx, mut rx) = ws.split();
loop {
tokio::select! {
relayed = from_relay.recv() => match relayed {
Ok(text) => { if tx.send(Message::Text(text)).await.is_err() { break } }
Err(RecvError::Lagged(_)) => continue,
Err(RecvError::Closed) => break,
},
incoming = rx.next() => match incoming {
Some(Ok(Message::Text(text))) => {
let v: serde_json::Value = match serde_json::from_str(&text) {
Ok(v) => v,
Err(_) => continue,
};
let route = { state.lock().await.route_client_command(&v) };
match route {
ClientRoute::Local(reply) => {
if tx.send(Message::Text(reply.to_string())).await.is_err() { break }
}
ClientRoute::Forward(env) => {
let _ = to_ext.send(env.to_string().into_bytes()).await;
}
}
}
Some(Ok(Message::Close(_))) | None => break,
_ => {}
},
}
}
nm_log("[nm-host] cdp client disconnected");
}
+244
View File
@@ -0,0 +1,244 @@
//! `find-url` — search the user's local Chrome/Edge **bookmarks** for pages they
//! saved, by keyword. Borrowed from web-access's `find-url.mjs`; lets an agent
//! locate an internal system or a previously-saved page that public search
//! can't reach, without opening a browser.
//!
//! v1 covers bookmarks only (a zero-dependency JSON read). Visited-history lives
//! in a locked SQLite DB and would need a SQLite dependency — not included yet.
use std::path::PathBuf;
use serde_json::Value;
use crate::color;
struct Hit {
name: String,
url: String,
folder: String,
date_added: i64,
}
/// Entry point for the `find-url` subcommand. `args` is the full cleaned argv
/// (including the leading "find-url").
pub fn run_find_url(args: &[String], json: bool) {
// Parse flags out of args[1..]; everything else is a keyword.
let mut browser = "chrome".to_string();
let mut profile = "Default".to_string();
let mut limit: usize = 20;
let mut keywords: Vec<String> = Vec::new();
let mut i = 1;
while i < args.len() {
match args[i].as_str() {
"--browser" => {
if let Some(v) = args.get(i + 1) {
browser = v.to_lowercase();
i += 1;
}
}
"--profile" => {
if let Some(v) = args.get(i + 1) {
profile = v.clone();
i += 1;
}
}
"--limit" => {
if let Some(v) = args.get(i + 1).and_then(|s| s.parse::<usize>().ok()) {
limit = v;
i += 1;
}
}
"--json" => {}
other if other.starts_with("--") => {}
other => keywords.push(other.to_lowercase()),
}
i += 1;
}
let path = match bookmarks_path(&browser, &profile) {
Some(p) => p,
None => {
emit_error(
json,
&format!("Could not locate {browser} bookmarks for profile '{profile}'"),
);
return;
}
};
let raw = match std::fs::read_to_string(&path) {
Ok(r) => r,
Err(e) => {
emit_error(json, &format!("Failed to read {}: {e}", path.display()));
return;
}
};
let root: Value = match serde_json::from_str(&raw) {
Ok(v) => v,
Err(e) => {
emit_error(json, &format!("Failed to parse bookmarks JSON: {e}"));
return;
}
};
let mut hits: Vec<Hit> = Vec::new();
if let Some(roots) = root.get("roots").and_then(|r| r.as_object()) {
for node in roots.values() {
walk(node, "", &keywords, &mut hits);
}
}
// Most-recently-added first (date_added is microseconds since 1601).
hits.sort_by(|a, b| b.date_added.cmp(&a.date_added));
hits.truncate(limit);
if json {
let arr: Vec<Value> = hits
.iter()
.map(|h| {
serde_json::json!({
"name": h.name,
"url": h.url,
"folder": h.folder,
})
})
.collect();
println!(
"{}",
serde_json::to_string(&serde_json::json!({
"success": true,
"data": { "results": arr, "count": hits.len() },
}))
.unwrap_or_default()
);
return;
}
if hits.is_empty() {
let kw = if keywords.is_empty() {
String::new()
} else {
format!(" matching {:?}", keywords.join(" "))
};
println!("No {browser} bookmarks found{kw}.");
return;
}
for h in &hits {
if h.folder.is_empty() {
println!("{}\n {}", h.name, h.url);
} else {
println!("{} ({})\n {}", h.name, h.folder, h.url);
}
}
}
/// Recursively walk a bookmark node, collecting URL entries that match every
/// keyword (in name or url). Empty keyword list matches everything.
fn walk(node: &Value, folder: &str, keywords: &[String], out: &mut Vec<Hit>) {
match node.get("type").and_then(|t| t.as_str()) {
Some("url") => {
let name = node.get("name").and_then(|v| v.as_str()).unwrap_or("");
let url = node.get("url").and_then(|v| v.as_str()).unwrap_or("");
// Skip non-navigable bookmarks: javascript: bookmarklets and data:
// URIs aren't pages you can visit, and their bodies can be huge.
if url.is_empty()
|| url.starts_with("javascript:")
|| url.starts_with("data:")
{
return;
}
let hay = format!("{} {}", name.to_lowercase(), url.to_lowercase());
if keywords.iter().all(|k| hay.contains(k.as_str())) {
let date_added = node
.get("date_added")
.and_then(|v| v.as_str())
.and_then(|s| s.parse::<i64>().ok())
.unwrap_or(0);
out.push(Hit {
name: name.to_string(),
url: url.to_string(),
folder: folder.to_string(),
date_added,
});
}
}
Some("folder") => {
let fname = node.get("name").and_then(|v| v.as_str()).unwrap_or("");
let child_folder = if folder.is_empty() {
fname.to_string()
} else {
format!("{folder}/{fname}")
};
if let Some(children) = node.get("children").and_then(|c| c.as_array()) {
for child in children {
walk(child, &child_folder, keywords, out);
}
}
}
_ => {}
}
}
/// Resolve the Bookmarks file path for a browser + profile across platforms.
fn bookmarks_path(browser: &str, profile: &str) -> Option<PathBuf> {
let base = browser_user_data_dir(browser)?;
let path = base.join(profile).join("Bookmarks");
if path.exists() {
Some(path)
} else {
None
}
}
/// The "User Data" directory that holds per-profile folders, per OS/browser.
fn browser_user_data_dir(browser: &str) -> Option<PathBuf> {
let is_edge = browser == "edge" || browser == "msedge";
#[cfg(target_os = "macos")]
{
let app_support = dirs::config_dir()?; // ~/Library/Application Support
let sub = if is_edge {
"Microsoft Edge"
} else {
"Google/Chrome"
};
Some(app_support.join(sub))
}
#[cfg(target_os = "windows")]
{
let local = dirs::data_local_dir()?; // %LOCALAPPDATA%
let sub = if is_edge {
"Microsoft/Edge/User Data"
} else {
"Google/Chrome/User Data"
};
Some(local.join(sub))
}
#[cfg(all(unix, not(target_os = "macos")))]
{
let config = dirs::config_dir()?; // ~/.config
let sub = if is_edge {
"microsoft-edge"
} else {
"google-chrome"
};
Some(config.join(sub))
}
}
fn emit_error(json: bool, msg: &str) {
if json {
println!(
"{}",
serde_json::to_string(&serde_json::json!({
"success": false,
"error": msg,
}))
.unwrap_or_default()
);
} else {
eprintln!("{} {msg}", color::error_indicator());
}
std::process::exit(1);
}
+45 -1
View File
@@ -2,7 +2,9 @@ mod chat;
mod color;
mod commands;
mod connection;
mod connect;
mod doctor;
mod findurl;
mod flags;
mod install;
mod native;
@@ -502,6 +504,14 @@ fn main() {
env::set_var("MSYS2_ARG_CONV_EXCL", "*");
}
// Native-messaging host mode: Chrome launches `agent-browser __nm-host
// <extension-origin> [...]` for the ab-connect extension. Must run before
// ANY stdout write — stdout is the Chrome native-messaging channel.
if env::args().nth(1).as_deref() == Some("__nm-host") {
connect::run_nm_host();
return;
}
// Native daemon mode: when AGENT_BROWSER_DAEMON is set, run as the daemon process
if env::var("AGENT_BROWSER_DAEMON").is_ok() {
// Ignore SIGPIPE so the daemon isn't killed when the parent drops
@@ -529,7 +539,7 @@ fn main() {
let args: Vec<String> = env::args().skip(1).collect();
let mut flags = parse_flags(&args);
let clean = clean_args(&args);
let mut clean = clean_args(&args);
// Loudly warn when launching a fresh browser with no profile: it gets a
// temporary EMPTY profile (no cookies / no login). For logged-in sites the
@@ -631,6 +641,40 @@ fn main() {
return;
}
// Handle find-url (doesn't need daemon): search local bookmarks
if matches!(
clean.first().map(|s| s.as_str()),
Some("find-url") | Some("findurl")
) {
findurl::run_find_url(&clean, flags.json);
return;
}
// Handle extension: native-messaging host install/status, and
// `extension connect` which attaches to the live relay (auto-discovers the
// CDP url the host wrote) by rewriting into the normal `connect <url>` flow.
// (`connect <port>` stays the plain CDP-attach command.)
if clean.first().map(|s| s.as_str()) == Some("extension") {
if clean.get(1).map(|s| s.as_str()) == Some("connect") {
match connect::relay_url() {
Some(url) => {
clean = vec!["connect".to_string(), url];
// fall through to the normal connect handling below
}
None => {
eprintln!(
"{} extension not connected. Run `agent-browser extension install`, load the\n ab-connect extension in Chrome (chrome://extensions → Developer mode →\n Load unpacked → extensions/ab-connect), then retry.",
color::error_indicator()
);
exit(1);
}
}
} else {
connect::run_connect(&clean, flags.json);
return;
}
}
// Handle session separately (doesn't need daemon)
if clean.first().map(|s| s.as_str()) == Some("session") {
run_session(&clean, &flags.session, flags.json);
+120 -2
View File
@@ -15,7 +15,111 @@ pub async fn click(
click_count: i32,
iframe_sessions: &HashMap<String, String>,
) -> Result<(), String> {
let (x, y, effective_session_id) = resolve_element_center(
// AGENT_BROWSER_CLICK_MODE: "" (default) = coordinate click with a DOM
// fallback; "coord" = strict coordinate only (no fallback); "dom" = always
// dispatch through the DOM.
let mode = std::env::var("AGENT_BROWSER_CLICK_MODE").unwrap_or_default();
// (A) Scroll the target into view first so the computed coordinates land
// inside the viewport. Without this, an element below the fold (or revealed
// after scroll/popup) yields off-viewport coordinates and the click lands on
// whatever currently occupies that point. Best-effort: ignore failures.
scroll_into_view_if_needed(client, session_id, ref_map, selector_or_ref, iframe_sessions).await;
if mode == "dom" {
return dom_click(client, session_id, ref_map, selector_or_ref, iframe_sessions).await;
}
let resolved = resolve_element_center(
client,
session_id,
ref_map,
selector_or_ref,
iframe_sessions,
)
.await;
match resolved {
Ok((x, y, effective_session_id)) => {
dispatch_click(client, &effective_session_id, x, y, button, click_count).await
}
Err(e) => {
// (B) The coordinate path failed — typically a persistent overlay
// failing the occlusion guard, or coordinates that won't resolve.
// Fall back to a DOM-dispatched `.click()` on the intended element,
// which targets the element directly instead of a screen point.
// Skipped for strict "coord" mode and for non-left / multi-clicks
// (a DOM `.click()` can't express right/middle/double semantics).
if mode == "coord" || button != "left" || click_count != 1 {
return Err(e);
}
eprintln!(
"[click] coordinate click failed ({e}); falling back to DOM dispatch \
(set AGENT_BROWSER_CLICK_MODE=coord to disable)"
);
dom_click(client, session_id, ref_map, selector_or_ref, iframe_sessions)
.await
.map_err(|dom_err| format!("{e}\n(DOM-dispatch fallback also failed: {dom_err})"))
}
}
}
/// Best-effort scroll-into-view before a coordinate click. Uses Chrome's
/// `scrollIntoViewIfNeeded` (only scrolls when not already fully visible),
/// falling back to centered `scrollIntoView`. Resolution failures are ignored —
/// the subsequent resolve will surface a real "not found" error.
async fn scroll_into_view_if_needed(
client: &CdpClient,
session_id: &str,
ref_map: &RefMap,
selector_or_ref: &str,
iframe_sessions: &HashMap<String, String>,
) {
let Ok((object_id, effective_session_id)) = resolve_element_object_id(
client,
session_id,
ref_map,
selector_or_ref,
iframe_sessions,
)
.await
else {
return;
};
let js = "function() { try { \
if (typeof this.scrollIntoViewIfNeeded === 'function') { this.scrollIntoViewIfNeeded(true); } \
else { this.scrollIntoView({ block: 'center', inline: 'center' }); } \
} catch (e) {} }";
let _ = client
.send_command_typed::<_, Value>(
"Runtime.callFunctionOn",
&CallFunctionOnParams {
function_declaration: js.to_string(),
object_id: Some(object_id),
arguments: None,
return_by_value: Some(true),
await_promise: Some(false),
},
Some(&effective_session_id),
)
.await;
// Let the scroll settle so the following getBoxModel sees final coordinates.
wait_for_paint_settled(client, &effective_session_id).await;
}
/// Dispatch a click through the DOM (`element.click()`) instead of via screen
/// coordinates. Targets the intended element directly, so it works when a
/// floating layer occludes the click point or the element sits in a portal that
/// confuses `elementFromPoint`. Used as the fallback for `click` and when
/// `AGENT_BROWSER_CLICK_MODE=dom`.
async fn dom_click(
client: &CdpClient,
session_id: &str,
ref_map: &RefMap,
selector_or_ref: &str,
iframe_sessions: &HashMap<String, String>,
) -> Result<(), String> {
let (object_id, effective_session_id) = resolve_element_object_id(
client,
session_id,
ref_map,
@@ -23,7 +127,21 @@ pub async fn click(
iframe_sessions,
)
.await?;
dispatch_click(client, &effective_session_id, x, y, button, click_count).await
client
.send_command_typed::<_, Value>(
"Runtime.callFunctionOn",
&CallFunctionOnParams {
function_declaration: "function() { this.click(); }".to_string(),
object_id: Some(object_id),
arguments: None,
return_by_value: Some(true),
await_promise: Some(false),
},
Some(&effective_session_id),
)
.await?;
wait_for_paint_settled(client, &effective_session_id).await;
Ok(())
}
pub async fn dblclick(
+2
View File
@@ -27,6 +27,8 @@ pub mod policy;
#[allow(dead_code)]
pub mod providers;
#[allow(dead_code)]
pub mod relay;
#[allow(dead_code)]
pub mod react;
#[allow(dead_code)]
pub mod recording;
+362
View File
@@ -0,0 +1,362 @@
//! Relay between the `ab-connect` browser extension and the daemon's `CdpClient`.
//!
//! The extension speaks a small CDP-over-WebSocket "envelope" protocol (adapted
//! from openclaw-browser-relay) and drives the user's real tabs via per-tab
//! `chrome.debugger`. The daemon's `CdpClient`, however, expects a **browser-
//! level** CDP endpoint (`Target.getTargets` / `Target.attachToTarget` → a
//! `sessionId`, then per-session commands). This relay bridges the two: it
//! tracks the targets the extension reports, answers the browser-level
//! `Target.*` discovery commands LOCALLY, and forwards everything else to the
//! extension as `forwardCDPCommand`. That keeps `CdpClient` and `browser.rs`
//! unchanged.
//!
//! This module is the pure translation core (no I/O) so the protocol can be
//! unit-tested; the tokio WebSocket server that drives it lives alongside.
use std::collections::HashMap;
use serde_json::{json, Value};
/// Protocol version advertised in the connect handshake (matches the extension).
pub const RELAY_PROTOCOL: i64 = 3;
/// One target (tab) the extension has attached, as the relay tracks it.
#[derive(Clone)]
struct TargetEntry {
session_id: String,
target_info: Value,
}
/// Relay translation state: the set of targets the extension currently exposes.
#[derive(Default)]
pub struct RelayState {
/// targetId -> entry
targets: HashMap<String, TargetEntry>,
}
/// What to do with a raw CDP command received from the `CdpClient`.
#[derive(Debug, PartialEq)]
pub enum ClientRoute {
/// Answer locally; the value is a raw CDP response `{id, result}`.
Local(Value),
/// Forward to the extension; the value is a `forwardCDPCommand` envelope.
Forward(Value),
}
/// An output the relay emits while handling an extension message.
#[derive(Debug, PartialEq)]
pub enum RelayOut {
/// Send this raw CDP message to the `CdpClient`.
ToClient(Value),
/// Send this envelope message back to the extension.
ToExt(Value),
}
impl RelayState {
pub fn new() -> Self {
Self::default()
}
/// The challenge the relay sends to the extension as soon as it connects,
/// kicking off the connect handshake.
pub fn connect_challenge(nonce: &str) -> Value {
json!({ "type": "event", "event": "connect.challenge", "payload": { "nonce": nonce } })
}
/// A keepalive ping for the extension.
pub fn ping() -> Value {
json!({ "method": "ping" })
}
/// Route a raw CDP command `{id, method, params?, sessionId?}` from the
/// `CdpClient`: answer browser-level `Target.*` discovery locally, forward
/// the rest to the extension.
pub fn route_client_command(&self, raw: &Value) -> ClientRoute {
let id = raw.get("id").cloned().unwrap_or(Value::Null);
let method = raw.get("method").and_then(|m| m.as_str()).unwrap_or("");
let params = raw.get("params").cloned().unwrap_or_else(|| json!({}));
let session_id = raw.get("sessionId").and_then(|s| s.as_str());
match method {
// Discovery is best-effort and event-driven in real CDP; abs only
// reads the getTargets result, so an empty ack is enough here.
"Target.setDiscoverTargets" | "Target.setAutoAttach" => {
ClientRoute::Local(json!({ "id": id, "result": {} }))
}
"Target.getTargets" => {
let infos: Vec<Value> =
self.targets.values().map(|t| t.target_info.clone()).collect();
ClientRoute::Local(json!({ "id": id, "result": { "targetInfos": infos } }))
}
"Target.attachToTarget" => {
let target_id = params.get("targetId").and_then(|t| t.as_str()).unwrap_or("");
match self.targets.get(target_id) {
Some(entry) => ClientRoute::Local(
json!({ "id": id, "result": { "sessionId": entry.session_id } }),
),
None => ClientRoute::Local(json!({
"id": id,
"error": { "code": -32602, "message": format!("No such target {target_id}") }
})),
}
}
// Everything else goes to the extension's chrome.debugger.
_ => ClientRoute::Forward(json!({
"id": id,
"method": "forwardCDPCommand",
"params": { "method": method, "params": params, "sessionId": session_id },
})),
}
}
/// Handle one decoded message from the extension. Updates target state and
/// returns the messages to emit (to the client and/or back to the
/// extension). `expected_token` is matched against the connect handshake.
pub fn handle_ext_message(&mut self, msg: &Value, expected_token: &str) -> Vec<RelayOut> {
// Connect handshake request from the extension.
if msg.get("type").and_then(|t| t.as_str()) == Some("req")
&& msg.get("method").and_then(|m| m.as_str()) == Some("connect")
{
let id = msg.get("id").cloned().unwrap_or(Value::Null);
let token = msg
.get("params")
.and_then(|p| p.get("auth"))
.and_then(|a| a.get("token"))
.and_then(|t| t.as_str())
.unwrap_or("");
let ok = !expected_token.is_empty() && token == expected_token;
let mut res = json!({ "type": "res", "id": id, "ok": ok });
if !ok {
res["error"] = json!({ "message": "invalid relay token" });
}
return vec![RelayOut::ToExt(res)];
}
// Keepalive.
if msg.get("method").and_then(|m| m.as_str()) == Some("pong") {
return vec![];
}
// Response to a forwardCDPCommand we sent → raw CDP response to client.
if msg.get("id").is_some()
&& (msg.get("result").is_some() || msg.get("error").is_some())
&& msg.get("method").is_none()
{
let mut out = json!({ "id": msg.get("id").cloned().unwrap_or(Value::Null) });
if let Some(r) = msg.get("result") {
out["result"] = r.clone();
}
if let Some(e) = msg.get("error") {
// CdpClient expects an error object; wrap a bare string.
out["error"] = match e {
Value::String(s) => json!({ "code": -32000, "message": s }),
other => other.clone(),
};
}
return vec![RelayOut::ToClient(out)];
}
// CDP event forwarded from a tab.
if msg.get("method").and_then(|m| m.as_str()) == Some("forwardCDPEvent") {
let p = msg.get("params").cloned().unwrap_or_else(|| json!({}));
let inner_method = p.get("method").and_then(|m| m.as_str()).unwrap_or("");
let inner_params = p.get("params").cloned().unwrap_or_else(|| json!({}));
let session_id = p.get("sessionId").and_then(|s| s.as_str());
// Learn/forget targets from the extension's synthesized Target events.
// We consume these to maintain state and do NOT forward them: abs
// discovers targets by pulling getTargets, and forwarding a second
// attachedToTarget would duplicate the one attachToTarget emits.
match inner_method {
"Target.attachedToTarget" => {
if let Some(info) = inner_params.get("targetInfo") {
if let Some(tid) = info.get("targetId").and_then(|t| t.as_str()) {
let sid = inner_params
.get("sessionId")
.and_then(|s| s.as_str())
.unwrap_or("")
.to_string();
self.targets.insert(
tid.to_string(),
TargetEntry { session_id: sid, target_info: info.clone() },
);
}
}
return vec![];
}
"Target.detachedFromTarget" => {
let gone = inner_params.get("sessionId").and_then(|s| s.as_str());
if let Some(gone) = gone {
self.targets.retain(|_, e| e.session_id != gone);
}
return vec![];
}
_ => {}
}
// Regular CDP event → deliver to the client with its sessionId.
let mut ev = json!({ "method": inner_method, "params": inner_params });
if let Some(sid) = session_id {
ev["sessionId"] = json!(sid);
}
return vec![RelayOut::ToClient(ev)];
}
vec![]
}
#[cfg(test)]
fn seed_target(&mut self, target_id: &str, session_id: &str) {
self.targets.insert(
target_id.to_string(),
TargetEntry {
session_id: session_id.to_string(),
target_info: json!({
"targetId": target_id,
"type": "page",
"title": "",
"url": "about:blank",
"attached": true,
}),
},
);
}
}
#[cfg(test)]
mod tests {
use super::*;
fn attached_event(target_id: &str, session_id: &str) -> Value {
json!({
"method": "forwardCDPEvent",
"params": {
"sessionId": session_id,
"method": "Target.attachedToTarget",
"params": {
"sessionId": session_id,
"targetInfo": { "targetId": target_id, "type": "page", "url": "https://x", "title": "X" }
}
}
})
}
#[test]
fn learns_target_from_attached_event_and_does_not_forward_it() {
let mut s = RelayState::new();
let out = s.handle_ext_message(&attached_event("T1", "cb-tab-1"), "tok");
assert!(out.is_empty(), "attachedToTarget should be consumed, not forwarded");
// Now getTargets must report it.
let route = s.route_client_command(&json!({ "id": 1, "method": "Target.getTargets" }));
match route {
ClientRoute::Local(v) => {
let infos = v["result"]["targetInfos"].as_array().unwrap();
assert_eq!(infos.len(), 1);
assert_eq!(infos[0]["targetId"], "T1");
}
_ => panic!("getTargets must be local"),
}
}
#[test]
fn attach_to_target_returns_known_session() {
let mut s = RelayState::new();
s.seed_target("T1", "cb-tab-1");
let route = s.route_client_command(
&json!({ "id": 5, "method": "Target.attachToTarget", "params": { "targetId": "T1", "flatten": true } }),
);
assert_eq!(
route,
ClientRoute::Local(json!({ "id": 5, "result": { "sessionId": "cb-tab-1" } }))
);
}
#[test]
fn attach_to_unknown_target_errors_locally() {
let s = RelayState::new();
let route = s.route_client_command(
&json!({ "id": 6, "method": "Target.attachToTarget", "params": { "targetId": "nope" } }),
);
match route {
ClientRoute::Local(v) => assert!(v.get("error").is_some()),
_ => panic!("should answer locally"),
}
}
#[test]
fn other_commands_forward_as_envelope() {
let s = RelayState::new();
let route = s.route_client_command(&json!({
"id": 9, "method": "Page.navigate",
"params": { "url": "https://x" }, "sessionId": "cb-tab-1"
}));
match route {
ClientRoute::Forward(v) => {
assert_eq!(v["method"], "forwardCDPCommand");
assert_eq!(v["id"], 9);
assert_eq!(v["params"]["method"], "Page.navigate");
assert_eq!(v["params"]["sessionId"], "cb-tab-1");
assert_eq!(v["params"]["params"]["url"], "https://x");
}
_ => panic!("Page.navigate must forward"),
}
}
#[test]
fn forward_command_response_becomes_client_response() {
let mut s = RelayState::new();
let out = s.handle_ext_message(&json!({ "id": 9, "result": { "frameId": "F1" } }), "tok");
assert_eq!(
out,
vec![RelayOut::ToClient(json!({ "id": 9, "result": { "frameId": "F1" } }))]
);
}
#[test]
fn forward_command_error_is_wrapped() {
let mut s = RelayState::new();
let out = s.handle_ext_message(&json!({ "id": 9, "error": "boom" }), "tok");
match &out[0] {
RelayOut::ToClient(v) => {
assert_eq!(v["id"], 9);
assert_eq!(v["error"]["message"], "boom");
}
_ => panic!("expected ToClient"),
}
}
#[test]
fn regular_event_is_forwarded_with_session() {
let mut s = RelayState::new();
let ev = json!({
"method": "forwardCDPEvent",
"params": { "sessionId": "cb-tab-1", "method": "Page.loadEventFired", "params": { "timestamp": 1.0 } }
});
let out = s.handle_ext_message(&ev, "tok");
assert_eq!(
out,
vec![RelayOut::ToClient(json!({
"method": "Page.loadEventFired",
"params": { "timestamp": 1.0 },
"sessionId": "cb-tab-1"
}))]
);
}
#[test]
fn connect_handshake_validates_token() {
let mut s = RelayState::new();
let req = json!({ "type": "req", "id": "c1", "method": "connect", "params": { "auth": { "token": "good" } } });
let ok = s.handle_ext_message(&req, "good");
assert_eq!(ok, vec![RelayOut::ToExt(json!({ "type": "res", "id": "c1", "ok": true }))]);
let bad = s.handle_ext_message(&req, "different");
match &bad[0] {
RelayOut::ToExt(v) => {
assert_eq!(v["ok"], false);
assert!(v.get("error").is_some());
}
_ => panic!("expected ToExt"),
}
}
}
Binary file not shown.
Binary file not shown.
+11
View File
@@ -0,0 +1,11 @@
# Attribution
The chrome.debugger attach + CDP Target handling in `background.js` is adapted
from **openclaw-browser-relay** by chengyixu
(https://github.com/chengyixu/openclaw-browser-relay, MIT per its README).
Changes for agent-browser-stealth: rebranded to "agent-browser connect"; the
transport is rewritten from a localhost WebSocket + shared token to Chrome
**native messaging** (host `com.agent_browser.connect`) — no port, no token,
Chrome authenticates the extension to the host by id. WebSocket/token/options
code removed.
+318
View File
@@ -0,0 +1,318 @@
// agent-browser connect — MV3 service worker.
//
// Bridges the user's real Chrome tabs to the local agent-browser daemon over a
// Chrome **native messaging** channel (no localhost port, no token: Chrome
// authenticates this extension to the host by id). It attaches chrome.debugger
// to eligible tabs and relays CDP both ways via a tiny envelope:
// host → ext : {id, method:"forwardCDPCommand", params:{method,params,sessionId}}
// ext → host : {id, result|error} (command reply)
// ext → host : {method:"forwardCDPEvent", params:{sessionId,method,params}}
//
// Target/discovery semantics (getTargets/attachToTarget) are emulated on the
// daemon side; here we just attach tabs and announce them as
// Target.attachedToTarget so the daemon's CDP client sees them appear.
//
// Adapted from openclaw-browser-relay (MIT, chengyixu) — the chrome.debugger
// attach + Target handling; the transport is rewritten from WebSocket+token to
// native messaging.
const HOST_NAME = 'com.agent_browser.connect'
const SKIP_URL = /^(chrome|chrome-extension|devtools|chrome-untrusted|edge|about):/i
/** @type {chrome.runtime.Port|null} */
let port = null
let nextSession = 1
/** tabId -> { sessionId, targetId } */
const tabs = new Map()
/** sessionId -> tabId (main session per tab) */
const sessionToTab = new Map()
/** child (OOPIF/worker) sessionId -> tabId */
const childSessionToTab = new Map()
function postToHost(msg) {
try {
if (port) port.postMessage(msg)
} catch (e) {
// port died; onDisconnect will reconnect.
}
}
function setBadge(tabId, kind) {
const map = { on: '', connecting: '…', error: '!' }
const colors = { on: '#16a34a', connecting: '#d97706', error: '#b91c1c' }
try {
chrome.action.setBadgeText({ tabId, text: map[kind] ?? '' })
if (colors[kind]) chrome.action.setBadgeBackgroundColor({ tabId, color: colors[kind] })
} catch {}
}
// ---- native messaging transport ------------------------------------------
function connectHost() {
if (port) return
try {
port = chrome.runtime.connectNative(HOST_NAME)
} catch (e) {
port = null
return
}
port.onMessage.addListener((msg) => void whenReady(() => onHostMessage(msg)))
port.onDisconnect.addListener(() => {
port = null
// Sessions are stale once the host is gone; the daemon re-discovers on
// reconnect. Keep chrome.debugger attached so reconnect is cheap.
for (const tabId of tabs.keys()) setBadge(tabId, 'connecting')
})
// Tell the daemon about everything we already have attached, then attach
// anything new.
reannounceAttachedTabs()
void attachAllTabs()
}
async function onHostMessage(msg) {
if (!msg || typeof msg !== 'object') return
// Optional keepalive.
if (msg.method === 'ping') {
postToHost({ method: 'pong' })
return
}
// Daemon (re)connected — (re)attach and announce every tab so it discovers
// the user's existing tabs rather than racing an empty target list.
if (msg.method === 'attachAll') {
reannounceAttachedTabs()
await attachAllTabs()
return
}
if (typeof msg.id !== 'undefined' && msg.method === 'forwardCDPCommand') {
try {
const result = await handleForwardCdpCommand(msg)
postToHost({ id: msg.id, result })
} catch (err) {
postToHost({ id: msg.id, error: err instanceof Error ? err.message : String(err) })
}
}
}
// ---- CDP command dispatch -------------------------------------------------
function tabForSession(sessionId) {
return sessionToTab.get(sessionId) ?? childSessionToTab.get(sessionId) ?? null
}
function tabForTarget(targetId) {
for (const [tabId, t] of tabs.entries()) if (t.targetId === targetId) return tabId
return null
}
function anyConnectedTab() {
const it = tabs.keys().next()
return it.done ? null : it.value
}
async function handleForwardCdpCommand(msg) {
const method = String(msg?.params?.method || '')
const params = msg?.params?.params || undefined
const sessionId = typeof msg?.params?.sessionId === 'string' ? msg.params.sessionId : undefined
// Browser-level Target methods that map onto chrome.tabs.
if (method === 'Target.createTarget') {
const url = typeof params?.url === 'string' && params.url ? params.url : 'about:blank'
const tab = await chrome.tabs.create({ url, active: false })
if (!tab.id) throw new Error('createTarget: no tab id')
await new Promise((r) => setTimeout(r, 100))
const t = await attachTab(tab.id)
return { targetId: t.targetId }
}
if (method === 'Target.closeTarget') {
const tid = typeof params?.targetId === 'string' ? params.targetId : ''
const tabId = tid ? tabForTarget(tid) : null
if (!tabId) return { success: false }
try {
await chrome.tabs.remove(tabId)
} catch {
return { success: false }
}
return { success: true }
}
if (method === 'Target.activateTarget') {
const tid = typeof params?.targetId === 'string' ? params.targetId : ''
const tabId = tid ? tabForTarget(tid) : null
if (tabId) {
const tab = await chrome.tabs.get(tabId).catch(() => null)
if (tab?.windowId) await chrome.windows.update(tab.windowId, { focused: true }).catch(() => {})
await chrome.tabs.update(tabId, { active: true }).catch(() => {})
}
return {}
}
// Everything else → chrome.debugger on the resolved tab.
const tabId =
(sessionId ? tabForSession(sessionId) : null) ??
(typeof params?.targetId === 'string' ? tabForTarget(params.targetId) : null) ??
anyConnectedTab()
if (!tabId) throw new Error(`no attached tab for ${method}`)
const dbg = { tabId }
// Re-enabling Runtime can leave a stale state; bounce it (matches upstream).
if (method === 'Runtime.enable') {
try {
await chrome.debugger.sendCommand(dbg, 'Runtime.disable')
await new Promise((r) => setTimeout(r, 30))
} catch {}
return await chrome.debugger.sendCommand(dbg, 'Runtime.enable', params)
}
return await chrome.debugger.sendCommand(dbg, method, params)
}
// ---- attach / detach ------------------------------------------------------
async function attachTab(tabId) {
const existing = tabs.get(tabId)
if (existing) return existing
const dbg = { tabId }
try {
await chrome.debugger.attach(dbg, '1.3')
} catch (e) {
// After a service-worker restart, chrome.debugger may still be bound to
// this tab from the previous instance — "Another debugger is already
// attached". The tab is still controllable via {tabId}, so don't skip it
// (skipping is why existing tabs went un-announced and the daemon opened a
// blank tab instead). Re-announce it. Any other error (restricted page) is
// surfaced and the caller skips this tab.
const msg = String((e && e.message) || e)
if (!/already attached|already being debugged/i.test(msg)) throw e
}
await chrome.debugger.sendCommand(dbg, 'Page.enable').catch(() => {})
const info = /** @type {any} */ (await chrome.debugger.sendCommand(dbg, 'Target.getTargetInfo'))
const targetInfo = info?.targetInfo
const targetId = String(targetInfo?.targetId || '')
if (!targetId) throw new Error('attachTab: no targetId')
const sessionId = `cb-tab-${nextSession++}`
const entry = { sessionId, targetId }
tabs.set(tabId, entry)
sessionToTab.set(sessionId, tabId)
setBadge(tabId, port ? 'on' : 'connecting')
postToHost({
method: 'forwardCDPEvent',
params: {
sessionId,
method: 'Target.attachedToTarget',
params: { sessionId, targetInfo: { ...targetInfo, attached: true } },
},
})
return entry
}
function detachTab(tabId, notify) {
const entry = tabs.get(tabId)
if (!entry) return
tabs.delete(tabId)
sessionToTab.delete(entry.sessionId)
for (const [sid, tid] of childSessionToTab.entries()) if (tid === tabId) childSessionToTab.delete(sid)
if (notify) {
postToHost({
method: 'forwardCDPEvent',
params: { sessionId: entry.sessionId, method: 'Target.detachedFromTarget', params: { sessionId: entry.sessionId } },
})
}
}
function eligible(tab) {
return !!tab && !!tab.id && typeof tab.url === 'string' && !SKIP_URL.test(tab.url)
}
async function attachAllTabs() {
let all = []
try {
all = await chrome.tabs.query({})
} catch {
return
}
for (const tab of all) {
if (eligible(tab) && !tabs.has(tab.id)) {
try {
await attachTab(tab.id)
} catch {
// Tab may be a restricted page or already attached elsewhere.
}
}
}
}
function reannounceAttachedTabs() {
for (const [, entry] of tabs.entries()) {
postToHost({
method: 'forwardCDPEvent',
params: {
sessionId: entry.sessionId,
method: 'Target.attachedToTarget',
params: { sessionId: entry.sessionId, targetInfo: { targetId: entry.targetId, type: 'page', attached: true } },
},
})
}
}
// ---- chrome.debugger events ----------------------------------------------
chrome.debugger.onEvent.addListener((source, method, params) =>
void whenReady(() => {
const tabId = source.tabId
if (!tabId) return
const entry = tabs.get(tabId)
if (!entry) return
if (method === 'Target.attachedToTarget' && params?.sessionId) {
childSessionToTab.set(String(params.sessionId), tabId)
}
if (method === 'Target.detachedFromTarget' && params?.sessionId) {
childSessionToTab.delete(String(params.sessionId))
}
postToHost({
method: 'forwardCDPEvent',
params: { sessionId: source.sessionId || entry.sessionId, method, params },
})
}),
)
chrome.debugger.onDetach.addListener((source) =>
void whenReady(() => {
if (source.tabId) detachTab(source.tabId, true)
}),
)
// ---- tab lifecycle --------------------------------------------------------
chrome.tabs.onUpdated.addListener((tabId, changeInfo, tab) =>
void whenReady(async () => {
if (changeInfo.status === 'complete' && eligible(tab) && !tabs.has(tabId) && port) {
try {
await attachTab(tabId)
} catch {}
}
}),
)
chrome.tabs.onRemoved.addListener((tabId) => void whenReady(() => detachTab(tabId, true)))
// ---- bootstrap + keepalive ------------------------------------------------
chrome.runtime.onInstalled.addListener(() => void whenReady(connectHost))
chrome.runtime.onStartup.addListener(() => void whenReady(connectHost))
chrome.action.onClicked.addListener(() => void whenReady(connectHost))
// MV3 service workers get suspended; an alarm wakes us to keep the host link
// and badges fresh.
chrome.alarms.create('keepalive', { periodInMinutes: 0.4 })
chrome.alarms.onAlarm.addListener((a) => {
if (a.name !== 'keepalive') return
void whenReady(() => {
if (!port) connectHost()
else void attachAllTabs()
})
})
// Gate placeholder so future async state-rehydration can hook in.
async function whenReady(fn) {
return fn()
}
// Kick a connection attempt as soon as the worker starts.
connectHost()
+28
View File
@@ -0,0 +1,28 @@
{
"manifest_version": 3,
"name": "agent-browser connect",
"version": "0.3.0",
"description": "Let agent-browser drive your logged-in Chrome — install once, no token, no per-use confirmation.",
"key": "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA6vQIyscGIPYPZdSpPwPL0+0gxUROyRgCpmvCSDoc8XUm4qm97VbKnD9Ijc1lV22lNWZtE78gaRjt6BeSfuMgnBymnhLKjN1gU6AI5QUU0mrJyeHdWKvrKQR5FmsM2A7Xr1ykE2SiiS8zNUS3Y/6O5l+Nva7wrVy6E4a2dkBVQkOsu+DV+nEZvhIyuDY5D5SPXqNwUTWTaglwj5mjvHz36xSwCWlPmrtJ+ED0AUyrb2z4GIOmvk4kqtBVrh/UD058klLo4CkYOnIybB5aV6WYuwarfPY4bF/dLggPem+ewLNTUNBuwrxj/A4nUv0LJTuRO8rR7f8WR9qnRCY0Ic5saQIDAQAB",
"icons": {
"16": "icons/icon16.png",
"32": "icons/icon32.png",
"48": "icons/icon48.png",
"128": "icons/icon128.png"
},
"permissions": [
"debugger",
"tabs",
"nativeMessaging",
"storage",
"alarms",
"webNavigation"
],
"background": {
"service_worker": "background.js",
"type": "module"
},
"action": {
"default_title": "agent-browser connect"
}
}
+131
View File
@@ -0,0 +1,131 @@
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Chrome Web Store 提交指南 — agent-browser connect</title>
<style>
:root{--fg:#1a1a1a;--muted:#5c5c5c;--accent:#2563eb;--warn:#b45309;--ok:#15803d;--border:#e2e2e2;--bg:#fff;--code:#f5f5f7}
*{box-sizing:border-box}
body{font-family:-apple-system,BlinkMacSystemFont,"PingFang SC","Microsoft YaHei",sans-serif;color:var(--fg);background:var(--bg);max-width:880px;margin:0 auto;padding:48px 24px;line-height:1.65}
header{border-bottom:2px solid var(--fg);padding-bottom:16px;margin-bottom:24px}
h1{font-size:1.7rem;margin:0 0 4px}
.sub{color:var(--muted)}
h2{font-size:1.2rem;margin:34px 0 10px;border-left:3px solid var(--accent);padding-left:10px}
h3{font-size:1rem;margin:20px 0 6px}
code{background:var(--code);padding:1px 5px;border-radius:4px;font-size:.88em}
pre{background:var(--code);border:1px solid var(--border);border-radius:8px;padding:12px 14px;overflow:auto;font-size:.86rem;white-space:pre-wrap}
table{border-collapse:collapse;width:100%;margin:12px 0;font-size:.92rem}
th,td{border:1px solid var(--border);padding:8px 10px;text-align:left;vertical-align:top}
th{background:var(--code)}
ol li,ul li{margin:6px 0}
.warn{background:#fffbeb;border:1px solid #fde68a;border-left:4px solid var(--warn);padding:12px 14px;border-radius:6px;margin:16px 0}
.ok{background:#f0fdf4;border:1px solid #bbf7d0;border-left:4px solid var(--ok);padding:12px 14px;border-radius:6px;margin:16px 0}
.field{font-weight:600;color:var(--accent)}
footer{margin-top:40px;padding-top:16px;border-top:1px solid var(--border);color:var(--muted);font-size:.85rem}
</style>
</head>
<body>
<header>
<h1>Chrome Web Store 提交指南</h1>
<div class="sub">agent-browser connect · 上传包 <code>extensions/ab-connect.zip</code> · id 锁定为 <code>ciiljdlhdpfckdcfkphgmfalanpdejep</code></div>
</header>
<p>为什么必须走商店:实测 Chrome 149 在<strong>非企业托管</strong>的 Mac 上,会把"非 Web Store"的 force-install 扩展直接标成 <code>[BLOCKED]</code>。商店扩展不受此限。这也是 codex / claude 扩展都发商店的原因。</p>
<div class="warn">
<strong>评审风险(务必知道):</strong> 本扩展用了 <code>debugger</code> 权限,这是 Chrome Web Store 审核最严的权限之一。理由必须写清楚"只在用户本机、用户主动发指令时驱动用户自己的标签页,无远程服务器"。类似工具(如 claude-in-chrome)能过审,但可能被多问一轮、审核时间偏长(几天到一两周)。
</div>
<h2>一、前置(你来做,一次性)</h2>
<ol>
<li>用一个 Google 账号登录 <code>https://chrome.google.com/webstore/devconsole</code></li>
<li>首次需付 <strong>$5</strong> 一次性开发者注册费</li>
<li>(隐私政策需要一个公开 URL,见第四节 —— 我可以帮你开 GitHub Pages 托管 <code>privacy.html</code>)</li>
</ol>
<h2>二、上传</h2>
<ol>
<li>devconsole → <span class="field">New item</span> → 上传 <code>extensions/ab-connect.zip</code></li>
<li>上传后确认分配到的 Item ID = <code>ciiljdlhdpfckdcfkphgmfalanpdejep</code>(因为 manifest 里保留了 <code>key</code>,id 会被锁成这个,native messaging 的 allowed_origins 才对得上)。<strong>若 id 不是这个,告诉我,我重签。</strong></li>
</ol>
<h2>三、商店信息(直接复制以下文案)</h2>
<h3>名称 / Name</h3>
<pre>agent-browser connect</pre>
<h3>简介 / Summary(≤132 字符)</h3>
<pre>Let your own agent-browser CLI drive your logged-in Chrome — a local automation bridge. No remote server, no token.</pre>
<h3>详细描述 / Description</h3>
<pre>agent-browser connect is the in-browser half of the open-source agent-browser CLI. It lets the
command-line tool you installed on this same computer automate the Chrome you're already logged
into — opening pages, clicking, filling forms, reading the DOM — driven entirely by you.
How it works
- The extension talks ONLY to the local agent-browser CLI over Chrome native messaging (a local
inter-process channel — no network socket, no token, no remote server).
- When you run an automation command, the extension relays Chrome DevTools Protocol operations to
the tab you target, then returns the result to the CLI.
Privacy
- No analytics, no trackers, no data collection.
- Nothing is sent to any remote server. The only message peer is the local CLI.
- Source is open (Apache-2.0): https://github.com/leeguooooo/agent-browser-stealth
You need the agent-browser CLI installed and paired (run: agent-browser extension install) for this
extension to do anything.</pre>
<h3>类别 / Category</h3>
<pre>Developer Tools</pre>
<h3>语言 / Language</h3>
<pre>English</pre>
<h2>四、隐私实践(Privacy practices 标签页 —— 必填)</h2>
<h3>Single purpose(单一用途)</h3>
<pre>Bridge the user's locally-installed agent-browser CLI to their own logged-in Chrome so the CLI can
automate pages the user is working with, entirely on the user's machine and at the user's command.</pre>
<h3>各权限理由 / Permission justifications</h3>
<table>
<tr><th>权限</th><th>理由(复制到对应输入框)</th></tr>
<tr><td class="field">debugger</td><td>Attaches the Chrome DevTools Protocol to the user's own active tab so the paired local agent-browser CLI can automate it (navigate, click, read DOM) only while the user is running a command. Commands arrive solely from the local CLI via native messaging; there is no remote endpoint.</td></tr>
<tr><td class="field">tabs</td><td>Enumerate and target the correct open tab to attach automation to.</td></tr>
<tr><td class="field">nativeMessaging</td><td>The sole communication channel: a local native-messaging connection to the agent-browser CLI installed on the same machine. No network is used.</td></tr>
<tr><td class="field">storage</td><td>Persist small local pairing/configuration state for the extension.</td></tr>
<tr><td class="field">alarms</td><td>Keep the MV3 service worker alive during longer automation sessions.</td></tr>
<tr><td class="field">webNavigation</td><td>Detect page loads/navigations so automation can wait for the right moment before acting.</td></tr>
<tr><td class="field">host permissions(若被问)</td><td>The extension declares none; tab access is mediated through the debugger attach the user initiates.</td></tr>
</table>
<h3>数据用途勾选 / Data usage</h3>
<ul>
<li>不勾选任何"collects user data"类别。</li>
<li>三个合规声明全部勾选可以为真:不卖数据 / 不挪作无关用途 / 不用于判断信用资质。</li>
<li><span class="field">Privacy policy URL</span>:填 <code>privacy.html</code> 的公开地址(见下)。</li>
</ul>
<h2>五、隐私政策 URL</h2>
<p>商店要求一个公开可访问的隐私政策地址。GitHub Pages <strong>已开启</strong>,直接填这个(渲染好看):</p>
<pre>https://leeguooooo.github.io/agent-browser-stealth/extensions/store/privacy.html</pre>
<p>(部署需 1–2 分钟生效。raw 备用直链:<code>https://raw.githubusercontent.com/leeguooooo/agent-browser-stealth/main/extensions/store/privacy.html</code>。)</p>
<h2>六、截图 / Screenshots(至少 1 张,1280×800 或 640×400</h2>
<p>可以截一张 CLI + Chrome 并排的演示图。<em>需要的话我用 cua-driver 截一张合规尺寸的图给你。</em></p>
<h2>七、提交后</h2>
<ol>
<li>提交审核 → 等几天。审核通过且状态变 <em>Published</em> 后告诉我。</li>
<li>我会把 <code>extension install</code> 的 force-install <code>update_url</code> 切到商店地址并发布新 fork;之后用户 <code>extension install</code> → 批准一次描述文件 → 静默装好(商店扩展不再 <code>[BLOCKED]</code>);或者用户在商店页一键 <span class="field">Add to Chrome</span></li>
</ol>
<div class="ok">
<strong>今天的临时可用方案:</strong> 在你这台 Mac 上 <code>chrome://extensions</code> → 打开开发者模式 → Load unpacked → 选 <code>extensions/ab-connect</code>,30 秒手动装一次,native messaging + <code>extension connect</code> 立即可用。等商店过审再切静默路径。
</div>
<footer>agent-browser-stealth · 提交包与文案随扩展版本更新;改扩展后重跑 <code>scripts/pack-extension.sh</code> 并重打 <code>ab-connect.zip</code></footer>
</body>
</html>
+77
View File
@@ -0,0 +1,77 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Privacy Policy — agent-browser connect</title>
<style>
:root{
--fg:#1a1a1a; --muted:#5c5c5c; --accent:#2563eb; --border:#e2e2e2; --bg:#fff; --code:#f5f5f5;
}
*{box-sizing:border-box}
body{font-family:-apple-system,BlinkMacSystemFont,"PingFang SC","Microsoft YaHei",sans-serif;
color:var(--fg);background:var(--bg);max-width:820px;margin:0 auto;padding:48px 24px;line-height:1.65}
header{border-bottom:2px solid var(--fg);padding-bottom:16px;margin-bottom:28px}
h1{font-size:1.7rem;margin:0 0 4px}
.sub{color:var(--muted);font-size:.95rem}
h2{font-size:1.15rem;margin:32px 0 8px;border-left:3px solid var(--accent);padding-left:10px}
code{background:var(--code);padding:1px 5px;border-radius:4px;font-size:.88em}
table{border-collapse:collapse;width:100%;margin:12px 0;font-size:.92rem}
th,td{border:1px solid var(--border);padding:8px 10px;text-align:left;vertical-align:top}
th{background:var(--code)}
.key{font-weight:600;color:var(--accent)}
footer{margin-top:40px;padding-top:16px;border-top:1px solid var(--border);color:var(--muted);font-size:.85rem}
strong{color:var(--fg)}
</style>
</head>
<body>
<header>
<h1>Privacy Policy — agent-browser connect</h1>
<div class="sub">Chrome extension (id <code>ciiljdlhdpfckdcfkphgmfalanpdejep</code>) · Last updated 2026-06-09</div>
</header>
<p><strong>Summary: this extension collects no personal data, contains no analytics or
trackers, and sends nothing to any remote server.</strong> It is a local bridge that lets the
user's own <code>agent-browser</code> command-line tool, running on the same computer, drive the
user's logged-in Chrome.</p>
<h2>What the extension does</h2>
<p>agent-browser connect pairs Chrome with the locally-installed <code>agent-browser</code> CLI over
Chrome <em>native messaging</em> (a local inter-process channel; no network socket, no token). When
the user issues an automation command in the CLI, the extension relays Chrome DevTools Protocol
operations to the tab the user targets. Everything happens on the user's machine, initiated by the
user.</p>
<h2>Data collection &amp; use</h2>
<table>
<tr><th>Category</th><th>Collected?</th><th>Detail</th></tr>
<tr><td class="key">Personally identifiable information</td><td>No</td><td>Never read, stored, or transmitted.</td></tr>
<tr><td class="key">Browsing history</td><td>No</td><td>Not collected. Page content is acted on transiently only while the user is running an automation command, and is never stored or sent off-device.</td></tr>
<tr><td class="key">Authentication / cookies / credentials</td><td>No</td><td>Not read or exported by the extension.</td></tr>
<tr><td class="key">Analytics / telemetry</td><td>No</td><td>The extension contains no analytics, tracking, or crash-reporting code.</td></tr>
<tr><td class="key">Remote transmission</td><td>No</td><td>The extension's only message peer is the local <code>agent-browser</code> CLI via native messaging. It makes no outbound network requests of its own.</td></tr>
</table>
<h2>Permissions &amp; why they are needed</h2>
<table>
<tr><th>Permission</th><th>Purpose</th></tr>
<tr><td class="key">debugger</td><td>Attach the Chrome DevTools Protocol to the user's own tab so the local CLI can automate it, only while the user is actively running a command.</td></tr>
<tr><td class="key">tabs</td><td>Enumerate and target the correct open tab to automate.</td></tr>
<tr><td class="key">nativeMessaging</td><td>The local transport to the paired <code>agent-browser</code> CLI — the extension's sole communication channel.</td></tr>
<tr><td class="key">storage</td><td>Persist small local pairing/state values.</td></tr>
<tr><td class="key">alarms</td><td>Keep the MV3 service worker alive during longer automation sessions.</td></tr>
<tr><td class="key">webNavigation</td><td>Detect page loads so automation can wait for the right moment.</td></tr>
</table>
<h2>Data sharing</h2>
<p>None. No data is sold, shared, or transferred to third parties. There are no third parties — the
extension talks only to a program the user installed on the same computer.</p>
<h2>Contact</h2>
<p>Source code, issues, and contact: <code>https://github.com/leeguooooo/agent-browser-stealth</code></p>
<footer>
agent-browser connect is open source (Apache-2.0). This policy applies to the extension only.
</footer>
</body>
</html>
Binary file not shown.

After

Width:  |  Height:  |  Size: 249 KiB

+45
View File
@@ -0,0 +1,45 @@
<!DOCTYPE html>
<html lang="en"><head><meta charset="UTF-8">
<style>
html,body{margin:0;width:1280px;height:800px;overflow:hidden;
font-family:-apple-system,BlinkMacSystemFont,"SF Pro Text",sans-serif;
background:linear-gradient(135deg,#0f172a 0%,#1e293b 100%);color:#e2e8f0}
.wrap{display:flex;flex-direction:column;height:100%;padding:56px 64px;box-sizing:border-box}
h1{font-size:46px;margin:0 0 6px;font-weight:700;letter-spacing:-.5px;color:#fff}
.tag{font-size:21px;color:#94a3b8;margin:0 0 32px;font-weight:400}
.accent{color:#38bdf8}
.term{background:#0b1220;border:1px solid #334155;border-radius:14px;
box-shadow:0 24px 60px rgba(0,0,0,.45);overflow:hidden;flex:1;display:flex;flex-direction:column}
.bar{background:#1e293b;padding:13px 18px;display:flex;gap:9px;align-items:center;border-bottom:1px solid #334155}
.dot{width:13px;height:13px;border-radius:50%}
.r{background:#ff5f56}.y{background:#ffbd2e}.g{background:#27c93f}
.bartitle{color:#64748b;font-size:14px;margin-left:12px;font-family:ui-monospace,monospace}
pre{margin:0;padding:26px 30px;font-family:ui-monospace,"SF Mono",Menlo,monospace;
font-size:19.5px;line-height:1.72;flex:1}
.p{color:#38bdf8}.c{color:#f1f5f9;font-weight:600}.o{color:#94a3b8}.ok{color:#4ade80}.dim{color:#475569}
.foot{display:flex;gap:40px;margin-top:30px;font-size:18px;color:#cbd5e1}
.foot b{color:#fff}
.pill{display:inline-block;background:#0c4a6e;color:#7dd3fc;font-size:15px;padding:4px 13px;
border-radius:999px;margin-left:14px;vertical-align:middle;font-weight:600}
</style></head>
<body><div class="wrap">
<h1>agent-browser&nbsp;connect <span class="pill">local · no token · no remote</span></h1>
<p class="tag">Let your own <span class="accent">agent-browser</span> CLI drive the Chrome you're already logged into.</p>
<div class="term">
<div class="bar"><span class="dot r"></span><span class="dot y"></span><span class="dot g"></span><span class="bartitle">zsh — agent-browser</span></div>
<pre><span class="p">$</span> <span class="c">agent-browser extension install</span>
<span class="ok"></span> <span class="o">native-messaging host installed (com.agent_browser.connect)</span>
<span class="ok"></span> <span class="o">extension ready — add it from the Chrome Web Store</span>
<span class="p">$</span> <span class="c">agent-browser open</span> <span class="o">"https://mail.google.com"</span> <span class="dim"># your logged-in tab</span>
<span class="p">$</span> <span class="c">agent-browser snapshot -i</span> <span class="dim"># read the page</span>
<span class="p">$</span> <span class="c">agent-browser click</span> <span class="o">@e42</span> <span class="dim"># act on it</span>
<span class="ok"></span> <span class="o">driving your real session — no re-login, no confirmation</span>
</pre>
</div>
<div class="foot">
<span>🔌 <b>Native messaging</b> — local only</span>
<span>🧩 <b>chrome.debugger</b> — on your command</span>
<span>🔓 <b>Open source</b> · Apache-2.0</span>
</div>
</div></body></html>
+2 -2
View File
@@ -1,7 +1,7 @@
{
"name": "agent-browser-stealth",
"version": "0.27.0-fork.15",
"description": "Browser automation CLI for AI agents stealth fork with anti-detection",
"version": "0.27.0-fork.19",
"description": "Browser automation CLI for AI agents \u2014 stealth fork with anti-detection",
"type": "module",
"packageManager": "pnpm@11.1.3",
"files": [
+39
View File
@@ -0,0 +1,39 @@
#!/bin/sh
# Build the Chrome Web Store upload package extensions/ab-connect.zip (and a signed
# extensions/ab-connect.crx for reference) from extensions/ab-connect, keeping the
# extension id constant via the stable signing key + manifest "key".
#
# The id MUST stay ciiljdlhdpfckdcfkphgmfalanpdejep so the native-messaging
# allowed_origins and the force-install policy keep matching. The id is pinned by
# the "key" field in manifest.json (kept in the uploaded zip on purpose).
#
# The private key lives at .secrets/ab-connect.pem and is git-ignored.
#
# After changing the extension:
# 1. bump "version" in extensions/ab-connect/manifest.json
# 2. run this script
# 3. commit extensions/ab-connect.zip (+ .crx) + manifest.json
# 4. upload ab-connect.zip to the Web Store (see extensions/store/SUBMISSION.html)
set -e
cd "$(dirname "$0")/.."
KEY=.secrets/ab-connect.pem
EXT=extensions/ab-connect
CHROME="${CHROME_BIN:-/Applications/Google Chrome.app/Contents/MacOS/Google Chrome}"
# Web Store upload package (zip of the unpacked extension, dotfiles excluded).
rm -f extensions/ab-connect.zip
( cd "$EXT" && zip -rq ../ab-connect.zip . -x '.*' )
[ -f extensions/ab-connect.zip ] || { echo "error: zip failed" >&2; exit 1; }
# Signed crx (reference / non-store force-install for managed setups).
if [ -f "$KEY" ]; then
rm -f extensions/ab-connect.crx
"$CHROME" --pack-extension="$PWD/$EXT" --pack-extension-key="$PWD/$KEY" >/dev/null 2>&1 || true
ID=$(openssl rsa -in "$KEY" -pubout -outform DER 2>/dev/null \
| openssl dgst -sha256 -binary | xxd -p -c256 | head -c32 | tr '0-9a-f' 'a-p')
echo "extension id: $ID"
else
echo "note: $KEY missing — built zip only (no crx)."
fi
echo "packed extensions/ab-connect.zip"
echo "manifest version: $(grep -o '"version"[^,]*' "$EXT/manifest.json" | head -1)"
+100 -3
View File
@@ -29,6 +29,57 @@ Refs (`@e1`, `@e2`, ...) are assigned fresh on every snapshot. They become
submits, dynamic re-renders, dialog opens. Always re-snapshot before your
next ref interaction.
## Before you automate: pick the cheapest tool
Driving a browser is the heavy option. agent-browser earns its keep when you
need a **real, logged-in browser** — not for reading text off a public page.
| You need | Use |
|---|---|
| Discover what exists / find sources | `WebSearch` |
| Specific facts from a static or public page | `WebFetch` or `curl` (no browser) |
| Login state, interaction, JS-rendered or anti-bot pages | **agent-browser** (this skill) |
| A page the user saved before / an internal system | `agent-browser find-url <keywords>` (their bookmarks), then open it |
| The user's **own already-open, logged-in** Chrome window | the **extension connect** flow (below) |
Don't hand-build deep URLs with query params — links discovered by *interacting*
with the site carry the right hidden context and dodge anti-bot checks; a
hand-constructed URL often doesn't.
### Driving the user's real, already-open Chrome (extension)
When the task needs the user's *live* logged-in window (their real session, the
window they're looking at — not a fresh browser), use the extension connect flow:
`agent-browser extension install` once, load `extensions/ab-connect` in
`chrome://extensions` once (a GUI step you can perform with a **computer-use /
GUI-automation tool** like the `cua-driver` skill — see
`references/commands.md` → "Drive your real, logged-in Chrome"), then
`agent-browser extension connect`. After that it's zero-confirmation, zero-token
CLI. Use `--launch` instead when a fresh, isolated browser is fine.
## Two ways to drive a page — and when to drop to `eval`
You have a **real Chrome with the user's DOM**. Two layers, mix them freely:
1. **Structured** (`snapshot` + `@ref`, `find`, typed actions) — convenient and
readable; best for straightforward forms and navigation. But the a11y view is
*lossy and fragile*: refs go stale on any change, hidden inputs never show up,
overlays can block coordinate clicks.
2. **eval-first** (`agent-browser eval "<js>"`) — your eyes and hands on the real
DOM: read hidden inputs, reach into Shadow DOM / iframes, inspect
`form.elements` and `.validity`, extract the exact shape you want, or call
`el.click()` directly. **The moment the structured path fights you, drop to
`eval` instead of retrying it** — it's the fast way to find *why* something
failed (e.g. a hidden `point_choice=none` the UI never exposes).
```bash
# "what's actually in this form / why won't it submit?"
agent-browser eval "[...document.forms[0].elements].map(e=>[e.name,e.type,e.value,e.checked])"
agent-browser eval "document.querySelector('[name=point_choice]')?.value"
agent-browser eval "[...document.forms[0].elements].filter(e=>!e.validity.valid).map(e=>e.name+': '+e.validationMessage)"
agent-browser eval "document.querySelector('#stubborn').click()" # direct DOM click, bypasses overlays
```
## Quickstart
```bash
@@ -137,9 +188,17 @@ agent-browser fill "input[name=email]" "user@test.com"
agent-browser click "button.primary"
```
Rule of thumb: snapshot + `@eN` refs are fastest and most reliable for
AI agents. `find role/text/label` is next best and doesn't require a prior
snapshot. Raw CSS is a fallback when the others fail.
Escalation ladder: snapshot + `@eN` refs are quickest for straightforward
pages → `find role/text/label` when you'd rather skip the snapshot → raw CSS
**`eval` the moment any of those fight you** (stale refs, hidden state,
occluded clicks). Don't retry a flaky structured locator three times; drop to
`eval` and act on the DOM directly.
`click` auto-scrolls into view and, if the coordinate click is occluded, falls
back to a DOM `.click()`. If a click *reports success but nothing happened*
classic for an autocomplete/menu `<li>` that closes on the input's blur — retry
that one with `AGENT_BROWSER_CLICK_MODE=dom agent-browser click ...`, or just
`agent-browser eval "<select the item via JS>"`.
## Waiting (read this)
@@ -209,6 +268,44 @@ AGENT_BROWSER_SESSION_NAME=my-app agent-browser open https://app.example.com
# State is auto-saved and restored on subsequent runs with the same name.
```
### Remember a site's quirks (site notes)
A site behaves the same every time you visit it. When you work out something
durable — a working selector, a URL pattern, a hidden field a form needs, an
anti-bot trap, what requires login — **write it down so the next run doesn't
re-discover it.** Keep one markdown file per domain (these are your own notes,
not shipped with the skill):
```
~/.agent-browser/site-patterns/<domain>.md
```
**Before** working on a domain, read its file if it exists (use your normal file
tools — this is plain markdown you own). Treat it as *hints, not guarantees*
sites change; verify before relying. **After** a successful session that taught
you something durable, create or update it. Suggested shape:
```markdown
---
domain: app.example.com
updated: 2026-06-05
---
## Platform traits
SPA; form renders ~1s after load (wait --text). Cloudflare on /login.
## Working patterns
- Address pick: the `<li>` closes on blur — select with CLICK_MODE=dom.
- Submit needs hidden `point_choice` set (eval), the UI never exposes it.
- Stable selector for "Continue": button[data-testid=submit]
## Known traps (date them)
- 2026-06-05: @ref to the basket button goes stale after the mini-cart opens;
re-snapshot or use `find role button --name "Checkout"`.
```
This is how repeat visits get fast and reliable instead of re-solving the same
page every time.
### Extract data
```bash
+114 -2
View File
@@ -318,15 +318,58 @@ agent-browser --version # Show version (-V)
agent-browser <command> --help # Show detailed help for a command
```
## Drive your real, logged-in Chrome (extension — zero confirmation)
Chrome 136 blocked `--remote-debugging-port` on the default profile, so to drive
the user's *existing* logged-in window, agent-browser uses a Chrome **extension**
over native messaging — no port, no token, no per-use confirmation (the
codex/claude approach).
One-time setup:
```bash
agent-browser extension install # writes the native-messaging host manifest
```
Then load the extension **once** — this is a GUI step (Chrome's `chrome://extensions`
is privileged; the CLI can't load an unpacked extension):
> chrome://extensions → enable **Developer mode** (top-right) → **Load unpacked**
> select `<repo>/extensions/ab-connect`
**You can do this load step yourself with a computer-use / GUI-automation tool**
(e.g. the `cua-driver` skill) — drive `chrome://extensions`, toggle Developer
mode, click *Load unpacked*, pick the folder in the Open dialog. If the extension
is already loaded, clicking its **Reload** (↻) button after a code change is
enough. Notes from doing this live: tools that send *synthetic keystrokes* (e.g.
peekaboo) often don't reach Chrome — **`cua-driver` works** because it reads
Chrome's accessibility tree and clicks real elements. The native "Open" file
dialog is the fiddly part; if keystroke entry there fails, ask the user to pick
the folder (one click). After it loads, Chrome assigns the extension a fixed id
(pinned in its manifest) and auto-connects the host.
Then, any time (pure CLI, zero confirmation):
```bash
agent-browser extension connect # auto-attaches to the live, logged-in tabs
agent-browser tab # list the real tabs it now controls
agent-browser tab t3 # switch the session to one of them
agent-browser snapshot -i / eval / click ... # drive it like any session
agent-browser extension status # is the host installed?
agent-browser extension uninstall # remove the host manifest
```
Security: the extension↔host link is authenticated by Chrome (extension id); the
host↔agent-browser CDP link uses an unguessable URL in a 0600 file. Use this when
you need the user's real cookies/login on their actual machine. (`--extension
<path>` is unrelated — that loads an extension into a *launched* browser.)
## Debugging
```bash
agent-browser --headed open example.com # Show browser window
agent-browser --cdp 9222 snapshot # Connect via CDP port
agent-browser connect 9222 # Alternative: connect command
agent-browser console # View console messages
agent-browser console # View console messages (needs AGENT_BROWSER_CAPTURE_CONSOLE=1)
agent-browser console --clear # Clear console
agent-browser errors # View page errors
agent-browser errors # View page errors (needs AGENT_BROWSER_CAPTURE_CONSOLE=1)
agent-browser errors --clear # Clear errors
agent-browser highlight @e1 # Highlight element
agent-browser inspect # Open Chrome DevTools for this session
@@ -336,6 +379,41 @@ agent-browser profiler start # Start Chrome DevTools profiling
agent-browser profiler stop trace.json # Stop and save profile
```
### Finding a page the user saved (`find-url`)
Search the user's local Chrome/Edge **bookmarks** by keyword — for internal
systems or previously-saved pages that public search can't reach. Local read, no
browser/daemon needed.
```bash
agent-browser find-url jira board # all keywords must match (name or url)
agent-browser find-url --limit 10 invoices
agent-browser find-url --browser edge --profile "Profile 1" wiki
agent-browser find-url grafana --json # {results:[{name,url,folder}], count}
```
Results are most-recently-added first. `javascript:`/`data:` bookmarklets are
skipped. (Visited-history search isn't included yet — bookmarks only.)
### Debugging forms / hidden state with `eval`
The a11y `snapshot` shows visible, interactive elements — it does **not** show
hidden inputs or a control's actual submitted value. When a form "looks filled"
but submit-validation rejects it, go straight to the DOM with `eval` instead of
guessing from the snapshot. This is usually the fastest way to find the real
problem (e.g. a hidden `point_choice=none` that the visible UI never exposes):
```bash
# Dump every field's name → value, including hidden inputs and unchecked radios
agent-browser eval "JSON.stringify([...document.forms[0].elements].map(e=>({name:e.name,type:e.type,value:e.value,checked:e.checked})).filter(e=>e.name))"
# Inspect one hidden field directly
agent-browser eval "document.querySelector('[name=point_choice]')?.value"
# Why won't it submit? Ask the browser's own validity API
agent-browser eval "[...document.forms[0].elements].filter(e=>!e.validity?.valid).map(e=>e.name+': '+e.validationMessage)"
```
## React / Web Vitals
Requires `--enable react-devtools` at launch for the `react ...` commands.
@@ -391,4 +469,38 @@ AGENT_BROWSER_HIDE_SCROLLBARS="false" # Keep native scrollbars visible in
AGENT_BROWSER_PROVIDER="browserbase" # Cloud browser provider
AGENT_BROWSER_STREAM_PORT="9223" # Override WebSocket streaming port (default: OS-assigned)
AGENT_BROWSER_HOME="/path/to/agent-browser" # Custom install location
AGENT_BROWSER_CLICK_MODE="dom" # Click strategy: "" (default: scroll-in + coordinate
# click, DOM-dispatch fallback), "coord" (strict
# coordinate only), "dom" (always element.click())
```
### Click reliability
`click` auto-scrolls the target into view first, then dispatches a coordinate
click. If that fails (a floating layer fails the occlusion guard, or the point
won't resolve) it falls back to a DOM-dispatched `.click()` on the intended
element. If a click *reports success but the page didn't react* — common for
autocomplete/menu `<li>` items that close on the input's blur — retry that one
with `AGENT_BROWSER_CLICK_MODE=dom` (a DOM dispatch doesn't move focus the way a
real pointer press does, so the item still selects). `=coord` disables the
fallback when you specifically want a hard failure on occlusion.
### Stealth / anti-detection knobs (fork)
```bash
AGENT_BROWSER_CAPTURE_CONSOLE="1" # Enable `console`/`errors` capture. OFF by default:
# a live CDP Runtime domain is a detectable bot signal,
# so console/errors return empty (with a hint) until set.
AGENT_BROWSER_TIMEZONE="Asia/Tokyo" # --launch only. Native timezone override (IANA id, or
# "auto" to derive from locale). Aligns Intl+Date to a proxy.
AGENT_BROWSER_BLOCK_WEBRTC="1" # --launch only. Hide local IP via WebRTC. Auto-forces WebRTC
# through the proxy when one is set; "0" opts out.
AGENT_BROWSER_HIDE_CANVAS="1" # --launch only. Session-stable canvas/audio fingerprint noise.
AGENT_BROWSER_ADAPTIVE_REF="0" # Disable adaptive @ref relocation (on by default; relocates a
# moved element by fingerprint when role/name re-query fails).
```
> **Heads-up for `console` / `errors`:** capture is **off by default** in this stealth
> fork. Both commands return `{"messages":[]}` / `{"errors":[]}` plus a `hint` until you
> launch the session with `AGENT_BROWSER_CAPTURE_CONSOLE=1`. This keeps the CDP `Runtime`
> domain disabled (a known bot signal) for the common automation path.
+1 -1
View File
@@ -96,7 +96,7 @@ Read [references/issue-taxonomy.md](references/issue-taxonomy.md) for the full l
- Within each section, test interactive elements: click buttons, fill forms, open dropdowns/modals.
- Check edge cases: empty states, error handling, boundary inputs.
- Try realistic end-to-end workflows (create, edit, delete flows).
- Check the browser console for errors periodically.
- Check the browser console for errors periodically. **Console/error capture is off by default in this stealth fork** — start the dogfood session with `AGENT_BROWSER_CAPTURE_CONSOLE=1` (e.g. `AGENT_BROWSER_CAPTURE_CONSOLE=1 agent-browser --session {SESSION} open <url>`) or `console`/`errors` will return empty.
**At each page:**
+3
View File
@@ -230,6 +230,9 @@ agent-browser snapshot -i | grep -c "treeitem"
### Check console for errors
Console/error capture is off by default in this stealth fork — launch the session with
`AGENT_BROWSER_CAPTURE_CONSOLE=1` first, or these return empty.
```bash
agent-browser console
agent-browser errors