--- name: core description: Core chrome-use usage guide. Read this before running any chrome-use commands. Covers the snapshot-and-ref workflow, navigating pages, interacting with elements (click, fill, type, select), extracting text and data, taking screenshots, managing tabs, handling forms and auth, waiting for content, running multiple browser sessions in parallel, and troubleshooting common failures. Use when the user asks to interact with a website, fill a form, click something, extract data, take a screenshot, log into a site, test a web app, or automate any browser task. allowed-tools: Bash(chrome-use:*), Bash(chrome-use:*), Bash(abs:*), Bash(npx chrome-use:*), Bash(npx chrome-use:*) --- # chrome-use core Fast browser automation CLI for AI agents. Chrome/Chromium via CDP, no Playwright or Puppeteer dependency. Accessibility-tree snapshots with compact `@eN` refs let agents interact with pages in ~200-400 tokens instead of parsing raw HTML. Most normal web tasks (navigate, read, click, fill, extract, screenshot) are covered here. Load a specialized skill when the task falls outside browser web pages — see [When to load another skill](#when-to-load-another-skill). > **Hit a rough edge? Please report it.** If a command surprised you — a > confusing error, a stale `@ref`, an occluded click, a flaky wait, a missing > feature, or anything that cost you extra turns — open a quick issue at > **** with the exact > command and what happened vs. what you expected. Agent-filed friction reports > are how this tool gets sharper; a 30-second issue is genuinely valuable. ## The core loop ```bash chrome-use open # 1. Open a page chrome-use snapshot -i # 2. See what's on it (interactive elements only) chrome-use click @e3 # 3. Act on refs from the snapshot chrome-use snapshot -i # 4. Re-snapshot after any page change ``` Refs (`@e1`, `@e2`, ...) are assigned fresh on every snapshot. They become **stale the moment the page changes** — after clicks that navigate, form submits, dynamic re-renders, dialog opens. Always re-snapshot before your next ref interaction. > **Hard rule: snapshot-first, never screenshot-to-locate.** For form fields and > buttons, ALWAYS `snapshot -i` and act on refs/selectors. Do **not** reach for > `screenshot` + coordinate clicks to find or hit an element — `snapshot -i` now > pierces **cross-origin iframes** (embedded Google Payments / Stripe / checkout / > KYC forms) and lists their elements by `@ref`, including input values. Use > coordinates only for canvas/WebGL, or when `snapshot` genuinely returns nothing > for your target. Screenshots are for *visual verification you report*, never the > agent's own input — and a full-page `screenshot` of a real retina browser is > often too large for an image reader anyway. (If you ever feel you *need* a > screenshot to read state or locate something, that's a bug — please file it.) > **Snapshot-first, always. Never default to `screenshot` + coordinate clicking > for form fields or buttons.** Run `snapshot -i` and act on `@refs`. Use > coordinates only for canvas/WebGL, or when `snapshot` genuinely returns nothing > for your target. This holds **even inside cross-origin embedded iframes** — > since v1.5.12 `snapshot -i` pierces out-of-process iframes (Google Payments, > Stripe, embedded checkout/KYC) and lists their elements with refs, so > `click @e` / `type @e` / `fill @e` work directly. A screenshot is for a genuine > *visual* check you report to the user — not your own input. (Full-page > screenshots of a real retina Chrome are often too large for the image reader > anyway.) Driving off pixels on the relay also risks a coordinate event drifting > onto the user's foreground tab — refs never do. See issue #37. > **Two different intents — only one is discouraged.** The rule above is about > *screenshot-to-locate* (using a picture to find/hit an element) — that's the bug. > *screenshot-to-capture* — saving a region or element to a file as a **reusable > image asset** (maps, charts, og-images, visual-diff baselines, report figures) — > is fully supported and encouraged: `screenshot [selector] [--clip x,y,w,h] `. > Capturing a rendered map region to a PNG for a blog post is the right tool, not a > smell. Screenshots are auto-downscaled to ≤2000px (longest edge) so they fit an > image reader and their pixels line up with `click x y`; override with > `--max-width`/`--max-height`/`--scale`. To click something you couldn't hit by > ref, `box @ref` gives the element's CSS-px box + `centerX/centerY` to feed > straight into `click ` — no screenshot needed. ## Before you automate: pick the cheapest tool Driving a browser is the heavy option. chrome-use earns its keep when you need a **real, logged-in browser** — not for reading text off a public page. | You need | Use | |---|---| | Discover what exists / find sources | `WebSearch` | | Specific facts from a static or public page | `WebFetch` or `curl` (no browser) | | **Structured data from a known site** (GitHub issues, Reddit/HN search, Bilibili/Twitter feed, …) — esp. behind login | `chrome-use site /` (see below) — skip snapshot+click entirely | | Login state, interaction, JS-rendered or anti-bot pages | **chrome-use** (this skill) | | A page the user saved before / an internal system | `chrome-use find-url ` (their bookmarks), then open it | | The user's **own already-open, logged-in** Chrome window | the **extension connect** flow (below) | Don't hand-build deep URLs with query params — links discovered by *interacting* with the site carry the right hidden context and dodge anti-bot checks; a hand-constructed URL often doesn't. ### Driving the user's real, already-open Chrome (extension) When the task needs the user's *live* logged-in window (their real session, the window they're looking at — not a fresh browser), use the extension connect flow. One-time setup: 1. `chrome-use extension install` — registers the native-messaging host. 2. Install the **chrome-use** extension. Easiest (and restart-stable): the **Chrome Web Store**, one-click *Add to Chrome*: (Dev fallback: `chrome://extensions` → Developer mode → *Load unpacked* → `extensions/ab-connect`. Load-unpacked can be disabled on Chrome restart, so prefer the Store build for unattended setups.) Once installed, plain `chrome-use open ` auto-connects through the extension relay — `auto_connect_cdp` **prefers the live relay over a raw `--remote-debugging-port`**, so Chrome 136+'s "Allow remote debugging?" consent popup never fires. `chrome-use extension connect` is the explicit form of the same path. Zero-confirmation, zero-token. Use `--launch` instead when a fresh, isolated browser is fine. `--launch` opens an **isolated, empty test profile** — no cookies, no login, no extensions (so the extension-relay path is off). Its window is labelled `chrome-use ()` in Chrome's profile menu so a human watching the desktop knows which session owns it. If a launched session needs more: - **Real cookies / login / extensions** → drop `--launch`, use `--profile auto` (reuses the user's real Chrome profile), or set `AGENT_BROWSER_PROFILE=auto` once so every call does it by default. - **A specific unpacked extension in the test profile** → `--launch --args "--load-extension="`. **If you DO hit the "Allow remote debugging?" dialog**, don't keep retrying (every attempt re-pops it). One of two things is true: 1. **You're on a stale build.** The relay-preference that avoids this dialog landed in **fork.30**. Run `chrome-use --version`: if it's below `0.27.0-fork.30`, upgrade and retry: ```bash curl -fsSL https://raw.githubusercontent.com/leeguooooo/chrome-use/main/install.sh | sh ``` If `which -a chrome-use` shows more than one install, an old **npm/pnpm** copy (the npm registry lags behind — Releases are the source of truth) may be shadowing the upgraded one; remove the stale copy (`npm rm -g chrome-use` / `pnpm rm -g chrome-use`) so the `install.sh` build wins. A tool that bundles its *own* pinned copy (e.g. `node .../chrome-use@0.24.x/.../chrome-use`) needs that copy upgraded too. 2. **The extension/relay isn't live.** Tell the user to install the Store extension (one click, above); after that the relay stays up and the dialog never returns. Each `--session` that connects gets its **own colored Chrome tab group** (named after the session) and drives only its own tabs — multiple agents share the one real browser without cross-talk, and the user's own tabs are never grouped. CDP drives the page without moving the user's mouse/keyboard, so it doesn't fight them for control. **Strict multi-agent isolation.** A session over the relay tracks and drives **only the tabs it created** (its own group). It does **not** adopt the user's existing tabs, other agents' tabs, or pop-ups (e.g. an OAuth/login window — that's the user's), so several agents (and other tools opening tabs) can work in the same real Chrome concurrently without ever dropping or stealing each other's tabs — another agent's tab churn can't make your bound tab vanish or drift your commands onto the wrong page. Consequence: `tab list` shows only *your* session's tabs; to drive a specific page, navigate to it in your own tab instead of expecting a pre-existing or popped-up tab to appear in the list. **Anti-detection ranking: this real logged-in Chrome (extension connect) > a headed launched browser > headless (forbidden).** A genuine human browser has no headless/automation tells at all, so prefer it for anything anti-bot-sensitive. **Silent by default.** When driving the user's real Chrome the agent works entirely in the background — new tabs open un-focused, the agent never force- fronts a tab, and focus is emulated so the page still renders and reports `visibilityState: 'visible'`. You don't need to do anything; just don't expect the user's view to follow you (use the explicit `bringToFront` only if you deliberately want to surface a tab). **Human-like input for behavioural anti-bot.** Beyond fingerprint stealth, `--humanize off|fast|human` (or `AGENT_BROWSER_HUMANIZE`) makes clicks follow a curved, decelerating path with in-element landing jitter, typing use variable cadence, and scroll/drag ease. Default `off`; a per-navigation detector auto-escalates pages guarded by Akamai/PerimeterX/DataDome to `human`. Leave it on auto; force `human` only when you already know the target scores behaviour. **Cloudflare clearance — solve once, reuse.** Passing a Cloudflare challenge mints a `cf_clearance` cookie (HttpOnly — invisible to `eval`/`document.cookie`; read it via `chrome-use cookies`). It's bound to your **IP + User-Agent**: reuse the same exit IP and UA and you skip the challenge until it expires. Driving the user's real Chrome (relay) persists it natively; for isolated sessions, `--session-name ` save/restores it. Before spending effort solving, run `chrome-use cf-status` (aliases `cf`, `clearance`): it reports whether the page is *currently* a Cloudflare challenge and whether a still-valid `cf_clearance` exists, with a recommendation — `proceed` (already cleared, don't re-solve), `solve` (challenge up, no clearance), or `reissue` (clearance present but page still blocks → IP/UA drifted, re-solve). Use it as a preflight to avoid re-solving what you already cleared. ## Two ways to drive a page — and when to drop to `eval` You have a **real Chrome with the user's DOM**. Two layers, mix them freely: 1. **Structured** (`snapshot` + `@ref`, `find`, typed actions) — convenient and readable; best for straightforward forms and navigation. But the a11y view is *lossy and fragile*: refs go stale on any change, hidden inputs never show up, overlays can block coordinate clicks. 2. **eval-first** (`chrome-use eval ""`) — your eyes and hands on the real DOM: read hidden inputs, reach into Shadow DOM / iframes, inspect `form.elements` and `.validity`, extract the exact shape you want, or call `el.click()` directly. **The moment the structured path fights you, drop to `eval` instead of retrying it** — it's the fast way to find *why* something failed (e.g. a hidden `point_choice=none` the UI never exposes). ```bash # "what's actually in this form / why won't it submit?" chrome-use eval "[...document.forms[0].elements].map(e=>[e.name,e.type,e.value,e.checked])" chrome-use eval "document.querySelector('[name=point_choice]')?.value" chrome-use eval "[...document.forms[0].elements].filter(e=>!e.validity.valid).map(e=>e.name+': '+e.validationMessage)" chrome-use eval "document.querySelector('#stubborn').click()" # direct DOM click, bypasses overlays ``` ## Site adapters — the cheapest path for "read structured data from site X" Before you `open` + `snapshot` + click your way through GitHub/Reddit/Bilibili/etc., check whether a **site adapter** already exists. An adapter is a community-written JS function that hits the site's own JSON API *from inside your logged-in tab* and returns clean structured data — no clicking, no scraping, no screenshots. It's the same idea as `eval`, packaged per-site. ```bash chrome-use site update # one-time: fetch the adapter pack (~145 cmds) chrome-use site list # what's installed (github/issues, reddit/search, …) chrome-use site info github/issues # an adapter's args + which domain it runs on chrome-use site github/issues owner/repo --json # run it → JSON (navigates there for you) ``` - Positional args fill the adapter's declared args **in order**; `--key value` overrides by name. - It navigates to the adapter's domain (reusing the current tab if you're already on it), so login-gated feeds (`bilibili/feed`, `twitter/...`) work because they run as *you*. - If no adapter fits, fall back to the normal `snapshot`/`eval` loop. Adapters come from the [bb-sites](https://github.com/epiral/bb-sites) community pack; chrome-use fetches & runs them. > **Auto-trigger — act on it.** chrome-use keeps the pack synced automatically (first use + > weekly), and when you `open`/`navigate`/`snapshot` a page whose domain has adapters it tells > you: a `💡 site adapters for ` line on stderr, and a `siteAdapters: {domain, commands}` > field in `--json`. **When you see that, prefer the listed `site /` over snapshot+click > for reading data** — it's the cheaper, more reliable path and it's already installed. You don't > need to run `site update` yourself; just use the command it names. ## Quickstart ```bash # Install once npm i -g chrome-use && chrome-use install # Take a screenshot of a page chrome-use open https://example.com chrome-use screenshot home.png chrome-use close # Search, click a result, and capture it chrome-use open https://duckduckgo.com chrome-use snapshot -i # find the search box ref chrome-use fill @e1 "chrome-use cli" chrome-use press Enter chrome-use wait --load networkidle chrome-use snapshot -i # refs now reflect results chrome-use click @e5 # click a result chrome-use screenshot result.png ``` The browser stays running across commands so these feel like a single session. Use `chrome-use close` (or `close --all`) when you're done. ## Reading a page ```bash chrome-use snapshot # full tree (verbose) chrome-use snapshot -i # interactive elements only (preferred) chrome-use snapshot -i -u # include href urls on links chrome-use snapshot -i -c # compact (no empty structural nodes) chrome-use snapshot -i -d 3 # cap depth at 3 levels chrome-use snapshot -s "#main" # scope to a CSS selector chrome-use snapshot -i --json # machine-readable output ``` Snapshot output looks like: ``` Page: Example - Log in URL: https://example.com/login - heading "Log in" [level=1, ref=e1] - textbox "Email" [ref=e2] - textbox "Password" [ref=e3] - button "Continue" [ref=e4] - link "Forgot password?" [ref=e5] ``` Each line is `- "" [, ref=eN]`, indented by nesting depth. You pass the ref to commands as `@eN` (e.g. `click @e4`). Refs are assigned fresh on every snapshot. For unstructured reading (no refs needed): ```bash chrome-use get text # WHOLE PAGE — all frames by default (see below) chrome-use get text @e1 # visible text of one element (or a CSS selector) chrome-use get text --main # main content only — skip nav/header/sidebar chrome-use get text --pierce # read through CLOSED shadow DOM (injected panels) chrome-use frames # list every frame + where the text lives chrome-use get html @e1 # innerHTML chrome-use get attr @e1 href # any attribute chrome-use get value @e1 # input value chrome-use get title # page title chrome-use get url # current URL chrome-use get count ".item" # count matching elements ``` **Whole-page text is cross-frame by default.** `chrome-use get text` with no selector aggregates visible text across **every** frame — top document plus same-process child frames plus cross-origin iframes — so you never silently miss content that lives in an iframe (Yahoo Auctions / Rakuten / Mercari shop descriptions, embedded checkout/spec frames). Each child frame is delimited with a `----- frame [kind] url -----` marker. You do **not** need to remember a flag — the default already reads all frames. (`--all-frames` is still accepted as an explicit alias.) So: when text looks missing or wrong, you don't have to guess — just `chrome-use get text` reads everything. To **see** the structure (which frame holds what), run `chrome-use frames`. To **cut boilerplate** (global nav/header/ footer, "related items" sidebars), use `chrome-use get text --main`. If content is lazy-loaded, `scroll` it into view first, then read. **Closed shadow DOM.** Some injected UI (browser-extension debug panels, web components) renders into a *closed* shadow root that `eval`/`innerText` cannot read. `chrome-use get text --pierce` reads through closed shadow roots and child documents via the CDP DOM tree — use it when content is clearly on screen (you see it in a screenshot) but `get text`/`eval` come back empty. ## Interacting ```bash chrome-use click @e1 # click chrome-use click @e1 --new-tab # open link in new tab instead of navigating chrome-use dblclick @e1 # double-click chrome-use hover @e1 # hover chrome-use focus @e1 # focus (useful before keyboard input) chrome-use fill @e2 "hello" # clear then type chrome-use type @e2 " world" # type without clearing chrome-use type @e5 "201-0001" --key-events # real keystrokes (not insertText) — # use for autocomplete/combobox fields that # only react to key events (e.g. a postal box # that auto-fills city/prefecture, Google Places) chrome-use press Enter # press a key at current focus (down+up) chrome-use press Control+a # key combination chrome-use keydown d # HOLD a key down (no auto-release) chrome-use keyup d # release it — pair them to hold-to-move # in a game: `keydown d; sleep; keyup d` chrome-use check @e3 # check checkbox chrome-use uncheck @e3 # uncheck chrome-use select @e4 "option-value" # native s and drop/paste composers (e.g. X). chrome-use scroll down 500 # scroll page (up/down/left/right) chrome-use scroll down 700 --at 640,400 # wheel at a pixel — scrolls a cross-origin # iframe (Payments/Stripe/checkout/KYC) that # plain page scroll can't reach chrome-use scroll down 700 --frame 2 # scroll frame 2 from `chrome-use frames` chrome-use scrollintoview @e1 # scroll element into view chrome-use drag @e1 @e2 # drag and drop ``` **Cross-origin iframes (embedded payment / checkout / KYC widgets — Google Payments, Stripe, etc.) — drive them by ref, never by screenshot.** `snapshot -i` pierces these out-of-process iframes and lists their elements by `@ref` (including input values); `get text --all-frames` reads their text. Then just act on the refs: `click @e`, `type @e`, `hover @e`, `dblclick @e`, `drag @a @b` all work into the iframe. Over the extension relay these are dispatched through the DOM (in the element's own frame), so they hit the right element in the right tab — a coordinate click/scroll there can drift onto whatever tab is in the foreground, so prefer refs. For below-the-fold content in such a frame, scroll it with `scroll down N --at x,y` (a pixel over the frame) or `--frame n`. For a postal/autocomplete box inside the frame, `type @e "…" --key-events`. > **Caveat: `find text "…"` can't reach into a cross-origin iframe** — it errors > "Element not found" even though `snapshot -i` lists those nodes and > `get text` reads them. Inside cross-origin iframes, target elements by their > **snapshot `@ref`**, not by `find`. (`box @ref` also works on iframe refs when > you need a coordinate fallback.) ### When refs don't work or you don't want to snapshot Use semantic locators: ```bash chrome-use find role button click --name "Submit" chrome-use find text "Sign In" click chrome-use find text "Sign In" click --exact # exact match only chrome-use find label "Email" fill "user@test.com" chrome-use find placeholder "Search" type "query" chrome-use find testid "submit-btn" click chrome-use find first ".card" click chrome-use find nth 2 ".card" hover ``` Or a raw CSS selector: ```bash chrome-use click "#submit" chrome-use fill "input[name=email]" "user@test.com" chrome-use click "button.primary" ``` Escalation ladder: snapshot + `@eN` refs are quickest for straightforward pages → `find role/text/label` when you'd rather skip the snapshot → raw CSS → **`eval` the moment any of those fight you** (stale refs, hidden state, occluded clicks). Don't retry a flaky structured locator three times; drop to `eval` and act on the DOM directly. `click` auto-scrolls into view and, if the coordinate click is occluded, falls back to a DOM `.click()`. If a click *reports success but nothing happened* — classic for an autocomplete/menu `
  • ` that closes on the input's blur — retry that one with `AGENT_BROWSER_CLICK_MODE=dom chrome-use click ...`, or just `chrome-use eval "