Compare commits

...
Author SHA1 Message Date
Chris Tate 2c0c5a2acc v0.6.0 changelog 2026-01-18 11:44:29 -06:00
Chris Tate 818d9fa95e format code (#152) 2026-01-18 11:21:28 -06:00
Kye Burchard a8dcbb1222 feat: add connect command for persistent CDP sessions (#127)
Adds a `connect <port>` command that establishes a CDP connection
to a running browser. The daemon remembers the connection, so
subsequent commands work without needing --cdp on every call.

Example:
  agent-browser connect 9222
  agent-browser snapshot  # works without --cdp
  agent-browser tab
  agent-browser close
2026-01-18 10:54:12 -06:00
Mikhail Beliakovvercel[bot] <35613825+vercel[bot]@users.noreply.github.com>google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
a9fcef4579 fix: support libasound2t64 on newer Ubuntu versions (#112)
* fix: support libasound2t64 on newer Ubuntu versions

Updates the install logic to check if `libasound2t64` is available using
`apt-cache` before falling back to `libasound2`. This fixes installation
on Ubuntu 24.04 and other systems affected by the 64-bit time_t transition.

* Update cli/src/install.rs

Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>

---------

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
2026-01-18 10:50:57 -06:00
Zhiwei Li 59baf97e51 fix: allow additional URL schemes in parse_command function (#125)
* fix: allow additional URL schemes in parse_command function

* fix: enhance URL validation in parse_command function to support lowercase schemes
2026-01-18 10:36:38 -06:00
0okay d02ef66c89 Refactor connection logic for Windows and hash calculationfix(cli): fix windows daemon startup and port calculation inconsistency (#79) 2026-01-18 09:14:03 -06:00
Danila Poyarkov 1689cf9eca fix(cli): handle SIGPIPE to prevent panic when piping output (#144) 2026-01-18 08:57:02 -06:00
Matthew KingandClaude Opus 4.5 03a53c9f36 feat: add Claude marketplace plugin (#106)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-18 07:49:27 -06:00
Zhiwei Li b1c0c6a366 feat: enhance response output with network request details and cleared status (#117) 2026-01-18 07:35:25 -06:00
Ryan DaigleandClaude Opus 4.5 c88734da89 feat: add NO_COLOR environment variable support (#122)
Add a centralized color module (cli/src/color.rs) that respects the
NO_COLOR environment variable per https://no-color.org/

Changes:
- Add color.rs module with helper functions for colored output
- Refactor all hardcoded ANSI escape codes to use the color module
- Add tests for color formatting functions
- Update AGENTS.md with color module usage guidelines

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-17 20:37:00 -06:00
Nicenonecb 28740acecf Fix CLI/protocol mismatches for select, frame main, and headers (#45)
* fix: align CLI command payloads with protocol

* fix(cli): support multi-value select in CLI
2026-01-17 20:25:19 -06:00
jaydenfyi 4112234371 fix(cli): Output screenshot as base64 string when no path provided (#83)
* fix(cli): print screenshot base64 when no path

* chore(docs): update docs and SKILL.md

* add test for screenshot with path arg

* more minimal readme + skill change
2026-01-17 20:18:06 -06:00
Li Yang 5e08e5d077 fix: detect stale unix socket by attempting connection (#114) 2026-01-17 19:42:48 -06:00
Sanchay 42879c337a fix: respect AGENT_BROWSER_HEADED env var for headed mode (#92)
The headless option was hardcoded to true in the auto-launch section,
ignoring the AGENT_BROWSER_HEADED environment variable. This fix checks
the env var so users can run the browser in headed mode by setting
AGENT_BROWSER_HEADED=1.

Fixes #90
2026-01-17 19:22:59 -06:00
Leon Gao 412ac63b68 fix: resolve refs in input value (#139) 2026-01-17 19:05:06 -06:00
Danila Poyarkov e6e832d2bc feat: add 'get styles' command for computed styles extraction (#142) 2026-01-17 18:56:52 -06:00
Dharma b19ca760aa fix: support URL parameter in tab new command (#64)
* fix: support URL parameter in tab new command

The CLI was correctly sending the URL parameter when running
`agent-browser tab new <url>`, but the TypeScript daemon was
ignoring it because:

1. The schema didn't include the url field (stripped during validation)
2. The TabNewCommand type didn't have a url property
3. The handler didn't pass the URL to browser.newTab()
4. browser.newTab() didn't accept or use a URL parameter

This fix adds URL support throughout the chain so that
`agent-browser tab new https://example.com` now correctly
opens a new tab and navigates to the specified URL.

Fixes #62

* fix: omit url field when not provided in tab new command

Previously, the CLI always sent "url": null when no URL was provided,
which caused Zod validation to fail with "Expected string, received null".

Now the url field is only included when a URL is actually provided.

Fixes issue reported by @ctate in PR review.

* refactor: move navigation logic from BrowserManager to handleTabNew

Address review feedback:
- Add .min(1) to URL validation for consistency with navigateSchema
- Keep BrowserManager.newTab() simple (single responsibility)
- Handle navigation in handleTabNew following same pattern as handleNavigate
2026-01-17 18:53:24 -06:00
Sheingandgoogle-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> e7c4936bc7 fix(cli): allow null path in screenshot command validation (#101)
The Rust CLI sends `null` for the `path` argument when it is not provided,
but the Zod schema only accepted `undefined`. This change updates the
`screenshotSchema` to allow `null` values for `path`, enabling the
`screenshot` command to work without a file path argument (outputting to stdout).

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
2026-01-17 18:41:55 -06:00
Sheinggoogle-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>sheing-google
7aad47d3bd fix: Prevent CDP timeout on empty URL tabs (#102)
When connecting to a browser via CDP, particularly on Android, tabs with an empty URL can cause Playwright commands to hang indefinitely. This leads to a timeout in agent-browser.

This commit fixes the issue by filtering out any pages that have an empty `page.url()` during the CDP connection process. This prevents agent-browser from attempting to interact with these problematic tabs, resolving the timeout while preserving normal pages.

Added a unit test to verify that pages with empty URLs are correctly ignored. Also increased the timeout for a flaky screencast test to improve test suite stability.

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: sheing-google <231310897+sheing-google@users.noreply.github.com>
2026-01-17 18:16:37 -06:00
edx.eth e196ed3e35 fix(cli): align protocol action names for wheel, emulatemedia, and find locators (#143)
- mouse wheel: send 'wheel' instead of 'mousewheel'
- set media: send 'emulatemedia' instead of 'media', fix reducedMotion to be string enum
- find locators: omit 'value' field when not provided (Zod .optional() expects undefined, not null)
  - Consistently applied to: role, label, placeholder, testid, first, last, nth

Fixes #131
2026-01-17 18:11:11 -06:00
1f31452fea feat: Add video recording with Playwright native video (#116)
* feat: add video recording with Playwright native video

Adds `record start/stop` commands using Playwright's built-in video
recording. No external dependencies required (no FFmpeg).

Usage:
  agent-browser record start ./demo.webm https://example.com
  agent-browser click @e1
  agent-browser record stop

Recording creates a fresh browser context with video enabled. For smooth
demos, explore the page first to plan actions, then start recording.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: auto-capture URL and transfer state for recording

When starting a recording without a URL:
- Automatically captures current page URL
- Preserves cookies and localStorage from current session

This enables a seamless workflow:
  agent-browser open https://app.example.com
  agent-browser snapshot -i  # explore, plan
  agent-browser record start ./demo.webm  # picks up URL + auth state
  agent-browser click @e3
  agent-browser record stop

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: error on non-webm recording path instead of silent coercion

Previously, specifying a non-.webm path like ./demo.mp4 would silently
change it to ./demo.webm. Now it throws a clear error telling the user
that Playwright native recording only supports WebM format.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: clean up recording temp directory after stopRecording

Previously the temp directory was created but never deleted, relying on
OS cleanup. Now we explicitly remove it after saving the video, in both
success and error paths.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add record restart command

Adds `record restart` command that stops the current recording (if any)
and starts a new one. Also improves the error message when trying to
start recording while already recording.

Changes:
- Add restartRecording method to BrowserManager
- Add recording_restart action to protocol, types, and actions
- Add CLI parsing for `record restart <path> [url]`
- Update help text and skill documentation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add CLI tests for record restart command

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Chris Tate <chris@ctate.dev>
2026-01-16 12:27:58 -06:00
NMW 3675e6bd7a feat: add --proxy flag for browser proxy configuration (#16)
* feat: add --proxy flag for browser proxy support

Add CLI flag to configure HTTP/SOCKS proxy for Playwright browser context.
Supports URL format with optional credentials: http://user:pass@host:port

* fix: improve proxy parsing error handling

- Handle malformed credentials (@ without :) by ignoring incomplete creds
- Replace unwrap() with expect() for better error messages
- Addresses Vercel bot code review suggestions

* Restaura cambios locales: soporte AGENT_BROWSER_HOME y timeout aumentado

- Agrega soporte para variable de entorno AGENT_BROWSER_HOME en connection.rs
- Aumenta timeout por defecto de 10s a 60s para conexiones más lentas

* feat: add --proxy flag for browser proxy configuration

Implements proxy support based on PR #16 with reviewer feedback:

Features:
- Parse proxy URLs: http://[user:pass@]host:port
- Support for HTTP, HTTPS, and SOCKS5 protocols
- Handle username-only auth (preserves username with empty password)
- Apply proxy to both standard and persistent contexts

Changes:
- cli/src/flags.rs: Add proxy flag parsing
- cli/src/main.rs: Add parse_proxy() with comprehensive tests
- cli/src/output.rs: Add --proxy to help output
- cli/src/commands.rs: Fix test helper to include proxy field
- src/types.ts: Add proxy to LaunchCommand interface
- src/protocol.ts: Add proxy validation schema
- src/browser.ts: Apply proxy to context creation

Tests:
- 7 unit tests for parse_proxy() covering all edge cases
- All Rust tests passing (69 tests)
- All TypeScript tests passing (168 tests)
- TypeScript typecheck passing

Resolves feedback from PR #16:
- Fixed username-only proxy handling (issue #2681046975)
- Added comprehensive unit tests
- Added --proxy to help documentation
- Used expect() instead of unwrap() for better error messages

* refactor: simplify parse_proxy function

- Remove redundant comments
- Extract server variable to reduce duplication
- Inline trivial username/password variables

All 7 proxy tests still passing.
2026-01-16 11:56:35 -06:00
Andrew GadzikandClaude Opus 4.5 fff9a146bd docs: update agent-browser skill with comprehensive command reference (#121)
Add documentation for new commands including focus, drag/drop, upload,
keydown/keyup, mouse control, cookies/storage, network interception,
tabs/windows, frames, dialogs, and browser settings.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 11:40:57 -06:00
edx.eth 34dcb7195a fix(cli): align console output field name with daemon response (#133)
The CLI expected a 'logs' field but the daemon returns 'messages'.
This caused 'agent-browser console' to display nothing.

Changed cli/src/output.rs to read 'messages' instead of 'logs',
matching the actual response from handleConsole in src/actions.ts.
2026-01-16 11:38:29 -06:00
Matthew KingandClaude Opus 4.5 7bdfcf8541 feat: add --version flag to CLI (#94)
Print the current version when `agent-browser --version` is passed.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 11:35:06 -06:00
Chris Tate 6abee37641 v0.5.0 (#78) 2026-01-13 21:54:20 -06:00
NoelandClaude Sonnet 4.5 7b43d408da fix: improve error message when element is blocked by overlay (#59)
When clicking an element that is blocked by a cookie banner or modal overlay,
the error message incorrectly showed "Element not found or not visible" even
though the element was found and visible.

The issue was in toAIFriendlyError(): the check for "Timeout" was evaluated
before "intercepts pointer events", causing the wrong error message to be
returned.

Changes:
- Reorder error detection to check "intercepts pointer events" before "Timeout"
- Improve error message to suggest dismissing modals/cookie banners
- Export toAIFriendlyError for testing
- Add focused tests for overlay blocking behavior

Before:
  Element "@e4" not found or not visible. Run 'snapshot' to see current page elements.

After:
  Element "@e4" is blocked by another element (likely a modal or overlay).
  Try dismissing any modals/cookie banners first.

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-13 15:25:46 -06:00
Chris TateandVercel <vercel[bot]@users.noreply.github.com> 2dc093cd62 add screencast (#67)
* docs

* updates

* Fix: The handleCopy function fails to handle errors from navigator.clipboard.writeText(), causing unhandled exceptions and misleading UI feedback when clipboard operations fail.

Co-authored-by: ctate <chris@ctate.dev>

* Fix: The benchmark file uses emojis (📊, 🚀, 🔨, 📈, 📋, , ⏱️, ⚠) in console output, violating repository guidelines that forbid emojis in code and output.

Co-authored-by: ctate <chris@ctate.dev>

* Remove benchmark/run.ts from PR

* screencast

* update docs

* address comments

---------

Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com>
2026-01-13 14:53:27 -06:00
Shirshak 673e2e266e feat: Add extension support (#48)
* Rebase: Add extension support

* Fix logs
2026-01-13 14:34:59 -06:00
Chris TateandVercel <vercel[bot]@users.noreply.github.com> 4713c8b520 add docs (#54)
* docs

* updates

* Fix: The handleCopy function fails to handle errors from navigator.clipboard.writeText(), causing unhandled exceptions and misleading UI feedback when clipboard operations fail.

Co-authored-by: ctate <chris@ctate.dev>

* Fix: The benchmark file uses emojis (📊, 🚀, 🔨, 📈, 📋, , ⏱️, ⚠) in console output, violating repository guidelines that forbid emojis in code and output.

Co-authored-by: ctate <chris@ctate.dev>

* Remove benchmark/run.ts from PR

---------

Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com>
2026-01-13 02:54:56 -06:00
Chris Tate b4bc761168 fix builds (#55) 2026-01-13 02:41:39 -06:00
Bryan LeeandChris Tate 6eafe50952 fix incomplete build-from-source instructions (#40) (#41)
Co-authored-by: Chris Tate <chris@ctate.dev>
2026-01-13 02:21:26 -06:00
Alan JeonClaude Opus 4.5vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
95675e9d55 feat: add CDP connection support for external browsers (#24)
* feat: add CDP connection support for external browsers

Add --cdp flag to connect to browsers via Chrome DevTools Protocol.
This enables control of Electron apps, Chrome instances, or any browser
exposing a CDP endpoint.

- Add cdpPort option to launch command schema
- Implement connectViaCDP() using chromium.connectOverCDP()
- Track browser connection type for proper reconnection handling
- Collect all pages from all contexts for CDP connections

Usage: agent-browser --cdp 9222 snapshot

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: enhance CDP connection handling and improve page tracking

* main.rs update

Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>

* fix: verify CDP connection is alive before early return in launch()

Prevents misleading errors when the remote browser crashes by checking
isConnected() before reusing an existing browser reference.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: reconnect when CDP port changes instead of reusing existing browser

Ensures --cdp flag is respected even when a browser session already exists.
Adds tests for launch() reconnection behavior.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Update src/browser.ts

Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>

* fix: improve CDP connection handling and validation

* feat: add CDP connection validation to ensure browser context accessibility

* Update src/browser.ts

Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>

* feat: enhance CDP connection handling and add reconnect logic

* fix: improve CDP connection handling during browser closure

* fix: reset cdpPort to null during browser initialization

* feat: enhance browser launch logic to handle CDP connection switching

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
2026-01-13 00:52:47 -06:00
Byonghun Lee 97b17c98fb Update installation instructions in README (#51)
Add native build step and global link command
2026-01-13 00:39:31 -06:00
Chris Tate 57a04385c1 v0.4.4 (#32)
* v0.4.4

* 0.4.4
2026-01-12 12:12:16 -06:00
Chris Tate 1a88d7f585 custom headers via --headers (#30)
* add custom headers via --headers

* add tests

* better parsing
2026-01-12 12:01:19 -06:00
Chris Tate 4f6fd8ec5c support serverless environments (#29)
* add --executable-path

* tests

* test vercel

* fixes
2026-01-12 11:41:25 -06:00
52 changed files with 10005 additions and 167 deletions
+19
View File
@@ -0,0 +1,19 @@
{
"$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
"name": "agent-browser",
"description": "Headless browser automation for AI agents",
"owner": {
"name": "Vercel",
"email": "support@vercel.com"
},
"plugins": [
{
"name": "agent-browser",
"description": "Automates browser interactions for web testing, form filling, screenshots, and data extraction",
"source": "./",
"strict": false,
"skills": ["./skills/agent-browser"],
"category": "development"
}
]
}
+31
View File
@@ -155,3 +155,34 @@ jobs:
exit 1
}
shell: pwsh
serverless-chromium:
name: Serverless Chromium (@sparticuz/chromium)
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup pnpm
uses: pnpm/action-setup@v4
with:
version: 9
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: 22
cache: pnpm
- name: Install dependencies
run: pnpm install
- name: Install @sparticuz/chromium
run: pnpm add -D @sparticuz/chromium
- name: Build TypeScript
run: pnpm build
- name: Run serverless integration test
run: pnpm exec vitest run test/serverless.test.ts
+6
View File
@@ -42,3 +42,9 @@ yarn.lock
# opensrc - source code for packages
opensrc/
# Docs site
docs/node_modules/
docs/.next/
docs/out/
docs/package-lock.json
+1
View File
@@ -5,6 +5,7 @@ Instructions for AI coding agents working with this codebase.
## Code Style
- Do not use emojis in code, output, or documentation. Unicode symbols (✓, ✗, →, ⚠) are acceptable.
- CLI colored output uses `cli/src/color.rs`. This module respects the `NO_COLOR` environment variable. Never use hardcoded ANSI color codes.
<!-- opensrc:start -->
+205 -1
View File
@@ -18,6 +18,8 @@ git clone https://github.com/vercel-labs/agent-browser
cd agent-browser
pnpm install
pnpm build
pnpm build:native # Requires Rust (https://rustup.rs)
pnpm link --global # Makes agent-browser available globally
agent-browser install
```
@@ -72,10 +74,11 @@ agent-browser scroll <dir> [px] # Scroll (up/down/left/right)
agent-browser scrollintoview <sel> # Scroll element into view (alias: scrollinto)
agent-browser drag <src> <tgt> # Drag and drop
agent-browser upload <sel> <files> # Upload files
agent-browser screenshot [path] # Take screenshot (--full for full page)
agent-browser screenshot [path] # Take screenshot (--full for full page, base64 png to stdout if no path)
agent-browser pdf <path> # Save as PDF
agent-browser snapshot # Accessibility tree with refs (best for AI)
agent-browser eval <js> # Run JavaScript
agent-browser connect <port> # Connect to browser via CDP
agent-browser close # Close browser (aliases: quit, exit)
```
@@ -293,11 +296,14 @@ agent-browser snapshot -i -c -d 5 # Combine options
| Option | Description |
|--------|-------------|
| `--session <name>` | Use isolated session (or `AGENT_BROWSER_SESSION` env) |
| `--headers <json>` | Set HTTP headers scoped to the URL's origin |
| `--executable-path <path>` | Custom browser executable (or `AGENT_BROWSER_EXECUTABLE_PATH` env) |
| `--json` | JSON output (for agents) |
| `--full, -f` | Full page screenshot |
| `--name, -n` | Locator name filter |
| `--exact` | Exact text match |
| `--headed` | Show browser window (not headless) |
| `--cdp <port>` | Connect via Chrome DevTools Protocol |
| `--debug` | Debug output |
## Selectors
@@ -387,6 +393,204 @@ agent-browser open example.com --headed
This opens a visible browser window instead of running headless.
## Authenticated Sessions
Use `--headers` to set HTTP headers for a specific origin, enabling authentication without login flows:
```bash
# Headers are scoped to api.example.com only
agent-browser open api.example.com --headers '{"Authorization": "Bearer <token>"}'
# Requests to api.example.com include the auth header
agent-browser snapshot -i --json
agent-browser click @e2
# Navigate to another domain - headers are NOT sent (safe!)
agent-browser open other-site.com
```
This is useful for:
- **Skipping login flows** - Authenticate via headers instead of UI
- **Switching users** - Start new sessions with different auth tokens
- **API testing** - Access protected endpoints directly
- **Security** - Headers are scoped to the origin, not leaked to other domains
To set headers for multiple origins, use `--headers` with each `open` command:
```bash
agent-browser open api.example.com --headers '{"Authorization": "Bearer token1"}'
agent-browser open api.acme.com --headers '{"Authorization": "Bearer token2"}'
```
For global headers (all domains), use `set headers`:
```bash
agent-browser set headers '{"X-Custom-Header": "value"}'
```
## Custom Browser Executable
Use a custom browser executable instead of the bundled Chromium. This is useful for:
- **Serverless deployment**: Use lightweight Chromium builds like `@sparticuz/chromium` (~50MB vs ~684MB)
- **System browsers**: Use an existing Chrome/Chromium installation
- **Custom builds**: Use modified browser builds
### CLI Usage
```bash
# Via flag
agent-browser --executable-path /path/to/chromium open example.com
# Via environment variable
AGENT_BROWSER_EXECUTABLE_PATH=/path/to/chromium agent-browser open example.com
```
### Serverless Example (Vercel/AWS Lambda)
```typescript
import chromium from '@sparticuz/chromium';
import { BrowserManager } from 'agent-browser';
export async function handler() {
const browser = new BrowserManager();
await browser.launch({
executablePath: await chromium.executablePath(),
headless: true,
});
// ... use browser
}
```
## CDP Mode
Connect to an existing browser via Chrome DevTools Protocol:
```bash
# Start Chrome with: google-chrome --remote-debugging-port=9222
# Connect once, then run commands without --cdp
agent-browser connect 9222
agent-browser snapshot
agent-browser tab
agent-browser close
# Or pass --cdp on each command
agent-browser --cdp 9222 snapshot
```
This enables control of:
- Electron apps
- Chrome/Chromium instances with remote debugging
- WebView2 applications
- Any browser exposing a CDP endpoint
## Streaming (Browser Preview)
Stream the browser viewport via WebSocket for live preview or "pair browsing" where a human can watch and interact alongside an AI agent.
### Enable Streaming
Set the `AGENT_BROWSER_STREAM_PORT` environment variable:
```bash
AGENT_BROWSER_STREAM_PORT=9223 agent-browser open example.com
```
This starts a WebSocket server on the specified port that streams the browser viewport and accepts input events.
### WebSocket Protocol
Connect to `ws://localhost:9223` to receive frames and send input:
**Receive frames:**
```json
{
"type": "frame",
"data": "<base64-encoded-jpeg>",
"metadata": {
"deviceWidth": 1280,
"deviceHeight": 720,
"pageScaleFactor": 1,
"offsetTop": 0,
"scrollOffsetX": 0,
"scrollOffsetY": 0
}
}
```
**Send mouse events:**
```json
{
"type": "input_mouse",
"eventType": "mousePressed",
"x": 100,
"y": 200,
"button": "left",
"clickCount": 1
}
```
**Send keyboard events:**
```json
{
"type": "input_keyboard",
"eventType": "keyDown",
"key": "Enter",
"code": "Enter"
}
```
**Send touch events:**
```json
{
"type": "input_touch",
"eventType": "touchStart",
"touchPoints": [{ "x": 100, "y": 200 }]
}
```
### Programmatic API
For advanced use, control streaming directly via the protocol:
```typescript
import { BrowserManager } from 'agent-browser';
const browser = new BrowserManager();
await browser.launch({ headless: true });
await browser.navigate('https://example.com');
// Start screencast
await browser.startScreencast((frame) => {
// frame.data is base64-encoded image
// frame.metadata contains viewport info
console.log('Frame received:', frame.metadata.deviceWidth, 'x', frame.metadata.deviceHeight);
}, {
format: 'jpeg',
quality: 80,
maxWidth: 1280,
maxHeight: 720,
});
// Inject mouse events
await browser.injectMouseEvent({
type: 'mousePressed',
x: 100,
y: 200,
button: 'left',
});
// Inject keyboard events
await browser.injectKeyboardEvent({
type: 'keyDown',
key: 'Enter',
code: 'Enter',
});
// Stop when done
await browser.stopScreencast();
```
## Architecture
agent-browser uses a client-daemon architecture:
+1 -1
View File
@@ -4,7 +4,7 @@ version = 4
[[package]]
name = "agent-browser"
version = "0.4.3"
version = "0.5.0"
dependencies = [
"libc",
"serde",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "agent-browser"
version = "0.4.3"
version = "0.5.0"
edition = "2021"
description = "Fast browser automation CLI for AI agents"
license = "Apache-2.0"
+158
View File
@@ -0,0 +1,158 @@
//! Color output utilities respecting NO_COLOR environment variable.
//!
//! When the NO_COLOR environment variable is present (regardless of value),
//! all color formatting is disabled per https://no-color.org/
use std::env;
use std::sync::OnceLock;
/// Returns true if color output is enabled (NO_COLOR is NOT set)
pub fn is_enabled() -> bool {
static COLORS_ENABLED: OnceLock<bool> = OnceLock::new();
*COLORS_ENABLED.get_or_init(|| env::var("NO_COLOR").is_err())
}
/// Format text in red (errors)
pub fn red(text: &str) -> String {
if is_enabled() {
format!("\x1b[31m{}\x1b[0m", text)
} else {
text.to_string()
}
}
/// Format text in green (success)
pub fn green(text: &str) -> String {
if is_enabled() {
format!("\x1b[32m{}\x1b[0m", text)
} else {
text.to_string()
}
}
/// Format text in yellow (warnings)
pub fn yellow(text: &str) -> String {
if is_enabled() {
format!("\x1b[33m{}\x1b[0m", text)
} else {
text.to_string()
}
}
/// Format text in cyan (info/progress)
pub fn cyan(text: &str) -> String {
if is_enabled() {
format!("\x1b[36m{}\x1b[0m", text)
} else {
text.to_string()
}
}
/// Format text in bold
pub fn bold(text: &str) -> String {
if is_enabled() {
format!("\x1b[1m{}\x1b[0m", text)
} else {
text.to_string()
}
}
/// Format text in dim
pub fn dim(text: &str) -> String {
if is_enabled() {
format!("\x1b[2m{}\x1b[0m", text)
} else {
text.to_string()
}
}
/// Red X error indicator
pub fn error_indicator() -> &'static str {
static INDICATOR: OnceLock<String> = OnceLock::new();
INDICATOR.get_or_init(|| {
if is_enabled() {
"\x1b[31m✗\x1b[0m".to_string()
} else {
"".to_string()
}
})
}
/// Green checkmark success indicator
pub fn success_indicator() -> &'static str {
static INDICATOR: OnceLock<String> = OnceLock::new();
INDICATOR.get_or_init(|| {
if is_enabled() {
"\x1b[32m✓\x1b[0m".to_string()
} else {
"".to_string()
}
})
}
/// Yellow warning indicator
pub fn warning_indicator() -> &'static str {
static INDICATOR: OnceLock<String> = OnceLock::new();
INDICATOR.get_or_init(|| {
if is_enabled() {
"\x1b[33m⚠\x1b[0m".to_string()
} else {
"".to_string()
}
})
}
/// Get console log color prefix by level
pub fn console_level_prefix(level: &str) -> String {
if !is_enabled() {
return format!("[{}]", level);
}
let color = match level {
"error" => "\x1b[31m",
"warning" => "\x1b[33m",
"info" => "\x1b[36m",
_ => "",
};
if color.is_empty() {
format!("[{}]", level)
} else {
format!("{}[{}]\x1b[0m", color, level)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_red_contains_ansi_codes() {
// Test the format structure (actual color depends on NO_COLOR env)
let formatted = format!("\x1b[31m{}\x1b[0m", "error");
assert!(formatted.contains("\x1b[31m"));
assert!(formatted.contains("\x1b[0m"));
}
#[test]
fn test_green_contains_ansi_codes() {
let formatted = format!("\x1b[32m{}\x1b[0m", "success");
assert!(formatted.contains("\x1b[32m"));
}
#[test]
fn test_console_level_prefix_contains_level() {
// Regardless of color state, the level text should be present
assert!(console_level_prefix("error").contains("error"));
assert!(console_level_prefix("warning").contains("warning"));
assert!(console_level_prefix("info").contains("info"));
assert!(console_level_prefix("log").contains("log"));
}
#[test]
fn test_indicators_contain_symbols() {
// Regardless of color state, symbols should be present
assert!(error_indicator().contains('✗'));
assert!(success_indicator().contains('✓'));
assert!(warning_indicator().contains('⚠'));
}
}
+437 -28
View File
@@ -75,12 +75,24 @@ pub fn parse_command(args: &[String], flags: &Flags) -> Result<Value, ParseError
context: cmd.to_string(),
usage: "open <url>",
})?;
let url = if url.starts_with("http") {
let url_lower = url.to_lowercase();
let url = if url_lower.starts_with("http://")
|| url_lower.starts_with("https://")
|| url_lower.starts_with("about:")
|| url_lower.starts_with("data:")
|| url_lower.starts_with("file:") {
url.to_string()
} else {
format!("https://{}", url)
};
Ok(json!({ "id": id, "action": "navigate", "url": url }))
let mut nav_cmd = json!({ "id": id, "action": "navigate", "url": url });
// If --headers flag is set, include headers (scoped to this origin)
if let Some(ref headers_json) = flags.headers {
if let Ok(headers) = serde_json::from_str::<serde_json::Value>(headers_json) {
nav_cmd["headers"] = headers;
}
}
Ok(nav_cmd)
}
"back" => Ok(json!({ "id": id, "action": "back" })),
"forward" => Ok(json!({ "id": id, "action": "forward" })),
@@ -146,13 +158,18 @@ pub fn parse_command(args: &[String], flags: &Flags) -> Result<Value, ParseError
"select" => {
let sel = rest.get(0).ok_or_else(|| ParseError::MissingArguments {
context: "select".to_string(),
usage: "select <selector> <value>",
usage: "select <selector> <value...>",
})?;
let val = rest.get(1).ok_or_else(|| ParseError::MissingArguments {
let _val = rest.get(1).ok_or_else(|| ParseError::MissingArguments {
context: "select".to_string(),
usage: "select <selector> <value>",
usage: "select <selector> <value...>",
})?;
Ok(json!({ "id": id, "action": "select", "selector": sel, "value": val }))
let values = &rest[1..];
if values.len() == 1 {
Ok(json!({ "id": id, "action": "select", "selector": sel, "values": values[0] }))
} else {
Ok(json!({ "id": id, "action": "select", "selector": sel, "values": values }))
}
}
"drag" => {
let src = rest.get(0).ok_or_else(|| ParseError::MissingArguments {
@@ -266,7 +283,11 @@ pub fn parse_command(args: &[String], flags: &Flags) -> Result<Value, ParseError
// === Screenshot/PDF ===
"screenshot" => {
Ok(json!({ "id": id, "action": "screenshot", "path": rest.get(0), "fullPage": flags.full }))
let mut cmd = json!({ "id": id, "action": "screenshot", "fullPage": flags.full });
if let Some(path) = rest.get(0) {
cmd["path"] = json!(path);
}
Ok(cmd)
}
"pdf" => {
let path = rest.get(0).ok_or_else(|| ParseError::MissingArguments {
@@ -316,6 +337,19 @@ pub fn parse_command(args: &[String], flags: &Flags) -> Result<Value, ParseError
// === Close ===
"close" | "quit" | "exit" => Ok(json!({ "id": id, "action": "close" })),
// === Connect (CDP) ===
"connect" => {
let port_str = rest.get(0).ok_or_else(|| ParseError::MissingArguments {
context: "connect".to_string(),
usage: "connect <port>",
})?;
let port: u16 = port_str.parse().map_err(|_| ParseError::MissingArguments {
context: format!("connect: invalid port '{}'", port_str),
usage: "connect <port>",
})?;
Ok(json!({ "id": id, "action": "launch", "cdpPort": port }))
}
// === Get ===
"get" => parse_get(&rest, &id),
@@ -360,10 +394,20 @@ pub fn parse_command(args: &[String], flags: &Flags) -> Result<Value, ParseError
// === Tabs ===
"tab" => {
match rest.get(0).map(|s| *s) {
Some("new") => Ok(json!({ "id": id, "action": "tab_new", "url": rest.get(1) })),
Some("new") => {
let mut cmd = json!({ "id": id, "action": "tab_new" });
if let Some(url) = rest.get(1) {
cmd["url"] = json!(url);
}
Ok(cmd)
}
Some("list") => Ok(json!({ "id": id, "action": "tab_list" })),
Some("close") => {
Ok(json!({ "id": id, "action": "tab_close", "index": rest.get(1).and_then(|s| s.parse::<i32>().ok()) }))
let mut cmd = json!({ "id": id, "action": "tab_close" });
if let Some(index) = rest.get(1).and_then(|s| s.parse::<i32>().ok()) {
cmd["index"] = json!(index);
}
Ok(cmd)
}
Some(n) if n.parse::<i32>().is_ok() => {
Ok(json!({ "id": id, "action": "tab_switch", "index": n.parse::<i32>().unwrap() }))
@@ -391,7 +435,7 @@ pub fn parse_command(args: &[String], flags: &Flags) -> Result<Value, ParseError
// === Frame ===
"frame" => {
if rest.get(0).map(|s| *s) == Some("main") {
Ok(json!({ "id": id, "action": "frame_main" }))
Ok(json!({ "id": id, "action": "mainframe" }))
} else {
let sel = rest.get(0).ok_or_else(|| ParseError::MissingArguments {
context: "frame".to_string(),
@@ -406,7 +450,11 @@ pub fn parse_command(args: &[String], flags: &Flags) -> Result<Value, ParseError
const VALID: &[&str] = &["accept", "dismiss"];
match rest.get(0).map(|s| *s) {
Some("accept") => {
Ok(json!({ "id": id, "action": "dialog", "response": "accept", "promptText": rest.get(1) }))
let mut cmd = json!({ "id": id, "action": "dialog", "response": "accept" });
if let Some(prompt_text) = rest.get(1) {
cmd["promptText"] = json!(prompt_text);
}
Ok(cmd)
}
Some("dismiss") => Ok(json!({ "id": id, "action": "dialog", "response": "dismiss" })),
Some(sub) => Err(ParseError::UnknownSubcommand {
@@ -424,8 +472,14 @@ pub fn parse_command(args: &[String], flags: &Flags) -> Result<Value, ParseError
"trace" => {
const VALID: &[&str] = &["start", "stop"];
match rest.get(0).map(|s| *s) {
Some("start") => Ok(json!({ "id": id, "action": "trace_start", "path": rest.get(1) })),
Some("stop") => Ok(json!({ "id": id, "action": "trace_stop", "path": rest.get(1) })),
Some("start") => Ok(json!({ "id": id, "action": "trace_start" })),
Some("stop") => {
let path = rest.get(1).ok_or_else(|| ParseError::MissingArguments {
context: "trace stop".to_string(),
usage: "trace stop <path>",
})?;
Ok(json!({ "id": id, "action": "trace_stop", "path": path }))
},
Some(sub) => Err(ParseError::UnknownSubcommand {
subcommand: sub.to_string(),
valid_options: VALID,
@@ -436,6 +490,60 @@ pub fn parse_command(args: &[String], flags: &Flags) -> Result<Value, ParseError
}),
}
}
// === Recording (Playwright native video recording) ===
"record" => {
const VALID: &[&str] = &["start", "stop", "restart"];
match rest.get(0).map(|s| *s) {
Some("start") => {
let path = rest.get(1).ok_or_else(|| ParseError::MissingArguments {
context: "record start".to_string(),
usage: "record start <output.webm> [url]",
})?;
// Optional URL parameter
let url = rest.get(2);
let mut cmd = json!({ "id": id, "action": "recording_start", "path": path });
if let Some(u) = url {
// Add https:// prefix if needed
let url_str = if u.starts_with("http") {
u.to_string()
} else {
format!("https://{}", u)
};
cmd["url"] = json!(url_str);
}
Ok(cmd)
}
Some("stop") => Ok(json!({ "id": id, "action": "recording_stop" })),
Some("restart") => {
let path = rest.get(1).ok_or_else(|| ParseError::MissingArguments {
context: "record restart".to_string(),
usage: "record restart <output.webm> [url]",
})?;
// Optional URL parameter
let url = rest.get(2);
let mut cmd = json!({ "id": id, "action": "recording_restart", "path": path });
if let Some(u) = url {
// Add https:// prefix if needed
let url_str = if u.starts_with("http") {
u.to_string()
} else {
format!("https://{}", u)
};
cmd["url"] = json!(url_str);
}
Ok(cmd)
}
Some(sub) => Err(ParseError::UnknownSubcommand {
subcommand: sub.to_string(),
valid_options: VALID,
}),
None => Err(ParseError::MissingArguments {
context: "record".to_string(),
usage: "record <start|stop|restart> [path] [url]",
}),
}
}
"console" => {
let clear = rest.iter().any(|&s| s == "--clear");
Ok(json!({ "id": id, "action": "console", "clear": clear }))
@@ -488,7 +596,7 @@ pub fn parse_command(args: &[String], flags: &Flags) -> Result<Value, ParseError
}
fn parse_get(rest: &[&str], id: &str) -> Result<Value, ParseError> {
const VALID: &[&str] = &["text", "html", "value", "attr", "url", "title", "count", "box"];
const VALID: &[&str] = &["text", "html", "value", "attr", "url", "title", "count", "box", "styles"];
match rest.get(0).map(|s| *s) {
Some("text") => {
@@ -539,13 +647,20 @@ fn parse_get(rest: &[&str], id: &str) -> Result<Value, ParseError> {
})?;
Ok(json!({ "id": id, "action": "boundingbox", "selector": sel }))
}
Some("styles") => {
let sel = rest.get(1).ok_or_else(|| ParseError::MissingArguments {
context: "get styles".to_string(),
usage: "get styles <selector>",
})?;
Ok(json!({ "id": id, "action": "styles", "selector": sel }))
}
Some(sub) => Err(ParseError::UnknownSubcommand {
subcommand: sub.to_string(),
valid_options: VALID,
}),
None => Err(ParseError::MissingArguments {
context: "get".to_string(),
usage: "get <text|html|value|attr|url|title|count|box> [args...]",
usage: "get <text|html|value|attr|url|title|count|box|styles> [args...]",
}),
}
}
@@ -623,15 +738,39 @@ fn parse_find(rest: &[&str], id: &str) -> Result<Value, ParseError> {
};
match *locator {
"role" => Ok(json!({ "id": id, "action": "getbyrole", "role": value, "subaction": subaction, "value": fill_value, "name": name, "exact": exact })),
"role" => {
let mut cmd = json!({ "id": id, "action": "getbyrole", "role": value, "subaction": subaction, "name": name, "exact": exact });
if let Some(v) = fill_value { cmd["value"] = json!(v); }
Ok(cmd)
}
"text" => Ok(json!({ "id": id, "action": "getbytext", "text": value, "subaction": subaction, "exact": exact })),
"label" => Ok(json!({ "id": id, "action": "getbylabel", "label": value, "subaction": subaction, "value": fill_value, "exact": exact })),
"placeholder" => Ok(json!({ "id": id, "action": "getbyplaceholder", "placeholder": value, "subaction": subaction, "value": fill_value, "exact": exact })),
"label" => {
let mut cmd = json!({ "id": id, "action": "getbylabel", "label": value, "subaction": subaction, "exact": exact });
if let Some(v) = fill_value { cmd["value"] = json!(v); }
Ok(cmd)
}
"placeholder" => {
let mut cmd = json!({ "id": id, "action": "getbyplaceholder", "placeholder": value, "subaction": subaction, "exact": exact });
if let Some(v) = fill_value { cmd["value"] = json!(v); }
Ok(cmd)
}
"alt" => Ok(json!({ "id": id, "action": "getbyalttext", "text": value, "subaction": subaction, "exact": exact })),
"title" => Ok(json!({ "id": id, "action": "getbytitle", "text": value, "subaction": subaction, "exact": exact })),
"testid" => Ok(json!({ "id": id, "action": "getbytestid", "testId": value, "subaction": subaction, "value": fill_value })),
"first" => Ok(json!({ "id": id, "action": "nth", "selector": value, "index": 0, "subaction": subaction, "value": fill_value })),
"last" => Ok(json!({ "id": id, "action": "nth", "selector": value, "index": -1, "subaction": subaction, "value": fill_value })),
"testid" => {
let mut cmd = json!({ "id": id, "action": "getbytestid", "testId": value, "subaction": subaction });
if let Some(v) = fill_value { cmd["value"] = json!(v); }
Ok(cmd)
}
"first" => {
let mut cmd = json!({ "id": id, "action": "nth", "selector": value, "index": 0, "subaction": subaction });
if let Some(v) = fill_value { cmd["value"] = json!(v); }
Ok(cmd)
}
"last" => {
let mut cmd = json!({ "id": id, "action": "nth", "selector": value, "index": -1, "subaction": subaction });
if let Some(v) = fill_value { cmd["value"] = json!(v); }
Ok(cmd)
}
_ => unreachable!(),
}
}
@@ -654,7 +793,9 @@ fn parse_find(rest: &[&str], id: &str) -> Result<Value, ParseError> {
} else {
None
};
Ok(json!({ "id": id, "action": "nth", "selector": sel, "index": idx, "subaction": sub, "value": fv }))
let mut cmd = json!({ "id": id, "action": "nth", "selector": sel, "index": idx, "subaction": sub });
if let Some(v) = fv { cmd["value"] = json!(v); }
Ok(cmd)
}
_ => Err(ParseError::UnknownSubcommand {
subcommand: locator.to_string(),
@@ -695,7 +836,7 @@ fn parse_mouse(rest: &[&str], id: &str) -> Result<Value, ParseError> {
Some("wheel") => {
let dy = rest.get(1).and_then(|s| s.parse::<i32>().ok()).unwrap_or(100);
let dx = rest.get(2).and_then(|s| s.parse::<i32>().ok()).unwrap_or(0);
Ok(json!({ "id": id, "action": "mousewheel", "deltaX": dx, "deltaY": dy }))
Ok(json!({ "id": id, "action": "wheel", "deltaX": dx, "deltaY": dy }))
}
Some(sub) => Err(ParseError::UnknownSubcommand {
subcommand: sub.to_string(),
@@ -766,7 +907,13 @@ fn parse_set(rest: &[&str], id: &str) -> Result<Value, ParseError> {
context: "set headers".to_string(),
usage: "set headers <json>",
})?;
Ok(json!({ "id": id, "action": "headers", "headers": headers_json }))
// Parse the JSON string into an object
let headers: serde_json::Value = serde_json::from_str(headers_json)
.map_err(|_| ParseError::MissingArguments {
context: "set headers".to_string(),
usage: "set headers <json> (must be valid JSON object)",
})?;
Ok(json!({ "id": id, "action": "headers", "headers": headers }))
}
Some("credentials") | Some("auth") => {
let user = rest.get(1).ok_or_else(|| ParseError::MissingArguments {
@@ -787,8 +934,12 @@ fn parse_set(rest: &[&str], id: &str) -> Result<Value, ParseError> {
} else {
"no-preference"
};
let reduced = rest.iter().any(|&s| s == "reduced-motion");
Ok(json!({ "id": id, "action": "media", "colorScheme": color, "reducedMotion": reduced }))
let reduced = if rest.iter().any(|&s| s == "reduced-motion") {
"reduce"
} else {
"no-preference"
};
Ok(json!({ "id": id, "action": "emulatemedia", "colorScheme": color, "reducedMotion": reduced }))
}
Some(sub) => Err(ParseError::UnknownSubcommand {
subcommand: sub.to_string(),
@@ -815,12 +966,22 @@ fn parse_network(rest: &[&str], id: &str) -> Result<Value, ParseError> {
let body = body_idx.and_then(|i| rest.get(i + 1).map(|s| *s));
Ok(json!({ "id": id, "action": "route", "url": url, "abort": abort, "body": body }))
}
Some("unroute") => Ok(json!({ "id": id, "action": "unroute", "url": rest.get(1) })),
Some("unroute") => {
let mut cmd = json!({ "id": id, "action": "unroute" });
if let Some(url) = rest.get(1) {
cmd["url"] = json!(url);
}
Ok(cmd)
},
Some("requests") => {
let clear = rest.iter().any(|&s| s == "--clear");
let filter_idx = rest.iter().position(|&s| s == "--filter");
let filter = filter_idx.and_then(|i| rest.get(i + 1).map(|s| *s));
Ok(json!({ "id": id, "action": "requests", "clear": clear, "filter": filter }))
let mut cmd = json!({ "id": id, "action": "requests", "clear": clear });
if let Some(f) = filter {
cmd["filter"] = json!(f);
}
Ok(cmd)
}
Some(sub) => Err(ParseError::UnknownSubcommand {
subcommand: sub.to_string(),
@@ -886,6 +1047,11 @@ mod tests {
full: false,
headed: false,
debug: false,
headers: None,
executable_path: None,
extensions: Vec::new(),
cdp: None,
proxy: None,
}
}
@@ -1012,6 +1178,81 @@ mod tests {
assert_eq!(cmd["url"], "https://example.com");
}
#[test]
fn test_navigate_with_headers() {
let mut flags = default_flags();
flags.headers = Some(r#"{"Authorization": "Bearer token"}"#.to_string());
let cmd = parse_command(&args("open api.example.com"), &flags).unwrap();
assert_eq!(cmd["action"], "navigate");
assert_eq!(cmd["url"], "https://api.example.com");
assert_eq!(cmd["headers"]["Authorization"], "Bearer token");
}
#[test]
fn test_navigate_with_multiple_headers() {
let mut flags = default_flags();
flags.headers = Some(r#"{"Authorization": "Bearer token", "X-Custom": "value"}"#.to_string());
let cmd = parse_command(&args("open api.example.com"), &flags).unwrap();
assert_eq!(cmd["headers"]["Authorization"], "Bearer token");
assert_eq!(cmd["headers"]["X-Custom"], "value");
}
#[test]
fn test_navigate_without_headers_flag() {
let cmd = parse_command(&args("open example.com"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "navigate");
// headers should not be present when flag is not set
assert!(cmd.get("headers").is_none());
}
#[test]
fn test_navigate_with_invalid_headers_json() {
let mut flags = default_flags();
flags.headers = Some("not valid json".to_string());
let cmd = parse_command(&args("open api.example.com"), &flags).unwrap();
// Invalid JSON should result in no headers field (graceful handling)
assert!(cmd.get("headers").is_none());
}
// === Set Headers Tests ===
#[test]
fn test_set_headers_parses_json() {
let input: Vec<String> = vec![
"set".to_string(),
"headers".to_string(),
r#"{"Authorization":"Bearer token"}"#.to_string(),
];
let cmd = parse_command(&input, &default_flags()).unwrap();
assert_eq!(cmd["action"], "headers");
// Headers should be an object, not a string
assert!(cmd["headers"].is_object());
assert_eq!(cmd["headers"]["Authorization"], "Bearer token");
}
#[test]
fn test_set_headers_with_multiple_values() {
let input: Vec<String> = vec![
"set".to_string(),
"headers".to_string(),
r#"{"Authorization": "Bearer token", "X-Custom": "value"}"#.to_string(),
];
let cmd = parse_command(&input, &default_flags()).unwrap();
assert_eq!(cmd["headers"]["Authorization"], "Bearer token");
assert_eq!(cmd["headers"]["X-Custom"], "value");
}
#[test]
fn test_set_headers_invalid_json_error() {
let input: Vec<String> = vec![
"set".to_string(),
"headers".to_string(),
"not-valid-json".to_string(),
];
let result = parse_command(&input, &default_flags());
assert!(result.is_err());
}
#[test]
fn test_back() {
let cmd = parse_command(&args("back"), &default_flags()).unwrap();
@@ -1055,12 +1296,46 @@ mod tests {
assert_eq!(cmd["text"], "some text");
}
#[test]
fn test_select() {
let cmd = parse_command(&args("select #menu option1"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "select");
assert_eq!(cmd["selector"], "#menu");
assert_eq!(cmd["values"], "option1");
}
#[test]
fn test_select_multiple_values() {
let cmd = parse_command(
&args("select #menu opt1 opt2 opt3"),
&default_flags(),
)
.unwrap();
assert_eq!(cmd["action"], "select");
assert_eq!(cmd["selector"], "#menu");
assert_eq!(cmd["values"], json!(["opt1", "opt2", "opt3"]));
}
#[test]
fn test_frame_main() {
let cmd = parse_command(&args("frame main"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "mainframe");
}
// === Tabs ===
#[test]
fn test_tab_new() {
let cmd = parse_command(&args("tab new"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "tab_new");
assert!(cmd.get("url").is_none(), "url should not be present when not provided");
}
#[test]
fn test_tab_new_with_url() {
let cmd = parse_command(&args("tab new https://example.com"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "tab_new");
assert_eq!(cmd["url"], "https://example.com");
}
#[test]
@@ -1088,6 +1363,14 @@ mod tests {
fn test_screenshot() {
let cmd = parse_command(&args("screenshot"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "screenshot");
assert!(cmd.get("path").is_none());
}
#[test]
fn test_screenshot_path() {
let cmd = parse_command(&args("screenshot out.png"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "screenshot");
assert_eq!(cmd["path"], "out.png");
}
#[test]
@@ -1181,6 +1464,82 @@ mod tests {
// === Unknown command ===
// === Record Tests ===
#[test]
fn test_record_start() {
let cmd = parse_command(&args("record start output.webm"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "recording_start");
assert_eq!(cmd["path"], "output.webm");
assert!(cmd.get("url").is_none());
}
#[test]
fn test_record_start_with_url() {
let cmd = parse_command(&args("record start demo.webm https://example.com"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "recording_start");
assert_eq!(cmd["path"], "demo.webm");
assert_eq!(cmd["url"], "https://example.com");
}
#[test]
fn test_record_start_with_url_no_protocol() {
let cmd = parse_command(&args("record start demo.webm example.com"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "recording_start");
assert_eq!(cmd["path"], "demo.webm");
assert_eq!(cmd["url"], "https://example.com");
}
#[test]
fn test_record_start_missing_path() {
let result = parse_command(&args("record start"), &default_flags());
assert!(result.is_err());
assert!(matches!(result.unwrap_err(), ParseError::MissingArguments { .. }));
}
#[test]
fn test_record_stop() {
let cmd = parse_command(&args("record stop"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "recording_stop");
}
#[test]
fn test_record_restart() {
let cmd = parse_command(&args("record restart output.webm"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "recording_restart");
assert_eq!(cmd["path"], "output.webm");
assert!(cmd.get("url").is_none());
}
#[test]
fn test_record_restart_with_url() {
let cmd = parse_command(&args("record restart demo.webm https://example.com"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "recording_restart");
assert_eq!(cmd["path"], "demo.webm");
assert_eq!(cmd["url"], "https://example.com");
}
#[test]
fn test_record_restart_missing_path() {
let result = parse_command(&args("record restart"), &default_flags());
assert!(result.is_err());
assert!(matches!(result.unwrap_err(), ParseError::MissingArguments { .. }));
}
#[test]
fn test_record_invalid_subcommand() {
let result = parse_command(&args("record foo"), &default_flags());
assert!(result.is_err());
assert!(matches!(result.unwrap_err(), ParseError::UnknownSubcommand { .. }));
}
#[test]
fn test_record_missing_subcommand() {
let result = parse_command(&args("record"), &default_flags());
assert!(result.is_err());
assert!(matches!(result.unwrap_err(), ParseError::MissingArguments { .. }));
}
#[test]
fn test_unknown_command() {
let result = parse_command(&args("unknowncommand"), &default_flags());
@@ -1224,4 +1583,54 @@ mod tests {
assert!(matches!(err, ParseError::MissingArguments { .. }));
assert!(err.format().contains("get text"));
}
// === Protocol alignment tests ===
#[test]
fn test_mouse_wheel() {
let cmd = parse_command(&args("mouse wheel 100 50"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "wheel");
assert_eq!(cmd["deltaY"], 100);
assert_eq!(cmd["deltaX"], 50);
}
#[test]
fn test_set_media() {
let cmd = parse_command(&args("set media dark"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "emulatemedia");
assert_eq!(cmd["colorScheme"], "dark");
assert_eq!(cmd["reducedMotion"], "no-preference");
}
#[test]
fn test_set_media_reduced_motion() {
let cmd = parse_command(&args("set media light reduced-motion"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "emulatemedia");
assert_eq!(cmd["colorScheme"], "light");
assert_eq!(cmd["reducedMotion"], "reduce");
}
#[test]
fn test_find_first_no_value() {
let cmd = parse_command(&args("find first a click"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "nth");
assert_eq!(cmd["index"], 0);
assert!(cmd.get("value").is_none());
}
#[test]
fn test_find_first_with_value() {
let cmd = parse_command(&args("find first input fill hello"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "nth");
assert_eq!(cmd["index"], 0);
assert_eq!(cmd["value"], "hello");
}
#[test]
fn test_find_nth_no_value() {
let cmd = parse_command(&args("find nth 2 a click"), &default_flags()).unwrap();
assert_eq!(cmd["action"], "nth");
assert_eq!(cmd["index"], 2);
assert!(cmd.get("value").is_none());
}
}
+50 -14
View File
@@ -104,7 +104,9 @@ fn get_port_for_session(session: &str) -> u16 {
for c in session.chars() {
hash = ((hash << 5).wrapping_sub(hash)).wrapping_add(c as i32);
}
49152 + ((hash.abs() as u16) % 16383)
// Correct logic: first take absolute modulo, then cast to u16
// Using unsigned_abs() to safely handle i32::MIN
49152 + ((hash.unsigned_abs() as u32 % 16383) as u16)
}
#[cfg(unix)]
@@ -140,7 +142,8 @@ fn is_daemon_running(session: &str) -> bool {
fn daemon_ready(session: &str) -> bool {
#[cfg(unix)]
{
get_socket_path(session).exists()
let socket_path = get_socket_path(session);
UnixStream::connect(&socket_path).is_ok()
}
#[cfg(windows)]
{
@@ -153,24 +156,44 @@ fn daemon_ready(session: &str) -> bool {
}
}
pub fn ensure_daemon(session: &str, headed: bool) -> Result<(), String> {
/// Result of ensure_daemon indicating whether a new daemon was started
pub struct DaemonResult {
/// True if we connected to an existing daemon, false if we started a new one
pub already_running: bool,
}
pub fn ensure_daemon(
session: &str,
headed: bool,
executable_path: Option<&str>,
extensions: &[String],
) -> Result<DaemonResult, String> {
if is_daemon_running(session) && daemon_ready(session) {
return Ok(());
return Ok(DaemonResult {
already_running: true,
});
}
let exe_path = env::current_exe().map_err(|e| e.to_string())?;
let exe_dir = exe_path.parent().unwrap();
let daemon_paths = [
let mut daemon_paths = vec![
exe_dir.join("daemon.js"),
exe_dir.join("../dist/daemon.js"),
PathBuf::from("dist/daemon.js"),
];
// Check AGENT_BROWSER_HOME environment variable
if let Ok(home) = env::var("AGENT_BROWSER_HOME") {
let home_path = PathBuf::from(&home);
daemon_paths.insert(0, home_path.join("dist/daemon.js"));
daemon_paths.insert(1, home_path.join("daemon.js"));
}
let daemon_path = daemon_paths
.iter()
.find(|p| p.exists())
.ok_or("Daemon not found. Run from project directory or ensure daemon.js is alongside binary.")?;
.ok_or("Daemon not found. Set AGENT_BROWSER_HOME environment variable or run from project directory.")?;
// Spawn daemon as a fully detached background process
#[cfg(unix)]
@@ -186,6 +209,14 @@ pub fn ensure_daemon(session: &str, headed: bool) -> Result<(), String> {
cmd.env("AGENT_BROWSER_HEADED", "1");
}
if let Some(path) = executable_path {
cmd.env("AGENT_BROWSER_EXECUTABLE_PATH", path);
}
if !extensions.is_empty() {
cmd.env("AGENT_BROWSER_EXTENSIONS", extensions.join(","));
}
// Create new process group and session to fully detach
unsafe {
cmd.pre_exec(|| {
@@ -206,13 +237,10 @@ pub fn ensure_daemon(session: &str, headed: bool) -> Result<(), String> {
{
use std::os::windows::process::CommandExt;
// On Windows, use cmd.exe to run node to ensure proper PATH resolution.
// This handles cases where node.exe isn't directly in PATH but node.cmd is.
// Pass the entire command as a single string to /c to handle paths with spaces.
let cmd_string = format!("node \"{}\"", daemon_path.display());
let mut cmd = Command::new("cmd");
cmd.arg("/c")
.arg(&cmd_string)
// On Windows, call node directly. Command::new handles PATH resolution (node.exe or node.cmd)
// and automatically quotes arguments containing spaces.
let mut cmd = Command::new("node");
cmd.arg(daemon_path)
.env("AGENT_BROWSER_DAEMON", "1")
.env("AGENT_BROWSER_SESSION", session);
@@ -220,6 +248,14 @@ pub fn ensure_daemon(session: &str, headed: bool) -> Result<(), String> {
cmd.env("AGENT_BROWSER_HEADED", "1");
}
if let Some(path) = executable_path {
cmd.env("AGENT_BROWSER_EXECUTABLE_PATH", path);
}
if !extensions.is_empty() {
cmd.env("AGENT_BROWSER_EXTENSIONS", extensions.join(","));
}
// CREATE_NEW_PROCESS_GROUP | DETACHED_PROCESS
const CREATE_NEW_PROCESS_GROUP: u32 = 0x00000200;
const DETACHED_PROCESS: u32 = 0x00000008;
@@ -234,7 +270,7 @@ pub fn ensure_daemon(session: &str, headed: bool) -> Result<(), String> {
for _ in 0..50 {
if daemon_ready(session) {
return Ok(());
return Ok(DaemonResult { already_running: false });
}
thread::sleep(Duration::from_millis(100));
}
+156 -1
View File
@@ -6,15 +6,30 @@ pub struct Flags {
pub headed: bool,
pub debug: bool,
pub session: String,
pub headers: Option<String>,
pub executable_path: Option<String>,
pub cdp: Option<String>,
pub extensions: Vec<String>,
pub proxy: Option<String>,
}
pub fn parse_flags(args: &[String]) -> Flags {
let extensions_env = env::var("AGENT_BROWSER_EXTENSIONS")
.ok()
.map(|s| s.split(',').map(|p| p.trim().to_string()).filter(|p| !p.is_empty()).collect::<Vec<_>>())
.unwrap_or_default();
let mut flags = Flags {
json: false,
full: false,
headed: false,
debug: false,
session: env::var("AGENT_BROWSER_SESSION").unwrap_or_else(|_| "default".to_string()),
headers: None,
executable_path: env::var("AGENT_BROWSER_EXECUTABLE_PATH").ok(),
cdp: None,
extensions: extensions_env,
proxy: None,
};
let mut i = 0;
@@ -30,6 +45,36 @@ pub fn parse_flags(args: &[String]) -> Flags {
i += 1;
}
}
"--headers" => {
if let Some(h) = args.get(i + 1) {
flags.headers = Some(h.clone());
i += 1;
}
}
"--executable-path" => {
if let Some(s) = args.get(i + 1) {
flags.executable_path = Some(s.clone());
i += 1;
}
},
"--extension" => {
if let Some(s) = args.get(i + 1) {
flags.extensions.push(s.clone());
i += 1;
}
},
"--cdp" => {
if let Some(s) = args.get(i + 1) {
flags.cdp = Some(s.clone());
i += 1;
}
}
"--proxy" => {
if let Some(p) = args.get(i + 1) {
flags.proxy = Some(p.clone());
i += 1;
}
}
_ => {}
}
i += 1;
@@ -43,13 +88,15 @@ pub fn clean_args(args: &[String]) -> Vec<String> {
// Global flags that should be stripped from command args
const GLOBAL_FLAGS: &[&str] = &["--json", "--full", "--headed", "--debug"];
// Global flags that take a value (need to skip the next arg too)
const GLOBAL_FLAGS_WITH_VALUE: &[&str] = &["--session", "--headers", "--executable-path", "--cdp", "--extension", "--proxy"];
for arg in args.iter() {
if skip_next {
skip_next = false;
continue;
}
if arg == "--session" {
if GLOBAL_FLAGS_WITH_VALUE.contains(&arg.as_str()) {
skip_next = true;
continue;
}
@@ -61,3 +108,111 @@ pub fn clean_args(args: &[String]) -> Vec<String> {
}
result
}
#[cfg(test)]
mod tests {
use super::*;
fn args(s: &str) -> Vec<String> {
s.split_whitespace().map(String::from).collect()
}
#[test]
fn test_parse_headers_flag() {
let flags = parse_flags(&args(r#"open example.com --headers {"Auth":"token"}"#));
assert_eq!(flags.headers, Some(r#"{"Auth":"token"}"#.to_string()));
}
#[test]
fn test_parse_headers_flag_with_spaces() {
// Headers JSON is passed as a single quoted argument in shell
let input: Vec<String> = vec![
"open".to_string(),
"example.com".to_string(),
"--headers".to_string(),
r#"{"Authorization": "Bearer token"}"#.to_string(),
];
let flags = parse_flags(&input);
assert_eq!(flags.headers, Some(r#"{"Authorization": "Bearer token"}"#.to_string()));
}
#[test]
fn test_parse_no_headers_flag() {
let flags = parse_flags(&args("open example.com"));
assert!(flags.headers.is_none());
}
#[test]
fn test_clean_args_removes_headers() {
let input: Vec<String> = vec![
"open".to_string(),
"example.com".to_string(),
"--headers".to_string(),
r#"{"Auth":"token"}"#.to_string(),
];
let clean = clean_args(&input);
assert_eq!(clean, vec!["open", "example.com"]);
}
#[test]
fn test_clean_args_removes_headers_at_start() {
let input: Vec<String> = vec![
"--headers".to_string(),
r#"{"Auth":"token"}"#.to_string(),
"open".to_string(),
"example.com".to_string(),
];
let clean = clean_args(&input);
assert_eq!(clean, vec!["open", "example.com"]);
}
#[test]
fn test_headers_with_other_flags() {
let input: Vec<String> = vec![
"open".to_string(),
"example.com".to_string(),
"--headers".to_string(),
r#"{"Auth":"token"}"#.to_string(),
"--json".to_string(),
"--headed".to_string(),
];
let flags = parse_flags(&input);
assert_eq!(flags.headers, Some(r#"{"Auth":"token"}"#.to_string()));
assert!(flags.json);
assert!(flags.headed);
let clean = clean_args(&input);
assert_eq!(clean, vec!["open", "example.com"]);
}
#[test]
fn test_parse_executable_path_flag() {
let flags = parse_flags(&args("--executable-path /path/to/chromium open example.com"));
assert_eq!(flags.executable_path, Some("/path/to/chromium".to_string()));
}
#[test]
fn test_parse_executable_path_flag_no_value() {
let flags = parse_flags(&args("--executable-path"));
assert_eq!(flags.executable_path, None);
}
#[test]
fn test_clean_args_removes_executable_path() {
let cleaned = clean_args(&args("--executable-path /path/to/chromium open example.com"));
assert_eq!(cleaned, vec!["open", "example.com"]);
}
#[test]
fn test_clean_args_removes_executable_path_with_other_flags() {
let cleaned = clean_args(&args("--json --executable-path /path/to/chromium --headed open example.com"));
assert_eq!(cleaned, vec!["open", "example.com"]);
}
#[test]
fn test_parse_flags_with_session_and_executable_path() {
let flags = parse_flags(&args("--session test --executable-path /custom/chrome open example.com"));
assert_eq!(flags.session, "test");
assert_eq!(flags.executable_path, Some("/custom/chrome".to_string()));
}
}
+32 -13
View File
@@ -1,3 +1,4 @@
use crate::color;
use std::process::{exit, Command, Stdio};
pub fn run_install(with_deps: bool) {
@@ -5,9 +6,15 @@ pub fn run_install(with_deps: bool) {
if is_linux {
if with_deps {
println!("\x1b[36mInstalling system dependencies...\x1b[0m");
println!("{}", color::cyan("Installing system dependencies..."));
let (pkg_mgr, deps) = if which_exists("apt-get") {
let libasound = if package_exists_apt("libasound2t64") {
"libasound2t64"
} else {
"libasound2"
};
(
"apt-get",
vec![
@@ -30,7 +37,7 @@ pub fn run_install(with_deps: bool) {
"libcairo2",
"libgdk-pixbuf-2.0-0",
"libxrender1",
"libasound2",
libasound,
"libfreetype6",
"libfontconfig1",
"libdbus-1-3",
@@ -93,7 +100,7 @@ pub fn run_install(with_deps: bool) {
],
)
} else {
eprintln!("\x1b[31m✗\x1b[0m No supported package manager found (apt-get, dnf, or yum)");
eprintln!("{} No supported package manager found (apt-get, dnf, or yum)", color::error_indicator());
exit(1);
};
@@ -112,22 +119,23 @@ pub fn run_install(with_deps: bool) {
match status {
Ok(s) if s.success() => {
println!("\x1b[32m✓\x1b[0m System dependencies installed")
println!("{} System dependencies installed", color::success_indicator())
}
Ok(_) => eprintln!(
"\x1b[33m⚠\x1b[0m Failed to install some dependencies. You may need to run manually with sudo."
"{} Failed to install some dependencies. You may need to run manually with sudo.",
color::warning_indicator()
),
Err(e) => eprintln!("\x1b[33m⚠\x1b[0m Could not run install command: {}", e),
Err(e) => eprintln!("{} Could not run install command: {}", color::warning_indicator(), e),
}
} else {
println!("\x1b[33m⚠\x1b[0m Linux detected. If browser fails to launch, run:");
println!("{} Linux detected. If browser fails to launch, run:", color::warning_indicator());
println!(" agent-browser install --with-deps");
println!(" or: npx playwright install-deps chromium");
println!();
}
}
println!("\x1b[36mInstalling Chromium browser...\x1b[0m");
println!("{}", color::cyan("Installing Chromium browser..."));
// On Windows, we need to use cmd.exe to run npx because npx is actually npx.cmd
// and Command::new() doesn't resolve .cmd files the way the shell does.
@@ -144,23 +152,23 @@ pub fn run_install(with_deps: bool) {
match status {
Ok(s) if s.success() => {
println!("\x1b[32m✓\x1b[0m Chromium installed successfully");
println!("{} Chromium installed successfully", color::success_indicator());
if is_linux && !with_deps {
println!();
println!("\x1b[33mNote:\x1b[0m If you see \"shared library\" errors when running, use:");
println!("{} If you see \"shared library\" errors when running, use:", color::yellow("Note:"));
println!(" agent-browser install --with-deps");
}
}
Ok(_) => {
eprintln!("\x1b[31m✗\x1b[0m Failed to install browser");
eprintln!("{} Failed to install browser", color::error_indicator());
if is_linux {
println!("\x1b[33mTip:\x1b[0m Try installing system dependencies first:");
println!("{} Try installing system dependencies first:", color::yellow("Tip:"));
println!(" agent-browser install --with-deps");
}
exit(1);
}
Err(e) => {
eprintln!("\x1b[31m✗\x1b[0m Failed to run npx: {}", e);
eprintln!("{} Failed to run npx: {}", color::error_indicator(), e);
eprintln!("Make sure Node.js is installed and npx is in your PATH");
exit(1);
}
@@ -189,3 +197,14 @@ fn which_exists(cmd: &str) -> bool {
.unwrap_or(false)
}
}
fn package_exists_apt(pkg: &str) -> bool {
Command::new("apt-cache")
.arg("show")
.arg(pkg)
.stdout(Stdio::null())
.stderr(Stdio::null())
.status()
.map(|s| s.success())
.unwrap_or(false)
}
+203 -19
View File
@@ -1,3 +1,4 @@
mod color;
mod commands;
mod connection;
mod flags;
@@ -21,7 +22,37 @@ use commands::{gen_id, parse_command, ParseError};
use connection::{ensure_daemon, send_command};
use flags::{clean_args, parse_flags};
use install::run_install;
use output::{print_command_help, print_help, print_response};
use output::{print_command_help, print_help, print_response, print_version};
fn parse_proxy(proxy_str: &str) -> serde_json::Value {
let Some(protocol_end) = proxy_str.find("://") else {
return json!({ "server": proxy_str });
};
let protocol = &proxy_str[..protocol_end + 3];
let rest = &proxy_str[protocol_end + 3..];
let Some(at_pos) = rest.rfind('@') else {
return json!({ "server": proxy_str });
};
let creds = &rest[..at_pos];
let server_part = &rest[at_pos + 1..];
let server = format!("{}{}", protocol, server_part);
let Some(colon_pos) = creds.find(':') else {
return json!({
"server": server,
"username": creds,
"password": ""
});
};
json!({
"server": server,
"username": &creds[..colon_pos],
"password": &creds[colon_pos + 1..]
})
}
fn run_session(args: &[String], session: &str, json_mode: bool) {
let subcommand = args.get(1).map(|s| s.as_str());
@@ -77,7 +108,7 @@ fn run_session(args: &[String], session: &str, json_mode: bool) {
} else {
println!("Active sessions:");
for s in &sessions {
let marker = if s == session { "" } else { " " };
let marker = if s == session { color::cyan("") } else { " ".to_string() };
println!("{} {}", marker, s);
}
}
@@ -94,16 +125,18 @@ fn run_session(args: &[String], session: &str, json_mode: bool) {
}
fn main() {
// Ignore SIGPIPE to prevent panic when piping to head/tail
#[cfg(unix)]
unsafe {
libc::signal(libc::SIGPIPE, libc::SIG_DFL);
}
let args: Vec<String> = env::args().skip(1).collect();
let flags = parse_flags(&args);
let clean = clean_args(&args);
let has_help = args.iter().any(|a| a == "--help" || a == "-h");
if clean.is_empty() {
print_help();
return;
}
let has_version = args.iter().any(|a| a == "--version" || a == "-V");
if has_help {
if let Some(cmd) = clean.get(0) {
@@ -115,6 +148,16 @@ fn main() {
return;
}
if has_version {
print_version();
return;
}
if clean.is_empty() {
print_help();
return;
}
// Handle install separately
if clean.get(0).map(|s| s.as_str()) == Some("install") {
let with_deps = args.iter().any(|a| a == "--with-deps" || a == "-d");
@@ -143,27 +186,109 @@ fn main() {
error_type
);
} else {
eprintln!("\x1b[31m{}\x1b[0m", e.format());
eprintln!("{}", color::red(&e.format()));
}
exit(1);
}
};
if let Err(e) = ensure_daemon(&flags.session, flags.headed) {
if flags.json {
println!(r#"{{"success":false,"error":"{}"}}"#, e);
} else {
eprintln!("\x1b[31m✗\x1b[0m {}", e);
let daemon_result = match ensure_daemon(&flags.session, flags.headed, flags.executable_path.as_deref(), &flags.extensions) {
Ok(result) => result,
Err(e) => {
if flags.json {
println!(r#"{{"success":false,"error":"{}"}}"#, e);
} else {
eprintln!("{} {}", color::error_indicator(), e);
}
exit(1);
}
};
// Warn if executable_path was specified but daemon was already running
if daemon_result.already_running && (flags.executable_path.is_some() || !flags.extensions.is_empty()) {
if !flags.json {
if flags.executable_path.is_some() {
eprintln!("{} --executable-path ignored: daemon already running. Use 'agent-browser close' first to restart with new path.", color::warning_indicator());
}
if !flags.extensions.is_empty() {
eprintln!("{} --extension ignored: daemon already running. Use 'agent-browser close' first to restart with extensions.", color::warning_indicator());
}
}
exit(1);
}
// If --headed flag is set, send launch command first to switch to headed mode
if flags.headed {
let launch_cmd = json!({ "id": gen_id(), "action": "launch", "headless": false });
// Connect via CDP if --cdp flag is set
if let Some(ref port) = flags.cdp {
let cdp_port: u16 = match port.parse::<u32>() {
Ok(p) if p == 0 => {
let msg = "Invalid CDP port: port must be greater than 0".to_string();
if flags.json {
println!(r#"{{"success":false,"error":"{}"}}"#, msg);
} else {
eprintln!("{} {}", color::error_indicator(), msg);
}
exit(1);
}
Ok(p) if p > 65535 => {
let msg = format!("Invalid CDP port: {} is out of range (valid range: 1-65535)", p);
if flags.json {
println!(r#"{{"success":false,"error":"{}"}}"#, msg);
} else {
eprintln!("{} {}", color::error_indicator(), msg);
}
exit(1);
}
Ok(p) => p as u16,
Err(_) => {
let msg = format!("Invalid CDP port: '{}' is not a valid number. Port must be a number between 1 and 65535", port);
if flags.json {
println!(r#"{{"success":false,"error":"{}"}}"#, msg);
} else {
eprintln!("{} {}", color::error_indicator(), msg);
}
exit(1);
}
};
let launch_cmd = json!({
"id": gen_id(),
"action": "launch",
"cdpPort": cdp_port
});
let err = match send_command(launch_cmd, &flags.session) {
Ok(resp) if resp.success => None,
Ok(resp) => Some(resp.error.unwrap_or_else(|| "CDP connection failed".to_string())),
Err(e) => Some(e.to_string()),
};
if let Some(msg) = err {
if flags.json {
println!(r#"{{"success":false,"error":"{}"}}"#, msg);
} else {
eprintln!("{} {}", color::error_indicator(), msg);
}
exit(1);
}
}
// Launch headed browser or proxy if flags are set (without CDP)
if (flags.headed || flags.proxy.is_some()) && flags.cdp.is_none() {
let mut launch_cmd = json!({
"id": gen_id(),
"action": "launch",
"headless": !flags.headed
});
if let Some(ref proxy_str) = flags.proxy {
let proxy_obj = parse_proxy(proxy_str);
launch_cmd.as_object_mut()
.expect("json! macro guarantees object type")
.insert("proxy".to_string(), proxy_obj);
}
if let Err(e) = send_command(launch_cmd, &flags.session) {
if !flags.json {
eprintln!("\x1b[33m⚠\x1b[0m Could not switch to headed mode: {}", e);
eprintln!("{} Could not configure browser: {}", color::warning_indicator(), e);
}
}
}
@@ -180,9 +305,68 @@ fn main() {
if flags.json {
println!(r#"{{"success":false,"error":"{}"}}"#, e);
} else {
eprintln!("\x1b[31m✗\x1b[0m {}", e);
eprintln!("{} {}", color::error_indicator(), e);
}
exit(1);
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_parse_proxy_simple() {
let result = parse_proxy("http://proxy.com:8080");
assert_eq!(result["server"], "http://proxy.com:8080");
assert!(result.get("username").is_none());
assert!(result.get("password").is_none());
}
#[test]
fn test_parse_proxy_with_auth() {
let result = parse_proxy("http://user:pass@proxy.com:8080");
assert_eq!(result["server"], "http://proxy.com:8080");
assert_eq!(result["username"], "user");
assert_eq!(result["password"], "pass");
}
#[test]
fn test_parse_proxy_username_only() {
let result = parse_proxy("http://user@proxy.com:8080");
assert_eq!(result["server"], "http://proxy.com:8080");
assert_eq!(result["username"], "user");
assert_eq!(result["password"], "");
}
#[test]
fn test_parse_proxy_no_protocol() {
let result = parse_proxy("proxy.com:8080");
assert_eq!(result["server"], "proxy.com:8080");
assert!(result.get("username").is_none());
}
#[test]
fn test_parse_proxy_socks5() {
let result = parse_proxy("socks5://proxy.com:1080");
assert_eq!(result["server"], "socks5://proxy.com:1080");
assert!(result.get("username").is_none());
}
#[test]
fn test_parse_proxy_socks5_with_auth() {
let result = parse_proxy("socks5://admin:secret@proxy.com:1080");
assert_eq!(result["server"], "socks5://proxy.com:1080");
assert_eq!(result["username"], "admin");
assert_eq!(result["password"], "secret");
}
#[test]
fn test_parse_proxy_complex_password() {
let result = parse_proxy("http://user:p@ss:w0rd@proxy.com:8080");
assert_eq!(result["server"], "http://proxy.com:8080");
assert_eq!(result["username"], "user");
assert_eq!(result["password"], "p@ss:w0rd");
}
}
+175 -23
View File
@@ -1,3 +1,4 @@
use crate::color;
use crate::connection::Response;
pub fn print_response(resp: &Response, json_mode: bool) {
@@ -8,7 +9,8 @@ pub fn print_response(resp: &Response, json_mode: bool) {
if !resp.success {
eprintln!(
"\x1b[31m✗\x1b[0m {}",
"{} {}",
color::error_indicator(),
resp.error.as_deref().unwrap_or("Unknown error")
);
return;
@@ -18,8 +20,8 @@ pub fn print_response(resp: &Response, json_mode: bool) {
// Navigation response
if let Some(url) = data.get("url").and_then(|v| v.as_str()) {
if let Some(title) = data.get("title").and_then(|v| v.as_str()) {
println!("\x1b[32m✓\x1b[0m \x1b[1m{}\x1b[0m", title);
println!("\x1b[2m {}\x1b[0m", url);
println!("{} {}", color::success_indicator(), color::bold(title));
println!(" {}", color::dim(url));
return;
}
println!("{}", url);
@@ -85,23 +87,17 @@ pub fn print_response(resp: &Response, json_mode: bool) {
.unwrap_or("Untitled");
let url = tab.get("url").and_then(|v| v.as_str()).unwrap_or("");
let active = tab.get("active").and_then(|v| v.as_bool()).unwrap_or(false);
let marker = if active { "" } else { " " };
let marker = if active { color::cyan("") } else { " ".to_string() };
println!("{} [{}] {} - {}", marker, i, title, url);
}
return;
}
// Console logs
if let Some(logs) = data.get("logs").and_then(|v| v.as_array()) {
if let Some(logs) = data.get("messages").and_then(|v| v.as_array()) {
for log in logs {
let level = log.get("type").and_then(|v| v.as_str()).unwrap_or("log");
let text = log.get("text").and_then(|v| v.as_str()).unwrap_or("");
let color = match level {
"error" => "\x1b[31m",
"warning" => "\x1b[33m",
"info" => "\x1b[36m",
_ => "\x1b[0m",
};
println!("{}[{}]\x1b[0m {}", color, level, text);
println!("{} {}", color::console_level_prefix(level), text);
}
return;
}
@@ -109,7 +105,7 @@ pub fn print_response(resp: &Response, json_mode: bool) {
if let Some(errors) = data.get("errors").and_then(|v| v.as_array()) {
for err in errors {
let msg = err.get("message").and_then(|v| v.as_str()).unwrap_or("");
println!("\x1b[31m✗\x1b[0m {}", msg);
println!("{} {}", color::error_indicator(), msg);
}
return;
}
@@ -122,6 +118,27 @@ pub fn print_response(resp: &Response, json_mode: bool) {
}
return;
}
// Network requests
if let Some(requests) = data.get("requests").and_then(|v| v.as_array()) {
if requests.is_empty() {
println!("No requests captured");
} else {
for req in requests {
let method = req.get("method").and_then(|v| v.as_str()).unwrap_or("GET");
let url = req.get("url").and_then(|v| v.as_str()).unwrap_or("");
let resource_type = req.get("resourceType").and_then(|v| v.as_str()).unwrap_or("");
println!("{} {} ({})", method, url, resource_type);
}
}
return;
}
// Cleared requests
if let Some(cleared) = data.get("cleared").and_then(|v| v.as_bool()) {
if cleared {
println!("\x1b[32m✓\x1b[0m Request log cleared");
return;
}
}
// Bounding box
if let Some(box_data) = data.get("box") {
println!(
@@ -130,18 +147,91 @@ pub fn print_response(resp: &Response, json_mode: bool) {
);
return;
}
// Closed
if data.get("closed").is_some() {
println!("\x1b[32m✓\x1b[0m Browser closed");
// Element styles
if let Some(elements) = data.get("elements").and_then(|v| v.as_array()) {
for (i, el) in elements.iter().enumerate() {
let tag = el.get("tag").and_then(|v| v.as_str()).unwrap_or("?");
let text = el.get("text").and_then(|v| v.as_str()).unwrap_or("");
println!("[{}] {} \"{}\"", i, tag, text);
if let Some(box_data) = el.get("box") {
let w = box_data.get("width").and_then(|v| v.as_i64()).unwrap_or(0);
let h = box_data.get("height").and_then(|v| v.as_i64()).unwrap_or(0);
let x = box_data.get("x").and_then(|v| v.as_i64()).unwrap_or(0);
let y = box_data.get("y").and_then(|v| v.as_i64()).unwrap_or(0);
println!(" box: {}x{} at ({}, {})", w, h, x, y);
}
if let Some(styles) = el.get("styles") {
let font_size = styles.get("fontSize").and_then(|v| v.as_str()).unwrap_or("");
let font_weight = styles.get("fontWeight").and_then(|v| v.as_str()).unwrap_or("");
let font_family = styles.get("fontFamily").and_then(|v| v.as_str()).unwrap_or("");
let color = styles.get("color").and_then(|v| v.as_str()).unwrap_or("");
let bg = styles.get("backgroundColor").and_then(|v| v.as_str()).unwrap_or("");
let radius = styles.get("borderRadius").and_then(|v| v.as_str()).unwrap_or("");
println!(" font: {} {} {}", font_size, font_weight, font_family);
println!(" color: {}", color);
println!(" background: {}", bg);
if radius != "0px" {
println!(" border-radius: {}", radius);
}
}
println!();
}
return;
}
// Screenshot path
// Closed
if data.get("closed").is_some() {
println!("{} Browser closed", color::success_indicator());
return;
}
// Recording start (has "started" field)
if let Some(started) = data.get("started").and_then(|v| v.as_bool()) {
if started {
if let Some(path) = data.get("path").and_then(|v| v.as_str()) {
println!("{} Recording started: {}", color::success_indicator(), path);
} else {
println!("{} Recording started", color::success_indicator());
}
return;
}
}
// Recording restart (has "stopped" field - from recording_restart action)
if data.get("stopped").is_some() {
let path = data.get("path").and_then(|v| v.as_str()).unwrap_or("unknown");
if let Some(prev_path) = data.get("previousPath").and_then(|v| v.as_str()) {
println!("{} Recording restarted: {} (previous saved to {})", color::success_indicator(), path, prev_path);
} else {
println!("{} Recording started: {}", color::success_indicator(), path);
}
return;
}
// Recording stop (has "frames" field - from recording_stop action)
if data.get("frames").is_some() {
if let Some(path) = data.get("path").and_then(|v| v.as_str()) {
if let Some(error) = data.get("error").and_then(|v| v.as_str()) {
println!("{} Recording saved to {} - {}", color::warning_indicator(), path, error);
} else {
println!("{} Recording saved to {}", color::success_indicator(), path);
}
} else {
println!("{} Recording stopped", color::success_indicator());
}
return;
}
// Screenshot path (no "started" or "frames" field)
if let Some(path) = data.get("path").and_then(|v| v.as_str()) {
println!("\x1b[32m✓\x1b[0m Screenshot saved to {}", path);
println!("{} Screenshot saved to {}", color::success_indicator(), color::green(path));
return;
}
// Screenshot base64
if let Some(base64) = data.get("base64").and_then(|v| v.as_str()) {
println!("{}", base64);
return;
}
// Default success
println!("\x1b[32m✓\x1b[0m Done");
println!("{} Done", color::success_indicator());
}
}
@@ -162,12 +252,15 @@ Aliases: goto, navigate
Global Options:
--json Output as JSON
--session <name> Use specific session
--headers <json> Set HTTP headers (scoped to this origin)
--headed Show browser window
Examples:
agent-browser open example.com
agent-browser open https://github.com
agent-browser open localhost:3000
agent-browser open api.example.com --headers '{"Authorization": "Bearer token"}'
# ^ Headers only sent to api.example.com, not other domains
"##,
"back" => r##"
agent-browser back - Navigate back in history
@@ -347,9 +440,9 @@ Examples:
"select" => r##"
agent-browser select - Select a dropdown option
Usage: agent-browser select <selector> <value>
Usage: agent-browser select <selector> <value...>
Selects an option in a <select> dropdown by its value attribute.
Selects one or more options in a <select> dropdown by value.
Global Options:
--json Output as JSON
@@ -358,6 +451,7 @@ Global Options:
Examples:
agent-browser select "#country" "US"
agent-browser select @e5 "option2"
agent-browser select "#menu" "opt1" "opt2" "opt3"
"##,
"drag" => r##"
agent-browser drag - Drag and drop
@@ -639,6 +733,7 @@ Subcommands:
url Get current URL
count <selector> Count matching elements
box <selector> Get bounding box (x, y, width, height)
styles <selector> Get computed styles of elements
Global Options:
--json Output as JSON
@@ -653,6 +748,8 @@ Examples:
agent-browser get url
agent-browser get count "li.item"
agent-browser get box "#header"
agent-browser get styles "button"
agent-browser get styles @e1
"##,
// === Is ===
@@ -976,6 +1073,42 @@ Examples:
agent-browser trace stop ./debug-trace.zip
"##,
// === Record (video) ===
"record" => r##"
agent-browser record - Record browser session to video
Usage: agent-browser record start <path.webm> [url]
agent-browser record stop
agent-browser record restart <path.webm> [url]
Record the browser to a WebM video file using Playwright's native recording.
Creates a fresh browser context but preserves cookies and localStorage.
If no URL is provided, automatically navigates to your current page.
Operations:
start <path> [url] Start recording (defaults to current URL if omitted)
stop Stop recording and save video
restart <path> [url] Stop current recording (if any) and start a new one
Global Options:
--json Output as JSON
--session <name> Use specific session
Examples:
# Record from current page (preserves login state)
agent-browser open https://app.example.com/dashboard
agent-browser snapshot -i # Explore and plan
agent-browser record start ./demo.webm
agent-browser click @e3 # Execute planned actions
agent-browser record stop
# Or specify a different URL
agent-browser record start ./demo.webm https://example.com
# Restart recording with a new file (stops previous, starts new)
agent-browser record restart ./take2.webm
"##,
// === Console/Errors ===
"console" => r##"
agent-browser console - View console logs
@@ -1118,7 +1251,7 @@ Core Commands:
focus <sel> Focus element
check <sel> Check checkbox
uncheck <sel> Uncheck checkbox
select <sel> <val> Select dropdown option
select <sel> <val...> Select dropdown option
drag <src> <dst> Drag and drop
upload <sel> <files...> Upload files
scroll <dir> [px] Scroll (up/down/left/right)
@@ -1128,6 +1261,7 @@ Core Commands:
pdf <path> Save as PDF
snapshot Accessibility tree with refs (for AI)
eval <js> Run JavaScript
connect <port> Connect to browser via CDP (e.g., connect 9222)
close Close browser
Navigation:
@@ -1136,7 +1270,7 @@ Navigation:
reload Reload page
Get Info: agent-browser get <what> [selector]
text, html, value, attr <name>, title, url, count, box
text, html, value, attr <name>, title, url, count, box, styles
Check State: agent-browser is <what> <selector>
visible, enabled, checked
@@ -1166,6 +1300,8 @@ Tabs:
Debug:
trace start|stop [path] Record trace
record start <path> [url] Start video recording (WebM)
record stop Stop and save video
console [--clear] View console logs
errors [--clear] View page errors
highlight <sel> Highlight element
@@ -1186,10 +1322,21 @@ Snapshot Options:
Options:
--session <name> Isolated session (or AGENT_BROWSER_SESSION env)
--headers <json> HTTP headers scoped to URL's origin (for auth)
--executable-path <path> Custom browser executable (or AGENT_BROWSER_EXECUTABLE_PATH)
--extension <path> Load browser extensions (repeatable).
--proxy <url> Proxy server (http://[user:pass@]host:port)
--json JSON output
--full, -f Full page screenshot
--headed Show browser window (not headless)
--cdp <port> Connect via CDP (Chrome DevTools Protocol)
--debug Debug output
--version, -V Show version
Environment:
AGENT_BROWSER_SESSION Session name (default: "default")
AGENT_BROWSER_EXECUTABLE_PATH Custom browser executable path
AGENT_BROWSER_STREAM_PORT Enable WebSocket streaming on port (e.g., 9223)
Examples:
agent-browser open example.com
@@ -1199,6 +1346,11 @@ Examples:
agent-browser find role button click --name Submit
agent-browser get text @e1
agent-browser screenshot --full
agent-browser --cdp 9222 snapshot # Connect via CDP port
"#
);
}
pub fn print_version() {
println!("agent-browser {}", env!("CARGO_PKG_VERSION"));
}
+41
View File
@@ -0,0 +1,41 @@
# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
# dependencies
/node_modules
/.pnp
.pnp.*
.yarn/*
!.yarn/patches
!.yarn/plugins
!.yarn/releases
!.yarn/versions
# testing
/coverage
# next.js
/.next/
/out/
# production
/build
# misc
.DS_Store
*.pem
# debug
npm-debug.log*
yarn-debug.log*
yarn-error.log*
.pnpm-debug.log*
# env files (can opt-in for committing if needed)
.env*
# vercel
.vercel
# typescript
*.tsbuildinfo
next-env.d.ts
+18
View File
@@ -0,0 +1,18 @@
import { defineConfig, globalIgnores } from "eslint/config";
import nextVitals from "eslint-config-next/core-web-vitals";
import nextTs from "eslint-config-next/typescript";
const eslintConfig = defineConfig([
...nextVitals,
...nextTs,
// Override default ignores of eslint-config-next.
globalIgnores([
// Default ignores of eslint-config-next:
".next/**",
"out/**",
"build/**",
"next-env.d.ts",
]),
]);
export default eslintConfig;
+7
View File
@@ -0,0 +1,7 @@
import type { NextConfig } from "next";
const nextConfig: NextConfig = {
/* config options here */
};
export default nextConfig;
+27
View File
@@ -0,0 +1,27 @@
{
"name": "docs",
"version": "0.1.0",
"private": true,
"scripts": {
"dev": "next dev",
"build": "next build",
"start": "next start",
"lint": "eslint"
},
"dependencies": {
"next": "16.1.1",
"react": "19.2.3",
"react-dom": "19.2.3",
"shiki": "^3.21.0"
},
"devDependencies": {
"@tailwindcss/postcss": "^4",
"@types/node": "^20",
"@types/react": "^19",
"@types/react-dom": "^19",
"eslint": "^9",
"eslint-config-next": "16.1.1",
"tailwindcss": "^4",
"typescript": "^5"
}
}
+4327
View File
File diff suppressed because it is too large Load Diff
+7
View File
@@ -0,0 +1,7 @@
const config = {
plugins: {
"@tailwindcss/postcss": {},
},
};
export default config;
+72
View File
@@ -0,0 +1,72 @@
import { CodeBlock } from "@/components/code-block";
export default function AgentMode() {
return (
<div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
<div className="prose">
<h1>Agent Mode</h1>
<p>
agent-browser works with any AI coding agent. Use <code>--json</code> for machine-readable output.
</p>
<h2>Compatible agents</h2>
<ul>
<li>Claude Code</li>
<li>Cursor</li>
<li>GitHub Copilot</li>
<li>OpenAI Codex</li>
<li>Google Gemini</li>
<li>opencode</li>
<li>Any agent that can run shell commands</li>
</ul>
<h2>JSON output</h2>
<CodeBlock code={`agent-browser snapshot --json
# {"success":true,"data":{"snapshot":"...","refs":{...}}}
agent-browser get text @e1 --json
agent-browser is visible @e2 --json`} />
<h2>Optimal workflow</h2>
<CodeBlock code={`# 1. Navigate and get snapshot
agent-browser open example.com
agent-browser snapshot -i --json # AI parses tree and refs
# 2. AI identifies target refs from snapshot
# 3. Execute actions using refs
agent-browser click @e2
agent-browser fill @e3 "input text"
# 4. Get new snapshot if page changed
agent-browser snapshot -i --json`} />
<h2>Integration</h2>
<h3>Just ask</h3>
<p>The simplest approach:</p>
<CodeBlock lang="text" code="Use agent-browser to test the login flow. Run agent-browser --help to see available commands." />
<p>The <code>--help</code> output is comprehensive.</p>
<h3>AGENTS.md / CLAUDE.md</h3>
<p>For consistent results, add to your instructions file:</p>
<CodeBlock lang="markdown" code={`## Browser Automation
Use \`agent-browser\` for web automation. Run \`agent-browser --help\` for all commands.
Core workflow:
1. \`agent-browser open <url>\` - Navigate to page
2. \`agent-browser snapshot -i\` - Get interactive elements with refs (@e1, @e2)
3. \`agent-browser click @e1\` / \`fill @e2 "text"\` - Interact using refs
4. Re-snapshot after page changes`} />
<h3>Claude Code skill</h3>
<p>For richer context:</p>
<CodeBlock code="cp -r node_modules/agent-browser/skills/agent-browser .claude/skills/" />
<p>Or download:</p>
<CodeBlock code={`mkdir -p .claude/skills/agent-browser
curl -o .claude/skills/agent-browser/SKILL.md \\
https://raw.githubusercontent.com/vercel-labs/agent-browser/main/skills/agent-browser/SKILL.md`} />
</div>
</div>
);
}
+83
View File
@@ -0,0 +1,83 @@
import { CodeBlock } from "@/components/code-block";
export default function CDPMode() {
return (
<div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
<div className="prose">
<h1>CDP Mode</h1>
<p>Connect to an existing browser via Chrome DevTools Protocol:</p>
<CodeBlock code={`# Start Chrome with: google-chrome --remote-debugging-port=9222
# Connect once, then run commands without --cdp
agent-browser connect 9222
agent-browser snapshot
agent-browser tab
agent-browser close
# Or pass --cdp on each command
agent-browser --cdp 9222 snapshot`} />
<h2>Use cases</h2>
<p>This enables control of:</p>
<ul>
<li>Electron apps</li>
<li>Chrome/Chromium with remote debugging</li>
<li>WebView2 applications</li>
<li>Any browser exposing a CDP endpoint</li>
</ul>
<h2>Global options</h2>
<table>
<thead>
<tr>
<th>Option</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>--session &lt;name&gt;</code></td>
<td>Use isolated session</td>
</tr>
<tr>
<td><code>--headers &lt;json&gt;</code></td>
<td>HTTP headers scoped to origin</td>
</tr>
<tr>
<td><code>--executable-path</code></td>
<td>Custom browser executable</td>
</tr>
<tr>
<td><code>--json</code></td>
<td>JSON output for agents</td>
</tr>
<tr>
<td><code>--full, -f</code></td>
<td>Full page screenshot</td>
</tr>
<tr>
<td><code>--name, -n</code></td>
<td>Locator name filter</td>
</tr>
<tr>
<td><code>--exact</code></td>
<td>Exact text match</td>
</tr>
<tr>
<td><code>--headed</code></td>
<td>Show browser window</td>
</tr>
<tr>
<td><code>--cdp &lt;port&gt;</code></td>
<td>CDP connection port</td>
</tr>
<tr>
<td><code>--debug</code></td>
<td>Debug output</td>
</tr>
</tbody>
</table>
</div>
</div>
);
}
+79
View File
@@ -0,0 +1,79 @@
import { CodeBlock } from "@/components/code-block";
export default function Changelog() {
return (
<div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
<div className="prose">
<h1>Changelog</h1>
<h2 id="v0.6.0">v0.6.0</h2>
<p className="text-[#888] text-sm">January 2026</p>
<h3>New Features</h3>
<ul>
<li>
<strong>Video recording</strong> - Record browser sessions to WebM using Playwright&apos;s native recording
<CodeBlock code={`agent-browser record start ./demo.webm
agent-browser click @e1
agent-browser record stop`} />
</li>
<li>
<strong><code>connect</code> command</strong> - Connect to a browser via CDP and persist the connection for subsequent commands
<CodeBlock code={`agent-browser connect 9222
agent-browser snapshot # No --cdp needed after connect`} />
</li>
<li>
<strong><code>--proxy</code> flag</strong> - Configure browser proxy with optional authentication
<CodeBlock code="agent-browser --proxy http://user:pass@proxy.com:8080 open example.com" />
</li>
<li>
<strong><code>get styles</code> command</strong> - Extract computed styles from elements
<CodeBlock code={`agent-browser get styles "button"`} />
</li>
<li>
<strong>Claude marketplace plugin</strong> - Added <code>.claude-plugin/marketplace.json</code> for Claude Code integration
</li>
<li>
<strong>Enhanced network output</strong> - <code>network requests</code> now shows method, URL, and resource type
</li>
<li>
<strong><code>--version</code> flag</strong> - Display CLI version
</li>
</ul>
<h3>Bug Fixes</h3>
<ul>
<li>Fix Windows daemon startup and port calculation</li>
<li>Support <code>libasound2t64</code> on newer Ubuntu versions (24.04+)</li>
<li>Prevent CDP timeout on empty URL tabs</li>
<li>Output screenshot as base64 when no path provided</li>
<li>Resolve refs in <code>get value</code> command</li>
<li>Support URL parameter in <code>tab new</code> command</li>
<li>Allow <code>about:</code>, <code>data:</code>, and <code>file:</code> URL schemes</li>
<li>Detect stale unix socket by attempting connection</li>
<li>Respect <code>AGENT_BROWSER_HEADED</code> environment variable</li>
<li>Handle SIGPIPE to prevent panic when piping to <code>head</code>/<code>tail</code></li>
<li>Fix null path validation in screenshot command</li>
</ul>
<h3>Protocol Alignment</h3>
<p>These changes align the CLI with the daemon protocol for consistency:</p>
<ul>
<li><code>select</code> command now uses <code>values</code> field (supports multiple selections)</li>
<li><code>frame main</code> uses <code>mainframe</code> action</li>
<li><code>mouse wheel</code> uses <code>wheel</code> action</li>
<li><code>set media</code> uses <code>emulatemedia</code> action</li>
<li>Console output uses <code>messages</code> field</li>
</ul>
<h3>Documentation</h3>
<ul>
<li>Expanded SKILL.md with comprehensive command reference</li>
<li>Updated README with new commands and options</li>
<li>Updated CDP mode documentation with <code>connect</code> workflow</li>
</ul>
</div>
</div>
);
}
+121
View File
@@ -0,0 +1,121 @@
import { CodeBlock } from "@/components/code-block";
export default function Commands() {
return (
<div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
<div className="prose">
<h1>Commands</h1>
<h2>Core</h2>
<CodeBlock code={`agent-browser open <url> # Navigate (aliases: goto, navigate)
agent-browser click <sel> # Click element
agent-browser dblclick <sel> # Double-click
agent-browser fill <sel> <text> # Clear and fill
agent-browser type <sel> <text> # Type into element
agent-browser press <key> # Press key (Enter, Tab, Control+a)
agent-browser hover <sel> # Hover element
agent-browser select <sel> <val> # Select dropdown option
agent-browser check <sel> # Check checkbox
agent-browser uncheck <sel> # Uncheck checkbox
agent-browser scroll <dir> [px] # Scroll (up/down/left/right)
agent-browser screenshot [path] # Screenshot (--full for full page)
agent-browser snapshot # Accessibility tree with refs
agent-browser eval <js> # Run JavaScript
agent-browser close # Close browser`} />
<h2>Get info</h2>
<CodeBlock code={`agent-browser get text <sel> # Get text content
agent-browser get html <sel> # Get innerHTML
agent-browser get value <sel> # Get input value
agent-browser get attr <sel> <attr> # Get attribute
agent-browser get title # Get page title
agent-browser get url # Get current URL
agent-browser get count <sel> # Count matching elements
agent-browser get box <sel> # Get bounding box`} />
<h2>Check state</h2>
<CodeBlock code={`agent-browser is visible <sel> # Check if visible
agent-browser is enabled <sel> # Check if enabled
agent-browser is checked <sel> # Check if checked`} />
<h2>Find elements</h2>
<p>Semantic locators with actions (<code>click</code>, <code>fill</code>, <code>check</code>, <code>hover</code>, <code>text</code>):</p>
<CodeBlock code={`agent-browser find role <role> <action> [value]
agent-browser find text <text> <action>
agent-browser find label <label> <action> [value]
agent-browser find placeholder <ph> <action> [value]
agent-browser find testid <id> <action> [value]
agent-browser find first <sel> <action> [value]
agent-browser find nth <n> <sel> <action> [value]`} />
<p>Examples:</p>
<CodeBlock code={`agent-browser find role button click --name "Submit"
agent-browser find label "Email" fill "test@test.com"
agent-browser find first ".item" click`} />
<h2>Wait</h2>
<CodeBlock code={`agent-browser wait <selector> # Wait for element
agent-browser wait <ms> # Wait for time
agent-browser wait --text "Welcome" # Wait for text
agent-browser wait --url "**/dash" # Wait for URL pattern
agent-browser wait --load networkidle # Wait for load state
agent-browser wait --fn "condition" # Wait for JS condition`} />
<h2>Mouse</h2>
<CodeBlock code={`agent-browser mouse move <x> <y> # Move mouse
agent-browser mouse down [button] # Press button
agent-browser mouse up [button] # Release button
agent-browser mouse wheel <dy> [dx] # Scroll wheel`} />
<h2>Settings</h2>
<CodeBlock code={`agent-browser set viewport <w> <h> # Set viewport size
agent-browser set device <name> # Emulate device ("iPhone 14")
agent-browser set geo <lat> <lng> # Set geolocation
agent-browser set offline [on|off] # Toggle offline mode
agent-browser set headers <json> # Extra HTTP headers
agent-browser set credentials <u> <p> # HTTP basic auth
agent-browser set media [dark|light] # Emulate color scheme`} />
<h2>Cookies & storage</h2>
<CodeBlock code={`agent-browser cookies # Get all cookies
agent-browser cookies set <name> <val> # Set cookie
agent-browser cookies clear # Clear cookies
agent-browser storage local # Get all localStorage
agent-browser storage local <key> # Get specific key
agent-browser storage local set <k> <v> # Set value
agent-browser storage local clear # Clear all
agent-browser storage session # Same for sessionStorage`} />
<h2>Network</h2>
<CodeBlock code={`agent-browser network route <url> # Intercept requests
agent-browser network route <url> --abort # Block requests
agent-browser network route <url> --body <json> # Mock response
agent-browser network unroute [url] # Remove routes
agent-browser network requests # View tracked requests`} />
<h2>Tabs & frames</h2>
<CodeBlock code={`agent-browser tab # List tabs
agent-browser tab new [url] # New tab
agent-browser tab <n> # Switch to tab
agent-browser tab close [n] # Close tab
agent-browser frame <sel> # Switch to iframe
agent-browser frame main # Back to main frame`} />
<h2>Debug</h2>
<CodeBlock code={`agent-browser trace start [path] # Start trace
agent-browser trace stop [path] # Stop and save trace
agent-browser console # View console messages
agent-browser errors # View page errors
agent-browser highlight <sel> # Highlight element
agent-browser state save <path> # Save auth state
agent-browser state load <path> # Load auth state`} />
<h2>Navigation</h2>
<CodeBlock code={`agent-browser back # Go back
agent-browser forward # Go forward
agent-browser reload # Reload page`} />
</div>
</div>
);
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 25 KiB

+182
View File
@@ -0,0 +1,182 @@
@import "tailwindcss";
:root {
--background: #000000;
--foreground: #ededed;
--muted: #888888;
--border: #222222;
--accent: #ededed;
}
@theme inline {
--color-background: var(--background);
--color-foreground: var(--foreground);
--color-muted: var(--muted);
--color-border: var(--border);
--color-accent: var(--accent);
--font-sans: var(--font-geist);
--font-mono: var(--font-geist-mono);
}
body {
background: var(--background);
color: var(--foreground);
font-family: var(--font-geist), system-ui, sans-serif;
}
/* Scrollbar */
::-webkit-scrollbar {
width: 6px;
height: 6px;
}
::-webkit-scrollbar-track {
background: transparent;
}
::-webkit-scrollbar-thumb {
background: #333;
border-radius: 3px;
}
::-webkit-scrollbar-thumb:hover {
background: #444;
}
/* Code blocks */
pre {
background: #111 !important;
border: 1px solid var(--border);
border-radius: 4px;
padding: 0.875rem;
overflow-x: auto;
font-family: var(--font-geist-mono), monospace;
font-size: 0.8125rem;
line-height: 1.7;
}
.code-block pre {
margin: 0;
}
.code-block {
margin-bottom: 1.25rem;
}
@media (max-width: 640px) {
pre {
font-size: 0.75rem;
padding: 0.75rem;
}
}
code {
font-family: var(--font-geist-mono), monospace;
}
:not(pre) > code {
background: #1a1a1a;
padding: 0.125rem 0.375rem;
border-radius: 3px;
font-size: 0.875em;
}
/* Prose */
.prose {
max-width: 100%;
}
.prose h1 {
font-size: 1.5rem;
font-weight: 500;
letter-spacing: -0.02em;
margin-bottom: 0.5rem;
color: #fff;
}
@media (min-width: 640px) {
.prose h1 {
font-size: 1.75rem;
}
}
.prose h2 {
font-size: 0.875rem;
font-weight: 500;
letter-spacing: 0;
text-transform: uppercase;
color: var(--muted);
margin-top: 3rem;
margin-bottom: 1rem;
}
.prose h3 {
font-size: 0.875rem;
font-weight: 500;
margin-top: 2rem;
margin-bottom: 0.75rem;
color: #ccc;
}
.prose p {
margin-bottom: 1.25rem;
line-height: 1.7;
color: var(--muted);
font-size: 0.9375rem;
}
.prose ul, .prose ol {
margin-bottom: 1.25rem;
padding-left: 1.25rem;
}
.prose li {
margin-bottom: 0.5rem;
color: var(--muted);
font-size: 0.9375rem;
line-height: 1.6;
}
.prose li strong {
color: #ccc;
font-weight: 500;
}
.prose a {
color: var(--foreground);
text-decoration: underline;
text-underline-offset: 2px;
}
.prose a:hover {
color: #fff;
}
.prose table {
width: 100%;
border-collapse: collapse;
margin: 1.5rem 0;
font-size: 0.8125rem;
}
.prose th, .prose td {
text-align: left;
padding: 0.625rem 0.875rem;
border-bottom: 1px solid var(--border);
}
.prose th {
font-weight: 500;
color: var(--muted);
text-transform: uppercase;
font-size: 0.75rem;
letter-spacing: 0.025em;
}
.prose td {
color: var(--muted);
}
.prose td code {
color: var(--foreground);
}
+58
View File
@@ -0,0 +1,58 @@
import { CodeBlock } from "@/components/code-block";
export default function Installation() {
return (
<div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
<div className="prose">
<h1>Installation</h1>
<h2>npm (recommended)</h2>
<CodeBlock code={`npm install -g agent-browser
agent-browser install # Download Chromium`} />
<h2>From source</h2>
<CodeBlock code={`git clone https://github.com/vercel-labs/agent-browser
cd agent-browser
pnpm install
pnpm build
pnpm build:native
./bin/agent-browser install
pnpm link --global`} />
<h2>Linux dependencies</h2>
<p>On Linux, install system dependencies:</p>
<CodeBlock code={`agent-browser install --with-deps
# or manually: npx playwright install-deps chromium`} />
<h2>Custom browser</h2>
<p>
Use a custom browser executable instead of bundled Chromium:
</p>
<ul>
<li><strong>Serverless</strong> - Use <code>@sparticuz/chromium</code> (~50MB vs ~684MB)</li>
<li><strong>System browser</strong> - Use existing Chrome installation</li>
<li><strong>Custom builds</strong> - Use modified browser builds</li>
</ul>
<CodeBlock code={`# Via flag
agent-browser --executable-path /path/to/chromium open example.com
# Via environment variable
AGENT_BROWSER_EXECUTABLE_PATH=/path/to/chromium agent-browser open example.com`} />
<h3>Serverless example</h3>
<CodeBlock lang="typescript" code={`import chromium from '@sparticuz/chromium';
import { BrowserManager } from 'agent-browser';
export async function handler() {
const browser = new BrowserManager();
await browser.launch({
executablePath: await chromium.executablePath(),
headless: true,
});
// ... use browser
}`} />
</div>
</div>
);
}
+40
View File
@@ -0,0 +1,40 @@
import type { Metadata } from "next";
import { Geist, Geist_Mono } from "next/font/google";
import "./globals.css";
import { Sidebar } from "@/components/sidebar";
const geist = Geist({
variable: "--font-geist",
subsets: ["latin"],
});
const geistMono = Geist_Mono({
variable: "--font-geist-mono",
subsets: ["latin"],
});
export const metadata: Metadata = {
title: "agent-browser",
description: "Headless browser automation CLI for AI agents",
};
export default function RootLayout({
children,
}: Readonly<{
children: React.ReactNode;
}>) {
return (
<html lang="en" className="dark">
<body
className={`${geist.variable} ${geistMono.variable} antialiased bg-zinc-950 text-zinc-100`}
>
<div className="flex min-h-screen">
<Sidebar />
<main className="flex-1 overflow-auto pt-14 lg:pt-0">
{children}
</main>
</div>
</body>
</html>
);
}
+69
View File
@@ -0,0 +1,69 @@
import { CodeBlock } from "@/components/code-block";
export default function Home() {
return (
<div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
<div className="prose">
<h1>agent-browser</h1>
<p>
Headless browser automation CLI for AI agents. Fast Rust CLI with Node.js fallback.
</p>
<CodeBlock code="npm install -g agent-browser" />
<h2>Features</h2>
<ul>
<li><strong>Universal</strong> - Works with any AI agent: Claude Code, Cursor, Codex, Copilot, Gemini, opencode, and more</li>
<li><strong>AI-first</strong> - Snapshot returns accessibility tree with refs for deterministic element selection</li>
<li><strong>Fast</strong> - Native Rust CLI for instant command parsing</li>
<li><strong>Complete</strong> - 50+ commands for navigation, forms, screenshots, network, storage</li>
<li><strong>Sessions</strong> - Multiple isolated browser instances with separate auth</li>
<li><strong>Cross-platform</strong> - macOS, Linux, Windows with native binaries</li>
<li><strong>Serverless</strong> - Custom executable path for lightweight Chromium builds</li>
</ul>
<h2>Example</h2>
<CodeBlock code={`# Navigate and get snapshot
agent-browser open example.com
agent-browser snapshot -i
# Output:
# - heading "Example Domain" [ref=e1]
# - link "More information..." [ref=e2]
# Interact using refs
agent-browser click @e2
agent-browser screenshot page.png
agent-browser close`} />
<h2>Why refs?</h2>
<p>
The <code>snapshot</code> command returns an accessibility tree where each element
has a unique ref like <code>@e1</code>, <code>@e2</code>. This provides:
</p>
<ul>
<li><strong>Deterministic</strong> - Ref points to exact element from snapshot</li>
<li><strong>Fast</strong> - No DOM re-query needed</li>
<li><strong>AI-friendly</strong> - LLMs can reliably parse and use refs</li>
</ul>
<h2>Architecture</h2>
<p>
Client-daemon architecture for optimal performance:
</p>
<ol>
<li><strong>Rust CLI</strong> - Parses commands, communicates with daemon</li>
<li><strong>Node.js Daemon</strong> - Manages Playwright browser instance</li>
</ol>
<p>
Daemon starts automatically and persists between commands.
</p>
<h2>Platforms</h2>
<p>
Native Rust binaries for macOS (ARM64, x64), Linux (ARM64, x64), and Windows (x64).
</p>
</div>
</div>
);
}
+51
View File
@@ -0,0 +1,51 @@
import { CodeBlock } from "@/components/code-block";
export default function QuickStart() {
return (
<div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
<div className="prose">
<h1>Quick Start</h1>
<h2>Basic workflow</h2>
<CodeBlock code={`agent-browser open example.com
agent-browser snapshot # Get accessibility tree with refs
agent-browser click @e2 # Click by ref from snapshot
agent-browser fill @e3 "test@example.com" # Fill by ref
agent-browser get text @e1 # Get text by ref
agent-browser screenshot # Base64 png to stdout
agent-browser screenshot page.png # Save to file
agent-browser close`} />
<h2>Traditional selectors</h2>
<p>CSS selectors and semantic locators also supported:</p>
<CodeBlock code={`agent-browser click "#submit"
agent-browser fill "#email" "test@example.com"
agent-browser find role button click --name "Submit"`} />
<h2>AI workflow</h2>
<p>Optimal workflow for AI agents:</p>
<CodeBlock code={`# 1. Navigate and get snapshot
agent-browser open example.com
agent-browser snapshot -i --json # AI parses tree and refs
# 2. AI identifies target refs from snapshot
# 3. Execute actions using refs
agent-browser click @e2
agent-browser fill @e3 "input text"
# 4. Get new snapshot if page changed
agent-browser snapshot -i --json`} />
<h2>Headed mode</h2>
<p>Show browser window for debugging:</p>
<CodeBlock code="agent-browser open example.com --headed" />
<h2>JSON output</h2>
<p>Use <code>--json</code> for machine-readable output:</p>
<CodeBlock code={`agent-browser snapshot --json
agent-browser get text @e1 --json
agent-browser is visible @e2 --json`} />
</div>
</div>
);
}
+53
View File
@@ -0,0 +1,53 @@
import { CodeBlock } from "@/components/code-block";
export default function Selectors() {
return (
<div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
<div className="prose">
<h1>Selectors</h1>
<h2>Refs (recommended)</h2>
<p>
Refs provide deterministic element selection from snapshots. Best for AI agents.
</p>
<CodeBlock code={`# 1. Get snapshot with refs
agent-browser snapshot
# Output:
# - heading "Example Domain" [ref=e1] [level=1]
# - button "Submit" [ref=e2]
# - textbox "Email" [ref=e3]
# - link "Learn more" [ref=e4]
# 2. Use refs to interact
agent-browser click @e2 # Click the button
agent-browser fill @e3 "test@example.com" # Fill the textbox
agent-browser get text @e1 # Get heading text
agent-browser hover @e4 # Hover the link`} />
<h3>Why refs?</h3>
<ul>
<li><strong>Deterministic</strong> - Ref points to exact element from snapshot</li>
<li><strong>Fast</strong> - No DOM re-query needed</li>
<li><strong>AI-friendly</strong> - LLMs can reliably parse and use refs</li>
</ul>
<h2>CSS selectors</h2>
<CodeBlock code={`agent-browser click "#id"
agent-browser click ".class"
agent-browser click "div > button"
agent-browser click "[data-testid='submit']"`} />
<h2>Text & XPath</h2>
<CodeBlock code={`agent-browser click "text=Submit"
agent-browser click "xpath=//button[@type='submit']"`} />
<h2>Semantic locators</h2>
<p>Find elements by role, label, or other semantic properties:</p>
<CodeBlock code={`agent-browser find role button click --name "Submit"
agent-browser find label "Email" fill "test@test.com"
agent-browser find placeholder "Search..." fill "query"
agent-browser find testid "submit-btn" click`} />
</div>
</div>
);
}
+66
View File
@@ -0,0 +1,66 @@
import { CodeBlock } from "@/components/code-block";
export default function Sessions() {
return (
<div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
<div className="prose">
<h1>Sessions</h1>
<p>Run multiple isolated browser instances:</p>
<CodeBlock code={`# Different sessions
agent-browser --session agent1 open site-a.com
agent-browser --session agent2 open site-b.com
# Or via environment variable
AGENT_BROWSER_SESSION=agent1 agent-browser click "#btn"
# List active sessions
agent-browser session list
# Output:
# Active sessions:
# -> default
# agent1
# Show current session
agent-browser session`} />
<h2>Session isolation</h2>
<p>Each session has its own:</p>
<ul>
<li>Browser instance</li>
<li>Cookies and storage</li>
<li>Navigation history</li>
<li>Authentication state</li>
</ul>
<h2>Authenticated sessions</h2>
<p>
Use <code>--headers</code> to set HTTP headers for a specific origin:
</p>
<CodeBlock code={`# Headers scoped to api.example.com only
agent-browser open api.example.com --headers '{"Authorization": "Bearer <token>"}'
# Requests to api.example.com include the auth header
agent-browser snapshot -i --json
agent-browser click @e2
# Navigate to another domain - headers NOT sent
agent-browser open other-site.com`} />
<p>Useful for:</p>
<ul>
<li><strong>Skipping login flows</strong> - Authenticate via headers</li>
<li><strong>Switching users</strong> - Different auth tokens per session</li>
<li><strong>API testing</strong> - Access protected endpoints</li>
<li><strong>Security</strong> - Headers scoped to origin, not leaked</li>
</ul>
<h2>Multiple origins</h2>
<CodeBlock code={`agent-browser open api.example.com --headers '{"Authorization": "Bearer token1"}'
agent-browser open api.acme.com --headers '{"Authorization": "Bearer token2"}'`} />
<h2>Global headers</h2>
<p>For headers on all domains:</p>
<CodeBlock code={`agent-browser set headers '{"X-Custom-Header": "value"}'`} />
</div>
</div>
);
}
+71
View File
@@ -0,0 +1,71 @@
import { CodeBlock } from "@/components/code-block";
export default function Snapshots() {
return (
<div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
<div className="prose">
<h1>Snapshots</h1>
<p>
The <code>snapshot</code> command returns the accessibility tree with refs for AI-friendly interaction.
</p>
<h2>Options</h2>
<p>Filter output to reduce size:</p>
<CodeBlock code={`agent-browser snapshot # Full accessibility tree
agent-browser snapshot -i # Interactive elements only
agent-browser snapshot -c # Compact (remove empty elements)
agent-browser snapshot -d 3 # Limit depth to 3 levels
agent-browser snapshot -s "#main" # Scope to CSS selector
agent-browser snapshot -i -c -d 5 # Combine options`} />
<table>
<thead>
<tr>
<th>Option</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>-i, --interactive</code></td>
<td>Only interactive elements (buttons, links, inputs)</td>
</tr>
<tr>
<td><code>-c, --compact</code></td>
<td>Remove empty structural elements</td>
</tr>
<tr>
<td><code>-d, --depth</code></td>
<td>Limit tree depth</td>
</tr>
<tr>
<td><code>-s, --selector</code></td>
<td>Scope to CSS selector</td>
</tr>
</tbody>
</table>
<h2>Output format</h2>
<CodeBlock code={`agent-browser snapshot
# Output:
# - heading "Example Domain" [ref=e1] [level=1]
# - button "Submit" [ref=e2]
# - textbox "Email" [ref=e3]
# - link "Learn more" [ref=e4]`} />
<h2>JSON output</h2>
<p>Use <code>--json</code> for machine-readable output:</p>
<CodeBlock code={`agent-browser snapshot --json
# {"success":true,"data":{"snapshot":"...","refs":{"e1":{"role":"heading","name":"Title"},...}}}`} />
<h2>Best practices</h2>
<ol>
<li>Use <code>-i</code> to reduce output to actionable elements</li>
<li>Use <code>--json</code> for structured parsing</li>
<li>Re-snapshot after page changes to get updated refs</li>
<li>Scope with <code>-s</code> for specific page sections</li>
</ol>
</div>
</div>
);
}
+217
View File
@@ -0,0 +1,217 @@
import { CodeBlock } from "@/components/code-block";
export default function Streaming() {
return (
<div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
<div className="prose">
<h1>Streaming</h1>
<p>
Stream the browser viewport via WebSocket for live preview or &quot;pair browsing&quot;
where a human can watch and interact alongside an AI agent.
</p>
<h2>Enable streaming</h2>
<p>
Set the <code>AGENT_BROWSER_STREAM_PORT</code> environment variable to start
a WebSocket server:
</p>
<CodeBlock code={`AGENT_BROWSER_STREAM_PORT=9223 agent-browser open example.com`} />
<p>
The server streams viewport frames and accepts input events (mouse, keyboard, touch).
</p>
<h2>WebSocket protocol</h2>
<p>Connect to <code>ws://localhost:9223</code> to receive frames and send input.</p>
<h3>Frame messages</h3>
<p>The server sends frame messages with base64-encoded images:</p>
<CodeBlock code={`{
"type": "frame",
"data": "<base64-encoded-jpeg>",
"metadata": {
"deviceWidth": 1280,
"deviceHeight": 720,
"pageScaleFactor": 1,
"offsetTop": 0,
"scrollOffsetX": 0,
"scrollOffsetY": 0
}
}`} />
<h3>Status messages</h3>
<p>Connection and screencast status:</p>
<CodeBlock code={`{
"type": "status",
"connected": true,
"screencasting": true,
"viewportWidth": 1280,
"viewportHeight": 720
}`} />
<h2>Input injection</h2>
<p>Send input events to control the browser remotely.</p>
<h3>Mouse events</h3>
<CodeBlock code={`// Click
{
"type": "input_mouse",
"eventType": "mousePressed",
"x": 100,
"y": 200,
"button": "left",
"clickCount": 1
}
// Release
{
"type": "input_mouse",
"eventType": "mouseReleased",
"x": 100,
"y": 200,
"button": "left"
}
// Move
{
"type": "input_mouse",
"eventType": "mouseMoved",
"x": 150,
"y": 250
}
// Scroll
{
"type": "input_mouse",
"eventType": "mouseWheel",
"x": 100,
"y": 200,
"deltaX": 0,
"deltaY": 100
}`} />
<h3>Keyboard events</h3>
<CodeBlock code={`// Key down
{
"type": "input_keyboard",
"eventType": "keyDown",
"key": "Enter",
"code": "Enter"
}
// Key up
{
"type": "input_keyboard",
"eventType": "keyUp",
"key": "Enter",
"code": "Enter"
}
// Type character
{
"type": "input_keyboard",
"eventType": "char",
"text": "a"
}
// With modifiers (1=Alt, 2=Ctrl, 4=Meta, 8=Shift)
{
"type": "input_keyboard",
"eventType": "keyDown",
"key": "c",
"code": "KeyC",
"modifiers": 2
}`} />
<h3>Touch events</h3>
<CodeBlock code={`// Touch start
{
"type": "input_touch",
"eventType": "touchStart",
"touchPoints": [{ "x": 100, "y": 200 }]
}
// Touch move
{
"type": "input_touch",
"eventType": "touchMove",
"touchPoints": [{ "x": 150, "y": 250 }]
}
// Touch end
{
"type": "input_touch",
"eventType": "touchEnd",
"touchPoints": []
}
// Multi-touch (pinch zoom)
{
"type": "input_touch",
"eventType": "touchStart",
"touchPoints": [
{ "x": 100, "y": 200, "id": 0 },
{ "x": 200, "y": 200, "id": 1 }
]
}`} />
<h2>Programmatic API</h2>
<p>For advanced use, control streaming directly via the TypeScript API:</p>
<CodeBlock code={`import { BrowserManager } from 'agent-browser';
const browser = new BrowserManager();
await browser.launch({ headless: true });
await browser.navigate('https://example.com');
// Start screencast with callback
await browser.startScreencast((frame) => {
console.log('Frame:', frame.metadata.deviceWidth, 'x', frame.metadata.deviceHeight);
// frame.data is base64-encoded image
}, {
format: 'jpeg', // or 'png'
quality: 80, // 0-100, jpeg only
maxWidth: 1280,
maxHeight: 720,
everyNthFrame: 1
});
// Inject mouse event
await browser.injectMouseEvent({
type: 'mousePressed',
x: 100,
y: 200,
button: 'left',
clickCount: 1
});
// Inject keyboard event
await browser.injectKeyboardEvent({
type: 'keyDown',
key: 'Enter',
code: 'Enter'
});
// Inject touch event
await browser.injectTouchEvent({
type: 'touchStart',
touchPoints: [{ x: 100, y: 200 }]
});
// Check if screencasting
console.log('Active:', browser.isScreencasting());
// Stop screencast
await browser.stopScreencast();`} />
<h2>Use cases</h2>
<ul>
<li><strong>Pair browsing</strong> - Human watches and assists AI agent in real-time</li>
<li><strong>Remote preview</strong> - View browser output in a separate UI</li>
<li><strong>Recording</strong> - Capture frames for video generation</li>
<li><strong>Mobile testing</strong> - Inject touch events for mobile emulation</li>
<li><strong>Accessibility testing</strong> - Manual interaction during automated tests</li>
</ul>
</div>
</div>
);
}
+22
View File
@@ -0,0 +1,22 @@
import { codeToHtml } from "shiki";
import { CopyButton } from "./copy-button";
interface CodeBlockProps {
code: string;
lang?: string;
}
export async function CodeBlock({ code, lang = "bash" }: CodeBlockProps) {
const trimmedCode = code.trim();
const html = await codeToHtml(trimmedCode, {
lang,
theme: "github-dark-default",
});
return (
<div className="code-block relative group">
<CopyButton code={trimmedCode} />
<div dangerouslySetInnerHTML={{ __html: html }} />
</div>
);
}
+40
View File
@@ -0,0 +1,40 @@
"use client";
import { useState } from "react";
interface CopyButtonProps {
code: string;
}
export function CopyButton({ code }: CopyButtonProps) {
const [copied, setCopied] = useState(false);
const handleCopy = async () => {
try {
await navigator.clipboard.writeText(code);
setCopied(true);
setTimeout(() => setCopied(false), 2000);
} catch (error) {
console.error("Failed to copy to clipboard:", error);
// Optionally, you could set an error state or show a toast notification here
}
};
return (
<button
onClick={handleCopy}
className="absolute top-2 right-2 p-1.5 rounded text-[#666] hover:text-[#999] hover:bg-[#333] opacity-0 group-hover:opacity-100 transition-all"
aria-label="Copy code"
>
{copied ? (
<svg className="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M5 13l4 4L19 7" />
</svg>
) : (
<svg className="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M8 16H6a2 2 0 01-2-2V6a2 2 0 012-2h8a2 2 0 012 2v2m-6 12h8a2 2 0 002-2v-8a2 2 0 00-2-2h-8a2 2 0 00-2 2v8a2 2 0 002 2z" />
</svg>
)}
</button>
);
}
+132
View File
@@ -0,0 +1,132 @@
"use client";
import Link from "next/link";
import { usePathname } from "next/navigation";
import { useState, useEffect } from "react";
const navigation = [
{ name: "Introduction", href: "/" },
{ name: "Installation", href: "/installation" },
{ name: "Quick Start", href: "/quick-start" },
{ name: "Commands", href: "/commands" },
{ name: "Selectors", href: "/selectors" },
{ name: "Sessions", href: "/sessions" },
{ name: "Snapshots", href: "/snapshots" },
{ name: "Streaming", href: "/streaming" },
{ name: "Agent Mode", href: "/agent-mode" },
{ name: "CDP Mode", href: "/cdp-mode" },
{ name: "Changelog", href: "/changelog" },
];
export function Sidebar() {
const pathname = usePathname();
const [isOpen, setIsOpen] = useState(false);
useEffect(() => {
setIsOpen(false);
}, [pathname]);
useEffect(() => {
const handleEscape = (e: KeyboardEvent) => {
if (e.key === "Escape") setIsOpen(false);
};
document.addEventListener("keydown", handleEscape);
return () => document.removeEventListener("keydown", handleEscape);
}, []);
return (
<>
{/* Mobile header */}
<header className="lg:hidden fixed top-0 left-0 right-0 z-50 bg-black/90 backdrop-blur-sm border-b border-[#222] px-4 py-3">
<div className="flex items-center justify-between">
<Link href="/" className="text-sm font-medium">
agent-browser
</Link>
<button
onClick={() => setIsOpen(!isOpen)}
className="p-2 -mr-2 text-[#888] hover:text-white transition-colors"
aria-label="Toggle menu"
>
{isOpen ? (
<svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M6 18L18 6M6 6l12 12" />
</svg>
) : (
<svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M4 6h16M4 12h16M4 18h16" />
</svg>
)}
</button>
</div>
</header>
{/* Mobile overlay */}
{isOpen && (
<div
className="lg:hidden fixed inset-0 z-40 bg-black/80"
onClick={() => setIsOpen(false)}
/>
)}
{/* Sidebar */}
<aside
className={`
fixed lg:sticky top-0 left-0 z-50 lg:z-auto
w-56 lg:w-48 h-screen
bg-black border-r border-[#222]
transform transition-transform duration-150 ease-out
${isOpen ? "translate-x-0" : "-translate-x-full lg:translate-x-0"}
pt-14 lg:pt-0
`}
>
<div className="h-full overflow-y-auto p-5">
{/* Desktop header */}
<div className="mb-8 hidden lg:block">
<Link href="/" className="text-sm font-medium">
agent-browser
</Link>
</div>
<nav className="space-y-0.5">
{navigation.map((item) => {
const isActive = pathname === item.href;
return (
<Link
key={item.name}
href={item.href}
className={`block px-2 py-1.5 text-[13px] transition-colors ${
isActive
? "text-white"
: "text-[#666] hover:text-[#999]"
}`}
>
{item.name}
</Link>
);
})}
</nav>
<div className="mt-8 pt-4 border-t border-[#222] space-y-0.5">
<a
href="https://github.com/vercel-labs/agent-browser"
target="_blank"
rel="noopener noreferrer"
className="block px-2 py-1.5 text-[13px] text-[#666] hover:text-[#999] transition-colors"
>
GitHub
</a>
<a
href="https://www.npmjs.com/package/agent-browser"
target="_blank"
rel="noopener noreferrer"
className="block px-2 py-1.5 text-[13px] text-[#666] hover:text-[#999] transition-colors"
>
npm
</a>
</div>
</div>
</aside>
</>
);
}
+34
View File
@@ -0,0 +1,34 @@
{
"compilerOptions": {
"target": "ES2017",
"lib": ["dom", "dom.iterable", "esnext"],
"allowJs": true,
"skipLibCheck": true,
"strict": true,
"noEmit": true,
"esModuleInterop": true,
"module": "esnext",
"moduleResolution": "bundler",
"resolveJsonModule": true,
"isolatedModules": true,
"jsx": "react-jsx",
"incremental": true,
"plugins": [
{
"name": "next"
}
],
"paths": {
"@/*": ["./src/*"]
}
},
"include": [
"next-env.d.ts",
"**/*.ts",
"**/*.tsx",
".next/types/**/*.ts",
".next/dev/types/**/*.ts",
"**/*.mts"
],
"exclude": ["node_modules"]
}
+3 -1
View File
@@ -1,6 +1,6 @@
{
"name": "agent-browser",
"version": "0.4.3",
"version": "0.5.0",
"description": "Headless browser automation CLI for AI agents",
"type": "module",
"main": "dist/daemon.js",
@@ -53,10 +53,12 @@
"homepage": "https://github.com/vercel-labs/agent-browser#readme",
"dependencies": {
"playwright-core": "^1.57.0",
"ws": "^8.19.0",
"zod": "^3.22.4"
},
"devDependencies": {
"@types/node": "^20.10.0",
"@types/ws": "^8.18.1",
"husky": "^9.1.7",
"lint-staged": "^15.2.11",
"playwright": "^1.57.0",
+27
View File
@@ -11,6 +11,9 @@ importers:
playwright-core:
specifier: ^1.57.0
version: 1.57.0
ws:
specifier: ^8.19.0
version: 8.19.0
zod:
specifier: ^3.22.4
version: 3.25.76
@@ -18,6 +21,9 @@ importers:
'@types/node':
specifier: ^20.10.0
version: 20.19.28
'@types/ws':
specifier: ^8.18.1
version: 8.18.1
husky:
specifier: ^9.1.7
version: 9.1.7
@@ -341,6 +347,9 @@ packages:
'@types/node@20.19.28':
resolution: {integrity: sha512-VyKBr25BuFDzBFCK5sUM6ZXiWfqgCTwTAOK8qzGV/m9FCirXYDlmczJ+d5dXBAQALGCdRRdbteKYfJ84NGEusw==}
'@types/ws@8.18.1':
resolution: {integrity: sha512-ThVF6DCVhA8kUGy+aazFQ4kXQ7E1Ty7A3ypFOe0IcJV8O/M511G99AW24irKrW56Wt44yG9+ij8FaqoBGkuBXg==}
'@vitest/expect@4.0.16':
resolution: {integrity: sha512-eshqULT2It7McaJkQGLkPjPjNph+uevROGuIMJdG3V+0BSR2w9u6J9Lwu+E8cK5TETlfou8GRijhafIMhXsimA==}
@@ -805,6 +814,18 @@ packages:
resolution: {integrity: sha512-42AtmgqjV+X1VpdOfyTGOYRi0/zsoLqtXQckTmqTeybT+BDIbM/Guxo7x3pE2vtpr1ok6xRqM9OpBe+Jyoqyww==}
engines: {node: '>=18'}
ws@8.19.0:
resolution: {integrity: sha512-blAT2mjOEIi0ZzruJfIhb3nps74PRWTCz1IjglWEEpQl5XS/UNama6u2/rjFkDDouqr4L67ry+1aGIALViWjDg==}
engines: {node: '>=10.0.0'}
peerDependencies:
bufferutil: ^4.0.1
utf-8-validate: '>=5.0.2'
peerDependenciesMeta:
bufferutil:
optional: true
utf-8-validate:
optional: true
yaml@2.8.2:
resolution: {integrity: sha512-mplynKqc1C2hTVYxd0PU2xQAc22TI1vShAYGksCCfxbn/dFwnHTNi1bvYsBTkhdUNtGIf5xNOg938rrSSYvS9A==}
engines: {node: '>= 14.6'}
@@ -985,6 +1006,10 @@ snapshots:
dependencies:
undici-types: 6.21.0
'@types/ws@8.18.1':
dependencies:
'@types/node': 20.19.28
'@vitest/expect@4.0.16':
dependencies:
'@standard-schema/spec': 1.1.0
@@ -1427,6 +1452,8 @@ snapshots:
string-width: 7.2.0
strip-ansi: 7.1.2
ws@8.19.0: {}
yaml@2.8.2: {}
zod@3.25.76: {}
+115 -6
View File
@@ -1,6 +1,7 @@
---
name: agent-browser
description: Automates browser interactions for web testing, form filling, screenshots, and data extraction. Use when the user needs to navigate websites, interact with web pages, fill forms, take screenshots, test web applications, or extract information from web pages.
allowed-tools: Bash(agent-browser:*)
---
# Browser Automation with agent-browser
@@ -28,56 +29,93 @@ agent-browser close # Close browser
```bash
agent-browser open <url> # Navigate to URL
agent-browser back # Go back
agent-browser forward # Go forward
agent-browser forward # Go forward
agent-browser reload # Reload page
agent-browser close # Close browser
```
### Snapshot (page analysis)
```bash
agent-browser snapshot # Full accessibility tree
agent-browser snapshot -i # Interactive elements only (recommended)
agent-browser snapshot -c # Compact output
agent-browser snapshot -d 3 # Limit depth to 3
agent-browser snapshot # Full accessibility tree
agent-browser snapshot -i # Interactive elements only (recommended)
agent-browser snapshot -c # Compact output
agent-browser snapshot -d 3 # Limit depth to 3
agent-browser snapshot -s "#main" # Scope to CSS selector
```
### Interactions (use @refs from snapshot)
```bash
agent-browser click @e1 # Click
agent-browser dblclick @e1 # Double-click
agent-browser focus @e1 # Focus element
agent-browser fill @e2 "text" # Clear and type
agent-browser type @e2 "text" # Type without clearing
agent-browser press Enter # Press key
agent-browser press Control+a # Key combination
agent-browser keydown Shift # Hold key down
agent-browser keyup Shift # Release key
agent-browser hover @e1 # Hover
agent-browser check @e1 # Check checkbox
agent-browser uncheck @e1 # Uncheck checkbox
agent-browser select @e1 "value" # Select dropdown
agent-browser scroll down 500 # Scroll page
agent-browser scrollintoview @e1 # Scroll element into view
agent-browser drag @e1 @e2 # Drag and drop
agent-browser upload @e1 file.pdf # Upload files
```
### Get information
```bash
agent-browser get text @e1 # Get element text
agent-browser get html @e1 # Get innerHTML
agent-browser get value @e1 # Get input value
agent-browser get attr @e1 href # Get attribute
agent-browser get title # Get page title
agent-browser get url # Get current URL
agent-browser get count ".item" # Count matching elements
agent-browser get box @e1 # Get bounding box
```
### Screenshots
### Check state
```bash
agent-browser is visible @e1 # Check if visible
agent-browser is enabled @e1 # Check if enabled
agent-browser is checked @e1 # Check if checked
```
### Screenshots & PDF
```bash
agent-browser screenshot # Screenshot to stdout
agent-browser screenshot path.png # Save to file
agent-browser screenshot --full # Full page
agent-browser pdf output.pdf # Save as PDF
```
### Video recording
```bash
agent-browser record start ./demo.webm # Start recording (uses current URL + state)
agent-browser click @e1 # Perform actions
agent-browser record stop # Stop and save video
agent-browser record restart ./take2.webm # Stop current + start new recording
```
Recording creates a fresh context but preserves cookies/storage from your session. If no URL is provided, it automatically returns to your current page. For smooth demos, explore first, then start recording.
### Wait
```bash
agent-browser wait @e1 # Wait for element
agent-browser wait 2000 # Wait milliseconds
agent-browser wait --text "Success" # Wait for text
agent-browser wait --url "**/dashboard" # Wait for URL pattern
agent-browser wait --load networkidle # Wait for network idle
agent-browser wait --fn "window.ready" # Wait for JS condition
```
### Mouse control
```bash
agent-browser mouse move 100 200 # Move mouse
agent-browser mouse down left # Press button
agent-browser mouse up left # Release button
agent-browser mouse wheel 100 # Scroll wheel
```
### Semantic locators (alternative to refs)
@@ -85,6 +123,66 @@ agent-browser wait --load networkidle # Wait for network idle
agent-browser find role button click --name "Submit"
agent-browser find text "Sign In" click
agent-browser find label "Email" fill "user@test.com"
agent-browser find first ".item" click
agent-browser find nth 2 "a" text
```
### Browser settings
```bash
agent-browser set viewport 1920 1080 # Set viewport size
agent-browser set device "iPhone 14" # Emulate device
agent-browser set geo 37.7749 -122.4194 # Set geolocation
agent-browser set offline on # Toggle offline mode
agent-browser set headers '{"X-Key":"v"}' # Extra HTTP headers
agent-browser set credentials user pass # HTTP basic auth
agent-browser set media dark # Emulate color scheme
```
### Cookies & Storage
```bash
agent-browser cookies # Get all cookies
agent-browser cookies set name value # Set cookie
agent-browser cookies clear # Clear cookies
agent-browser storage local # Get all localStorage
agent-browser storage local key # Get specific key
agent-browser storage local set k v # Set value
agent-browser storage local clear # Clear all
```
### Network
```bash
agent-browser network route <url> # Intercept requests
agent-browser network route <url> --abort # Block requests
agent-browser network route <url> --body '{}' # Mock response
agent-browser network unroute [url] # Remove routes
agent-browser network requests # View tracked requests
agent-browser network requests --filter api # Filter requests
```
### Tabs & Windows
```bash
agent-browser tab # List tabs
agent-browser tab new [url] # New tab
agent-browser tab 2 # Switch to tab
agent-browser tab close # Close tab
agent-browser window new # New window
```
### Frames
```bash
agent-browser frame "#iframe" # Switch to iframe
agent-browser frame main # Back to main frame
```
### Dialogs
```bash
agent-browser dialog accept [text] # Accept dialog
agent-browser dialog dismiss # Dismiss dialog
```
### JavaScript
```bash
agent-browser eval "document.title" # Run JavaScript
```
## Example: Form submission
@@ -137,7 +235,18 @@ agent-browser get text @e1 --json
## Debugging
```bash
agent-browser open example.com --headed # Show browser window
agent-browser console # View console messages
agent-browser errors # View page errors
agent-browser record start ./debug.webm # Record from current page
agent-browser record stop # Save recording
agent-browser open example.com --headed # Show browser window
agent-browser --cdp 9222 snapshot # Connect via CDP
agent-browser console # View console messages
agent-browser console --clear # Clear console
agent-browser errors # View page errors
agent-browser errors --clear # Clear errors
agent-browser highlight @e1 # Highlight element
agent-browser trace start # Start recording trace
agent-browser trace stop trace.zip # Stop and save trace
```
+39
View File
@@ -0,0 +1,39 @@
import { describe, it, expect } from 'vitest';
import { toAIFriendlyError } from './actions.js';
describe('toAIFriendlyError', () => {
describe('element blocked by overlay', () => {
it('should detect intercepts pointer events even when Timeout is in message', () => {
// This is the exact error from Playwright when a cookie banner blocks an element
// Bug: Previously this was incorrectly reported as "not found or not visible"
const error = new Error(
'TimeoutError: locator.click: Timeout 10000ms exceeded.\n' +
'Call log:\n' +
" - waiting for getByRole('link', { name: 'Anmelden', exact: true }).first()\n" +
' - locator resolved to <a href="https://example.com/login">Anmelden</a>\n' +
' - attempting click action\n' +
' 2 x waiting for element to be visible, enabled and stable\n' +
' - element is visible, enabled and stable\n' +
' - scrolling into view if needed\n' +
' - done scrolling\n' +
' - <body class="font-sans antialiased">...</body> intercepts pointer events\n' +
' - retrying click action'
);
const result = toAIFriendlyError(error, '@e4');
// Must NOT say "not found" - the element WAS found
expect(result.message).not.toContain('not found');
// Must indicate the element is blocked
expect(result.message).toContain('blocked by another element');
expect(result.message).toContain('modal or overlay');
});
it('should suggest dismissing cookie banners', () => {
const error = new Error('<div class="cookie-overlay"> intercepts pointer events');
const result = toAIFriendlyError(error, '@e1');
expect(result.message).toContain('cookie banners');
});
});
});
+252 -15
View File
@@ -1,5 +1,5 @@
import type { Page, Frame } from 'playwright-core';
import type { BrowserManager } from './browser.js';
import type { BrowserManager, ScreencastFrame } from './browser.js';
import type {
Command,
Response,
@@ -26,6 +26,7 @@ import type {
SelectCommand,
HoverCommand,
ContentCommand,
TabNewCommand,
TabSwitchCommand,
TabCloseCommand,
WindowNewCommand,
@@ -49,6 +50,7 @@ import type {
IsCheckedCommand,
CountCommand,
BoundingBoxCommand,
StylesCommand,
TraceStartCommand,
TraceStopCommand,
HarStopCommand,
@@ -94,6 +96,14 @@ import type {
MultiSelectCommand,
WaitForDownloadCommand,
ResponseBodyCommand,
ScreencastStartCommand,
ScreencastStopCommand,
InputMouseCommand,
InputKeyboardCommand,
InputTouchCommand,
RecordingStartCommand,
RecordingStopCommand,
RecordingRestartCommand,
NavigateData,
ScreenshotData,
EvaluateData,
@@ -102,9 +112,29 @@ import type {
TabNewData,
TabSwitchData,
TabCloseData,
ScreencastStartData,
ScreencastStopData,
RecordingStartData,
RecordingStopData,
RecordingRestartData,
InputEventData,
StylesData,
} from './types.js';
import { successResponse, errorResponse } from './protocol.js';
// Callback for screencast frames - will be set by the daemon when streaming is active
let screencastFrameCallback: ((frame: ScreencastFrame) => void) | null = null;
/**
* Set the callback for screencast frames
* This is called by the daemon to set up frame streaming
*/
export function setScreencastFrameCallback(
callback: ((frame: ScreencastFrame) => void) | null
): void {
screencastFrameCallback = callback;
}
// Snapshot response type
interface SnapshotData {
snapshot: string;
@@ -113,8 +143,9 @@ interface SnapshotData {
/**
* Convert Playwright errors to AI-friendly messages
* @internal Exported for testing
*/
function toAIFriendlyError(error: unknown, selector: string): Error {
export function toAIFriendlyError(error: unknown, selector: string): Error {
const message = error instanceof Error ? error.message : String(error);
// Handle strict mode violation (multiple elements match)
@@ -129,7 +160,24 @@ function toAIFriendlyError(error: unknown, selector: string): Error {
);
}
// Handle element not found
// Handle element not interactable (must be checked BEFORE timeout case)
// This includes cases where an overlay/modal blocks the element
if (message.includes('intercepts pointer events')) {
return new Error(
`Element "${selector}" is blocked by another element (likely a modal or overlay). ` +
`Try dismissing any modals/cookie banners first.`
);
}
// Handle element not visible
if (message.includes('not visible') && !message.includes('Timeout')) {
return new Error(
`Element "${selector}" is not visible. ` +
`Try scrolling it into view or check if it's hidden.`
);
}
// Handle element not found (timeout waiting for element)
if (
message.includes('waiting for') &&
(message.includes('to be visible') || message.includes('Timeout'))
@@ -140,14 +188,6 @@ function toAIFriendlyError(error: unknown, selector: string): Error {
);
}
// Handle element not interactable
if (message.includes('intercepts pointer events') || message.includes('not visible')) {
return new Error(
`Element "${selector}" is not interactable (may be hidden or covered). ` +
`Try scrolling it into view or check if a modal/overlay is blocking it.`
);
}
// Return original error for unknown cases
return error instanceof Error ? error : new Error(message);
}
@@ -280,6 +320,8 @@ export async function executeCommand(command: Command, browser: BrowserManager):
return await handleCount(command, browser);
case 'boundingbox':
return await handleBoundingBox(command, browser);
case 'styles':
return await handleStyles(command, browser);
case 'video_start':
return await handleVideoStart(command, browser);
case 'video_stop':
@@ -386,6 +428,22 @@ export async function executeCommand(command: Command, browser: BrowserManager):
return await handleWaitForDownload(command, browser);
case 'responsebody':
return await handleResponseBody(command, browser);
case 'screencast_start':
return await handleScreencastStart(command, browser);
case 'screencast_stop':
return await handleScreencastStop(command, browser);
case 'input_mouse':
return await handleInputMouse(command, browser);
case 'input_keyboard':
return await handleInputKeyboard(command, browser);
case 'input_touch':
return await handleInputTouch(command, browser);
case 'recording_start':
return await handleRecordingStart(command, browser);
case 'recording_stop':
return await handleRecordingStop(command, browser);
case 'recording_restart':
return await handleRecordingRestart(command, browser);
default: {
// TypeScript narrows to never here, but we handle it for safety
const unknownCommand = command as { id: string; action: string };
@@ -411,6 +469,12 @@ async function handleNavigate(
browser: BrowserManager
): Promise<Response<NavigateData>> {
const page = browser.getPage();
// If headers are provided, set up scoped headers for this origin
if (command.headers && Object.keys(command.headers).length > 0) {
await browser.setScopedHeaders(command.url, command.headers);
}
await page.goto(command.url, {
waitUntil: command.waitUntil ?? 'load',
});
@@ -650,10 +714,17 @@ async function handleClose(
}
async function handleTabNew(
command: Command & { action: 'tab_new' },
command: TabNewCommand,
browser: BrowserManager
): Promise<Response<TabNewData>> {
const result = await browser.newTab();
// Navigate to URL if provided (same pattern as handleNavigate)
if (command.url) {
const page = browser.getPage();
await page.goto(command.url, { waitUntil: 'domcontentloaded' });
}
return successResponse(command.id, result);
}
@@ -672,7 +743,7 @@ async function handleTabSwitch(
command: TabSwitchCommand,
browser: BrowserManager
): Promise<Response<TabSwitchData>> {
const result = browser.switchTo(command.index);
const result = await browser.switchTo(command.index);
const page = browser.getPage();
return successResponse(command.id, {
...result,
@@ -1179,6 +1250,62 @@ async function handleBoundingBox(
return successResponse(command.id, { box });
}
async function handleStyles(
command: StylesCommand,
browser: BrowserManager
): Promise<Response<StylesData>> {
const page = browser.getPage();
// Shared extraction logic as a string to be eval'd in browser context
const extractStylesScript = `(function(el) {
const s = getComputedStyle(el);
const r = el.getBoundingClientRect();
return {
tag: el.tagName.toLowerCase(),
text: el.innerText?.trim().slice(0, 80) || null,
box: {
x: Math.round(r.x),
y: Math.round(r.y),
width: Math.round(r.width),
height: Math.round(r.height),
},
styles: {
fontSize: s.fontSize,
fontWeight: s.fontWeight,
fontFamily: s.fontFamily.split(',')[0].trim().replace(/"/g, ''),
color: s.color,
backgroundColor: s.backgroundColor,
borderRadius: s.borderRadius,
border: s.border !== 'none' && s.borderWidth !== '0px' ? s.border : null,
boxShadow: s.boxShadow !== 'none' ? s.boxShadow : null,
padding: s.padding,
},
};
})`;
// Check if it's a ref - single element
if (browser.isRef(command.selector)) {
const locator = browser.getLocator(command.selector);
const element = (await locator.evaluate((el, script) => {
const fn = eval(script);
return fn(el);
}, extractStylesScript)) as StylesData['elements'][0];
return successResponse(command.id, { elements: [element] });
}
// CSS selector - can match multiple elements
const elements = (await page.$$eval(
command.selector,
(els, script) => {
const fn = eval(script);
return els.map((el) => fn(el));
},
extractStylesScript
)) as StylesData['elements'];
return successResponse(command.id, { elements });
}
// Advanced handlers
async function handleVideoStart(
@@ -1377,8 +1504,8 @@ async function handleInputValue(
command: InputValueCommand,
browser: BrowserManager
): Promise<Response> {
const page = browser.getPage();
const value = await page.locator(command.selector).inputValue();
const locator = browser.getLocator(command.selector);
const value = await locator.inputValue();
return successResponse(command.id, { value });
}
@@ -1763,3 +1890,113 @@ async function handleResponseBody(
body: parsed,
});
}
// Screencast and input injection handlers
async function handleScreencastStart(
command: ScreencastStartCommand,
browser: BrowserManager
): Promise<Response<ScreencastStartData>> {
if (!screencastFrameCallback) {
throw new Error('Screencast frame callback not set. Start the streaming server first.');
}
await browser.startScreencast(screencastFrameCallback, {
format: command.format,
quality: command.quality,
maxWidth: command.maxWidth,
maxHeight: command.maxHeight,
everyNthFrame: command.everyNthFrame,
});
return successResponse(command.id, {
started: true,
format: command.format ?? 'jpeg',
quality: command.quality ?? 80,
});
}
async function handleScreencastStop(
command: ScreencastStopCommand,
browser: BrowserManager
): Promise<Response<ScreencastStopData>> {
await browser.stopScreencast();
return successResponse(command.id, { stopped: true });
}
async function handleInputMouse(
command: InputMouseCommand,
browser: BrowserManager
): Promise<Response<InputEventData>> {
await browser.injectMouseEvent({
type: command.type,
x: command.x,
y: command.y,
button: command.button,
clickCount: command.clickCount,
deltaX: command.deltaX,
deltaY: command.deltaY,
modifiers: command.modifiers,
});
return successResponse(command.id, { injected: true });
}
async function handleInputKeyboard(
command: InputKeyboardCommand,
browser: BrowserManager
): Promise<Response<InputEventData>> {
await browser.injectKeyboardEvent({
type: command.type,
key: command.key,
code: command.code,
text: command.text,
modifiers: command.modifiers,
});
return successResponse(command.id, { injected: true });
}
async function handleInputTouch(
command: InputTouchCommand,
browser: BrowserManager
): Promise<Response<InputEventData>> {
await browser.injectTouchEvent({
type: command.type,
touchPoints: command.touchPoints,
modifiers: command.modifiers,
});
return successResponse(command.id, { injected: true });
}
// Recording handlers (Playwright native video recording)
async function handleRecordingStart(
command: RecordingStartCommand,
browser: BrowserManager
): Promise<Response<RecordingStartData>> {
await browser.startRecording(command.path, command.url);
return successResponse(command.id, {
started: true,
path: command.path,
});
}
async function handleRecordingStop(
command: RecordingStopCommand,
browser: BrowserManager
): Promise<Response<RecordingStopData>> {
const result = await browser.stopRecording();
return successResponse(command.id, result);
}
async function handleRecordingRestart(
command: RecordingRestartCommand,
browser: BrowserManager
): Promise<Response<RecordingRestartData>> {
const result = await browser.restartRecording(command.path, command.url);
return successResponse(command.id, {
started: true,
path: command.path,
previousPath: result.previousPath,
stopped: result.stopped,
});
}
+367 -1
View File
@@ -1,5 +1,6 @@
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { describe, it, expect, beforeAll, afterAll, vi } from 'vitest';
import { BrowserManager } from './browser.js';
import { chromium } from 'playwright-core';
describe('BrowserManager', () => {
let browser: BrowserManager;
@@ -22,6 +23,35 @@ describe('BrowserManager', () => {
const page = browser.getPage();
expect(page).toBeDefined();
});
it('should reject invalid executablePath', async () => {
const testBrowser = new BrowserManager();
await expect(
testBrowser.launch({
headless: true,
executablePath: '/nonexistent/path/to/chromium',
})
).rejects.toThrow();
});
it('should be no-op when relaunching with same options', async () => {
const browserInstance = browser.getBrowser();
await browser.launch({ id: 'test', action: 'launch', headless: true });
expect(browser.getBrowser()).toBe(browserInstance);
});
it('should reconnect when CDP port changes', async () => {
const newBrowser = new BrowserManager();
await newBrowser.launch({ id: 'test', action: 'launch', headless: true });
expect(newBrowser.getBrowser()).not.toBeNull();
await expect(
newBrowser.launch({ id: 'test', action: 'launch', cdpPort: 59999 })
).rejects.toThrow();
expect(newBrowser.getBrowser()).toBeNull();
await newBrowser.close();
});
});
describe('navigation', () => {
@@ -294,4 +324,340 @@ describe('BrowserManager', () => {
expect(h1).toBe('Example Domain');
});
});
describe('scoped headers', () => {
it('should register route for scoped headers', async () => {
// Test that setScopedHeaders doesn't throw and completes successfully
await browser.clearScopedHeaders();
await expect(
browser.setScopedHeaders('https://example.com', { 'X-Test': 'value' })
).resolves.not.toThrow();
await browser.clearScopedHeaders();
});
it('should handle full URL origin', async () => {
await browser.clearScopedHeaders();
await expect(
browser.setScopedHeaders('https://api.example.com/path', { Authorization: 'Bearer token' })
).resolves.not.toThrow();
await browser.clearScopedHeaders();
});
it('should handle hostname-only origin', async () => {
await browser.clearScopedHeaders();
await expect(
browser.setScopedHeaders('example.com', { 'X-Custom': 'value' })
).resolves.not.toThrow();
await browser.clearScopedHeaders();
});
it('should clear scoped headers for specific origin', async () => {
await browser.clearScopedHeaders();
await browser.setScopedHeaders('https://example.com', { 'X-Test': 'value' });
await expect(browser.clearScopedHeaders('https://example.com')).resolves.not.toThrow();
});
it('should clear all scoped headers', async () => {
await browser.setScopedHeaders('https://example.com', { 'X-Test-1': 'value1' });
await browser.setScopedHeaders('https://example.org', { 'X-Test-2': 'value2' });
await expect(browser.clearScopedHeaders()).resolves.not.toThrow();
});
it('should replace headers when called twice for same origin', async () => {
await browser.clearScopedHeaders();
await browser.setScopedHeaders('https://example.com', { 'X-First': 'first' });
// Second call should replace, not add
await expect(
browser.setScopedHeaders('https://example.com', { 'X-Second': 'second' })
).resolves.not.toThrow();
await browser.clearScopedHeaders();
});
it('should handle clearing non-existent origin gracefully', async () => {
await browser.clearScopedHeaders();
// Should not throw when clearing headers that were never set
await expect(browser.clearScopedHeaders('https://never-set.com')).resolves.not.toThrow();
});
});
describe('CDP session', () => {
it('should create CDP session on demand', async () => {
const cdp = await browser.getCDPSession();
expect(cdp).toBeDefined();
});
it('should reuse existing CDP session', async () => {
const cdp1 = await browser.getCDPSession();
const cdp2 = await browser.getCDPSession();
expect(cdp1).toBe(cdp2);
});
it('should filter out pages with empty URLs during CDP connection', async () => {
const mockBrowser = {
contexts: () => [
{
pages: () => [
{ url: () => 'http://example.com', on: vi.fn() },
{ url: () => '', on: vi.fn() }, // This page should be filtered out
{ url: () => 'http://anothersite.com', on: vi.fn() },
],
on: vi.fn(),
},
],
close: vi.fn(),
};
const spy = vi.spyOn(chromium, 'connectOverCDP').mockResolvedValue(mockBrowser as any);
const cdpBrowser = new BrowserManager();
await cdpBrowser.launch({ cdpPort: 9222 });
// Should have 2 pages, not 3
expect(cdpBrowser.getPages().length).toBe(2);
// Verify that the empty URL page is not in the list
const urls = cdpBrowser.getPages().map((p) => p.url());
expect(urls).not.toContain('');
expect(urls).toContain('http://example.com');
spy.mockRestore();
});
});
describe('screencast', () => {
it('should report screencasting state correctly', () => {
expect(browser.isScreencasting()).toBe(false);
});
it('should start screencast', async () => {
const frames: Array<{ data: string }> = [];
await browser.startScreencast((frame) => {
frames.push(frame);
});
expect(browser.isScreencasting()).toBe(true);
// Wait a bit for at least one frame
await new Promise((resolve) => setTimeout(resolve, 1000));
await browser.stopScreencast();
expect(browser.isScreencasting()).toBe(false);
expect(frames.length).toBeGreaterThan(0);
});
it('should start screencast with custom options', async () => {
const frames: Array<{ data: string }> = [];
await browser.startScreencast(
(frame) => {
frames.push(frame);
},
{
format: 'png',
quality: 100,
maxWidth: 800,
maxHeight: 600,
everyNthFrame: 1,
}
);
expect(browser.isScreencasting()).toBe(true);
// Wait for a frame
await new Promise((resolve) => setTimeout(resolve, 200));
await browser.stopScreencast();
expect(frames.length).toBeGreaterThan(0);
});
it('should throw when starting screencast twice', async () => {
await browser.startScreencast(() => {});
await expect(browser.startScreencast(() => {})).rejects.toThrow('Screencast already active');
await browser.stopScreencast();
});
it('should handle stop when not screencasting', async () => {
// Should not throw
await expect(browser.stopScreencast()).resolves.not.toThrow();
});
});
describe('tab switch invalidates CDP session', () => {
// Clean up any extra tabs before each test
beforeEach(async () => {
// Close all tabs except the first one
const tabs = await browser.listTabs();
for (let i = tabs.length - 1; i > 0; i--) {
await browser.closeTab(i);
}
// Ensure we're on tab 0
await browser.switchTo(0);
// Stop any active screencast
if (browser.isScreencasting()) {
await browser.stopScreencast();
}
});
it('should not invalidate CDP when switching to same tab', async () => {
// Get CDP session for current tab
const cdp1 = await browser.getCDPSession();
// Switch to same tab - should NOT invalidate
await browser.switchTo(0);
// Should be the same session
const cdp2 = await browser.getCDPSession();
expect(cdp2).toBe(cdp1);
});
it('should invalidate CDP session on tab switch', async () => {
// Get CDP session for tab 0
const cdp1 = await browser.getCDPSession();
expect(cdp1).toBeDefined();
// Create new tab - this switches to the new tab automatically
await browser.newTab();
// Get CDP session - should be different since we're on a new page
const cdp2 = await browser.getCDPSession();
expect(cdp2).toBeDefined();
// Sessions should be different objects (different pages have different CDP sessions)
expect(cdp2).not.toBe(cdp1);
});
it('should stop screencast on tab switch', async () => {
// Start screencast on tab 0
await browser.startScreencast(() => {});
expect(browser.isScreencasting()).toBe(true);
// Create new tab and switch
await browser.newTab();
await browser.switchTo(1);
// Screencast should be stopped (it's page-specific)
expect(browser.isScreencasting()).toBe(false);
});
});
describe('input injection', () => {
it('should inject mouse move event', async () => {
await expect(
browser.injectMouseEvent({
type: 'mouseMoved',
x: 100,
y: 100,
})
).resolves.not.toThrow();
});
it('should inject mouse click events', async () => {
await expect(
browser.injectMouseEvent({
type: 'mousePressed',
x: 100,
y: 100,
button: 'left',
clickCount: 1,
})
).resolves.not.toThrow();
await expect(
browser.injectMouseEvent({
type: 'mouseReleased',
x: 100,
y: 100,
button: 'left',
})
).resolves.not.toThrow();
});
it('should inject mouse wheel event', async () => {
await expect(
browser.injectMouseEvent({
type: 'mouseWheel',
x: 100,
y: 100,
deltaX: 0,
deltaY: 100,
})
).resolves.not.toThrow();
});
it('should inject keyboard events', async () => {
await expect(
browser.injectKeyboardEvent({
type: 'keyDown',
key: 'a',
code: 'KeyA',
})
).resolves.not.toThrow();
await expect(
browser.injectKeyboardEvent({
type: 'keyUp',
key: 'a',
code: 'KeyA',
})
).resolves.not.toThrow();
});
it('should inject char event', async () => {
// CDP char events only accept single characters
await expect(
browser.injectKeyboardEvent({
type: 'char',
text: 'h',
})
).resolves.not.toThrow();
});
it('should inject keyboard with modifiers', async () => {
await expect(
browser.injectKeyboardEvent({
type: 'keyDown',
key: 'c',
code: 'KeyC',
modifiers: 2, // Ctrl
})
).resolves.not.toThrow();
});
it('should inject touch events', async () => {
await expect(
browser.injectTouchEvent({
type: 'touchStart',
touchPoints: [{ x: 100, y: 100 }],
})
).resolves.not.toThrow();
await expect(
browser.injectTouchEvent({
type: 'touchMove',
touchPoints: [{ x: 150, y: 150 }],
})
).resolves.not.toThrow();
await expect(
browser.injectTouchEvent({
type: 'touchEnd',
touchPoints: [],
})
).resolves.not.toThrow();
});
it('should inject multi-touch events', async () => {
await expect(
browser.injectTouchEvent({
type: 'touchStart',
touchPoints: [
{ x: 100, y: 100, id: 0 },
{ x: 200, y: 200, id: 1 },
],
})
).resolves.not.toThrow();
await expect(
browser.injectTouchEvent({
type: 'touchEnd',
touchPoints: [],
})
).resolves.not.toThrow();
});
});
});
+733 -36
View File
@@ -11,10 +11,39 @@ import {
type Request,
type Route,
type Locator,
type CDPSession,
type Video,
} from 'playwright-core';
import path from 'node:path';
import os from 'node:os';
import { existsSync, mkdirSync, rmSync } from 'node:fs';
import type { LaunchCommand } from './types.js';
import { type RefMap, type EnhancedSnapshot, getEnhancedSnapshot, parseRef } from './snapshot.js';
// Screencast frame data from CDP
export interface ScreencastFrame {
data: string; // base64 encoded image
metadata: {
offsetTop: number;
pageScaleFactor: number;
deviceWidth: number;
deviceHeight: number;
scrollOffsetX: number;
scrollOffsetY: number;
timestamp?: number;
};
sessionId: number;
}
// Screencast options
export interface ScreencastOptions {
format?: 'jpeg' | 'png';
quality?: number; // 0-100, only for jpeg
maxWidth?: number;
maxHeight?: number;
everyNthFrame?: number;
}
interface TrackedRequest {
url: string;
method: string;
@@ -39,6 +68,8 @@ interface PageError {
*/
export class BrowserManager {
private browser: Browser | null = null;
private cdpPort: number | null = null;
private isPersistentContext: boolean = false;
private contexts: BrowserContext[] = [];
private pages: Page[] = [];
private activePageIndex: number = 0;
@@ -51,12 +82,26 @@ export class BrowserManager {
private isRecordingHar: boolean = false;
private refMap: RefMap = {};
private lastSnapshot: string = '';
private scopedHeaderRoutes: Map<string, (route: Route) => Promise<void>> = new Map();
// CDP session for screencast and input injection
private cdpSession: CDPSession | null = null;
private screencastActive: boolean = false;
private screencastSessionId: number = 0;
private frameCallback: ((frame: ScreencastFrame) => void) | null = null;
private screencastFrameHandler: ((params: any) => void) | null = null;
// Video recording (Playwright native)
private recordingContext: BrowserContext | null = null;
private recordingPage: Page | null = null;
private recordingOutputPath: string = '';
private recordingTempDir: string = '';
/**
* Check if browser is launched
*/
isLaunched(): boolean {
return this.browser !== null;
return this.browser !== null || this.isPersistentContext;
}
/**
@@ -439,7 +484,7 @@ export class BrowserManager {
}
/**
* Set extra HTTP headers
* Set extra HTTP headers (global - all requests)
*/
async setExtraHeaders(headers: Record<string, string>): Promise<void> {
const context = this.contexts[0];
@@ -448,6 +493,76 @@ export class BrowserManager {
}
}
/**
* Set scoped HTTP headers (only for requests matching the origin)
* Uses route interception to add headers only to matching requests
*/
async setScopedHeaders(origin: string, headers: Record<string, string>): Promise<void> {
const page = this.getPage();
// Build URL pattern from origin (e.g., "api.example.com" -> "**://api.example.com/**")
// Handle both full URLs and just hostnames
let urlPattern: string;
try {
const url = new URL(origin.startsWith('http') ? origin : `https://${origin}`);
// Match any protocol, the host, and any path
urlPattern = `**://${url.host}/**`;
} catch {
// If parsing fails, treat as hostname pattern
urlPattern = `**://${origin}/**`;
}
// Remove existing route for this origin if any
const existingHandler = this.scopedHeaderRoutes.get(urlPattern);
if (existingHandler) {
await page.unroute(urlPattern, existingHandler);
}
// Create handler that adds headers to matching requests
const handler = async (route: Route) => {
const requestHeaders = route.request().headers();
await route.continue({
headers: {
...requestHeaders,
...headers,
},
});
};
// Store and register the route
this.scopedHeaderRoutes.set(urlPattern, handler);
await page.route(urlPattern, handler);
}
/**
* Clear scoped headers for an origin (or all if no origin specified)
*/
async clearScopedHeaders(origin?: string): Promise<void> {
const page = this.getPage();
if (origin) {
let urlPattern: string;
try {
const url = new URL(origin.startsWith('http') ? origin : `https://${origin}`);
urlPattern = `**://${url.host}/**`;
} catch {
urlPattern = `**://${origin}/**`;
}
const handler = this.scopedHeaderRoutes.get(urlPattern);
if (handler) {
await page.unroute(urlPattern, handler);
this.scopedHeaderRoutes.delete(urlPattern);
}
} else {
// Clear all scoped header routes
for (const [pattern, handler] of this.scopedHeaderRoutes) {
await page.unroute(pattern, handler);
}
this.scopedHeaderRoutes.clear();
}
}
/**
* Start tracing
*/
@@ -502,47 +617,158 @@ export class BrowserManager {
return this.browser;
}
/**
* Check if an existing CDP connection is still alive
* by verifying we can access browser contexts and that at least one has pages
*/
private isCdpConnectionAlive(): boolean {
if (!this.browser) return false;
try {
const contexts = this.browser.contexts();
if (contexts.length === 0) return false;
return contexts.some((context) => context.pages().length > 0);
} catch {
return false;
}
}
/**
* Check if CDP connection needs to be re-established
*/
private needsCdpReconnect(cdpPort: number): boolean {
if (!this.browser?.isConnected()) return true;
if (this.cdpPort !== cdpPort) return true;
if (!this.isCdpConnectionAlive()) return true;
return false;
}
/**
* Launch the browser with the specified options
* If already launched, this is a no-op (browser stays open)
*/
async launch(options: LaunchCommand): Promise<void> {
// If already launched, don't relaunch
if (this.browser) {
const cdpPort = options.cdpPort;
const hasExtensions = !!options.extensions?.length;
if (hasExtensions && cdpPort) {
throw new Error('Extensions cannot be used with CDP connection');
}
if (this.isLaunched()) {
const needsRelaunch =
(!cdpPort && this.cdpPort !== null) || (!!cdpPort && this.needsCdpReconnect(cdpPort));
if (needsRelaunch) {
await this.close();
} else {
return;
}
}
if (cdpPort) {
await this.connectViaCDP(cdpPort);
return;
}
// Select browser type
const browserType = options.browser ?? 'chromium';
if (hasExtensions && browserType !== 'chromium') {
throw new Error('Extensions are only supported in Chromium');
}
const launcher =
browserType === 'firefox' ? firefox : browserType === 'webkit' ? webkit : chromium;
const viewport = options.viewport ?? { width: 1280, height: 720 };
// Launch browser
this.browser = await launcher.launch({
headless: options.headless ?? true,
});
// Create context with viewport
const context = await this.browser.newContext({
viewport: options.viewport ?? { width: 1280, height: 720 },
});
// Set default timeout to 10 seconds (Playwright default is 30s)
context.setDefaultTimeout(10000);
let context: BrowserContext;
if (hasExtensions) {
const extPaths = options.extensions!.join(',');
const session = process.env.AGENT_BROWSER_SESSION || 'default';
context = await launcher.launchPersistentContext(
path.join(os.tmpdir(), `agent-browser-ext-${session}`),
{
headless: false,
executablePath: options.executablePath,
args: [`--disable-extensions-except=${extPaths}`, `--load-extension=${extPaths}`],
viewport,
extraHTTPHeaders: options.headers,
...(options.proxy && { proxy: options.proxy }),
}
);
this.isPersistentContext = true;
} else {
this.browser = await launcher.launch({
headless: options.headless ?? true,
executablePath: options.executablePath,
});
this.cdpPort = null;
context = await this.browser.newContext({
viewport,
extraHTTPHeaders: options.headers,
...(options.proxy && { proxy: options.proxy }),
});
}
context.setDefaultTimeout(60000);
this.contexts.push(context);
// Create initial page
const page = await context.newPage();
const page = context.pages()[0] ?? (await context.newPage());
this.pages.push(page);
this.activePageIndex = 0;
// Automatically start console and error tracking
this.setupPageTracking(page);
}
/**
* Set up console and error tracking for a page
* Connect to a running browser via CDP (Chrome DevTools Protocol)
*/
private async connectViaCDP(cdpPort: number | undefined): Promise<void> {
if (!cdpPort) {
throw new Error('cdpPort is required for CDP connection');
}
const browser = await chromium.connectOverCDP(`http://localhost:${cdpPort}`).catch(() => {
throw new Error(
`Failed to connect via CDP on port ${cdpPort}. ` +
`Make sure the app is running with --remote-debugging-port=${cdpPort}`
);
});
// Validate and set up state, cleaning up browser connection if anything fails
try {
const contexts = browser.contexts();
if (contexts.length === 0) {
throw new Error('No browser context found. Make sure the app has an open window.');
}
// Filter out pages with empty URLs, which can cause Playwright to hang
const allPages = contexts.flatMap((context) => context.pages()).filter((page) => page.url());
if (allPages.length === 0) {
throw new Error('No page found. Make sure the app has loaded content.');
}
// All validation passed - commit state
this.browser = browser;
this.cdpPort = cdpPort;
for (const context of contexts) {
this.contexts.push(context);
this.setupContextTracking(context);
}
for (const page of allPages) {
this.pages.push(page);
this.setupPageTracking(page);
}
this.activePageIndex = 0;
} catch (error) {
// Clean up browser connection if validation or setup failed
await browser.close().catch(() => {});
throw error;
}
}
/**
* Set up console, error, and close tracking for a page
*/
private setupPageTracking(page: Page): void {
page.on('console', (msg) => {
@@ -559,6 +785,26 @@ export class BrowserManager {
timestamp: Date.now(),
});
});
page.on('close', () => {
const index = this.pages.indexOf(page);
if (index !== -1) {
this.pages.splice(index, 1);
if (this.activePageIndex >= this.pages.length) {
this.activePageIndex = Math.max(0, this.pages.length - 1);
}
}
});
}
/**
* Set up tracking for new pages in a context (for CDP connections)
*/
private setupContextTracking(context: BrowserContext): void {
context.on('page', (page) => {
this.pages.push(page);
this.setupPageTracking(page);
});
}
/**
@@ -569,6 +815,9 @@ export class BrowserManager {
throw new Error('Browser not launched');
}
// Invalidate CDP session since we're switching to a new page
await this.invalidateCDPSession();
const context = this.contexts[0]; // Use first context for tabs
const page = await context.newPage();
this.pages.push(page);
@@ -594,7 +843,7 @@ export class BrowserManager {
const context = await this.browser.newContext({
viewport: viewport ?? { width: 1280, height: 720 },
});
context.setDefaultTimeout(10000);
context.setDefaultTimeout(60000);
this.contexts.push(context);
const page = await context.newPage();
@@ -607,14 +856,36 @@ export class BrowserManager {
return { index: this.activePageIndex, total: this.pages.length };
}
/**
* Invalidate the current CDP session (must be called before switching pages)
* This ensures screencast and input injection work correctly after tab switch
*/
private async invalidateCDPSession(): Promise<void> {
// Stop screencast if active (it's tied to the current page's CDP session)
if (this.screencastActive) {
await this.stopScreencast();
}
// Detach and clear the CDP session
if (this.cdpSession) {
await this.cdpSession.detach().catch(() => {});
this.cdpSession = null;
}
}
/**
* Switch to a specific tab/page by index
*/
switchTo(index: number): { index: number; url: string; title: string } {
async switchTo(index: number): Promise<{ index: number; url: string; title: string }> {
if (index < 0 || index >= this.pages.length) {
throw new Error(`Invalid tab index: ${index}. Available: 0-${this.pages.length - 1}`);
}
// Invalidate CDP session before switching (it's page-specific)
if (index !== this.activePageIndex) {
await this.invalidateCDPSession();
}
this.activePageIndex = index;
const page = this.pages[index];
@@ -639,6 +910,11 @@ export class BrowserManager {
throw new Error('Cannot close the last tab. Use "close" to close the browser.');
}
// If closing the active tab, invalidate CDP session first
if (targetIndex === this.activePageIndex) {
await this.invalidateCDPSession();
}
const page = this.pages[targetIndex];
await page.close();
this.pages.splice(targetIndex, 1);
@@ -668,27 +944,448 @@ export class BrowserManager {
return tabs;
}
/**
* Get or create a CDP session for the current page
* Only works with Chromium-based browsers
*/
async getCDPSession(): Promise<CDPSession> {
if (this.cdpSession) {
return this.cdpSession;
}
const page = this.getPage();
const context = page.context();
// Create a new CDP session attached to the page
this.cdpSession = await context.newCDPSession(page);
return this.cdpSession;
}
/**
* Check if screencast is currently active
*/
isScreencasting(): boolean {
return this.screencastActive;
}
/**
* Start screencast - streams viewport frames via CDP
* @param callback Function called for each frame
* @param options Screencast options
*/
async startScreencast(
callback: (frame: ScreencastFrame) => void,
options?: ScreencastOptions
): Promise<void> {
if (this.screencastActive) {
throw new Error('Screencast already active');
}
const cdp = await this.getCDPSession();
this.frameCallback = callback;
this.screencastActive = true;
// Create and store the frame handler so we can remove it later
this.screencastFrameHandler = async (params: any) => {
const frame: ScreencastFrame = {
data: params.data,
metadata: params.metadata,
sessionId: params.sessionId,
};
// Acknowledge the frame to receive the next one
await cdp.send('Page.screencastFrameAck', { sessionId: params.sessionId });
// Call the callback with the frame
if (this.frameCallback) {
this.frameCallback(frame);
}
};
// Listen for screencast frames
cdp.on('Page.screencastFrame', this.screencastFrameHandler);
// Start the screencast
await cdp.send('Page.startScreencast', {
format: options?.format ?? 'jpeg',
quality: options?.quality ?? 80,
maxWidth: options?.maxWidth ?? 1280,
maxHeight: options?.maxHeight ?? 720,
everyNthFrame: options?.everyNthFrame ?? 1,
});
}
/**
* Stop screencast
*/
async stopScreencast(): Promise<void> {
if (!this.screencastActive) {
return;
}
try {
const cdp = await this.getCDPSession();
await cdp.send('Page.stopScreencast');
// Remove the event listener to prevent accumulation
if (this.screencastFrameHandler) {
cdp.off('Page.screencastFrame', this.screencastFrameHandler);
}
} catch {
// Ignore errors when stopping
}
this.screencastActive = false;
this.frameCallback = null;
this.screencastFrameHandler = null;
}
/**
* Inject a mouse event via CDP
*/
async injectMouseEvent(params: {
type: 'mousePressed' | 'mouseReleased' | 'mouseMoved' | 'mouseWheel';
x: number;
y: number;
button?: 'left' | 'right' | 'middle' | 'none';
clickCount?: number;
deltaX?: number;
deltaY?: number;
modifiers?: number; // 1=Alt, 2=Ctrl, 4=Meta, 8=Shift
}): Promise<void> {
const cdp = await this.getCDPSession();
const cdpButton =
params.button === 'left'
? 'left'
: params.button === 'right'
? 'right'
: params.button === 'middle'
? 'middle'
: 'none';
await cdp.send('Input.dispatchMouseEvent', {
type: params.type,
x: params.x,
y: params.y,
button: cdpButton,
clickCount: params.clickCount ?? 1,
deltaX: params.deltaX ?? 0,
deltaY: params.deltaY ?? 0,
modifiers: params.modifiers ?? 0,
});
}
/**
* Inject a keyboard event via CDP
*/
async injectKeyboardEvent(params: {
type: 'keyDown' | 'keyUp' | 'char';
key?: string;
code?: string;
text?: string;
modifiers?: number; // 1=Alt, 2=Ctrl, 4=Meta, 8=Shift
}): Promise<void> {
const cdp = await this.getCDPSession();
await cdp.send('Input.dispatchKeyEvent', {
type: params.type,
key: params.key,
code: params.code,
text: params.text,
modifiers: params.modifiers ?? 0,
});
}
/**
* Inject touch event via CDP (for mobile emulation)
*/
async injectTouchEvent(params: {
type: 'touchStart' | 'touchEnd' | 'touchMove' | 'touchCancel';
touchPoints: Array<{ x: number; y: number; id?: number }>;
modifiers?: number;
}): Promise<void> {
const cdp = await this.getCDPSession();
await cdp.send('Input.dispatchTouchEvent', {
type: params.type,
touchPoints: params.touchPoints.map((tp, i) => ({
x: tp.x,
y: tp.y,
id: tp.id ?? i,
})),
modifiers: params.modifiers ?? 0,
});
}
/**
* Check if video recording is currently active
*/
isRecording(): boolean {
return this.recordingContext !== null;
}
/**
* Start recording to a video file using Playwright's native video recording.
* Creates a fresh browser context with video recording enabled.
* Automatically captures current URL and transfers cookies/storage if no URL provided.
*
* @param outputPath - Path to the output video file (will be .webm)
* @param url - Optional URL to navigate to (defaults to current page URL)
*/
async startRecording(outputPath: string, url?: string): Promise<void> {
if (this.recordingContext) {
throw new Error(
"Recording already in progress. Run 'record stop' first, or use 'record restart' to stop and start a new recording."
);
}
if (!this.browser) {
throw new Error('Browser not launched. Call launch first.');
}
// Check if output file already exists
if (existsSync(outputPath)) {
throw new Error(`Output file already exists: ${outputPath}`);
}
// Validate output path is .webm (Playwright native format)
if (!outputPath.endsWith('.webm')) {
throw new Error(
'Playwright native recording only supports WebM format. Please use a .webm extension.'
);
}
// Auto-capture current URL if none provided
const currentPage = this.pages.length > 0 ? this.pages[this.activePageIndex] : null;
const currentContext = this.contexts.length > 0 ? this.contexts[0] : null;
if (!url && currentPage) {
const currentUrl = currentPage.url();
if (currentUrl && currentUrl !== 'about:blank') {
url = currentUrl;
}
}
// Capture state from current context (cookies + storage)
let storageState:
| {
cookies: Array<{
name: string;
value: string;
domain: string;
path: string;
expires: number;
httpOnly: boolean;
secure: boolean;
sameSite: 'Strict' | 'Lax' | 'None';
}>;
origins: Array<{
origin: string;
localStorage: Array<{ name: string; value: string }>;
}>;
}
| undefined;
if (currentContext) {
try {
storageState = await currentContext.storageState();
} catch {
// Ignore errors - context might be closed or invalid
}
}
// Create a temp directory for video recording
const session = process.env.AGENT_BROWSER_SESSION || 'default';
this.recordingTempDir = path.join(
os.tmpdir(),
`agent-browser-recording-${session}-${Date.now()}`
);
mkdirSync(this.recordingTempDir, { recursive: true });
this.recordingOutputPath = outputPath;
// Create a new context with video recording enabled and restored state
const viewport = { width: 1280, height: 720 };
this.recordingContext = await this.browser.newContext({
viewport,
recordVideo: {
dir: this.recordingTempDir,
size: viewport,
},
storageState,
});
this.recordingContext.setDefaultTimeout(10000);
// Create a page in the recording context
this.recordingPage = await this.recordingContext.newPage();
// Add the recording context and page to our managed lists
this.contexts.push(this.recordingContext);
this.pages.push(this.recordingPage);
this.activePageIndex = this.pages.length - 1;
// Set up page tracking
this.setupPageTracking(this.recordingPage);
// Invalidate CDP session since we switched pages
await this.invalidateCDPSession();
// Navigate to URL if provided or captured
if (url) {
await this.recordingPage.goto(url, { waitUntil: 'load' });
}
}
/**
* Stop recording and save the video file
* @returns Recording result with path
*/
async stopRecording(): Promise<{ path: string; frames: number; error?: string }> {
if (!this.recordingContext || !this.recordingPage) {
return { path: '', frames: 0, error: 'No recording in progress' };
}
const outputPath = this.recordingOutputPath;
try {
// Get the video object before closing the page
const video = this.recordingPage.video();
// Remove recording page/context from our managed lists before closing
const pageIndex = this.pages.indexOf(this.recordingPage);
if (pageIndex !== -1) {
this.pages.splice(pageIndex, 1);
}
const contextIndex = this.contexts.indexOf(this.recordingContext);
if (contextIndex !== -1) {
this.contexts.splice(contextIndex, 1);
}
// Close the page to finalize the video
await this.recordingPage.close();
// Save the video to the desired output path
if (video) {
await video.saveAs(outputPath);
}
// Clean up temp directory
if (this.recordingTempDir) {
rmSync(this.recordingTempDir, { recursive: true, force: true });
}
// Close the recording context
await this.recordingContext.close();
// Reset recording state
this.recordingContext = null;
this.recordingPage = null;
this.recordingOutputPath = '';
this.recordingTempDir = '';
// Adjust active page index
if (this.pages.length > 0) {
this.activePageIndex = Math.min(this.activePageIndex, this.pages.length - 1);
} else {
this.activePageIndex = 0;
}
// Invalidate CDP session since we may have switched pages
await this.invalidateCDPSession();
return { path: outputPath, frames: 0 }; // Playwright doesn't expose frame count
} catch (error) {
// Clean up temp directory on error
if (this.recordingTempDir) {
rmSync(this.recordingTempDir, { recursive: true, force: true });
}
// Reset state on error
this.recordingContext = null;
this.recordingPage = null;
this.recordingOutputPath = '';
this.recordingTempDir = '';
const message = error instanceof Error ? error.message : String(error);
return { path: outputPath, frames: 0, error: message };
}
}
/**
* Restart recording - stops current recording (if any) and starts a new one.
* Convenience method that combines stopRecording and startRecording.
*
* @param outputPath - Path to the output video file (must be .webm)
* @param url - Optional URL to navigate to (defaults to current page URL)
* @returns Result from stopping the previous recording (if any)
*/
async restartRecording(
outputPath: string,
url?: string
): Promise<{ previousPath?: string; stopped: boolean }> {
let previousPath: string | undefined;
let stopped = false;
// Stop current recording if active
if (this.recordingContext) {
const result = await this.stopRecording();
previousPath = result.path;
stopped = true;
}
// Start new recording
await this.startRecording(outputPath, url);
return { previousPath, stopped };
}
/**
* Close the browser and clean up
*/
async close(): Promise<void> {
for (const page of this.pages) {
await page.close().catch(() => {});
// Stop recording if active (saves video)
if (this.recordingContext) {
await this.stopRecording();
}
// Stop screencast if active
if (this.screencastActive) {
await this.stopScreencast();
}
// Clean up CDP session
if (this.cdpSession) {
await this.cdpSession.detach().catch(() => {});
this.cdpSession = null;
}
// CDP: only disconnect, don't close external app's pages
if (this.cdpPort !== null) {
if (this.browser) {
await this.browser.close().catch(() => {});
this.browser = null;
}
} else {
// Regular browser: close everything
for (const page of this.pages) {
await page.close().catch(() => {});
}
for (const context of this.contexts) {
await context.close().catch(() => {});
}
if (this.browser) {
await this.browser.close().catch(() => {});
this.browser = null;
}
}
this.pages = [];
for (const context of this.contexts) {
await context.close().catch(() => {});
}
this.contexts = [];
if (this.browser) {
await this.browser.close().catch(() => {});
this.browser = null;
}
this.cdpPort = null;
this.isPersistentContext = false;
this.activePageIndex = 0;
this.refMap = {};
this.lastSnapshot = '';
this.frameCallback = null;
}
}
+62 -3
View File
@@ -5,6 +5,7 @@ import * as os from 'os';
import { BrowserManager } from './browser.js';
import { parseCommand, serializeResponse, errorResponse } from './protocol.js';
import { executeCommand } from './actions.js';
import { StreamServer } from './stream-server.js';
// Platform detection
const isWindows = process.platform === 'win32';
@@ -12,6 +13,12 @@ const isWindows = process.platform === 'win32';
// Session support - each session gets its own socket/pid
let currentSession = process.env.AGENT_BROWSER_SESSION || 'default';
// Stream server for browser preview
let streamServer: StreamServer | null = null;
// Default stream port (can be overridden with AGENT_BROWSER_STREAM_PORT)
const DEFAULT_STREAM_PORT = 9223;
/**
* Set the current session
*/
@@ -105,8 +112,10 @@ export function getConnectionInfo(
*/
export function cleanupSocket(session?: string): void {
const pidFile = getPidFile(session);
const streamPortFile = getStreamPortFile(session);
try {
if (fs.existsSync(pidFile)) fs.unlinkSync(pidFile);
if (fs.existsSync(streamPortFile)) fs.unlinkSync(streamPortFile);
if (isWindows) {
const portFile = getPortFile(session);
if (fs.existsSync(portFile)) fs.unlinkSync(portFile);
@@ -120,15 +129,40 @@ export function cleanupSocket(session?: string): void {
}
/**
* Start the daemon server
* Get the stream port file path
*/
export async function startDaemon(): Promise<void> {
export function getStreamPortFile(session?: string): string {
const sess = session ?? currentSession;
return path.join(os.tmpdir(), `agent-browser-${sess}.stream`);
}
/**
* Start the daemon server
* @param options.streamPort Port for WebSocket stream server (0 to disable)
*/
export async function startDaemon(options?: { streamPort?: number }): Promise<void> {
// Clean up any stale socket
cleanupSocket();
const browser = new BrowserManager();
let shuttingDown = false;
// Start stream server if port is specified (or use default if env var is set)
const streamPort =
options?.streamPort ??
(process.env.AGENT_BROWSER_STREAM_PORT
? parseInt(process.env.AGENT_BROWSER_STREAM_PORT, 10)
: 0);
if (streamPort > 0) {
streamServer = new StreamServer(browser, streamPort);
await streamServer.start();
// Write stream port to file for clients to discover
const streamPortFile = getStreamPortFile();
fs.writeFileSync(streamPortFile, streamPort.toString());
}
const server = net.createServer((socket) => {
let buffer = '';
@@ -158,7 +192,18 @@ export async function startDaemon(): Promise<void> {
parseResult.command.action !== 'launch' &&
parseResult.command.action !== 'close'
) {
await browser.launch({ id: 'auto', action: 'launch', headless: true });
const extensions = process.env.AGENT_BROWSER_EXTENSIONS
? process.env.AGENT_BROWSER_EXTENSIONS.split(',')
.map((p) => p.trim())
.filter(Boolean)
: undefined;
await browser.launch({
id: 'auto',
action: 'launch',
headless: process.env.AGENT_BROWSER_HEADED !== '1',
executablePath: process.env.AGENT_BROWSER_EXECUTABLE_PATH,
extensions: extensions,
});
}
// Handle close command specially
@@ -222,6 +267,20 @@ export async function startDaemon(): Promise<void> {
const shutdown = async () => {
if (shuttingDown) return;
shuttingDown = true;
// Stop stream server if running
if (streamServer) {
await streamServer.stop();
streamServer = null;
// Clean up stream port file
const streamPortFile = getStreamPortFile();
try {
if (fs.existsSync(streamPortFile)) fs.unlinkSync(streamPortFile);
} catch {
// Ignore cleanup errors
}
}
await browser.close();
server.close();
cleanupSocket();
+427
View File
@@ -15,6 +15,22 @@ describe('parseCommand', () => {
}
});
it('should parse navigate with headers', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'navigate',
url: 'https://example.com',
headers: { Authorization: 'Bearer token' },
})
);
expect(result.success).toBe(true);
if (result.success) {
expect(result.command.action).toBe('navigate');
expect(result.command.headers).toEqual({ Authorization: 'Bearer token' });
}
});
it('should reject navigate without url', () => {
const result = parseCommand(cmd({ id: '1', action: 'navigate' }));
expect(result.success).toBe(false);
@@ -373,6 +389,14 @@ describe('parseCommand', () => {
expect(result.success).toBe(true);
});
it('should parse tab_new with url', () => {
const result = parseCommand(cmd({ id: '1', action: 'tab_new', url: 'https://example.com' }));
expect(result.success).toBe(true);
if (result.success) {
expect((result.command as { url?: string }).url).toBe('https://example.com');
}
});
it('should parse tab_list', () => {
const result = parseCommand(cmd({ id: '1', action: 'tab_list' }));
expect(result.success).toBe(true);
@@ -461,6 +485,24 @@ describe('parseCommand', () => {
expect(result.command.headless).toBe(false);
}
});
it('should parse launch with cdpPort', () => {
const result = parseCommand(cmd({ id: '1', action: 'launch', cdpPort: 9222 }));
expect(result.success).toBe(true);
if (result.success) {
expect(result.command.cdpPort).toBe(9222);
}
});
it('should reject launch with invalid cdpPort', () => {
const result = parseCommand(cmd({ id: '1', action: 'launch', cdpPort: -1 }));
expect(result.success).toBe(false);
});
it('should reject launch with non-numeric cdpPort', () => {
const result = parseCommand(cmd({ id: '1', action: 'launch', cdpPort: 'invalid' }));
expect(result.success).toBe(false);
});
});
describe('mouse actions', () => {
@@ -602,6 +644,391 @@ describe('parseCommand', () => {
});
});
describe('screencast', () => {
it('should parse screencast_start with defaults', () => {
const result = parseCommand(cmd({ id: '1', action: 'screencast_start' }));
expect(result.success).toBe(true);
if (result.success) {
expect(result.command.action).toBe('screencast_start');
}
});
it('should parse screencast_start with all options', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'screencast_start',
format: 'png',
quality: 90,
maxWidth: 1920,
maxHeight: 1080,
everyNthFrame: 2,
})
);
expect(result.success).toBe(true);
if (result.success) {
expect(result.command.format).toBe('png');
expect(result.command.quality).toBe(90);
expect(result.command.maxWidth).toBe(1920);
expect(result.command.maxHeight).toBe(1080);
expect(result.command.everyNthFrame).toBe(2);
}
});
it('should reject screencast_start with invalid format', () => {
const result = parseCommand(cmd({ id: '1', action: 'screencast_start', format: 'gif' }));
expect(result.success).toBe(false);
});
it('should reject screencast_start with quality out of range', () => {
const result = parseCommand(cmd({ id: '1', action: 'screencast_start', quality: 150 }));
expect(result.success).toBe(false);
});
it('should reject screencast_start with negative maxWidth', () => {
const result = parseCommand(cmd({ id: '1', action: 'screencast_start', maxWidth: -100 }));
expect(result.success).toBe(false);
});
it('should parse screencast_stop', () => {
const result = parseCommand(cmd({ id: '1', action: 'screencast_stop' }));
expect(result.success).toBe(true);
if (result.success) {
expect(result.command.action).toBe('screencast_stop');
}
});
});
describe('input injection', () => {
describe('input_mouse', () => {
it('should parse mousePressed event', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_mouse',
type: 'mousePressed',
x: 100,
y: 200,
button: 'left',
})
);
expect(result.success).toBe(true);
if (result.success) {
expect(result.command.action).toBe('input_mouse');
expect(result.command.type).toBe('mousePressed');
expect(result.command.x).toBe(100);
expect(result.command.y).toBe(200);
expect(result.command.button).toBe('left');
}
});
it('should parse mouseReleased event', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_mouse',
type: 'mouseReleased',
x: 100,
y: 200,
})
);
expect(result.success).toBe(true);
});
it('should parse mouseMoved event', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_mouse',
type: 'mouseMoved',
x: 150,
y: 250,
})
);
expect(result.success).toBe(true);
});
it('should parse mouseWheel event with deltas', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_mouse',
type: 'mouseWheel',
x: 100,
y: 200,
deltaX: 0,
deltaY: 100,
})
);
expect(result.success).toBe(true);
if (result.success) {
expect(result.command.deltaX).toBe(0);
expect(result.command.deltaY).toBe(100);
}
});
it('should parse mouse event with modifiers', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_mouse',
type: 'mousePressed',
x: 100,
y: 200,
modifiers: 6, // Ctrl + Meta
})
);
expect(result.success).toBe(true);
if (result.success) {
expect(result.command.modifiers).toBe(6);
}
});
it('should parse mouse event with clickCount', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_mouse',
type: 'mousePressed',
x: 100,
y: 200,
clickCount: 2,
})
);
expect(result.success).toBe(true);
if (result.success) {
expect(result.command.clickCount).toBe(2);
}
});
it('should reject input_mouse with invalid type', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_mouse',
type: 'invalid',
x: 100,
y: 200,
})
);
expect(result.success).toBe(false);
});
it('should reject input_mouse without x coordinate', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_mouse',
type: 'mousePressed',
y: 200,
})
);
expect(result.success).toBe(false);
});
it('should reject input_mouse without y coordinate', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_mouse',
type: 'mousePressed',
x: 100,
})
);
expect(result.success).toBe(false);
});
});
describe('input_keyboard', () => {
it('should parse keyDown event', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_keyboard',
type: 'keyDown',
key: 'Enter',
code: 'Enter',
})
);
expect(result.success).toBe(true);
if (result.success) {
expect(result.command.action).toBe('input_keyboard');
expect(result.command.type).toBe('keyDown');
expect(result.command.key).toBe('Enter');
expect(result.command.code).toBe('Enter');
}
});
it('should parse keyUp event', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_keyboard',
type: 'keyUp',
key: 'a',
})
);
expect(result.success).toBe(true);
});
it('should parse char event with text', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_keyboard',
type: 'char',
text: 'hello',
})
);
expect(result.success).toBe(true);
if (result.success) {
expect(result.command.text).toBe('hello');
}
});
it('should parse keyboard event with modifiers', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_keyboard',
type: 'keyDown',
key: 'c',
modifiers: 2, // Ctrl
})
);
expect(result.success).toBe(true);
if (result.success) {
expect(result.command.modifiers).toBe(2);
}
});
it('should reject input_keyboard with invalid type', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_keyboard',
type: 'invalid',
})
);
expect(result.success).toBe(false);
});
});
describe('input_touch', () => {
it('should parse touchStart event', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_touch',
type: 'touchStart',
touchPoints: [{ x: 100, y: 200 }],
})
);
expect(result.success).toBe(true);
if (result.success) {
expect(result.command.action).toBe('input_touch');
expect(result.command.type).toBe('touchStart');
expect(result.command.touchPoints).toHaveLength(1);
expect(result.command.touchPoints[0].x).toBe(100);
expect(result.command.touchPoints[0].y).toBe(200);
}
});
it('should parse touchEnd event', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_touch',
type: 'touchEnd',
touchPoints: [],
})
);
expect(result.success).toBe(true);
});
it('should parse touchMove event', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_touch',
type: 'touchMove',
touchPoints: [{ x: 150, y: 250 }],
})
);
expect(result.success).toBe(true);
});
it('should parse touchCancel event', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_touch',
type: 'touchCancel',
touchPoints: [],
})
);
expect(result.success).toBe(true);
});
it('should parse multi-touch event', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_touch',
type: 'touchStart',
touchPoints: [
{ x: 100, y: 200, id: 0 },
{ x: 300, y: 400, id: 1 },
],
})
);
expect(result.success).toBe(true);
if (result.success) {
expect(result.command.touchPoints).toHaveLength(2);
}
});
it('should parse touch event with modifiers', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_touch',
type: 'touchStart',
touchPoints: [{ x: 100, y: 200 }],
modifiers: 8, // Shift
})
);
expect(result.success).toBe(true);
if (result.success) {
expect(result.command.modifiers).toBe(8);
}
});
it('should reject input_touch with invalid type', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_touch',
type: 'invalid',
touchPoints: [],
})
);
expect(result.success).toBe(false);
});
it('should reject input_touch without touchPoints', () => {
const result = parseCommand(
cmd({
id: '1',
action: 'input_touch',
type: 'touchStart',
})
);
expect(result.success).toBe(false);
});
});
});
describe('invalid commands', () => {
it('should reject unknown action', () => {
const result = parseCommand(cmd({ id: '1', action: 'unknown' }));
+95 -1
View File
@@ -18,12 +18,25 @@ const launchSchema = baseCommandSchema.extend({
})
.optional(),
browser: z.enum(['chromium', 'firefox', 'webkit']).optional(),
cdpPort: z.number().positive().optional(),
executablePath: z.string().optional(),
extensions: z.array(z.string()).optional(),
headers: z.record(z.string()).optional(),
proxy: z
.object({
server: z.string().min(1),
bypass: z.string().optional(),
username: z.string().optional(),
password: z.string().optional(),
})
.optional(),
});
const navigateSchema = baseCommandSchema.extend({
action: z.literal('navigate'),
url: z.string().min(1),
waitUntil: z.enum(['load', 'domcontentloaded', 'networkidle']).optional(),
headers: z.record(z.string()).optional(),
});
const clickSchema = baseCommandSchema.extend({
@@ -294,6 +307,11 @@ const boundingBoxSchema = baseCommandSchema.extend({
selector: z.string().min(1),
});
const stylesSchema = baseCommandSchema.extend({
action: z.literal('styles'),
selector: z.string().min(1),
});
const videoStartSchema = baseCommandSchema.extend({
action: z.literal('video_start'),
path: z.string().min(1),
@@ -303,6 +321,23 @@ const videoStopSchema = baseCommandSchema.extend({
action: z.literal('video_stop'),
});
// Recording schemas (Playwright native video recording)
const recordingStartSchema = baseCommandSchema.extend({
action: z.literal('recording_start'),
path: z.string().min(1),
url: z.string().min(1).optional(),
});
const recordingStopSchema = baseCommandSchema.extend({
action: z.literal('recording_stop'),
});
const recordingRestartSchema = baseCommandSchema.extend({
action: z.literal('recording_restart'),
path: z.string().min(1),
url: z.string().min(1).optional(),
});
const traceStartSchema = baseCommandSchema.extend({
action: z.literal('trace_start'),
screenshots: z.boolean().optional(),
@@ -584,6 +619,55 @@ const responseBodySchema = baseCommandSchema.extend({
timeout: z.number().positive().optional(),
});
// Screencast schemas for streaming browser viewport
const screencastStartSchema = baseCommandSchema.extend({
action: z.literal('screencast_start'),
format: z.enum(['jpeg', 'png']).optional(),
quality: z.number().min(0).max(100).optional(),
maxWidth: z.number().positive().optional(),
maxHeight: z.number().positive().optional(),
everyNthFrame: z.number().positive().optional(),
});
const screencastStopSchema = baseCommandSchema.extend({
action: z.literal('screencast_stop'),
});
// Input injection schemas for pair browsing
const inputMouseSchema = baseCommandSchema.extend({
action: z.literal('input_mouse'),
type: z.enum(['mousePressed', 'mouseReleased', 'mouseMoved', 'mouseWheel']),
x: z.number(),
y: z.number(),
button: z.enum(['left', 'right', 'middle', 'none']).optional(),
clickCount: z.number().positive().optional(),
deltaX: z.number().optional(),
deltaY: z.number().optional(),
modifiers: z.number().optional(),
});
const inputKeyboardSchema = baseCommandSchema.extend({
action: z.literal('input_keyboard'),
type: z.enum(['keyDown', 'keyUp', 'char']),
key: z.string().optional(),
code: z.string().optional(),
text: z.string().optional(),
modifiers: z.number().optional(),
});
const inputTouchSchema = baseCommandSchema.extend({
action: z.literal('input_touch'),
type: z.enum(['touchStart', 'touchEnd', 'touchMove', 'touchCancel']),
touchPoints: z.array(
z.object({
x: z.number(),
y: z.number(),
id: z.number().optional(),
})
),
modifiers: z.number().optional(),
});
const pressSchema = baseCommandSchema.extend({
action: z.literal('press'),
key: z.string().min(1),
@@ -592,7 +676,7 @@ const pressSchema = baseCommandSchema.extend({
const screenshotSchema = baseCommandSchema.extend({
action: z.literal('screenshot'),
path: z.string().optional(),
path: z.string().nullable().optional(),
fullPage: z.boolean().optional(),
selector: z.string().min(1).optional(),
format: z.enum(['png', 'jpeg']).optional(),
@@ -652,6 +736,7 @@ const closeSchema = baseCommandSchema.extend({
// Tab/Window schemas
const tabNewSchema = baseCommandSchema.extend({
action: z.literal('tab_new'),
url: z.string().min(1).optional(),
});
const tabListSchema = baseCommandSchema.extend({
@@ -741,8 +826,12 @@ const commandSchema = z.discriminatedUnion('action', [
isCheckedSchema,
countSchema,
boundingBoxSchema,
stylesSchema,
videoStartSchema,
videoStopSchema,
recordingStartSchema,
recordingStopSchema,
recordingRestartSchema,
traceStartSchema,
traceStopSchema,
harStartSchema,
@@ -794,6 +883,11 @@ const commandSchema = z.discriminatedUnion('action', [
multiSelectSchema,
waitForDownloadSchema,
responseBodySchema,
screencastStartSchema,
screencastStopSchema,
inputMouseSchema,
inputKeyboardSchema,
inputTouchSchema,
]);
// Parse result type
+364
View File
@@ -0,0 +1,364 @@
import { WebSocketServer, WebSocket } from 'ws';
import type { BrowserManager, ScreencastFrame } from './browser.js';
import { setScreencastFrameCallback } from './actions.js';
// Message types for WebSocket communication
export interface FrameMessage {
type: 'frame';
data: string; // base64 encoded image
metadata: {
offsetTop: number;
pageScaleFactor: number;
deviceWidth: number;
deviceHeight: number;
scrollOffsetX: number;
scrollOffsetY: number;
timestamp?: number;
};
}
export interface InputMouseMessage {
type: 'input_mouse';
eventType: 'mousePressed' | 'mouseReleased' | 'mouseMoved' | 'mouseWheel';
x: number;
y: number;
button?: 'left' | 'right' | 'middle' | 'none';
clickCount?: number;
deltaX?: number;
deltaY?: number;
modifiers?: number;
}
export interface InputKeyboardMessage {
type: 'input_keyboard';
eventType: 'keyDown' | 'keyUp' | 'char';
key?: string;
code?: string;
text?: string;
modifiers?: number;
}
export interface InputTouchMessage {
type: 'input_touch';
eventType: 'touchStart' | 'touchEnd' | 'touchMove' | 'touchCancel';
touchPoints: Array<{ x: number; y: number; id?: number }>;
modifiers?: number;
}
export interface StatusMessage {
type: 'status';
connected: boolean;
screencasting: boolean;
viewportWidth?: number;
viewportHeight?: number;
}
export interface ErrorMessage {
type: 'error';
message: string;
}
export type StreamMessage =
| FrameMessage
| InputMouseMessage
| InputKeyboardMessage
| InputTouchMessage
| StatusMessage
| ErrorMessage;
/**
* WebSocket server for streaming browser viewport and receiving input
*/
export class StreamServer {
private wss: WebSocketServer | null = null;
private clients: Set<WebSocket> = new Set();
private browser: BrowserManager;
private port: number;
private isScreencasting: boolean = false;
constructor(browser: BrowserManager, port: number = 9223) {
this.browser = browser;
this.port = port;
}
/**
* Start the WebSocket server
*/
start(): Promise<void> {
return new Promise((resolve, reject) => {
try {
this.wss = new WebSocketServer({ port: this.port });
this.wss.on('connection', (ws) => {
this.handleConnection(ws);
});
this.wss.on('error', (error) => {
console.error('[StreamServer] WebSocket error:', error);
reject(error);
});
this.wss.on('listening', () => {
console.log(`[StreamServer] Listening on port ${this.port}`);
// Set up the screencast frame callback
setScreencastFrameCallback((frame) => {
this.broadcastFrame(frame);
});
resolve();
});
} catch (error) {
reject(error);
}
});
}
/**
* Stop the WebSocket server
*/
async stop(): Promise<void> {
// Stop screencasting
if (this.isScreencasting) {
await this.stopScreencast();
}
// Clear the callback
setScreencastFrameCallback(null);
// Close all clients
for (const client of this.clients) {
client.close();
}
this.clients.clear();
// Close the server
if (this.wss) {
return new Promise((resolve) => {
this.wss!.close(() => {
this.wss = null;
resolve();
});
});
}
}
/**
* Handle a new WebSocket connection
*/
private handleConnection(ws: WebSocket): void {
console.log('[StreamServer] Client connected');
this.clients.add(ws);
// Send initial status
this.sendStatus(ws);
// Start screencasting if this is the first client
if (this.clients.size === 1 && !this.isScreencasting) {
this.startScreencast().catch((error) => {
console.error('[StreamServer] Failed to start screencast:', error);
this.sendError(ws, error.message);
});
}
// Handle messages from client
ws.on('message', (data) => {
try {
const message = JSON.parse(data.toString()) as StreamMessage;
this.handleMessage(message, ws);
} catch (error) {
console.error('[StreamServer] Failed to parse message:', error);
}
});
// Handle client disconnect
ws.on('close', () => {
console.log('[StreamServer] Client disconnected');
this.clients.delete(ws);
// Stop screencasting if no more clients
if (this.clients.size === 0 && this.isScreencasting) {
this.stopScreencast().catch((error) => {
console.error('[StreamServer] Failed to stop screencast:', error);
});
}
});
ws.on('error', (error) => {
console.error('[StreamServer] Client error:', error);
this.clients.delete(ws);
});
}
/**
* Handle incoming messages from clients
*/
private async handleMessage(message: StreamMessage, ws: WebSocket): Promise<void> {
try {
switch (message.type) {
case 'input_mouse':
await this.browser.injectMouseEvent({
type: message.eventType,
x: message.x,
y: message.y,
button: message.button,
clickCount: message.clickCount,
deltaX: message.deltaX,
deltaY: message.deltaY,
modifiers: message.modifiers,
});
break;
case 'input_keyboard':
await this.browser.injectKeyboardEvent({
type: message.eventType,
key: message.key,
code: message.code,
text: message.text,
modifiers: message.modifiers,
});
break;
case 'input_touch':
await this.browser.injectTouchEvent({
type: message.eventType,
touchPoints: message.touchPoints,
modifiers: message.modifiers,
});
break;
case 'status':
// Client is requesting status
this.sendStatus(ws);
break;
}
} catch (error) {
const errorMessage = error instanceof Error ? error.message : String(error);
this.sendError(ws, errorMessage);
}
}
/**
* Broadcast a frame to all connected clients
*/
private broadcastFrame(frame: ScreencastFrame): void {
const message: FrameMessage = {
type: 'frame',
data: frame.data,
metadata: frame.metadata,
};
const payload = JSON.stringify(message);
for (const client of this.clients) {
if (client.readyState === WebSocket.OPEN) {
client.send(payload);
}
}
}
/**
* Send status to a client
*/
private sendStatus(ws: WebSocket): void {
let viewportWidth: number | undefined;
let viewportHeight: number | undefined;
try {
const page = this.browser.getPage();
const viewport = page.viewportSize();
viewportWidth = viewport?.width;
viewportHeight = viewport?.height;
} catch {
// Browser not launched yet
}
const message: StatusMessage = {
type: 'status',
connected: true,
screencasting: this.isScreencasting,
viewportWidth,
viewportHeight,
};
if (ws.readyState === WebSocket.OPEN) {
ws.send(JSON.stringify(message));
}
}
/**
* Send an error to a client
*/
private sendError(ws: WebSocket, errorMessage: string): void {
const message: ErrorMessage = {
type: 'error',
message: errorMessage,
};
if (ws.readyState === WebSocket.OPEN) {
ws.send(JSON.stringify(message));
}
}
/**
* Start screencasting
*/
private async startScreencast(): Promise<void> {
// Set flag immediately to prevent race conditions with concurrent calls
if (this.isScreencasting) return;
this.isScreencasting = true;
try {
// Check if browser is launched
if (!this.browser.isLaunched()) {
throw new Error('Browser not launched');
}
await this.browser.startScreencast((frame) => this.broadcastFrame(frame), {
format: 'jpeg',
quality: 80,
maxWidth: 1280,
maxHeight: 720,
everyNthFrame: 1,
});
// Notify all clients
for (const client of this.clients) {
this.sendStatus(client);
}
} catch (error) {
// Reset flag on failure so caller can retry
this.isScreencasting = false;
throw error;
}
}
/**
* Stop screencasting
*/
private async stopScreencast(): Promise<void> {
if (!this.isScreencasting) return;
await this.browser.stopScreencast();
this.isScreencasting = false;
// Notify all clients
for (const client of this.clients) {
this.sendStatus(client);
}
}
/**
* Get the port the server is running on
*/
getPort(): number {
return this.port;
}
/**
* Get the number of connected clients
*/
getClientCount(): number {
return this.clients.size;
}
}
+143 -2
View File
@@ -12,12 +12,23 @@ export interface LaunchCommand extends BaseCommand {
headless?: boolean;
viewport?: { width: number; height: number };
browser?: 'chromium' | 'firefox' | 'webkit';
headers?: Record<string, string>;
executablePath?: string;
cdpPort?: number;
extensions?: string[];
proxy?: {
server: string;
bypass?: string;
username?: string;
password?: string;
};
}
export interface NavigateCommand extends BaseCommand {
action: 'navigate';
url: string;
waitUntil?: 'load' | 'domcontentloaded' | 'networkidle';
headers?: Record<string, string>;
}
export interface ClickCommand extends BaseCommand {
@@ -305,6 +316,12 @@ export interface BoundingBoxCommand extends BaseCommand {
selector: string;
}
// Computed styles
export interface StylesCommand extends BaseCommand {
action: 'styles';
selector: string;
}
// More semantic locators
export interface GetByAltTextCommand extends BaseCommand {
action: 'getbyalttext';
@@ -454,7 +471,50 @@ export interface ResponseBodyCommand extends BaseCommand {
timeout?: number;
}
// Video recording
// Screencast commands for streaming browser viewport
export interface ScreencastStartCommand extends BaseCommand {
action: 'screencast_start';
format?: 'jpeg' | 'png';
quality?: number; // 0-100, jpeg only
maxWidth?: number;
maxHeight?: number;
everyNthFrame?: number;
}
export interface ScreencastStopCommand extends BaseCommand {
action: 'screencast_stop';
}
// Input injection commands for pair browsing
export interface InputMouseCommand extends BaseCommand {
action: 'input_mouse';
type: 'mousePressed' | 'mouseReleased' | 'mouseMoved' | 'mouseWheel';
x: number;
y: number;
button?: 'left' | 'right' | 'middle' | 'none';
clickCount?: number;
deltaX?: number;
deltaY?: number;
modifiers?: number;
}
export interface InputKeyboardCommand extends BaseCommand {
action: 'input_keyboard';
type: 'keyDown' | 'keyUp' | 'char';
key?: string;
code?: string;
text?: string;
modifiers?: number;
}
export interface InputTouchCommand extends BaseCommand {
action: 'input_touch';
type: 'touchStart' | 'touchEnd' | 'touchMove' | 'touchCancel';
touchPoints: Array<{ x: number; y: number; id?: number }>;
modifiers?: number;
}
// Video recording (Playwright native - requires launch-time setup)
export interface VideoStartCommand extends BaseCommand {
action: 'video_start';
path: string;
@@ -464,6 +524,23 @@ export interface VideoStopCommand extends BaseCommand {
action: 'video_stop';
}
// Screen recording (Playwright native - creates fresh recording context)
export interface RecordingStartCommand extends BaseCommand {
action: 'recording_start';
path: string;
url?: string;
}
export interface RecordingStopCommand extends BaseCommand {
action: 'recording_stop';
}
export interface RecordingRestartCommand extends BaseCommand {
action: 'recording_restart';
path: string;
url?: string;
}
// Tracing
export interface TraceStartCommand extends BaseCommand {
action: 'trace_start';
@@ -701,6 +778,7 @@ export interface CloseCommand extends BaseCommand {
// Tab/Window commands
export interface TabNewCommand extends BaseCommand {
action: 'tab_new';
url?: string;
}
export interface TabListCommand extends BaseCommand {
@@ -785,8 +863,12 @@ export type Command =
| IsCheckedCommand
| CountCommand
| BoundingBoxCommand
| StylesCommand
| VideoStartCommand
| VideoStopCommand
| RecordingStartCommand
| RecordingStopCommand
| RecordingRestartCommand
| TraceStartCommand
| TraceStopCommand
| HarStartCommand
@@ -837,7 +919,12 @@ export type Command =
| InsertTextCommand
| MultiSelectCommand
| WaitForDownloadCommand
| ResponseBodyCommand;
| ResponseBodyCommand
| ScreencastStartCommand
| ScreencastStopCommand
| InputMouseCommand
| InputKeyboardCommand
| InputTouchCommand;
// Response types
export interface SuccessResponse<T = unknown> {
@@ -905,6 +992,60 @@ export interface TabCloseData {
remaining: number;
}
export interface ScreencastStartData {
started: boolean;
format: string;
quality: number;
}
export interface ScreencastStopData {
stopped: boolean;
}
export interface RecordingStartData {
started: boolean;
path: string;
}
export interface RecordingStopData {
path: string;
frames: number;
error?: string;
}
export interface RecordingRestartData {
started: boolean;
path: string;
previousPath?: string;
stopped: boolean;
}
export interface InputEventData {
injected: boolean;
}
// Element styles data
export interface ElementStyleInfo {
tag: string;
text: string | null;
box: { x: number; y: number; width: number; height: number };
styles: {
fontSize: string;
fontWeight: string;
fontFamily: string;
color: string;
backgroundColor: string;
borderRadius: string;
border: string | null;
boxShadow: string | null;
padding: string;
};
}
export interface StylesData {
elements: ElementStyleInfo[];
}
// Browser state
export interface BrowserState {
browser: Browser | null;
+85
View File
@@ -0,0 +1,85 @@
/**
* Integration test for @sparticuz/chromium compatibility
* This tests the executablePath option with a serverless-optimized Chromium build
*
* Note: @sparticuz/chromium only works on Linux (designed for AWS Lambda).
* This test will skip on non-Linux platforms.
*/
import { describe, it, expect, afterAll } from 'vitest';
import { BrowserManager } from '../src/browser.js';
import * as os from 'os';
const isLinux = os.platform() === 'linux';
// Only run if @sparticuz/chromium is available AND we're on Linux
const canRunTest = await (async () => {
if (!isLinux) {
console.log('Skipping @sparticuz/chromium test: only runs on Linux');
return false;
}
try {
await import('@sparticuz/chromium');
return true;
} catch {
console.log('Skipping @sparticuz/chromium test: package not installed');
return false;
}
})();
describe.skipIf(!canRunTest)('Serverless Chromium Integration', () => {
let browser: BrowserManager;
let chromiumPath: string;
it('should get executable path from @sparticuz/chromium', async () => {
const chromium = await import('@sparticuz/chromium');
chromiumPath = await chromium.default.executablePath();
expect(chromiumPath).toBeTruthy();
expect(typeof chromiumPath).toBe('string');
console.log('Chromium executable path:', chromiumPath);
});
it('should launch browser with custom executablePath', async () => {
const chromium = await import('@sparticuz/chromium');
chromiumPath = await chromium.default.executablePath();
browser = new BrowserManager();
await browser.launch({
headless: true,
executablePath: chromiumPath,
});
expect(browser.isLaunched()).toBe(true);
});
it('should navigate to a page', async () => {
const page = browser.getPage();
await page.goto('https://example.com');
expect(page.url()).toBe('https://example.com/');
});
it('should get page title', async () => {
const page = browser.getPage();
const title = await page.title();
expect(title).toBe('Example Domain');
});
it('should take snapshot with refs', async () => {
const { tree, refs } = await browser.getSnapshot();
expect(tree).toContain('Example Domain');
expect(typeof refs).toBe('object');
expect(Object.keys(refs).length).toBeGreaterThan(0);
});
it('should take screenshot', async () => {
const page = browser.getPage();
const buffer = await page.screenshot();
expect(buffer).toBeInstanceOf(Buffer);
expect(buffer.length).toBeGreaterThan(0);
});
afterAll(async () => {
if (browser?.isLaunched()) {
await browser.close();
}
});
});
+1 -1
View File
@@ -3,7 +3,7 @@ import { defineConfig } from 'vitest/config';
export default defineConfig({
test: {
globals: true,
include: ['src/**/*.test.ts'],
include: ['src/**/*.test.ts', 'test/**/*.test.ts'],
testTimeout: 30000,
},
});