v0.5.0

When clicking an element that is blocked by a cookie banner or modal overlay,
the error message incorrectly showed "Element not found or not visible" even
though the element was found and visible.

The issue was in toAIFriendlyError(): the check for "Timeout" was evaluated
before "intercepts pointer events", causing the wrong error message to be
returned.

Changes:
- Reorder error detection to check "intercepts pointer events" before "Timeout"
- Improve error message to suggest dismissing modals/cookie banners
- Export toAIFriendlyError for testing
- Add focused tests for overlay blocking behavior

Before:
  Element "@e4" not found or not visible. Run 'snapshot' to see current page elements.

After:
  Element "@e4" is blocked by another element (likely a modal or overlay).
  Try dismissing any modals/cookie banners first.

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>

.gitignore

@@ -42,3 +42,9 @@ yarn.lock
 
 # opensrc - source code for packages
 opensrc/
+
+# Docs site
+docs/node_modules/
+docs/.next/
+docs/out/
+docs/package-lock.json

README.md

@@ -18,6 +18,8 @@ git clone https://github.com/vercel-labs/agent-browser
 cd agent-browser
 pnpm install
 pnpm build
+pnpm build:native   # Requires Rust (https://rustup.rs)
+pnpm link --global  # Makes agent-browser available globally
 agent-browser install
 ```
 
@@ -293,12 +295,14 @@ agent-browser snapshot -i -c -d 5         # Combine options
 | Option | Description |
 |--------|-------------|
 | `--session <name>` | Use isolated session (or `AGENT_BROWSER_SESSION` env) |
+| `--headers <json>` | Set HTTP headers scoped to the URL's origin |
 | `--executable-path <path>` | Custom browser executable (or `AGENT_BROWSER_EXECUTABLE_PATH` env) |
 | `--json` | JSON output (for agents) |
 | `--full, -f` | Full page screenshot |
 | `--name, -n` | Locator name filter |
 | `--exact` | Exact text match |
 | `--headed` | Show browser window (not headless) |
+| `--cdp <port>` | Connect via Chrome DevTools Protocol |
 | `--debug` | Debug output |
 
 ## Selectors
@@ -388,6 +392,41 @@ agent-browser open example.com --headed
 
 This opens a visible browser window instead of running headless.
 
+## Authenticated Sessions
+
+Use `--headers` to set HTTP headers for a specific origin, enabling authentication without login flows:
+
+```bash
+# Headers are scoped to api.example.com only
+agent-browser open api.example.com --headers '{"Authorization": "Bearer <token>"}'
+
+# Requests to api.example.com include the auth header
+agent-browser snapshot -i --json
+agent-browser click @e2
+
+# Navigate to another domain - headers are NOT sent (safe!)
+agent-browser open other-site.com
+```
+
+This is useful for:
+- **Skipping login flows** - Authenticate via headers instead of UI
+- **Switching users** - Start new sessions with different auth tokens
+- **API testing** - Access protected endpoints directly
+- **Security** - Headers are scoped to the origin, not leaked to other domains
+
+To set headers for multiple origins, use `--headers` with each `open` command:
+
+```bash
+agent-browser open api.example.com --headers '{"Authorization": "Bearer token1"}'
+agent-browser open api.acme.com --headers '{"Authorization": "Bearer token2"}'
+```
+
+For global headers (all domains), use `set headers`:
+
+```bash
+agent-browser set headers '{"X-Custom-Header": "value"}'
+```
+
 ## Custom Browser Executable
 
 Use a custom browser executable instead of the bundled Chromium. This is useful for:
@@ -421,6 +460,132 @@ export async function handler() {
 }
 ```
 
+## CDP Mode
+
+Connect to an existing browser via Chrome DevTools Protocol:
+
+```bash
+# Connect to Electron app
+agent-browser --cdp 9222 snapshot
+
+# Connect to Chrome with remote debugging
+# (Start Chrome with: google-chrome --remote-debugging-port=9222)
+agent-browser --cdp 9222 open about:blank
+```
+
+This enables control of:
+- Electron apps
+- Chrome/Chromium instances with remote debugging
+- WebView2 applications
+- Any browser exposing a CDP endpoint
+
+## Streaming (Browser Preview)
+
+Stream the browser viewport via WebSocket for live preview or "pair browsing" where a human can watch and interact alongside an AI agent.
+
+### Enable Streaming
+
+Set the `AGENT_BROWSER_STREAM_PORT` environment variable:
+
+```bash
+AGENT_BROWSER_STREAM_PORT=9223 agent-browser open example.com
+```
+
+This starts a WebSocket server on the specified port that streams the browser viewport and accepts input events.
+
+### WebSocket Protocol
+
+Connect to `ws://localhost:9223` to receive frames and send input:
+
+**Receive frames:**
+```json
+{
+  "type": "frame",
+  "data": "<base64-encoded-jpeg>",
+  "metadata": {
+    "deviceWidth": 1280,
+    "deviceHeight": 720,
+    "pageScaleFactor": 1,
+    "offsetTop": 0,
+    "scrollOffsetX": 0,
+    "scrollOffsetY": 0
+  }
+}
+```
+
+**Send mouse events:**
+```json
+{
+  "type": "input_mouse",
+  "eventType": "mousePressed",
+  "x": 100,
+  "y": 200,
+  "button": "left",
+  "clickCount": 1
+}
+```
+
+**Send keyboard events:**
+```json
+{
+  "type": "input_keyboard",
+  "eventType": "keyDown",
+  "key": "Enter",
+  "code": "Enter"
+}
+```
+
+**Send touch events:**
+```json
+{
+  "type": "input_touch",
+  "eventType": "touchStart",
+  "touchPoints": [{ "x": 100, "y": 200 }]
+}
+```
+
+### Programmatic API
+
+For advanced use, control streaming directly via the protocol:
+
+```typescript
+import { BrowserManager } from 'agent-browser';
+
+const browser = new BrowserManager();
+await browser.launch({ headless: true });
+await browser.navigate('https://example.com');
+
+// Start screencast
+await browser.startScreencast((frame) => {
+  // frame.data is base64-encoded image
+  // frame.metadata contains viewport info
+  console.log('Frame received:', frame.metadata.deviceWidth, 'x', frame.metadata.deviceHeight);
+}, {
+  format: 'jpeg',
+  quality: 80,
+  maxWidth: 1280,
+  maxHeight: 720,
+});
+
+// Inject mouse events
+await browser.injectMouseEvent({
+  type: 'mousePressed',
+  x: 100,
+  y: 200,
+  button: 'left',
+});
+
+// Inject keyboard events
+await browser.injectKeyboardEvent({
+  type: 'keyDown',
+  key: 'Enter',
+  code: 'Enter',
+});
+
+// Stop when done
+await browser.stopScreencast();
+```
+
 ## Architecture
 
 agent-browser uses a client-daemon architecture:

cli/Cargo.lock

@@ -4,7 +4,7 @@ version = 4
 
 [[package]]
 name = "agent-browser"
-version = "0.4.3"
+version = "0.5.0"
 dependencies = [
  "libc",
  "serde",

cli/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "agent-browser"
-version = "0.4.3"
+version = "0.5.0"
 edition = "2021"
 description = "Fast browser automation CLI for AI agents"
 license = "Apache-2.0"

cli/src/commands.rs

@@ -80,7 +80,14 @@ pub fn parse_command(args: &[String], flags: &Flags) -> Result<Value, ParseError
             } else {
                 format!("https://{}", url)
             };
-            Ok(json!({ "id": id, "action": "navigate", "url": url }))
+            let mut nav_cmd = json!({ "id": id, "action": "navigate", "url": url });
+            // If --headers flag is set, include headers (scoped to this origin)
+            if let Some(ref headers_json) = flags.headers {
+                if let Ok(headers) = serde_json::from_str::<serde_json::Value>(headers_json) {
+                    nav_cmd["headers"] = headers;
+                }
+            }
+            Ok(nav_cmd)
         }
         "back" => Ok(json!({ "id": id, "action": "back" })),
         "forward" => Ok(json!({ "id": id, "action": "forward" })),
@@ -766,7 +773,13 @@ fn parse_set(rest: &[&str], id: &str) -> Result<Value, ParseError> {
                 context: "set headers".to_string(),
                 usage: "set headers <json>",
             })?;
-            Ok(json!({ "id": id, "action": "headers", "headers": headers_json }))
+            // Parse the JSON string into an object
+            let headers: serde_json::Value = serde_json::from_str(headers_json)
+                .map_err(|_| ParseError::MissingArguments {
+                    context: "set headers".to_string(),
+                    usage: "set headers <json> (must be valid JSON object)",
+                })?;
+            Ok(json!({ "id": id, "action": "headers", "headers": headers }))
         }
         Some("credentials") | Some("auth") => {
             let user = rest.get(1).ok_or_else(|| ParseError::MissingArguments {
@@ -886,7 +899,10 @@ mod tests {
             full: false,
             headed: false,
             debug: false,
+            headers: None,
             executable_path: None,
+            extensions: Vec::new(),
+            cdp: None,
         }
     }
 
@@ -1013,6 +1029,81 @@ mod tests {
         assert_eq!(cmd["url"], "https://example.com");
     }
 
+    #[test]
+    fn test_navigate_with_headers() {
+        let mut flags = default_flags();
+        flags.headers = Some(r#"{"Authorization": "Bearer token"}"#.to_string());
+        let cmd = parse_command(&args("open api.example.com"), &flags).unwrap();
+        assert_eq!(cmd["action"], "navigate");
+        assert_eq!(cmd["url"], "https://api.example.com");
+        assert_eq!(cmd["headers"]["Authorization"], "Bearer token");
+    }
+
+    #[test]
+    fn test_navigate_with_multiple_headers() {
+        let mut flags = default_flags();
+        flags.headers = Some(r#"{"Authorization": "Bearer token", "X-Custom": "value"}"#.to_string());
+        let cmd = parse_command(&args("open api.example.com"), &flags).unwrap();
+        assert_eq!(cmd["headers"]["Authorization"], "Bearer token");
+        assert_eq!(cmd["headers"]["X-Custom"], "value");
+    }
+
+    #[test]
+    fn test_navigate_without_headers_flag() {
+        let cmd = parse_command(&args("open example.com"), &default_flags()).unwrap();
+        assert_eq!(cmd["action"], "navigate");
+        // headers should not be present when flag is not set
+        assert!(cmd.get("headers").is_none());
+    }
+
+    #[test]
+    fn test_navigate_with_invalid_headers_json() {
+        let mut flags = default_flags();
+        flags.headers = Some("not valid json".to_string());
+        let cmd = parse_command(&args("open api.example.com"), &flags).unwrap();
+        // Invalid JSON should result in no headers field (graceful handling)
+        assert!(cmd.get("headers").is_none());
+    }
+
+    // === Set Headers Tests ===
+
+    #[test]
+    fn test_set_headers_parses_json() {
+        let input: Vec<String> = vec![
+            "set".to_string(),
+            "headers".to_string(),
+            r#"{"Authorization":"Bearer token"}"#.to_string(),
+        ];
+        let cmd = parse_command(&input, &default_flags()).unwrap();
+        assert_eq!(cmd["action"], "headers");
+        // Headers should be an object, not a string
+        assert!(cmd["headers"].is_object());
+        assert_eq!(cmd["headers"]["Authorization"], "Bearer token");
+    }
+
+    #[test]
+    fn test_set_headers_with_multiple_values() {
+        let input: Vec<String> = vec![
+            "set".to_string(),
+            "headers".to_string(),
+            r#"{"Authorization": "Bearer token", "X-Custom": "value"}"#.to_string(),
+        ];
+        let cmd = parse_command(&input, &default_flags()).unwrap();
+        assert_eq!(cmd["headers"]["Authorization"], "Bearer token");
+        assert_eq!(cmd["headers"]["X-Custom"], "value");
+    }
+
+    #[test]
+    fn test_set_headers_invalid_json_error() {
+        let input: Vec<String> = vec![
+            "set".to_string(),
+            "headers".to_string(),
+            "not-valid-json".to_string(),
+        ];
+        let result = parse_command(&input, &default_flags());
+        assert!(result.is_err());
+    }
+
     #[test]
     fn test_back() {
         let cmd = parse_command(&args("back"), &default_flags()).unwrap();

cli/src/connection.rs

@@ -159,9 +159,16 @@ pub struct DaemonResult {
     pub already_running: bool,
 }
 
-pub fn ensure_daemon(session: &str, headed: bool, executable_path: Option<&str>) -> Result<DaemonResult, String> {
+pub fn ensure_daemon(
+    session: &str,
+    headed: bool,
+    executable_path: Option<&str>,
+    extensions: &[String],
+) -> Result<DaemonResult, String> {
     if is_daemon_running(session) && daemon_ready(session) {
-        return Ok(DaemonResult { already_running: true });
+        return Ok(DaemonResult {
+            already_running: true,
+        });
     }
 
     let exe_path = env::current_exe().map_err(|e| e.to_string())?;
@@ -196,6 +203,10 @@ pub fn ensure_daemon(session: &str, headed: bool, executable_path: Option<&str>)
             cmd.env("AGENT_BROWSER_EXECUTABLE_PATH", path);
         }
 
+        if !extensions.is_empty() {
+            cmd.env("AGENT_BROWSER_EXTENSIONS", extensions.join(","));
+        }
+
         // Create new process group and session to fully detach
         unsafe {
             cmd.pre_exec(|| {
@@ -234,6 +245,10 @@ pub fn ensure_daemon(session: &str, headed: bool, executable_path: Option<&str>)
             cmd.env("AGENT_BROWSER_EXECUTABLE_PATH", path);
         }
 
+        if !extensions.is_empty() {
+            cmd.env("AGENT_BROWSER_EXTENSIONS", extensions.join(","));
+        }
+
         // CREATE_NEW_PROCESS_GROUP | DETACHED_PROCESS
         const CREATE_NEW_PROCESS_GROUP: u32 = 0x00000200;
         const DETACHED_PROCESS: u32 = 0x00000008;

cli/src/flags.rs

@@ -6,17 +6,28 @@ pub struct Flags {
     pub headed: bool,
     pub debug: bool,
     pub session: String,
+    pub headers: Option<String>,
     pub executable_path: Option<String>,
+    pub cdp: Option<String>,
+    pub extensions: Vec<String>,
 }
 
 pub fn parse_flags(args: &[String]) -> Flags {
+    let extensions_env = env::var("AGENT_BROWSER_EXTENSIONS")
+        .ok()
+        .map(|s| s.split(',').map(|p| p.trim().to_string()).filter(|p| !p.is_empty()).collect::<Vec<_>>())
+        .unwrap_or_default();
+
     let mut flags = Flags {
         json: false,
         full: false,
         headed: false,
         debug: false,
         session: env::var("AGENT_BROWSER_SESSION").unwrap_or_else(|_| "default".to_string()),
+        headers: None,
         executable_path: env::var("AGENT_BROWSER_EXECUTABLE_PATH").ok(),
+        cdp: None,
+        extensions: extensions_env,
     };
 
     let mut i = 0;
@@ -32,11 +43,29 @@ pub fn parse_flags(args: &[String]) -> Flags {
                     i += 1;
                 }
             }
+            "--headers" => {
+                if let Some(h) = args.get(i + 1) {
+                    flags.headers = Some(h.clone());
+                    i += 1;
+                }
+            }
             "--executable-path" => {
                 if let Some(s) = args.get(i + 1) {
                     flags.executable_path = Some(s.clone());
                     i += 1;
                 }
+            },
+            "--extension" => {
+                if let Some(s) = args.get(i + 1) {
+                    flags.extensions.push(s.clone());
+                    i += 1;
+                }
+            },
+            "--cdp" => {
+                if let Some(s) = args.get(i + 1) {
+                    flags.cdp = Some(s.clone());
+                    i += 1;
+                }
             }
             _ => {}
         }
@@ -51,15 +80,15 @@ pub fn clean_args(args: &[String]) -> Vec<String> {
 
     // Global flags that should be stripped from command args
     const GLOBAL_FLAGS: &[&str] = &["--json", "--full", "--headed", "--debug"];
-    // Flags that take a value (skip both the flag and the next arg)
+    // Global flags that take a value (need to skip the next arg too)
-    const VALUE_FLAGS: &[&str] = &["--session", "--executable-path"];
+    const GLOBAL_FLAGS_WITH_VALUE: &[&str] = &["--session", "--headers", "--executable-path", "--cdp", "--extension"];
 
     for arg in args.iter() {
         if skip_next {
             skip_next = false;
             continue;
         }
-        if VALUE_FLAGS.contains(&arg.as_str()) {
+        if GLOBAL_FLAGS_WITH_VALUE.contains(&arg.as_str()) {
             skip_next = true;
             continue;
         }
@@ -80,6 +109,74 @@ mod tests {
         s.split_whitespace().map(String::from).collect()
     }
 
+    #[test]
+    fn test_parse_headers_flag() {
+        let flags = parse_flags(&args(r#"open example.com --headers {"Auth":"token"}"#));
+        assert_eq!(flags.headers, Some(r#"{"Auth":"token"}"#.to_string()));
+    }
+
+    #[test]
+    fn test_parse_headers_flag_with_spaces() {
+        // Headers JSON is passed as a single quoted argument in shell
+        let input: Vec<String> = vec![
+            "open".to_string(),
+            "example.com".to_string(),
+            "--headers".to_string(),
+            r#"{"Authorization": "Bearer token"}"#.to_string(),
+        ];
+        let flags = parse_flags(&input);
+        assert_eq!(flags.headers, Some(r#"{"Authorization": "Bearer token"}"#.to_string()));
+    }
+
+    #[test]
+    fn test_parse_no_headers_flag() {
+        let flags = parse_flags(&args("open example.com"));
+        assert!(flags.headers.is_none());
+    }
+
+    #[test]
+    fn test_clean_args_removes_headers() {
+        let input: Vec<String> = vec![
+            "open".to_string(),
+            "example.com".to_string(),
+            "--headers".to_string(),
+            r#"{"Auth":"token"}"#.to_string(),
+        ];
+        let clean = clean_args(&input);
+        assert_eq!(clean, vec!["open", "example.com"]);
+    }
+
+    #[test]
+    fn test_clean_args_removes_headers_at_start() {
+        let input: Vec<String> = vec![
+            "--headers".to_string(),
+            r#"{"Auth":"token"}"#.to_string(),
+            "open".to_string(),
+            "example.com".to_string(),
+        ];
+        let clean = clean_args(&input);
+        assert_eq!(clean, vec!["open", "example.com"]);
+    }
+
+    #[test]
+    fn test_headers_with_other_flags() {
+        let input: Vec<String> = vec![
+            "open".to_string(),
+            "example.com".to_string(),
+            "--headers".to_string(),
+            r#"{"Auth":"token"}"#.to_string(),
+            "--json".to_string(),
+            "--headed".to_string(),
+        ];
+        let flags = parse_flags(&input);
+        assert_eq!(flags.headers, Some(r#"{"Auth":"token"}"#.to_string()));
+        assert!(flags.json);
+        assert!(flags.headed);
+        
+        let clean = clean_args(&input);
+        assert_eq!(clean, vec!["open", "example.com"]);
+    }
+
     #[test]
     fn test_parse_executable_path_flag() {
         let flags = parse_flags(&args("--executable-path /path/to/chromium open example.com"));

cli/src/main.rs

@@ -149,7 +149,7 @@ fn main() {
         }
     };
 
-    let daemon_result = match ensure_daemon(&flags.session, flags.headed, flags.executable_path.as_deref()) {
+    let daemon_result = match ensure_daemon(&flags.session, flags.headed, flags.executable_path.as_deref(), &flags.extensions) {
         Ok(result) => result,
         Err(e) => {
             if flags.json {
@@ -162,18 +162,83 @@ fn main() {
     };
 
     // Warn if executable_path was specified but daemon was already running
-    if daemon_result.already_running && flags.executable_path.is_some() {
+    if daemon_result.already_running && (flags.executable_path.is_some() || !flags.extensions.is_empty()) {
         if !flags.json {
-            eprintln!("\x1b[33m⚠\x1b[0m --executable-path ignored: daemon already running. Use 'agent-browser close' first to restart with new path.");
+            if flags.executable_path.is_some() {
+                eprintln!("\x1b[33m⚠\x1b[0m --executable-path ignored: daemon already running. Use 'agent-browser close' first to restart with new path.");
+            }
+            if !flags.extensions.is_empty() {
+                eprintln!("\x1b[33m⚠\x1b[0m --extension ignored: daemon already running. Use 'agent-browser close' first to restart with extensions.");
+            }
         }
     }
 
-    // If --headed flag is set, send launch command first to switch to headed mode
+    // Connect via CDP if --cdp flag is set
-    if flags.headed {
+    if let Some(ref port) = flags.cdp {
-        let launch_cmd = json!({ "id": gen_id(), "action": "launch", "headless": false });
+        let cdp_port: u16 = match port.parse::<u32>() {
+            Ok(p) if p == 0 => {
+                let msg = "Invalid CDP port: port must be greater than 0".to_string();
+                if flags.json {
+                    println!(r#"{{"success":false,"error":"{}"}}"#, msg);
+                } else {
+                    eprintln!("\x1b[31m✗\x1b[0m {}", msg);
+                }
+                exit(1);
+            }
+            Ok(p) if p > 65535 => {
+                let msg = format!("Invalid CDP port: {} is out of range (valid range: 1-65535)", p);
+                if flags.json {
+                    println!(r#"{{"success":false,"error":"{}"}}"#, msg);
+                } else {
+                    eprintln!("\x1b[31m✗\x1b[0m {}", msg);
+                }
+                exit(1);
+            }
+            Ok(p) => p as u16,
+            Err(_) => {
+                let msg = format!("Invalid CDP port: '{}' is not a valid number. Port must be a number between 1 and 65535", port);
+                if flags.json {
+                    println!(r#"{{"success":false,"error":"{}"}}"#, msg);
+                } else {
+                    eprintln!("\x1b[31m✗\x1b[0m {}", msg);
+                }
+                exit(1);
+            }
+        };
+
+        let launch_cmd = json!({
+            "id": gen_id(),
+            "action": "launch",
+            "cdpPort": cdp_port
+        });
+
+        let err = match send_command(launch_cmd, &flags.session) {
+            Ok(resp) if resp.success => None,
+            Ok(resp) => Some(resp.error.unwrap_or_else(|| "CDP connection failed".to_string())),
+            Err(e) => Some(e.to_string()),
+        };
+
+        if let Some(msg) = err {
+            if flags.json {
+                println!(r#"{{"success":false,"error":"{}"}}"#, msg);
+            } else {
+                eprintln!("\x1b[31m✗\x1b[0m {}", msg);
+            }
+            exit(1);
+        }
+    }
+
+    // Launch headed browser if --headed flag is set (without CDP)
+    if flags.headed && flags.cdp.is_none() {
+        let launch_cmd = json!({
+            "id": gen_id(),
+            "action": "launch",
+            "headless": false
+        });
+
         if let Err(e) = send_command(launch_cmd, &flags.session) {
             if !flags.json {
-                eprintln!("\x1b[33m⚠\x1b[0m Could not switch to headed mode: {}", e);
+                eprintln!("\x1b[33m⚠\x1b[0m Could not launch headed browser: {}", e);
             }
         }
     }

cli/src/output.rs

@@ -162,12 +162,15 @@ Aliases: goto, navigate
 Global Options:
   --json               Output as JSON
   --session <name>     Use specific session
+  --headers <json>     Set HTTP headers (scoped to this origin)
   --headed             Show browser window
 
 Examples:
   agent-browser open example.com
   agent-browser open https://github.com
   agent-browser open localhost:3000
+  agent-browser open api.example.com --headers '{"Authorization": "Bearer token"}'
+    # ^ Headers only sent to api.example.com, not other domains
 "##,
         "back" => r##"
 agent-browser back - Navigate back in history
@@ -1186,12 +1189,20 @@ Snapshot Options:
 
 Options:
   --session <name>           Isolated session (or AGENT_BROWSER_SESSION env)
+  --headers <json>           HTTP headers scoped to URL's origin (for auth)
   --executable-path <path>   Custom browser executable (or AGENT_BROWSER_EXECUTABLE_PATH)
+  --extension <path>         Load browser extensions (repeatable).
   --json                     JSON output
   --full, -f                 Full page screenshot
   --headed                   Show browser window (not headless)
+  --cdp <port>               Connect via CDP (Chrome DevTools Protocol)
   --debug                    Debug output
 
+Environment:
+  AGENT_BROWSER_SESSION          Session name (default: "default")
+  AGENT_BROWSER_EXECUTABLE_PATH  Custom browser executable path
+  AGENT_BROWSER_STREAM_PORT      Enable WebSocket streaming on port (e.g., 9223)
+
 Examples:
   agent-browser open example.com
   agent-browser snapshot -i              # Interactive elements only
@@ -1200,6 +1211,7 @@ Examples:
   agent-browser find role button click --name Submit
   agent-browser get text @e1
   agent-browser screenshot --full
+  agent-browser --cdp 9222 snapshot      # Connect via CDP port
 "#
     );
 }

docs/.gitignore

@@ -0,0 +1,41 @@
+# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
+
+# dependencies
+/node_modules
+/.pnp
+.pnp.*
+.yarn/*
+!.yarn/patches
+!.yarn/plugins
+!.yarn/releases
+!.yarn/versions
+
+# testing
+/coverage
+
+# next.js
+/.next/
+/out/
+
+# production
+/build
+
+# misc
+.DS_Store
+*.pem
+
+# debug
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+.pnpm-debug.log*
+
+# env files (can opt-in for committing if needed)
+.env*
+
+# vercel
+.vercel
+
+# typescript
+*.tsbuildinfo
+next-env.d.ts

docs/eslint.config.mjs

@@ -0,0 +1,18 @@
+import { defineConfig, globalIgnores } from "eslint/config";
+import nextVitals from "eslint-config-next/core-web-vitals";
+import nextTs from "eslint-config-next/typescript";
+
+const eslintConfig = defineConfig([
+  ...nextVitals,
+  ...nextTs,
+  // Override default ignores of eslint-config-next.
+  globalIgnores([
+    // Default ignores of eslint-config-next:
+    ".next/**",
+    "out/**",
+    "build/**",
+    "next-env.d.ts",
+  ]),
+]);
+
+export default eslintConfig;

docs/next.config.ts

@@ -0,0 +1,7 @@
+import type { NextConfig } from "next";
+
+const nextConfig: NextConfig = {
+  /* config options here */
+};
+
+export default nextConfig;

docs/package.json

@@ -0,0 +1,27 @@
+{
+  "name": "docs",
+  "version": "0.1.0",
+  "private": true,
+  "scripts": {
+    "dev": "next dev",
+    "build": "next build",
+    "start": "next start",
+    "lint": "eslint"
+  },
+  "dependencies": {
+    "next": "16.1.1",
+    "react": "19.2.3",
+    "react-dom": "19.2.3",
+    "shiki": "^3.21.0"
+  },
+  "devDependencies": {
+    "@tailwindcss/postcss": "^4",
+    "@types/node": "^20",
+    "@types/react": "^19",
+    "@types/react-dom": "^19",
+    "eslint": "^9",
+    "eslint-config-next": "16.1.1",
+    "tailwindcss": "^4",
+    "typescript": "^5"
+  }
+}

docs/postcss.config.mjs

@@ -0,0 +1,7 @@
+const config = {
+  plugins: {
+    "@tailwindcss/postcss": {},
+  },
+};
+
+export default config;

docs/src/app/agent-mode/page.tsx

@@ -0,0 +1,72 @@
+import { CodeBlock } from "@/components/code-block";
+
+export default function AgentMode() {
+  return (
+    <div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
+      <div className="prose">
+        <h1>Agent Mode</h1>
+        <p>
+          agent-browser works with any AI coding agent. Use <code>--json</code> for machine-readable output.
+        </p>
+
+        <h2>Compatible agents</h2>
+        <ul>
+          <li>Claude Code</li>
+          <li>Cursor</li>
+          <li>GitHub Copilot</li>
+          <li>OpenAI Codex</li>
+          <li>Google Gemini</li>
+          <li>opencode</li>
+          <li>Any agent that can run shell commands</li>
+        </ul>
+
+        <h2>JSON output</h2>
+        <CodeBlock code={`agent-browser snapshot --json
+# {"success":true,"data":{"snapshot":"...","refs":{...}}}
+
+agent-browser get text @e1 --json
+agent-browser is visible @e2 --json`} />
+
+        <h2>Optimal workflow</h2>
+        <CodeBlock code={`# 1. Navigate and get snapshot
+agent-browser open example.com
+agent-browser snapshot -i --json   # AI parses tree and refs
+
+# 2. AI identifies target refs from snapshot
+# 3. Execute actions using refs
+agent-browser click @e2
+agent-browser fill @e3 "input text"
+
+# 4. Get new snapshot if page changed
+agent-browser snapshot -i --json`} />
+
+        <h2>Integration</h2>
+
+        <h3>Just ask</h3>
+        <p>The simplest approach:</p>
+        <CodeBlock lang="text" code="Use agent-browser to test the login flow. Run agent-browser --help to see available commands." />
+        <p>The <code>--help</code> output is comprehensive.</p>
+
+        <h3>AGENTS.md / CLAUDE.md</h3>
+        <p>For consistent results, add to your instructions file:</p>
+        <CodeBlock lang="markdown" code={`## Browser Automation
+
+Use \`agent-browser\` for web automation. Run \`agent-browser --help\` for all commands.
+
+Core workflow:
+1. \`agent-browser open <url>\` - Navigate to page
+2. \`agent-browser snapshot -i\` - Get interactive elements with refs (@e1, @e2)
+3. \`agent-browser click @e1\` / \`fill @e2 "text"\` - Interact using refs
+4. Re-snapshot after page changes`} />
+
+        <h3>Claude Code skill</h3>
+        <p>For richer context:</p>
+        <CodeBlock code="cp -r node_modules/agent-browser/skills/agent-browser .claude/skills/" />
+        <p>Or download:</p>
+        <CodeBlock code={`mkdir -p .claude/skills/agent-browser
+curl -o .claude/skills/agent-browser/SKILL.md \\
+  https://raw.githubusercontent.com/vercel-labs/agent-browser/main/skills/agent-browser/SKILL.md`} />
+      </div>
+    </div>
+  );
+}

docs/src/app/cdp-mode/page.tsx

@@ -0,0 +1,79 @@
+import { CodeBlock } from "@/components/code-block";
+
+export default function CDPMode() {
+  return (
+    <div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
+      <div className="prose">
+        <h1>CDP Mode</h1>
+        <p>Connect to an existing browser via Chrome DevTools Protocol:</p>
+        <CodeBlock code={`# Connect to Electron app
+agent-browser --cdp 9222 snapshot
+
+# Connect to Chrome with remote debugging
+# (Start Chrome with: google-chrome --remote-debugging-port=9222)
+agent-browser --cdp 9222 open about:blank`} />
+
+        <h2>Use cases</h2>
+        <p>This enables control of:</p>
+        <ul>
+          <li>Electron apps</li>
+          <li>Chrome/Chromium with remote debugging</li>
+          <li>WebView2 applications</li>
+          <li>Any browser exposing a CDP endpoint</li>
+        </ul>
+
+        <h2>Global options</h2>
+        <table>
+          <thead>
+            <tr>
+              <th>Option</th>
+              <th>Description</th>
+            </tr>
+          </thead>
+          <tbody>
+            <tr>
+              <td><code>--session &lt;name&gt;</code></td>
+              <td>Use isolated session</td>
+            </tr>
+            <tr>
+              <td><code>--headers &lt;json&gt;</code></td>
+              <td>HTTP headers scoped to origin</td>
+            </tr>
+            <tr>
+              <td><code>--executable-path</code></td>
+              <td>Custom browser executable</td>
+            </tr>
+            <tr>
+              <td><code>--json</code></td>
+              <td>JSON output for agents</td>
+            </tr>
+            <tr>
+              <td><code>--full, -f</code></td>
+              <td>Full page screenshot</td>
+            </tr>
+            <tr>
+              <td><code>--name, -n</code></td>
+              <td>Locator name filter</td>
+            </tr>
+            <tr>
+              <td><code>--exact</code></td>
+              <td>Exact text match</td>
+            </tr>
+            <tr>
+              <td><code>--headed</code></td>
+              <td>Show browser window</td>
+            </tr>
+            <tr>
+              <td><code>--cdp &lt;port&gt;</code></td>
+              <td>CDP connection port</td>
+            </tr>
+            <tr>
+              <td><code>--debug</code></td>
+              <td>Debug output</td>
+            </tr>
+          </tbody>
+        </table>
+      </div>
+    </div>
+  );
+}

docs/src/app/commands/page.tsx

@@ -0,0 +1,121 @@
+import { CodeBlock } from "@/components/code-block";
+
+export default function Commands() {
+  return (
+    <div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
+      <div className="prose">
+        <h1>Commands</h1>
+
+        <h2>Core</h2>
+        <CodeBlock code={`agent-browser open <url>              # Navigate (aliases: goto, navigate)
+agent-browser click <sel>             # Click element
+agent-browser dblclick <sel>          # Double-click
+agent-browser fill <sel> <text>       # Clear and fill
+agent-browser type <sel> <text>       # Type into element
+agent-browser press <key>             # Press key (Enter, Tab, Control+a)
+agent-browser hover <sel>             # Hover element
+agent-browser select <sel> <val>      # Select dropdown option
+agent-browser check <sel>             # Check checkbox
+agent-browser uncheck <sel>           # Uncheck checkbox
+agent-browser scroll <dir> [px]       # Scroll (up/down/left/right)
+agent-browser screenshot [path]       # Screenshot (--full for full page)
+agent-browser snapshot                # Accessibility tree with refs
+agent-browser eval <js>               # Run JavaScript
+agent-browser close                   # Close browser`} />
+
+        <h2>Get info</h2>
+        <CodeBlock code={`agent-browser get text <sel>          # Get text content
+agent-browser get html <sel>          # Get innerHTML
+agent-browser get value <sel>         # Get input value
+agent-browser get attr <sel> <attr>   # Get attribute
+agent-browser get title               # Get page title
+agent-browser get url                 # Get current URL
+agent-browser get count <sel>         # Count matching elements
+agent-browser get box <sel>           # Get bounding box`} />
+
+        <h2>Check state</h2>
+        <CodeBlock code={`agent-browser is visible <sel>        # Check if visible
+agent-browser is enabled <sel>        # Check if enabled
+agent-browser is checked <sel>        # Check if checked`} />
+
+        <h2>Find elements</h2>
+        <p>Semantic locators with actions (<code>click</code>, <code>fill</code>, <code>check</code>, <code>hover</code>, <code>text</code>):</p>
+        <CodeBlock code={`agent-browser find role <role> <action> [value]
+agent-browser find text <text> <action>
+agent-browser find label <label> <action> [value]
+agent-browser find placeholder <ph> <action> [value]
+agent-browser find testid <id> <action> [value]
+agent-browser find first <sel> <action> [value]
+agent-browser find nth <n> <sel> <action> [value]`} />
+        <p>Examples:</p>
+        <CodeBlock code={`agent-browser find role button click --name "Submit"
+agent-browser find label "Email" fill "test@test.com"
+agent-browser find first ".item" click`} />
+
+        <h2>Wait</h2>
+        <CodeBlock code={`agent-browser wait <selector>         # Wait for element
+agent-browser wait <ms>               # Wait for time
+agent-browser wait --text "Welcome"   # Wait for text
+agent-browser wait --url "**/dash"    # Wait for URL pattern
+agent-browser wait --load networkidle # Wait for load state
+agent-browser wait --fn "condition"   # Wait for JS condition`} />
+
+        <h2>Mouse</h2>
+        <CodeBlock code={`agent-browser mouse move <x> <y>      # Move mouse
+agent-browser mouse down [button]     # Press button
+agent-browser mouse up [button]       # Release button
+agent-browser mouse wheel <dy> [dx]   # Scroll wheel`} />
+
+        <h2>Settings</h2>
+        <CodeBlock code={`agent-browser set viewport <w> <h>    # Set viewport size
+agent-browser set device <name>       # Emulate device ("iPhone 14")
+agent-browser set geo <lat> <lng>     # Set geolocation
+agent-browser set offline [on|off]    # Toggle offline mode
+agent-browser set headers <json>      # Extra HTTP headers
+agent-browser set credentials <u> <p> # HTTP basic auth
+agent-browser set media [dark|light]  # Emulate color scheme`} />
+
+        <h2>Cookies & storage</h2>
+        <CodeBlock code={`agent-browser cookies                 # Get all cookies
+agent-browser cookies set <name> <val> # Set cookie
+agent-browser cookies clear           # Clear cookies
+
+agent-browser storage local           # Get all localStorage
+agent-browser storage local <key>     # Get specific key
+agent-browser storage local set <k> <v>  # Set value
+agent-browser storage local clear     # Clear all
+
+agent-browser storage session         # Same for sessionStorage`} />
+
+        <h2>Network</h2>
+        <CodeBlock code={`agent-browser network route <url>              # Intercept requests
+agent-browser network route <url> --abort      # Block requests
+agent-browser network route <url> --body <json>  # Mock response
+agent-browser network unroute [url]            # Remove routes
+agent-browser network requests                 # View tracked requests`} />
+
+        <h2>Tabs & frames</h2>
+        <CodeBlock code={`agent-browser tab                     # List tabs
+agent-browser tab new [url]           # New tab
+agent-browser tab <n>                 # Switch to tab
+agent-browser tab close [n]           # Close tab
+agent-browser frame <sel>             # Switch to iframe
+agent-browser frame main              # Back to main frame`} />
+
+        <h2>Debug</h2>
+        <CodeBlock code={`agent-browser trace start [path]      # Start trace
+agent-browser trace stop [path]       # Stop and save trace
+agent-browser console                 # View console messages
+agent-browser errors                  # View page errors
+agent-browser highlight <sel>         # Highlight element
+agent-browser state save <path>       # Save auth state
+agent-browser state load <path>       # Load auth state`} />
+
+        <h2>Navigation</h2>
+        <CodeBlock code={`agent-browser back                    # Go back
+agent-browser forward                 # Go forward
+agent-browser reload                  # Reload page`} />
+      </div>
+    </div>
+  );
+}

docs/src/app/globals.css

@@ -0,0 +1,182 @@
+@import "tailwindcss";
+
+:root {
+  --background: #000000;
+  --foreground: #ededed;
+  --muted: #888888;
+  --border: #222222;
+  --accent: #ededed;
+}
+
+@theme inline {
+  --color-background: var(--background);
+  --color-foreground: var(--foreground);
+  --color-muted: var(--muted);
+  --color-border: var(--border);
+  --color-accent: var(--accent);
+  --font-sans: var(--font-geist);
+  --font-mono: var(--font-geist-mono);
+}
+
+body {
+  background: var(--background);
+  color: var(--foreground);
+  font-family: var(--font-geist), system-ui, sans-serif;
+}
+
+/* Scrollbar */
+::-webkit-scrollbar {
+  width: 6px;
+  height: 6px;
+}
+
+::-webkit-scrollbar-track {
+  background: transparent;
+}
+
+::-webkit-scrollbar-thumb {
+  background: #333;
+  border-radius: 3px;
+}
+
+::-webkit-scrollbar-thumb:hover {
+  background: #444;
+}
+
+/* Code blocks */
+pre {
+  background: #111 !important;
+  border: 1px solid var(--border);
+  border-radius: 4px;
+  padding: 0.875rem;
+  overflow-x: auto;
+  font-family: var(--font-geist-mono), monospace;
+  font-size: 0.8125rem;
+  line-height: 1.7;
+}
+
+.code-block pre {
+  margin: 0;
+}
+
+.code-block {
+  margin-bottom: 1.25rem;
+}
+
+@media (max-width: 640px) {
+  pre {
+    font-size: 0.75rem;
+    padding: 0.75rem;
+  }
+}
+
+code {
+  font-family: var(--font-geist-mono), monospace;
+}
+
+:not(pre) > code {
+  background: #1a1a1a;
+  padding: 0.125rem 0.375rem;
+  border-radius: 3px;
+  font-size: 0.875em;
+}
+
+/* Prose */
+.prose {
+  max-width: 100%;
+}
+
+.prose h1 {
+  font-size: 1.5rem;
+  font-weight: 500;
+  letter-spacing: -0.02em;
+  margin-bottom: 0.5rem;
+  color: #fff;
+}
+
+@media (min-width: 640px) {
+  .prose h1 {
+    font-size: 1.75rem;
+  }
+}
+
+.prose h2 {
+  font-size: 0.875rem;
+  font-weight: 500;
+  letter-spacing: 0;
+  text-transform: uppercase;
+  color: var(--muted);
+  margin-top: 3rem;
+  margin-bottom: 1rem;
+}
+
+.prose h3 {
+  font-size: 0.875rem;
+  font-weight: 500;
+  margin-top: 2rem;
+  margin-bottom: 0.75rem;
+  color: #ccc;
+}
+
+.prose p {
+  margin-bottom: 1.25rem;
+  line-height: 1.7;
+  color: var(--muted);
+  font-size: 0.9375rem;
+}
+
+.prose ul, .prose ol {
+  margin-bottom: 1.25rem;
+  padding-left: 1.25rem;
+}
+
+.prose li {
+  margin-bottom: 0.5rem;
+  color: var(--muted);
+  font-size: 0.9375rem;
+  line-height: 1.6;
+}
+
+.prose li strong {
+  color: #ccc;
+  font-weight: 500;
+}
+
+.prose a {
+  color: var(--foreground);
+  text-decoration: underline;
+  text-underline-offset: 2px;
+}
+
+.prose a:hover {
+  color: #fff;
+}
+
+.prose table {
+  width: 100%;
+  border-collapse: collapse;
+  margin: 1.5rem 0;
+  font-size: 0.8125rem;
+}
+
+.prose th, .prose td {
+  text-align: left;
+  padding: 0.625rem 0.875rem;
+  border-bottom: 1px solid var(--border);
+}
+
+.prose th {
+  font-weight: 500;
+  color: var(--muted);
+  text-transform: uppercase;
+  font-size: 0.75rem;
+  letter-spacing: 0.025em;
+}
+
+.prose td {
+  color: var(--muted);
+}
+
+.prose td code {
+  color: var(--foreground);
+}

docs/src/app/installation/page.tsx

@@ -0,0 +1,58 @@
+import { CodeBlock } from "@/components/code-block";
+
+export default function Installation() {
+  return (
+    <div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
+      <div className="prose">
+        <h1>Installation</h1>
+
+        <h2>npm (recommended)</h2>
+        <CodeBlock code={`npm install -g agent-browser
+agent-browser install  # Download Chromium`} />
+
+        <h2>From source</h2>
+        <CodeBlock code={`git clone https://github.com/vercel-labs/agent-browser
+cd agent-browser
+pnpm install
+pnpm build
+pnpm build:native
+./bin/agent-browser install
+pnpm link --global`} />
+
+        <h2>Linux dependencies</h2>
+        <p>On Linux, install system dependencies:</p>
+        <CodeBlock code={`agent-browser install --with-deps
+# or manually: npx playwright install-deps chromium`} />
+
+        <h2>Custom browser</h2>
+        <p>
+          Use a custom browser executable instead of bundled Chromium:
+        </p>
+        <ul>
+          <li><strong>Serverless</strong> - Use <code>@sparticuz/chromium</code> (~50MB vs ~684MB)</li>
+          <li><strong>System browser</strong> - Use existing Chrome installation</li>
+          <li><strong>Custom builds</strong> - Use modified browser builds</li>
+        </ul>
+
+        <CodeBlock code={`# Via flag
+agent-browser --executable-path /path/to/chromium open example.com
+
+# Via environment variable
+AGENT_BROWSER_EXECUTABLE_PATH=/path/to/chromium agent-browser open example.com`} />
+
+        <h3>Serverless example</h3>
+        <CodeBlock lang="typescript" code={`import chromium from '@sparticuz/chromium';
+import { BrowserManager } from 'agent-browser';
+
+export async function handler() {
+  const browser = new BrowserManager();
+  await browser.launch({
+    executablePath: await chromium.executablePath(),
+    headless: true,
+  });
+  // ... use browser
+}`} />
+      </div>
+    </div>
+  );
+}

docs/src/app/layout.tsx

@@ -0,0 +1,40 @@
+import type { Metadata } from "next";
+import { Geist, Geist_Mono } from "next/font/google";
+import "./globals.css";
+import { Sidebar } from "@/components/sidebar";
+
+const geist = Geist({
+  variable: "--font-geist",
+  subsets: ["latin"],
+});
+
+const geistMono = Geist_Mono({
+  variable: "--font-geist-mono",
+  subsets: ["latin"],
+});
+
+export const metadata: Metadata = {
+  title: "agent-browser",
+  description: "Headless browser automation CLI for AI agents",
+};
+
+export default function RootLayout({
+  children,
+}: Readonly<{
+  children: React.ReactNode;
+}>) {
+  return (
+    <html lang="en" className="dark">
+      <body
+        className={`${geist.variable} ${geistMono.variable} antialiased bg-zinc-950 text-zinc-100`}
+      >
+        <div className="flex min-h-screen">
+          <Sidebar />
+          <main className="flex-1 overflow-auto pt-14 lg:pt-0">
+            {children}
+          </main>
+        </div>
+      </body>
+    </html>
+  );
+}

docs/src/app/page.tsx

@@ -0,0 +1,69 @@
+import { CodeBlock } from "@/components/code-block";
+
+export default function Home() {
+  return (
+    <div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
+      <div className="prose">
+        <h1>agent-browser</h1>
+        <p>
+          Headless browser automation CLI for AI agents. Fast Rust CLI with Node.js fallback.
+        </p>
+
+        <CodeBlock code="npm install -g agent-browser" />
+
+        <h2>Features</h2>
+        <ul>
+          <li><strong>Universal</strong> - Works with any AI agent: Claude Code, Cursor, Codex, Copilot, Gemini, opencode, and more</li>
+          <li><strong>AI-first</strong> - Snapshot returns accessibility tree with refs for deterministic element selection</li>
+          <li><strong>Fast</strong> - Native Rust CLI for instant command parsing</li>
+          <li><strong>Complete</strong> - 50+ commands for navigation, forms, screenshots, network, storage</li>
+          <li><strong>Sessions</strong> - Multiple isolated browser instances with separate auth</li>
+          <li><strong>Cross-platform</strong> - macOS, Linux, Windows with native binaries</li>
+          <li><strong>Serverless</strong> - Custom executable path for lightweight Chromium builds</li>
+        </ul>
+
+        <h2>Example</h2>
+        <CodeBlock code={`# Navigate and get snapshot
+agent-browser open example.com
+agent-browser snapshot -i
+
+# Output:
+# - heading "Example Domain" [ref=e1]
+# - link "More information..." [ref=e2]
+
+# Interact using refs
+agent-browser click @e2
+agent-browser screenshot page.png
+agent-browser close`} />
+
+        <h2>Why refs?</h2>
+        <p>
+          The <code>snapshot</code> command returns an accessibility tree where each element 
+          has a unique ref like <code>@e1</code>, <code>@e2</code>. This provides:
+        </p>
+        <ul>
+          <li><strong>Deterministic</strong> - Ref points to exact element from snapshot</li>
+          <li><strong>Fast</strong> - No DOM re-query needed</li>
+          <li><strong>AI-friendly</strong> - LLMs can reliably parse and use refs</li>
+        </ul>
+
+        <h2>Architecture</h2>
+        <p>
+          Client-daemon architecture for optimal performance:
+        </p>
+        <ol>
+          <li><strong>Rust CLI</strong> - Parses commands, communicates with daemon</li>
+          <li><strong>Node.js Daemon</strong> - Manages Playwright browser instance</li>
+        </ol>
+        <p>
+          Daemon starts automatically and persists between commands.
+        </p>
+
+        <h2>Platforms</h2>
+        <p>
+          Native Rust binaries for macOS (ARM64, x64), Linux (ARM64, x64), and Windows (x64).
+        </p>
+      </div>
+    </div>
+  );
+}

docs/src/app/quick-start/page.tsx

@@ -0,0 +1,50 @@
+import { CodeBlock } from "@/components/code-block";
+
+export default function QuickStart() {
+  return (
+    <div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
+      <div className="prose">
+        <h1>Quick Start</h1>
+
+        <h2>Basic workflow</h2>
+        <CodeBlock code={`agent-browser open example.com
+agent-browser snapshot                    # Get accessibility tree with refs
+agent-browser click @e2                   # Click by ref from snapshot
+agent-browser fill @e3 "test@example.com" # Fill by ref
+agent-browser get text @e1                # Get text by ref
+agent-browser screenshot page.png
+agent-browser close`} />
+
+        <h2>Traditional selectors</h2>
+        <p>CSS selectors and semantic locators also supported:</p>
+        <CodeBlock code={`agent-browser click "#submit"
+agent-browser fill "#email" "test@example.com"
+agent-browser find role button click --name "Submit"`} />
+
+        <h2>AI workflow</h2>
+        <p>Optimal workflow for AI agents:</p>
+        <CodeBlock code={`# 1. Navigate and get snapshot
+agent-browser open example.com
+agent-browser snapshot -i --json   # AI parses tree and refs
+
+# 2. AI identifies target refs from snapshot
+# 3. Execute actions using refs
+agent-browser click @e2
+agent-browser fill @e3 "input text"
+
+# 4. Get new snapshot if page changed
+agent-browser snapshot -i --json`} />
+
+        <h2>Headed mode</h2>
+        <p>Show browser window for debugging:</p>
+        <CodeBlock code="agent-browser open example.com --headed" />
+
+        <h2>JSON output</h2>
+        <p>Use <code>--json</code> for machine-readable output:</p>
+        <CodeBlock code={`agent-browser snapshot --json
+agent-browser get text @e1 --json
+agent-browser is visible @e2 --json`} />
+      </div>
+    </div>
+  );
+}

docs/src/app/selectors/page.tsx

@@ -0,0 +1,53 @@
+import { CodeBlock } from "@/components/code-block";
+
+export default function Selectors() {
+  return (
+    <div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
+      <div className="prose">
+        <h1>Selectors</h1>
+
+        <h2>Refs (recommended)</h2>
+        <p>
+          Refs provide deterministic element selection from snapshots. Best for AI agents.
+        </p>
+        <CodeBlock code={`# 1. Get snapshot with refs
+agent-browser snapshot
+# Output:
+# - heading "Example Domain" [ref=e1] [level=1]
+# - button "Submit" [ref=e2]
+# - textbox "Email" [ref=e3]
+# - link "Learn more" [ref=e4]
+
+# 2. Use refs to interact
+agent-browser click @e2                   # Click the button
+agent-browser fill @e3 "test@example.com" # Fill the textbox
+agent-browser get text @e1                # Get heading text
+agent-browser hover @e4                   # Hover the link`} />
+
+        <h3>Why refs?</h3>
+        <ul>
+          <li><strong>Deterministic</strong> - Ref points to exact element from snapshot</li>
+          <li><strong>Fast</strong> - No DOM re-query needed</li>
+          <li><strong>AI-friendly</strong> - LLMs can reliably parse and use refs</li>
+        </ul>
+
+        <h2>CSS selectors</h2>
+        <CodeBlock code={`agent-browser click "#id"
+agent-browser click ".class"
+agent-browser click "div > button"
+agent-browser click "[data-testid='submit']"`} />
+
+        <h2>Text & XPath</h2>
+        <CodeBlock code={`agent-browser click "text=Submit"
+agent-browser click "xpath=//button[@type='submit']"`} />
+
+        <h2>Semantic locators</h2>
+        <p>Find elements by role, label, or other semantic properties:</p>
+        <CodeBlock code={`agent-browser find role button click --name "Submit"
+agent-browser find label "Email" fill "test@test.com"
+agent-browser find placeholder "Search..." fill "query"
+agent-browser find testid "submit-btn" click`} />
+      </div>
+    </div>
+  );
+}

docs/src/app/sessions/page.tsx

@@ -0,0 +1,66 @@
+import { CodeBlock } from "@/components/code-block";
+
+export default function Sessions() {
+  return (
+    <div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
+      <div className="prose">
+        <h1>Sessions</h1>
+        <p>Run multiple isolated browser instances:</p>
+        <CodeBlock code={`# Different sessions
+agent-browser --session agent1 open site-a.com
+agent-browser --session agent2 open site-b.com
+
+# Or via environment variable
+AGENT_BROWSER_SESSION=agent1 agent-browser click "#btn"
+
+# List active sessions
+agent-browser session list
+# Output:
+# Active sessions:
+# -> default
+#    agent1
+
+# Show current session
+agent-browser session`} />
+
+        <h2>Session isolation</h2>
+        <p>Each session has its own:</p>
+        <ul>
+          <li>Browser instance</li>
+          <li>Cookies and storage</li>
+          <li>Navigation history</li>
+          <li>Authentication state</li>
+        </ul>
+
+        <h2>Authenticated sessions</h2>
+        <p>
+          Use <code>--headers</code> to set HTTP headers for a specific origin:
+        </p>
+        <CodeBlock code={`# Headers scoped to api.example.com only
+agent-browser open api.example.com --headers '{"Authorization": "Bearer <token>"}'
+
+# Requests to api.example.com include the auth header
+agent-browser snapshot -i --json
+agent-browser click @e2
+
+# Navigate to another domain - headers NOT sent
+agent-browser open other-site.com`} />
+        <p>Useful for:</p>
+        <ul>
+          <li><strong>Skipping login flows</strong> - Authenticate via headers</li>
+          <li><strong>Switching users</strong> - Different auth tokens per session</li>
+          <li><strong>API testing</strong> - Access protected endpoints</li>
+          <li><strong>Security</strong> - Headers scoped to origin, not leaked</li>
+        </ul>
+
+        <h2>Multiple origins</h2>
+        <CodeBlock code={`agent-browser open api.example.com --headers '{"Authorization": "Bearer token1"}'
+agent-browser open api.acme.com --headers '{"Authorization": "Bearer token2"}'`} />
+
+        <h2>Global headers</h2>
+        <p>For headers on all domains:</p>
+        <CodeBlock code={`agent-browser set headers '{"X-Custom-Header": "value"}'`} />
+      </div>
+    </div>
+  );
+}

docs/src/app/snapshots/page.tsx

@@ -0,0 +1,71 @@
+import { CodeBlock } from "@/components/code-block";
+
+export default function Snapshots() {
+  return (
+    <div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
+      <div className="prose">
+        <h1>Snapshots</h1>
+        <p>
+          The <code>snapshot</code> command returns the accessibility tree with refs for AI-friendly interaction.
+        </p>
+
+        <h2>Options</h2>
+        <p>Filter output to reduce size:</p>
+        <CodeBlock code={`agent-browser snapshot                    # Full accessibility tree
+agent-browser snapshot -i                 # Interactive elements only
+agent-browser snapshot -c                 # Compact (remove empty elements)
+agent-browser snapshot -d 3               # Limit depth to 3 levels
+agent-browser snapshot -s "#main"         # Scope to CSS selector
+agent-browser snapshot -i -c -d 5         # Combine options`} />
+
+        <table>
+          <thead>
+            <tr>
+              <th>Option</th>
+              <th>Description</th>
+            </tr>
+          </thead>
+          <tbody>
+            <tr>
+              <td><code>-i, --interactive</code></td>
+              <td>Only interactive elements (buttons, links, inputs)</td>
+            </tr>
+            <tr>
+              <td><code>-c, --compact</code></td>
+              <td>Remove empty structural elements</td>
+            </tr>
+            <tr>
+              <td><code>-d, --depth</code></td>
+              <td>Limit tree depth</td>
+            </tr>
+            <tr>
+              <td><code>-s, --selector</code></td>
+              <td>Scope to CSS selector</td>
+            </tr>
+          </tbody>
+        </table>
+
+        <h2>Output format</h2>
+        <CodeBlock code={`agent-browser snapshot
+# Output:
+# - heading "Example Domain" [ref=e1] [level=1]
+# - button "Submit" [ref=e2]
+# - textbox "Email" [ref=e3]
+# - link "Learn more" [ref=e4]`} />
+
+        <h2>JSON output</h2>
+        <p>Use <code>--json</code> for machine-readable output:</p>
+        <CodeBlock code={`agent-browser snapshot --json
+# {"success":true,"data":{"snapshot":"...","refs":{"e1":{"role":"heading","name":"Title"},...}}}`} />
+
+        <h2>Best practices</h2>
+        <ol>
+          <li>Use <code>-i</code> to reduce output to actionable elements</li>
+          <li>Use <code>--json</code> for structured parsing</li>
+          <li>Re-snapshot after page changes to get updated refs</li>
+          <li>Scope with <code>-s</code> for specific page sections</li>
+        </ol>
+      </div>
+    </div>
+  );
+}

docs/src/app/streaming/page.tsx

@@ -0,0 +1,217 @@
+import { CodeBlock } from "@/components/code-block";
+
+export default function Streaming() {
+  return (
+    <div className="max-w-2xl mx-auto px-4 sm:px-6 py-8 sm:py-12">
+      <div className="prose">
+        <h1>Streaming</h1>
+        <p>
+          Stream the browser viewport via WebSocket for live preview or &quot;pair browsing&quot;
+          where a human can watch and interact alongside an AI agent.
+        </p>
+
+        <h2>Enable streaming</h2>
+        <p>
+          Set the <code>AGENT_BROWSER_STREAM_PORT</code> environment variable to start
+          a WebSocket server:
+        </p>
+        <CodeBlock code={`AGENT_BROWSER_STREAM_PORT=9223 agent-browser open example.com`} />
+
+        <p>
+          The server streams viewport frames and accepts input events (mouse, keyboard, touch).
+        </p>
+
+        <h2>WebSocket protocol</h2>
+        <p>Connect to <code>ws://localhost:9223</code> to receive frames and send input.</p>
+
+        <h3>Frame messages</h3>
+        <p>The server sends frame messages with base64-encoded images:</p>
+        <CodeBlock code={`{
+  "type": "frame",
+  "data": "<base64-encoded-jpeg>",
+  "metadata": {
+    "deviceWidth": 1280,
+    "deviceHeight": 720,
+    "pageScaleFactor": 1,
+    "offsetTop": 0,
+    "scrollOffsetX": 0,
+    "scrollOffsetY": 0
+  }
+}`} />
+
+        <h3>Status messages</h3>
+        <p>Connection and screencast status:</p>
+        <CodeBlock code={`{
+  "type": "status",
+  "connected": true,
+  "screencasting": true,
+  "viewportWidth": 1280,
+  "viewportHeight": 720
+}`} />
+
+        <h2>Input injection</h2>
+        <p>Send input events to control the browser remotely.</p>
+
+        <h3>Mouse events</h3>
+        <CodeBlock code={`// Click
+{
+  "type": "input_mouse",
+  "eventType": "mousePressed",
+  "x": 100,
+  "y": 200,
+  "button": "left",
+  "clickCount": 1
+}
+
+// Release
+{
+  "type": "input_mouse",
+  "eventType": "mouseReleased",
+  "x": 100,
+  "y": 200,
+  "button": "left"
+}
+
+// Move
+{
+  "type": "input_mouse",
+  "eventType": "mouseMoved",
+  "x": 150,
+  "y": 250
+}
+
+// Scroll
+{
+  "type": "input_mouse",
+  "eventType": "mouseWheel",
+  "x": 100,
+  "y": 200,
+  "deltaX": 0,
+  "deltaY": 100
+}`} />
+
+        <h3>Keyboard events</h3>
+        <CodeBlock code={`// Key down
+{
+  "type": "input_keyboard",
+  "eventType": "keyDown",
+  "key": "Enter",
+  "code": "Enter"
+}
+
+// Key up
+{
+  "type": "input_keyboard",
+  "eventType": "keyUp",
+  "key": "Enter",
+  "code": "Enter"
+}
+
+// Type character
+{
+  "type": "input_keyboard",
+  "eventType": "char",
+  "text": "a"
+}
+
+// With modifiers (1=Alt, 2=Ctrl, 4=Meta, 8=Shift)
+{
+  "type": "input_keyboard",
+  "eventType": "keyDown",
+  "key": "c",
+  "code": "KeyC",
+  "modifiers": 2
+}`} />
+
+        <h3>Touch events</h3>
+        <CodeBlock code={`// Touch start
+{
+  "type": "input_touch",
+  "eventType": "touchStart",
+  "touchPoints": [{ "x": 100, "y": 200 }]
+}
+
+// Touch move
+{
+  "type": "input_touch",
+  "eventType": "touchMove",
+  "touchPoints": [{ "x": 150, "y": 250 }]
+}
+
+// Touch end
+{
+  "type": "input_touch",
+  "eventType": "touchEnd",
+  "touchPoints": []
+}
+
+// Multi-touch (pinch zoom)
+{
+  "type": "input_touch",
+  "eventType": "touchStart",
+  "touchPoints": [
+    { "x": 100, "y": 200, "id": 0 },
+    { "x": 200, "y": 200, "id": 1 }
+  ]
+}`} />
+
+        <h2>Programmatic API</h2>
+        <p>For advanced use, control streaming directly via the TypeScript API:</p>
+        <CodeBlock code={`import { BrowserManager } from 'agent-browser';
+
+const browser = new BrowserManager();
+await browser.launch({ headless: true });
+await browser.navigate('https://example.com');
+
+// Start screencast with callback
+await browser.startScreencast((frame) => {
+  console.log('Frame:', frame.metadata.deviceWidth, 'x', frame.metadata.deviceHeight);
+  // frame.data is base64-encoded image
+}, {
+  format: 'jpeg',  // or 'png'
+  quality: 80,     // 0-100, jpeg only
+  maxWidth: 1280,
+  maxHeight: 720,
+  everyNthFrame: 1
+});
+
+// Inject mouse event
+await browser.injectMouseEvent({
+  type: 'mousePressed',
+  x: 100,
+  y: 200,
+  button: 'left',
+  clickCount: 1
+});
+
+// Inject keyboard event
+await browser.injectKeyboardEvent({
+  type: 'keyDown',
+  key: 'Enter',
+  code: 'Enter'
+});
+
+// Inject touch event
+await browser.injectTouchEvent({
+  type: 'touchStart',
+  touchPoints: [{ x: 100, y: 200 }]
+});
+
+// Check if screencasting
+console.log('Active:', browser.isScreencasting());
+
+// Stop screencast
+await browser.stopScreencast();`} />
+
+        <h2>Use cases</h2>
+        <ul>
+          <li><strong>Pair browsing</strong> - Human watches and assists AI agent in real-time</li>
+          <li><strong>Remote preview</strong> - View browser output in a separate UI</li>
+          <li><strong>Recording</strong> - Capture frames for video generation</li>
+          <li><strong>Mobile testing</strong> - Inject touch events for mobile emulation</li>
+          <li><strong>Accessibility testing</strong> - Manual interaction during automated tests</li>
+        </ul>
+      </div>
+    </div>
+  );
+}

docs/src/components/code-block.tsx

@@ -0,0 +1,22 @@
+import { codeToHtml } from "shiki";
+import { CopyButton } from "./copy-button";
+
+interface CodeBlockProps {
+  code: string;
+  lang?: string;
+}
+
+export async function CodeBlock({ code, lang = "bash" }: CodeBlockProps) {
+  const trimmedCode = code.trim();
+  const html = await codeToHtml(trimmedCode, {
+    lang,
+    theme: "github-dark-default",
+  });
+
+  return (
+    <div className="code-block relative group">
+      <CopyButton code={trimmedCode} />
+      <div dangerouslySetInnerHTML={{ __html: html }} />
+    </div>
+  );
+}

docs/src/components/copy-button.tsx

@@ -0,0 +1,40 @@
+"use client";
+
+import { useState } from "react";
+
+interface CopyButtonProps {
+  code: string;
+}
+
+export function CopyButton({ code }: CopyButtonProps) {
+  const [copied, setCopied] = useState(false);
+
+  const handleCopy = async () => {
+    try {
+      await navigator.clipboard.writeText(code);
+      setCopied(true);
+      setTimeout(() => setCopied(false), 2000);
+    } catch (error) {
+      console.error("Failed to copy to clipboard:", error);
+      // Optionally, you could set an error state or show a toast notification here
+    }
+  };
+
+  return (
+    <button
+      onClick={handleCopy}
+      className="absolute top-2 right-2 p-1.5 rounded text-[#666] hover:text-[#999] hover:bg-[#333] opacity-0 group-hover:opacity-100 transition-all"
+      aria-label="Copy code"
+    >
+      {copied ? (
+        <svg className="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
+          <path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M5 13l4 4L19 7" />
+        </svg>
+      ) : (
+        <svg className="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
+          <path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M8 16H6a2 2 0 01-2-2V6a2 2 0 012-2h8a2 2 0 012 2v2m-6 12h8a2 2 0 002-2v-8a2 2 0 00-2-2h-8a2 2 0 00-2 2v8a2 2 0 002 2z" />
+        </svg>
+      )}
+    </button>
+  );
+}

docs/src/components/sidebar.tsx

@@ -0,0 +1,131 @@
+"use client";
+
+import Link from "next/link";
+import { usePathname } from "next/navigation";
+import { useState, useEffect } from "react";
+
+const navigation = [
+  { name: "Introduction", href: "/" },
+  { name: "Installation", href: "/installation" },
+  { name: "Quick Start", href: "/quick-start" },
+  { name: "Commands", href: "/commands" },
+  { name: "Selectors", href: "/selectors" },
+  { name: "Sessions", href: "/sessions" },
+  { name: "Snapshots", href: "/snapshots" },
+  { name: "Streaming", href: "/streaming" },
+  { name: "Agent Mode", href: "/agent-mode" },
+  { name: "CDP Mode", href: "/cdp-mode" },
+];
+
+export function Sidebar() {
+  const pathname = usePathname();
+  const [isOpen, setIsOpen] = useState(false);
+
+  useEffect(() => {
+    setIsOpen(false);
+  }, [pathname]);
+
+  useEffect(() => {
+    const handleEscape = (e: KeyboardEvent) => {
+      if (e.key === "Escape") setIsOpen(false);
+    };
+    document.addEventListener("keydown", handleEscape);
+    return () => document.removeEventListener("keydown", handleEscape);
+  }, []);
+
+  return (
+    <>
+      {/* Mobile header */}
+      <header className="lg:hidden fixed top-0 left-0 right-0 z-50 bg-black/90 backdrop-blur-sm border-b border-[#222] px-4 py-3">
+        <div className="flex items-center justify-between">
+          <Link href="/" className="text-sm font-medium">
+            agent-browser
+          </Link>
+          <button
+            onClick={() => setIsOpen(!isOpen)}
+            className="p-2 -mr-2 text-[#888] hover:text-white transition-colors"
+            aria-label="Toggle menu"
+          >
+            {isOpen ? (
+              <svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
+                <path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M6 18L18 6M6 6l12 12" />
+              </svg>
+            ) : (
+              <svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
+                <path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M4 6h16M4 12h16M4 18h16" />
+              </svg>
+            )}
+          </button>
+        </div>
+      </header>
+
+      {/* Mobile overlay */}
+      {isOpen && (
+        <div
+          className="lg:hidden fixed inset-0 z-40 bg-black/80"
+          onClick={() => setIsOpen(false)}
+        />
+      )}
+
+      {/* Sidebar */}
+      <aside
+        className={`
+          fixed lg:sticky top-0 left-0 z-50 lg:z-auto
+          w-56 lg:w-48 h-screen
+          bg-black border-r border-[#222]
+          transform transition-transform duration-150 ease-out
+          ${isOpen ? "translate-x-0" : "-translate-x-full lg:translate-x-0"}
+          pt-14 lg:pt-0
+        `}
+      >
+        <div className="h-full overflow-y-auto p-5">
+          {/* Desktop header */}
+          <div className="mb-8 hidden lg:block">
+            <Link href="/" className="text-sm font-medium">
+              agent-browser
+            </Link>
+          </div>
+
+          <nav className="space-y-0.5">
+            {navigation.map((item) => {
+              const isActive = pathname === item.href;
+
+              return (
+                <Link
+                  key={item.name}
+                  href={item.href}
+                  className={`block px-2 py-1.5 text-[13px] transition-colors ${
+                    isActive
+                      ? "text-white"
+                      : "text-[#666] hover:text-[#999]"
+                  }`}
+                >
+                  {item.name}
+                </Link>
+              );
+            })}
+          </nav>
+
+          <div className="mt-8 pt-4 border-t border-[#222] space-y-0.5">
+            <a
+              href="https://github.com/vercel-labs/agent-browser"
+              target="_blank"
+              rel="noopener noreferrer"
+              className="block px-2 py-1.5 text-[13px] text-[#666] hover:text-[#999] transition-colors"
+            >
+              GitHub
+            </a>
+            <a
+              href="https://www.npmjs.com/package/agent-browser"
+              target="_blank"
+              rel="noopener noreferrer"
+              className="block px-2 py-1.5 text-[13px] text-[#666] hover:text-[#999] transition-colors"
+            >
+              npm
+            </a>
+          </div>
+        </div>
+      </aside>
+    </>
+  );
+}

docs/tsconfig.json

@@ -0,0 +1,34 @@
+{
+  "compilerOptions": {
+    "target": "ES2017",
+    "lib": ["dom", "dom.iterable", "esnext"],
+    "allowJs": true,
+    "skipLibCheck": true,
+    "strict": true,
+    "noEmit": true,
+    "esModuleInterop": true,
+    "module": "esnext",
+    "moduleResolution": "bundler",
+    "resolveJsonModule": true,
+    "isolatedModules": true,
+    "jsx": "react-jsx",
+    "incremental": true,
+    "plugins": [
+      {
+        "name": "next"
+      }
+    ],
+    "paths": {
+      "@/*": ["./src/*"]
+    }
+  },
+  "include": [
+    "next-env.d.ts",
+    "**/*.ts",
+    "**/*.tsx",
+    ".next/types/**/*.ts",
+    ".next/dev/types/**/*.ts",
+    "**/*.mts"
+  ],
+  "exclude": ["node_modules"]
+}

package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-browser",
-  "version": "0.4.3",
+  "version": "0.5.0",
   "description": "Headless browser automation CLI for AI agents",
   "type": "module",
   "main": "dist/daemon.js",
@@ -53,10 +53,12 @@
   "homepage": "https://github.com/vercel-labs/agent-browser#readme",
   "dependencies": {
     "playwright-core": "^1.57.0",
+    "ws": "^8.19.0",
     "zod": "^3.22.4"
   },
   "devDependencies": {
     "@types/node": "^20.10.0",
+    "@types/ws": "^8.18.1",
     "husky": "^9.1.7",
     "lint-staged": "^15.2.11",
     "playwright": "^1.57.0",

pnpm-lock.yaml

@@ -11,6 +11,9 @@ importers:
       playwright-core:
         specifier: ^1.57.0
         version: 1.57.0
+      ws:
+        specifier: ^8.19.0
+        version: 8.19.0
       zod:
         specifier: ^3.22.4
         version: 3.25.76
@@ -18,6 +21,9 @@ importers:
       '@types/node':
         specifier: ^20.10.0
         version: 20.19.28
+      '@types/ws':
+        specifier: ^8.18.1
+        version: 8.18.1
       husky:
         specifier: ^9.1.7
         version: 9.1.7
@@ -341,6 +347,9 @@ packages:
   '@types/node@20.19.28':
     resolution: {integrity: sha512-VyKBr25BuFDzBFCK5sUM6ZXiWfqgCTwTAOK8qzGV/m9FCirXYDlmczJ+d5dXBAQALGCdRRdbteKYfJ84NGEusw==}
 
+  '@types/ws@8.18.1':
+    resolution: {integrity: sha512-ThVF6DCVhA8kUGy+aazFQ4kXQ7E1Ty7A3ypFOe0IcJV8O/M511G99AW24irKrW56Wt44yG9+ij8FaqoBGkuBXg==}
+
   '@vitest/expect@4.0.16':
     resolution: {integrity: sha512-eshqULT2It7McaJkQGLkPjPjNph+uevROGuIMJdG3V+0BSR2w9u6J9Lwu+E8cK5TETlfou8GRijhafIMhXsimA==}
 
@@ -805,6 +814,18 @@ packages:
     resolution: {integrity: sha512-42AtmgqjV+X1VpdOfyTGOYRi0/zsoLqtXQckTmqTeybT+BDIbM/Guxo7x3pE2vtpr1ok6xRqM9OpBe+Jyoqyww==}
     engines: {node: '>=18'}
 
+  ws@8.19.0:
+    resolution: {integrity: sha512-blAT2mjOEIi0ZzruJfIhb3nps74PRWTCz1IjglWEEpQl5XS/UNama6u2/rjFkDDouqr4L67ry+1aGIALViWjDg==}
+    engines: {node: '>=10.0.0'}
+    peerDependencies:
+      bufferutil: ^4.0.1
+      utf-8-validate: '>=5.0.2'
+    peerDependenciesMeta:
+      bufferutil:
+        optional: true
+      utf-8-validate:
+        optional: true
+
   yaml@2.8.2:
     resolution: {integrity: sha512-mplynKqc1C2hTVYxd0PU2xQAc22TI1vShAYGksCCfxbn/dFwnHTNi1bvYsBTkhdUNtGIf5xNOg938rrSSYvS9A==}
     engines: {node: '>= 14.6'}
@@ -985,6 +1006,10 @@ snapshots:
     dependencies:
       undici-types: 6.21.0
 
+  '@types/ws@8.18.1':
+    dependencies:
+      '@types/node': 20.19.28
+
   '@vitest/expect@4.0.16':
     dependencies:
       '@standard-schema/spec': 1.1.0
@@ -1427,6 +1452,8 @@ snapshots:
       string-width: 7.2.0
       strip-ansi: 7.1.2
 
+  ws@8.19.0: {}
+
   yaml@2.8.2: {}
 
   zod@3.25.76: {}

src/actions.test.ts

@@ -0,0 +1,39 @@
+import { describe, it, expect } from 'vitest';
+import { toAIFriendlyError } from './actions.js';
+
+describe('toAIFriendlyError', () => {
+  describe('element blocked by overlay', () => {
+    it('should detect intercepts pointer events even when Timeout is in message', () => {
+      // This is the exact error from Playwright when a cookie banner blocks an element
+      // Bug: Previously this was incorrectly reported as "not found or not visible"
+      const error = new Error(
+        'TimeoutError: locator.click: Timeout 10000ms exceeded.\n' +
+          'Call log:\n' +
+          "  - waiting for getByRole('link', { name: 'Anmelden', exact: true }).first()\n" +
+          '    - locator resolved to <a href="https://example.com/login">Anmelden</a>\n' +
+          '  - attempting click action\n' +
+          '    2 x waiting for element to be visible, enabled and stable\n' +
+          '      - element is visible, enabled and stable\n' +
+          '      - scrolling into view if needed\n' +
+          '      - done scrolling\n' +
+          '      - <body class="font-sans antialiased">...</body> intercepts pointer events\n' +
+          '    - retrying click action'
+      );
+
+      const result = toAIFriendlyError(error, '@e4');
+
+      // Must NOT say "not found" - the element WAS found
+      expect(result.message).not.toContain('not found');
+      // Must indicate the element is blocked
+      expect(result.message).toContain('blocked by another element');
+      expect(result.message).toContain('modal or overlay');
+    });
+
+    it('should suggest dismissing cookie banners', () => {
+      const error = new Error('<div class="cookie-overlay"> intercepts pointer events');
+      const result = toAIFriendlyError(error, '@e1');
+
+      expect(result.message).toContain('cookie banners');
+    });
+  });
+});

src/actions.ts

@@ -1,5 +1,5 @@
 import type { Page, Frame } from 'playwright-core';
-import type { BrowserManager } from './browser.js';
+import type { BrowserManager, ScreencastFrame } from './browser.js';
 import type {
   Command,
   Response,
@@ -94,6 +94,11 @@ import type {
   MultiSelectCommand,
   WaitForDownloadCommand,
   ResponseBodyCommand,
+  ScreencastStartCommand,
+  ScreencastStopCommand,
+  InputMouseCommand,
+  InputKeyboardCommand,
+  InputTouchCommand,
   NavigateData,
   ScreenshotData,
   EvaluateData,
@@ -102,9 +107,25 @@ import type {
   TabNewData,
   TabSwitchData,
   TabCloseData,
+  ScreencastStartData,
+  ScreencastStopData,
+  InputEventData,
 } from './types.js';
 import { successResponse, errorResponse } from './protocol.js';
 
+// Callback for screencast frames - will be set by the daemon when streaming is active
+let screencastFrameCallback: ((frame: ScreencastFrame) => void) | null = null;
+
+/**
+ * Set the callback for screencast frames
+ * This is called by the daemon to set up frame streaming
+ */
+export function setScreencastFrameCallback(
+  callback: ((frame: ScreencastFrame) => void) | null
+): void {
+  screencastFrameCallback = callback;
+}
+
 // Snapshot response type
 interface SnapshotData {
   snapshot: string;
@@ -113,8 +134,9 @@ interface SnapshotData {
 
 /**
  * Convert Playwright errors to AI-friendly messages
+ * @internal Exported for testing
  */
-function toAIFriendlyError(error: unknown, selector: string): Error {
+export function toAIFriendlyError(error: unknown, selector: string): Error {
   const message = error instanceof Error ? error.message : String(error);
 
   // Handle strict mode violation (multiple elements match)
@@ -129,7 +151,24 @@ function toAIFriendlyError(error: unknown, selector: string): Error {
     );
   }
 
-  // Handle element not found
+  // Handle element not interactable (must be checked BEFORE timeout case)
+  // This includes cases where an overlay/modal blocks the element
+  if (message.includes('intercepts pointer events')) {
+    return new Error(
+      `Element "${selector}" is blocked by another element (likely a modal or overlay). ` +
+        `Try dismissing any modals/cookie banners first.`
+    );
+  }
+
+  // Handle element not visible
+  if (message.includes('not visible') && !message.includes('Timeout')) {
+    return new Error(
+      `Element "${selector}" is not visible. ` +
+        `Try scrolling it into view or check if it's hidden.`
+    );
+  }
+
+  // Handle element not found (timeout waiting for element)
   if (
     message.includes('waiting for') &&
     (message.includes('to be visible') || message.includes('Timeout'))
@@ -140,14 +179,6 @@ function toAIFriendlyError(error: unknown, selector: string): Error {
     );
   }
 
-  // Handle element not interactable
-  if (message.includes('intercepts pointer events') || message.includes('not visible')) {
-    return new Error(
-      `Element "${selector}" is not interactable (may be hidden or covered). ` +
-        `Try scrolling it into view or check if a modal/overlay is blocking it.`
-    );
-  }
-
   // Return original error for unknown cases
   return error instanceof Error ? error : new Error(message);
 }
@@ -386,6 +417,16 @@ export async function executeCommand(command: Command, browser: BrowserManager):
         return await handleWaitForDownload(command, browser);
       case 'responsebody':
         return await handleResponseBody(command, browser);
+      case 'screencast_start':
+        return await handleScreencastStart(command, browser);
+      case 'screencast_stop':
+        return await handleScreencastStop(command, browser);
+      case 'input_mouse':
+        return await handleInputMouse(command, browser);
+      case 'input_keyboard':
+        return await handleInputKeyboard(command, browser);
+      case 'input_touch':
+        return await handleInputTouch(command, browser);
       default: {
         // TypeScript narrows to never here, but we handle it for safety
         const unknownCommand = command as { id: string; action: string };
@@ -411,6 +452,12 @@ async function handleNavigate(
   browser: BrowserManager
 ): Promise<Response<NavigateData>> {
   const page = browser.getPage();
+
+  // If headers are provided, set up scoped headers for this origin
+  if (command.headers && Object.keys(command.headers).length > 0) {
+    await browser.setScopedHeaders(command.url, command.headers);
+  }
+
   await page.goto(command.url, {
     waitUntil: command.waitUntil ?? 'load',
   });
@@ -672,7 +719,7 @@ async function handleTabSwitch(
   command: TabSwitchCommand,
   browser: BrowserManager
 ): Promise<Response<TabSwitchData>> {
-  const result = browser.switchTo(command.index);
+  const result = await browser.switchTo(command.index);
   const page = browser.getPage();
   return successResponse(command.id, {
     ...result,
@@ -1763,3 +1810,79 @@ async function handleResponseBody(
     body: parsed,
   });
 }
+
+// Screencast and input injection handlers
+
+async function handleScreencastStart(
+  command: ScreencastStartCommand,
+  browser: BrowserManager
+): Promise<Response<ScreencastStartData>> {
+  if (!screencastFrameCallback) {
+    throw new Error('Screencast frame callback not set. Start the streaming server first.');
+  }
+
+  await browser.startScreencast(screencastFrameCallback, {
+    format: command.format,
+    quality: command.quality,
+    maxWidth: command.maxWidth,
+    maxHeight: command.maxHeight,
+    everyNthFrame: command.everyNthFrame,
+  });
+
+  return successResponse(command.id, {
+    started: true,
+    format: command.format ?? 'jpeg',
+    quality: command.quality ?? 80,
+  });
+}
+
+async function handleScreencastStop(
+  command: ScreencastStopCommand,
+  browser: BrowserManager
+): Promise<Response<ScreencastStopData>> {
+  await browser.stopScreencast();
+  return successResponse(command.id, { stopped: true });
+}
+
+async function handleInputMouse(
+  command: InputMouseCommand,
+  browser: BrowserManager
+): Promise<Response<InputEventData>> {
+  await browser.injectMouseEvent({
+    type: command.type,
+    x: command.x,
+    y: command.y,
+    button: command.button,
+    clickCount: command.clickCount,
+    deltaX: command.deltaX,
+    deltaY: command.deltaY,
+    modifiers: command.modifiers,
+  });
+  return successResponse(command.id, { injected: true });
+}
+
+async function handleInputKeyboard(
+  command: InputKeyboardCommand,
+  browser: BrowserManager
+): Promise<Response<InputEventData>> {
+  await browser.injectKeyboardEvent({
+    type: command.type,
+    key: command.key,
+    code: command.code,
+    text: command.text,
+    modifiers: command.modifiers,
+  });
+  return successResponse(command.id, { injected: true });
+}
+
+async function handleInputTouch(
+  command: InputTouchCommand,
+  browser: BrowserManager
+): Promise<Response<InputEventData>> {
+  await browser.injectTouchEvent({
+    type: command.type,
+    touchPoints: command.touchPoints,
+    modifiers: command.modifiers,
+  });
+  return successResponse(command.id, { injected: true });
+}

src/browser.test.ts

@@ -32,6 +32,25 @@ describe('BrowserManager', () => {
         })
       ).rejects.toThrow();
     });
+
+    it('should be no-op when relaunching with same options', async () => {
+      const browserInstance = browser.getBrowser();
+      await browser.launch({ id: 'test', action: 'launch', headless: true });
+      expect(browser.getBrowser()).toBe(browserInstance);
+    });
+
+    it('should reconnect when CDP port changes', async () => {
+      const newBrowser = new BrowserManager();
+      await newBrowser.launch({ id: 'test', action: 'launch', headless: true });
+      expect(newBrowser.getBrowser()).not.toBeNull();
+
+      await expect(
+        newBrowser.launch({ id: 'test', action: 'launch', cdpPort: 59999 })
+      ).rejects.toThrow();
+
+      expect(newBrowser.getBrowser()).toBeNull();
+      await newBrowser.close();
+    });
   });
 
   describe('navigation', () => {
@@ -304,4 +323,311 @@ describe('BrowserManager', () => {
       expect(h1).toBe('Example Domain');
     });
   });
+
+  describe('scoped headers', () => {
+    it('should register route for scoped headers', async () => {
+      // Test that setScopedHeaders doesn't throw and completes successfully
+      await browser.clearScopedHeaders();
+      await expect(
+        browser.setScopedHeaders('https://example.com', { 'X-Test': 'value' })
+      ).resolves.not.toThrow();
+      await browser.clearScopedHeaders();
+    });
+
+    it('should handle full URL origin', async () => {
+      await browser.clearScopedHeaders();
+      await expect(
+        browser.setScopedHeaders('https://api.example.com/path', { Authorization: 'Bearer token' })
+      ).resolves.not.toThrow();
+      await browser.clearScopedHeaders();
+    });
+
+    it('should handle hostname-only origin', async () => {
+      await browser.clearScopedHeaders();
+      await expect(
+        browser.setScopedHeaders('example.com', { 'X-Custom': 'value' })
+      ).resolves.not.toThrow();
+      await browser.clearScopedHeaders();
+    });
+
+    it('should clear scoped headers for specific origin', async () => {
+      await browser.clearScopedHeaders();
+      await browser.setScopedHeaders('https://example.com', { 'X-Test': 'value' });
+      await expect(browser.clearScopedHeaders('https://example.com')).resolves.not.toThrow();
+    });
+
+    it('should clear all scoped headers', async () => {
+      await browser.setScopedHeaders('https://example.com', { 'X-Test-1': 'value1' });
+      await browser.setScopedHeaders('https://example.org', { 'X-Test-2': 'value2' });
+      await expect(browser.clearScopedHeaders()).resolves.not.toThrow();
+    });
+
+    it('should replace headers when called twice for same origin', async () => {
+      await browser.clearScopedHeaders();
+      await browser.setScopedHeaders('https://example.com', { 'X-First': 'first' });
+      // Second call should replace, not add
+      await expect(
+        browser.setScopedHeaders('https://example.com', { 'X-Second': 'second' })
+      ).resolves.not.toThrow();
+      await browser.clearScopedHeaders();
+    });
+
+    it('should handle clearing non-existent origin gracefully', async () => {
+      await browser.clearScopedHeaders();
+      // Should not throw when clearing headers that were never set
+      await expect(browser.clearScopedHeaders('https://never-set.com')).resolves.not.toThrow();
+    });
+  });
+
+  describe('CDP session', () => {
+    it('should create CDP session on demand', async () => {
+      const cdp = await browser.getCDPSession();
+      expect(cdp).toBeDefined();
+    });
+
+    it('should reuse existing CDP session', async () => {
+      const cdp1 = await browser.getCDPSession();
+      const cdp2 = await browser.getCDPSession();
+      expect(cdp1).toBe(cdp2);
+    });
+  });
+
+  describe('screencast', () => {
+    it('should report screencasting state correctly', () => {
+      expect(browser.isScreencasting()).toBe(false);
+    });
+
+    it('should start screencast', async () => {
+      const frames: Array<{ data: string }> = [];
+      await browser.startScreencast((frame) => {
+        frames.push(frame);
+      });
+      expect(browser.isScreencasting()).toBe(true);
+
+      // Wait a bit for at least one frame
+      await new Promise((resolve) => setTimeout(resolve, 200));
+
+      await browser.stopScreencast();
+      expect(browser.isScreencasting()).toBe(false);
+      expect(frames.length).toBeGreaterThan(0);
+    });
+
+    it('should start screencast with custom options', async () => {
+      const frames: Array<{ data: string }> = [];
+      await browser.startScreencast(
+        (frame) => {
+          frames.push(frame);
+        },
+        {
+          format: 'png',
+          quality: 100,
+          maxWidth: 800,
+          maxHeight: 600,
+          everyNthFrame: 1,
+        }
+      );
+      expect(browser.isScreencasting()).toBe(true);
+
+      // Wait for a frame
+      await new Promise((resolve) => setTimeout(resolve, 200));
+
+      await browser.stopScreencast();
+      expect(frames.length).toBeGreaterThan(0);
+    });
+
+    it('should throw when starting screencast twice', async () => {
+      await browser.startScreencast(() => {});
+      await expect(browser.startScreencast(() => {})).rejects.toThrow('Screencast already active');
+      await browser.stopScreencast();
+    });
+
+    it('should handle stop when not screencasting', async () => {
+      // Should not throw
+      await expect(browser.stopScreencast()).resolves.not.toThrow();
+    });
+  });
+
+  describe('tab switch invalidates CDP session', () => {
+    // Clean up any extra tabs before each test
+    beforeEach(async () => {
+      // Close all tabs except the first one
+      const tabs = await browser.listTabs();
+      for (let i = tabs.length - 1; i > 0; i--) {
+        await browser.closeTab(i);
+      }
+      // Ensure we're on tab 0
+      await browser.switchTo(0);
+      // Stop any active screencast
+      if (browser.isScreencasting()) {
+        await browser.stopScreencast();
+      }
+    });
+
+    it('should not invalidate CDP when switching to same tab', async () => {
+      // Get CDP session for current tab
+      const cdp1 = await browser.getCDPSession();
+
+      // Switch to same tab - should NOT invalidate
+      await browser.switchTo(0);
+
+      // Should be the same session
+      const cdp2 = await browser.getCDPSession();
+      expect(cdp2).toBe(cdp1);
+    });
+
+    it('should invalidate CDP session on tab switch', async () => {
+      // Get CDP session for tab 0
+      const cdp1 = await browser.getCDPSession();
+      expect(cdp1).toBeDefined();
+
+      // Create new tab - this switches to the new tab automatically
+      await browser.newTab();
+
+      // Get CDP session - should be different since we're on a new page
+      const cdp2 = await browser.getCDPSession();
+      expect(cdp2).toBeDefined();
+
+      // Sessions should be different objects (different pages have different CDP sessions)
+      expect(cdp2).not.toBe(cdp1);
+    });
+
+    it('should stop screencast on tab switch', async () => {
+      // Start screencast on tab 0
+      await browser.startScreencast(() => {});
+      expect(browser.isScreencasting()).toBe(true);
+
+      // Create new tab and switch
+      await browser.newTab();
+      await browser.switchTo(1);
+
+      // Screencast should be stopped (it's page-specific)
+      expect(browser.isScreencasting()).toBe(false);
+    });
+  });
+
+  describe('input injection', () => {
+    it('should inject mouse move event', async () => {
+      await expect(
+        browser.injectMouseEvent({
+          type: 'mouseMoved',
+          x: 100,
+          y: 100,
+        })
+      ).resolves.not.toThrow();
+    });
+
+    it('should inject mouse click events', async () => {
+      await expect(
+        browser.injectMouseEvent({
+          type: 'mousePressed',
+          x: 100,
+          y: 100,
+          button: 'left',
+          clickCount: 1,
+        })
+      ).resolves.not.toThrow();
+
+      await expect(
+        browser.injectMouseEvent({
+          type: 'mouseReleased',
+          x: 100,
+          y: 100,
+          button: 'left',
+        })
+      ).resolves.not.toThrow();
+    });
+
+    it('should inject mouse wheel event', async () => {
+      await expect(
+        browser.injectMouseEvent({
+          type: 'mouseWheel',
+          x: 100,
+          y: 100,
+          deltaX: 0,
+          deltaY: 100,
+        })
+      ).resolves.not.toThrow();
+    });
+
+    it('should inject keyboard events', async () => {
+      await expect(
+        browser.injectKeyboardEvent({
+          type: 'keyDown',
+          key: 'a',
+          code: 'KeyA',
+        })
+      ).resolves.not.toThrow();
+
+      await expect(
+        browser.injectKeyboardEvent({
+          type: 'keyUp',
+          key: 'a',
+          code: 'KeyA',
+        })
+      ).resolves.not.toThrow();
+    });
+
+    it('should inject char event', async () => {
+      // CDP char events only accept single characters
+      await expect(
+        browser.injectKeyboardEvent({
+          type: 'char',
+          text: 'h',
+        })
+      ).resolves.not.toThrow();
+    });
+
+    it('should inject keyboard with modifiers', async () => {
+      await expect(
+        browser.injectKeyboardEvent({
+          type: 'keyDown',
+          key: 'c',
+          code: 'KeyC',
+          modifiers: 2, // Ctrl
+        })
+      ).resolves.not.toThrow();
+    });
+
+    it('should inject touch events', async () => {
+      await expect(
+        browser.injectTouchEvent({
+          type: 'touchStart',
+          touchPoints: [{ x: 100, y: 100 }],
+        })
+      ).resolves.not.toThrow();
+
+      await expect(
+        browser.injectTouchEvent({
+          type: 'touchMove',
+          touchPoints: [{ x: 150, y: 150 }],
+        })
+      ).resolves.not.toThrow();
+
+      await expect(
+        browser.injectTouchEvent({
+          type: 'touchEnd',
+          touchPoints: [],
+        })
+      ).resolves.not.toThrow();
+    });
+
+    it('should inject multi-touch events', async () => {
+      await expect(
+        browser.injectTouchEvent({
+          type: 'touchStart',
+          touchPoints: [
+            { x: 100, y: 100, id: 0 },
+            { x: 200, y: 200, id: 1 },
+          ],
+        })
+      ).resolves.not.toThrow();
+
+      await expect(
+        browser.injectTouchEvent({
+          type: 'touchEnd',
+          touchPoints: [],
+        })
+      ).resolves.not.toThrow();
+    });
+  });
 });

src/browser.ts

@@ -11,10 +11,37 @@ import {
   type Request,
   type Route,
   type Locator,
+  type CDPSession,
 } from 'playwright-core';
+import path from 'node:path';
+import os from 'node:os';
 import type { LaunchCommand } from './types.js';
 import { type RefMap, type EnhancedSnapshot, getEnhancedSnapshot, parseRef } from './snapshot.js';
 
+// Screencast frame data from CDP
+export interface ScreencastFrame {
+  data: string; // base64 encoded image
+  metadata: {
+    offsetTop: number;
+    pageScaleFactor: number;
+    deviceWidth: number;
+    deviceHeight: number;
+    scrollOffsetX: number;
+    scrollOffsetY: number;
+    timestamp?: number;
+  };
+  sessionId: number;
+}
+
+// Screencast options
+export interface ScreencastOptions {
+  format?: 'jpeg' | 'png';
+  quality?: number; // 0-100, only for jpeg
+  maxWidth?: number;
+  maxHeight?: number;
+  everyNthFrame?: number;
+}
+
 interface TrackedRequest {
   url: string;
   method: string;
@@ -39,6 +66,8 @@ interface PageError {
  */
 export class BrowserManager {
   private browser: Browser | null = null;
+  private cdpPort: number | null = null;
+  private isPersistentContext: boolean = false;
   private contexts: BrowserContext[] = [];
   private pages: Page[] = [];
   private activePageIndex: number = 0;
@@ -51,12 +80,20 @@ export class BrowserManager {
   private isRecordingHar: boolean = false;
   private refMap: RefMap = {};
   private lastSnapshot: string = '';
+  private scopedHeaderRoutes: Map<string, (route: Route) => Promise<void>> = new Map();
+
+  // CDP session for screencast and input injection
+  private cdpSession: CDPSession | null = null;
+  private screencastActive: boolean = false;
+  private screencastSessionId: number = 0;
+  private frameCallback: ((frame: ScreencastFrame) => void) | null = null;
+  private screencastFrameHandler: ((params: any) => void) | null = null;
 
   /**
    * Check if browser is launched
    */
   isLaunched(): boolean {
-    return this.browser !== null;
+    return this.browser !== null || this.isPersistentContext;
   }
 
   /**
@@ -439,7 +476,7 @@ export class BrowserManager {
   }
 
   /**
-   * Set extra HTTP headers
+   * Set extra HTTP headers (global - all requests)
    */
   async setExtraHeaders(headers: Record<string, string>): Promise<void> {
     const context = this.contexts[0];
@@ -448,6 +485,76 @@ export class BrowserManager {
     }
   }
 
+  /**
+   * Set scoped HTTP headers (only for requests matching the origin)
+   * Uses route interception to add headers only to matching requests
+   */
+  async setScopedHeaders(origin: string, headers: Record<string, string>): Promise<void> {
+    const page = this.getPage();
+
+    // Build URL pattern from origin (e.g., "api.example.com" -> "**://api.example.com/**")
+    // Handle both full URLs and just hostnames
+    let urlPattern: string;
+    try {
+      const url = new URL(origin.startsWith('http') ? origin : `https://${origin}`);
+      // Match any protocol, the host, and any path
+      urlPattern = `**://${url.host}/**`;
+    } catch {
+      // If parsing fails, treat as hostname pattern
+      urlPattern = `**://${origin}/**`;
+    }
+
+    // Remove existing route for this origin if any
+    const existingHandler = this.scopedHeaderRoutes.get(urlPattern);
+    if (existingHandler) {
+      await page.unroute(urlPattern, existingHandler);
+    }
+
+    // Create handler that adds headers to matching requests
+    const handler = async (route: Route) => {
+      const requestHeaders = route.request().headers();
+      await route.continue({
+        headers: {
+          ...requestHeaders,
+          ...headers,
+        },
+      });
+    };
+
+    // Store and register the route
+    this.scopedHeaderRoutes.set(urlPattern, handler);
+    await page.route(urlPattern, handler);
+  }
+
+  /**
+   * Clear scoped headers for an origin (or all if no origin specified)
+   */
+  async clearScopedHeaders(origin?: string): Promise<void> {
+    const page = this.getPage();
+
+    if (origin) {
+      let urlPattern: string;
+      try {
+        const url = new URL(origin.startsWith('http') ? origin : `https://${origin}`);
+        urlPattern = `**://${url.host}/**`;
+      } catch {
+        urlPattern = `**://${origin}/**`;
+      }
+
+      const handler = this.scopedHeaderRoutes.get(urlPattern);
+      if (handler) {
+        await page.unroute(urlPattern, handler);
+        this.scopedHeaderRoutes.delete(urlPattern);
+      }
+    } else {
+      // Clear all scoped header routes
+      for (const [pattern, handler] of this.scopedHeaderRoutes) {
+        await page.unroute(pattern, handler);
+      }
+      this.scopedHeaderRoutes.clear();
+    }
+  }
+
   /**
    * Start tracing
    */
@@ -502,48 +609,151 @@ export class BrowserManager {
     return this.browser;
   }
 
+  /**
+   * Check if an existing CDP connection is still alive
+   * by verifying we can access browser contexts and that at least one has pages
+   */
+  private isCdpConnectionAlive(): boolean {
+    if (!this.browser) return false;
+    try {
+      const contexts = this.browser.contexts();
+      if (contexts.length === 0) return false;
+      return contexts.some((context) => context.pages().length > 0);
+    } catch {
+      return false;
+    }
+  }
+
+  /**
+   * Check if CDP connection needs to be re-established
+   */
+  private needsCdpReconnect(cdpPort: number): boolean {
+    if (!this.browser?.isConnected()) return true;
+    if (this.cdpPort !== cdpPort) return true;
+    if (!this.isCdpConnectionAlive()) return true;
+    return false;
+  }
+
   /**
    * Launch the browser with the specified options
    * If already launched, this is a no-op (browser stays open)
    */
   async launch(options: LaunchCommand): Promise<void> {
-    // If already launched, don't relaunch
+    const cdpPort = options.cdpPort;
-    if (this.browser) {
+    const hasExtensions = !!options.extensions?.length;
+
+    if (hasExtensions && cdpPort) {
+      throw new Error('Extensions cannot be used with CDP connection');
+    }
+
+    if (this.isLaunched()) {
+      const needsRelaunch =
+        (!cdpPort && this.cdpPort !== null) || (!!cdpPort && this.needsCdpReconnect(cdpPort));
+      if (needsRelaunch) {
+        await this.close();
+      } else {
+        return;
+      }
+    }
+
+    if (cdpPort) {
+      await this.connectViaCDP(cdpPort);
       return;
     }
 
-    // Select browser type
     const browserType = options.browser ?? 'chromium';
+    if (hasExtensions && browserType !== 'chromium') {
+      throw new Error('Extensions are only supported in Chromium');
+    }
+
     const launcher =
       browserType === 'firefox' ? firefox : browserType === 'webkit' ? webkit : chromium;
+    const viewport = options.viewport ?? { width: 1280, height: 720 };
 
-    // Launch browser
+    let context: BrowserContext;
-    this.browser = await launcher.launch({
+    if (hasExtensions) {
-      headless: options.headless ?? true,
+      const extPaths = options.extensions!.join(',');
-      executablePath: options.executablePath,
+      const session = process.env.AGENT_BROWSER_SESSION || 'default';
-    });
+      context = await launcher.launchPersistentContext(
+        path.join(os.tmpdir(), `agent-browser-ext-${session}`),
+        {
+          headless: false,
+          executablePath: options.executablePath,
+          args: [`--disable-extensions-except=${extPaths}`, `--load-extension=${extPaths}`],
+          viewport,
+          extraHTTPHeaders: options.headers,
+        }
+      );
+      this.isPersistentContext = true;
+    } else {
+      this.browser = await launcher.launch({
+        headless: options.headless ?? true,
+        executablePath: options.executablePath,
+      });
+      this.cdpPort = null;
+      context = await this.browser.newContext({ viewport, extraHTTPHeaders: options.headers });
+    }
 
-    // Create context with viewport
-    const context = await this.browser.newContext({
-      viewport: options.viewport ?? { width: 1280, height: 720 },
-    });
-
-    // Set default timeout to 10 seconds (Playwright default is 30s)
     context.setDefaultTimeout(10000);
-
     this.contexts.push(context);
 
-    // Create initial page
+    const page = context.pages()[0] ?? (await context.newPage());
-    const page = await context.newPage();
     this.pages.push(page);
     this.activePageIndex = 0;
-
-    // Automatically start console and error tracking
     this.setupPageTracking(page);
   }
 
   /**
-   * Set up console and error tracking for a page
+   * Connect to a running browser via CDP (Chrome DevTools Protocol)
+   */
+  private async connectViaCDP(cdpPort: number | undefined): Promise<void> {
+    if (!cdpPort) {
+      throw new Error('cdpPort is required for CDP connection');
+    }
+
+    const browser = await chromium.connectOverCDP(`http://localhost:${cdpPort}`).catch(() => {
+      throw new Error(
+        `Failed to connect via CDP on port ${cdpPort}. ` +
+          `Make sure the app is running with --remote-debugging-port=${cdpPort}`
+      );
+    });
+
+    // Validate and set up state, cleaning up browser connection if anything fails
+    try {
+      const contexts = browser.contexts();
+      if (contexts.length === 0) {
+        throw new Error('No browser context found. Make sure the app has an open window.');
+      }
+
+      const allPages = contexts.flatMap((context) => context.pages());
+      if (allPages.length === 0) {
+        throw new Error('No page found. Make sure the app has loaded content.');
+      }
+
+      // All validation passed - commit state
+      this.browser = browser;
+      this.cdpPort = cdpPort;
+
+      for (const context of contexts) {
+        this.contexts.push(context);
+        this.setupContextTracking(context);
+      }
+
+      for (const page of allPages) {
+        this.pages.push(page);
+        this.setupPageTracking(page);
+      }
+
+      this.activePageIndex = 0;
+    } catch (error) {
+      // Clean up browser connection if validation or setup failed
+      await browser.close().catch(() => {});
+      throw error;
+    }
+  }
+
+  /**
+   * Set up console, error, and close tracking for a page
    */
   private setupPageTracking(page: Page): void {
     page.on('console', (msg) => {
@@ -560,6 +770,26 @@ export class BrowserManager {
         timestamp: Date.now(),
       });
     });
+
+    page.on('close', () => {
+      const index = this.pages.indexOf(page);
+      if (index !== -1) {
+        this.pages.splice(index, 1);
+        if (this.activePageIndex >= this.pages.length) {
+          this.activePageIndex = Math.max(0, this.pages.length - 1);
+        }
+      }
+    });
+  }
+
+  /**
+   * Set up tracking for new pages in a context (for CDP connections)
+   */
+  private setupContextTracking(context: BrowserContext): void {
+    context.on('page', (page) => {
+      this.pages.push(page);
+      this.setupPageTracking(page);
+    });
   }
 
   /**
@@ -570,6 +800,9 @@ export class BrowserManager {
       throw new Error('Browser not launched');
     }
 
+    // Invalidate CDP session since we're switching to a new page
+    await this.invalidateCDPSession();
+
     const context = this.contexts[0]; // Use first context for tabs
     const page = await context.newPage();
     this.pages.push(page);
@@ -608,14 +841,36 @@ export class BrowserManager {
     return { index: this.activePageIndex, total: this.pages.length };
   }
 
+  /**
+   * Invalidate the current CDP session (must be called before switching pages)
+   * This ensures screencast and input injection work correctly after tab switch
+   */
+  private async invalidateCDPSession(): Promise<void> {
+    // Stop screencast if active (it's tied to the current page's CDP session)
+    if (this.screencastActive) {
+      await this.stopScreencast();
+    }
+
+    // Detach and clear the CDP session
+    if (this.cdpSession) {
+      await this.cdpSession.detach().catch(() => {});
+      this.cdpSession = null;
+    }
+  }
+
   /**
    * Switch to a specific tab/page by index
    */
-  switchTo(index: number): { index: number; url: string; title: string } {
+  async switchTo(index: number): Promise<{ index: number; url: string; title: string }> {
     if (index < 0 || index >= this.pages.length) {
       throw new Error(`Invalid tab index: ${index}. Available: 0-${this.pages.length - 1}`);
     }
 
+    // Invalidate CDP session before switching (it's page-specific)
+    if (index !== this.activePageIndex) {
+      await this.invalidateCDPSession();
+    }
+
     this.activePageIndex = index;
     const page = this.pages[index];
 
@@ -640,6 +895,11 @@ export class BrowserManager {
       throw new Error('Cannot close the last tab. Use "close" to close the browser.');
     }
 
+    // If closing the active tab, invalidate CDP session first
+    if (targetIndex === this.activePageIndex) {
+      await this.invalidateCDPSession();
+    }
+
     const page = this.pages[targetIndex];
     await page.close();
     this.pages.splice(targetIndex, 1);
@@ -669,27 +929,222 @@ export class BrowserManager {
     return tabs;
   }
 
+  /**
+   * Get or create a CDP session for the current page
+   * Only works with Chromium-based browsers
+   */
+  async getCDPSession(): Promise<CDPSession> {
+    if (this.cdpSession) {
+      return this.cdpSession;
+    }
+
+    const page = this.getPage();
+    const context = page.context();
+
+    // Create a new CDP session attached to the page
+    this.cdpSession = await context.newCDPSession(page);
+    return this.cdpSession;
+  }
+
+  /**
+   * Check if screencast is currently active
+   */
+  isScreencasting(): boolean {
+    return this.screencastActive;
+  }
+
+  /**
+   * Start screencast - streams viewport frames via CDP
+   * @param callback Function called for each frame
+   * @param options Screencast options
+   */
+  async startScreencast(
+    callback: (frame: ScreencastFrame) => void,
+    options?: ScreencastOptions
+  ): Promise<void> {
+    if (this.screencastActive) {
+      throw new Error('Screencast already active');
+    }
+
+    const cdp = await this.getCDPSession();
+    this.frameCallback = callback;
+    this.screencastActive = true;
+
+    // Create and store the frame handler so we can remove it later
+    this.screencastFrameHandler = async (params: any) => {
+      const frame: ScreencastFrame = {
+        data: params.data,
+        metadata: params.metadata,
+        sessionId: params.sessionId,
+      };
+
+      // Acknowledge the frame to receive the next one
+      await cdp.send('Page.screencastFrameAck', { sessionId: params.sessionId });
+
+      // Call the callback with the frame
+      if (this.frameCallback) {
+        this.frameCallback(frame);
+      }
+    };
+
+    // Listen for screencast frames
+    cdp.on('Page.screencastFrame', this.screencastFrameHandler);
+
+    // Start the screencast
+    await cdp.send('Page.startScreencast', {
+      format: options?.format ?? 'jpeg',
+      quality: options?.quality ?? 80,
+      maxWidth: options?.maxWidth ?? 1280,
+      maxHeight: options?.maxHeight ?? 720,
+      everyNthFrame: options?.everyNthFrame ?? 1,
+    });
+  }
+
+  /**
+   * Stop screencast
+   */
+  async stopScreencast(): Promise<void> {
+    if (!this.screencastActive) {
+      return;
+    }
+
+    try {
+      const cdp = await this.getCDPSession();
+      await cdp.send('Page.stopScreencast');
+
+      // Remove the event listener to prevent accumulation
+      if (this.screencastFrameHandler) {
+        cdp.off('Page.screencastFrame', this.screencastFrameHandler);
+      }
+    } catch {
+      // Ignore errors when stopping
+    }
+
+    this.screencastActive = false;
+    this.frameCallback = null;
+    this.screencastFrameHandler = null;
+  }
+
+  /**
+   * Inject a mouse event via CDP
+   */
+  async injectMouseEvent(params: {
+    type: 'mousePressed' | 'mouseReleased' | 'mouseMoved' | 'mouseWheel';
+    x: number;
+    y: number;
+    button?: 'left' | 'right' | 'middle' | 'none';
+    clickCount?: number;
+    deltaX?: number;
+    deltaY?: number;
+    modifiers?: number; // 1=Alt, 2=Ctrl, 4=Meta, 8=Shift
+  }): Promise<void> {
+    const cdp = await this.getCDPSession();
+
+    const cdpButton =
+      params.button === 'left'
+        ? 'left'
+        : params.button === 'right'
+          ? 'right'
+          : params.button === 'middle'
+            ? 'middle'
+            : 'none';
+
+    await cdp.send('Input.dispatchMouseEvent', {
+      type: params.type,
+      x: params.x,
+      y: params.y,
+      button: cdpButton,
+      clickCount: params.clickCount ?? 1,
+      deltaX: params.deltaX ?? 0,
+      deltaY: params.deltaY ?? 0,
+      modifiers: params.modifiers ?? 0,
+    });
+  }
+
+  /**
+   * Inject a keyboard event via CDP
+   */
+  async injectKeyboardEvent(params: {
+    type: 'keyDown' | 'keyUp' | 'char';
+    key?: string;
+    code?: string;
+    text?: string;
+    modifiers?: number; // 1=Alt, 2=Ctrl, 4=Meta, 8=Shift
+  }): Promise<void> {
+    const cdp = await this.getCDPSession();
+
+    await cdp.send('Input.dispatchKeyEvent', {
+      type: params.type,
+      key: params.key,
+      code: params.code,
+      text: params.text,
+      modifiers: params.modifiers ?? 0,
+    });
+  }
+
+  /**
+   * Inject touch event via CDP (for mobile emulation)
+   */
+  async injectTouchEvent(params: {
+    type: 'touchStart' | 'touchEnd' | 'touchMove' | 'touchCancel';
+    touchPoints: Array<{ x: number; y: number; id?: number }>;
+    modifiers?: number;
+  }): Promise<void> {
+    const cdp = await this.getCDPSession();
+
+    await cdp.send('Input.dispatchTouchEvent', {
+      type: params.type,
+      touchPoints: params.touchPoints.map((tp, i) => ({
+        x: tp.x,
+        y: tp.y,
+        id: tp.id ?? i,
+      })),
+      modifiers: params.modifiers ?? 0,
+    });
+  }
+
   /**
    * Close the browser and clean up
    */
   async close(): Promise<void> {
-    for (const page of this.pages) {
+    // Stop screencast if active
-      await page.close().catch(() => {});
+    if (this.screencastActive) {
+      await this.stopScreencast();
     }
+
+    // Clean up CDP session
+    if (this.cdpSession) {
+      await this.cdpSession.detach().catch(() => {});
+      this.cdpSession = null;
+    }
+
+    // CDP: only disconnect, don't close external app's pages
+    if (this.cdpPort !== null) {
+      if (this.browser) {
+        await this.browser.close().catch(() => {});
+        this.browser = null;
+      }
+    } else {
+      // Regular browser: close everything
+      for (const page of this.pages) {
+        await page.close().catch(() => {});
+      }
+      for (const context of this.contexts) {
+        await context.close().catch(() => {});
+      }
+      if (this.browser) {
+        await this.browser.close().catch(() => {});
+        this.browser = null;
+      }
+    }
+
     this.pages = [];
-
-    for (const context of this.contexts) {
-      await context.close().catch(() => {});
-    }
     this.contexts = [];
-
+    this.cdpPort = null;
-    if (this.browser) {
+    this.isPersistentContext = false;
-      await this.browser.close().catch(() => {});
-      this.browser = null;
-    }
-
     this.activePageIndex = 0;
     this.refMap = {};
     this.lastSnapshot = '';
+    this.frameCallback = null;
   }
 }

src/daemon.ts

@@ -5,6 +5,7 @@ import * as os from 'os';
 import { BrowserManager } from './browser.js';
 import { parseCommand, serializeResponse, errorResponse } from './protocol.js';
 import { executeCommand } from './actions.js';
+import { StreamServer } from './stream-server.js';
 
 // Platform detection
 const isWindows = process.platform === 'win32';
@@ -12,6 +13,12 @@ const isWindows = process.platform === 'win32';
 // Session support - each session gets its own socket/pid
 let currentSession = process.env.AGENT_BROWSER_SESSION || 'default';
 
+// Stream server for browser preview
+let streamServer: StreamServer | null = null;
+
+// Default stream port (can be overridden with AGENT_BROWSER_STREAM_PORT)
+const DEFAULT_STREAM_PORT = 9223;
+
 /**
  * Set the current session
  */
@@ -105,8 +112,10 @@ export function getConnectionInfo(
  */
 export function cleanupSocket(session?: string): void {
   const pidFile = getPidFile(session);
+  const streamPortFile = getStreamPortFile(session);
   try {
     if (fs.existsSync(pidFile)) fs.unlinkSync(pidFile);
+    if (fs.existsSync(streamPortFile)) fs.unlinkSync(streamPortFile);
     if (isWindows) {
       const portFile = getPortFile(session);
       if (fs.existsSync(portFile)) fs.unlinkSync(portFile);
@@ -120,15 +129,40 @@ export function cleanupSocket(session?: string): void {
 }
 
 /**
- * Start the daemon server
+ * Get the stream port file path
  */
-export async function startDaemon(): Promise<void> {
+export function getStreamPortFile(session?: string): string {
+  const sess = session ?? currentSession;
+  return path.join(os.tmpdir(), `agent-browser-${sess}.stream`);
+}
+
+/**
+ * Start the daemon server
+ * @param options.streamPort Port for WebSocket stream server (0 to disable)
+ */
+export async function startDaemon(options?: { streamPort?: number }): Promise<void> {
   // Clean up any stale socket
   cleanupSocket();
 
   const browser = new BrowserManager();
   let shuttingDown = false;
 
+  // Start stream server if port is specified (or use default if env var is set)
+  const streamPort =
+    options?.streamPort ??
+    (process.env.AGENT_BROWSER_STREAM_PORT
+      ? parseInt(process.env.AGENT_BROWSER_STREAM_PORT, 10)
+      : 0);
+
+  if (streamPort > 0) {
+    streamServer = new StreamServer(browser, streamPort);
+    await streamServer.start();
+
+    // Write stream port to file for clients to discover
+    const streamPortFile = getStreamPortFile();
+    fs.writeFileSync(streamPortFile, streamPort.toString());
+  }
+
   const server = net.createServer((socket) => {
     let buffer = '';
 
@@ -158,11 +192,17 @@ export async function startDaemon(): Promise<void> {
             parseResult.command.action !== 'launch' &&
             parseResult.command.action !== 'close'
           ) {
+            const extensions = process.env.AGENT_BROWSER_EXTENSIONS
+              ? process.env.AGENT_BROWSER_EXTENSIONS.split(',')
+                  .map((p) => p.trim())
+                  .filter(Boolean)
+              : undefined;
             await browser.launch({
               id: 'auto',
               action: 'launch',
               headless: true,
               executablePath: process.env.AGENT_BROWSER_EXECUTABLE_PATH,
+              extensions: extensions,
             });
           }
 
@@ -227,6 +267,20 @@ export async function startDaemon(): Promise<void> {
   const shutdown = async () => {
     if (shuttingDown) return;
     shuttingDown = true;
+
+    // Stop stream server if running
+    if (streamServer) {
+      await streamServer.stop();
+      streamServer = null;
+      // Clean up stream port file
+      const streamPortFile = getStreamPortFile();
+      try {
+        if (fs.existsSync(streamPortFile)) fs.unlinkSync(streamPortFile);
+      } catch {
+        // Ignore cleanup errors
+      }
+    }
+
     await browser.close();
     server.close();
     cleanupSocket();

src/protocol.test.ts

@@ -461,6 +461,24 @@ describe('parseCommand', () => {
         expect(result.command.headless).toBe(false);
       }
     });
+
+    it('should parse launch with cdpPort', () => {
+      const result = parseCommand(cmd({ id: '1', action: 'launch', cdpPort: 9222 }));
+      expect(result.success).toBe(true);
+      if (result.success) {
+        expect(result.command.cdpPort).toBe(9222);
+      }
+    });
+
+    it('should reject launch with invalid cdpPort', () => {
+      const result = parseCommand(cmd({ id: '1', action: 'launch', cdpPort: -1 }));
+      expect(result.success).toBe(false);
+    });
+
+    it('should reject launch with non-numeric cdpPort', () => {
+      const result = parseCommand(cmd({ id: '1', action: 'launch', cdpPort: 'invalid' }));
+      expect(result.success).toBe(false);
+    });
   });
 
   describe('mouse actions', () => {
@@ -602,6 +620,391 @@ describe('parseCommand', () => {
     });
   });
 
+  describe('screencast', () => {
+    it('should parse screencast_start with defaults', () => {
+      const result = parseCommand(cmd({ id: '1', action: 'screencast_start' }));
+      expect(result.success).toBe(true);
+      if (result.success) {
+        expect(result.command.action).toBe('screencast_start');
+      }
+    });
+
+    it('should parse screencast_start with all options', () => {
+      const result = parseCommand(
+        cmd({
+          id: '1',
+          action: 'screencast_start',
+          format: 'png',
+          quality: 90,
+          maxWidth: 1920,
+          maxHeight: 1080,
+          everyNthFrame: 2,
+        })
+      );
+      expect(result.success).toBe(true);
+      if (result.success) {
+        expect(result.command.format).toBe('png');
+        expect(result.command.quality).toBe(90);
+        expect(result.command.maxWidth).toBe(1920);
+        expect(result.command.maxHeight).toBe(1080);
+        expect(result.command.everyNthFrame).toBe(2);
+      }
+    });
+
+    it('should reject screencast_start with invalid format', () => {
+      const result = parseCommand(cmd({ id: '1', action: 'screencast_start', format: 'gif' }));
+      expect(result.success).toBe(false);
+    });
+
+    it('should reject screencast_start with quality out of range', () => {
+      const result = parseCommand(cmd({ id: '1', action: 'screencast_start', quality: 150 }));
+      expect(result.success).toBe(false);
+    });
+
+    it('should reject screencast_start with negative maxWidth', () => {
+      const result = parseCommand(cmd({ id: '1', action: 'screencast_start', maxWidth: -100 }));
+      expect(result.success).toBe(false);
+    });
+
+    it('should parse screencast_stop', () => {
+      const result = parseCommand(cmd({ id: '1', action: 'screencast_stop' }));
+      expect(result.success).toBe(true);
+      if (result.success) {
+        expect(result.command.action).toBe('screencast_stop');
+      }
+    });
+  });
+
+  describe('input injection', () => {
+    describe('input_mouse', () => {
+      it('should parse mousePressed event', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_mouse',
+            type: 'mousePressed',
+            x: 100,
+            y: 200,
+            button: 'left',
+          })
+        );
+        expect(result.success).toBe(true);
+        if (result.success) {
+          expect(result.command.action).toBe('input_mouse');
+          expect(result.command.type).toBe('mousePressed');
+          expect(result.command.x).toBe(100);
+          expect(result.command.y).toBe(200);
+          expect(result.command.button).toBe('left');
+        }
+      });
+
+      it('should parse mouseReleased event', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_mouse',
+            type: 'mouseReleased',
+            x: 100,
+            y: 200,
+          })
+        );
+        expect(result.success).toBe(true);
+      });
+
+      it('should parse mouseMoved event', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_mouse',
+            type: 'mouseMoved',
+            x: 150,
+            y: 250,
+          })
+        );
+        expect(result.success).toBe(true);
+      });
+
+      it('should parse mouseWheel event with deltas', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_mouse',
+            type: 'mouseWheel',
+            x: 100,
+            y: 200,
+            deltaX: 0,
+            deltaY: 100,
+          })
+        );
+        expect(result.success).toBe(true);
+        if (result.success) {
+          expect(result.command.deltaX).toBe(0);
+          expect(result.command.deltaY).toBe(100);
+        }
+      });
+
+      it('should parse mouse event with modifiers', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_mouse',
+            type: 'mousePressed',
+            x: 100,
+            y: 200,
+            modifiers: 6, // Ctrl + Meta
+          })
+        );
+        expect(result.success).toBe(true);
+        if (result.success) {
+          expect(result.command.modifiers).toBe(6);
+        }
+      });
+
+      it('should parse mouse event with clickCount', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_mouse',
+            type: 'mousePressed',
+            x: 100,
+            y: 200,
+            clickCount: 2,
+          })
+        );
+        expect(result.success).toBe(true);
+        if (result.success) {
+          expect(result.command.clickCount).toBe(2);
+        }
+      });
+
+      it('should reject input_mouse with invalid type', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_mouse',
+            type: 'invalid',
+            x: 100,
+            y: 200,
+          })
+        );
+        expect(result.success).toBe(false);
+      });
+
+      it('should reject input_mouse without x coordinate', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_mouse',
+            type: 'mousePressed',
+            y: 200,
+          })
+        );
+        expect(result.success).toBe(false);
+      });
+
+      it('should reject input_mouse without y coordinate', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_mouse',
+            type: 'mousePressed',
+            x: 100,
+          })
+        );
+        expect(result.success).toBe(false);
+      });
+    });
+
+    describe('input_keyboard', () => {
+      it('should parse keyDown event', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_keyboard',
+            type: 'keyDown',
+            key: 'Enter',
+            code: 'Enter',
+          })
+        );
+        expect(result.success).toBe(true);
+        if (result.success) {
+          expect(result.command.action).toBe('input_keyboard');
+          expect(result.command.type).toBe('keyDown');
+          expect(result.command.key).toBe('Enter');
+          expect(result.command.code).toBe('Enter');
+        }
+      });
+
+      it('should parse keyUp event', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_keyboard',
+            type: 'keyUp',
+            key: 'a',
+          })
+        );
+        expect(result.success).toBe(true);
+      });
+
+      it('should parse char event with text', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_keyboard',
+            type: 'char',
+            text: 'hello',
+          })
+        );
+        expect(result.success).toBe(true);
+        if (result.success) {
+          expect(result.command.text).toBe('hello');
+        }
+      });
+
+      it('should parse keyboard event with modifiers', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_keyboard',
+            type: 'keyDown',
+            key: 'c',
+            modifiers: 2, // Ctrl
+          })
+        );
+        expect(result.success).toBe(true);
+        if (result.success) {
+          expect(result.command.modifiers).toBe(2);
+        }
+      });
+
+      it('should reject input_keyboard with invalid type', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_keyboard',
+            type: 'invalid',
+          })
+        );
+        expect(result.success).toBe(false);
+      });
+    });
+
+    describe('input_touch', () => {
+      it('should parse touchStart event', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_touch',
+            type: 'touchStart',
+            touchPoints: [{ x: 100, y: 200 }],
+          })
+        );
+        expect(result.success).toBe(true);
+        if (result.success) {
+          expect(result.command.action).toBe('input_touch');
+          expect(result.command.type).toBe('touchStart');
+          expect(result.command.touchPoints).toHaveLength(1);
+          expect(result.command.touchPoints[0].x).toBe(100);
+          expect(result.command.touchPoints[0].y).toBe(200);
+        }
+      });
+
+      it('should parse touchEnd event', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_touch',
+            type: 'touchEnd',
+            touchPoints: [],
+          })
+        );
+        expect(result.success).toBe(true);
+      });
+
+      it('should parse touchMove event', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_touch',
+            type: 'touchMove',
+            touchPoints: [{ x: 150, y: 250 }],
+          })
+        );
+        expect(result.success).toBe(true);
+      });
+
+      it('should parse touchCancel event', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_touch',
+            type: 'touchCancel',
+            touchPoints: [],
+          })
+        );
+        expect(result.success).toBe(true);
+      });
+
+      it('should parse multi-touch event', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_touch',
+            type: 'touchStart',
+            touchPoints: [
+              { x: 100, y: 200, id: 0 },
+              { x: 300, y: 400, id: 1 },
+            ],
+          })
+        );
+        expect(result.success).toBe(true);
+        if (result.success) {
+          expect(result.command.touchPoints).toHaveLength(2);
+        }
+      });
+
+      it('should parse touch event with modifiers', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_touch',
+            type: 'touchStart',
+            touchPoints: [{ x: 100, y: 200 }],
+            modifiers: 8, // Shift
+          })
+        );
+        expect(result.success).toBe(true);
+        if (result.success) {
+          expect(result.command.modifiers).toBe(8);
+        }
+      });
+
+      it('should reject input_touch with invalid type', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_touch',
+            type: 'invalid',
+            touchPoints: [],
+          })
+        );
+        expect(result.success).toBe(false);
+      });
+
+      it('should reject input_touch without touchPoints', () => {
+        const result = parseCommand(
+          cmd({
+            id: '1',
+            action: 'input_touch',
+            type: 'touchStart',
+          })
+        );
+        expect(result.success).toBe(false);
+      });
+    });
+  });
+
   describe('invalid commands', () => {
     it('should reject unknown action', () => {
       const result = parseCommand(cmd({ id: '1', action: 'unknown' }));

src/protocol.ts

@@ -18,6 +18,7 @@ const launchSchema = baseCommandSchema.extend({
     })
     .optional(),
   browser: z.enum(['chromium', 'firefox', 'webkit']).optional(),
+  cdpPort: z.number().positive().optional(),
 });
 
 const navigateSchema = baseCommandSchema.extend({
@@ -584,6 +585,55 @@ const responseBodySchema = baseCommandSchema.extend({
   timeout: z.number().positive().optional(),
 });
 
+// Screencast schemas for streaming browser viewport
+const screencastStartSchema = baseCommandSchema.extend({
+  action: z.literal('screencast_start'),
+  format: z.enum(['jpeg', 'png']).optional(),
+  quality: z.number().min(0).max(100).optional(),
+  maxWidth: z.number().positive().optional(),
+  maxHeight: z.number().positive().optional(),
+  everyNthFrame: z.number().positive().optional(),
+});
+
+const screencastStopSchema = baseCommandSchema.extend({
+  action: z.literal('screencast_stop'),
+});
+
+// Input injection schemas for pair browsing
+const inputMouseSchema = baseCommandSchema.extend({
+  action: z.literal('input_mouse'),
+  type: z.enum(['mousePressed', 'mouseReleased', 'mouseMoved', 'mouseWheel']),
+  x: z.number(),
+  y: z.number(),
+  button: z.enum(['left', 'right', 'middle', 'none']).optional(),
+  clickCount: z.number().positive().optional(),
+  deltaX: z.number().optional(),
+  deltaY: z.number().optional(),
+  modifiers: z.number().optional(),
+});
+
+const inputKeyboardSchema = baseCommandSchema.extend({
+  action: z.literal('input_keyboard'),
+  type: z.enum(['keyDown', 'keyUp', 'char']),
+  key: z.string().optional(),
+  code: z.string().optional(),
+  text: z.string().optional(),
+  modifiers: z.number().optional(),
+});
+
+const inputTouchSchema = baseCommandSchema.extend({
+  action: z.literal('input_touch'),
+  type: z.enum(['touchStart', 'touchEnd', 'touchMove', 'touchCancel']),
+  touchPoints: z.array(
+    z.object({
+      x: z.number(),
+      y: z.number(),
+      id: z.number().optional(),
+    })
+  ),
+  modifiers: z.number().optional(),
+});
+
 const pressSchema = baseCommandSchema.extend({
   action: z.literal('press'),
   key: z.string().min(1),
@@ -794,6 +844,11 @@ const commandSchema = z.discriminatedUnion('action', [
   multiSelectSchema,
   waitForDownloadSchema,
   responseBodySchema,
+  screencastStartSchema,
+  screencastStopSchema,
+  inputMouseSchema,
+  inputKeyboardSchema,
+  inputTouchSchema,
 ]);
 
 // Parse result type

src/stream-server.ts

@@ -0,0 +1,364 @@
+import { WebSocketServer, WebSocket } from 'ws';
+import type { BrowserManager, ScreencastFrame } from './browser.js';
+import { setScreencastFrameCallback } from './actions.js';
+
+// Message types for WebSocket communication
+export interface FrameMessage {
+  type: 'frame';
+  data: string; // base64 encoded image
+  metadata: {
+    offsetTop: number;
+    pageScaleFactor: number;
+    deviceWidth: number;
+    deviceHeight: number;
+    scrollOffsetX: number;
+    scrollOffsetY: number;
+    timestamp?: number;
+  };
+}
+
+export interface InputMouseMessage {
+  type: 'input_mouse';
+  eventType: 'mousePressed' | 'mouseReleased' | 'mouseMoved' | 'mouseWheel';
+  x: number;
+  y: number;
+  button?: 'left' | 'right' | 'middle' | 'none';
+  clickCount?: number;
+  deltaX?: number;
+  deltaY?: number;
+  modifiers?: number;
+}
+
+export interface InputKeyboardMessage {
+  type: 'input_keyboard';
+  eventType: 'keyDown' | 'keyUp' | 'char';
+  key?: string;
+  code?: string;
+  text?: string;
+  modifiers?: number;
+}
+
+export interface InputTouchMessage {
+  type: 'input_touch';
+  eventType: 'touchStart' | 'touchEnd' | 'touchMove' | 'touchCancel';
+  touchPoints: Array<{ x: number; y: number; id?: number }>;
+  modifiers?: number;
+}
+
+export interface StatusMessage {
+  type: 'status';
+  connected: boolean;
+  screencasting: boolean;
+  viewportWidth?: number;
+  viewportHeight?: number;
+}
+
+export interface ErrorMessage {
+  type: 'error';
+  message: string;
+}
+
+export type StreamMessage =
+  | FrameMessage
+  | InputMouseMessage
+  | InputKeyboardMessage
+  | InputTouchMessage
+  | StatusMessage
+  | ErrorMessage;
+
+/**
+ * WebSocket server for streaming browser viewport and receiving input
+ */
+export class StreamServer {
+  private wss: WebSocketServer | null = null;
+  private clients: Set<WebSocket> = new Set();
+  private browser: BrowserManager;
+  private port: number;
+  private isScreencasting: boolean = false;
+
+  constructor(browser: BrowserManager, port: number = 9223) {
+    this.browser = browser;
+    this.port = port;
+  }
+
+  /**
+   * Start the WebSocket server
+   */
+  start(): Promise<void> {
+    return new Promise((resolve, reject) => {
+      try {
+        this.wss = new WebSocketServer({ port: this.port });
+
+        this.wss.on('connection', (ws) => {
+          this.handleConnection(ws);
+        });
+
+        this.wss.on('error', (error) => {
+          console.error('[StreamServer] WebSocket error:', error);
+          reject(error);
+        });
+
+        this.wss.on('listening', () => {
+          console.log(`[StreamServer] Listening on port ${this.port}`);
+
+          // Set up the screencast frame callback
+          setScreencastFrameCallback((frame) => {
+            this.broadcastFrame(frame);
+          });
+
+          resolve();
+        });
+      } catch (error) {
+        reject(error);
+      }
+    });
+  }
+
+  /**
+   * Stop the WebSocket server
+   */
+  async stop(): Promise<void> {
+    // Stop screencasting
+    if (this.isScreencasting) {
+      await this.stopScreencast();
+    }
+
+    // Clear the callback
+    setScreencastFrameCallback(null);
+
+    // Close all clients
+    for (const client of this.clients) {
+      client.close();
+    }
+    this.clients.clear();
+
+    // Close the server
+    if (this.wss) {
+      return new Promise((resolve) => {
+        this.wss!.close(() => {
+          this.wss = null;
+          resolve();
+        });
+      });
+    }
+  }
+
+  /**
+   * Handle a new WebSocket connection
+   */
+  private handleConnection(ws: WebSocket): void {
+    console.log('[StreamServer] Client connected');
+    this.clients.add(ws);
+
+    // Send initial status
+    this.sendStatus(ws);
+
+    // Start screencasting if this is the first client
+    if (this.clients.size === 1 && !this.isScreencasting) {
+      this.startScreencast().catch((error) => {
+        console.error('[StreamServer] Failed to start screencast:', error);
+        this.sendError(ws, error.message);
+      });
+    }
+
+    // Handle messages from client
+    ws.on('message', (data) => {
+      try {
+        const message = JSON.parse(data.toString()) as StreamMessage;
+        this.handleMessage(message, ws);
+      } catch (error) {
+        console.error('[StreamServer] Failed to parse message:', error);
+      }
+    });
+
+    // Handle client disconnect
+    ws.on('close', () => {
+      console.log('[StreamServer] Client disconnected');
+      this.clients.delete(ws);
+
+      // Stop screencasting if no more clients
+      if (this.clients.size === 0 && this.isScreencasting) {
+        this.stopScreencast().catch((error) => {
+          console.error('[StreamServer] Failed to stop screencast:', error);
+        });
+      }
+    });
+
+    ws.on('error', (error) => {
+      console.error('[StreamServer] Client error:', error);
+      this.clients.delete(ws);
+    });
+  }
+
+  /**
+   * Handle incoming messages from clients
+   */
+  private async handleMessage(message: StreamMessage, ws: WebSocket): Promise<void> {
+    try {
+      switch (message.type) {
+        case 'input_mouse':
+          await this.browser.injectMouseEvent({
+            type: message.eventType,
+            x: message.x,
+            y: message.y,
+            button: message.button,
+            clickCount: message.clickCount,
+            deltaX: message.deltaX,
+            deltaY: message.deltaY,
+            modifiers: message.modifiers,
+          });
+          break;
+
+        case 'input_keyboard':
+          await this.browser.injectKeyboardEvent({
+            type: message.eventType,
+            key: message.key,
+            code: message.code,
+            text: message.text,
+            modifiers: message.modifiers,
+          });
+          break;
+
+        case 'input_touch':
+          await this.browser.injectTouchEvent({
+            type: message.eventType,
+            touchPoints: message.touchPoints,
+            modifiers: message.modifiers,
+          });
+          break;
+
+        case 'status':
+          // Client is requesting status
+          this.sendStatus(ws);
+          break;
+      }
+    } catch (error) {
+      const errorMessage = error instanceof Error ? error.message : String(error);
+      this.sendError(ws, errorMessage);
+    }
+  }
+
+  /**
+   * Broadcast a frame to all connected clients
+   */
+  private broadcastFrame(frame: ScreencastFrame): void {
+    const message: FrameMessage = {
+      type: 'frame',
+      data: frame.data,
+      metadata: frame.metadata,
+    };
+
+    const payload = JSON.stringify(message);
+
+    for (const client of this.clients) {
+      if (client.readyState === WebSocket.OPEN) {
+        client.send(payload);
+      }
+    }
+  }
+
+  /**
+   * Send status to a client
+   */
+  private sendStatus(ws: WebSocket): void {
+    let viewportWidth: number | undefined;
+    let viewportHeight: number | undefined;
+
+    try {
+      const page = this.browser.getPage();
+      const viewport = page.viewportSize();
+      viewportWidth = viewport?.width;
+      viewportHeight = viewport?.height;
+    } catch {
+      // Browser not launched yet
+    }
+
+    const message: StatusMessage = {
+      type: 'status',
+      connected: true,
+      screencasting: this.isScreencasting,
+      viewportWidth,
+      viewportHeight,
+    };
+
+    if (ws.readyState === WebSocket.OPEN) {
+      ws.send(JSON.stringify(message));
+    }
+  }
+
+  /**
+   * Send an error to a client
+   */
+  private sendError(ws: WebSocket, errorMessage: string): void {
+    const message: ErrorMessage = {
+      type: 'error',
+      message: errorMessage,
+    };
+
+    if (ws.readyState === WebSocket.OPEN) {
+      ws.send(JSON.stringify(message));
+    }
+  }
+
+  /**
+   * Start screencasting
+   */
+  private async startScreencast(): Promise<void> {
+    // Set flag immediately to prevent race conditions with concurrent calls
+    if (this.isScreencasting) return;
+    this.isScreencasting = true;
+
+    try {
+      // Check if browser is launched
+      if (!this.browser.isLaunched()) {
+        throw new Error('Browser not launched');
+      }
+
+      await this.browser.startScreencast((frame) => this.broadcastFrame(frame), {
+        format: 'jpeg',
+        quality: 80,
+        maxWidth: 1280,
+        maxHeight: 720,
+        everyNthFrame: 1,
+      });
+
+      // Notify all clients
+      for (const client of this.clients) {
+        this.sendStatus(client);
+      }
+    } catch (error) {
+      // Reset flag on failure so caller can retry
+      this.isScreencasting = false;
+      throw error;
+    }
+  }
+
+  /**
+   * Stop screencasting
+   */
+  private async stopScreencast(): Promise<void> {
+    if (!this.isScreencasting) return;
+
+    await this.browser.stopScreencast();
+    this.isScreencasting = false;
+
+    // Notify all clients
+    for (const client of this.clients) {
+      this.sendStatus(client);
+    }
+  }
+
+  /**
+   * Get the port the server is running on
+   */
+  getPort(): number {
+    return this.port;
+  }
+
+  /**
+   * Get the number of connected clients
+   */
+  getClientCount(): number {
+    return this.clients.size;
+  }
+}

src/types.ts

@@ -12,13 +12,17 @@ export interface LaunchCommand extends BaseCommand {
   headless?: boolean;
   viewport?: { width: number; height: number };
   browser?: 'chromium' | 'firefox' | 'webkit';
+  headers?: Record<string, string>;
   executablePath?: string;
+  cdpPort?: number;
+  extensions?: string[];
 }
 
 export interface NavigateCommand extends BaseCommand {
   action: 'navigate';
   url: string;
   waitUntil?: 'load' | 'domcontentloaded' | 'networkidle';
+  headers?: Record<string, string>;
 }
 
 export interface ClickCommand extends BaseCommand {
@@ -455,6 +459,49 @@ export interface ResponseBodyCommand extends BaseCommand {
   timeout?: number;
 }
 
+// Screencast commands for streaming browser viewport
+export interface ScreencastStartCommand extends BaseCommand {
+  action: 'screencast_start';
+  format?: 'jpeg' | 'png';
+  quality?: number; // 0-100, jpeg only
+  maxWidth?: number;
+  maxHeight?: number;
+  everyNthFrame?: number;
+}
+
+export interface ScreencastStopCommand extends BaseCommand {
+  action: 'screencast_stop';
+}
+
+// Input injection commands for pair browsing
+export interface InputMouseCommand extends BaseCommand {
+  action: 'input_mouse';
+  type: 'mousePressed' | 'mouseReleased' | 'mouseMoved' | 'mouseWheel';
+  x: number;
+  y: number;
+  button?: 'left' | 'right' | 'middle' | 'none';
+  clickCount?: number;
+  deltaX?: number;
+  deltaY?: number;
+  modifiers?: number;
+}
+
+export interface InputKeyboardCommand extends BaseCommand {
+  action: 'input_keyboard';
+  type: 'keyDown' | 'keyUp' | 'char';
+  key?: string;
+  code?: string;
+  text?: string;
+  modifiers?: number;
+}
+
+export interface InputTouchCommand extends BaseCommand {
+  action: 'input_touch';
+  type: 'touchStart' | 'touchEnd' | 'touchMove' | 'touchCancel';
+  touchPoints: Array<{ x: number; y: number; id?: number }>;
+  modifiers?: number;
+}
+
 // Video recording
 export interface VideoStartCommand extends BaseCommand {
   action: 'video_start';
@@ -838,7 +885,12 @@ export type Command =
   | InsertTextCommand
   | MultiSelectCommand
   | WaitForDownloadCommand
-  | ResponseBodyCommand;
+  | ResponseBodyCommand
+  | ScreencastStartCommand
+  | ScreencastStopCommand
+  | InputMouseCommand
+  | InputKeyboardCommand
+  | InputTouchCommand;
 
 // Response types
 export interface SuccessResponse<T = unknown> {
@@ -906,6 +958,20 @@ export interface TabCloseData {
   remaining: number;
 }
 
+export interface ScreencastStartData {
+  started: boolean;
+  format: string;
+  quality: number;
+}
+
+export interface ScreencastStopData {
+  stopped: boolean;
+}
+
+export interface InputEventData {
+  injected: boolean;
+}
+
 // Browser state
 export interface BrowserState {
   browser: Browser | null;