dashboard (#1034)
* dashboard * fix: re-apply download behavior on recording context (#1019) * fix: re-apply download behavior on recording context record start creates a new browser context via Target.createBrowserContext. Browser.setDownloadBehavior called at launch only applies to the default context, so downloads in the recording context are silently dropped. Fix: 1. Store download_path on BrowserManager (from LaunchOptions) 2. After creating the recording context, call Browser.setDownloadBehavior with the new browserContextId This ensures downloads work during recording. Fixes #1018 * fix: add download_path to third BrowserManager constructor (auto_connect_cdp) * fix: reap zombie Chrome process and fast-detect crash for auto-restart (#1023) When Chrome crashes (e.g. SIGTRAP from CHECK() assertion), the daemon now: 1. Reaps the zombie immediately via a SIGCHLD handler in the event loop that calls waitpid(-1, WNOHANG) 2. Detects the crash instantly on the next command via a non-blocking try_wait() check (has_process_exited), avoiding the 3-second CDP timeout that is_connection_alive() would incur 3. Auto-relaunches Chrome transparently for the caller Fixes #1017 Co-authored-by: ctate <366502+ctate@users.noreply.github.com> * fix: route keyboard type through text input (#1014) * fix: handle --clear flag in console command (#1015) The console and errors commands parsed --clear from CLI args but the action handlers silently ignored the flag. The handlers did not accept the cmd parameter so they had no way to read the clear field. Changes: - Add clear_console() method to EventTracker in network.rs - Update handle_console to accept cmd, read the clear field, and clear the buffer when --clear is passed (returns {cleared: true}) - Update call site in execute_command to pass cmd Co-authored-by: xuyongliang <yongliang.xyl@alibaba-inc.com> * chore: patch release - ### Bug Fixes - **Re-apply download behavior on r... (#1025) * Add runtime stream enable/disable/status commands (#951) * Add runtime stream management commands * Run rustfmt and satisfy clippy * Fix stream disable cleanup semantics * Format stream disable regression tests * fix: retain radio/checkbox elements in compact snapshot tree (#1008) compact_tree() checked for "[ref=" to identify lines worth keeping, but radio and checkbox elements render as e.g. [checked=false, ref=e1] where the "[" opens before "checked=", not "ref=". Dropping the leading bracket so the check is just "ref=" fixes the match for all elements with refs. Fixes #1006 Co-authored-by: ctate <366502+ctate@users.noreply.github.com> * chore: version packages (#1027) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fixes * dashboard * fixes * remove observe * fmt * fixes * fixes * jotai * fmt * upload dashboard --------- Co-authored-by: Stefan Smiljkovic <stefan@vanila.io> Co-authored-by: ctate <366502+ctate@users.noreply.github.com> Co-authored-by: zhanba <c5e1856@gmail.com> Co-authored-by: xuyongliang <478439790@qq.com> Co-authored-by: xuyongliang <yongliang.xyl@alibaba-inc.com> Co-authored-by: Thomas Kosiewski <thoma471@googlemail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This commit is contained in:
Chris Tate
co-authored by
ctate
github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Stefan Smiljkovic
zhanba
xuyongliang
xuyongliang
Thomas Kosiewski
ctate
github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Stefan Smiljkovic
zhanba
xuyongliang
xuyongliang
Thomas Kosiewski
parent
63f03b8e06
commit
f9174513c2
@@ -34,6 +34,7 @@ agent-browser stream enable [--port <port>] # Start runtime WebSocket streaming
|
||||
agent-browser stream status # Show runtime streaming state and bound port
|
||||
agent-browser stream disable # Stop runtime WebSocket streaming
|
||||
agent-browser close # Close browser (aliases: quit, exit)
|
||||
agent-browser close --all # Close all active sessions
|
||||
```
|
||||
|
||||
## Get info
|
||||
@@ -240,7 +241,7 @@ agent-browser stream status # Show enabled state, port, browser connec
|
||||
agent-browser stream disable # Stop runtime streaming and remove the .stream metadata file
|
||||
```
|
||||
|
||||
Use `stream enable` for sessions that are already running. If you need streaming from daemon startup, set `AGENT_BROWSER_STREAM_PORT` before the first command in that session.
|
||||
Streaming is enabled automatically for all sessions. Use these commands to check status, re-enable on a specific port, or disable streaming.
|
||||
|
||||
## Debug
|
||||
|
||||
@@ -325,6 +326,15 @@ agent-browser session # Show current session name
|
||||
agent-browser session list # List active sessions
|
||||
```
|
||||
|
||||
## Dashboard
|
||||
|
||||
```bash
|
||||
agent-browser dashboard [start] # Start the dashboard server (default port: 4848)
|
||||
agent-browser dashboard start --port <n> # Start on a specific port
|
||||
agent-browser dashboard stop # Stop the dashboard server
|
||||
agent-browser dashboard install # Install the dashboard files
|
||||
```
|
||||
|
||||
## Navigation
|
||||
|
||||
```bash
|
||||
|
||||
@@ -173,7 +173,7 @@ These environment variables configure additional daemon and runtime behavior:
|
||||
<tr><td><code>AGENT_BROWSER_ENCRYPTION_KEY</code></td><td>64-char hex key for AES-256-GCM session encryption.</td><td>(none)</td></tr>
|
||||
<tr><td><code>AGENT_BROWSER_EXTENSIONS</code></td><td>Comma-separated browser extension paths. Extensions work in both headed and headless mode.</td><td>(none)</td></tr>
|
||||
<tr><td><code>AGENT_BROWSER_HEADED</code></td><td>Show browser window instead of running headless (<code>1</code> to enable).</td><td>(disabled)</td></tr>
|
||||
<tr><td><code>AGENT_BROWSER_STREAM_PORT</code></td><td>Enable WebSocket streaming at daemon startup on the specified port (e.g., <code>9223</code>). For an already-running session, use <code>agent-browser stream enable</code>.</td><td>(disabled)</td></tr>
|
||||
<tr><td><code>AGENT_BROWSER_STREAM_PORT</code></td><td>Override the WebSocket streaming port. By default, an OS-assigned port is used. Set this to bind to a specific port (e.g., <code>9223</code>).</td><td>OS-assigned</td></tr>
|
||||
<tr><td><code>AGENT_BROWSER_IDLE_TIMEOUT_MS</code></td><td>Auto-shutdown the daemon after N ms of inactivity (no commands received). Useful for ephemeral environments.</td><td>(disabled)</td></tr>
|
||||
<tr><td><code>AGENT_BROWSER_IOS_DEVICE</code></td><td>Default iOS device name for the <code>ios</code> provider.</td><td>(none)</td></tr>
|
||||
<tr><td><code>AGENT_BROWSER_IOS_UDID</code></td><td>Default iOS device UDID for the <code>ios</code> provider.</td><td>(none)</td></tr>
|
||||
|
||||
@@ -0,0 +1,130 @@
|
||||
# Observability Dashboard
|
||||
|
||||
Monitor agent-browser sessions in real time with a local web dashboard showing a live browser viewport and command activity feed.
|
||||
|
||||
## Install
|
||||
|
||||
Download the dashboard once:
|
||||
|
||||
```bash
|
||||
agent-browser dashboard install
|
||||
```
|
||||
|
||||
This downloads the dashboard to `~/.agent-browser/dashboard/` and is served directly by the daemon when streaming is enabled.
|
||||
|
||||
## Usage
|
||||
|
||||
Start the dashboard server and open any session -- it appears automatically:
|
||||
|
||||
```bash
|
||||
agent-browser dashboard start
|
||||
agent-browser open example.com
|
||||
```
|
||||
|
||||
Then open `http://localhost:4848` in your browser to see the live dashboard.
|
||||
|
||||
All sessions automatically stream to the dashboard. No extra flags are needed.
|
||||
|
||||
### Custom stream port
|
||||
|
||||
By default each session binds its WebSocket stream server to an OS-assigned port. To use a specific port, set the `AGENT_BROWSER_STREAM_PORT` environment variable:
|
||||
|
||||
```bash
|
||||
AGENT_BROWSER_STREAM_PORT=9223 agent-browser open example.com
|
||||
```
|
||||
|
||||
You can also use the runtime commands to control streaming on a running session:
|
||||
|
||||
```bash
|
||||
agent-browser stream enable --port 9223
|
||||
agent-browser stream status
|
||||
agent-browser stream disable
|
||||
```
|
||||
|
||||
## Dashboard features
|
||||
|
||||
The dashboard is a single-page web app with three areas:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Area</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><strong>Live viewport</strong></td>
|
||||
<td>Real-time JPEG frames from the browser, rendered to a canvas element</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>Activity feed</strong></td>
|
||||
<td>Chronological stream of commands, results, and console messages with expandable details</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>Status bar</strong></td>
|
||||
<td>Connection status, viewport dimensions, and WebSocket endpoint</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## WebSocket protocol
|
||||
|
||||
The dashboard connects to the same WebSocket endpoint used by [Streaming](/streaming), with additional message types for observability:
|
||||
|
||||
### Command events
|
||||
|
||||
Sent when a command begins executing:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "command",
|
||||
"action": "click",
|
||||
"id": "r123",
|
||||
"params": { "selector": "@e5" },
|
||||
"timestamp": 1711367000000
|
||||
}
|
||||
```
|
||||
|
||||
### Result events
|
||||
|
||||
Sent when a command finishes:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "result",
|
||||
"id": "r123",
|
||||
"action": "click",
|
||||
"success": true,
|
||||
"data": {},
|
||||
"duration_ms": 45,
|
||||
"timestamp": 1711367000045
|
||||
}
|
||||
```
|
||||
|
||||
### Console events
|
||||
|
||||
Sent when the browser logs to the console:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "console",
|
||||
"level": "log",
|
||||
"text": "Page loaded",
|
||||
"timestamp": 1711367000100
|
||||
}
|
||||
```
|
||||
|
||||
These are in addition to the existing `frame`, `status`, and `error` message types documented on the [Streaming](/streaming) page.
|
||||
|
||||
## Architecture
|
||||
|
||||
The dashboard is a Next.js static export (`output: 'export'`) that produces plain HTML, CSS, and JS. It lives at `packages/dashboard/` in the monorepo and is built with:
|
||||
|
||||
```bash
|
||||
pnpm build:dashboard
|
||||
```
|
||||
|
||||
The built files are served by the daemon's stream server on the same port used for WebSocket connections. Plain HTTP requests serve the dashboard, while WebSocket upgrade requests are handled as before.
|
||||
|
||||
When the dashboard is not installed, visiting the HTTP endpoint shows instructions to run `agent-browser dashboard install`.
|
||||
@@ -3,27 +3,25 @@
|
||||
Stream the browser viewport via WebSocket for live preview or "pair browsing"
|
||||
where a human can watch and interact alongside an AI agent.
|
||||
|
||||
## Enable streaming
|
||||
## Streaming
|
||||
|
||||
For an already-running session, enable streaming at runtime:
|
||||
Every session automatically starts a WebSocket stream server on an OS-assigned port. The server streams viewport frames and accepts input events (mouse, keyboard, touch).
|
||||
|
||||
```bash
|
||||
agent-browser stream enable
|
||||
agent-browser stream status
|
||||
agent-browser stream disable
|
||||
```
|
||||
|
||||
`stream enable` binds an available localhost port automatically unless you pass `--port <port>`. `stream status` returns the enabled state, active port, browser connection state, and whether screencasting is active. `stream disable` tears the server down and removes the session's `.stream` metadata file.
|
||||
|
||||
If you want the WebSocket server to exist from daemon startup, set `AGENT_BROWSER_STREAM_PORT` before the first command in that session:
|
||||
To bind to a specific port, set `AGENT_BROWSER_STREAM_PORT`:
|
||||
|
||||
```bash
|
||||
AGENT_BROWSER_STREAM_PORT=9223 agent-browser open example.com
|
||||
```
|
||||
|
||||
The environment variable only affects daemon startup. For sessions that are already running, use `agent-browser stream enable` instead.
|
||||
You can also manage streaming at runtime:
|
||||
|
||||
Once enabled, the server streams viewport frames and accepts input events (mouse, keyboard, touch).
|
||||
```bash
|
||||
agent-browser stream status # Show streaming state and bound port
|
||||
agent-browser stream enable --port 9223 # Re-enable on a specific port
|
||||
agent-browser stream disable # Stop streaming for the session
|
||||
```
|
||||
|
||||
`stream status` returns the enabled state, active port, browser connection state, and whether screencasting is active. `stream disable` tears the server down and removes the session's `.stream` metadata file.
|
||||
|
||||
## Runtime status response
|
||||
|
||||
|
||||
Reference in New Issue
Block a user