Add AI chat to dashboard, refactor stream module, snapshot --urls, batch argument mode (#1160)
* chat * refactor * fixes * fixes * fixes * fixes * improvements * download chat * batch * fixes * fixes * fixes * fmt * fixes * fixes * fixes * fmt
This commit is contained in:
@@ -61,6 +61,9 @@ docs/package-lock.json
|
||||
# pnpm
|
||||
.pnpm-store/
|
||||
|
||||
# TypeScript
|
||||
*.tsbuildinfo
|
||||
|
||||
# next
|
||||
.next/
|
||||
out/
|
||||
|
||||
@@ -203,21 +203,24 @@ agent-browser wait "#spinner" --state hidden
|
||||
|
||||
### Batch Execution
|
||||
|
||||
Execute multiple commands in a single invocation by piping a JSON array of
|
||||
string arrays to `batch`. This avoids per-command process startup overhead
|
||||
when running multi-step workflows.
|
||||
Execute multiple commands in a single invocation. Commands can be passed as
|
||||
quoted arguments or piped as JSON via stdin. This avoids per-command process
|
||||
startup overhead when running multi-step workflows.
|
||||
|
||||
```bash
|
||||
# Pipe commands as JSON
|
||||
# Argument mode: each quoted argument is a full command
|
||||
agent-browser batch "open https://example.com" "snapshot -i" "screenshot"
|
||||
|
||||
# With --bail to stop on first error
|
||||
agent-browser batch --bail "open https://example.com" "click @e1" "screenshot"
|
||||
|
||||
# Stdin mode: pipe commands as JSON
|
||||
echo '[
|
||||
["open", "https://example.com"],
|
||||
["snapshot", "-i"],
|
||||
["click", "@e1"],
|
||||
["screenshot", "result.png"]
|
||||
]' | agent-browser batch --json
|
||||
|
||||
# Stop on first error
|
||||
agent-browser batch --bail < commands.json
|
||||
```
|
||||
|
||||
### Clipboard
|
||||
@@ -548,6 +551,7 @@ The `snapshot` command supports filtering to reduce output size:
|
||||
```bash
|
||||
agent-browser snapshot # Full accessibility tree
|
||||
agent-browser snapshot -i # Interactive elements only (buttons, inputs, links)
|
||||
agent-browser snapshot -i --urls # Interactive elements with link URLs
|
||||
agent-browser snapshot -c # Compact (remove empty structural elements)
|
||||
agent-browser snapshot -d 3 # Limit depth to 3 levels
|
||||
agent-browser snapshot -s "#main" # Scope to CSS selector
|
||||
@@ -557,6 +561,7 @@ agent-browser snapshot -i -c -d 5 # Combine options
|
||||
| Option | Description |
|
||||
| ---------------------- | ----------------------------------------------------------------------- |
|
||||
| `-i, --interactive` | Only show interactive elements (buttons, links, inputs) |
|
||||
| `-u, --urls` | Include href URLs for link elements |
|
||||
| `-c, --compact` | Remove empty structural elements |
|
||||
| `-d, --depth <n>` | Limit tree depth |
|
||||
| `-s, --selector <sel>` | Scope to CSS selector |
|
||||
@@ -650,6 +655,19 @@ The dashboard displays:
|
||||
- **Activity feed** -- chronological command/result stream with timing and expandable details
|
||||
- **Console output** -- browser console messages (log, warn, error)
|
||||
- **Session creation** -- create new sessions from the UI with local engines (Chrome, Lightpanda) or cloud providers (AgentCore, Browserbase, Browserless, Browser Use, Kernel)
|
||||
- **AI Chat** -- chat with an AI assistant directly in the dashboard (requires Vercel AI Gateway configuration)
|
||||
|
||||
### AI Chat
|
||||
|
||||
The dashboard includes an optional AI chat panel powered by the Vercel AI Gateway. Set these environment variables to enable it:
|
||||
|
||||
```bash
|
||||
export AI_GATEWAY_API_KEY=gw_your_key_here
|
||||
export AI_GATEWAY_MODEL=anthropic/claude-sonnet-4.6 # optional, this is the default
|
||||
export AI_GATEWAY_URL=https://ai-gateway.vercel.sh # optional, this is the default
|
||||
```
|
||||
|
||||
The Chat tab is always visible in the dashboard. When `AI_GATEWAY_API_KEY` is set, the Rust server proxies requests to the gateway and streams responses back using the Vercel AI SDK's UI Message Stream protocol. Without the key, sending a message shows an error inline.
|
||||
|
||||
## Configuration
|
||||
|
||||
|
||||
Generated
+37
@@ -605,6 +605,12 @@ version = "0.3.32"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7e3450815272ef58cec6d564423f6e755e25379b217b0bc688e295ba24df6b1d"
|
||||
|
||||
[[package]]
|
||||
name = "futures-io"
|
||||
version = "0.3.32"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "cecba35d7ad927e23624b22ad55235f2239cfa44fd10428eecbeba6d6a717718"
|
||||
|
||||
[[package]]
|
||||
name = "futures-macro"
|
||||
version = "0.3.32"
|
||||
@@ -635,9 +641,11 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "389ca41296e6190b48053de0321d02a77f32f8a5d2461dd38762c0593805c6d6"
|
||||
dependencies = [
|
||||
"futures-core",
|
||||
"futures-io",
|
||||
"futures-macro",
|
||||
"futures-sink",
|
||||
"futures-task",
|
||||
"memchr",
|
||||
"pin-project-lite",
|
||||
"slab",
|
||||
]
|
||||
@@ -1678,6 +1686,7 @@ dependencies = [
|
||||
"base64",
|
||||
"bytes",
|
||||
"futures-core",
|
||||
"futures-util",
|
||||
"http",
|
||||
"http-body",
|
||||
"http-body-util",
|
||||
@@ -1697,12 +1706,14 @@ dependencies = [
|
||||
"sync_wrapper",
|
||||
"tokio",
|
||||
"tokio-rustls",
|
||||
"tokio-util",
|
||||
"tower",
|
||||
"tower-http",
|
||||
"tower-service",
|
||||
"url",
|
||||
"wasm-bindgen",
|
||||
"wasm-bindgen-futures",
|
||||
"wasm-streams",
|
||||
"web-sys",
|
||||
"webpki-roots 1.0.5",
|
||||
]
|
||||
@@ -2128,6 +2139,19 @@ dependencies = [
|
||||
"webpki-roots 0.26.11",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tokio-util"
|
||||
version = "0.7.18"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "9ae9cec805b01e8fc3fd2fe289f89149a9b66dd16786abd8b19cfa7b48cb0098"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"futures-core",
|
||||
"futures-sink",
|
||||
"pin-project-lite",
|
||||
"tokio",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tower"
|
||||
version = "0.5.3"
|
||||
@@ -2430,6 +2454,19 @@ dependencies = [
|
||||
"wasmparser",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "wasm-streams"
|
||||
version = "0.4.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "15053d8d85c7eccdbefef60f06769760a563c7f0a9d6902a13d35c7800b0ad65"
|
||||
dependencies = [
|
||||
"futures-util",
|
||||
"js-sys",
|
||||
"wasm-bindgen",
|
||||
"wasm-bindgen-futures",
|
||||
"web-sys",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "wasmparser"
|
||||
version = "0.244.0"
|
||||
|
||||
+1
-1
@@ -22,7 +22,7 @@ futures-util = "0.3"
|
||||
url = "2"
|
||||
uuid = { version = "1", features = ["v4"] }
|
||||
image = "0.25"
|
||||
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls-webpki-roots"] }
|
||||
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls-webpki-roots", "stream"] }
|
||||
sha2 = "0.10"
|
||||
aes-gcm = "0.10"
|
||||
async-trait = "0.1"
|
||||
|
||||
+20
-5
@@ -1,15 +1,30 @@
|
||||
//! Color output utilities respecting NO_COLOR environment variable.
|
||||
//! Color output utilities.
|
||||
//!
|
||||
//! When the NO_COLOR environment variable is present (regardless of value),
|
||||
//! all color formatting is disabled per https://no-color.org/
|
||||
//! Colors are off by default (agent-friendly). Enable with
|
||||
//! `AGENT_BROWSER_COLOR=1`. Setting `NO_COLOR` to any value disables
|
||||
//! colors per <https://no-color.org/>.
|
||||
|
||||
use std::env;
|
||||
use std::sync::OnceLock;
|
||||
|
||||
/// Returns true if color output is enabled (NO_COLOR is NOT set)
|
||||
fn env_is_truthy(name: &str) -> Option<bool> {
|
||||
env::var(name)
|
||||
.ok()
|
||||
.map(|val| !matches!(val.to_lowercase().as_str(), "0" | "false" | "no"))
|
||||
}
|
||||
|
||||
/// Returns true if color output is enabled.
|
||||
///
|
||||
/// Priority: `NO_COLOR` (presence disables, per spec) >
|
||||
/// `AGENT_BROWSER_COLOR` (truthy enables) > default (off).
|
||||
pub fn is_enabled() -> bool {
|
||||
static COLORS_ENABLED: OnceLock<bool> = OnceLock::new();
|
||||
*COLORS_ENABLED.get_or_init(|| env::var("NO_COLOR").is_err())
|
||||
*COLORS_ENABLED.get_or_init(|| {
|
||||
if env::var_os("NO_COLOR").is_some() {
|
||||
return false;
|
||||
}
|
||||
env_is_truthy("AGENT_BROWSER_COLOR").unwrap_or(false)
|
||||
})
|
||||
}
|
||||
|
||||
/// Format text in red (errors)
|
||||
|
||||
+93
-2
@@ -552,9 +552,11 @@ fn parse_command_inner(args: &[String], flags: &Flags) -> Result<Value, ParseErr
|
||||
obj.insert("compact".to_string(), json!(true));
|
||||
}
|
||||
"-C" | "--cursor" => {
|
||||
// deprecated, cursor-interactive elements are referred by default now
|
||||
obj.insert("cursor".to_string(), json!(true));
|
||||
}
|
||||
"-u" | "--urls" => {
|
||||
obj.insert("urls".to_string(), json!(true));
|
||||
}
|
||||
"-d" | "--depth" => {
|
||||
if let Some(d) = rest.get(i + 1) {
|
||||
if let Ok(n) = d.parse::<i32>() {
|
||||
@@ -1409,7 +1411,12 @@ fn parse_command_inner(args: &[String], flags: &Flags) -> Result<Value, ParseErr
|
||||
// === Batch ===
|
||||
"batch" => {
|
||||
let bail = rest.contains(&"--bail");
|
||||
Ok(json!({ "id": id, "action": "batch", "bail": bail }))
|
||||
let commands: Vec<&str> = rest.iter().filter(|a| **a != "--bail").copied().collect();
|
||||
let mut cmd = json!({ "id": id, "action": "batch", "bail": bail });
|
||||
if !commands.is_empty() {
|
||||
cmd["commands"] = json!(commands);
|
||||
}
|
||||
Ok(cmd)
|
||||
}
|
||||
|
||||
_ => Err(ParseError::UnknownCommand {
|
||||
@@ -2289,6 +2296,38 @@ fn parse_storage(rest: &[&str], id: &str) -> Result<Value, ParseError> {
|
||||
}
|
||||
}
|
||||
|
||||
/// Split a string into arguments respecting shell quoting (double/single quotes, backslash escapes).
|
||||
pub fn shell_words_split(s: &str) -> Vec<String> {
|
||||
let mut args = Vec::new();
|
||||
let mut current = String::new();
|
||||
let mut in_double = false;
|
||||
let mut in_single = false;
|
||||
let mut chars = s.chars().peekable();
|
||||
|
||||
while let Some(c) = chars.next() {
|
||||
match c {
|
||||
'\\' if !in_single => {
|
||||
if let Some(&next) = chars.peek() {
|
||||
chars.next();
|
||||
current.push(next);
|
||||
}
|
||||
}
|
||||
'"' if !in_single => in_double = !in_double,
|
||||
'\'' if !in_double => in_single = !in_single,
|
||||
' ' if !in_double && !in_single => {
|
||||
if !current.is_empty() {
|
||||
args.push(std::mem::take(&mut current));
|
||||
}
|
||||
}
|
||||
_ => current.push(c),
|
||||
}
|
||||
}
|
||||
if !current.is_empty() {
|
||||
args.push(current);
|
||||
}
|
||||
args
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
@@ -3005,6 +3044,21 @@ mod tests {
|
||||
assert_eq!(cmd["maxDepth"], 3);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_snapshot_urls() {
|
||||
let cmd = parse_command(&args("snapshot -i --urls"), &default_flags()).unwrap();
|
||||
assert_eq!(cmd["action"], "snapshot");
|
||||
assert_eq!(cmd["interactive"], true);
|
||||
assert_eq!(cmd["urls"], true);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_snapshot_urls_short() {
|
||||
let cmd = parse_command(&args("snapshot -i -u"), &default_flags()).unwrap();
|
||||
assert_eq!(cmd["action"], "snapshot");
|
||||
assert_eq!(cmd["urls"], true);
|
||||
}
|
||||
|
||||
// === Wait ===
|
||||
|
||||
#[test]
|
||||
@@ -4360,4 +4414,41 @@ mod tests {
|
||||
assert_eq!(cmd["action"], "batch");
|
||||
assert_eq!(cmd["bail"], true);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_batch_with_args() {
|
||||
let cmd_args = vec![
|
||||
"batch".to_string(),
|
||||
"open https://example.com".to_string(),
|
||||
"screenshot".to_string(),
|
||||
];
|
||||
let cmd = parse_command(&cmd_args, &default_flags()).unwrap();
|
||||
assert_eq!(cmd["action"], "batch");
|
||||
assert_eq!(cmd["bail"], false);
|
||||
let commands = cmd["commands"].as_array().unwrap();
|
||||
assert_eq!(commands.len(), 2);
|
||||
assert_eq!(commands[0], "open https://example.com");
|
||||
assert_eq!(commands[1], "screenshot");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_batch_with_args_and_bail() {
|
||||
let cmd_args = vec![
|
||||
"batch".to_string(),
|
||||
"--bail".to_string(),
|
||||
"open https://example.com".to_string(),
|
||||
"screenshot".to_string(),
|
||||
];
|
||||
let cmd = parse_command(&cmd_args, &default_flags()).unwrap();
|
||||
assert_eq!(cmd["action"], "batch");
|
||||
assert_eq!(cmd["bail"], true);
|
||||
let commands = cmd["commands"].as_array().unwrap();
|
||||
assert_eq!(commands.len(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_batch_no_args_no_commands_field() {
|
||||
let cmd = parse_command(&args("batch"), &default_flags()).unwrap();
|
||||
assert!(cmd.get("commands").is_none());
|
||||
}
|
||||
}
|
||||
|
||||
+36
-26
@@ -1257,10 +1257,16 @@ fn main() {
|
||||
}
|
||||
}
|
||||
|
||||
// Handle batch command: read commands from stdin, execute sequentially
|
||||
// Handle batch command: from args or stdin
|
||||
if cmd.get("action").and_then(|v| v.as_str()) == Some("batch") {
|
||||
let bail = cmd.get("bail").and_then(|v| v.as_bool()).unwrap_or(false);
|
||||
run_batch(&flags, bail);
|
||||
let arg_commands = cmd.get("commands").and_then(|v| v.as_array()).map(|arr| {
|
||||
arr.iter()
|
||||
.filter_map(|v| v.as_str())
|
||||
.map(commands::shell_words_split)
|
||||
.collect::<Vec<Vec<String>>>()
|
||||
});
|
||||
run_batch(&flags, bail, arg_commands);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1340,36 +1346,40 @@ fn main() {
|
||||
}
|
||||
}
|
||||
|
||||
fn run_batch(flags: &Flags, bail: bool) {
|
||||
use std::io::Read as _;
|
||||
fn run_batch(flags: &Flags, bail: bool, arg_commands: Option<Vec<Vec<String>>>) {
|
||||
let commands: Vec<Vec<String>> = if let Some(cmds) = arg_commands {
|
||||
cmds
|
||||
} else {
|
||||
use std::io::Read as _;
|
||||
|
||||
let mut input = String::new();
|
||||
if let Err(e) = std::io::stdin().read_to_string(&mut input) {
|
||||
if flags.json {
|
||||
print_json_error(format!("Failed to read stdin: {}", e));
|
||||
} else {
|
||||
eprintln!("{} Failed to read stdin: {}", color::error_indicator(), e);
|
||||
}
|
||||
exit(1);
|
||||
}
|
||||
|
||||
let commands: Vec<Vec<String>> = match serde_json::from_str(&input) {
|
||||
Ok(c) => c,
|
||||
Err(e) => {
|
||||
let mut input = String::new();
|
||||
if let Err(e) = std::io::stdin().read_to_string(&mut input) {
|
||||
if flags.json {
|
||||
print_json_error(format!(
|
||||
"Invalid JSON input: {}. Expected an array of string arrays, e.g. [[\"open\", \"https://example.com\"], [\"snapshot\"]]",
|
||||
e
|
||||
));
|
||||
print_json_error(format!("Failed to read stdin: {}", e));
|
||||
} else {
|
||||
eprintln!(
|
||||
"{} Invalid JSON input: {}. Expected an array of string arrays.",
|
||||
color::error_indicator(),
|
||||
e
|
||||
);
|
||||
eprintln!("{} Failed to read stdin: {}", color::error_indicator(), e);
|
||||
}
|
||||
exit(1);
|
||||
}
|
||||
|
||||
match serde_json::from_str(&input) {
|
||||
Ok(c) => c,
|
||||
Err(e) => {
|
||||
if flags.json {
|
||||
print_json_error(format!(
|
||||
"Invalid JSON input: {}. Expected an array of string arrays, e.g. [[\"open\", \"https://example.com\"], [\"snapshot\"]]",
|
||||
e
|
||||
));
|
||||
} else {
|
||||
eprintln!(
|
||||
"{} Invalid JSON input: {}. Expected an array of string arrays.",
|
||||
color::error_indicator(),
|
||||
e
|
||||
);
|
||||
}
|
||||
exit(1);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
if commands.is_empty() {
|
||||
|
||||
@@ -2304,6 +2304,7 @@ async fn handle_snapshot(cmd: &Value, state: &mut DaemonState) -> Result<Value,
|
||||
.get("maxDepth")
|
||||
.and_then(|v| v.as_u64())
|
||||
.map(|d| d as usize),
|
||||
urls: cmd.get("urls").and_then(|v| v.as_bool()).unwrap_or(false),
|
||||
};
|
||||
|
||||
state.ref_map.clear();
|
||||
|
||||
@@ -80,6 +80,7 @@ pub struct SnapshotOptions {
|
||||
pub interactive: bool,
|
||||
pub compact: bool,
|
||||
pub depth: Option<usize>,
|
||||
pub urls: bool,
|
||||
}
|
||||
|
||||
struct TreeNode {
|
||||
@@ -98,7 +99,8 @@ struct TreeNode {
|
||||
has_ref: bool,
|
||||
ref_id: Option<String>,
|
||||
depth: usize,
|
||||
cursor_info: Option<CursorElementInfo>, // cursor-interactive information
|
||||
cursor_info: Option<CursorElementInfo>,
|
||||
url: Option<String>,
|
||||
}
|
||||
|
||||
impl TreeNode {
|
||||
@@ -121,10 +123,10 @@ impl TreeNode {
|
||||
ref_id: None,
|
||||
depth: 0,
|
||||
cursor_info: None,
|
||||
url: None,
|
||||
}
|
||||
}
|
||||
|
||||
// Clear node content
|
||||
fn clear(&mut self) {
|
||||
self.role = String::new();
|
||||
self.name = String::new();
|
||||
@@ -139,6 +141,7 @@ impl TreeNode {
|
||||
self.children.clear();
|
||||
self.parent_idx = None;
|
||||
self.has_ref = false;
|
||||
self.url = None;
|
||||
self.ref_id = None;
|
||||
self.depth = 0;
|
||||
self.cursor_info = None;
|
||||
@@ -383,6 +386,75 @@ pub async fn take_snapshot(
|
||||
|
||||
ref_map.set_next_ref_num(next_ref);
|
||||
|
||||
if options.urls {
|
||||
let link_nodes: Vec<(usize, i64)> = tree_nodes
|
||||
.iter()
|
||||
.enumerate()
|
||||
.filter(|(_, n)| n.role == "link" && n.has_ref && n.backend_node_id.is_some())
|
||||
.filter_map(|(i, n)| n.backend_node_id.map(|bid| (i, bid)))
|
||||
.collect();
|
||||
|
||||
if !link_nodes.is_empty() {
|
||||
// CDP has no batch resolve API, so we parallelize individual calls.
|
||||
// Phase 1: resolve all backend node IDs to JS object IDs in parallel.
|
||||
let resolve_futs = link_nodes.iter().map(|&(idx, bid)| async move {
|
||||
let resolved = client
|
||||
.send_command(
|
||||
"DOM.resolveNode",
|
||||
Some(serde_json::json!({ "backendNodeId": bid })),
|
||||
Some(session_id),
|
||||
)
|
||||
.await;
|
||||
let obj_id = resolved.ok().and_then(|r| {
|
||||
r.get("object")
|
||||
.and_then(|o| o.get("objectId"))
|
||||
.and_then(|v| v.as_str())
|
||||
.map(|s| s.to_string())
|
||||
});
|
||||
(idx, obj_id)
|
||||
});
|
||||
let resolved: Vec<(usize, Option<String>)> =
|
||||
futures_util::future::join_all(resolve_futs).await;
|
||||
|
||||
// Phase 2: fetch hrefs for all resolved objects in parallel.
|
||||
let href_futs: Vec<_> = resolved
|
||||
.iter()
|
||||
.filter_map(|(idx, obj_id)| {
|
||||
let oid = obj_id.as_ref()?;
|
||||
Some(async move {
|
||||
let result = client
|
||||
.send_command(
|
||||
"Runtime.callFunctionOn",
|
||||
Some(serde_json::json!({
|
||||
"objectId": oid,
|
||||
"functionDeclaration": "function() { return this.href || ''; }",
|
||||
"returnByValue": true,
|
||||
})),
|
||||
Some(session_id),
|
||||
)
|
||||
.await;
|
||||
let href = result.ok().and_then(|r| {
|
||||
r.get("result")
|
||||
.and_then(|r| r.get("value"))
|
||||
.and_then(|v| v.as_str())
|
||||
.filter(|s| !s.is_empty())
|
||||
.map(|s| s.to_string())
|
||||
});
|
||||
(*idx, href)
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
let hrefs: Vec<(usize, Option<String>)> =
|
||||
futures_util::future::join_all(href_futs).await;
|
||||
|
||||
for (idx, href) in hrefs {
|
||||
if let Some(url) = href {
|
||||
tree_nodes[idx].url = Some(url);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let mut output = String::new();
|
||||
for &root_idx in &effective_roots {
|
||||
render_tree(&tree_nodes, root_idx, 0, &mut output, options);
|
||||
@@ -797,6 +869,7 @@ fn build_tree(nodes: &[AXNode]) -> (Vec<TreeNode>, Vec<usize>) {
|
||||
ref_id: None,
|
||||
depth: 0,
|
||||
cursor_info: None,
|
||||
url: None,
|
||||
});
|
||||
id_to_idx.insert(node.node_id.clone(), i);
|
||||
}
|
||||
@@ -993,6 +1066,10 @@ fn render_tree(
|
||||
attrs.push(format!("ref={}", ref_id));
|
||||
}
|
||||
|
||||
if let Some(ref url) = node.url {
|
||||
attrs.push(format!("url={}", url));
|
||||
}
|
||||
|
||||
if !attrs.is_empty() {
|
||||
line.push_str(&format!(" [{}]", attrs.join(", ")));
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,325 @@
|
||||
use serde_json::{json, Value};
|
||||
use std::sync::Arc;
|
||||
|
||||
use tokio::sync::{broadcast, watch, Mutex, RwLock};
|
||||
|
||||
use crate::native::cdp::client::CdpClient;
|
||||
use crate::native::network;
|
||||
|
||||
use super::timestamp_ms;
|
||||
|
||||
/// Background task that subscribes to CDP events and broadcasts screencast frames in real-time.
|
||||
/// Also handles auto-start/stop of screencast based on WebSocket client count.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub(super) async fn cdp_event_loop(
|
||||
frame_tx: broadcast::Sender<String>,
|
||||
client_slot: Arc<RwLock<Option<Arc<CdpClient>>>>,
|
||||
client_notify: Arc<tokio::sync::Notify>,
|
||||
screencasting: Arc<Mutex<bool>>,
|
||||
client_count: Arc<Mutex<usize>>,
|
||||
cdp_session_id: Arc<RwLock<Option<String>>>,
|
||||
viewport_width: Arc<Mutex<u32>>,
|
||||
viewport_height: Arc<Mutex<u32>>,
|
||||
last_frame: Arc<RwLock<Option<String>>>,
|
||||
last_tabs: Arc<RwLock<Vec<Value>>>,
|
||||
last_engine: Arc<RwLock<String>>,
|
||||
recording: Arc<Mutex<bool>>,
|
||||
mut shutdown_rx: watch::Receiver<bool>,
|
||||
) {
|
||||
loop {
|
||||
tokio::select! {
|
||||
changed = shutdown_rx.changed() => {
|
||||
if changed.is_err() || *shutdown_rx.borrow() {
|
||||
let session_id = cdp_session_id.read().await.clone();
|
||||
if *screencasting.lock().await {
|
||||
if let Some(ref client) = *client_slot.read().await {
|
||||
let _ = client
|
||||
.send_command_no_params("Page.stopScreencast", session_id.as_deref())
|
||||
.await;
|
||||
}
|
||||
let mut sc = screencasting.lock().await;
|
||||
*sc = false;
|
||||
}
|
||||
return;
|
||||
}
|
||||
}
|
||||
_ = client_notify.notified() => {}
|
||||
}
|
||||
|
||||
let count = *client_count.lock().await;
|
||||
let guard = client_slot.read().await;
|
||||
|
||||
if count > 0 {
|
||||
if let Some(ref client) = *guard {
|
||||
let mut event_rx = client.subscribe();
|
||||
let client_arc = Arc::clone(client);
|
||||
drop(guard);
|
||||
|
||||
let session_id = cdp_session_id.read().await.clone();
|
||||
|
||||
let vw = *viewport_width.lock().await;
|
||||
let vh = *viewport_height.lock().await;
|
||||
|
||||
let eng = last_engine.read().await.clone();
|
||||
let supports_screencast = eng == "chrome";
|
||||
|
||||
if supports_screencast {
|
||||
let _ = client_arc
|
||||
.send_command(
|
||||
"Page.startScreencast",
|
||||
Some(json!({
|
||||
"format": "jpeg",
|
||||
"quality": 80,
|
||||
"maxWidth": vw,
|
||||
"maxHeight": vh,
|
||||
"everyNthFrame": 1,
|
||||
})),
|
||||
session_id.as_deref(),
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
{
|
||||
let mut sc = screencasting.lock().await;
|
||||
*sc = supports_screencast;
|
||||
}
|
||||
|
||||
let rec = *recording.lock().await;
|
||||
let status = json!({
|
||||
"type": "status",
|
||||
"connected": true,
|
||||
"screencasting": supports_screencast,
|
||||
"viewportWidth": vw,
|
||||
"viewportHeight": vh,
|
||||
"engine": eng,
|
||||
"recording": rec,
|
||||
});
|
||||
let _ = frame_tx.send(status.to_string());
|
||||
|
||||
loop {
|
||||
tokio::select! {
|
||||
changed = shutdown_rx.changed() => {
|
||||
if changed.is_err() || *shutdown_rx.borrow() {
|
||||
if supports_screencast {
|
||||
let session_id = cdp_session_id.read().await.clone();
|
||||
let _ = client_arc
|
||||
.send_command_no_params("Page.stopScreencast", session_id.as_deref())
|
||||
.await;
|
||||
}
|
||||
let mut sc = screencasting.lock().await;
|
||||
*sc = false;
|
||||
return;
|
||||
}
|
||||
}
|
||||
event = event_rx.recv() => {
|
||||
match event {
|
||||
Ok(evt) => {
|
||||
if evt.method == "Page.frameNavigated" {
|
||||
if let Some(frame) = evt.params.get("frame") {
|
||||
let is_main = frame
|
||||
.get("parentId")
|
||||
.and_then(|v| v.as_str())
|
||||
.is_none_or(|s| s.is_empty());
|
||||
if is_main {
|
||||
if let Some(url) = frame.get("url").and_then(|v| v.as_str()) {
|
||||
{
|
||||
let mut tabs = last_tabs.write().await;
|
||||
for tab in tabs.iter_mut() {
|
||||
if tab.get("active").and_then(|v| v.as_bool()).unwrap_or(false) {
|
||||
tab.as_object_mut().map(|o| o.insert("url".to_string(), json!(url)));
|
||||
}
|
||||
}
|
||||
}
|
||||
let msg = json!({
|
||||
"type": "url",
|
||||
"url": url,
|
||||
"timestamp": timestamp_ms(),
|
||||
});
|
||||
let _ = frame_tx.send(msg.to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
} else if evt.method == "Page.screencastFrame" {
|
||||
if let Some(sid) = evt.params.get("sessionId").and_then(|v| v.as_i64()) {
|
||||
let _ = client_arc.send_command(
|
||||
"Page.screencastFrameAck",
|
||||
Some(json!({ "sessionId": sid })),
|
||||
evt.session_id.as_deref(),
|
||||
).await;
|
||||
}
|
||||
|
||||
if let Some(data) = evt.params.get("data").and_then(|v| v.as_str()) {
|
||||
let meta = evt.params.get("metadata");
|
||||
let msg = json!({
|
||||
"type": "frame",
|
||||
"data": data,
|
||||
"metadata": {
|
||||
"offsetTop": meta.and_then(|m| m.get("offsetTop")).and_then(|v| v.as_f64()).unwrap_or(0.0),
|
||||
"pageScaleFactor": meta.and_then(|m| m.get("pageScaleFactor")).and_then(|v| v.as_f64()).unwrap_or(1.0),
|
||||
"deviceWidth": meta.and_then(|m| m.get("deviceWidth")).and_then(|v| v.as_u64()).unwrap_or(1280),
|
||||
"deviceHeight": meta.and_then(|m| m.get("deviceHeight")).and_then(|v| v.as_u64()).unwrap_or(720),
|
||||
"scrollOffsetX": meta.and_then(|m| m.get("scrollOffsetX")).and_then(|v| v.as_f64()).unwrap_or(0.0),
|
||||
"scrollOffsetY": meta.and_then(|m| m.get("scrollOffsetY")).and_then(|v| v.as_f64()).unwrap_or(0.0),
|
||||
"timestamp": meta.and_then(|m| m.get("timestamp")).and_then(|v| v.as_u64()).unwrap_or(0),
|
||||
}
|
||||
});
|
||||
let msg_str = msg.to_string();
|
||||
{
|
||||
let mut lf = last_frame.write().await;
|
||||
*lf = Some(msg_str.clone());
|
||||
}
|
||||
let _ = frame_tx.send(msg_str);
|
||||
}
|
||||
} else if evt.method == "Runtime.consoleAPICalled" {
|
||||
let level = evt.params.get("type")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("log");
|
||||
let raw_args = evt.params.get("args")
|
||||
.and_then(|v| v.as_array())
|
||||
.cloned()
|
||||
.unwrap_or_default();
|
||||
let text = network::format_console_args(&raw_args);
|
||||
if !text.is_empty() {
|
||||
let mut msg = json!({
|
||||
"type": "console",
|
||||
"level": level,
|
||||
"text": text,
|
||||
"timestamp": timestamp_ms(),
|
||||
});
|
||||
if !raw_args.is_empty() {
|
||||
msg.as_object_mut().unwrap().insert(
|
||||
"args".to_string(),
|
||||
Value::Array(raw_args),
|
||||
);
|
||||
}
|
||||
let _ = frame_tx.send(msg.to_string());
|
||||
}
|
||||
} else if evt.method == "Runtime.exceptionThrown" {
|
||||
let text = evt.params.get("exceptionDetails")
|
||||
.and_then(|d| {
|
||||
d.get("exception")
|
||||
.and_then(|e| e.get("description").and_then(|v| v.as_str()))
|
||||
.or_else(|| d.get("text").and_then(|v| v.as_str()))
|
||||
})
|
||||
.unwrap_or("Unknown error");
|
||||
let line = evt.params.get("exceptionDetails")
|
||||
.and_then(|d| d.get("lineNumber").and_then(|v| v.as_i64()));
|
||||
let column = evt.params.get("exceptionDetails")
|
||||
.and_then(|d| d.get("columnNumber").and_then(|v| v.as_i64()));
|
||||
let msg = json!({
|
||||
"type": "page_error",
|
||||
"text": text,
|
||||
"line": line,
|
||||
"column": column,
|
||||
"timestamp": timestamp_ms(),
|
||||
});
|
||||
let _ = frame_tx.send(msg.to_string());
|
||||
}
|
||||
}
|
||||
Err(broadcast::error::RecvError::Lagged(_)) => continue,
|
||||
Err(broadcast::error::RecvError::Closed) => break,
|
||||
}
|
||||
}
|
||||
_ = client_notify.notified() => {
|
||||
let count = *client_count.lock().await;
|
||||
let new_session_id = cdp_session_id.read().await.clone();
|
||||
if count == 0 {
|
||||
if supports_screencast {
|
||||
let _ = client_arc
|
||||
.send_command_no_params("Page.stopScreencast", session_id.as_deref())
|
||||
.await;
|
||||
}
|
||||
let mut sc = screencasting.lock().await;
|
||||
*sc = false;
|
||||
break;
|
||||
}
|
||||
let client_changed = {
|
||||
let guard = client_slot.read().await;
|
||||
let same = guard
|
||||
.as_ref()
|
||||
.is_some_and(|c| Arc::ptr_eq(c, &client_arc));
|
||||
!same
|
||||
};
|
||||
let session_changed = new_session_id != session_id;
|
||||
let new_vw = *viewport_width.lock().await;
|
||||
let new_vh = *viewport_height.lock().await;
|
||||
let viewport_changed = new_vw != vw || new_vh != vh;
|
||||
if client_changed || session_changed || viewport_changed {
|
||||
if supports_screencast {
|
||||
let _ = client_arc
|
||||
.send_command_no_params("Page.stopScreencast", session_id.as_deref())
|
||||
.await;
|
||||
}
|
||||
let mut sc = screencasting.lock().await;
|
||||
*sc = false;
|
||||
client_notify.notify_one();
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
drop(guard);
|
||||
}
|
||||
} else {
|
||||
let was_screencasting = *screencasting.lock().await;
|
||||
if was_screencasting {
|
||||
if let Some(ref client) = *guard {
|
||||
let session_id = cdp_session_id.read().await.clone();
|
||||
let _ = client
|
||||
.send_command_no_params("Page.stopScreencast", session_id.as_deref())
|
||||
.await;
|
||||
}
|
||||
let mut sc = screencasting.lock().await;
|
||||
*sc = false;
|
||||
}
|
||||
drop(guard);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub async fn start_screencast(
|
||||
client: &CdpClient,
|
||||
session_id: &str,
|
||||
format: &str,
|
||||
quality: i32,
|
||||
max_width: i32,
|
||||
max_height: i32,
|
||||
) -> Result<(), String> {
|
||||
client
|
||||
.send_command(
|
||||
"Page.startScreencast",
|
||||
Some(json!({
|
||||
"format": format,
|
||||
"quality": quality,
|
||||
"maxWidth": max_width,
|
||||
"maxHeight": max_height,
|
||||
"everyNthFrame": 1,
|
||||
})),
|
||||
Some(session_id),
|
||||
)
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub async fn stop_screencast(client: &CdpClient, session_id: &str) -> Result<(), String> {
|
||||
client
|
||||
.send_command_no_params("Page.stopScreencast", Some(session_id))
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub async fn ack_screencast_frame(
|
||||
client: &CdpClient,
|
||||
session_id: &str,
|
||||
screencast_session_id: i64,
|
||||
) -> Result<(), String> {
|
||||
client
|
||||
.send_command(
|
||||
"Page.screencastFrameAck",
|
||||
Some(json!({ "sessionId": screencast_session_id })),
|
||||
Some(session_id),
|
||||
)
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
@@ -0,0 +1,970 @@
|
||||
use std::sync::OnceLock;
|
||||
|
||||
use serde_json::{json, Value};
|
||||
|
||||
use tokio::io::AsyncWriteExt;
|
||||
|
||||
use super::http::cors_headers_for_origin;
|
||||
|
||||
const DEFAULT_AI_GATEWAY_URL: &str = "https://ai-gateway.vercel.sh";
|
||||
|
||||
static HTTP_CLIENT: OnceLock<reqwest::Client> = OnceLock::new();
|
||||
|
||||
fn http_client() -> &'static reqwest::Client {
|
||||
HTTP_CLIENT.get_or_init(reqwest::Client::new)
|
||||
}
|
||||
|
||||
fn is_chat_enabled() -> bool {
|
||||
std::env::var("AI_GATEWAY_API_KEY").is_ok()
|
||||
}
|
||||
|
||||
pub(super) fn chat_status_json() -> String {
|
||||
let enabled = is_chat_enabled();
|
||||
let mut obj = json!({ "enabled": enabled });
|
||||
if enabled {
|
||||
if let Ok(model) = std::env::var("AI_GATEWAY_MODEL") {
|
||||
obj["model"] = Value::String(model);
|
||||
}
|
||||
}
|
||||
obj.to_string()
|
||||
}
|
||||
|
||||
pub(super) async fn handle_models_request(
|
||||
stream: &mut tokio::net::TcpStream,
|
||||
origin: Option<&str>,
|
||||
) {
|
||||
let cors = cors_headers_for_origin(origin);
|
||||
let gateway_url = std::env::var("AI_GATEWAY_URL")
|
||||
.unwrap_or_else(|_| DEFAULT_AI_GATEWAY_URL.to_string())
|
||||
.trim_end_matches('/')
|
||||
.to_string();
|
||||
let api_key = match std::env::var("AI_GATEWAY_API_KEY") {
|
||||
Ok(k) => k,
|
||||
Err(_) => {
|
||||
let body = r#"{"data":[]}"#;
|
||||
let resp = format!(
|
||||
"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n{cors}\r\n",
|
||||
body.len()
|
||||
);
|
||||
let _ = stream.write_all(resp.as_bytes()).await;
|
||||
let _ = stream.write_all(body.as_bytes()).await;
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
let url = format!("{}/v1/models", gateway_url);
|
||||
let client = http_client();
|
||||
let result = client
|
||||
.get(&url)
|
||||
.header("Authorization", format!("Bearer {}", api_key))
|
||||
.send()
|
||||
.await;
|
||||
|
||||
let body = match result {
|
||||
Ok(r) if r.status().is_success() => r
|
||||
.text()
|
||||
.await
|
||||
.unwrap_or_else(|_| r#"{"data":[]}"#.to_string()),
|
||||
_ => r#"{"data":[]}"#.to_string(),
|
||||
};
|
||||
|
||||
let resp = format!(
|
||||
"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n{cors}\r\n",
|
||||
body.len()
|
||||
);
|
||||
let _ = stream.write_all(resp.as_bytes()).await;
|
||||
let _ = stream.write_all(body.as_bytes()).await;
|
||||
}
|
||||
|
||||
const SKILL_NAMES: &[&str] = &["agent-browser", "slack", "electron", "dogfood", "agentcore"];
|
||||
|
||||
/// Locate the `skills/` directory by walking up from the executable.
|
||||
/// Works for npm installs (binary in `bin/`, skills at `../skills/`) and
|
||||
/// dev builds (binary deep in `cli/target/`, skills at repo root).
|
||||
fn find_skills_dir() -> Option<std::path::PathBuf> {
|
||||
let exe = std::env::current_exe().ok()?;
|
||||
let real = exe.canonicalize().unwrap_or(exe);
|
||||
let mut dir = real.parent();
|
||||
while let Some(d) = dir {
|
||||
let candidate = d.join("skills");
|
||||
if candidate.join("agent-browser").join("SKILL.md").exists() {
|
||||
return Some(candidate);
|
||||
}
|
||||
dir = d.parent();
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
fn load_skills() -> Vec<(String, String)> {
|
||||
let Some(skills_dir) = find_skills_dir() else {
|
||||
return Vec::new();
|
||||
};
|
||||
SKILL_NAMES
|
||||
.iter()
|
||||
.filter_map(|name| {
|
||||
let path = skills_dir.join(name).join("SKILL.md");
|
||||
let content = std::fs::read_to_string(&path).ok()?;
|
||||
Some((name.to_string(), content))
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn strip_frontmatter(s: &str) -> &str {
|
||||
if !s.starts_with("---") {
|
||||
return s;
|
||||
}
|
||||
if let Some(end) = s[3..].find("---") {
|
||||
let after = &s[3 + end + 3..];
|
||||
after.trim_start_matches(['\n', '\r'])
|
||||
} else {
|
||||
s
|
||||
}
|
||||
}
|
||||
|
||||
fn get_system_prompt() -> &'static str {
|
||||
static PROMPT: OnceLock<String> = OnceLock::new();
|
||||
PROMPT.get_or_init(|| {
|
||||
let skills = load_skills();
|
||||
|
||||
let mut sections = String::new();
|
||||
for (name, content) in &skills {
|
||||
let body = strip_frontmatter(content);
|
||||
sections.push_str(&format!("\n\n<skill name=\"{}\">\n{}\n</skill>", name, body.trim()));
|
||||
}
|
||||
|
||||
format!(
|
||||
r#"You are an AI assistant that controls a browser through agent-browser. You have an active browser session, but you can also create new sessions.
|
||||
|
||||
RULES:
|
||||
- You MUST use the agent_browser tool for every browser action. NEVER claim you performed an action without calling the tool.
|
||||
- If the user asks you to do something, call the tool first, then describe the result.
|
||||
- If a request is outside your capabilities (e.g. system operations), say so honestly. Do not improvise or pretend.
|
||||
- One tool call per command. Do not chain with `&&` or `;`.
|
||||
- Do not add `--json`.
|
||||
- Do not run non-agent-browser programs.
|
||||
- Keep responses concise.
|
||||
- For screenshots, omit the path argument so they save to the default location (which will be displayed inline). Screenshots from tool calls are ALREADY shown to the user. Do NOT re-display them with markdown image syntax in your text response. Never use `![...]()` to reference screenshots.
|
||||
- To create a new session: add `--session <name>` to any command (e.g. `agent-browser --session my-session open https://example.com`). If the session does not exist, it will be created automatically.
|
||||
- To use a different browser engine: add `--engine <engine>` (e.g. `agent-browser --session lp-session --engine lightpanda open https://example.com`). Supported engines: chrome (default), lightpanda.
|
||||
|
||||
The following skill references describe agent-browser capabilities in detail. Use them when deciding which commands to run and how to approach tasks.
|
||||
{sections}"#,
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
const CHAT_TOOLS: &str = r#"[{"type":"function","function":{"name":"agent_browser","description":"Execute an agent-browser command. Runs against the active session by default. Add --session <name> to target or create a different session, and --engine <engine> to choose a browser engine.","parameters":{"type":"object","properties":{"command":{"type":"string","description":"The command to execute, e.g. 'agent-browser open https://google.com' or 'agent-browser --session new-session open https://example.com' or 'agent-browser snapshot -i' or 'agent-browser click @e3'"}},"required":["command"]}}}]"#;
|
||||
|
||||
const COMPACT_THRESHOLD_CHARS: usize = 200_000;
|
||||
const KEEP_RECENT_MESSAGES: usize = 6;
|
||||
|
||||
fn estimate_chars(messages: &[Value]) -> usize {
|
||||
messages
|
||||
.iter()
|
||||
.map(|m| {
|
||||
let content_len = m
|
||||
.get("content")
|
||||
.map(|c| {
|
||||
if let Some(s) = c.as_str() {
|
||||
s.len()
|
||||
} else {
|
||||
c.to_string().len()
|
||||
}
|
||||
})
|
||||
.unwrap_or(0);
|
||||
let tc_len = m
|
||||
.get("tool_calls")
|
||||
.map(|t| t.to_string().len())
|
||||
.unwrap_or(0);
|
||||
content_len + tc_len
|
||||
})
|
||||
.sum()
|
||||
}
|
||||
|
||||
fn find_safe_split(messages: &[Value], keep_recent: usize) -> usize {
|
||||
if messages.len() <= keep_recent + 1 {
|
||||
return 1;
|
||||
}
|
||||
let desired = messages.len() - keep_recent;
|
||||
for i in (1..=desired).rev() {
|
||||
if messages[i].get("role").and_then(|r| r.as_str()) == Some("user") {
|
||||
return i;
|
||||
}
|
||||
}
|
||||
desired.max(1)
|
||||
}
|
||||
|
||||
fn build_summary_text(messages: &[Value]) -> String {
|
||||
let mut text = String::new();
|
||||
for msg in messages {
|
||||
let role = msg
|
||||
.get("role")
|
||||
.and_then(|r| r.as_str())
|
||||
.unwrap_or("unknown");
|
||||
if let Some(content) = msg.get("content").and_then(|c| c.as_str()) {
|
||||
if !content.is_empty() {
|
||||
let truncated = if content.len() > 2000 {
|
||||
format!("{}...[truncated]", &content[..2000])
|
||||
} else {
|
||||
content.to_string()
|
||||
};
|
||||
text.push_str(&format!("[{}] {}\n\n", role, truncated));
|
||||
}
|
||||
}
|
||||
if let Some(tcs) = msg.get("tool_calls").and_then(|t| t.as_array()) {
|
||||
for tc in tcs {
|
||||
let name = tc
|
||||
.get("function")
|
||||
.and_then(|f| f.get("name"))
|
||||
.and_then(|n| n.as_str())
|
||||
.unwrap_or("");
|
||||
let args = tc
|
||||
.get("function")
|
||||
.and_then(|f| f.get("arguments"))
|
||||
.and_then(|a| a.as_str())
|
||||
.unwrap_or("");
|
||||
text.push_str(&format!("[assistant tool:{}] {}\n", name, args));
|
||||
}
|
||||
}
|
||||
}
|
||||
text
|
||||
}
|
||||
|
||||
async fn summarize_for_compaction(
|
||||
client: &reqwest::Client,
|
||||
url: &str,
|
||||
api_key: &str,
|
||||
model: &str,
|
||||
messages: &[Value],
|
||||
) -> Option<String> {
|
||||
let conversation = build_summary_text(messages);
|
||||
if conversation.is_empty() {
|
||||
return None;
|
||||
}
|
||||
|
||||
let body = json!({
|
||||
"model": model,
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"content": "Summarize this browser automation conversation concisely. Preserve: URLs visited, actions performed, current page state, errors encountered, and user goals. Output only the summary."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": conversation
|
||||
}
|
||||
],
|
||||
"max_tokens": 1024,
|
||||
"stream": false,
|
||||
});
|
||||
|
||||
let resp = client
|
||||
.post(url)
|
||||
.header("Authorization", format!("Bearer {}", api_key))
|
||||
.header("Content-Type", "application/json")
|
||||
.body(body.to_string())
|
||||
.send()
|
||||
.await
|
||||
.ok()?;
|
||||
|
||||
if !resp.status().is_success() {
|
||||
return None;
|
||||
}
|
||||
|
||||
let result: Value = resp.json().await.ok()?;
|
||||
result
|
||||
.get("choices")
|
||||
.and_then(|c| c.get(0))
|
||||
.and_then(|c| c.get("message"))
|
||||
.and_then(|m| m.get("content"))
|
||||
.and_then(|c| c.as_str())
|
||||
.map(|s| s.to_string())
|
||||
}
|
||||
|
||||
const SCREENSHOT_MAX_WIDTH: u32 = 1024;
|
||||
const SCREENSHOT_JPEG_QUALITY: u8 = 40;
|
||||
|
||||
fn compress_image_to_jpeg(raw_bytes: &[u8]) -> Option<Vec<u8>> {
|
||||
let img = image::load_from_memory(raw_bytes).ok()?;
|
||||
let img = if img.width() > SCREENSHOT_MAX_WIDTH {
|
||||
img.resize(
|
||||
SCREENSHOT_MAX_WIDTH,
|
||||
u32::MAX,
|
||||
image::imageops::FilterType::Triangle,
|
||||
)
|
||||
} else {
|
||||
img
|
||||
};
|
||||
let mut buf = std::io::Cursor::new(Vec::new());
|
||||
let encoder =
|
||||
image::codecs::jpeg::JpegEncoder::new_with_quality(&mut buf, SCREENSHOT_JPEG_QUALITY);
|
||||
img.write_with_encoder(encoder).ok()?;
|
||||
Some(buf.into_inner())
|
||||
}
|
||||
|
||||
fn has_image_extension(s: &str) -> bool {
|
||||
let lower = s.to_lowercase();
|
||||
lower.ends_with(".png") || lower.ends_with(".jpg") || lower.ends_with(".jpeg")
|
||||
}
|
||||
|
||||
fn extract_image_path(text: &str) -> Option<String> {
|
||||
for line in text.lines() {
|
||||
let trimmed = line.trim();
|
||||
// Whole line is a path (handles paths with spaces)
|
||||
if has_image_extension(trimmed) && std::path::Path::new(trimmed).exists() {
|
||||
return Some(trimmed.to_string());
|
||||
}
|
||||
for suffix in [".png", ".jpg", ".jpeg"] {
|
||||
if let Some(pos) = trimmed.to_lowercase().rfind(suffix) {
|
||||
let end = pos + suffix.len();
|
||||
let candidate = &trimmed[..end];
|
||||
let start = candidate
|
||||
.rfind(|c: char| c.is_whitespace())
|
||||
.map(|i| i + 1)
|
||||
.unwrap_or(0);
|
||||
let path = &candidate[start..];
|
||||
if !path.is_empty() && std::path::Path::new(path).exists() {
|
||||
return Some(path.to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
fn enrich_tool_output(result: &str) -> String {
|
||||
let Some(path) = extract_image_path(result) else {
|
||||
return result.to_string();
|
||||
};
|
||||
|
||||
let Ok(raw_bytes) = std::fs::read(&path) else {
|
||||
return result.to_string();
|
||||
};
|
||||
|
||||
let (jpeg_bytes, mime) = match compress_image_to_jpeg(&raw_bytes) {
|
||||
Some(compressed) => (compressed, "image/jpeg"),
|
||||
None => {
|
||||
let lower = path.to_lowercase();
|
||||
(
|
||||
raw_bytes,
|
||||
if lower.ends_with(".png") {
|
||||
"image/png"
|
||||
} else {
|
||||
"image/jpeg"
|
||||
},
|
||||
)
|
||||
}
|
||||
};
|
||||
|
||||
let b64 = base64::Engine::encode(&base64::engine::general_purpose::STANDARD, &jpeg_bytes);
|
||||
let data_url = format!("data:{};base64,{}", mime, b64);
|
||||
|
||||
json!({
|
||||
"text": result,
|
||||
"image": data_url
|
||||
})
|
||||
.to_string()
|
||||
}
|
||||
|
||||
const ALLOWED_COMMANDS: &[&str] = &[
|
||||
"open",
|
||||
"goto",
|
||||
"navigate",
|
||||
"back",
|
||||
"forward",
|
||||
"reload",
|
||||
"click",
|
||||
"dblclick",
|
||||
"fill",
|
||||
"type",
|
||||
"hover",
|
||||
"focus",
|
||||
"check",
|
||||
"uncheck",
|
||||
"select",
|
||||
"drag",
|
||||
"upload",
|
||||
"download",
|
||||
"press",
|
||||
"key",
|
||||
"keydown",
|
||||
"keyup",
|
||||
"keyboard",
|
||||
"scroll",
|
||||
"scrollintoview",
|
||||
"scrollinto",
|
||||
"wait",
|
||||
"screenshot",
|
||||
"pdf",
|
||||
"snapshot",
|
||||
"eval",
|
||||
"close",
|
||||
"quit",
|
||||
"exit",
|
||||
"inspect",
|
||||
"auth",
|
||||
"confirm",
|
||||
"deny",
|
||||
"connect",
|
||||
"cookies",
|
||||
"storage",
|
||||
"window",
|
||||
"frame",
|
||||
"dialog",
|
||||
"trace",
|
||||
"profiler",
|
||||
"record",
|
||||
"har",
|
||||
"network",
|
||||
"title",
|
||||
"url",
|
||||
"console",
|
||||
"errors",
|
||||
"highlight",
|
||||
"state",
|
||||
"emulate",
|
||||
"video",
|
||||
"tap",
|
||||
"swipe",
|
||||
"device",
|
||||
"batch",
|
||||
"diff",
|
||||
"find",
|
||||
"role",
|
||||
"text",
|
||||
"label",
|
||||
"placeholder",
|
||||
"alt",
|
||||
"testid",
|
||||
"first",
|
||||
"last",
|
||||
"nth",
|
||||
"mouse",
|
||||
"touchscreen",
|
||||
"attribute",
|
||||
"property",
|
||||
"set",
|
||||
"get",
|
||||
"is",
|
||||
"stream",
|
||||
"tab",
|
||||
"clipboard",
|
||||
"session",
|
||||
];
|
||||
|
||||
const ALLOWED_GLOBAL_FLAGS: &[&str] = &["--session", "--engine"];
|
||||
|
||||
async fn execute_chat_tool(session: &str, command: &str) -> String {
|
||||
let exe = match std::env::current_exe() {
|
||||
Ok(p) => p,
|
||||
Err(e) => return format!("Failed to resolve executable: {}", e),
|
||||
};
|
||||
|
||||
let single = command.split("&&").next().unwrap_or(command);
|
||||
let single = single.split(';').next().unwrap_or(single).trim();
|
||||
let stripped = single.strip_prefix("agent-browser ").unwrap_or(single);
|
||||
let words = crate::commands::shell_words_split(stripped);
|
||||
|
||||
let mut global_flags: Vec<String> = Vec::new();
|
||||
let mut cmd_words: Vec<String> = Vec::new();
|
||||
let mut has_session_flag = false;
|
||||
let mut i = 0;
|
||||
while i < words.len() {
|
||||
if ALLOWED_GLOBAL_FLAGS.contains(&words[i].as_str()) {
|
||||
if words[i] == "--session" {
|
||||
has_session_flag = true;
|
||||
}
|
||||
global_flags.push(words[i].clone());
|
||||
if i + 1 < words.len() {
|
||||
global_flags.push(words[i + 1].clone());
|
||||
i += 2;
|
||||
} else {
|
||||
i += 1;
|
||||
}
|
||||
} else {
|
||||
cmd_words.push(words[i].clone());
|
||||
i += 1;
|
||||
}
|
||||
}
|
||||
|
||||
let first_cmd = cmd_words.first().map(|s| s.as_str()).unwrap_or("");
|
||||
if !ALLOWED_COMMANDS.contains(&first_cmd) {
|
||||
return format!(
|
||||
"Blocked: '{}' is not a valid agent-browser command.",
|
||||
first_cmd
|
||||
);
|
||||
}
|
||||
|
||||
let mut args: Vec<String> = Vec::new();
|
||||
if !has_session_flag {
|
||||
args.push("--session".into());
|
||||
args.push(session.into());
|
||||
}
|
||||
args.extend(global_flags);
|
||||
args.extend(cmd_words);
|
||||
|
||||
let mut cmd = tokio::process::Command::new(&exe);
|
||||
cmd.args(&args)
|
||||
.env_remove("AGENT_BROWSER_DASHBOARD")
|
||||
.env_remove("AGENT_BROWSER_DASHBOARD_PORT")
|
||||
.env_remove("AGENT_BROWSER_STREAM_PORT");
|
||||
|
||||
match cmd.output().await {
|
||||
Ok(output) => {
|
||||
let stdout = String::from_utf8_lossy(&output.stdout).trim().to_string();
|
||||
let stderr = String::from_utf8_lossy(&output.stderr).trim().to_string();
|
||||
if stdout.is_empty() && !stderr.is_empty() {
|
||||
stderr
|
||||
} else if stdout.is_empty() {
|
||||
"Command completed with no output.".to_string()
|
||||
} else {
|
||||
stdout
|
||||
}
|
||||
}
|
||||
Err(e) => format!("Failed to execute command: {}", e),
|
||||
}
|
||||
}
|
||||
|
||||
async fn stream_gateway_response(
|
||||
stream: &mut tokio::net::TcpStream,
|
||||
gw_response: reqwest::Response,
|
||||
) -> Vec<(String, String, String)> {
|
||||
use futures_util::StreamExt as _;
|
||||
|
||||
let mut text_part_id = uuid::Uuid::new_v4().to_string();
|
||||
let mut text_started = false;
|
||||
let mut tool_calls: Vec<(String, String, String)> = Vec::new();
|
||||
let mut tool_call_args: std::collections::HashMap<usize, (String, String, String)> =
|
||||
std::collections::HashMap::new();
|
||||
let mut byte_stream = gw_response.bytes_stream();
|
||||
let mut buffer = String::new();
|
||||
|
||||
while let Some(chunk_result) = byte_stream.next().await {
|
||||
let chunk = match chunk_result {
|
||||
Ok(c) => c,
|
||||
Err(_) => break,
|
||||
};
|
||||
|
||||
buffer.push_str(&String::from_utf8_lossy(&chunk));
|
||||
|
||||
while let Some(newline_pos) = buffer.find('\n') {
|
||||
let line = buffer[..newline_pos].trim_end_matches('\r').to_string();
|
||||
buffer = buffer[newline_pos + 1..].to_string();
|
||||
|
||||
if line.is_empty() {
|
||||
continue;
|
||||
}
|
||||
let Some(data) = line.strip_prefix("data: ") else {
|
||||
continue;
|
||||
};
|
||||
if data == "[DONE]" {
|
||||
if text_started {
|
||||
let ev = format!("data: {}\n\n", json!({"type":"text-end","id":text_part_id}));
|
||||
let _ = stream.write_all(ev.as_bytes()).await;
|
||||
}
|
||||
let mut indices: Vec<usize> = tool_call_args.keys().copied().collect();
|
||||
indices.sort();
|
||||
for idx in indices {
|
||||
if let Some(tc) = tool_call_args.remove(&idx) {
|
||||
tool_calls.push(tc);
|
||||
}
|
||||
}
|
||||
return tool_calls;
|
||||
}
|
||||
let Ok(sse_json) = serde_json::from_str::<Value>(data) else {
|
||||
continue;
|
||||
};
|
||||
let delta = sse_json
|
||||
.get("choices")
|
||||
.and_then(|c| c.get(0))
|
||||
.and_then(|c| c.get("delta"));
|
||||
let Some(delta) = delta else { continue };
|
||||
|
||||
if let Some(text) = delta.get("content").and_then(|c| c.as_str()) {
|
||||
if !text.is_empty() {
|
||||
if !text_started {
|
||||
let ev = format!(
|
||||
"data: {}\n\n",
|
||||
json!({"type":"text-start","id":text_part_id})
|
||||
);
|
||||
if stream.write_all(ev.as_bytes()).await.is_err() {
|
||||
return tool_calls;
|
||||
}
|
||||
text_started = true;
|
||||
}
|
||||
let ev = format!(
|
||||
"data: {}\n\n",
|
||||
json!({"type":"text-delta","id":text_part_id,"delta":text})
|
||||
);
|
||||
if stream.write_all(ev.as_bytes()).await.is_err() {
|
||||
return tool_calls;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if let Some(tcs) = delta.get("tool_calls").and_then(|t| t.as_array()) {
|
||||
if text_started {
|
||||
let ev = format!("data: {}\n\n", json!({"type":"text-end","id":text_part_id}));
|
||||
let _ = stream.write_all(ev.as_bytes()).await;
|
||||
text_started = false;
|
||||
text_part_id = uuid::Uuid::new_v4().to_string();
|
||||
}
|
||||
|
||||
for tc in tcs {
|
||||
let idx = tc.get("index").and_then(|i| i.as_u64()).unwrap_or(0) as usize;
|
||||
if let std::collections::hash_map::Entry::Vacant(e) = tool_call_args.entry(idx)
|
||||
{
|
||||
let id = tc
|
||||
.get("id")
|
||||
.and_then(|i| i.as_str())
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
let name = tc
|
||||
.get("function")
|
||||
.and_then(|f| f.get("name"))
|
||||
.and_then(|n| n.as_str())
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
let ev = format!(
|
||||
"data: {}\n\n",
|
||||
json!({"type":"tool-input-start","toolCallId":id,"toolName":name})
|
||||
);
|
||||
let _ = stream.write_all(ev.as_bytes()).await;
|
||||
e.insert((id, name, String::new()));
|
||||
}
|
||||
if let Some(arg_delta) = tc
|
||||
.get("function")
|
||||
.and_then(|f| f.get("arguments"))
|
||||
.and_then(|a| a.as_str())
|
||||
{
|
||||
let entry = tool_call_args.get_mut(&idx).unwrap();
|
||||
entry.2.push_str(arg_delta);
|
||||
let ev = format!(
|
||||
"data: {}\n\n",
|
||||
json!({"type":"tool-input-delta","toolCallId":entry.0,"inputTextDelta":arg_delta})
|
||||
);
|
||||
let _ = stream.write_all(ev.as_bytes()).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if text_started {
|
||||
let ev = format!("data: {}\n\n", json!({"type":"text-end","id":text_part_id}));
|
||||
let _ = stream.write_all(ev.as_bytes()).await;
|
||||
}
|
||||
let mut indices: Vec<usize> = tool_call_args.keys().copied().collect();
|
||||
indices.sort();
|
||||
for idx in indices {
|
||||
if let Some(tc) = tool_call_args.remove(&idx) {
|
||||
tool_calls.push(tc);
|
||||
}
|
||||
}
|
||||
tool_calls
|
||||
}
|
||||
|
||||
pub(super) async fn handle_chat_request(
|
||||
stream: &mut tokio::net::TcpStream,
|
||||
body: &str,
|
||||
origin: Option<&str>,
|
||||
) {
|
||||
let cors = cors_headers_for_origin(origin);
|
||||
let gateway_url = std::env::var("AI_GATEWAY_URL")
|
||||
.unwrap_or_else(|_| DEFAULT_AI_GATEWAY_URL.to_string())
|
||||
.trim_end_matches('/')
|
||||
.to_string();
|
||||
let api_key = match std::env::var("AI_GATEWAY_API_KEY") {
|
||||
Ok(k) => k,
|
||||
Err(_) => {
|
||||
let err = r#"{"error":"AI_GATEWAY_API_KEY not set. Set the AI_GATEWAY_API_KEY environment variable to enable AI chat."}"#;
|
||||
let resp = format!(
|
||||
"HTTP/1.1 500 Internal Server Error\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n{cors}\r\n",
|
||||
err.len()
|
||||
);
|
||||
let _ = stream.write_all(resp.as_bytes()).await;
|
||||
let _ = stream.write_all(err.as_bytes()).await;
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
let default_model = std::env::var("AI_GATEWAY_MODEL")
|
||||
.unwrap_or_else(|_| "anthropic/claude-sonnet-4.6".to_string());
|
||||
|
||||
let parsed: Value = match serde_json::from_str(body) {
|
||||
Ok(v) => v,
|
||||
Err(e) => {
|
||||
let err = format!(r#"{{"error":"Invalid JSON: {}"}}"#, e);
|
||||
let resp = format!(
|
||||
"HTTP/1.1 400 Bad Request\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n{cors}\r\n",
|
||||
err.len()
|
||||
);
|
||||
let _ = stream.write_all(resp.as_bytes()).await;
|
||||
let _ = stream.write_all(err.as_bytes()).await;
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
let messages = parsed.get("messages").cloned().unwrap_or(json!([]));
|
||||
let model = parsed
|
||||
.get("model")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or(&default_model)
|
||||
.to_string();
|
||||
let session = parsed
|
||||
.get("session")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("default")
|
||||
.to_string();
|
||||
|
||||
let mut openai_messages: Vec<Value> =
|
||||
vec![json!({"role": "system", "content": get_system_prompt()})];
|
||||
let mut frontend_boundaries: Vec<usize> = Vec::new();
|
||||
let frontend_arr = messages.as_array();
|
||||
let frontend_count = frontend_arr.map(|a| a.len()).unwrap_or(0);
|
||||
if let Some(arr) = frontend_arr {
|
||||
for msg in arr {
|
||||
frontend_boundaries.push(openai_messages.len());
|
||||
let Some(role) = msg.get("role").and_then(|r| r.as_str()) else {
|
||||
continue;
|
||||
};
|
||||
if let Some(parts) = msg.get("parts").and_then(|p| p.as_array()) {
|
||||
let mut content_parts: Vec<Value> = Vec::new();
|
||||
for part in parts {
|
||||
match part.get("type").and_then(|t| t.as_str()) {
|
||||
Some("text") => {
|
||||
if let Some(text) = part.get("text").and_then(|t| t.as_str()) {
|
||||
if !text.is_empty() {
|
||||
content_parts.push(json!({"type": "text", "text": text}));
|
||||
}
|
||||
}
|
||||
}
|
||||
Some("file") => {
|
||||
if let (Some(url), Some(media_type)) = (
|
||||
part.get("url").and_then(|u| u.as_str()),
|
||||
part.get("mediaType").and_then(|m| m.as_str()),
|
||||
) {
|
||||
if media_type.starts_with("image/") {
|
||||
content_parts.push(json!({
|
||||
"type": "image_url",
|
||||
"image_url": { "url": url }
|
||||
}));
|
||||
}
|
||||
}
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
if !content_parts.is_empty() {
|
||||
let content = if content_parts.len() == 1
|
||||
&& content_parts[0].get("type").and_then(|t| t.as_str()) == Some("text")
|
||||
{
|
||||
content_parts[0]["text"].clone()
|
||||
} else {
|
||||
json!(content_parts)
|
||||
};
|
||||
openai_messages.push(json!({"role": role, "content": content}));
|
||||
}
|
||||
} else if let Some(content) = msg.get("content").and_then(|c| c.as_str()) {
|
||||
openai_messages.push(json!({"role": role, "content": content}));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let tools: Value = serde_json::from_str(CHAT_TOOLS).unwrap();
|
||||
let url = format!("{}/v1/chat/completions", gateway_url);
|
||||
let client = http_client();
|
||||
|
||||
let total_chars = estimate_chars(&openai_messages);
|
||||
let mut compaction_summary: Option<String> = None;
|
||||
let mut compaction_failed = false;
|
||||
let mut keep_last_n: usize = frontend_count;
|
||||
|
||||
if total_chars > COMPACT_THRESHOLD_CHARS && openai_messages.len() > KEEP_RECENT_MESSAGES + 2 {
|
||||
let split = find_safe_split(&openai_messages, KEEP_RECENT_MESSAGES);
|
||||
let to_summarize = &openai_messages[1..split];
|
||||
|
||||
if let Some(summary) =
|
||||
summarize_for_compaction(client, &url, &api_key, &model, to_summarize).await
|
||||
{
|
||||
let summary_msg = json!({
|
||||
"role": "system",
|
||||
"content": format!("[Conversation summary]\n{}", summary)
|
||||
});
|
||||
let recent = openai_messages[split..].to_vec();
|
||||
openai_messages = vec![openai_messages[0].clone(), summary_msg];
|
||||
openai_messages.extend(recent);
|
||||
|
||||
let kept_frontend = frontend_boundaries
|
||||
.iter()
|
||||
.filter(|&&boundary| boundary >= split)
|
||||
.count();
|
||||
keep_last_n = kept_frontend;
|
||||
compaction_summary = Some(summary);
|
||||
} else {
|
||||
compaction_failed = true;
|
||||
}
|
||||
}
|
||||
|
||||
let headers = format!(
|
||||
"HTTP/1.1 200 OK\r\nContent-Type: text/event-stream\r\nCache-Control: no-cache\r\nConnection: keep-alive\r\nx-vercel-ai-ui-message-stream: v1\r\n{cors}\r\n"
|
||||
);
|
||||
if stream.write_all(headers.as_bytes()).await.is_err() {
|
||||
return;
|
||||
}
|
||||
|
||||
let message_id = uuid::Uuid::new_v4().to_string();
|
||||
let start_ev = format!(
|
||||
"data: {}\n\n",
|
||||
json!({"type":"start","messageId":message_id})
|
||||
);
|
||||
if stream.write_all(start_ev.as_bytes()).await.is_err() {
|
||||
return;
|
||||
}
|
||||
|
||||
if let Some(ref summary) = compaction_summary {
|
||||
let ev = format!(
|
||||
"data: {}\n\n",
|
||||
json!({
|
||||
"type": "message-metadata",
|
||||
"messageMetadata": {
|
||||
"compacted": true,
|
||||
"summary": summary,
|
||||
"keepLastN": keep_last_n
|
||||
}
|
||||
})
|
||||
);
|
||||
let _ = stream.write_all(ev.as_bytes()).await;
|
||||
} else if compaction_failed {
|
||||
let ev = format!(
|
||||
"data: {}\n\n",
|
||||
json!({
|
||||
"type": "message-metadata",
|
||||
"messageMetadata": {
|
||||
"compacted": false,
|
||||
"warning": "Conversation is large but compaction failed. Responses may be degraded."
|
||||
}
|
||||
})
|
||||
);
|
||||
let _ = stream.write_all(ev.as_bytes()).await;
|
||||
}
|
||||
|
||||
let total_deadline = tokio::time::Instant::now() + std::time::Duration::from_secs(300);
|
||||
const TOOL_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(60);
|
||||
|
||||
for _step in 0..50 {
|
||||
if tokio::time::Instant::now() >= total_deadline {
|
||||
let ev = format!(
|
||||
"data: {}\n\n",
|
||||
json!({"type":"error","errorText":"Chat session timed out (5 minute limit)."})
|
||||
);
|
||||
let _ = stream.write_all(ev.as_bytes()).await;
|
||||
break;
|
||||
}
|
||||
|
||||
let step_ev = "data: {\"type\":\"start-step\"}\n\n";
|
||||
if stream.write_all(step_ev.as_bytes()).await.is_err() {
|
||||
return;
|
||||
}
|
||||
|
||||
let gateway_body = json!({
|
||||
"model": model,
|
||||
"messages": openai_messages,
|
||||
"tools": tools,
|
||||
"stream": true,
|
||||
});
|
||||
|
||||
let gw_response = match client
|
||||
.post(&url)
|
||||
.header("Authorization", format!("Bearer {}", api_key))
|
||||
.header("Content-Type", "application/json")
|
||||
.body(gateway_body.to_string())
|
||||
.send()
|
||||
.await
|
||||
{
|
||||
Ok(r) => r,
|
||||
Err(e) => {
|
||||
let ev = format!(
|
||||
"data: {}\n\n",
|
||||
json!({"type":"error","errorText":format!("Gateway request failed: {}", e)})
|
||||
);
|
||||
let _ = stream.write_all(ev.as_bytes()).await;
|
||||
break;
|
||||
}
|
||||
};
|
||||
|
||||
if !gw_response.status().is_success() {
|
||||
let body_text = gw_response.text().await.unwrap_or_default();
|
||||
let ev = format!(
|
||||
"data: {}\n\n",
|
||||
json!({"type":"error","errorText":body_text})
|
||||
);
|
||||
let _ = stream.write_all(ev.as_bytes()).await;
|
||||
break;
|
||||
}
|
||||
|
||||
let tool_calls = stream_gateway_response(stream, gw_response).await;
|
||||
|
||||
if tool_calls.is_empty() {
|
||||
let finish_step_ev = "data: {\"type\":\"finish-step\"}\n\n";
|
||||
let _ = stream.write_all(finish_step_ev.as_bytes()).await;
|
||||
break;
|
||||
}
|
||||
|
||||
let tc_values: Vec<Value> = tool_calls.iter().map(|(id, name, args)| {
|
||||
json!({"id": id, "type": "function", "function": {"name": name, "arguments": args}})
|
||||
}).collect();
|
||||
openai_messages.push(json!({"role": "assistant", "tool_calls": tc_values}));
|
||||
|
||||
for (tc_id, tc_name, tc_args) in &tool_calls {
|
||||
let input: Value = serde_json::from_str(tc_args).unwrap_or(json!({}));
|
||||
let command = input.get("command").and_then(|c| c.as_str()).unwrap_or("");
|
||||
|
||||
let ev = format!(
|
||||
"data: {}\n\n",
|
||||
json!({
|
||||
"type": "tool-input-available",
|
||||
"toolCallId": tc_id,
|
||||
"toolName": tc_name,
|
||||
"input": input
|
||||
})
|
||||
);
|
||||
let _ = stream.write_all(ev.as_bytes()).await;
|
||||
|
||||
let result = match tokio::time::timeout(
|
||||
TOOL_TIMEOUT,
|
||||
execute_chat_tool(&session, command),
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(r) => r,
|
||||
Err(_) => "Tool execution timed out after 60 seconds.".to_string(),
|
||||
};
|
||||
|
||||
let frontend_output = enrich_tool_output(&result);
|
||||
let ev = format!(
|
||||
"data: {}\n\n",
|
||||
json!({
|
||||
"type": "tool-output-available",
|
||||
"toolCallId": tc_id,
|
||||
"output": frontend_output
|
||||
})
|
||||
);
|
||||
let _ = stream.write_all(ev.as_bytes()).await;
|
||||
|
||||
openai_messages.push(json!({
|
||||
"role": "tool",
|
||||
"tool_call_id": tc_id,
|
||||
"content": result
|
||||
}));
|
||||
}
|
||||
|
||||
let finish_step_ev = "data: {\"type\":\"finish-step\"}\n\n";
|
||||
let _ = stream.write_all(finish_step_ev.as_bytes()).await;
|
||||
}
|
||||
|
||||
let finish_ev = "data: {\"type\":\"finish\"}\n\n";
|
||||
let _ = stream.write_all(finish_ev.as_bytes()).await;
|
||||
let done_ev = "data: [DONE]\n\n";
|
||||
let _ = stream.write_all(done_ev.as_bytes()).await;
|
||||
}
|
||||
@@ -0,0 +1,310 @@
|
||||
use serde_json::{json, Value};
|
||||
use std::path::PathBuf;
|
||||
use std::sync::Arc;
|
||||
|
||||
use tokio::io::AsyncWriteExt;
|
||||
use tokio::net::TcpListener;
|
||||
|
||||
use crate::connection::get_socket_dir;
|
||||
use crate::install::get_dashboard_dir;
|
||||
|
||||
use super::chat::{chat_status_json, handle_chat_request, handle_models_request};
|
||||
use super::discovery::discover_sessions;
|
||||
use super::http::{serve_static_file, CORS_HEADERS, DASHBOARD_NOT_INSTALLED_HTML};
|
||||
|
||||
pub async fn run_dashboard_server(port: u16) {
|
||||
let addr = format!("127.0.0.1:{}", port);
|
||||
let listener = match TcpListener::bind(&addr).await {
|
||||
Ok(l) => l,
|
||||
Err(e) => {
|
||||
eprintln!("Failed to bind dashboard server on {}: {}", addr, e);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
let dashboard_dir: Arc<PathBuf> = Arc::from(get_dashboard_dir());
|
||||
|
||||
loop {
|
||||
let Ok((stream, _addr)) = listener.accept().await else {
|
||||
break;
|
||||
};
|
||||
let dash_dir = dashboard_dir.clone();
|
||||
tokio::spawn(async move {
|
||||
handle_dashboard_connection(stream, dash_dir).await;
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
async fn handle_dashboard_connection(
|
||||
mut stream: tokio::net::TcpStream,
|
||||
dashboard_dir: Arc<PathBuf>,
|
||||
) {
|
||||
use tokio::io::AsyncReadExt;
|
||||
|
||||
let mut buf = vec![0u8; 8192];
|
||||
let n = match stream.read(&mut buf).await {
|
||||
Ok(n) if n > 0 => n,
|
||||
_ => return,
|
||||
};
|
||||
|
||||
let header_str = std::str::from_utf8(&buf[..n]).unwrap_or("");
|
||||
let first_line = header_str.lines().next().unwrap_or("").to_string();
|
||||
let method = first_line.split_whitespace().next().unwrap_or("GET");
|
||||
let path = first_line.split_whitespace().nth(1).unwrap_or("/");
|
||||
let origin = header_str.lines().find_map(|line| {
|
||||
if line.len() > 8 && line[..8].eq_ignore_ascii_case("origin: ") {
|
||||
Some(line[8..].trim().to_string())
|
||||
} else {
|
||||
None
|
||||
}
|
||||
});
|
||||
|
||||
if method == "OPTIONS" {
|
||||
let response = format!(
|
||||
"HTTP/1.1 204 No Content\r\n{CORS_HEADERS}Access-Control-Max-Age: 86400\r\nContent-Length: 0\r\nConnection: close\r\n\r\n"
|
||||
);
|
||||
let _ = stream.write_all(response.as_bytes()).await;
|
||||
return;
|
||||
}
|
||||
|
||||
if method == "POST" && path == "/api/chat" {
|
||||
let body_str = read_post_body(&mut stream, &buf, n).await;
|
||||
handle_chat_request(&mut stream, &body_str, origin.as_deref()).await;
|
||||
return;
|
||||
}
|
||||
|
||||
if method == "GET" && path == "/api/models" {
|
||||
handle_models_request(&mut stream, origin.as_deref()).await;
|
||||
return;
|
||||
}
|
||||
|
||||
if method == "POST" && (path == "/api/sessions" || path == "/api/exec" || path == "/api/kill") {
|
||||
let body_str = read_post_body(&mut stream, &buf, n).await;
|
||||
let result = if path == "/api/exec" {
|
||||
exec_cli(&body_str).await
|
||||
} else if path == "/api/kill" {
|
||||
kill_session(&body_str).await
|
||||
} else {
|
||||
spawn_session(&body_str).await
|
||||
};
|
||||
let (status, resp_body) = match result {
|
||||
Ok(msg) => ("200 OK", msg),
|
||||
Err(e) => (
|
||||
"400 Bad Request",
|
||||
format!(
|
||||
r#"{{"success":false,"error":{}}}"#,
|
||||
serde_json::to_string(&e).unwrap_or_else(|_| format!("\"{}\"", e))
|
||||
),
|
||||
),
|
||||
};
|
||||
let response = format!(
|
||||
"HTTP/1.1 {status}\r\nContent-Type: application/json; charset=utf-8\r\nContent-Length: {}\r\nConnection: close\r\n{CORS_HEADERS}\r\n",
|
||||
resp_body.len()
|
||||
);
|
||||
let _ = stream.write_all(response.as_bytes()).await;
|
||||
let _ = stream.write_all(resp_body.as_bytes()).await;
|
||||
return;
|
||||
}
|
||||
|
||||
let (status, content_type, body): (&str, &str, Vec<u8>) = if path == "/api/sessions" {
|
||||
(
|
||||
"200 OK",
|
||||
"application/json; charset=utf-8",
|
||||
discover_sessions().into_bytes(),
|
||||
)
|
||||
} else if path == "/api/chat/status" {
|
||||
(
|
||||
"200 OK",
|
||||
"application/json; charset=utf-8",
|
||||
chat_status_json().into_bytes(),
|
||||
)
|
||||
} else if dashboard_dir.join("index.html").exists() {
|
||||
serve_static_file(&dashboard_dir, path)
|
||||
} else {
|
||||
(
|
||||
"200 OK",
|
||||
"text/html; charset=utf-8",
|
||||
DASHBOARD_NOT_INSTALLED_HTML.as_bytes().to_vec(),
|
||||
)
|
||||
};
|
||||
|
||||
let response = format!(
|
||||
"HTTP/1.1 {}\r\nContent-Type: {}\r\nContent-Length: {}\r\nConnection: close\r\n{CORS_HEADERS}\r\n",
|
||||
status,
|
||||
content_type,
|
||||
body.len()
|
||||
);
|
||||
let _ = stream.write_all(response.as_bytes()).await;
|
||||
let _ = stream.write_all(&body).await;
|
||||
}
|
||||
|
||||
async fn read_post_body(stream: &mut tokio::net::TcpStream, initial: &[u8], n: usize) -> String {
|
||||
use tokio::io::AsyncReadExt;
|
||||
|
||||
let header_end = initial[..n]
|
||||
.windows(4)
|
||||
.position(|w| w == b"\r\n\r\n")
|
||||
.map(|p| p + 4)
|
||||
.or_else(|| {
|
||||
initial[..n]
|
||||
.windows(2)
|
||||
.position(|w| w == b"\n\n")
|
||||
.map(|p| p + 2)
|
||||
});
|
||||
let Some(header_end) = header_end else {
|
||||
return String::new();
|
||||
};
|
||||
|
||||
let header_str = String::from_utf8_lossy(&initial[..header_end]);
|
||||
let content_length: usize = header_str
|
||||
.lines()
|
||||
.find_map(|l| {
|
||||
if l.len() > 16 && l[..16].eq_ignore_ascii_case("content-length: ") {
|
||||
l[16..].trim().parse().ok()
|
||||
} else {
|
||||
let lower = l.to_lowercase();
|
||||
lower
|
||||
.strip_prefix("content-length:")
|
||||
.and_then(|v| v.trim().parse().ok())
|
||||
}
|
||||
})
|
||||
.unwrap_or(0);
|
||||
|
||||
if content_length == 0 {
|
||||
return String::new();
|
||||
}
|
||||
|
||||
let read_body = &initial[header_end..n];
|
||||
let already_read = read_body.len().min(content_length);
|
||||
|
||||
let mut body = Vec::with_capacity(content_length);
|
||||
body.extend_from_slice(&read_body[..already_read]);
|
||||
|
||||
let remaining = content_length - already_read;
|
||||
if remaining > 0 {
|
||||
let mut rest = vec![0u8; remaining];
|
||||
if stream.read_exact(&mut rest).await.is_ok() {
|
||||
body.extend_from_slice(&rest);
|
||||
}
|
||||
}
|
||||
|
||||
String::from_utf8(body).unwrap_or_default()
|
||||
}
|
||||
|
||||
async fn exec_cli(body: &str) -> Result<String, String> {
|
||||
let parsed: Value = serde_json::from_str(body).map_err(|e| format!("Invalid JSON: {}", e))?;
|
||||
let args: Vec<String> = parsed
|
||||
.get("args")
|
||||
.and_then(|v| v.as_array())
|
||||
.ok_or("Missing \"args\" array")?
|
||||
.iter()
|
||||
.filter_map(|v| v.as_str().map(|s| s.to_string()))
|
||||
.collect();
|
||||
|
||||
if args.is_empty() {
|
||||
return Err("Empty args array".to_string());
|
||||
}
|
||||
|
||||
let exe = std::env::current_exe().map_err(|e| format!("Cannot resolve executable: {}", e))?;
|
||||
|
||||
let mut cmd = tokio::process::Command::new(&exe);
|
||||
cmd.args(&args)
|
||||
.arg("--json")
|
||||
.env_remove("AGENT_BROWSER_DASHBOARD")
|
||||
.env_remove("AGENT_BROWSER_DASHBOARD_PORT")
|
||||
.env_remove("AGENT_BROWSER_STREAM_PORT");
|
||||
|
||||
let output = cmd
|
||||
.output()
|
||||
.await
|
||||
.map_err(|e| format!("Failed to execute: {}", e))?;
|
||||
|
||||
let stdout = String::from_utf8_lossy(&output.stdout).trim().to_string();
|
||||
let stderr = String::from_utf8_lossy(&output.stderr).trim().to_string();
|
||||
|
||||
Ok(json!({
|
||||
"success": output.status.success(),
|
||||
"exit_code": output.status.code(),
|
||||
"stdout": stdout,
|
||||
"stderr": stderr,
|
||||
})
|
||||
.to_string())
|
||||
}
|
||||
|
||||
async fn kill_session(body: &str) -> Result<String, String> {
|
||||
let parsed: Value = serde_json::from_str(body).map_err(|e| format!("Invalid JSON: {}", e))?;
|
||||
let session = parsed
|
||||
.get("session")
|
||||
.and_then(|v| v.as_str())
|
||||
.ok_or("Missing \"session\" field")?;
|
||||
|
||||
if session.is_empty() || session.len() > 64 {
|
||||
return Err("Session name must be 1-64 characters".to_string());
|
||||
}
|
||||
|
||||
let dir = get_socket_dir();
|
||||
let pid_path = dir.join(format!("{}.pid", session));
|
||||
|
||||
let pid_str = std::fs::read_to_string(&pid_path)
|
||||
.map_err(|_| format!("No PID file for session '{}'", session))?;
|
||||
let pid: u32 = pid_str
|
||||
.trim()
|
||||
.parse()
|
||||
.map_err(|_| format!("Invalid PID in file: {}", pid_str.trim()))?;
|
||||
|
||||
#[cfg(unix)]
|
||||
{
|
||||
unsafe {
|
||||
libc::kill(pid as i32, libc::SIGTERM);
|
||||
}
|
||||
tokio::time::sleep(std::time::Duration::from_millis(500)).await;
|
||||
if unsafe { libc::kill(pid as i32, 0) } == 0 {
|
||||
unsafe {
|
||||
libc::kill(pid as i32, libc::SIGKILL);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for ext in &["pid", "sock", "stream", "engine", "extensions"] {
|
||||
let _ = std::fs::remove_file(dir.join(format!("{}.{}", session, ext)));
|
||||
}
|
||||
|
||||
Ok(json!({ "success": true, "killed_pid": pid }).to_string())
|
||||
}
|
||||
|
||||
pub(super) async fn spawn_session(body: &str) -> Result<String, String> {
|
||||
let parsed: Value = serde_json::from_str(body).map_err(|e| format!("Invalid JSON: {}", e))?;
|
||||
let session = parsed
|
||||
.get("session")
|
||||
.and_then(|v| v.as_str())
|
||||
.ok_or("Missing \"session\" field")?;
|
||||
|
||||
if session.is_empty() || session.len() > 64 {
|
||||
return Err("Session name must be 1-64 characters".to_string());
|
||||
}
|
||||
|
||||
let exe = std::env::current_exe().map_err(|e| format!("Cannot resolve executable: {}", e))?;
|
||||
|
||||
let mut cmd = tokio::process::Command::new(&exe);
|
||||
cmd.arg("open")
|
||||
.arg("about:blank")
|
||||
.arg("--session")
|
||||
.arg(session);
|
||||
|
||||
cmd.stdout(std::process::Stdio::null());
|
||||
cmd.stderr(std::process::Stdio::null());
|
||||
|
||||
let status = cmd
|
||||
.status()
|
||||
.await
|
||||
.map_err(|e| format!("Failed to spawn session: {}", e))?;
|
||||
|
||||
if status.success() {
|
||||
Ok(format!(
|
||||
r#"{{"success":true,"session":{}}}"#,
|
||||
serde_json::to_string(session).unwrap_or_default()
|
||||
))
|
||||
} else {
|
||||
Err(format!("Session process exited with {}", status))
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,118 @@
|
||||
use serde_json::{json, Value};
|
||||
use std::path::Path;
|
||||
|
||||
use crate::connection::get_socket_dir;
|
||||
|
||||
pub(super) fn discover_sessions() -> String {
|
||||
let dir = get_socket_dir();
|
||||
let mut sessions = Vec::new();
|
||||
|
||||
if let Ok(entries) = std::fs::read_dir(&dir) {
|
||||
for entry in entries.flatten() {
|
||||
let name = entry.file_name();
|
||||
let name_str = name.to_string_lossy();
|
||||
if let Some(session) = name_str.strip_suffix(".stream") {
|
||||
if let Ok(port_str) = std::fs::read_to_string(entry.path()) {
|
||||
if let Ok(port) = port_str.trim().parse::<u16>() {
|
||||
let pid_path = dir.join(format!("{}.pid", session));
|
||||
if is_process_alive(&pid_path) {
|
||||
let engine_path = dir.join(format!("{}.engine", session));
|
||||
let engine = std::fs::read_to_string(&engine_path)
|
||||
.ok()
|
||||
.filter(|s| !s.trim().is_empty())
|
||||
.unwrap_or_else(|| "chrome".to_string());
|
||||
|
||||
let provider_path = dir.join(format!("{}.provider", session));
|
||||
let provider = std::fs::read_to_string(&provider_path)
|
||||
.ok()
|
||||
.filter(|s| !s.trim().is_empty());
|
||||
|
||||
let extensions = read_extensions_metadata(&dir, session);
|
||||
|
||||
let mut entry = json!({
|
||||
"session": session,
|
||||
"port": port,
|
||||
"engine": engine.trim(),
|
||||
});
|
||||
if let Some(ref p) = provider {
|
||||
entry["provider"] = json!(p.trim());
|
||||
}
|
||||
if !extensions.is_empty() {
|
||||
entry["extensions"] = json!(extensions);
|
||||
}
|
||||
sessions.push(entry);
|
||||
} else {
|
||||
let _ = std::fs::remove_file(entry.path());
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
serde_json::to_string(&sessions).unwrap_or_else(|_| "[]".to_string())
|
||||
}
|
||||
|
||||
fn read_extensions_metadata(dir: &std::path::Path, session: &str) -> Vec<Value> {
|
||||
let ext_path = dir.join(format!("{}.extensions", session));
|
||||
let ext_str = match std::fs::read_to_string(&ext_path) {
|
||||
Ok(s) => s,
|
||||
Err(_) => return Vec::new(),
|
||||
};
|
||||
|
||||
ext_str
|
||||
.split(',')
|
||||
.map(|p| p.trim())
|
||||
.filter(|p| !p.is_empty())
|
||||
.filter_map(|path| {
|
||||
let manifest_path = std::path::Path::new(path).join("manifest.json");
|
||||
let manifest_str = std::fs::read_to_string(&manifest_path).ok()?;
|
||||
let manifest: Value = serde_json::from_str(&manifest_str).ok()?;
|
||||
|
||||
let name = manifest
|
||||
.get("name")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("Unknown")
|
||||
.to_string();
|
||||
let version = manifest
|
||||
.get("version")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
let description = manifest
|
||||
.get("description")
|
||||
.and_then(|v| v.as_str())
|
||||
.map(|s| s.to_string());
|
||||
|
||||
let mut ext = json!({
|
||||
"name": name,
|
||||
"version": version,
|
||||
"path": path,
|
||||
});
|
||||
if let Some(desc) = description {
|
||||
ext["description"] = json!(desc);
|
||||
}
|
||||
Some(ext)
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn is_process_alive(pid_path: &Path) -> bool {
|
||||
let pid_str = match std::fs::read_to_string(pid_path) {
|
||||
Ok(s) => s,
|
||||
Err(_) => return false,
|
||||
};
|
||||
let pid: u32 = match pid_str.trim().parse() {
|
||||
Ok(p) => p,
|
||||
Err(_) => return false,
|
||||
};
|
||||
#[cfg(unix)]
|
||||
{
|
||||
unsafe { libc::kill(pid as i32, 0) == 0 }
|
||||
}
|
||||
#[cfg(not(unix))]
|
||||
{
|
||||
let _ = pid;
|
||||
true
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,339 @@
|
||||
use serde_json::{json, Value};
|
||||
use std::path::Path;
|
||||
use std::sync::Arc;
|
||||
|
||||
use tokio::io::{AsyncReadExt, AsyncWriteExt};
|
||||
use tokio::sync::RwLock;
|
||||
|
||||
use crate::connection::get_socket_dir;
|
||||
#[cfg(windows)]
|
||||
use crate::connection::resolve_port;
|
||||
|
||||
use super::chat::{chat_status_json, handle_chat_request, handle_models_request};
|
||||
use super::dashboard::spawn_session;
|
||||
use super::discovery::discover_sessions;
|
||||
|
||||
pub(super) const CORS_HEADERS: &str = "Access-Control-Allow-Origin: *\r\nAccess-Control-Allow-Methods: GET, POST, OPTIONS\r\nAccess-Control-Allow-Headers: Content-Type\r\n";
|
||||
|
||||
/// Build CORS headers that reflect the request origin only when it passes
|
||||
/// `is_allowed_origin`. Used for sensitive endpoints (chat, models) so the
|
||||
/// API key is not accessible from arbitrary web pages.
|
||||
pub(super) fn cors_headers_for_origin(origin: Option<&str>) -> String {
|
||||
let allowed_origin = match origin {
|
||||
Some(o) if super::is_allowed_origin(Some(o)) => o,
|
||||
_ => "http://localhost",
|
||||
};
|
||||
format!(
|
||||
"Access-Control-Allow-Origin: {}\r\nAccess-Control-Allow-Methods: GET, POST, OPTIONS\r\nAccess-Control-Allow-Headers: Content-Type\r\n",
|
||||
allowed_origin
|
||||
)
|
||||
}
|
||||
|
||||
fn parse_origin(peeked: &[u8]) -> Option<String> {
|
||||
let header_str = std::str::from_utf8(peeked).ok()?;
|
||||
for line in header_str.lines() {
|
||||
if line.len() > 8 && line[..8].eq_ignore_ascii_case("origin: ") {
|
||||
return Some(line[8..].trim().to_string());
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
pub(super) async fn handle_http_request(
|
||||
mut stream: tokio::net::TcpStream,
|
||||
peeked: &[u8],
|
||||
dashboard_dir: Option<&Path>,
|
||||
last_tabs: &Arc<RwLock<Vec<Value>>>,
|
||||
last_engine: &Arc<RwLock<String>>,
|
||||
session_name: &str,
|
||||
) {
|
||||
let peeked_len = peeked.len();
|
||||
let mut discard = vec![0u8; peeked_len];
|
||||
let _ = stream.read_exact(&mut discard).await;
|
||||
|
||||
let request = String::from_utf8_lossy(peeked);
|
||||
let first_line = request.lines().next().unwrap_or("");
|
||||
let method = first_line.split_whitespace().next().unwrap_or("GET");
|
||||
let path = first_line.split_whitespace().nth(1).unwrap_or("/");
|
||||
let origin = parse_origin(peeked);
|
||||
|
||||
if method == "OPTIONS" {
|
||||
let response = format!(
|
||||
"HTTP/1.1 204 No Content\r\n{CORS_HEADERS}Access-Control-Max-Age: 86400\r\nContent-Length: 0\r\nConnection: close\r\n\r\n"
|
||||
);
|
||||
let _ = stream.write_all(response.as_bytes()).await;
|
||||
return;
|
||||
}
|
||||
|
||||
if method == "POST" {
|
||||
let full_body = read_full_body(&mut stream, peeked).await;
|
||||
if full_body.is_none()
|
||||
&& (path == "/api/chat" || path == "/api/sessions" || path == "/api/command")
|
||||
{
|
||||
let body = r#"{"error":"Request body too large"}"#;
|
||||
let response = format!(
|
||||
"HTTP/1.1 413 Payload Too Large\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n{CORS_HEADERS}\r\n",
|
||||
body.len()
|
||||
);
|
||||
let _ = stream.write_all(response.as_bytes()).await;
|
||||
let _ = stream.write_all(body.as_bytes()).await;
|
||||
return;
|
||||
}
|
||||
let body_str = full_body.as_deref().unwrap_or("");
|
||||
|
||||
if path == "/api/sessions" {
|
||||
let result = spawn_session(body_str).await;
|
||||
let (status, resp_body) = match result {
|
||||
Ok(msg) => ("200 OK", msg),
|
||||
Err(e) => (
|
||||
"400 Bad Request",
|
||||
format!(
|
||||
r#"{{"success":false,"error":{}}}"#,
|
||||
serde_json::to_string(&e).unwrap_or_else(|_| format!("\"{}\"", e))
|
||||
),
|
||||
),
|
||||
};
|
||||
let response = format!(
|
||||
"HTTP/1.1 {status}\r\nContent-Type: application/json; charset=utf-8\r\nContent-Length: {}\r\nConnection: close\r\n{CORS_HEADERS}\r\n",
|
||||
resp_body.len()
|
||||
);
|
||||
let _ = stream.write_all(response.as_bytes()).await;
|
||||
let _ = stream.write_all(resp_body.as_bytes()).await;
|
||||
return;
|
||||
}
|
||||
|
||||
if path == "/api/command" {
|
||||
let result = relay_command_to_daemon(session_name, body_str).await;
|
||||
let (status, resp_body) = match result {
|
||||
Ok(resp) => ("200 OK", resp),
|
||||
Err(e) => (
|
||||
"502 Bad Gateway",
|
||||
format!(
|
||||
r#"{{"success":false,"error":{}}}"#,
|
||||
serde_json::to_string(&e).unwrap_or_else(|_| format!("\"{}\"", e))
|
||||
),
|
||||
),
|
||||
};
|
||||
let response = format!(
|
||||
"HTTP/1.1 {status}\r\nContent-Type: application/json; charset=utf-8\r\nContent-Length: {}\r\nConnection: close\r\n{CORS_HEADERS}\r\n",
|
||||
resp_body.len()
|
||||
);
|
||||
let _ = stream.write_all(response.as_bytes()).await;
|
||||
let _ = stream.write_all(resp_body.as_bytes()).await;
|
||||
return;
|
||||
}
|
||||
|
||||
if path == "/api/chat" {
|
||||
handle_chat_request(&mut stream, body_str, origin.as_deref()).await;
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
if method == "GET" && path == "/api/models" {
|
||||
handle_models_request(&mut stream, origin.as_deref()).await;
|
||||
return;
|
||||
}
|
||||
|
||||
let (status, content_type, body): (&str, &str, Vec<u8>) = if path == "/api/sessions" {
|
||||
(
|
||||
"200 OK",
|
||||
"application/json; charset=utf-8",
|
||||
discover_sessions().into_bytes(),
|
||||
)
|
||||
} else if path == "/api/tabs" {
|
||||
let tabs = last_tabs.read().await;
|
||||
(
|
||||
"200 OK",
|
||||
"application/json; charset=utf-8",
|
||||
serde_json::to_string(&*tabs)
|
||||
.unwrap_or_else(|_| "[]".to_string())
|
||||
.into_bytes(),
|
||||
)
|
||||
} else if path == "/api/status" {
|
||||
let engine = last_engine.read().await;
|
||||
(
|
||||
"200 OK",
|
||||
"application/json; charset=utf-8",
|
||||
format!(r#"{{"engine":"{}"}}"#, *engine).into_bytes(),
|
||||
)
|
||||
} else if path == "/api/chat/status" {
|
||||
(
|
||||
"200 OK",
|
||||
"application/json; charset=utf-8",
|
||||
chat_status_json().into_bytes(),
|
||||
)
|
||||
} else {
|
||||
match dashboard_dir {
|
||||
Some(dir) => serve_static_file(dir, path),
|
||||
None => (
|
||||
"200 OK",
|
||||
"text/html; charset=utf-8",
|
||||
DASHBOARD_NOT_INSTALLED_HTML.as_bytes().to_vec(),
|
||||
),
|
||||
}
|
||||
};
|
||||
|
||||
let response = format!(
|
||||
"HTTP/1.1 {}\r\nContent-Type: {}\r\nContent-Length: {}\r\nConnection: close\r\n{CORS_HEADERS}\r\n",
|
||||
status,
|
||||
content_type,
|
||||
body.len()
|
||||
);
|
||||
let _ = stream.write_all(response.as_bytes()).await;
|
||||
let _ = stream.write_all(&body).await;
|
||||
}
|
||||
|
||||
fn find_header_end(buf: &[u8]) -> Option<usize> {
|
||||
buf.windows(4)
|
||||
.position(|w| w == b"\r\n\r\n")
|
||||
.map(|p| p + 4)
|
||||
.or_else(|| buf.windows(2).position(|w| w == b"\n\n").map(|p| p + 2))
|
||||
}
|
||||
|
||||
fn parse_content_length_bytes(headers: &[u8]) -> Option<usize> {
|
||||
let header_str = std::str::from_utf8(headers).ok()?;
|
||||
for line in header_str.lines() {
|
||||
if line.len() > 16 && line[..16].eq_ignore_ascii_case("content-length: ") {
|
||||
return line[16..].trim().parse().ok();
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
const MAX_BODY_SIZE: usize = 10 * 1024 * 1024;
|
||||
|
||||
async fn read_full_body(stream: &mut tokio::net::TcpStream, peeked: &[u8]) -> Option<String> {
|
||||
let body_offset = find_header_end(peeked)?;
|
||||
let content_length = parse_content_length_bytes(&peeked[..body_offset])?;
|
||||
if content_length == 0 {
|
||||
return Some(String::new());
|
||||
}
|
||||
if content_length > MAX_BODY_SIZE {
|
||||
return None;
|
||||
}
|
||||
|
||||
let peeked_body = &peeked[body_offset..];
|
||||
let peeked_body_len = peeked_body.len().min(content_length);
|
||||
|
||||
let mut body = Vec::with_capacity(content_length);
|
||||
body.extend_from_slice(&peeked_body[..peeked_body_len]);
|
||||
|
||||
let remaining = content_length - peeked_body_len;
|
||||
if remaining > 0 {
|
||||
let mut rest = vec![0u8; remaining];
|
||||
if stream.read_exact(&mut rest).await.is_err() {
|
||||
return String::from_utf8(body).ok();
|
||||
}
|
||||
body.extend_from_slice(&rest);
|
||||
}
|
||||
|
||||
String::from_utf8(body).ok()
|
||||
}
|
||||
|
||||
pub(super) async fn relay_command_to_daemon(
|
||||
session_name: &str,
|
||||
body: &str,
|
||||
) -> Result<String, String> {
|
||||
let mut cmd: Value = serde_json::from_str(body).map_err(|e| format!("Invalid JSON: {}", e))?;
|
||||
|
||||
if cmd.get("id").is_none() {
|
||||
let id = format!(
|
||||
"dash-{}",
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_millis()
|
||||
);
|
||||
cmd["id"] = json!(id);
|
||||
}
|
||||
|
||||
let mut json_str = serde_json::to_string(&cmd).map_err(|e| e.to_string())?;
|
||||
json_str.push('\n');
|
||||
|
||||
#[cfg(unix)]
|
||||
let stream = {
|
||||
let socket_path = get_socket_dir().join(format!("{}.sock", session_name));
|
||||
tokio::net::UnixStream::connect(&socket_path)
|
||||
.await
|
||||
.map_err(|e| format!("Failed to connect to daemon: {}", e))?
|
||||
};
|
||||
|
||||
#[cfg(windows)]
|
||||
let stream = {
|
||||
let port = resolve_port(session_name);
|
||||
tokio::net::TcpStream::connect(format!("127.0.0.1:{}", port))
|
||||
.await
|
||||
.map_err(|e| format!("Failed to connect to daemon: {}", e))?
|
||||
};
|
||||
|
||||
let (reader, mut writer) = tokio::io::split(stream);
|
||||
|
||||
writer
|
||||
.write_all(json_str.as_bytes())
|
||||
.await
|
||||
.map_err(|e| format!("Failed to send command: {}", e))?;
|
||||
|
||||
let mut buf_reader = tokio::io::BufReader::new(reader);
|
||||
let mut response_line = String::new();
|
||||
tokio::io::AsyncBufReadExt::read_line(&mut buf_reader, &mut response_line)
|
||||
.await
|
||||
.map_err(|e| format!("Failed to read response: {}", e))?;
|
||||
|
||||
Ok(response_line.trim().to_string())
|
||||
}
|
||||
|
||||
pub(super) fn serve_static_file(
|
||||
dir: &Path,
|
||||
url_path: &str,
|
||||
) -> (&'static str, &'static str, Vec<u8>) {
|
||||
let clean = url_path.trim_start_matches('/');
|
||||
let file_path = if clean.is_empty() {
|
||||
dir.join("index.html")
|
||||
} else {
|
||||
let joined = dir.join(clean);
|
||||
if joined.is_file() {
|
||||
joined
|
||||
} else {
|
||||
dir.join("index.html")
|
||||
}
|
||||
};
|
||||
|
||||
match std::fs::read(&file_path) {
|
||||
Ok(content) => {
|
||||
let ext = file_path.extension().and_then(|e| e.to_str()).unwrap_or("");
|
||||
let ct = match ext {
|
||||
"html" => "text/html; charset=utf-8",
|
||||
"js" => "application/javascript; charset=utf-8",
|
||||
"css" => "text/css; charset=utf-8",
|
||||
"json" => "application/json; charset=utf-8",
|
||||
"svg" => "image/svg+xml",
|
||||
"png" => "image/png",
|
||||
"ico" => "image/x-icon",
|
||||
_ => "application/octet-stream",
|
||||
};
|
||||
("200 OK", ct, content)
|
||||
}
|
||||
Err(_) => (
|
||||
"404 Not Found",
|
||||
"text/html; charset=utf-8",
|
||||
b"<html><body><p>404 Not Found</p></body></html>".to_vec(),
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
pub(super) const DASHBOARD_NOT_INSTALLED_HTML: &str = r#"<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head><meta charset="utf-8"><title>agent-browser</title>
|
||||
<style>
|
||||
body { font-family: system-ui, sans-serif; display: flex; justify-content: center; align-items: center; height: 100vh; margin: 0; background: #0a0a0a; color: #e5e5e5; }
|
||||
.card { text-align: center; max-width: 400px; }
|
||||
code { background: #262626; padding: 2px 8px; border-radius: 4px; font-size: 14px; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="card">
|
||||
<h2>Dashboard not installed</h2>
|
||||
<p>Run <code>agent-browser dashboard install</code> to download the dashboard.</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>"#;
|
||||
@@ -0,0 +1,501 @@
|
||||
mod cdp_loop;
|
||||
mod chat;
|
||||
mod dashboard;
|
||||
mod discovery;
|
||||
mod http;
|
||||
mod websocket;
|
||||
|
||||
pub use cdp_loop::{ack_screencast_frame, start_screencast, stop_screencast};
|
||||
pub use dashboard::run_dashboard_server;
|
||||
|
||||
use serde_json::{json, Value};
|
||||
use std::path::PathBuf;
|
||||
use std::sync::Arc;
|
||||
|
||||
use tokio::net::TcpListener;
|
||||
use tokio::sync::{broadcast, watch, Mutex, Notify, RwLock};
|
||||
|
||||
use super::cdp::client::CdpClient;
|
||||
|
||||
/// Frame metadata from CDP Page.screencastFrame events.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct FrameMetadata {
|
||||
pub offset_top: f64,
|
||||
pub page_scale_factor: f64,
|
||||
pub device_width: u32,
|
||||
pub device_height: u32,
|
||||
pub scroll_offset_x: f64,
|
||||
pub scroll_offset_y: f64,
|
||||
pub timestamp: u64,
|
||||
}
|
||||
|
||||
impl Default for FrameMetadata {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
offset_top: 0.0,
|
||||
page_scale_factor: 1.0,
|
||||
device_width: 1280,
|
||||
device_height: 720,
|
||||
scroll_offset_x: 0.0,
|
||||
scroll_offset_y: 0.0,
|
||||
timestamp: 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub struct StreamServer {
|
||||
port: u16,
|
||||
session_name: String,
|
||||
frame_tx: broadcast::Sender<String>,
|
||||
client_count: Arc<Mutex<usize>>,
|
||||
client_slot: Arc<RwLock<Option<Arc<CdpClient>>>>,
|
||||
/// The active CDP page session ID (from Target.attachToTarget).
|
||||
cdp_session_id: Arc<RwLock<Option<String>>>,
|
||||
client_notify: Arc<Notify>,
|
||||
screencasting: Arc<Mutex<bool>>,
|
||||
viewport_width: Arc<Mutex<u32>>,
|
||||
viewport_height: Arc<Mutex<u32>>,
|
||||
dashboard_dir: Option<PathBuf>,
|
||||
last_tabs: Arc<RwLock<Vec<Value>>>,
|
||||
last_engine: Arc<RwLock<String>>,
|
||||
last_frame: Arc<RwLock<Option<String>>>,
|
||||
recording: Arc<Mutex<bool>>,
|
||||
shutdown_tx: watch::Sender<bool>,
|
||||
accept_task: Mutex<Option<tokio::task::JoinHandle<()>>>,
|
||||
cdp_task: Mutex<Option<tokio::task::JoinHandle<()>>>,
|
||||
}
|
||||
|
||||
impl StreamServer {
|
||||
pub async fn start(
|
||||
preferred_port: u16,
|
||||
client: Arc<CdpClient>,
|
||||
session_id: String,
|
||||
) -> Result<Self, String> {
|
||||
let client_slot = Arc::new(RwLock::new(Some(client)));
|
||||
let (server, _) = Self::start_inner(preferred_port, client_slot, session_id, true).await?;
|
||||
Ok(server)
|
||||
}
|
||||
|
||||
/// Start the stream server without a CDP client.
|
||||
/// Returns the server and a shared slot to set the client when the browser launches.
|
||||
/// Input messages are ignored until the client is set.
|
||||
/// When `allow_port_fallback` is true, binding to an occupied port falls back to an
|
||||
/// OS-assigned port (used by daemon startup). When false, the error propagates
|
||||
/// (used by the runtime `stream_enable` command).
|
||||
pub async fn start_without_client(
|
||||
preferred_port: u16,
|
||||
session_id: String,
|
||||
allow_port_fallback: bool,
|
||||
) -> Result<(Self, Arc<RwLock<Option<Arc<CdpClient>>>>), String> {
|
||||
let client_slot = Arc::new(RwLock::new(None::<Arc<CdpClient>>));
|
||||
Self::start_inner(preferred_port, client_slot, session_id, allow_port_fallback).await
|
||||
}
|
||||
|
||||
/// Resolve the dashboard directory if it exists.
|
||||
fn resolve_dashboard_dir() -> Option<PathBuf> {
|
||||
let dir = dirs::home_dir()?.join(".agent-browser").join("dashboard");
|
||||
if dir.join("index.html").exists() {
|
||||
Some(dir)
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
/// Notify the background CDP listener that the client has changed (browser launched/closed).
|
||||
pub fn notify_client_changed(&self) {
|
||||
self.client_notify.notify_one();
|
||||
}
|
||||
|
||||
/// Update the active CDP page session ID used for screencast commands.
|
||||
pub async fn set_cdp_session_id(&self, session_id: Option<String>) {
|
||||
let mut guard = self.cdp_session_id.write().await;
|
||||
*guard = session_id;
|
||||
}
|
||||
|
||||
/// Check whether the server currently has active screencast running.
|
||||
pub async fn is_screencasting(&self) -> bool {
|
||||
*self.screencasting.lock().await
|
||||
}
|
||||
|
||||
/// Update the stored viewport dimensions used by status messages and screencast.
|
||||
/// Also notifies the screencast event loop to restart with the new dimensions.
|
||||
pub async fn set_viewport(&self, width: u32, height: u32) {
|
||||
*self.viewport_width.lock().await = width;
|
||||
*self.viewport_height.lock().await = height;
|
||||
self.client_notify.notify_one();
|
||||
}
|
||||
|
||||
/// Get the current viewport dimensions.
|
||||
pub async fn viewport(&self) -> (u32, u32) {
|
||||
let w = *self.viewport_width.lock().await;
|
||||
let h = *self.viewport_height.lock().await;
|
||||
(w, h)
|
||||
}
|
||||
|
||||
/// Override the cached screencast state for explicit CLI start/stop commands.
|
||||
pub async fn set_screencasting(&self, active: bool) {
|
||||
let mut guard = self.screencasting.lock().await;
|
||||
*guard = active;
|
||||
}
|
||||
|
||||
/// Update and broadcast the recording state.
|
||||
pub async fn set_recording(&self, active: bool, engine: &str) {
|
||||
*self.recording.lock().await = active;
|
||||
let connected = self.client_slot.read().await.is_some();
|
||||
let sc = *self.screencasting.lock().await;
|
||||
let (vw, vh) = self.viewport().await;
|
||||
self.broadcast_status(connected, sc, vw, vh, engine).await;
|
||||
}
|
||||
|
||||
/// Shut down the accept loop and background CDP listener, releasing the bound port.
|
||||
pub async fn shutdown(&self) {
|
||||
let _ = self.shutdown_tx.send(true);
|
||||
|
||||
if let Some(task) = self.accept_task.lock().await.take() {
|
||||
let _ = task.await;
|
||||
}
|
||||
if let Some(task) = self.cdp_task.lock().await.take() {
|
||||
let _ = task.await;
|
||||
}
|
||||
}
|
||||
|
||||
async fn start_inner(
|
||||
preferred_port: u16,
|
||||
client_slot: Arc<RwLock<Option<Arc<CdpClient>>>>,
|
||||
session_id: String,
|
||||
allow_port_fallback: bool,
|
||||
) -> Result<(Self, Arc<RwLock<Option<Arc<CdpClient>>>>), String> {
|
||||
let addr = format!("127.0.0.1:{}", preferred_port);
|
||||
let listener = match TcpListener::bind(&addr).await {
|
||||
Ok(l) => l,
|
||||
Err(_) if allow_port_fallback && preferred_port != 0 => {
|
||||
TcpListener::bind("127.0.0.1:0")
|
||||
.await
|
||||
.map_err(|e| format!("Failed to bind stream server: {}", e))?
|
||||
}
|
||||
Err(e) => return Err(format!("Failed to bind stream server: {}", e)),
|
||||
};
|
||||
|
||||
let actual_addr = listener
|
||||
.local_addr()
|
||||
.map_err(|e| format!("Failed to get stream address: {}", e))?;
|
||||
let port = actual_addr.port();
|
||||
|
||||
let dashboard_dir = Self::resolve_dashboard_dir();
|
||||
|
||||
let (frame_tx, _) = broadcast::channel::<String>(64);
|
||||
let client_count = Arc::new(Mutex::new(0usize));
|
||||
let client_notify = Arc::new(Notify::new());
|
||||
let screencasting = Arc::new(Mutex::new(false));
|
||||
let cdp_session_id = Arc::new(RwLock::new(None::<String>));
|
||||
let viewport_width = Arc::new(Mutex::new(1280u32));
|
||||
let viewport_height = Arc::new(Mutex::new(720u32));
|
||||
let last_tabs = Arc::new(RwLock::new(Vec::<Value>::new()));
|
||||
let last_engine = Arc::new(RwLock::new("chrome".to_string()));
|
||||
let last_frame = Arc::new(RwLock::new(None::<String>));
|
||||
let recording = Arc::new(Mutex::new(false));
|
||||
let (shutdown_tx, shutdown_rx) = watch::channel(false);
|
||||
|
||||
let frame_tx_clone = frame_tx.clone();
|
||||
let client_count_clone = client_count.clone();
|
||||
let client_slot_clone = client_slot.clone();
|
||||
let notify_clone = client_notify.clone();
|
||||
let screencasting_clone = screencasting.clone();
|
||||
let cdp_session_clone = cdp_session_id.clone();
|
||||
|
||||
let vw_clone = viewport_width.clone();
|
||||
let vh_clone = viewport_height.clone();
|
||||
let dashboard_dir_clone = dashboard_dir.clone();
|
||||
let last_tabs_clone = last_tabs.clone();
|
||||
let last_engine_clone = last_engine.clone();
|
||||
let last_frame_clone = last_frame.clone();
|
||||
let recording_clone = recording.clone();
|
||||
let accept_shutdown_rx = shutdown_rx.clone();
|
||||
let session_name_clone = session_id.clone();
|
||||
let accept_task = tokio::spawn(async move {
|
||||
websocket::accept_loop(
|
||||
listener,
|
||||
frame_tx_clone,
|
||||
client_count_clone,
|
||||
client_slot_clone,
|
||||
notify_clone,
|
||||
screencasting_clone,
|
||||
cdp_session_clone,
|
||||
vw_clone,
|
||||
vh_clone,
|
||||
dashboard_dir_clone,
|
||||
last_tabs_clone,
|
||||
last_engine_clone,
|
||||
last_frame_clone,
|
||||
recording_clone,
|
||||
accept_shutdown_rx,
|
||||
session_name_clone,
|
||||
)
|
||||
.await;
|
||||
});
|
||||
|
||||
let frame_tx_bg = frame_tx.clone();
|
||||
let client_slot_bg = client_slot.clone();
|
||||
let client_notify_bg = client_notify.clone();
|
||||
let screencasting_bg = screencasting.clone();
|
||||
let client_count_bg = client_count.clone();
|
||||
let cdp_session_bg = cdp_session_id.clone();
|
||||
let vw_bg = viewport_width.clone();
|
||||
let vh_bg = viewport_height.clone();
|
||||
let last_frame_bg = last_frame.clone();
|
||||
let last_tabs_bg = last_tabs.clone();
|
||||
let last_engine_bg = last_engine.clone();
|
||||
let recording_bg = recording.clone();
|
||||
let cdp_task = tokio::spawn(async move {
|
||||
cdp_loop::cdp_event_loop(
|
||||
frame_tx_bg,
|
||||
client_slot_bg,
|
||||
client_notify_bg,
|
||||
screencasting_bg,
|
||||
client_count_bg,
|
||||
cdp_session_bg,
|
||||
vw_bg,
|
||||
vh_bg,
|
||||
last_frame_bg,
|
||||
last_tabs_bg,
|
||||
last_engine_bg,
|
||||
recording_bg,
|
||||
shutdown_rx,
|
||||
)
|
||||
.await;
|
||||
});
|
||||
|
||||
Ok((
|
||||
Self {
|
||||
port,
|
||||
session_name: session_id,
|
||||
frame_tx,
|
||||
client_count,
|
||||
client_slot: client_slot.clone(),
|
||||
cdp_session_id,
|
||||
client_notify,
|
||||
screencasting,
|
||||
viewport_width,
|
||||
viewport_height,
|
||||
dashboard_dir,
|
||||
last_tabs,
|
||||
last_engine,
|
||||
last_frame,
|
||||
recording,
|
||||
shutdown_tx,
|
||||
accept_task: Mutex::new(Some(accept_task)),
|
||||
cdp_task: Mutex::new(Some(cdp_task)),
|
||||
},
|
||||
client_slot,
|
||||
))
|
||||
}
|
||||
|
||||
pub fn port(&self) -> u16 {
|
||||
self.port
|
||||
}
|
||||
|
||||
/// Broadcast a raw frame string (legacy).
|
||||
pub fn broadcast_frame(&self, frame_json: &str) {
|
||||
let s = frame_json.to_string();
|
||||
if let Ok(mut lf) = self.last_frame.try_write() {
|
||||
*lf = Some(s.clone());
|
||||
}
|
||||
let _ = self.frame_tx.send(s);
|
||||
}
|
||||
|
||||
/// Broadcast a screencast frame with structured metadata.
|
||||
pub fn broadcast_screencast_frame(&self, base64_data: &str, metadata: &FrameMetadata) {
|
||||
let msg = json!({
|
||||
"type": "frame",
|
||||
"data": base64_data,
|
||||
"metadata": {
|
||||
"offsetTop": metadata.offset_top,
|
||||
"pageScaleFactor": metadata.page_scale_factor,
|
||||
"deviceWidth": metadata.device_width,
|
||||
"deviceHeight": metadata.device_height,
|
||||
"scrollOffsetX": metadata.scroll_offset_x,
|
||||
"scrollOffsetY": metadata.scroll_offset_y,
|
||||
"timestamp": metadata.timestamp,
|
||||
}
|
||||
});
|
||||
let s = msg.to_string();
|
||||
if let Ok(mut lf) = self.last_frame.try_write() {
|
||||
*lf = Some(s.clone());
|
||||
}
|
||||
let _ = self.frame_tx.send(s);
|
||||
}
|
||||
|
||||
/// Broadcast a status message to all connected clients.
|
||||
pub async fn broadcast_status(
|
||||
&self,
|
||||
connected: bool,
|
||||
screencasting: bool,
|
||||
viewport_width: u32,
|
||||
viewport_height: u32,
|
||||
engine: &str,
|
||||
) {
|
||||
{
|
||||
let mut guard = self.last_engine.write().await;
|
||||
*guard = engine.to_string();
|
||||
}
|
||||
let rec = *self.recording.lock().await;
|
||||
let msg = json!({
|
||||
"type": "status",
|
||||
"connected": connected,
|
||||
"screencasting": screencasting,
|
||||
"viewportWidth": viewport_width,
|
||||
"viewportHeight": viewport_height,
|
||||
"engine": engine,
|
||||
"recording": rec,
|
||||
});
|
||||
let _ = self.frame_tx.send(msg.to_string());
|
||||
}
|
||||
|
||||
/// Broadcast an error message to all connected clients.
|
||||
pub fn broadcast_error(&self, message: &str) {
|
||||
let msg = json!({
|
||||
"type": "error",
|
||||
"message": message,
|
||||
});
|
||||
let _ = self.frame_tx.send(msg.to_string());
|
||||
}
|
||||
|
||||
/// Broadcast a command event when a command begins executing.
|
||||
pub fn broadcast_command(&self, action: &str, id: &str, params: &Value) {
|
||||
let msg = json!({
|
||||
"type": "command",
|
||||
"action": action,
|
||||
"id": id,
|
||||
"params": params,
|
||||
"timestamp": timestamp_ms(),
|
||||
});
|
||||
let _ = self.frame_tx.send(msg.to_string());
|
||||
}
|
||||
|
||||
/// Broadcast a result event after a command finishes executing.
|
||||
pub fn broadcast_result(
|
||||
&self,
|
||||
id: &str,
|
||||
action: &str,
|
||||
success: bool,
|
||||
data: &Value,
|
||||
duration_ms: u64,
|
||||
) {
|
||||
let msg = json!({
|
||||
"type": "result",
|
||||
"id": id,
|
||||
"action": action,
|
||||
"success": success,
|
||||
"data": data,
|
||||
"duration_ms": duration_ms,
|
||||
"timestamp": timestamp_ms(),
|
||||
});
|
||||
let _ = self.frame_tx.send(msg.to_string());
|
||||
}
|
||||
|
||||
/// Broadcast a console event from the browser.
|
||||
pub fn broadcast_console(&self, level: &str, text: &str, args: &[Value]) {
|
||||
let mut msg = json!({
|
||||
"type": "console",
|
||||
"level": level,
|
||||
"text": text,
|
||||
"timestamp": timestamp_ms(),
|
||||
});
|
||||
if !args.is_empty() {
|
||||
msg.as_object_mut()
|
||||
.unwrap()
|
||||
.insert("args".to_string(), Value::Array(args.to_vec()));
|
||||
}
|
||||
let _ = self.frame_tx.send(msg.to_string());
|
||||
}
|
||||
|
||||
/// Broadcast a page error (uncaught exception) from the browser.
|
||||
pub fn broadcast_page_error(&self, text: &str, line: Option<i64>, column: Option<i64>) {
|
||||
let msg = json!({
|
||||
"type": "page_error",
|
||||
"text": text,
|
||||
"line": line,
|
||||
"column": column,
|
||||
"timestamp": timestamp_ms(),
|
||||
});
|
||||
let _ = self.frame_tx.send(msg.to_string());
|
||||
}
|
||||
|
||||
/// Broadcast the current tab list so the dashboard can render a tab bar.
|
||||
/// Also caches the list so newly connected WebSocket clients receive it immediately.
|
||||
pub async fn broadcast_tabs(&self, tabs: &[Value]) {
|
||||
{
|
||||
let mut guard = self.last_tabs.write().await;
|
||||
*guard = tabs.to_vec();
|
||||
}
|
||||
let msg = json!({
|
||||
"type": "tabs",
|
||||
"tabs": tabs,
|
||||
"timestamp": timestamp_ms(),
|
||||
});
|
||||
let _ = self.frame_tx.send(msg.to_string());
|
||||
}
|
||||
|
||||
/// Whether the dashboard directory is available.
|
||||
pub fn has_dashboard(&self) -> bool {
|
||||
self.dashboard_dir.is_some()
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn timestamp_ms() -> u64 {
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.map(|d| d.as_millis() as u64)
|
||||
.unwrap_or(0)
|
||||
}
|
||||
|
||||
pub fn is_allowed_origin(origin: Option<&str>) -> bool {
|
||||
match origin {
|
||||
None => true,
|
||||
Some(o) => {
|
||||
if o.starts_with("file://") {
|
||||
return true;
|
||||
}
|
||||
if let Ok(url) = url::Url::parse(o) {
|
||||
let host = url.host_str().unwrap_or("");
|
||||
host == "localhost" || host == "127.0.0.1" || host == "::1" || host == "[::1]"
|
||||
} else {
|
||||
false
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn test_allowed_origin_none() {
|
||||
assert!(is_allowed_origin(None));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_allowed_origin_file() {
|
||||
assert!(is_allowed_origin(Some("file:///path/to/file")));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_allowed_origin_localhost() {
|
||||
assert!(is_allowed_origin(Some("http://localhost:3000")));
|
||||
assert!(is_allowed_origin(Some("http://127.0.0.1:8080")));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_disallowed_origin() {
|
||||
assert!(!is_allowed_origin(Some("http://evil.com")));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_frame_metadata_default() {
|
||||
let meta = FrameMetadata::default();
|
||||
assert_eq!(meta.device_width, 1280);
|
||||
assert_eq!(meta.device_height, 720);
|
||||
assert_eq!(meta.page_scale_factor, 1.0);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,352 @@
|
||||
use serde_json::{json, Value};
|
||||
use std::net::SocketAddr;
|
||||
use std::path::PathBuf;
|
||||
use std::sync::Arc;
|
||||
|
||||
use futures_util::{SinkExt, StreamExt};
|
||||
use tokio::net::TcpListener;
|
||||
use tokio::sync::{broadcast, watch, Mutex, Notify, RwLock};
|
||||
use tokio_tungstenite::tungstenite::Message;
|
||||
|
||||
use crate::native::cdp::client::CdpClient;
|
||||
|
||||
use super::http::handle_http_request;
|
||||
use super::{is_allowed_origin, timestamp_ms};
|
||||
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub(super) async fn accept_loop(
|
||||
listener: TcpListener,
|
||||
frame_tx: broadcast::Sender<String>,
|
||||
client_count: Arc<Mutex<usize>>,
|
||||
client_slot: Arc<RwLock<Option<Arc<CdpClient>>>>,
|
||||
client_notify: Arc<Notify>,
|
||||
screencasting: Arc<Mutex<bool>>,
|
||||
cdp_session_id: Arc<RwLock<Option<String>>>,
|
||||
viewport_width: Arc<Mutex<u32>>,
|
||||
viewport_height: Arc<Mutex<u32>>,
|
||||
dashboard_dir: Option<PathBuf>,
|
||||
last_tabs: Arc<RwLock<Vec<Value>>>,
|
||||
last_engine: Arc<RwLock<String>>,
|
||||
last_frame: Arc<RwLock<Option<String>>>,
|
||||
recording: Arc<Mutex<bool>>,
|
||||
mut shutdown_rx: watch::Receiver<bool>,
|
||||
session_name: String,
|
||||
) {
|
||||
let dashboard_dir = dashboard_dir.map(Arc::from);
|
||||
let session_name: Arc<str> = Arc::from(session_name);
|
||||
loop {
|
||||
tokio::select! {
|
||||
changed = shutdown_rx.changed() => {
|
||||
if changed.is_err() || *shutdown_rx.borrow() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
accept_result = listener.accept() => {
|
||||
let Ok((stream, addr)) = accept_result else {
|
||||
break;
|
||||
};
|
||||
let frame_tx = frame_tx.clone();
|
||||
let client_count = client_count.clone();
|
||||
let client_slot = client_slot.clone();
|
||||
let client_notify = client_notify.clone();
|
||||
let screencasting = screencasting.clone();
|
||||
let cdp_session_id = cdp_session_id.clone();
|
||||
let vw = viewport_width.clone();
|
||||
let vh = viewport_height.clone();
|
||||
let dd = dashboard_dir.clone();
|
||||
let lt = last_tabs.clone();
|
||||
let le = last_engine.clone();
|
||||
let lf = last_frame.clone();
|
||||
let rec = recording.clone();
|
||||
let shutdown_rx = shutdown_rx.clone();
|
||||
let sn = session_name.clone();
|
||||
|
||||
tokio::spawn(async move {
|
||||
handle_connection(
|
||||
stream,
|
||||
addr,
|
||||
frame_tx,
|
||||
client_count,
|
||||
client_slot,
|
||||
client_notify,
|
||||
screencasting,
|
||||
cdp_session_id,
|
||||
vw,
|
||||
vh,
|
||||
dd,
|
||||
lt,
|
||||
le,
|
||||
lf,
|
||||
rec,
|
||||
shutdown_rx,
|
||||
sn,
|
||||
)
|
||||
.await;
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn is_websocket_upgrade(request: &str) -> bool {
|
||||
request.lines().any(|line| {
|
||||
if let Some((name, value)) = line.split_once(':') {
|
||||
name.trim().eq_ignore_ascii_case("upgrade")
|
||||
&& value.trim().eq_ignore_ascii_case("websocket")
|
||||
} else {
|
||||
false
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
/// Peek at the TCP stream to dispatch between WebSocket upgrade and plain HTTP.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
async fn handle_connection(
|
||||
stream: tokio::net::TcpStream,
|
||||
addr: SocketAddr,
|
||||
frame_tx: broadcast::Sender<String>,
|
||||
client_count: Arc<Mutex<usize>>,
|
||||
client_slot: Arc<RwLock<Option<Arc<CdpClient>>>>,
|
||||
client_notify: Arc<Notify>,
|
||||
screencasting: Arc<Mutex<bool>>,
|
||||
cdp_session_id: Arc<RwLock<Option<String>>>,
|
||||
viewport_width: Arc<Mutex<u32>>,
|
||||
viewport_height: Arc<Mutex<u32>>,
|
||||
dashboard_dir: Option<Arc<PathBuf>>,
|
||||
last_tabs: Arc<RwLock<Vec<Value>>>,
|
||||
last_engine: Arc<RwLock<String>>,
|
||||
last_frame: Arc<RwLock<Option<String>>>,
|
||||
recording: Arc<Mutex<bool>>,
|
||||
shutdown_rx: watch::Receiver<bool>,
|
||||
session_name: Arc<str>,
|
||||
) {
|
||||
let mut buf = [0u8; 4096];
|
||||
let n = match stream.peek(&mut buf).await {
|
||||
Ok(n) => n,
|
||||
Err(_) => return,
|
||||
};
|
||||
let request = String::from_utf8_lossy(&buf[..n]);
|
||||
|
||||
if is_websocket_upgrade(&request) {
|
||||
let frame_rx = frame_tx.subscribe();
|
||||
handle_ws_client(
|
||||
stream,
|
||||
addr,
|
||||
frame_rx,
|
||||
client_count,
|
||||
client_slot,
|
||||
client_notify,
|
||||
screencasting,
|
||||
cdp_session_id,
|
||||
viewport_width,
|
||||
viewport_height,
|
||||
last_tabs,
|
||||
last_engine,
|
||||
last_frame,
|
||||
recording,
|
||||
shutdown_rx,
|
||||
)
|
||||
.await;
|
||||
} else {
|
||||
handle_http_request(
|
||||
stream,
|
||||
&buf[..n],
|
||||
dashboard_dir.as_deref().map(|p| p.as_path()),
|
||||
&last_tabs,
|
||||
&last_engine,
|
||||
&session_name,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
}
|
||||
|
||||
#[allow(clippy::result_large_err, clippy::too_many_arguments)]
|
||||
async fn handle_ws_client(
|
||||
stream: tokio::net::TcpStream,
|
||||
_addr: SocketAddr,
|
||||
mut frame_rx: broadcast::Receiver<String>,
|
||||
client_count: Arc<Mutex<usize>>,
|
||||
client_slot: Arc<RwLock<Option<Arc<CdpClient>>>>,
|
||||
client_notify: Arc<Notify>,
|
||||
screencasting: Arc<Mutex<bool>>,
|
||||
cdp_session_id: Arc<RwLock<Option<String>>>,
|
||||
viewport_width: Arc<Mutex<u32>>,
|
||||
viewport_height: Arc<Mutex<u32>>,
|
||||
last_tabs: Arc<RwLock<Vec<Value>>>,
|
||||
last_engine: Arc<RwLock<String>>,
|
||||
last_frame: Arc<RwLock<Option<String>>>,
|
||||
recording: Arc<Mutex<bool>>,
|
||||
mut shutdown_rx: watch::Receiver<bool>,
|
||||
) {
|
||||
let callback =
|
||||
|req: &tokio_tungstenite::tungstenite::handshake::server::Request,
|
||||
resp: tokio_tungstenite::tungstenite::handshake::server::Response| {
|
||||
let origin = req
|
||||
.headers()
|
||||
.get("origin")
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.map(|s| s.to_string());
|
||||
if !is_allowed_origin(origin.as_deref()) {
|
||||
let mut reject =
|
||||
tokio_tungstenite::tungstenite::handshake::server::ErrorResponse::new(Some(
|
||||
"Origin not allowed".to_string(),
|
||||
));
|
||||
*reject.status_mut() = tokio_tungstenite::tungstenite::http::StatusCode::FORBIDDEN;
|
||||
return Err(reject);
|
||||
}
|
||||
Ok(resp)
|
||||
};
|
||||
|
||||
let ws_stream = match tokio_tungstenite::accept_hdr_async(stream, callback).await {
|
||||
Ok(ws) => ws,
|
||||
Err(_) => return,
|
||||
};
|
||||
|
||||
{
|
||||
let mut count = client_count.lock().await;
|
||||
*count += 1;
|
||||
}
|
||||
|
||||
let (mut ws_tx, mut ws_rx) = ws_stream.split();
|
||||
|
||||
{
|
||||
let guard = client_slot.read().await;
|
||||
let connected = guard.is_some();
|
||||
let sc = *screencasting.lock().await;
|
||||
let vw = *viewport_width.lock().await;
|
||||
let vh = *viewport_height.lock().await;
|
||||
let eng = last_engine.read().await.clone();
|
||||
let rec = *recording.lock().await;
|
||||
let status = json!({
|
||||
"type": "status",
|
||||
"connected": connected,
|
||||
"screencasting": sc,
|
||||
"viewportWidth": vw,
|
||||
"viewportHeight": vh,
|
||||
"engine": eng,
|
||||
"recording": rec,
|
||||
});
|
||||
let _ = ws_tx.send(Message::Text(status.to_string())).await;
|
||||
|
||||
let tabs = last_tabs.read().await;
|
||||
if !tabs.is_empty() {
|
||||
let tabs_msg = json!({
|
||||
"type": "tabs",
|
||||
"tabs": *tabs,
|
||||
"timestamp": timestamp_ms(),
|
||||
});
|
||||
let _ = ws_tx.send(Message::Text(tabs_msg.to_string())).await;
|
||||
}
|
||||
|
||||
if let Some(ref cached) = *last_frame.read().await {
|
||||
let _ = ws_tx.send(Message::Text(cached.clone())).await;
|
||||
}
|
||||
}
|
||||
|
||||
client_notify.notify_one();
|
||||
|
||||
loop {
|
||||
tokio::select! {
|
||||
changed = shutdown_rx.changed() => {
|
||||
if changed.is_err() || *shutdown_rx.borrow() {
|
||||
let _ = ws_tx.send(Message::Close(None)).await;
|
||||
break;
|
||||
}
|
||||
}
|
||||
frame = frame_rx.recv() => {
|
||||
match frame {
|
||||
Ok(data) => {
|
||||
if ws_tx.send(Message::Text(data)).await.is_err() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
Err(broadcast::error::RecvError::Lagged(_)) => {
|
||||
continue;
|
||||
}
|
||||
Err(broadcast::error::RecvError::Closed) => break,
|
||||
}
|
||||
}
|
||||
msg = ws_rx.next() => {
|
||||
match msg {
|
||||
Some(Ok(Message::Text(text))) => {
|
||||
let guard = client_slot.read().await;
|
||||
if let Some(ref client) = *guard {
|
||||
let sid = cdp_session_id.read().await;
|
||||
handle_client_message(&text, client.as_ref(), sid.as_deref()).await;
|
||||
}
|
||||
}
|
||||
Some(Ok(Message::Close(_))) | None => break,
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
{
|
||||
let mut count = client_count.lock().await;
|
||||
*count = count.saturating_sub(1);
|
||||
}
|
||||
|
||||
client_notify.notify_one();
|
||||
}
|
||||
|
||||
async fn handle_client_message(msg: &str, client: &CdpClient, session_id: Option<&str>) {
|
||||
let parsed: Value = match serde_json::from_str(msg) {
|
||||
Ok(v) => v,
|
||||
Err(_) => return,
|
||||
};
|
||||
|
||||
let msg_type = parsed.get("type").and_then(|v| v.as_str()).unwrap_or("");
|
||||
|
||||
match msg_type {
|
||||
"input_mouse" => {
|
||||
let _ = client
|
||||
.send_command(
|
||||
"Input.dispatchMouseEvent",
|
||||
Some(json!({
|
||||
"type": parsed.get("eventType").and_then(|v| v.as_str()).unwrap_or("mouseMoved"),
|
||||
"x": parsed.get("x").and_then(|v| v.as_f64()).unwrap_or(0.0),
|
||||
"y": parsed.get("y").and_then(|v| v.as_f64()).unwrap_or(0.0),
|
||||
"button": parsed.get("button").and_then(|v| v.as_str()).unwrap_or("none"),
|
||||
"clickCount": parsed.get("clickCount").and_then(|v| v.as_i64()).unwrap_or(0),
|
||||
"deltaX": parsed.get("deltaX").and_then(|v| v.as_f64()).unwrap_or(0.0),
|
||||
"deltaY": parsed.get("deltaY").and_then(|v| v.as_f64()).unwrap_or(0.0),
|
||||
"modifiers": parsed.get("modifiers").and_then(|v| v.as_i64()).unwrap_or(0),
|
||||
})),
|
||||
session_id,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
"input_keyboard" => {
|
||||
let _ = client
|
||||
.send_command(
|
||||
"Input.dispatchKeyEvent",
|
||||
Some(json!({
|
||||
"type": parsed.get("eventType").and_then(|v| v.as_str()).unwrap_or("keyDown"),
|
||||
"key": parsed.get("key"),
|
||||
"code": parsed.get("code"),
|
||||
"text": parsed.get("text"),
|
||||
"windowsVirtualKeyCode": parsed.get("windowsVirtualKeyCode").and_then(|v| v.as_i64()).unwrap_or(0),
|
||||
"modifiers": parsed.get("modifiers").and_then(|v| v.as_i64()).unwrap_or(0),
|
||||
})),
|
||||
session_id,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
"input_touch" => {
|
||||
let _ = client
|
||||
.send_command(
|
||||
"Input.dispatchTouchEvent",
|
||||
Some(json!({
|
||||
"type": parsed.get("eventType").and_then(|v| v.as_str()).unwrap_or("touchStart"),
|
||||
"touchPoints": parsed.get("touchPoints").unwrap_or(&json!([])),
|
||||
"modifiers": parsed.get("modifiers").and_then(|v| v.as_i64()).unwrap_or(0),
|
||||
})),
|
||||
session_id,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
"status" => {}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
+23
-13
@@ -1544,6 +1544,7 @@ Designed for AI agents to understand page structure.
|
||||
|
||||
Options:
|
||||
-i, --interactive Only include interactive elements
|
||||
-u, --urls Include href URLs for link elements
|
||||
-c, --compact Remove empty structural elements
|
||||
-d, --depth <n> Limit tree depth
|
||||
-s, --selector <sel> Scope snapshot to CSS selector
|
||||
@@ -1555,6 +1556,7 @@ Global Options:
|
||||
Examples:
|
||||
agent-browser snapshot
|
||||
agent-browser snapshot -i
|
||||
agent-browser snapshot -i --urls
|
||||
agent-browser snapshot --compact --depth 5
|
||||
agent-browser snapshot -s "#main-content"
|
||||
"##
|
||||
@@ -2622,20 +2624,24 @@ Examples:
|
||||
|
||||
"batch" => {
|
||||
r##"
|
||||
agent-browser batch - Execute multiple commands from stdin
|
||||
agent-browser batch - Execute multiple commands sequentially
|
||||
|
||||
Usage: echo '<json>' | agent-browser batch [options]
|
||||
Usage: agent-browser batch [options] "<cmd1>" "<cmd2>" ...
|
||||
echo '<json>' | agent-browser batch [options]
|
||||
|
||||
Reads a JSON array of commands from stdin and executes them sequentially.
|
||||
Each command is an array of strings matching normal CLI arguments.
|
||||
Results are printed in order, separated by blank lines (or as a JSON array
|
||||
with --json).
|
||||
Runs multiple commands in sequence. Commands can be passed as quoted
|
||||
arguments or piped as JSON via stdin. Results are printed in order,
|
||||
separated by blank lines (or as a JSON array with --json).
|
||||
|
||||
Options:
|
||||
--bail Stop on first error (default: continue all commands)
|
||||
--json Output results as a JSON array
|
||||
|
||||
Input Format:
|
||||
Argument Mode:
|
||||
Each quoted argument is a full command string:
|
||||
agent-browser batch "open https://example.com" "snapshot -i" "screenshot"
|
||||
|
||||
Stdin Mode (JSON):
|
||||
A JSON array of string arrays. Each inner array is one command:
|
||||
[
|
||||
["open", "https://example.com"],
|
||||
@@ -2646,8 +2652,9 @@ Input Format:
|
||||
]
|
||||
|
||||
Examples:
|
||||
agent-browser batch "open https://example.com" "screenshot"
|
||||
agent-browser batch --bail "open https://example.com" "click @e1" "screenshot"
|
||||
echo '[["open", "https://example.com"], ["snapshot"]]' | agent-browser batch
|
||||
echo '[["open", "https://example.com"], ["get", "title"]]' | agent-browser batch --json
|
||||
agent-browser batch --bail < commands.json
|
||||
"##
|
||||
}
|
||||
@@ -2769,8 +2776,8 @@ Streaming:
|
||||
stream status Show streaming status and active port
|
||||
|
||||
Batch:
|
||||
batch [--bail] Execute commands from stdin (JSON array of string arrays)
|
||||
--bail stops on first error (default: continue all)
|
||||
batch [--bail] ["cmd" ...] Execute multiple commands sequentially (args or stdin)
|
||||
--bail stops on first error (default: continue all)
|
||||
|
||||
Auth Vault:
|
||||
auth save <name> [opts] Save auth profile (--url, --username, --password/--password-stdin)
|
||||
@@ -2912,6 +2919,9 @@ Environment:
|
||||
AGENT_BROWSER_SCREENSHOT_DIR Default screenshot output directory
|
||||
AGENT_BROWSER_SCREENSHOT_QUALITY JPEG quality 0-100
|
||||
AGENT_BROWSER_SCREENSHOT_FORMAT Screenshot format: png, jpeg
|
||||
AI_GATEWAY_URL Vercel AI Gateway base URL (default: https://ai-gateway.vercel.sh)
|
||||
AI_GATEWAY_API_KEY API key for the AI Gateway (enables dashboard AI chat)
|
||||
AI_GATEWAY_MODEL Default AI model (default: anthropic/claude-sonnet-4.6)
|
||||
|
||||
Install:
|
||||
npm install -g agent-browser # npm
|
||||
@@ -2928,7 +2938,7 @@ Examples:
|
||||
agent-browser get text @e1
|
||||
agent-browser screenshot --full
|
||||
agent-browser screenshot --annotate # Labeled screenshot for vision models
|
||||
agent-browser wait --load networkidle # Wait for slow pages to load
|
||||
agent-browser wait 2000 # Wait for slow pages to settle
|
||||
agent-browser --cdp 9222 snapshot # Connect via CDP port
|
||||
agent-browser --auto-connect snapshot # Auto-discover running Chrome
|
||||
agent-browser stream enable # Start runtime streaming on an auto-selected port
|
||||
@@ -2942,9 +2952,9 @@ Examples:
|
||||
Command Chaining:
|
||||
Chain commands with && in a single shell call (browser persists via daemon):
|
||||
|
||||
agent-browser open example.com && agent-browser wait --load networkidle && agent-browser snapshot -i
|
||||
agent-browser open example.com && agent-browser snapshot -i
|
||||
agent-browser fill @e1 "user@example.com" && agent-browser fill @e2 "pass" && agent-browser click @e3
|
||||
agent-browser open example.com && agent-browser wait --load networkidle && agent-browser screenshot page.png
|
||||
agent-browser open example.com && agent-browser screenshot
|
||||
|
||||
iOS Simulator (requires Xcode and Appium):
|
||||
agent-browser -p ios open example.com # Use default iPhone
|
||||
|
||||
@@ -10,7 +10,7 @@ import { minuteRateLimit, dailyRateLimit } from "@/lib/rate-limit";
|
||||
|
||||
export const maxDuration = 60;
|
||||
|
||||
const DEFAULT_MODEL = "anthropic/claude-haiku-4.5";
|
||||
const DEFAULT_MODEL = "anthropic/claude-sonnet-4.6";
|
||||
|
||||
const SYSTEM_PROMPT = `You are a helpful documentation assistant for agent-browser, a browser automation CLI designed for AI agents.
|
||||
|
||||
|
||||
@@ -394,18 +394,22 @@ agent-browser reload # Reload page
|
||||
|
||||
## Batch execution
|
||||
|
||||
Execute multiple commands in a single invocation by piping a JSON array of string arrays to `batch`:
|
||||
Execute multiple commands in a single invocation. Commands can be passed as quoted arguments or piped as JSON via stdin.
|
||||
|
||||
```bash
|
||||
# Argument mode: each quoted argument is a full command
|
||||
agent-browser batch "open https://example.com" "snapshot -i" "screenshot"
|
||||
|
||||
# With --bail to stop on first error
|
||||
agent-browser batch --bail "open https://example.com" "click @e1" "screenshot"
|
||||
|
||||
# Stdin mode: pipe commands as JSON
|
||||
echo '[
|
||||
["open", "https://example.com"],
|
||||
["snapshot", "-i"],
|
||||
["click", "@e1"],
|
||||
["screenshot", "result.png"]
|
||||
]' | agent-browser batch --json
|
||||
|
||||
# Stop on first error
|
||||
agent-browser batch --bail < commands.json
|
||||
```
|
||||
|
||||
<table>
|
||||
|
||||
@@ -187,6 +187,9 @@ These environment variables configure additional daemon and runtime behavior:
|
||||
<tr><td><code>AGENT_BROWSER_CONFIRM_INTERACTIVE</code></td><td>Enable interactive confirmation prompts (auto-denies if stdin is not a TTY).</td><td>(disabled)</td></tr>
|
||||
<tr><td><code>AGENT_BROWSER_ENGINE</code></td><td>Browser engine to use: <code>chrome</code> (default), <code>lightpanda</code>.</td><td><code>chrome</code></td></tr>
|
||||
<tr><td><code>AGENT_BROWSER_NO_AUTO_DIALOG</code></td><td>Disable automatic dismissal of <code>alert</code>/<code>beforeunload</code> dialogs.</td><td>(disabled)</td></tr>
|
||||
<tr><td><code>AI_GATEWAY_URL</code></td><td>Vercel AI Gateway base URL.</td><td><code>https://ai-gateway.vercel.sh</code></td></tr>
|
||||
<tr><td><code>AI_GATEWAY_API_KEY</code></td><td>API key for the Vercel AI Gateway. Required to enable AI chat.</td><td>(none)</td></tr>
|
||||
<tr><td><code>AI_GATEWAY_MODEL</code></td><td>Default AI model for dashboard chat.</td><td><code>anthropic/claude-sonnet-4.6</code></td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
@@ -135,3 +135,38 @@ pnpm build:dashboard
|
||||
The built files are served by the daemon's stream server on the same port used for WebSocket connections. Plain HTTP requests serve the dashboard, while WebSocket upgrade requests are handled as before.
|
||||
|
||||
When the dashboard is not installed, visiting the HTTP endpoint shows instructions to run `agent-browser dashboard install`.
|
||||
|
||||
## AI Chat
|
||||
|
||||
The dashboard includes an optional AI chat panel powered by the [Vercel AI Gateway](https://vercel.com/docs/ai-gateway). When enabled, a **Chat** tab appears in the right pane alongside Activity, Console, Network, Storage, and Extensions.
|
||||
|
||||
### Setup
|
||||
|
||||
The Chat tab is always visible. Set the API key to enable responses:
|
||||
|
||||
```bash
|
||||
export AI_GATEWAY_API_KEY=gw_your_key_here
|
||||
agent-browser dashboard start
|
||||
```
|
||||
|
||||
Optionally override the gateway URL or model:
|
||||
|
||||
```bash
|
||||
export AI_GATEWAY_URL=https://ai-gateway.vercel.sh # this is the default
|
||||
export AI_GATEWAY_MODEL=openai/gpt-4o-mini # default: anthropic/claude-sonnet-4.6
|
||||
```
|
||||
|
||||
### How it works
|
||||
|
||||
The Rust server proxies chat requests from the dashboard to the Vercel AI Gateway and streams responses back using the Vercel AI SDK's UI Message Stream protocol. The dashboard frontend uses `useChat` from `@ai-sdk/react` with `DefaultChatTransport`.
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr><th>Variable</th><th>Description</th><th>Default</th></tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr><td><code>AI_GATEWAY_URL</code></td><td>Vercel AI Gateway base URL.</td><td><code>https://ai-gateway.vercel.sh</code></td></tr>
|
||||
<tr><td><code>AI_GATEWAY_API_KEY</code></td><td>API key for the AI Gateway. Required to enable AI chat responses.</td><td>(none)</td></tr>
|
||||
<tr><td><code>AI_GATEWAY_MODEL</code></td><td>Default AI model for chat requests.</td><td><code>anthropic/claude-sonnet-4.6</code></td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
@@ -21,6 +21,7 @@ agent-browser snapshot -i -c -d 5 # Combine options
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr><td><code>-i, --interactive</code></td><td>Only interactive elements (buttons, links, inputs)</td></tr>
|
||||
<tr><td><code>-u, --urls</code></td><td>Include href URLs for link elements</td></tr>
|
||||
<tr><td><code>-c, --compact</code></td><td>Remove empty structural elements</td></tr>
|
||||
<tr><td><code>-d, --depth</code></td><td>Limit tree depth</td></tr>
|
||||
<tr><td><code>-s, --selector</code></td><td>Scope to CSS selector</td></tr>
|
||||
|
||||
@@ -68,7 +68,7 @@ export function Header() {
|
||||
>
|
||||
<path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z" />
|
||||
</svg>
|
||||
<span>25k</span>
|
||||
<span>27k</span>
|
||||
</a>
|
||||
<a
|
||||
href="https://www.npmjs.com/package/agent-browser"
|
||||
|
||||
@@ -1,9 +1,22 @@
|
||||
import type { NextConfig } from "next";
|
||||
|
||||
const DAEMON_ORIGIN = process.env.DAEMON_URL || "http://localhost:4848";
|
||||
|
||||
const config: NextConfig = {
|
||||
output: "export",
|
||||
images: { unoptimized: true },
|
||||
devIndicators: false,
|
||||
env: {
|
||||
NEXT_PUBLIC_DAEMON_URL: DAEMON_ORIGIN,
|
||||
},
|
||||
async rewrites() {
|
||||
return [
|
||||
{
|
||||
source: "/api/:path*",
|
||||
destination: `${DAEMON_ORIGIN}/api/:path*`,
|
||||
},
|
||||
];
|
||||
},
|
||||
};
|
||||
|
||||
export default config;
|
||||
|
||||
@@ -8,16 +8,22 @@
|
||||
"start": "next start"
|
||||
},
|
||||
"dependencies": {
|
||||
"@ai-sdk/react": "^3.0.148",
|
||||
"@radix-ui/react-popover": "^1.1.15",
|
||||
"ai": "^6.0.146",
|
||||
"class-variance-authority": "^0.7.1",
|
||||
"clsx": "^2.1.1",
|
||||
"cmdk": "^1.1.1",
|
||||
"jotai": "^2.19.0",
|
||||
"lucide-react": "^1.7.0",
|
||||
"next": "16.1.1",
|
||||
"next-themes": "^0.4.6",
|
||||
"radix-ui": "^1.4.3",
|
||||
"react": "^19.1.0",
|
||||
"react-dom": "^19.1.0",
|
||||
"react-resizable-panels": "^4.7.6",
|
||||
"shadcn": "^4.1.0",
|
||||
"streamdown": "^2.5.0",
|
||||
"tailwind-merge": "^3.5.0",
|
||||
"tw-animate-css": "^1.4.0"
|
||||
},
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 25 KiB |
@@ -186,3 +186,22 @@ button {
|
||||
:is(.dark *).json-punct {
|
||||
color: #a1a1a1;
|
||||
}
|
||||
|
||||
@keyframes shimmer {
|
||||
0% { background-position: 200% 0; }
|
||||
100% { background-position: -200% 0; }
|
||||
}
|
||||
|
||||
.shimmer-text {
|
||||
background: linear-gradient(
|
||||
90deg,
|
||||
currentColor 25%,
|
||||
color-mix(in srgb, currentColor 40%, transparent) 50%,
|
||||
currentColor 75%
|
||||
);
|
||||
background-size: 200% 100%;
|
||||
-webkit-background-clip: text;
|
||||
background-clip: text;
|
||||
-webkit-text-fill-color: transparent;
|
||||
animation: shimmer 2s ease-in-out infinite;
|
||||
}
|
||||
|
||||
@@ -4,6 +4,7 @@ import { Geist } from "next/font/google";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { TooltipProvider } from "@/components/ui/tooltip";
|
||||
import { JotaiProvider } from "@/store/provider";
|
||||
import { ThemeProvider } from "@/components/theme-provider";
|
||||
|
||||
const geist = Geist({ subsets: ["latin"], variable: "--font-sans" });
|
||||
|
||||
@@ -18,11 +19,13 @@ export default function RootLayout({
|
||||
children: React.ReactNode;
|
||||
}) {
|
||||
return (
|
||||
<html lang="en" className={cn("dark font-sans antialiased", geist.variable)}>
|
||||
<html lang="en" className={cn("font-sans antialiased", geist.variable)} suppressHydrationWarning>
|
||||
<body>
|
||||
<JotaiProvider>
|
||||
<TooltipProvider>{children}</TooltipProvider>
|
||||
</JotaiProvider>
|
||||
<ThemeProvider attribute="class" defaultTheme="dark" enableSystem disableTransitionOnChange>
|
||||
<JotaiProvider>
|
||||
<TooltipProvider>{children}</TooltipProvider>
|
||||
</JotaiProvider>
|
||||
</ThemeProvider>
|
||||
</body>
|
||||
</html>
|
||||
);
|
||||
|
||||
@@ -1,14 +1,16 @@
|
||||
"use client";
|
||||
|
||||
import { useAtomValue } from "jotai/react";
|
||||
import { activePortAtom } from "@/store/sessions";
|
||||
import { useAtomValue, useSetAtom } from "jotai/react";
|
||||
import { activePortAtom, sessionsAtom, newSessionDialogAtom } from "@/store/sessions";
|
||||
import { useSessionsSync } from "@/store/sessions";
|
||||
import { useStreamSync, hasConsoleErrorsAtom, consoleLogsAtom } from "@/store/stream";
|
||||
import { useActivitySync } from "@/store/activity";
|
||||
import { activeExtensionsAtom } from "@/store/sessions";
|
||||
import { useChatStatusSync } from "@/store/chat";
|
||||
import { useMediaQuery } from "@/hooks/use-media-query";
|
||||
import { Viewport } from "@/components/viewport";
|
||||
import { ActivityFeed } from "@/components/activity-feed";
|
||||
import { ChatPanel } from "@/components/chat-panel";
|
||||
import { ConsolePanel } from "@/components/console-panel";
|
||||
import { StoragePanel } from "@/components/storage-panel";
|
||||
import { ExtensionsPanel } from "@/components/extensions-panel";
|
||||
@@ -20,21 +22,28 @@ import {
|
||||
ResizableHandle,
|
||||
} from "@/components/ui/resizable";
|
||||
import { Tabs, TabsList, TabsTrigger, TabsContent } from "@/components/ui/tabs";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { Plus } from "lucide-react";
|
||||
|
||||
export default function DashboardPage() {
|
||||
const activePort = useAtomValue(activePortAtom);
|
||||
useStreamSync(activePort);
|
||||
useSessionsSync();
|
||||
useActivitySync();
|
||||
useChatStatusSync();
|
||||
|
||||
const sessions = useAtomValue(sessionsAtom);
|
||||
const hasSessions = sessions.length > 0;
|
||||
const setNewSessionDialog = useSetAtom(newSessionDialogAtom);
|
||||
const isDesktop = useMediaQuery("(min-width: 768px)");
|
||||
const hasConsoleErrors = useAtomValue(hasConsoleErrorsAtom);
|
||||
const activeExtensions = useAtomValue(activeExtensionsAtom);
|
||||
|
||||
const sidePanel = (
|
||||
<Tabs defaultValue="activity" className="flex h-full flex-col">
|
||||
<Tabs defaultValue="chat" className="flex h-full flex-col">
|
||||
<div className="shrink-0 px-2 pt-1">
|
||||
<TabsList variant="line" className="h-7 w-full">
|
||||
<TabsTrigger value="chat" className="text-[11px]">Chat</TabsTrigger>
|
||||
<TabsTrigger value="activity" className="text-[11px]">Activity</TabsTrigger>
|
||||
<TabsTrigger value="console" className="text-[11px]">
|
||||
Console
|
||||
@@ -67,10 +76,46 @@ export default function DashboardPage() {
|
||||
<TabsContent value="extensions" className="min-h-0 flex-1 overflow-hidden">
|
||||
<ExtensionsPanel />
|
||||
</TabsContent>
|
||||
<TabsContent value="chat" className="min-h-0 flex-1 overflow-hidden">
|
||||
<ChatPanel />
|
||||
</TabsContent>
|
||||
</Tabs>
|
||||
);
|
||||
|
||||
if (isDesktop) {
|
||||
if (!hasSessions) {
|
||||
return (
|
||||
<div className="flex h-screen flex-col bg-background">
|
||||
<ResizablePanelGroup
|
||||
orientation="horizontal"
|
||||
className="min-h-0 flex-1"
|
||||
>
|
||||
<ResizablePanel id="sessions" defaultSize="15%" minSize="10%" maxSize="30%">
|
||||
<SessionTree />
|
||||
</ResizablePanel>
|
||||
<ResizableHandle />
|
||||
<ResizablePanel id="empty" defaultSize="85%">
|
||||
<div className="flex h-full items-center justify-center">
|
||||
<div className="text-center space-y-4">
|
||||
<div className="space-y-2">
|
||||
<p className="text-sm text-muted-foreground">No active sessions</p>
|
||||
<p className="text-xs text-muted-foreground/60">Create a session to get started</p>
|
||||
</div>
|
||||
<Button
|
||||
size="sm"
|
||||
onClick={() => setNewSessionDialog(true)}
|
||||
>
|
||||
<Plus className="size-3.5" />
|
||||
New session
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
</ResizablePanel>
|
||||
</ResizablePanelGroup>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="flex h-screen flex-col bg-background">
|
||||
<ResizablePanelGroup
|
||||
|
||||
@@ -0,0 +1,822 @@
|
||||
"use client";
|
||||
|
||||
import { useRef, useEffect, useState, useCallback, useMemo } from "react";
|
||||
import { useAtomValue } from "jotai/react";
|
||||
import { useChat } from "@ai-sdk/react";
|
||||
import { DefaultChatTransport } from "ai";
|
||||
import { Streamdown } from "streamdown";
|
||||
import { getChatApiUrl, chatModelAtom, availableModelsAtom } from "@/store/chat";
|
||||
import { activeSessionNameAtom } from "@/store/sessions";
|
||||
import { ModelSelector } from "@/components/model-selector";
|
||||
import { shikiTheme } from "@/lib/shiki-theme";
|
||||
import { ScrollArea } from "@/components/ui/scroll-area";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { ArrowUp, Square, Trash2, ChevronRight, ImagePlus, X, Loader, Copy, Check, Download } from "lucide-react";
|
||||
|
||||
type ExtraProps = { node?: unknown };
|
||||
type MdImgProps = React.ImgHTMLAttributes<HTMLImageElement> & ExtraProps;
|
||||
type MdHeadingProps = React.HTMLAttributes<HTMLHeadingElement> & ExtraProps;
|
||||
type MdAnchorProps = React.AnchorHTMLAttributes<HTMLAnchorElement> & ExtraProps;
|
||||
type MdPreProps = React.HTMLAttributes<HTMLPreElement> & ExtraProps;
|
||||
type MdCodeProps = React.HTMLAttributes<HTMLElement> & ExtraProps;
|
||||
|
||||
const chatComponents = {
|
||||
img: ({ node: _node, src, alt, ...props }: MdImgProps) => {
|
||||
if (typeof src === "string" && src.startsWith("data:image/")) {
|
||||
return <img src={src} alt={alt} className="rounded-md border border-border max-w-full my-1" {...props} />;
|
||||
}
|
||||
return null;
|
||||
},
|
||||
h1: ({ node: _node, ...props }: MdHeadingProps) => <p className="font-bold" {...props} />,
|
||||
h2: ({ node: _node, ...props }: MdHeadingProps) => <p className="font-bold" {...props} />,
|
||||
h3: ({ node: _node, ...props }: MdHeadingProps) => <p className="font-bold" {...props} />,
|
||||
h4: ({ node: _node, ...props }: MdHeadingProps) => <p className="font-bold" {...props} />,
|
||||
h5: ({ node: _node, ...props }: MdHeadingProps) => <p className="font-bold" {...props} />,
|
||||
h6: ({ node: _node, ...props }: MdHeadingProps) => <p className="font-bold" {...props} />,
|
||||
a: ({ node: _node, href, children, ...props }: MdAnchorProps) => (
|
||||
<a
|
||||
href={href}
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
className="text-primary underline underline-offset-2"
|
||||
{...props}
|
||||
>
|
||||
{children}
|
||||
</a>
|
||||
),
|
||||
pre: ({ node: _node, ...props }: MdPreProps) => (
|
||||
<pre
|
||||
className="text-[11px] bg-background border border-border rounded-md p-2 my-1.5 whitespace-pre-wrap break-all"
|
||||
{...props}
|
||||
/>
|
||||
),
|
||||
code: ({ className, children, node: _node, ...props }: MdCodeProps) => {
|
||||
if (className?.includes("language-")) {
|
||||
return <code className={className} {...props}>{children}</code>;
|
||||
}
|
||||
return (
|
||||
<span
|
||||
className="text-[11px] bg-secondary/60 px-1 py-0.5 rounded text-foreground font-mono break-all"
|
||||
{...props}
|
||||
>
|
||||
{children}
|
||||
</span>
|
||||
);
|
||||
},
|
||||
};
|
||||
|
||||
const STORAGE_PREFIX = "dashboard-chat-";
|
||||
const IMAGE_DATA_URL_RE = /data:image\/[^;]+;base64,[A-Za-z0-9+/=]+/g;
|
||||
|
||||
function stripImagesForStorage(messages: unknown[]): unknown[] {
|
||||
const json = JSON.stringify(messages);
|
||||
return JSON.parse(json.replace(IMAGE_DATA_URL_RE, "[image stripped]"));
|
||||
}
|
||||
|
||||
const SUGGESTIONS = [
|
||||
"Go to vercel.com",
|
||||
"Take a screenshot",
|
||||
"What's on the page?",
|
||||
"Click the first link",
|
||||
];
|
||||
|
||||
interface ToolInvocationPart {
|
||||
type: string;
|
||||
toolCallId: string;
|
||||
state: string;
|
||||
input?: Record<string, unknown>;
|
||||
output?: unknown;
|
||||
}
|
||||
|
||||
function isToolPart(part: { type: string }): part is ToolInvocationPart {
|
||||
return part.type.startsWith("tool-");
|
||||
}
|
||||
|
||||
function truncateOutput(text: string, maxLines = 30): string {
|
||||
const lines = text.split("\n");
|
||||
if (lines.length <= maxLines) return text;
|
||||
return lines.slice(0, maxLines).join("\n") + `\n... (${lines.length - maxLines} more lines)`;
|
||||
}
|
||||
|
||||
function parseOutputObject(raw: unknown): Record<string, unknown> | null {
|
||||
if (typeof raw === "string") {
|
||||
try {
|
||||
const parsed = JSON.parse(raw);
|
||||
if (typeof parsed === "object" && parsed !== null) return parsed;
|
||||
} catch { /* not JSON */ }
|
||||
return null;
|
||||
}
|
||||
if (typeof raw === "object" && raw !== null) return raw as Record<string, unknown>;
|
||||
return null;
|
||||
}
|
||||
|
||||
function formatOutput(raw: unknown): string | null {
|
||||
if (typeof raw === "string") {
|
||||
if (!raw.trim()) return null;
|
||||
const obj = parseOutputObject(raw);
|
||||
if (obj) {
|
||||
if (typeof obj.text === "string" && obj.image) return obj.text as string;
|
||||
const { image: _, ...rest } = obj;
|
||||
return JSON.stringify(rest, null, 2);
|
||||
}
|
||||
return raw;
|
||||
}
|
||||
if (typeof raw === "object" && raw !== null) {
|
||||
const r = raw as Record<string, unknown>;
|
||||
if (typeof r.text === "string" && r.image) return r.text as string;
|
||||
const { image: _, ...rest } = r;
|
||||
return JSON.stringify(rest, null, 2);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function extractImageUrl(raw: unknown): string | null {
|
||||
const obj = parseOutputObject(raw);
|
||||
if (!obj) return null;
|
||||
const img = obj.image;
|
||||
if (typeof img === "string" && img.startsWith("data:image/")) return img;
|
||||
return null;
|
||||
}
|
||||
|
||||
function ToolCallBlock({ part, onImageLoad }: { part: ToolInvocationPart; onImageLoad?: () => void }) {
|
||||
const [expanded, setExpanded] = useState(false);
|
||||
const toolName = part.type.split("-").slice(1).join("-");
|
||||
const command = (part.input as { command?: string })?.command ?? toolName;
|
||||
const isDone = part.state === "output-available";
|
||||
const isRunning = !isDone;
|
||||
const output = isDone ? formatOutput(part.output) : null;
|
||||
const hasOutput = !!output;
|
||||
const imageUrl = isDone ? extractImageUrl(part.output) : null;
|
||||
const canExpand = hasOutput && !isRunning;
|
||||
|
||||
return (
|
||||
<div className="space-y-1.5">
|
||||
<div
|
||||
className={cn(
|
||||
"rounded-md text-[10px] font-mono overflow-hidden border border-border",
|
||||
canExpand && "cursor-pointer",
|
||||
)}
|
||||
onClick={() => canExpand && setExpanded(!expanded)}
|
||||
>
|
||||
<div className={cn(
|
||||
"px-2 py-1 flex items-center gap-2",
|
||||
expanded && hasOutput ? "border-b border-border bg-secondary/30" : "bg-secondary/30",
|
||||
)}>
|
||||
{isRunning ? (
|
||||
<Loader className="size-3 shrink-0 animate-spin text-muted-foreground" />
|
||||
) : (
|
||||
<ChevronRight
|
||||
className={cn(
|
||||
"size-3 shrink-0 text-muted-foreground transition-transform duration-200",
|
||||
expanded && "rotate-90",
|
||||
)}
|
||||
/>
|
||||
)}
|
||||
<span className={cn(
|
||||
"truncate",
|
||||
isRunning ? "text-foreground/80 shimmer-text" : "text-foreground/80",
|
||||
)}>{command}</span>
|
||||
</div>
|
||||
{expanded && hasOutput && (
|
||||
<div className="max-h-[300px] overflow-y-auto">
|
||||
<pre className="px-2 py-1.5 text-foreground/80 whitespace-pre-wrap break-all leading-relaxed">
|
||||
{truncateOutput(output)}
|
||||
</pre>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
{imageUrl && (
|
||||
<img
|
||||
src={imageUrl}
|
||||
alt="Screenshot"
|
||||
className="rounded-md border border-border max-w-full"
|
||||
onLoad={onImageLoad}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
const DEFAULT_CONTEXT_WINDOW = 128000;
|
||||
|
||||
function estimateTokens(text: string): number {
|
||||
return Math.ceil(text.length / 4);
|
||||
}
|
||||
|
||||
function formatTokenCount(n: number): string {
|
||||
if (n >= 1_000_000) return `${(n / 1_000_000).toFixed(1)}M`;
|
||||
if (n >= 1_000) return `${(n / 1_000).toFixed(0)}K`;
|
||||
return `${n}`;
|
||||
}
|
||||
|
||||
function ContextMeter({ used, total }: { used: number; total: number }) {
|
||||
const ratio = Math.min(used / total, 1);
|
||||
const size = 16;
|
||||
const strokeWidth = 2;
|
||||
const r = (size - strokeWidth) / 2;
|
||||
const circumference = 2 * Math.PI * r;
|
||||
const offset = circumference * (1 - ratio);
|
||||
const color =
|
||||
ratio > 0.9 ? "text-destructive" : ratio > 0.7 ? "text-yellow-500" : "text-muted-foreground/50";
|
||||
|
||||
return (
|
||||
<div
|
||||
className="relative shrink-0"
|
||||
title={`${formatTokenCount(used)} / ${formatTokenCount(total)} tokens`}
|
||||
>
|
||||
<svg width={size} height={size} viewBox={`0 0 ${size} ${size}`}>
|
||||
<circle
|
||||
cx={size / 2}
|
||||
cy={size / 2}
|
||||
r={r}
|
||||
fill="none"
|
||||
stroke="currentColor"
|
||||
strokeWidth={strokeWidth}
|
||||
className="text-border"
|
||||
/>
|
||||
<circle
|
||||
cx={size / 2}
|
||||
cy={size / 2}
|
||||
r={r}
|
||||
fill="none"
|
||||
stroke="currentColor"
|
||||
strokeWidth={strokeWidth}
|
||||
strokeDasharray={circumference}
|
||||
strokeDashoffset={offset}
|
||||
strokeLinecap="round"
|
||||
className={cn(color, "transition-[stroke-dashoffset] duration-300")}
|
||||
transform={`rotate(-90 ${size / 2} ${size / 2})`}
|
||||
/>
|
||||
</svg>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
const DEFAULT_MODEL = "anthropic/claude-sonnet-4.6";
|
||||
|
||||
function useTimeAgo(ts: number | undefined) {
|
||||
const [, setTick] = useState(0);
|
||||
useEffect(() => {
|
||||
if (!ts) return;
|
||||
const id = setInterval(() => setTick((t) => t + 1), 30_000);
|
||||
return () => clearInterval(id);
|
||||
}, [ts]);
|
||||
if (!ts) return "";
|
||||
const diff = Math.floor((Date.now() - ts) / 1000);
|
||||
if (diff < 5) return "just now";
|
||||
if (diff < 60) return `${diff}s ago`;
|
||||
const mins = Math.floor(diff / 60);
|
||||
if (mins < 60) return `${mins}m ago`;
|
||||
const hrs = Math.floor(mins / 60);
|
||||
return `${hrs}h ago`;
|
||||
}
|
||||
|
||||
function MessageFooter({ model, timestamp, text }: { model: string; timestamp?: number; text: string }) {
|
||||
const [copied, setCopied] = useState(false);
|
||||
const timeAgo = useTimeAgo(timestamp);
|
||||
const shortModel = model.includes("/") ? model.split("/").pop()! : model;
|
||||
|
||||
const handleCopy = useCallback(() => {
|
||||
navigator.clipboard.writeText(text).then(() => {
|
||||
setCopied(true);
|
||||
setTimeout(() => setCopied(false), 2000);
|
||||
});
|
||||
}, [text]);
|
||||
|
||||
return (
|
||||
<div className="flex items-center gap-2 pt-0.5 text-[10px] text-muted-foreground/50">
|
||||
<span>{shortModel}</span>
|
||||
{timeAgo && (
|
||||
<>
|
||||
<span>·</span>
|
||||
<span>{timeAgo}</span>
|
||||
</>
|
||||
)}
|
||||
<button
|
||||
type="button"
|
||||
onClick={handleCopy}
|
||||
className="ml-auto hover:text-muted-foreground transition-colors"
|
||||
aria-label="Copy message"
|
||||
>
|
||||
{copied ? <Check className="size-3" /> : <Copy className="size-3" />}
|
||||
</button>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
interface PendingImage {
|
||||
file: File;
|
||||
preview: string;
|
||||
}
|
||||
|
||||
export function ChatPanel() {
|
||||
const [input, setInput] = useState("");
|
||||
const [errorDismissed, setErrorDismissed] = useState(false);
|
||||
const [pendingImages, setPendingImages] = useState<PendingImage[]>([]);
|
||||
const fileInputRef = useRef<HTMLInputElement>(null);
|
||||
const defaultModel = useAtomValue(chatModelAtom);
|
||||
const [selectedModel, setSelectedModel] = useState<string>(defaultModel || DEFAULT_MODEL);
|
||||
const messagesEndRef = useRef<HTMLDivElement>(null);
|
||||
const inputRef = useRef<HTMLTextAreaElement>(null);
|
||||
const sessionName = useAtomValue(activeSessionNameAtom);
|
||||
const chatId = sessionName || "default";
|
||||
const storageKey = `${STORAGE_PREFIX}${chatId}`;
|
||||
const sessionRef = useRef(chatId);
|
||||
sessionRef.current = chatId;
|
||||
const modelRef = useRef(selectedModel);
|
||||
modelRef.current = selectedModel;
|
||||
const messageTimestamps = useRef<Record<string, number>>({});
|
||||
|
||||
useEffect(() => {
|
||||
if (defaultModel) setSelectedModel(defaultModel);
|
||||
}, [defaultModel]);
|
||||
|
||||
const transport = useRef(
|
||||
new DefaultChatTransport({
|
||||
api: getChatApiUrl(),
|
||||
body: () => ({
|
||||
session: sessionRef.current,
|
||||
model: modelRef.current,
|
||||
}),
|
||||
}),
|
||||
).current;
|
||||
|
||||
const { messages, sendMessage, stop, status, setMessages, error } = useChat({
|
||||
chatId,
|
||||
transport,
|
||||
onError: () => setErrorDismissed(false),
|
||||
});
|
||||
|
||||
const visibleError = error && !errorDismissed ? error : undefined;
|
||||
const isLoading = status === "streaming" || status === "submitted";
|
||||
const hasMessages = messages.length > 0 || !!visibleError;
|
||||
|
||||
useEffect(() => {
|
||||
for (const msg of messages) {
|
||||
if (msg.role === "assistant" && !messageTimestamps.current[msg.id]) {
|
||||
messageTimestamps.current[msg.id] = Date.now();
|
||||
}
|
||||
}
|
||||
}, [messages]);
|
||||
|
||||
const models = useAtomValue(availableModelsAtom);
|
||||
const estimatedTokens = useMemo(() => {
|
||||
let total = 0;
|
||||
for (const msg of messages) {
|
||||
for (const part of msg.parts) {
|
||||
if (part.type === "text") total += estimateTokens(part.text);
|
||||
else if (isToolPart(part)) {
|
||||
if (part.input) total += estimateTokens(JSON.stringify(part.input));
|
||||
if (part.output) {
|
||||
const raw = typeof part.output === "string" ? part.output : JSON.stringify(part.output);
|
||||
const stripped = raw.replace(/"image"\s*:\s*"data:[^"]*"/g, '"image":"[omitted]"');
|
||||
total += estimateTokens(stripped);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return total;
|
||||
}, [messages]);
|
||||
const contextWindow = useMemo(() => {
|
||||
const match = models.find((m) => m.id === selectedModel);
|
||||
return match?.context_window ?? DEFAULT_CONTEXT_WINDOW;
|
||||
}, [models, selectedModel]);
|
||||
|
||||
useEffect(() => {
|
||||
inputRef.current?.focus();
|
||||
}, []);
|
||||
|
||||
const scrollToBottom = useCallback(() => {
|
||||
messagesEndRef.current?.scrollIntoView({ behavior: "smooth" });
|
||||
}, []);
|
||||
|
||||
useEffect(() => {
|
||||
scrollToBottom();
|
||||
}, [messages, visibleError, scrollToBottom]);
|
||||
|
||||
// Restore messages from localStorage when chatId changes
|
||||
useEffect(() => {
|
||||
try {
|
||||
const stored = localStorage.getItem(storageKey);
|
||||
if (stored) {
|
||||
const parsed = JSON.parse(stored);
|
||||
if (Array.isArray(parsed) && parsed.length > 0) {
|
||||
setMessages(parsed);
|
||||
return;
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
setMessages([]);
|
||||
}, [chatId, storageKey, setMessages]);
|
||||
|
||||
// Persist messages to localStorage (strip base64 images to save space)
|
||||
useEffect(() => {
|
||||
if (isLoading) return;
|
||||
if (messages.length === 0) {
|
||||
localStorage.removeItem(storageKey);
|
||||
return;
|
||||
}
|
||||
try {
|
||||
localStorage.setItem(storageKey, JSON.stringify(stripImagesForStorage(messages)));
|
||||
} catch {
|
||||
// ignore quota
|
||||
}
|
||||
}, [messages, isLoading, storageKey]);
|
||||
|
||||
const addImages = useCallback((files: FileList | null) => {
|
||||
if (!files) return;
|
||||
const images = Array.from(files).filter((f) => f.type.startsWith("image/"));
|
||||
setPendingImages((prev) => [
|
||||
...prev,
|
||||
...images.map((file) => ({ file, preview: URL.createObjectURL(file) })),
|
||||
]);
|
||||
}, []);
|
||||
|
||||
const removeImage = useCallback((index: number) => {
|
||||
setPendingImages((prev) => {
|
||||
const next = [...prev];
|
||||
URL.revokeObjectURL(next[index].preview);
|
||||
next.splice(index, 1);
|
||||
return next;
|
||||
});
|
||||
}, []);
|
||||
|
||||
const handleSubmit = useCallback(
|
||||
(e: React.FormEvent) => {
|
||||
e.preventDefault();
|
||||
if ((!input.trim() && pendingImages.length === 0) || isLoading) return;
|
||||
const dt = new DataTransfer();
|
||||
for (const img of pendingImages) dt.items.add(img.file);
|
||||
const files = dt.files.length > 0 ? dt.files : undefined;
|
||||
sendMessage({ text: input, files });
|
||||
setInput("");
|
||||
setPendingImages((prev) => {
|
||||
for (const p of prev) URL.revokeObjectURL(p.preview);
|
||||
return [];
|
||||
});
|
||||
},
|
||||
[input, isLoading, sendMessage, pendingImages],
|
||||
);
|
||||
|
||||
const lastCompactedId = useRef<string | null>(null);
|
||||
useEffect(() => {
|
||||
if (isLoading || messages.length === 0) return;
|
||||
const lastAssistant = [...messages].reverse().find((m) => m.role === "assistant");
|
||||
if (!lastAssistant) return;
|
||||
if (lastAssistant.id === lastCompactedId.current) return;
|
||||
const meta = (lastAssistant as any).metadata as
|
||||
| { compacted?: boolean; summary?: string; keepLastN?: number }
|
||||
| undefined;
|
||||
if (!meta?.compacted || typeof meta.keepLastN !== "number") return;
|
||||
|
||||
lastCompactedId.current = lastAssistant.id;
|
||||
const keep = meta.keepLastN;
|
||||
if (keep >= messages.length) return;
|
||||
|
||||
const summaryMsg = {
|
||||
id: `compaction-${Date.now()}`,
|
||||
role: "assistant" as const,
|
||||
parts: [
|
||||
{
|
||||
type: "text" as const,
|
||||
text: `*Earlier messages were summarized to stay within the context window.*`,
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
const kept = messages.slice(messages.length - keep);
|
||||
setMessages([summaryMsg as any, ...kept]);
|
||||
}, [isLoading, messages, setMessages]);
|
||||
|
||||
const handleClear = useCallback(() => {
|
||||
setMessages([]);
|
||||
setErrorDismissed(true);
|
||||
localStorage.removeItem(storageKey);
|
||||
requestAnimationFrame(() => inputRef.current?.focus());
|
||||
}, [setMessages, storageKey]);
|
||||
|
||||
const handleDownload = useCallback(() => {
|
||||
const data = messages.map((msg) => ({
|
||||
id: msg.id,
|
||||
role: msg.role,
|
||||
parts: msg.parts.map((p) => {
|
||||
if (p.type === "text") return { type: "text", text: p.text };
|
||||
if (p.type === "file") return { type: "file", filename: (p as any).filename };
|
||||
if (isToolPart(p)) {
|
||||
const out = typeof p.output === "string" ? p.output : JSON.stringify(p.output);
|
||||
const stripped = out?.replace(/"image":"data:[^"]*"/g, '"image":"[stripped]"');
|
||||
return {
|
||||
type: p.type,
|
||||
toolName: (p as any).toolName,
|
||||
state: (p as any).state,
|
||||
input: (p as any).input,
|
||||
output: stripped,
|
||||
};
|
||||
}
|
||||
return { type: p.type };
|
||||
}),
|
||||
}));
|
||||
const json = JSON.stringify({ session: chatId, model: selectedModel, messages: data }, null, 2);
|
||||
const blob = new Blob([json], { type: "application/json" });
|
||||
const url = URL.createObjectURL(blob);
|
||||
const a = document.createElement("a");
|
||||
a.href = url;
|
||||
a.download = `chat-${chatId}-${Date.now()}.json`;
|
||||
a.click();
|
||||
URL.revokeObjectURL(url);
|
||||
}, [messages, chatId, selectedModel]);
|
||||
|
||||
const hasVisibleContent = (parts: (typeof messages)[number]["parts"]): boolean => {
|
||||
return parts.some(
|
||||
(p) => (p.type === "text" && p.text.length > 0) || p.type === "file" || isToolPart(p),
|
||||
);
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="flex h-full flex-col">
|
||||
{hasMessages && (
|
||||
<div className="flex items-center justify-end gap-2 px-3 py-1.5 shrink-0 border-b border-border/40">
|
||||
<button
|
||||
onClick={handleDownload}
|
||||
className="text-muted-foreground hover:text-foreground transition-colors shrink-0"
|
||||
aria-label="Download conversation"
|
||||
>
|
||||
<Download className="size-3" />
|
||||
</button>
|
||||
<button
|
||||
onClick={handleClear}
|
||||
className="text-muted-foreground hover:text-foreground transition-colors shrink-0"
|
||||
aria-label="Clear conversation"
|
||||
>
|
||||
<Trash2 className="size-3" />
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<ScrollArea className="flex-1 min-h-0">
|
||||
<div className="p-3 space-y-3">
|
||||
{!hasMessages && !isLoading && (
|
||||
<div className="space-y-2 pt-2">
|
||||
<p className="text-[11px] text-muted-foreground">
|
||||
Control the browser with natural language:
|
||||
</p>
|
||||
<div className="flex flex-wrap gap-1.5">
|
||||
{SUGGESTIONS.map((s) => (
|
||||
<button
|
||||
key={s}
|
||||
type="button"
|
||||
onClick={() => sendMessage({ text: s })}
|
||||
className="text-[10px] px-2 py-1 rounded-md border bg-secondary/50 text-muted-foreground hover:text-foreground hover:bg-secondary transition-colors"
|
||||
>
|
||||
{s}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{messages.map((message) => {
|
||||
if (message.id.startsWith("compaction-")) {
|
||||
return (
|
||||
<div key={message.id} className="flex items-center gap-2 text-[10px] text-muted-foreground/60">
|
||||
<div className="flex-1 border-t border-border/40" />
|
||||
<span>Earlier messages summarized</span>
|
||||
<div className="flex-1 border-t border-border/40" />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
if (!hasVisibleContent(message.parts)) return null;
|
||||
return (
|
||||
<div key={message.id}>
|
||||
{message.role === "user" ? (
|
||||
<div className="space-y-1.5">
|
||||
{message.parts.some((p) => p.type === "file") && (
|
||||
<div className="flex flex-wrap gap-1.5">
|
||||
{message.parts
|
||||
.filter((p): p is Extract<typeof p, { type: "file" }> => p.type === "file")
|
||||
.map((p, i) => (
|
||||
<img
|
||||
key={i}
|
||||
src={p.url}
|
||||
alt={p.filename ?? "uploaded image"}
|
||||
className="max-h-24 rounded-md border border-border object-cover"
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
<div className="text-xs text-muted-foreground whitespace-pre-wrap leading-relaxed">
|
||||
{message.parts
|
||||
.filter((p): p is Extract<typeof p, { type: "text" }> => p.type === "text")
|
||||
.map((p) => p.text)
|
||||
.join("")}
|
||||
</div>
|
||||
</div>
|
||||
) : (
|
||||
<div className="space-y-1.5">
|
||||
{(() => {
|
||||
type Group = { type: "tools" | "text"; items: (typeof message.parts)[number][] };
|
||||
const groups: Group[] = [];
|
||||
for (const part of message.parts) {
|
||||
const groupType = isToolPart(part) ? "tools" : "text";
|
||||
const last = groups[groups.length - 1];
|
||||
if (last && last.type === groupType) {
|
||||
last.items.push(part);
|
||||
} else {
|
||||
groups.push({ type: groupType, items: [part] });
|
||||
}
|
||||
}
|
||||
|
||||
return groups.map((group, gi) => {
|
||||
if (group.type === "tools") {
|
||||
return (
|
||||
<div key={gi} className="space-y-0.5">
|
||||
{group.items.map((part) => {
|
||||
if (!isToolPart(part)) return null;
|
||||
return <ToolCallBlock key={part.toolCallId} part={part} onImageLoad={scrollToBottom} />;
|
||||
})}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
const combinedText = group.items
|
||||
.filter((p): p is Extract<typeof p, { type: "text" }> => p.type === "text" && !!p.text)
|
||||
.map((p) => p.text)
|
||||
.join("");
|
||||
if (!combinedText) return null;
|
||||
return (
|
||||
<div key={gi} className="text-xs text-foreground">
|
||||
<Streamdown
|
||||
shikiTheme={shikiTheme}
|
||||
controls={false}
|
||||
components={chatComponents}
|
||||
>
|
||||
{combinedText}
|
||||
</Streamdown>
|
||||
</div>
|
||||
);
|
||||
});
|
||||
})()}
|
||||
{(() => {
|
||||
const isLast = message === messages[messages.length - 1];
|
||||
const isComplete = !isLast || !isLoading;
|
||||
if (!isComplete) return null;
|
||||
const fullText = message.parts
|
||||
.filter((p): p is Extract<typeof p, { type: "text" }> => p.type === "text" && !!p.text)
|
||||
.map((p) => p.text)
|
||||
.join("");
|
||||
return (
|
||||
<MessageFooter
|
||||
model={selectedModel}
|
||||
timestamp={messageTimestamps.current[message.id]}
|
||||
text={fullText}
|
||||
/>
|
||||
);
|
||||
})()}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
|
||||
{isLoading && messages.length > 0 && (() => {
|
||||
const lastMsg = messages[messages.length - 1];
|
||||
const lastPart = lastMsg?.parts[lastMsg.parts.length - 1];
|
||||
const noVisibleContent = !lastMsg || !hasVisibleContent(lastMsg.parts);
|
||||
const lastIsCompletedTool = lastPart && isToolPart(lastPart) && lastPart.state === "output-available";
|
||||
if (noVisibleContent || lastIsCompletedTool) {
|
||||
return (
|
||||
<span className="text-[11px] text-muted-foreground shimmer-text">
|
||||
Working...
|
||||
</span>
|
||||
);
|
||||
}
|
||||
return null;
|
||||
})()}
|
||||
|
||||
{visibleError && (
|
||||
<div className="text-[10px] text-destructive/80 bg-destructive/10 rounded-md px-2 py-1.5">
|
||||
{(() => {
|
||||
try {
|
||||
const parsed = JSON.parse(visibleError.message);
|
||||
return parsed.message || parsed.error || visibleError.message;
|
||||
} catch {
|
||||
return visibleError.message || "Something went wrong.";
|
||||
}
|
||||
})()}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div ref={messagesEndRef} />
|
||||
</div>
|
||||
</ScrollArea>
|
||||
|
||||
<div className="shrink-0 border-t border-border">
|
||||
<form onSubmit={handleSubmit}>
|
||||
{pendingImages.length > 0 && (
|
||||
<div className="flex flex-wrap gap-1.5 px-3 pt-2">
|
||||
{pendingImages.map((img, i) => (
|
||||
<div key={img.preview} className="group relative">
|
||||
<img
|
||||
src={img.preview}
|
||||
alt={img.file.name}
|
||||
className="h-14 rounded-md border border-border object-cover"
|
||||
/>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => removeImage(i)}
|
||||
className="absolute -top-1.5 -right-1.5 hidden group-hover:flex size-4 items-center justify-center rounded-full bg-background border border-border text-muted-foreground hover:text-foreground"
|
||||
>
|
||||
<X className="size-2.5" />
|
||||
</button>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
<div className="px-3 pt-2 pb-1.5">
|
||||
<textarea
|
||||
ref={inputRef}
|
||||
value={input}
|
||||
onChange={(e) => {
|
||||
setInput(e.target.value);
|
||||
e.target.style.height = "auto";
|
||||
e.target.style.height = `${e.target.scrollHeight}px`;
|
||||
}}
|
||||
rows={1}
|
||||
placeholder="Ask something..."
|
||||
onKeyDown={(e) => {
|
||||
if (e.key === "Enter" && !e.shiftKey) {
|
||||
e.preventDefault();
|
||||
handleSubmit(e);
|
||||
}
|
||||
}}
|
||||
onPaste={(e) => {
|
||||
const items = e.clipboardData?.items;
|
||||
if (!items) return;
|
||||
const imageFiles: File[] = [];
|
||||
for (const item of items) {
|
||||
if (item.type.startsWith("image/")) {
|
||||
const file = item.getAsFile();
|
||||
if (file) imageFiles.push(file);
|
||||
}
|
||||
}
|
||||
if (imageFiles.length > 0) {
|
||||
const dt = new DataTransfer();
|
||||
for (const f of imageFiles) dt.items.add(f);
|
||||
addImages(dt.files);
|
||||
}
|
||||
}}
|
||||
className="w-full bg-transparent text-xs text-foreground outline-none resize-none max-h-24 leading-relaxed placeholder:text-muted-foreground"
|
||||
/>
|
||||
</div>
|
||||
<div className="flex items-center justify-between px-3 pb-2">
|
||||
<ModelSelector value={selectedModel} onChange={setSelectedModel} />
|
||||
<div className="flex items-center gap-2">
|
||||
{hasMessages && (
|
||||
<ContextMeter used={estimatedTokens} total={contextWindow} />
|
||||
)}
|
||||
<input
|
||||
ref={fileInputRef}
|
||||
type="file"
|
||||
accept="image/*"
|
||||
multiple
|
||||
className="hidden"
|
||||
onChange={(e) => {
|
||||
addImages(e.target.files);
|
||||
e.target.value = "";
|
||||
}}
|
||||
/>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => fileInputRef.current?.click()}
|
||||
className="text-muted-foreground hover:text-foreground transition-colors shrink-0 p-1"
|
||||
aria-label="Attach image"
|
||||
>
|
||||
<ImagePlus className="size-3.5" />
|
||||
</button>
|
||||
{isLoading ? (
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => stop()}
|
||||
className="bg-primary text-primary-foreground rounded-full p-1 hover:bg-primary/90 transition-colors shrink-0"
|
||||
aria-label="Stop"
|
||||
>
|
||||
<Square className="size-3 fill-current" />
|
||||
</button>
|
||||
) : (
|
||||
<button
|
||||
type="submit"
|
||||
disabled={!input.trim() && pendingImages.length === 0}
|
||||
className="bg-primary text-primary-foreground rounded-full p-1 hover:bg-primary/90 transition-colors disabled:opacity-30 shrink-0"
|
||||
aria-label="Send message"
|
||||
>
|
||||
<ArrowUp className="size-3" />
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,97 @@
|
||||
"use client";
|
||||
|
||||
import { useState } from "react";
|
||||
import { useAtomValue } from "jotai/react";
|
||||
import { availableModelsAtom } from "@/store/chat";
|
||||
import { ChevronDown, Check } from "lucide-react";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { Popover, PopoverContent, PopoverTrigger } from "@/components/ui/popover";
|
||||
import {
|
||||
Command,
|
||||
CommandInput,
|
||||
CommandList,
|
||||
CommandEmpty,
|
||||
CommandGroup,
|
||||
CommandItem,
|
||||
} from "@/components/ui/command";
|
||||
|
||||
function formatModelLabel(id: string): string {
|
||||
const parts = id.split("/");
|
||||
return parts.length > 1 ? parts.slice(1).join("/") : id;
|
||||
}
|
||||
|
||||
function formatProvider(id: string): string {
|
||||
const parts = id.split("/");
|
||||
if (parts.length > 1) return parts[0];
|
||||
return "";
|
||||
}
|
||||
|
||||
interface ModelSelectorProps {
|
||||
value: string;
|
||||
onChange: (model: string) => void;
|
||||
}
|
||||
|
||||
export function ModelSelector({ value, onChange }: ModelSelectorProps) {
|
||||
const [open, setOpen] = useState(false);
|
||||
const models = useAtomValue(availableModelsAtom);
|
||||
|
||||
const providers = new Map<string, typeof models>();
|
||||
for (const m of models) {
|
||||
const provider = formatProvider(m.id) || "other";
|
||||
if (!providers.has(provider)) providers.set(provider, []);
|
||||
providers.get(provider)!.push(m);
|
||||
}
|
||||
|
||||
return (
|
||||
<Popover open={open} onOpenChange={setOpen}>
|
||||
<PopoverTrigger asChild>
|
||||
<button
|
||||
className="flex items-center gap-0.5 text-[10px] text-muted-foreground hover:text-foreground transition-colors truncate max-w-[180px]"
|
||||
aria-label="Select model"
|
||||
>
|
||||
<span className="truncate">{formatModelLabel(value)}</span>
|
||||
<ChevronDown className="h-2.5 w-2.5 shrink-0 opacity-50" />
|
||||
</button>
|
||||
</PopoverTrigger>
|
||||
<PopoverContent className="w-64 p-0" align="start" side="top">
|
||||
<Command>
|
||||
<CommandInput placeholder="Filter models..." />
|
||||
<CommandList>
|
||||
<CommandEmpty>No models found.</CommandEmpty>
|
||||
{models.length > 0 ? (
|
||||
Array.from(providers.entries()).map(([provider, providerModels]) => (
|
||||
<CommandGroup key={provider} heading={provider}>
|
||||
{providerModels.map((m) => (
|
||||
<CommandItem
|
||||
key={m.id}
|
||||
value={m.id}
|
||||
onSelect={() => {
|
||||
onChange(m.id);
|
||||
setOpen(false);
|
||||
}}
|
||||
>
|
||||
<Check
|
||||
className={cn(
|
||||
"h-3 w-3 shrink-0",
|
||||
value === m.id ? "opacity-100" : "opacity-0",
|
||||
)}
|
||||
/>
|
||||
<span className="truncate">{formatModelLabel(m.id)}</span>
|
||||
</CommandItem>
|
||||
))}
|
||||
</CommandGroup>
|
||||
))
|
||||
) : (
|
||||
<CommandGroup>
|
||||
<CommandItem value={value} onSelect={() => setOpen(false)}>
|
||||
<Check className="h-3 w-3 shrink-0 opacity-100" />
|
||||
<span className="truncate">{value}</span>
|
||||
</CommandItem>
|
||||
</CommandGroup>
|
||||
)}
|
||||
</CommandList>
|
||||
</Command>
|
||||
</PopoverContent>
|
||||
</Popover>
|
||||
);
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
"use client";
|
||||
|
||||
import { useCallback, useRef, useState, type SyntheticEvent } from "react";
|
||||
import { useCallback, useEffect, useRef, useState, type SyntheticEvent } from "react";
|
||||
import { useAtom } from "jotai/react";
|
||||
import { useAtomValue, useSetAtom } from "jotai/react";
|
||||
import type { SessionInfo, TabInfo } from "@/types";
|
||||
import {
|
||||
@@ -13,9 +14,11 @@ import {
|
||||
closeTabAtom,
|
||||
addTabAtom,
|
||||
switchTabAtom,
|
||||
newSessionDialogAtom,
|
||||
} from "@/store/sessions";
|
||||
import { tabsForPortAtom, engineForPortAtom } from "@/store/tabs";
|
||||
import { ChevronRight, Loader2, Plus, Trash2 } from "lucide-react";
|
||||
import { ThemeToggle } from "@/components/theme-toggle";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { Badge } from "@/components/ui/badge";
|
||||
import { Button } from "@/components/ui/button";
|
||||
@@ -116,7 +119,7 @@ function getFaviconUrl(url: string): string | null {
|
||||
function TabFavicon({ url }: { url: string }) {
|
||||
const src = getFaviconUrl(url);
|
||||
if (!src) {
|
||||
return <span className="flex size-3.5 shrink-0 items-center justify-center rounded-sm bg-muted text-[8px] text-muted-foreground">●</span>;
|
||||
return <span className="flex size-4 shrink-0 items-center justify-center rounded-sm bg-muted text-[8px] text-muted-foreground">●</span>;
|
||||
}
|
||||
const handleError = (e: SyntheticEvent<HTMLImageElement>) => {
|
||||
(e.target as HTMLImageElement).style.display = "none";
|
||||
@@ -125,9 +128,9 @@ function TabFavicon({ url }: { url: string }) {
|
||||
<img
|
||||
src={src}
|
||||
alt=""
|
||||
width={14}
|
||||
height={14}
|
||||
className="size-3.5 shrink-0 rounded-sm"
|
||||
width={16}
|
||||
height={16}
|
||||
className="size-4 shrink-0 rounded-sm"
|
||||
onError={handleError}
|
||||
/>
|
||||
);
|
||||
@@ -149,7 +152,7 @@ function TabNode({ tab, isViewed, isSessionActive, onClose, onSwitch, onSelectSe
|
||||
<button
|
||||
onClick={isClickable ? handleClick : undefined}
|
||||
className={cn(
|
||||
"flex w-full min-w-0 items-center gap-1.5 py-1 pr-1 pl-7 text-left text-xs",
|
||||
"flex w-full min-w-0 items-center gap-2 py-1 pr-1 pl-7 text-left text-xs",
|
||||
isViewed
|
||||
? "bg-card text-foreground"
|
||||
: "text-muted-foreground cursor-pointer hover:text-foreground",
|
||||
@@ -321,9 +324,9 @@ function SessionNode({
|
||||
))}
|
||||
<button
|
||||
onClick={onAddTab}
|
||||
className="flex w-full items-center gap-1.5 py-1 pr-1 pl-7 text-xs text-muted-foreground hover:text-foreground"
|
||||
className="flex w-full items-center gap-2 py-1 pr-1 pl-7 text-xs text-muted-foreground hover:text-foreground"
|
||||
>
|
||||
<Plus className="size-3.5" />
|
||||
<Plus className="size-4" />
|
||||
Add tab
|
||||
</button>
|
||||
</div>
|
||||
@@ -347,7 +350,7 @@ export function SessionTree() {
|
||||
const dispatchSwitchTab = useSetAtom(switchTabAtom);
|
||||
|
||||
const [expandedMap, setExpandedMap] = useState<Record<number, boolean>>({});
|
||||
const [newSessionOpen, setNewSessionOpen] = useState(false);
|
||||
const [newSessionOpen, setNewSessionOpen] = useAtom(newSessionDialogAtom);
|
||||
const [closeAllOpen, setCloseAllOpen] = useState(false);
|
||||
const [newSessionName, setNewSessionName] = useState("");
|
||||
const [newSessionBrowser, setNewSessionBrowser] = useState("chrome");
|
||||
@@ -384,11 +387,21 @@ export function SessionTree() {
|
||||
}
|
||||
}, [newSessionName, newSessionBrowser, creating, dispatchCreateSession]);
|
||||
|
||||
useEffect(() => {
|
||||
if (newSessionOpen && !newSessionName) {
|
||||
const existing = new Set(sessions.map((s) => s.session));
|
||||
let n = sessions.length + 1;
|
||||
while (existing.has(`session-${n}`)) n++;
|
||||
setNewSessionName(`session-${n}`);
|
||||
}
|
||||
}, [newSessionOpen, newSessionName, sessions]);
|
||||
|
||||
return (
|
||||
<div className="flex h-full flex-col">
|
||||
<div className="flex shrink-0 items-center px-3 py-2">
|
||||
<span className="text-xs text-muted-foreground">Sessions</span>
|
||||
<div className="ml-auto flex items-center gap-0.5">
|
||||
<ThemeToggle />
|
||||
{sessions.some((s) => !s.pending) && (
|
||||
<button
|
||||
type="button"
|
||||
@@ -401,7 +414,13 @@ export function SessionTree() {
|
||||
)}
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setNewSessionOpen(true)}
|
||||
onClick={() => {
|
||||
const existing = new Set(sessions.map((s) => s.session));
|
||||
let n = sessions.length + 1;
|
||||
while (existing.has(`session-${n}`)) n++;
|
||||
setNewSessionName(`session-${n}`);
|
||||
setNewSessionOpen(true);
|
||||
}}
|
||||
className="flex size-5 items-center justify-center rounded text-muted-foreground hover:bg-muted hover:text-foreground"
|
||||
title="New session"
|
||||
>
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
"use client";
|
||||
|
||||
import { ThemeProvider as NextThemesProvider } from "next-themes";
|
||||
|
||||
export function ThemeProvider({
|
||||
children,
|
||||
...props
|
||||
}: React.ComponentProps<typeof NextThemesProvider>) {
|
||||
return <NextThemesProvider {...props}>{children}</NextThemesProvider>;
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
"use client";
|
||||
|
||||
import { useTheme } from "next-themes";
|
||||
import { Moon, Sun } from "lucide-react";
|
||||
|
||||
export function ThemeToggle() {
|
||||
const { resolvedTheme, setTheme } = useTheme();
|
||||
|
||||
return (
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setTheme(resolvedTheme === "dark" ? "light" : "dark")}
|
||||
className="flex size-5 items-center justify-center rounded text-muted-foreground hover:bg-muted hover:text-foreground"
|
||||
title={resolvedTheme === "dark" ? "Switch to light mode" : "Switch to dark mode"}
|
||||
>
|
||||
{resolvedTheme === "dark" ? (
|
||||
<Sun className="size-3" />
|
||||
) : (
|
||||
<Moon className="size-3" />
|
||||
)}
|
||||
</button>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,98 @@
|
||||
"use client";
|
||||
|
||||
import * as React from "react";
|
||||
import { Command as CommandPrimitive } from "cmdk";
|
||||
import { Search } from "lucide-react";
|
||||
import { cn } from "@/lib/utils";
|
||||
|
||||
const Command = React.forwardRef<
|
||||
React.ComponentRef<typeof CommandPrimitive>,
|
||||
React.ComponentPropsWithoutRef<typeof CommandPrimitive>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<CommandPrimitive
|
||||
ref={ref}
|
||||
className={cn(
|
||||
"flex h-full w-full flex-col overflow-hidden rounded-md bg-popover text-popover-foreground",
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
/>
|
||||
));
|
||||
Command.displayName = CommandPrimitive.displayName;
|
||||
|
||||
const CommandInput = React.forwardRef<
|
||||
React.ComponentRef<typeof CommandPrimitive.Input>,
|
||||
React.ComponentPropsWithoutRef<typeof CommandPrimitive.Input>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<div className="flex items-center border-b px-2" cmdk-input-wrapper="">
|
||||
<Search className="mr-1.5 h-3 w-3 shrink-0 opacity-50" />
|
||||
<CommandPrimitive.Input
|
||||
ref={ref}
|
||||
className={cn(
|
||||
"flex h-8 w-full rounded-md bg-transparent py-1.5 text-xs outline-none placeholder:text-muted-foreground disabled:cursor-not-allowed disabled:opacity-50",
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
/>
|
||||
</div>
|
||||
));
|
||||
CommandInput.displayName = CommandPrimitive.Input.displayName;
|
||||
|
||||
const CommandList = React.forwardRef<
|
||||
React.ComponentRef<typeof CommandPrimitive.List>,
|
||||
React.ComponentPropsWithoutRef<typeof CommandPrimitive.List>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<CommandPrimitive.List
|
||||
ref={ref}
|
||||
className={cn("max-h-[200px] overflow-y-auto overflow-x-hidden", className)}
|
||||
{...props}
|
||||
/>
|
||||
));
|
||||
CommandList.displayName = CommandPrimitive.List.displayName;
|
||||
|
||||
const CommandEmpty = React.forwardRef<
|
||||
React.ComponentRef<typeof CommandPrimitive.Empty>,
|
||||
React.ComponentPropsWithoutRef<typeof CommandPrimitive.Empty>
|
||||
>((props, ref) => (
|
||||
<CommandPrimitive.Empty ref={ref} className="py-4 text-center text-xs text-muted-foreground" {...props} />
|
||||
));
|
||||
CommandEmpty.displayName = CommandPrimitive.Empty.displayName;
|
||||
|
||||
const CommandGroup = React.forwardRef<
|
||||
React.ComponentRef<typeof CommandPrimitive.Group>,
|
||||
React.ComponentPropsWithoutRef<typeof CommandPrimitive.Group>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<CommandPrimitive.Group
|
||||
ref={ref}
|
||||
className={cn(
|
||||
"overflow-hidden p-1 text-foreground [&_[cmdk-group-heading]]:px-2 [&_[cmdk-group-heading]]:py-1 [&_[cmdk-group-heading]]:text-[10px] [&_[cmdk-group-heading]]:font-medium [&_[cmdk-group-heading]]:text-muted-foreground",
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
/>
|
||||
));
|
||||
CommandGroup.displayName = CommandPrimitive.Group.displayName;
|
||||
|
||||
const CommandItem = React.forwardRef<
|
||||
React.ComponentRef<typeof CommandPrimitive.Item>,
|
||||
React.ComponentPropsWithoutRef<typeof CommandPrimitive.Item>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<CommandPrimitive.Item
|
||||
ref={ref}
|
||||
className={cn(
|
||||
"relative flex cursor-default gap-2 select-none items-center rounded-sm px-2 py-1 text-xs outline-none data-[selected=true]:bg-accent data-[selected=true]:text-accent-foreground data-[disabled=true]:pointer-events-none data-[disabled=true]:opacity-50",
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
/>
|
||||
));
|
||||
CommandItem.displayName = CommandPrimitive.Item.displayName;
|
||||
|
||||
export {
|
||||
Command,
|
||||
CommandInput,
|
||||
CommandList,
|
||||
CommandEmpty,
|
||||
CommandGroup,
|
||||
CommandItem,
|
||||
};
|
||||
@@ -0,0 +1,30 @@
|
||||
"use client";
|
||||
|
||||
import * as React from "react";
|
||||
import * as PopoverPrimitive from "@radix-ui/react-popover";
|
||||
import { cn } from "@/lib/utils";
|
||||
|
||||
const Popover = PopoverPrimitive.Root;
|
||||
const PopoverTrigger = PopoverPrimitive.Trigger;
|
||||
const PopoverAnchor = PopoverPrimitive.Anchor;
|
||||
|
||||
const PopoverContent = React.forwardRef<
|
||||
React.ComponentRef<typeof PopoverPrimitive.Content>,
|
||||
React.ComponentPropsWithoutRef<typeof PopoverPrimitive.Content>
|
||||
>(({ className, align = "center", sideOffset = 4, ...props }, ref) => (
|
||||
<PopoverPrimitive.Portal>
|
||||
<PopoverPrimitive.Content
|
||||
ref={ref}
|
||||
align={align}
|
||||
sideOffset={sideOffset}
|
||||
className={cn(
|
||||
"z-50 w-72 rounded-md border bg-popover p-4 text-popover-foreground shadow-md outline-none data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0 data-[state=closed]:zoom-out-95 data-[state=open]:zoom-in-95 data-[side=bottom]:slide-in-from-top-2 data-[side=left]:slide-in-from-right-2 data-[side=right]:slide-in-from-left-2 data-[side=top]:slide-in-from-bottom-2",
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
/>
|
||||
</PopoverPrimitive.Portal>
|
||||
));
|
||||
PopoverContent.displayName = PopoverPrimitive.Content.displayName;
|
||||
|
||||
export { Popover, PopoverTrigger, PopoverContent, PopoverAnchor };
|
||||
@@ -1,5 +1,3 @@
|
||||
const DASHBOARD_PORT = 4848;
|
||||
|
||||
export interface ExecResult {
|
||||
success: boolean;
|
||||
exit_code: number | null;
|
||||
@@ -9,7 +7,7 @@ export interface ExecResult {
|
||||
|
||||
export async function execCommand(args: string[]): Promise<ExecResult> {
|
||||
try {
|
||||
const resp = await fetch(`http://localhost:${DASHBOARD_PORT}/api/exec`, {
|
||||
const resp = await fetch("/api/exec", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ args }),
|
||||
@@ -31,7 +29,7 @@ export function sessionArgs(session: string, ...args: string[]): string[] {
|
||||
|
||||
export async function killSession(session: string): Promise<{ success: boolean; killed_pid?: number }> {
|
||||
try {
|
||||
const resp = await fetch(`http://localhost:${DASHBOARD_PORT}/api/kill`, {
|
||||
const resp = await fetch("/api/kill", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ session }),
|
||||
|
||||
@@ -0,0 +1,160 @@
|
||||
import type { ThemeRegistrationAny } from "streamdown";
|
||||
|
||||
const lightTheme: ThemeRegistrationAny = {
|
||||
name: "dashboard-light",
|
||||
type: "light",
|
||||
colors: {
|
||||
"editor.background": "transparent",
|
||||
"editor.foreground": "#171717",
|
||||
},
|
||||
settings: [
|
||||
{
|
||||
scope: ["comment", "punctuation.definition.comment"],
|
||||
settings: { foreground: "#6B7280" },
|
||||
},
|
||||
{
|
||||
scope: [
|
||||
"string",
|
||||
"string.quoted",
|
||||
"string.template",
|
||||
"punctuation.definition.string",
|
||||
],
|
||||
settings: { foreground: "#067A6E" },
|
||||
},
|
||||
{
|
||||
scope: [
|
||||
"constant.numeric",
|
||||
"constant.language.boolean",
|
||||
"constant.language.null",
|
||||
],
|
||||
settings: { foreground: "#0070C0" },
|
||||
},
|
||||
{
|
||||
scope: ["keyword", "storage.type", "storage.modifier"],
|
||||
settings: { foreground: "#D6409F" },
|
||||
},
|
||||
{
|
||||
scope: ["keyword.operator", "keyword.control"],
|
||||
settings: { foreground: "#D6409F" },
|
||||
},
|
||||
{
|
||||
scope: ["entity.name.function", "support.function", "meta.function-call"],
|
||||
settings: { foreground: "#6E56CF" },
|
||||
},
|
||||
{
|
||||
scope: ["variable", "variable.other"],
|
||||
settings: { foreground: "#171717" },
|
||||
},
|
||||
{
|
||||
scope: ["variable.parameter"],
|
||||
settings: { foreground: "#B45309" },
|
||||
},
|
||||
{
|
||||
scope: ["entity.name.tag", "support.class.component", "entity.name.type"],
|
||||
settings: { foreground: "#D6409F" },
|
||||
},
|
||||
{
|
||||
scope: ["punctuation", "meta.brace", "meta.bracket"],
|
||||
settings: { foreground: "#6B7280" },
|
||||
},
|
||||
{
|
||||
scope: [
|
||||
"support.type.property-name",
|
||||
"entity.name.tag.json",
|
||||
"meta.object-literal.key",
|
||||
"punctuation.support.type.property-name",
|
||||
],
|
||||
settings: { foreground: "#D6409F" },
|
||||
},
|
||||
{
|
||||
scope: ["entity.other.attribute-name"],
|
||||
settings: { foreground: "#067A6E" },
|
||||
},
|
||||
{
|
||||
scope: ["support.type.primitive", "entity.name.type.primitive"],
|
||||
settings: { foreground: "#067A6E" },
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
const darkTheme: ThemeRegistrationAny = {
|
||||
name: "dashboard-dark",
|
||||
type: "dark",
|
||||
colors: {
|
||||
"editor.background": "transparent",
|
||||
"editor.foreground": "#EDEDED",
|
||||
},
|
||||
settings: [
|
||||
{
|
||||
scope: ["comment", "punctuation.definition.comment"],
|
||||
settings: { foreground: "#A1A1A1" },
|
||||
},
|
||||
{
|
||||
scope: [
|
||||
"string",
|
||||
"string.quoted",
|
||||
"string.template",
|
||||
"punctuation.definition.string",
|
||||
],
|
||||
settings: { foreground: "#00CA50" },
|
||||
},
|
||||
{
|
||||
scope: [
|
||||
"constant.numeric",
|
||||
"constant.language.boolean",
|
||||
"constant.language.null",
|
||||
],
|
||||
settings: { foreground: "#47A8FF" },
|
||||
},
|
||||
{
|
||||
scope: ["keyword", "storage.type", "storage.modifier"],
|
||||
settings: { foreground: "#FF4D8D" },
|
||||
},
|
||||
{
|
||||
scope: ["keyword.operator", "keyword.control"],
|
||||
settings: { foreground: "#FF4D8D" },
|
||||
},
|
||||
{
|
||||
scope: ["entity.name.function", "support.function", "meta.function-call"],
|
||||
settings: { foreground: "#C472FB" },
|
||||
},
|
||||
{
|
||||
scope: ["variable", "variable.other"],
|
||||
settings: { foreground: "#EDEDED" },
|
||||
},
|
||||
{
|
||||
scope: ["variable.parameter"],
|
||||
settings: { foreground: "#FF9300" },
|
||||
},
|
||||
{
|
||||
scope: ["entity.name.tag", "support.class.component", "entity.name.type"],
|
||||
settings: { foreground: "#FF4D8D" },
|
||||
},
|
||||
{
|
||||
scope: ["punctuation", "meta.brace", "meta.bracket"],
|
||||
settings: { foreground: "#EDEDED" },
|
||||
},
|
||||
{
|
||||
scope: [
|
||||
"support.type.property-name",
|
||||
"entity.name.tag.json",
|
||||
"meta.object-literal.key",
|
||||
"punctuation.support.type.property-name",
|
||||
],
|
||||
settings: { foreground: "#FF4D8D" },
|
||||
},
|
||||
{
|
||||
scope: ["entity.other.attribute-name"],
|
||||
settings: { foreground: "#00CA50" },
|
||||
},
|
||||
{
|
||||
scope: ["support.type.primitive", "entity.name.type.primitive"],
|
||||
settings: { foreground: "#00CA50" },
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
export const shikiTheme: [ThemeRegistrationAny, ThemeRegistrationAny] = [
|
||||
lightTheme,
|
||||
darkTheme,
|
||||
];
|
||||
@@ -0,0 +1,81 @@
|
||||
"use client";
|
||||
|
||||
import { atom } from "jotai";
|
||||
import { useEffect } from "react";
|
||||
import { useAtomCallback } from "jotai/utils";
|
||||
import { useCallback } from "react";
|
||||
|
||||
const DAEMON_URL = process.env.NEXT_PUBLIC_DAEMON_URL || "";
|
||||
|
||||
function daemonBase(): string {
|
||||
if (typeof window === "undefined" || !DAEMON_URL) return "";
|
||||
try {
|
||||
const daemon = new URL(DAEMON_URL);
|
||||
if (window.location.host === daemon.host) return "";
|
||||
return DAEMON_URL;
|
||||
} catch {
|
||||
return "";
|
||||
}
|
||||
}
|
||||
|
||||
function getChatStatusUrl(): string {
|
||||
return `${daemonBase()}/api/chat/status`;
|
||||
}
|
||||
|
||||
export function getChatApiUrl(): string {
|
||||
return `${daemonBase()}/api/chat`;
|
||||
}
|
||||
|
||||
export function getModelsApiUrl(): string {
|
||||
return `${daemonBase()}/api/models`;
|
||||
}
|
||||
|
||||
export interface ModelInfo {
|
||||
id: string;
|
||||
name?: string;
|
||||
owned_by?: string;
|
||||
context_window?: number;
|
||||
}
|
||||
|
||||
export const chatEnabledAtom = atom(false);
|
||||
export const chatModelAtom = atom<string | undefined>(undefined);
|
||||
export const availableModelsAtom = atom<ModelInfo[]>([]);
|
||||
|
||||
export function useChatStatusSync() {
|
||||
const fetchStatus = useAtomCallback(
|
||||
useCallback(async (_get, set) => {
|
||||
try {
|
||||
const resp = await fetch(getChatStatusUrl());
|
||||
if (resp.ok) {
|
||||
const data = await resp.json();
|
||||
set(chatEnabledAtom, !!data.enabled);
|
||||
if (data.model) set(chatModelAtom, data.model);
|
||||
}
|
||||
} catch {
|
||||
set(chatEnabledAtom, false);
|
||||
}
|
||||
try {
|
||||
const resp = await fetch(getModelsApiUrl());
|
||||
if (resp.ok) {
|
||||
const data = await resp.json();
|
||||
if (Array.isArray(data?.data)) {
|
||||
const models: ModelInfo[] = data.data.map((m: Record<string, unknown>) => ({
|
||||
id: m.id as string,
|
||||
name: (m.name as string) || undefined,
|
||||
owned_by: (m.owned_by as string) || undefined,
|
||||
context_window: typeof m.context_window === "number" ? m.context_window : undefined,
|
||||
}));
|
||||
models.sort((a, b) => a.id.localeCompare(b.id));
|
||||
set(availableModelsAtom, models);
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// models fetch failed, leave empty
|
||||
}
|
||||
}, []),
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
fetchStatus();
|
||||
}, [fetchStatus]);
|
||||
}
|
||||
@@ -15,16 +15,10 @@ function getPort(): number {
|
||||
return p ? parseInt(p, 10) || 9223 : 9223;
|
||||
}
|
||||
|
||||
const DASHBOARD_PORT = 4848;
|
||||
export const newSessionDialogAtom = atom(false);
|
||||
|
||||
function getSessionsUrl(): string {
|
||||
if (typeof window !== "undefined") {
|
||||
const origin = window.location.origin;
|
||||
if (origin.includes(`:${DASHBOARD_PORT}`)) {
|
||||
return "/api/sessions";
|
||||
}
|
||||
}
|
||||
return `http://localhost:${DASHBOARD_PORT}/api/sessions`;
|
||||
return "/api/sessions";
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -125,12 +119,21 @@ function parseExecError(result: ExecResult): string {
|
||||
return "";
|
||||
}
|
||||
|
||||
const CHAT_STORAGE_PREFIX = "dashboard-chat-";
|
||||
|
||||
function clearChatStorage(sessionName: string) {
|
||||
try {
|
||||
sessionStorage.removeItem(`${CHAT_STORAGE_PREFIX}${sessionName}`);
|
||||
} catch { /* ignore */ }
|
||||
}
|
||||
|
||||
export const closeSessionAtom = atom(null, (get, set, port: number) => {
|
||||
const sessions = get(sessionsAtom);
|
||||
const s = sessions.find((x) => x.port === port)?.session;
|
||||
if (s) {
|
||||
set(closingSessionsAtom, (prev) => new Set(prev).add(s));
|
||||
execCommand(sessionArgs(s, "close"));
|
||||
clearChatStorage(s);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -140,6 +143,7 @@ export const killSessionAtom = atom(null, (get, set, port: number) => {
|
||||
if (s) {
|
||||
set(closingSessionsAtom, (prev) => new Set(prev).add(s));
|
||||
killSession(s);
|
||||
clearChatStorage(s);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -149,6 +153,7 @@ export const closeAllSessionsAtom = atom(null, (get, set) => {
|
||||
if (!s.pending && !s.closing) {
|
||||
set(closingSessionsAtom, (prev) => new Set(prev).add(s.session));
|
||||
execCommand(sessionArgs(s.session, "close"));
|
||||
clearChatStorage(s.session);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
File diff suppressed because one or more lines are too long
Generated
+95
-27
@@ -12,7 +12,7 @@ importers:
|
||||
dependencies:
|
||||
'@ai-sdk/react':
|
||||
specifier: ^3.0.80
|
||||
version: 3.0.140(react@19.2.3)(zod@4.3.6)
|
||||
version: 3.0.148(react@19.2.3)(zod@4.3.6)
|
||||
'@mdx-js/loader':
|
||||
specifier: ^3.1.1
|
||||
version: 3.1.1
|
||||
@@ -42,10 +42,10 @@ importers:
|
||||
version: 1.3.1(next@16.1.1(@opentelemetry/api@1.9.0)(react-dom@19.2.3(react@19.2.3))(react@19.2.3))(react@19.2.3)
|
||||
ai:
|
||||
specifier: ^6.0.78
|
||||
version: 6.0.138(zod@4.3.6)
|
||||
version: 6.0.146(zod@4.3.6)
|
||||
bash-tool:
|
||||
specifier: ^1.3.14
|
||||
version: 1.3.15(ai@6.0.138(zod@4.3.6))(just-bash@2.14.0)
|
||||
version: 1.3.15(ai@6.0.146(zod@4.3.6))(just-bash@2.14.0)
|
||||
clsx:
|
||||
specifier: ^2.1.1
|
||||
version: 2.1.1
|
||||
@@ -113,12 +113,24 @@ importers:
|
||||
|
||||
packages/dashboard:
|
||||
dependencies:
|
||||
'@ai-sdk/react':
|
||||
specifier: ^3.0.148
|
||||
version: 3.0.148(react@19.2.3)(zod@3.25.76)
|
||||
'@radix-ui/react-popover':
|
||||
specifier: ^1.1.15
|
||||
version: 1.1.15(@types/react-dom@19.2.3(@types/react@19.2.14))(@types/react@19.2.14)(react-dom@19.2.3(react@19.2.3))(react@19.2.3)
|
||||
ai:
|
||||
specifier: ^6.0.146
|
||||
version: 6.0.146(zod@3.25.76)
|
||||
class-variance-authority:
|
||||
specifier: ^0.7.1
|
||||
version: 0.7.1
|
||||
clsx:
|
||||
specifier: ^2.1.1
|
||||
version: 2.1.1
|
||||
cmdk:
|
||||
specifier: ^1.1.1
|
||||
version: 1.1.1(@types/react-dom@19.2.3(@types/react@19.2.14))(@types/react@19.2.14)(react-dom@19.2.3(react@19.2.3))(react@19.2.3)
|
||||
jotai:
|
||||
specifier: ^2.19.0
|
||||
version: 2.19.0(@babel/core@7.29.0)(@babel/template@7.28.6)(@types/react@19.2.14)(react@19.2.3)
|
||||
@@ -128,6 +140,9 @@ importers:
|
||||
next:
|
||||
specifier: 16.1.1
|
||||
version: 16.1.1(@babel/core@7.29.0)(@opentelemetry/api@1.9.0)(react-dom@19.2.3(react@19.2.3))(react@19.2.3)
|
||||
next-themes:
|
||||
specifier: ^0.4.6
|
||||
version: 0.4.6(react-dom@19.2.3(react@19.2.3))(react@19.2.3)
|
||||
radix-ui:
|
||||
specifier: ^1.4.3
|
||||
version: 1.4.3(@types/react-dom@19.2.3(@types/react@19.2.14))(@types/react@19.2.14)(react-dom@19.2.3(react@19.2.3))(react@19.2.3)
|
||||
@@ -143,6 +158,9 @@ importers:
|
||||
shadcn:
|
||||
specifier: ^4.1.0
|
||||
version: 4.1.0(@types/node@22.19.15)(typescript@5.9.3)
|
||||
streamdown:
|
||||
specifier: ^2.5.0
|
||||
version: 2.5.0(react-dom@19.2.3(react@19.2.3))(react@19.2.3)
|
||||
tailwind-merge:
|
||||
specifier: ^3.5.0
|
||||
version: 3.5.0
|
||||
@@ -171,14 +189,14 @@ importers:
|
||||
|
||||
packages:
|
||||
|
||||
'@ai-sdk/gateway@3.0.80':
|
||||
resolution: {integrity: sha512-uM7kpZB5l977lW7+2X1+klBUxIZQ78+1a9jHlaHFEzcOcmmslTl3sdP0QqfuuBcO0YBM2gwOiqVdp8i4TRQYcw==}
|
||||
'@ai-sdk/gateway@3.0.88':
|
||||
resolution: {integrity: sha512-AFoj7xdWAtCQcy0jJ235ENSakYM8D28qBX+rB+/rX4r8qe/LXgl0e5UivOqxAlIM5E9jnQdYxIPuj3XFtGk/yg==}
|
||||
engines: {node: '>=18'}
|
||||
peerDependencies:
|
||||
zod: ^3.25.76 || ^4.1.8
|
||||
|
||||
'@ai-sdk/provider-utils@4.0.21':
|
||||
resolution: {integrity: sha512-MtFUYI1/8mgDvRmaBDjbLJPFFrMG777AvSgyIFQtZHIMzm88R/12vYBBpnk7pfiWLFE1DSZzY4WDYzGbKAcmiw==}
|
||||
'@ai-sdk/provider-utils@4.0.22':
|
||||
resolution: {integrity: sha512-B2OTFcRw/Pdka9ZTjpXv6T6qZ6RruRuLokyb8HwW+aoW9ndJ3YasA3/mVswyJw7VMBF8ofXgqvcrCt9KYvFifg==}
|
||||
engines: {node: '>=18'}
|
||||
peerDependencies:
|
||||
zod: ^3.25.76 || ^4.1.8
|
||||
@@ -187,8 +205,8 @@ packages:
|
||||
resolution: {integrity: sha512-oGMAgGoQdBXbZqNG0Ze56CHjDZ1IDYOwGYxYjO5KLSlz5HiNQ9udIXsPZ61VWaHGZ5XW/jyjmr6t2xz2jGVwbQ==}
|
||||
engines: {node: '>=18'}
|
||||
|
||||
'@ai-sdk/react@3.0.140':
|
||||
resolution: {integrity: sha512-wCL9iTrzoW8ppYVrz7BFCurCEqW6hjCoFFyxkrS3us2pSbUlQpXHmrprFQevzfa2PJXiF5hss7ZN8ZNdiZW//A==}
|
||||
'@ai-sdk/react@3.0.148':
|
||||
resolution: {integrity: sha512-bxKtS3KINzjtEf9xrAhORRN0HIMgqlI1Nwhd0eaAXL3Eljf3XVl9Bw+HXiCLVNzyOcyOwwBLlvq8SZ0amys7eA==}
|
||||
engines: {node: '>=18'}
|
||||
peerDependencies:
|
||||
react: ^18 || ~19.0.1 || ~19.1.2 || ^19.2.1
|
||||
@@ -2031,8 +2049,8 @@ packages:
|
||||
resolution: {integrity: sha512-MnA+YT8fwfJPgBx3m60MNqakm30XOkyIoH1y6huTQvC0PwZG7ki8NacLBcrPbNoo8vEZy7Jpuk7+jMO+CUovTQ==}
|
||||
engines: {node: '>= 14'}
|
||||
|
||||
ai@6.0.138:
|
||||
resolution: {integrity: sha512-49OfPe0f5uxJ6jUdA5BBXjIinP6+ZdYfAtpF2aEH64GA5wPcxH2rf/TBUQQ0bbamBz/D+TLMV18xilZqOC+zaA==}
|
||||
ai@6.0.146:
|
||||
resolution: {integrity: sha512-70DE8k1rR0N3mXxyyfjYAx/FxRln/kQ5ym18lt1ys1eUklcPuoIXGbUBwdfCbmkt6YF3jCDZ5+OgkWieP/NGDw==}
|
||||
engines: {node: '>=18'}
|
||||
peerDependencies:
|
||||
zod: ^3.25.76 || ^4.1.8
|
||||
@@ -2284,6 +2302,12 @@ packages:
|
||||
resolution: {integrity: sha512-eYm0QWBtUrBWZWG0d386OGAw16Z995PiOVo2B7bjWSbHedGl5e0ZWaq65kOGgUSNesEIDkB9ISbTg/JK9dhCZA==}
|
||||
engines: {node: '>=6'}
|
||||
|
||||
cmdk@1.1.1:
|
||||
resolution: {integrity: sha512-Vsv7kFaXm+ptHDMZ7izaRsP70GgrW9NBNGswt9OZaVBLlE0SNpDq8eu/VGXyF9r7M0azK3Wy7OlYXsuyYLFzHg==}
|
||||
peerDependencies:
|
||||
react: ^18 || ^19 || ^19.0.0-rc
|
||||
react-dom: ^18 || ^19 || ^19.0.0-rc
|
||||
|
||||
code-block-writer@13.0.3:
|
||||
resolution: {integrity: sha512-Oofo0pq3IKnsFtuHqSF7TqBfr71aeyZDVJ0HpmqB7FBM2qEigL0iPONSCZSO9pE9dZTAxANe5XHG9Uy0YMv8cg==}
|
||||
|
||||
@@ -5167,14 +5191,28 @@ packages:
|
||||
|
||||
snapshots:
|
||||
|
||||
'@ai-sdk/gateway@3.0.80(zod@4.3.6)':
|
||||
'@ai-sdk/gateway@3.0.88(zod@3.25.76)':
|
||||
dependencies:
|
||||
'@ai-sdk/provider': 3.0.8
|
||||
'@ai-sdk/provider-utils': 4.0.21(zod@4.3.6)
|
||||
'@ai-sdk/provider-utils': 4.0.22(zod@3.25.76)
|
||||
'@vercel/oidc': 3.1.0
|
||||
zod: 3.25.76
|
||||
|
||||
'@ai-sdk/gateway@3.0.88(zod@4.3.6)':
|
||||
dependencies:
|
||||
'@ai-sdk/provider': 3.0.8
|
||||
'@ai-sdk/provider-utils': 4.0.22(zod@4.3.6)
|
||||
'@vercel/oidc': 3.1.0
|
||||
zod: 4.3.6
|
||||
|
||||
'@ai-sdk/provider-utils@4.0.21(zod@4.3.6)':
|
||||
'@ai-sdk/provider-utils@4.0.22(zod@3.25.76)':
|
||||
dependencies:
|
||||
'@ai-sdk/provider': 3.0.8
|
||||
'@standard-schema/spec': 1.1.0
|
||||
eventsource-parser: 3.0.6
|
||||
zod: 3.25.76
|
||||
|
||||
'@ai-sdk/provider-utils@4.0.22(zod@4.3.6)':
|
||||
dependencies:
|
||||
'@ai-sdk/provider': 3.0.8
|
||||
'@standard-schema/spec': 1.1.0
|
||||
@@ -5185,10 +5223,20 @@ snapshots:
|
||||
dependencies:
|
||||
json-schema: 0.4.0
|
||||
|
||||
'@ai-sdk/react@3.0.140(react@19.2.3)(zod@4.3.6)':
|
||||
'@ai-sdk/react@3.0.148(react@19.2.3)(zod@3.25.76)':
|
||||
dependencies:
|
||||
'@ai-sdk/provider-utils': 4.0.21(zod@4.3.6)
|
||||
ai: 6.0.138(zod@4.3.6)
|
||||
'@ai-sdk/provider-utils': 4.0.22(zod@3.25.76)
|
||||
ai: 6.0.146(zod@3.25.76)
|
||||
react: 19.2.3
|
||||
swr: 2.4.1(react@19.2.3)
|
||||
throttleit: 2.1.0
|
||||
transitivePeerDependencies:
|
||||
- zod
|
||||
|
||||
'@ai-sdk/react@3.0.148(react@19.2.3)(zod@4.3.6)':
|
||||
dependencies:
|
||||
'@ai-sdk/provider-utils': 4.0.22(zod@4.3.6)
|
||||
ai: 6.0.146(zod@4.3.6)
|
||||
react: 19.2.3
|
||||
swr: 2.4.1(react@19.2.3)
|
||||
throttleit: 2.1.0
|
||||
@@ -7105,11 +7153,19 @@ snapshots:
|
||||
|
||||
agent-base@7.1.4: {}
|
||||
|
||||
ai@6.0.138(zod@4.3.6):
|
||||
ai@6.0.146(zod@3.25.76):
|
||||
dependencies:
|
||||
'@ai-sdk/gateway': 3.0.80(zod@4.3.6)
|
||||
'@ai-sdk/gateway': 3.0.88(zod@3.25.76)
|
||||
'@ai-sdk/provider': 3.0.8
|
||||
'@ai-sdk/provider-utils': 4.0.21(zod@4.3.6)
|
||||
'@ai-sdk/provider-utils': 4.0.22(zod@3.25.76)
|
||||
'@opentelemetry/api': 1.9.0
|
||||
zod: 3.25.76
|
||||
|
||||
ai@6.0.146(zod@4.3.6):
|
||||
dependencies:
|
||||
'@ai-sdk/gateway': 3.0.88(zod@4.3.6)
|
||||
'@ai-sdk/provider': 3.0.8
|
||||
'@ai-sdk/provider-utils': 4.0.22(zod@4.3.6)
|
||||
'@opentelemetry/api': 1.9.0
|
||||
zod: 4.3.6
|
||||
|
||||
@@ -7249,9 +7305,9 @@ snapshots:
|
||||
|
||||
baseline-browser-mapping@2.10.10: {}
|
||||
|
||||
bash-tool@1.3.15(ai@6.0.138(zod@4.3.6))(just-bash@2.14.0):
|
||||
bash-tool@1.3.15(ai@6.0.146(zod@4.3.6))(just-bash@2.14.0):
|
||||
dependencies:
|
||||
ai: 6.0.138(zod@4.3.6)
|
||||
ai: 6.0.146(zod@4.3.6)
|
||||
fast-glob: 3.3.3
|
||||
gray-matter: 4.0.3
|
||||
zod: 3.25.76
|
||||
@@ -7389,6 +7445,18 @@ snapshots:
|
||||
|
||||
clsx@2.1.1: {}
|
||||
|
||||
cmdk@1.1.1(@types/react-dom@19.2.3(@types/react@19.2.14))(@types/react@19.2.14)(react-dom@19.2.3(react@19.2.3))(react@19.2.3):
|
||||
dependencies:
|
||||
'@radix-ui/react-compose-refs': 1.1.2(@types/react@19.2.14)(react@19.2.3)
|
||||
'@radix-ui/react-dialog': 1.1.15(@types/react-dom@19.2.3(@types/react@19.2.14))(@types/react@19.2.14)(react-dom@19.2.3(react@19.2.3))(react@19.2.3)
|
||||
'@radix-ui/react-id': 1.1.1(@types/react@19.2.14)(react@19.2.3)
|
||||
'@radix-ui/react-primitive': 2.1.3(@types/react-dom@19.2.3(@types/react@19.2.14))(@types/react@19.2.14)(react-dom@19.2.3(react@19.2.3))(react@19.2.3)
|
||||
react: 19.2.3
|
||||
react-dom: 19.2.3(react@19.2.3)
|
||||
transitivePeerDependencies:
|
||||
- '@types/react'
|
||||
- '@types/react-dom'
|
||||
|
||||
code-block-writer@13.0.3: {}
|
||||
|
||||
collapse-white-space@2.1.0: {}
|
||||
@@ -7921,7 +7989,7 @@ snapshots:
|
||||
'@next/eslint-plugin-next': 16.1.1
|
||||
eslint: 9.39.4(jiti@2.6.1)
|
||||
eslint-import-resolver-node: 0.3.9
|
||||
eslint-import-resolver-typescript: 3.10.1(eslint-plugin-import@2.32.0)(eslint@9.39.4(jiti@2.6.1))
|
||||
eslint-import-resolver-typescript: 3.10.1(eslint-plugin-import@2.32.0(@typescript-eslint/parser@8.57.2(eslint@9.39.4(jiti@2.6.1))(typescript@5.9.3))(eslint@9.39.4(jiti@2.6.1)))(eslint@9.39.4(jiti@2.6.1))
|
||||
eslint-plugin-import: 2.32.0(@typescript-eslint/parser@8.57.2(eslint@9.39.4(jiti@2.6.1))(typescript@5.9.3))(eslint-import-resolver-typescript@3.10.1)(eslint@9.39.4(jiti@2.6.1))
|
||||
eslint-plugin-jsx-a11y: 6.10.2(eslint@9.39.4(jiti@2.6.1))
|
||||
eslint-plugin-react: 7.37.5(eslint@9.39.4(jiti@2.6.1))
|
||||
@@ -7944,7 +8012,7 @@ snapshots:
|
||||
transitivePeerDependencies:
|
||||
- supports-color
|
||||
|
||||
eslint-import-resolver-typescript@3.10.1(eslint-plugin-import@2.32.0)(eslint@9.39.4(jiti@2.6.1)):
|
||||
eslint-import-resolver-typescript@3.10.1(eslint-plugin-import@2.32.0(@typescript-eslint/parser@8.57.2(eslint@9.39.4(jiti@2.6.1))(typescript@5.9.3))(eslint@9.39.4(jiti@2.6.1)))(eslint@9.39.4(jiti@2.6.1)):
|
||||
dependencies:
|
||||
'@nolyfill/is-core-module': 1.0.39
|
||||
debug: 4.4.3
|
||||
@@ -7959,14 +8027,14 @@ snapshots:
|
||||
transitivePeerDependencies:
|
||||
- supports-color
|
||||
|
||||
eslint-module-utils@2.12.1(@typescript-eslint/parser@8.57.2(eslint@9.39.4(jiti@2.6.1))(typescript@5.9.3))(eslint-import-resolver-node@0.3.9)(eslint-import-resolver-typescript@3.10.1)(eslint@9.39.4(jiti@2.6.1)):
|
||||
eslint-module-utils@2.12.1(@typescript-eslint/parser@8.57.2(eslint@9.39.4(jiti@2.6.1))(typescript@5.9.3))(eslint-import-resolver-node@0.3.9)(eslint-import-resolver-typescript@3.10.1(eslint-plugin-import@2.32.0(@typescript-eslint/parser@8.57.2(eslint@9.39.4(jiti@2.6.1))(typescript@5.9.3))(eslint@9.39.4(jiti@2.6.1)))(eslint@9.39.4(jiti@2.6.1)))(eslint@9.39.4(jiti@2.6.1)):
|
||||
dependencies:
|
||||
debug: 3.2.7
|
||||
optionalDependencies:
|
||||
'@typescript-eslint/parser': 8.57.2(eslint@9.39.4(jiti@2.6.1))(typescript@5.9.3)
|
||||
eslint: 9.39.4(jiti@2.6.1)
|
||||
eslint-import-resolver-node: 0.3.9
|
||||
eslint-import-resolver-typescript: 3.10.1(eslint-plugin-import@2.32.0)(eslint@9.39.4(jiti@2.6.1))
|
||||
eslint-import-resolver-typescript: 3.10.1(eslint-plugin-import@2.32.0(@typescript-eslint/parser@8.57.2(eslint@9.39.4(jiti@2.6.1))(typescript@5.9.3))(eslint@9.39.4(jiti@2.6.1)))(eslint@9.39.4(jiti@2.6.1))
|
||||
transitivePeerDependencies:
|
||||
- supports-color
|
||||
|
||||
@@ -7981,7 +8049,7 @@ snapshots:
|
||||
doctrine: 2.1.0
|
||||
eslint: 9.39.4(jiti@2.6.1)
|
||||
eslint-import-resolver-node: 0.3.9
|
||||
eslint-module-utils: 2.12.1(@typescript-eslint/parser@8.57.2(eslint@9.39.4(jiti@2.6.1))(typescript@5.9.3))(eslint-import-resolver-node@0.3.9)(eslint-import-resolver-typescript@3.10.1)(eslint@9.39.4(jiti@2.6.1))
|
||||
eslint-module-utils: 2.12.1(@typescript-eslint/parser@8.57.2(eslint@9.39.4(jiti@2.6.1))(typescript@5.9.3))(eslint-import-resolver-node@0.3.9)(eslint-import-resolver-typescript@3.10.1(eslint-plugin-import@2.32.0(@typescript-eslint/parser@8.57.2(eslint@9.39.4(jiti@2.6.1))(typescript@5.9.3))(eslint@9.39.4(jiti@2.6.1)))(eslint@9.39.4(jiti@2.6.1)))(eslint@9.39.4(jiti@2.6.1))
|
||||
hasown: 2.0.2
|
||||
is-core-module: 2.16.1
|
||||
is-glob: 4.0.3
|
||||
|
||||
@@ -25,7 +25,7 @@ agent-browser snapshot -i
|
||||
agent-browser fill @e1 "user@example.com"
|
||||
agent-browser fill @e2 "password123"
|
||||
agent-browser click @e3
|
||||
agent-browser wait --load networkidle
|
||||
agent-browser wait 2000
|
||||
agent-browser snapshot -i # Check result
|
||||
```
|
||||
|
||||
@@ -34,14 +34,14 @@ agent-browser snapshot -i # Check result
|
||||
Commands can be chained with `&&` in a single shell invocation. The browser persists between commands via a background daemon, so chaining is safe and more efficient than separate calls.
|
||||
|
||||
```bash
|
||||
# Chain open + wait + snapshot in one call
|
||||
agent-browser open https://example.com && agent-browser wait --load networkidle && agent-browser snapshot -i
|
||||
# Chain open + snapshot in one call (open already waits for page load)
|
||||
agent-browser open https://example.com && agent-browser snapshot -i
|
||||
|
||||
# Chain multiple interactions
|
||||
agent-browser fill @e1 "user@example.com" && agent-browser fill @e2 "password123" && agent-browser click @e3
|
||||
|
||||
# Navigate and capture
|
||||
agent-browser open https://example.com && agent-browser wait --load networkidle && agent-browser screenshot page.png
|
||||
agent-browser open https://example.com && agent-browser screenshot
|
||||
```
|
||||
|
||||
**When to chain:** Use `&&` when you don't need to read the output of an intermediate command before proceeding (e.g., open + wait + screenshot). Run commands separately when you need to parse the output first (e.g., snapshot to discover refs, then interact using those refs).
|
||||
@@ -117,6 +117,11 @@ See [references/authentication.md](references/authentication.md) for OAuth, 2FA,
|
||||
## Essential Commands
|
||||
|
||||
```bash
|
||||
# Batch: ALWAYS use batch for 2+ sequential commands. Commands run in order.
|
||||
agent-browser batch "open https://example.com" "snapshot -i"
|
||||
agent-browser batch "open https://example.com" "screenshot"
|
||||
agent-browser batch "click @e1" "wait 1000" "screenshot"
|
||||
|
||||
# Navigation
|
||||
agent-browser open <url> # Navigate (aliases: goto, navigate)
|
||||
agent-browser close # Close browser
|
||||
@@ -124,6 +129,7 @@ agent-browser close --all # Close all active sessions
|
||||
|
||||
# Snapshot
|
||||
agent-browser snapshot -i # Interactive elements with refs (recommended)
|
||||
agent-browser snapshot -i --urls # Include href URLs for links
|
||||
agent-browser snapshot -s "#selector" # Scope to CSS selector
|
||||
|
||||
# Interaction (use @refs from snapshot)
|
||||
@@ -147,10 +153,10 @@ agent-browser get cdp-url # Get CDP WebSocket URL
|
||||
|
||||
# Wait
|
||||
agent-browser wait @e1 # Wait for element
|
||||
agent-browser wait --load networkidle # Wait for network idle
|
||||
agent-browser wait --url "**/page" # Wait for URL pattern
|
||||
agent-browser wait 2000 # Wait milliseconds
|
||||
agent-browser wait --text "Welcome" # Wait for text to appear (substring match)
|
||||
agent-browser wait --url "**/page" # Wait for URL pattern
|
||||
agent-browser wait --text "Welcome" # Wait for text to appear (substring match)
|
||||
agent-browser wait --load networkidle # Wait for network idle (caution: see Pitfalls)
|
||||
agent-browser wait --fn "!document.body.innerText.includes('Loading...')" # Wait for text to disappear
|
||||
agent-browser wait "#spinner" --state hidden # Wait for element to disappear
|
||||
|
||||
@@ -159,6 +165,14 @@ agent-browser download @e1 ./file.pdf # Click element to trigger downlo
|
||||
agent-browser wait --download ./output.zip # Wait for any download to complete
|
||||
agent-browser --download-path ./downloads open <url> # Set default download directory
|
||||
|
||||
# Tab management
|
||||
agent-browser tab list # List all open tabs
|
||||
agent-browser tab new # Open a blank new tab
|
||||
agent-browser tab new https://example.com # Open URL in a new tab
|
||||
agent-browser tab 2 # Switch to tab by index (0-based)
|
||||
agent-browser tab close # Close the current tab
|
||||
agent-browser tab close 2 # Close tab by index
|
||||
|
||||
# Network
|
||||
agent-browser network requests # Inspect tracked requests
|
||||
agent-browser network requests --type xhr,fetch # Filter by resource type
|
||||
@@ -218,35 +232,62 @@ Every session automatically starts a WebSocket stream server on an OS-assigned p
|
||||
|
||||
## Batch Execution
|
||||
|
||||
Execute multiple commands in a single invocation by piping a JSON array of string arrays to `batch`. This avoids per-command process startup overhead when running multi-step workflows.
|
||||
ALWAYS use `batch` when running 2+ commands in sequence. Batch executes commands in order, so dependent commands (like navigate then screenshot) work correctly. Each quoted argument is a separate command.
|
||||
|
||||
```bash
|
||||
echo '[
|
||||
["open", "https://example.com"],
|
||||
["snapshot", "-i"],
|
||||
["click", "@e1"],
|
||||
["screenshot", "result.png"]
|
||||
]' | agent-browser batch --json
|
||||
# Navigate and take a snapshot
|
||||
agent-browser batch "open https://example.com" "snapshot -i"
|
||||
|
||||
# Stop on first error
|
||||
# Navigate, snapshot, and screenshot in one call
|
||||
agent-browser batch "open https://example.com" "snapshot -i" "screenshot"
|
||||
|
||||
# Click, wait, then screenshot
|
||||
agent-browser batch "click @e1" "wait 1000" "screenshot"
|
||||
|
||||
# With --bail to stop on first error
|
||||
agent-browser batch --bail "open https://example.com" "click @e1" "screenshot"
|
||||
```
|
||||
|
||||
Only use a single command (not batch) when you need to read the output before deciding the next command. For example, you must run `snapshot -i` as a single command when you need to read the refs to decide what to click. After reading the snapshot, batch the remaining steps.
|
||||
|
||||
Stdin mode is also supported for programmatic use:
|
||||
|
||||
```bash
|
||||
echo '[["open","https://example.com"],["screenshot"]]' | agent-browser batch --json
|
||||
agent-browser batch --bail < commands.json
|
||||
```
|
||||
|
||||
Use `batch` when you have a known sequence of commands that don't depend on intermediate output. Use separate commands or `&&` chaining when you need to parse output between steps (e.g., snapshot to discover refs, then interact).
|
||||
## Efficiency Strategies
|
||||
|
||||
These patterns minimize tool calls and token usage.
|
||||
|
||||
**Use `--urls` to avoid re-navigation.** When you need to visit links from a page, use `snapshot -i --urls` to get all href URLs upfront. Then `open` each URL directly instead of clicking refs and navigating back.
|
||||
|
||||
**Snapshot once, act many times.** Never re-snapshot the same page. Extract all needed info (refs, URLs, text) from a single snapshot, then batch the remaining actions.
|
||||
|
||||
**Multi-page workflow (e.g. "visit N sites and screenshot each"):**
|
||||
|
||||
```bash
|
||||
# 1. Get all URLs in one call
|
||||
agent-browser batch "open https://news.ycombinator.com" "snapshot -i --urls"
|
||||
# Read output to extract URLs, then visit each directly:
|
||||
# 2. One batch per target site
|
||||
agent-browser batch "open https://github.com/example/repo" "screenshot"
|
||||
agent-browser batch "open https://example.com/article" "screenshot"
|
||||
agent-browser batch "open https://other.com/page" "screenshot"
|
||||
```
|
||||
|
||||
This approach uses 4 tool calls instead of 14+. Never go back to the listing page between visits.
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Form Submission
|
||||
|
||||
```bash
|
||||
agent-browser open https://example.com/signup
|
||||
agent-browser snapshot -i
|
||||
agent-browser fill @e1 "Jane Doe"
|
||||
agent-browser fill @e2 "jane@example.com"
|
||||
agent-browser select @e3 "California"
|
||||
agent-browser check @e4
|
||||
agent-browser click @e5
|
||||
agent-browser wait --load networkidle
|
||||
# Navigate and get the form structure
|
||||
agent-browser batch "open https://example.com/signup" "snapshot -i"
|
||||
# Read the snapshot output to identify form refs, then fill and submit
|
||||
agent-browser batch "fill @e1 \"Jane Doe\"" "fill @e2 \"jane@example.com\"" "select @e3 \"California\"" "check @e4" "click @e5" "wait 2000"
|
||||
```
|
||||
|
||||
### Authentication with Auth Vault (Recommended)
|
||||
@@ -271,17 +312,12 @@ agent-browser auth delete github
|
||||
|
||||
```bash
|
||||
# Login once and save state
|
||||
agent-browser open https://app.example.com/login
|
||||
agent-browser snapshot -i
|
||||
agent-browser fill @e1 "$USERNAME"
|
||||
agent-browser fill @e2 "$PASSWORD"
|
||||
agent-browser click @e3
|
||||
agent-browser wait --url "**/dashboard"
|
||||
agent-browser state save auth.json
|
||||
agent-browser batch "open https://app.example.com/login" "snapshot -i"
|
||||
# Read snapshot to find form refs, then fill and submit
|
||||
agent-browser batch "fill @e1 \"$USERNAME\"" "fill @e2 \"$PASSWORD\"" "click @e3" "wait --url **/dashboard" "state save auth.json"
|
||||
|
||||
# Reuse in future sessions
|
||||
agent-browser state load auth.json
|
||||
agent-browser open https://app.example.com/dashboard
|
||||
agent-browser batch "state load auth.json" "open https://app.example.com/dashboard"
|
||||
```
|
||||
|
||||
### Session Persistence
|
||||
@@ -311,8 +347,7 @@ agent-browser state clean --older-than 7
|
||||
Iframe content is automatically inlined in snapshots. Refs inside iframes carry frame context, so you can interact with them directly.
|
||||
|
||||
```bash
|
||||
agent-browser open https://example.com/checkout
|
||||
agent-browser snapshot -i
|
||||
agent-browser batch "open https://example.com/checkout" "snapshot -i"
|
||||
# @e1 [heading] "Checkout"
|
||||
# @e2 [Iframe] "payment-frame"
|
||||
# @e3 [input] "Card number"
|
||||
@@ -320,23 +355,19 @@ agent-browser snapshot -i
|
||||
# @e5 [button] "Pay"
|
||||
|
||||
# Interact directly — no frame switch needed
|
||||
agent-browser fill @e3 "4111111111111111"
|
||||
agent-browser fill @e4 "12/28"
|
||||
agent-browser click @e5
|
||||
agent-browser batch "fill @e3 \"4111111111111111\"" "fill @e4 \"12/28\"" "click @e5"
|
||||
|
||||
# To scope a snapshot to one iframe:
|
||||
agent-browser frame @e2
|
||||
agent-browser snapshot -i # Only iframe content
|
||||
agent-browser batch "frame @e2" "snapshot -i"
|
||||
agent-browser frame main # Return to main frame
|
||||
```
|
||||
|
||||
### Data Extraction
|
||||
|
||||
```bash
|
||||
agent-browser open https://example.com/products
|
||||
agent-browser snapshot -i
|
||||
agent-browser batch "open https://example.com/products" "snapshot -i"
|
||||
# Read snapshot to find element refs, then extract
|
||||
agent-browser get text @e5 # Get specific element text
|
||||
agent-browser get text body > page.txt # Get all page text
|
||||
|
||||
# JSON output for parsing
|
||||
agent-browser snapshot -i --json
|
||||
@@ -530,27 +561,29 @@ agent-browser diff url https://staging.example.com https://prod.example.com --sc
|
||||
|
||||
## Timeouts and Slow Pages
|
||||
|
||||
The default timeout is 25 seconds. This can be overridden with the `AGENT_BROWSER_DEFAULT_TIMEOUT` environment variable (value in milliseconds). For slow websites or large pages, use explicit waits instead of relying on the default timeout:
|
||||
The default timeout is 25 seconds. This can be overridden with the `AGENT_BROWSER_DEFAULT_TIMEOUT` environment variable (value in milliseconds).
|
||||
|
||||
**Important:** `open` already waits for the page `load` event before returning. In most cases, no additional wait is needed before taking a snapshot or screenshot. Only add an explicit wait when content loads asynchronously after the initial page load.
|
||||
|
||||
```bash
|
||||
# Wait for network activity to settle (best for slow pages)
|
||||
agent-browser wait --load networkidle
|
||||
|
||||
# Wait for a specific element to appear
|
||||
# Wait for a specific element to appear (preferred for dynamic content)
|
||||
agent-browser wait "#content"
|
||||
agent-browser wait @e1
|
||||
|
||||
# Wait a fixed duration (good default for slow SPAs)
|
||||
agent-browser wait 2000
|
||||
|
||||
# Wait for a specific URL pattern (useful after redirects)
|
||||
agent-browser wait --url "**/dashboard"
|
||||
|
||||
# Wait for a JavaScript condition
|
||||
agent-browser wait --fn "document.readyState === 'complete'"
|
||||
# Wait for text to appear on the page
|
||||
agent-browser wait --text "Results loaded"
|
||||
|
||||
# Wait a fixed duration (milliseconds) as a last resort
|
||||
agent-browser wait 5000
|
||||
# Wait for a JavaScript condition
|
||||
agent-browser wait --fn "document.querySelectorAll('.item').length > 0"
|
||||
```
|
||||
|
||||
When dealing with consistently slow websites, use `wait --load networkidle` after `open` to ensure the page is fully loaded before taking a snapshot. If a specific element is slow to render, wait for it directly with `wait <selector>` or `wait @ref`.
|
||||
**Avoid `wait --load networkidle`** unless you are certain the site has no persistent network activity. Ad-heavy sites, sites with analytics/tracking, and sites with websockets will cause `networkidle` to hang indefinitely. Prefer `wait 2000` or `wait <selector>` instead.
|
||||
|
||||
## JavaScript Dialogs (alert / confirm / prompt)
|
||||
|
||||
@@ -764,6 +797,18 @@ agent-browser dashboard stop
|
||||
|
||||
The dashboard runs independently of browser sessions on port 4848 (configurable with `--port`). All sessions automatically stream to the dashboard. Sessions can also be created from the dashboard UI with local engines or cloud providers.
|
||||
|
||||
### Dashboard AI Chat
|
||||
|
||||
The dashboard has an optional AI chat tab powered by the Vercel AI Gateway. Enable it by setting:
|
||||
|
||||
```bash
|
||||
export AI_GATEWAY_API_KEY=gw_your_key_here
|
||||
export AI_GATEWAY_MODEL=anthropic/claude-sonnet-4.6 # optional default
|
||||
export AI_GATEWAY_URL=https://ai-gateway.vercel.sh # optional default
|
||||
```
|
||||
|
||||
The Chat tab is always visible in the dashboard. Set `AI_GATEWAY_API_KEY` to enable AI responses.
|
||||
|
||||
## Ready-to-Use Templates
|
||||
|
||||
| Template | Description |
|
||||
|
||||
Reference in New Issue
Block a user