full native (#754)
* full native * fix: apply cargo fmt formatting * fix: prevent zip path traversal in Chromium installer Use enclosed_name() to sanitize zip entry paths, preventing malicious archives from writing outside the extraction directory. * improvements * fix: apply cargo fmt formatting * benchmarks * bench * updates * fixes
This commit is contained in:
@@ -0,0 +1,4 @@
|
||||
# Vercel Sandbox credentials
|
||||
SANDBOX_VERCEL_TOKEN=
|
||||
SANDBOX_VERCEL_TEAM_ID=
|
||||
SANDBOX_VERCEL_PROJECT_ID=
|
||||
@@ -0,0 +1,2 @@
|
||||
node_modules/
|
||||
results.json
|
||||
@@ -0,0 +1,76 @@
|
||||
# agent-browser Daemon Benchmarks
|
||||
|
||||
Compares command latency and system metrics between the **Node.js daemon** (published npm version) and the **Rust native daemon** (built from source), running inside a [Vercel Sandbox](https://vercel.com/docs/sandbox) microVM.
|
||||
|
||||
## What it measures
|
||||
|
||||
**Command latency** -- per-scenario timing with warmup, multiple iterations, and stddev:
|
||||
|
||||
- `navigate` -- page load round-trip
|
||||
- `snapshot` -- accessibility tree generation
|
||||
- `screenshot` -- viewport capture
|
||||
- `evaluate` -- JavaScript execution
|
||||
- `click` -- element interaction
|
||||
- `fill` -- form input
|
||||
- `agent-loop` -- snapshot/click/snapshot cycle (typical AI agent pattern)
|
||||
- `full-workflow` -- realistic 7-command sequence
|
||||
|
||||
**System metrics** -- collected while the daemon is running:
|
||||
|
||||
- Cold start time (daemon spawn + browser launch)
|
||||
- Binary size and total distribution size (including browser download)
|
||||
- Daemon RSS and peak RSS (separated from browser process memory)
|
||||
- Browser RSS (Chrome processes, same for both daemons)
|
||||
- Daemon CPU time
|
||||
- Process counts
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Node.js 18+
|
||||
- pnpm
|
||||
- Vercel Sandbox credentials (token, team ID, project ID)
|
||||
|
||||
## Setup
|
||||
|
||||
```bash
|
||||
cd benchmarks
|
||||
pnpm install
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
Fill in your Vercel Sandbox credentials in `.env`:
|
||||
|
||||
```
|
||||
SANDBOX_VERCEL_TOKEN=your_token
|
||||
SANDBOX_VERCEL_TEAM_ID=your_team_id
|
||||
SANDBOX_VERCEL_PROJECT_ID=your_project_id
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
pnpm bench # 10 iterations, 1 warmup, 8 vCPUs
|
||||
pnpm bench -- --iterations 20 # more iterations for tighter stats
|
||||
pnpm bench -- --warmup 2 # extra warmup iterations
|
||||
pnpm bench -- --json # write results.json
|
||||
pnpm bench -- --branch main # build native from a different branch
|
||||
pnpm bench -- --vcpus 16 # more vCPUs (faster Rust build)
|
||||
```
|
||||
|
||||
## How it works
|
||||
|
||||
1. Creates a Vercel Sandbox (Amazon Linux, configurable vCPUs)
|
||||
2. Installs Chromium system dependencies
|
||||
3. **Phase 1 -- Node.js daemon**: installs `agent-browser` from npm (last version with the Node daemon), runs all scenarios, collects metrics
|
||||
4. **Phase 2 -- Rust native daemon**: installs Rust toolchain, clones the repo, runs `cargo build --release`, replaces the binary, runs the same scenarios, collects metrics
|
||||
5. Prints comparison tables and optionally writes `results.json`
|
||||
|
||||
## Interpreting results
|
||||
|
||||
**Command latency** is dominated by Chrome (CDP round-trips), not the daemon. Both daemons are thin relays between the CLI and Chrome, so per-command speedups are typically small. The stddev column helps distinguish real differences from noise.
|
||||
|
||||
**Where the native daemon wins** is in cold start (no Node.js runtime to boot), daemon memory (single Rust binary vs V8 heap), and distribution size (no Playwright dependency).
|
||||
|
||||
The **daemon RSS** metric isolates the daemon process memory from Chrome. This is the apples-to-apples comparison -- both daemons talk to the same Chrome, but Node.js adds ~140 MB of V8 overhead while the Rust daemon uses ~7 MB.
|
||||
|
||||
**Distribution size** includes the daemon plus its browser download. The Node version includes the npm package + Playwright's bundled Chromium. The Rust version is just the binary + Chrome for Testing.
|
||||
@@ -0,0 +1,900 @@
|
||||
/**
|
||||
* Node.js Daemon vs Rust Native Daemon benchmark.
|
||||
*
|
||||
* Compares the last published npm version (Node.js daemon) against the
|
||||
* Rust-only build from a given branch, running real agent-browser commands
|
||||
* inside a Vercel Sandbox.
|
||||
*
|
||||
* Captures:
|
||||
* - Command latency (per-scenario, with warmup + measured iterations + stddev)
|
||||
* - Cold start time (first launch to daemon ready)
|
||||
* - Daemon memory (RSS, peak RSS) separated from browser memory
|
||||
* - Daemon CPU time
|
||||
* - Process tree (daemon + browser children)
|
||||
* - Binary and distribution size on disk
|
||||
*
|
||||
* Usage:
|
||||
* pnpm bench # default: 10 iterations, 1 warmup
|
||||
* pnpm bench -- --iterations 20 # override iterations
|
||||
* pnpm bench -- --warmup 2 # override warmup count
|
||||
* pnpm bench -- --json # write results.json
|
||||
* pnpm bench -- --branch my-branch # override native branch (default: ctate/native-2)
|
||||
* pnpm bench -- --vcpus 8 # sandbox vCPUs (default: 8, higher = faster Rust build)
|
||||
*/
|
||||
|
||||
import { Sandbox } from "@vercel/sandbox";
|
||||
import { readFileSync, writeFileSync } from "fs";
|
||||
import { scenarios, type Scenario } from "./scenarios.js";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Env
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function loadEnv() {
|
||||
try {
|
||||
const content = readFileSync(".env", "utf-8");
|
||||
for (const line of content.split("\n")) {
|
||||
const trimmed = line.trim();
|
||||
if (!trimmed || trimmed.startsWith("#")) continue;
|
||||
const eq = trimmed.indexOf("=");
|
||||
if (eq === -1) continue;
|
||||
const key = trimmed.slice(0, eq);
|
||||
let val = trimmed.slice(eq + 1);
|
||||
if (
|
||||
(val.startsWith('"') && val.endsWith('"')) ||
|
||||
(val.startsWith("'") && val.endsWith("'"))
|
||||
) {
|
||||
val = val.slice(1, -1);
|
||||
}
|
||||
process.env[key] = val;
|
||||
}
|
||||
} catch {}
|
||||
}
|
||||
loadEnv();
|
||||
|
||||
const credentials = {
|
||||
token: process.env.SANDBOX_VERCEL_TOKEN!,
|
||||
teamId: process.env.SANDBOX_VERCEL_TEAM_ID!,
|
||||
projectId: process.env.SANDBOX_VERCEL_PROJECT_ID!,
|
||||
};
|
||||
|
||||
if (!credentials.token || !credentials.teamId || !credentials.projectId) {
|
||||
console.error(
|
||||
"Missing credentials. Set SANDBOX_VERCEL_TOKEN, SANDBOX_VERCEL_TEAM_ID, SANDBOX_VERCEL_PROJECT_ID in .env",
|
||||
);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// CLI args
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function parseArgs() {
|
||||
const args = process.argv.slice(2);
|
||||
let iterations = 10;
|
||||
let warmup = 1;
|
||||
let json = false;
|
||||
let branch = "ctate/native-2";
|
||||
let vcpus = 8;
|
||||
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
if (args[i] === "--iterations" && args[i + 1]) {
|
||||
iterations = parseInt(args[++i], 10);
|
||||
} else if (args[i] === "--warmup" && args[i + 1]) {
|
||||
warmup = parseInt(args[++i], 10);
|
||||
} else if (args[i] === "--json") {
|
||||
json = true;
|
||||
} else if (args[i] === "--branch" && args[i + 1]) {
|
||||
branch = args[++i];
|
||||
} else if (args[i] === "--vcpus" && args[i + 1]) {
|
||||
vcpus = parseInt(args[++i], 10);
|
||||
}
|
||||
}
|
||||
|
||||
return { iterations, warmup, json, branch, vcpus };
|
||||
}
|
||||
|
||||
const config = parseArgs();
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Constants
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const TIMEOUT_MS = 30 * 60 * 1000;
|
||||
const REPO_URL = "https://github.com/vercel-labs/agent-browser.git";
|
||||
|
||||
const CHROMIUM_SYSTEM_DEPS = [
|
||||
"nss",
|
||||
"nspr",
|
||||
"libxkbcommon",
|
||||
"atk",
|
||||
"at-spi2-atk",
|
||||
"at-spi2-core",
|
||||
"libXcomposite",
|
||||
"libXdamage",
|
||||
"libXrandr",
|
||||
"libXfixes",
|
||||
"libXcursor",
|
||||
"libXi",
|
||||
"libXtst",
|
||||
"libXScrnSaver",
|
||||
"libXext",
|
||||
"mesa-libgbm",
|
||||
"libdrm",
|
||||
"mesa-libGL",
|
||||
"mesa-libEGL",
|
||||
"cups-libs",
|
||||
"alsa-lib",
|
||||
"pango",
|
||||
"cairo",
|
||||
"gtk3",
|
||||
"dbus-libs",
|
||||
];
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Sandbox helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
type SandboxInstance = InstanceType<typeof Sandbox>;
|
||||
|
||||
async function run(
|
||||
sandbox: SandboxInstance,
|
||||
cmd: string,
|
||||
args: string[],
|
||||
): Promise<string> {
|
||||
const result = await sandbox.runCommand(cmd, args);
|
||||
const stdout = await result.stdout();
|
||||
const stderr = await result.stderr();
|
||||
if (result.exitCode !== 0) {
|
||||
throw new Error(
|
||||
`Command failed (exit ${result.exitCode}): ${cmd} ${args.join(" ")}\n${stderr || stdout}`,
|
||||
);
|
||||
}
|
||||
return stdout;
|
||||
}
|
||||
|
||||
async function shell(sandbox: SandboxInstance, script: string): Promise<string> {
|
||||
return run(sandbox, "sh", ["-c", script]);
|
||||
}
|
||||
|
||||
async function shellSafe(sandbox: SandboxInstance, script: string): Promise<string> {
|
||||
const result = await sandbox.runCommand("sh", ["-c", script]);
|
||||
return (await result.stdout()).trim();
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Stats
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface Stats {
|
||||
avgMs: number;
|
||||
stddevMs: number;
|
||||
minMs: number;
|
||||
maxMs: number;
|
||||
p50Ms: number;
|
||||
samples: number[];
|
||||
}
|
||||
|
||||
function computeStats(samples: number[]): Stats {
|
||||
const sorted = [...samples].sort((a, b) => a - b);
|
||||
const sum = sorted.reduce((a, b) => a + b, 0);
|
||||
const avg = sum / sorted.length;
|
||||
const variance =
|
||||
sorted.reduce((acc, v) => acc + (v - avg) ** 2, 0) / sorted.length;
|
||||
return {
|
||||
avgMs: Math.round(avg),
|
||||
stddevMs: Math.round(Math.sqrt(variance)),
|
||||
minMs: sorted[0],
|
||||
maxMs: sorted[sorted.length - 1],
|
||||
p50Ms: sorted[Math.floor(sorted.length / 2)],
|
||||
samples: sorted,
|
||||
};
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Metrics collection
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface ProcessMetrics {
|
||||
pid: number;
|
||||
rssKb: number;
|
||||
vszKb: number;
|
||||
cpuPercent: number;
|
||||
memPercent: number;
|
||||
cpuTimeSec: number;
|
||||
command: string;
|
||||
}
|
||||
|
||||
interface DaemonMetrics {
|
||||
coldStartMs: number;
|
||||
binarySizeBytes: number;
|
||||
distributionSizeBytes: number;
|
||||
daemonProcesses: ProcessMetrics[];
|
||||
browserProcesses: ProcessMetrics[];
|
||||
daemonRssKb: number;
|
||||
browserRssKb: number;
|
||||
daemonPeakRssKb: number;
|
||||
daemonCpuTimeSec: number;
|
||||
totalCpuTimeSec: number;
|
||||
}
|
||||
|
||||
async function findDaemonPids(
|
||||
sandbox: SandboxInstance,
|
||||
_session: string,
|
||||
): Promise<number[]> {
|
||||
// The daemon process name is "agent-browser" but session/daemon flags are
|
||||
// env vars, not command-line args, so we can't grep them from `ps`.
|
||||
// Instead, find all agent-browser processes that look like long-running daemons
|
||||
// (not short-lived CLI invocations -- those exit immediately).
|
||||
const raw = await shellSafe(
|
||||
sandbox,
|
||||
`pgrep -x agent-browser 2>/dev/null || true`,
|
||||
);
|
||||
if (!raw) {
|
||||
// Fallback: broader match on process name
|
||||
const fallback = await shellSafe(
|
||||
sandbox,
|
||||
`pgrep -f 'agent-browser' 2>/dev/null | head -5 || true`,
|
||||
);
|
||||
if (!fallback) return [];
|
||||
return fallback.split("\n").map(Number).filter(Boolean);
|
||||
}
|
||||
return raw.split("\n").map(Number).filter(Boolean);
|
||||
}
|
||||
|
||||
async function collectProcessMetrics(
|
||||
sandbox: SandboxInstance,
|
||||
pid: number,
|
||||
): Promise<ProcessMetrics | null> {
|
||||
const raw = await shellSafe(
|
||||
sandbox,
|
||||
`ps -p ${pid} -o pid=,rss=,vsz=,%cpu=,%mem=,cputime=,comm= 2>/dev/null || true`,
|
||||
);
|
||||
if (!raw) return null;
|
||||
|
||||
const parts = raw.trim().split(/\s+/);
|
||||
if (parts.length < 7) return null;
|
||||
|
||||
// Parse cputime "HH:MM:SS" or "MM:SS" to seconds
|
||||
const timeParts = parts[5].split(":").map(Number);
|
||||
let cpuTimeSec = 0;
|
||||
if (timeParts.length === 3) {
|
||||
cpuTimeSec = timeParts[0] * 3600 + timeParts[1] * 60 + timeParts[2];
|
||||
} else if (timeParts.length === 2) {
|
||||
cpuTimeSec = timeParts[0] * 60 + timeParts[1];
|
||||
}
|
||||
|
||||
return {
|
||||
pid: Number(parts[0]),
|
||||
rssKb: Number(parts[1]),
|
||||
vszKb: Number(parts[2]),
|
||||
cpuPercent: Number(parts[3]),
|
||||
memPercent: Number(parts[4]),
|
||||
cpuTimeSec,
|
||||
command: parts.slice(6).join(" "),
|
||||
};
|
||||
}
|
||||
|
||||
async function getPeakRssKb(
|
||||
sandbox: SandboxInstance,
|
||||
pid: number,
|
||||
): Promise<number> {
|
||||
const raw = await shellSafe(
|
||||
sandbox,
|
||||
`cat /proc/${pid}/status 2>/dev/null | grep VmHWM | awk '{print $2}' || echo 0`,
|
||||
);
|
||||
return Number(raw) || 0;
|
||||
}
|
||||
|
||||
async function getChildPids(
|
||||
sandbox: SandboxInstance,
|
||||
pid: number,
|
||||
): Promise<number[]> {
|
||||
const raw = await shellSafe(
|
||||
sandbox,
|
||||
`pgrep -P ${pid} 2>/dev/null || true`,
|
||||
);
|
||||
if (!raw) return [];
|
||||
return raw.split("\n").map(Number).filter(Boolean);
|
||||
}
|
||||
|
||||
async function getAllDescendantPids(
|
||||
sandbox: SandboxInstance,
|
||||
pid: number,
|
||||
): Promise<number[]> {
|
||||
const all: number[] = [];
|
||||
const queue = [pid];
|
||||
while (queue.length > 0) {
|
||||
const current = queue.shift()!;
|
||||
all.push(current);
|
||||
const children = await getChildPids(sandbox, current);
|
||||
queue.push(...children);
|
||||
}
|
||||
return all;
|
||||
}
|
||||
|
||||
async function collectDaemonMetrics(
|
||||
sandbox: SandboxInstance,
|
||||
session: string,
|
||||
coldStartMs: number,
|
||||
binarySizeBytes: number,
|
||||
distributionSizeBytes: number,
|
||||
): Promise<DaemonMetrics> {
|
||||
// Find daemon PIDs -- the agent-browser process itself
|
||||
const daemonPids = await findDaemonPids(sandbox, session);
|
||||
|
||||
// Also find the full process tree (daemon + Chrome children)
|
||||
let allPids: number[] = [];
|
||||
for (const pid of daemonPids) {
|
||||
const descendants = await getAllDescendantPids(sandbox, pid);
|
||||
allPids.push(...descendants);
|
||||
}
|
||||
allPids = [...new Set(allPids)];
|
||||
|
||||
// If no daemon PIDs found via pgrep, fall back to grabbing all
|
||||
// agent-browser and chrome processes for metrics
|
||||
if (allPids.length === 0) {
|
||||
const fallback = await shellSafe(
|
||||
sandbox,
|
||||
`ps -eo pid,comm | grep -E 'agent-browser|chrome' | grep -v grep | awk '{print $1}' || true`,
|
||||
);
|
||||
if (fallback) {
|
||||
allPids = fallback.split("\n").map(Number).filter(Boolean);
|
||||
}
|
||||
}
|
||||
|
||||
const daemonProcs: ProcessMetrics[] = [];
|
||||
const browserProcs: ProcessMetrics[] = [];
|
||||
let daemonPeakRssKb = 0;
|
||||
|
||||
for (const pid of allPids) {
|
||||
const metrics = await collectProcessMetrics(sandbox, pid);
|
||||
if (!metrics) continue;
|
||||
|
||||
const isBrowser = /chrome|chromium/i.test(metrics.command);
|
||||
if (isBrowser) {
|
||||
browserProcs.push(metrics);
|
||||
} else {
|
||||
daemonProcs.push(metrics);
|
||||
const peak = await getPeakRssKb(sandbox, pid);
|
||||
daemonPeakRssKb = Math.max(daemonPeakRssKb, peak);
|
||||
}
|
||||
}
|
||||
|
||||
const daemonRssKb = daemonProcs.reduce((sum, p) => sum + p.rssKb, 0);
|
||||
const browserRssKb = browserProcs.reduce((sum, p) => sum + p.rssKb, 0);
|
||||
const daemonCpuTimeSec = daemonProcs.reduce((sum, p) => sum + p.cpuTimeSec, 0);
|
||||
const allProcs = [...daemonProcs, ...browserProcs];
|
||||
const totalCpuTimeSec = allProcs.reduce((sum, p) => sum + p.cpuTimeSec, 0);
|
||||
|
||||
return {
|
||||
coldStartMs,
|
||||
binarySizeBytes,
|
||||
distributionSizeBytes,
|
||||
daemonProcesses: daemonProcs,
|
||||
browserProcesses: browserProcs,
|
||||
daemonRssKb,
|
||||
browserRssKb,
|
||||
daemonPeakRssKb,
|
||||
daemonCpuTimeSec,
|
||||
totalCpuTimeSec,
|
||||
};
|
||||
}
|
||||
|
||||
async function getBinarySize(
|
||||
sandbox: SandboxInstance,
|
||||
): Promise<number> {
|
||||
// Follow symlinks to get the real binary/script size
|
||||
const raw = await shellSafe(
|
||||
sandbox,
|
||||
`stat -L -c %s "$(readlink -f "$(which agent-browser)")" 2>/dev/null || echo 0`,
|
||||
);
|
||||
return Number(raw) || 0;
|
||||
}
|
||||
|
||||
async function getDistributionSize(
|
||||
sandbox: SandboxInstance,
|
||||
mode: DaemonMode,
|
||||
): Promise<number> {
|
||||
if (mode === "node") {
|
||||
// Total size of the npm package + Playwright browser
|
||||
const npmPkg = await shellSafe(
|
||||
sandbox,
|
||||
`du -sb "$(npm root -g)/agent-browser" 2>/dev/null | awk '{print $1}' || echo 0`,
|
||||
);
|
||||
const pwBrowser = await shellSafe(
|
||||
sandbox,
|
||||
`du -sb "$HOME/.cache/ms-playwright" 2>/dev/null | awk '{print $1}' || echo 0`,
|
||||
);
|
||||
return (Number(npmPkg) || 0) + (Number(pwBrowser) || 0);
|
||||
} else {
|
||||
// Rust binary + Chrome for Testing (checks multiple possible cache paths)
|
||||
const binary = await shellSafe(
|
||||
sandbox,
|
||||
`stat -L -c %s "$(readlink -f "$(which agent-browser)")" 2>/dev/null || echo 0`,
|
||||
);
|
||||
const chrome = await shellSafe(
|
||||
sandbox,
|
||||
[
|
||||
`size=0`,
|
||||
`for d in "$HOME/.cache/agent-browser" "$HOME/.cache/ms-playwright" "$HOME/.agent-browser/chrome"; do`,
|
||||
` if [ -d "$d" ]; then size=$(du -sb "$d" 2>/dev/null | awk '{print $1}'); break; fi`,
|
||||
`done`,
|
||||
`echo $size`,
|
||||
].join("; "),
|
||||
);
|
||||
return (Number(binary) || 0) + (Number(chrome) || 0);
|
||||
}
|
||||
}
|
||||
|
||||
function formatBytes(bytes: number): string {
|
||||
if (bytes >= 1024 * 1024) return `${(bytes / 1024 / 1024).toFixed(1)} MB`;
|
||||
if (bytes >= 1024) return `${(bytes / 1024).toFixed(1)} KB`;
|
||||
return `${bytes} B`;
|
||||
}
|
||||
|
||||
function formatKb(kb: number): string {
|
||||
if (kb >= 1024) return `${(kb / 1024).toFixed(1)} MB`;
|
||||
return `${kb} KB`;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Scenario runner
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
type DaemonMode = "node" | "native";
|
||||
|
||||
function daemonEnv(mode: DaemonMode): Record<string, string> {
|
||||
return { AGENT_BROWSER_SESSION: `bench-${mode}` };
|
||||
}
|
||||
|
||||
async function agentBrowser(
|
||||
sandbox: SandboxInstance,
|
||||
args: string[],
|
||||
mode: DaemonMode,
|
||||
): Promise<void> {
|
||||
const result = await sandbox.runCommand({
|
||||
cmd: "agent-browser",
|
||||
args,
|
||||
env: daemonEnv(mode),
|
||||
});
|
||||
if (result.exitCode !== 0) {
|
||||
const stderr = await result.stderr();
|
||||
const stdout = await result.stdout();
|
||||
throw new Error(
|
||||
`agent-browser ${args.join(" ")} failed (exit ${result.exitCode}): ${stderr || stdout}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
async function timedAgentBrowser(
|
||||
sandbox: SandboxInstance,
|
||||
args: string[],
|
||||
mode: DaemonMode,
|
||||
): Promise<number> {
|
||||
const start = Date.now();
|
||||
const result = await sandbox.runCommand({
|
||||
cmd: "agent-browser",
|
||||
args,
|
||||
env: daemonEnv(mode),
|
||||
});
|
||||
const elapsed = Date.now() - start;
|
||||
if (result.exitCode !== 0) {
|
||||
const stderr = await result.stderr();
|
||||
const stdout = await result.stdout();
|
||||
throw new Error(
|
||||
`agent-browser ${args.join(" ")} failed (exit ${result.exitCode}): ${stderr || stdout}`,
|
||||
);
|
||||
}
|
||||
return elapsed;
|
||||
}
|
||||
|
||||
interface ScenarioResult {
|
||||
name: string;
|
||||
description: string;
|
||||
stats: Stats;
|
||||
error?: string;
|
||||
}
|
||||
|
||||
async function runScenario(
|
||||
sandbox: SandboxInstance,
|
||||
scenario: Scenario,
|
||||
mode: DaemonMode,
|
||||
iterations: number,
|
||||
warmup: number,
|
||||
): Promise<ScenarioResult> {
|
||||
try {
|
||||
if (scenario.setup) {
|
||||
for (const cmd of scenario.setup) {
|
||||
await agentBrowser(sandbox, cmd, mode);
|
||||
}
|
||||
}
|
||||
|
||||
for (let w = 0; w < warmup; w++) {
|
||||
for (const cmd of scenario.commands) {
|
||||
await agentBrowser(sandbox, cmd, mode);
|
||||
}
|
||||
}
|
||||
|
||||
const samples: number[] = [];
|
||||
for (let i = 0; i < iterations; i++) {
|
||||
let totalMs = 0;
|
||||
for (const cmd of scenario.commands) {
|
||||
totalMs += await timedAgentBrowser(sandbox, cmd, mode);
|
||||
}
|
||||
samples.push(totalMs);
|
||||
}
|
||||
|
||||
if (scenario.teardown) {
|
||||
for (const cmd of scenario.teardown) {
|
||||
await agentBrowser(sandbox, cmd, mode);
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
name: scenario.name,
|
||||
description: scenario.description,
|
||||
stats: computeStats(samples),
|
||||
};
|
||||
} catch (err: unknown) {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
return {
|
||||
name: scenario.name,
|
||||
description: scenario.description,
|
||||
stats: { avgMs: -1, stddevMs: -1, minMs: -1, maxMs: -1, p50Ms: -1, samples: [] },
|
||||
error: message,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Benchmark phases
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface DaemonResults {
|
||||
mode: DaemonMode;
|
||||
label: string;
|
||||
scenarios: ScenarioResult[];
|
||||
metrics: DaemonMetrics;
|
||||
}
|
||||
|
||||
async function benchmarkDaemon(
|
||||
sandbox: SandboxInstance,
|
||||
mode: DaemonMode,
|
||||
label: string,
|
||||
): Promise<DaemonResults> {
|
||||
console.log(`\n--- ${label} ---`);
|
||||
|
||||
// Measure sizes before launch
|
||||
const binarySizeBytes = await getBinarySize(sandbox);
|
||||
const distributionSizeBytes = await getDistributionSize(sandbox, mode);
|
||||
|
||||
// Cold start: time the first launch (daemon spawn + browser launch)
|
||||
const coldStartBegin = Date.now();
|
||||
await agentBrowser(sandbox, ["open", "about:blank"], mode);
|
||||
const coldStartMs = Date.now() - coldStartBegin;
|
||||
console.log(` Cold start: ${coldStartMs}ms`);
|
||||
console.log(` Binary size: ${formatBytes(binarySizeBytes)}`);
|
||||
console.log(` Distribution size: ${formatBytes(distributionSizeBytes)}`);
|
||||
|
||||
// Run all scenarios
|
||||
const results: ScenarioResult[] = [];
|
||||
for (const scenario of scenarios) {
|
||||
process.stdout.write(` ${scenario.name} `);
|
||||
const result = await runScenario(
|
||||
sandbox,
|
||||
scenario,
|
||||
mode,
|
||||
config.iterations,
|
||||
config.warmup,
|
||||
);
|
||||
if (result.error) {
|
||||
console.log(`FAILED: ${result.error.slice(0, 120)}`);
|
||||
} else {
|
||||
const dots = ".".repeat(Math.max(1, 30 - scenario.name.length));
|
||||
const s = result.stats;
|
||||
console.log(
|
||||
`${dots} ${s.avgMs}ms avg +/-${s.stddevMs}ms (p50: ${s.p50Ms}ms, min: ${s.minMs}ms, max: ${s.maxMs}ms)`,
|
||||
);
|
||||
}
|
||||
results.push(result);
|
||||
}
|
||||
|
||||
// Collect system metrics after scenarios (daemon is still running)
|
||||
const session = `bench-${mode}`;
|
||||
const metrics = await collectDaemonMetrics(
|
||||
sandbox,
|
||||
session,
|
||||
coldStartMs,
|
||||
binarySizeBytes,
|
||||
distributionSizeBytes,
|
||||
);
|
||||
|
||||
// Also grab a full process snapshot for context
|
||||
const psOutput = await shellSafe(
|
||||
sandbox,
|
||||
`ps aux --sort=-rss | head -20`,
|
||||
);
|
||||
console.log(`\n Process snapshot (top by RSS):`);
|
||||
for (const line of psOutput.split("\n").slice(0, 10)) {
|
||||
console.log(` ${line}`);
|
||||
}
|
||||
|
||||
console.log(`\n Daemon processes (${metrics.daemonProcesses.length}):`);
|
||||
console.log(` RSS: ${formatKb(metrics.daemonRssKb)} (peak: ${formatKb(metrics.daemonPeakRssKb)})`);
|
||||
console.log(` CPU time: ${metrics.daemonCpuTimeSec.toFixed(1)}s`);
|
||||
for (const p of metrics.daemonProcesses) {
|
||||
console.log(` PID ${p.pid}: ${p.command} (RSS: ${formatKb(p.rssKb)}, CPU: ${p.cpuPercent}%)`);
|
||||
}
|
||||
console.log(` Browser processes (${metrics.browserProcesses.length}):`);
|
||||
console.log(` RSS: ${formatKb(metrics.browserRssKb)}`);
|
||||
for (const p of metrics.browserProcesses) {
|
||||
console.log(` PID ${p.pid}: ${p.command} (RSS: ${formatKb(p.rssKb)}, CPU: ${p.cpuPercent}%)`);
|
||||
}
|
||||
|
||||
await agentBrowser(sandbox, ["close"], mode);
|
||||
console.log(` Browser closed.`);
|
||||
|
||||
return { mode, label, scenarios: results, metrics };
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Install helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function installChromiumDeps(sandbox: SandboxInstance) {
|
||||
console.log("Installing Chromium system dependencies...");
|
||||
await shell(
|
||||
sandbox,
|
||||
`sudo dnf clean all 2>&1 && sudo dnf install -y --skip-broken ${CHROMIUM_SYSTEM_DEPS.join(" ")} 2>&1 && sudo ldconfig 2>&1`,
|
||||
);
|
||||
}
|
||||
|
||||
async function installNodeDaemon(sandbox: SandboxInstance) {
|
||||
console.log("Installing agent-browser from npm (Node.js daemon)...");
|
||||
await run(sandbox, "npm", ["install", "-g", "agent-browser"]);
|
||||
await run(sandbox, "npx", ["agent-browser", "install"]);
|
||||
const version = await shell(sandbox, "agent-browser --version 2>&1 || true");
|
||||
console.log(` version: ${version.trim()}`);
|
||||
}
|
||||
|
||||
async function installNativeDaemon(sandbox: SandboxInstance, branch: string) {
|
||||
console.log(`\nBuilding native daemon from ${branch}...`);
|
||||
|
||||
console.log(" Installing build tools and Rust toolchain...");
|
||||
const rustStart = Date.now();
|
||||
await shell(
|
||||
sandbox,
|
||||
"sudo dnf install -y gcc gcc-c++ make perl-core openssl-devel 2>&1",
|
||||
);
|
||||
await shell(
|
||||
sandbox,
|
||||
"curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y 2>&1",
|
||||
);
|
||||
console.log(` Rust + build tools installed (${Math.round((Date.now() - rustStart) / 1000)}s)`);
|
||||
|
||||
console.log(` Cloning repo (branch: ${branch})...`);
|
||||
const cloneStart = Date.now();
|
||||
await shell(
|
||||
sandbox,
|
||||
`git clone --depth 1 --branch ${branch} ${REPO_URL} /tmp/agent-browser 2>&1`,
|
||||
);
|
||||
console.log(` Cloned (${Math.round((Date.now() - cloneStart) / 1000)}s)`);
|
||||
|
||||
console.log(" Building release binary (cargo build --release)...");
|
||||
const buildStart = Date.now();
|
||||
await shell(
|
||||
sandbox,
|
||||
"source $HOME/.cargo/env && cd /tmp/agent-browser/cli && cargo build --release 2>&1",
|
||||
);
|
||||
console.log(` Built (${Math.round((Date.now() - buildStart) / 1000)}s)`);
|
||||
|
||||
const npmBinPath = (await shell(sandbox, "which agent-browser")).trim();
|
||||
console.log(` Replacing ${npmBinPath} with native build...`);
|
||||
await shell(
|
||||
sandbox,
|
||||
`sudo cp /tmp/agent-browser/cli/target/release/agent-browser ${npmBinPath}`,
|
||||
);
|
||||
|
||||
const version = await shell(sandbox, "agent-browser --version 2>&1 || true");
|
||||
console.log(` version: ${version.trim()}`);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Output
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function printResults(node: DaemonResults, native: DaemonResults) {
|
||||
console.log("\n\n========== COMMAND LATENCY ==========\n");
|
||||
|
||||
const header =
|
||||
"Scenario".padEnd(20) + "| Node avg +/-sd | Rust avg +/-sd | Speedup";
|
||||
const sep = "-".repeat(20) + "|-----------------|-----------------|--------";
|
||||
console.log(header);
|
||||
console.log(sep);
|
||||
|
||||
for (let i = 0; i < node.scenarios.length; i++) {
|
||||
const n = node.scenarios[i];
|
||||
const r = native.scenarios[i];
|
||||
const name = n.name.padEnd(20);
|
||||
|
||||
if (n.error || r.error) {
|
||||
const nodeVal = n.error ? "FAILED".padEnd(15) : `${n.stats.avgMs}ms`.padEnd(15);
|
||||
const rustVal = r.error ? "FAILED".padEnd(15) : `${r.stats.avgMs}ms`.padEnd(15);
|
||||
console.log(`${name}| ${nodeVal} | ${rustVal} | --`);
|
||||
continue;
|
||||
}
|
||||
|
||||
const nodeVal = `${n.stats.avgMs} +/-${n.stats.stddevMs}ms`.padEnd(15);
|
||||
const rustVal = `${r.stats.avgMs} +/-${r.stats.stddevMs}ms`.padEnd(15);
|
||||
const speedup =
|
||||
r.stats.avgMs > 0
|
||||
? (n.stats.avgMs / r.stats.avgMs).toFixed(2) + "x"
|
||||
: "--";
|
||||
console.log(`${name}| ${nodeVal} | ${rustVal} | ${speedup.padStart(6)}`);
|
||||
}
|
||||
|
||||
console.log("\n\n========== SYSTEM METRICS ==========\n");
|
||||
|
||||
const nm = node.metrics;
|
||||
const rm = native.metrics;
|
||||
|
||||
function ratio(a: number, b: number): string {
|
||||
if (b <= 0) return "--";
|
||||
return (a / b).toFixed(2) + "x";
|
||||
}
|
||||
|
||||
const metricRows: [string, string, string, string][] = [
|
||||
[
|
||||
"Cold start",
|
||||
`${nm.coldStartMs}ms`,
|
||||
`${rm.coldStartMs}ms`,
|
||||
ratio(nm.coldStartMs, rm.coldStartMs),
|
||||
],
|
||||
[
|
||||
"Binary size",
|
||||
formatBytes(nm.binarySizeBytes),
|
||||
formatBytes(rm.binarySizeBytes),
|
||||
ratio(nm.binarySizeBytes, rm.binarySizeBytes),
|
||||
],
|
||||
[
|
||||
"Distribution size",
|
||||
formatBytes(nm.distributionSizeBytes),
|
||||
formatBytes(rm.distributionSizeBytes),
|
||||
ratio(nm.distributionSizeBytes, rm.distributionSizeBytes),
|
||||
],
|
||||
[
|
||||
"Daemon RSS",
|
||||
formatKb(nm.daemonRssKb),
|
||||
formatKb(rm.daemonRssKb),
|
||||
ratio(nm.daemonRssKb, rm.daemonRssKb),
|
||||
],
|
||||
[
|
||||
"Daemon peak RSS",
|
||||
formatKb(nm.daemonPeakRssKb),
|
||||
formatKb(rm.daemonPeakRssKb),
|
||||
ratio(nm.daemonPeakRssKb, rm.daemonPeakRssKb),
|
||||
],
|
||||
[
|
||||
"Browser RSS",
|
||||
formatKb(nm.browserRssKb),
|
||||
formatKb(rm.browserRssKb),
|
||||
ratio(nm.browserRssKb, rm.browserRssKb),
|
||||
],
|
||||
[
|
||||
"Daemon CPU time",
|
||||
`${nm.daemonCpuTimeSec.toFixed(1)}s`,
|
||||
`${rm.daemonCpuTimeSec.toFixed(1)}s`,
|
||||
ratio(nm.daemonCpuTimeSec, rm.daemonCpuTimeSec),
|
||||
],
|
||||
[
|
||||
"Daemon processes",
|
||||
String(nm.daemonProcesses.length),
|
||||
String(rm.daemonProcesses.length),
|
||||
"--",
|
||||
],
|
||||
[
|
||||
"Browser processes",
|
||||
String(nm.browserProcesses.length),
|
||||
String(rm.browserProcesses.length),
|
||||
"--",
|
||||
],
|
||||
];
|
||||
|
||||
const mHeader =
|
||||
"Metric".padEnd(20) + "| Node".padEnd(14) + "| Rust".padEnd(14) + "| Ratio";
|
||||
const mSep = "-".repeat(20) + "|" + "-".repeat(13) + "|" + "-".repeat(13) + "|--------";
|
||||
console.log(mHeader);
|
||||
console.log(mSep);
|
||||
for (const [metric, nodeVal, rustVal, ratio] of metricRows) {
|
||||
console.log(
|
||||
`${metric.padEnd(20)}| ${nodeVal.padEnd(12)}| ${rustVal.padEnd(12)}| ${ratio}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Main
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function main() {
|
||||
console.log("agent-browser Daemon Benchmark (Node.js vs Rust Native)");
|
||||
console.log(`Branch: ${config.branch}`);
|
||||
console.log(`Iterations: ${config.iterations} (+ ${config.warmup} warmup)`);
|
||||
console.log(`vCPUs: ${config.vcpus}\n`);
|
||||
|
||||
console.log("Creating sandbox...");
|
||||
const sandbox = await Sandbox.create({
|
||||
...credentials,
|
||||
timeout: TIMEOUT_MS,
|
||||
runtime: "node22",
|
||||
networkPolicy: "allow-all" as const,
|
||||
resources: { vcpus: config.vcpus },
|
||||
});
|
||||
console.log(`Sandbox: ${sandbox.sandboxId}`);
|
||||
|
||||
try {
|
||||
await installChromiumDeps(sandbox);
|
||||
|
||||
// Phase 1: Node.js daemon (from published npm package)
|
||||
await installNodeDaemon(sandbox);
|
||||
const nodeResults = await benchmarkDaemon(
|
||||
sandbox,
|
||||
"node",
|
||||
"Node.js Daemon (npm)",
|
||||
);
|
||||
|
||||
// Phase 2: Rust native daemon (built from branch)
|
||||
await installNativeDaemon(sandbox, config.branch);
|
||||
const nativeResults = await benchmarkDaemon(
|
||||
sandbox,
|
||||
"native",
|
||||
`Rust Native Daemon (${config.branch})`,
|
||||
);
|
||||
|
||||
printResults(nodeResults, nativeResults);
|
||||
|
||||
if (config.json) {
|
||||
const output = {
|
||||
timestamp: new Date().toISOString(),
|
||||
branch: config.branch,
|
||||
vcpus: config.vcpus,
|
||||
iterations: config.iterations,
|
||||
warmup: config.warmup,
|
||||
node: {
|
||||
scenarios: nodeResults.scenarios.map((s) => ({
|
||||
name: s.name,
|
||||
description: s.description,
|
||||
...s.stats,
|
||||
error: s.error,
|
||||
})),
|
||||
metrics: nodeResults.metrics,
|
||||
},
|
||||
native: {
|
||||
scenarios: nativeResults.scenarios.map((s) => ({
|
||||
name: s.name,
|
||||
description: s.description,
|
||||
...s.stats,
|
||||
error: s.error,
|
||||
})),
|
||||
metrics: nativeResults.metrics,
|
||||
},
|
||||
};
|
||||
writeFileSync("results.json", JSON.stringify(output, null, 2));
|
||||
console.log("\nResults written to results.json");
|
||||
}
|
||||
} catch (err: unknown) {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
console.error(`\nFatal error: ${message}`);
|
||||
process.exit(1);
|
||||
} finally {
|
||||
try {
|
||||
await sandbox.stop();
|
||||
console.log("\nSandbox stopped.");
|
||||
} catch {
|
||||
console.warn("Warning: failed to stop sandbox.");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
main();
|
||||
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"name": "agent-browser-benchmarks",
|
||||
"version": "1.0.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"bench": "tsx bench.ts"
|
||||
},
|
||||
"dependencies": {
|
||||
"@vercel/sandbox": "^1.8.0",
|
||||
"tsx": "^4.19.0"
|
||||
}
|
||||
}
|
||||
Generated
+472
@@ -0,0 +1,472 @@
|
||||
lockfileVersion: '9.0'
|
||||
|
||||
settings:
|
||||
autoInstallPeers: true
|
||||
excludeLinksFromLockfile: false
|
||||
|
||||
importers:
|
||||
|
||||
.:
|
||||
dependencies:
|
||||
'@vercel/sandbox':
|
||||
specifier: ^1.8.0
|
||||
version: 1.8.1
|
||||
tsx:
|
||||
specifier: ^4.19.0
|
||||
version: 4.21.0
|
||||
|
||||
packages:
|
||||
|
||||
'@esbuild/aix-ppc64@0.27.4':
|
||||
resolution: {integrity: sha512-cQPwL2mp2nSmHHJlCyoXgHGhbEPMrEEU5xhkcy3Hs/O7nGZqEpZ2sUtLaL9MORLtDfRvVl2/3PAuEkYZH0Ty8Q==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [ppc64]
|
||||
os: [aix]
|
||||
|
||||
'@esbuild/android-arm64@0.27.4':
|
||||
resolution: {integrity: sha512-gdLscB7v75wRfu7QSm/zg6Rx29VLdy9eTr2t44sfTW7CxwAtQghZ4ZnqHk3/ogz7xao0QAgrkradbBzcqFPasw==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [arm64]
|
||||
os: [android]
|
||||
|
||||
'@esbuild/android-arm@0.27.4':
|
||||
resolution: {integrity: sha512-X9bUgvxiC8CHAGKYufLIHGXPJWnr0OCdR0anD2e21vdvgCI8lIfqFbnoeOz7lBjdrAGUhqLZLcQo6MLhTO2DKQ==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [arm]
|
||||
os: [android]
|
||||
|
||||
'@esbuild/android-x64@0.27.4':
|
||||
resolution: {integrity: sha512-PzPFnBNVF292sfpfhiyiXCGSn9HZg5BcAz+ivBuSsl6Rk4ga1oEXAamhOXRFyMcjwr2DVtm40G65N3GLeH1Lvw==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [x64]
|
||||
os: [android]
|
||||
|
||||
'@esbuild/darwin-arm64@0.27.4':
|
||||
resolution: {integrity: sha512-b7xaGIwdJlht8ZFCvMkpDN6uiSmnxxK56N2GDTMYPr2/gzvfdQN8rTfBsvVKmIVY/X7EM+/hJKEIbbHs9oA4tQ==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [arm64]
|
||||
os: [darwin]
|
||||
|
||||
'@esbuild/darwin-x64@0.27.4':
|
||||
resolution: {integrity: sha512-sR+OiKLwd15nmCdqpXMnuJ9W2kpy0KigzqScqHI3Hqwr7IXxBp3Yva+yJwoqh7rE8V77tdoheRYataNKL4QrPw==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [x64]
|
||||
os: [darwin]
|
||||
|
||||
'@esbuild/freebsd-arm64@0.27.4':
|
||||
resolution: {integrity: sha512-jnfpKe+p79tCnm4GVav68A7tUFeKQwQyLgESwEAUzyxk/TJr4QdGog9sqWNcUbr/bZt/O/HXouspuQDd9JxFSw==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [arm64]
|
||||
os: [freebsd]
|
||||
|
||||
'@esbuild/freebsd-x64@0.27.4':
|
||||
resolution: {integrity: sha512-2kb4ceA/CpfUrIcTUl1wrP/9ad9Atrp5J94Lq69w7UwOMolPIGrfLSvAKJp0RTvkPPyn6CIWrNy13kyLikZRZQ==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [x64]
|
||||
os: [freebsd]
|
||||
|
||||
'@esbuild/linux-arm64@0.27.4':
|
||||
resolution: {integrity: sha512-7nQOttdzVGth1iz57kxg9uCz57dxQLHWxopL6mYuYthohPKEK0vU0C3O21CcBK6KDlkYVcnDXY099HcCDXd9dA==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [arm64]
|
||||
os: [linux]
|
||||
|
||||
'@esbuild/linux-arm@0.27.4':
|
||||
resolution: {integrity: sha512-aBYgcIxX/wd5n2ys0yESGeYMGF+pv6g0DhZr3G1ZG4jMfruU9Tl1i2Z+Wnj9/KjGz1lTLCcorqE2viePZqj4Eg==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [arm]
|
||||
os: [linux]
|
||||
|
||||
'@esbuild/linux-ia32@0.27.4':
|
||||
resolution: {integrity: sha512-oPtixtAIzgvzYcKBQM/qZ3R+9TEUd1aNJQu0HhGyqtx6oS7qTpvjheIWBbes4+qu1bNlo2V4cbkISr8q6gRBFA==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [ia32]
|
||||
os: [linux]
|
||||
|
||||
'@esbuild/linux-loong64@0.27.4':
|
||||
resolution: {integrity: sha512-8mL/vh8qeCoRcFH2nM8wm5uJP+ZcVYGGayMavi8GmRJjuI3g1v6Z7Ni0JJKAJW+m0EtUuARb6Lmp4hMjzCBWzA==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [loong64]
|
||||
os: [linux]
|
||||
|
||||
'@esbuild/linux-mips64el@0.27.4':
|
||||
resolution: {integrity: sha512-1RdrWFFiiLIW7LQq9Q2NES+HiD4NyT8Itj9AUeCl0IVCA459WnPhREKgwrpaIfTOe+/2rdntisegiPWn/r/aAw==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [mips64el]
|
||||
os: [linux]
|
||||
|
||||
'@esbuild/linux-ppc64@0.27.4':
|
||||
resolution: {integrity: sha512-tLCwNG47l3sd9lpfyx9LAGEGItCUeRCWeAx6x2Jmbav65nAwoPXfewtAdtbtit/pJFLUWOhpv0FpS6GQAmPrHA==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [ppc64]
|
||||
os: [linux]
|
||||
|
||||
'@esbuild/linux-riscv64@0.27.4':
|
||||
resolution: {integrity: sha512-BnASypppbUWyqjd1KIpU4AUBiIhVr6YlHx/cnPgqEkNoVOhHg+YiSVxM1RLfiy4t9cAulbRGTNCKOcqHrEQLIw==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [riscv64]
|
||||
os: [linux]
|
||||
|
||||
'@esbuild/linux-s390x@0.27.4':
|
||||
resolution: {integrity: sha512-+eUqgb/Z7vxVLezG8bVB9SfBie89gMueS+I0xYh2tJdw3vqA/0ImZJ2ROeWwVJN59ihBeZ7Tu92dF/5dy5FttA==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [s390x]
|
||||
os: [linux]
|
||||
|
||||
'@esbuild/linux-x64@0.27.4':
|
||||
resolution: {integrity: sha512-S5qOXrKV8BQEzJPVxAwnryi2+Iq5pB40gTEIT69BQONqR7JH1EPIcQ/Uiv9mCnn05jff9umq/5nqzxlqTOg9NA==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [x64]
|
||||
os: [linux]
|
||||
|
||||
'@esbuild/netbsd-arm64@0.27.4':
|
||||
resolution: {integrity: sha512-xHT8X4sb0GS8qTqiwzHqpY00C95DPAq7nAwX35Ie/s+LO9830hrMd3oX0ZMKLvy7vsonee73x0lmcdOVXFzd6Q==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [arm64]
|
||||
os: [netbsd]
|
||||
|
||||
'@esbuild/netbsd-x64@0.27.4':
|
||||
resolution: {integrity: sha512-RugOvOdXfdyi5Tyv40kgQnI0byv66BFgAqjdgtAKqHoZTbTF2QqfQrFwa7cHEORJf6X2ht+l9ABLMP0dnKYsgg==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [x64]
|
||||
os: [netbsd]
|
||||
|
||||
'@esbuild/openbsd-arm64@0.27.4':
|
||||
resolution: {integrity: sha512-2MyL3IAaTX+1/qP0O1SwskwcwCoOI4kV2IBX1xYnDDqthmq5ArrW94qSIKCAuRraMgPOmG0RDTA74mzYNQA9ow==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [arm64]
|
||||
os: [openbsd]
|
||||
|
||||
'@esbuild/openbsd-x64@0.27.4':
|
||||
resolution: {integrity: sha512-u8fg/jQ5aQDfsnIV6+KwLOf1CmJnfu1ShpwqdwC0uA7ZPwFws55Ngc12vBdeUdnuWoQYx/SOQLGDcdlfXhYmXQ==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [x64]
|
||||
os: [openbsd]
|
||||
|
||||
'@esbuild/openharmony-arm64@0.27.4':
|
||||
resolution: {integrity: sha512-JkTZrl6VbyO8lDQO3yv26nNr2RM2yZzNrNHEsj9bm6dOwwu9OYN28CjzZkH57bh4w0I2F7IodpQvUAEd1mbWXg==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [arm64]
|
||||
os: [openharmony]
|
||||
|
||||
'@esbuild/sunos-x64@0.27.4':
|
||||
resolution: {integrity: sha512-/gOzgaewZJfeJTlsWhvUEmUG4tWEY2Spp5M20INYRg2ZKl9QPO3QEEgPeRtLjEWSW8FilRNacPOg8R1uaYkA6g==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [x64]
|
||||
os: [sunos]
|
||||
|
||||
'@esbuild/win32-arm64@0.27.4':
|
||||
resolution: {integrity: sha512-Z9SExBg2y32smoDQdf1HRwHRt6vAHLXcxD2uGgO/v2jK7Y718Ix4ndsbNMU/+1Qiem9OiOdaqitioZwxivhXYg==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [arm64]
|
||||
os: [win32]
|
||||
|
||||
'@esbuild/win32-ia32@0.27.4':
|
||||
resolution: {integrity: sha512-DAyGLS0Jz5G5iixEbMHi5KdiApqHBWMGzTtMiJ72ZOLhbu/bzxgAe8Ue8CTS3n3HbIUHQz/L51yMdGMeoxXNJw==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [ia32]
|
||||
os: [win32]
|
||||
|
||||
'@esbuild/win32-x64@0.27.4':
|
||||
resolution: {integrity: sha512-+knoa0BDoeXgkNvvV1vvbZX4+hizelrkwmGJBdT17t8FNPwG2lKemmuMZlmaNQ3ws3DKKCxpb4zRZEIp3UxFCg==}
|
||||
engines: {node: '>=18'}
|
||||
cpu: [x64]
|
||||
os: [win32]
|
||||
|
||||
'@vercel/oidc@3.2.0':
|
||||
resolution: {integrity: sha512-UycprH3T6n3jH0k44NHMa7pnFHGu/N05MjojYr+Mc6I7obkoLIJujSWwin1pCvdy/eOxrI/l3uDLQsmcrOb4ug==}
|
||||
engines: {node: '>= 20'}
|
||||
|
||||
'@vercel/sandbox@1.8.1':
|
||||
resolution: {integrity: sha512-txohjI20aMxZiAzBL/KJi5EqTYsesBdOyIOtpTIyebPLTqYtDYfNhQ4OeYiUcPMUo0XBt8gSet/rIdLQEjj3/A==}
|
||||
|
||||
async-retry@1.3.3:
|
||||
resolution: {integrity: sha512-wfr/jstw9xNi/0teMHrRW7dsz3Lt5ARhYNZ2ewpadnhaIp5mbALhOAP+EAdsC7t4Z6wqsDVv9+W6gm1Dk9mEyw==}
|
||||
|
||||
b4a@1.8.0:
|
||||
resolution: {integrity: sha512-qRuSmNSkGQaHwNbM7J78Wwy+ghLEYF1zNrSeMxj4Kgw6y33O3mXcQ6Ie9fRvfU/YnxWkOchPXbaLb73TkIsfdg==}
|
||||
peerDependencies:
|
||||
react-native-b4a: '*'
|
||||
peerDependenciesMeta:
|
||||
react-native-b4a:
|
||||
optional: true
|
||||
|
||||
bare-events@2.8.2:
|
||||
resolution: {integrity: sha512-riJjyv1/mHLIPX4RwiK+oW9/4c3TEUeORHKefKAKnZ5kyslbN+HXowtbaVEqt4IMUB7OXlfixcs6gsFeo/jhiQ==}
|
||||
peerDependencies:
|
||||
bare-abort-controller: '*'
|
||||
peerDependenciesMeta:
|
||||
bare-abort-controller:
|
||||
optional: true
|
||||
|
||||
esbuild@0.27.4:
|
||||
resolution: {integrity: sha512-Rq4vbHnYkK5fws5NF7MYTU68FPRE1ajX7heQ/8QXXWqNgqqJ/GkmmyxIzUnf2Sr/bakf8l54716CcMGHYhMrrQ==}
|
||||
engines: {node: '>=18'}
|
||||
hasBin: true
|
||||
|
||||
events-universal@1.0.1:
|
||||
resolution: {integrity: sha512-LUd5euvbMLpwOF8m6ivPCbhQeSiYVNb8Vs0fQ8QjXo0JTkEHpz8pxdQf0gStltaPpw0Cca8b39KxvK9cfKRiAw==}
|
||||
|
||||
fast-fifo@1.3.2:
|
||||
resolution: {integrity: sha512-/d9sfos4yxzpwkDkuN7k2SqFKtYNmCTzgfEpz82x34IM9/zc8KGxQoXg1liNC/izpRM/MBdt44Nmx41ZWqk+FQ==}
|
||||
|
||||
fsevents@2.3.3:
|
||||
resolution: {integrity: sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==}
|
||||
engines: {node: ^8.16.0 || ^10.6.0 || >=11.0.0}
|
||||
os: [darwin]
|
||||
|
||||
get-tsconfig@4.13.6:
|
||||
resolution: {integrity: sha512-shZT/QMiSHc/YBLxxOkMtgSid5HFoauqCE3/exfsEcwg1WkeqjG+V40yBbBrsD+jW2HDXcs28xOfcbm2jI8Ddw==}
|
||||
|
||||
jsonlines@0.1.1:
|
||||
resolution: {integrity: sha512-ekDrAGso79Cvf+dtm+mL8OBI2bmAOt3gssYs833De/C9NmIpWDWyUO4zPgB5x2/OhY366dkhgfPMYfwZF7yOZA==}
|
||||
|
||||
ms@2.1.3:
|
||||
resolution: {integrity: sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==}
|
||||
|
||||
os-paths@4.4.0:
|
||||
resolution: {integrity: sha512-wrAwOeXp1RRMFfQY8Sy7VaGVmPocaLwSFOYCGKSyo8qmJ+/yaafCl5BCA1IQZWqFSRBrKDYFeR9d/VyQzfH/jg==}
|
||||
engines: {node: '>= 6.0'}
|
||||
|
||||
picocolors@1.1.1:
|
||||
resolution: {integrity: sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==}
|
||||
|
||||
resolve-pkg-maps@1.0.0:
|
||||
resolution: {integrity: sha512-seS2Tj26TBVOC2NIc2rOe2y2ZO7efxITtLZcGSOnHHNOQ7CkiUBfw0Iw2ck6xkIhPwLhKNLS8BO+hEpngQlqzw==}
|
||||
|
||||
retry@0.13.1:
|
||||
resolution: {integrity: sha512-XQBQ3I8W1Cge0Seh+6gjj03LbmRFWuoszgK9ooCpwYIrhhoO80pfq4cUkU5DkknwfOfFteRwlZ56PYOGYyFWdg==}
|
||||
engines: {node: '>= 4'}
|
||||
|
||||
streamx@2.23.0:
|
||||
resolution: {integrity: sha512-kn+e44esVfn2Fa/O0CPFcex27fjIL6MkVae0Mm6q+E6f0hWv578YCERbv+4m02cjxvDsPKLnmxral/rR6lBMAg==}
|
||||
|
||||
tar-stream@3.1.7:
|
||||
resolution: {integrity: sha512-qJj60CXt7IU1Ffyc3NJMjh6EkuCFej46zUqJ4J7pqYlThyd9bO0XBTmcOIhSzZJVWfsLks0+nle/j538YAW9RQ==}
|
||||
|
||||
text-decoder@1.2.7:
|
||||
resolution: {integrity: sha512-vlLytXkeP4xvEq2otHeJfSQIRyWxo/oZGEbXrtEEF9Hnmrdly59sUbzZ/QgyWuLYHctCHxFF4tRQZNQ9k60ExQ==}
|
||||
|
||||
tsx@4.21.0:
|
||||
resolution: {integrity: sha512-5C1sg4USs1lfG0GFb2RLXsdpXqBSEhAaA/0kPL01wxzpMqLILNxIxIOKiILz+cdg/pLnOUxFYOR5yhHU666wbw==}
|
||||
engines: {node: '>=18.0.0'}
|
||||
hasBin: true
|
||||
|
||||
undici@7.24.1:
|
||||
resolution: {integrity: sha512-5xoBibbmnjlcR3jdqtY2Lnx7WbrD/tHlT01TmvqZUFVc9Q1w4+j5hbnapTqbcXITMH1ovjq/W7BkqBilHiVAaA==}
|
||||
engines: {node: '>=20.18.1'}
|
||||
|
||||
xdg-app-paths@5.1.0:
|
||||
resolution: {integrity: sha512-RAQ3WkPf4KTU1A8RtFx3gWywzVKe00tfOPFfl2NDGqbIFENQO4kqAJp7mhQjNj/33W5x5hiWWUdyfPq/5SU3QA==}
|
||||
engines: {node: '>=6'}
|
||||
|
||||
xdg-portable@7.3.0:
|
||||
resolution: {integrity: sha512-sqMMuL1rc0FmMBOzCpd0yuy9trqF2yTTVe+E9ogwCSWQCdDEtQUwrZPT6AxqtsFGRNxycgncbP/xmOOSPw5ZUw==}
|
||||
engines: {node: '>= 6.0'}
|
||||
|
||||
zod@3.24.4:
|
||||
resolution: {integrity: sha512-OdqJE9UDRPwWsrHjLN2F8bPxvwJBK22EHLWtanu0LSYr5YqzsaaW3RMgmjwr8Rypg5k+meEJdSPXJZXE/yqOMg==}
|
||||
|
||||
snapshots:
|
||||
|
||||
'@esbuild/aix-ppc64@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/android-arm64@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/android-arm@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/android-x64@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/darwin-arm64@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/darwin-x64@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/freebsd-arm64@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/freebsd-x64@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/linux-arm64@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/linux-arm@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/linux-ia32@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/linux-loong64@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/linux-mips64el@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/linux-ppc64@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/linux-riscv64@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/linux-s390x@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/linux-x64@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/netbsd-arm64@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/netbsd-x64@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/openbsd-arm64@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/openbsd-x64@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/openharmony-arm64@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/sunos-x64@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/win32-arm64@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/win32-ia32@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@esbuild/win32-x64@0.27.4':
|
||||
optional: true
|
||||
|
||||
'@vercel/oidc@3.2.0': {}
|
||||
|
||||
'@vercel/sandbox@1.8.1':
|
||||
dependencies:
|
||||
'@vercel/oidc': 3.2.0
|
||||
async-retry: 1.3.3
|
||||
jsonlines: 0.1.1
|
||||
ms: 2.1.3
|
||||
picocolors: 1.1.1
|
||||
tar-stream: 3.1.7
|
||||
undici: 7.24.1
|
||||
xdg-app-paths: 5.1.0
|
||||
zod: 3.24.4
|
||||
transitivePeerDependencies:
|
||||
- bare-abort-controller
|
||||
- react-native-b4a
|
||||
|
||||
async-retry@1.3.3:
|
||||
dependencies:
|
||||
retry: 0.13.1
|
||||
|
||||
b4a@1.8.0: {}
|
||||
|
||||
bare-events@2.8.2: {}
|
||||
|
||||
esbuild@0.27.4:
|
||||
optionalDependencies:
|
||||
'@esbuild/aix-ppc64': 0.27.4
|
||||
'@esbuild/android-arm': 0.27.4
|
||||
'@esbuild/android-arm64': 0.27.4
|
||||
'@esbuild/android-x64': 0.27.4
|
||||
'@esbuild/darwin-arm64': 0.27.4
|
||||
'@esbuild/darwin-x64': 0.27.4
|
||||
'@esbuild/freebsd-arm64': 0.27.4
|
||||
'@esbuild/freebsd-x64': 0.27.4
|
||||
'@esbuild/linux-arm': 0.27.4
|
||||
'@esbuild/linux-arm64': 0.27.4
|
||||
'@esbuild/linux-ia32': 0.27.4
|
||||
'@esbuild/linux-loong64': 0.27.4
|
||||
'@esbuild/linux-mips64el': 0.27.4
|
||||
'@esbuild/linux-ppc64': 0.27.4
|
||||
'@esbuild/linux-riscv64': 0.27.4
|
||||
'@esbuild/linux-s390x': 0.27.4
|
||||
'@esbuild/linux-x64': 0.27.4
|
||||
'@esbuild/netbsd-arm64': 0.27.4
|
||||
'@esbuild/netbsd-x64': 0.27.4
|
||||
'@esbuild/openbsd-arm64': 0.27.4
|
||||
'@esbuild/openbsd-x64': 0.27.4
|
||||
'@esbuild/openharmony-arm64': 0.27.4
|
||||
'@esbuild/sunos-x64': 0.27.4
|
||||
'@esbuild/win32-arm64': 0.27.4
|
||||
'@esbuild/win32-ia32': 0.27.4
|
||||
'@esbuild/win32-x64': 0.27.4
|
||||
|
||||
events-universal@1.0.1:
|
||||
dependencies:
|
||||
bare-events: 2.8.2
|
||||
transitivePeerDependencies:
|
||||
- bare-abort-controller
|
||||
|
||||
fast-fifo@1.3.2: {}
|
||||
|
||||
fsevents@2.3.3:
|
||||
optional: true
|
||||
|
||||
get-tsconfig@4.13.6:
|
||||
dependencies:
|
||||
resolve-pkg-maps: 1.0.0
|
||||
|
||||
jsonlines@0.1.1: {}
|
||||
|
||||
ms@2.1.3: {}
|
||||
|
||||
os-paths@4.4.0: {}
|
||||
|
||||
picocolors@1.1.1: {}
|
||||
|
||||
resolve-pkg-maps@1.0.0: {}
|
||||
|
||||
retry@0.13.1: {}
|
||||
|
||||
streamx@2.23.0:
|
||||
dependencies:
|
||||
events-universal: 1.0.1
|
||||
fast-fifo: 1.3.2
|
||||
text-decoder: 1.2.7
|
||||
transitivePeerDependencies:
|
||||
- bare-abort-controller
|
||||
- react-native-b4a
|
||||
|
||||
tar-stream@3.1.7:
|
||||
dependencies:
|
||||
b4a: 1.8.0
|
||||
fast-fifo: 1.3.2
|
||||
streamx: 2.23.0
|
||||
transitivePeerDependencies:
|
||||
- bare-abort-controller
|
||||
- react-native-b4a
|
||||
|
||||
text-decoder@1.2.7:
|
||||
dependencies:
|
||||
b4a: 1.8.0
|
||||
transitivePeerDependencies:
|
||||
- react-native-b4a
|
||||
|
||||
tsx@4.21.0:
|
||||
dependencies:
|
||||
esbuild: 0.27.4
|
||||
get-tsconfig: 4.13.6
|
||||
optionalDependencies:
|
||||
fsevents: 2.3.3
|
||||
|
||||
undici@7.24.1: {}
|
||||
|
||||
xdg-app-paths@5.1.0:
|
||||
dependencies:
|
||||
xdg-portable: 7.3.0
|
||||
|
||||
xdg-portable@7.3.0:
|
||||
dependencies:
|
||||
os-paths: 4.4.0
|
||||
|
||||
zod@3.24.4: {}
|
||||
@@ -0,0 +1,105 @@
|
||||
/**
|
||||
* Benchmark scenarios for comparing Node.js daemon vs Rust native daemon.
|
||||
*
|
||||
* Each scenario defines CLI commands run via `sandbox.runCommand("agent-browser", args)`.
|
||||
* Setup/teardown commands run once and are not timed.
|
||||
* The `commands` array is timed over N iterations.
|
||||
*/
|
||||
|
||||
export interface Scenario {
|
||||
name: string;
|
||||
description: string;
|
||||
setup?: string[][];
|
||||
commands: string[][];
|
||||
teardown?: string[][];
|
||||
}
|
||||
|
||||
const FORM_HTML = [
|
||||
"<html><head><title>Bench</title></head><body>",
|
||||
"<h1>Benchmark Page</h1>",
|
||||
"<input id='name' type='text' placeholder='Name'>",
|
||||
"<input id='email' type='email' placeholder='Email'>",
|
||||
"<select id='color'><option value='red'>Red</option><option value='blue'>Blue</option></select>",
|
||||
"<input id='agree' type='checkbox'>",
|
||||
"<textarea id='bio' placeholder='Bio'></textarea>",
|
||||
"<button id='submit'>Submit</button>",
|
||||
"<p id='status'>Ready</p>",
|
||||
"<a id='link' href='javascript:void(0)' onclick=\"document.getElementById('status').textContent='Clicked'\">Click me</a>",
|
||||
"<ul>",
|
||||
...Array.from({ length: 20 }, (_, i) => `<li class='item'>Item ${i + 1}</li>`),
|
||||
"</ul>",
|
||||
"</body></html>",
|
||||
].join("");
|
||||
|
||||
const INJECT_FORM_SCRIPT = `document.open(); document.write(${JSON.stringify(FORM_HTML)}); document.close(); 'ok'`;
|
||||
|
||||
const SETUP_PAGE: string[][] = [
|
||||
["open", "about:blank"],
|
||||
["eval", INJECT_FORM_SCRIPT],
|
||||
];
|
||||
|
||||
export const scenarios: Scenario[] = [
|
||||
{
|
||||
name: "navigate",
|
||||
description: "Page navigation (about:blank round-trip)",
|
||||
commands: [["open", "about:blank"]],
|
||||
},
|
||||
{
|
||||
name: "snapshot",
|
||||
description: "DOM snapshot (accessibility tree)",
|
||||
setup: SETUP_PAGE,
|
||||
commands: [["snapshot"]],
|
||||
},
|
||||
{
|
||||
name: "screenshot",
|
||||
description: "Screenshot capture",
|
||||
setup: SETUP_PAGE,
|
||||
commands: [["screenshot"]],
|
||||
},
|
||||
{
|
||||
name: "evaluate",
|
||||
description: "JavaScript evaluation",
|
||||
setup: SETUP_PAGE,
|
||||
commands: [
|
||||
[
|
||||
"eval",
|
||||
"document.title + ' ' + document.querySelectorAll('li').length",
|
||||
],
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "click",
|
||||
description: "Element click interaction",
|
||||
setup: SETUP_PAGE,
|
||||
commands: [["click", "#link"]],
|
||||
},
|
||||
{
|
||||
name: "fill",
|
||||
description: "Form field fill",
|
||||
setup: SETUP_PAGE,
|
||||
commands: [["fill", "#name", "Benchmark User"]],
|
||||
},
|
||||
{
|
||||
name: "agent-loop",
|
||||
description: "AI agent loop: snapshot -> click -> snapshot (typical agent cycle)",
|
||||
setup: SETUP_PAGE,
|
||||
commands: [["snapshot"], ["click", "#link"], ["snapshot"]],
|
||||
},
|
||||
{
|
||||
name: "full-workflow",
|
||||
description:
|
||||
"Realistic workflow: navigate, inject form, snapshot, click, fill, evaluate, screenshot",
|
||||
commands: [
|
||||
["open", "about:blank"],
|
||||
["eval", INJECT_FORM_SCRIPT],
|
||||
["snapshot"],
|
||||
["click", "#link"],
|
||||
["fill", "#name", "Agent User"],
|
||||
[
|
||||
"eval",
|
||||
"document.getElementById('name').value",
|
||||
],
|
||||
["screenshot"],
|
||||
],
|
||||
},
|
||||
];
|
||||
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "bundler",
|
||||
"esModuleInterop": true,
|
||||
"strict": true,
|
||||
"skipLibCheck": true,
|
||||
"outDir": "dist",
|
||||
"declaration": true
|
||||
},
|
||||
"include": ["*.ts"]
|
||||
}
|
||||
Reference in New Issue
Block a user