Chris Tate
4cc6ca40b7
Before this change, the main skill served by the CLI (`agent-browser
skills get agent-browser`) was a ~40-line discovery stub whose content
was essentially "run `agent-browser skills get <name>` before doing
anything." Agents already inside the CLI got no signal from it — the
content they needed to actually use the tool lived only in the `--full`
references.
Split the two jobs apart:
- **`skill-data/core/`** (new) — the runtime usage guide. 420-line
`SKILL.md` covering the snapshot-and-ref loop, common workflows
(login, extract, screenshot, multi-tab, sessions, iframes, dialogs),
waiting strategies, element selection strategies, troubleshooting,
and when to load a specialized skill. Supplementary `references/` and
`templates/` (moved from `skills/agent-browser/`) provide the full
command reference under `--full`.
- **`skills/agent-browser/SKILL.md`** — still the discovery stub that
`npx skills add` installs, now marked `hidden: true` so it stays out
of `skills list` inside the CLI. Body is a clean pointer to
`agent-browser skills get core` and the specialized skills.
The `hidden: true` frontmatter flag is a new, general mechanism: skills
marked hidden are omitted from `skills list` and `skills get --all` but
can still be fetched by explicit name. This keeps the stub reachable
for anyone who installed via `npx skills add` without polluting the
CLI-side skill listing.
## Behavior
```
$ agent-browser skills list
agentcore Run agent-browser on AWS Bedrock AgentCore cloud browsers...
core Core agent-browser usage guide. Read this before running...
dogfood Systematically explore and test a web application...
electron Automate Electron desktop apps (VS Code, Slack, Discord...)
slack Interact with Slack workspaces using browser automation...
vercel-sandbox Run agent-browser + Chrome inside Vercel Sandbox microVMs...
$ agent-browser skills get core # the actual usage guide
# ~420 lines of workflows, patterns, troubleshooting
$ agent-browser skills get agent-browser # still works if called explicitly
# the thin stub, now pointing at `core`
```
External `npx skills add vercel-labs/agent-browser` behavior is
unchanged: it finds and installs the thin `agent-browser` stub, which
tells the agent to run `agent-browser skills get core` for real
content. Version drift protection is preserved — the stub is the only
thing that gets copied; the real content is always runtime-fetched.
## Updated
- `cli/src/skills.rs` — `SkillInfo.hidden: bool`, parsed from
frontmatter; `run_list` and `run_get --all` filter it. 3 new unit
tests for the frontmatter parser.
- `cli/src/output.rs` — top-level `--help` and `skills` subcommand help
reference `skills get core` / `skills get core --full`.
- `AGENTS.md` — "update these files for user-facing features" now
points at `skill-data/core/` instead of the stub, with a note that
the stub is not the right place for feature content.
- `README.md`, `docs/src/app/skills/page.mdx` — describe the new
split and `skills get core --full` as the recommended entry point.
- `evals/cases/{command-usage,skill-selection}.ts` — expect
`skills get core` in agent output instead of `skills get
agent-browser`. Eval lib still reads `skills/agent-browser/SKILL.md`
(simulating what an agent sees after `npx skills add`).
All 11 skills unit tests pass. `cargo clippy -- -D warnings` and
`cargo fmt --check` clean. Verified end-to-end: `skills list` shows
`core` + specialized (no stub), `skills get core` returns the new
content, `skills get agent-browser` still returns the stub on explicit
request.
95 lines
2.6 KiB
TypeScript
95 lines
2.6 KiB
TypeScript
import type { EvalCase } from "../lib/types.ts";
|
|
|
|
const RUBRIC = `
|
|
1 - Agent does not load any skill or loads a completely wrong one
|
|
2 - Agent loads the generic agent-browser skill when a specialized one exists
|
|
3 - Agent loads a related but suboptimal skill
|
|
4 - Agent loads the correct specialized skill
|
|
5 - Agent loads the correct skill and explains why it chose it
|
|
`.trim();
|
|
|
|
export const cases: EvalCase[] = [
|
|
{
|
|
id: "ss-01",
|
|
name: "Selects slack skill for Slack tasks",
|
|
category: "skill-selection",
|
|
prompt: "Check my Slack unreads and summarize any messages mentioning me",
|
|
expectedPatterns: [
|
|
"skills get slack",
|
|
],
|
|
rubric: RUBRIC,
|
|
},
|
|
{
|
|
id: "ss-02",
|
|
name: "Selects electron skill for VS Code automation",
|
|
category: "skill-selection",
|
|
prompt: "Automate VS Code to open a project and run a terminal command",
|
|
expectedPatterns: [
|
|
"skills get electron",
|
|
],
|
|
rubric: RUBRIC,
|
|
},
|
|
{
|
|
id: "ss-03",
|
|
name: "Selects dogfood skill for QA/testing",
|
|
category: "skill-selection",
|
|
prompt: "QA test http://localhost:3000 and find any bugs or UX issues",
|
|
expectedPatterns: [
|
|
"skills get dogfood",
|
|
],
|
|
rubric: RUBRIC,
|
|
},
|
|
{
|
|
id: "ss-04",
|
|
name: "Selects agentcore skill for AWS cloud browsers",
|
|
category: "skill-selection",
|
|
prompt:
|
|
"Run browser automation on AWS using AgentCore cloud browsers",
|
|
expectedPatterns: [
|
|
"skills get agentcore",
|
|
],
|
|
rubric: RUBRIC,
|
|
},
|
|
{
|
|
id: "ss-05",
|
|
name: "Selects vercel-sandbox skill for Vercel environments",
|
|
category: "skill-selection",
|
|
prompt:
|
|
"Run headless Chrome inside a Vercel Sandbox microVM to test my deployed Next.js app",
|
|
expectedPatterns: [
|
|
"skills get vercel-sandbox",
|
|
],
|
|
rubric: RUBRIC,
|
|
},
|
|
{
|
|
id: "ss-06",
|
|
name: "Selects electron skill for Discord automation",
|
|
category: "skill-selection",
|
|
prompt: "Automate the Discord desktop app to send a message in a channel",
|
|
expectedPatterns: [
|
|
"skills get electron",
|
|
],
|
|
rubric: RUBRIC,
|
|
},
|
|
{
|
|
id: "ss-07",
|
|
name: "Selects dogfood skill for exploratory testing",
|
|
category: "skill-selection",
|
|
prompt: "Dogfood vercel.com and write up a bug report",
|
|
expectedPatterns: [
|
|
"skills get dogfood",
|
|
],
|
|
rubric: RUBRIC,
|
|
},
|
|
{
|
|
id: "ss-08",
|
|
name: "Selects core skill for general browser tasks",
|
|
category: "skill-selection",
|
|
prompt: "Navigate to hacker news and screenshot the front page",
|
|
expectedPatterns: [
|
|
"skills get core",
|
|
],
|
|
rubric: RUBRIC,
|
|
},
|
|
];
|