@vercel/agent eval Test AI coding agents on your framework. Measure what actually works. Why? You're building a frontend framework and want AI agents to work well with it. But how do you know if: Your documentation helps agents write correct code? Adding an MCP server improves agent success rates? Sonnet performs as well as Opus for your use cases? Your latest API changes broke agent compatibility? This framework gives you answers. Run controlled experiments, measure pass rates, compare techniques. Quick Start [安装/使用命令见下方] CLI Run all experiments [安装/使用命令见下方] With no arguments, the CLI discovers every experiments/ .ts file and runs them all. Each experiment runs in parallel. Results with matching fingerprints are reused automatically (see Result Reuse). Run a single experiment [安装/使用命令见下方] The argument is the experiment filename without .ts . This resolves to experiments/cc.ts . Flags Flag Description dry Preview what would run without executing. No API calls, no cost. smoke Quick setup verification. Picks the first eval alphabetically, runs once per model. force Ignore cached fingerprints and re run everything. Only applies when running all. ack failures Keep non model failures as final results instead of deleting them. Flags work with both modes: [安装/使用命令见下方] Other commands [安装/使用命令见下方] Creating Evals Each eval tests one specific task an agent should be able to do with your framework. Directory structure [安装/使用命令见下方] PROMPT.md what you want the agent to do: [安装/使用命令见下方] EVAL.ts how you verify it worked: [安装/使用命令见下方] Use EVAL.tsx when your tests require JSX syntax (React Testing Library, component rendering). You only need one eval file per fixture choose .tsx if any test needs JSX. Asserting on agent behavior EVAL.ts tests can assert not just on the files the agent produced, but on how it worked — which shell commands it ran, which files it read, how many tool calls it made, etc. The framework automatically parses the agent's transcript and writes the results to a