# Router Benchmark (Stage 2)

Tests whether the activation-scenario descriptions in v2.2.0 skill frontmatter are good enough to route the right skill to a given prompt.

See `researcher/benchmarks/PLAN.md` for full methodology.

## Files

- `prompts.jsonl`: ground-truth prompts. Each line has `prompt_id`, `prompt`, `expected_primary_skill`, optional `acceptable_secondary_skills` and `rejected_skills`, and a `reason`.
- `routing-prompt.md`: the template given to the LLM. Uses `{{SKILL_BLOCK}}`, `{{USER_PROMPT}}`, `{{SKILL_COUNT}}` placeholders.
- `results/<date>-<seed>/`: per-run JSON outputs (gitignored).

## Running

From the SDK runner:

```bash
cd researcher/benchmarks/sdk-runner
npm install
npm run router:dry-run                       # see the plan and cost forecast
npm run router:run -- --max-budget-usd 5     # execute (after exporting CURSOR_API_KEY)
```

## Ground truth

Initial fixtures are 50 prompts covering:

- Single-skill positive controls (one per skill, 15 cases)
- Adversarial boundary pairs from the v2.2.0 boundary-confusion list (15 cases across 5 pairs x 3 variants)
- Combined-skill prompts where multiple are acceptable (10 cases)
- Negative controls where no skill should fit well (5 cases)
- Subtle activation cases that should still resolve (5 cases)

Expand to 100 by adding prompts as new boundary confusions surface in the wild.