# proxy — protocol translation & parameter injection

A general-purpose inference proxy, built on FastAPI, that sits between the harness under test and the model backend. Evaluations turn it on with
`model_connection: local_proxy`. It exists to solve two problems:

1. **Protocol translation** — bridges the two sides when the harness and the backend speak different protocols (today that means Anthropic→OpenAI).
2. **`extra_body` injection** — folds in the sampling parameters that cbc/cc can't forward natively (`top_k` / `min_p` /
   `chat_template_kwargs`, etc.) before the request reaches the backend.

It works purely at the transport layer and **never alters model content**: it doesn't truncate tool_call, and it doesn't override max_tokens.

## Runtime roles

- **job-private (default)**: `scripts/run.sh` brings up a short-lived proxy for each run on `:3456` (bumping to a free port if that one is taken), and tears it down when the run finishes
  (the trap only kills the PID this run started). It carries a single route, and single-route fallback stays enabled.
- **shared (`SHARED_PROXY=1`)**: one long-lived proxy runs on a fixed `:3456` and accumulates routes as jobs are added. You manage it separately with
  `scripts/proxy/proxy-shared.sh {start,stop}`; a run only appends its routes and never starts or stops the proxy itself.
  In shared mode, single-route fallback is **off** — every request must match a route by slug/token or it gets a 404. That's deliberate: with many routes in play, falling back to "the one route" would send one model's request to another.

The config is generated by the runner (`prepare_job` / `resolve_manifest`) from four config layers and written to disk; the proxy loads it with
`--config <yaml>`. Secrets never enter the YAML — only their `*_env` names are written, and `load_config_from_yaml` resolves them at runtime.

## Request pipeline

```
client(harness) → route resolution → [interceptors.on_request] → _prepare_upstream
  → (A2O) Anthropic→OpenAI conversion + extra_body injection → UpstreamSender(retry)
  → backend → (A2O) OpenAI→Anthropic conversion back → [interceptors.on_response] → client
```

## Route resolution (token / body.model → slug)

Each route is keyed by its **`slug`** (= model slug). `resolve_route` picks one like this:

1. Take the request's **bearer token** (`Authorization: Bearer <x>` or `x-api-key`) and look up a route by it;
2. If that misses, fall back to **body.model**;
3. Only an exact `slug` match counts as a hit. A **non-empty value that matches nothing → 404** — misconfiguration surfaces right away instead of being silently misrouted.
4. Only when the lookup value is **empty** (the harness sent neither a token nor a body.model) does a job-private
   proxy fall back to its sole route. The **shared proxy never falls back**, since with multiple routes accumulated it couldn't know which one you meant.

Once matched, the proxy rewrites the upstream body's `model` to the route's **`effective_model`**
(`backend_model`, defaulting to slug) — this is the point where slug maps to the real backend model id.

## Parameter override / injection (`_prepare_upstream` is the single source of truth)

`model.params` (`configs/models/<slug>.yaml`) is the **single place** where model request parameters are declared. The runner's
`model_params.flatten_params` flattens it into the route's `extra_body` (internal field
`injected_params`), and the proxy then shallow-updates it into the upstream body:

- **Top-level sampling parameters** (`temperature` / `top_p`, etc.) go directly into the body.
- **`max_output_tokens` → `max_tokens`** (mapped first).
- **`params.extra_body.*`** (`top_k` / `min_p` / `chat_template_kwargs`, etc.) are applied **last**,
  and on a key conflict the **extra_body content wins** (it is more specific and provider-targeted) — so an explicit
  `extra_body.max_tokens` overrides the mapped `max_output_tokens`.

Injection happens only at **inference endpoints** (chat/completions or Anthropic's messages); auxiliary endpoints
(`/v1/models`, etc.) are forwarded untouched. **Anthropic backends skip all conversion beyond model rewriting/injection.** The proxy injects only
what the model config declares — whether the backend actually accepts a given key (say `min_p`) is up to the backend, and the proxy won't paper over a rejection.

> `ExtraBodyInterceptor` is now a no-op. It runs before prepare, so under A2O its changes get thrown away when prepare rebuilds the body,
> and under OpenAI passthrough it just repeats what prepare already does. It stays registrable only so legacy route configs keep loading.

## Model mapping when running cc (claude-code)

Unlike cbc, claude has no models.json — it reads endpoint/model/key entirely from **env**, which `cc_agent.run()` assembles:

| env | local_proxy | direct |
|---|---|---|
| `ANTHROPIC_BASE_URL` | host proxy (`proxy_url`) | real backend URL |
| `ANTHROPIC_MODEL` | **route slug** (the proxy selects a route from it, then does A→O conversion + rewrites body.model) | real model id |
| `ANTHROPIC_API_KEY` | fake key (`dummy-for-proxy`, the real key lives on the proxy side) | real backend key |

**What to set for cc's settings / auth key**: under local_proxy, `ANTHROPIC_API_KEY` is just a
**placeholder fake key** — the proxy selects a route by slug (from `ANTHROPIC_MODEL` = bearer token) and hits the backend with the route's own
`backend.key`, ignoring this fake key. So the cc side does not need real credentials.

**Alias pin**: once a custom `ANTHROPIC_BASE_URL` is set (proxy or self-hosted), cc_agent pins
`ANTHROPIC_DEFAULT_{SONNET,OPUS,HAIKU}_MODEL` and `CLAUDE_CODE_SUBAGENT_MODEL` **all to the
same `model` (= slug)**, preventing claude from substituting the request with a built-in Anthropic model id and getting a 404.

> Sampling parameters don't flow through cc at all: under local_proxy the proxy is the sole source of truth for them, and cc only passes down endpoint/model/key
> plus native fallback env like max_output_tokens/compaction. cc's tool deny list lives in settings.json's
> `permissions.deny` (see `configs/harnesses/claude-code/CONFIG.md`).

## Modules

| file | responsibility |
|---|---|
| `main.py` | FastAPI app, endpoints (`/v1/chat/completions` dedicated + `/{path:path}` catch-all for `/v1/messages`, etc., `/health`, `/v1/models`, `/admin/reload`), route resolution, protocol guard |
| `config.py` | `ProxyConfig` / `RouteConfig` / `ProxyMode`; `load_config_from_yaml` (`*_env` resolution, O2A fail-fast, `openai_passthrough`→`passthrough` normalization) |
| `pipeline.py` | orchestrates route → intercept → protocol mapping → forward; scrubs Anthropic-only request headers |
| `protocols/a2o.py` | Anthropic Messages ↔ OpenAI Chat Completions conversion (including streaming); when a tool name exceeds 64 characters, hash-truncates it and maps it back |
| `sender.py` | upstream HTTP forwarding, streaming-aware + retry |
| `interceptors/logger.py` | JSONL logging of every request/response (log-viewer compatible), split per trial |
| `interceptors/extra_body.py` | injection marker (now a no-op, see above) |

## ProxyMode

| mode | meaning | status |
|---|---|---|
| `passthrough` | same-protocol forwarding | ✓ |
| `a2o` | Anthropic client → OpenAI backend | ✓ |
| `openai_passthrough` | legacy OpenAI-only alias (input compatibility only, not an enum member) | normalized to `passthrough` on load |
| `o2a` | OpenAI client → Anthropic backend | **not implemented, fail-fast** |

## Running the proxy locally

```bash
python3 -m workbuddy_bench.proxy --config <config.yaml>
```

You never need to start it by hand in the evaluation flow — under `local_proxy`, `scripts/run.sh` brings up a job-private proxy for you.