proxy — protocol translation & parameter injection

A general-purpose inference proxy, built on FastAPI, that sits between the harness under test and the model backend. Evaluations turn it on with model_connection: local_proxy. It exists to solve two problems:

1. Protocol translation — bridges the two sides when the harness and the backend speak different protocols (today that means Anthropic→OpenAI).
2. extra_body injection — folds in the sampling parameters that cbc/cc can't forward natively (top_k / min_p / chat_template_kwargs, etc.) before the request reaches the backend.

It works purely at the transport layer and never alters model content: it doesn't truncate tool_call, and it doesn't override max_tokens.

Runtime roles

• job-private (default): scripts/run.sh brings up a short-lived proxy for each run on :3456 (bumping to a free port if that one is taken), and tears it down when the run finishes (the trap only kills the PID this run started). It carries a single route, and single-route fallback stays enabled.
• shared (SHARED_PROXY=1): one long-lived proxy runs on a fixed :3456 and accumulates routes as jobs are added. You manage it separately with scripts/proxy/proxy-shared.sh {start,stop}; a run only appends its routes and never starts or stops the proxy itself. In shared mode, single-route fallback is off — every request must match a route by slug/token or it gets a 404. That's deliberate: with many routes in play, falling back to "the one route" would send one model's request to another.

The config is generated by the runner (prepare_job / resolve_manifest) from four config layers and written to disk; the proxy loads it with --config <yaml>. Secrets never enter the YAML — only their *_env names are written, and load_config_from_yaml resolves them at runtime.

Request pipeline

client(harness) → route resolution → [interceptors.on_request] → _prepare_upstream
  → (A2O) Anthropic→OpenAI conversion + extra_body injection → UpstreamSender(retry)
  → backend → (A2O) OpenAI→Anthropic conversion back → [interceptors.on_response] → client

Route resolution (token / body.model → slug)

Each route is keyed by its slug (= model slug). resolve_route picks one like this:

1. Take the request's bearer token (Authorization: Bearer <x> or x-api-key) and look up a route by it;
2. If that misses, fall back to body.model;
3. Only an exact slug match counts as a hit. A non-empty value that matches nothing → 404 — misconfiguration surfaces right away instead of being silently misrouted.
4. Only when the lookup value is empty (the harness sent neither a token nor a body.model) does a job-private proxy fall back to its sole route. The shared proxy never falls back, since with multiple routes accumulated it couldn't know which one you meant.

Once matched, the proxy rewrites the upstream body's model to the route's effective_model (backend_model, defaulting to slug) — this is the point where slug maps to the real backend model id.

Parameter override / injection (_prepare_upstream is the single source of truth)

model.params (configs/models/<slug>.yaml) is the single place where model request parameters are declared. The runner's model_params.flatten_params flattens it into the route's extra_body (internal field injected_params), and the proxy then shallow-updates it into the upstream body:

• Top-level sampling parameters (temperature / top_p, etc.) go directly into the body.
• max_output_tokens → max_tokens (mapped first).
• params.extra_body.* (top_k / min_p / chat_template_kwargs, etc.) are applied last, and on a key conflict the extra_body content wins (it is more specific and provider-targeted) — so an explicit extra_body.max_tokens overrides the mapped max_output_tokens.

Injection happens only at inference endpoints (chat/completions or Anthropic's messages); auxiliary endpoints (/v1/models, etc.) are forwarded untouched. Anthropic backends skip all conversion beyond model rewriting/injection. The proxy injects only what the model config declares — whether the backend actually accepts a given key (say min_p) is up to the backend, and the proxy won't paper over a rejection.

ExtraBodyInterceptor is now a no-op. It runs before prepare, so under A2O its changes get thrown away when prepare rebuilds the body, and under OpenAI passthrough it just repeats what prepare already does. It stays registrable only so legacy route configs keep loading.

Model mapping when running cc (claude-code)

Unlike cbc, claude has no models.json — it reads endpoint/model/key entirely from env, which cc_agent.run() assembles:

env	local_proxy	direct
ANTHROPIC_BASE_URL	host proxy (proxy_url)	real backend URL
ANTHROPIC_MODEL	route slug (the proxy selects a route from it, then does A→O conversion + rewrites body.model)	real model id
ANTHROPIC_API_KEY	fake key (dummy-for-proxy, the real key lives on the proxy side)	real backend key

What to set for cc's settings / auth key: under local_proxy, ANTHROPIC_API_KEY is just a placeholder fake key — the proxy selects a route by slug (from ANTHROPIC_MODEL = bearer token) and hits the backend with the route's own backend.key, ignoring this fake key. So the cc side does not need real credentials.

Alias pin: once a custom ANTHROPIC_BASE_URL is set (proxy or self-hosted), cc_agent pins ANTHROPIC_DEFAULT_{SONNET,OPUS,HAIKU}_MODEL and CLAUDE_CODE_SUBAGENT_MODEL all to the same model (= slug), preventing claude from substituting the request with a built-in Anthropic model id and getting a 404.

Sampling parameters don't flow through cc at all: under local_proxy the proxy is the sole source of truth for them, and cc only passes down endpoint/model/key plus native fallback env like max_output_tokens/compaction. cc's tool deny list lives in settings.json's permissions.deny (see configs/harnesses/claude-code/CONFIG.md).

Modules

file	responsibility
main.py	FastAPI app, endpoints (/v1/chat/completions dedicated + /{path:path} catch-all for /v1/messages, etc., /health, /v1/models, /admin/reload), route resolution, protocol guard
config.py	ProxyConfig / RouteConfig / ProxyMode; load_config_from_yaml (*_env resolution, O2A fail-fast, openai_passthrough→passthrough normalization)
pipeline.py	orchestrates route → intercept → protocol mapping → forward; scrubs Anthropic-only request headers
protocols/a2o.py	Anthropic Messages ↔ OpenAI Chat Completions conversion (including streaming); when a tool name exceeds 64 characters, hash-truncates it and maps it back
sender.py	upstream HTTP forwarding, streaming-aware + retry
interceptors/logger.py	JSONL logging of every request/response (log-viewer compatible), split per trial
interceptors/extra_body.py	injection marker (now a no-op, see above)

ProxyMode

mode	meaning	status
passthrough	same-protocol forwarding	✓
a2o	Anthropic client → OpenAI backend	✓
openai_passthrough	legacy OpenAI-only alias (input compatibility only, not an enum member)	normalized to passthrough on load
o2a	OpenAI client → Anthropic backend	not implemented, fail-fast

Running the proxy locally

python3 -m workbuddy_bench.proxy --config <config.yaml>

You never need to start it by hand in the evaluation flow — under local_proxy, scripts/run.sh brings up a job-private proxy for you.