OpenCode Workflow

OpenCode is the lightweight control plane for this inference project.

The target surface is local llama.cpp inference:

• llama-server is the preferred runtime.
• The benchmark endpoint is OpenAI-compatible chat completions.
• .agents/skills/huggingface-local-models/ defines the GGUF discovery and launch workflow.
• research/results.tsv is the append-only local benchmark ledger.

Setup

Install project tooling:

uv sync

Install or confirm llama.cpp:

llama-server --version
llama-cli --version

On macOS, Homebrew is the simplest path:

brew install llama.cpp

Authenticate if the target model is gated:

hf auth login

Parent Session Workflow

1. Inspect local hardware and tools:

uv run scripts/inspect_llama_toolchain.py

2. Resolve a target GGUF:

uv run scripts/resolve_hf_gguf.py --repo <owner/repo> --quant Q4_K_M

3. Start OpenCode in inference/:

opencode

4. Use the inference-lab primary agent.

Delegate to:

• optimizer for the next one-variable speed candidate
• benchmarker for baseline and candidate measurements

Benchmarking

Benchmark an existing endpoint:

uv run scripts/benchmark_llama.py \
  --base-url http://127.0.0.1:8080/v1 \
  --runs 3 \
  --append-tsv research/results.tsv

Launch a server for one benchmark and stop it afterward:

uv run scripts/benchmark_llama.py \
  --server-cmd "llama-server --hf-repo <owner/repo> --hf-file <file.gguf> -c 4096 --port 8080" \
  --base-url http://127.0.0.1:8080/v1 \
  --runs 3 \
  --append-tsv research/results.tsv