11. Loop Closure: PR-Based Curation

The hidden failure mode at Level 5 is the open loop: the Curator generates suggestions but nothing gets merged. On the Aristote project over ten weeks of ACE-v1 operation, two curator reports separated by ten weeks proposed the same two rule candidates. Neither was merged. The loop was open, and the system produced reports instead of progress.

Closing the loop requires making the Curator's output easy to act on. The mechanism: the Curator generates a Git PR rather than a plain report.

PR Anatomy

Each Curator PR contains four things:

1. Config diff: the exact rule or skill change proposed (a git diff-ready patch, not prose)
2. Friction evidence: the 3-5 friction events that drove the suggestion, with event IDs linking back to the signal files
3. Canary results: a before/after comparison on 10-20 probe prompts (see below)
4. Escalation note: if this suggestion was already proposed in a previous report and not acted on, the PR includes a counter ("This suggestion appeared in 2 prior reports without action")

A human reviews and merges or closes. The Curator never modifies rules directly. This is the "Augmented" in a mature context engineering workflow: the loop closes through human judgment, not automation.

A/B Canary Probes

Before proposing a change, the Curator runs a small set of probe prompts against both the current config and the proposed config. Probes are simple, task-representative inputs that exercise the rule being changed.

# canary-ab.sh
OLD_CONFIG="$1"
NEW_CONFIG="$2"
PROBES_FILE="${3:-canary-probes.yaml}"

for probe in $(yq '.probes[].id' "$PROBES_FILE"); do
  question=$(yq ".probes[] | select(.id == \"$probe\") | .prompt" "$PROBES_FILE")
  old_out=$(run_with_config "$OLD_CONFIG" "$question")
  new_out=$(run_with_config "$NEW_CONFIG" "$question")
  compare_outputs "$probe" "$old_out" "$new_out"
done

Ten to twenty probes per PR is sufficient. Use cosine similarity for a first pass; run LLM-as-judge only on probes where similarity drops below 0.85. This keeps canary costs near zero for most PRs and reserving judgment for the edge cases that actually warrant it.

Multi-timescale Operation

Running the Curator on a single cadence produces two failure modes: too frequent and you overwhelm reviewers, too infrequent and friction accumulates invisibly. Use three loops:

Loop	Trigger	Action
Real-time	PostStop hook fires	Append friction event to local signal store
Weekly	Cron (Saturday 02:00)	Curator aggregates the week, generates PR if signal threshold met
Quarterly	Manual	Constitutional audit: check rule overlap, archive dormant skills, review profile consolidation candidates

The quarterly loop is not automatable in any useful way. It requires reading the system's actual behavior, not just its logged signals.

Signal Locality Decision

Where friction signals live determines what the Curator can access. Three options:

Option	How it works	Best for
A. Local cron	Signals stay on the developer's machine; Curator runs as a macOS launchd job or local cron	Solo dev, privacy-first, no infra to maintain
B. Pushed signal store	PostStop hook pushes anonymized signals to a private repo or S3 bucket; Curator runs in CI	Teams of 5+, multi-dev reconciliation required (Section 14)
C. Hosted dev env	Signals land in a shared environment (Codespaces, Coder) by default	Teams already on hosted dev infrastructure

Option A is the right default for solo developers. Option B is necessary for any team that wants cross-developer pattern analysis or multi-dev profile reconciliation; the signal store should be a private repo, not a SaaS platform, to keep sensitive path and tooling data off third-party servers. Option C is only worth considering if the team is already committed to hosted dev environments for other reasons.

Suggestion Suppression

A suggestion that appears in three consecutive reports without any action taken should change state: it either moves to "pending human decision" with a blocking flag in the next PR, or it gets closed as "won't fix" with a documented reason. Allowing suggestions to repeat silently is the same failure mode as the open loop, just more subtle.