AI Engineering from Scratch(英文原版)
Given a repo and any existing prose instructions (AGENTS.md, CONTRIBUTING.md, onboarding docs), produce a five-category rule set the workbench can execute.
The five categories:
startup— what must be true before work begins.forbidden— what must never happen.definition_of_done— what proves the task is complete.uncertainty— what the agent does when not sure.approval— what requires human sign-off.
Produce:
docs/agent-rules.mdwith one##heading per rule. Each rule carriescategory,check, and a one-line description.tools/rule_checker.pywith aRuleCheckerclass exposing one method percheck. Each method takes aTurnTracedataclass and returnsbool.tools/rule_report.pyrunner that loads rules, runs the checker on a trace, emits arule_report.json.- A migration notes file: which prose lines became which rule, which were dropped as aspirational, why.
Hard rejects:
- Rules without a
checkfield. Aspirational-only rules belong in onboarding docs, not in the workbench rule set. - A single "be careful" rule. Specify a category and a check or remove it.
- Checks that require LLM calls. Rule checks must be deterministic and cheap so they can run every turn.
- Rule files over 200 lines. Split by category into
agent-rules.\{startup,forbidden,done,uncertainty,approval\}.mdand route from a parent index.
Refusal rules:
- If the agent product cannot supply a
TurnTrace(no instrumentation), refuse to wire the checker until at leastread_state_file,edited_files, andtests_exit_codeare recorded. - If existing instructions are mostly aspirational (>50%), surface that finding before emitting rules. The rule set will look thin; that is correct.
- If a rule is added because of a single past incident, attach the incident id so future review can decide if it is still needed.
Output structure:
<repo>/
├── docs/
│ └── agent-rules.md
├── tools/
│ ├── rule_checker.py
│ └── rule_report.py
└── docs/migration-notes.mdEnd with "what to read next" pointing to:
- Lesson 36 for per-task scope contracts that extend the forbidden category.
- Lesson 38 for verification gates that consume the rule report.
- Lesson 39 for the reviewer agent that scores rule compliance.