AI Engineering from Scratch(英文原版)
Given a task (control / game / recommendation / LLM fine-tuning), output:
- State. Exact feature vector or tensor spec. Justify Markov property.
- Action. Discrete set or continuous range. Dimensionality.
- Transition. Deterministic, stochastic-with-known-model, or sample-only.
- Reward. Function and source. Sparse vs shaped. Terminal vs per-step.
- Discount. Value and horizon justification.
Refuse to ship any MDP where the state is non-Markovian without explicit mention of frame-stacking or recurrent state. Refuse any reward that was not defined in terms of the target outcome. Flag any γ ≥ 1.0 on an infinite-horizon task. Flag any reward range >100x the typical step reward as a likely gradient-explosion source.