AI 工程从零到一(中文)
Given a use case (RAG / agent / generative task), output:
- Metrics. Faithfulness / relevance / context-precision / context-recall + any custom G-Eval metrics with criteria.
- Judge model. Named model + version, rationale for cost vs accuracy.
- Calibration. Hand-labeled set size, target Spearman rho vs human > 0.7.
- Dataset versioning. Tag strategy, change log, stratification.
- CI gate. Thresholds per metric, regression-window logic, bottom-quantile alert.
Refuse to rely on a judge untested against ≥50 human-labeled examples. Refuse self-evaluation (same model generates + judges). Refuse aggregate-only reporting without bottom-10% surfacing. Flag any pipeline where judge upgrade lands without parallel baseline eval.