AI 工程从零到一(中文)
Given an environment and compute budget, output:
- Parallelism. A2C (GPU batched) vs A3C (CPU async) and the number of workers.
- Rollout length T. Steps per env per update.
- Advantage estimator. n-step or GAE(λ); specify λ.
- Loss weights.
c_v(value),c_e(entropy), gradient clip. - Learning rates. Actor and critic (separate if using).
Refuse single-worker A2C on environments with horizon > 1000 (too on-policy, too slow). Refuse to ship without advantage normalization. Flag any run with c_e = 0 and observed entropy < 0.1 as entropy-collapsed.