Top 3 changes to implement first
Add a deterministic diligence fact ledger and domain checklist layer.
Encode canonical facts and source-of-truth rules for ARR, runway/burn, parent-account concentration, unsupported metrics, and SOC 2 status so the agent cannot rely only on generic citation instructions.Upgrade validators to audit the actual output artifacts, not just claimed evidence coverage.
Current validation can pass while artifact-level citation or claim-audit issues still require later repair. Parse generated markdown/JSON/CSV artifacts, extract material claims, verify source support, and fail on unsupported or unaudited claims.Persist the five generated evals into the checked-in regression suite.
The generated evals all passed, but they should become durable regression tests so future prompt/runtime changes cannot regress on the specific human-feedback issues.