AI 原生开放教材

外观

Harness Engineering 学习指南工程手册·中文原文 ↗原件 ↓仓库 ↗

附录 E 扩展相关工作 ​

本附录扩展了第 2 节中的简要讨论，并将 Meta-Harness 与若干相邻研究方向进行了定位，这些方向在正文中无法详细涵盖。一个反复出现的区分是：Meta-Harness 优化的是可执行的 Harness 实现，并通过文件系统为提议器提供对先前代码、分数和执行轨迹的选择性访问。

AlphaEvolve / OpenEvolve ​

AlphaEvolve [^34] 和 OpenEvolve [^42] 通过 LLM 引导的变异和结构化反馈来进化代码：提议器接收一个包含标量分数的程序数据库（每步 4-22K token；Table 1），并对锦标赛选择的父代应用固定的变异策略。这些方法专为算法发现和优化而设计（数学猜想、调度启发式、硬件内核），搜索目标是具有清晰标量目标的单一无状态函数，变异是局部的。Harness Engineering 是一个不同的范式：Harness 是跨多个样本累积经验的有状态程序，单个设计选择（如在记忆中存储什么）可以级联影响整个评估序列。Meta-Harness 通过给予非结构化编码智能体完整的文件系统访问权限来解决这一问题，使其能选择性地读取任何先前候选方案的源代码、执行轨迹和分数。

GEPA ​

GEPA [^1] 在反馈丰富度方面是最接近的文本优化器（text optimizer），为每个候选方案提供 Rollout 轨迹。它专为具有短反馈循环的提示优化任务设计（数学问题、指令遵循、代码优化），其中每次 Rollout 是单次 LLM 调用或短流水线。在这一范式中，逐候选反思效果良好：一个提示、一个答案、一个分数。Harness Engineering 需要同时跨多个样本和多个候选方案进行推理：理解为什么某个检索策略在一类问题上有效但在另一类问题上退化，需要跨整个种群比较执行轨迹。GEPA 每次操作一个候选方案（每步 2-8K token；Table 1），使用必须预先预测哪些信息相关的固定批评格式。Meta-Harness 让提议器同时访问所有先前候选方案，并让智能体自行决定检查什么。

提示编排框架 ​

若干系统为组合多阶段 LLM 程序提供了结构化抽象。LMQL [^7]、LangChain [^12] 和 DSPy [^22] 通过为提示模板、控制流和模块化 LLM 流水线提供更高层次的接口，使提示工程更加系统化。这些框架帮助开发者指定和组织 LLM 程序，但它们通常仍需要手动设计检索策略、记忆更新和编排逻辑。Meta-Harness 在不同的层次上运作：它在可执行代码中搜索这些策略的实现，将 Harness 本身作为优化目标。

[^1]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/L. A. Agrawal, S. Tan, D. Soylu, N. Ziems, R. Khare, K. Opsahl-Ong, A. Singhvi, H. Shandilya, M. J. Ryan, M. Jiang, et al. (2025) Gepa: reflective prompt evolution can outperform reinforcement learning. arXiv preprint arXiv:2507.19457. Cited by: Appendix E, Table 1, §1, §2, Table 4.

[^2]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/M. Andrychowicz, M. Denil, S. Gomez, M. W. Hoffman, D. Pfau, T. Schaul, B. Shillingford, and N. De Freitas (2016) Learning to learn by gradient descent by gradient descent. Advances in neural information processing systems 29. Cited by: §2.

[^3]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/Anthropic/README.md and community contributors Agentskills/agentskills. Note: GitHub repository https://github.com/agentskills/agentskills Specification and documentation for Agent Skills, accessed March 27, 2026 Cited by: 1st item.

[^4]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/Anthropic/README.md (2025) Claude code: an agentic coding tool. Note: https://www.anthropic.com/claude-code Cited by: §3.

[^5]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/M. Balunović, J. Dekoninck, I. Petrov, N. Jovanović, and M. Vechev (2025-02) MathArena: evaluating llms on uncontaminated math competitions. SRI Lab, ETH Zurich. External Links: Link Cited by: §4.2, §4.2.

[^6]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/F. Barbieri, J. Camacho-Collados, L. Neves, and L. Espinosa-Anke (2020) TweetEval: unified benchmark and comparative evaluation for tweet classification. External Links: 2010.12421, Link Cited by: 9th item.

[^7]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/L. Beurer-Kellner, M. Fischer, and M. Vechev (2023-06) Prompting is programming: a query language for large language models. Proceedings of the ACM on Programming Languages 7 (PLDI), pp. 1946–1969. External Links: ISSN 2475-1421, Link, Document Cited by: Appendix E.

[^8]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/B. Böckeler (2026-03) Harness engineering. Note: https://martinfowler.com/articles/exploring-gen-ai/harness-engineering.html martinfowler.com Cited by: §1.

[^9]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/C. Bölük (2026-02) I improved 15 LLMs at coding in one afternoon. only the harness changed.. Note: https://blog.can.ac/2026/02/12/the-harness-problem/ Cited by: §1.

[^10]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/I. Casanueva, T. Temčinas, D. Gerz, M. Henderson, and I. Vulić (2020) Efficient intent detection with dual sentence encoders. External Links: 2003.04807, Link Cited by: 6th item.

[^11]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/M. Cemri, S. Agrawal, A. Gupta, S. Liu, A. Cheng, Q. Mang, A. Naren, L. E. Erdogan, K. Sen, M. Zaharia, et al. (2026) AdaEvolve: adaptive llm driven zeroth-order optimization. arXiv preprint arXiv:2602.20133. Cited by: §1.

[^12]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/LangChain/README.md Note: Software, released 2022-10-17 External Links: Link Cited by: Appendix E.

[^13]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/A. Cohan, W. Ammar, M. van Zuylen, and F. Cady (2019) Structural scaffolds for citation intent classification in scientific publications. External Links: 1904.01608, Link Cited by: 1st item.

[^14]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/D. Demszky, D. Movshovitz-Attias, J. Ko, A. Cowen, G. Nemade, and S. Ravi (2020) GoEmotions: a dataset of fine-grained emotions. External Links: 2005.00547, Link Cited by: 5th item.

[^15]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/Z. Fei, X. Shen, D. Zhu, F. Zhou, Z. Han, A. Huang, S. Zhang, K. Chen, Z. Yin, Z. Shen, et al. (2024) Lawbench: benchmarking legal knowledge of large language models. In Proceedings of the 2024 conference on empirical methods in natural language processing, pp. 7933–7962. Cited by: §4.1.

[^16]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/C. Finn, P. Abbeel, and S. Levine (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning, Cited by: §2.

[^17]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/ForgeCode/README.md (2025) Benchmarks don't matter. External Links: Link Cited by: §4.3.

[^18]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/Gretel/README.md AI (2023) Symptom to diagnosis dataset. Note: https://huggingface.co/datasets/gretelai/symptom/_to/_diagnosis Accessed: 2026-01-22 Cited by: §4.1.

[^19]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/S. Hu, C. Lu, and J. Clune (2025) Automated design of agentic systems. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §2.

[^20]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/A. Justin Young (2025-11) Effective harnesses for long-running agents. Note: https://anthropic.com/engineering/effective-harnesses-for-long-running-agents Anthropic Engineering Blog Cited by: §1.

[^21]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/P. Keung, Y. Lu, G. Szarvas, and N. A. Smith (2020) The multilingual amazon reviews corpus. External Links: 2010.02573, Link Cited by: 3rd item.

[^22]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/O. Khattab, A. Singhvi, P. Maheshwari, Z. Zhang, K. Santhanam, S. Vardhamanan, S. Haq, A. Sharma, T. T. Joshi, H. Moazam, H. Miller, M. Zaharia, and C. Potts (2023) DSPy: compiling declarative language model calls into self-improving pipelines. External Links: 2310.03714, Link Cited by: Appendix E.

[^23]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/T. Khot, A. Sabharwal, and P. Clark (2018-Apr.) SciTaiL: a textual entailment dataset from science question answering. Proceedings of the AAAI Conference on Artificial Intelligence 32 (1). External Links: Link, Document Cited by: 8th item.

[^24]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/KRAFTON/README.md AI and Ludo Robotics (2026) Terminus-kira: boosting frontier model performance on terminal-bench with minimal harness. External Links: Link Cited by: §B.3, §4.3.

[^25]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/Y. Lee, J. Boen, and C. Finn (2025) Feedback descent: open-ended text optimization via pairwise comparison. In arXiv preprint arXiv:2511.07919, Cited by: Table 1, §1, §2.

[^26]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/J. Lehman, J. Gordon, S. Jain, K. Ndousse, C. Yeh, and K. O. Stanley (2022) Evolution through large models. External Links: 2206.08896, Link Cited by: §2.

[^27]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, et al. (2020) Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems 33, pp. 9459–9474. Cited by: §1, §2.

[^28]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/L. Loukas, M. Fergadiotis, I. Chalkidis, E. Spyropoulou, P. Malakasiotis, I. Androutsopoulos, and G. Paliouras (2022) FiNER: financial numeric entity recognition for xbrl tagging. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 4419–4431. External Links: Link, Document Cited by: 2nd item.

[^29]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/T. Luong, D. Hwang, H. H. Nguyen, G. Ghiasi, Y. Chervonyi, I. Seo, J. Kim, G. Bingham, J. Lee, S. Mishra, A. Zhai, C. H. Hu, H. Michalewski, J. Kim, J. Ahn, J. Bae, X. Song, T. H. Trinh, Q. V. Le, and J. Jung (2025) Towards robust mathematical reasoning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, External Links: Link Cited by: §4.2.

[^30]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang, et al. (2023) Self-refine: iterative refinement with self-feedback. Advances in neural information processing systems 36, pp. 46534–46594. Cited by: §1, §2.

[^31]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/P. Malo, A. Sinha, P. Takala, P. Korhonen, and J. Wallenius (2013) Good debt or bad debt: detecting semantic orientations in economic texts. External Links: 1307.5336, Link Cited by: 4th item.

[^32]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/M. A. Merrill, A. G. Shaw, N. Carlini, B. Li, H. Raj, I. Bercovich, L. Shi, J. Y. Shin, T. Walshe, E. K. Buchanan, et al. (2026) Terminal-bench: benchmarking agents on hard, realistic tasks in command line interfaces. arXiv preprint arXiv:2601.11868. Cited by: §4.3.

[^33]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/J. Nichols (2025-06) How we scored #1 on terminal-bench (52%). External Links: Link Cited by: §4.3.

[^34]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/A. Novikov, N. Vũ, M. Eisenberger, E. Dupont, P. Huang, A. Z. Wagner, S. Shirobokov, B. Kozlovskii, F. J. Ruiz, A. Mehrabian, et al. (2025) Alphaevolve: a coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.13131. Cited by: Appendix E, Table 1, §1, §2.

[^35]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/OpenAI/README.md (2026-02) Harness engineering: leveraging Codex in an agent-first world. Note: https://openai.com/index/harness-engineering/ OpenAI Blog Cited by: §1.

[^36]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/C. Packer, V. Fang, S. Patil, K. Lin, S. Wooders, and J. Gonzalez (2023) MemGPT: towards llms as operating systems.. Cited by: §1, §2.

[^37]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/R. Pryzant, D. Iter, J. Li, Y. T. Lee, C. Zhu, and M. Zeng (2023) Automatic prompt optimization with "gradient descent" and beam search. arXiv preprint arXiv:2305.03495. Cited by: §1, §2.

[^38]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/B. Romera-Paredes, M. Barekatain, A. Novikov, M. Balog, M. P. Kumar, E. Dupont, F. J. Ruiz, J. S. Ellenberg, P. Wang, O. Fawzi, et al. (2024) Mathematical discoveries from program search with large language models. Nature 625 (7995), pp. 468–475. Cited by: §1, §2.

[^39]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/J. Schmidhuber (1993) A neural network that embeds its own meta-levels. In IEEE International Conference on Neural Networks, Cited by: §2.

[^40]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/N. Schneider, N. Stiefl, and G. A. Landrum (2016) What's what: the (nearly) definitive guide to reaction role assignment. Journal of chemical information and modeling 56 (12), pp. 2336–2346. Cited by: §4.1.

[^41]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/S. Shakya, A. Hartl, S. Hochreiter, and K. Pöppel (2026) Adaptive retrieval helps reasoning in llms – but mostly if it's not used. External Links: 2602.07213, Link Cited by: §4.2.

[^42]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/A. Sharma (2025) OpenEvolve: an open-source evolutionary coding agent. Note: https://github.com/algorithmicsuperintelligence/openevolve GitHub repository External Links: Link Cited by: Appendix E, §2, 2nd item, Table 4.

[^43]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/J. Snell, K. Swersky, and R. S. Zemel (2017) Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems, Cited by: §2.

[^44]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/R. Sutton (2019) The bitter lesson, 2019. URL http://www./ incompleteideas. net/IncIdeas/BitterLesson. html. Cited by: §5.

[^45]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/S. Thrun and L. Pratt (1998) Learning to learn: introduction and overview. In Learning to learn, pp. 3–17. Cited by: §2.

[^46]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/M. Tian, Z. Wang, B. Yang, Z. Tang, K. Zhu, H. Dong, H. Li, X. Xie, G. Wang, and J. You (2026) SWE-bench mobile: can large language model agents develop industry-level mobile applications?. In arXiv preprint, External Links: Link Cited by: §1.

[^47]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/H. Trivedi, N. Balasubramanian, T. Khot, and A. Sabharwal (2023) Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. External Links: 2212.10509, Link Cited by: §1, §2.

[^48]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/C. Xiao, G. T. Hudson, and N. A. Moubayed (2024) RAR-b: reasoning as retrieval benchmark. External Links: 2404.06347, Link Cited by: §4.2.

[^49]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/Y. Xiong, S. Hu, and J. Clune (2026) Learning to continually learn via meta-learning agentic memory designs. In OpenReview, External Links: Link Cited by: §2.

[^50]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/C. Yang, X. Wang, Y. Lu, H. Liu, Q. V. Le, D. Zhou, and X. Chen (2023) Large language models as optimizers. In The Twelfth International Conference on Learning Representations, Cited by: Table 1, §1, §2.

[^51]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/H. Ye, X. He, V. Arak, H. Dong, and G. Song (2026) Meta context engineering via agentic skill evolution. arXiv preprint arXiv:2601.21557. Cited by: Table 2, §4.1, §4.1, Table 2.

[^52]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/M. Yuksekgonul, F. Bianchi, J. Boen, S. Liu, Z. Huang, C. Guestrin, and J. Zou (2024) TextGrad: automatic "differentiation" via text. External Links: 2406.07496, Link Cited by: Table 1, §1, §2.

[^53]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/M. Yuksekgonul, D. Koceja, X. Li, F. Bianchi, J. McCaleb, X. Wang, J. Kautz, Y. Choi, J. Zou, C. Guestrin, et al. (2026) Learning to discover at test time. arXiv preprint arXiv:2601.16175. Cited by: 3rd item, Table 4.

[^54]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/M. Yuksekgonul, D. Koceja, X. Li, F. Bianchi, J. McCaleb, X. Wang, J. Kautz, Y. Choi, J. Zou, C. Guestrin, and Y. Sun (2026) Learning to discover at test time. External Links: 2601.16175, Link Cited by: Table 1, §4.3.

[^55]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/A. L. Zhang, T. Kraska, and O. Khattab (2026) Recursive language models. External Links: 2512.24601, Link Cited by: §1, §2.

[^56]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/G. Zhang, H. Ren, C. Zhan, Z. Zhou, J. Wang, H. Zhu, W. Zhou, and S. Yan (2025) Memevolve: meta-evolution of agent memory systems. arXiv preprint arXiv:2512.18746. Cited by: §2.

[^57]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/J. Zhang, J. Xiang, Z. Yu, F. Teng, X. Chen, J. Chen, M. Zhuge, X. Cheng, S. Hong, J. Wang, B. Zheng, B. Liu, Y. Luo, and C. Wu (2025) AFlow: automating agentic workflow generation. External Links: 2410.10762, Link Cited by: §2.

[^58]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/Q. Zhang, C. Hu, S. Upasani, B. Ma, F. Hong, V. Kamanuru, J. Rainton, C. Wu, M. Ji, H. Li, U. Thakker, J. Zou, and K. Olukotun (2025) Agentic context engineering: evolving contexts for self-improving language models. In arXiv preprint arXiv:2510.04618, Cited by: §1, Table 2, §4.1, §4.1, Table 5.

[^59]: https://github.com/deusyu/harness-engineering/blob/858c0da6570aad32947c09c7e83fb04f46d22ebe/works/X. Zhang, J. Zhao, and Y. LeCun (2016) Character-level convolutional networks for text classification. External Links: 1509.01626, Link Cited by: 7th item.