9.1 What is Context Engineering

After years of Prompt Engineering becoming the focus of applied AI, a new term has come to the forefront: Context Engineering. Today, building systems with language models is no longer just about finding the right phrasing and wording in prompts, but about answering a more macro question: What kind of context configuration is most likely to make the model produce the behavior we expect?

The so-called "context" refers to the set of tokens included when sampling a large language model (LLM). The engineering problem at hand is to optimize the utility of these tokens under the inherent constraints of the LLM, in order to stably obtain expected results. To effectively harness LLMs, it is often necessary to "think in context"—that is: at any call, examine the overall state visible to the LLM and predict the behavior this state might induce.

Figure 9.1 Prompt engineering vs Context engineering

This section will explore the emerging context engineering and provide a refined mental model for building controllable and effective agents.

Context Engineering vs. Prompt Engineering

As shown in Figure 9.1, from the perspective of leading model vendors, context engineering is the natural evolution of prompt engineering. Prompt engineering focuses on how to write and organize LLM instructions to obtain better results (such as system prompt writing and structured strategies); while context engineering is how to plan and maintain the "optimal information set (tokens)" during the inference stage, which includes not only the prompt itself, but also all other information that will enter the context window.

In the early stages of LLM engineering, prompts were often the main work, because most use cases (except daily chat) required fine-tuned prompt optimization for single-turn classification or text generation. As the name suggests, the core of prompt engineering is "how to write effective prompts", especially system prompts. However, as we begin to engineer stronger agents that work over longer time spans and across multiple inference rounds, we need strategies that can manage the entire context state—including system instructions, tools, MCP (Model Context Protocol), external data, message history, etc.

An agent running in a loop will continuously generate data that may be relevant to the next round of inference. This information must be periodically refined. Therefore, the "art and technique" of context engineering lies in identifying which content should enter the limited context window from the continuously expanding "candidate information universe".