1. What is Context Engineering
The Definition
Andrej Karpathy coined the phrase: "Context engineering is the art of filling the context window with the right information at the right time."
That single sentence contains three non-obvious requirements:
- Filling: the context window should be populated deliberately, not accidentally. Leaving it mostly empty wastes the model's capacity; leaving it chaotically full wastes your tokens and degrades output quality.
- Right information: not all information is equal. Architecture decisions are more valuable than linting preferences. Negative constraints ("never return raw SQL errors to the client") are more actionable than aspirational goals ("write clean code").
- Right time: path-scoped rules for backend code have no value when editing a frontend component. Loading everything always is the lazy approach that degrades adherence.
A Contested Origin
Karpathy's mid-2025 post is the commonly credited source, but the attribution resurfaced as a live debate at the AI Engineer conferences in 2026. Dex Horthy, founder of HumanLayer and author of "12-Factor Agents," is now described in the AI Engineer conference's own materials as having coined the term, with an earlier antecedent: the "dumb zone," the 40 to 60% band of a large context window where recall degrades and reasoning weakens, based on an analysis of roughly 100,000 developer sessions. Treat this as a genuinely open question rather than a settled fact. What matters practically is not who said it first, but that two independent lines of observation, Karpathy's framing and Horthy's session-scale data, converge on the same conclusion: mid-window content is where instruction-following breaks down first.
Prompt Engineering vs. Context Engineering
These terms are often conflated. The distinction matters:
| Dimension | Prompt Engineering | Context Engineering |
|---|---|---|
| Scope | One request | Entire session or system |
| Duration | Single interaction | Persistent across interactions |
| Effort | Per-request crafting | Upfront system design |
| Scale | Individual | Team-wide or organization-wide |
| Artifact | A prompt string | A configuration system |
Prompt engineering is about crafting the right question for one task. Context engineering is the system that ensures Claude has the right background knowledge before any task begins. You can have excellent prompts on top of poor context engineering and still get mediocre results, because the model lacks the structural understanding of your project, conventions, and constraints.
A practical analogy: prompt engineering is writing a good email to a contractor. Context engineering is the onboarding process, code style guide, architecture documentation, and team norms that ensure the contractor understands the project before reading a single email.
Context Engineering vs. Context Optimization
Both terms appear in the literature and are sometimes used interchangeably. They are not the same.
| Dimension | Context Engineering | Context Optimization |
|---|---|---|
| Core question | What information should be in context? | What is the minimum set of high-signal tokens that maximizes the outcome? |
| Goal | Completeness and correctness | Efficiency and signal density |
| Method | Identify what the model needs to know | Remove everything it does not need to know |
| Failure mode | Missing critical information | Overshooting (too much irrelevant content) |
| Output | A context system | A trimmed, high-fidelity prompt or config |
A useful mental model: context engineering answers "what to include," context optimization answers "what to cut."
In practice, you do both. The engineering pass builds the complete picture: architecture decisions, conventions, constraints. The optimization pass prunes it: removes redundancy, compresses verbose rules, archives outdated entries, path-scopes subsystem-specific content. The reduction techniques in Section 8 are the optimization pass.
Synthesis vs. reasoning
A related distinction worth naming explicitly:
- Context synthesis is stateful and iterative. It accumulates knowledge across sessions, updates when conventions change, and reflects project history. CLAUDE.md is context synthesis.
- Reasoning is ephemeral and disposable. Each inference step uses the context to produce an output, then discards the intermediate state. Claude's chain-of-thought is reasoning.
Treating reasoning artifacts (intermediate thoughts, debug traces, error outputs) as context synthesis material is a common mistake. It pollutes the context with ephemeral state and accelerates context rot. Separate what should persist (synthesis) from what should be discarded (reasoning noise).
Why It Matters
LLMs are context-window computers. The quality of output is bounded by the quality of input. This is not a soft claim. It has a hard technical basis:
- The model has no persistent memory between sessions (without explicit tooling). Every session starts from zero unless context is deliberately provided.
- The model cannot infer unstated conventions. If you want TypeScript interfaces instead of
typealiases, that must be stated. If you want errors logged before being thrown, that must be stated. - Models are sensitive to instruction placement and framing. An instruction buried in line 400 of a 500-line CLAUDE.md is less likely to be followed than one in the first 50 lines.
Teams that invest in context engineering consistently report fewer revision cycles, better adherence to conventions, and more predictable outputs. The investment is front-loaded (building the system), but the returns compound across every interaction.
A useful diagnostic reframe: most AI output failures are context failures, not model failures. When Claude generates a generic response, ignores a convention, or produces code that doesn't match your stack, the model is almost never broken. The context it received was incomplete, contradictory, or missing the right information at the right time. This reframe shifts troubleshooting from "the AI is bad at this" to "what is missing from the context?"
The Three Layers
Context engineering in Claude Code operates across three distinct layers:
| Layer | Mechanism | Scope | When Loaded |
|---|---|---|---|
| Global config | ~/.claude/CLAUDE.md | All projects | Always |
| Project config | ./CLAUDE.md + path-scoped modules | Current project | Per session |
| Session | Inline instructions, /add, flags | Current session only | Runtime |
Each layer has different tradeoffs. Global config is always-on but cannot reference project-specific details. Session instructions are flexible but ephemeral. Project config is the workhorse: structured, versioned, reviewable.
Good context engineering means putting each piece of information in the right layer, not cramming everything into one file, and not leaving critical knowledge in the session layer where it evaporates after every conversation.
Markdown files as shared persistent memory work better than a dedicated database for this purpose. A plain Markdown file that multiple agents or sessions can read and append to serves as durable, human-readable memory without the operational overhead of standing up a store. Pairing that memory with a rules file that documents project conventions (naming, code style, architectural boundaries) gives the model the same onboarding a new engineer would need; without it, output quality and adherence to project style both suffer. (Alex Gavrilescu, Devoxx, 2025. Konstantin Pavlov, 2025, reaches the same conclusion independently.)
A dedicated "research, then plan, then implement" sequence, each stage handed off through its own document, reduces rework on non-trivial tasks. The pattern: a research document capturing findings with file and line-number references, then a plan document reviewed before any code changes, then implementation against that plan. Splitting research from planning from execution keeps each stage's context focused and gives a natural checkpoint to catch a wrong assumption before it propagates into code. (Emmanuel Sciara, Dev With AI Meetup, 2026)
Static vs. Dynamic Context
The three-layer system above is static context: configuration files that are assembled before a session begins and remain stable throughout. Claude Code is primarily a static context system, which is why CLAUDE.md structure and path-scoping matter so much.
As you move toward agent workflows, a second category appears: dynamic context, assembled at inference time as the agent operates.
| Type | How assembled | Examples in Claude Code |
|---|---|---|
| Static | Before session, from files | CLAUDE.md, path-scoped modules, skills |
| Dynamic | At runtime, from tools | Tool outputs, file reads, web fetches, MCP data |
In practice, every Claude Code session uses both. The static context (your configuration) sets the behavioral envelope; the dynamic context (files Claude reads, tool results it processes) provides the specific information for each task. Context engineering covers both, but the failure modes differ: static context problems manifest as consistent convention violations; dynamic context problems manifest as Claude acting on stale or incomplete information mid-task.
A centralized context layer can reduce fragmentation while increasing the blast radius of stale, poisoned, or incorrectly related data. Require provenance, freshness, scoped read permissions, and a revocation path before treating a shared context lake as authoritative. Memory Systems documents the corresponding poisoning, staleness, and multi-writer risks. Product demonstrations that show retrieval working do not measure those failure modes.
For teams building automated pipelines and agents, Anthropic's September 2025 engineering post "Effective context engineering for AI agents" covers the dynamic side in depth.
Anthropic's April 2026 Managed Agents architecture pushes this further with a named separation: the harness and model ("brain"), the sandboxed execution layer ("hands"), and a durable session event log that lives entirely outside the context window. The harness reads from that log through a getEvents()-style interface, pulling positional slices back in on demand rather than keeping everything resident. Anthropic's stated rationale is that they cannot predict which context-management techniques future models will need, so durability belongs in the session layer while the context logic stays in the harness, which is expected to change. The companion post, "Effective harnesses for long-running agents", covers the open problem of preserving coherent progress across repeated context resets on multi-hour and multi-day tasks. See AI Ecosystem §14 for the full architecture, including the append-only event log and checkpoint/restore mechanics already documented in this guide.
Why Context Rot is Structural, Not Accidental
Transformer models attend to all tokens pairwise. That means the number of attention relationships in a context window grows as n², not n. Double the context length and you quadruple the number of relationships the model must weigh. At 200K tokens, this means billions of pairwise computations, and the model's attention becomes increasingly diffuse.
This is not a bug that future models will eliminate. It is a consequence of the architecture itself. Context rot, the progressive degradation of instruction adherence as context grows, is structurally baked in. The implication: you cannot solve context rot by relying on a larger context window. You solve it by keeping context lean and loading information just in time.
The effect has been refined, not refuted. A recurring critique through 2026 is that Needle-in-a-Haystack benchmarks measure only lexical retrieval. Once the question and the needle stop sharing obvious vocabulary (realistic, semantically ambiguous queries), degradation with length is worse than NIAH numbers suggest. One data point circulating in 2026, specific to its setup and not a general constant: Opus 4.6, used as a trajectory monitor on MonitorBench, dropped from 98.6% to 88% recall once 800K tokens of benign prior actions were prepended. Read it as evidence that the failure mode also hits monitor and classifier models, not just generation, rather than as a number to generalize past its own conditions.
Just-in-time retrieval vs. pre-loading
There are two strategies for giving Claude the information it needs:
| Strategy | Mechanism | When to use |
|---|---|---|
| Pre-loading (RAG) | Retrieve and inject all potentially relevant context before inference | Known, stable context requirements |
| Just-in-time retrieval | Retrieve context on demand, exactly when and only when needed | Dynamic, task-specific context |
Pre-loading is the familiar RAG pattern: build a retrieval index, pull relevant chunks into the prompt upfront. It works when you know in advance what information the model will need.
Just-in-time retrieval is more demanding to implement but more effective at scale: the model retrieves information dynamically as the task demands it, using tool calls, MCP servers, or file reads. Only the information needed for the current step is in context.
Claude Code's behavior reflects this pattern: CLAUDE.md loads upfront (pre-loaded, always relevant), while file contents and tool results are retrieved at inference time via read_file, glob, grep, and MCP calls. The glob and grep tools are the JIT retrieval layer. They put specific file contents into context only when a task touches those files.
In-context generation, plain RAG or a well-built prompt, often beats fine-tuning when the goal is teaching a model new knowledge. Fine-tuning bakes facts into the weights through gradient updates, but it runs into a documented failure mode called the reversal curse: a model fine-tuned on "A is B" frequently cannot answer "what is B," even though the fact is logically symmetric. Retrieval keeps the fact as text in context rather than asking the model to generalize from a weight update, which sidesteps the problem entirely. Two independent Stanford courses converge on this point when advising teams choosing between the two approaches. (Stanford CS25, 2026; CS230 Lecture 8, 2025)
Memory tool (beta)
As of Claude Sonnet 4.5, Anthropic released a Memory tool in public beta. It allows Claude to store and retrieve persistent facts across sessions without manual CLAUDE.md management. The tool maintains a structured knowledge store that Claude queries when relevant context is needed.
This is distinct from CLAUDE.md: CLAUDE.md is static configuration (always loaded), while the Memory tool is dynamic retrieval (queried on demand). For teams building agents, the Memory tool reduces the need to manually encode knowledge in config files.
Chain-of-thought in long tasks
Chain-of-thought (CoT) prompting improves model reasoning on isolated tasks. However, Anthropic's engineering data shows it can hurt performance in long agentic tasks. The mechanism: CoT generates additional tokens, which extend context length, which accelerates context rot for subsequent steps. On tasks spanning 20+ tool calls, this effect is measurable.
The practical rule: use CoT for complex isolated reasoning steps, not as a blanket strategy for agentic workflows. In long runs, prefer compressed intermediate outputs over extended reasoning traces.