Context Engineering

Prompt engineering asks how to word a prompt. Context engineering asks a bigger question: across a long, multi-step task, what should be in the model's window right now? It is the practice of curating and maintaining the optimal set of tokens during inference, spanning system instructions, tool definitions, retrieved data, and the growing message history (Anthropic, 2025, Effective context engineering for AI agents). The term, popularized by Andrej Karpathy in mid-2025, named something practitioners had been doing ad hoc, and by 2026 it was the dominant topic in agent development.

Why a bigger window did not end the problem

The comforting assumption was that million-token windows would make context management obsolete: just put everything in. Agents broke that. An agent reasons, calls a tool, reads the result, and repeats for hundreds of steps, generating its own context as it goes. Tool outputs alone can be thousands of tokens each. The window that felt infinite for a document fills in an afternoon of autonomous work, and model quality degrades well before the hard limit is reached. Context is a finite budget, not a box to fill.

The core techniques

Context engineering is a small, general toolkit for keeping the window small and clean:

• Compaction. When the window nears its limit, summarize the trace and restart from the summary plus the most recent turns. Tune for recall first (keep everything that might matter), then precision.
• Structured note-taking. Write durable facts and decisions to external storage and keep only a pointer in the window, so memory grows on cheap disk instead of in expensive context.
• Sub-agent isolation. Delegate a scoped sub-task to a sub-agent with its own fresh window; it does the heavy work and returns a compact digest (roughly 1,000 to 2,000 tokens), so the orchestrator's window holds summaries, not raw traces.
• Just-in-time retrieval. Hold lightweight identifiers (file paths, query handles) and load the actual data only when a step needs it, then let it leave the window.
• Tool-result clearing. Once a tool output has been used, drop it from the history rather than carrying it forever.

grow the window -> budget check -> keep / drop / offload / compact -> continue

The frame that helps

Treat context as a cache you actively evict from, not a transcript you passively append to. The default of "keep everything" is a bug, because every stale token both costs money (attention is quadratic in sequence length) and risks pulling the model off course. A disproven hypothesis left in the window is what makes an agent loop on a bad plan.

What it is not

Context engineering is not retrieval-augmented generation, though it uses retrieval. RAG front-loads relevant passages before generation; context engineering governs the whole context state continuously, including the model's own outputs and tool results, across an entire run. RAG is one tool inside the discipline, not a substitute for it.

Why it is now core, not advanced

As agent task horizons lengthen, the window becomes the binding constraint on how far an agent can go before it drifts, loops, or forgets its goal. Frameworks now ship compaction and sub-agent orchestration as built-ins, and the same isolate-and-digest pattern scales up to coordinating fleets of agents. The lever that decides whether a long-running agent stays coherent is not the model; it is what you keep in front of it.

Further reading

• Anthropic, 2025, Effective context engineering for AI agents - the definitional treatment of the techniques above.
• Breunig, 2025, How Long Contexts Fail - the failure modes context engineering exists to prevent.