Context Engineering for Long-Horizon Agents: Managing the Only Resource That Runs Out

In March 2025, METR measured how long a task a frontier model could finish on its own and found the answer doubling every seven months. By early 2026 the doubling time had compressed to roughly 4.3 months, and the best agents were clearing software tasks that take a human expert the better part of a working day (METR, 2025, Measuring AI Ability to Complete Long Tasks). An agent that runs for hours sounds like a scaling story about smarter models. It is really a story about a resource that does not scale: the context window. A model that can act for eight hours still has to decide, at every single step, what to keep in front of itself, because the window that holds its working memory fills long before the task ends.

Managing that window used to be an afterthought bolted onto prompt engineering. Over the past year it has become the thing production teams spend most of their time on, and it acquired a name. Andrej Karpathy popularized "context engineering" in mid-2025 to describe the discipline of filling the context window with exactly the right tokens, and by mid-2026 it was the dominant topic in agent development, with Anthropic, Chroma, LangChain, and a wave of arXiv papers all converging on the same problem from different angles.

Why this matters: The model is not the bottleneck for most agent failures anymore. The bottleneck is what you put in front of the model, and long-horizon agents put too much in front of it. Context engineering is how you keep an agent coherent across a thousand steps instead of watching it drift, loop, and forget its own goal by step three hundred.

TL;DR

• A long-horizon agent accumulates tool outputs, its own reasoning, and message history until the context window is full or, worse, until it is full of noise. The window is a finite budget, and spending it well is the core engineering task.
• Bigger windows do not fix this. Model accuracy degrades non-uniformly as input grows ("context rot"), and the degradation hits cliffs rather than sloping gently (Chroma, 2025, Context Rot).
• Long contexts fail in four distinct ways: poisoning (a false fact takes root), distraction (the model imitates its own history instead of reasoning), confusion (irrelevant tokens pull the answer off course), and clash (accumulated context contradicts itself) (Breunig, 2025).
• The mitigation toolkit is small and general: compaction (summarize and restart), structured note-taking (offload memory to disk), sub-agent isolation (spend a fresh window on a sub-task, return a digest), just-in-time retrieval (load data by reference, not by value), and tool-result clearing.
• These techniques are not free. Compaction can silently drop safety constraints or the one detail that mattered; sub-agents multiply cost and coordination complexity. Knowing what each one throws away is the skill.
• The frame that helps: treat context as a cache you actively evict from, not a transcript you passively append to.

At a Glance

A long-horizon agent is a loop. It reads its context, decides on an action, calls a tool, and the result is appended back into the context. Left alone, that loop is a monotonically growing transcript. Context engineering inserts a management step that decides what survives each turn.

flowchart LR
  U[Goal + instructions] --> CTX[Context window]
  CTX --> M[Model step]
  M -->|action| T[Tool call]
  T -->|observation| CM[Context manager]
  M -->|reasoning| CM
  CM -->|keep / drop / summarize| CTX
  CM -.->|offload| MEM[(External memory)]
  MEM -.->|load by reference| CTX
  CTX --> OUT[Task result]
  class U,MEM blue
  class M purple
  class T slate
  class CM amber
  class CTX,OUT teal
  classDef blue fill:#1e40af,stroke:#3b82f6,stroke-width:1px,color:#fff
  classDef purple fill:#6d28d9,stroke:#a78bfa,stroke-width:1px,color:#fff
  classDef teal fill:#0e7490,stroke:#22d3ee,stroke-width:1px,color:#fff
  classDef amber fill:#b45309,stroke:#fbbf24,stroke-width:1px,color:#fff
  classDef slate fill:#334155,stroke:#64748b,stroke-width:1px,color:#e2e8f0

The dashed edges are the whole game. Without them you have a chatbot with tools. With them you have an agent that can run past the point where its raw history would overflow.

[IMAGE: side-by-side timeline of two agent runs on the same 400-step task, one with a naively growing transcript that overflows and derails around step 180, one with active context management that completes, token count plotted on the y-axis]

Before context engineering

The context window started as a number you rarely touched. The original Transformer trained on sentence pairs of a few dozen tokens (Vaswani et al., 2017, Attention Is All You Need, arXiv:1706.03762). GPT-3 shipped a 2,048-token window, and the entire prompt-engineering craft of 2020 to 2022 was about fitting a task into that space: pick the best few-shot examples, phrase the instruction tightly, and hope the answer arrived before you ran out of room.

Retrieval-augmented generation was the first real acknowledgment that the window was a budget to be managed rather than a box to be filled. Instead of stuffing a knowledge base into the prompt, you retrieved the few passages that mattered and inserted those (Lewis et al., 2020, Retrieval-Augmented Generation, arXiv:2005.11401). Then windows grew, fast, and a comforting myth grew with them: a million-token context would make retrieval and management obsolete, because you could simply put everything in.

Agents killed that myth. A single-shot question answered from a large document is one thing; an agent that reasons, acts, observes, and repeats for hundreds of turns generates its own context, and it generates a lot of it. Each tool call can return thousands of tokens. Each reasoning step adds more. The window that felt infinite for a document fills in an afternoon of autonomous work, and the model's behavior degrades well before the hard limit.

timeline
  title From prompt engineering to context engineering
  2020 : GPT-3, 2048-token window : prompts hand-tuned to fit
  2020 : RAG : retrieve the few passages that matter
  2023 : 100k+ windows : "just put everything in"
  2023 : Lost in the Middle : recall sags in the center of long inputs
  2025 : Context rot quantified : accuracy degrades non-uniformly with length
  2025 : "Context engineering" named : curate the token budget as a discipline
  2026 : Long-horizon agents : compaction and sub-agents become default

How context engineering actually works

Context engineering is the practice of deciding, continuously, which tokens occupy the model's window during inference. Anthropic frames it as the natural successor to prompt engineering: prompt engineering optimizes the words you write once, context engineering optimizes the entire context state (system instructions, tools, retrieved data, and the growing message history) across many turns (Anthropic, 2025, Effective context engineering for AI agents). The shift is from authoring a static prompt to running a live memory policy.

Why the window is not just a size limit

The naive model of a context window is a bucket: it holds N tokens, and trouble starts only when you exceed N. The real behavior is worse. Chroma's controlled study across 18 models (including GPT-4.1, Claude 4, and Gemini 2.5) showed that accuracy on even simple retrieval tasks falls as input length grows, and it falls unevenly: some models are fine at 32k tokens and collapse at 64k, others hold and then drop off a cliff (Chroma, 2025, Context Rot). Anthropic points to the mechanism: attention creates \(n^2\) pairwise relationships across \(n\) tokens, and a model trained mostly on shorter sequences has thinner experience with genuinely long-range dependencies, so its attention gets spread thin.

The practical consequence is a reframe. You are not trying to stay under a hard cap. You are trying to keep the window in the region where the model still reasons well, which is often a fraction of the advertised length. Tokens are not free even when they fit.

The four ways long contexts fail

Drew Breunig's taxonomy is the sharpest tool for reasoning about agent failure, because it names failure modes that a raw token count hides (Breunig, 2025, How Long Contexts Fail).

• Context poisoning. A hallucination or an error enters the context and is never removed, so every subsequent step treats the false fact as established and builds on it. In a long agent run, one bad tool observation can compound for hundreds of turns.
• Context distraction. As the history grows, the model starts leaning on that history instead of its trained knowledge, reproducing the pattern of its past actions rather than forming a new plan. Agents in this state loop: they repeat a failing action because the context is full of that action.
• Context confusion. Irrelevant material in the window degrades the answer because the model tries to use what it was given. A large, mostly-unused tool set is a classic source; the model reaches for a tool that is present but wrong.
• Context clash. Information accumulated across turns starts to contradict itself. The "Lost in Conversation" study made this concrete: splitting a single well-specified prompt into sequential turns caused an average 39% quality drop across six tasks, because early, incomplete attempts stayed in the context and dragged down the final answer (Laban et al., 2025, LLMs Get Lost in Multi-Turn Conversation, arXiv:2505.06120).

flowchart TD
  G[Growing context] --> P[Poisoning: false fact takes root]
  G --> D[Distraction: imitates own history]
  G --> C[Confusion: irrelevant tokens mislead]
  G --> X[Clash: context contradicts itself]
  P --> F[Agent derails]
  D --> F
  C --> F
  X --> F
  F --> R[Context engineering: keep the window small and clean]
  class G,P,D,C,X rose
  class F slate
  class R emerald
  classDef rose fill:#be123c,stroke:#fb7185,stroke-width:1px,color:#fff
  classDef slate fill:#334155,stroke:#64748b,stroke-width:1px,color:#e2e8f0
  classDef emerald fill:#047857,stroke:#34d399,stroke-width:1px,color:#fff

These four modes share a cause and a cure. The cause is unmanaged accumulation. The cure is a policy that actively removes, compresses, or relocates tokens before they do damage.

Compaction: summarize and restart

Compaction is the dominant technique in current agent frameworks. When the context approaches its limit, the agent hands the transcript to a model, asks for a summary that preserves the load-bearing details, and restarts a fresh window seeded with that summary plus the most recent turns (Anthropic, 2025). The window resets from, say, 180k tokens back down to 15k, and the agent keeps going.

The design tension is recall versus precision. Anthropic's guidance is to tune first for recall (make sure the summary keeps everything that might matter, even at the cost of length) and only then for precision (trim the redundant). A summary that is too aggressive drops the decision the agent made forty steps ago that it needs at step four hundred; a summary that is too timid barely buys any room.

The frontier here is deciding when to compact. Fixed-interval schedules ("summarize every 50k tokens") are simple but blunt, because they can fire in the middle of a derivation, discarding working state the agent is about to use. Recent work lets the model itself trigger compaction: SelfCompact gives the agent a compaction tool plus a lightweight rubric for when to fire it (near task completion, on convergence) and when to hold off (mid-derivation, when stuck), reporting up to 18.1-point gains on math benchmarks and 30 to 70% lower token cost than fixed-interval summarization across seven models (Li et al., 2026, Self-Compacting Language Model Agents, arXiv:2606.23525). The paper's own framing is telling: unprompted models cannot reliably sense when their context is rotting, but a small rubric closes the gap.

Structured note-taking: offload to memory

Compaction compresses what is in the window. Note-taking moves it out entirely. The agent writes durable facts, decisions, and progress to an external store (a file, a scratchpad, a database) and reads them back on demand, so the window holds a pointer instead of the payload. Anthropic's example is an agent playing Pokemon that keeps precise tallies across thousands of game steps by writing them to a note rather than holding them in context (Anthropic, 2025). The window stays small; the memory grows on disk, which is cheap.

The connection to human cognition is not decoration. You do not hold a project's full history in working memory; you keep a lightweight index and pull the relevant page when you need it. That is exactly the just-in-time pattern below.

Just-in-time retrieval: load by reference

Instead of pre-loading everything an agent might need, give it lightweight identifiers (file paths, query handles, URLs) and tools to expand them at runtime. The agent loads a file's contents only when it decides to read that file, and the contents leave the window when they are no longer needed. This inverts the RAG default of retrieving up front; here retrieval is an action the agent takes mid-task, so the window holds only what the current step requires.

Sub-agent isolation: spend a fresh window, return a digest

The most structural technique is to not grow one window at all. An orchestrator holds the high-level plan and delegates a narrowly scoped sub-task to a sub-agent that runs in its own clean context, does the work (which may itself consume a large window), and returns only a condensed result, typically 1,000 to 2,000 tokens (Anthropic, 2025). The orchestrator's window accumulates digests, not raw traces, so it stays legible even when the total work is enormous. This is context engineering as an architecture rather than a per-turn policy.

Seeing it in motion

The orchestrator-plus-sub-agent pattern is easiest to see as a sequence. The orchestrator never sees the sub-agent's tool spam; it sees a summary.

sequenceDiagram
  participant O as Orchestrator
  participant S as Sub-agent
  participant T as Tools
  participant Mem as Memory store
  O->>Mem: read plan + progress note
  O->>S: delegate scoped sub-task
  S->>T: many tool calls (fresh window)
  T-->>S: raw observations (large)
  S->>S: reason over its own context
  S-->>O: return 1-2k token digest
  O->>Mem: write updated progress note
  Note over O,Mem: orchestrator window holds digests, not raw traces
  O->>O: decide next sub-task or finish

Compaction, by contrast, is a decision made inside a single agent's loop. The policy is a small state machine: keep appending until a trigger fires, then either compact or offload, then continue.

stateDiagram-v2
  [*] --> Act
  Act --> Observe: call tool
  Observe --> Append: add result to window
  Append --> Check: budget check
  Check --> Act: under threshold
  Check --> Offload: durable fact
  Check --> Compact: near limit
  Offload --> Act: pointer stays, payload to memory
  Compact --> Act: summarize + reset window
  Act --> [*]: goal met

When to take which branch is the crux. A durable fact (a decision, a result, a constraint) should be offloaded so it survives every future compaction. Transient bulk (a raw file dump the agent has already parsed) should be cleared. Only when neither applies and the window is genuinely full do you pay for a full compaction pass.

The architecture of a context-managed runtime

Put the pieces together and the runtime looks like a hub-and-spoke system. A context manager sits between the model and everything that wants to write into its window, arbitrating what gets in and what gets evicted.

graph TD
  ORCH[Orchestrator model]
  CTXMGR[Context manager]
  MEM[(Memory store)]
  SUB[Sub-agent pool]
  TOOLS[Tool layer]
  RET[Retrieval by reference]
  ORCH --> CTXMGR
  CTXMGR --> MEM
  CTXMGR --> RET
  ORCH --> SUB
  SUB --> TOOLS
  SUB --> CTXMGR
  TOOLS --> CTXMGR
  RET --> MEM
  class ORCH purple
  class SUB purple
  class CTXMGR amber
  class MEM blue
  class TOOLS slate
  class RET teal
  classDef purple fill:#6d28d9,stroke:#a78bfa,stroke-width:1px,color:#fff
  classDef amber fill:#b45309,stroke:#fbbf24,stroke-width:1px,color:#fff
  classDef blue fill:#1e40af,stroke:#3b82f6,stroke-width:1px,color:#fff
  classDef slate fill:#334155,stroke:#64748b,stroke-width:1px,color:#e2e8f0
  classDef teal fill:#0e7490,stroke:#22d3ee,stroke-width:1px,color:#fff

The context manager is the amber node for a reason: it is where cost and risk concentrate. Every eviction decision is a bet that the evicted tokens will not be needed, and every compaction is a lossy compression of the agent's own memory.

Watch it run

The loop below is the article's central mechanism in motion, exported from the companion diagram (context-engineering-long-horizon-agents.drawio). Arrows animate in the direction tokens flow, the compaction self-loop on the context manager shows the window folding back on itself, and the dashed feedback edge from memory shows facts being reloaded by reference.

By the Numbers

The numbers that matter for context engineering are not model sizes; they are the shapes of degradation and the cost of the fixes.

Two of these deserve emphasis. The 39% multi-turn drop is not a long-context result; those conversations were short. It is a context-hygiene result: the mere presence of the agent's own earlier, wrong attempts poisoned the final answer. And the METR doubling time is the reason all of this is urgent rather than academic. Task horizons that double roughly twice a year mean the window pressure gets worse on a predictable clock.

The complexity math also cuts against the naive fix. Doubling the window quadruples attention cost, so "just use a bigger context" is the most expensive possible answer to a problem that compaction solves for the price of one extra model call.

A Concrete Example

Trace a coding agent asked to fix a failing integration test in a mid-size repository. Budget the window at a soft limit of 150k tokens, where the team has measured quality starting to sag.

Steps 1 to 20. The agent greps the codebase, opens six files, and runs the test suite. The failing test's stack trace is 4k tokens; the six files are 40k; the agent's reasoning adds another 8k. Window: ~55k tokens. Everything is still coherent.

Step 21, offload. The agent identifies the root cause: a timezone assumption in billing/invoice.py. That is a durable fact. Instead of trusting it to survive in the transcript, the agent writes a progress note to disk: root_cause: naive datetime in invoice.py:88, expected tz-aware. The note costs 30 tokens in the window; the reasoning that produced it can now be dropped later without losing the conclusion.

Steps 22 to 60. The agent edits the file, re-runs the suite, and hits a second, unrelated failure in a caching test. It reads three more files and their fixtures. Two of those file dumps, 22k tokens, were needed only to confirm the fix does not touch caching. Window climbs to ~140k, nearing the limit. Worse, the transcript now contains the agent's first, abandoned hypothesis (a currency-rounding theory it disproved at step 34). That dead hypothesis is exactly the "clash" fuel from the Lost in Conversation study.

Step 61, compaction. The budget check fires. The context manager summarizes: it keeps the confirmed root cause, the successful edit, and the current caching-test failure at high fidelity (recall first), and it drops the disproven currency theory and the two file dumps that are no longer relevant (precision second). The window resets from 140k to about 18k: the progress note, the summary, and the last few turns.

Steps 62 to 90. Running in the compacted window, the agent fixes the caching test cleanly. Because the disproven theory is gone, it never revisits it; because the root-cause note survived compaction, it never re-derives the timezone bug. It runs the full suite green and reports.

Now replay the same run with no context management. By step 61 the window holds the stack trace, all nine files, both live and dead hypotheses, and 40 steps of reasoning, roughly 180k tokens of mostly-stale content. The agent, distracted by its own history, re-opens invoice.py "to check," re-runs a theory it already disproved, and burns twenty steps going in a circle before either overflowing or timing out. Same model, same task, opposite outcome. The difference is entirely in what stayed in the window.

Where It Breaks

Context engineering is a set of lossy operations, and each one fails in a specific way.

Compaction is silent forgetting. A summary is a guess about what matters later. When the guess is wrong, the failure is invisible: the agent does not error, it simply proceeds without the dropped fact and produces a confidently wrong result. The most unsettling version is a safety one. Recent work argues that compaction can quietly erase safety constraints that were established early in a run, because a summary optimized for task progress does not know to preserve a guardrail stated three hundred steps ago (Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents, arXiv:2606.22528). A constraint is exactly the kind of thing a progress-focused summarizer treats as noise.

Note-taking rots too. External memory is only as good as its consistency. An agent that writes a note at step 20 and a contradicting note at step 200 has manufactured its own context clash, now durable on disk. Memory needs its own hygiene: updates, not just appends.

Sub-agent digests hide the wrong thing. A 1,500-token digest is a compression of a large trace, and compression chosen by the sub-agent may omit the detail the orchestrator most needed. The orchestrator cannot ask about what it never saw, so a subtle omission surfaces only as a downstream failure with no obvious cause.

Just-in-time retrieval can starve the agent. If the agent misjudges what it needs and does not retrieve it, it reasons from a gap. This trades the confusion failure (too much context) for a different one (too little), and tuning the balance is task-specific.

The eval problem underlies all of it. Needle-in-a-haystack tests, where one planted fact is retrieved from filler, flatter every one of these techniques, because real agent tasks require integrating many facts across a long, self-generated, partly-contradictory history. A context strategy that aces NIAH can still fall over on a real 400-step run. Measuring context engineering honestly means measuring it on long agent trajectories, which the field is only starting to do (Slipstream: Trajectory-Grounded Compaction Validation for Long-Horizon Agents, arXiv:2605.08580).

Alternative Designs

Not everyone manages context the same way, and the disagreements are real.

The honest summary is that no single strategy dominates. Compaction and memory are complements, not rivals: you offload the durable facts so that compaction can be aggressive about everything else. Multi-agent isolation is the heaviest hammer, worth its coordination cost only when a task genuinely decomposes; applied to an indivisible task it just adds latency and a lossy digest boundary. The teams that do this well tend to layer the cheap techniques (note-taking, tool clearing) as defaults and reserve compaction and sub-agents for when the budget check actually fires.

How It Is Used in Practice

Context engineering left the blog posts and entered shipping products over the past year. Agent frameworks now treat compaction as a built-in rather than a user's problem, and the pattern of an orchestrator delegating to isolated sub-agents is standard in production coding and research agents. Anthropic ships context-engineering guidance and tooling (compaction, memory, tool clearing) as first-class parts of its agent platform, a signal that it considers the discipline core rather than advanced (Anthropic, 2025).

The industrial scale is arriving too. At GTC Taipei in June 2026, Foxconn described MoMClaw, a manufacturing operations system built on Nvidia's Factory Operations blueprint that coordinates hundreds of specialized agents (quality, logistics, safety) under a central orchestrator, exactly the isolate-and-digest architecture applied to a factory floor (Nvidia, 2026). Whether the reported operational gains hold up under independent measurement is an open question; the architecture, though, is the same one this article describes, which is the point. The context-management patterns that started as tricks for keeping a coding agent coherent are now the reference design for coordinating fleets of agents.

The operational reality for teams is unglamorous. Context engineering shows up as budget dashboards (tokens per run, compaction frequency), as regression tests that replay long trajectories to catch a compaction that started dropping the wrong thing, and as on-call incidents where an agent looped because a stale note poisoned its plan. It is infrastructure work, and it is where a growing share of agent-engineering effort now goes.

Insights Worth Remembering

• The context window is the one resource a long-horizon agent cannot scale its way out of. Smarter models make the task longer, which makes the window pressure worse, not better.
• A bigger window is a bigger desk, not a better memory. Past a model-specific point, extra tokens degrade the answer instead of improving it.
• Most agent "reasoning failures" are context failures in disguise. Before you blame the model, look at what was in its window when it went wrong.
• Treat context as a cache you evict from, not a transcript you append to. The default of "keep everything" is a bug, not a safe baseline.
• Offload durable facts, compact transient bulk, and never let a disproven hypothesis linger; the dead theory in the window is what makes an agent loop.
• Every context operation is lossy. The skill is not applying the techniques; it is knowing precisely what each one throws away.
• Compaction that optimizes for progress will happily discard a safety constraint. If a rule must survive, it belongs in durable memory, not in the summarizable transcript.

Open Questions

What is established: long contexts degrade non-uniformly, multi-turn accumulation hurts quality, and compaction plus isolation measurably extend how far an agent can run. Those are measured results.

What is still open is largely about control and evaluation. Can a model reliably learn when to compact rather than following a fixed schedule? SelfCompact suggests a rubric helps, but whether models can internalize that judgment without hand-written rules is unresolved (Li et al., 2026). How do we evaluate a context strategy without a good long-trajectory benchmark, given that needle-in-a-haystack tests systematically overstate performance? Trajectory-grounded validation is an early attempt, not a settled answer (arXiv:2605.08580). And the safety question is genuinely unsettled: if compaction can erase constraints, what is the right way to make certain invariants non-compactable without bloating every window with a growing list of rules (arXiv:2606.22528)?

The speculative but plausible near-term direction is that context management stops being application code and becomes a runtime layer, a "context OS" that routes, compresses, and persists on the agent's behalf, the way an operating system manages physical memory a process never sees directly. The early routing systems point that way, but calling it inevitable would be getting ahead of the evidence.

Sources and Further Reading

Foundational and primary sources

• Anthropic, 2025, Effective context engineering for AI agents - the definitional treatment: context as a finite resource, compaction, note-taking, sub-agents, just-in-time retrieval.
• Breunig, 2025, How Long Contexts Fail (and How to Fix Them) - the four failure modes (poisoning, distraction, confusion, clash).
• Chroma Research, 2025, Context Rot: How Increasing Input Tokens Impacts LLM Performance - controlled measurement of non-uniform degradation across 18 models.
• Laban et al., 2025, LLMs Get Lost in Multi-Turn Conversation, arXiv:2505.06120 - the 39% multi-turn drop and its decomposition into aptitude vs reliability loss.
• METR, 2025, Measuring AI Ability to Complete Long Tasks - the task-length time-horizon methodology and doubling trend.

Important follow-up work

• Li et al., 2026, Self-Compacting Language Model Agents, arXiv:2606.23525 - model-triggered compaction with a rubric; token-cost and accuracy gains.
• Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents, arXiv:2606.22528 - the safety failure mode of compaction.
• Slipstream: Trajectory-Grounded Compaction Validation for Long-Horizon Agents, arXiv:2605.08580 - evaluating compaction on real trajectories rather than synthetic haystacks.
• AgentSwing: Adaptive Parallel Context Management Routing for Long-Horizon Web Agents, arXiv:2603.27490 - context routing as a runtime layer.

Technical blogs and tooling

• Anthropic Cookbook, Context engineering: memory, compaction, and tool clearing - runnable patterns for the techniques above.
• Willison, 2025, How to Fix Your Context - a practitioner's synthesis of the failure modes and fixes.
• LangChain, How to Fix Your Context - the write / select / compress / isolate framing as code.

Background

• Vaswani et al., 2017, Attention Is All You Need, arXiv:1706.03762 - the \(O(n^2)\) attention cost that makes long windows expensive.
• Lewis et al., 2020, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, arXiv:2005.11401 - retrieval as the first acknowledgment that the window is a budget.
• Liu et al., 2023, Lost in the Middle, arXiv:2307.03172 - positional recall loss in long inputs, the precursor to context rot.