Context Windows and Long-Context Models

The advertised context length is a marketing number and an engineering ceiling; the usable context is something smaller and harder to measure. A model can accept a million tokens, keep its perplexity stable across all of them, and still fail to answer a question whose evidence sat in the middle of the input. Two failure surfaces hide behind one number. The first is whether the model can represent and process a sequence that long at all. The second is whether it actually attends to the right part of it. Confusing the two is the most common mistake teams make when they buy "long context" and wonder why retrieval still beats it.

What sets the ceiling

Three pressures bound how long a window you can train and serve.