What is Context window?
The context window is the amount of text a language model can process per call, measured in tokens. Everything counts towards it: the system prompt, the conversation so far, attached documents and the answer being written. Once the window is full, something drops off the front or the call fails. A model has no memory between two calls, only this window - anything a system should remember has to be placed back into it next time.
Two practical consequences follow. First, context costs money and time: every call pays for the entire contents of the window, not just the new question. A chat that has been running for two hours is considerably more expensive per answer than the first turn. Second, a larger window is no substitute for selection. Tipping in 200 pages does not buy the best answer, it buys the most expensive one.
The better strategy is selection over volume: load only the relevant passages through retrieval, summarise long histories instead of carrying them wholesale, and put important rules at the start or the end rather than the middle. This is precisely why tasks that break into clearly separated steps are often cheaper and steadier in a fixed workflow than in an agent that drags its whole history along.
Why does Context window matter?
That a large window is not automatically used well has been measured: the study "Lost in the Middle" by Nelson Liu and colleagues shows a U-shaped curve - models retrieve information reliably at the beginning and the end of the context and markedly less well in the middle, with overall accuracy falling as the context grows longer. The position of your most important rule is therefore not a formatting detail.
Context window in practice
- 01A chatbot is given the three relevant sections instead of the entire manual, and answers faster, cheaper and more precisely as a result.
- 02A team notices that answers drift away from the style guide in long chats, and starts a fresh session with the same rules for each article instead of carrying on.
- 03An automation run condenses the history into five sentences after every step, so the window does not fill up with stale intermediate states.
