Context window
Also: context length
What is Context window?
The context window is the maximum amount of text, measured in tokens, that a language model can consider in a single request, covering the instructions, supplied documents, conversation history and its own answer.
What Context window means
The context window is the model's working memory for one call. Everything the model can use must fit inside it: the system prompt, retrieved documents, prior conversation turns, tool results and the response it generates. Sizes range from a few thousand tokens to over a million depending on the model, and the limit is hard: exceed it and the request fails or content is silently cut.
In practice, how you fill the window matters as much as its size. Models attend unevenly across long inputs, so critical instructions and evidence should be placed deliberately, retrieval should return the few most relevant chunks rather than whole manuals, and long conversations need summarisation or truncation strategies. Every token in the window is also billed, so a large window used carelessly is a cost problem.
The context window is not persistent memory. Nothing survives between requests unless your application re-sends it, which is why agent memory and retrieval exist. And a bigger window is not a substitute for retrieval: stuffing everything in raises cost and latency and often lowers accuracy compared with sending only what is relevant.
Who it really matters to
- CTO / Head of Engineering: window size shapes the retrieval design, chunking strategy and conversation-management logic.
- CFO: tokens in the window are the largest driver of per-request cost, so context discipline is cost control.
- Product manager: decides what the assistant can "see" at once, such as how much of a customer's history or a document.
- Data lead: chunk sizes and document structure need to be designed around what fits and what the model attends to.
Why it exists
The context window exists as a hard architectural limit of transformer models, and the term matters to buyers because it explains many product constraints: why an assistant forgets earlier turns, why a whole contract cannot always be pasted in, why costs rise with conversation length. Larger windows have eased the constraint but created a new trade-off: sending more is easy, but it costs more, is slower and does not reliably improve answers. Good engineering treats the window as a budget to be spent on the most relevant material, not as a bin to fill.
Where it is applied
- A contract-review assistant that chunks long agreements and retrieves only the clauses relevant to each question.
- A support copilot that summarises older conversation turns so a long ticket thread still fits with room for the answer.
- A clinical summarisation tool that selects the relevant sections of a patient record rather than the whole history.
- A code-modernisation pipeline that feeds a legacy module plus its dependencies within a fixed token budget.
- A logistics exception agent that includes current shipment state and policy snippets, not the full manual, per request.
Is Context window a skill?
ConceptA concept engineers design around rather than a skill on its own. Eazyware's retrieval and knowledge engineering and LLM applications work sets chunking, retrieval and conversation-management strategy to the window and the cost budget.
Eazyware service that covers it: LLM Application Development. Starting prices are on the pricing page.
Frequently asked questions
If models now have huge context windows, do we still need RAG?
Usually yes. A large window lets you send more, but every token costs money and time, and models attend less reliably to very long inputs. Retrieval that sends the few relevant passages is cheaper, faster and often more accurate.
Why does our assistant forget what was said earlier?
Because each request only contains what your application re-sends. Once history exceeds the window or the budget you set, older turns are dropped or summarised. Persistent memory has to be engineered outside the model.