Context Window

Latest update: 26/04/27

Definition

A context window is the maximum amount of text an AI model can read and hold in memory at one time – everything it can “see” when generating a response.

What Is a Context Window?

Every AI language model has a working memory limit. The context window is that limit. It covers everything the model has access to in a single session: your instructions, the conversation history, any documents you’ve pasted in, and the model’s own previous responses.

Once you exceed that limit, older content starts to fall out of scope. The model doesn’t delete it – it just can’t see it anymore. It’s as if the conversation never happened.

Context windows are measured in tokens, not words. A small context window might hold 4,000 tokens (roughly 3,000 words). The largest models today can handle well over 100,000 – enough to fit a full novel.

💡 How Does It Work?

Think of the context window like a whiteboard in a meeting room. You can write a lot on it, but it has a fixed size. Once it fills up, writing something new means erasing something old. The person in the room can only use what’s currently on the board.

When you send a message, the model reads everything on its whiteboard – your current message, all previous messages, any system instructions, and any documents – then generates a response. If the whiteboard is full, early content gets pushed off the edge and the model can no longer refer to it.

This is why long conversations sometimes feel like the AI has “forgotten” what you told it at the beginning. It hasn’t malfunctioned – it simply can’t see that far back anymore.

Why It Matters for Your Prompts

The context window shapes almost everything about how you structure a conversation or a task.

Put your most important instructions at the top of your prompt – or repeat them near the end if you’re deep into a long session. Models pay more attention to content that’s close to where they’re currently generating, and they can lose track of instructions buried far back in the thread.

Pasting in long documents eats context fast. A 10-page PDF might use 5,000–7,000 tokens before you’ve even typed a question. If your context window is 8,000 tokens, there’s almost no room left for back-and-forth.

For multi-step workflows, context management becomes a real skill. Breaking tasks into shorter sessions, summarizing earlier output before starting a new one, and keeping prompts tight all help you get more done within the limits you have.

🌐 Real-World Example

A consultant is drafting a 15-page report with an AI tool. She pastes the full brief, adds background documents, and starts writing section by section. Everything goes well for the first few sections.

By section seven, the AI starts contradicting things it said earlier. It repeats recommendations it already gave. The tone shifts.

The problem: the context window filled up. The early sections, the brief, and the background documents are no longer visible to the model. It’s generating responses based only on what fits in the remaining space.

Her fix: start a fresh session for each major section, pasting only the relevant context for that part. Less convenient – but far more reliable.

Related Terms

Token – Tokens are what fills the context window; understanding them helps you manage space.
Large Language Model (LLM) – LLMs are the models that have context windows; each model has its own limit.
Prompt – Your prompt is the first thing consuming space in the context window.
System Prompt – System prompts sit inside the context window too, often before you’ve typed a single word.
Context Caching – A technique for reusing parts of the context window across requests to save time and cost.

Encyclopedia Prompting Techniques Architecture & Technical Advanced Concepts

Frequently Asked Questions

What happens when I hit the context window limit?

Different models handle this differently. Some stop and tell you the limit has been reached. Others silently drop the oldest content and keep going. The second behavior is more common – and more confusing, because the AI will keep responding confidently even as it loses access to earlier parts of the conversation.

Does a bigger context window always mean better results?

Not necessarily. Larger context windows let you work with more material, which is genuinely useful. But models can sometimes perform worse with very long contexts because relevant information gets diluted by surrounding text. Focused, well-structured prompts tend to outperform bloated ones even when you have plenty of space.

Is the context window the same as the AI’s memory?

Not quite. The context window is temporary – it resets every session. It’s not memory in the human sense; it’s more like a scratchpad that gets wiped clean. Some tools offer persistent memory features that sit outside the context window, but the context window itself doesn’t carry over between conversations.

Why do some AI tools cost more for longer conversations?

Most AI APIs charge per token processed. Longer conversations mean more tokens in the context window – and the entire window gets re-processed with each new message. That’s why a 50-message thread costs more per response than a fresh one: the model is re-reading everything every time.

References

Anthropic – Claude Model Documentation (anthropic.com/docs) – Lists context window sizes for Claude models and how context is handled.
OpenAI – GPT-4 Technical Report (openai.com) – Covers context window specifications and performance at different context lengths.