The single most useful thing to understand about a chat model: it has no memory between messages. Every time you send something, the entire conversation is re-sent. What the model "knows" is only what is in front of it at that moment.
The context window is the limit on how much can be in front of it.
What is in the window
- The system prompt — standing instructions
- The conversation so far — every previous message, yours and its own
- Anything attached — documents, code, retrieved passages
- Your current message
- Room reserved for the reply
All of it measured in tokens. That last item matters: if the window is nearly full of input, there is little room left for output, and answers get truncated.
What happens at the edge
When a conversation grows past the window, something has to go. Most applications drop the oldest turns.
That produces behaviour people misread as the model being unreliable:
- It contradicts something agreed twenty messages ago
- It re-asks for information you already gave
- It forgets a constraint you set at the start
- A long document you pasted seems partly ignored
None of this is the model changing its mind. That text is simply no longer being sent.
Bigger windows do not solve everything
Windows have grown a great deal, and it is tempting to conclude the problem is gone. Two things persist.
Position matters. Material at the very start and very end of a long context tends to be used more reliably than material buried in the middle. If something is critical, put it near your instruction rather than in the middle of a long paste.
Cost and latency scale with what you send. Every token in the window is paid for on every turn. A conversation carrying an enormous document costs that document again with each message.
Working with the limit
- Put the instruction near the end, after a long document, so it is not buried
- Restate constraints that must hold in a long conversation, rather than trusting the opening message to survive
- Start a new conversation when the topic changes — carrying twenty irrelevant turns is cost without benefit
- Summarise and continue for genuinely long work: ask for a summary of decisions so far, then begin fresh with it
Why this explains so much
A large share of "the AI is being stupid" moments are context problems. Once you picture the window — a finite space that everything must fit into, refilled from scratch on every turn — the behaviour stops looking arbitrary and starts being predictable.