Why does a model forget what you said earlier?
A context window is the token capacity a model runtime allows it to process at once. A base model does not retain conversational state in its weights between turns; a surrounding system may resend, summarize, or select prior messages. Text beyond the active context may be omitted.
The model has no memory between turns
A base language model processes the tokens supplied for an inference request; ordinary inference does not update its weights. Conversation state, if retained, is usually managed separately by surrounding software. Ask a second question without including prior context and the model does not receive that exchange.
A conversation can feel continuous because surrounding software supplies prior messages as context. Some systems resend a transcript, while others select or summarize earlier turns to fit the active window. The model uses the text it receives; it does not retrieve every past exchange from its weights.
Memory is measured in tokens, and a token is not a word
A token is the unit a tokenizer produces: a short piece drawn from a fixed vocabulary that was fixed when the model was built, often a common word, a word fragment, a space, or a mark of punctuation. The split is not judged fresh each time. The same sentence always breaks into the same tokens for a given model, though a different model, with a different vocabulary, can slice the identical sentence differently. For ordinary English prose the usual rule of thumb is around 750 words per 1,000 tokens. Code, names, numbers and other writing systems can run much denser, so the same page of text can cost very different amounts.
The context window is the configured token limit for the model’s input and output in a given system. A long input may leave less room for a reply, though systems can reserve space, truncate, or manage context in other ways. Published limits vary widely, so “how much can it remember” has no single answer.
Why does a longer memory cost more?
In standard full self-attention, each token can be compared with other tokens in the sequence, so attention work can grow roughly with the square of sequence length. Architectures and inference optimizations differ, but longer inputs generally require more computation and memory.
Generating a reply adds a second cost. To avoid recomputing the whole transcript for every new word, intermediate values for each token are held in memory, in what is usually called the KV cache. It grows with the length of the conversation, and on hardware where memory is already spent holding the model itself, that growth is what runs out first.
What happens when a conversation outgrows the window
When a conversation exceeds the context available to a request, the surrounding system may drop older turns, summarize them, split the work, or use another context-management strategy. The model answers from the information supplied for that step, so omitted context may not be apparent in its response.
Some models and tasks show reduced recall for details placed in the middle of long inputs, even when those inputs fit. Results vary by model and task. If inaccurate content remains in supplied conversation history, it may influence later answers unless corrected or omitted.
Remembering between sessions is retrieval, not memory
Features that let an assistant recall you across days do not change the model. They write notes to a store outside it, then pick some of those notes and place them in the window ahead of your next message. It is the same mechanism as the transcript, with a search step in front of it.
The weights themselves do not move while the model is running. Learning happens in training, a separate and far more expensive process; using a model does not update it. So memory of every kind comes down to what text can be afforded on the way in, and the window is the limit on that.
Between requests, systems may resend or summarize text; ordinary inference does not store conversation history in the base model’s weights.