What Is a Context Window? Tokens, Limits, Context Rot

Picture a desk that only fits so much paper. You can pile on notes, a few documents, and the reply you are drafting, but past a certain point there is no more room. That desk is a context window. So what is a context window in plain terms? It is the total amount of text an AI model can hold in mind during a single request. Everything you typed, everything it said back, and the answer it is writing right now all share that one surface.

What is a context window, measured in tokens

The desk is measured in tokens, not pages. A token is the small chunk a model reads text in: sometimes a whole word, sometimes a fragment, sometimes a single character or a piece of punctuation. A rough English rule of thumb is that one token runs about four characters, or roughly three quarters of a word, so 100 tokens land near 75 words. Treat that as an approximation. Code, rare words, and non-English text break the ratio.

Here is the part most people miss. The context window is one combined budget for input and output. The system instructions, your prompt, the chat history, and any files you uploaded all eat into it on every turn. Whatever the input spends gets subtracted from what is left for the reply. Fill the desk with paper and there is no space to write the answer.

A quick example makes it concrete. Say you paste a 40-page report into a model with a 128,000 token window. At roughly four characters per token, those pages might run 60,000 tokens. Add a few thousand tokens of system instructions and chat history, and you have used more than half the desk before the model writes a single word. There is still plenty of room here, but you can see how a couple of long documents and a chatty session start to crowd the surface. The budget is shared, and the reply has to fit in whatever is left.

How big are context windows today?

Big, and getting bigger. A few years ago a 128,000 token window felt generous. In 2026 the numbers are much larger. Here are some advertised maximums from the model makers themselves.

Those numbers look like the answer to every long-document problem. They are not. A 10 million token window sounds like a warehouse for paper. The catch is whether the model can actually read all of it well. That is where the myth falls apart.

The myth: bigger is always better

A bigger window means the model can take in more. It does not mean the model uses all of it equally well. The advertised size and the usable size are two different things, and the gap can be wide.

Start with a well-documented effect researchers call lost in the middle. When important information sits at the very start or the very end of a long input, models recall it well. Bury that same information in the middle and accuracy drops, sometimes sharply. The result is a U-shaped curve: strong at the edges, soft in the belly. This shows up even in models built specifically for long context.

There is a reason for the gap between the two tests. A needle-in-a-haystack test asks the model to find one planted fact in a long document, which is pure recall and something big windows handle well. Real work asks the model to connect facts that sit pages apart and reason across them, which is much harder. Artificial Analysis built a benchmark for exactly that case, using real documents of 10,000 to 100,000 tokens instead of synthetic filler.

The scores are humbling. As of August 2026, the leaders on that long-context reasoning benchmark score around 83 percent, which still leaves nearly one in five long-document questions wrong. Artificial Analysis also notes that the frontier models of mid-2024 scored under 50 percent on its hardest tasks. Compare that to the near-perfect numbers these same models post on short benchmarks. A large window does not guarantee the model can reason across everything inside it.

Context rot: when more text makes answers worse

Practitioners have a name for the slow decline you feel in a long session: context rot. It is an observed phenomenon rather than a single peer-reviewed result, but it lines up with what the benchmarks show. Quality can degrade as the input grows even when the window is nowhere near full. Reporting points to a few compounding causes:

• Lost in the middle. The attention gaps described above, where the model overlooks what sits between the bookends.

• Attention dilution. As the token count climbs, the model's focus spreads thinner across everything it has to weigh.

• Distractor interference. Old, similar-sounding but irrelevant text in the history pulls the answer off course.

This is the practical reason a chat that started sharp can drift after an hour. You are not imagining it. The desk filled up, the older notes are still on it, and the model is now reading through a stack of half-relevant scraps to find the one line that matters.

What this means for YOUR chat

When a normal chat gets long, two things happen. First, you pay for the whole history on every turn, because all of it is re-read each time you hit send. Second, the answers can get vaguer, repeat themselves, or forget a detail you gave it 40 messages ago. Eventually the conversation runs up against the window, and the oldest turns start to fall off the edge of the desk. That is what it means to exceed the context window: the model keeps the recent stuff and loses the early stuff.

A few habits keep long chats useful:

1. Put the important stuff at the start or the end of your message, not buried in a wall of text.

2. Start a fresh chat when the topic changes. A clean desk reads faster and answers sharper.

3. Summarize the thread so far, then carry the summary forward instead of dragging the whole transcript.

4. Save the facts you want remembered somewhere persistent, so the model does not have to hold them in the window at all.

MultiChats builds the last two of those into the app. When a thread gets heavy, it flags that the chat is getting long and offers to summarize and start fresh, so you carry the gist forward without the bloat. For the facts you never want lost, Memories and Folders keep them out of the running window and pull them back in when they matter. The summarize option and Memories come with paid plans.

Frequently asked questions

How many tokens is a context window?

It depends entirely on the model. In 2026 many frontier models advertise 1 million tokens or more, including the Gemini 3 family, and Llama 4 Scout advertises 10 million. Older or smaller models commonly sit at 128,000 tokens. Remember that this single budget covers your prompt, the chat history, any files, and the reply together.

Is a bigger context window always better?

No. A bigger window lets the model take in more, but it does not guarantee the model reasons over all of it well. Lost-in-the-middle gaps and context rot mean accuracy can fall as input grows, and long-context benchmarks show even top models slipping well below their short-context scores. Use the room when you genuinely need it, not by default.

What happens when a chat exceeds the context window?

The model cannot hold everything, so the oldest turns drop off the edge first. It keeps recent messages and the current reply, and the early details effectively vanish from its view. That is why a model can suddenly forget something you said at the start of a very long thread.

How do I keep a long chat from getting worse?

Keep key details near the start or end of your messages, start a fresh chat when the subject shifts, summarize the conversation and continue from the summary, and store must-remember facts in a memory feature rather than the live thread. Those four moves fight context rot directly.

The short version

A context window is the desk: the total tokens a model can work with in one go, shared by your input and its output. The headline sizes are huge now, but the usable space is smaller than the sticker, and quality can rot as a chat drags on. Knowing that changes how you work. Keep important details at the edges, reset when the topic turns, and lean on summaries and memory for the rest.

If you want to feel the difference between models with big windows and ones that hold a thread well, that is exactly what a multi-model app is for. See the best AI chat app comparison, or check the MultiChats plans to try 25+ models, with long-chat summaries and memory, under one subscription.