What is a context window?

A context window is the maximum amount of text a language model can consider at once, counted in tokens (roughly three quarters of a word each). Everything in one request has to fit: instructions, the documents you supply, the conversation and the answer. A short paper can fit whole; a long paper or a library needs retrieval to pick what goes in.

Local models often have smaller windows than the largest cloud models, which makes choosing the right passages more important.

Filling the window with barely relevant text can make answers worse, not better.