What is a context window?

A context window is the maximum amount of text a model can consider at once, measured in tokens. Windows on the current catalog run from 8K to 1M tokens: the large ones fit entire policy books, case histories, or codebases. Larger windows cost more per request but remove the need to fragment documents.

Long context changes system design: instead of retrieval pipelines that fetch fragments, an agent can read the whole source and answer with full context. The tradeoff is cost and latency, which grow with the tokens actually processed.

Choosing by context is a routing decision: send whole-corpus tasks to 1M-token models like Llama 4 Scout or Gemini 3.5 Flash, and short transactional turns to cheaper windows.

Related terms