Context Window

The context window is the model's working memory limit. Apr 2026 numbers: Claude Opus/Sonnet 4.6 = 200K tokens, Claude with extended caching = 1M, GPT-5.4 = 256K, Gemini 2.5 Pro = 2M. Window size matters most for: long documents (legal contracts, codebases), multi-turn conversations (chat history grows), and agent loops (each iteration adds observations). Hitting the limit causes a hard error or silent truncation depending on the API. Production agents always need a strategy for what to drop when context fills up.

# Approximate token counts before sending
import tiktoken
enc = tiktoken.encoding_for_model("gpt-4")
total = sum(len(enc.encode(m["content"])) for m in messages)
if total > 180_000:  # Leave headroom for response
    messages = compress_old_turns(messages)

Always count tokens before sending. Truncation strategies vary; compress > drop > error.

Plan for context limits from day one. The cheapest fix is summarization of old turns. The most reliable is sliding-window plus a separate long-term memory store.

Related Terms