Prompt Caching

Prompt caching lets you mark a prefix of your prompt as "cacheable." On subsequent calls within ~5 minutes (Anthropic) the model reads the cached tokens instead of reprocessing. Cached input tokens cost ~90 percent less. Latency drops because the model skips the prefill step on cached tokens. Best fit: RAG agents that send the same system prompt and retrieved context across many tool-calling iterations. Real-world savings: 30 to 70 percent of total cost for typical agent workloads.

# Anthropic prompt caching
response = client.messages.create(
    model="claude-sonnet-4-6",
    system=[
        {"type": "text", "text": LONG_SYSTEM_PROMPT,
         "cache_control": {"type": "ephemeral"}},
    ],
    messages=conversation,
)

Add cache_control to any prefix. The model auto-detects what to cache.

Turn it on for any agent where the system prompt + tools + retrieved context is large and reused. Pure cost win. No quality tradeoff.

Related Terms