Chain-of-Thought (CoT)

Chain-of-thought prompting was introduced by Google Research in early 2022 and remains the most reliable prompting technique for improving reasoning quality. The pattern: instead of asking the model for an answer, ask it to "think step by step" and write out the intermediate reasoning. Modern frontier models (Claude Opus, GPT-5, Gemini 2.5) often do this implicitly, but explicit CoT prompting still improves accuracy on math, multi-hop QA, planning, and tool selection. The cost is more output tokens. The benefit is materially fewer wrong answers, especially on tasks the model could not solve in one pass.

CoT is the parent of every reasoning pattern in this glossary. ReAct adds tool calls between reasoning steps. Tree-of-thought explores multiple branches. Reflexion adds self-critique. Plan-and-execute splits planning from execution. All of them inherit CoT's core insight: the model reasons better when it writes its reasoning down. In production, the practical question is whether to keep the reasoning visible to end users (transparent, helpful for trust) or hide it as scratch (cleaner UX, hides model uncertainty). Most production agents do both: log the full chain for debugging and audit, surface only the final answer to users.

# Without CoT
Q: I have 23 apples. I give 5 to Alice and buy 8 more. How many?
A: 26   ← sometimes right, sometimes off-by-one

# With CoT ("think step by step")
Q: ... Let's work through this step by step.
A: Start with 23. Give away 5: 23 - 5 = 18. Buy 8 more: 18 + 8 = 26.
   Answer: 26.   ← reliably correct

Visible reasoning is what makes CoT work. The model outputs more tokens but gets the answer right.

Default for any non-trivial reasoning task. Skip only when the task is simple lookup or classification, or when latency budget cannot afford the extra output tokens.

• Wei et al. 2022 — original CoT paper
• Anthropic: Building Effective Agents

Related Terms