Large Language Models
Chain of Thought Prompting
Why telling the model to think step by step radically improves reasoning, and when it actively hurts.
beginner · 5 min read
Adding "Let's think step by step" before the model answers, or showing it worked examples that include reasoning, dramatically improves performance on math and multi-step problems. The 2022 paper made it famous; modern models often do it implicitly.
How to use it
- Zero-shot CoT. Just append "Let's think step by step" to the prompt.
- Few-shot CoT. Show 2-5 example problems with explicit reasoning chains before the target problem.
- Self-consistency. Sample multiple reasoning chains, take the majority answer. Bigger latency win than you would expect.
When it hurts
For tasks that are reflexive ("what's the capital of France?"), forced reasoning adds latency without improving accuracy and sometimes leads the model astray. Match the prompt to the task complexity.
Modern models
Frontier models (Claude, GPT-4, Gemini) trained heavily on reasoning traces often produce chain-of-thought spontaneously. Some expose it as an explicit "thinking" output (Claude extended thinking, OpenAI o1). The capability is built in; the question is just whether you let the user see it.