Chapter 8: Chain-of-Thought and Reasoning
Enabling AI to Show Its Work and Reason Through Problems
Learning Objectives
After completing this chapter, you will be able to:
- Explain what chain-of-thought prompting is
- Distinguish between zero-shot and few-shot chain-of-thought
- Apply step-by-step reasoning techniques
- Know when chain-of-thought improves results
- Debug and refine reasoning chains
What Is Chain-of-Thought Prompting?
Definition
Chain-of-Thought (CoT) prompting is a technique that encourages AI to break down complex problems into intermediate reasoning steps before arriving at a final answer. Instead of jumping directly to a conclusion, the model “thinks out loud.”
The Core Insight
Figure 8.1: Standard prompting jumps directly to answers, while chain-of-thought breaks problems into visible reasoning steps.
Why It Works
- Decomposition: Complex problems become manageable parts
- Error Detection: Mistakes become visible in the reasoning
- Guidance: Each step constrains the next
- Verification: You can check the logic, not just the answer
Types of Chain-of-Thought
Zero-Shot Chain-of-Thought
Zero-shot CoT uses a simple trigger phrase without providing examples:
Question: If a car travels at 60 mph for 2 hours and then
at 40 mph for 3 hours, what is the total distance traveled?
Think through this step by step.
Common Trigger Phrases:
- “Let’s think step by step”
- “Think through this carefully”
- “Show your reasoning”
- “Work through this problem”
- “Explain your thought process”
Few-Shot Chain-of-Thought
Few-shot CoT provides examples of reasoning before the actual question:
Example:
Question: How many apples do I have if I start with 5 and eat 2?
Reasoning: I start with 5 apples. I eat 2 apples. 5 - 2 = 3.
Answer: 3 apples.
Question: How many apples do I have if I start with 8, eat 3,
and buy 4 more?
Reasoning: [Model continues with similar step-by-step format]
Comparison
| Type | Pros | Cons |
|---|---|---|
| Zero-Shot | Simple, no examples needed | Less control over reasoning format |
| Few-Shot | More consistent format | Requires good examples |
Step-by-Step Reasoning Techniques
Technique 1: Explicit Step Instructions
Solve this problem by following these steps:
STEP 1: Identify all the given information
STEP 2: Determine what we need to find
STEP 3: Choose an appropriate method
STEP 4: Execute the solution
STEP 5: Verify the answer
Problem: [Your problem here]
Technique 2: Think-Plan-Execute
For this task, please:
THINK: What is being asked? What do I know?
PLAN: What approach will I take?
EXECUTE: Carry out the plan step by step
CHECK: Verify the result makes sense
Task: [Your task here]
Technique 3: Structured Decomposition
Break down and solve this problem:
1. UNDERSTAND the problem
- What are the inputs?
- What is the expected output?
- What constraints exist?
2. PLAN the approach
- What method will work?
- What are the sub-problems?
3. SOLVE step by step
- [Work through each sub-problem]
4. VALIDATE
- Does the answer satisfy the original problem?
- Are there edge cases to consider?
Problem: [Your problem here]
The Reasoning Chain Diagram
Figure 8.2: The complete chain-of-thought process showing how problems are decomposed into steps before reaching a conclusion.
When to Use Chain-of-Thought
Best Use Cases
| Task Type | Why CoT Helps | Example |
|---|---|---|
| Math problems | Shows calculation steps | Multi-step arithmetic |
| Logic puzzles | Reveals deductive process | Syllogisms, riddles |
| Code debugging | Traces execution | Finding bug sources |
| Complex analysis | Structures thinking | Multi-factor decisions |
| Planning tasks | Breaks into phases | Project planning |
| Causal reasoning | Shows cause-effect chains | Root cause analysis |
When NOT to Use
| Task Type | Why CoT May Not Help |
|---|---|
| Simple factual recall | No reasoning needed |
| Creative writing | Can interrupt flow |
| Quick lookups | Adds unnecessary overhead |
| Highly subjective tasks | No objective steps |
Decision Guide
Figure 8.3: Decision guide for determining when chain-of-thought prompting will improve results.
Chain-of-Thought Examples
Example 1: Mathematical Reasoning
Prompt:
A store sells apples in bags of 6 and oranges in bags of 4.
If Maria buys 3 bags of apples and some bags of oranges,
and she has exactly 30 pieces of fruit total, how many
bags of oranges did she buy?
Think through this step by step.
Response:
Let me work through this step by step.
Step 1: Calculate the apples
Maria buys 3 bags of apples
Each bag has 6 apples
Total apples = 3 × 6 = 18 apples
Step 2: Determine remaining fruit
Total fruit = 30
Fruit from apples = 18
Remaining must be oranges = 30 - 18 = 12 oranges
Step 3: Calculate orange bags
Each bag has 4 oranges
Number of bags = 12 ÷ 4 = 3 bags
Answer: Maria bought 3 bags of oranges.
Example 2: Logical Deduction
Prompt:
In a room, there are 4 people: Alex, Beth, Carlos, and Diana.
- Alex is taller than Beth
- Carlos is shorter than Diana
- Diana is shorter than Beth
Who is the tallest? Show your reasoning.
Response:
Let me reason through the height relationships.
Step 1: List known relationships
- Alex > Beth (Alex taller than Beth)
- Diana > Carlos (Carlos shorter than Diana)
- Beth > Diana (Diana shorter than Beth)
Step 2: Build the ordering
From Beth > Diana and Diana > Carlos: Beth > Diana > Carlos
From Alex > Beth: Alex > Beth > Diana > Carlos
Step 3: Determine the tallest
The complete ordering from tallest to shortest:
Alex > Beth > Diana > Carlos
Answer: Alex is the tallest.
Example 3: Code Analysis
Prompt:
This function should return the sum of even numbers in a list,
but it's returning incorrect results. Trace through the logic
step by step to find the bug.
def sum_even(numbers):
total = 0
for n in numbers:
if n % 2 == 0:
total = n
return total
Response:
Let me trace through the function step by step.
Step 1: Identify the purpose
The function should sum all even numbers in the list.
Step 2: Trace execution with example [2, 4, 6]
- Initialize: total = 0
- n = 2: 2 % 2 == 0 is True, total = 2 (replaces 0)
- n = 4: 4 % 2 == 0 is True, total = 4 (replaces 2!)
- n = 6: 6 % 2 == 0 is True, total = 6 (replaces 4!)
- Returns 6
Step 3: Identify the bug
The line "total = n" REPLACES the total instead of
ADDING to it. It should be "total += n" or "total = total + n"
Step 4: Corrected code
def sum_even(numbers):
total = 0
for n in numbers:
if n % 2 == 0:
total += n # Changed from total = n
return total
Answer: The bug is on line 5. Use "total += n" instead of "total = n".
Advanced CoT Techniques
Self-Consistency (Multiple Chains)
Solve this problem three different ways, then compare
your answers to find the most reliable solution.
Problem: [Your problem]
Approach 1: [First method]
Approach 2: [Second method]
Approach 3: [Third method]
Final Answer: [Most consistent result with explanation]
Verification Chain
After solving, verify your answer:
1. Solve the problem
2. Check: Does the answer satisfy all constraints?
3. Reverse check: Work backwards from answer to input
4. Sanity check: Is the answer reasonable?
Decomposition Chain
Break this complex problem into sub-problems:
Main Problem: [Complex problem]
Sub-problem 1: [First component]
Solution: [Solve independently]
Sub-problem 2: [Second component]
Solution: [Solve independently]
Integration: Combine solutions to answer main problem
Debugging Reasoning Chains
Common Chain Errors
| Error Type | Example | Fix |
|---|---|---|
| Skipped step | Jumping to conclusion | Request explicit steps |
| Wrong assumption | Incorrect premise | Ask to state assumptions |
| Calculation error | Math mistake in chain | Request verification |
| Logic gap | Unjustified conclusion | Ask “why” at each step |
Debugging Prompts
When reasoning seems wrong:
Your step 3 seems incorrect. Can you:
1. Re-examine the logic from step 2 to step 3
2. Explain why you made that conclusion
3. Consider if there's an alternative interpretation
When steps are unclear:
Please clarify your reasoning:
- What did you assume in step 2?
- How did step 2 lead to step 3?
- What would happen if that assumption were wrong?
Key Takeaways
- Chain-of-Thought makes AI reasoning visible and verifiable
- Zero-shot CoT uses trigger phrases; few-shot CoT uses examples
- CoT is most valuable for complex, multi-step problems
- Step-by-step structures help organize reasoning
- Verification chains catch errors before final answers
- Not all tasks benefit from CoT—use it strategically
Summary
Chain-of-thought prompting transforms AI from a black box into a transparent reasoning system. By encouraging step-by-step thinking, you get not just answers but the logic behind them. This makes outputs more accurate, more verifiable, and easier to debug. Master this technique for any task involving logic, calculation, or complex analysis.
Review Questions
- What distinguishes zero-shot from few-shot chain-of-thought?
- Name three task types where CoT is particularly valuable.
- What are common trigger phrases for zero-shot CoT?
- How does the verification chain technique work?
- When might CoT be unnecessary or counterproductive?
Practical Exercise
Exercise 8.1: CoT Application
Use chain-of-thought to solve: “A farmer has chickens and cows. Together they have 30 heads and 74 legs. How many of each animal does the farmer have?”
Write the prompt and expected reasoning chain.
Exercise 8.2: Debugging a Chain
Given this incorrect reasoning, identify and fix the error: “To find 15% of 80: 15 + 80 = 95, so 15% of 80 is 95.”