Avoiding Errors: When Automatic Thinking Fails

Picking up where we left off, we've established that your mind operates using two distinct but interconnected systems: the fast and automatic System 1, and the slow and deliberate System 2. This distinction is the starting point for developing more robust thinking skills. Now, we will delve into the crucial next step: learning to recognize the specific situations where your efficient mental autopilot can steer you wrong.

In this lesson, we will explore the predictable errors that arise from automatic thinking. You will learn about mental shortcuts, known as heuristics, and see firsthand through a series of puzzles and scenarios how they can lead to cognitive biases. By the end of our session, you will be better equipped to spot these cognitive traps in real-time, creating an opportunity to pause and engage your more analytical mind. This is a foundational skill for challenging the hidden assumptions you're keen to overcome.

When the Fast Path Fails

In our last lesson, we drew an analogy between System 1 and a network's "fast path" processor. It's incredibly efficient for routine tasks, but its efficiency comes from using simplified rules and patterns—what psychologists call heuristics. These mental shortcuts work brilliantly most of the time, saving us precious mental energy.

The problem arises when System 1 applies a shortcut to a situation where it doesn't fit. It's like a routing rule that correctly handles 99% of data packets but misdirects a specific, unusual type, causing an error. The key to better thinking is learning to recognize these "unusual packets" before the error occurs.

Let's test this with a couple of quick brain-teasers. As before, go with your first, intuitive answer.

If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets?
In a lake, there is a patch of lily pads. Every day, the patch doubles in size. If it takes 48 days for the patch to cover the entire lake, how long would it take for the patch to cover half the lake?

Did the answers "100 minutes" and "24 days" immediately come to mind? These are classic System 1 responses. They feel right because they match a simple pattern. But if we engage System 2, we find the correct answers:

It would take 5 minutes. Each machine produces one widget in 5 minutes, so 100 machines can produce 100 widgets in the same amount of time.
It would take 47 days. If the patch doubles in size every day, it must have been half the size of the lake the day before it covered it completely.

These puzzles highlight a critical principle: System 1's suggestions are often accepted without scrutiny by the lazy System 2. Our goal is to train System 2 to be a little more vigilant.

Heuristics and Biases: The Source Code of Error

The puzzles above are fun demonstrations, but the underlying mechanisms are at play in serious, real-world decisions. To understand them better, let's formally define heuristics and their side effect, cognitive biases.

Heuristics and biases in decision making

This article provides clear, simple definitions for these core concepts and gives a practical example of how they play out.

Start by reading the section "What are heuristics". Focus on the definition of heuristics as problem-solving shortcuts. Next, read the section "Cognitive biases" to understand how these shortcuts lead to errors. Pay attention to the idea that a bias is a systematic error that occurs when a heuristic doesn't match the environment. Finally, read the "Fast and slow examples" section. This provides an excellent real-world scenario of a job interview to illustrate how System 1's initial heuristic-based judgments can unconsciously steer System 2's "deliberate" analysis.

As the article explains, a heuristic isn't inherently bad; it's a tool. A cognitive bias is the predictable error that occurs when you use the wrong tool for the job. In the job interview example, System 1 uses a shortcut ("glasses = smart") and presents this as a fact to System 2. A lazy System 2 then proceeds not by evaluating the candidate objectively, but by looking for evidence that confirms System 1's initial, biased assessment. This particular trap is called confirmation bias, and it is one of the most pervasive obstacles to clear thinking.

The Invisible Gorilla: When Focus Creates Blindness

Errors don't just happen because System 2 is lazy. Sometimes, System 2 can be working so hard on a specific task that it renders System 1 completely blind to otherwise obvious information. This phenomenon is called "inattentional blindness."

System 1 and System 2 Thinking

This case study of a famous psychology experiment is a powerful demonstration of how our focus can make us miss the obvious.

Please read the case study titled 'The Invisible Gorilla'. As you read, notice how engaging System 2 on a specific task (counting passes) completely suppresses System 1's ability to spot a major anomaly (the gorilla).

This experiment is a profound metaphor for real-world problem-solving. When you are deep in a complex systems engineering task—debugging code, analyzing logs, or designing an architecture—your System 2 is intensely focused. This is necessary, but it also creates the risk that you will completely miss a "gorilla": a fundamental, high-level flaw in your approach, a mistaken assumption, or a piece of contradictory information that is "obvious" to an outside observer. Recognizing this vulnerability is the first step toward mitigating it.

The Confirmation Bias Trap: An Interactive Puzzle

Now, let's see if you can spot and overcome the confirmation bias in your own thinking. The next resource is a video that presents a simple number-sequence puzzle. Your task is to figure out the rule.

I encourage you to pause the video after the rule is presented and think about what three-number sequences you would propose to test your hypothesis before you see what the people in the video do.

The Most Common Cognitive Bias

This Veritasium video is a brilliant interactive demonstration of confirmation bias. It cleverly guides you into the very trap it's explaining.

Watch from the introduction where the host explains the puzzle. The sequence is 2, 4, 8. Then, watch as participants repeatedly test sequences that align with their initial hypothesis. Notice their strategy. Are they trying to prove their rule is right, or are they trying to find out where it might be wrong? The host then guides them toward a more effective strategy starting around this point. See how testing for a "no" provides much more information than getting another "yes". Finally, watch the explanation of the bias and how it connects to the scientific method of trying to disprove theories.

How did you do? Did you, like most people, immediately think the rule was "doubling the previous number" and then propose sequences like (10, 20, 40) to confirm it?

The more powerful strategy is to try to falsify your hypothesis. If you think the rule is "doubling," you should test a sequence that doesn't follow that rule, like (2, 4, 7) or (10, 9, 8). Getting a "yes" for (2, 4, 7) and a "no" for (10, 9, 8) tells you that "doubling" is wrong but "ascending order" might be right. This is directly analogous to debugging: the goal is not to prove your code works, but to actively find the inputs that break it. That's where you find the bugs.

Test Your Intuition

Let's end with a few quick questions that often trip up System 1. These are drawn from cognitive research and illustrate common thinking errors.

This table presents several questions that reveal common errors in intuitive thinking. We'll look at a few of them.

Consider these two questions from the table:

Question 3: In an English dictionary, are there more words with the form _ _ _ _ _ N _ (six-letter word with N as the fifth letter) or _ _ _ _ I N G?
- System 1's trap: It is much easier to recall words ending in "ING" than it is to recall words with 'n' as the fifth letter. This ease of recall (the availability heuristic) makes us assume "ING" words are more common. In reality, any seven-letter word ending in "ING" also has 'n' as its fifth letter, so the _ _ _ _ _ N _ category must be larger.
Question 7: Linda is 31, single, outspoken, and very bright. She majored in philosophy and was deeply concerned with social justice. Which is more probable? (a) Linda is a bank teller. (b) Linda is a bank teller and is active in the feminist movement.
- System 1's trap: The description of Linda fits the stereotype of a feminist activist far better than the stereotype of a bank teller. System 1 latches onto this plausible story. However, simple logic dictates that the set of "bank tellers" is much larger than the subset of "bank tellers who are also feminist activists." Therefore, (a) is statistically more probable. Choosing (b) is an error called the conjunction fallacy.

These examples show how System 1 prefers a plausible, easy-to-process narrative over cold, hard logic—a tendency you must learn to watch out for.

Conclusion

In this lesson, we moved from knowing that there are two systems of thought to seeing how the fast, automatic system can lead to predictable mistakes.

Key Takeaways:

System 1 relies on heuristics (mental shortcuts) to make fast, efficient judgments.
When a heuristic is applied to the wrong situation, it results in a systematic error, known as a cognitive bias.
Confirmation bias is the tendency to seek out and favor information that confirms our existing beliefs, while ignoring information that contradicts them. Trying to falsify your assumptions is a more powerful strategy for finding the truth.
Inattentional blindness shows that even when we are focused, our concentration on one task can make us blind to other obvious information.

You are now equipped to start recognizing the tell-tale signs of System 1 errors. This is not about eliminating System 1—that would be impossible and undesirable. The goal is to cultivate a "warning system" that tells you when to slow down and consciously engage the analytical power of System 2.

In our next lesson, we will continue this process by formally identifying and naming some of the most common and impactful cognitive traps you'll encounter: confirmation bias, anchoring bias, and the availability heuristic.