Concurrency Models: Threading vs. Multiprocessing

1. The Two Worlds: I/O-Bound vs. CPU-Bound

Before we start our experiments, let's formally define the two types of problems that benefit from concurrency. Understanding this distinction is the first and most important step in choosing the right tool.

To summarize the key distinction:

• I/O-Bound: The program's speed is limited by the time it spends waiting for slow external resources (e.g., network, disk drive, database). The goal of concurrency here is to overlap these waiting times.
• CPU-Bound: The program's speed is limited by the processor's speed. It spends most of its time performing computations. The goal of concurrency here is to perform more computations in the same amount of time by using multiple CPU cores.

With these definitions in mind, let's conduct our first experiment.

2. The I/O-Bound Experiment: Downloading Websites

Our first task is a classic I/O-bound problem: downloading the content of multiple websites. The limiting factor is not our CPU; it's the network latency and the speed of the remote servers.

We will test three approaches: synchronous, multi-threaded, and multi-process.

Let's analyze the results from the article:

• Synchronous: The baseline. It took over 14 seconds. Each request happens one after the other.
• Multi-threaded (ThreadPoolExecutor): This was the big winner, finishing in just over 3 seconds. While one thread was waiting for a server to respond, the GIL was released, and another thread could initiate a different download. This overlap in I/O wait times is what provides the massive speedup. This is the correct model for this task.
• Multi-process (ProcessPoolExecutor): This version was also fast (around 3.4 seconds), but slightly slower than the threaded version. Why? Because creating and managing separate processes has a higher overhead than creating threads. For an I/O-bound task where the GIL is not a bottleneck, this extra overhead provides no benefit and makes it a suboptimal choice.

This experiment confirms our rule: for I/O-bound work, threading is the superior choice.

3. The CPU-Bound Experiment: Heavy Computation

Now, let's switch to a CPU-bound problem: calculating Fibonacci numbers recursively. This task involves no I/O; it's pure computation. The CPU is the bottleneck.

We'll repeat the same three tests: synchronous, multi-threaded, and multi-process.

Let's break down these results, which are much more dramatic:

• Synchronous: The baseline. Took about 35 seconds.
• Multi-threaded (ThreadPoolExecutor): This version was slower than the synchronous one, taking nearly 40 seconds. This is the GIL in action. Because the task is CPU-bound, the threads can't run in parallel. They are forced to take turns on a single CPU core, and the overhead of switching between threads and contending for the GIL actually adds to the total execution time. This is the incorrect model for this task.
• Multi-process (ProcessPoolExecutor): This was the clear winner, taking about 10 seconds. By creating separate processes, each with its own GIL, the program was able to execute the calculations in true parallel across multiple CPU cores. This is the correct model for this task.

To visualize exactly why threading fails here and multiprocessing succeeds, the following video provides an excellent demonstration with activity plots.

This experiment proves the second half of our rule: for CPU-bound work, multiprocessing is the only way to achieve a speedup using standard libraries.

4. A Decision Framework and Comparison to Java

You now have the empirical evidence to build a robust decision-making framework. The choice between threading and multiprocessing is not a matter of preference but a direct consequence of the task's nature and the GIL's behavior.

This flowchart and comparison table provide a concise summary.

Given your Java background, it's useful to contrast Python's model with Java's. In Java, threads are "real" OS threads, and there is no GIL. Therefore, Java's threading can be used for both CPU-bound and I/O-bound tasks. This is a fundamental difference in design philosophy.

5. What About asyncio?

In our discussion of I/O-bound concurrency, you might have noticed the mention of asyncio. It represents a third, very important concurrency model in Python. While a full exploration of asyncio is a topic for another lesson, it's crucial to understand its place in this landscape.

• asyncio is a single-threaded, single-process model that uses cooperative multitasking. Instead of the OS switching threads preemptively, tasks explicitly yield control to an event loop when they are waiting for I/O.
• For I/O-bound work, asyncio is often even faster than threading because it avoids the overhead of OS thread management. The community guideline is often: "Use asyncio when you can, threading when you must."
• threading remains essential when you need to perform concurrent I/O using blocking libraries (like the standard requests library) that are not asyncio-compatible.

You can even use threading and multiprocessing from within an asyncio program to run blocking code without stalling the event loop.

This gives you the complete picture. You have three primary tools for concurrency in Python, each with a specific purpose:

1. asyncio: Best for high-performance, I/O-bound tasks using compatible async libraries.
2. threading: Best for I/O-bound tasks when using blocking libraries or for simpler concurrent I/O.
3. multiprocessing: The only choice for CPU-bound tasks that need to bypass the GIL and achieve true parallelism.

Conclusion

This lesson has armed you with the practical knowledge to choose the correct concurrency model in Python. You've gone beyond the rule of thumb and proven it with experiments, observing the performance implications of both good and bad choices.

Key Takeaways:

• The choice between threading and multiprocessing is dictated by whether your task is I/O-bound or CPU-bound.
• For I/O-bound tasks, threading is effective because the GIL is released during I/O waits, allowing threads to overlap their waiting periods. multiprocessing works but has higher overhead.
• For CPU-bound tasks, threading is ineffective and can even be slower due to the GIL. multiprocessing is the correct solution as it bypasses the GIL to achieve true parallelism on multiple cores.
• asyncio is a powerful, often faster, alternative for I/O-bound concurrency, but threading remains vital for handling blocking I/O code.

What's Next?

We've focused on how to make your code run on multiple cores. But what if the code running on each core could be made orders of magnitude faster? Many computational problems have already been solved and highly optimized in C or Fortran. In the next lesson, we'll explore when to stop writing our own Python loops and instead offload heavy numeric work to optimized libraries like NumPy and Cython, another critical strategy in the performance optimization toolkit.