OpenMP Fork-Join Model

1. The Fork-Join Execution Model

At its heart, the fork-join model is a simple yet powerful concept for structuring a parallel program. An OpenMP program doesn't run in parallel from start to finish. Instead, it alternates between sequential and parallel phases.

Here’s the flow:

1. Start Sequentially: The program begins as a standard C++ application, with a single thread of execution. This initial thread is called the master thread.
2. Fork: When the master thread encounters a designated parallel region, it creates, or "forks," a new team of threads. The master thread itself becomes part of this team, typically with thread ID 0.
3. Execute in Parallel: The entire team of threads executes the code within the parallel region concurrently.
4. Join: At the end of the parallel region, all threads in the team synchronize. The threads created at the fork are then either destroyed or put to sleep, and they "join" back with the master thread.
5. Resume Sequentially: Execution continues with only the master thread until the next parallel region is encountered or the program ends.

This entire lifecycle is illustrated perfectly in the following diagram.

The sections of code executed by the single master thread are called serial regions, and the sections executed by the team of threads are parallel regions.

2. The parallel Directive: Creating a Parallel Region

The concept of a fork-join model is abstract. In C++, you trigger it with a specific compiler directive. The most fundamental OpenMP directive is #pragma omp parallel. When the compiler sees this line, it knows that the structured block of code that follows is a parallel region.

Let's watch a video that explains the fork-join model and introduces the syntax for creating a parallel region.

As the video highlighted, the join is not just about disbanding the team; it's also an implicit barrier. The master thread will wait at the end of the parallel region until all other threads in the team have finished their work. This is a crucial safety feature that prevents the program from moving on while parallel tasks are still in progress.

To get another perspective, let's read a short web page that connects the #pragma omp parallel directive directly to the fork-join diagram we saw earlier.

3. "Hello, Parallel World": A Practical Example

Now, let's see the fork-join model in action. The simplest way to do this is with a "Hello, World" program where every thread in the team introduces itself.

The following video segment will walk you through this example. It demonstrates how a single printf statement inside a parallel region is executed by every thread. It also introduces two handy OpenMP library functions:

• omp_get_thread_num(): Returns the unique ID of the calling thread (from 0 to N-1).
• omp_get_num_threads(): Returns the total number of threads in the current team.

Here is the code from the example for your reference. Note that to use OpenMP functions and directives, you must include the omp.h header.

#include <iostream>
#include <omp.h>

int main() {
    // This is a serial region. Only the master thread executes this.
    std::cout << "In the serial region, before the fork." << std::endl;

    // This pragma marks the beginning of a parallel region.
    // The master thread will now fork a team of threads.



    #pragma omp parallel
    {
        // This code is executed by ALL threads in the team.
        int thread_id = omp_get_thread_num();
        int num_threads = omp_get_num_threads();
        
        // Using printf for thread-safety, as mentioned in the video.
        printf("Hello from thread %d of %d\n", thread_id, num_threads);
    }
    // This is an implicit join. All threads synchronize here.
    // The worker threads are disbanded, and only the master continues.

    // This is a serial region again.
    std::cout << "In the serial region, after the join." << std::endl;

    return 0;
}

When you compile and run this code, you'll see output from multiple threads. As the video pointed out, the order in which the "Hello" messages appear is unpredictable. This is a direct consequence of the operating system scheduling the threads independently—a perfect, benign demonstration of the non-determinism we discussed in the context of race conditions.

Conclusion

In this lesson, we have unpacked the core execution model of OpenMP. This fork-join paradigm is the foundation upon which all other OpenMP features are built.

Key Takeaways:

• OpenMP programs alternate between serial regions (executed by one master thread) and parallel regions (executed by a team of threads).
• The transition from serial to parallel is called a fork, and the transition back is called a join. This is the fork-join model.
• The #pragma omp parallel directive is used in C++ to define a parallel region.
• The code inside a parallel block is executed redundantly by all threads in the team unless instructed otherwise.
• The join at the end of a parallel region acts as an implicit synchronization barrier, ensuring all parallel work is complete before sequential execution resumes.

Preview of the Next Lesson

We now know how to create a team of threads. However, having every thread do the exact same thing is rarely useful. The real power of parallelism comes from dividing the work.

In the next lesson, we will learn how to do just that. We will explore the #pragma omp for directive, which allows us to automatically split the iterations of a loop among the team of threads. This will be our first step into true work-sharing, and it will naturally lead us to the critical concepts of shared and private variables, which are essential for preventing race conditions.