Basic MPI Program Structure

1. The Core MPI Functions and Program Structure

Every MPI program follows a standard structure. It begins by initializing the MPI environment, then performs its parallel work, and finally, shuts down the environment. This entire parallel section of your code is enclosed by MPI_Init and MPI_Finalize.

Within this parallel region, two essential pieces of information are needed for the SPMD model to work:

1. How many total processes are part of this job?
2. Who am I within that group of processes?

Let's watch a video that walks through the C++ code for a basic MPI application and explains the functions that provide this structure and information.

To summarize the structure discussed in the video, here is a flowchart that visualizes the essential components of an MPI program.

2. Your First MPI Program: "Hello, World!"

Now, let's put these pieces together into a complete C++ program. The following code is a classic "Hello, World!" example adapted for MPI. It perfectly demonstrates the SPMD model: every process runs this exact same code, but because MPI_Comm_rank gives each one a different my_rank, the output from each process is unique.

Here is the C++ code from the resource for your reference.

#include <iostream>
#include <mpi.h>

int main(int argc, char **argv) {
    // Initialize the MPI environment.
    // This must always be called before any other MPI functions.
    MPI_Init(&argc, &argv);

    // Get the number of processes in MPI_COMM_WORLD.
    int world_size;
    MPI_Comm_size(MPI_COMM_WORLD, &world_size);

    // Get the rank of this process in MPI_COMM_WORLD.
    int my_rank;
    MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);

    // Print out a message.
    std::cout << "Hello from rank " << my_rank 
              << " out of " << world_size << " processes." << std::endl;

    // Finalize the MPI environment.
    // This must always be called after all other MPI functions.
    MPI_Finalize();

    return 0;
}

Notice MPI_Init takes the address of argc and argv. This allows the MPI system to parse command-line arguments that are meant for it (e.g., network configuration details) and remove them, leaving only the arguments intended for your application.

3. Compiling and Running your MPI Program

You can't compile and run an MPI program like a standard C++ application. You need special tools that know how to link the MPI libraries and launch processes across a system.

Compiling with mpic++

MPI implementations provide a compiler wrapper, typically named mpic++ for C++ code. This wrapper calls a standard C++ compiler (like g++ or clang++) but automatically adds the necessary flags to include MPI headers and link against the MPI libraries. For you, the process is simple: just replace your usual g++ command with mpic++.

Running with mpirun or mpiexec

To execute your compiled program, you use a launcher like mpirun or mpiexec. Its most important job is to start multiple copies of your program. The -n flag specifies the number of processes to launch.

Let's return to the video, which demonstrates this workflow.

To summarize the commands, if your file is named hello.cpp, the workflow is:

1. Compile:

   mpic++ hello.cpp -o hello_mpi
   

2. Run with 4 processes:

   mpirun -n 4 ./hello_mpi
   

You will likely see an output like this, though the order may vary:

Hello from rank 2 out of 4 processes.
Hello from rank 0 out of 4 processes.
Hello from rank 3 out of 4 processes.
Hello from rank 1 out of 4 processes.

If you have an MPI implementation installed on your system, I encourage you to save the code above, compile it, and run it yourself. Try changing the number of processes with the -n flag.

4. A Note on Error Handling

In a distributed application, if one process encounters a fatal error, you typically want the entire job to terminate gracefully, rather than leaving "zombie" processes running. Simply calling exit() or return 1 from one process is not sufficient, as it only terminates that single process.

The correct way to abort an entire MPI job is with MPI_Abort.

MPI_Abort(MPI_COMM_WORLD, error_code);

This function terminates all processes in the MPI_COMM_WORLD communicator. It's good practice to use this for handling unrecoverable errors in your MPI applications.

Conclusion

Congratulations! You have just learned how to write, compile, and run a parallel program using MPI. This is a significant first step into the world of distributed-memory programming.

Key Takeaways:

• Program Structure: All MPI C++ programs must include <mpi.h>, and all parallel code must be contained between MPI_Init() and MPI_Finalize().
• Core Functions: MPI_Comm_size() reports the total number of processes, and MPI_Comm_rank() provides the unique ID for the current process. These are the building blocks of the SPMD model.
• Workflow: MPI code is compiled with mpic++ (or a similar wrapper) and executed with mpirun or mpiexec, using the -n flag to specify the process count.
• Execution is Asynchronous: Processes run independently. Without explicit synchronization, the order of operations (like printing to the screen) between processes is not guaranteed.

Preview of the Next Lesson

Our "Hello, World!" program demonstrated the SPMD model, but the processes didn't actually communicate with each other. They each ran in isolation. In the next lesson, we will change that. You will learn how to implement point-to-point communication using blocking MPI_Send and MPI_Recv. This will allow your processes to explicitly exchange data, which is the heart of the message-passing paradigm.