Distributed-Memory Parallelism with MPI

Explain the message-passing paradigm and the SPMD (Single Program, Multiple Data) execution model

Write basic MPI programs using MPI_Init, MPI_Finalize, MPI_Comm_rank, and MPI_Comm_size

Implement point-to-point communication using blocking MPI_Send and MPI_Recv

Use non-blocking communication (MPI_Isend, MPI_Irecv, MPI_Wait) to overlap communication with computation

Implement one-to-all broadcast (MPI_Bcast) and all-to-one reduction (MPI_Reduce) operations

Distribute and collect data arrays using MPI_Scatter and MPI_Gather

Implement parallel matrix-vector multiplication using MPI collectives

Model communication overhead and apply strategies like message aggregation to minimize it