About the Message Passing Interface (MPI)

The Message Passing Interface is a specification for one particular approach to enabling communication and coordination among processes running in parallel on multiple independent but connected computing nodes. Whether the Message Passing Interface (MPI) is the best approach for a particular computational problem is not a question which can be answered in general, but MPI is widely used on Beowulf clusters and is an approach which allows developers of parallel-processing algorithms flexibility in algorithm design along with opportunities to keep the overhead of inter-node communication relatively low. (The realization of these opportunities, of course, is dependent on the developer's skill and the efficiency of the MPI implementation.) Power and flexibility come at the cost of complexity — the effective use of the Message Passing Interface in designing and implementing parallel algorithms and software requires that an already proficient scientific programmer become familiar with the concepts of MPI and then gain experience in applying MPI to solve a variety of computational problems. Two languages widely used for Beowulf software development (with or without MPI) are FORTRAN and C; there are a number of implementations of MPI, both commercial and open-source, available for use with these two high-level, procedural programming languages.

The MPI topic page in the General Information Section provides additional introductory information about MPI.

Why Two Implementations of MPI?

The Message Passing Interface is a specification: it is a statement of the syntax and semantics of additional statements to be added to the standard statements of a high-level, procedural programming language (such as FORTRAN or C) in order to extend the expressive range of the language with parallel-processing constructs. The MPI specification does not dictate how an MPI implementation is to get from source code, with MPI statements embedded, to real, functioning software; it only dictates the functionality to be added by MPI.

MPI specifies that a program written as if it were a single program can be initiated (launched) on any number of the nodes of a Beowulf or other computational cluster to run in parallel in such a way that each copy of the program can identify itself individually, perform blocks of computations without outside interference, but also communicate with all of the other copies. Conceptually, the cleanest way to implement MPI might be: first, to create an operating system (OS) specifically designed for parallel cluster processing and equipped with low-level system calls to support interprocess communication among processes distributed over a number of nodes; and second, to create new language compilers which understand MPI constructs and syntax and create executables tailored for use on the underlying parallel-cluster-processing operating system. Although the custom-OS/custom-compiler approach has intellectual appeal, it implies a substantial amount of development for implementers and significant constraints and specialization for users.