Safe Parallel Libraries



next up previous contents
Next: Caching Up: Communicators Previous: Communicator Destructor

Safe Parallel Libraries

  libraries, safety modularity

This section illustrates the design of parallel libraries, and the use of communicators to ensure the safety of internal library communications.

Assume that a new parallel library function is needed that is similar to the MPI broadcast function, except that it is not required that all processes provide the rank of the root process. Instead of the root argument of MPI_BCAST, the function takes a Boolean flag input that is true if the calling process is the root, false, otherwise. To simplify the example we make another assumption: namely that the datatype of the send buffer is identical to the datatype of the receive buffer, so that only one datatype argument is needed. A possible code for such a modified broadcast function is shown below.

A (correct) execution of this code is illustrated in Figure gif, with arrows used to indicate communications.


Figure: Correct invocation of mcast

Since the invocation of mcast at the three processes is not simultaneous, it may actually happen that mcast is invoked at process 0 before process 1 executed the receive in the caller code. This receive, rather than being matched by the caller code send at process 2, might be matched by the first send of process 0 within mcast. The erroneous execution illustrated in Figure gif results.


Figure: Erroneous invocation of mcast

How can such erroneous execution be prevented? One option is to enforce synchronization at the entry to mcast, and, for symmetric reasons, at the exit from mcast. E.g., the first and last executable statements within the code of mcast would be a call to MPI_Barrier(comm). This, however, introduces two superfluous synchronizations that will slow down execution. Furthermore, this synchronization works only if the caller code obeys the convention that messages sent before a collective invocation should also be received at their destination before the matching invocation. Consider an invocation to mcast() in a context that does not obey this restriction, as shown in the example below.

The desired execution of the code in this example is illustrated in Figure gif.


Figure: Correct invocation of mcast

However, a more likely matching of sends with receives will lead to the erroneous execution is illustrated in Figure gif.


Figure: Erroneous invocation of mcast

Erroneous results may also occur if a process that is not in the group of comm and does not participate in the collective invocation of mcast sends a message to processes one or two in the group of comm.

A more robust solution to this problem is to use a distinct communication domain for communication within the library, which is not used by the caller code. This will ensure that messages sent by the library are not received outside the library, and vice-versa. The modified code of the function mcast is shown below.

This code suffers the penalty of one communicator allocation and deallocation at each invocation. We show in the next section, in Example gif, how to avoid this overhead, by using a preallocated communicator.



next up previous contents
Next: Caching Up: Communicators Previous: Communicator Destructor



Jack Dongarra
Fri Sep 1 06:16:55 EDT 1995