Irregular Collective Communications

Adlib has some support for irregular communications in the form of collective gather() and scatter() operations. The simplest form of the gather operation for one-dimensional arrays has prototypes

$$\begin{minipage}[t]{\linewidth}\small\begin{tabbing} \verb$ void ... ..., $$T$\verb$ [[-]] source, int [[-]] subscripts) ;$ \end{tabbing}\end{minipage}$$

The subscripts array should have the same shape as, and be aligned with, the destination array. In pseudocode, the gather operation is equivalent to

$$\begin{minipage}[t]{\linewidth}\small\verb$ for all $$i$\verb$ in... ...b$ destination [$$i$\verb$] = source [subscripts [$$i$\verb$]] ;$\end{minipage}$$

where $N$ is the size of the destination (and subscripts) array. If we are implementing a parallel algorithm that involves a stage like

$$\begin{minipage}[t]{\linewidth}\small\verb$ for all $$i$\verb$ in... ...rallel do$ \\ \verb$ a [$$i$\verb$] = b [${\em fun(i)}\verb$] ;$\end{minipage}$$

where fun is an arbitrary function, it can be expressed in HPJava as

$$\begin{minipage}[t]{\linewidth}\small\verb$ int [[-]] tmp = new i... ... fun(}\verb$i${\em )} ; \\ \\ \verb$ Adlib.gather(a, b, tmp) ;$\end{minipage}$$

where p and x are the distribution group and range of a. The source array may have a completely unrelated mapping.

The one-dimensional case generalizes to give a complicated family of prototypes for multidimensional arrays:

Currently the highest rank of source array with a gather() method is 3. The source and destination arrays can have different ranks (just as they have unrelated distribution formats). But the destination and subscript arrays are all the same shape, and all are aligned with one another. The number of subscript array arguments is equal to the rank of the source array. If the rank of the destination array is $R$, the general behaviour of these methods is:

$$\begin{minipage}[t]{\linewidth}\small\begin{tabbing} \verb$for al... ...verb$, ..., $$i_{R-1}$\verb$],$ \\ \>\verb$...] ;$ \end{tabbing}\end{minipage}$$

where $(N_0, \ldots, N_{R-1})$ is the shape of destination array.

The basic scatter function has very similar prototypes, but the names source and destination are switched. For example the one-dimensional case is

$$\begin{minipage}[t]{\linewidth}\small\begin{tabbing} \verb$ void ... ...stination,$ \\ \verb$ int [[-]] subscripts) ;$ \\ \end{tabbing}\end{minipage}$$

and it behaves like

$$\begin{minipage}[t]{\linewidth}\small\begin{tabbing} \verb$ for a... ... [subscripts [$$i$\verb$]] = source [$$i$\verb$] ;$ \end{tabbing}\end{minipage}$$

Currently the HPJava version of Adlib does not support combining scatters, although these could be added in later releases.

All the gather() and scatter() operations take an optional final argument which is a boolean array acting as a mask, e.g.

$$\begin{minipage}[t]{\linewidth}\small\begin{tabbing} \verb$ void ... ... \verb$ int [[-]] subscripts, boolean [[-]] mask)$ \end{tabbing}\end{minipage}$$

The mask should have the same shape as and be aligned with the subscript arrays. Assignment to a destination element is conditional on the value of the element of the mask associated with the subscripts.