HPJava Home Page

HPJava language
SC '02 demo

Project flier

Java HPC Course
HPspmd lectures

Community Grids
Bloomington photos

Java Grande Home
Numerics working group


Viewing these pages

Communication in HPJava

Returning to our matrix multiplication example, we saw how the simple implementation depended on the fact that there was a very special alignment relation between the three distributed arrays a, b and c. Every process already held copies of all the operands needed to compute every locally held result element.

We would be quite lucky to find three arrays with such a convenient alignment of elements in the real world. On the other hand, given any three distributed arrays of compatible shape, we can always copy the input arrays to temporary arrays that do have the desired alignment to the result array, then do the computation. This reshuffling of data is an example of a communication.

Here is a completely general HPJava method for multiplying together two matrices, implemented as distributed arrays:

void matmul(double [[-,-]] c, double [[-,-]] a, double [[-,-]] b) {

  Group p = c.grp() ;

  Range x = c.rng(0) ;
  Range y = c.rng(1) ;

  int N = a.rng(1).size() ;

  double [[-,*]] ta = new double [[x, N]] on p ;
  double [[*,-]] tb = new double [[N, y]] on p ;

  Adlib.remap(ta, a) ;
  Adlib.remap(tb, b) ;

    overall(i = x for :)
      overall(j = y for :) {

        float sum = 0 ;
        for(int k = 0 ; k < N ; k++)
          sum += ta [i, k] * tb [k, j] ;

        c [i, j] = sum ;
The method introduces two temporary arrays ta, tb with the desired distribution format. This is determined from c by using the inquiry functions grp() and rng() to get the distribution group and index ranges of the array c. The method Adlib.remap() does the actual communication needed to remap the data. The collective communication method remap() is a static method of the class Adlib. It takes two distributed arrays as arguments. These arrays must have the same type and shape; however they can have any, unrelated, distribution pattern. The elements of the second array argument are copied to the elements of the first.

The example also illustrates a more general form of the distributed array constructor, as in:

  double [[-,*]] ta = new double [[x, N]] on p ;
The clause on p explicitly specifies the group of elements over which the array is distributed. This defaults to the current APG, but in general a new array may be distributed over any subgroup of the APG.

Our implementation of matrix multiplication has some performance issues associated with its memory usage. These issues can be patched up. Meanwhile, besides introducing an important communication pattern, the simple version given here encapsulates some interesting principles of library construction with HPJava - in particular how arrays can be created and manipulated, even though the distribution formats are only determined at run-time.

Next: More on Ranges

Bryan Carpenter, (dbc@ecs.soton.ac.uk). Last updated May 2007.