In this dissertation, we gave a historical review of data-parallel languages such as High Performance Fortran (HPF), some message-passing frameworks including p4, PARMACS, and PVM, as well as the MPI standard, and high-level libraries for multiarrays such as PARTI, the Global Array (GA) Toolkit, and Adlib. Moreover, we introduced an HPspmd programming language model--a SPMD framework for using libraries based on multiarrays. It adopted the model of distributed arrays standardized by the HPF Forum, but relinquished the high-level single-threaded model of the HPF language. Also, we described a Java extension of HPspmd programming language model, called HPJava. Via examples, we reviewed the specific syntax and a few new control constructs of HPJava. Most importantly we discussed the compilation strategies such as type-analysis, pre-translation, basic translation schemes, and some optimizations of HPJava. To verify our compilation schemes are appropriate for the current HPJava system, we experimented with and benchmarked some scientific and engineering applications in HPJava. The result of the experimental studies and benchmarks was quite satisfactory as an initial benchmark for a brand-new language. Moreover, we proved that the performance of Java applications are almost caught up with that of C applications in several machines. Finally, we compared and contrasted our HPspmd programming language model with several other language projects such as Co-Array Fortran, ZPL, JavaParty, Titanium, and timber, similar to our HPspmd programming language model, as well as a parallel C++ library as a superset of STL, called STAPL.