Figure 5.13 illustrates multithreaded implementation. In this implementation, communications are involved between Java threads. The set of all threads is stored as an array. Each index in this array represents node id.
Two different static queues send and receive queue are maintained to store early arrival of send and receive requests. Each thread also maintains a wait set in the Request class. Communication of any thread that is stroed in this set will be blocked untile complete transaction. If tasks of a non-blocking send or receive are not completed by the time to call completion method of non-blocking communication, like iwait() or iwaitany(), that particular send or receive is stored into the wait set and is blocked for its completion.
Current version of send() and recv() methods are implemented using isend() and irecv() with iwait() method call. When a send request is created by the send thread, it looks for a matching receive request in the receive queue. If a matching receive request is exist, it copies data from the send buffer to the receive request buffer. It also checks if any other thread is iwait-ing on ``matching receive'' and removes all requests from wait set, and signal the waiting thread. This signal makes the waiting thread awake and continue its operation. The send request will be added into the send queue if it fails to find matching receive queue. A receive request work similarly as the send request. The receive request searches the send queue instead of receive queue for a matching request.