IMPI_P_PID

Each client's array contains identification numbers for its procs. Each value is an IMPI_Uint4, and must be unique among other procs that share the same IPV6 address.

Example: per-client labels

Consider the same three-client job from above. After receiving the concatenated IMPI buffer from the server, the clients now exchange their startup parameters.

The first parameter is the number of local hosts maintained by each client. In the above example, each client would first send an IMPI_Cmd to the server containing {IMPI_CMD_COLL, 8}. After the IMPI_Cmd, the clients each send the IMPI_C_NHOSTS label followed by their local host count. (For a total of 8 bytes, thus the value of 8 in the len field of the command.) This exchange is shown in Figure 2.2.

  

Figure: COLL command IMPI_C_NHOSTS exchange.

The server, upon receiving all of the IMPI_C_NHOSTS data, passes back the concatenated values to the clients:

        IMPI_Int4 cmd         = IMPI_CMD_COLL
        IMPI_Int4 len         = 20
        IMPI_Int4 label       = IMPI_C_NHOSTS
        IMPI_Int4 client_mask = 0x7
        IMPI_Int4 value       = 3             // from client 0
        IMPI_Int4 value       = 2             // from client 1
        IMPI_Int4 value       = 2             // from client 2

All of the per-client values follow this same pattern. For another example, assume that client 0 has a packet length of 8000, while clients 1 and 2 have a packet length of 4000. They therefore first send an IMPI_Cmd to the server containing {IMPI_CMD_COLL, 8}. After the IMPI_Cmd, the clients each send the IMPI_C_PKTLEN label followed by their local pktlen value.

The server, upon receiving all of the IMPI_C_PKTLEN data, passes back the concatenated values to the clients:

        IMPI_Int4 cmd         = IMPI_CMD_COLL
        IMPI_Int4 len         = 20
        IMPI_Int4 label       = IMPI_C_PKTLEN
        IMPI_Int4 client_mask = 0x7
        IMPI_Int4 value       = 8000          // from client 0
        IMPI_Int4 value       = 4000          // from client 1
        IMPI_Int4 value       = 4000          // from client 2

Each client can then independently determine the global minimum packet length. Similarly, the IMPI_C_TAGUB value will be used by the clients to determine the minimum tag_ub among the clients. The server, being a passive collecting and broadcasting process, knows nothing of the meaning of any of these COLL labels.

Note: Some tags may be optional, and therefore may not be sent by all clients. When this happens, the server will zero the appropriate bit(s) in the client_mask field and just concatenate the values from the participating clients. For example, if client 1 had not passed in a packet length, the above would instead have been:

        IMPI_Int4 cmd         = IMPI_CMD_COLL
        IMPI_Int4 len         = 16
        IMPI_Int4 label       = IMPI_C_PKTLEN
        IMPI_Int4 client_mask = 0x5
        IMPI_Int4 value       = 8000          // from client 0
        IMPI_Int4 value       = 4000          // from client 2

This is sent to all clients, regardless of whether they submitted a value for this label. Clients are free to ignore non-mandated COLL labels for the IMPI protocol version(s) that they are using, as well as commands they do not understand.

Discussion.Is it useful for all clients to receive this broadcast, even those that did not send in a value? Although such clients may always ignore it if they wish, is there a reason they would not want to ignore it?(End of discussion.)

Example: per-host labels

  

Figure: COLL command IMPI_H_PORT exchange.

Again using the same three-client example, let's say that it is now time for the clients to submit the port numbers for their hosts. This exchange is shown in Figure 2.3. These would be submitted as follows:

    client 0:
        IMPI_Int4 cmd         = IMPI_CMD_COLL
        IMPI_Int4 len         = 16
        IMPI_Int4 label       = IMPI_H_PORT
        IMPI_Int4 port        = 5001 // (or whatever)
        IMPI_Int4 port        = 5002
        IMPI_Int4 port        = 5003

    client 1:
        IMPI_Int4 cmd         = IMPI_CMD_COLL
        IMPI_Int4 len         = 12
        IMPI_Int4 label       = IMPI_H_PORT
        IMPI_Int4 port        = 6001 // (or whatever)
        IMPI_Int4 port        = 6002

    client 2:
        IMPI_Int4 cmd         = IMPI_CMD_COLL
        IMPI_Int4 len         = 12
        IMPI_Int4 label       = IMPI_H_PORT
        IMPI_Int4 port        = 7001 // (or whatever)
        IMPI_Int4 port        = 7002

In return, the server would send back:

        IMPI_Int4 cmd         = IMPI_CMD_COLL
        IMPI_Int4 len         = 36
        IMPI_Int4 label       = IMPI_H_PORT
        IMPI_Int4 client_mask = 0x7
        IMPI_Int4 port        = 5001 // from client 0
        IMPI_Int4 port        = 5002
        IMPI_Int4 port        = 5003
        IMPI_Int4 port        = 6001 // from client 1
        IMPI_Int4 port        = 6002
        IMPI_Int4 port        = 7001 // from client 2
        IMPI_Int4 port        = 7002