Message Packets

MPI requires that messages of active requests be uniquely identified to allow for their cancellation. Requests that have been completed or are otherwise inactive cannot be cancelled. As a result, an IMPI message is identified by its source and destination processes, and by a source request identifier unique for every active request at the source process. The request identifier is of type IMPI_Uint8. The total message length is represented by a value of type IMPI_Uint8.

Advice to implementors.On systems with 64-bit memory addressing or less, the address of the request object at the source process may be used as the unique identifier of an active request. On systems with wider memory addressing, the source process would need to maintain a mapping of active requests to identifiers.(End of advice to implementors.)

A message is divided into packets of a given maximum size IMPI_Pk_maxlen, negotiated at startup time. All message packets are sent in the same channel, in sequential order. A fixed-length packet header, IMPI_Packet, holds the message information and identifies the type of transfer: data packet (synchronous message or not), synchronization or protocol acknowledgment (ACK), cancel request, or cancel reply (successful or not).

In addition to identifying messages for cancellation, the source request identifier is used by the sender to access the request handle in the rendezvous message protocol. Similarly, an optional destination request identifier, of type IMPI_Uint8, may be used to accelerate the receiver's access to the request handle. Its usage and the required support by the peer agent is discussed in a later section.

Three quality-of-service fields are made available: pk_count holds the message ``count'' argument (number of datatype elements), pk_dtype identifies the sender's datatype, and pk_seqnum holds a sequence number uniquely identifying a message after it can no longer be cancelled. pk_dtype is an opaque handle.

typedef struct {
        IMPI_Uint4              pk_type;     /* packet type */
#define IMPI_PK_DATA            0            /* message data */
#define IMPI_PK_DATASYNC        1            /* message data (sync) */
#define IMPI_PK_PROTOACK        2            /* protocol ACK */
#define IMPI_PK_SYNCACK         3            /* synchronization ACK */
#define IMPI_PK_CANCEL          4            /* cancel request */
#define IMPI_PK_CANCELYES       5            /* 'yes' cancel reply */
#define IMPI_PK_CANCELNO        6            /* 'no' cancel reply */
        IMPI_Uint4              pk_len;      /* packet data length */
        IMPI_Proc               pk_src;      /* source process */
        IMPI_Proc               pk_dest;     /* destination process */
        IMPI_Uint8              pk_srqid;    /* source request ID */
        IMPI_Uint8              pk_drqid;    /* destination request ID */
        IMPI_Uint8              pk_msglen;   /* total message length */

        IMPI_Int8               pk_tag;      /* message tag */
        IMPI_Uint8              pk_cid;      /* context ID */
        IMPI_Uint8              pk_seqnum;   /* message sequence # */
        IMPI_Int8               pk_count;    /* QoS: message count */
        IMPI_Uint8              pk_dtype;    /* QoS: message datatype */
        IMPI_Uint8              pk_reserved; /* for future use */

} IMPI_Packet;

The data packet is made of a header followed by up to IMPI_Pk_maxlen bytes of packed user data, and uses all the header fields. The four other packet types are header-only, and use a subset of the header fields. The list of fields used by each packet is given in table 3.1. Network byte-order is used in the header.

 

Table 3.1: Packet field usage.

Packet Type	Fields Used
data	all fields (QoS fields optional)
data sync.	all fields (QoS fields optional)
sync. ACK	pk_type, pk_src, pk_dest, pk_srqid, pk_drqid
protocol ACK	pk_type, pk_src, pk_dest
cancel request	pk_type, pk_src, pk_dest, pk_srqid
cancel reply	pk_type, pk_src, pk_dest, pk_srqid