Tests are written as scripts that are interpreted within a standard C MPI program on the system under test. These test scripts are delivered to the MPI processes as arrays of characters via MPI_Send(). The first test to be run each time the interpreter is started is a test that is embedded in the interpreter. This initial test verifies that this delivery mechanism is working. If this initial test fails, no other tests are attempted and diagnostic messages are sent to the Test Tool. For the special case of the Startup Only testing, this initial test of communications is not attempted.
Once the basic delivery of test scripts has been verified, the tester may select tests to be run. At this point, in the event of a test failure, it is most important to help testers diagnose the error. For this reason, the contents of each script will be available on request via the Test Tool. This section describes the operation of the interpreter and the language that it accepts so that the tester can read these scripts and hopefully diagnose the failure. Error messages produced by these scripts will also be more understandable given the test script that produced them.
The test interpreter is an C MPI program that runs on the SUT. This program runs a loop that repeatedly receives and interprets small scripts that superficially look like standard C and MPI code. The interpreter repeats this loop until a script calls a special function which signals the interpreter to exit. The language that the interpreter understands is limited to a small subset of C with MPI function calls available. The MPI routines in the interpreter are wrappers to the actual MPI library routines in the IUT. Declarations for variables and single dimensioned arrays of the basic data types are allowed as well as control structures such as if, while, and for. In addition to the MPI routines, other routines are available such as printf() for printing to stdout and report()for passing information to the Test Manager. Details of the language accepted by this interpreter are available in the source code.
A test will be comprised of a block of statements in this language. Some sample tests are shown below.
The same interpreter, with some modifications, is executed within one or more simulated MPI processes which are running as part of the Test Manager. These simulated MPI processes execute the same script as the IUT. The MPI routines in this case are linked to routines, internal to the Test manager, which implement the communications between the simulated processes as well as external MPI processes. Other modifications to the interpreter allow these simulated MPI processes to interact directly with the Test Manager, supplying test status and error information as the script is interpreted.
Here is a simple test that exercises MPI_Send. Some of the comments refer to the test while other comments refer to the syntax of the interpreted language.
{
int i; /* Only one declaration allowed per line */
int in;
int my_rank;
int nprocs;
int status[3];
int nerrors;
/* Note: & not used in interpreter (no pointers). */
MPI_Comm_rank(MPI_COMM_WORLD, my_rank);
MPI_Comm_size(MPI_COMM_WORLD, nprocs);
if (my_rank > 0) {
/* All but node 0 sends their rank to node 0. */
MPI_Send(my_rank, 1, MPI_INT, 0, my_rank, MPI_COMM_WORLD);
} else {
nerrors = 0;
for (i=1; i<nprocs; i++) {
MPI_Recv(in, 1, MPI_INT, MPI_ANY_SOURCE, MPI_ANY_TAG,
MPI_COMM_WORLD, status);
if (status[MPI_TAG] != status[MPI_SOURCE]) {
nerrors = nerrors + 1; /* ++ and -- are not available */
/* report() acts like a printf statement when executed
* on the IUT. Otherwise, report() sends this info to the
* Test Manager, which will pass this info on to the
* Test Tool. All numeric values are printed using the
* %f format.
*/
report("Error: source:%.0f != Tag:%.0f",
status[MPI_SOURCE], status[MPI_TAG]);
} else if (in != status[MPI_TAG]) {
nerrors = nerrors + 1;
report("Error: received:%.0f != Source,Tag:%.0f",
in, status[MPI_TAG]);
}
}
if (nerrors == 0) {
report("Result: Pass");
} else {
report("Result: Fail: %.0f error out of %.0f messages",
nerrors, nprocs-1);
}
}
} /* end of script */
For the preceding test to pass, MPI process rank 0 must receive a message from each other process with the tag and the single integer both matching the source process rank. This test will usually be run with rank 0 owned by the Test Manager, although this is not required. Tests may have requirements as to the number of processes and the ordering of them in MPI_COMM_WORLD. These restrictions will be enforced either within the Test Manager or within the script itself when the tests are executed.
{
int i;
int p;
int a[5];
int answer;
int correct_answer;
int root;
int rank;
int nprocs;
int scale;
root = 0;
MPI_Comm_rank(MPI_COMM_WORLD, rank);
MPI_Comm_size(MPI_COMM_WORLD, nprocs);
if (nprocs > 5) {
report("Test skipped. Too many processes");
return;
}
scale = 1;
for (i=0;i<rank;i=i+1) {
scale = scale * 10;
}
for (i=0;i<5;i=i+1) {
a[i] = scale;
}
MPI_Reduce(a, answer, 10, MPI_INT, MPI_SUM, root, MPI_COMM_WORLD);
if (rank == root) {
/* Calculate the correct answer */
scale = 1;
correct_answer = 0;
for (p=0;p<nprocs;p=p+1) {
correct_answer = correct_answer + 5 * scale;
scale = scale * 10;
}
if (answer != correct_answer) {
report("Result: Fail, expected %.0f != %.0f", correct_answer, answer);
} else {
report("Result: Pass");
}
}
}
Each node in the preceding test fills an array and then calls the MPI_Reduce() routine. The correct answer is computed, based on the number of nodes involved in the test, and the root node verifies that the correct value has been received. The Test Manager's simulated MPI processes will need to actively participate in the algorithms used to perform this, and other, collective operations.
The interpreter must be able to execute test scripts that exercise all aspects of the IMPI protocol. Many test scripts have been written and more will be added. The first tests to be included focussed on the IMPI start-up protocol and the basic MPI routines, MPI_Init(), MPI_Finalize(), MPI_Send(), and MPI_Recv(). As of this writing, the testing of non-blocking send/recv calls are supported and testing of collectives will be available soon.