Introduction :

  The ior_survey script can be used to test the performance of the lustre
file systems. It uses IOR (Interleaved Or Random), a script used for testing
performance of parallel file systems using various interfaces and access 
patterns.  IOR uses MPI for process synchronization.

General Description:

  ior_mpiio is a parallel file system test developed by the SIOP (Scalable
I/O Project) at LLNL. This parallel program performs parallel writes and
reads to/from a file using MPI-IO and reporting the throughput rates.

  MPI is used for process synchronization.  Under the control of compile-time
defined constants (and, to a lesser extent, environment variables), I/O is done
via MPI-IO. The data are written and read using independent parallel transfers
of equal-sized blocks of contiguous bytes that cover the file with no gaps and
that do not overlap each other. The test consists of creating a new file, 
writing it with data, then reading the data back.

  The data written are C integers. If the program runs successfully to
completion, it returns 0. If a problem is detected with any I/O routine, the
program exits with a value of IO_ERR.

  If a non-I/O problem is detected, the program exits with a value of
INTERNAL_ERR (this can be caused by a bug in the test program, or a problem in
MPI, or by inconsistencies in the environment variable settings).

Requirements :
	To run the ior_survey script following items are required.

1: IOR

  The IOR test should be obtained at
  ftp://ftp.llnl.gov/pub/siop/ior/

2: pdsh
	The tarball can be obtained from
   http://sourceforge.net/project/showfiles.php?group_id=33530&package_id=183641

3: pdsh-rcmd-ssh module
	The rpm for this could be found at
   http://sourceforge.net/project/showfiles.php?group_id=33530&package_id=183641

4: lam/mpi
	The tarball can be obtained from
   http://www.lam-mpi.org/7.1/download.php

5: You need to be a non-root user to execute the script and should have the 
   super-user privileges.

6: The user should have login on all the nodes without password on which the
   test is going to be run.



To make an entry into the sudoers file :

1: Become super user (root)

2: type visudo

3: make an entry as
  username   ALL=(ALL) NOPASSWD: ALL //(username is the name of the user)
 

Building IOR :

  Type 'gmake mpiio' from the IOR/ directory.  In
  IOR/src/C, the file Makefile.config currently has settings for AIX, Linux,
  OSF1 (TRU64), and IRIX64 to model on.  Note that MPI must be present for
  building/running IOR, and that MPI I/O must be available for MPI I/O, HDF5,
  and Parallel netCDF builds.  As well, HDF5 and Parallel netCDF libraries are
  necessary for those builds.  All IOR builds include the POSIX interface.

  Copy the IOR binary file in IOR/src/C/ to /usr/local/sbin/ using

	sudo cp IOR/src/C/IOR /usr/local/sbin/



Installing pdsh and pdsh-rcmd-ssh module :

1: Download the pdsh tarball

2: untar it using tar -xzvf (if tar.gz) or tar -xjvf(if tar.bz2) 

3: go to the pdsh directory and type ./bootstrap

4: configure it using the following command

	./configure --with-ssh

5: Build it using "make"

6: Install it using "sudo make install"

7: Download the pdsh-rcmd-ssh rpm

8: Install the rpm using "rpm -ivh pdsh-rcmd-ssh*"


Installing lam/mpi :

1: Download the lam tarball

2: untar it using tar -xzvf (if tar.gz) or tar -xjvf(if tar.bz2)

3: go to the lam directory and type ./configure

4: Build it using "make"

5: Install it using "sudo make install"

	The lam, IOR, pdsh should be installed on all the nodes on which the
	test is going to be run.
	
Note: Please make sure that you have installed the same version of lam on all
the nodes on which the test is going to be run.



Running the ior_survey script :

1: Lustre should be mounted at /mnt/lustre. Do 
	"touch /mnt/lustre/ior_survey_testfile"

2: Make a hostfile in which the ip addresses of all the nodes are present on
   the node from where the script is going to be executed.

3: run the lam using "lamboot -v -d hostfile". This will start lamd on all the
   nodes.

4: run the ior_survey script using "./ior_survey"

Note:
   The node names of the clients should be like rhea1, rhea2, rhea3, so on.
   The name of the cluster (1st part of the node name) should be set in the 
   ior_survey script in the cluster name field.
   e.g.  cluster=rhea //name of the cluster

   The client node numbers should be set as last part of the node name i.e.
   numeral part.
   e.g. client=(1)   //to run test on one node only node1.
	client=(1-2) //to run test on two nodes node1, node2.

	Please note that the hostfile should contain the ip addresses of only
   those nodes on which the lustre filesystem is mounted i.e. clients are 
   mounted.

	The details of the test can be found on the node from where the
   test was run as /tmp/ior_survey_run_date@start_time_nodename.detail

	The output of the IOR looks like
	
host1: access    bw(MiB/s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   iter
host1: ------    ---------  ---------- ---------  --------   --------   --------   ----
host1: write     1.58       2097152    1024.00    0.000873   1299.37    0.000132   0
host1:
host1: Max Write: 1.58 MiB/sec (1.65 MB/sec)
 
	where, 
		host1 : node on which the test is run
		access: the test which is run (write, rewrite, read, reread)
		bw    : band width
		block : total size to be written
		xfer  : block size to transfer here 1MB
		open  : time taken for open
		close : time taken for close
		wr/rd : time taken for read/write
		iteration : iteration no.
		Max write : Max_write speed obtained
		
Note : MB is defined as 1,000,000 bytes and MiB is 1,048,576 bytes.

	The summary of the test can be found on the node from where the
   test was run as /tmp/ior_survey_run_date@start_time_nodename.summary
   It contains the tests run and the status of those tests.


Instructions for graphing IOR results

   The plot-ior.pl script will plot the results from the .detail file
   generated by ior-survery. It will create a data file for writes as
   /tmp/ior_survey_run_date@start_time_nodename.detail.dat1 and for reads
   as /tmp/ior_survey_run_date@start_time_nodename.detail.dat2 and gnuplot
   file as /tmp/ior_survey_run_date@start_time_nodename.detail.scr.

   $ perl iokit-parse-ior /tmp/ior_survey_run_date@start_time_nodename.detail

