== racer test 1: racer on clients: oleg457-client.virtnet DURATION=300 ========================================================== 16:33:02 (1741897982) racers pids: 10406 10409 10410 10411 10412 10413 ./dir_create.sh: line 1: kill: (29492) - No such process kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] ./file_symlink.sh: line 1: kill: (30474) - No such process ./file_list.sh: line 1: kill: (27181) - No such process ./file_list.sh: line 1: kill: (27442) - No such process ./file_list.sh: line 1: kill: (19404) - No such process ./file_list.sh: line 1: kill: (27496) - No such process ./file_list.sh: line 1: kill: (28203) - No such process ./file_list.sh: line 1: kill: (27481) - No such process ./file_list.sh: line 1: kill: (28191) - No such process ./file_concat.sh: line 1: kill: (30822) - No such process ./file_exec.sh: line 1: kill: (10496) - No such process ./file_exec.sh: line 1: kill: (10642) - No such process ./file_exec.sh: line 1: kill: (10703) - No such process ./file_exec.sh: line 1: kill: (11171) - No such process ./file_exec.sh: line 1: kill: (11097) - No such process ./file_exec.sh: line 1: kill: (11410) - No such process ./file_exec.sh: line 1: kill: (11266) - No such process ./file_exec.sh: line 1: kill: (11669) - No such process ./file_exec.sh: line 1: kill: (11509) - No such process ./file_exec.sh: line 1: kill: (11045) - No such process ./file_exec.sh: line 1: kill: (11499) - No such process kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] ./file_chmod.sh: line 1: kill: (31076) - No such process kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] ./file_truncate.sh: line 1: kill: (31547) - No such process ./file_truncate.sh: line 1: kill: (31417) - No such process ./file_truncate.sh: line 1: kill: (31513) - No such process ./file_truncate.sh: line 1: kill: (31488) - No such process ./file_truncate.sh: line 1: kill: (31496) - No such process ./file_getxattr.sh: line 1: kill: (31700) - No such process ./file_getxattr.sh: line 1: kill: (31726) - No such process ./file_setxattr.sh: line 1: kill: (31954) - No such process ./file_setxattr.sh: line 1: kill: (31977) - No such process ./file_setxattr.sh: line 1: kill: (31944) - No such process ./dir_remote.sh: line 1: kill: (32065) - No such process ./dir_remote.sh: line 1: kill: (32062) - No such process ./dir_remote.sh: line 1: kill: (32059) - No such process ./file_fallocate.sh: line 1: kill: (32207) - No such process ./file_fallocate.sh: line 1: kill: (32215) - No such process ./file_fallocate.sh: line 1: kill: (32219) - No such process ./file_fallocate.sh: line 1: kill: (32212) - No such process ./file_fallocate.sh: line 1: kill: (32218) - No such process ./file_fallocate.sh: line 1: kill: (32231) - No such process ./file_fallocate.sh: line 1: kill: (32217) - No such process ./file_fallocate.sh: line 1: kill: (32220) - No such process file_create.sh: no process found file_create.sh: no process found file_create.sh: no process found file_create.sh: no process found file_rm.sh: no process found file_rm.sh: no process found file_rename.sh: no process found file_create.sh: no process found file_create.sh: no process found file_rm.sh: no process found file_link.sh: no process found file_rename.sh: no process found file_rm.sh: no process found file_rm.sh: no process found file_rename.sh: no process found file_symlink.sh: no process found file_rm.sh: no process found file_link.sh: no process found file_rename.sh: no process found file_rename.sh: no process found file_rename.sh: no process found file_link.sh: no process found file_list.sh: no process found file_link.sh: no process found file_symlink.sh: no process found file_symlink.sh: no process found file_link.sh: no process found file_link.sh: no process found file_symlink.sh: no process found file_list.sh: no process found file_list.sh: no process found file_symlink.sh: no process found file_list.sh: no process found file_symlink.sh: no process found file_exec.sh: no process found file_list.sh: no process found file_exec.sh: no process found file_chown.sh: no process found file_exec.sh: no process found file_exec.sh: no process found file_list.sh: no process found file_chown.sh: no process found file_chown.sh: no process found file_chmod.sh: no process found file_chmod.sh: no process found file_chmod.sh: no process found file_chown.sh: no process found file_mknod.sh: no process found file_mknod.sh: no process found file_exec.sh: no process found file_exec.sh: no process found file_chmod.sh: no process found file_mknod.sh: no process found file_chown.sh: no process found file_chown.sh: no process found file_truncate.sh: no process found file_chmod.sh: no process found file_mknod.sh: no process found file_truncate.sh: no process found file_truncate.sh: no process found file_chmod.sh: no process found file_mknod.sh: no process found file_truncate.sh: no process found file_delxattr.sh: no process found file_delxattr.sh: no process found file_delxattr.sh: no process found file_truncate.sh: no process found file_mknod.sh: no process found file_delxattr.sh: no process found file_delxattr.sh: no process found file_getxattr.sh: no process found file_getxattr.sh: no process found file_getxattr.sh: no process found file_truncate.sh: no process found file_getxattr.sh: no process found file_getxattr.sh: no process found file_setxattr.sh: no process found file_setxattr.sh: no process found file_setxattr.sh: no process found dir_remote.sh: no process found file_delxattr.sh: no process found file_setxattr.sh: no process found file_setxattr.sh: no process found dir_remote.sh: no process found dir_remote.sh: no process found file_getxattr.sh: no process found dir_migrate.sh: no process found dir_remote.sh: no process found dir_remote.sh: no process found file_setxattr.sh: no process found dir_migrate.sh: no process found dir_migrate.sh: no process found dir_migrate.sh: no process found file_fallocate.sh: no process found file_fallocate.sh: no process found dir_migrate.sh: no process found dir_remote.sh: no process found file_fallocate.sh: no process found file_fallocate.sh: no process found file_fallocate.sh: no process found dir_migrate.sh: no process found file_fallocate.sh: no process found file_create.sh: no process found file_create.sh: no process found file_create.sh: no process found file_create.sh: no process found file_rm.sh: no process found file_create.sh: no process found file_rm.sh: no process found file_rename.sh: no process found file_create.sh: no process found file_rm.sh: no process found file_link.sh: no process found file_rename.sh: no process found file_rename.sh: no process found file_rm.sh: no process found file_rm.sh: no process found file_symlink.sh: no process found file_rm.sh: no process found file_link.sh: no process found file_rename.sh: no process found file_link.sh: no process found file_symlink.sh: no process found file_list.sh: no process found file_list.sh: no process found file_rename.sh: no process found file_rename.sh: no process found file_link.sh: no process found file_exec.sh: no process found file_symlink.sh: no process found file_symlink.sh: no process found file_link.sh: no process found file_link.sh: no process found file_chown.sh: no process found file_list.sh: no process found file_list.sh: no process found file_symlink.sh: no process found file_symlink.sh: no process found file_chmod.sh: no process found file_exec.sh: no process found file_mknod.sh: no process found file_list.sh: no process found file_exec.sh: no process found file_list.sh: no process found file_chown.sh: no process found file_truncate.sh: no process found file_exec.sh: no process found file_chown.sh: no process found file_chmod.sh: no process found file_chown.sh: no process found file_mknod.sh: no process found file_delxattr.sh: no process found file_exec.sh: no process found file_exec.sh: no process found file_chmod.sh: no process found file_chmod.sh: no process found file_getxattr.sh: no process found file_mknod.sh: no process found file_chown.sh: no process found file_mknod.sh: no process found file_truncate.sh: no process found file_chown.sh: no process found file_truncate.sh: no process found file_setxattr.sh: no process found file_truncate.sh: no process found file_delxattr.sh: no process found file_chmod.sh: no process found file_delxattr.sh: no process found file_chmod.sh: no process found file_getxattr.sh: no process found dir_remote.sh: no process found file_mknod.sh: no process found file_delxattr.sh: no process found file_getxattr.sh: no process found file_setxattr.sh: no process found file_mknod.sh: no process found file_setxattr.sh: no process found file_getxattr.sh: no process found file_truncate.sh: no process found dir_migrate.sh: no process found dir_remote.sh: no process found dir_remote.sh: no process found file_setxattr.sh: no process found file_delxattr.sh: no process found dir_migrate.sh: no process found file_fallocate.sh: no process found file_truncate.sh: no process found dir_remote.sh: no process found dir_migrate.sh: no process found file_getxattr.sh: no process found file_fallocate.sh: no process found file_delxattr.sh: no process found file_fallocate.sh: no process found dir_migrate.sh: no process found file_getxattr.sh: no process found file_setxattr.sh: no process found file_fallocate.sh: no process found dir_remote.sh: no process found file_setxattr.sh: no process found dir_remote.sh: no process found dir_migrate.sh: no process found dir_migrate.sh: no process found file_fallocate.sh: no process found file_fallocate.sh: no process found file_create.sh: no process found file_create.sh: no process found file_create.sh: no process found file_rm.sh: no process found file_rm.sh: no process found file_rename.sh: no process found file_rename.sh: no process found file_link.sh: no process found file_create.sh: no process found file_rm.sh: no process found file_create.sh: no process found file_create.sh: no process found file_link.sh: no process found file_rename.sh: no process found file_symlink.sh: no process found file_rm.sh: no process found file_link.sh: no process found file_rm.sh: no process found file_list.sh: no process found file_symlink.sh: no process found file_symlink.sh: no process found file_rename.sh: no process found file_rename.sh: no process found file_rm.sh: no process found file_exec.sh: no process found file_list.sh: no process found file_chown.sh: no process found file_link.sh: no process found file_link.sh: no process found file_list.sh: no process found file_rename.sh: no process found file_symlink.sh: no process found file_link.sh: no process found file_list.sh: no process found file_chmod.sh: no process found file_symlink.sh: no process found file_symlink.sh: no process found file_exec.sh: no process found file_list.sh: no process found file_exec.sh: no process found file_mknod.sh: no process found file_list.sh: no process found file_chown.sh: no process found file_chown.sh: no process found file_chmod.sh: no process found file_truncate.sh: no process found file_exec.sh: no process found file_exec.sh: no process found file_mknod.sh: no process found file_chmod.sh: no process found file_exec.sh: no process found file_chown.sh: no process found file_truncate.sh: no process found file_delxattr.sh: no process found file_mknod.sh: no process found file_chown.sh: no process found file_chown.sh: no process found file_truncate.sh: no process found file_chmod.sh: no process found file_chmod.sh: no process found file_mknod.sh: no process found file_delxattr.sh: no process found file_getxattr.sh: no process found file_delxattr.sh: no process found file_chmod.sh: no process found file_mknod.sh: no process found file_truncate.sh: no process found file_getxattr.sh: no process found file_getxattr.sh: no process found file_setxattr.sh: no process found file_truncate.sh: no process found file_mknod.sh: no process found file_delxattr.sh: no process found file_setxattr.sh: no process found file_delxattr.sh: no process found dir_remote.sh: no process found file_setxattr.sh: no process found file_getxattr.sh: no process found file_truncate.sh: no process found dir_remote.sh: no process found dir_migrate.sh: no process found file_delxattr.sh: no process found dir_remote.sh: no process found file_getxattr.sh: no process found file_fallocate.sh: no process found dir_migrate.sh: no process found file_setxattr.sh: no process found file_setxattr.sh: no process found file_getxattr.sh: no process found dir_migrate.sh: no process found dir_remote.sh: no process found file_fallocate.sh: no process found file_setxattr.sh: no process found dir_remote.sh: no process found file_fallocate.sh: no process found dir_migrate.sh: no process found dir_remote.sh: no process found dir_migrate.sh: no process found file_fallocate.sh: no process found dir_migrate.sh: no process found file_fallocate.sh: no process found file_fallocate.sh: no process found file_create.sh: no process found file_rm.sh: no process found file_rename.sh: no process found file_link.sh: no process found file_create.sh: no process found file_create.sh: no process found file_symlink.sh: no process found file_create.sh: no process found file_list.sh: no process found file_create.sh: no process found file_rm.sh: no process found file_rm.sh: no process found file_create.sh: no process found file_rm.sh: no process found file_rename.sh: no process found file_rename.sh: no process found file_rename.sh: no process found file_rm.sh: no process found file_link.sh: no process found file_exec.sh: no process found file_link.sh: no process found file_symlink.sh: no process found file_link.sh: no process found file_symlink.sh: no process found file_rename.sh: no process found file_list.sh: no process found file_rm.sh: no process found file_link.sh: no process found file_chown.sh: no process found file_symlink.sh: no process found file_symlink.sh: no process found file_rename.sh: no process found file_list.sh: no process found file_link.sh: no process found file_list.sh: no process found file_list.sh: no process found file_chmod.sh: no process found file_symlink.sh: no process found file_exec.sh: no process found file_list.sh: no process found file_mknod.sh: no process found file_chown.sh: no process found file_exec.sh: no process found file_truncate.sh: no process found file_delxattr.sh: no process found file_chmod.sh: no process found file_chown.sh: no process found file_exec.sh: no process found file_exec.sh: no process found file_mknod.sh: no process found file_exec.sh: no process found file_truncate.sh: no process found file_getxattr.sh: no process found file_chown.sh: no process found file_chown.sh: no process found file_chmod.sh: no process found file_chown.sh: no process found file_setxattr.sh: no process found file_mknod.sh: no process found file_delxattr.sh: no process found file_chmod.sh: no process found file_chmod.sh: no process found file_chmod.sh: no process found dir_remote.sh: no process found file_getxattr.sh: no process found file_truncate.sh: no process found dir_migrate.sh: no process found file_mknod.sh: no process found file_mknod.sh: no process found file_mknod.sh: no process found file_delxattr.sh: no process found file_fallocate.sh: no process found file_truncate.sh: no process found file_delxattr.sh: no process found file_truncate.sh: no process found file_setxattr.sh: no process found file_truncate.sh: no process found file_getxattr.sh: no process found file_getxattr.sh: no process found file_setxattr.sh: no process found dir_remote.sh: no process found file_delxattr.sh: no process found file_delxattr.sh: no process found file_setxattr.sh: no process found dir_remote.sh: no process found file_getxattr.sh: no process found dir_migrate.sh: no process found dir_remote.sh: no process found file_getxattr.sh: no process found file_fallocate.sh: no process found file_setxattr.sh: no process found dir_migrate.sh: no process found file_setxattr.sh: no process found dir_migrate.sh: no process found file_fallocate.sh: no process found dir_remote.sh: no process found dir_remote.sh: no process found file_fallocate.sh: no process found dir_migrate.sh: no process found dir_migrate.sh: no process found file_fallocate.sh: no process found file_fallocate.sh: no process found kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] file_create.sh: no process found dir_create.sh: no process found file_rm.sh: no process found file_rename.sh: no process found file_link.sh: no process found file_symlink.sh: no process found file_list.sh: no process found file_concat.sh: no process found file_exec.sh: no process found file_chown.sh: no process found file_create.sh: no process found file_chmod.sh: no process found file_create.sh: no process found dir_create.sh: no process found dir_create.sh: no process found file_mknod.sh: no process found file_rm.sh: no process found file_rm.sh: no process found file_create.sh: no process found file_truncate.sh: no process found file_rename.sh: no process found dir_create.sh: no process found file_delxattr.sh: no process found file_create.sh: no process found file_rm.sh: no process found file_create.sh: no process found file_rename.sh: no process found file_getxattr.sh: no process found file_rename.sh: no process found dir_create.sh: no process found file_link.sh: no process found file_setxattr.sh: no process found dir_create.sh: no process found file_link.sh: no process found file_symlink.sh: no process found file_rm.sh: no process found file_symlink.sh: no process found file_rm.sh: no process found file_link.sh: no process found dir_remote.sh: no process found file_rename.sh: no process found file_list.sh: no process found file_symlink.sh: no process found file_list.sh: no process found file_concat.sh: no process found file_link.sh: no process found dir_migrate.sh: no process found file_concat.sh: no process found file_rename.sh: no process found file_symlink.sh: no process found file_exec.sh: no process found file_exec.sh: no process found file_fallocate.sh: no process found file_list.sh: no process found file_chown.sh: no process found file_link.sh: no process found file_list.sh: no process found file_chown.sh: no process found file_symlink.sh: no process found file_chmod.sh: no process found file_concat.sh: no process found file_concat.sh: no process found file_chmod.sh: no process found file_list.sh: no process found file_exec.sh: no process found file_mknod.sh: no process found file_concat.sh: no process found file_mknod.sh: no process found file_exec.sh: no process found file_chown.sh: no process found file_truncate.sh: no process found file_exec.sh: no process found file_truncate.sh: no process found file_chown.sh: no process found file_chmod.sh: no process found file_delxattr.sh: no process found file_delxattr.sh: no process found file_chmod.sh: no process found file_getxattr.sh: no process found file_chown.sh: no process found file_mknod.sh: no process found file_chmod.sh: no process found file_setxattr.sh: no process found file_getxattr.sh: no process found file_mknod.sh: no process found file_mknod.sh: no process found dir_remote.sh: no process found file_setxattr.sh: no process found file_truncate.sh: no process found file_truncate.sh: no process found file_truncate.sh: no process found file_delxattr.sh: no process found dir_migrate.sh: no process found dir_remote.sh: no process found file_getxattr.sh: no process found file_delxattr.sh: no process found file_fallocate.sh: no process found dir_migrate.sh: no process found file_delxattr.sh: no process found file_getxattr.sh: no process found file_setxattr.sh: no process found file_getxattr.sh: no process found file_setxattr.sh: no process found dir_remote.sh: no process found file_setxattr.sh: no process found dir_remote.sh: no process found file_fallocate.sh: no process found dir_migrate.sh: no process found dir_remote.sh: no process found dir_migrate.sh: no process found file_fallocate.sh: no process found dir_migrate.sh: no process found file_fallocate.sh: no process found file_fallocate.sh: no process found Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel racer cleanup ./file_exec.sh: line 16: 998 Illegal instruction $DIR/$file 0.$((RANDOM % 5 + 1)) 2> /dev/null sleeping 5 sec ... Waited 5, rc=2 sleeping 10 sec ... Waited 15, rc=2 sleeping 20 sec ... Waited 35, rc=2 sleeping 40 sec ... Waited 75, rc=2 sleeping 80 sec ... there should be NO racer processes: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.157@tcp:/lustre 7666232 51272 7162740 1% /mnt/lustre We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel racer cleanup sleeping 5 sec ... Waited 5, rc=2 sleeping 10 sec ... Waited 15, rc=2 sleeping 20 sec ... Waited 35, rc=2 sleeping 40 sec ... Waited 75, rc=2 sleeping 80 sec ... there should be NO racer processes: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.157@tcp:/lustre 7666232 51272 7162740 1% /mnt/lustre We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel racer cleanup sleeping 5 sec ... Waited 5, rc=2 sleeping 10 sec ... Waited 15, rc=2 sleeping 20 sec ... Waited 35, rc=2 sleeping 40 sec ... Waited 75, rc=2 sleeping 80 sec ... there should be NO racer processes: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.157@tcp:/lustre 7666232 51272 7162740 1% /mnt/lustre We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. pid=10406 rc=0 pid=10409 rc=0 pid=10410 rc=0 Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel racer cleanup sleeping 5 sec ... Waited 5, rc=2 sleeping 10 sec ... Waited 15, rc=2 sleeping 20 sec ... Waited 35, rc=2 sleeping 40 sec ... Waited 75, rc=2 sleeping 80 sec ... there should be NO racer processes: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.157@tcp:/lustre 7666232 51272 7162740 1% /mnt/lustre2 We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. pid=10411 rc=0 Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel racer cleanup sleeping 5 sec ... Waited 5, rc=2 sleeping 10 sec ... Waited 15, rc=2 sleeping 20 sec ... Waited 35, rc=2 sleeping 40 sec ... Waited 75, rc=2 sleeping 80 sec ... there should be NO racer processes: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.157@tcp:/lustre 7666232 51272 7162740 1% /mnt/lustre2 We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds.