-----============= acceptance-small: racer ============----- Fri Aug 9 21:17:45 EDT 2024 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 excepting tests: racer: /home/green/git/lustre-release/lustre/tests/racer/racer.sh with 2 MDTs === racer: start setup 21:17:50 (1723252670) === Starting client oleg410-client.virtnet: -o user_xattr,flock 192.168.204.110@tcp:/lustre /mnt/lustre2 Started clients oleg410-client.virtnet: 192.168.204.110@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project) oleg410-client.virtnet: executing check_config_client /mnt/lustre oleg410-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg410-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff880000020800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800a9fd5800.idle_timeout=debug osc.lustre-OST0001-osc-ffff880000020800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800a9fd5800.idle_timeout=debug disable quota as required oleg410-server: oleg410-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all osd-ldiskfs.track_declares_assert=1 === racer: finish setup 21:17:57 (1723252677) === RACERDIRS=/mnt/lustre /mnt/lustre2 keep default fallocate mode: 0 == racer test 1: racer on clients: oleg410-client.virtnet DURATION=300 ========================================================== 21:17:58 (1723252678) racers pids: 9714 9715 9716 9717 9720 9723 ./file_create.sh: line 60: ((: 2/0: division by 0 (error token is "0") ./file_create.sh: line 1: kill: (22212) - No such process ./file_create.sh: line 1: kill: (22329) - No such process kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] ./file_rm.sh: line 1: kill: (22511) - No such process kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] ./file_symlink.sh: line 1: kill: (23937) - No such process ./file_symlink.sh: line 1: kill: (24206) - No such process kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] ./file_symlink.sh: line 1: kill: (23995) - No such process ./file_list.sh: line 1: kill: (21634) - No such process ./file_list.sh: line 1: kill: (20200) - No such process ./file_list.sh: line 1: kill: (20294) - No such process ./file_list.sh: line 1: kill: (19051) - No such process ./file_list.sh: line 1: kill: (19834) - No such process ./file_list.sh: line 1: kill: (19062) - No such process ./file_list.sh: line 1: kill: (20173) - No such process ./file_list.sh: line 1: kill: (18166) - No such process ./file_concat.sh: line 1: kill: (24679) - No such process ./file_exec.sh: line 1: kill: (9992) - No such process ./file_exec.sh: line 1: kill: (9899) - No such process ./file_exec.sh: line 1: kill: (10227) - No such process ./file_exec.sh: line 1: kill: (10302) - No such process ./file_exec.sh: line 1: kill: (10617) - No such process ./file_exec.sh: line 1: kill: (10663) - No such process ./file_exec.sh: line 1: kill: (10679) - No such process ./file_exec.sh: line 1: kill: (10913) - No such process ./file_exec.sh: line 1: kill: (10789) - No such process ./file_exec.sh: line 1: kill: (10698) - No such process ./file_chown.sh: line 1: kill: (24937) - No such process ./file_chown.sh: line 1: kill: (24798) - No such process ./file_chown.sh: line 1: kill: (24868) - No such process ./file_chmod.sh: line 1: kill: (24849) - No such process kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] ./file_chmod.sh: line 1: kill: (24923) - No such process ./file_chmod.sh: line 1: kill: (25032) - No such process kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] ./file_mknod.sh: line 1: kill: (25214) - No such process ./file_mknod.sh: line 1: kill: (25277) - No such process ./file_mknod.sh: line 1: kill: (25363) - No such process ./file_truncate.sh: line 1: kill: (25538) - No such process ./file_truncate.sh: line 1: kill: (25535) - No such process ./file_truncate.sh: line 1: kill: (25450) - No such process ./file_truncate.sh: line 1: kill: (25427) - No such process ./file_truncate.sh: line 1: kill: (25485) - No such process ./file_delxattr.sh: line 1: kill: (25606) - No such process ./file_delxattr.sh: line 1: kill: (25504) - No such process ./file_delxattr.sh: line 1: kill: (25657) - No such process ./file_delxattr.sh: line 1: kill: (25530) - No such process ./file_delxattr.sh: line 1: kill: (25677) - No such process ./file_delxattr.sh: line 1: kill: (25680) - No such process ./file_getxattr.sh: line 1: kill: (25470) - No such process ./file_getxattr.sh: line 1: kill: (25770) - No such process ./file_getxattr.sh: line 1: kill: (25780) - No such process ./file_getxattr.sh: line 1: kill: (25777) - No such process ./file_getxattr.sh: line 1: kill: (25759) - No such process ./file_setxattr.sh: line 1: kill: (25884) - No such process ./file_setxattr.sh: line 1: kill: (25853) - No such process ./file_setxattr.sh: line 1: kill: (25871) - No such process ./file_setxattr.sh: line 1: kill: (25887) - No such process ./file_setxattr.sh: line 1: kill: (25855) - No such process ./file_setxattr.sh: line 1: kill: (25865) - No such process ./file_setxattr.sh: line 1: kill: (25849) - No such process ./file_setxattr.sh: line 1: kill: (25924) - No such process ./file_setxattr.sh: line 1: kill: (25895) - No such process ./dir_remote.sh: line 1: kill: (25989) - No such process ./dir_remote.sh: line 1: kill: (26009) - No such process ./dir_remote.sh: line 1: kill: (25979) - No such process ./dir_remote.sh: line 1: kill: (26002) - No such process ./dir_remote.sh: line 1: kill: (26014) - No such process ./dir_remote.sh: line 1: kill: (25985) - No such process ./dir_remote.sh: line 1: kill: (26030) - No such process ./dir_remote.sh: line 1: kill: (26032) - No such process ./dir_migrate.sh: line 1: kill: (26097) - No such process ./file_fallocate.sh: line 1: kill: (26170) - No such process ./file_fallocate.sh: line 1: kill: (26172) - No such process ./file_fallocate.sh: line 1: kill: (26181) - No such process ./file_fallocate.sh: line 1: kill: (26177) - No such process ./file_fallocate.sh: line 1: kill: (26169) - No such process ./file_fallocate.sh: line 1: kill: (26171) - No such process ./file_fallocate.sh: line 1: kill: (26183) - No such process ./file_fallocate.sh: line 1: kill: (26173) - No such process ./file_fallocate.sh: line 1: kill: (26174) - No such process file_create.sh: no process found dir_create.sh: no process found file_create.sh: no process found file_create.sh: no process found file_create.sh: no process found file_rm.sh: no process found dir_create.sh: no process found file_create.sh: no process found file_create.sh: no process found dir_create.sh: no process found file_rename.sh: no process found dir_create.sh: no process found file_rm.sh: no process found dir_create.sh: no process found dir_create.sh: no process found file_rm.sh: no process found file_rename.sh: no process found file_link.sh: no process found file_rename.sh: no process found file_rm.sh: no process found file_rm.sh: no process found file_rm.sh: no process found file_link.sh: no process found file_rename.sh: no process found file_symlink.sh: no process found file_rename.sh: no process found file_symlink.sh: no process found file_link.sh: no process found file_list.sh: no process found file_rename.sh: no process found file_link.sh: no process found file_link.sh: no process found file_concat.sh: no process found file_symlink.sh: no process found file_symlink.sh: no process found file_symlink.sh: no process found file_link.sh: no process found file_list.sh: no process found file_exec.sh: no process found file_list.sh: no process found file_concat.sh: no process found file_list.sh: no process found file_concat.sh: no process found file_list.sh: no process found file_symlink.sh: no process found file_exec.sh: no process found file_chown.sh: no process found file_exec.sh: no process found file_concat.sh: no process found file_list.sh: no process found file_concat.sh: no process found file_chown.sh: no process found file_exec.sh: no process found file_chown.sh: no process found file_exec.sh: no process found file_chmod.sh: no process found file_chown.sh: no process found file_concat.sh: no process found file_chmod.sh: no process found file_mknod.sh: no process found file_chmod.sh: no process found file_exec.sh: no process found file_mknod.sh: no process found file_chown.sh: no process found file_chown.sh: no process found file_mknod.sh: no process found file_truncate.sh: no process found file_chmod.sh: no process found file_chmod.sh: no process found file_truncate.sh: no process found file_chmod.sh: no process found file_mknod.sh: no process found file_truncate.sh: no process found file_delxattr.sh: no process found file_delxattr.sh: no process found file_truncate.sh: no process found file_mknod.sh: no process found file_getxattr.sh: no process found file_delxattr.sh: no process found file_getxattr.sh: no process found file_mknod.sh: no process found file_delxattr.sh: no process found file_setxattr.sh: no process found file_getxattr.sh: no process found file_setxattr.sh: no process found file_truncate.sh: no process found file_truncate.sh: no process found dir_remote.sh: no process found file_getxattr.sh: no process found file_delxattr.sh: no process found file_delxattr.sh: no process found file_setxattr.sh: no process found dir_migrate.sh: no process found file_getxattr.sh: no process found dir_remote.sh: no process found file_setxattr.sh: no process found file_fallocate.sh: no process found dir_remote.sh: no process found file_getxattr.sh: no process found dir_migrate.sh: no process found file_setxattr.sh: no process found dir_migrate.sh: no process found dir_remote.sh: no process found file_fallocate.sh: no process found file_setxattr.sh: no process found file_fallocate.sh: no process found dir_remote.sh: no process found dir_remote.sh: no process found dir_migrate.sh: no process found dir_migrate.sh: no process found dir_migrate.sh: no process found file_fallocate.sh: no process found file_fallocate.sh: no process found file_fallocate.sh: no process found Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel racer cleanup sleeping 5 sec ... there should be NO racer processes: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.110@tcp:/lustre 7666232 56648 7157224 1% /mnt/lustre2 We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel racer cleanup sleeping 5 sec ... there should be NO racer processes: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.110@tcp:/lustre 7666232 56648 7157224 1% /mnt/lustre We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel racer cleanup sleeping 5 sec ... there should be NO racer processes: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.110@tcp:/lustre 7666232 56648 7157224 1% /mnt/lustre We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel racer cleanup sleeping 5 sec ... there should be NO racer processes: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.110@tcp:/lustre 7666232 56648 7157224 1% /mnt/lustre2 We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel racer cleanup sleeping 5 sec ... there should be NO racer processes: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.110@tcp:/lustre 7666232 56648 7157224 1% /mnt/lustre We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. pid=9714 rc=0 pid=9715 rc=0 pid=9716 rc=0 pid=9717 rc=0 Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel racer cleanup sleeping 5 sec ... there should be NO racer processes: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.110@tcp:/lustre 7666232 56648 7157224 1% /mnt/lustre2 We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. pid=9720 rc=0 pid=9723 rc=0 PASS 1 (311s) == racer test 2: racer rename: oleg410-client.virtnet DURATION=300 ========================================================== 21:23:09 (1723252989) starting on oleg410-client.virtnet:/mnt/lustre/racer0 with: dir_create dir_remote file_rename file_rename file_create file_rm starting on oleg410-client.virtnet:/mnt/lustre/racer1 with: dir_create dir_remote file_rename file_rename file_create file_rm starting on oleg410-client.virtnet:/mnt/lustre2/racer0 with: dir_create dir_remote file_rename file_rename file_create file_rm starting on oleg410-client.virtnet:/mnt/lustre2/racer1 with: dir_create dir_remote file_rename file_rename file_create file_rm racers pids: 27026 27027 27028 27030 ./file_create.sh: line 60: ((: 10/0: division by 0 (error token is "0") kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] ./file_rename.sh: line 1: kill: (15566) - No such process ./file_rename.sh: line 1: kill: (15499) - No such process ./file_rename.sh: line 1: kill: (15344) - No such process kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] dir_create.sh: no process found dir_remote.sh: no process found dir_create.sh: no process found file_rename.sh: no process found dir_create.sh: no process found dir_create.sh: no process found dir_remote.sh: no process found file_rename.sh: no process found dir_remote.sh: no process found file_rename.sh: no process found dir_remote.sh: no process found file_create.sh: no process found file_rename.sh: no process found file_rename.sh: no process found file_rm.sh: no process found file_rename.sh: no process found file_rename.sh: no process found file_rename.sh: no process found file_create.sh: no process found file_create.sh: no process found file_create.sh: no process found file_rm.sh: no process found file_rm.sh: no process found file_rm.sh: no process found Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit layout: raid0 raid0 pfl pfl pfl layout: raid0 raid0 pfl pfl pfl layout: raid0 raid0 pfl pfl pfl racer cleanup sleeping 5 sec ... there should be NO racer processes: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.110@tcp:/lustre 7666232 17312 7194792 1% /mnt/lustre2 We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit layout: raid0 raid0 pfl pfl pfl layout: raid0 raid0 pfl pfl pfl layout: raid0 raid0 pfl pfl pfl racer cleanup sleeping 5 sec ... there should be NO racer processes: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.110@tcp:/lustre 7666232 17312 7194792 1% /mnt/lustre We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. pid=27026 rc=0 Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit layout: raid0 raid0 pfl pfl pfl layout: raid0 raid0 pfl pfl pfl layout: raid0 raid0 pfl pfl pfl racer cleanup sleeping 5 sec ... there should be NO racer processes: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.110@tcp:/lustre 7666232 17312 7194792 1% /mnt/lustre2 We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit layout: raid0 raid0 pfl pfl pfl layout: raid0 raid0 pfl pfl pfl layout: raid0 raid0 pfl pfl pfl racer cleanup sleeping 5 sec ... there should be NO racer processes: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.110@tcp:/lustre 7666232 17312 7194792 1% /mnt/lustre We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. pid=27027 rc=0 pid=27028 rc=0 pid=27030 rc=0 PASS 2 (308s) == racer test complete, duration 631 sec ================= 21:28:17 (1723253297) === racer: start cleanup 21:28:17 (1723253297) === Stopping clients: oleg410-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg410-client.virtnet /mnt/lustre2 opts: === racer: finish cleanup 21:28:28 (1723253308) ===