-----============= acceptance-small: racer ============----- Tue Nov  5 05:16:47 EST 2024
mgs: CentOS Linux release 7.9.2009 (Core)
MGS_OS_VERSION_ID=7
MGS_OS_ID=centos
MGS_OS_VERSION_CODE=117440512
MGS_OS_ID_LIKE=rhel fedora centos
mds1: CentOS Linux release 7.9.2009 (Core)
MDS1_OS_ID_LIKE=rhel fedora centos
MDS1_OS_ID=centos
MDS1_OS_VERSION_ID=7
MDS1_OS_VERSION_CODE=117440512
ost1: CentOS Linux release 7.9.2009 (Core)
OST1_OS_VERSION_CODE=117440512
OST1_OS_VERSION_ID=7
OST1_OS_ID_LIKE=rhel fedora centos
OST1_OS_ID=centos
client: CentOS Linux release 7.9.2009 (Core)
CLIENT_OS_ID=centos
CLIENT_OS_ID_LIKE=rhel fedora centos
CLIENT_OS_VERSION_ID=7
CLIENT_OS_VERSION_CODE=117440512
excepting tests: 
racer: /home/green/git/lustre-release/lustre/tests/racer/racer.sh with 2 MDTs
=== racer: start setup 05:16:55 (1730801815) ===
Starting client oleg243-client.virtnet:  -o user_xattr,flock 192.168.202.143@tcp:/lustre /mnt/lustre2
Started clients oleg243-client.virtnet: 
192.168.202.143@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project)
oleg243-client.virtnet: executing check_config_client /mnt/lustre
oleg243-client.virtnet: Checking config lustre mounted on /mnt/lustre
Checking servers environments
Checking clients oleg243-client.virtnet environments
Using TIMEOUT=20
osc.lustre-OST0000-osc-ffff8800aa6d9800.idle_timeout=debug
osc.lustre-OST0000-osc-ffff88012a540000.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800aa6d9800.idle_timeout=debug
osc.lustre-OST0001-osc-ffff88012a540000.idle_timeout=debug
disable quota as required
oleg243-server: oleg243-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
osd-ldiskfs.track_declares_assert=1
=== racer: finish setup 05:17:10 (1730801830) ===
RACERDIRS=/mnt/lustre /mnt/lustre2
keep default fallocate mode: 0

== racer test 1: racer on clients: oleg243-client.virtnet DURATION=300 ========================================================== 05:17:12 (1730801832)
racers pids: 10211 10212 10213 10214 10216 10218
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
./file_rm.sh: line 1: kill: (4610) - No such process
./file_rm.sh: line 1: kill: (3955) - No such process
./file_list.sh: line 1: kill: (30151) - No such process
./file_exec.sh: line 1: kill: (10680) - No such process
./file_exec.sh: line 1: kill: (10750) - No such process
./file_exec.sh: line 1: kill: (10802) - No such process
./file_exec.sh: line 1: kill: (10501) - No such process
./file_exec.sh: line 1: kill: (10979) - No such process
./file_exec.sh: line 1: kill: (11007) - No such process
./file_exec.sh: line 1: kill: (11307) - No such process
./file_exec.sh: line 1: kill: (11376) - No such process
./file_chmod.sh: line 1: kill: (5932) - No such process
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
./file_mknod.sh: line 1: kill: (5976) - No such process
./file_mknod.sh: line 1: kill: (5980) - No such process
./file_truncate.sh: line 1: kill: (6168) - No such process
./file_truncate.sh: line 1: kill: (6196) - No such process
./file_delxattr.sh: line 1: kill: (6170) - No such process
./file_delxattr.sh: line 1: kill: (6265) - No such process
./file_delxattr.sh: line 1: kill: (6240) - No such process
./file_delxattr.sh: line 1: kill: (6248) - No such process
./file_delxattr.sh: line 1: kill: (6259) - No such process
./file_getxattr.sh: line 1: kill: (6380) - No such process
./file_getxattr.sh: line 1: kill: (6001) - No such process
./file_setxattr.sh: line 1: kill: (6414) - No such process
./file_setxattr.sh: line 1: kill: (6451) - No such process
./dir_remote.sh: line 1: kill: (6554) - No such process
./dir_remote.sh: line 1: kill: (6399) - No such process
./dir_remote.sh: line 1: kill: (6516) - No such process
./dir_migrate.sh: line 1: kill: (6125) - No such process
./file_fallocate.sh: line 1: kill: (6666) - No such process
./file_fallocate.sh: line 1: kill: (6622) - No such process
./file_fallocate.sh: line 1: kill: (6591) - No such process
./file_fallocate.sh: line 1: kill: (6654) - No such process
file_create.sh: no process found
dir_create.sh: no process found
file_rm.sh: no process found
file_create.sh: no process found
file_rename.sh: no process found
dir_create.sh: no process found
file_link.sh: no process found
file_rm.sh: no process found
file_symlink.sh: no process found
file_list.sh: no process found
file_rename.sh: no process found
file_create.sh: no process found
file_create.sh: no process found
file_concat.sh: no process found
file_create.sh: no process found
file_link.sh: no process found
file_exec.sh: no process found
dir_create.sh: no process found
dir_create.sh: no process found
dir_create.sh: no process found
file_symlink.sh: no process found
file_rm.sh: no process found
file_rm.sh: no process found
file_list.sh: no process found
file_chown.sh: no process found
file_rm.sh: no process found
file_rename.sh: no process found
file_chmod.sh: no process found
file_create.sh: no process found
file_rename.sh: no process found
file_concat.sh: no process found
file_link.sh: no process found
file_link.sh: no process found
dir_create.sh: no process found
file_rename.sh: no process found
file_exec.sh: no process found
file_mknod.sh: no process found
file_symlink.sh: no process found
file_rm.sh: no process found
file_symlink.sh: no process found
file_link.sh: no process found
file_truncate.sh: no process found
file_list.sh: no process found
file_chown.sh: no process found
file_symlink.sh: no process found
file_rename.sh: no process found
file_concat.sh: no process found
file_delxattr.sh: no process found
file_chmod.sh: no process found
file_list.sh: no process found
file_list.sh: no process found
file_exec.sh: no process found
file_link.sh: no process found
file_mknod.sh: no process found
file_getxattr.sh: no process found
file_truncate.sh: no process found
file_chown.sh: no process found
file_concat.sh: no process found
file_concat.sh: no process found
file_symlink.sh: no process found
file_setxattr.sh: no process found
file_exec.sh: no process found
file_list.sh: no process found
file_exec.sh: no process found
dir_remote.sh: no process found
file_delxattr.sh: no process found
file_concat.sh: no process found
file_chown.sh: no process found
dir_migrate.sh: no process found
file_chmod.sh: no process found
file_chown.sh: no process found
file_exec.sh: no process found
file_chmod.sh: no process found
file_getxattr.sh: no process found
file_fallocate.sh: no process found
file_mknod.sh: no process found
file_mknod.sh: no process found
file_chown.sh: no process found
file_chmod.sh: no process found
file_truncate.sh: no process found
file_truncate.sh: no process found
file_setxattr.sh: no process found
file_mknod.sh: no process found
file_chmod.sh: no process found
file_delxattr.sh: no process found
file_truncate.sh: no process found
file_delxattr.sh: no process found
dir_remote.sh: no process found
file_mknod.sh: no process found
file_delxattr.sh: no process found
file_getxattr.sh: no process found
dir_migrate.sh: no process found
file_getxattr.sh: no process found
file_getxattr.sh: no process found
file_truncate.sh: no process found
file_fallocate.sh: no process found
file_setxattr.sh: no process found
file_setxattr.sh: no process found
file_setxattr.sh: no process found
file_delxattr.sh: no process found
dir_remote.sh: no process found
dir_remote.sh: no process found
file_getxattr.sh: no process found
dir_migrate.sh: no process found
dir_migrate.sh: no process found
dir_remote.sh: no process found
Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit
layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel
layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel
layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel
racer cleanup
sleeping 5 sec ...
there should be NO racer processes:
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
Filesystem                  1K-blocks  Used Available Use% Mounted on
192.168.202.143@tcp:/lustre   7666232 70336   7143284   1% /mnt/lustre
We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds.
file_setxattr.sh: no process found
file_fallocate.sh: no process found
dir_migrate.sh: no process found
dir_remote.sh: no process found
file_fallocate.sh: no process found
file_fallocate.sh: no process found
dir_migrate.sh: no process found
file_fallocate.sh: no process found
Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit
layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel
layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel
layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel
racer cleanup
./file_exec.sh: line 16: 19993 Illegal instruction     $DIR/$file 0.$((RANDOM % 5 + 1)) 2> /dev/null
sleeping 5 sec ...
there should be NO racer processes:
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
Filesystem                  1K-blocks  Used Available Use% Mounted on
192.168.202.143@tcp:/lustre   7666232 70336   7143284   1% /mnt/lustre2
We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds.
Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit
layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel
layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel
layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel
racer cleanup
sleeping 5 sec ...
there should be NO racer processes:
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
Filesystem                  1K-blocks  Used Available Use% Mounted on
192.168.202.143@tcp:/lustre   7666232 70336   7143284   1% /mnt/lustre2
We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds.
Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit
layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel
layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel
layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel
racer cleanup
sleeping 5 sec ...
there should be NO racer processes:
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
Filesystem                  1K-blocks  Used Available Use% Mounted on
192.168.202.143@tcp:/lustre   7666232 70336   7143284   1% /mnt/lustre2
We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds.
Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit
layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel
layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel
layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel
racer cleanup
sleeping 5 sec ...
there should be NO racer processes:
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
Filesystem                  1K-blocks  Used Available Use% Mounted on
192.168.202.143@tcp:/lustre   7666232 70336   7143284   1% /mnt/lustre
We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds.
Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit
layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel
layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel
layout: raid0 raid0 pfl pfl pfl flr flr flr sel sel sel
racer cleanup
sleeping 5 sec ...
there should be NO racer processes:
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
Filesystem                  1K-blocks  Used Available Use% Mounted on
192.168.202.143@tcp:/lustre   7666232 70336   7143284   1% /mnt/lustre
We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds.
pid=10211 rc=0
pid=10212 rc=0
pid=10213 rc=0
pid=10214 rc=0
pid=10216 rc=0
pid=10218 rc=0
PASS 1 (313s)

== racer test 2: racer rename: oleg243-client.virtnet DURATION=300 ========================================================== 05:22:25 (1730802145)
starting on oleg243-client.virtnet:/mnt/lustre/racer0 with: dir_create dir_remote file_rename file_rename file_create file_rm
starting on oleg243-client.virtnet:/mnt/lustre/racer1 with: dir_create dir_remote file_rename file_rename file_create file_rm
starting on oleg243-client.virtnet:/mnt/lustre2/racer0 with: dir_create dir_remote file_rename file_rename file_create file_rm
starting on oleg243-client.virtnet:/mnt/lustre2/racer1 with: dir_create dir_remote file_rename file_rename file_create file_rm
racers pids: 7517 7518 7519 7520
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
./dir_create.sh: line 1: kill: (19547) - No such process
./dir_create.sh: line 1: kill: (19458) - No such process
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
dir_create.sh: no process found
dir_create.sh: no process found
dir_remote.sh: no process found
dir_remote.sh: no process found
file_rename.sh: no process found
file_rename.sh: no process found
file_rename.sh: no process found
dir_create.sh: no process found
file_rename.sh: no process found
file_create.sh: no process found
dir_remote.sh: no process found
file_rm.sh: no process found
file_create.sh: no process found
dir_create.sh: no process found
file_rename.sh: no process found
file_rm.sh: no process found
file_rename.sh: no process found
dir_remote.sh: no process found
file_create.sh: no process found
file_rename.sh: no process found
file_rename.sh: no process found
file_rm.sh: no process found
file_create.sh: no process found
file_rm.sh: no process found
Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit
layout: raid0 raid0 pfl pfl pfl
layout: raid0 raid0 pfl pfl pfl
layout: raid0 raid0 pfl pfl pfl
racer cleanup
sleeping 5 sec ...
there should be NO racer processes:
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
Filesystem                  1K-blocks  Used Available Use% Mounted on
192.168.202.143@tcp:/lustre   7666232 43060   7170056   1% /mnt/lustre2
We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds.
Running /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds. CTRL-C to exit
layout: raid0 raid0 pfl pfl pfl
layout: raid0 raid0 pfl pfl pfl
layout: raid0 raid0 pfl pfl pfl
racer cleanup
sleeping 5 sec ...
there should be NO racer processes:
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
Filesystem                  1K-blocks  Used Available Use% Mounted on
192.168.202.143@tcp:/lustre   7666232 43060   7170056   1% /mnt/lustre2
We survived /home/green/git/lustre-release/lustre/tests/racer/racer.sh for 300 seconds.