-----============= acceptance-small: conf-sanity ============----- Thu Jun 5 07:06:59 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg401-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex: No such file or directory excepting tests: 32b 32c 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg401-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg401-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg401-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg401-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg401-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg401-server oleg401-server: oleg401-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory loading modules on: 'oleg401-server' oleg401-server: oleg401-server.virtnet: executing load_modules_local oleg401-server: Loading modules from /home/green/git/lustre-release/lustre oleg401-server: detected 4 online CPUs by sysfs oleg401-server: MODOPTS_LIBCFS= oleg401-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg401-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg401-server: oleg401-server.virtnet: executing set_default_debug -1 all pdsh@oleg401-client: oleg401-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg401-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg401-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg401-server: oleg401-server.virtnet: executing set_default_debug -1 all pdsh@oleg401-client: oleg401-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg401-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg401-server: oleg401-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg401-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg401-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg401-server stop mds service on oleg401-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg401-server == conf-sanity test 45: long unlink handling in ptlrpcd == 07:08:19 (1749121699) start mds service on oleg401-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg401-server: oleg401-server.virtnet: executing set_default_debug -1 all pdsh@oleg401-client: oleg401-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg401-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg401-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg401-server: oleg401-server.virtnet: executing set_default_debug -1 all pdsh@oleg401-client: oleg401-server: ssh exited with exit code 1 Started lustre-OST0000 oleg401-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg401-client.virtnet: -o user_xattr,flock 192.168.204.101@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg401-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg401-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: '/mnt/lustre': Cannot send after transport endpoint shutdown fail_loc=0x0 start mds service on oleg401-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg401-server: oleg401-server.virtnet: executing set_default_debug -1 all pdsh@oleg401-client: oleg401-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg401-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg401-client.virtnet: -o user_xattr,flock 192.168.204.101@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg401-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg401-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg401-server stop mds service on oleg401-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg401-server unloading modules via unload_modules_local on: 'oleg401-server' oleg401-server: oleg401-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg401-client: oleg401-client: ssh exited with exit code 2 pdsh@oleg401-client: oleg401-server: ssh exited with exit code 2 pdsh@oleg401-client: oleg401-client: ssh exited with exit code 2 pdsh@oleg401-client: oleg401-server: ssh exited with exit code 2 PASS 45 (136s) == conf-sanity test 69: replace an OST with the same index ========================================================== 07:10:35 (1749121835) start mds service on oleg401-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg401-server' oleg401-server: oleg401-server.virtnet: executing load_modules_local oleg401-server: Loading modules from /home/green/git/lustre-release/lustre oleg401-server: detected 4 online CPUs by sysfs oleg401-server: MODOPTS_LIBCFS= oleg401-server: Force libcfs to create 2 CPU partitions oleg401-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg401-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg401-server: oleg401-server.virtnet: executing set_default_debug -1 all pdsh@oleg401-client: oleg401-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg401-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg401-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg401-server: oleg401-server.virtnet: executing set_default_debug -1 all pdsh@oleg401-client: oleg401-server: ssh exited with exit code 1 Started lustre-OST0000 oleg401-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg401-client.virtnet: -o user_xattr,flock 192.168.204.101@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 11872 inodes available. Want 99872. rc=0 - open/close 1618 (time 1749121874.96 total 10.00 last 161.80) - open/close 3634 (time 1749121884.96 total 20.00 last 201.54) - open/close 5440 (time 1749121894.96 total 30.01 last 180.56) - open/close 7127 (time 1749121904.97 total 40.01 last 168.64) - open/close 9231 (time 1749121914.97 total 50.01 last 210.38) total: 10000 open/close in 53.24 seconds: 187.82 ops/second - unlinked 0 (time 1749121919 ; total 0 ; last 0) total: 10000 unlinks in 29 seconds: 344.827576 unlinks/second - open/close 2112 (time 1749121960.11 total 10.00 last 211.17) - open/close 4588 (time 1749121970.12 total 20.00 last 247.54) - open/close 6746 (time 1749121980.12 total 30.01 last 215.70) - open/close 8904 (time 1749121990.12 total 40.01 last 215.80) total: 10000 open/close in 45.52 seconds: 219.69 ops/second - unlinked 0 (time 1749121996 ; total 0 ; last 0) total: 10000 unlinks in 29 seconds: 344.827576 unlinks/second - open/close 1809 (time 1749122036.59 total 10.00 last 180.87) - open/close 3801 (time 1749122046.59 total 20.00 last 199.15) - open/close 5936 (time 1749122056.60 total 30.01 last 213.40) - open/close 8258 (time 1749122066.60 total 40.02 last 232.05) total: 10000 open/close in 48.98 seconds: 204.18 ops/second - unlinked 0 (time 1749122076 ; total 0 ; last 0) total: 10000 unlinks in 24 seconds: 416.666656 unlinks/second - open/close 2871 (time 1749122111.84 total 10.00 last 287.09) - open/close 5898 (time 1749122121.84 total 20.00 last 302.59) - open/close 8100 (time 1749122131.84 total 30.00 last 220.20) - open/close 9945 (time 1749122141.85 total 40.01 last 184.45) total: 10000 open/close in 40.32 seconds: 248.00 ops/second - unlinked 0 (time 1749122143 ; total 0 ; last 0) total: 10000 unlinks in 28 seconds: 357.142853 unlinks/second - open/close 2358 (time 1749122183.13 total 10.00 last 235.72) - open/close 4531 (time 1749122193.13 total 20.01 last 217.25) - open/close 7248 (time 1749122203.14 total 30.01 last 271.62) - open/close 9795 (time 1749122213.14 total 40.01 last 254.66) total: 10000 open/close in 40.90 seconds: 244.52 ops/second - unlinked 0 (time 1749122215 ; total 0 ; last 0) total: 10000 unlinks in 20 seconds: 500.000000 unlinks/second - open/close 2940 (time 1749122246.85 total 10.00 last 293.88) - open/close 5980 (time 1749122256.85 total 20.01 last 303.94) - open/close 8420 (time 1749122266.85 total 30.01 last 243.99) total: 10000 open/close in 35.48 seconds: 281.85 ops/second - unlinked 0 (time 1749122273 ; total 0 ; last 0) total: 10000 unlinks in 21 seconds: 476.190491 unlinks/second - open/close 2270 (time 1749122305.72 total 10.00 last 226.95) - open/close 5581 (time 1749122315.72 total 20.00 last 331.02) - open/close 8552 (time 1749122325.72 total 30.01 last 297.04) total: 10000 open/close in 34.36 seconds: 291.01 ops/second - unlinked 0 (time 1749122331 ; total 0 ; last 0) total: 10000 unlinks in 19 seconds: 526.315796 unlinks/second - open/close 3309 (time 1749122361.59 total 10.00 last 330.85) - open/close 5862 (time 1749122371.59 total 20.00 last 255.22) - open/close 7960 (time 1749122381.59 total 30.00 last 209.80) total: 10000 open/close in 38.48 seconds: 259.85 ops/second - unlinked 0 (time 1749122391 ; total 0 ; last 0) total: 10000 unlinks in 20 seconds: 500.000000 unlinks/second - open/close 3178 (time 1749122422.58 total 10.00 last 317.79) - open/close 6372 (time 1749122432.58 total 20.00 last 319.35) - open/close 9283 (time 1749122442.58 total 30.00 last 291.07) total: 10000 open/close in 33.11 seconds: 301.99 ops/second - unlinked 0 (time 1749122446 ; total 0 ; last 0) total: 10000 unlinks in 22 seconds: 454.545441 unlinks/second - open/close 2304 (time 1749122479.79 total 10.00 last 230.38) - open/close 4325 (time 1749122489.79 total 20.00 last 202.05) - open/close 6680 (time 1749122499.79 total 30.00 last 235.48) - open/close 9534 (time 1749122509.80 total 40.01 last 285.26) total: 10000 open/close in 41.84 seconds: 238.99 ops/second - unlinked 0 (time 1749122512 ; total 0 ; last 0) total: 10000 unlinks in 21 seconds: 476.190491 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg401-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg401-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg401-server Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.204.101@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.204.101@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg401-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg401-server: oleg401-server.virtnet: executing set_default_debug -1 all pdsh@oleg401-client: oleg401-server: ssh exited with exit code 1 Started lustre-OST0000 oleg401-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg401-server: oleg401-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg401-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 8 sec mount lustre on /mnt/lustre..... Starting client: oleg401-client.virtnet: -o user_xattr,flock 192.168.204.101@tcp:/lustre /mnt/lustre On OST0, 17701 used inodes rc=0 umount lustre on /mnt/lustre..... Stopping client oleg401-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg401-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg401-server stop mds service on oleg401-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg401-server unloading modules via unload_modules_local on: 'oleg401-server' oleg401-server: oleg401-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg401-client: oleg401-client: ssh exited with exit code 2 pdsh@oleg401-client: oleg401-server: ssh exited with exit code 2 pdsh@oleg401-client: oleg401-client: ssh exited with exit code 2 pdsh@oleg401-client: oleg401-server: ssh exited with exit code 2 PASS 69 (736s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 07:22:51 (1749122571) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (1s) Stopping clients: oleg401-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg401-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg401-client: oleg401-server: ssh exited with exit code 2 oleg401-server: oleg401-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg401-server' oleg401-server: oleg401-server.virtnet: executing load_modules_local oleg401-server: Loading modules from /home/green/git/lustre-release/lustre oleg401-server: detected 4 online CPUs by sysfs oleg401-server: MODOPTS_LIBCFS= oleg401-server: Force libcfs to create 2 CPU partitions oleg401-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg401-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 972 sec =========== 07:23:12 (1749122592) === conf-sanity: start cleanup 07:23:13 (1749122593) === === conf-sanity: finish cleanup 07:23:13 (1749122593) ===