-----============= acceptance-small: conf-sanity ============----- Wed May 7 09:45:42 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg356-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex: No such file or directory excepting tests: 32b 32c 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg356-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg356-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg356-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg356-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg356-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg356-server oleg356-server: oleg356-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: MODOPTS_LIBCFS= oleg356-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg356-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg356-server: oleg356-server.virtnet: executing set_default_debug -1 all pdsh@oleg356-client: oleg356-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg356-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg356-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg356-server: oleg356-server.virtnet: executing set_default_debug -1 all pdsh@oleg356-client: oleg356-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg356-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg356-server: oleg356-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg356-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg356-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg356-server stop mds service on oleg356-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg356-server == conf-sanity test 45: long unlink handling in ptlrpcd == 09:46:44 (1746625604) start mds service on oleg356-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg356-server: oleg356-server.virtnet: executing set_default_debug -1 all pdsh@oleg356-client: oleg356-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg356-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg356-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg356-server: oleg356-server.virtnet: executing set_default_debug -1 all pdsh@oleg356-client: oleg356-server: ssh exited with exit code 1 Started lustre-OST0000 oleg356-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg356-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg356-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: '/mnt/lustre': Cannot send after transport endpoint shutdown fail_loc=0x0 start mds service on oleg356-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg356-server: oleg356-server.virtnet: executing set_default_debug -1 all pdsh@oleg356-client: oleg356-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg356-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg356-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg356-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg356-server stop mds service on oleg356-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg356-server unloading modules via unload_modules_local on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg356-client: oleg356-client: ssh exited with exit code 2 pdsh@oleg356-client: oleg356-server: ssh exited with exit code 2 pdsh@oleg356-client: oleg356-client: ssh exited with exit code 2 pdsh@oleg356-client: oleg356-server: ssh exited with exit code 2 PASS 45 (127s) == conf-sanity test 69: replace an OST with the same index ========================================================== 09:48:51 (1746625731) start mds service on oleg356-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: MODOPTS_LIBCFS= oleg356-server: Force libcfs to create 2 CPU partitions oleg356-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg356-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg356-server: oleg356-server.virtnet: executing set_default_debug -1 all pdsh@oleg356-client: oleg356-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg356-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg356-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg356-server: oleg356-server.virtnet: executing set_default_debug -1 all pdsh@oleg356-client: oleg356-server: ssh exited with exit code 1 Started lustre-OST0000 oleg356-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 11872 inodes available. Want 99872. rc=0 - open/close 2688 (time 1746625771.51 total 10.01 last 268.63) - open/close 6049 (time 1746625781.51 total 20.01 last 335.99) - open/close 8069 (time 1746625791.52 total 30.01 last 201.93) total: 10000 open/close in 39.89 seconds: 250.69 ops/second - unlinked 0 (time 1746625802 ; total 0 ; last 0) total: 10000 unlinks in 26 seconds: 384.615387 unlinks/second - open/close 2396 (time 1746625839.25 total 10.00 last 239.59) - open/close 5023 (time 1746625849.25 total 20.00 last 262.61) - open/close 7249 (time 1746625859.25 total 30.01 last 222.54) - open/close 9517 (time 1746625869.25 total 40.01 last 226.80) total: 10000 open/close in 41.85 seconds: 238.93 ops/second - unlinked 0 (time 1746625872 ; total 0 ; last 0) total: 10000 unlinks in 22 seconds: 454.545441 unlinks/second - open/close 2623 (time 1746625905.36 total 10.00 last 262.20) - open/close 5392 (time 1746625915.37 total 20.01 last 276.73) - open/close 8267 (time 1746625925.37 total 30.01 last 287.49) total: 10000 open/close in 36.47 seconds: 274.22 ops/second - unlinked 0 (time 1746625932 ; total 0 ; last 0) total: 10000 unlinks in 19 seconds: 526.315796 unlinks/second - open/close 2352 (time 1746625962.70 total 10.01 last 234.87) - open/close 4618 (time 1746625972.70 total 20.02 last 226.54) - open/close 7606 (time 1746625982.70 total 30.02 last 298.78) total: 10000 open/close in 36.85 seconds: 271.37 ops/second - unlinked 0 (time 1746625990 ; total 0 ; last 0) total: 10000 unlinks in 19 seconds: 526.315796 unlinks/second - open/close 3383 (time 1746626020.22 total 10.00 last 338.25) - open/close 7498 (time 1746626030.22 total 20.00 last 411.42) total: 10000 open/close in 27.42 seconds: 364.72 ops/second - unlinked 0 (time 1746626038 ; total 0 ; last 0) total: 10000 unlinks in 17 seconds: 588.235291 unlinks/second - open/close 3638 (time 1746626066.27 total 10.00 last 363.74) - open/close 6628 (time 1746626076.27 total 20.00 last 298.93) - open/close 9390 (time 1746626086.28 total 30.01 last 276.05) total: 10000 open/close in 32.64 seconds: 306.34 ops/second - unlinked 0 (time 1746626089 ; total 0 ; last 0) total: 10000 unlinks in 18 seconds: 555.555542 unlinks/second - open/close 2692 (time 1746626118.47 total 10.00 last 269.18) - open/close 5694 (time 1746626128.47 total 20.00 last 300.17) - open/close 9056 (time 1746626138.47 total 30.00 last 336.18) total: 10000 open/close in 32.63 seconds: 306.43 ops/second - unlinked 0 (time 1746626141 ; total 0 ; last 0) total: 10000 unlinks in 21 seconds: 476.190491 unlinks/second - open/close 2887 (time 1746626173.61 total 10.00 last 288.67) - open/close 6389 (time 1746626183.62 total 20.00 last 350.10) - open/close 9994 (time 1746626193.62 total 30.00 last 360.48) total: 10000 open/close in 30.01 seconds: 333.17 ops/second - unlinked 0 (time 1746626194 ; total 0 ; last 0) total: 10000 unlinks in 16 seconds: 625.000000 unlinks/second - open/close 2746 (time 1746626221.68 total 10.00 last 274.57) - open/close 5258 (time 1746626231.68 total 20.00 last 251.15) - open/close 7354 (time 1746626241.68 total 30.01 last 209.53) total: 10000 open/close in 39.45 seconds: 253.50 ops/second - unlinked 0 (time 1746626252 ; total 0 ; last 0) total: 10000 unlinks in 21 seconds: 476.190491 unlinks/second - open/close 2587 (time 1746626284.70 total 10.00 last 258.68) - open/close 5218 (time 1746626294.70 total 20.00 last 263.05) - open/close 8187 (time 1746626304.71 total 30.01 last 296.78) total: 10000 open/close in 36.03 seconds: 277.52 ops/second - unlinked 0 (time 1746626311 ; total 0 ; last 0) total: 10000 unlinks in 20 seconds: 500.000000 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg356-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg356-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg356-server Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.203.156@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.203.156@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg356-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg356-server: oleg356-server.virtnet: executing set_default_debug -1 all pdsh@oleg356-client: oleg356-server: ssh exited with exit code 1 Started lustre-OST0000 oleg356-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg356-server: oleg356-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg356-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 8 sec mount lustre on /mnt/lustre..... Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre On OST0, 17741 used inodes rc=0 umount lustre on /mnt/lustre..... Stopping client oleg356-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg356-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg356-server stop mds service on oleg356-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg356-server unloading modules via unload_modules_local on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg356-client: oleg356-client: ssh exited with exit code 2 pdsh@oleg356-client: oleg356-server: ssh exited with exit code 2 pdsh@oleg356-client: oleg356-client: ssh exited with exit code 2 pdsh@oleg356-client: oleg356-server: ssh exited with exit code 2 PASS 69 (636s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 09:59:27 (1746626367) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (1s) Stopping clients: oleg356-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg356-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg356-client: oleg356-server: ssh exited with exit code 2 oleg356-server: oleg356-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: MODOPTS_LIBCFS= oleg356-server: Force libcfs to create 2 CPU partitions oleg356-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg356-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 844 sec =========== 09:59:46 (1746626386) === conf-sanity: start cleanup 09:59:46 (1746626386) === === conf-sanity: finish cleanup 09:59:47 (1746626387) ===