-----============= acceptance-small: conf-sanity ============----- Sun Jun 22 07:11:03 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg258-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex: No such file or directory excepting tests: 32b 32c 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg258-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg258-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg258-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg258-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg258-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg258-server oleg258-server: oleg258-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: MODOPTS_LIBCFS= oleg258-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg258-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg258-server: oleg258-server.virtnet: executing set_default_debug -1 all pdsh@oleg258-client: oleg258-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg258-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg258-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg258-server: oleg258-server.virtnet: executing set_default_debug -1 all pdsh@oleg258-client: oleg258-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg258-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg258-server: oleg258-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg258-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg258-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg258-server stop mds service on oleg258-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg258-server == conf-sanity test 45: long unlink handling in ptlrpcd == 07:12:36 (1750590756) start mds service on oleg258-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg258-server: oleg258-server.virtnet: executing set_default_debug -1 all pdsh@oleg258-client: oleg258-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg258-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg258-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg258-server: oleg258-server.virtnet: executing set_default_debug -1 all pdsh@oleg258-client: oleg258-server: ssh exited with exit code 1 Started lustre-OST0000 oleg258-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg258-client.virtnet: -o user_xattr,flock 192.168.202.158@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg258-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg258-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: '/mnt/lustre': Cannot send after transport endpoint shutdown fail_loc=0x0 start mds service on oleg258-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg258-server: oleg258-server.virtnet: executing set_default_debug -1 all pdsh@oleg258-client: oleg258-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg258-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg258-client.virtnet: -o user_xattr,flock 192.168.202.158@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg258-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg258-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg258-server stop mds service on oleg258-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg258-server unloading modules via unload_modules_local on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg258-client: oleg258-client: ssh exited with exit code 2 pdsh@oleg258-client: oleg258-server: ssh exited with exit code 2 pdsh@oleg258-client: oleg258-client: ssh exited with exit code 2 pdsh@oleg258-client: oleg258-server: ssh exited with exit code 2 PASS 45 (139s) == conf-sanity test 69: replace an OST with the same index ========================================================== 07:14:56 (1750590896) start mds service on oleg258-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: MODOPTS_LIBCFS= oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg258-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg258-server: oleg258-server.virtnet: executing set_default_debug -1 all pdsh@oleg258-client: oleg258-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg258-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg258-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg258-server: oleg258-server.virtnet: executing set_default_debug -1 all pdsh@oleg258-client: oleg258-server: ssh exited with exit code 1 Started lustre-OST0000 oleg258-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg258-client.virtnet: -o user_xattr,flock 192.168.202.158@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 11872 inodes available. Want 99872. rc=0 - open/close 1502 (time 1750590943.95 total 10.00 last 150.19) - open/close 2990 (time 1750590953.95 total 20.00 last 148.80) - open/close 4494 (time 1750590963.95 total 30.00 last 150.37) - open/close 5977 (time 1750590973.95 total 40.01 last 148.26) - open/close 7448 (time 1750590983.95 total 50.01 last 147.10) - open/close 9051 (time 1750590993.96 total 60.01 last 160.24) total: 10000 open/close in 66.30 seconds: 150.84 ops/second - unlinked 0 (time 1750591001 ; total 0 ; last 0) total: 10000 unlinks in 36 seconds: 277.777771 unlinks/second - open/close 1660 (time 1750591048.97 total 10.00 last 166.00) - open/close 3122 (time 1750591058.98 total 20.00 last 146.16) - open/close 4759 (time 1750591068.98 total 30.01 last 163.66) - open/close 6296 (time 1750591078.98 total 40.01 last 153.68) - open/close 7855 (time 1750591088.98 total 50.01 last 155.89) - open/close 9425 (time 1750591098.98 total 60.01 last 156.92) total: 10000 open/close in 63.28 seconds: 158.03 ops/second - unlinked 0 (time 1750591103 ; total 0 ; last 0) total: 10000 unlinks in 38 seconds: 263.157898 unlinks/second - open/close 1413 (time 1750591153.15 total 10.00 last 141.29) - open/close 2804 (time 1750591163.17 total 20.02 last 138.84) - open/close 4212 (time 1750591173.17 total 30.02 last 140.75) - open/close 5597 (time 1750591183.17 total 40.02 last 138.48) - open/close 7026 (time 1750591193.17 total 50.03 last 142.87) - open/close 8484 (time 1750591203.18 total 60.03 last 145.73) total: 10000 open/close in 69.26 seconds: 144.38 ops/second - unlinked 0 (time 1750591213 ; total 0 ; last 0) total: 10000 unlinks in 31 seconds: 322.580658 unlinks/second - open/close 1796 (time 1750591256.70 total 10.00 last 179.51) - open/close 3327 (time 1750591266.70 total 20.01 last 153.09) - open/close 4939 (time 1750591276.70 total 30.01 last 161.14) - open/close 6471 (time 1750591286.70 total 40.01 last 153.17) - open/close 7904 (time 1750591296.70 total 50.01 last 143.27) - open/close 9693 (time 1750591306.71 total 60.02 last 178.83) total: 10000 open/close in 62.10 seconds: 161.04 ops/second - unlinked 0 (time 1750591310 ; total 0 ; last 0) total: 10000 unlinks in 38 seconds: 263.157898 unlinks/second - open/close 1796 (time 1750591360.42 total 10.00 last 179.53) - open/close 3339 (time 1750591370.42 total 20.01 last 154.24) - open/close 4843 (time 1750591380.42 total 30.01 last 150.40) - open/close 6371 (time 1750591390.42 total 40.01 last 152.78) - open/close 8180 (time 1750591400.43 total 50.02 last 180.78) - open/close 9931 (time 1750591410.43 total 60.02 last 175.06) total: 10000 open/close in 60.39 seconds: 165.60 ops/second - unlinked 0 (time 1750591412 ; total 0 ; last 0) total: 10000 unlinks in 33 seconds: 303.030304 unlinks/second - open/close 1466 (time 1750591457.05 total 10.00 last 146.54) - open/close 2896 (time 1750591467.05 total 20.01 last 142.97) - open/close 4366 (time 1750591477.05 total 30.01 last 146.93) - open/close 6003 (time 1750591487.06 total 40.02 last 163.61) - open/close 7438 (time 1750591497.06 total 50.02 last 143.45) - open/close 8723 (time 1750591507.06 total 60.02 last 128.46) total: 10000 open/close in 69.60 seconds: 143.68 ops/second - unlinked 0 (time 1750591518 ; total 0 ; last 0) total: 10000 unlinks in 46 seconds: 217.391312 unlinks/second - open/close 1333 (time 1750591576.38 total 10.00 last 133.29) - open/close 2731 (time 1750591586.38 total 20.00 last 139.80) - open/close 4145 (time 1750591596.38 total 30.00 last 141.38) - open/close 5527 (time 1750591606.39 total 40.01 last 138.12) - open/close 6894 (time 1750591616.39 total 50.01 last 136.65) - open/close 8238 (time 1750591626.39 total 60.01 last 134.38) - open/close 9534 (time 1750591636.39 total 70.02 last 129.55) total: 10000 open/close in 73.40 seconds: 136.25 ops/second - unlinked 0 (time 1750591641 ; total 0 ; last 0) total: 10000 unlinks in 37 seconds: 270.270264 unlinks/second - open/close 1485 (time 1750591690.50 total 10.01 last 148.41) - open/close 2792 (time 1750591700.51 total 20.01 last 130.65) - open/close 4203 (time 1750591710.51 total 30.01 last 141.08) - open/close 5590 (time 1750591720.51 total 40.01 last 138.67) - open/close 7012 (time 1750591730.52 total 50.02 last 142.10) - open/close 8452 (time 1750591740.52 total 60.02 last 143.99) - open/close 9960 (time 1750591750.52 total 70.02 last 150.76) total: 10000 open/close in 70.29 seconds: 142.26 ops/second - unlinked 0 (time 1750591752 ; total 0 ; last 0) total: 10000 unlinks in 34 seconds: 294.117645 unlinks/second - open/close 2098 (time 1750591797.90 total 10.00 last 209.75) - open/close 4160 (time 1750591807.90 total 20.00 last 206.18) - open/close 6371 (time 1750591817.90 total 30.01 last 221.04) - open/close 8686 (time 1750591827.90 total 40.01 last 231.50) total: 10000 open/close in 45.66 seconds: 219.01 ops/second - unlinked 0 (time 1750591834 ; total 0 ; last 0) total: 10000 unlinks in 36 seconds: 277.777771 unlinks/second - open/close 1633 (time 1750591882.44 total 10.00 last 163.24) - open/close 3359 (time 1750591892.44 total 20.00 last 172.59) - open/close 4860 (time 1750591902.45 total 30.01 last 150.02) - open/close 6468 (time 1750591912.45 total 40.01 last 160.77) - open/close 8112 (time 1750591922.45 total 50.02 last 164.34) - open/close 9967 (time 1750591932.46 total 60.02 last 185.41) total: 10000 open/close in 60.18 seconds: 166.16 ops/second - unlinked 0 (time 1750591933 ; total 0 ; last 0) total: 10000 unlinks in 32 seconds: 312.500000 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg258-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg258-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg258-server Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.202.158@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.202.158@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg258-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg258-server: oleg258-server.virtnet: executing set_default_debug -1 all pdsh@oleg258-client: oleg258-server: ssh exited with exit code 1 Started lustre-OST0000 oleg258-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg258-server: oleg258-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg258-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 10 sec mount lustre on /mnt/lustre..... Starting client: oleg258-client.virtnet: -o user_xattr,flock 192.168.202.158@tcp:/lustre /mnt/lustre On OST0, 17685 used inodes rc=0 umount lustre on /mnt/lustre..... Stopping client oleg258-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg258-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg258-server stop mds service on oleg258-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg258-server unloading modules via unload_modules_local on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg258-client: oleg258-client: ssh exited with exit code 2 pdsh@oleg258-client: oleg258-server: ssh exited with exit code 2 pdsh@oleg258-client: oleg258-client: ssh exited with exit code 2 pdsh@oleg258-client: oleg258-server: ssh exited with exit code 2 PASS 69 (1123s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 07:33:39 (1750592019) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (1s) Stopping clients: oleg258-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg258-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg258-client: oleg258-server: ssh exited with exit code 2 oleg258-server: oleg258-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: MODOPTS_LIBCFS= oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg258-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 1383 sec ========== 07:34:07 (1750592047) === conf-sanity: start cleanup 07:34:08 (1750592048) === === conf-sanity: finish cleanup 07:34:09 (1750592049) ===