-----============= acceptance-small: conf-sanity ============----- Mon Mar 16 14:24:05 EDT 2026 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg340-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32b 32c 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg340-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg340-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg340-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg340-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg340-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg340-server oleg340-server: oleg340-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy oleg340-server: detected 4 online CPUs by sysfs oleg340-server: MODOPTS_LIBCFS= oleg340-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg340-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg340-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg340-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg340-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg340-server: oleg340-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg340-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg340-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg340-server stop mds service on oleg340-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg340-server == conf-sanity test 45: long unlink handling in ptlrpcd == 14:26:05 (1773685565) start mds service on oleg340-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg340-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg340-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Started lustre-OST0000 oleg340-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg340-client.virtnet: -o user_xattr,flock 192.168.203.140@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg340-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg340-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg340-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg340-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg340-client.virtnet: -o user_xattr,flock 192.168.203.140@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg340-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg340-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg340-server stop mds service on oleg340-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg340-server unloading modules via unload_modules_local on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing unload_modules_local oleg340-server: modules unloaded. pdsh@oleg340-client: oleg340-client: ssh exited with exit code 2 pdsh@oleg340-client: oleg340-server: ssh exited with exit code 2 pdsh@oleg340-client: oleg340-client: ssh exited with exit code 2 pdsh@oleg340-client: oleg340-server: ssh exited with exit code 2 PASS 45 (156s) == conf-sanity test 69: replace an OST with the same index ========================================================== 14:28:41 (1773685721) start mds service on oleg340-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy oleg340-server: detected 4 online CPUs by sysfs oleg340-server: MODOPTS_LIBCFS= oleg340-server: Force libcfs to create 2 CPU partitions oleg340-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg340-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg340-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg340-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Started lustre-OST0000 oleg340-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg340-client.virtnet: -o user_xattr,flock 192.168.203.140@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 11872 inodes available. Want 99872. rc=0 - open/close 1125 (time 1773685787.90 total 10.00 last 112.49) - open/close 2118 (time 1773685797.90 total 20.01 last 99.26) - open/close 3221 (time 1773685807.90 total 30.01 last 110.28) - open/close 4284 (time 1773685817.91 total 40.01 last 106.24) - open/close 5500 (time 1773685827.91 total 50.02 last 121.54) - open/close 6782 (time 1773685837.92 total 60.03 last 128.10) - open/close 8024 (time 1773685847.92 total 70.03 last 124.19) - open/close 9311 (time 1773685857.92 total 80.03 last 128.65) total: 10000 open/close in 84.73 seconds: 118.02 ops/second - unlinked 0 (time 1773685864 ; total 0 ; last 0) total: 10000 unlinks in 46 seconds: 217.391312 unlinks/second - open/close 1359 (time 1773685923.13 total 10.00 last 135.85) - open/close 2650 (time 1773685933.13 total 20.00 last 129.09) - open/close 3879 (time 1773685943.13 total 30.01 last 122.83) - open/close 5149 (time 1773685953.13 total 40.01 last 127.00) - open/close 6517 (time 1773685963.13 total 50.01 last 136.77) - open/close 7850 (time 1773685973.14 total 60.01 last 133.29) - open/close 9148 (time 1773685983.14 total 70.02 last 129.72) total: 10000 open/close in 77.15 seconds: 129.61 ops/second - unlinked 0 (time 1773685992 ; total 0 ; last 0) total: 10000 unlinks in 38 seconds: 263.157898 unlinks/second - open/close 1264 (time 1773686042.66 total 10.01 last 126.23) - open/close 2756 (time 1773686052.66 total 20.02 last 149.15) - open/close 4321 (time 1773686062.66 total 30.02 last 156.45) - open/close 5657 (time 1773686072.67 total 40.02 last 133.53) - open/close 6919 (time 1773686082.68 total 50.03 last 126.11) - open/close 8188 (time 1773686092.68 total 60.03 last 126.87) - open/close 9423 (time 1773686102.69 total 70.04 last 123.40) total: 10000 open/close in 74.29 seconds: 134.61 ops/second - unlinked 0 (time 1773686109 ; total 0 ; last 0) total: 10000 unlinks in 39 seconds: 256.410248 unlinks/second - open/close 1471 (time 1773686160.94 total 10.00 last 147.10) - open/close 2934 (time 1773686170.95 total 20.01 last 146.23) - open/close 4327 (time 1773686180.95 total 30.01 last 139.24) - open/close 5720 (time 1773686190.96 total 40.01 last 139.23) - open/close 7269 (time 1773686200.96 total 50.02 last 154.86) - open/close 8687 (time 1773686210.96 total 60.02 last 141.78) total: 10000 open/close in 68.75 seconds: 145.45 ops/second - unlinked 0 (time 1773686221 ; total 0 ; last 0) total: 10000 unlinks in 39 seconds: 256.410248 unlinks/second - open/close 1508 (time 1773686272.68 total 10.00 last 150.78) - open/close 2972 (time 1773686282.69 total 20.01 last 146.32) - open/close 4430 (time 1773686292.69 total 30.01 last 145.76) - open/close 5971 (time 1773686302.70 total 40.01 last 154.02) - open/close 7427 (time 1773686312.70 total 50.02 last 145.58) - open/close 8940 (time 1773686322.70 total 60.02 last 151.29) total: 10000 open/close in 67.00 seconds: 149.24 ops/second - unlinked 0 (time 1773686331 ; total 0 ; last 0) total: 10000 unlinks in 36 seconds: 277.777771 unlinks/second - open/close 1769 (time 1773686379.67 total 10.00 last 176.83) - open/close 3423 (time 1773686389.67 total 20.01 last 165.28) - open/close 4974 (time 1773686399.68 total 30.02 last 155.03) - open/close 6568 (time 1773686409.68 total 40.02 last 159.33) - open/close 8134 (time 1773686419.68 total 50.02 last 156.60) - open/close 9469 (time 1773686429.68 total 60.02 last 133.47) total: 10000 open/close in 63.68 seconds: 157.03 ops/second - unlinked 0 (time 1773686435 ; total 0 ; last 0) total: 10000 unlinks in 36 seconds: 277.777771 unlinks/second - open/close 1425 (time 1773686484.30 total 10.00 last 142.49) - open/close 3017 (time 1773686494.30 total 20.00 last 159.19) - open/close 4327 (time 1773686504.31 total 30.01 last 130.94) - open/close 5754 (time 1773686514.31 total 40.01 last 142.69) - open/close 7119 (time 1773686524.31 total 50.01 last 136.40) - open/close 8915 (time 1773686534.32 total 60.02 last 179.54) total: 10000 open/close in 67.44 seconds: 148.29 ops/second - unlinked 0 (time 1773686543 ; total 0 ; last 0) total: 10000 unlinks in 38 seconds: 263.157898 unlinks/second - open/close 1617 (time 1773686593.13 total 10.01 last 161.61) - open/close 2875 (time 1773686603.13 total 20.01 last 125.79) - open/close 4263 (time 1773686613.13 total 30.01 last 138.73) - open/close 6369 (time 1773686623.14 total 40.02 last 210.49) - open/close 8420 (time 1773686633.14 total 50.02 last 205.01) total: 10000 open/close in 58.47 seconds: 171.02 ops/second - unlinked 0 (time 1773686643 ; total 0 ; last 0) total: 10000 unlinks in 27 seconds: 370.370361 unlinks/second - open/close 1806 (time 1773686681.84 total 10.00 last 180.59) - open/close 3667 (time 1773686691.84 total 20.00 last 186.07) - open/close 5532 (time 1773686701.84 total 30.01 last 186.42) - open/close 7584 (time 1773686711.85 total 40.01 last 205.16) - open/close 9558 (time 1773686721.85 total 50.01 last 197.31) total: 10000 open/close in 52.82 seconds: 189.32 ops/second - unlinked 0 (time 1773686726 ; total 0 ; last 0) total: 10000 unlinks in 25 seconds: 400.000000 unlinks/second - open/close 2065 (time 1773686762.66 total 10.01 last 206.35) - open/close 4592 (time 1773686772.66 total 20.01 last 252.69) - open/close 6579 (time 1773686782.67 total 30.01 last 198.62) - open/close 9125 (time 1773686792.67 total 40.01 last 254.58) total: 10000 open/close in 42.99 seconds: 232.61 ops/second - unlinked 0 (time 1773686797 ; total 0 ; last 0) total: 10000 unlinks in 20 seconds: 500.000000 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg340-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg340-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg340-server Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.203.140@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.203.140@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg340-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Started lustre-OST0000 oleg340-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg340-server: oleg340-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg340-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 8 sec mount lustre on /mnt/lustre..... Starting client: oleg340-client.virtnet: -o user_xattr,flock 192.168.203.140@tcp:/lustre /mnt/lustre On OST0, 10053 used inodes rc=0 pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 umount lustre on /mnt/lustre..... Stopping client oleg340-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg340-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg340-server Permanent disk data: Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: mgsnode=192.168.203.140@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.203.140@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=98 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre:OST0000 checking for existing Lustre data: found Read previous values: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: sys.timeout=20 autodegrade=on mgsnode=192.168.203.140@tcp Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: sys.timeout=20 autodegrade=on mgsnode=192.168.203.140@tcp autodegrade=on lustre:svname=lustre-OST0000 Writing lustre-ost1/ost1 properties lustre:sys.timeout=20 lustre:autodegrade=on lustre:mgsnode=192.168.203.140@tcp lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg340-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Started lustre-OST0000 oleg340-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg340-server: oleg340-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg340-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg340-client.virtnet: -o user_xattr,flock 192.168.203.140@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg340-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg340-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg340-server stop mds service on oleg340-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg340-server unloading modules via unload_modules_local on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing unload_modules_local oleg340-server: modules unloaded. pdsh@oleg340-client: oleg340-client: ssh exited with exit code 2 pdsh@oleg340-client: oleg340-server: ssh exited with exit code 2 pdsh@oleg340-client: oleg340-client: ssh exited with exit code 2 pdsh@oleg340-client: oleg340-server: ssh exited with exit code 2 PASS 69 (1159s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 14:48:01 (1773686881) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (1s) Stopping clients: oleg340-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg340-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg340-client: oleg340-server: ssh exited with exit code 2 oleg340-server: oleg340-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy oleg340-server: detected 4 online CPUs by sysfs oleg340-server: MODOPTS_LIBCFS= oleg340-server: Force libcfs to create 2 CPU partitions oleg340-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg340-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 1464 sec ========== 14:48:30 (1773686910) === conf-sanity: start cleanup 14:48:31 (1773686911) === === conf-sanity: finish cleanup 14:48:32 (1773686912) ===