-----============= acceptance-small: conf-sanity ============----- Wed Dec 3 01:51:55 EST 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg424-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32b 32c 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg424-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg424-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg424-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg424-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg424-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg424-server oleg424-server: oleg424-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg424-server' oleg424-server: oleg424-server.virtnet: executing load_modules_local oleg424-server: Loading modules from /home/green/git/lustre-release/lustre oleg424-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg424-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg424-server: detected 4 online CPUs by sysfs oleg424-server: MODOPTS_LIBCFS= oleg424-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg424-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg424-server: oleg424-server.virtnet: executing set_default_debug -1 all pdsh@oleg424-client: oleg424-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg424-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg424-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg424-server: oleg424-server.virtnet: executing set_default_debug -1 all pdsh@oleg424-client: oleg424-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg424-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg424-server: oleg424-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg424-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg424-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg424-server stop mds service on oleg424-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg424-server == conf-sanity test 45: long unlink handling in ptlrpcd == 01:53:05 (1764744785) start mds service on oleg424-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg424-server: oleg424-server.virtnet: executing set_default_debug -1 all pdsh@oleg424-client: oleg424-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg424-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg424-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg424-server: oleg424-server.virtnet: executing set_default_debug -1 all pdsh@oleg424-client: oleg424-server: ssh exited with exit code 1 Started lustre-OST0000 oleg424-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg424-client.virtnet: -o user_xattr,flock 192.168.204.124@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg424-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg424-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg424-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg424-server: oleg424-server.virtnet: executing set_default_debug -1 all pdsh@oleg424-client: oleg424-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg424-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg424-client.virtnet: -o user_xattr,flock 192.168.204.124@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg424-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg424-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg424-server stop mds service on oleg424-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg424-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg424-server' oleg424-server: oleg424-server.virtnet: executing unload_modules_local oleg424-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg424-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg424-client: oleg424-client: ssh exited with exit code 2 pdsh@oleg424-client: oleg424-server: ssh exited with exit code 2 pdsh@oleg424-client: oleg424-client: ssh exited with exit code 2 pdsh@oleg424-client: oleg424-server: ssh exited with exit code 2 PASS 45 (137s) == conf-sanity test 69: replace an OST with the same index ========================================================== 01:55:22 (1764744922) start mds service on oleg424-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg424-server' oleg424-server: oleg424-server.virtnet: executing load_modules_local oleg424-server: Loading modules from /home/green/git/lustre-release/lustre oleg424-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg424-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg424-server: detected 4 online CPUs by sysfs oleg424-server: MODOPTS_LIBCFS= oleg424-server: Force libcfs to create 2 CPU partitions oleg424-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg424-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg424-server: oleg424-server.virtnet: executing set_default_debug -1 all pdsh@oleg424-client: oleg424-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg424-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg424-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg424-server: oleg424-server.virtnet: executing set_default_debug -1 all pdsh@oleg424-client: oleg424-server: ssh exited with exit code 1 Started lustre-OST0000 oleg424-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg424-client.virtnet: -o user_xattr,flock 192.168.204.124@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 11872 inodes available. Want 99872. rc=0 - open/close 2730 (time 1764744966.43 total 10.00 last 272.90) - open/close 5289 (time 1764744976.43 total 20.01 last 255.86) - open/close 7466 (time 1764744986.43 total 30.01 last 217.61) - open/close 9432 (time 1764744996.43 total 40.01 last 196.58) total: 10000 open/close in 42.17 seconds: 237.13 ops/second - unlinked 0 (time 1764745000 ; total 0 ; last 0) total: 10000 unlinks in 21 seconds: 476.190491 unlinks/second - open/close 2046 (time 1764745032.74 total 10.00 last 204.58) - open/close 3755 (time 1764745042.74 total 20.00 last 170.85) - open/close 5805 (time 1764745052.75 total 30.01 last 204.94) - open/close 7960 (time 1764745062.75 total 40.01 last 215.47) - open/close 9775 (time 1764745072.75 total 50.01 last 181.50) total: 10000 open/close in 50.97 seconds: 196.20 ops/second - unlinked 0 (time 1764745075 ; total 0 ; last 0) total: 10000 unlinks in 28 seconds: 357.142853 unlinks/second - open/close 2454 (time 1764745114.77 total 10.00 last 245.29) - open/close 4860 (time 1764745124.77 total 20.01 last 240.51) - open/close 6995 (time 1764745134.78 total 30.01 last 213.42) - open/close 9469 (time 1764745144.78 total 40.01 last 247.37) total: 10000 open/close in 42.05 seconds: 237.81 ops/second - unlinked 0 (time 1764745148 ; total 0 ; last 0) total: 10000 unlinks in 20 seconds: 500.000000 unlinks/second - open/close 2885 (time 1764745180.34 total 10.00 last 288.48) - open/close 5776 (time 1764745190.35 total 20.00 last 289.07) - open/close 8834 (time 1764745200.35 total 30.00 last 305.76) total: 10000 open/close in 34.69 seconds: 288.26 ops/second - unlinked 0 (time 1764745206 ; total 0 ; last 0) total: 10000 unlinks in 22 seconds: 454.545441 unlinks/second - open/close 2495 (time 1764745240.67 total 10.00 last 249.50) - open/close 5018 (time 1764745250.67 total 20.00 last 252.26) - open/close 8130 (time 1764745260.67 total 30.00 last 311.11) total: 10000 open/close in 35.10 seconds: 284.90 ops/second - unlinked 0 (time 1764745267 ; total 0 ; last 0) total: 10000 unlinks in 15 seconds: 666.666687 unlinks/second - open/close 3958 (time 1764745293.74 total 10.00 last 395.72) - open/close 8041 (time 1764745303.74 total 20.00 last 408.22) total: 10000 open/close in 24.77 seconds: 403.64 ops/second - unlinked 0 (time 1764745309 ; total 0 ; last 0) total: 10000 unlinks in 15 seconds: 666.666687 unlinks/second - open/close 3459 (time 1764745335.91 total 10.00 last 345.82) - open/close 6594 (time 1764745345.91 total 20.00 last 313.47) - open/close 9527 (time 1764745355.91 total 30.00 last 293.28) total: 10000 open/close in 31.61 seconds: 316.37 ops/second - unlinked 0 (time 1764745358 ; total 0 ; last 0) total: 10000 unlinks in 18 seconds: 555.555542 unlinks/second - open/close 3906 (time 1764745388.48 total 10.00 last 390.54) - open/close 7535 (time 1764745398.48 total 20.00 last 362.87) total: 10000 open/close in 28.30 seconds: 353.32 ops/second - unlinked 0 (time 1764745408 ; total 0 ; last 0) total: 10000 unlinks in 17 seconds: 588.235291 unlinks/second - open/close 3168 (time 1764745436.62 total 10.00 last 316.76) - open/close 6313 (time 1764745446.63 total 20.00 last 314.42) - open/close 9221 (time 1764745456.63 total 30.01 last 290.76) total: 10000 open/close in 32.52 seconds: 307.52 ops/second - unlinked 0 (time 1764745460 ; total 0 ; last 0) total: 10000 unlinks in 17 seconds: 588.235291 unlinks/second - open/close 3329 (time 1764745489.01 total 10.00 last 332.87) - open/close 6628 (time 1764745499.01 total 20.00 last 329.87) - open/close 9589 (time 1764745509.01 total 30.01 last 295.92) total: 10000 open/close in 31.44 seconds: 318.04 ops/second - unlinked 0 (time 1764745511 ; total 0 ; last 0) total: 10000 unlinks in 23 seconds: 434.782623 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg424-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg424-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg424-server Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.204.124@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.204.124@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg424-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg424-server: oleg424-server.virtnet: executing set_default_debug -1 all pdsh@oleg424-client: oleg424-server: ssh exited with exit code 1 Started lustre-OST0000 oleg424-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg424-server: oleg424-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg424-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 6 sec mount lustre on /mnt/lustre..... Starting client: oleg424-client.virtnet: -o user_xattr,flock 192.168.204.124@tcp:/lustre /mnt/lustre On OST0, 18781 used inodes rc=0 pdsh@oleg424-client: oleg424-server: ssh exited with exit code 22 umount lustre on /mnt/lustre..... Stopping client oleg424-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg424-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg424-server Permanent disk data: Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: mgsnode=192.168.204.124@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.204.124@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=98 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre:OST0000 checking for existing Lustre data: found Read previous values: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: sys.timeout=20 autodegrade=on mgsnode=192.168.204.124@tcp Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: sys.timeout=20 autodegrade=on mgsnode=192.168.204.124@tcp autodegrade=on lustre:svname=lustre-OST0000 Writing lustre-ost1/ost1 properties lustre:sys.timeout=20 lustre:autodegrade=on lustre:mgsnode=192.168.204.124@tcp lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg424-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg424-server: oleg424-server.virtnet: executing set_default_debug -1 all pdsh@oleg424-client: oleg424-server: ssh exited with exit code 1 Started lustre-OST0000 oleg424-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg424-server: oleg424-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg424-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 6 sec mount lustre on /mnt/lustre..... Starting client: oleg424-client.virtnet: -o user_xattr,flock 192.168.204.124@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg424-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg424-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg424-server stop mds service on oleg424-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg424-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg424-server' oleg424-server: oleg424-server.virtnet: executing unload_modules_local oleg424-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg424-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg424-client: oleg424-client: ssh exited with exit code 2 pdsh@oleg424-client: oleg424-server: ssh exited with exit code 2 pdsh@oleg424-client: oleg424-client: ssh exited with exit code 2 pdsh@oleg424-client: oleg424-server: ssh exited with exit code 2 PASS 69 (682s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 02:06:45 (1764745605) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (1s) Stopping clients: oleg424-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg424-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg424-client: oleg424-server: ssh exited with exit code 2 oleg424-server: oleg424-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg424-server' oleg424-server: oleg424-server.virtnet: executing load_modules_local oleg424-server: Loading modules from /home/green/git/lustre-release/lustre oleg424-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg424-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg424-server: detected 4 online CPUs by sysfs oleg424-server: MODOPTS_LIBCFS= oleg424-server: Force libcfs to create 2 CPU partitions oleg424-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg424-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 917 sec =========== 02:07:12 (1764745632) === conf-sanity: start cleanup 02:07:13 (1764745633) === === conf-sanity: finish cleanup 02:07:14 (1764745634) ===