-----============= acceptance-small: conf-sanity ============----- Fri Oct 10 19:07:52 EDT 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg136-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32b 32c 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg136-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg136-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg136-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg136-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg136-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg136-server oleg136-server: oleg136-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg136-server' oleg136-server: oleg136-server.virtnet: executing load_modules_local oleg136-server: Loading modules from /home/green/git/lustre-release/lustre oleg136-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg136-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg136-server: detected 4 online CPUs by sysfs oleg136-server: MODOPTS_LIBCFS= oleg136-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg136-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg136-server: oleg136-server.virtnet: executing set_default_debug -1 all pdsh@oleg136-client: oleg136-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg136-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg136-server: oleg136-server.virtnet: executing set_default_debug -1 all pdsh@oleg136-client: oleg136-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg136-server: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg136-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg136-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg136-server stop mds service on oleg136-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg136-server == conf-sanity test 45: long unlink handling in ptlrpcd == 19:09:01 (1760137741) start mds service on oleg136-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg136-server: oleg136-server.virtnet: executing set_default_debug -1 all pdsh@oleg136-client: oleg136-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg136-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg136-server: oleg136-server.virtnet: executing set_default_debug -1 all pdsh@oleg136-client: oleg136-server: ssh exited with exit code 1 Started lustre-OST0000 oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg136-client.virtnet: -o user_xattr,flock 192.168.201.136@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg136-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg136-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg136-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg136-server: oleg136-server.virtnet: executing set_default_debug -1 all pdsh@oleg136-client: oleg136-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg136-client.virtnet: -o user_xattr,flock 192.168.201.136@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg136-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg136-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg136-server stop mds service on oleg136-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg136-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg136-server' oleg136-server: oleg136-server.virtnet: executing unload_modules_local oleg136-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg136-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg136-client: oleg136-client: ssh exited with exit code 2 pdsh@oleg136-client: oleg136-server: ssh exited with exit code 2 pdsh@oleg136-client: oleg136-server: ssh exited with exit code 2 pdsh@oleg136-client: oleg136-client: ssh exited with exit code 2 PASS 45 (131s) == conf-sanity test 69: replace an OST with the same index ========================================================== 19:11:13 (1760137873) start mds service on oleg136-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg136-server' oleg136-server: oleg136-server.virtnet: executing load_modules_local oleg136-server: Loading modules from /home/green/git/lustre-release/lustre oleg136-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg136-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg136-server: detected 4 online CPUs by sysfs oleg136-server: MODOPTS_LIBCFS= oleg136-server: Force libcfs to create 2 CPU partitions oleg136-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg136-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg136-server: oleg136-server.virtnet: executing set_default_debug -1 all pdsh@oleg136-client: oleg136-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg136-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg136-server: oleg136-server.virtnet: executing set_default_debug -1 all pdsh@oleg136-client: oleg136-server: ssh exited with exit code 1 Started lustre-OST0000 oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg136-client.virtnet: -o user_xattr,flock 192.168.201.136@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 11872 inodes available. Want 99872. rc=0 - open/close 3901 (time 1760137914.52 total 10.00 last 390.03) - open/close 8180 (time 1760137924.52 total 20.00 last 427.88) total: 10000 open/close in 23.84 seconds: 419.49 ops/second - unlinked 0 (time 1760137929 ; total 0 ; last 0) total: 10000 unlinks in 14 seconds: 714.285706 unlinks/second - open/close 4452 (time 1760137954.37 total 10.00 last 445.14) - open/close 8946 (time 1760137964.38 total 20.00 last 449.37) total: 10000 open/close in 22.23 seconds: 449.84 ops/second - unlinked 0 (time 1760137967 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4513 (time 1760137991.89 total 10.00 last 451.27) - open/close 8979 (time 1760138001.89 total 20.00 last 446.59) total: 10000 open/close in 22.33 seconds: 447.86 ops/second - unlinked 0 (time 1760138005 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4428 (time 1760138030.09 total 10.00 last 442.72) - open/close 9157 (time 1760138040.10 total 20.00 last 472.85) total: 10000 open/close in 21.75 seconds: 459.83 ops/second - unlinked 0 (time 1760138043 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4579 (time 1760138067.53 total 10.00 last 457.88) - open/close 9244 (time 1760138077.53 total 20.00 last 466.40) total: 10000 open/close in 21.56 seconds: 463.90 ops/second - unlinked 0 (time 1760138080 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4718 (time 1760138104.61 total 10.00 last 471.77) - open/close 9414 (time 1760138114.61 total 20.00 last 469.57) total: 10000 open/close in 21.27 seconds: 470.13 ops/second - unlinked 0 (time 1760138117 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4588 (time 1760138141.88 total 10.00 last 458.76) - open/close 9306 (time 1760138151.88 total 20.00 last 471.71) total: 10000 open/close in 21.55 seconds: 464.01 ops/second - unlinked 0 (time 1760138154 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4614 (time 1760138178.78 total 10.00 last 461.31) - open/close 9286 (time 1760138188.78 total 20.00 last 467.12) total: 10000 open/close in 21.50 seconds: 465.12 ops/second - unlinked 0 (time 1760138191 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4527 (time 1760138215.50 total 10.00 last 452.68) - open/close 9122 (time 1760138225.50 total 20.00 last 459.46) total: 10000 open/close in 21.87 seconds: 457.27 ops/second - unlinked 0 (time 1760138228 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4457 (time 1760138252.77 total 10.00 last 445.66) - open/close 8801 (time 1760138262.77 total 20.00 last 434.32) total: 10000 open/close in 22.82 seconds: 438.19 ops/second - unlinked 0 (time 1760138266 ; total 0 ; last 0) total: 10000 unlinks in 15 seconds: 666.666687 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg136-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg136-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg136-server Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.201.136@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.201.136@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg136-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg136-server: oleg136-server.virtnet: executing set_default_debug -1 all pdsh@oleg136-client: oleg136-server: ssh exited with exit code 1 Started lustre-OST0000 oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg136-server: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg136-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 11 sec mount lustre on /mnt/lustre..... Starting client: oleg136-client.virtnet: -o user_xattr,flock 192.168.201.136@tcp:/lustre /mnt/lustre On OST0, 20325 used inodes rc=0 pdsh@oleg136-client: oleg136-server: ssh exited with exit code 22 umount lustre on /mnt/lustre..... Stopping client oleg136-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg136-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg136-server Permanent disk data: Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: mgsnode=192.168.201.136@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.201.136@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=98 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre:OST0000 checking for existing Lustre data: found Read previous values: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: autodegrade=on sys.timeout=20 mgsnode=192.168.201.136@tcp Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: autodegrade=on sys.timeout=20 mgsnode=192.168.201.136@tcp autodegrade=on lustre:svname=lustre-OST0000 Writing lustre-ost1/ost1 properties lustre:autodegrade=on lustre:sys.timeout=20 lustre:mgsnode=192.168.201.136@tcp lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg136-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg136-server: oleg136-server.virtnet: executing set_default_debug -1 all pdsh@oleg136-client: oleg136-server: ssh exited with exit code 1 Started lustre-OST0000 oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg136-server: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg136-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 9 sec mount lustre on /mnt/lustre..... Starting client: oleg136-client.virtnet: -o user_xattr,flock 192.168.201.136@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg136-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg136-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg136-server stop mds service on oleg136-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg136-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg136-server' oleg136-server: oleg136-server.virtnet: executing unload_modules_local oleg136-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg136-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg136-client: oleg136-client: ssh exited with exit code 2 pdsh@oleg136-client: oleg136-server: ssh exited with exit code 2 pdsh@oleg136-client: oleg136-server: ssh exited with exit code 2 pdsh@oleg136-client: oleg136-client: ssh exited with exit code 2 PASS 69 (489s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 19:19:23 (1760138363) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (1s) Stopping clients: oleg136-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg136-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg136-client: oleg136-server: ssh exited with exit code 2 oleg136-server: oleg136-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg136-server' oleg136-server: oleg136-server.virtnet: executing load_modules_local oleg136-server: Loading modules from /home/green/git/lustre-release/lustre oleg136-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg136-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg136-server: detected 4 online CPUs by sysfs oleg136-server: MODOPTS_LIBCFS= oleg136-server: Force libcfs to create 2 CPU partitions oleg136-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg136-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 715 sec =========== 19:19:48 (1760138388) === conf-sanity: start cleanup 19:19:48 (1760138388) === === conf-sanity: finish cleanup 19:19:49 (1760138389) ===