-----============= acceptance-small: conf-sanity ============----- Wed Mar 11 09:18:45 EDT 2026 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg111-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32b 32c 32newtarball Stopping clients: oleg111-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg111-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg111-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg111-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg111-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg111-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg111-server oleg111-server: oleg111-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg111-server' oleg111-server: oleg111-server.virtnet: executing load_modules_local oleg111-server: Loading modules from /home/green/git/lustre-release/lustre oleg111-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg111-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg111-server: detected 4 online CPUs by sysfs oleg111-server: MODOPTS_LIBCFS= oleg111-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg111-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg111-server: oleg111-server.virtnet: executing set_default_debug -1 all pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg111-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg111-server: oleg111-server.virtnet: executing set_default_debug -1 all pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg111-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg111-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg111-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg111-server: oleg111-server.virtnet: executing set_default_debug -1 all pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg111-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg111-server: oleg111-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg111-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg111-server: oleg111-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg111-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg111-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg111-server stop mds service on oleg111-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg111-server stop mds service on oleg111-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg111-server == conf-sanity test 45: long unlink handling in ptlrpcd == 09:21:34 (1773235294) start mds service on oleg111-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg111-server: oleg111-server.virtnet: executing set_default_debug -1 all pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg111-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg111-server: oleg111-server.virtnet: executing set_default_debug -1 all pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg111-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg111-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg111-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg111-server: oleg111-server.virtnet: executing set_default_debug -1 all pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Started lustre-OST0000 oleg111-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg111-client.virtnet: -o user_xattr,flock 192.168.201.111@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg111-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg111-server stop mds service on oleg111-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg111-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg111-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg111-server: oleg111-server.virtnet: executing set_default_debug -1 all pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg111-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg111-server: oleg111-server.virtnet: executing set_default_debug -1 all pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg111-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg111-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg111-client.virtnet: -o user_xattr,flock 192.168.201.111@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg111-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg111-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg111-server stop mds service on oleg111-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg111-server stop mds service on oleg111-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg111-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg111-server' oleg111-server: oleg111-server.virtnet: executing unload_modules_local oleg111-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg111-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg111-client: oleg111-client: ssh exited with exit code 2 pdsh@oleg111-client: oleg111-server: ssh exited with exit code 2 pdsh@oleg111-client: oleg111-server: ssh exited with exit code 2 pdsh@oleg111-client: oleg111-client: ssh exited with exit code 2 PASS 45 (231s) == conf-sanity test 69: replace an OST with the same index ========================================================== 09:25:25 (1773235525) start mds service on oleg111-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg111-server' oleg111-server: oleg111-server.virtnet: executing load_modules_local oleg111-server: Loading modules from /home/green/git/lustre-release/lustre oleg111-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg111-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg111-server: detected 4 online CPUs by sysfs oleg111-server: MODOPTS_LIBCFS= oleg111-server: Force libcfs to create 2 CPU partitions oleg111-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg111-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg111-server: oleg111-server.virtnet: executing set_default_debug -1 all pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg111-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg111-server: oleg111-server.virtnet: executing set_default_debug -1 all pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg111-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg111-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg111-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg111-server: oleg111-server.virtnet: executing set_default_debug -1 all pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Started lustre-OST0000 oleg111-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg111-client.virtnet: -o user_xattr,flock 192.168.201.111@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 49504 inodes available. Want 99872. rc=0 - open/close 956 (time 1773235595.32 total 10.00 last 95.57) - open/close 2018 (time 1773235605.32 total 20.01 last 106.12) - open/close 2966 (time 1773235615.33 total 30.02 last 94.74) - open/close 4013 (time 1773235625.33 total 40.02 last 104.68) - open/close 4999 (time 1773235635.33 total 50.02 last 98.57) - open/close 6177 (time 1773235645.34 total 60.02 last 117.78) - open/close 7551 (time 1773235655.34 total 70.03 last 137.32) - open/close 8911 (time 1773235665.34 total 80.03 last 135.98) - open/close 9827 (time 1773235675.35 total 90.03 last 91.56) total: 10000 open/close in 91.89 seconds: 108.82 ops/second - unlinked 0 (time 1773235680 ; total 0 ; last 0) total: 10000 unlinks in 55 seconds: 181.818176 unlinks/second - open/close 970 (time 1773235749.39 total 10.00 last 96.99) - open/close 1983 (time 1773235759.39 total 20.00 last 101.30) - open/close 2998 (time 1773235769.40 total 30.01 last 101.42) - open/close 3927 (time 1773235779.41 total 40.02 last 92.79) - open/close 4883 (time 1773235789.42 total 50.03 last 95.56) - open/close 5741 (time 1773235799.42 total 60.03 last 85.73) - open/close 6635 (time 1773235809.44 total 70.05 last 89.27) - open/close 7593 (time 1773235819.44 total 80.05 last 95.73) - open/close 8517 (time 1773235829.45 total 90.06 last 92.34) - open/close 9419 (time 1773235839.46 total 100.07 last 90.15) total: 10000 open/close in 105.28 seconds: 94.99 ops/second - unlinked 0 (time 1773235847 ; total 0 ; last 0) total: 10000 unlinks in 52 seconds: 192.307693 unlinks/second - open/close 1004 (time 1773235911.91 total 10.00 last 100.39) - open/close 2151 (time 1773235921.91 total 20.00 last 114.66) - open/close 3497 (time 1773235931.92 total 30.01 last 134.58) - open/close 4771 (time 1773235941.92 total 40.01 last 127.35) - open/close 5769 (time 1773235951.93 total 50.02 last 99.68) - open/close 7103 (time 1773235961.93 total 60.02 last 133.37) - open/close 8147 (time 1773235971.94 total 70.03 last 104.33) - open/close 9089 (time 1773235981.96 total 80.05 last 94.05) total: 10000 open/close in 89.17 seconds: 112.14 ops/second - unlinked 0 (time 1773235993 ; total 0 ; last 0) total: 10000 unlinks in 44 seconds: 227.272720 unlinks/second - open/close 1152 (time 1773236050.65 total 10.00 last 115.16) - open/close 2260 (time 1773236060.67 total 20.03 last 110.55) - open/close 3292 (time 1773236070.67 total 30.03 last 103.18) - open/close 4406 (time 1773236080.67 total 40.03 last 111.39) - open/close 5552 (time 1773236090.68 total 50.03 last 114.56) - open/close 6546 (time 1773236100.68 total 60.03 last 99.38) - open/close 7529 (time 1773236110.68 total 70.03 last 98.29) - open/close 8581 (time 1773236120.68 total 80.04 last 105.18) - open/close 9684 (time 1773236130.69 total 90.05 last 110.19) total: 10000 open/close in 92.53 seconds: 108.07 ops/second - unlinked 0 (time 1773236135 ; total 0 ; last 0) total: 10000 unlinks in 46 seconds: 217.391312 unlinks/second - open/close 1197 (time 1773236194.24 total 10.00 last 119.65) - open/close 2201 (time 1773236204.24 total 20.01 last 100.31) - open/close 3382 (time 1773236214.25 total 30.02 last 118.03) - open/close 4538 (time 1773236224.25 total 40.02 last 115.59) - open/close 5843 (time 1773236234.26 total 50.03 last 130.38) - open/close 7138 (time 1773236244.26 total 60.03 last 129.45) - open/close 8353 (time 1773236254.27 total 70.04 last 121.44) - open/close 9581 (time 1773236264.27 total 80.04 last 122.78) total: 10000 open/close in 83.81 seconds: 119.32 ops/second - unlinked 0 (time 1773236270 ; total 0 ; last 0) total: 10000 unlinks in 52 seconds: 192.307693 unlinks/second - open/close 1096 (time 1773236334.79 total 10.01 last 109.54) - open/close 2211 (time 1773236344.79 total 20.01 last 111.49) - open/close 3257 (time 1773236354.79 total 30.01 last 104.58) - open/close 4385 (time 1773236364.80 total 40.01 last 112.74) - open/close 5592 (time 1773236374.81 total 50.02 last 120.62) - open/close 6850 (time 1773236384.81 total 60.02 last 125.79) - open/close 7968 (time 1773236394.82 total 70.03 last 111.70) - open/close 9092 (time 1773236404.82 total 80.03 last 112.37) total: 10000 open/close in 88.01 seconds: 113.62 ops/second - unlinked 0 (time 1773236415 ; total 0 ; last 0) total: 10000 unlinks in 44 seconds: 227.272720 unlinks/second - open/close 1453 (time 1773236471.61 total 10.00 last 145.28) - open/close 2830 (time 1773236481.61 total 20.01 last 137.64) - open/close 4374 (time 1773236491.62 total 30.01 last 154.36) - open/close 5824 (time 1773236501.62 total 40.01 last 144.94) - open/close 7160 (time 1773236511.62 total 50.02 last 133.57) - open/close 8496 (time 1773236521.63 total 60.02 last 133.52) total: 10000 open/close in 67.74 seconds: 147.61 ops/second - unlinked 0 (time 1773236531 ; total 0 ; last 0) total: 10000 unlinks in 36 seconds: 277.777771 unlinks/second - open/close 1733 (time 1773236579.48 total 10.00 last 173.29) - open/close 2853 (time 1773236589.49 total 20.01 last 111.92) - open/close 4364 (time 1773236599.49 total 30.01 last 151.05) - open/close 5652 (time 1773236609.50 total 40.02 last 128.74) - open/close 6874 (time 1773236619.50 total 50.02 last 122.10) - open/close 8117 (time 1773236629.51 total 60.03 last 124.27) - open/close 9985 (time 1773236639.51 total 70.03 last 186.74) total: 10000 open/close in 70.08 seconds: 142.70 ops/second - unlinked 0 (time 1773236641 ; total 0 ; last 0) total: 10000 unlinks in 28 seconds: 357.142853 unlinks/second - open/close 1774 (time 1773236681.75 total 10.00 last 177.40) - open/close 3861 (time 1773236691.75 total 20.01 last 208.59) - open/close 5788 (time 1773236701.76 total 30.01 last 192.61) - open/close 7974 (time 1773236711.76 total 40.01 last 218.58) total: 10000 open/close in 49.08 seconds: 203.73 ops/second - unlinked 0 (time 1773236722 ; total 0 ; last 0) total: 10000 unlinks in 22 seconds: 454.545441 unlinks/second - open/close 2289 (time 1773236756.52 total 10.00 last 228.85) - open/close 4724 (time 1773236766.52 total 20.00 last 243.48) - open/close 7374 (time 1773236776.52 total 30.01 last 264.80) - open/close 9464 (time 1773236786.53 total 40.01 last 208.94) total: 10000 open/close in 42.57 seconds: 234.92 ops/second - unlinked 0 (time 1773236790 ; total 0 ; last 0) total: 10000 unlinks in 29 seconds: 344.827576 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg111-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg111-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg111-server Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x42 (OST update ) Persistent mount opts: ,errors=remount-ro Parameters: mgsnode=192.168.201.111@tcp sys.timeout=20 device size = 4096MB formatting backing filesystem ldiskfs on /dev/mapper/ost1_flakey target name lustre-OST0000 kilobytes 200000 options -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F mkfs_cmd = mke2fs -j -b 4096 -L lustre-OST0000 -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/mapper/ost1_flakey 200000k Writing CONFIGS/mountdata start ost1 service on oleg111-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg111-server: oleg111-server.virtnet: executing set_default_debug -1 all pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Started lustre-OST0000 oleg111-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg111-server: oleg111-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg111-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec oleg111-server: oleg111-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg111-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg111-client.virtnet: -o user_xattr,flock 192.168.201.111@tcp:/lustre /mnt/lustre On OST0, 16122 used inodes rc=0 pdsh@oleg111-client: oleg111-server: ssh exited with exit code 22 umount lustre on /mnt/lustre..... Stopping client oleg111-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg111-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg111-server Permanent disk data: Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x62 (OST first_time update ) Persistent mount opts: ,errors=remount-ro Parameters: mgsnode=192.168.201.111@tcp sys.timeout=20 device size = 4096MB formatting backing filesystem ldiskfs on /dev/mapper/ost1_flakey target name lustre:OST0000 kilobytes 200000 options -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F mkfs_cmd = mke2fs -j -b 4096 -L lustre:OST0000 -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/mapper/ost1_flakey 200000k Writing CONFIGS/mountdata checking for existing Lustre data: found Read previous values: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x62 (OST first_time update ) Persistent mount opts: ,errors=remount-ro Parameters: mgsnode=192.168.201.111@tcp sys.timeout=20 Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x42 (OST update ) Persistent mount opts: ,errors=remount-ro Parameters: mgsnode=192.168.201.111@tcp sys.timeout=20 Writing CONFIGS/mountdata start ost1 service on oleg111-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg111-server: oleg111-server.virtnet: executing set_default_debug -1 all pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Started lustre-OST0000 oleg111-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg111-server: oleg111-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg111-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 4 sec oleg111-server: oleg111-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg111-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg111-client.virtnet: -o user_xattr,flock 192.168.201.111@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg111-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg111-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg111-server stop mds service on oleg111-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg111-server stop mds service on oleg111-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg111-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg111-server' oleg111-server: oleg111-server.virtnet: executing unload_modules_local oleg111-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg111-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg111-client: oleg111-client: ssh exited with exit code 2 pdsh@oleg111-client: oleg111-server: ssh exited with exit code 2 pdsh@oleg111-client: oleg111-client: ssh exited with exit code 2 pdsh@oleg111-client: oleg111-server: ssh exited with exit code 2 PASS 69 (1369s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 09:48:15 (1773236895) oleg111-server: debugfs 1.47.3-wc2 (11-Nov-2025) Supported features: dir_prealloc imagic_inodes has_journal ext_attr resize_inode dir_index sparse_super2 fast_commit stable_inodes orphan_file filetype needs_recovery journal_dev meta_bg extent 64bit mmp flex_bg ea_inode dirdata metadata_csum_seed large_dir inline_data encrypt casefold sparse_super large_file huge_file uninit_bg dir_nlink extra_isize quota bigalloc metadata_csum read-only project shared_blocks verity orphan_present Stopping clients: oleg111-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg111-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg111-client: oleg111-server: ssh exited with exit code 2 oleg111-server: oleg111-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg111-server' oleg111-server: oleg111-server.virtnet: executing load_modules_local oleg111-server: Loading modules from /home/green/git/lustre-release/lustre oleg111-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg111-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg111-server: detected 4 online CPUs by sysfs oleg111-server: MODOPTS_LIBCFS= oleg111-server: Force libcfs to create 2 CPU partitions oleg111-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg111-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format ost1: /dev/mapper/ost1_flakey pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 SKIP: conf-sanity test_111 /dev/mapper/mds1_flakey too small for 5000000kB MDS Stopping clients: oleg111-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg111-client.virtnet /mnt/lustre2 (opts:-f) oleg111-server: oleg111-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg111-server' oleg111-server: oleg111-server.virtnet: executing load_modules_local oleg111-server: Loading modules from /home/green/git/lustre-release/lustre oleg111-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg111-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg111-server: detected 4 online CPUs by sysfs oleg111-server: MODOPTS_LIBCFS= oleg111-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg111-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg111-server: oleg111-server.virtnet: executing set_default_debug -1 all pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg111-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg111-server: oleg111-server.virtnet: executing set_default_debug -1 all pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg111-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg111-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg111-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg111-server: oleg111-server.virtnet: executing set_default_debug -1 all pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg111-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg111-server: oleg111-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg111-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg111-server: oleg111-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg111-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg111-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg111-server stop mds service on oleg111-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg111-server stop mds service on oleg111-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg111-server SKIP 111 (98s) Stopping clients: oleg111-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg111-client.virtnet /mnt/lustre2 (opts:-f) oleg111-server: oleg111-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg111-server' oleg111-server: oleg111-server.virtnet: executing load_modules_local oleg111-server: Loading modules from /home/green/git/lustre-release/lustre oleg111-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg111-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg111-server: detected 4 online CPUs by sysfs oleg111-server: MODOPTS_LIBCFS= oleg111-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey == conf-sanity test complete, duration 1889 sec ========== 09:50:16 (1773237016) === conf-sanity: start cleanup 09:50:17 (1773237017) === === conf-sanity: finish cleanup 09:50:18 (1773237018) ===