-----============= acceptance-small: conf-sanity ============----- Wed Aug 5 08:11:07 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg127-server: /home/green/git/lustre-release/lustre/tests/except/conf-sanity.ex - see CLIENT_VERSION > v2_15_63-53-g1a6ef725c2 (34682930 > 34553653) for LU-16938, go 81 excepting tests: 32b 32c 32newtarball Stopping clients: oleg127-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg127-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg127-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg127-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg127-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg127-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg127-server oleg127-server: oleg127-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg127-server' oleg127-server: oleg127-server.virtnet: executing load_modules_local oleg127-server: Loading modules from /home/green/git/lustre-release/lustre oleg127-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg127-server: detected 4 online CPUs by sysfs oleg127-server: MODOPTS_LIBCFS= oleg127-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg127-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg127-server: oleg127-server.virtnet: executing set_default_debug -1 all pdsh@oleg127-client: oleg127-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg127-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg127-server: oleg127-server.virtnet: executing set_default_debug -1 all pdsh@oleg127-client: oleg127-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg127-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg127-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg127-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg127-server: oleg127-server.virtnet: executing set_default_debug -1 all pdsh@oleg127-client: oleg127-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg127-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg127-server: oleg127-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg127-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg127-server: oleg127-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg127-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg127-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg127-server stop mds service on oleg127-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg127-server stop mds service on oleg127-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg127-server == conf-sanity test 45: long unlink handling in ptlrpcd == 08:15:50 (1785932150) start mds service on oleg127-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg127-server: oleg127-server.virtnet: executing set_default_debug -1 all pdsh@oleg127-client: oleg127-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg127-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg127-server: oleg127-server.virtnet: executing set_default_debug -1 all pdsh@oleg127-client: oleg127-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg127-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg127-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg127-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg127-server: oleg127-server.virtnet: executing set_default_debug -1 all pdsh@oleg127-client: oleg127-server: ssh exited with exit code 1 Started lustre-OST0000 oleg127-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg127-client.virtnet: -o user_xattr,flock 192.168.201.127@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg127-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg127-server stop mds service on oleg127-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg127-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg127-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg127-server: oleg127-server.virtnet: executing set_default_debug -1 all pdsh@oleg127-client: oleg127-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg127-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg127-server: oleg127-server.virtnet: executing set_default_debug -1 all pdsh@oleg127-client: oleg127-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg127-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg127-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg127-client.virtnet: -o user_xattr,flock 192.168.201.127@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg127-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg127-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg127-server stop mds service on oleg127-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg127-server stop mds service on oleg127-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg127-server unloading modules via unload_modules_local on: 'oleg127-server' oleg127-server: oleg127-server.virtnet: executing unload_modules_local oleg127-server: modules unloaded. pdsh@oleg127-client: oleg127-server: ssh exited with exit code 2 pdsh@oleg127-client: oleg127-client: ssh exited with exit code 2 pdsh@oleg127-client: oleg127-client: ssh exited with exit code 2 pdsh@oleg127-client: oleg127-server: ssh exited with exit code 2 PASS 45 (340s) == conf-sanity test 69: replace an OST with the same index ========================================================== 08:21:30 (1785932490) start mds service on oleg127-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg127-server' oleg127-server: oleg127-server.virtnet: executing load_modules_local oleg127-server: Loading modules from /home/green/git/lustre-release/lustre oleg127-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg127-server: detected 4 online CPUs by sysfs oleg127-server: MODOPTS_LIBCFS= oleg127-server: Force libcfs to create 2 CPU partitions oleg127-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg127-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg127-server: mdt/mdt options: 'mdt_enable_flr_ec=1' Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg127-server: oleg127-server.virtnet: executing set_default_debug -1 all pdsh@oleg127-client: oleg127-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg127-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg127-server: oleg127-server.virtnet: executing set_default_debug -1 all pdsh@oleg127-client: oleg127-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg127-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg127-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg127-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg127-server: oleg127-server.virtnet: executing set_default_debug -1 all pdsh@oleg127-client: oleg127-server: ssh exited with exit code 1 Started lustre-OST0000 oleg127-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg127-client.virtnet: -o user_xattr,flock 192.168.201.127@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 49503 inodes available. Want 99872. rc=0 - open/close 868 (time 1785932626.96 total 10.03 last 86.58) - open/close 1586 (time 1785932636.97 total 20.03 last 71.77) - open/close 2343 (time 1785932646.97 total 30.04 last 75.64) - open/close 2848 (time 1785932656.98 total 40.05 last 50.44) - open/close 3505 (time 1785932667.00 total 50.06 last 65.61) - open/close 4221 (time 1785932677.01 total 60.07 last 71.53) - open/close 4797 (time 1785932687.02 total 70.08 last 57.54) - open/close 5309 (time 1785932697.02 total 80.09 last 51.16) - open/close 5814 (time 1785932707.05 total 90.11 last 50.38) - open/close 6482 (time 1785932717.06 total 100.12 last 66.72) - open/close 7138 (time 1785932727.06 total 110.13 last 65.58) - open/close 7748 (time 1785932737.07 total 120.13 last 60.99) - open/close 8378 (time 1785932747.07 total 130.13 last 62.97) - open/close 9077 (time 1785932757.09 total 140.16 last 69.73) - open/close 9624 (time 1785932767.12 total 150.19 last 54.55) total: 10000 open/close in 156.46 seconds: 63.91 ops/second - unlinked 0 (time 1785932779 ; total 0 ; last 0) total: 10000 unlinks in 81 seconds: 123.456787 unlinks/second - open/close 459 (time 1785932876.84 total 10.02 last 45.79) - open/close 1036 (time 1785932886.84 total 20.03 last 57.67) - open/close 1566 (time 1785932896.85 total 30.03 last 52.97) - open/close 2051 (time 1785932906.87 total 40.05 last 48.42) - open/close 2533 (time 1785932916.87 total 50.05 last 48.19) - open/close 3025 (time 1785932926.87 total 60.05 last 49.19) - open/close 3587 (time 1785932936.87 total 70.06 last 56.20) - open/close 4445 (time 1785932946.87 total 80.06 last 85.77) - open/close 5203 (time 1785932956.89 total 90.08 last 75.68) - open/close 6078 (time 1785932966.90 total 100.08 last 87.45) - open/close 6771 (time 1785932976.92 total 110.10 last 69.15) - open/close 7254 (time 1785932986.93 total 120.11 last 48.25) - open/close 7884 (time 1785932996.94 total 130.13 last 62.90) - open/close 8659 (time 1785933006.95 total 140.13 last 77.47) - open/close 9464 (time 1785933016.95 total 150.14 last 80.46) total: 10000 open/close in 158.55 seconds: 63.07 ops/second - unlinked 0 (time 1785933030 ; total 0 ; last 0) total: 10000 unlinks in 79 seconds: 126.582275 unlinks/second - open/close 701 (time 1785933125.94 total 10.01 last 70.06) - open/close 1367 (time 1785933135.97 total 20.03 last 66.44) - open/close 2085 (time 1785933145.98 total 30.04 last 71.71) - open/close 2718 (time 1785933155.98 total 40.04 last 63.30) - open/close 3308 (time 1785933166.00 total 50.06 last 58.87) - open/close 3841 (time 1785933176.02 total 60.08 last 53.21) - open/close 4281 (time 1785933186.04 total 70.10 last 43.91) - open/close 4758 (time 1785933196.05 total 80.12 last 47.63) - open/close 5220 (time 1785933206.07 total 90.13 last 46.13) - open/close 5975 (time 1785933216.08 total 100.14 last 75.41) - open/close 6547 (time 1785933226.10 total 110.16 last 57.10) - open/close 7040 (time 1785933236.11 total 120.17 last 49.26) - open/close 7452 (time 1785933246.12 total 130.18 last 41.16) - open/close 7878 (time 1785933256.14 total 140.20 last 42.52) - open/close 8419 (time 1785933266.14 total 150.21 last 54.05) - open/close 9166 (time 1785933276.15 total 160.22 last 74.64) - open/close 9934 (time 1785933286.17 total 170.24 last 76.64) total: 10000 open/close in 171.31 seconds: 58.37 ops/second - unlinked 0 (time 1785933293 ; total 0 ; last 0) total: 10000 unlinks in 91 seconds: 109.890106 unlinks/second - open/close 550 (time 1785933400.61 total 10.00 last 54.98) - open/close 1181 (time 1785933410.63 total 20.03 last 62.97) - open/close 1645 (time 1785933420.63 total 30.03 last 46.39) - open/close 2020 (time 1785933430.64 total 40.03 last 37.48) - open/close 2629 (time 1785933440.64 total 50.04 last 60.88) - open/close 3359 (time 1785933450.67 total 60.06 last 72.84) - open/close 4156 (time 1785933460.68 total 70.07 last 79.62) - open/close 4805 (time 1785933470.69 total 80.08 last 64.83) - open/close 5331 (time 1785933480.71 total 90.10 last 52.47) - open/close 5842 (time 1785933490.72 total 100.11 last 51.06) - open/close 6390 (time 1785933500.72 total 110.12 last 54.78) - open/close 6985 (time 1785933510.72 total 120.12 last 59.49) - open/close 7863 (time 1785933520.73 total 130.13 last 87.73) - open/close 8520 (time 1785933530.75 total 140.14 last 65.58) - open/close 8939 (time 1785933540.78 total 150.18 last 41.76) - open/close 9357 (time 1785933550.81 total 160.21 last 41.68) - open/close 9799 (time 1785933560.82 total 170.21 last 44.19) total: 10000 open/close in 173.93 seconds: 57.49 ops/second - unlinked 0 (time 1785933570 ; total 0 ; last 0) total: 10000 unlinks in 94 seconds: 106.382980 unlinks/second - open/close 508 (time 1785933680.55 total 10.01 last 50.75) - open/close 1155 (time 1785933690.55 total 20.01 last 64.69) - open/close 1707 (time 1785933700.58 total 30.05 last 55.01) - open/close 2124 (time 1785933710.62 total 40.09 last 41.54) - open/close 2643 (time 1785933720.63 total 50.09 last 51.88) - open/close 3213 (time 1785933730.63 total 60.09 last 56.99) - open/close 3839 (time 1785933740.65 total 70.11 last 62.48) - open/close 4388 (time 1785933750.66 total 80.12 last 54.86) - open/close 4905 (time 1785933760.66 total 90.13 last 51.66) - open/close 5437 (time 1785933770.68 total 100.15 last 53.08) - open/close 5877 (time 1785933780.69 total 110.15 last 43.98) - open/close 6419 (time 1785933790.73 total 120.19 last 53.99) - open/close 6985 (time 1785933800.75 total 130.22 last 56.46) - open/close 7540 (time 1785933810.77 total 140.24 last 55.39) - open/close 8096 (time 1785933820.80 total 150.26 last 55.45) - open/close 8563 (time 1785933830.81 total 160.28 last 46.64) - open/close 8932 (time 1785933840.84 total 170.30 last 36.80) - open/close 9438 (time 1785933850.85 total 180.32 last 50.54) total: 10000 open/close in 187.09 seconds: 53.45 ops/second - unlinked 0 (time 1785933863 ; total 0 ; last 0) total: 10000 unlinks in 88 seconds: 113.636360 unlinks/second - open/close 468 (time 1785933968.15 total 10.03 last 46.64) - open/close 982 (time 1785933978.16 total 20.05 last 51.32) - open/close 1542 (time 1785933988.18 total 30.07 last 55.89) - open/close 2254 (time 1785933998.20 total 40.08 last 71.10) - open/close 2784 (time 1785934008.20 total 50.08 last 53.00) - open/close 3298 (time 1785934018.21 total 60.09 last 51.35) - open/close 3856 (time 1785934028.22 total 70.10 last 55.73) - open/close 4462 (time 1785934038.22 total 80.11 last 60.57) - open/close 5015 (time 1785934048.23 total 90.11 last 55.28) - open/close 5567 (time 1785934058.24 total 100.13 last 55.11) - open/close 6079 (time 1785934068.24 total 110.13 last 51.19) - open/close 6698 (time 1785934078.26 total 120.14 last 61.82) - open/close 7159 (time 1785934088.26 total 130.15 last 46.07) - open/close 7690 (time 1785934098.46 total 140.34 last 52.10) - open/close 8117 (time 1785934108.47 total 150.36 last 42.62) - open/close 8668 (time 1785934118.48 total 160.37 last 55.04) - open/close 9232 (time 1785934128.49 total 170.37 last 56.38) total: 10000 open/close in 179.59 seconds: 55.68 ops/second - unlinked 0 (time 1785934143 ; total 0 ; last 0) total: 10000 unlinks in 94 seconds: 106.382980 unlinks/second - open/close 832 (time 1785934253.65 total 10.00 last 83.18) - open/close 1585 (time 1785934263.66 total 20.01 last 75.24) - open/close 2082 (time 1785934273.67 total 30.02 last 49.66) - open/close 2722 (time 1785934283.69 total 40.04 last 63.87) - open/close 3746 (time 1785934293.70 total 50.05 last 102.32) - open/close 4254 (time 1785934303.70 total 60.05 last 50.79) - open/close 4922 (time 1785934313.72 total 70.07 last 66.65) - open/close 5336 (time 1785934323.72 total 80.07 last 41.39) - open/close 5920 (time 1785934333.73 total 90.08 last 58.37) - open/close 6594 (time 1785934343.74 total 100.09 last 67.35) - open/close 7167 (time 1785934353.74 total 110.09 last 57.29) - open/close 8140 (time 1785934363.74 total 120.09 last 97.26) - open/close 8814 (time 1785934373.74 total 130.09 last 67.40) - open/close 9396 (time 1785934383.75 total 140.10 last 58.14) - open/close 9934 (time 1785934393.78 total 150.13 last 53.67) total: 10000 open/close in 151.37 seconds: 66.06 ops/second - unlinked 0 (time 1785934401 ; total 0 ; last 0) total: 10000 unlinks in 72 seconds: 138.888885 unlinks/second - open/close 544 (time 1785934490.18 total 10.01 last 54.36) - open/close 933 (time 1785934500.21 total 20.04 last 38.76) - open/close 1403 (time 1785934510.22 total 30.05 last 46.95) - open/close 1887 (time 1785934520.22 total 40.06 last 48.39) - open/close 2693 (time 1785934530.27 total 50.10 last 80.20) - open/close 3188 (time 1785934540.28 total 60.11 last 49.49) - open/close 3799 (time 1785934550.28 total 70.11 last 61.07) - open/close 4516 (time 1785934560.30 total 80.13 last 71.60) - open/close 5411 (time 1785934570.30 total 90.13 last 89.49) - open/close 5998 (time 1785934580.31 total 100.14 last 58.62) - open/close 6735 (time 1785934590.31 total 110.14 last 73.69) - open/close 7404 (time 1785934600.32 total 120.15 last 66.83) - open/close 8192 (time 1785934610.33 total 130.16 last 78.74) - open/close 9103 (time 1785934620.33 total 140.17 last 91.05) total: 10000 open/close in 149.61 seconds: 66.84 ops/second - unlinked 0 (time 1785934636 ; total 0 ; last 0) total: 10000 unlinks in 79 seconds: 126.582275 unlinks/second - open/close 566 (time 1785934731.48 total 10.02 last 56.51) - open/close 1119 (time 1785934741.48 total 20.02 last 55.28) - open/close 1665 (time 1785934751.49 total 30.03 last 54.54) - open/close 2200 (time 1785934761.50 total 40.03 last 53.49) - open/close 3133 (time 1785934771.50 total 50.03 last 93.29) - open/close 3624 (time 1785934781.50 total 60.04 last 49.07) - open/close 4134 (time 1785934791.50 total 70.04 last 51.00) - open/close 4709 (time 1785934801.51 total 80.04 last 57.47) - open/close 5085 (time 1785934811.51 total 90.05 last 37.58) - open/close 5456 (time 1785934821.52 total 100.05 last 37.09) - open/close 6136 (time 1785934831.52 total 110.06 last 67.97) - open/close 6697 (time 1785934841.53 total 120.07 last 56.03) - open/close 7282 (time 1785934851.62 total 130.15 last 58.00) - open/close 7890 (time 1785934861.62 total 140.16 last 60.77) - open/close 8360 (time 1785934871.63 total 150.16 last 46.98) - open/close 8923 (time 1785934881.63 total 160.16 last 56.30) - open/close 9555 (time 1785934891.63 total 170.17 last 63.14) total: 10000 open/close in 175.24 seconds: 57.06 ops/second - unlinked 0 (time 1785934902 ; total 0 ; last 0) total: 10000 unlinks in 76 seconds: 131.578949 unlinks/second - open/close 466 (time 1785934994.18 total 10.00 last 46.60) - open/close 913 (time 1785935004.19 total 20.01 last 44.68) - open/close 1500 (time 1785935014.19 total 30.01 last 58.66) - open/close 2359 (time 1785935024.21 total 40.03 last 85.73) - open/close 2959 (time 1785935034.21 total 50.03 last 59.99) - open/close 3633 (time 1785935044.23 total 60.05 last 67.25) - open/close 4343 (time 1785935054.24 total 70.06 last 70.98) - open/close 5036 (time 1785935064.25 total 80.07 last 69.20) - open/close 5563 (time 1785935074.26 total 90.08 last 52.68) - open/close 6352 (time 1785935084.27 total 100.09 last 78.79) - open/close 7011 (time 1785935094.29 total 110.11 last 65.79) - open/close 7462 (time 1785935104.30 total 120.12 last 45.05) - open/close 7945 (time 1785935114.33 total 130.15 last 48.13) - open/close 8561 (time 1785935124.34 total 140.16 last 61.55) - open/close 9465 (time 1785935134.34 total 150.16 last 90.37) total: 10000 open/close in 154.72 seconds: 64.63 ops/second - unlinked 0 (time 1785935143 ; total 0 ; last 0) total: 10000 unlinks in 82 seconds: 121.951218 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg127-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg127-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg127-server Permanent disk data: Device: /dev/mapper/ost1_flakey Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x42 (OST update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.201.127@tcp sys.timeout=20 device size = 4096MB formatting backing filesystem ldiskfs on /dev/mapper/ost1_flakey target name lustre-OST0000 kilobytes 200000 options -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F mkfs_cmd = mke2fs -j -b 4096 -L lustre-OST0000 -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/mapper/ost1_flakey 200000k start ost1 service on oleg127-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg127-server: oleg127-server.virtnet: executing set_default_debug -1 all pdsh@oleg127-client: oleg127-server: ssh exited with exit code 1 Started lustre-OST0000 oleg127-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg127-server: oleg127-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg127-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg127-server: oleg127-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg127-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg127-client.virtnet: -o user_xattr,flock 192.168.201.127@tcp:/lustre /mnt/lustre On OST0, 20430 used inodes rc=0 pdsh@oleg127-client: oleg127-server: ssh exited with exit code 22 umount lustre on /mnt/lustre..... Stopping client oleg127-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg127-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg127-server Permanent disk data: Device: /dev/mapper/ost1_flakey Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x62 (OST first_time update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.201.127@tcp sys.timeout=20 device size = 4096MB formatting backing filesystem ldiskfs on /dev/mapper/ost1_flakey target name lustre:OST0000 kilobytes 200000 options -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F mkfs_cmd = mke2fs -j -b 4096 -L lustre:OST0000 -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/mapper/ost1_flakey 200000k checking for existing Lustre data: found Read previous values: Device: /dev/mapper/ost1_flakey Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x62 (OST first_time update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.201.127@tcp sys.timeout=20 Permanent disk data: Device: /dev/mapper/ost1_flakey Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x42 (OST update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.201.127@tcp sys.timeout=20 start ost1 service on oleg127-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg127-server: oleg127-server.virtnet: executing set_default_debug -1 all pdsh@oleg127-client: oleg127-server: ssh exited with exit code 1 Started lustre-OST0000 oleg127-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg127-server: oleg127-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg127-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg127-server: oleg127-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg127-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg127-client.virtnet: -o user_xattr,flock 192.168.201.127@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg127-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg127-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg127-server stop mds service on oleg127-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg127-server stop mds service on oleg127-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg127-server unloading modules via unload_modules_local on: 'oleg127-server' oleg127-server: oleg127-server.virtnet: executing unload_modules_local oleg127-server: modules unloaded. pdsh@oleg127-client: oleg127-server: ssh exited with exit code 2 pdsh@oleg127-client: oleg127-client: ssh exited with exit code 2 pdsh@oleg127-client: oleg127-server: ssh exited with exit code 2 pdsh@oleg127-client: oleg127-client: ssh exited with exit code 2 PASS 69 (2956s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 09:10:48 (1785935448) oleg127-server: debugfs 1.47.3-wc2 (11-Nov-2025) Supported features: dir_prealloc imagic_inodes has_journal ext_attr resize_inode dir_index sparse_super2 fast_commit stable_inodes orphan_file filetype needs_recovery journal_dev meta_bg extent 64bit mmp flex_bg ea_inode dirdata metadata_csum_seed large_dir inline_data encrypt casefold sparse_super large_file huge_file uninit_bg dir_nlink extra_isize quota bigalloc metadata_csum read-only project shared_blocks verity orphan_present Stopping clients: oleg127-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg127-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg127-client: oleg127-server: ssh exited with exit code 2 oleg127-server: oleg127-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg127-server' oleg127-server: oleg127-server.virtnet: executing load_modules_local oleg127-server: Loading modules from /home/green/git/lustre-release/lustre oleg127-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg127-server: detected 4 online CPUs by sysfs oleg127-server: MODOPTS_LIBCFS= oleg127-server: Force libcfs to create 2 CPU partitions oleg127-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg127-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg127-server: mdt/mdt options: 'mdt_enable_flr_ec=1' Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format ost1: /dev/mapper/ost1_flakey pdsh@oleg127-client: oleg127-server: ssh exited with exit code 1 SKIP: conf-sanity test_111 /dev/mapper/mds1_flakey too small for 5000000kB MDS Stopping clients: oleg127-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg127-client.virtnet /mnt/lustre2 (opts:-f) oleg127-server: oleg127-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg127-server' oleg127-server: oleg127-server.virtnet: executing load_modules_local oleg127-server: Loading modules from /home/green/git/lustre-release/lustre oleg127-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg127-server: detected 4 online CPUs by sysfs oleg127-server: MODOPTS_LIBCFS= oleg127-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg127-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg127-server: oleg127-server.virtnet: executing set_default_debug -1 all pdsh@oleg127-client: oleg127-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg127-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg127-server: oleg127-server.virtnet: executing set_default_debug -1 all pdsh@oleg127-client: oleg127-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg127-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg127-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg127-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg127-server: oleg127-server.virtnet: executing set_default_debug -1 all pdsh@oleg127-client: oleg127-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg127-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg127-server: oleg127-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg127-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg127-server: oleg127-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg127-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg127-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg127-server stop mds service on oleg127-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg127-server stop mds service on oleg127-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg127-server SKIP 111 (307s) Stopping clients: oleg127-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg127-client.virtnet /mnt/lustre2 (opts:-f) oleg127-server: oleg127-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg127-server' oleg127-server: oleg127-server.virtnet: executing load_modules_local oleg127-server: Loading modules from /home/green/git/lustre-release/lustre oleg127-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg127-server: detected 4 online CPUs by sysfs oleg127-server: MODOPTS_LIBCFS= oleg127-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey == conf-sanity test complete, duration 3963 sec ========== 09:17:12 (1785935832) === conf-sanity: start cleanup 09:17:15 (1785935835) === === conf-sanity: finish cleanup 09:17:20 (1785935840) ===