-----============= acceptance-small: conf-sanity ============----- Thu Sep 3 16:51:05 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg627-server: /home/green/git/lustre-release/lustre/tests/except/conf-sanity.ex - see CLIENT_VERSION >= v2_15_63-53-g1a6ef725c2 (34683396 >= 34553653) for LU-16938, go 81 - see CLIENT_VERSION >= v2_15_65-31-g74a5b9efaa (34683396 >= 34554143) for LU-18348, go 32b 32c - see CLIENT_VERSION >= v2_16_55-19-gf69f27cc35 (34683396 >= 34617107) for LU-17920, go 123ae 123ag - see CLIENT_VERSION >= v2_17_52-94-g86904ef943 (34683396 >= 34681950) for LU-19740, go 115 - see CLIENT_VERSION >= v2_17_57-14-g2924d62f60 (34683396 >= 34683150) for LU-20214, go 104a excepting tests: 32b 32c 32newtarball Stopping clients: oleg627-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg627-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg627-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg627-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg627-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg627-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg627-server oleg627-server: oleg627-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg627-server' oleg627-server: oleg627-server.virtnet: executing load_modules_local oleg627-server: kmemleak disabled oleg627-server: Loading modules from /home/green/git/lustre-release/lustre oleg627-server: detected 4 online CPUs by sysfs oleg627-server: MODOPTS_LIBCFS= oleg627-server: Force libcfs to create 2 CPU partitions oleg627-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg627-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg627-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg627-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg627-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg627-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg627-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg627-server: oleg627-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg627-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg627-server: oleg627-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg627-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg627-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg627-server stop mds service on oleg627-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg627-server stop mds service on oleg627-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg627-server == conf-sanity test 45: long unlink handling in ptlrpcd == 16:54:20 (1788468860) start mds service on oleg627-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg627-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg627-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg627-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg627-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Started lustre-OST0000 oleg627-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg627-client.virtnet: -o user_xattr,flock 192.168.206.127@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg627-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg627-server stop mds service on oleg627-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg627-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg627-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg627-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg627-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg627-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg627-client.virtnet: -o user_xattr,flock 192.168.206.127@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg627-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg627-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg627-server stop mds service on oleg627-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg627-server stop mds service on oleg627-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg627-server unloading modules via unload_modules_local on: 'oleg627-server' oleg627-server: oleg627-server.virtnet: executing unload_modules_local oleg627-server: kmemleak disabled modules unloaded. pdsh@oleg627-client: oleg627-server: ssh exited with exit code 2 pdsh@oleg627-client: oleg627-client: ssh exited with exit code 2 pdsh@oleg627-client: oleg627-server: ssh exited with exit code 2 pdsh@oleg627-client: oleg627-client: ssh exited with exit code 2 PASS 45 (293s) == conf-sanity test 69: replace an OST with the same index ========================================================== 16:59:13 (1788469153) start mds service on oleg627-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' -rwxr-xr-x. 1 green green 1921072 Sep 3 15:45 /home/green/git/lustre-release/lustre/utils/mount.lustre -rwxr-xr-x. 1 root root 12 Jan 31 2019 /usr/sbin/mount.lustre mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre loading modules on: 'oleg627-server' oleg627-server: oleg627-server.virtnet: executing load_modules_local oleg627-server: kmemleak disabled oleg627-server: Loading modules from /home/green/git/lustre-release/lustre oleg627-server: detected 4 online CPUs by sysfs oleg627-server: MODOPTS_LIBCFS= oleg627-server: Force libcfs to create 2 CPU partitions oleg627-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg627-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg627-server: mdt/mdt options: 'mdt_enable_flr_ec=1' oleg627-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg627-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg627-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg627-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg627-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Started lustre-OST0000 oleg627-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg627-client.virtnet: -o user_xattr,flock 192.168.206.127@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 49502 inodes available. Want 99872. rc=0 - open/close 754 (time 1788469249.19 total 10.00 last 75.40) - open/close 1481 (time 1788469259.20 total 20.01 last 72.65) - open/close 2261 (time 1788469269.20 total 30.01 last 77.97) - open/close 3007 (time 1788469279.21 total 40.01 last 74.58) - open/close 3779 (time 1788469289.21 total 50.02 last 77.15) - open/close 4429 (time 1788469299.22 total 60.03 last 64.96) - open/close 5258 (time 1788469309.22 total 70.03 last 82.88) - open/close 6056 (time 1788469319.22 total 80.03 last 79.77) - open/close 6828 (time 1788469329.24 total 90.04 last 77.11) - open/close 7621 (time 1788469339.24 total 100.05 last 79.27) - open/close 8336 (time 1788469349.24 total 110.05 last 71.49) - open/close 9082 (time 1788469359.24 total 120.05 last 74.59) - open/close 9696 (time 1788469369.26 total 130.07 last 61.28) total: 10000 open/close in 134.50 seconds: 74.35 ops/second - unlinked 0 (time 1788469378 ; total 0 ; last 0) total: 10000 unlinks in 69 seconds: 144.927536 unlinks/second - open/close 719 (time 1788469462.64 total 10.01 last 71.84) - open/close 1513 (time 1788469472.66 total 20.02 last 79.30) - open/close 2243 (time 1788469482.67 total 30.03 last 72.94) - open/close 3072 (time 1788469492.67 total 40.04 last 82.82) - open/close 3808 (time 1788469502.68 total 50.04 last 73.59) - open/close 4689 (time 1788469512.68 total 60.05 last 88.06) - open/close 5389 (time 1788469522.70 total 70.06 last 69.88) - open/close 6191 (time 1788469532.70 total 80.07 last 80.17) - open/close 6948 (time 1788469542.71 total 90.08 last 75.64) - open/close 7707 (time 1788469552.73 total 100.10 last 75.75) - open/close 8503 (time 1788469562.74 total 110.10 last 79.56) - open/close 9278 (time 1788469572.74 total 120.10 last 77.49) - open/close 9947 (time 1788469582.76 total 130.12 last 66.76) total: 10000 open/close in 130.61 seconds: 76.56 ops/second - unlinked 0 (time 1788469587 ; total 0 ; last 0) total: 10000 unlinks in 68 seconds: 147.058823 unlinks/second - open/close 586 (time 1788469671.26 total 10.02 last 58.50) - open/close 1132 (time 1788469681.28 total 20.03 last 54.51) - open/close 1907 (time 1788469691.29 total 30.04 last 77.44) - open/close 2668 (time 1788469701.29 total 40.04 last 76.09) - open/close 3517 (time 1788469711.30 total 50.05 last 84.81) - open/close 4451 (time 1788469721.31 total 60.06 last 93.32) - open/close 5171 (time 1788469731.31 total 70.07 last 71.94) - open/close 5915 (time 1788469741.33 total 80.09 last 74.28) - open/close 6425 (time 1788469751.35 total 90.10 last 50.92) - open/close 7017 (time 1788469761.35 total 100.10 last 59.19) - open/close 7849 (time 1788469771.36 total 110.11 last 83.14) - open/close 8593 (time 1788469781.36 total 120.11 last 74.37) - open/close 9229 (time 1788469791.36 total 130.12 last 63.57) - open/close 9746 (time 1788469801.37 total 140.12 last 51.68) total: 10000 open/close in 146.25 seconds: 68.38 ops/second - unlinked 0 (time 1788469812 ; total 0 ; last 0) total: 10000 unlinks in 77 seconds: 129.870132 unlinks/second - open/close 796 (time 1788469903.51 total 10.00 last 79.56) - open/close 1378 (time 1788469913.52 total 20.01 last 58.17) - open/close 2162 (time 1788469923.53 total 30.02 last 78.31) - open/close 2869 (time 1788469933.54 total 40.03 last 70.66) - open/close 3644 (time 1788469943.56 total 50.05 last 77.31) - open/close 4392 (time 1788469953.57 total 60.06 last 74.72) - open/close 5008 (time 1788469963.59 total 70.08 last 61.50) - open/close 5623 (time 1788469973.61 total 80.10 last 61.40) - open/close 6175 (time 1788469983.61 total 90.10 last 55.18) - open/close 6723 (time 1788469993.64 total 100.13 last 54.66) - open/close 7362 (time 1788470003.64 total 110.13 last 63.87) - open/close 8034 (time 1788470013.65 total 120.14 last 67.10) - open/close 8732 (time 1788470023.71 total 130.20 last 69.44) - open/close 9216 (time 1788470033.71 total 140.20 last 48.37) - open/close 9807 (time 1788470043.72 total 150.21 last 59.03) total: 10000 open/close in 153.11 seconds: 65.31 ops/second - unlinked 0 (time 1788470050 ; total 0 ; last 0) total: 10000 unlinks in 80 seconds: 125.000000 unlinks/second - open/close 499 (time 1788470146.62 total 10.06 last 49.60) - open/close 1049 (time 1788470156.62 total 20.07 last 54.96) - open/close 1541 (time 1788470166.64 total 30.08 last 49.14) - open/close 2137 (time 1788470176.64 total 40.08 last 59.59) - open/close 2727 (time 1788470186.65 total 50.09 last 58.96) - open/close 3184 (time 1788470196.65 total 60.09 last 45.70) - open/close 3565 (time 1788470206.65 total 70.10 last 38.07) - open/close 3947 (time 1788470216.68 total 80.12 last 38.11) - open/close 4444 (time 1788470226.68 total 90.12 last 49.70) - open/close 4985 (time 1788470236.69 total 100.13 last 54.04) - open/close 5625 (time 1788470246.70 total 110.14 last 63.95) - open/close 6368 (time 1788470256.71 total 120.16 last 74.20) - open/close 6962 (time 1788470266.72 total 130.17 last 59.32) - open/close 7416 (time 1788470276.73 total 140.17 last 45.39) - open/close 7872 (time 1788470286.75 total 150.19 last 45.50) - open/close 8307 (time 1788470296.76 total 160.21 last 43.43) - open/close 8915 (time 1788470306.78 total 170.23 last 60.68) - open/close 9615 (time 1788470316.79 total 180.23 last 69.97) total: 10000 open/close in 187.23 seconds: 53.41 ops/second - unlinked 0 (time 1788470329 ; total 0 ; last 0) total: 10000 unlinks in 94 seconds: 106.382980 unlinks/second - open/close 861 (time 1788470438.56 total 10.01 last 86.05) - open/close 1645 (time 1788470448.56 total 20.01 last 78.40) - open/close 2433 (time 1788470458.57 total 30.01 last 78.77) - open/close 3221 (time 1788470468.57 total 40.01 last 78.79) - open/close 3976 (time 1788470478.58 total 50.02 last 75.44) - open/close 4651 (time 1788470488.58 total 60.02 last 67.48) - open/close 5337 (time 1788470498.59 total 70.03 last 68.52) - open/close 6095 (time 1788470508.59 total 80.03 last 75.80) - open/close 6782 (time 1788470518.62 total 90.06 last 68.53) - open/close 7459 (time 1788470528.62 total 100.06 last 67.70) - open/close 8212 (time 1788470538.62 total 110.07 last 75.26) - open/close 9002 (time 1788470548.62 total 120.07 last 79.00) - open/close 9840 (time 1788470558.63 total 130.07 last 83.76) total: 10000 open/close in 131.89 seconds: 75.82 ops/second - unlinked 0 (time 1788470564 ; total 0 ; last 0) total: 10000 unlinks in 72 seconds: 138.888885 unlinks/second - open/close 746 (time 1788470650.48 total 10.01 last 74.56) - open/close 1490 (time 1788470660.49 total 20.02 last 74.32) - open/close 2347 (time 1788470670.50 total 30.03 last 85.60) - open/close 3109 (time 1788470680.50 total 40.03 last 76.19) - open/close 3841 (time 1788470690.50 total 50.03 last 73.18) - open/close 4593 (time 1788470700.51 total 60.04 last 75.16) - open/close 5384 (time 1788470710.54 total 70.06 last 78.88) - open/close 6134 (time 1788470720.54 total 80.07 last 74.93) - open/close 6880 (time 1788470730.56 total 90.08 last 74.52) - open/close 7665 (time 1788470740.56 total 100.09 last 78.49) - open/close 8441 (time 1788470750.56 total 110.09 last 77.59) - open/close 9325 (time 1788470760.56 total 120.09 last 88.33) total: 10000 open/close in 128.16 seconds: 78.03 ops/second - unlinked 0 (time 1788470773 ; total 0 ; last 0) total: 10000 unlinks in 67 seconds: 149.253738 unlinks/second - open/close 777 (time 1788470855.30 total 10.01 last 77.62) - open/close 1538 (time 1788470865.31 total 20.02 last 76.01) - open/close 2290 (time 1788470875.32 total 30.03 last 75.18) - open/close 3037 (time 1788470885.33 total 40.04 last 74.61) - open/close 3675 (time 1788470895.34 total 50.05 last 63.74) - open/close 4416 (time 1788470905.34 total 60.05 last 74.09) - open/close 5195 (time 1788470915.35 total 70.06 last 77.80) - open/close 5970 (time 1788470925.36 total 80.07 last 77.44) - open/close 6694 (time 1788470935.37 total 90.07 last 72.36) - open/close 7274 (time 1788470945.38 total 100.09 last 57.90) - open/close 7923 (time 1788470955.40 total 110.11 last 64.79) - open/close 8482 (time 1788470965.40 total 120.11 last 55.89) - open/close 9256 (time 1788470975.41 total 130.11 last 77.36) total: 10000 open/close in 139.43 seconds: 71.72 ops/second - unlinked 0 (time 1788470989 ; total 0 ; last 0) total: 10000 unlinks in 72 seconds: 138.888885 unlinks/second - open/close 748 (time 1788471075.62 total 10.01 last 74.72) - open/close 1620 (time 1788471085.64 total 20.03 last 87.04) - open/close 2447 (time 1788471095.65 total 30.03 last 82.66) - open/close 3223 (time 1788471105.65 total 40.04 last 77.57) - open/close 4022 (time 1788471115.65 total 50.04 last 79.87) - open/close 4851 (time 1788471125.66 total 60.05 last 82.86) - open/close 5638 (time 1788471135.67 total 70.05 last 78.65) - open/close 6485 (time 1788471145.67 total 80.06 last 84.67) - open/close 7302 (time 1788471155.68 total 90.06 last 81.65) - open/close 8108 (time 1788471165.70 total 100.09 last 80.43) - open/close 8936 (time 1788471175.71 total 110.10 last 82.71) - open/close 9815 (time 1788471185.71 total 120.10 last 87.84) total: 10000 open/close in 122.12 seconds: 81.89 ops/second - unlinked 0 (time 1788471191 ; total 0 ; last 0) total: 10000 unlinks in 73 seconds: 136.986298 unlinks/second - open/close 795 (time 1788471278.24 total 10.01 last 79.44) - open/close 1542 (time 1788471288.25 total 20.01 last 74.66) - open/close 2351 (time 1788471298.25 total 30.02 last 80.88) - open/close 3189 (time 1788471308.26 total 40.02 last 83.74) - open/close 3987 (time 1788471318.26 total 50.03 last 79.77) - open/close 4811 (time 1788471328.26 total 60.03 last 82.37) - open/close 5689 (time 1788471338.26 total 70.03 last 87.80) - open/close 6482 (time 1788471348.29 total 80.05 last 79.12) - open/close 7248 (time 1788471358.29 total 90.06 last 76.57) - open/close 7964 (time 1788471368.29 total 100.06 last 71.58) - open/close 8828 (time 1788471378.30 total 110.07 last 86.34) - open/close 9583 (time 1788471388.30 total 120.07 last 75.48) total: 10000 open/close in 125.71 seconds: 79.55 ops/second - unlinked 0 (time 1788471398 ; total 0 ; last 0) total: 10000 unlinks in 70 seconds: 142.857147 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg627-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg627-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg627-server Permanent disk data: Device: /dev/mapper/ost1_flakey Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x42 (OST update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.206.127@tcp sys.timeout=20 device size = 4096MB formatting backing filesystem ldiskfs on /dev/mapper/ost1_flakey target name lustre-OST0000 kilobytes 200000 options -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F mkfs_cmd = mke2fs -j -b 4096 -L lustre-OST0000 -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/mapper/ost1_flakey 200000k start ost1 service on oleg627-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Started lustre-OST0000 oleg627-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg627-server: oleg627-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg627-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg627-server: oleg627-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg627-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg627-client.virtnet: -o user_xattr,flock 192.168.206.127@tcp:/lustre /mnt/lustre On OST0, 20430 used inodes rc=0 pdsh@oleg627-client: oleg627-server: ssh exited with exit code 22 umount lustre on /mnt/lustre..... Stopping client oleg627-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg627-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg627-server Permanent disk data: Device: /dev/mapper/ost1_flakey Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x62 (OST first_time update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.206.127@tcp sys.timeout=20 device size = 4096MB formatting backing filesystem ldiskfs on /dev/mapper/ost1_flakey target name lustre:OST0000 kilobytes 200000 options -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F mkfs_cmd = mke2fs -j -b 4096 -L lustre:OST0000 -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/mapper/ost1_flakey 200000k checking for existing Lustre data: found Read previous values: Device: /dev/mapper/ost1_flakey Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x62 (OST first_time update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.206.127@tcp sys.timeout=20 Permanent disk data: Device: /dev/mapper/ost1_flakey Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x42 (OST update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.206.127@tcp sys.timeout=20 start ost1 service on oleg627-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Started lustre-OST0000 oleg627-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg627-server: oleg627-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg627-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg627-server: oleg627-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg627-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg627-client.virtnet: -o user_xattr,flock 192.168.206.127@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg627-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg627-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg627-server stop mds service on oleg627-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg627-server stop mds service on oleg627-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg627-server unloading modules via unload_modules_local on: 'oleg627-server' oleg627-server: oleg627-server.virtnet: executing unload_modules_local oleg627-server: kmemleak disabled modules unloaded. pdsh@oleg627-client: oleg627-server: ssh exited with exit code 2 pdsh@oleg627-client: oleg627-client: ssh exited with exit code 2 pdsh@oleg627-client: oleg627-client: ssh exited with exit code 2 pdsh@oleg627-client: oleg627-server: ssh exited with exit code 2 PASS 69 (2472s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 17:40:26 (1788471626) oleg627-server: debugfs 1.47.3-wc2 (11-Nov-2025) Supported features: dir_prealloc imagic_inodes has_journal ext_attr resize_inode dir_index sparse_super2 fast_commit stable_inodes orphan_file filetype needs_recovery journal_dev meta_bg extent 64bit mmp flex_bg ea_inode dirdata metadata_csum_seed large_dir inline_data encrypt casefold sparse_super large_file huge_file uninit_bg dir_nlink extra_isize quota bigalloc metadata_csum read-only project shared_blocks verity orphan_present Stopping clients: oleg627-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg627-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg627-client: oleg627-server: ssh exited with exit code 2 oleg627-server: oleg627-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' -rwxr-xr-x. 1 green green 1921072 Sep 3 15:45 /home/green/git/lustre-release/lustre/utils/mount.lustre -rwxr-xr-x. 1 root root 12 Jan 31 2019 /usr/sbin/mount.lustre mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre loading modules on: 'oleg627-server' oleg627-server: oleg627-server.virtnet: executing load_modules_local oleg627-server: kmemleak disabled oleg627-server: Loading modules from /home/green/git/lustre-release/lustre oleg627-server: detected 4 online CPUs by sysfs oleg627-server: MODOPTS_LIBCFS= oleg627-server: Force libcfs to create 2 CPU partitions oleg627-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg627-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg627-server: mdt/mdt options: 'mdt_enable_flr_ec=1' oleg627-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format ost1: /dev/mapper/ost1_flakey pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 SKIP: conf-sanity test_111 /dev/mapper/mds1_flakey too small for 5000000kB MDS Stopping clients: oleg627-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg627-client.virtnet /mnt/lustre2 (opts:-f) oleg627-server: oleg627-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg627-server' oleg627-server: oleg627-server.virtnet: executing load_modules_local oleg627-server: kmemleak disabled oleg627-server: Loading modules from /home/green/git/lustre-release/lustre oleg627-server: detected 4 online CPUs by sysfs oleg627-server: MODOPTS_LIBCFS= oleg627-server: Force libcfs to create 2 CPU partitions oleg627-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg627-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg627-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg627-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg627-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg627-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg627-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg627-server: oleg627-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg627-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg627-server: oleg627-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg627-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg627-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg627-server stop mds service on oleg627-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg627-server stop mds service on oleg627-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg627-server SKIP 111 (221s) Stopping clients: oleg627-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg627-client.virtnet /mnt/lustre2 (opts:-f) oleg627-server: oleg627-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg627-server' oleg627-server: oleg627-server.virtnet: executing load_modules_local oleg627-server: kmemleak disabled oleg627-server: Loading modules from /home/green/git/lustre-release/lustre oleg627-server: detected 4 online CPUs by sysfs oleg627-server: MODOPTS_LIBCFS= oleg627-server: Force libcfs to create 2 CPU partitions oleg627-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey == conf-sanity test complete, duration 3239 sec ========== 17:45:06 (1788471906) === conf-sanity: start cleanup 17:45:08 (1788471908) === === conf-sanity: finish cleanup 17:45:12 (1788471912) ===