-----============= acceptance-small: conf-sanity ============----- Thu Sep 3 08:39:52 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg428-server: /home/green/git/lustre-release/lustre/tests/except/conf-sanity.ex - see CLIENT_VERSION >= v2_15_63-53-g1a6ef725c2 (34683394 >= 34553653) for LU-16938, go 81 - see CLIENT_VERSION >= v2_15_65-31-g74a5b9efaa (34683394 >= 34554143) for LU-18348, go 32b 32c - see CLIENT_VERSION >= v2_16_55-19-gf69f27cc35 (34683394 >= 34617107) for LU-17920, go 123ae 123ag - see CLIENT_VERSION >= v2_17_52-94-g86904ef943 (34683394 >= 34681950) for LU-19740, go 115 excepting tests: 32b 32c 32newtarball Stopping clients: oleg428-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg428-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg428-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg428-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg428-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg428-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg428-server oleg428-server: oleg428-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg428-server' oleg428-server: oleg428-server.virtnet: executing load_modules_local oleg428-server: kmemleak disabled oleg428-server: Loading modules from /home/green/git/lustre-release/lustre oleg428-server: detected 4 online CPUs by sysfs oleg428-server: MODOPTS_LIBCFS= oleg428-server: Force libcfs to create 2 CPU partitions oleg428-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg428-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg428-server: oleg428-server.virtnet: executing set_default_debug -1 all pdsh@oleg428-client: oleg428-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg428-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg428-server: oleg428-server.virtnet: executing set_default_debug -1 all pdsh@oleg428-client: oleg428-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg428-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg428-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg428-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg428-server: oleg428-server.virtnet: executing set_default_debug -1 all pdsh@oleg428-client: oleg428-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg428-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg428-server: oleg428-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg428-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg428-server: oleg428-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg428-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg428-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg428-server stop mds service on oleg428-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg428-server stop mds service on oleg428-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg428-server == conf-sanity test 45: long unlink handling in ptlrpcd == 08:43:52 (1788439432) start mds service on oleg428-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg428-server: oleg428-server.virtnet: executing set_default_debug -1 all pdsh@oleg428-client: oleg428-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg428-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg428-server: oleg428-server.virtnet: executing set_default_debug -1 all pdsh@oleg428-client: oleg428-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg428-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg428-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg428-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg428-server: oleg428-server.virtnet: executing set_default_debug -1 all pdsh@oleg428-client: oleg428-server: ssh exited with exit code 1 Started lustre-OST0000 oleg428-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg428-client.virtnet: -o user_xattr,flock 192.168.204.128@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg428-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg428-server stop mds service on oleg428-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg428-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg428-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg428-server: oleg428-server.virtnet: executing set_default_debug -1 all pdsh@oleg428-client: oleg428-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg428-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg428-server: oleg428-server.virtnet: executing set_default_debug -1 all pdsh@oleg428-client: oleg428-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg428-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg428-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg428-client.virtnet: -o user_xattr,flock 192.168.204.128@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg428-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg428-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg428-server stop mds service on oleg428-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg428-server stop mds service on oleg428-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg428-server unloading modules via unload_modules_local on: 'oleg428-server' oleg428-server: oleg428-server.virtnet: executing unload_modules_local oleg428-server: kmemleak disabled modules unloaded. pdsh@oleg428-client: oleg428-server: ssh exited with exit code 2 pdsh@oleg428-client: oleg428-client: ssh exited with exit code 2 pdsh@oleg428-client: oleg428-server: ssh exited with exit code 2 pdsh@oleg428-client: oleg428-client: ssh exited with exit code 2 PASS 45 (331s) == conf-sanity test 69: replace an OST with the same index ========================================================== 08:49:24 (1788439764) start mds service on oleg428-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' -rwxr-xr-x 1 green green 1921056 Sep 3 07:51 /home/green/git/lustre-release/lustre/utils/mount.lustre -rwxr-xr-x. 1 root root 12 Jan 31 2019 /usr/sbin/mount.lustre mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre loading modules on: 'oleg428-server' oleg428-server: oleg428-server.virtnet: executing load_modules_local oleg428-server: kmemleak disabled oleg428-server: Loading modules from /home/green/git/lustre-release/lustre oleg428-server: detected 4 online CPUs by sysfs oleg428-server: MODOPTS_LIBCFS= oleg428-server: Force libcfs to create 2 CPU partitions oleg428-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg428-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg428-server: mdt/mdt options: 'mdt_enable_flr_ec=1' oleg428-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg428-server: oleg428-server.virtnet: executing set_default_debug -1 all pdsh@oleg428-client: oleg428-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg428-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg428-server: oleg428-server.virtnet: executing set_default_debug -1 all pdsh@oleg428-client: oleg428-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg428-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg428-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg428-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg428-server: oleg428-server.virtnet: executing set_default_debug -1 all pdsh@oleg428-client: oleg428-server: ssh exited with exit code 1 Started lustre-OST0000 oleg428-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg428-client.virtnet: -o user_xattr,flock 192.168.204.128@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 49503 inodes available. Want 99872. rc=0 - open/close 624 (time 1788439901.87 total 10.00 last 62.40) - open/close 1278 (time 1788439911.88 total 20.01 last 65.37) - open/close 1913 (time 1788439921.90 total 30.02 last 63.40) - open/close 2552 (time 1788439931.90 total 40.02 last 63.88) - open/close 3161 (time 1788439941.90 total 50.03 last 60.87) - open/close 3821 (time 1788439951.91 total 60.04 last 65.93) - open/close 4396 (time 1788439961.93 total 70.05 last 57.42) - open/close 4904 (time 1788439971.95 total 80.07 last 50.71) - open/close 5526 (time 1788439981.95 total 90.08 last 62.15) - open/close 6288 (time 1788439991.98 total 100.10 last 76.03) - open/close 6960 (time 1788440001.99 total 110.11 last 67.14) - open/close 7617 (time 1788440011.99 total 120.12 last 65.64) - open/close 8284 (time 1788440022.01 total 130.13 last 66.61) - open/close 8907 (time 1788440032.02 total 140.14 last 62.25) - open/close 9596 (time 1788440042.03 total 150.16 last 68.77) total: 10000 open/close in 155.94 seconds: 64.13 ops/second - unlinked 0 (time 1788440053 ; total 0 ; last 0) total: 10000 unlinks in 83 seconds: 120.481926 unlinks/second - open/close 600 (time 1788440152.58 total 10.03 last 59.82) - open/close 1266 (time 1788440162.59 total 20.03 last 66.58) - open/close 1955 (time 1788440172.59 total 30.04 last 68.88) - open/close 2637 (time 1788440182.60 total 40.04 last 68.16) - open/close 3405 (time 1788440192.60 total 50.04 last 76.78) - open/close 4083 (time 1788440202.60 total 60.04 last 67.79) - open/close 4738 (time 1788440212.61 total 70.06 last 65.43) - open/close 5449 (time 1788440222.62 total 80.06 last 71.05) - open/close 6129 (time 1788440232.63 total 90.07 last 67.93) - open/close 6804 (time 1788440242.64 total 100.08 last 67.45) - open/close 7380 (time 1788440252.67 total 110.11 last 57.42) - open/close 8028 (time 1788440262.67 total 120.12 last 64.77) - open/close 8620 (time 1788440272.69 total 130.13 last 59.12) - open/close 9170 (time 1788440282.70 total 140.14 last 54.94) - open/close 9931 (time 1788440292.70 total 150.14 last 76.08) total: 10000 open/close in 151.06 seconds: 66.20 ops/second - unlinked 0 (time 1788440298 ; total 0 ; last 0) total: 10000 unlinks in 85 seconds: 117.647057 unlinks/second - open/close 511 (time 1788440400.03 total 10.01 last 51.04) - open/close 1134 (time 1788440410.04 total 20.01 last 62.28) - open/close 1692 (time 1788440420.04 total 30.02 last 55.77) - open/close 2259 (time 1788440430.05 total 40.03 last 56.64) - open/close 2910 (time 1788440440.06 total 50.04 last 65.04) - open/close 3535 (time 1788440450.07 total 60.05 last 62.45) - open/close 4189 (time 1788440460.07 total 70.05 last 65.40) - open/close 4804 (time 1788440470.07 total 80.05 last 61.48) - open/close 5510 (time 1788440480.08 total 90.06 last 70.56) - open/close 6178 (time 1788440490.10 total 100.08 last 66.67) - open/close 6806 (time 1788440500.10 total 110.08 last 62.79) - open/close 7539 (time 1788440510.12 total 120.09 last 73.17) - open/close 8101 (time 1788440520.12 total 130.10 last 56.18) - open/close 8753 (time 1788440530.14 total 140.11 last 65.10) - open/close 9397 (time 1788440540.14 total 150.12 last 64.35) total: 10000 open/close in 158.98 seconds: 62.90 ops/second - unlinked 0 (time 1788440554 ; total 0 ; last 0) total: 10000 unlinks in 90 seconds: 111.111115 unlinks/second - open/close 635 (time 1788440660.20 total 10.00 last 63.48) - open/close 1351 (time 1788440670.20 total 20.01 last 71.58) - open/close 2040 (time 1788440680.21 total 30.01 last 68.85) - open/close 2751 (time 1788440690.25 total 40.05 last 70.80) - open/close 3450 (time 1788440700.25 total 50.06 last 69.89) - open/close 4161 (time 1788440710.26 total 60.06 last 71.04) - open/close 4806 (time 1788440720.27 total 70.08 last 64.42) - open/close 5400 (time 1788440730.29 total 80.09 last 59.29) - open/close 5954 (time 1788440740.30 total 90.10 last 55.35) - open/close 6556 (time 1788440750.32 total 100.13 last 60.05) - open/close 7062 (time 1788440760.33 total 110.14 last 50.56) - open/close 7697 (time 1788440770.35 total 120.16 last 63.36) - open/close 8320 (time 1788440780.36 total 130.17 last 62.24) - open/close 9035 (time 1788440790.37 total 140.17 last 71.47) - open/close 9802 (time 1788440800.38 total 150.18 last 76.61) total: 10000 open/close in 153.05 seconds: 65.34 ops/second - unlinked 0 (time 1788440809 ; total 0 ; last 0) total: 10000 unlinks in 84 seconds: 119.047623 unlinks/second - open/close 677 (time 1788440908.77 total 10.00 last 67.67) - open/close 1378 (time 1788440918.77 total 20.01 last 70.08) - open/close 2060 (time 1788440928.77 total 30.01 last 68.19) - open/close 2758 (time 1788440938.78 total 40.01 last 69.76) - open/close 3430 (time 1788440948.79 total 50.02 last 67.15) - open/close 4103 (time 1788440958.81 total 60.04 last 67.17) - open/close 4770 (time 1788440968.81 total 70.05 last 66.65) - open/close 5356 (time 1788440978.81 total 80.05 last 58.59) - open/close 5902 (time 1788440988.81 total 90.05 last 54.60) - open/close 6438 (time 1788440998.82 total 100.05 last 53.58) - open/close 7128 (time 1788441008.83 total 110.06 last 68.93) - open/close 7854 (time 1788441018.84 total 120.07 last 72.54) - open/close 8506 (time 1788441028.84 total 130.07 last 65.20) - open/close 9145 (time 1788441038.85 total 140.09 last 63.81) - open/close 9691 (time 1788441048.85 total 150.09 last 54.58) total: 10000 open/close in 154.84 seconds: 64.58 ops/second - unlinked 0 (time 1788441057 ; total 0 ; last 0) total: 10000 unlinks in 80 seconds: 125.000000 unlinks/second - open/close 790 (time 1788441152.51 total 10.02 last 78.84) - open/close 1405 (time 1788441162.51 total 20.03 last 61.46) - open/close 2093 (time 1788441172.52 total 30.03 last 68.75) - open/close 2814 (time 1788441182.53 total 40.04 last 72.04) - open/close 3498 (time 1788441192.56 total 50.08 last 68.16) - open/close 4212 (time 1788441202.57 total 60.08 last 71.36) - open/close 4796 (time 1788441212.58 total 70.09 last 58.33) - open/close 5320 (time 1788441222.60 total 80.11 last 52.32) - open/close 5931 (time 1788441232.60 total 90.11 last 61.09) - open/close 6692 (time 1788441242.60 total 100.11 last 76.09) - open/close 7388 (time 1788441252.61 total 110.12 last 69.52) - open/close 7868 (time 1788441262.64 total 120.15 last 47.85) - open/close 8295 (time 1788441272.64 total 130.16 last 42.69) - open/close 8837 (time 1788441282.66 total 140.18 last 54.09) - open/close 9386 (time 1788441292.67 total 150.18 last 54.85) - open/close 9964 (time 1788441302.67 total 160.19 last 57.79) total: 10000 open/close in 160.76 seconds: 62.21 ops/second - unlinked 0 (time 1788441308 ; total 0 ; last 0) total: 10000 unlinks in 84 seconds: 119.047623 unlinks/second - open/close 657 (time 1788441407.40 total 10.00 last 65.67) - open/close 1242 (time 1788441417.42 total 20.02 last 58.41) - open/close 1857 (time 1788441427.43 total 30.03 last 61.46) - open/close 2540 (time 1788441437.43 total 40.03 last 68.28) - open/close 3222 (time 1788441447.45 total 50.05 last 68.08) - open/close 3874 (time 1788441457.45 total 60.05 last 65.20) - open/close 4623 (time 1788441467.45 total 70.05 last 74.90) - open/close 5340 (time 1788441477.45 total 80.05 last 71.68) - open/close 6043 (time 1788441487.46 total 90.06 last 70.20) - open/close 6683 (time 1788441497.47 total 100.07 last 63.97) - open/close 7347 (time 1788441507.49 total 110.09 last 66.27) - open/close 8007 (time 1788441517.50 total 120.10 last 65.92) - open/close 8683 (time 1788441527.50 total 130.10 last 67.58) - open/close 9314 (time 1788441537.52 total 140.12 last 63.00) total: 10000 open/close in 149.22 seconds: 67.02 ops/second - unlinked 0 (time 1788441551 ; total 0 ; last 0) total: 10000 unlinks in 83 seconds: 120.481926 unlinks/second - open/close 770 (time 1788441650.10 total 10.00 last 76.98) - open/close 1418 (time 1788441660.10 total 20.01 last 64.75) - open/close 2082 (time 1788441670.11 total 30.02 last 66.34) - open/close 2808 (time 1788441680.12 total 40.02 last 72.59) - open/close 3481 (time 1788441690.12 total 50.03 last 67.27) - open/close 4161 (time 1788441700.13 total 60.04 last 67.90) - open/close 4867 (time 1788441710.14 total 70.05 last 70.53) - open/close 5412 (time 1788441720.16 total 80.07 last 54.41) - open/close 6026 (time 1788441730.17 total 90.08 last 61.32) - open/close 6706 (time 1788441740.18 total 100.09 last 67.94) - open/close 7321 (time 1788441750.18 total 110.09 last 61.49) - open/close 8082 (time 1788441760.21 total 120.12 last 75.90) - open/close 8641 (time 1788441770.22 total 130.12 last 55.86) - open/close 9266 (time 1788441780.23 total 140.13 last 62.44) - open/close 9935 (time 1788441790.24 total 150.15 last 66.80) total: 10000 open/close in 151.09 seconds: 66.19 ops/second - unlinked 0 (time 1788441795 ; total 0 ; last 0) total: 10000 unlinks in 80 seconds: 125.000000 unlinks/second - open/close 683 (time 1788441889.99 total 10.00 last 68.29) - open/close 1383 (time 1788441900.00 total 20.01 last 69.92) - open/close 2120 (time 1788441910.01 total 30.02 last 73.65) - open/close 2757 (time 1788441920.01 total 40.02 last 63.68) - open/close 3379 (time 1788441930.01 total 50.03 last 62.18) - open/close 4037 (time 1788441940.02 total 60.03 last 65.76) - open/close 4770 (time 1788441950.03 total 70.04 last 73.22) - open/close 5443 (time 1788441960.04 total 80.05 last 67.27) - open/close 5995 (time 1788441970.04 total 90.05 last 55.20) - open/close 6669 (time 1788441980.04 total 100.06 last 67.36) - open/close 7267 (time 1788441990.06 total 110.07 last 59.71) - open/close 7935 (time 1788442000.06 total 120.07 last 66.78) - open/close 8633 (time 1788442010.06 total 130.07 last 69.80) - open/close 9292 (time 1788442020.07 total 140.08 last 65.85) total: 10000 open/close in 149.92 seconds: 66.70 ops/second - unlinked 0 (time 1788442034 ; total 0 ; last 0) total: 10000 unlinks in 83 seconds: 120.481926 unlinks/second - open/close 698 (time 1788442133.96 total 10.01 last 69.75) - open/close 1369 (time 1788442143.96 total 20.01 last 67.08) - open/close 2042 (time 1788442153.97 total 30.02 last 67.23) - open/close 2760 (time 1788442163.98 total 40.03 last 71.75) - open/close 3424 (time 1788442173.99 total 50.03 last 66.35) - open/close 4113 (time 1788442183.99 total 60.04 last 68.87) - open/close 4810 (time 1788442194.00 total 70.05 last 69.64) - open/close 5503 (time 1788442204.01 total 80.05 last 69.27) - open/close 6155 (time 1788442214.01 total 90.06 last 65.17) - open/close 6870 (time 1788442224.01 total 100.06 last 71.49) - open/close 7601 (time 1788442234.01 total 110.06 last 73.09) - open/close 8280 (time 1788442244.01 total 120.06 last 67.90) - open/close 8946 (time 1788442254.03 total 130.08 last 66.47) - open/close 9578 (time 1788442264.05 total 140.09 last 63.11) total: 10000 open/close in 146.17 seconds: 68.41 ops/second - unlinked 0 (time 1788442274 ; total 0 ; last 0) total: 10000 unlinks in 82 seconds: 121.951218 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg428-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg428-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg428-server Permanent disk data: Device: /dev/mapper/ost1_flakey Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x42 (OST update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.204.128@tcp sys.timeout=20 device size = 4096MB formatting backing filesystem ldiskfs on /dev/mapper/ost1_flakey target name lustre-OST0000 kilobytes 200000 options -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F mkfs_cmd = mke2fs -j -b 4096 -L lustre-OST0000 -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/mapper/ost1_flakey 200000k start ost1 service on oleg428-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg428-server: oleg428-server.virtnet: executing set_default_debug -1 all pdsh@oleg428-client: oleg428-server: ssh exited with exit code 1 Started lustre-OST0000 oleg428-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg428-server: oleg428-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg428-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg428-server: oleg428-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg428-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg428-client.virtnet: -o user_xattr,flock 192.168.204.128@tcp:/lustre /mnt/lustre On OST0, 20430 used inodes rc=0 pdsh@oleg428-client: oleg428-server: ssh exited with exit code 22 umount lustre on /mnt/lustre..... Stopping client oleg428-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg428-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg428-server Permanent disk data: Device: /dev/mapper/ost1_flakey Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x62 (OST first_time update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.204.128@tcp sys.timeout=20 device size = 4096MB formatting backing filesystem ldiskfs on /dev/mapper/ost1_flakey target name lustre:OST0000 kilobytes 200000 options -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F mkfs_cmd = mke2fs -j -b 4096 -L lustre:OST0000 -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/mapper/ost1_flakey 200000k checking for existing Lustre data: found Read previous values: Device: /dev/mapper/ost1_flakey Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x62 (OST first_time update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.204.128@tcp sys.timeout=20 Permanent disk data: Device: /dev/mapper/ost1_flakey Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x42 (OST update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.204.128@tcp sys.timeout=20 start ost1 service on oleg428-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg428-server: oleg428-server.virtnet: executing set_default_debug -1 all pdsh@oleg428-client: oleg428-server: ssh exited with exit code 1 Started lustre-OST0000 oleg428-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg428-server: oleg428-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg428-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg428-server: oleg428-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg428-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg428-client.virtnet: -o user_xattr,flock 192.168.204.128@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg428-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg428-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg428-server stop mds service on oleg428-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg428-server stop mds service on oleg428-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg428-server unloading modules via unload_modules_local on: 'oleg428-server' oleg428-server: oleg428-server.virtnet: executing unload_modules_local oleg428-server: kmemleak disabled modules unloaded. pdsh@oleg428-client: oleg428-server: ssh exited with exit code 2 pdsh@oleg428-client: oleg428-client: ssh exited with exit code 2 pdsh@oleg428-client: oleg428-server: ssh exited with exit code 2 pdsh@oleg428-client: oleg428-client: ssh exited with exit code 2 PASS 69 (2765s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 09:35:31 (1788442531) oleg428-server: debugfs 1.47.3-wc2 (11-Nov-2025) Supported features: dir_prealloc imagic_inodes has_journal ext_attr resize_inode dir_index sparse_super2 fast_commit stable_inodes orphan_file filetype needs_recovery journal_dev meta_bg extent 64bit mmp flex_bg ea_inode dirdata metadata_csum_seed large_dir inline_data encrypt casefold sparse_super large_file huge_file uninit_bg dir_nlink extra_isize quota bigalloc metadata_csum read-only project shared_blocks verity orphan_present Stopping clients: oleg428-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg428-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg428-client: oleg428-server: ssh exited with exit code 2 oleg428-server: oleg428-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' -rwxr-xr-x 1 green green 1921056 Sep 3 07:51 /home/green/git/lustre-release/lustre/utils/mount.lustre -rwxr-xr-x. 1 root root 12 Jan 31 2019 /usr/sbin/mount.lustre mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre loading modules on: 'oleg428-server' oleg428-server: oleg428-server.virtnet: executing load_modules_local oleg428-server: kmemleak disabled oleg428-server: Loading modules from /home/green/git/lustre-release/lustre oleg428-server: detected 4 online CPUs by sysfs oleg428-server: MODOPTS_LIBCFS= oleg428-server: Force libcfs to create 2 CPU partitions oleg428-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg428-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg428-server: mdt/mdt options: 'mdt_enable_flr_ec=1' oleg428-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format ost1: /dev/mapper/ost1_flakey pdsh@oleg428-client: oleg428-server: ssh exited with exit code 1 SKIP: conf-sanity test_111 /dev/mapper/mds1_flakey too small for 5000000kB MDS Stopping clients: oleg428-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg428-client.virtnet /mnt/lustre2 (opts:-f) oleg428-server: oleg428-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg428-server' oleg428-server: oleg428-server.virtnet: executing load_modules_local oleg428-server: kmemleak disabled oleg428-server: Loading modules from /home/green/git/lustre-release/lustre oleg428-server: detected 4 online CPUs by sysfs oleg428-server: MODOPTS_LIBCFS= oleg428-server: Force libcfs to create 2 CPU partitions oleg428-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg428-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg428-server: oleg428-server.virtnet: executing set_default_debug -1 all pdsh@oleg428-client: oleg428-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg428-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg428-server: oleg428-server.virtnet: executing set_default_debug -1 all pdsh@oleg428-client: oleg428-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg428-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg428-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg428-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg428-server: oleg428-server.virtnet: executing set_default_debug -1 all pdsh@oleg428-client: oleg428-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg428-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg428-server: oleg428-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg428-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg428-server: oleg428-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg428-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg428-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg428-server stop mds service on oleg428-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg428-server stop mds service on oleg428-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg428-server SKIP 111 (246s) Stopping clients: oleg428-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg428-client.virtnet /mnt/lustre2 (opts:-f) oleg428-server: oleg428-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg428-server' oleg428-server: oleg428-server.virtnet: executing load_modules_local oleg428-server: kmemleak disabled oleg428-server: Loading modules from /home/green/git/lustre-release/lustre oleg428-server: detected 4 online CPUs by sysfs oleg428-server: MODOPTS_LIBCFS= oleg428-server: Force libcfs to create 2 CPU partitions oleg428-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey == conf-sanity test complete, duration 3655 sec ========== 09:40:49 (1788442849) === conf-sanity: start cleanup 09:40:51 (1788442851) === === conf-sanity: finish cleanup 09:40:54 (1788442854) ===