-----============= acceptance-small: conf-sanity ============----- Thu Jul 30 03:51:31 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg222-server: /home/green/git/lustre-release/lustre/tests/except/conf-sanity.ex - see CLIENT_VERSION > v2_15_63-53-g1a6ef725c2 (34682882 > 34553653) for LU-16938, go 81 excepting tests: 32b 32c 32newtarball Stopping clients: oleg222-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg222-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg222-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg222-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg222-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg222-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg222-server oleg222-server: oleg222-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg222-server' oleg222-server: oleg222-server.virtnet: executing load_modules_local oleg222-server: Loading modules from /home/green/git/lustre-release/lustre oleg222-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg222-server: detected 4 online CPUs by sysfs oleg222-server: MODOPTS_LIBCFS= oleg222-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg222-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg222-server: oleg222-server.virtnet: executing set_default_debug -1 all pdsh@oleg222-client: oleg222-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg222-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg222-server: oleg222-server.virtnet: executing set_default_debug -1 all pdsh@oleg222-client: oleg222-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg222-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg222-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg222-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg222-server: oleg222-server.virtnet: executing set_default_debug -1 all pdsh@oleg222-client: oleg222-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg222-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg222-server: oleg222-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg222-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg222-server: oleg222-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg222-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg222-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg222-server stop mds service on oleg222-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg222-server stop mds service on oleg222-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg222-server == conf-sanity test 45: long unlink handling in ptlrpcd == 03:56:26 (1785398186) start mds service on oleg222-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg222-server: oleg222-server.virtnet: executing set_default_debug -1 all pdsh@oleg222-client: oleg222-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg222-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg222-server: oleg222-server.virtnet: executing set_default_debug -1 all pdsh@oleg222-client: oleg222-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg222-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg222-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg222-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg222-server: oleg222-server.virtnet: executing set_default_debug -1 all pdsh@oleg222-client: oleg222-server: ssh exited with exit code 1 Started lustre-OST0000 oleg222-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg222-client.virtnet: -o user_xattr,flock 192.168.202.122@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg222-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg222-server stop mds service on oleg222-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg222-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg222-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg222-server: oleg222-server.virtnet: executing set_default_debug -1 all pdsh@oleg222-client: oleg222-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg222-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg222-server: oleg222-server.virtnet: executing set_default_debug -1 all pdsh@oleg222-client: oleg222-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg222-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg222-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg222-client.virtnet: -o user_xattr,flock 192.168.202.122@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg222-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg222-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg222-server stop mds service on oleg222-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg222-server stop mds service on oleg222-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg222-server unloading modules via unload_modules_local on: 'oleg222-server' oleg222-server: oleg222-server.virtnet: executing unload_modules_local oleg222-server: modules unloaded. pdsh@oleg222-client: oleg222-client: ssh exited with exit code 2 pdsh@oleg222-client: oleg222-server: ssh exited with exit code 2 pdsh@oleg222-client: oleg222-server: ssh exited with exit code 2 pdsh@oleg222-client: oleg222-client: ssh exited with exit code 2 PASS 45 (353s) == conf-sanity test 69: replace an OST with the same index ========================================================== 04:02:20 (1785398540) start mds service on oleg222-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg222-server' oleg222-server: oleg222-server.virtnet: executing load_modules_local oleg222-server: Loading modules from /home/green/git/lustre-release/lustre oleg222-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg222-server: detected 4 online CPUs by sysfs oleg222-server: MODOPTS_LIBCFS= oleg222-server: Force libcfs to create 2 CPU partitions oleg222-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg222-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg222-server: mdt/mdt options: 'mdt_enable_flr_ec=1' Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg222-server: oleg222-server.virtnet: executing set_default_debug -1 all pdsh@oleg222-client: oleg222-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg222-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg222-server: oleg222-server.virtnet: executing set_default_debug -1 all pdsh@oleg222-client: oleg222-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg222-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg222-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg222-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg222-server: oleg222-server.virtnet: executing set_default_debug -1 all pdsh@oleg222-client: oleg222-server: ssh exited with exit code 1 Started lustre-OST0000 oleg222-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg222-client.virtnet: -o user_xattr,flock 192.168.202.122@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 49503 inodes available. Want 99872. rc=0 - open/close 468 (time 1785398695.77 total 10.03 last 46.65) - open/close 1085 (time 1785398705.80 total 20.06 last 61.54) - open/close 1542 (time 1785398715.80 total 30.07 last 45.66) - open/close 2057 (time 1785398725.82 total 40.09 last 51.40) - open/close 2652 (time 1785398735.83 total 50.09 last 59.46) - open/close 3321 (time 1785398745.83 total 60.09 last 66.90) - open/close 3892 (time 1785398755.83 total 70.09 last 57.09) - open/close 4310 (time 1785398765.85 total 80.11 last 41.74) - open/close 4716 (time 1785398775.86 total 90.12 last 40.54) - open/close 5245 (time 1785398785.90 total 100.16 last 52.70) - open/close 5775 (time 1785398795.90 total 110.16 last 52.98) - open/close 6344 (time 1785398805.92 total 120.18 last 56.82) - open/close 6910 (time 1785398815.94 total 130.20 last 56.46) - open/close 7644 (time 1785398825.96 total 140.22 last 73.30) - open/close 8153 (time 1785398835.96 total 150.22 last 50.89) - open/close 8652 (time 1785398845.97 total 160.23 last 49.83) - open/close 9089 (time 1785398855.98 total 170.24 last 43.65) - open/close 9613 (time 1785398865.99 total 180.25 last 52.38) total: 10000 open/close in 187.94 seconds: 53.21 ops/second - unlinked 0 (time 1785398879 ; total 0 ; last 0) total: 10000 unlinks in 98 seconds: 102.040817 unlinks/second - open/close 506 (time 1785398994.22 total 10.01 last 50.53) - open/close 1224 (time 1785399004.23 total 20.02 last 71.77) - open/close 1870 (time 1785399014.24 total 30.03 last 64.52) - open/close 2227 (time 1785399024.25 total 40.04 last 35.66) - open/close 2620 (time 1785399034.28 total 50.08 last 39.16) - open/close 3084 (time 1785399044.30 total 60.09 last 46.33) - open/close 3650 (time 1785399054.31 total 70.11 last 56.51) - open/close 4083 (time 1785399064.34 total 80.14 last 43.17) - open/close 4651 (time 1785399074.36 total 90.16 last 56.69) - open/close 5028 (time 1785399084.38 total 100.17 last 37.65) - open/close 5486 (time 1785399094.38 total 110.17 last 45.78) - open/close 5932 (time 1785399104.39 total 120.18 last 44.57) - open/close 6356 (time 1785399114.40 total 130.19 last 42.34) - open/close 6869 (time 1785399124.43 total 140.22 last 51.15) - open/close 7347 (time 1785399134.44 total 150.23 last 47.76) - open/close 7869 (time 1785399144.46 total 160.26 last 52.07) - open/close 8301 (time 1785399154.48 total 170.27 last 43.13) - open/close 8771 (time 1785399164.49 total 180.28 last 46.97) - open/close 9262 (time 1785399174.49 total 190.29 last 49.07) - open/close 9614 (time 1785399184.50 total 200.29 last 35.18) total: 10000 open/close in 210.19 seconds: 47.58 ops/second - unlinked 0 (time 1785399202 ; total 0 ; last 0) total: 10000 unlinks in 97 seconds: 103.092781 unlinks/second - open/close 469 (time 1785399315.04 total 10.00 last 46.88) - open/close 884 (time 1785399325.06 total 20.03 last 41.41) - open/close 1635 (time 1785399335.08 total 30.04 last 75.01) - open/close 2160 (time 1785399345.09 total 40.06 last 52.41) - open/close 2783 (time 1785399355.11 total 50.07 last 62.19) - open/close 3285 (time 1785399365.13 total 60.10 last 50.08) - open/close 3669 (time 1785399375.15 total 70.11 last 38.34) - open/close 4104 (time 1785399385.16 total 80.12 last 43.46) - open/close 4507 (time 1785399395.18 total 90.14 last 40.24) - open/close 5080 (time 1785399405.18 total 100.15 last 57.26) - open/close 5502 (time 1785399415.19 total 110.15 last 42.16) - open/close 5999 (time 1785399425.22 total 120.18 last 49.56) - open/close 6478 (time 1785399435.24 total 130.20 last 47.80) - open/close 6960 (time 1785399445.24 total 140.21 last 48.19) - open/close 7410 (time 1785399455.27 total 150.23 last 44.89) - open/close 7865 (time 1785399465.28 total 160.24 last 45.46) - open/close 8314 (time 1785399475.30 total 170.27 last 44.78) - open/close 8772 (time 1785399485.30 total 180.27 last 45.80) - open/close 9191 (time 1785399495.31 total 190.27 last 41.87) - open/close 9649 (time 1785399505.32 total 200.28 last 45.77) - open/close 9985 (time 1785399515.34 total 210.30 last 33.53) total: 10000 open/close in 210.65 seconds: 47.47 ops/second - unlinked 0 (time 1785399523 ; total 0 ; last 0) total: 10000 unlinks in 118 seconds: 84.745766 unlinks/second - open/close 588 (time 1785399657.09 total 10.01 last 58.76) - open/close 1162 (time 1785399667.13 total 20.04 last 57.21) - open/close 1720 (time 1785399677.14 total 30.06 last 55.71) - open/close 2143 (time 1785399687.17 total 40.08 last 42.20) - open/close 2615 (time 1785399697.17 total 50.08 last 47.19) - open/close 3094 (time 1785399707.17 total 60.09 last 47.88) - open/close 3768 (time 1785399717.17 total 70.09 last 67.40) - open/close 4211 (time 1785399727.21 total 80.12 last 44.14) - open/close 4689 (time 1785399737.21 total 90.12 last 47.79) - open/close 5471 (time 1785399747.21 total 100.12 last 78.18) - open/close 6178 (time 1785399757.22 total 110.13 last 70.63) - open/close 6647 (time 1785399767.23 total 120.14 last 46.85) - open/close 7074 (time 1785399777.24 total 130.15 last 42.66) - open/close 7532 (time 1785399787.24 total 140.15 last 45.80) - open/close 8232 (time 1785399797.25 total 150.16 last 69.96) - open/close 8772 (time 1785399807.25 total 160.16 last 53.98) - open/close 9385 (time 1785399817.26 total 170.17 last 61.26) total: 10000 open/close in 176.49 seconds: 56.66 ops/second - unlinked 0 (time 1785399829 ; total 0 ; last 0) total: 10000 unlinks in 97 seconds: 103.092781 unlinks/second - open/close 487 (time 1785399943.04 total 10.01 last 48.63) - open/close 1079 (time 1785399953.09 total 20.06 last 58.93) - open/close 1679 (time 1785399963.09 total 30.07 last 59.97) - open/close 2350 (time 1785399973.10 total 40.07 last 67.04) - open/close 2874 (time 1785399983.11 total 50.09 last 52.33) - open/close 3560 (time 1785399993.12 total 60.09 last 68.57) - open/close 4087 (time 1785400003.12 total 70.09 last 52.69) - open/close 4878 (time 1785400013.13 total 80.10 last 79.04) - open/close 5375 (time 1785400023.14 total 90.12 last 49.62) - open/close 5939 (time 1785400033.15 total 100.13 last 56.35) - open/close 6286 (time 1785400043.17 total 110.15 last 34.63) - open/close 6734 (time 1785400053.26 total 120.23 last 44.42) - open/close 7160 (time 1785400063.30 total 130.28 last 42.42) - open/close 7589 (time 1785400073.31 total 140.28 last 42.87) - open/close 8117 (time 1785400083.34 total 150.31 last 52.65) - open/close 8708 (time 1785400093.35 total 160.33 last 59.01) - open/close 9342 (time 1785400103.37 total 170.35 last 63.27) - open/close 9867 (time 1785400113.37 total 180.35 last 52.50) total: 10000 open/close in 183.29 seconds: 54.56 ops/second - unlinked 0 (time 1785400123 ; total 0 ; last 0) total: 10000 unlinks in 110 seconds: 90.909088 unlinks/second - open/close 680 (time 1785400253.10 total 10.00 last 67.97) - open/close 1203 (time 1785400263.10 total 20.01 last 52.29) - open/close 1550 (time 1785400273.11 total 30.02 last 34.65) - open/close 1910 (time 1785400283.12 total 40.03 last 35.97) - open/close 2288 (time 1785400293.14 total 50.05 last 37.72) - open/close 2712 (time 1785400303.14 total 60.05 last 42.39) - open/close 3215 (time 1785400313.15 total 70.06 last 50.25) - open/close 3702 (time 1785400323.17 total 80.08 last 48.60) - open/close 4218 (time 1785400333.22 total 90.13 last 51.37) - open/close 4690 (time 1785400343.25 total 100.16 last 47.07) - open/close 5092 (time 1785400353.26 total 110.17 last 40.15) - open/close 5485 (time 1785400363.28 total 120.19 last 39.22) - open/close 6018 (time 1785400373.30 total 130.21 last 53.18) - open/close 6468 (time 1785400383.32 total 140.23 last 44.92) - open/close 6890 (time 1785400393.34 total 150.25 last 42.12) - open/close 7344 (time 1785400403.34 total 160.25 last 45.39) - open/close 7716 (time 1785400413.35 total 170.26 last 37.19) - open/close 8170 (time 1785400423.35 total 180.26 last 45.37) - open/close 8927 (time 1785400433.36 total 190.27 last 75.66) - open/close 9570 (time 1785400443.37 total 200.28 last 64.23) total: 10000 open/close in 206.74 seconds: 48.37 ops/second - unlinked 0 (time 1785400456 ; total 0 ; last 0) total: 10000 unlinks in 101 seconds: 99.009903 unlinks/second - open/close 552 (time 1785400573.48 total 10.01 last 55.17) - open/close 945 (time 1785400583.48 total 20.01 last 39.29) - open/close 1571 (time 1785400593.51 total 30.04 last 62.44) - open/close 2010 (time 1785400603.51 total 40.04 last 43.90) - open/close 2504 (time 1785400613.53 total 50.06 last 49.27) - open/close 3002 (time 1785400623.54 total 60.07 last 49.78) - open/close 3641 (time 1785400633.54 total 70.07 last 63.88) - open/close 4181 (time 1785400643.56 total 80.09 last 53.91) - open/close 4769 (time 1785400653.58 total 90.11 last 58.69) - open/close 5152 (time 1785400663.59 total 100.12 last 38.26) - open/close 5816 (time 1785400673.60 total 110.13 last 66.32) - open/close 6541 (time 1785400683.60 total 120.13 last 72.47) - open/close 7037 (time 1785400693.61 total 130.14 last 49.58) - open/close 7792 (time 1785400703.62 total 140.15 last 75.42) - open/close 8151 (time 1785400713.62 total 150.15 last 35.88) - open/close 8658 (time 1785400723.64 total 160.17 last 50.62) - open/close 9259 (time 1785400733.64 total 170.17 last 60.08) total: 10000 open/close in 178.10 seconds: 56.15 ops/second - unlinked 0 (time 1785400747 ; total 0 ; last 0) total: 10000 unlinks in 107 seconds: 93.457947 unlinks/second - open/close 677 (time 1785400870.56 total 10.00 last 67.69) - open/close 1208 (time 1785400880.56 total 20.01 last 53.06) - open/close 1895 (time 1785400890.60 total 30.05 last 68.44) - open/close 2351 (time 1785400900.61 total 40.05 last 45.56) - open/close 2827 (time 1785400910.63 total 50.07 last 47.52) - open/close 3285 (time 1785400920.63 total 60.07 last 45.79) - open/close 3755 (time 1785400930.63 total 70.08 last 46.99) - open/close 4287 (time 1785400940.64 total 80.08 last 53.17) - open/close 4652 (time 1785400950.73 total 90.18 last 36.15) - open/close 4988 (time 1785400960.75 total 100.19 last 33.56) - open/close 5347 (time 1785400970.76 total 110.20 last 35.86) - open/close 5945 (time 1785400980.76 total 120.20 last 59.80) - open/close 6465 (time 1785400990.78 total 130.22 last 51.90) - open/close 7002 (time 1785401000.80 total 140.25 last 53.57) - open/close 7361 (time 1785401010.81 total 150.25 last 35.88) - open/close 7661 (time 1785401020.83 total 160.27 last 29.93) - open/close 8041 (time 1785401030.83 total 170.28 last 37.99) - open/close 8802 (time 1785401040.84 total 180.28 last 76.06) - open/close 9475 (time 1785401050.85 total 190.30 last 67.20) total: 10000 open/close in 200.07 seconds: 49.98 ops/second - unlinked 0 (time 1785401067 ; total 0 ; last 0) total: 10000 unlinks in 119 seconds: 84.033615 unlinks/second - open/close 378 (time 1785401204.15 total 10.01 last 37.75) - open/close 953 (time 1785401214.16 total 20.02 last 57.47) - open/close 1516 (time 1785401224.16 total 30.02 last 56.28) - open/close 2226 (time 1785401234.17 total 40.03 last 70.93) - open/close 2883 (time 1785401244.19 total 50.05 last 65.58) - open/close 3388 (time 1785401254.21 total 60.07 last 50.41) - open/close 3718 (time 1785401264.22 total 70.08 last 32.95) - open/close 4218 (time 1785401274.25 total 80.11 last 49.85) - open/close 4715 (time 1785401284.25 total 90.11 last 49.69) - open/close 5150 (time 1785401294.27 total 100.13 last 43.42) - open/close 5548 (time 1785401304.27 total 110.13 last 39.80) - open/close 5955 (time 1785401314.29 total 120.15 last 40.63) - open/close 6299 (time 1785401324.32 total 130.18 last 34.31) - open/close 6872 (time 1785401334.33 total 140.19 last 57.20) - open/close 7422 (time 1785401344.37 total 150.23 last 54.79) - open/close 7923 (time 1785401354.42 total 160.28 last 49.87) - open/close 8300 (time 1785401364.45 total 170.31 last 37.57) - open/close 8833 (time 1785401374.48 total 180.33 last 53.18) - open/close 9204 (time 1785401384.49 total 190.35 last 37.05) - open/close 9779 (time 1785401394.50 total 200.36 last 57.43) total: 10000 open/close in 202.79 seconds: 49.31 ops/second - unlinked 0 (time 1785401403 ; total 0 ; last 0) total: 10000 unlinks in 97 seconds: 103.092781 unlinks/second - open/close 472 (time 1785401515.63 total 10.02 last 47.09) - open/close 911 (time 1785401525.65 total 20.05 last 43.80) - open/close 1644 (time 1785401535.66 total 30.06 last 73.21) - open/close 2069 (time 1785401545.67 total 40.07 last 42.47) - open/close 2476 (time 1785401555.74 total 50.14 last 40.41) - open/close 2725 (time 1785401565.77 total 60.17 last 24.82) - open/close 2957 (time 1785401575.82 total 70.22 last 23.09) - open/close 3329 (time 1785401585.83 total 80.22 last 37.19) - open/close 3840 (time 1785401595.84 total 90.24 last 51.00) - open/close 4406 (time 1785401605.85 total 100.25 last 56.55) - open/close 4854 (time 1785401615.89 total 110.28 last 44.64) - open/close 5440 (time 1785401625.90 total 120.29 last 58.54) - open/close 5913 (time 1785401635.93 total 130.32 last 47.16) - open/close 6340 (time 1785401645.94 total 140.34 last 42.64) - open/close 6747 (time 1785401655.96 total 150.36 last 40.62) - open/close 7215 (time 1785401665.97 total 160.36 last 46.77) - open/close 7893 (time 1785401675.98 total 170.37 last 67.72) - open/close 8416 (time 1785401686.00 total 180.39 last 52.19) - open/close 8847 (time 1785401696.00 total 190.40 last 43.09) - open/close 9435 (time 1785401706.00 total 200.40 last 58.79) total: 10000 open/close in 209.08 seconds: 47.83 ops/second - unlinked 0 (time 1785401720 ; total 0 ; last 0) total: 10000 unlinks in 115 seconds: 86.956520 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg222-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg222-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg222-server Permanent disk data: Device: /dev/mapper/ost1_flakey Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x42 (OST update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.202.122@tcp sys.timeout=20 device size = 4096MB formatting backing filesystem ldiskfs on /dev/mapper/ost1_flakey target name lustre-OST0000 kilobytes 200000 options -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F mkfs_cmd = mke2fs -j -b 4096 -L lustre-OST0000 -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/mapper/ost1_flakey 200000k start ost1 service on oleg222-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg222-server: oleg222-server.virtnet: executing set_default_debug -1 all pdsh@oleg222-client: oleg222-server: ssh exited with exit code 1 Started lustre-OST0000 oleg222-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg222-server: oleg222-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg222-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg222-server: oleg222-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg222-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg222-client.virtnet: -o user_xattr,flock 192.168.202.122@tcp:/lustre /mnt/lustre On OST0, 20430 used inodes rc=0 pdsh@oleg222-client: oleg222-server: ssh exited with exit code 22 umount lustre on /mnt/lustre..... Stopping client oleg222-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg222-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg222-server Permanent disk data: Device: /dev/mapper/ost1_flakey Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x62 (OST first_time update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.202.122@tcp sys.timeout=20 device size = 4096MB formatting backing filesystem ldiskfs on /dev/mapper/ost1_flakey target name lustre:OST0000 kilobytes 200000 options -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F mkfs_cmd = mke2fs -j -b 4096 -L lustre:OST0000 -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/mapper/ost1_flakey 200000k checking for existing Lustre data: found Read previous values: Device: /dev/mapper/ost1_flakey Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x62 (OST first_time update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.202.122@tcp sys.timeout=20 Permanent disk data: Device: /dev/mapper/ost1_flakey Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x42 (OST update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.202.122@tcp sys.timeout=20 start ost1 service on oleg222-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg222-server: oleg222-server.virtnet: executing set_default_debug -1 all pdsh@oleg222-client: oleg222-server: ssh exited with exit code 1 Started lustre-OST0000 oleg222-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg222-server: oleg222-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg222-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg222-server: oleg222-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg222-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg222-client.virtnet: -o user_xattr,flock 192.168.202.122@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg222-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg222-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg222-server stop mds service on oleg222-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg222-server stop mds service on oleg222-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg222-server unloading modules via unload_modules_local on: 'oleg222-server' oleg222-server: oleg222-server.virtnet: executing unload_modules_local oleg222-server: modules unloaded. pdsh@oleg222-client: oleg222-client: ssh exited with exit code 2 pdsh@oleg222-client: oleg222-server: ssh exited with exit code 2 pdsh@oleg222-client: oleg222-client: ssh exited with exit code 2 pdsh@oleg222-client: oleg222-server: ssh exited with exit code 2 PASS 69 (3555s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 05:01:38 (1785402098) oleg222-server: debugfs 1.47.3-wc2 (11-Nov-2025) Supported features: dir_prealloc imagic_inodes has_journal ext_attr resize_inode dir_index sparse_super2 fast_commit stable_inodes orphan_file filetype needs_recovery journal_dev meta_bg extent 64bit mmp flex_bg ea_inode dirdata metadata_csum_seed large_dir inline_data encrypt casefold sparse_super large_file huge_file uninit_bg dir_nlink extra_isize quota bigalloc metadata_csum read-only project shared_blocks verity orphan_present Stopping clients: oleg222-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg222-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg222-client: oleg222-server: ssh exited with exit code 2 oleg222-server: oleg222-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg222-server' oleg222-server: oleg222-server.virtnet: executing load_modules_local oleg222-server: Loading modules from /home/green/git/lustre-release/lustre oleg222-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg222-server: detected 4 online CPUs by sysfs oleg222-server: MODOPTS_LIBCFS= oleg222-server: Force libcfs to create 2 CPU partitions oleg222-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg222-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg222-server: mdt/mdt options: 'mdt_enable_flr_ec=1' Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format ost1: /dev/mapper/ost1_flakey pdsh@oleg222-client: oleg222-server: ssh exited with exit code 1 SKIP: conf-sanity test_111 /dev/mapper/mds1_flakey too small for 5000000kB MDS Stopping clients: oleg222-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg222-client.virtnet /mnt/lustre2 (opts:-f) oleg222-server: oleg222-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg222-server' oleg222-server: oleg222-server.virtnet: executing load_modules_local oleg222-server: Loading modules from /home/green/git/lustre-release/lustre oleg222-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg222-server: detected 4 online CPUs by sysfs oleg222-server: MODOPTS_LIBCFS= oleg222-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg222-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg222-server: oleg222-server.virtnet: executing set_default_debug -1 all pdsh@oleg222-client: oleg222-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg222-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg222-server: oleg222-server.virtnet: executing set_default_debug -1 all pdsh@oleg222-client: oleg222-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg222-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg222-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg222-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg222-server: oleg222-server.virtnet: executing set_default_debug -1 all pdsh@oleg222-client: oleg222-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg222-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg222-server: oleg222-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg222-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg222-server: oleg222-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg222-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg222-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg222-server stop mds service on oleg222-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg222-server stop mds service on oleg222-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg222-server SKIP 111 (317s) Stopping clients: oleg222-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg222-client.virtnet /mnt/lustre2 (opts:-f) oleg222-server: oleg222-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg222-server' oleg222-server: oleg222-server.virtnet: executing load_modules_local oleg222-server: Loading modules from /home/green/git/lustre-release/lustre oleg222-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg222-server: detected 4 online CPUs by sysfs oleg222-server: MODOPTS_LIBCFS= oleg222-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey == conf-sanity test complete, duration 4608 sec ========== 05:08:21 (1785402501) === conf-sanity: start cleanup 05:08:24 (1785402504) === === conf-sanity: finish cleanup 05:08:29 (1785402509) ===