-----============= acceptance-small: conf-sanity ============----- Sun Nov 16 22:47:02 EST 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg452-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32newtarball Stopping clients: oleg452-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg452-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg452-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg452-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg452-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg452-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg452-server oleg452-server: oleg452-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg452-server' oleg452-server: oleg452-server.virtnet: executing load_modules_local oleg452-server: Loading modules from /home/green/git/lustre-release/lustre oleg452-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg452-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg452-server: detected 4 online CPUs by sysfs oleg452-server: MODOPTS_LIBCFS= oleg452-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg452-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg452-server: oleg452-server.virtnet: executing set_default_debug -1 all pdsh@oleg452-client: oleg452-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg452-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg452-server: oleg452-server.virtnet: executing set_default_debug -1 all pdsh@oleg452-client: oleg452-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg452-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg452-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg452-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg452-server: oleg452-server.virtnet: executing set_default_debug -1 all pdsh@oleg452-client: oleg452-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg452-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg452-server: oleg452-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg452-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg452-server: oleg452-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg452-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg452-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg452-server stop mds service on oleg452-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg452-server stop mds service on oleg452-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg452-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 22:50:00 (1763351400) Stopping clients: oleg452-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg452-client.virtnet /mnt/lustre2 (opts:-f) oleg452-server: oleg452-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg452-server' oleg452-server: oleg452-server.virtnet: executing load_modules_local oleg452-server: Loading modules from /home/green/git/lustre-release/lustre oleg452-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg452-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg452-server: detected 4 online CPUs by sysfs oleg452-server: MODOPTS_LIBCFS= oleg452-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg452-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg452-server: oleg452-server.virtnet: executing set_default_debug -1 all pdsh@oleg452-client: oleg452-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg452-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg452-server: oleg452-server.virtnet: executing set_default_debug -1 all pdsh@oleg452-client: oleg452-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg452-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg452-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg452-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg452-server: oleg452-server.virtnet: executing set_default_debug -1 all pdsh@oleg452-client: oleg452-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg452-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg452-client.virtnet: -o user_xattr,flock 192.168.204.152@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 79992 274 79718 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 79992 255 79737 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 65808 368 65440 1% /mnt/lustre[OST:0] filesystem_summary: 65969 529 65440 1% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 1023 (time 1763351500.35 total 10.00 last 102.28) - open/close 2076 (time 1763351510.35 total 20.01 last 105.24) - open/close 3077 (time 1763351520.37 total 30.02 last 99.95) - open/close 4015 (time 1763351530.37 total 40.03 last 93.78) - open/close 5042 (time 1763351540.38 total 50.04 last 102.60) - open/close 6066 (time 1763351550.38 total 60.04 last 102.36) - open/close 7016 (time 1763351560.39 total 70.05 last 94.89) - open/close 7942 (time 1763351570.40 total 80.06 last 92.55) - open/close 8932 (time 1763351580.41 total 90.06 last 98.93) - open/close 9883 (time 1763351590.41 total 100.06 last 95.09) - open/close 10818 (time 1763351600.41 total 110.06 last 93.49) - open/close 11753 (time 1763351610.41 total 120.07 last 93.45) - open/close 12741 (time 1763351620.42 total 130.07 last 98.76) - open/close 13746 (time 1763351630.42 total 140.07 last 100.50) - open/close 14665 (time 1763351640.42 total 150.07 last 91.90) - open/close 15529 (time 1763351650.43 total 160.08 last 86.33) - open/close 16367 (time 1763351660.43 total 170.08 last 83.79) - open/close 17185 (time 1763351670.43 total 180.08 last 81.79) - open/close 18034 (time 1763351680.43 total 190.08 last 84.90) - open/close 18966 (time 1763351690.43 total 200.09 last 93.18) - open/close 19895 (time 1763351700.43 total 210.09 last 92.87) - open/close 20804 (time 1763351710.44 total 220.10 last 90.85) - open/close 21674 (time 1763351720.45 total 230.10 last 86.94) - open/close 22605 (time 1763351730.45 total 240.11 last 93.02) - open/close 23522 (time 1763351740.46 total 250.12 last 91.64) - open/close 24473 (time 1763351750.47 total 260.12 last 95.04) - open/close 25403 (time 1763351760.47 total 270.12 last 92.98) - open/close 26335 (time 1763351770.47 total 280.13 last 93.18) - open/close 27246 (time 1763351780.49 total 290.15 last 90.89) - open/close 28173 (time 1763351790.51 total 300.16 last 92.58) - open/close 29012 (time 1763351800.51 total 310.17 last 83.84) - open/close 29914 (time 1763351810.52 total 320.17 last 90.17) - open/close 30869 (time 1763351820.52 total 330.18 last 95.44) - open/close 31829 (time 1763351830.53 total 340.18 last 95.98) - open/close 32758 (time 1763351840.53 total 350.19 last 92.82) - open/close 33742 (time 1763351850.54 total 360.20 last 98.32) - open/close 34706 (time 1763351860.55 total 370.20 last 96.33) - open/close 35649 (time 1763351870.55 total 380.21 last 94.30) - open/close 36419 (time 1763351880.55 total 390.21 last 76.98) - open/close 37382 (time 1763351890.56 total 400.22 last 96.21) - open/close 38395 (time 1763351900.57 total 410.23 last 101.19) - open/close 39406 (time 1763351910.57 total 420.23 last 101.08) - open/close 40000 (time 1763351915.80 total 425.46 last 113.59) - open/close 41011 (time 1763351925.81 total 435.47 last 101.03) - open/close 42025 (time 1763351935.81 total 445.47 last 101.35) - open/close 43004 (time 1763351945.82 total 455.47 last 97.87) - open/close 43993 (time 1763351955.82 total 465.48 last 98.85) - open/close 44935 (time 1763351965.83 total 475.49 last 94.09) - open/close 45878 (time 1763351975.84 total 485.50 last 94.22) - open/close 46841 (time 1763351985.85 total 495.51 last 96.19) - open/close 47802 (time 1763351995.86 total 505.51 last 96.08) - open/close 48731 (time 1763352005.86 total 515.51 last 92.88) - open/close 49658 (time 1763352015.86 total 525.52 last 92.66) - open/close 50000 (time 1763352019.38 total 529.04 last 97.18) - open/close 50980 (time 1763352029.39 total 539.04 last 97.95) - open/close 52003 (time 1763352039.40 total 549.05 last 102.18) - open/close 53100 (time 1763352049.40 total 559.06 last 109.69) - open/close 54127 (time 1763352059.40 total 569.06 last 102.69) - open/close 55043 (time 1763352069.41 total 579.06 last 91.56) - open/close 56052 (time 1763352079.41 total 589.07 last 100.82) - open/close 57049 (time 1763352089.42 total 599.07 last 99.67) - open/close 58017 (time 1763352099.42 total 609.07 last 96.78) - open/close 58935 (time 1763352109.42 total 619.08 last 91.76) - open/close 59879 (time 1763352119.43 total 629.09 last 94.33) - open/close 60921 (time 1763352129.44 total 639.09 last 104.14) - open/close 61794 (time 1763352139.44 total 649.10 last 87.27) - open/close 62748 (time 1763352149.45 total 659.10 last 95.35) - open/close 63735 (time 1763352159.47 total 669.13 last 98.44) - open/close 64650 (time 1763352169.49 total 679.15 last 91.33) total: 64768 open/close in 680.43 seconds: 95.19 ops/second - /unlink 889 (time 1763352183.38 total 10.00 last 88.87) - /unlink 1831 (time 1763352193.40 total 20.02 last 94.06) - /unlink 2752 (time 1763352203.40 total 30.03 last 92.03) - /unlink 3710 (time 1763352213.41 total 40.03 last 95.73) - /unlink 4851 (time 1763352223.42 total 50.04 last 114.04) - /unlink 5819 (time 1763352233.42 total 60.04 last 96.76) - /unlink 6712 (time 1763352243.42 total 70.04 last 89.28) - /unlink 7697 (time 1763352253.42 total 80.04 last 98.50) - /unlink 8763 (time 1763352263.43 total 90.05 last 106.58) - /unlink 9678 (time 1763352273.43 total 100.05 last 91.46) - /unlink 10000 (time 1763352276.67 total 103.29 last 99.47) - /unlink 11135 (time 1763352286.67 total 113.29 last 113.49) - /unlink 12306 (time 1763352296.68 total 123.30 last 116.97) - /unlink 13505 (time 1763352306.68 total 133.30 last 119.90) - /unlink 14542 (time 1763352316.68 total 143.30 last 103.67) - /unlink 15781 (time 1763352326.69 total 153.31 last 123.84) - /unlink 17217 (time 1763352336.69 total 163.31 last 143.56) - /unlink 18682 (time 1763352346.69 total 173.31 last 146.48) - /unlink 20000 (time 1763352355.20 total 181.82 last 154.88) - /unlink 21444 (time 1763352365.21 total 191.83 last 144.32) - /unlink 22619 (time 1763352375.22 total 201.84 last 117.36) - /unlink 23666 (time 1763352385.23 total 211.85 last 104.58) - /unlink 24846 (time 1763352395.23 total 221.86 last 117.94) - /unlink 26235 (time 1763352405.25 total 231.87 last 138.68) - /unlink 27290 (time 1763352415.25 total 241.87 last 105.48) - /unlink 28361 (time 1763352425.25 total 251.87 last 107.09) - /unlink 29509 (time 1763352435.26 total 261.88 last 114.77) - /unlink 30000 (time 1763352439.18 total 265.80 last 125.03) - /unlink 31096 (time 1763352449.19 total 275.81 last 109.53) - /unlink 32233 (time 1763352459.19 total 285.81 last 113.66) - /unlink 33513 (time 1763352469.20 total 295.82 last 127.93) - /unlink 34756 (time 1763352479.20 total 305.82 last 124.24) - /unlink 35988 (time 1763352489.21 total 315.83 last 123.15) - /unlink 37207 (time 1763352499.22 total 325.84 last 121.78) - /unlink 38694 (time 1763352509.22 total 335.84 last 148.67) - /unlink 39985 (time 1763352519.22 total 345.85 last 129.04) - /unlink 41062 (time 1763352529.23 total 355.85 last 107.66) - /unlink 42334 (time 1763352539.23 total 365.85 last 127.19) - /unlink 44234 (time 1763352549.23 total 375.85 last 189.94) - /unlink 46593 (time 1763352559.24 total 385.86 last 235.79) - /unlink 48611 (time 1763352569.24 total 395.86 last 201.75) - /unlink 50000 (time 1763352575.14 total 401.76 last 235.48) - /unlink 52345 (time 1763352585.14 total 411.76 last 234.41) - /unlink 54558 (time 1763352595.14 total 421.76 last 221.30) - /unlink 56854 (time 1763352605.14 total 431.77 last 229.52) - /unlink 59094 (time 1763352615.15 total 441.77 last 223.95) - /unlink 60000 (time 1763352618.85 total 445.47 last 244.74) - /unlink 62605 (time 1763352628.85 total 455.47 last 260.46) total: 64768 /unlink in 464.59 seconds: 139.41 ops/second Waiting for MDT destroys to complete oleg452-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg452-client: oleg452-server: ssh exited with exit code 1 - open/close 1176 (time 1763352660.54 total 10.00 last 117.57) - open/close 2491 (time 1763352670.54 total 20.01 last 131.45) - open/close 3674 (time 1763352680.54 total 30.01 last 118.26) - open/close 5072 (time 1763352690.55 total 40.01 last 139.76) - open/close 6377 (time 1763352700.55 total 50.01 last 130.50) - open/close 7500 (time 1763352710.55 total 60.02 last 112.23) - open/close 8938 (time 1763352720.56 total 70.02 last 143.74) - open/close 10000 (time 1763352727.81 total 77.27 last 146.51) - open/close 11535 (time 1763352737.81 total 87.28 last 153.43) - open/close 13078 (time 1763352747.81 total 97.28 last 154.27) - open/close 14546 (time 1763352757.83 total 107.29 last 146.60) - open/close 15838 (time 1763352767.84 total 117.31 last 129.00) - open/close 16922 (time 1763352777.84 total 127.31 last 108.37) - open/close 17983 (time 1763352787.85 total 137.32 last 106.01) - open/close 19127 (time 1763352797.85 total 147.32 last 114.38) - open/close 20000 (time 1763352804.75 total 154.21 last 126.62) - open/close 21543 (time 1763352814.75 total 164.22 last 154.26) - open/close 22809 (time 1763352824.75 total 174.22 last 126.59) - open/close 24319 (time 1763352834.75 total 184.22 last 150.97) - open/close 25767 (time 1763352844.76 total 194.22 last 144.75) - open/close 27151 (time 1763352854.76 total 204.22 last 138.39) - open/close 28608 (time 1763352864.76 total 214.22 last 145.69) - open/close 29999 (time 1763352874.76 total 224.23 last 139.07) - open/close 31222 (time 1763352884.77 total 234.23 last 122.21) - open/close 32619 (time 1763352894.77 total 244.24 last 139.69) - open/close 33993 (time 1763352904.77 total 254.24 last 137.35) - open/close 35514 (time 1763352914.79 total 264.25 last 151.88) - open/close 36926 (time 1763352924.79 total 274.26 last 141.18) - open/close 38329 (time 1763352934.79 total 284.26 last 140.30) - open/close 40000 (time 1763352944.01 total 293.47 last 181.32) - open/close 41648 (time 1763352954.01 total 303.47 last 164.79) - open/close 42951 (time 1763352964.01 total 313.48 last 130.22) - open/close 44232 (time 1763352974.02 total 323.48 last 128.06) - open/close 45622 (time 1763352984.02 total 333.48 last 138.98) - open/close 46938 (time 1763352994.02 total 343.49 last 131.52) - open/close 48273 (time 1763353004.02 total 353.49 last 133.50) - open/close 49875 (time 1763353014.03 total 363.49 last 160.15) - open/close 51552 (time 1763353024.03 total 373.50 last 167.65) - open/close 53017 (time 1763353034.04 total 383.50 last 146.42) - open/close 54640 (time 1763353044.04 total 393.50 last 162.26) - open/close 56377 (time 1763353054.04 total 403.51 last 173.64) - open/close 58019 (time 1763353064.05 total 413.51 last 164.14) - open/close 59821 (time 1763353074.05 total 423.52 last 180.12) - open/close 61759 (time 1763353084.05 total 433.52 last 193.75) - open/close 63207 (time 1763353094.05 total 443.52 last 144.77) - open/close 64583 (time 1763353104.06 total 453.52 last 137.58) total: 64768 open/close in 455.17 seconds: 142.30 ops/second - /unlink 2652 (time 1763353117.73 total 10.00 last 265.14) - /unlink 4888 (time 1763353127.73 total 20.01 last 223.53) - /unlink 7695 (time 1763353137.73 total 30.01 last 280.63) - /unlink 10000 (time 1763353146.61 total 38.89 last 259.49) - /unlink 12600 (time 1763353156.62 total 48.90 last 259.87) - /unlink 15460 (time 1763353166.62 total 58.90 last 285.96) - /unlink 18793 (time 1763353176.62 total 68.90 last 333.27) - /unlink 20000 (time 1763353179.26 total 71.54 last 456.77) - /unlink 24285 (time 1763353189.27 total 81.54 last 428.45) - /unlink 27515 (time 1763353199.27 total 91.54 last 322.94) - /unlink 30000 (time 1763353207.58 total 99.85 last 299.07) - /unlink 32832 (time 1763353217.58 total 109.85 last 283.14) - /unlink 36133 (time 1763353227.58 total 119.86 last 329.99) - /unlink 39538 (time 1763353237.58 total 129.86 last 340.46) - /unlink 42812 (time 1763353247.59 total 139.86 last 327.32) - /unlink 46388 (time 1763353257.59 total 149.86 last 357.54) - /unlink 49411 (time 1763353267.59 total 159.86 last 302.26) - /unlink 50000 (time 1763353269.87 total 162.14 last 258.37) - /unlink 53011 (time 1763353279.87 total 172.15 last 301.05) - /unlink 56149 (time 1763353289.87 total 182.15 last 313.78) - /unlink 59290 (time 1763353299.87 total 192.15 last 314.03) - /unlink 63160 (time 1763353309.88 total 202.15 last 386.89) total: 64768 /unlink in 206.09 seconds: 314.26 ops/second Waiting for MDT destroys to complete oleg452-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg452-client: oleg452-server: ssh exited with exit code 1 - open/close 2102 (time 1763353335.84 total 10.00 last 210.19) - open/close 3958 (time 1763353345.85 total 20.01 last 185.48) - open/close 5568 (time 1763353355.85 total 30.01 last 160.98) - open/close 7077 (time 1763353365.85 total 40.01 last 150.83) - open/close 8762 (time 1763353375.86 total 50.02 last 168.44) - open/close 10000 (time 1763353383.34 total 57.50 last 165.47) - open/close 11762 (time 1763353393.34 total 67.50 last 176.13) - open/close 13532 (time 1763353403.34 total 77.50 last 176.96) - open/close 15558 (time 1763353413.35 total 87.51 last 202.51) - open/close 17398 (time 1763353423.35 total 97.51 last 183.90) - open/close 19292 (time 1763353433.36 total 107.52 last 189.35) - open/close 20000 (time 1763353437.05 total 111.21 last 191.85) - open/close 22119 (time 1763353447.05 total 121.21 last 211.85) - open/close 24195 (time 1763353457.05 total 131.21 last 207.59) - open/close 26141 (time 1763353467.05 total 141.21 last 194.54) - open/close 27996 (time 1763353477.05 total 151.21 last 185.49) - open/close 29686 (time 1763353487.05 total 161.22 last 168.98) - open/close 31766 (time 1763353497.06 total 171.22 last 207.92) - open/close 33885 (time 1763353507.06 total 181.22 last 211.88) - open/close 35917 (time 1763353517.06 total 191.22 last 203.14) - open/close 38135 (time 1763353527.06 total 201.22 last 221.77) - open/close 40000 (time 1763353535.79 total 209.95 last 213.83) - open/close 42908 (time 1763353545.79 total 219.95 last 290.74) - open/close 45574 (time 1763353555.79 total 229.95 last 266.59) - open/close 47800 (time 1763353565.79 total 239.95 last 222.55) - open/close 50000 (time 1763353574.99 total 249.15 last 239.18) - open/close 53068 (time 1763353584.99 total 259.15 last 306.72) - open/close 55626 (time 1763353594.99 total 269.16 last 255.70) - open/close 58963 (time 1763353605.00 total 279.16 last 333.66) - open/close 60000 (time 1763353607.74 total 281.90 last 378.37) - open/close 63624 (time 1763353617.74 total 291.90 last 362.30) total: 64768 open/close in 295.01 seconds: 219.54 ops/second - /unlink 5584 (time 1763353632.35 total 10.00 last 558.26) - /unlink 10000 (time 1763353642.07 total 19.73 last 454.06) - /unlink 14532 (time 1763353652.08 total 29.73 last 453.17) - /unlink 18271 (time 1763353662.08 total 39.73 last 373.88) - /unlink 20000 (time 1763353665.80 total 43.46 last 463.70) - /unlink 24165 (time 1763353675.81 total 53.46 last 416.46) - /unlink 29423 (time 1763353685.81 total 63.46 last 525.78) - /unlink 34596 (time 1763353695.81 total 73.46 last 517.20) - /unlink 39457 (time 1763353705.81 total 83.46 last 485.97) - /unlink 44517 (time 1763353715.81 total 93.46 last 505.95) - /unlink 50000 (time 1763353724.93 total 102.59 last 601.06) - /unlink 56363 (time 1763353734.93 total 112.59 last 636.29) - /unlink 60000 (time 1763353740.27 total 117.93 last 681.25) total: 64768 /unlink in 125.37 seconds: 516.62 ops/second Waiting for MDT destroys to complete oleg452-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg452-client: oleg452-server: ssh exited with exit code 1 - open/close 3676 (time 1763353767.70 total 10.00 last 367.54) - open/close 7952 (time 1763353777.70 total 20.00 last 427.52) - open/close 10000 (time 1763353782.91 total 25.21 last 393.17) - open/close 13320 (time 1763353792.92 total 35.22 last 331.89) - open/close 17536 (time 1763353802.92 total 45.22 last 421.54) - open/close 20000 (time 1763353808.04 total 50.33 last 481.47) - open/close 24233 (time 1763353818.04 total 60.34 last 423.29) - open/close 28057 (time 1763353828.04 total 70.34 last 382.33) - open/close 30000 (time 1763353833.88 total 76.18 last 332.42) - open/close 33077 (time 1763353843.89 total 86.18 last 307.61) - open/close 36565 (time 1763353853.89 total 96.19 last 348.76) - open/close 40000 (time 1763353862.44 total 104.74 last 401.54) - open/close 43830 (time 1763353872.44 total 114.74 last 382.93) - open/close 47686 (time 1763353882.44 total 124.74 last 385.58) - open/close 50000 (time 1763353889.18 total 131.48 last 343.54) - open/close 52791 (time 1763353899.18 total 141.48 last 279.07) - open/close 56624 (time 1763353909.18 total 151.48 last 383.26) - open/close 60000 (time 1763353916.95 total 159.25 last 434.73) - open/close 64000 (time 1763353926.95 total 169.25 last 399.94) total: 64768 open/close in 170.99 seconds: 378.78 ops/second - /unlink 7685 (time 1763353940.12 total 10.00 last 768.50) - /unlink 10000 (time 1763353943.24 total 13.12 last 741.85) - /unlink 17202 (time 1763353953.24 total 23.12 last 720.20) - /unlink 20000 (time 1763353957.07 total 26.95 last 730.63) - /unlink 28139 (time 1763353967.07 total 36.95 last 813.84) - /unlink 36146 (time 1763353977.07 total 46.95 last 800.67) - /unlink 40000 (time 1763353981.85 total 51.73 last 805.83) - /unlink 47173 (time 1763353991.85 total 61.73 last 717.26) - /unlink 50000 (time 1763353995.21 total 65.09 last 841.80) - /unlink 56984 (time 1763354005.21 total 75.09 last 698.35) - /unlink 60000 (time 1763354010.14 total 80.02 last 612.09) total: 64768 /unlink in 87.17 seconds: 743.03 ops/second Waiting for MDT destroys to complete oleg452-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg452-client: oleg452-server: ssh exited with exit code 1 - open/close 4750 (time 1763354081.89 total 10.00 last 474.94) - open/close 8326 (time 1763354091.89 total 20.00 last 357.60) - open/close 10000 (time 1763354095.61 total 23.72 last 450.02) - open/close 13749 (time 1763354105.61 total 33.72 last 374.88) - open/close 17890 (time 1763354115.61 total 43.72 last 414.02) - open/close 20000 (time 1763354120.30 total 48.41 last 450.43) - open/close 24002 (time 1763354130.30 total 58.41 last 400.10) - open/close 28980 (time 1763354140.30 total 68.41 last 497.73) - open/close 34276 (time 1763354150.30 total 78.41 last 529.57) - open/close 39281 (time 1763354160.30 total 88.41 last 500.41) - open/close 43892 (time 1763354170.30 total 98.42 last 460.96) - open/close 48485 (time 1763354180.31 total 108.42 last 459.24) - open/close 50000 (time 1763354183.44 total 111.55 last 483.92) - open/close 54483 (time 1763354193.44 total 121.55 last 448.21) - open/close 58478 (time 1763354203.44 total 131.55 last 399.45) - open/close 60000 (time 1763354207.17 total 135.29 last 407.49) - open/close 63984 (time 1763354217.17 total 145.29 last 398.40) total: 64768 open/close in 146.92 seconds: 440.83 ops/second - /unlink 7340 (time 1763354230.25 total 10.00 last 733.93) - /unlink 10000 (time 1763354234.24 total 13.99 last 666.44) - /unlink 18216 (time 1763354244.24 total 23.99 last 821.54) - /unlink 27380 (time 1763354254.25 total 34.00 last 915.44) - /unlink 30000 (time 1763354257.37 total 37.12 last 840.67) - /unlink 37257 (time 1763354267.37 total 47.12 last 725.66) - /unlink 40000 (time 1763354271.01 total 50.76 last 752.87) - /unlink 48444 (time 1763354281.01 total 60.76 last 844.35) - /unlink 56757 (time 1763354291.01 total 70.76 last 831.29) - /unlink 60000 (time 1763354294.53 total 74.29 last 921.24) total: 64768 /unlink in 79.62 seconds: 813.47 ops/second Waiting for MDT destroys to complete oleg452-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg452-client: oleg452-server: ssh exited with exit code 1 fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg452-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg452-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg452-server stop mds service on oleg452-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg452-server stop mds service on oleg452-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg452-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg452-server' oleg452-server: oleg452-server.virtnet: executing unload_modules_local oleg452-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg452-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg452-client: oleg452-client: ssh exited with exit code 2 pdsh@oleg452-client: oleg452-server: ssh exited with exit code 2 pdsh@oleg452-client: oleg452-client: ssh exited with exit code 2 pdsh@oleg452-client: oleg452-server: ssh exited with exit code 2 PASS 106 (2964s) Stopping clients: oleg452-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg452-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg452-client: oleg452-server: ssh exited with exit code 2 oleg452-server: oleg452-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg452-server' oleg452-server: oleg452-server.virtnet: executing load_modules_local oleg452-server: Loading modules from /home/green/git/lustre-release/lustre oleg452-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg452-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg452-server: detected 4 online CPUs by sysfs oleg452-server: MODOPTS_LIBCFS= oleg452-server: Force libcfs to create 2 CPU partitions oleg452-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg452-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey == conf-sanity test complete, duration 3164 sec ========== 23:39:48 (1763354388) === conf-sanity: start cleanup 23:39:49 (1763354389) === === conf-sanity: finish cleanup 23:39:50 (1763354390) ===