== conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 05:44:22 (1762166662) Stopping clients: oleg139-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg139-client.virtnet /mnt/lustre2 (opts:-f) oleg139-server: oleg139-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg139-server' oleg139-server: oleg139-server.virtnet: executing load_modules_local oleg139-server: Loading modules from /home/green/git/lustre-release/lustre oleg139-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg139-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg139-server: detected 4 online CPUs by sysfs oleg139-server: MODOPTS_LIBCFS= oleg139-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg139-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg139-server: oleg139-server.virtnet: executing set_default_debug -1 all pdsh@oleg139-client: oleg139-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg139-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg139-server: oleg139-server.virtnet: executing set_default_debug -1 all pdsh@oleg139-client: oleg139-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg139-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg139-server: oleg139-server.virtnet: executing set_default_debug -1 all pdsh@oleg139-client: oleg139-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg139-client.virtnet: -o user_xattr,flock 192.168.201.139@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 79992 274 79718 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 79992 255 79737 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 65808 368 65440 1% /mnt/lustre[OST:0] filesystem_summary: 65969 529 65440 1% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 1175 (time 1762166731.40 total 10.00 last 117.45) - open/close 2095 (time 1762166741.40 total 20.01 last 91.98) - open/close 3039 (time 1762166751.40 total 30.01 last 94.38) - open/close 4043 (time 1762166761.40 total 40.01 last 100.38) - open/close 5151 (time 1762166771.41 total 50.01 last 110.78) - open/close 6350 (time 1762166781.41 total 60.01 last 119.89) - open/close 7736 (time 1762166791.41 total 70.01 last 138.59) - open/close 9155 (time 1762166801.41 total 80.01 last 141.88) - open/close 10000 (time 1762166807.22 total 85.82 last 145.48) - open/close 11548 (time 1762166817.23 total 95.83 last 154.68) - open/close 13187 (time 1762166827.23 total 105.84 last 163.81) - open/close 14570 (time 1762166837.23 total 115.84 last 138.30) - open/close 15839 (time 1762166847.24 total 125.84 last 126.84) - open/close 17254 (time 1762166857.24 total 135.84 last 141.50) - open/close 18361 (time 1762166867.25 total 145.85 last 110.59) - open/close 19812 (time 1762166877.25 total 155.85 last 145.09) - open/close 21289 (time 1762166887.25 total 165.86 last 147.61) - open/close 22599 (time 1762166897.26 total 175.86 last 130.93) - open/close 24086 (time 1762166907.27 total 185.88 last 148.49) - open/close 26271 (time 1762166917.28 total 195.88 last 218.44) - open/close 28061 (time 1762166927.28 total 205.89 last 178.92) - open/close 30000 (time 1762166936.68 total 215.28 last 206.37) - open/close 31680 (time 1762166946.68 total 225.29 last 167.87) - open/close 33088 (time 1762166956.69 total 235.29 last 140.77) - open/close 34729 (time 1762166966.69 total 245.29 last 164.04) - open/close 36596 (time 1762166976.70 total 255.30 last 186.56) - open/close 38123 (time 1762166986.70 total 265.30 last 152.68) - open/close 40000 (time 1762166995.80 total 274.41 last 206.12) - open/close 42095 (time 1762167005.80 total 284.41 last 209.49) - open/close 44089 (time 1762167015.81 total 294.41 last 199.35) - open/close 45998 (time 1762167025.81 total 304.41 last 190.89) - open/close 47672 (time 1762167035.81 total 314.41 last 167.38) - open/close 49147 (time 1762167045.81 total 324.42 last 147.44) - open/close 50000 (time 1762167050.81 total 329.42 last 170.51) - open/close 51870 (time 1762167060.82 total 339.42 last 186.94) - open/close 53654 (time 1762167070.82 total 349.43 last 178.33) - open/close 55304 (time 1762167080.83 total 359.43 last 164.92) - open/close 57087 (time 1762167090.83 total 369.43 last 178.28) - open/close 58846 (time 1762167100.83 total 379.44 last 175.85) - open/close 60000 (time 1762167106.58 total 385.19 last 200.69) - open/close 61596 (time 1762167116.58 total 395.19 last 159.56) - open/close 63076 (time 1762167126.59 total 405.19 last 147.94) total: 64768 open/close in 414.56 seconds: 156.23 ops/second - /unlink 3013 (time 1762167147.73 total 10.00 last 301.26) - /unlink 6627 (time 1762167157.73 total 20.00 last 361.39) - /unlink 9924 (time 1762167167.73 total 30.00 last 329.66) - /unlink 13245 (time 1762167177.73 total 40.00 last 332.09) - /unlink 16762 (time 1762167187.73 total 50.01 last 351.62) - /unlink 19807 (time 1762167197.74 total 60.01 last 304.45) - /unlink 23015 (time 1762167207.74 total 70.01 last 320.72) - /unlink 25995 (time 1762167217.74 total 80.01 last 297.95) - /unlink 29738 (time 1762167227.74 total 90.01 last 374.21) - /unlink 33996 (time 1762167237.74 total 100.01 last 425.79) - /unlink 38505 (time 1762167247.74 total 110.01 last 450.86) - /unlink 40000 (time 1762167251.49 total 113.76 last 398.65) - /unlink 43563 (time 1762167261.49 total 123.77 last 356.29) - /unlink 47485 (time 1762167271.50 total 133.77 last 392.11) - /unlink 50000 (time 1762167278.22 total 140.49 last 374.01) - /unlink 53799 (time 1762167288.22 total 150.49 last 379.89) - /unlink 57695 (time 1762167298.22 total 160.49 last 389.56) - /unlink 60000 (time 1762167304.04 total 166.31 last 396.45) - /unlink 63639 (time 1762167314.04 total 176.31 last 363.82) total: 64768 /unlink in 179.93 seconds: 359.96 ops/second Waiting for MDT destroys to complete oleg139-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg139-client: oleg139-server: ssh exited with exit code 1 - open/close 2225 (time 1762167338.21 total 10.00 last 222.40) - open/close 4525 (time 1762167348.21 total 20.01 last 229.98) - open/close 7101 (time 1762167358.21 total 30.01 last 257.58) - open/close 9716 (time 1762167368.21 total 40.01 last 261.44) - open/close 11938 (time 1762167378.21 total 50.01 last 222.18) - open/close 14077 (time 1762167388.21 total 60.01 last 213.88) - open/close 16314 (time 1762167398.22 total 70.01 last 223.65) - open/close 18856 (time 1762167408.22 total 80.01 last 254.18) - open/close 20000 (time 1762167412.86 total 84.65 last 246.62) - open/close 22595 (time 1762167422.86 total 94.66 last 259.41) - open/close 25698 (time 1762167432.86 total 104.66 last 310.23) - open/close 28090 (time 1762167442.86 total 114.66 last 239.15) - open/close 30000 (time 1762167449.30 total 121.10 last 296.75) - open/close 33046 (time 1762167459.30 total 131.10 last 304.51) - open/close 36226 (time 1762167469.30 total 141.10 last 317.97) - open/close 39137 (time 1762167479.30 total 151.10 last 291.06) - open/close 40000 (time 1762167481.85 total 153.65 last 338.67) - open/close 43040 (time 1762167491.85 total 163.65 last 303.94) - open/close 45347 (time 1762167501.86 total 173.65 last 230.64) - open/close 47709 (time 1762167511.86 total 183.66 last 236.16) - open/close 50000 (time 1762167519.96 total 191.75 last 282.92) - open/close 52648 (time 1762167529.96 total 201.76 last 264.59) - open/close 55620 (time 1762167539.97 total 211.76 last 297.18) - open/close 58643 (time 1762167549.97 total 221.76 last 302.26) - open/close 60000 (time 1762167553.21 total 225.00 last 418.69) - open/close 63840 (time 1762167563.21 total 235.01 last 383.91) total: 64768 open/close in 237.39 seconds: 272.84 ops/second - /unlink 6083 (time 1762167577.11 total 10.00 last 608.28) - /unlink 10000 (time 1762167582.90 total 15.79 last 676.38) - /unlink 17152 (time 1762167592.90 total 25.79 last 715.13) - /unlink 20000 (time 1762167597.61 total 30.51 last 604.13) - /unlink 26657 (time 1762167607.62 total 40.51 last 665.62) - /unlink 30000 (time 1762167614.04 total 46.93 last 520.57) - /unlink 34658 (time 1762167624.04 total 56.93 last 465.68) - /unlink 40000 (time 1762167632.07 total 64.96 last 665.47) - /unlink 46293 (time 1762167642.07 total 74.96 last 629.26) - /unlink 50000 (time 1762167647.38 total 80.28 last 697.17) - /unlink 56938 (time 1762167657.39 total 90.28 last 693.71) - /unlink 60000 (time 1762167663.75 total 96.64 last 480.99) total: 64768 /unlink in 104.07 seconds: 622.33 ops/second Waiting for MDT destroys to complete oleg139-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg139-client: oleg139-server: ssh exited with exit code 1 - open/close 3160 (time 1762167698.29 total 10.00 last 316.00) - open/close 7234 (time 1762167708.30 total 20.00 last 407.25) - open/close 10000 (time 1762167716.44 total 28.15 last 339.53) - open/close 13362 (time 1762167726.45 total 38.15 last 336.08) - open/close 16504 (time 1762167736.45 total 48.16 last 314.08) - open/close 19137 (time 1762167746.45 total 58.16 last 263.24) - open/close 20000 (time 1762167750.15 total 61.86 last 233.34) - open/close 22628 (time 1762167760.15 total 71.86 last 262.78) - open/close 25542 (time 1762167770.15 total 81.86 last 291.38) - open/close 28324 (time 1762167780.15 total 91.86 last 278.14) - open/close 30000 (time 1762167786.18 total 97.89 last 278.20) - open/close 33039 (time 1762167796.18 total 107.89 last 303.85) - open/close 35710 (time 1762167806.18 total 117.89 last 267.05) - open/close 38757 (time 1762167816.18 total 127.89 last 304.63) - open/close 40000 (time 1762167820.31 total 132.02 last 301.36) - open/close 43535 (time 1762167830.31 total 142.02 last 353.28) - open/close 46728 (time 1762167840.32 total 152.02 last 319.25) - open/close 49311 (time 1762167850.32 total 162.02 last 258.30) - open/close 50000 (time 1762167853.37 total 165.08 last 225.65) - open/close 52372 (time 1762167863.37 total 175.08 last 237.11) - open/close 55426 (time 1762167873.38 total 185.08 last 305.34) - open/close 58584 (time 1762167883.38 total 195.08 last 315.78) - open/close 60000 (time 1762167887.92 total 199.63 last 311.75) - open/close 63542 (time 1762167897.92 total 209.63 last 354.09) total: 64768 open/close in 213.31 seconds: 303.64 ops/second - /unlink 5228 (time 1762167913.11 total 10.00 last 522.71) - /unlink 9469 (time 1762167923.11 total 20.00 last 424.03) - /unlink 14686 (time 1762167933.11 total 30.00 last 521.70) - /unlink 20000 (time 1762167942.46 total 39.36 last 568.18) - /unlink 26223 (time 1762167952.47 total 49.36 last 622.23) - /unlink 30000 (time 1762167958.33 total 55.22 last 643.75) - /unlink 37494 (time 1762167968.33 total 65.23 last 749.35) - /unlink 40000 (time 1762167971.46 total 68.35 last 801.18) - /unlink 47520 (time 1762167981.46 total 78.35 last 751.99) - /unlink 50000 (time 1762167984.82 total 81.71 last 739.41) - /unlink 57278 (time 1762167994.82 total 91.71 last 727.72) - /unlink 60000 (time 1762167999.03 total 95.92 last 646.60) total: 64768 /unlink in 101.80 seconds: 636.24 ops/second Waiting for MDT destroys to complete oleg139-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg139-client: oleg139-server: ssh exited with exit code 1 - open/close 971 (time 1762168036.82 total 10.02 last 96.95) - open/close 2220 (time 1762168046.82 total 20.02 last 124.82) - open/close 3216 (time 1762168056.83 total 30.03 last 99.50) - open/close 4195 (time 1762168066.83 total 40.03 last 97.89) - open/close 5080 (time 1762168076.83 total 50.03 last 88.49) - open/close 6112 (time 1762168086.84 total 60.03 last 103.19) - open/close 7140 (time 1762168096.84 total 70.04 last 102.77) - open/close 8445 (time 1762168106.85 total 80.04 last 130.41) - open/close 9817 (time 1762168116.85 total 90.04 last 137.19) - open/close 11102 (time 1762168126.85 total 100.05 last 128.48) - open/close 12286 (time 1762168136.85 total 110.05 last 118.40) - open/close 13611 (time 1762168146.85 total 120.05 last 132.40) - open/close 15439 (time 1762168156.86 total 130.05 last 182.77) - open/close 17214 (time 1762168166.86 total 140.06 last 177.42) - open/close 18758 (time 1762168176.86 total 150.06 last 154.35) - open/close 20000 (time 1762168185.29 total 158.48 last 147.46) - open/close 21428 (time 1762168195.29 total 168.49 last 142.80) - open/close 22860 (time 1762168205.30 total 178.49 last 143.07) - open/close 24158 (time 1762168215.30 total 188.50 last 129.75) - open/close 25748 (time 1762168225.30 total 198.50 last 158.95) - open/close 27464 (time 1762168235.31 total 208.51 last 171.54) - open/close 28661 (time 1762168245.31 total 218.51 last 119.68) - open/close 30000 (time 1762168254.98 total 228.18 last 138.42) - open/close 31177 (time 1762168265.01 total 238.21 last 117.39) - open/close 32536 (time 1762168275.01 total 248.21 last 135.86) - open/close 33807 (time 1762168285.01 total 258.21 last 127.07) - open/close 35030 (time 1762168295.02 total 268.22 last 122.26) - open/close 36245 (time 1762168305.02 total 278.22 last 121.49) - open/close 37620 (time 1762168315.02 total 288.22 last 137.47) - open/close 39139 (time 1762168325.02 total 298.22 last 151.88) - open/close 40000 (time 1762168331.34 total 304.54 last 136.22) - open/close 41380 (time 1762168341.35 total 314.55 last 137.92) - open/close 42820 (time 1762168351.35 total 324.55 last 143.93) - open/close 43961 (time 1762168361.36 total 334.55 last 114.07) - open/close 45097 (time 1762168371.36 total 344.56 last 113.55) - open/close 46224 (time 1762168381.36 total 354.56 last 112.70) - open/close 47331 (time 1762168391.36 total 364.56 last 110.68) - open/close 48350 (time 1762168401.37 total 374.57 last 101.82) - open/close 49657 (time 1762168411.37 total 384.57 last 130.66) - open/close 50000 (time 1762168413.85 total 387.05 last 138.49) - open/close 51451 (time 1762168423.85 total 397.05 last 145.08) - open/close 52805 (time 1762168433.85 total 407.05 last 135.36) - open/close 54057 (time 1762168443.86 total 417.05 last 125.16) - open/close 55354 (time 1762168453.86 total 427.06 last 129.60) - open/close 56647 (time 1762168463.87 total 437.06 last 129.26) - open/close 57738 (time 1762168473.87 total 447.07 last 109.05) - open/close 58961 (time 1762168483.88 total 457.08 last 122.22) - open/close 60000 (time 1762168488.04 total 461.24 last 249.42) - open/close 61696 (time 1762168498.05 total 471.24 last 169.55) - open/close 63248 (time 1762168508.05 total 481.25 last 155.18) total: 64768 open/close in 488.33 seconds: 132.63 ops/second - /unlink 2916 (time 1762168526.68 total 10.01 last 291.44) - /unlink 5295 (time 1762168536.68 total 20.01 last 237.85) - /unlink 8513 (time 1762168546.69 total 30.01 last 321.76) - /unlink 10000 (time 1762168552.36 total 35.69 last 261.85) - /unlink 13023 (time 1762168562.37 total 45.69 last 302.24) - /unlink 15725 (time 1762168572.37 total 55.69 last 270.16) - /unlink 19101 (time 1762168582.37 total 65.69 last 337.53) - /unlink 20000 (time 1762168585.19 total 68.52 last 318.53) - /unlink 22858 (time 1762168595.20 total 78.52 last 285.67) - /unlink 25134 (time 1762168605.20 total 88.52 last 227.60) - /unlink 27230 (time 1762168615.20 total 98.52 last 209.51) - /unlink 29608 (time 1762168625.20 total 108.53 last 237.75) - /unlink 31967 (time 1762168635.21 total 118.53 last 235.86) - /unlink 34606 (time 1762168645.21 total 128.53 last 263.84) - /unlink 37045 (time 1762168655.21 total 138.53 last 243.89) - /unlink 39414 (time 1762168665.21 total 148.53 last 236.85) - /unlink 40000 (time 1762168667.92 total 151.24 last 216.63) - /unlink 42545 (time 1762168677.92 total 161.24 last 254.48) - /unlink 45173 (time 1762168687.92 total 171.24 last 262.80) - /unlink 47338 (time 1762168697.92 total 181.24 last 216.43) - /unlink 49410 (time 1762168707.92 total 191.25 last 207.10) - /unlink 50000 (time 1762168710.45 total 193.78 last 233.32) - /unlink 52113 (time 1762168720.45 total 203.78 last 211.30) - /unlink 54303 (time 1762168730.46 total 213.78 last 218.90) - /unlink 56517 (time 1762168740.46 total 223.78 last 221.33) - /unlink 58661 (time 1762168750.46 total 233.78 last 214.39) - /unlink 60000 (time 1762168756.42 total 239.75 last 224.58) - /unlink 62696 (time 1762168766.43 total 249.75 last 269.48) total: 64768 /unlink in 257.19 seconds: 251.83 ops/second Waiting for MDT destroys to complete oleg139-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg139-client: oleg139-server: ssh exited with exit code 1 - open/close 1547 (time 1762168794.49 total 10.00 last 154.67) - open/close 3421 (time 1762168804.50 total 20.00 last 187.36) - open/close 4990 (time 1762168814.50 total 30.01 last 156.80) - open/close 6241 (time 1762168824.51 total 40.02 last 125.04) - open/close 7622 (time 1762168834.51 total 50.02 last 138.08) - open/close 8975 (time 1762168844.52 total 60.03 last 135.17) - open/close 10000 (time 1762168851.87 total 67.38 last 139.41) - open/close 11312 (time 1762168861.87 total 77.38 last 131.17) - open/close 12785 (time 1762168871.87 total 87.38 last 147.29) - open/close 14167 (time 1762168881.88 total 97.39 last 138.11) - open/close 15539 (time 1762168891.88 total 107.39 last 137.20) - open/close 16799 (time 1762168901.88 total 117.39 last 125.96) - open/close 18000 (time 1762168911.89 total 127.40 last 120.05) - open/close 19468 (time 1762168921.89 total 137.40 last 146.76) - open/close 20000 (time 1762168924.77 total 140.27 last 184.99) - open/close 21731 (time 1762168934.77 total 150.28 last 173.05) - open/close 23136 (time 1762168944.77 total 160.28 last 140.44) - open/close 24427 (time 1762168954.78 total 170.28 last 129.07) - open/close 25571 (time 1762168964.78 total 180.28 last 114.40) - open/close 26812 (time 1762168974.78 total 190.29 last 124.05) - open/close 28564 (time 1762168984.78 total 200.29 last 175.19) - open/close 30000 (time 1762168994.30 total 209.81 last 150.87) - open/close 31681 (time 1762169004.30 total 219.81 last 168.04) - open/close 33461 (time 1762169014.30 total 229.81 last 178.00) - open/close 34857 (time 1762169024.31 total 239.82 last 139.48) - open/close 36288 (time 1762169034.32 total 249.82 last 143.03) - open/close 37957 (time 1762169044.32 total 259.83 last 166.84) - open/close 39628 (time 1762169054.32 total 269.83 last 167.08) - open/close 41564 (time 1762169064.32 total 279.83 last 193.58) - open/close 43912 (time 1762169074.32 total 289.83 last 234.79) - open/close 46335 (time 1762169084.32 total 299.83 last 242.27) - open/close 48954 (time 1762169094.33 total 309.83 last 261.86) - open/close 50000 (time 1762169099.24 total 314.75 last 212.91) - open/close 51989 (time 1762169109.24 total 324.75 last 198.86) - open/close 53057 (time 1762169119.24 total 334.75 last 106.76) - open/close 53977 (time 1762169129.25 total 344.76 last 91.95) - open/close 54954 (time 1762169139.25 total 354.76 last 97.68) - open/close 55955 (time 1762169149.25 total 364.76 last 100.08) - open/close 57248 (time 1762169159.26 total 374.77 last 129.18) - open/close 58851 (time 1762169169.26 total 384.77 last 160.28) - open/close 60000 (time 1762169175.40 total 390.91 last 187.28) - open/close 61802 (time 1762169185.40 total 400.91 last 180.14) - open/close 63577 (time 1762169195.40 total 410.91 last 177.46) total: 64768 open/close in 419.33 seconds: 154.45 ops/second - /unlink 3328 (time 1762169215.66 total 10.00 last 332.77) - /unlink 6393 (time 1762169225.67 total 20.01 last 306.30) - /unlink 9080 (time 1762169235.67 total 30.01 last 268.68) - /unlink 10000 (time 1762169239.34 total 33.68 last 250.61) - /unlink 12439 (time 1762169249.34 total 43.68 last 243.80) - /unlink 15369 (time 1762169259.34 total 53.68 last 292.96) - /unlink 17337 (time 1762169269.35 total 63.69 last 196.78) - /unlink 19172 (time 1762169279.35 total 73.69 last 183.46) - /unlink 20000 (time 1762169283.44 total 77.78 last 202.32) - /unlink 22890 (time 1762169293.44 total 87.78 last 288.89) - /unlink 25421 (time 1762169303.45 total 97.79 last 252.93) - /unlink 27679 (time 1762169313.45 total 107.80 last 225.70) - /unlink 29668 (time 1762169323.46 total 117.80 last 198.83) - /unlink 32351 (time 1762169333.46 total 127.80 last 268.28) - /unlink 35118 (time 1762169343.46 total 137.80 last 276.65) - /unlink 37967 (time 1762169353.46 total 147.80 last 284.82) - /unlink 40000 (time 1762169360.23 total 154.57 last 300.48) - /unlink 43634 (time 1762169370.23 total 164.57 last 363.34) - /unlink 46819 (time 1762169380.23 total 174.57 last 318.49) - /unlink 50000 (time 1762169388.85 total 183.19 last 369.18) - /unlink 52692 (time 1762169398.85 total 193.19 last 269.08) - /unlink 56173 (time 1762169408.85 total 203.19 last 348.07) - /unlink 60000 (time 1762169418.31 total 212.66 last 404.49) - /unlink 64198 (time 1762169428.32 total 222.66 last 419.72) total: 64768 /unlink in 224.37 seconds: 288.67 ops/second Waiting for MDT destroys to complete oleg139-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg139-client: oleg139-server: ssh exited with exit code 1 fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg139-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg139-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg139-server stop mds service on oleg139-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg139-server stop mds service on oleg139-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg139-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg139-server' oleg139-server: oleg139-server.virtnet: executing unload_modules_local oleg139-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg139-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg139-client: oleg139-client: ssh exited with exit code 2 pdsh@oleg139-client: oleg139-server: ssh exited with exit code 2 pdsh@oleg139-client: oleg139-client: ssh exited with exit code 2 pdsh@oleg139-client: oleg139-server: ssh exited with exit code 2