== conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 22:00:35 (1783562435) start mds service on oleg203-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg203-server: oleg203-server.virtnet: executing set_default_debug -1 all pdsh@oleg203-client: oleg203-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg203-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg203-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg203-server: oleg203-server.virtnet: executing set_default_debug -1 all pdsh@oleg203-client: oleg203-server: ssh exited with exit code 1 Started lustre-OST0000 oleg203-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg203-client.virtnet: -o user_xattr,flock 192.168.202.103@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 92553 329 92224 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 12329 425 11904 4% /mnt/lustre[OST:0] filesystem_summary: 12233 329 11904 3% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close/unlink 577 (time 1783562470.48 total 10.01 last 57.66) - open/close/unlink 1142 (time 1783562480.49 total 20.01 last 56.48) - open/close/unlink 1809 (time 1783562490.50 total 30.02 last 66.64) - open/close/unlink 2468 (time 1783562500.51 total 40.03 last 65.84) - open/close/unlink 3072 (time 1783562510.52 total 50.05 last 60.30) - open/close/unlink 3645 (time 1783562520.53 total 60.05 last 57.29) - open/close/unlink 4302 (time 1783562530.54 total 70.06 last 65.63) - open/close/unlink 4926 (time 1783562540.56 total 80.08 last 62.27) - open/close/unlink 5505 (time 1783562550.58 total 90.10 last 57.77) - open/close/unlink 6058 (time 1783562560.59 total 100.11 last 55.23) - open/close/unlink 6578 (time 1783562570.61 total 110.14 last 51.88) - open/close/unlink 7062 (time 1783562580.63 total 120.15 last 48.33) - open/close/unlink 7595 (time 1783562590.63 total 130.16 last 53.27) - open/close/unlink 8212 (time 1783562600.64 total 140.16 last 61.68) - open/close/unlink 8840 (time 1783562610.65 total 150.17 last 62.73) - open/close/unlink 9462 (time 1783562620.65 total 160.17 last 62.19) - open/close/unlink 10000 (time 1783562628.25 total 167.77 last 70.84) - open/close/unlink 10679 (time 1783562638.26 total 177.78 last 67.83) - open/close/unlink 11346 (time 1783562648.26 total 187.79 last 66.65) - open/close/unlink 11937 (time 1783562658.27 total 197.79 last 59.08) - open/close/unlink 12503 (time 1783562668.28 total 207.80 last 56.52) - open/close/unlink 13141 (time 1783562678.28 total 217.80 last 63.78) - open/close/unlink 13869 (time 1783562688.29 total 227.81 last 72.77) - open/close/unlink 14715 (time 1783562698.29 total 237.81 last 84.56) - open/close/unlink 15461 (time 1783562708.30 total 247.82 last 74.53) - open/close/unlink 16157 (time 1783562718.31 total 257.83 last 69.54) - open/close/unlink 16946 (time 1783562728.33 total 267.85 last 78.76) - open/close/unlink 17627 (time 1783562738.34 total 277.86 last 68.02) - open/close/unlink 18360 (time 1783562748.35 total 287.87 last 73.22) - open/close/unlink 19147 (time 1783562758.35 total 297.87 last 78.68) - open/close/unlink 19835 (time 1783562768.36 total 307.88 last 68.77) - open/close/unlink 20000 (time 1783562770.58 total 310.10 last 74.35) - open/close/unlink 20761 (time 1783562780.58 total 320.10 last 76.10) - open/close/unlink 21435 (time 1783562790.58 total 330.10 last 67.39) - open/close/unlink 22060 (time 1783562800.59 total 340.11 last 62.44) - open/close/unlink 22638 (time 1783562810.60 total 350.12 last 57.75) - open/close/unlink 23164 (time 1783562820.60 total 360.12 last 52.58) - open/close/unlink 23814 (time 1783562830.60 total 370.13 last 64.97) - open/close/unlink 24373 (time 1783562840.61 total 380.13 last 55.88) - open/close/unlink 24975 (time 1783562850.62 total 390.14 last 60.14) - open/close/unlink 25573 (time 1783562860.63 total 400.15 last 59.72) - open/close/unlink 26225 (time 1783562870.65 total 410.17 last 65.08) - open/close/unlink 26818 (time 1783562880.65 total 420.17 last 59.28) - open/close/unlink 27435 (time 1783562890.68 total 430.20 last 61.51) - open/close/unlink 28108 (time 1783562900.69 total 440.21 last 67.25) - open/close/unlink 28864 (time 1783562910.70 total 450.22 last 75.54) - open/close/unlink 29607 (time 1783562920.70 total 460.22 last 74.27) - open/close/unlink 30000 (time 1783562925.23 total 464.75 last 86.81) - open/close/unlink 30879 (time 1783562935.23 total 474.75 last 87.89) - open/close/unlink 31752 (time 1783562945.23 total 484.75 last 87.27) - open/close/unlink 32487 (time 1783562955.24 total 494.76 last 73.43) - open/close/unlink 33222 (time 1783562965.25 total 504.77 last 73.48) - open/close/unlink 34015 (time 1783562975.25 total 514.78 last 79.23) - open/close/unlink 34730 (time 1783562985.26 total 524.78 last 71.45) - open/close/unlink 35573 (time 1783562995.27 total 534.79 last 84.26) - open/close/unlink 36356 (time 1783563005.27 total 544.79 last 78.27) - open/close/unlink 37251 (time 1783563015.27 total 554.80 last 89.46) - open/close/unli - open/close/unlink 584 (time 1783562470.69 total 10.02 last 58.30) - open/close/unlink 1149 (time 1783562480.69 total 20.02 last 56.50) - open/close/unlink 1809 (time 1783562490.70 total 30.02 last 65.98) - open/close/unlink 2453 (time 1783562500.71 total 40.04 last 64.31) - open/close/unlink 3057 (time 1783562510.72 total 50.04 last 60.34) - open/close/unlink 3628 (time 1783562520.73 total 60.05 last 57.04) - open/close/unlink 4283 (time 1783562530.74 total 70.07 last 65.42) - open/close/unlink 4909 (time 1783562540.75 total 80.07 last 62.58) - open/close/unlink 5487 (time 1783562550.76 total 90.09 last 57.71) - open/close/unlink 6039 (time 1783562560.76 total 100.09 last 55.20) - open/close/unlink 6555 (time 1783562570.77 total 110.09 last 51.57) - open/close/unlink 7029 (time 1783562580.78 total 120.10 last 47.35) - open/close/unlink 7565 (time 1783562590.79 total 130.11 last 53.56) - open/close/unlink 8193 (time 1783562600.80 total 140.12 last 62.73) - open/close/unlink 8810 (time 1783562610.80 total 150.12 last 61.69) - open/close/unlink 9436 (time 1783562620.81 total 160.13 last 62.55) - open/close/unlink 10000 (time 1783562628.90 total 168.22 last 69.73) - open/close/unlink 10689 (time 1783562638.90 total 178.23 last 68.85) - open/close/unlink 11349 (time 1783562648.91 total 188.24 last 65.93) - open/close/unlink 11931 (time 1783562658.93 total 198.25 last 58.13) - open/close/unlink 12498 (time 1783562668.93 total 208.26 last 56.66) - open/close/unlink 13160 (time 1783562678.93 total 218.26 last 66.19) - open/close/unlink 13881 (time 1783562688.94 total 228.26 last 72.08) - open/close/unlink 14728 (time 1783562698.94 total 238.27 last 84.64) - open/close/unlink 15475 (time 1783562708.95 total 248.28 last 74.63) - open/close/unlink 16173 (time 1783562718.95 total 258.28 last 69.79) - open/close/unlink 16969 (time 1783562728.96 total 268.29 last 79.54) - open/close/unlink 17644 (time 1783562738.97 total 278.29 last 67.48) - open/close/unlink 18380 (time 1783562748.97 total 288.30 last 73.55) - open/close/unlink 19143 (time 1783562758.98 total 298.31 last 76.24) - open/close/unlink 19838 (time 1783562768.98 total 308.31 last 69.49) - open/close/unlink 20000 (time 1783562771.08 total 310.40 last 77.26) - open/close/unlink 20758 (time 1783562781.08 total 320.40 last 75.80) - open/close/unlink 21446 (time 1783562791.09 total 330.41 last 68.76) - open/close/unlink 22048 (time 1783562801.10 total 340.43 last 60.11) - open/close/unlink 22632 (time 1783562811.11 total 350.43 last 58.36) - open/close/unlink 23160 (time 1783562821.13 total 360.46 last 52.66) - open/close/unlink 23837 (time 1783562831.16 total 370.48 last 67.53) - open/close/unlink 24402 (time 1783562841.16 total 380.49 last 56.48) - open/close/unlink 24982 (time 1783562851.16 total 390.49 last 58.00) - open/close/unlink 25590 (time 1783562861.17 total 400.50 last 60.75) - open/close/unlink 26231 (time 1783562871.17 total 410.50 last 64.09) - open/close/unlink 26828 (time 1783562881.21 total 420.53 last 59.51) - open/close/unlink 27442 (time 1783562891.21 total 430.53 last 61.38) - open/close/unlink 28132 (time 1783562901.21 total 440.54 last 68.99) - open/close/unlink 28893 (time 1783562911.22 total 450.54 last 76.06) - open/close/unlink 29638 (time 1783562921.22 total 460.54 last 74.50) - open/close/unlink 30000 (time 1783562925.41 total 464.74 last 86.27) - open/close/unlink 30882 (time 1783562935.42 total 474.75 last 88.13) - open/close/unlink 31758 (time 1783562945.43 total 484.75 last 87.56) - open/close/unlink 32487 (time 1783562955.43 total 494.75 last 72.89) - open/close/unlink 33236 (time 1783562965.43 total 504.76 last 74.85) - open/close/unlink 34024 (time 1783562975.44 total 514.77 last 78.73) - open/close/unlink 34728 (time 1783562985.46 total 524.78 last 70.29) - open/close/unlink 35572 (time 1783562995.47 total 534.79 last 84.35) - open/close/unlink 36363 (time 1783563005.47 total 544.80 last 79.06) - open/close/unlink 37253 (time 1783563015.49 total 554.81 last 88.86) - open/close/unli - open/close/unlink 575 (time 1783562470.70 total 10.01 last 57.45) - open/close/unlink 1144 (time 1783562480.72 total 20.03 last 56.81) - open/close/unlink 1804 (time 1783562490.72 total 30.03 last 65.94) - open/close/unlink 2453 (time 1783562500.73 total 40.04 last 64.87) - open/close/unlink 3054 (time 1783562510.75 total 50.05 last 60.00) - open/close/unlink 3629 (time 1783562520.76 total 60.07 last 57.44) - open/close/unlink 4285 (time 1783562530.76 total 70.07 last 65.56) - open/close/unlink 4906 (time 1783562540.76 total 80.07 last 62.09) - open/close/unlink 5484 (time 1783562550.78 total 90.09 last 57.71) - open/close/unlink 6035 (time 1783562560.79 total 100.10 last 55.02) - open/close/unlink 6555 (time 1783562570.80 total 110.11 last 51.98) - open/close/unlink 7038 (time 1783562580.80 total 120.11 last 48.27) - open/close/unlink 7563 (time 1783562590.81 total 130.12 last 52.45) - open/close/unlink 8184 (time 1783562600.82 total 140.13 last 62.04) - open/close/unlink 8800 (time 1783562610.82 total 150.13 last 61.60) - open/close/unlink 9430 (time 1783562620.83 total 160.14 last 62.99) - open/close/unlink 10000 (time 1783562628.92 total 168.23 last 70.39) - open/close/unlink 10694 (time 1783562638.93 total 178.24 last 69.33) - open/close/unlink 11344 (time 1783562648.95 total 188.26 last 64.92) - open/close/unlink 11921 (time 1783562658.96 total 198.27 last 57.64) - open/close/unlink 12491 (time 1783562668.96 total 208.27 last 56.99) - open/close/unlink 13148 (time 1783562678.97 total 218.27 last 65.65) - open/close/unlink 13874 (time 1783562688.97 total 228.28 last 72.55) - open/close/unlink 14730 (time 1783562698.97 total 238.28 last 85.59) - open/close/unlink 15475 (time 1783562708.98 total 248.29 last 74.43) - open/close/unlink 16171 (time 1783562718.98 total 258.29 last 69.58) - open/close/unlink 16957 (time 1783562728.99 total 268.30 last 78.54) - open/close/unlink 17626 (time 1783562739.00 total 278.31 last 66.85) - open/close/unlink 18355 (time 1783562749.01 total 288.32 last 72.81) - open/close/unlink 19113 (time 1783562759.03 total 298.34 last 75.69) - open/close/unlink 19813 (time 1783562769.03 total 308.34 last 69.96) - open/close/unlink 20000 (time 1783562771.44 total 310.75 last 77.64) - open/close/unlink 20754 (time 1783562781.44 total 320.75 last 75.39) - open/close/unlink 21444 (time 1783562791.45 total 330.76 last 68.93) - open/close/unlink 22035 (time 1783562801.46 total 340.77 last 59.06) - open/close/unlink 22634 (time 1783562811.46 total 350.77 last 59.89) - open/close/unlink 23144 (time 1783562821.47 total 360.78 last 50.95) - open/close/unlink 23818 (time 1783562831.47 total 370.78 last 67.38) - open/close/unlink 24381 (time 1783562841.49 total 380.80 last 56.22) - open/close/unlink 24961 (time 1783562851.49 total 390.80 last 57.96) - open/close/unlink 25574 (time 1783562861.50 total 400.81 last 61.26) - open/close/unlink 26219 (time 1783562871.51 total 410.82 last 64.44) - open/close/unlink 26820 (time 1783562881.52 total 420.83 last 60.00) - open/close/unlink 27434 (time 1783562891.54 total 430.85 last 61.33) - open/close/unlink 28123 (time 1783562901.54 total 440.85 last 68.88) - open/close/unlink 28900 (time 1783562911.55 total 450.86 last 77.64) - open/close/unlink 29638 (time 1783562921.56 total 460.87 last 73.75) - open/close/unlink 30000 (time 1783562925.63 total 464.94 last 88.86) - open/close/unlink 30876 (time 1783562935.64 total 474.95 last 87.50) - open/close/unlink 31748 (time 1783562945.65 total 484.96 last 87.10) - open/close/unlink 32473 (time 1783562955.67 total 494.98 last 72.40) - open/close/unlink 33224 (time 1783562965.67 total 504.98 last 75.07) - open/close/unlink 33995 (time 1783562975.68 total 514.99 last 77.00) - open/close/unlink 34695 (time 1783562985.70 total 525.01 last 69.89) - open/close/unlink 35543 (time 1783562995.71 total 535.02 last 84.68) - open/close/unlink 36350 (time 1783563005.71 total 545.02 last 80.69) - open/close/unlink 37230 (time 1783563015.72 total 555.03 last 87.94) - open/close/unli - open/close/unlink 592 (time 1783562470.48 total 10.00 last 59.18) - open/close/unlink 1154 (time 1783562480.49 total 20.01 last 56.17) - open/close/unlink 1810 (time 1783562490.49 total 30.01 last 65.59) - open/close/unlink 2462 (time 1783562500.50 total 40.02 last 65.15) - open/close/unlink 3059 (time 1783562510.51 total 50.03 last 59.62) - open/close/unlink 3631 (time 1783562520.53 total 60.05 last 57.10) - open/close/unlink 4289 (time 1783562530.55 total 70.07 last 65.68) - open/close/unlink 4925 (time 1783562540.56 total 80.08 last 63.53) - open/close/unlink 5498 (time 1783562550.57 total 90.09 last 57.23) - open/close/unlink 6050 (time 1783562560.58 total 100.10 last 55.13) - open/close/unlink 6581 (time 1783562570.59 total 110.11 last 53.09) - open/close/unlink 7058 (time 1783562580.60 total 120.11 last 47.66) - open/close/unlink 7592 (time 1783562590.60 total 130.12 last 53.35) - open/close/unlink 8217 (time 1783562600.61 total 140.13 last 62.48) - open/close/unlink 8850 (time 1783562610.62 total 150.14 last 63.25) - open/close/unlink 9470 (time 1783562620.62 total 160.14 last 61.99) - open/close/unlink 10000 (time 1783562628.01 total 167.53 last 71.73) - open/close/unlink 10684 (time 1783562638.02 total 177.54 last 68.28) - open/close/unlink 11349 (time 1783562648.03 total 187.55 last 66.47) - open/close/unlink 11943 (time 1783562658.03 total 197.55 last 59.38) - open/close/unlink 12513 (time 1783562668.04 total 207.56 last 56.98) - open/close/unlink 13150 (time 1783562678.05 total 217.57 last 63.61) - open/close/unlink 13883 (time 1783562688.05 total 227.57 last 73.27) - open/close/unlink 14739 (time 1783562698.06 total 237.58 last 85.53) - open/close/unlink 15489 (time 1783562708.07 total 247.59 last 74.94) - open/close/unlink 16177 (time 1783562718.07 total 257.59 last 68.80) - open/close/unlink 16971 (time 1783562728.07 total 267.59 last 79.39) - open/close/unlink 17644 (time 1783562738.08 total 277.60 last 67.23) - open/close/unlink 18382 (time 1783562748.08 total 287.60 last 73.79) - open/close/unlink 19175 (time 1783562758.09 total 297.61 last 79.22) - open/close/unlink 19864 (time 1783562768.09 total 307.61 last 68.89) - open/close/unlink 20631 (time 1783562778.10 total 317.62 last 76.65) - open/close/unlink 21274 (time 1783562788.11 total 327.63 last 64.25) - open/close/unlink 21949 (time 1783562798.11 total 337.63 last 67.44) - open/close/unlink 22540 (time 1783562808.14 total 347.66 last 58.94) - open/close/unlink 23112 (time 1783562818.15 total 357.67 last 57.16) - open/close/unlink 23683 (time 1783562828.16 total 367.68 last 57.07) - open/close/unlink 24293 (time 1783562838.16 total 377.68 last 60.99) - open/close/unlink 24896 (time 1783562848.16 total 387.68 last 60.27) - open/close/unlink 25482 (time 1783562858.16 total 397.68 last 58.58) - open/close/unlink 26105 (time 1783562868.17 total 407.69 last 62.27) - open/close/unlink 26750 (time 1783562878.18 total 417.70 last 64.46) - open/close/unlink 27353 (time 1783562888.19 total 427.71 last 60.20) - open/close/unlink 28012 (time 1783562898.20 total 437.72 last 65.83) - open/close/unlink 28739 (time 1783562908.20 total 447.72 last 72.69) - open/close/unlink 29475 (time 1783562918.22 total 457.74 last 73.48) - open/close/unlink 30000 (time 1783562924.33 total 463.85 last 85.96) - open/close/unlink 30888 (time 1783562934.33 total 473.85 last 88.75) - open/close/unlink 31755 (time 1783562944.34 total 483.86 last 86.64) - open/close/unlink 32516 (time 1783562954.34 total 493.86 last 76.06) - open/close/unlink 33243 (time 1783562964.36 total 503.88 last 72.62) - open/close/unlink 34053 (time 1783562974.37 total 513.89 last 80.90) - open/close/unlink 34772 (time 1783562984.38 total 523.90 last 71.84) - open/close/unlink 35600 (time 1783562994.38 total 533.90 last 82.77) - open/close/unlink 36370 (time 1783563004.38 total 543.90 last 77.00) - open/close/unlink 37261 (time 1783563014.38 total 553.90 last 89.08) - open/close/unlink 38176 (time 1783563024.39 total 563.91 last 91.40) - open/close/unli - open/close/unlink 593 (time 1783562470.83 total 10.01 last 59.24) - open/close/unlink 1159 (time 1783562480.84 total 20.02 last 56.57) - open/close/unlink 1827 (time 1783562490.84 total 30.02 last 66.76) - open/close/unlink 2474 (time 1783562500.85 total 40.03 last 64.66) - open/close/unlink 3058 (time 1783562510.85 total 50.03 last 58.38) - open/close/unlink 3628 (time 1783562520.85 total 60.03 last 56.99) - open/close/unlink 4287 (time 1783562530.86 total 70.04 last 65.87) - open/close/unlink 4907 (time 1783562540.88 total 80.06 last 61.86) - open/close/unlink 5486 (time 1783562550.89 total 90.07 last 57.84) - open/close/unlink 6036 (time 1783562560.91 total 100.09 last 54.90) - open/close/unlink 6560 (time 1783562570.93 total 110.10 last 52.32) - open/close/unlink 7040 (time 1783562580.94 total 120.12 last 47.94) - open/close/unlink 7571 (time 1783562590.94 total 130.12 last 53.07) - open/close/unlink 8200 (time 1783562600.95 total 140.13 last 62.85) - open/close/unlink 8824 (time 1783562610.96 total 150.14 last 62.31) - open/close/unlink 9448 (time 1783562620.98 total 160.15 last 62.33) - open/close/unlink 10000 (time 1783562628.90 total 168.07 last 69.68) - open/close/unlink 10690 (time 1783562638.91 total 178.09 last 68.91) - open/close/unlink 11345 (time 1783562648.92 total 188.10 last 65.43) - open/close/unlink 11926 (time 1783562658.93 total 198.11 last 58.03) - open/close/unlink 12497 (time 1783562668.94 total 208.12 last 57.04) - open/close/unlink 13162 (time 1783562678.95 total 218.12 last 66.48) - open/close/unlink 13888 (time 1783562688.95 total 228.13 last 72.57) - open/close/unlink 14752 (time 1783562698.96 total 238.14 last 86.35) - open/close/unlink 15494 (time 1783562708.96 total 248.14 last 74.19) - open/close/unlink 16191 (time 1783562718.96 total 258.14 last 69.67) - open/close/unlink 16990 (time 1783562728.98 total 268.15 last 79.81) - open/close/unlink 17665 (time 1783562738.98 total 278.16 last 67.49) - open/close/unlink 18404 (time 1783562748.98 total 288.16 last 73.87) - open/close/unlink 19175 (time 1783562758.99 total 298.17 last 77.00) - open/close/unlink 19876 (time 1783562769.01 total 308.18 last 70.01) - open/close/unlink 20644 (time 1783562779.03 total 318.20 last 76.66) - open/close/unlink 21288 (time 1783562789.03 total 328.21 last 64.34) - open/close/unlink 21969 (time 1783562799.05 total 338.23 last 68.00) - open/close/unlink 22555 (time 1783562809.05 total 348.23 last 58.56) - open/close/unlink 23108 (time 1783562819.08 total 358.26 last 55.15) - open/close/unlink 23710 (time 1783562829.08 total 368.26 last 60.20) - open/close/unlink 24313 (time 1783562839.10 total 378.27 last 60.22) - open/close/unlink 24917 (time 1783562849.11 total 388.29 last 60.30) - open/close/unlink 25521 (time 1783562859.12 total 398.30 last 60.35) - open/close/unlink 26139 (time 1783562869.13 total 408.30 last 61.76) - open/close/unlink 26771 (time 1783562879.13 total 418.31 last 63.19) - open/close/unlink 27379 (time 1783562889.13 total 428.31 last 60.76) - open/close/unlink 28036 (time 1783562899.14 total 438.32 last 65.64) - open/close/unlink 28780 (time 1783562909.15 total 448.33 last 74.34) - open/close/unlink 29503 (time 1783562919.16 total 458.34 last 72.22) - open/close/unlink 30000 (time 1783562924.88 total 464.05 last 86.96) - open/close/unlink 30889 (time 1783562934.89 total 474.06 last 88.81) - open/close/unlink 31765 (time 1783562944.90 total 484.07 last 87.52) - open/close/unlink 32514 (time 1783562954.90 total 494.08 last 74.86) - open/close/unlink 33249 (time 1783562964.90 total 504.08 last 73.48) - open/close/unlink 34045 (time 1783562974.91 total 514.09 last 79.54) - open/close/unlink 34754 (time 1783562984.92 total 524.10 last 70.84) - open/close/unlink 35589 (time 1783562994.92 total 534.10 last 83.46) - open/close/unlink 36370 (time 1783563004.93 total 544.11 last 78.05) - open/close/unlink 37263 (time 1783563014.94 total 554.12 last 89.23) - open/close/unlink 38178 (time 1783563024.94 total 564.12 last 91.46) - open/close/unlink 39302 (time 1783563034.40 total 573.92 last 112.51) - open/close/unlink 40000 (time 1783563041.30 total 580.82 last 101.22) - open/close/unlink 41132 (time 1783563051.31 total 590.83 last 113.04) - open/close/unlink 42236 (time 1783563061.31 total 600.83 last 110.39) - open/close/unlink 43086 (time 1783563071.33 total 610.85 last 84.88) - open/close/unlink 43941 (time 1783563081.34 total 620.86 last 85.42) - open/close/unlink 44772 (time 1783563091.35 total 630.87 last 83.00) - open/close/unlink 45544 (time 1783563101.36 total 640.88 last 77.12) - open/close/unlink 46505 (time 1783563111.37 total 650.88 last 96.03) - open/close/unlink 47609 (time 1783563121.37 total 660.89 last 110.40) - open/close/unlink 48516 (time 1783563131.37 total 670.89 last 90.61) - open/close/unlink 49400 (time 1783563141.38 total 680.90 last 88.32) - open/close/unlink 50000 (time 1783563148.32 total 687.84 last 86.46) - open/close/unlink 50891 (time 1783563158.33 total 697.85 last 89.03) - open/close/unlink 51983 (time 1783563168.34 total 707.86 last 109.16) - open/close/unlink 53005 (time 1783563178.34 total 717.86 last 102.18) - open/close/unlink 53922 (time 1783563188.34 total 727.86 last 91.70) - open/close/unlink 54801 (time 1783563198.34 total 737.86 last 87.87) - open/close/unlink 55860 (time 1783563208.34 total 747.86 last 105.87) - open/close/unlink 56909 (time 1783563218.35 total 757.87 last 104.84) - open/close/unlink 58135 (time 1783563228.36 total 767.88 last 122.46) - open/close/unlink 59396 (time 1783563238.36 total 777.88 last 126.06) - open/close/unlink 60000 (time 1783563242.63 total 782.15 last 141.63) - open/close/unlink 61336 (time 1783563252.64 total 792.16 last 133.52) - open/close/unlink 62594 (time 1783563262.64 total 802.16 last 125.75) - open/close/unlink 63930 (time 1783563272.64 total 812.16 last 133.56) total: 64768 open/close/unlink in 818.35 seconds: 79.14 ops/second nk 39296 (time 1783563034.95 total 574.12 last 111.77) - open/close/unlink 40000 (time 1783563041.67 total 580.84 last 104.75) - open/close/unlink 41133 (time 1783563051.67 total 590.85 last 113.22) - open/close/unlink 42252 (time 1783563061.68 total 600.86 last 111.84) - open/close/unlink 43095 (time 1783563071.68 total 610.86 last 84.28) - open/close/unlink 43925 (time 1783563081.68 total 620.86 last 82.99) - open/close/unlink 44762 (time 1783563091.69 total 630.87 last 83.66) - open/close/unlink 45533 (time 1783563101.69 total 640.87 last 77.09) - open/close/unlink 46491 (time 1783563111.70 total 650.88 last 95.71) - open/close/unlink 47604 (time 1783563121.70 total 660.88 last 111.25) - open/close/unlink 48517 (time 1783563131.70 total 670.88 last 91.29) - open/close/unlink 49423 (time 1783563141.70 total 680.88 last 90.60) - open/close/unlink 50000 (time 1783563148.48 total 687.66 last 85.12) - open/close/unlink 50897 (time 1783563158.49 total 697.67 last 89.65) - open/close/unlink 51978 (time 1783563168.49 total 707.67 last 108.07) - open/close/unlink 53000 (time 1783563178.49 total 717.67 last 102.20) - open/close/unlink 53918 (time 1783563188.50 total 727.68 last 91.70) - open/close/unlink 54788 (time 1783563198.51 total 737.69 last 86.94) - open/close/unlink 55853 (time 1783563208.51 total 747.69 last 106.44) - open/close/unlink 56897 (time 1783563218.52 total 757.70 last 104.35) - open/close/unlink 58115 (time 1783563228.53 total 767.71 last 121.68) - open/close/unlink 59388 (time 1783563238.53 total 777.71 last 127.26) - open/close/unlink 60000 (time 1783563242.87 total 782.05 last 141.14) - open/close/unlink 61329 (time 1783563252.87 total 792.05 last 132.82) - open/close/unlink 62589 (time 1783563262.89 total 802.06 last 125.85) - open/close/unlink 63926 (time 1783563272.89 total 812.07 last 133.64) total: 64768 open/close/unlink in 818.72 seconds: 79.11 ops/second nk 38178 (time 1783563025.28 total 564.80 last 92.65) - open/close/unlink 39270 (time 1783563035.28 total 574.80 last 109.18) - open/close/unlink 40000 (time 1783563042.15 total 581.67 last 106.30) - open/close/unlink 41132 (time 1783563052.15 total 591.67 last 113.18) - open/close/unlink 42257 (time 1783563062.15 total 601.67 last 112.48) - open/close/unlink 43079 (time 1783563072.15 total 611.68 last 82.18) - open/close/unlink 43916 (time 1783563082.17 total 621.69 last 83.57) - open/close/unlink 44756 (time 1783563092.18 total 631.70 last 83.90) - open/close/unlink 45509 (time 1783563102.18 total 641.71 last 75.29) - open/close/unlink 46489 (time 1783563112.19 total 651.71 last 97.97) - open/close/unlink 47581 (time 1783563122.19 total 661.71 last 109.13) - open/close/unlink 48480 (time 1783563132.22 total 671.74 last 89.65) - open/close/unlink 49392 (time 1783563142.22 total 681.74 last 91.18) - open/close/unlink 50000 (time 1783563149.24 total 688.77 last 86.58) - open/close/unlink 50900 (time 1783563159.24 total 698.77 last 90.00) - open/close/unlink 51974 (time 1783563169.25 total 708.77 last 107.37) - open/close/unlink 53015 (time 1783563179.26 total 718.78 last 104.02) - open/close/unlink 53913 (time 1783563189.27 total 728.79 last 89.65) - open/close/unlink 54786 (time 1783563199.27 total 738.80 last 87.29) - open/close/unlink 55848 (time 1783563209.27 total 748.80 last 106.18) - open/close/unlink 56909 (time 1783563219.28 total 758.80 last 106.04) - open/close/unlink 58130 (time 1783563229.28 total 768.80 last 122.09) - open/close/unlink 59442 (time 1783563239.28 total 778.80 last 131.19) - open/close/unlink 60000 (time 1783563243.27 total 782.79 last 140.03) - open/close/unlink 61324 (time 1783563253.27 total 792.79 last 132.34) - open/close/unlink 62570 (time 1783563263.27 total 802.80 last 124.58) - open/close/unlink 63894 (time 1783563273.28 total 812.80 last 132.32) total: 64768 open/close/unlink in 821.91 seconds: 78.80 ops/second nk 38203 (time 1783563025.49 total 564.82 last 94.95) - open/close/unlink 39292 (time 1783563035.49 total 574.82 last 108.88) - open/close/unlink 40000 (time 1783563042.09 total 581.41 last 107.38) - open/close/unlink 41121 (time 1783563052.09 total 591.42 last 112.05) - open/close/unlink 42236 (time 1783563062.10 total 601.42 last 111.40) - open/close/unlink 43068 (time 1783563072.11 total 611.43 last 83.16) - open/close/unlink 43901 (time 1783563082.12 total 621.44 last 83.21) - open/close/unlink 44738 (time 1783563092.12 total 631.44 last 83.67) - open/close/unlink 45503 (time 1783563102.13 total 641.46 last 76.39) - open/close/unlink 46475 (time 1783563112.14 total 651.47 last 97.14) - open/close/unlink 47573 (time 1783563122.15 total 661.47 last 109.73) - open/close/unlink 48471 (time 1783563132.15 total 671.47 last 89.78) - open/close/unlink 49388 (time 1783563142.15 total 681.47 last 91.69) - open/close/unlink 50000 (time 1783563149.22 total 688.54 last 86.59) - open/close/unlink 50899 (time 1783563159.22 total 698.54 last 89.89) - open/close/unlink 51964 (time 1783563169.22 total 708.55 last 106.46) - open/close/unlink 52995 (time 1783563179.23 total 718.55 last 103.03) - open/close/unlink 53901 (time 1783563189.24 total 728.57 last 90.47) - open/close/unlink 54775 (time 1783563199.25 total 738.57 last 87.36) - open/close/unlink 55829 (time 1783563209.25 total 748.58 last 105.34) - open/close/unlink 56879 (time 1783563219.26 total 758.59 last 104.91) - open/close/unlink 58104 (time 1783563229.27 total 768.60 last 122.40) - open/close/unlink 59421 (time 1783563239.27 total 778.60 last 131.67) - open/close/unlink 60000 (time 1783563243.32 total 782.64 last 143.07) - open/close/unlink 61318 (time 1783563253.33 total 792.65 last 131.73) - open/close/unlink 62568 (time 1783563263.33 total 802.66 last 124.92) - open/close/unlink 63888 (time 1783563273.34 total 812.66 last 131.91) total: 64768 open/close/unlink in 821.83 seconds: 78.81 ops/second nk 38181 (time 1783563025.72 total 565.03 last 95.10) - open/close/unlink 39251 (time 1783563035.73 total 575.04 last 106.92) - open/close/unlink 40000 (time 1783563042.54 total 581.85 last 110.00) - open/close/unlink 41120 (time 1783563052.54 total 591.85 last 111.98) - open/close/unlink 42241 (time 1783563062.54 total 601.85 last 112.05) - open/close/unlink 43059 (time 1783563072.55 total 611.86 last 81.75) - open/close/unlink 43890 (time 1783563082.55 total 621.86 last 83.06) - open/close/unlink 44722 (time 1783563092.56 total 631.87 last 83.13) - open/close/unlink 45486 (time 1783563102.57 total 641.88 last 76.33) - open/close/unlink 46473 (time 1783563112.58 total 651.89 last 98.63) - open/close/unlink 47580 (time 1783563122.59 total 661.90 last 110.63) - open/close/unlink 48470 (time 1783563132.59 total 671.90 last 88.93) - open/close/unlink 49377 (time 1783563142.60 total 681.91 last 90.63) - open/close/unlink 50000 (time 1783563149.60 total 688.91 last 88.98) - open/close/unlink 50912 (time 1783563159.61 total 698.92 last 91.14) - open/close/unlink 51977 (time 1783563169.61 total 708.92 last 106.50) - open/close/unlink 52994 (time 1783563179.61 total 718.92 last 101.66) - open/close/unlink 53901 (time 1783563189.62 total 728.93 last 90.60) - open/close/unlink 54781 (time 1783563199.63 total 738.94 last 87.97) - open/close/unlink 55863 (time 1783563209.63 total 748.94 last 108.18) - open/close/unlink 56911 (time 1783563219.63 total 758.94 last 104.76) - open/close/unlink 58119 (time 1783563229.64 total 768.95 last 120.73) - open/close/unlink 59426 (time 1783563239.65 total 778.96 last 130.57) - open/close/unlink 60000 (time 1783563243.57 total 782.88 last 146.26) - open/close/unlink 61329 (time 1783563253.57 total 792.88 last 132.89) - open/close/unlink 62590 (time 1783563263.57 total 802.88 last 126.10) - open/close/unlink 63910 (time 1783563273.58 total 812.89 last 131.94) total: 64768 open/close/unlink in 822.54 seconds: 78.74 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete oleg203-server: cannot open 'lustre-ost2': no such pool pdsh@oleg203-client: oleg203-server: ssh exited with exit code 255 fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg203-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg203-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg203-server stop mds service on oleg203-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg203-server unloading modules via unload_modules_local on: 'oleg203-server' oleg203-server: oleg203-server.virtnet: executing unload_modules_local oleg203-server: modules unloaded. pdsh@oleg203-client: oleg203-client: ssh exited with exit code 2 pdsh@oleg203-client: oleg203-server: ssh exited with exit code 2 pdsh@oleg203-client: oleg203-client: ssh exited with exit code 2 pdsh@oleg203-client: oleg203-server: ssh exited with exit code 2