== conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 01:25:06 (1784093106) start mds service on oleg156-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg156-server: oleg156-server.virtnet: executing set_default_debug -1 all pdsh@oleg156-client: oleg156-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg156-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg156-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg156-server: oleg156-server.virtnet: executing set_default_debug -1 all pdsh@oleg156-client: oleg156-server: ssh exited with exit code 1 Started lustre-OST0000 oleg156-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg156-client.virtnet: -o user_xattr,flock 192.168.201.156@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 92553 329 92224 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 12329 425 11904 4% /mnt/lustre[OST:0] filesystem_summary: 12233 329 11904 3% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close/unlink 765 (time 1784093139.60 total 10.01 last 76.44) - open/close/unlink 1407 (time 1784093149.60 total 20.02 last 64.14) - open/close/unlink 2133 (time 1784093159.62 total 30.04 last 72.45) - open/close/unlink 2802 (time 1784093169.65 total 40.06 last 66.76) - open/close/unlink 3383 (time 1784093179.65 total 50.06 last 58.06) - open/close/unlink 4068 (time 1784093189.65 total 60.07 last 68.49) - open/close/unlink 4749 (time 1784093199.66 total 70.07 last 68.08) - open/close/unlink 5465 (time 1784093209.66 total 80.07 last 71.59) - open/close/unlink 6085 (time 1784093219.67 total 90.08 last 61.93) - open/close/unlink 6727 (time 1784093229.68 total 100.09 last 64.16) - open/close/unlink 7199 (time 1784093239.69 total 110.10 last 47.14) - open/close/unlink 7715 (time 1784093249.69 total 120.11 last 51.57) - open/close/unlink 8475 (time 1784093259.70 total 130.12 last 75.93) - open/close/unlink 9212 (time 1784093269.71 total 140.13 last 73.62) - open/close/unlink 10000 (time 1784093278.63 total 149.04 last 88.44) - open/close/unlink 10711 (time 1784093288.63 total 159.04 last 71.08) - open/close/unlink 11419 (time 1784093298.64 total 169.05 last 70.72) - open/close/unlink 12192 (time 1784093308.65 total 179.06 last 77.24) - open/close/unlink 12879 (time 1784093318.65 total 189.07 last 68.65) - open/close/unlink 13601 (time 1784093328.67 total 199.08 last 72.09) - open/close/unlink 14285 (time 1784093338.69 total 209.10 last 68.27) - open/close/unlink 15027 (time 1784093348.69 total 219.10 last 74.18) - open/close/unlink 15895 (time 1784093358.69 total 229.10 last 86.80) - open/close/unlink 16599 (time 1784093368.70 total 239.11 last 70.37) - open/close/unlink 17213 (time 1784093378.71 total 249.12 last 61.34) - open/close/unlink 17804 (time 1784093388.71 total 259.12 last 59.10) - open/close/unlink 18609 (time 1784093398.72 total 269.13 last 80.41) - open/close/unlink 19283 (time 1784093408.72 total 279.13 last 67.38) - open/close/unlink 19951 (time 1784093418.73 total 289.14 last 66.74) - open/close/unlink 20715 (time 1784093428.74 total 299.15 last 76.31) - open/close/unlink 21554 (time 1784093438.75 total 309.16 last 83.85) - open/close/unlink 22347 (time 1784093448.75 total 319.17 last 79.25) - open/close/unlink 23217 (time 1784093458.76 total 329.17 last 86.95) - open/close/unlink 24096 (time 1784093468.76 total 339.17 last 87.88) - open/close/unlink 24896 (time 1784093478.77 total 349.18 last 79.95) - open/close/unlink 25710 (time 1784093488.78 total 359.19 last 81.34) - open/close/unlink 26389 (time 1784093498.78 total 369.19 last 67.86) - open/close/unlink 27128 (time 1784093508.79 total 379.20 last 73.82) - open/close/unlink 28068 (time 1784093518.80 total 389.21 last 93.93) - open/close/unlink 29050 (time 1784093528.81 total 399.23 last 98.06) - open/close/unlink 29763 (time 1784093538.82 total 409.24 last 71.23) - open/close/unlink 30000 (time 1784093542.20 total 412.61 last 70.22) - open/close/unlink 30915 (time 1784093552.20 total 422.61 last 91.49) - open/close/unlink 31654 (time 1784093562.20 total 432.62 last 73.87) - open/close/unlink 32370 (time 1784093572.21 total 442.63 last 71.54) - open/close/unlink 33124 (time 1784093582.22 total 452.63 last 75.36) - open/close/unlink 33842 (time 1784093592.22 total 462.63 last 71.79) - open/close/unlink 34661 (time 1784093602.22 total 472.63 last 81.89) - open/close/unlink 35424 (time 1784093612.22 total 482.63 last 76.29) - open/close/unlink 36378 (time 1784093622.24 total 492.65 last 95.28) - open/close/unlink 37179 (time 1784093632.24 total 502.65 last 80.06) - open/close/unlink 37969 (time 1784093642.25 total 512.66 last 78.95) - open/close/unlink 38615 (time 1784093652.25 total 522.66 last 64.56) - open/close/unlink 39234 (time 1784093662.25 total 532.66 last 61.89) - open/close/unlink 40000 (time 1784093671.97 total 542.38 last 78.86) - open/close/unlink 40805 (time 1784093681.97 total 552.38 last 80.49) - open/close/unlink 41694 (time 1784093691.97 total 562.38 last 88.90) - open/close/un - open/close/unlink 770 (time 1784093139.70 total 10.00 last 76.97) - open/close/unlink 1415 (time 1784093149.71 total 20.02 last 64.39) - open/close/unlink 2133 (time 1784093159.73 total 30.04 last 71.68) - open/close/unlink 2801 (time 1784093169.74 total 40.05 last 66.76) - open/close/unlink 3378 (time 1784093179.75 total 50.06 last 57.64) - open/close/unlink 4066 (time 1784093189.75 total 60.06 last 68.79) - open/close/unlink 4752 (time 1784093199.76 total 70.07 last 68.54) - open/close/unlink 5475 (time 1784093209.77 total 80.08 last 72.23) - open/close/unlink 6097 (time 1784093219.77 total 90.08 last 62.19) - open/close/unlink 6743 (time 1784093229.78 total 100.09 last 64.55) - open/close/unlink 7221 (time 1784093239.79 total 110.10 last 47.74) - open/close/unlink 7737 (time 1784093249.80 total 120.11 last 51.54) - open/close/unlink 8495 (time 1784093259.81 total 130.12 last 75.72) - open/close/unlink 9234 (time 1784093269.81 total 140.12 last 73.90) - open/close/unlink 10000 (time 1784093278.46 total 148.77 last 88.52) - open/close/unlink 10721 (time 1784093288.47 total 158.78 last 72.07) - open/close/unlink 11415 (time 1784093298.48 total 168.79 last 69.32) - open/close/unlink 12177 (time 1784093308.48 total 178.79 last 76.19) - open/close/unlink 12863 (time 1784093318.48 total 188.79 last 68.57) - open/close/unlink 13586 (time 1784093328.49 total 198.80 last 72.24) - open/close/unlink 14264 (time 1784093338.51 total 208.82 last 67.70) - open/close/unlink 15021 (time 1784093348.52 total 218.83 last 75.61) - open/close/unlink 15863 (time 1784093358.52 total 228.83 last 84.17) - open/close/unlink 16590 (time 1784093368.54 total 238.84 last 72.59) - open/close/unlink 17203 (time 1784093378.55 total 248.86 last 61.23) - open/close/unlink 17790 (time 1784093388.55 total 258.86 last 58.70) - open/close/unlink 18603 (time 1784093398.57 total 268.88 last 81.14) - open/close/unlink 19281 (time 1784093408.57 total 278.88 last 67.78) - open/close/unlink 19949 (time 1784093418.58 total 288.89 last 66.73) - open/close/unlink 20710 (time 1784093428.58 total 298.89 last 76.10) - open/close/unlink 21543 (time 1784093438.60 total 308.91 last 83.17) - open/close/unlink 22344 (time 1784093448.61 total 318.92 last 80.01) - open/close/unlink 23213 (time 1784093458.63 total 328.94 last 86.70) - open/close/unlink 24081 (time 1784093468.64 total 338.95 last 86.70) - open/close/unlink 24881 (time 1784093478.65 total 348.95 last 79.99) - open/close/unlink 25695 (time 1784093488.65 total 358.96 last 81.33) - open/close/unlink 26372 (time 1784093498.66 total 368.96 last 67.69) - open/close/unlink 27110 (time 1784093508.67 total 378.97 last 73.72) - open/close/unlink 28042 (time 1784093518.67 total 388.98 last 93.16) - open/close/unlink 29018 (time 1784093528.67 total 398.98 last 97.57) - open/close/unlink 29725 (time 1784093538.68 total 408.99 last 70.64) - open/close/unlink 30000 (time 1784093542.57 total 412.88 last 70.68) - open/close/unlink 30914 (time 1784093552.58 total 422.89 last 91.31) - open/close/unlink 31662 (time 1784093562.58 total 432.89 last 74.80) - open/close/unlink 32365 (time 1784093572.59 total 442.90 last 70.27) - open/close/unlink 33134 (time 1784093582.59 total 452.90 last 76.88) - open/close/unlink 33850 (time 1784093592.60 total 462.91 last 71.52) - open/close/unlink 34671 (time 1784093602.61 total 472.92 last 82.05) - open/close/unlink 35434 (time 1784093612.61 total 482.92 last 76.26) - open/close/unlink 36393 (time 1784093622.62 total 492.92 last 95.87) - open/close/unlink 37178 (time 1784093632.62 total 502.93 last 78.44) - open/close/unlink 37960 (time 1784093642.63 total 512.94 last 78.16) - open/close/unlink 38609 (time 1784093652.64 total 522.95 last 64.82) - open/close/unlink 39237 (time 1784093662.65 total 532.96 last 62.76) - open/close/unlink 40000 (time 1784093672.31 total 542.62 last 78.97) - open/close/unlink 40795 (time 1784093682.32 total 552.62 last 79.45) - open/close/unlink 41698 (time 1784093692.33 total 562.63 last 90.21) - open/close/un - open/close/unlink 778 (time 1784093139.88 total 10.01 last 77.75) - open/close/unlink 1423 (time 1784093149.88 total 20.01 last 64.48) - open/close/unlink 2145 (time 1784093159.89 total 30.02 last 72.14) - open/close/unlink 2814 (time 1784093169.90 total 40.02 last 66.86) - open/close/unlink 3392 (time 1784093179.92 total 50.04 last 57.69) - open/close/unlink 4087 (time 1784093189.92 total 60.05 last 69.47) - open/close/unlink 4770 (time 1784093199.92 total 70.05 last 68.29) - open/close/unlink 5484 (time 1784093209.94 total 80.07 last 71.28) - open/close/unlink 6090 (time 1784093219.95 total 90.07 last 60.55) - open/close/unlink 6739 (time 1784093229.95 total 100.08 last 64.86) - open/close/unlink 7203 (time 1784093239.96 total 110.09 last 46.36) - open/close/unlink 7723 (time 1784093249.96 total 120.09 last 52.00) - open/close/unlink 8484 (time 1784093259.97 total 130.09 last 76.07) - open/close/unlink 9213 (time 1784093269.98 total 140.10 last 72.84) - open/close/unlink 10000 (time 1784093278.92 total 149.04 last 88.03) - open/close/unlink 10709 (time 1784093288.93 total 159.06 last 70.80) - open/close/unlink 11401 (time 1784093298.93 total 169.06 last 69.18) - open/close/unlink 12175 (time 1784093308.94 total 179.07 last 77.34) - open/close/unlink 12880 (time 1784093318.94 total 189.07 last 70.48) - open/close/unlink 13595 (time 1784093328.94 total 199.07 last 71.49) - open/close/unlink 14282 (time 1784093338.96 total 209.09 last 68.57) - open/close/unlink 15036 (time 1784093348.96 total 219.09 last 75.38) - open/close/unlink 15908 (time 1784093358.97 total 229.10 last 87.14) - open/close/unlink 16615 (time 1784093368.99 total 239.11 last 70.59) - open/close/unlink 17229 (time 1784093378.99 total 249.12 last 61.38) - open/close/unlink 17827 (time 1784093388.99 total 259.12 last 59.78) - open/close/unlink 18634 (time 1784093399.00 total 269.12 last 80.66) - open/close/unlink 19307 (time 1784093409.00 total 279.13 last 67.29) - open/close/unlink 19973 (time 1784093419.01 total 289.13 last 66.54) - open/close/unlink 20753 (time 1784093429.02 total 299.14 last 77.94) - open/close/unlink 21583 (time 1784093439.02 total 309.15 last 82.94) - open/close/unlink 22379 (time 1784093449.04 total 319.17 last 79.46) - open/close/unlink 23241 (time 1784093459.05 total 329.18 last 86.10) - open/close/unlink 24099 (time 1784093469.06 total 339.19 last 85.75) - open/close/unlink 24897 (time 1784093479.06 total 349.19 last 79.78) - open/close/unlink 25709 (time 1784093489.06 total 359.19 last 81.19) - open/close/unlink 26393 (time 1784093499.07 total 369.20 last 68.32) - open/close/unlink 27143 (time 1784093509.07 total 379.20 last 75.00) - open/close/unlink 28086 (time 1784093519.08 total 389.20 last 94.28) - open/close/unlink 29058 (time 1784093529.08 total 399.20 last 97.19) - open/close/unlink 29767 (time 1784093539.08 total 409.21 last 70.88) - open/close/unlink 30000 (time 1784093542.35 total 412.47 last 71.37) - open/close/unlink 30910 (time 1784093552.35 total 422.48 last 90.91) - open/close/unlink 31649 (time 1784093562.37 total 432.50 last 73.79) - open/close/unlink 32351 (time 1784093572.37 total 442.50 last 70.20) - open/close/unlink 33105 (time 1784093582.38 total 452.51 last 75.29) - open/close/unlink 33822 (time 1784093592.39 total 462.52 last 71.64) - open/close/unlink 34639 (time 1784093602.40 total 472.53 last 81.65) - open/close/unlink 35405 (time 1784093612.41 total 482.54 last 76.53) - open/close/unlink 36350 (time 1784093622.41 total 492.54 last 94.45) - open/close/unlink 37149 (time 1784093632.43 total 502.55 last 79.80) - open/close/unlink 37924 (time 1784093642.43 total 512.55 last 77.50) - open/close/unlink 38568 (time 1784093652.44 total 522.57 last 64.31) - open/close/unlink 39189 (time 1784093662.44 total 532.57 last 62.10) - open/close/unlink 39973 (time 1784093672.45 total 542.57 last 78.35) - open/close/unlink 40781 (time 1784093682.45 total 552.58 last 80.75) - open/close/unlink 41685 (time 1784093692.46 total 562.59 last 90.33) - open/close/un - open/close/unlink 780 (time 1784093139.70 total 10.01 last 77.96) - open/close/unlink 1433 (time 1784093149.70 total 20.01 last 65.29) - open/close/unlink 2162 (time 1784093159.71 total 30.01 last 72.85) - open/close/unlink 2829 (time 1784093169.71 total 40.01 last 66.69) - open/close/unlink 3398 (time 1784093179.71 total 50.02 last 56.88) - open/close/unlink 4085 (time 1784093189.72 total 60.02 last 68.69) - open/close/unlink 4772 (time 1784093199.72 total 70.03 last 68.66) - open/close/unlink 5483 (time 1784093209.72 total 80.03 last 71.09) - open/close/unlink 6094 (time 1784093219.73 total 90.03 last 61.06) - open/close/unlink 6733 (time 1784093229.74 total 100.05 last 63.82) - open/close/unlink 7209 (time 1784093239.75 total 110.06 last 47.55) - open/close/unlink 7726 (time 1784093249.76 total 120.06 last 51.68) - open/close/unlink 8487 (time 1784093259.76 total 130.07 last 76.04) - open/close/unlink 9226 (time 1784093269.77 total 140.07 last 73.86) - open/close/unlink 10000 (time 1784093278.53 total 148.83 last 88.39) - open/close/unlink 10726 (time 1784093288.54 total 158.84 last 72.51) - open/close/unlink 11433 (time 1784093298.56 total 168.87 last 70.53) - open/close/unlink 12200 (time 1784093308.57 total 178.87 last 76.64) - open/close/unlink 12888 (time 1784093318.58 total 188.88 last 68.75) - open/close/unlink 13620 (time 1784093328.59 total 198.89 last 73.14) - open/close/unlink 14297 (time 1784093338.59 total 208.89 last 67.67) - open/close/unlink 15041 (time 1784093348.59 total 218.89 last 74.40) - open/close/unlink 15896 (time 1784093358.59 total 228.90 last 85.47) - open/close/unlink 16611 (time 1784093368.60 total 238.90 last 71.47) - open/close/unlink 17218 (time 1784093378.61 total 248.91 last 60.66) - open/close/unlink 17815 (time 1784093388.61 total 258.92 last 59.66) - open/close/unlink 18632 (time 1784093398.62 total 268.93 last 81.62) - open/close/unlink 19306 (time 1784093408.63 total 278.93 last 67.36) - open/close/unlink 19967 (time 1784093418.64 total 288.95 last 66.01) - open/close/unlink 20726 (time 1784093428.64 total 298.95 last 75.90) - open/close/unlink 21557 (time 1784093438.65 total 308.96 last 83.01) - open/close/unlink 22361 (time 1784093448.66 total 318.96 last 80.38) - open/close/unlink 23238 (time 1784093458.66 total 328.97 last 87.65) - open/close/unlink 24111 (time 1784093468.68 total 338.98 last 87.18) - open/close/unlink 24906 (time 1784093478.69 total 349.00 last 79.37) - open/close/unlink 25723 (time 1784093488.71 total 359.01 last 81.58) - open/close/unlink 26401 (time 1784093498.72 total 369.02 last 67.73) - open/close/unlink 27133 (time 1784093508.74 total 379.04 last 73.04) - open/close/unlink 28062 (time 1784093518.75 total 389.05 last 92.85) - open/close/unlink 29042 (time 1784093528.75 total 399.06 last 97.93) - open/close/unlink 29752 (time 1784093538.76 total 409.06 last 70.98) - open/close/unlink 30000 (time 1784093542.38 total 412.69 last 68.36) - open/close/unlink 30904 (time 1784093552.39 total 422.69 last 90.33) - open/close/unlink 31640 (time 1784093562.39 total 432.70 last 73.58) - open/close/unlink 32347 (time 1784093572.40 total 442.70 last 70.65) - open/close/unlink 33108 (time 1784093582.41 total 452.71 last 76.07) - open/close/unlink 33817 (time 1784093592.41 total 462.71 last 70.89) - open/close/unlink 34639 (time 1784093602.42 total 472.73 last 82.07) - open/close/unlink 35399 (time 1784093612.42 total 482.73 last 76.00) - open/close/unlink 36350 (time 1784093622.44 total 492.74 last 94.96) - open/close/unlink 37143 (time 1784093632.44 total 502.74 last 79.27) - open/close/unlink 37927 (time 1784093642.45 total 512.75 last 78.32) - open/close/unlink 38577 (time 1784093652.45 total 522.76 last 65.00) - open/close/unlink 39203 (time 1784093662.45 total 532.76 last 62.60) - open/close/unlink 39996 (time 1784093672.46 total 542.76 last 79.23) - open/close/unlink 40791 (time 1784093682.47 total 552.77 last 79.45) - open/close/unlink 41704 (time 1784093692.47 total 562.78 last 91.23) - open/close/un - open/close/unlink 765 (time 1784093139.59 total 10.02 last 76.37) - open/close/unlink 1417 (time 1784093149.59 total 20.02 last 65.18) - open/close/unlink 2145 (time 1784093159.60 total 30.03 last 72.72) - open/close/unlink 2818 (time 1784093169.60 total 40.03 last 67.29) - open/close/unlink 3397 (time 1784093179.61 total 50.04 last 57.85) - open/close/unlink 4085 (time 1784093189.61 total 60.05 last 68.78) - open/close/unlink 4767 (time 1784093199.62 total 70.05 last 68.15) - open/close/unlink 5482 (time 1784093209.63 total 80.06 last 71.46) - open/close/unlink 6094 (time 1784093219.64 total 90.07 last 61.10) - open/close/unlink 6738 (time 1784093229.65 total 100.08 last 64.38) - open/close/unlink 7207 (time 1784093239.65 total 110.08 last 46.89) - open/close/unlink 7715 (time 1784093249.65 total 120.08 last 50.79) - open/close/unlink 8462 (time 1784093259.66 total 130.09 last 74.64) - open/close/unlink 9208 (time 1784093269.66 total 140.09 last 74.57) - open/close/unlink 10000 (time 1784093278.56 total 148.99 last 88.98) - open/close/unlink 10717 (time 1784093288.57 total 159.00 last 71.67) - open/close/unlink 11416 (time 1784093298.57 total 169.00 last 69.90) - open/close/unlink 12185 (time 1784093308.57 total 179.00 last 76.89) - open/close/unlink 12885 (time 1784093318.58 total 189.01 last 69.92) - open/close/unlink 13610 (time 1784093328.58 total 199.01 last 72.47) - open/close/unlink 14288 (time 1784093338.58 total 209.01 last 67.79) - open/close/unlink 15044 (time 1784093348.60 total 219.03 last 75.51) - open/close/unlink 15901 (time 1784093358.60 total 229.03 last 85.69) - open/close/unlink 16615 (time 1784093368.60 total 239.03 last 71.38) - open/close/unlink 17225 (time 1784093378.62 total 249.05 last 60.90) - open/close/unlink 17815 (time 1784093388.63 total 259.06 last 58.92) - open/close/unlink 18630 (time 1784093398.64 total 269.07 last 81.45) - open/close/unlink 19309 (time 1784093408.64 total 279.07 last 67.86) - open/close/unlink 19973 (time 1784093418.64 total 289.07 last 66.40) - open/close/unlink 20745 (time 1784093428.66 total 299.09 last 77.08) - open/close/unlink 21576 (time 1784093438.66 total 309.09 last 83.06) - open/close/unlink 22376 (time 1784093448.66 total 319.09 last 80.00) - open/close/unlink 23236 (time 1784093458.68 total 329.11 last 85.86) - open/close/unlink 24105 (time 1784093468.70 total 339.13 last 86.71) - open/close/unlink 24903 (time 1784093478.71 total 349.14 last 79.69) - open/close/unlink 25711 (time 1784093488.72 total 359.15 last 80.72) - open/close/unlink 26387 (time 1784093498.73 total 369.16 last 67.59) - open/close/unlink 27124 (time 1784093508.73 total 379.16 last 73.70) - open/close/unlink 28052 (time 1784093518.73 total 389.16 last 92.74) - open/close/unlink 29026 (time 1784093528.74 total 399.17 last 97.34) - open/close/unlink 29728 (time 1784093538.74 total 409.17 last 70.17) - open/close/unlink 30000 (time 1784093542.60 total 413.03 last 70.42) - open/close/unlink 30906 (time 1784093552.61 total 423.04 last 90.53) - open/close/unlink 31645 (time 1784093562.62 total 433.05 last 73.88) - open/close/unlink 32347 (time 1784093572.62 total 443.05 last 70.18) - open/close/unlink 33108 (time 1784093582.62 total 453.05 last 76.10) - open/close/unlink 33819 (time 1784093592.63 total 463.06 last 71.04) - open/close/unlink 34641 (time 1784093602.63 total 473.06 last 82.17) - open/close/unlink 35411 (time 1784093612.63 total 483.06 last 76.99) - open/close/unlink 36369 (time 1784093622.64 total 493.07 last 95.74) - open/close/unlink 37155 (time 1784093632.64 total 503.07 last 78.55) - open/close/unlink 37936 (time 1784093642.66 total 513.09 last 78.01) - open/close/unlink 38578 (time 1784093652.66 total 523.09 last 64.19) - open/close/unlink 39212 (time 1784093662.66 total 533.09 last 63.35) - open/close/unlink 39998 (time 1784093672.67 total 543.10 last 78.54) - open/close/unlink 40800 (time 1784093682.68 total 553.11 last 80.14) - open/close/unlink 41715 (time 1784093692.68 total 563.11 last 91.50) - open/close/unlink 42799 (time 1784093701.97 total 572.38 last 110.47) - open/close/unlink 43673 (time 1784093711.97 total 582.38 last 87.39) - open/close/unlink 44642 (time 1784093721.98 total 592.39 last 96.81) - open/close/unlink 45845 (time 1784093731.98 total 602.39 last 120.29) - open/close/unlink 46810 (time 1784093741.99 total 612.40 last 96.46) - open/close/unlink 47819 (time 1784093751.99 total 622.40 last 100.88) - open/close/unlink 49069 (time 1784093761.99 total 632.40 last 124.96) - open/close/unlink 50000 (time 1784093769.01 total 639.42 last 132.64) - open/close/unlink 51100 (time 1784093779.02 total 649.43 last 109.90) - open/close/unlink 52055 (time 1784093789.03 total 659.44 last 95.43) - open/close/unlink 53276 (time 1784093799.03 total 669.44 last 122.10) - open/close/unlink 54508 (time 1784093809.03 total 679.44 last 123.19) - open/close/unlink 55689 (time 1784093819.03 total 689.44 last 118.06) - open/close/unlink 57023 (time 1784093829.03 total 699.45 last 133.35) - open/close/unlink 58332 (time 1784093839.04 total 709.45 last 130.84) - open/close/unlink 59550 (time 1784093849.05 total 719.46 last 121.73) - open/close/unlink 60000 (time 1784093851.76 total 722.17 last 165.67) - open/close/unlink 61192 (time 1784093861.77 total 732.18 last 119.13) - open/close/unlink 62462 (time 1784093871.77 total 742.18 last 126.99) - open/close/unlink 63888 (time 1784093881.77 total 752.18 last 142.56) total: 64768 open/close/unlink in 758.10 seconds: 85.43 ops/second link 42797 (time 1784093702.33 total 572.63 last 109.88) - open/close/unlink 43673 (time 1784093712.33 total 582.64 last 87.54) - open/close/unlink 44639 (time 1784093722.34 total 592.64 last 96.57) - open/close/unlink 45841 (time 1784093732.34 total 602.65 last 120.11) - open/close/unlink 46811 (time 1784093742.35 total 612.66 last 96.91) - open/close/unlink 47826 (time 1784093752.36 total 622.67 last 101.45) - open/close/unlink 49087 (time 1784093762.36 total 632.67 last 126.03) - open/close/unlink 50000 (time 1784093769.32 total 639.63 last 131.22) - open/close/unlink 51086 (time 1784093779.33 total 649.64 last 108.48) - open/close/unlink 52048 (time 1784093789.34 total 659.65 last 96.10) - open/close/unlink 53308 (time 1784093799.36 total 669.67 last 125.83) - open/close/unlink 54547 (time 1784093809.36 total 679.67 last 123.88) - open/close/unlink 55743 (time 1784093819.36 total 689.67 last 119.55) - open/close/unlink 57083 (time 1784093829.37 total 699.68 last 133.94) - open/close/unlink 58408 (time 1784093839.37 total 709.68 last 132.50) - open/close/unlink 59621 (time 1784093849.37 total 719.68 last 121.27) - open/close/unlink 60000 (time 1784093851.67 total 721.98 last 164.77) - open/close/unlink 61199 (time 1784093861.67 total 731.98 last 119.87) - open/close/unlink 62462 (time 1784093871.68 total 741.99 last 126.21) - open/close/unlink 63876 (time 1784093881.68 total 751.99 last 141.35) total: 64768 open/close/unlink in 758.02 seconds: 85.44 ops/second link 42796 (time 1784093702.48 total 572.78 last 109.19) - open/close/unlink 43667 (time 1784093712.50 total 582.81 last 86.87) - open/close/unlink 44643 (time 1784093722.51 total 592.82 last 97.51) - open/close/unlink 45855 (time 1784093732.52 total 602.82 last 121.11) - open/close/unlink 46823 (time 1784093742.52 total 612.83 last 96.77) - open/close/unlink 47835 (time 1784093752.53 total 622.83 last 101.17) - open/close/unlink 49098 (time 1784093762.53 total 632.84 last 126.19) - open/close/unlink 50000 (time 1784093769.38 total 639.68 last 131.74) - open/close/unlink 51087 (time 1784093779.39 total 649.69 last 108.60) - open/close/unlink 52033 (time 1784093789.40 total 659.70 last 94.50) - open/close/unlink 53294 (time 1784093799.40 total 669.71 last 126.08) - open/close/unlink 54536 (time 1784093809.40 total 679.71 last 124.18) - open/close/unlink 55719 (time 1784093819.41 total 689.71 last 118.27) - open/close/unlink 57037 (time 1784093829.42 total 699.72 last 131.68) - open/close/unlink 58364 (time 1784093839.42 total 709.72 last 132.64) - open/close/unlink 59574 (time 1784093849.42 total 719.72 last 121.00) - open/close/unlink 60000 (time 1784093851.96 total 722.26 last 167.73) - open/close/unlink 61181 (time 1784093861.96 total 732.27 last 118.06) - open/close/unlink 62461 (time 1784093871.97 total 742.27 last 127.96) - open/close/unlink 63891 (time 1784093881.97 total 752.27 last 142.93) total: 64768 open/close/unlink in 758.33 seconds: 85.41 ops/second link 42808 (time 1784093702.69 total 573.12 last 109.18) - open/close/unlink 43676 (time 1784093712.69 total 583.12 last 86.78) - open/close/unlink 44647 (time 1784093722.69 total 593.12 last 97.09) - open/close/unlink 45867 (time 1784093732.70 total 603.13 last 121.97) - open/close/unlink 46846 (time 1784093742.71 total 613.14 last 97.82) - open/close/unlink 47858 (time 1784093752.71 total 623.14 last 101.19) - open/close/unlink 49125 (time 1784093762.71 total 633.14 last 126.64) - open/close/unlink 50000 (time 1784093769.41 total 639.84 last 130.55) - open/close/unlink 51090 (time 1784093779.42 total 649.85 last 108.88) - open/close/unlink 52055 (time 1784093789.43 total 659.86 last 96.47) - open/close/unlink 53315 (time 1784093799.43 total 669.86 last 125.97) - open/close/unlink 54555 (time 1784093809.43 total 679.86 last 123.98) - open/close/unlink 55748 (time 1784093819.44 total 689.87 last 119.14) - open/close/unlink 57066 (time 1784093829.46 total 699.89 last 131.64) - open/close/unlink 58388 (time 1784093839.46 total 709.89 last 132.19) - open/close/unlink 59603 (time 1784093849.46 total 719.89 last 121.49) - open/close/unlink 60000 (time 1784093851.86 total 722.29 last 165.51) - open/close/unlink 61180 (time 1784093861.87 total 732.30 last 117.85) - open/close/unlink 62462 (time 1784093871.87 total 742.30 last 128.17) - open/close/unlink 63885 (time 1784093881.87 total 752.30 last 142.29) total: 64768 open/close/unlink in 758.48 seconds: 85.39 ops/second link 42787 (time 1784093702.46 total 572.59 last 110.20) - open/close/unlink 43658 (time 1784093712.47 total 582.60 last 87.03) - open/close/unlink 44633 (time 1784093722.47 total 592.60 last 97.44) - open/close/unlink 45837 (time 1784093732.48 total 602.60 last 120.38) - open/close/unlink 46806 (time 1784093742.48 total 612.61 last 96.84) - open/close/unlink 47801 (time 1784093752.48 total 622.61 last 99.49) - open/close/unlink 49067 (time 1784093762.48 total 632.61 last 126.58) - open/close/unlink 50000 (time 1784093769.54 total 639.67 last 132.24) - open/close/unlink 51074 (time 1784093779.54 total 649.67 last 107.39) - open/close/unlink 52043 (time 1784093789.54 total 659.67 last 96.90) - open/close/unlink 53305 (time 1784093799.54 total 669.67 last 126.19) - open/close/unlink 54542 (time 1784093809.55 total 679.67 last 123.64) - open/close/unlink 55731 (time 1784093819.55 total 689.68 last 118.87) - open/close/unlink 57050 (time 1784093829.56 total 699.69 last 131.74) - open/close/unlink 58392 (time 1784093839.56 total 709.69 last 134.19) - open/close/unlink 59607 (time 1784093849.57 total 719.69 last 121.45) - open/close/unlink 60000 (time 1784093851.94 total 722.06 last 165.82) - open/close/unlink 61184 (time 1784093861.95 total 732.08 last 118.25) - open/close/unlink 62465 (time 1784093871.96 total 742.08 last 128.02) - open/close/unlink 63878 (time 1784093881.96 total 752.09 last 141.25) total: 64768 open/close/unlink in 758.77 seconds: 85.36 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete oleg156-server: cannot open 'lustre-ost2': no such pool pdsh@oleg156-client: oleg156-server: ssh exited with exit code 255 fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg156-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg156-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg156-server stop mds service on oleg156-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg156-server unloading modules via unload_modules_local on: 'oleg156-server' oleg156-server: oleg156-server.virtnet: executing unload_modules_local oleg156-server: modules unloaded. pdsh@oleg156-client: oleg156-client: ssh exited with exit code 2 pdsh@oleg156-client: oleg156-server: ssh exited with exit code 2 pdsh@oleg156-client: oleg156-client: ssh exited with exit code 2 pdsh@oleg156-client: oleg156-server: ssh exited with exit code 2