== conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 16:16:34 (1784232994) start mds service on oleg414-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg414-server: oleg414-server.virtnet: executing set_default_debug -1 all pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg414-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg414-server: oleg414-server.virtnet: executing set_default_debug -1 all pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg414-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg414-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg414-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg414-server: oleg414-server.virtnet: executing set_default_debug -1 all pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1 Started lustre-OST0000 oleg414-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg414-client.virtnet: -o user_xattr,flock 192.168.204.114@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 79992 274 79718 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 79992 255 79737 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 50000 433 49567 1% /mnt/lustre[OST:0] filesystem_summary: 50096 529 49567 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close/unlink 585 (time 1784233045.50 total 10.01 last 58.45) - open/close/unlink 1137 (time 1784233055.51 total 20.01 last 55.16) - open/close/unlink 1659 (time 1784233065.52 total 30.03 last 52.11) - open/close/unlink 2126 (time 1784233075.53 total 40.03 last 46.69) - open/close/unlink 2718 (time 1784233085.53 total 50.04 last 59.19) - open/close/unlink 3321 (time 1784233095.53 total 60.04 last 60.27) - open/close/unlink 3985 (time 1784233105.54 total 70.05 last 66.37) - open/close/unlink 4607 (time 1784233115.54 total 80.05 last 62.20) - open/close/unlink 5304 (time 1784233125.54 total 90.05 last 69.66) - open/close/unlink 6074 (time 1784233135.56 total 100.07 last 76.88) - open/close/unlink 6765 (time 1784233145.57 total 110.08 last 69.03) - open/close/unlink 7589 (time 1784233155.58 total 120.09 last 82.31) - open/close/unlink 8515 (time 1784233165.59 total 130.10 last 92.54) - open/close/unlink 9532 (time 1784233175.59 total 140.10 last 101.66) - open/close/unlink 10000 (time 1784233180.27 total 144.78 last 100.02) - open/close/unlink 11115 (time 1784233190.28 total 154.79 last 111.40) - open/close/unlink 12209 (time 1784233200.29 total 164.79 last 109.33) - open/close/unlink 12987 (time 1784233210.30 total 174.81 last 77.71) - open/close/unlink 13864 (time 1784233220.30 total 184.81 last 87.68) - open/close/unlink 14605 (time 1784233230.31 total 194.82 last 74.01) - open/close/unlink 15328 (time 1784233240.32 total 204.83 last 72.24) - open/close/unlink 16020 (time 1784233250.32 total 214.83 last 69.16) - open/close/unlink 16643 (time 1784233260.35 total 224.86 last 62.16) - open/close/unlink 16968 (time 1784233270.35 total 234.86 last 32.48) - open/close/unlink 17252 (time 1784233280.35 total 244.86 last 28.39) - open/close/unlink 17450 (time 1784233290.44 total 254.95 last 19.62) - open/close/unlink 17723 (time 1784233300.45 total 264.96 last 27.28) - open/close/unlink 18134 (time 1784233310.46 total 274.97 last 41.07) - open/close/unlink 18526 (time 1784233320.47 total 284.98 last 39.17) - open/close/unlink 18872 (time 1784233330.51 total 295.02 last 34.47) - open/close/unlink 19363 (time 1784233340.52 total 305.03 last 49.03) - open/close/unlink 19896 (time 1784233350.53 total 315.04 last 53.23) - open/close/unlink 20000 (time 1784233353.30 total 317.81 last 37.60) - open/close/unlink 20443 (time 1784233363.31 total 327.82 last 44.25) - open/close/unlink 20972 (time 1784233373.33 total 337.83 last 52.83) - open/close/unlink 21515 (time 1784233383.33 total 347.83 last 54.30) - open/close/unlink 22111 (time 1784233393.33 total 357.84 last 59.59) - open/close/unlink 22735 (time 1784233403.33 total 367.84 last 62.37) - open/close/unlink 23319 (time 1784233413.35 total 377.86 last 58.30) - open/close/unlink 23857 (time 1784233423.38 total 387.89 last 53.65) - open/close/unlink 24360 (time 1784233433.39 total 397.90 last 50.25) - open/close/unlink 24806 (time 1784233443.39 total 407.90 last 44.59) - open/close/unlink 25269 (time 1784233453.40 total 417.91 last 46.26) - open/close/unlink 25876 (time 1784233463.41 total 427.92 last 60.61) - open/close/unlink 26441 (time 1784233473.41 total 437.92 last 56.49) - open/close/unlink 27005 (time 1784233483.42 total 447.92 last 56.40) - open/close/unlink 27628 (time 1784233493.42 total 457.92 last 62.30) - open/close/unlink 28303 (time 1784233503.42 total 467.93 last 67.47) - open/close/unlink 28975 (time 1784233513.42 total 477.93 last 67.17) - open/close/unlink 29715 (time 1784233523.43 total 487.94 last 73.97) - open/close/unlink 30000 (time 1784233527.67 total 492.18 last 67.18) - open/close/unlink 30905 (time 1784233537.67 total 502.18 last 90.49) - open/close/unlink 31820 (time 1784233547.69 total 512.20 last 91.37) - open/close/unlink 32865 (time 1784233557.69 total 522.20 last 104.45) - open/close/unlink 33910 (time 1784233567.69 total 532.20 last 104.49) - open/close/unlink 34759 (time 1784233577.70 total 542.21 last 84.84) - open/close/unlink 35542 (time 1784233587.71 total 552.22 last 78.19) - open/cl - open/close/unlink 580 (time 1784233045.61 total 10.02 last 57.88) - open/close/unlink 1133 (time 1784233055.62 total 20.02 last 55.28) - open/close/unlink 1655 (time 1784233065.63 total 30.03 last 52.15) - open/close/unlink 2112 (time 1784233075.64 total 40.04 last 45.65) - open/close/unlink 2701 (time 1784233085.64 total 50.05 last 58.88) - open/close/unlink 3303 (time 1784233095.65 total 60.06 last 60.15) - open/close/unlink 3960 (time 1784233105.65 total 70.06 last 65.69) - open/close/unlink 4576 (time 1784233115.65 total 80.06 last 61.56) - open/close/unlink 5268 (time 1784233125.66 total 90.07 last 69.16) - open/close/unlink 6043 (time 1784233135.68 total 100.09 last 77.33) - open/close/unlink 6728 (time 1784233145.69 total 110.10 last 68.46) - open/close/unlink 7551 (time 1784233155.69 total 120.10 last 82.25) - open/close/unlink 8479 (time 1784233165.70 total 130.10 last 92.79) - open/close/unlink 9485 (time 1784233175.70 total 140.11 last 100.54) - open/close/unlink 10000 (time 1784233180.89 total 145.29 last 99.35) - open/close/unlink 11132 (time 1784233190.89 total 155.30 last 113.11) - open/close/unlink 12215 (time 1784233200.90 total 165.30 last 108.27) - open/close/unlink 12983 (time 1784233210.90 total 175.31 last 76.78) - open/close/unlink 13865 (time 1784233220.91 total 185.31 last 88.14) - open/close/unlink 14607 (time 1784233230.92 total 195.32 last 74.12) - open/close/unlink 15330 (time 1784233240.92 total 205.33 last 72.26) - open/close/unlink 16026 (time 1784233250.92 total 215.33 last 69.58) - open/close/unlink 16639 (time 1784233260.94 total 225.35 last 61.22) - open/close/unlink 16947 (time 1784233270.97 total 235.38 last 30.69) - open/close/unlink 17234 (time 1784233280.98 total 245.39 last 28.68) - open/close/unlink 17435 (time 1784233290.99 total 255.40 last 20.08) - open/close/unlink 17717 (time 1784233301.01 total 265.42 last 28.15) - open/close/unlink 18121 (time 1784233311.01 total 275.42 last 40.39) - open/close/unlink 18509 (time 1784233321.03 total 285.44 last 38.74) - open/close/unlink 18849 (time 1784233331.04 total 295.45 last 33.95) - open/close/unlink 19369 (time 1784233341.05 total 305.46 last 51.97) - open/close/unlink 19897 (time 1784233351.06 total 315.47 last 52.74) - open/close/unlink 20000 (time 1784233353.77 total 318.18 last 37.96) - open/close/unlink 20441 (time 1784233363.78 total 328.19 last 44.07) - open/close/unlink 20970 (time 1784233373.78 total 338.19 last 52.88) - open/close/unlink 21525 (time 1784233383.79 total 348.20 last 55.43) - open/close/unlink 22128 (time 1784233393.79 total 358.20 last 60.30) - open/close/unlink 22754 (time 1784233403.80 total 368.20 last 62.59) - open/close/unlink 23337 (time 1784233413.80 total 378.21 last 58.26) - open/close/unlink 23868 (time 1784233423.82 total 388.23 last 52.99) - open/close/unlink 24370 (time 1784233433.84 total 398.25 last 50.09) - open/close/unlink 24826 (time 1784233443.85 total 408.25 last 45.59) - open/close/unlink 25297 (time 1784233453.85 total 418.26 last 47.09) - open/close/unlink 25894 (time 1784233463.86 total 428.27 last 59.60) - open/close/unlink 26467 (time 1784233473.87 total 438.28 last 57.24) - open/close/unlink 27033 (time 1784233483.88 total 448.28 last 56.59) - open/close/unlink 27649 (time 1784233493.88 total 458.29 last 61.56) - open/close/unlink 28320 (time 1784233503.90 total 468.31 last 66.98) - open/close/unlink 28998 (time 1784233513.91 total 478.32 last 67.74) - open/close/unlink 29723 (time 1784233523.91 total 488.32 last 72.47) - open/close/unlink 30000 (time 1784233527.93 total 492.34 last 68.92) - open/close/unlink 30910 (time 1784233537.95 total 502.35 last 90.87) - open/close/unlink 31834 (time 1784233547.95 total 512.36 last 92.38) - open/close/unlink 32874 (time 1784233557.95 total 522.36 last 103.98) - open/close/unlink 33930 (time 1784233567.95 total 532.36 last 105.55) - open/close/unlink 34782 (time 1784233577.96 total 542.37 last 85.13) - open/close/unlink 35564 (time 1784233587.96 total 552.37 last 78.19) - open/clo - open/close/unlink 581 (time 1784233045.79 total 10.01 last 58.01) - open/close/unlink 1137 (time 1784233055.80 total 20.02 last 55.55) - open/close/unlink 1655 (time 1784233065.83 total 30.05 last 51.66) - open/close/unlink 2109 (time 1784233075.85 total 40.07 last 45.32) - open/close/unlink 2707 (time 1784233085.85 total 50.07 last 59.80) - open/close/unlink 3321 (time 1784233095.85 total 60.07 last 61.38) - open/close/unlink 3975 (time 1784233105.85 total 70.08 last 65.38) - open/close/unlink 4591 (time 1784233115.86 total 80.08 last 61.54) - open/close/unlink 5285 (time 1784233125.87 total 90.09 last 69.36) - open/close/unlink 6057 (time 1784233135.88 total 100.10 last 77.13) - open/close/unlink 6747 (time 1784233145.90 total 110.13 last 68.81) - open/close/unlink 7574 (time 1784233155.92 total 120.14 last 82.59) - open/close/unlink 8497 (time 1784233165.92 total 130.15 last 92.23) - open/close/unlink 9509 (time 1784233175.93 total 140.16 last 101.11) - open/close/unlink 10000 (time 1784233180.88 total 145.10 last 99.23) - open/close/unlink 11130 (time 1784233190.89 total 155.11 last 112.92) - open/close/unlink 12219 (time 1784233200.90 total 165.12 last 108.79) - open/close/unlink 12991 (time 1784233210.90 total 175.12 last 77.19) - open/close/unlink 13870 (time 1784233220.91 total 185.13 last 87.82) - open/close/unlink 14610 (time 1784233230.91 total 195.14 last 73.98) - open/close/unlink 15329 (time 1784233240.92 total 205.14 last 71.87) - open/close/unlink 16020 (time 1784233250.92 total 215.15 last 69.06) - open/close/unlink 16625 (time 1784233260.93 total 225.15 last 60.47) - open/close/unlink 16932 (time 1784233270.97 total 235.19 last 30.58) - open/close/unlink 17219 (time 1784233281.00 total 245.22 last 28.62) - open/close/unlink 17418 (time 1784233291.00 total 255.22 last 19.90) - open/close/unlink 17707 (time 1784233301.01 total 265.23 last 28.87) - open/close/unlink 18109 (time 1784233311.02 total 275.24 last 40.15) - open/close/unlink 18502 (time 1784233321.05 total 285.27 last 39.20) - open/close/unlink 18841 (time 1784233331.08 total 295.30 last 33.80) - open/close/unlink 19362 (time 1784233341.08 total 305.30 last 52.09) - open/close/unlink 19881 (time 1784233351.09 total 315.31 last 51.85) - open/close/unlink 20000 (time 1784233354.13 total 318.36 last 39.08) - open/close/unlink 20450 (time 1784233364.15 total 328.37 last 44.93) - open/close/unlink 20977 (time 1784233374.16 total 338.38 last 52.65) - open/close/unlink 21529 (time 1784233384.18 total 348.41 last 55.07) - open/close/unlink 22128 (time 1784233394.19 total 358.41 last 59.88) - open/close/unlink 22748 (time 1784233404.21 total 368.43 last 61.88) - open/close/unlink 23320 (time 1784233414.21 total 378.43 last 57.18) - open/close/unlink 23848 (time 1784233424.23 total 388.46 last 52.68) - open/close/unlink 24345 (time 1784233434.25 total 398.47 last 49.64) - open/close/unlink 24818 (time 1784233444.26 total 408.48 last 47.24) - open/close/unlink 25299 (time 1784233454.29 total 418.52 last 47.94) - open/close/unlink 25895 (time 1784233464.31 total 428.53 last 59.52) - open/close/unlink 26468 (time 1784233474.32 total 438.54 last 57.24) - open/close/unlink 27047 (time 1784233484.32 total 448.54 last 57.86) - open/close/unlink 27667 (time 1784233494.33 total 458.56 last 61.92) - open/close/unlink 28349 (time 1784233504.34 total 468.56 last 68.19) - open/close/unlink 29025 (time 1784233514.34 total 478.56 last 67.59) - open/close/unlink 29736 (time 1784233524.34 total 488.57 last 71.05) - open/close/unlink 30000 (time 1784233528.15 total 492.38 last 69.32) - open/close/unlink 30911 (time 1784233538.16 total 502.38 last 91.07) - open/close/unlink 31833 (time 1784233548.16 total 512.38 last 92.19) - open/close/unlink 32872 (time 1784233558.16 total 522.38 last 103.87) - open/close/unlink 33927 (time 1784233568.18 total 532.40 last 105.32) - open/close/unlink 34771 (time 1784233578.19 total 542.41 last 84.33) - open/close/unlink 35567 (time 1784233588.19 total 552.42 last 79.53) - open/clo - open/close/unlink 584 (time 1784233045.61 total 10.00 last 58.40) - open/close/unlink 1136 (time 1784233055.62 total 20.00 last 55.20) - open/close/unlink 1662 (time 1784233065.62 total 30.01 last 52.58) - open/close/unlink 2127 (time 1784233075.62 total 40.01 last 46.48) - open/close/unlink 2719 (time 1784233085.63 total 50.02 last 59.14) - open/close/unlink 3324 (time 1784233095.64 total 60.02 last 60.46) - open/close/unlink 3979 (time 1784233105.64 total 70.03 last 65.48) - open/close/unlink 4596 (time 1784233115.64 total 80.03 last 61.68) - open/close/unlink 5293 (time 1784233125.65 total 90.03 last 69.68) - open/close/unlink 6065 (time 1784233135.67 total 100.06 last 77.02) - open/close/unlink 6758 (time 1784233145.68 total 110.06 last 69.26) - open/close/unlink 7582 (time 1784233155.68 total 120.07 last 82.36) - open/close/unlink 8503 (time 1784233165.68 total 130.07 last 92.10) - open/close/unlink 9528 (time 1784233175.69 total 140.07 last 102.45) - open/close/unlink 10000 (time 1784233180.42 total 144.81 last 99.68) - open/close/unlink 11116 (time 1784233190.43 total 154.81 last 111.54) - open/close/unlink 12208 (time 1784233200.43 total 164.82 last 109.16) - open/close/unlink 12974 (time 1784233210.44 total 174.83 last 76.53) - open/close/unlink 13844 (time 1784233220.45 total 184.84 last 86.91) - open/close/unlink 14581 (time 1784233230.46 total 194.85 last 73.62) - open/close/unlink 15312 (time 1784233240.46 total 204.85 last 73.07) - open/close/unlink 16009 (time 1784233250.47 total 214.86 last 69.63) - open/close/unlink 16622 (time 1784233260.48 total 224.86 last 61.28) - open/close/unlink 16949 (time 1784233270.50 total 234.89 last 32.63) - open/close/unlink 17234 (time 1784233280.51 total 244.90 last 28.46) - open/close/unlink 17435 (time 1784233290.54 total 254.93 last 20.04) - open/close/unlink 17711 (time 1784233300.54 total 264.93 last 27.60) - open/close/unlink 18114 (time 1784233310.56 total 274.94 last 40.24) - open/close/unlink 18503 (time 1784233320.56 total 284.95 last 38.89) - open/close/unlink 18850 (time 1784233330.56 total 294.95 last 34.69) - open/close/unlink 19346 (time 1784233340.57 total 304.96 last 49.56) - open/close/unlink 19875 (time 1784233350.58 total 314.97 last 52.83) - open/close/unlink 20000 (time 1784233353.72 total 318.11 last 39.79) - open/close/unlink 20440 (time 1784233363.73 total 328.12 last 43.96) - open/close/unlink 20973 (time 1784233373.75 total 338.13 last 53.22) - open/close/unlink 21528 (time 1784233383.75 total 348.14 last 55.48) - open/close/unlink 22127 (time 1784233393.75 total 358.14 last 59.88) - open/close/unlink 22752 (time 1784233403.76 total 368.14 last 62.50) - open/close/unlink 23335 (time 1784233413.77 total 378.16 last 58.22) - open/close/unlink 23869 (time 1784233423.79 total 388.18 last 53.29) - open/close/unlink 24366 (time 1784233433.80 total 398.19 last 49.65) - open/close/unlink 24828 (time 1784233443.81 total 408.19 last 46.17) - open/close/unlink 25293 (time 1784233453.81 total 418.20 last 46.48) - open/close/unlink 25894 (time 1784233463.82 total 428.20 last 60.07) - open/close/unlink 26465 (time 1784233473.83 total 438.21 last 57.03) - open/close/unlink 27028 (time 1784233483.84 total 448.23 last 56.23) - open/close/unlink 27645 (time 1784233493.85 total 458.24 last 61.64) - open/close/unlink 28313 (time 1784233503.86 total 468.24 last 66.73) - open/close/unlink 28977 (time 1784233513.86 total 478.25 last 66.39) - open/close/unlink 29705 (time 1784233523.86 total 488.25 last 72.78) - open/close/unlink 30000 (time 1784233528.18 total 492.57 last 68.31) - open/close/unlink 30903 (time 1784233538.19 total 502.58 last 90.22) - open/close/unlink 31825 (time 1784233548.19 total 512.58 last 92.16) - open/close/unlink 32867 (time 1784233558.20 total 522.59 last 104.15) - open/close/unlink 33925 (time 1784233568.21 total 532.60 last 105.67) - open/close/unlink 34777 (time 1784233578.22 total 542.60 last 85.15) - open/close/unlink 35571 (time 1784233588.22 total 552.61 last 79.38) - open/clo - open/close/unlink 587 (time 1784233045.79 total 10.01 last 58.66) - open/close/unlink 1141 (time 1784233055.79 total 20.01 last 55.40) - open/close/unlink 1663 (time 1784233065.82 total 30.03 last 52.08) - open/close/unlink 2125 (time 1784233075.82 total 40.03 last 46.18) - open/close/unlink 2721 (time 1784233085.83 total 50.04 last 59.56) - open/close/unlink 3337 (time 1784233095.83 total 60.04 last 61.59) - open/close/unlink 3992 (time 1784233105.85 total 70.06 last 65.38) - open/close/unlink 4605 (time 1784233115.86 total 80.07 last 61.25) - open/close/unlink 5299 (time 1784233125.86 total 90.07 last 69.39) - open/close/unlink 6060 (time 1784233135.86 total 100.08 last 76.06) - open/close/unlink 6753 (time 1784233145.86 total 110.08 last 69.29) - open/close/unlink 7572 (time 1784233155.87 total 120.08 last 81.87) - open/close/unlink 8491 (time 1784233165.88 total 130.09 last 91.83) - open/close/unlink 9500 (time 1784233175.88 total 140.10 last 100.81) - open/close/unlink 10000 (time 1784233180.96 total 145.17 last 98.60) - open/close/unlink 11124 (time 1784233190.97 total 155.19 last 112.21) - open/close/unlink 12205 (time 1784233200.98 total 165.19 last 108.06) - open/close/unlink 12975 (time 1784233210.99 total 175.20 last 76.92) - open/close/unlink 13853 (time 1784233220.99 total 185.20 last 87.80) - open/close/unlink 14592 (time 1784233230.99 total 195.20 last 73.89) - open/close/unlink 15312 (time 1784233241.00 total 205.22 last 71.89) - open/close/unlink 16002 (time 1784233251.02 total 215.23 last 68.90) - open/close/unlink 16604 (time 1784233261.04 total 225.25 last 60.09) - open/close/unlink 16906 (time 1784233271.08 total 235.29 last 30.08) - open/close/unlink 17192 (time 1784233281.09 total 245.30 last 28.58) - open/close/unlink 17389 (time 1784233291.09 total 255.30 last 19.70) - open/close/unlink 17670 (time 1784233301.09 total 265.31 last 28.08) - open/close/unlink 18078 (time 1784233311.11 total 275.32 last 40.75) - open/close/unlink 18468 (time 1784233321.11 total 285.32 last 38.98) - open/close/unlink 18808 (time 1784233331.12 total 295.34 last 33.96) - open/close/unlink 19324 (time 1784233341.14 total 305.35 last 51.54) - open/close/unlink 19852 (time 1784233351.15 total 315.37 last 52.70) - open/close/unlink 20000 (time 1784233354.76 total 318.98 last 41.04) - open/close/unlink 20440 (time 1784233364.78 total 328.99 last 43.92) - open/close/unlink 20978 (time 1784233374.79 total 339.01 last 53.73) - open/close/unlink 21524 (time 1784233384.82 total 349.03 last 54.45) - open/close/unlink 22138 (time 1784233394.83 total 359.04 last 61.35) - open/close/unlink 22769 (time 1784233404.84 total 369.06 last 63.02) - open/close/unlink 23330 (time 1784233414.84 total 379.06 last 56.10) - open/close/unlink 23854 (time 1784233424.86 total 389.08 last 52.30) - open/close/unlink 24342 (time 1784233434.86 total 399.08 last 48.79) - open/close/unlink 24810 (time 1784233444.87 total 409.08 last 46.79) - open/close/unlink 25304 (time 1784233454.87 total 419.08 last 49.38) - open/close/unlink 25888 (time 1784233464.88 total 429.09 last 58.33) - open/close/unlink 26460 (time 1784233474.88 total 439.09 last 57.20) - open/close/unlink 27044 (time 1784233484.88 total 449.10 last 58.39) - open/close/unlink 27654 (time 1784233494.90 total 459.11 last 60.90) - open/close/unlink 28338 (time 1784233504.90 total 469.11 last 68.38) - open/close/unlink 29014 (time 1784233514.90 total 479.12 last 67.58) - open/close/unlink 29724 (time 1784233524.91 total 489.13 last 70.92) - open/close/unlink 30000 (time 1784233528.76 total 492.98 last 71.72) - open/close/unlink 30930 (time 1784233538.77 total 502.98 last 92.94) - open/close/unlink 31862 (time 1784233548.78 total 512.99 last 93.13) - open/close/unlink 32911 (time 1784233558.79 total 523.00 last 104.80) - open/close/unlink 33945 (time 1784233568.79 total 533.00 last 103.37) - open/close/unlink 34804 (time 1784233578.79 total 543.01 last 85.87) - open/close/unlink 35607 (time 1784233588.79 total 553.01 last 80.29) - open/close/unlink 36732 (time 1784233598.20 total 562.42 last 116.45) - open/close/unlink 38116 (time 1784233608.20 total 572.42 last 138.39) - open/close/unlink 39070 (time 1784233618.20 total 582.43 last 95.34) - open/close/unlink 40000 (time 1784233626.10 total 590.32 last 117.86) - open/close/unlink 41306 (time 1784233636.10 total 600.32 last 130.54) - open/close/unlink 42816 (time 1784233646.11 total 610.33 last 150.88) - open/close/unlink 44082 (time 1784233656.11 total 620.33 last 126.57) - open/close/unlink 45217 (time 1784233666.11 total 630.33 last 113.49) - open/close/unlink 46192 (time 1784233676.12 total 640.34 last 97.41) - open/close/unlink 47305 (time 1784233686.12 total 650.34 last 111.28) - open/close/unlink 48439 (time 1784233696.13 total 660.35 last 113.37) - open/close/unlink 49580 (time 1784233706.13 total 670.35 last 114.04) - open/close/unlink 50000 (time 1784233709.54 total 673.76 last 123.29) - open/close/unlink 51286 (time 1784233719.54 total 683.77 last 128.51) - open/close/unlink 52718 (time 1784233729.55 total 693.77 last 143.11) - open/close/unlink 54087 (time 1784233739.55 total 703.78 last 136.87) - open/close/unlink 55278 (time 1784233749.56 total 713.78 last 119.05) - open/close/unlink 56463 (time 1784233759.56 total 723.78 last 118.44) - open/close/unlink 57888 (time 1784233769.56 total 733.79 last 142.48) - open/close/unlink 59336 (time 1784233779.57 total 743.79 last 144.71) - open/close/unlink 60000 (time 1784233784.40 total 748.62 last 137.48) - open/close/unlink 61290 (time 1784233794.40 total 758.62 last 128.98) - open/close/unlink 62627 (time 1784233804.40 total 768.63 last 133.67) - open/close/unlink 64079 (time 1784233814.41 total 778.63 last 145.08) total: 64768 open/close/unlink in 783.09 seconds: 82.71 ops/second se/unlink 36732 (time 1784233597.96 total 562.37 last 116.80) - open/close/unlink 38126 (time 1784233607.97 total 572.38 last 139.29) - open/close/unlink 39082 (time 1784233617.98 total 582.39 last 95.52) - open/close/unlink 40000 (time 1784233625.94 total 590.35 last 115.29) - open/close/unlink 41301 (time 1784233635.95 total 600.35 last 130.04) - open/close/unlink 42819 (time 1784233645.95 total 610.36 last 151.73) - open/close/unlink 44070 (time 1784233655.95 total 620.36 last 125.06) - open/close/unlink 45205 (time 1784233665.96 total 630.37 last 113.45) - open/close/unlink 46200 (time 1784233675.97 total 640.38 last 99.38) - open/close/unlink 47306 (time 1784233685.97 total 650.38 last 110.57) - open/close/unlink 48442 (time 1784233695.98 total 660.39 last 113.50) - open/close/unlink 49583 (time 1784233705.99 total 670.40 last 114.03) - open/close/unlink 50000 (time 1784233709.43 total 673.84 last 120.98) - open/close/unlink 51285 (time 1784233719.44 total 683.84 last 128.49) - open/close/unlink 52705 (time 1784233729.44 total 693.85 last 141.98) - open/close/unlink 54068 (time 1784233739.44 total 703.85 last 136.27) - open/close/unlink 55243 (time 1784233749.44 total 713.85 last 117.45) - open/close/unlink 56425 (time 1784233759.45 total 723.86 last 118.15) - open/close/unlink 57847 (time 1784233769.45 total 733.86 last 142.18) - open/close/unlink 59290 (time 1784233779.46 total 743.86 last 144.18) - open/close/unlink 60000 (time 1784233784.65 total 749.06 last 136.60) - open/close/unlink 61301 (time 1784233794.66 total 759.07 last 130.04) - open/close/unlink 62650 (time 1784233804.66 total 769.07 last 134.89) - open/close/unlink 64094 (time 1784233814.66 total 779.07 last 144.36) total: 64768 open/close/unlink in 783.80 seconds: 82.63 ops/second ose/unlink 36712 (time 1784233597.72 total 562.23 last 116.95) - open/close/unlink 38110 (time 1784233607.72 total 572.23 last 139.74) - open/close/unlink 39057 (time 1784233617.72 total 582.23 last 94.68) - open/close/unlink 40000 (time 1784233625.93 total 590.44 last 114.88) - open/close/unlink 41295 (time 1784233635.93 total 600.44 last 129.49) - open/close/unlink 42825 (time 1784233645.94 total 610.45 last 152.95) - open/close/unlink 44086 (time 1784233655.95 total 620.46 last 125.94) - open/close/unlink 45219 (time 1784233665.96 total 630.47 last 113.21) - open/close/unlink 46209 (time 1784233675.97 total 640.47 last 98.92) - open/close/unlink 47320 (time 1784233685.98 total 650.49 last 110.97) - open/close/unlink 48462 (time 1784233695.98 total 660.49 last 114.16) - open/close/unlink 49604 (time 1784233705.99 total 670.50 last 114.11) - open/close/unlink 50000 (time 1784233709.28 total 673.79 last 120.37) - open/close/unlink 51297 (time 1784233719.28 total 683.79 last 129.69) - open/close/unlink 52702 (time 1784233729.28 total 693.79 last 140.49) - open/close/unlink 54070 (time 1784233739.29 total 703.80 last 136.72) - open/close/unlink 55255 (time 1784233749.29 total 713.80 last 118.47) - open/close/unlink 56432 (time 1784233759.30 total 723.81 last 117.62) - open/close/unlink 57850 (time 1784233769.30 total 733.81 last 141.75) - open/close/unlink 59295 (time 1784233779.31 total 743.82 last 144.39) - open/close/unlink 60000 (time 1784233784.54 total 749.05 last 134.61) - open/close/unlink 61294 (time 1784233794.55 total 759.06 last 129.31) - open/close/unlink 62627 (time 1784233804.55 total 769.06 last 133.26) - open/close/unlink 64062 (time 1784233814.56 total 779.07 last 143.45) total: 64768 open/close/unlink in 784.24 seconds: 82.59 ops/second se/unlink 36770 (time 1784233598.80 total 563.01 last 116.23) - open/close/unlink 38144 (time 1784233608.82 total 573.03 last 137.15) - open/close/unlink 39100 (time 1784233618.83 total 583.04 last 95.49) - open/close/unlink 40000 (time 1784233626.25 total 590.47 last 121.22) - open/close/unlink 41285 (time 1784233636.26 total 600.48 last 128.40) - open/close/unlink 42791 (time 1784233646.26 total 610.48 last 150.57) - open/close/unlink 44069 (time 1784233656.27 total 620.48 last 127.73) - open/close/unlink 45194 (time 1784233666.28 total 630.49 last 112.41) - open/close/unlink 46172 (time 1784233676.28 total 640.49 last 97.79) - open/close/unlink 47281 (time 1784233686.28 total 650.50 last 110.85) - open/close/unlink 48413 (time 1784233696.29 total 660.50 last 113.17) - open/close/unlink 49559 (time 1784233706.29 total 670.50 last 114.59) - open/close/unlink 50000 (time 1784233709.87 total 674.09 last 122.99) - open/close/unlink 51298 (time 1784233719.87 total 684.09 last 129.78) - open/close/unlink 52711 (time 1784233729.90 total 694.12 last 140.92) - open/close/unlink 54082 (time 1784233739.90 total 704.12 last 137.07) - open/close/unlink 55261 (time 1784233749.92 total 714.13 last 117.76) - open/close/unlink 56438 (time 1784233759.92 total 724.13 last 117.64) - open/close/unlink 57870 (time 1784233769.93 total 734.14 last 143.08) - open/close/unlink 59332 (time 1784233779.93 total 744.15 last 146.17) - open/close/unlink 60000 (time 1784233784.92 total 749.13 last 133.92) - open/close/unlink 61323 (time 1784233794.92 total 759.13 last 132.28) - open/close/unlink 62654 (time 1784233804.93 total 769.14 last 133.04) - open/close/unlink 64077 (time 1784233814.93 total 779.15 last 142.19) total: 64768 open/close/unlink in 784.86 seconds: 82.52 ops/second se/unlink 36732 (time 1784233598.23 total 562.62 last 116.01) - open/close/unlink 38111 (time 1784233608.23 total 572.62 last 137.88) - open/close/unlink 39068 (time 1784233618.23 total 582.62 last 95.68) - open/close/unlink 40000 (time 1784233626.16 total 590.55 last 117.56) - open/close/unlink 41293 (time 1784233636.16 total 600.55 last 129.26) - open/close/unlink 42793 (time 1784233646.17 total 610.55 last 149.94) - open/close/unlink 44064 (time 1784233656.17 total 620.56 last 127.08) - open/close/unlink 45196 (time 1784233666.17 total 630.56 last 113.16) - open/close/unlink 46174 (time 1784233676.18 total 640.56 last 97.76) - open/close/unlink 47289 (time 1784233686.18 total 650.56 last 111.50) - open/close/unlink 48421 (time 1784233696.19 total 660.57 last 113.09) - open/close/unlink 49564 (time 1784233706.19 total 670.57 last 114.28) - open/close/unlink 50000 (time 1784233709.76 total 674.14 last 122.15) - open/close/unlink 51287 (time 1784233719.76 total 684.15 last 128.67) - open/close/unlink 52702 (time 1784233729.77 total 694.15 last 141.40) - open/close/unlink 54071 (time 1784233739.77 total 704.16 last 136.86) - open/close/unlink 55247 (time 1784233749.78 total 714.17 last 117.48) - open/close/unlink 56422 (time 1784233759.78 total 724.17 last 117.47) - open/close/unlink 57855 (time 1784233769.79 total 734.17 last 143.24) - open/close/unlink 59299 (time 1784233779.79 total 744.18 last 144.35) - open/close/unlink 60000 (time 1784233785.03 total 749.42 last 133.79) - open/close/unlink 61325 (time 1784233795.03 total 759.42 last 132.47) - open/close/unlink 62652 (time 1784233805.04 total 769.43 last 132.59) - open/close/unlink 64070 (time 1784233815.05 total 779.43 last 141.72) total: 64768 open/close/unlink in 785.25 seconds: 82.48 ops/second Waiting for MDT destroys to complete oleg414-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1 fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg414-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg414-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg414-server stop mds service on oleg414-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg414-server stop mds service on oleg414-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg414-server unloading modules via unload_modules_local on: 'oleg414-server' oleg414-server: oleg414-server.virtnet: executing unload_modules_local oleg414-server: modules unloaded. pdsh@oleg414-client: oleg414-client: ssh exited with exit code 2 pdsh@oleg414-client: oleg414-server: ssh exited with exit code 2 pdsh@oleg414-client: oleg414-client: ssh exited with exit code 2 pdsh@oleg414-client: oleg414-server: ssh exited with exit code 2