== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 21:35:00 (1788485700)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.70 seconds: 14.31 ops/second
 - mkdir 110 (time 1788485722.92 total 10.04 last 10.96)
 - mkdir 263 (time 1788485732.95 total 20.06 last 15.26)
 - mkdir 424 (time 1788485743.03 total 30.15 last 15.96)
 - mkdir 573 (time 1788485753.12 total 40.24 last 14.77)
 - mkdir 719 (time 1788485763.22 total 50.34 last 14.46)
 - mkdir 871 (time 1788485773.28 total 60.40 last 15.10)
total: 1000 mkdir in 67.36 seconds: 14.85 ops/second
 - mkdir 124 (time 1788485724.79 total 10.06 last 12.33)
 - mkdir 288 (time 1788485734.83 total 20.10 last 16.33)
 - mkdir 440 (time 1788485744.88 total 30.14 last 15.14)
 - mkdir 590 (time 1788485754.91 total 40.18 last 14.94)
 - mkdir 736 (time 1788485765.01 total 50.27 last 14.47)
 - mkdir 901 (time 1788485775.03 total 60.30 last 16.46)
total: 1000 mkdir in 66.13 seconds: 15.12 ops/second
 - mkdir 134 (time 1788485726.68 total 10.08 last 13.30)
 - mkdir 294 (time 1788485736.72 total 20.12 last 15.93)
 - mkdir 442 (time 1788485746.79 total 30.19 last 14.70)
 - mkdir 592 (time 1788485756.79 total 40.19 last 15.00)
 - mkdir 757 (time 1788485766.83 total 50.22 last 16.44)
 - mkdir 929 (time 1788485776.83 total 60.23 last 17.20)
total: 1000 mkdir in 65.25 seconds: 15.32 ops/second
 - mkdir 118 (time 1788485724.61 total 10.03 last 11.76)
 - mkdir 281 (time 1788485734.62 total 20.04 last 16.29)
 - mkdir 429 (time 1788485744.64 total 30.06 last 14.76)
 - mkdir 572 (time 1788485754.68 total 40.10 last 14.24)
 - mkdir 723 (time 1788485764.71 total 50.13 last 15.06)
 - mkdir 886 (time 1788485774.75 total 60.17 last 16.23)
total: 1000 mkdir in 67.48 seconds: 14.82 ops/second
 - mkdir 128 (time 1788485725.84 total 10.02 last 12.78)
 - mkdir 290 (time 1788485735.92 total 20.09 last 16.08)
 - mkdir 437 (time 1788485745.94 total 30.11 last 14.67)
 - mkdir 585 (time 1788485756.02 total 40.19 last 14.69)
 - mkdir 735 (time 1788485766.02 total 50.19 last 15.00)
 - mkdir 903 (time 1788485776.05 total 60.23 last 16.74)
total: 1000 mkdir in 66.90 seconds: 14.95 ops/second
 - mkdir 136 (time 1788485727.29 total 10.11 last 13.45)
 - mkdir 298 (time 1788485737.41 total 20.24 last 16.00)
 - mkdir 448 (time 1788485747.43 total 30.26 last 14.97)
 - mkdir 592 (time 1788485757.47 total 40.29 last 14.35)
 - mkdir 749 (time 1788485767.49 total 50.31 last 15.66)
 - mkdir 919 (time 1788485777.51 total 60.33 last 16.98)
total: 1000 mkdir in 67.50 seconds: 14.81 ops/second
 - mkdir 132 (time 1788485725.93 total 10.03 last 13.16)
 - mkdir 289 (time 1788485735.98 total 20.08 last 15.62)
 - mkdir 436 (time 1788485746.03 total 30.12 last 14.64)
 - mkdir 581 (time 1788485756.12 total 40.22 last 14.37)
 - mkdir 729 (time 1788485766.16 total 50.26 last 14.75)
 - mkdir 894 (time 1788485776.23 total 60.32 last 16.39)
total: 1000 mkdir in 69.02 seconds: 14.49 ops/second
 - mkdir 131 (time 1788485726.09 total 10.03 last 13.06)
 - mkdir 288 (time 1788485736.11 total 20.05 last 15.67)
 - mkdir 431 (time 1788485746.19 total 30.12 last 14.19)
 - mkdir 577 (time 1788485756.20 total 40.14 last 14.58)
 - mkdir 723 (time 1788485766.24 total 50.18 last 14.55)
 - mkdir 887 (time 1788485776.24 total 60.18 last 16.40)
total: 1000 mkdir in 69.46 seconds: 14.40 ops/second
 - mkdir 131 (time 1788485726.28 total 10.06 last 13.02)
 - mkdir 285 (time 1788485736.33 total 20.11 last 15.32)
 - mkdir 429 (time 1788485746.44 total 30.22 last 14.25)
 - mkdir 572 (time 1788485756.50 total 40.28 last 14.21)
 - mkdir 725 (time 1788485766.56 total 50.34 last 15.21)
 - mkdir 888 (time 1788485776.58 total 60.36 last 16.27)
 - mkdir 999 (time 1788485786.61 total 70.39 last 11.06)
total: 1000 mkdir in 70.39 seconds: 14.21 ops/second
 - mkdir 104 (time 1788485723.13 total 10.05 last 10.35)
 - mkdir 250 (time 1788485733.26 total 20.18 last 14.41)
 - mkdir 403 (time 1788485743.31 total 30.23 last 15.23)
 - mkdir 544 (time 1788485753.37 total 40.29 last 14.01)
 - mkdir 679 (time 1788485763.41 total 50.33 last 13.44)
 - mkdir 824 (time 1788485773.43 total 60.35 last 14.47)
 - mkdir 969 (time 1788485783.52 total 70.44 last 14.37)
total: 1000 mkdir in 73.70 seconds: 13.57 ops/second
Time for file creation 84 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 15 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.49 seconds: 20.57 ops/second
 - mkdir 146 (time 1788486332.95 total 10.04 last 14.54)
 - mkdir 283 (time 1788486342.97 total 20.06 last 13.67)
 - mkdir 448 (time 1788486353.06 total 30.15 last 16.36)
 - mkdir 590 (time 1788486363.08 total 40.17 last 14.18)
 - mkdir 746 (time 1788486373.09 total 50.18 last 15.57)
 - mkdir 911 (time 1788486383.10 total 60.19 last 16.48)
total: 1000 mkdir in 63.98 seconds: 15.63 ops/second
 - mkdir 147 (time 1788486334.68 total 10.01 last 14.69)
 - mkdir 290 (time 1788486344.73 total 20.05 last 14.24)
 - mkdir 450 (time 1788486354.86 total 30.18 last 15.79)
 - mkdir 602 (time 1788486364.87 total 40.19 last 15.18)
 - mkdir 766 (time 1788486374.90 total 50.23 last 16.35)
 - mkdir 953 (time 1788486384.94 total 60.26 last 18.63)
total: 1000 mkdir in 64.25 seconds: 15.56 ops/second
 - mkdir 162 (time 1788486336.82 total 10.08 last 16.08)
 - mkdir 307 (time 1788486346.88 total 20.13 last 14.42)
 - mkdir 470 (time 1788486356.95 total 30.21 last 16.18)
 - mkdir 631 (time 1788486366.96 total 40.22 last 16.08)
 - mkdir 788 (time 1788486376.99 total 50.25 last 15.66)
 - mkdir 978 (time 1788486387.00 total 60.26 last 18.97)
total: 1000 mkdir in 63.30 seconds: 15.80 ops/second
 - mkdir 154 (time 1788486336.21 total 10.01 last 15.38)
 - mkdir 295 (time 1788486346.25 total 20.05 last 14.05)
 - mkdir 453 (time 1788486356.33 total 30.13 last 15.68)
 - mkdir 614 (time 1788486366.38 total 40.18 last 16.01)
 - mkdir 780 (time 1788486376.47 total 50.27 last 16.45)
 - mkdir 969 (time 1788486386.55 total 60.35 last 18.75)
total: 1000 mkdir in 64.65 seconds: 15.47 ops/second
 - mkdir 160 (time 1788486336.61 total 10.01 last 15.98)
 - mkdir 306 (time 1788486346.63 total 20.04 last 14.56)
 - mkdir 463 (time 1788486356.68 total 30.09 last 15.63)
 - mkdir 622 (time 1788486366.69 total 40.10 last 15.88)
 - mkdir 780 (time 1788486376.78 total 50.19 last 15.66)
 - mkdir 973 (time 1788486386.83 total 60.23 last 19.21)
total: 1000 mkdir in 64.36 seconds: 15.54 ops/second
 - mkdir 161 (time 1788486336.97 total 10.07 last 15.99)
 - mkdir 306 (time 1788486346.98 total 20.08 last 14.49)
 - mkdir 464 (time 1788486357.03 total 30.13 last 15.72)
 - mkdir 624 (time 1788486367.03 total 40.13 last 15.99)
 - mkdir 782 (time 1788486377.10 total 50.20 last 15.69)
 - mkdir 973 (time 1788486387.22 total 60.32 last 18.88)
total: 1000 mkdir in 64.37 seconds: 15.53 ops/second
 - mkdir 156 (time 1788486337.02 total 10.05 last 15.52)
 - mkdir 299 (time 1788486347.04 total 20.08 last 14.26)
 - mkdir 454 (time 1788486357.16 total 30.19 last 15.33)
 - mkdir 618 (time 1788486367.20 total 40.23 last 16.33)
 - mkdir 780 (time 1788486377.21 total 50.24 last 16.18)
 - mkdir 971 (time 1788486387.37 total 60.41 last 18.79)
total: 1000 mkdir in 65.25 seconds: 15.33 ops/second
 - mkdir 155 (time 1788486336.88 total 10.03 last 15.45)
 - mkdir 298 (time 1788486346.94 total 20.09 last 14.22)
 - mkdir 451 (time 1788486356.96 total 30.12 last 15.26)
 - mkdir 610 (time 1788486366.97 total 40.12 last 15.89)
 - mkdir 768 (time 1788486377.01 total 50.16 last 15.74)
 - mkdir 955 (time 1788486387.04 total 60.19 last 18.65)
total: 1000 mkdir in 67.49 seconds: 14.82 ops/second
 - mkdir 143 (time 1788486335.94 total 10.04 last 14.24)
 - mkdir 283 (time 1788486345.96 total 20.06 last 13.97)
 - mkdir 430 (time 1788486355.98 total 30.08 last 14.67)
 - mkdir 590 (time 1788486366.01 total 40.11 last 15.95)
 - mkdir 754 (time 1788486376.03 total 50.13 last 16.37)
 - mkdir 945 (time 1788486386.07 total 60.16 last 19.04)
total: 1000 mkdir in 68.63 seconds: 14.57 ops/second
 - mkdir 153 (time 1788486336.86 total 10.02 last 15.27)
 - mkdir 287 (time 1788486346.91 total 20.07 last 13.33)
 - mkdir 436 (time 1788486356.92 total 30.08 last 14.89)
 - mkdir 595 (time 1788486366.95 total 40.11 last 15.85)
 - mkdir 755 (time 1788486376.97 total 50.13 last 15.96)
 - mkdir 944 (time 1788486387.01 total 60.18 last 18.82)
total: 1000 mkdir in 69.07 seconds: 14.48 ops/second
Time for file creation 81 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788486403 ; total 1 ; last 1)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788486405 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788486406 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1788486406 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1788486407 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1788486406 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1788486406 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1788486407 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1788486407 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1788486407 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
Time for file removal 50 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8