== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 14:19:46 (1789150786)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.73 seconds: 13.75 ops/second
 - mkdir 126 (time 1789150808.91 total 10.01 last 12.58)
 - mkdir 261 (time 1789150818.93 total 20.03 last 13.47)
 - mkdir 387 (time 1789150829.02 total 30.12 last 12.49)
 - mkdir 514 (time 1789150839.03 total 40.13 last 12.69)
 - mkdir 650 (time 1789150849.06 total 50.16 last 13.56)
 - mkdir 768 (time 1789150859.18 total 60.29 last 11.65)
 - mkdir 889 (time 1789150869.23 total 70.33 last 12.05)
total: 1000 mkdir in 79.97 seconds: 12.50 ops/second
 - mkdir 128 (time 1789150809.93 total 10.05 last 12.74)
 - mkdir 260 (time 1789150820.01 total 20.12 last 13.10)
 - mkdir 393 (time 1789150830.12 total 30.24 last 13.14)
 - mkdir 526 (time 1789150840.16 total 40.28 last 13.26)
 - mkdir 653 (time 1789150850.27 total 50.39 last 12.56)
 - mkdir 765 (time 1789150860.30 total 60.42 last 11.17)
 - mkdir 888 (time 1789150870.36 total 70.48 last 12.23)
total: 1000 mkdir in 80.46 seconds: 12.43 ops/second
 - mkdir 127 (time 1789150809.83 total 10.01 last 12.69)
 - mkdir 255 (time 1789150819.84 total 20.02 last 12.79)
 - mkdir 384 (time 1789150829.87 total 30.05 last 12.86)
 - mkdir 518 (time 1789150839.89 total 40.08 last 13.37)
 - mkdir 653 (time 1789150849.95 total 50.13 last 13.43)
 - mkdir 764 (time 1789150859.97 total 60.15 last 11.08)
 - mkdir 884 (time 1789150870.09 total 70.27 last 11.86)
 - mkdir 994 (time 1789150880.15 total 80.33 last 10.93)
total: 1000 mkdir in 81.01 seconds: 12.34 ops/second
 - mkdir 128 (time 1789150810.39 total 10.03 last 12.76)
 - mkdir 261 (time 1789150820.47 total 20.10 last 13.20)
 - mkdir 386 (time 1789150830.64 total 30.27 last 12.29)
 - mkdir 524 (time 1789150840.71 total 40.34 last 13.71)
 - mkdir 651 (time 1789150850.77 total 50.40 last 12.63)
 - mkdir 759 (time 1789150860.82 total 60.45 last 10.75)
 - mkdir 879 (time 1789150870.90 total 70.53 last 11.90)
 - mkdir 985 (time 1789150880.95 total 80.58 last 10.55)
total: 1000 mkdir in 82.85 seconds: 12.07 ops/second
 - mkdir 131 (time 1789150810.75 total 10.01 last 13.09)
 - mkdir 262 (time 1789150820.78 total 20.04 last 13.07)
 - mkdir 389 (time 1789150830.81 total 30.07 last 12.66)
 - mkdir 524 (time 1789150840.88 total 40.13 last 13.41)
 - mkdir 649 (time 1789150850.89 total 50.14 last 12.49)
 - mkdir 764 (time 1789150860.96 total 60.22 last 11.41)
 - mkdir 879 (time 1789150870.98 total 70.23 last 11.49)
 - mkdir 983 (time 1789150880.99 total 80.24 last 10.39)
total: 1000 mkdir in 82.55 seconds: 12.11 ops/second
 - mkdir 119 (time 1789150809.78 total 10.13 last 11.75)
 - mkdir 251 (time 1789150819.84 total 20.19 last 13.12)
 - mkdir 372 (time 1789150829.89 total 30.25 last 12.04)
 - mkdir 504 (time 1789150839.90 total 40.25 last 13.19)
 - mkdir 636 (time 1789150849.91 total 50.26 last 13.19)
 - mkdir 748 (time 1789150859.94 total 60.29 last 11.17)
 - mkdir 869 (time 1789150870.05 total 70.40 last 11.97)
 - mkdir 976 (time 1789150880.17 total 80.53 last 10.57)
total: 1000 mkdir in 83.95 seconds: 11.91 ops/second
 - mkdir 127 (time 1789150810.57 total 10.03 last 12.66)
 - mkdir 257 (time 1789150820.64 total 20.10 last 12.90)
 - mkdir 377 (time 1789150830.69 total 30.15 last 11.94)
 - mkdir 514 (time 1789150840.69 total 40.16 last 13.69)
 - mkdir 640 (time 1789150850.73 total 50.20 last 12.55)
 - mkdir 752 (time 1789150860.82 total 60.28 last 11.11)
 - mkdir 876 (time 1789150870.86 total 70.32 last 12.35)
 - mkdir 982 (time 1789150881.00 total 80.46 last 10.45)
total: 1000 mkdir in 83.23 seconds: 12.01 ops/second
 - mkdir 126 (time 1789150810.32 total 10.07 last 12.52)
 - mkdir 255 (time 1789150820.36 total 20.11 last 12.85)
 - mkdir 382 (time 1789150830.36 total 30.11 last 12.70)
 - mkdir 515 (time 1789150840.47 total 40.21 last 13.16)
 - mkdir 644 (time 1789150850.54 total 50.29 last 12.80)
 - mkdir 757 (time 1789150860.69 total 60.44 last 11.14)
 - mkdir 877 (time 1789150870.73 total 70.48 last 11.95)
 - mkdir 980 (time 1789150880.86 total 80.60 last 10.17)
total: 1000 mkdir in 83.72 seconds: 11.94 ops/second
 - mkdir 124 (time 1789150809.68 total 10.03 last 12.36)
 - mkdir 253 (time 1789150819.74 total 20.08 last 12.83)
 - mkdir 370 (time 1789150829.74 total 30.09 last 11.69)
 - mkdir 505 (time 1789150839.77 total 40.12 last 13.46)
 - mkdir 641 (time 1789150849.81 total 50.16 last 13.54)
 - mkdir 753 (time 1789150859.87 total 60.22 last 11.14)
 - mkdir 870 (time 1789150870.00 total 70.35 last 11.55)
 - mkdir 975 (time 1789150880.09 total 80.44 last 10.41)
total: 1000 mkdir in 84.42 seconds: 11.85 ops/second
 - mkdir 123 (time 1789150810.74 total 10.00 last 12.30)
 - mkdir 246 (time 1789150820.80 total 20.07 last 12.22)
 - mkdir 364 (time 1789150830.84 total 30.10 last 11.75)
 - mkdir 495 (time 1789150840.85 total 40.11 last 13.10)
 - mkdir 624 (time 1789150850.88 total 50.14 last 12.86)
 - mkdir 738 (time 1789150860.95 total 60.21 last 11.32)
 - mkdir 852 (time 1789150870.97 total 70.23 last 11.38)
 - mkdir 951 (time 1789150881.13 total 80.39 last 9.75)
total: 1000 mkdir in 86.50 seconds: 11.56 ops/second
Time for file creation 99 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 16 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.64 seconds: 15.68 ops/second
 - mkdir 151 (time 1789151525.45 total 10.03 last 15.05)
 - mkdir 302 (time 1789151535.53 total 20.11 last 14.98)
 - mkdir 444 (time 1789151545.61 total 30.19 last 14.10)
 - mkdir 595 (time 1789151555.61 total 40.19 last 15.10)
 - mkdir 760 (time 1789151565.62 total 50.20 last 16.47)
 - mkdir 900 (time 1789151575.63 total 60.21 last 13.99)
total: 1000 mkdir in 67.96 seconds: 14.71 ops/second
 - mkdir 147 (time 1789151525.52 total 10.05 last 14.63)
 - mkdir 302 (time 1789151535.56 total 20.09 last 15.44)
 - mkdir 445 (time 1789151545.59 total 30.12 last 14.26)
 - mkdir 597 (time 1789151555.61 total 40.14 last 15.17)
 - mkdir 759 (time 1789151565.63 total 50.16 last 16.17)
 - mkdir 900 (time 1789151575.68 total 60.21 last 14.02)
total: 1000 mkdir in 68.15 seconds: 14.67 ops/second
 - mkdir 147 (time 1789151527.22 total 10.03 last 14.65)
 - mkdir 311 (time 1789151537.23 total 20.04 last 16.39)
 - mkdir 461 (time 1789151547.25 total 30.06 last 14.96)
 - mkdir 613 (time 1789151557.27 total 40.08 last 15.18)
 - mkdir 769 (time 1789151567.33 total 50.14 last 15.51)
 - mkdir 910 (time 1789151577.45 total 60.26 last 13.93)
total: 1000 mkdir in 67.20 seconds: 14.88 ops/second
 - mkdir 145 (time 1789151526.75 total 10.02 last 14.47)
 - mkdir 305 (time 1789151536.80 total 20.07 last 15.92)
 - mkdir 447 (time 1789151546.81 total 30.09 last 14.18)
 - mkdir 598 (time 1789151556.88 total 40.15 last 15.01)
 - mkdir 762 (time 1789151566.98 total 50.26 last 16.23)
 - mkdir 901 (time 1789151576.99 total 60.27 last 13.89)
total: 1000 mkdir in 68.51 seconds: 14.60 ops/second
 - mkdir 148 (time 1789151527.46 total 10.03 last 14.75)
 - mkdir 306 (time 1789151537.56 total 20.14 last 15.64)
 - mkdir 448 (time 1789151547.57 total 30.14 last 14.19)
 - mkdir 606 (time 1789151557.60 total 40.18 last 15.75)
 - mkdir 764 (time 1789151567.65 total 50.23 last 15.72)
 - mkdir 904 (time 1789151577.75 total 60.33 last 13.86)
total: 1000 mkdir in 68.00 seconds: 14.71 ops/second
 - mkdir 147 (time 1789151527.58 total 10.02 last 14.68)
 - mkdir 303 (time 1789151537.68 total 20.11 last 15.46)
 - mkdir 446 (time 1789151547.68 total 30.11 last 14.30)
 - mkdir 607 (time 1789151557.69 total 40.12 last 16.08)
 - mkdir 765 (time 1789151567.75 total 50.19 last 15.70)
 - mkdir 903 (time 1789151577.79 total 60.22 last 13.75)
total: 1000 mkdir in 68.44 seconds: 14.61 ops/second
 - mkdir 146 (time 1789151527.32 total 10.00 last 14.59)
 - mkdir 304 (time 1789151537.39 total 20.07 last 15.69)
 - mkdir 451 (time 1789151547.47 total 30.15 last 14.59)
 - mkdir 607 (time 1789151557.49 total 40.18 last 15.56)
 - mkdir 762 (time 1789151567.50 total 50.18 last 15.49)
 - mkdir 897 (time 1789151577.56 total 60.24 last 13.42)
total: 1000 mkdir in 68.76 seconds: 14.54 ops/second
 - mkdir 147 (time 1789151527.73 total 10.06 last 14.62)
 - mkdir 305 (time 1789151537.80 total 20.13 last 15.69)
 - mkdir 447 (time 1789151547.84 total 30.16 last 14.15)
 - mkdir 599 (time 1789151557.86 total 40.19 last 15.17)
 - mkdir 755 (time 1789151567.87 total 50.20 last 15.58)
 - mkdir 896 (time 1789151577.92 total 60.25 last 14.03)
total: 1000 mkdir in 69.03 seconds: 14.49 ops/second
 - mkdir 140 (time 1789151527.40 total 10.05 last 13.93)
 - mkdir 297 (time 1789151537.42 total 20.07 last 15.68)
 - mkdir 441 (time 1789151547.46 total 30.11 last 14.33)
 - mkdir 594 (time 1789151557.47 total 40.12 last 15.29)
 - mkdir 751 (time 1789151567.51 total 50.16 last 15.64)
 - mkdir 890 (time 1789151577.57 total 60.22 last 13.82)
 - mkdir 997 (time 1789151587.66 total 70.31 last 10.61)
total: 1000 mkdir in 70.44 seconds: 14.20 ops/second
 - mkdir 136 (time 1789151527.31 total 10.02 last 13.57)
 - mkdir 291 (time 1789151537.31 total 20.02 last 15.50)
 - mkdir 429 (time 1789151547.34 total 30.05 last 13.77)
 - mkdir 588 (time 1789151557.39 total 40.09 last 15.82)
 - mkdir 744 (time 1789151567.39 total 50.10 last 15.60)
 - mkdir 884 (time 1789151577.43 total 60.14 last 13.94)
 - mkdir 995 (time 1789151587.52 total 70.23 last 11.00)
total: 1000 mkdir in 70.58 seconds: 14.17 ops/second
Time for file creation 82 sec for 1 RPC sent at a time
 - unlinked 0 (time 1789151597 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1789151598 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1789151599 ; total 1 ; last 1)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1789151598 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1789151599 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1789151599 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1789151599 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1789151599 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1789151600 ; total 1 ; last 1)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1789151600 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
Time for file removal 51 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8