== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 08:23:19 (1789042999)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.50 seconds: 20.10 ops/second
 - mkdir 182 (time 1789043019.73 total 10.01 last 18.19)
 - mkdir 323 (time 1789043029.75 total 20.03 last 14.07)
 - mkdir 505 (time 1789043039.77 total 30.05 last 18.16)
 - mkdir 650 (time 1789043049.85 total 40.13 last 14.38)
 - mkdir 806 (time 1789043059.89 total 50.17 last 15.53)
 - mkdir 952 (time 1789043069.93 total 60.21 last 14.55)
total: 1000 mkdir in 63.53 seconds: 15.74 ops/second
 - mkdir 173 (time 1789043019.89 total 10.00 last 17.30)
 - mkdir 313 (time 1789043030.01 total 20.12 last 13.84)
 - mkdir 494 (time 1789043040.02 total 30.13 last 18.07)
 - mkdir 633 (time 1789043050.02 total 40.13 last 13.90)
 - mkdir 788 (time 1789043060.12 total 50.23 last 15.36)
 - mkdir 935 (time 1789043070.19 total 60.30 last 14.60)
total: 1000 mkdir in 66.48 seconds: 15.04 ops/second
 - mkdir 185 (time 1789043021.37 total 10.08 last 18.35)
 - mkdir 334 (time 1789043031.38 total 20.09 last 14.88)
 - mkdir 512 (time 1789043041.39 total 30.10 last 17.79)
 - mkdir 653 (time 1789043051.43 total 40.14 last 14.05)
 - mkdir 798 (time 1789043061.46 total 50.17 last 14.45)
 - mkdir 950 (time 1789043071.48 total 60.19 last 15.18)
total: 1000 mkdir in 65.67 seconds: 15.23 ops/second
 - mkdir 174 (time 1789043020.88 total 10.01 last 17.38)
 - mkdir 314 (time 1789043030.93 total 20.05 last 13.94)
 - mkdir 498 (time 1789043040.97 total 30.10 last 18.32)
 - mkdir 636 (time 1789043051.01 total 40.14 last 13.74)
 - mkdir 791 (time 1789043061.07 total 50.20 last 15.41)
 - mkdir 943 (time 1789043071.13 total 60.26 last 15.11)
total: 1000 mkdir in 66.95 seconds: 14.94 ops/second
 - mkdir 177 (time 1789043021.05 total 10.04 last 17.62)
 - mkdir 319 (time 1789043031.07 total 20.06 last 14.18)
 - mkdir 500 (time 1789043041.12 total 30.11 last 18.00)
 - mkdir 639 (time 1789043051.13 total 40.12 last 13.89)
 - mkdir 786 (time 1789043061.17 total 50.16 last 14.63)
 - mkdir 936 (time 1789043071.23 total 60.22 last 14.92)
total: 1000 mkdir in 67.20 seconds: 14.88 ops/second
 - mkdir 176 (time 1789043020.25 total 10.00 last 17.60)
 - mkdir 315 (time 1789043030.26 total 20.01 last 13.89)
 - mkdir 497 (time 1789043040.26 total 30.02 last 18.18)
 - mkdir 633 (time 1789043050.30 total 40.05 last 13.55)
 - mkdir 787 (time 1789043060.33 total 50.08 last 15.36)
 - mkdir 926 (time 1789043070.36 total 60.11 last 13.86)
total: 1000 mkdir in 68.02 seconds: 14.70 ops/second
 - mkdir 179 (time 1789043021.41 total 10.03 last 17.84)
 - mkdir 324 (time 1789043031.43 total 20.06 last 14.46)
 - mkdir 502 (time 1789043041.54 total 30.16 last 17.62)
 - mkdir 647 (time 1789043051.54 total 40.16 last 14.50)
 - mkdir 795 (time 1789043061.56 total 50.19 last 14.76)
 - mkdir 943 (time 1789043071.57 total 60.20 last 14.79)
total: 1000 mkdir in 67.23 seconds: 14.87 ops/second
 - mkdir 177 (time 1789043021.16 total 10.03 last 17.65)
 - mkdir 321 (time 1789043031.20 total 20.07 last 14.34)
 - mkdir 500 (time 1789043041.22 total 30.09 last 17.86)
 - mkdir 639 (time 1789043051.23 total 40.09 last 13.89)
 - mkdir 790 (time 1789043061.25 total 50.12 last 15.06)
 - mkdir 936 (time 1789043071.32 total 60.18 last 14.51)
total: 1000 mkdir in 67.68 seconds: 14.78 ops/second
 - mkdir 166 (time 1789043021.18 total 10.01 last 16.58)
 - mkdir 307 (time 1789043031.20 total 20.04 last 14.06)
 - mkdir 485 (time 1789043041.22 total 30.06 last 17.76)
 - mkdir 627 (time 1789043051.23 total 40.07 last 14.18)
 - mkdir 772 (time 1789043061.28 total 50.11 last 14.44)
 - mkdir 918 (time 1789043071.30 total 60.14 last 14.56)
total: 1000 mkdir in 69.65 seconds: 14.36 ops/second
 - mkdir 164 (time 1789043020.63 total 10.01 last 16.39)
 - mkdir 296 (time 1789043030.63 total 20.01 last 13.19)
 - mkdir 467 (time 1789043040.66 total 30.04 last 17.05)
 - mkdir 607 (time 1789043050.68 total 40.05 last 13.98)
 - mkdir 762 (time 1789043060.74 total 50.12 last 15.41)
 - mkdir 909 (time 1789043070.77 total 60.15 last 14.65)
 - mkdir 993 (time 1789043080.80 total 70.18 last 8.37)
total: 1000 mkdir in 70.76 seconds: 14.13 ops/second
Time for file creation 81 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 14 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.46 seconds: 21.94 ops/second
 - mkdir 160 (time 1789043625.48 total 10.04 last 15.93)
 - mkdir 330 (time 1789043635.56 total 20.12 last 16.88)
 - mkdir 507 (time 1789043645.56 total 30.12 last 17.69)
 - mkdir 684 (time 1789043655.59 total 40.15 last 17.65)
 - mkdir 864 (time 1789043665.63 total 50.19 last 17.92)
 - mkdir 996 (time 1789043675.66 total 60.22 last 13.16)
total: 1000 mkdir in 60.37 seconds: 16.56 ops/second
 - mkdir 159 (time 1789043625.59 total 10.00 last 15.89)
 - mkdir 331 (time 1789043635.66 total 20.07 last 17.08)
 - mkdir 505 (time 1789043645.69 total 30.11 last 17.34)
 - mkdir 683 (time 1789043655.72 total 40.14 last 17.75)
 - mkdir 859 (time 1789043665.79 total 50.20 last 17.49)
 - mkdir 996 (time 1789043675.81 total 60.23 last 13.67)
total: 1000 mkdir in 60.47 seconds: 16.54 ops/second
 - mkdir 164 (time 1789043626.50 total 10.02 last 16.37)
 - mkdir 336 (time 1789043636.56 total 20.08 last 17.10)
 - mkdir 521 (time 1789043646.60 total 30.12 last 18.43)
 - mkdir 707 (time 1789043656.65 total 40.16 last 18.51)
 - mkdir 873 (time 1789043666.70 total 50.22 last 16.51)
total: 1000 mkdir in 59.85 seconds: 16.71 ops/second
 - mkdir 152 (time 1789043624.94 total 10.02 last 15.17)
 - mkdir 326 (time 1789043634.95 total 20.04 last 17.37)
 - mkdir 497 (time 1789043645.03 total 30.11 last 16.97)
 - mkdir 667 (time 1789043655.06 total 40.14 last 16.95)
 - mkdir 846 (time 1789043665.06 total 50.15 last 17.88)
 - mkdir 977 (time 1789043675.09 total 60.17 last 13.07)
total: 1000 mkdir in 61.80 seconds: 16.18 ops/second
 - mkdir 161 (time 1789043626.11 total 10.01 last 16.09)
 - mkdir 333 (time 1789043636.12 total 20.02 last 17.18)
 - mkdir 511 (time 1789043646.15 total 30.04 last 17.75)
 - mkdir 690 (time 1789043656.18 total 40.08 last 17.84)
 - mkdir 859 (time 1789043666.25 total 50.14 last 16.79)
 - mkdir 993 (time 1789043676.27 total 60.16 last 13.38)
total: 1000 mkdir in 60.71 seconds: 16.47 ops/second
 - mkdir 161 (time 1789043626.30 total 10.02 last 16.06)
 - mkdir 328 (time 1789043636.40 total 20.12 last 16.54)
 - mkdir 506 (time 1789043646.45 total 30.17 last 17.71)
 - mkdir 690 (time 1789043656.48 total 40.20 last 18.34)
 - mkdir 855 (time 1789043666.48 total 50.21 last 16.49)
 - mkdir 989 (time 1789043676.49 total 60.21 last 13.39)
total: 1000 mkdir in 61.31 seconds: 16.31 ops/second
 - mkdir 162 (time 1789043626.21 total 10.04 last 16.14)
 - mkdir 327 (time 1789043636.23 total 20.06 last 16.46)
 - mkdir 502 (time 1789043646.25 total 30.08 last 17.47)
 - mkdir 682 (time 1789043656.26 total 40.09 last 17.98)
 - mkdir 850 (time 1789043666.37 total 50.20 last 16.62)
 - mkdir 987 (time 1789043676.44 total 60.27 last 13.60)
total: 1000 mkdir in 61.70 seconds: 16.21 ops/second
 - mkdir 158 (time 1789043626.52 total 10.01 last 15.78)
 - mkdir 320 (time 1789043636.54 total 20.04 last 16.15)
 - mkdir 497 (time 1789043646.56 total 30.06 last 17.67)
 - mkdir 684 (time 1789043656.64 total 40.14 last 18.55)
 - mkdir 850 (time 1789043666.67 total 50.16 last 16.56)
 - mkdir 983 (time 1789043676.72 total 60.22 last 13.22)
total: 1000 mkdir in 61.82 seconds: 16.18 ops/second
 - mkdir 161 (time 1789043626.24 total 10.02 last 16.07)
 - mkdir 328 (time 1789043636.32 total 20.09 last 16.58)
 - mkdir 501 (time 1789043646.34 total 30.11 last 17.27)
 - mkdir 679 (time 1789043656.37 total 40.15 last 17.74)
 - mkdir 845 (time 1789043666.38 total 50.16 last 16.58)
 - mkdir 979 (time 1789043676.47 total 60.25 last 13.28)
total: 1000 mkdir in 62.18 seconds: 16.08 ops/second
 - mkdir 149 (time 1789043625.51 total 10.04 last 14.84)
 - mkdir 311 (time 1789043635.57 total 20.10 last 16.11)
 - mkdir 478 (time 1789043645.58 total 30.11 last 16.68)
 - mkdir 658 (time 1789043655.59 total 40.12 last 17.99)
 - mkdir 833 (time 1789043665.60 total 50.13 last 17.48)
 - mkdir 968 (time 1789043675.60 total 60.13 last 13.49)
total: 1000 mkdir in 63.16 seconds: 15.83 ops/second
Time for file creation 75 sec for 1 RPC sent at a time
 - unlinked 0 (time 1789043689 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789043689 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789043690 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1789043690 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789043691 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789043690 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1789043691 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789043690 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1789043691 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789043691 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
Time for file removal 42 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8