== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 11:03:18 (1788534198)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.57 seconds: 17.53 ops/second
 - mkdir 173 (time 1788534221.10 total 10.07 last 17.18)
 - mkdir 335 (time 1788534231.12 total 20.09 last 16.17)
 - mkdir 519 (time 1788534241.15 total 30.12 last 18.35)
 - mkdir 686 (time 1788534251.20 total 40.16 last 16.63)
 - mkdir 832 (time 1788534261.27 total 50.24 last 14.49)
total: 1000 mkdir in 59.76 seconds: 16.73 ops/second
 - mkdir 176 (time 1788534220.11 total 10.05 last 17.52)
 - mkdir 331 (time 1788534230.12 total 20.06 last 15.48)
 - mkdir 500 (time 1788534240.15 total 30.09 last 16.85)
 - mkdir 660 (time 1788534250.26 total 40.20 last 15.82)
 - mkdir 806 (time 1788534260.29 total 50.23 last 14.56)
 - mkdir 971 (time 1788534270.29 total 60.23 last 16.49)
total: 1000 mkdir in 63.01 seconds: 15.87 ops/second
 - mkdir 163 (time 1788534220.38 total 10.02 last 16.26)
 - mkdir 322 (time 1788534230.40 total 20.04 last 15.87)
 - mkdir 495 (time 1788534240.44 total 30.08 last 17.23)
 - mkdir 661 (time 1788534250.50 total 40.14 last 16.50)
 - mkdir 805 (time 1788534260.56 total 50.20 last 14.31)
 - mkdir 972 (time 1788534270.56 total 60.21 last 16.70)
total: 1000 mkdir in 62.97 seconds: 15.88 ops/second
 - mkdir 171 (time 1788534221.12 total 10.05 last 17.02)
 - mkdir 333 (time 1788534231.18 total 20.10 last 16.12)
 - mkdir 505 (time 1788534241.18 total 30.10 last 17.20)
 - mkdir 668 (time 1788534251.18 total 40.10 last 16.30)
 - mkdir 813 (time 1788534261.27 total 50.19 last 14.37)
 - mkdir 971 (time 1788534271.33 total 60.25 last 15.71)
total: 1000 mkdir in 63.30 seconds: 15.80 ops/second
 - mkdir 172 (time 1788534220.71 total 10.01 last 17.19)
 - mkdir 328 (time 1788534230.79 total 20.09 last 15.47)
 - mkdir 496 (time 1788534240.82 total 30.12 last 16.75)
 - mkdir 654 (time 1788534250.86 total 40.16 last 15.74)
 - mkdir 793 (time 1788534260.89 total 50.19 last 13.87)
 - mkdir 959 (time 1788534270.91 total 60.21 last 16.56)
total: 1000 mkdir in 64.63 seconds: 15.47 ops/second
 - mkdir 171 (time 1788534221.74 total 10.06 last 16.99)
 - mkdir 332 (time 1788534231.75 total 20.08 last 16.08)
 - mkdir 508 (time 1788534241.80 total 30.12 last 17.52)
 - mkdir 663 (time 1788534251.82 total 40.14 last 15.48)
 - mkdir 806 (time 1788534261.85 total 50.18 last 14.24)
 - mkdir 967 (time 1788534271.91 total 60.23 last 16.01)
total: 1000 mkdir in 63.80 seconds: 15.68 ops/second
 - mkdir 170 (time 1788534221.63 total 10.08 last 16.86)
 - mkdir 324 (time 1788534231.64 total 20.10 last 15.38)
 - mkdir 504 (time 1788534241.68 total 30.13 last 17.95)
 - mkdir 656 (time 1788534251.69 total 40.15 last 15.17)
 - mkdir 801 (time 1788534261.72 total 50.18 last 14.46)
 - mkdir 963 (time 1788534271.80 total 60.25 last 16.08)
total: 1000 mkdir in 64.12 seconds: 15.60 ops/second
 - mkdir 159 (time 1788534218.26 total 10.02 last 15.86)
 - mkdir 307 (time 1788534228.31 total 20.07 last 14.73)
 - mkdir 467 (time 1788534238.35 total 30.11 last 15.94)
 - mkdir 631 (time 1788534248.35 total 40.11 last 16.39)
 - mkdir 759 (time 1788534258.40 total 50.16 last 12.74)
 - mkdir 919 (time 1788534268.47 total 60.23 last 15.89)
total: 1000 mkdir in 68.00 seconds: 14.71 ops/second
 - mkdir 173 (time 1788534221.42 total 10.10 last 17.13)
 - mkdir 331 (time 1788534231.45 total 20.13 last 15.75)
 - mkdir 499 (time 1788534241.47 total 30.15 last 16.76)
 - mkdir 652 (time 1788534251.47 total 40.16 last 15.29)
 - mkdir 795 (time 1788534261.54 total 50.22 last 14.21)
 - mkdir 959 (time 1788534271.56 total 60.24 last 16.37)
total: 1000 mkdir in 64.99 seconds: 15.39 ops/second
 - mkdir 166 (time 1788534221.66 total 10.11 last 16.41)
 - mkdir 322 (time 1788534231.72 total 20.17 last 15.51)
 - mkdir 496 (time 1788534241.84 total 30.29 last 17.19)
 - mkdir 647 (time 1788534251.88 total 40.33 last 15.04)
 - mkdir 784 (time 1788534261.95 total 50.40 last 13.61)
 - mkdir 944 (time 1788534271.99 total 60.44 last 15.93)
total: 1000 mkdir in 66.05 seconds: 15.14 ops/second
Time for file creation 77 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 12 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.58 seconds: 17.35 ops/second
 - mkdir 126 (time 1788534790.46 total 10.02 last 12.57)
 - mkdir 255 (time 1788534800.51 total 20.07 last 12.83)
 - mkdir 404 (time 1788534810.56 total 30.12 last 14.83)
 - mkdir 564 (time 1788534820.59 total 40.15 last 15.96)
 - mkdir 738 (time 1788534830.61 total 50.17 last 17.37)
 - mkdir 899 (time 1788534840.65 total 60.21 last 16.02)
total: 1000 mkdir in 66.29 seconds: 15.08 ops/second
 - mkdir 118 (time 1788534795.69 total 10.05 last 11.74)
 - mkdir 262 (time 1788534805.78 total 20.15 last 14.27)
 - mkdir 421 (time 1788534815.83 total 30.19 last 15.82)
 - mkdir 600 (time 1788534825.85 total 40.22 last 17.86)
 - mkdir 762 (time 1788534835.85 total 50.22 last 16.20)
 - mkdir 934 (time 1788534845.87 total 60.23 last 17.17)
total: 1000 mkdir in 68.49 seconds: 14.60 ops/second
 - mkdir 131 (time 1788534797.18 total 10.06 last 13.02)
 - mkdir 280 (time 1788534807.18 total 20.06 last 14.89)
 - mkdir 444 (time 1788534817.22 total 30.10 last 16.33)
 - mkdir 624 (time 1788534827.25 total 40.13 last 17.95)
 - mkdir 781 (time 1788534837.26 total 50.14 last 15.67)
 - mkdir 944 (time 1788534847.29 total 60.17 last 16.25)
total: 1000 mkdir in 69.01 seconds: 14.49 ops/second
 - mkdir 117 (time 1788534796.32 total 10.03 last 11.67)
 - mkdir 263 (time 1788534806.33 total 20.04 last 14.58)
 - mkdir 424 (time 1788534816.34 total 30.05 last 16.09)
 - mkdir 603 (time 1788534826.37 total 40.08 last 17.85)
 - mkdir 768 (time 1788534836.46 total 50.17 last 16.34)
 - mkdir 934 (time 1788534846.53 total 60.24 last 16.49)
 - mkdir 999 (time 1788534856.54 total 70.25 last 6.50)
total: 1000 mkdir in 70.25 seconds: 14.24 ops/second
 - mkdir 109 (time 1788534795.76 total 10.01 last 10.89)
 - mkdir 249 (time 1788534805.77 total 20.02 last 13.99)
 - mkdir 410 (time 1788534815.80 total 30.05 last 16.05)
 - mkdir 587 (time 1788534825.84 total 40.08 last 17.64)
 - mkdir 754 (time 1788534835.92 total 50.16 last 16.57)
 - mkdir 923 (time 1788534846.01 total 60.26 last 16.74)
 - mkdir 995 (time 1788534856.12 total 70.37 last 7.12)
total: 1000 mkdir in 71.21 seconds: 14.04 ops/second
 - mkdir 115 (time 1788534796.59 total 10.01 last 11.49)
 - mkdir 266 (time 1788534806.62 total 20.04 last 15.06)
 - mkdir 424 (time 1788534816.65 total 30.07 last 15.75)
 - mkdir 603 (time 1788534826.69 total 40.12 last 17.82)
 - mkdir 762 (time 1788534836.71 total 50.14 last 15.87)
 - mkdir 934 (time 1788534846.80 total 60.22 last 17.05)
 - mkdir 996 (time 1788534856.84 total 70.27 last 6.17)
total: 1000 mkdir in 70.94 seconds: 14.10 ops/second
 - mkdir 118 (time 1788534796.69 total 10.04 last 11.76)
 - mkdir 263 (time 1788534806.69 total 20.04 last 14.49)
 - mkdir 428 (time 1788534816.72 total 30.07 last 16.46)
 - mkdir 604 (time 1788534826.79 total 40.14 last 17.48)
 - mkdir 765 (time 1788534836.83 total 50.18 last 16.03)
 - mkdir 934 (time 1788534846.88 total 60.24 last 16.81)
 - mkdir 996 (time 1788534856.99 total 70.34 last 6.14)
total: 1000 mkdir in 70.96 seconds: 14.09 ops/second
 - mkdir 116 (time 1788534795.83 total 10.06 last 11.53)
 - mkdir 254 (time 1788534805.84 total 20.07 last 13.79)
 - mkdir 410 (time 1788534815.85 total 30.08 last 15.58)
 - mkdir 586 (time 1788534825.85 total 40.09 last 17.59)
 - mkdir 746 (time 1788534835.87 total 50.11 last 15.97)
 - mkdir 918 (time 1788534845.90 total 60.14 last 17.14)
 - mkdir 990 (time 1788534855.94 total 70.17 last 7.18)
total: 1000 mkdir in 72.02 seconds: 13.89 ops/second
 - mkdir 121 (time 1788534796.65 total 10.03 last 12.06)
 - mkdir 267 (time 1788534806.69 total 20.07 last 14.54)
 - mkdir 430 (time 1788534816.71 total 30.10 last 16.26)
 - mkdir 607 (time 1788534826.77 total 40.16 last 17.60)
 - mkdir 762 (time 1788534836.81 total 50.19 last 15.45)
 - mkdir 930 (time 1788534846.83 total 60.21 last 16.77)
 - mkdir 992 (time 1788534856.92 total 70.31 last 6.14)
total: 1000 mkdir in 71.56 seconds: 13.97 ops/second
 - mkdir 116 (time 1788534794.87 total 10.02 last 11.58)
 - mkdir 242 (time 1788534804.89 total 20.05 last 12.57)
 - mkdir 391 (time 1788534814.94 total 30.10 last 14.82)
 - mkdir 565 (time 1788534825.05 total 40.20 last 17.22)
 - mkdir 730 (time 1788534835.09 total 50.25 last 16.42)
 - mkdir 893 (time 1788534845.12 total 60.27 last 16.25)
 - mkdir 974 (time 1788534855.27 total 70.42 last 7.98)
total: 1000 mkdir in 74.31 seconds: 13.46 ops/second
Time for file creation 88 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788534869 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788534870 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788534871 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788534871 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788534871 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788534871 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788534871 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788534872 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788534871 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788534871 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
Time for file removal 45 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8