== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 06:56:18 (1788951378)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.49 seconds: 20.57 ops/second
 - mkdir 153 (time 1788951399.73 total 10.00 last 15.30)
 - mkdir 312 (time 1788951409.79 total 20.06 last 15.81)
 - mkdir 476 (time 1788951419.80 total 30.07 last 16.38)
 - mkdir 666 (time 1788951429.81 total 40.08 last 18.98)
 - mkdir 834 (time 1788951439.87 total 50.14 last 16.71)
total: 1000 mkdir in 58.52 seconds: 17.09 ops/second
 - mkdir 151 (time 1788951400.10 total 10.06 last 15.01)
 - mkdir 316 (time 1788951410.11 total 20.08 last 16.47)
 - mkdir 480 (time 1788951420.11 total 30.08 last 16.40)
 - mkdir 667 (time 1788951430.15 total 40.12 last 18.63)
 - mkdir 839 (time 1788951440.16 total 50.12 last 17.19)
total: 1000 mkdir in 58.49 seconds: 17.10 ops/second
 - mkdir 153 (time 1788951399.17 total 10.01 last 15.29)
 - mkdir 299 (time 1788951409.22 total 20.06 last 14.52)
 - mkdir 447 (time 1788951419.25 total 30.09 last 14.76)
 - mkdir 638 (time 1788951429.26 total 40.10 last 19.08)
 - mkdir 809 (time 1788951439.29 total 50.13 last 17.06)
total: 1000 mkdir in 59.99 seconds: 16.67 ops/second
 - mkdir 151 (time 1788951400.45 total 10.04 last 15.05)
 - mkdir 314 (time 1788951410.50 total 20.09 last 16.22)
 - mkdir 470 (time 1788951420.55 total 30.14 last 15.52)
 - mkdir 662 (time 1788951430.59 total 40.18 last 19.12)
 - mkdir 840 (time 1788951440.63 total 50.21 last 17.73)
total: 1000 mkdir in 58.89 seconds: 16.98 ops/second
 - mkdir 149 (time 1788951400.30 total 10.02 last 14.87)
 - mkdir 308 (time 1788951410.32 total 20.04 last 15.87)
 - mkdir 466 (time 1788951420.37 total 30.09 last 15.71)
 - mkdir 655 (time 1788951430.38 total 40.10 last 18.89)
 - mkdir 834 (time 1788951440.40 total 50.12 last 17.86)
total: 1000 mkdir in 59.11 seconds: 16.92 ops/second
 - mkdir 148 (time 1788951400.33 total 10.08 last 14.68)
 - mkdir 308 (time 1788951410.38 total 20.13 last 15.92)
 - mkdir 464 (time 1788951420.42 total 30.17 last 15.54)
 - mkdir 657 (time 1788951430.45 total 40.20 last 19.24)
 - mkdir 836 (time 1788951440.46 total 50.22 last 17.87)
total: 1000 mkdir in 59.17 seconds: 16.90 ops/second
 - mkdir 153 (time 1788951400.69 total 10.07 last 15.20)
 - mkdir 314 (time 1788951410.75 total 20.12 last 16.01)
 - mkdir 471 (time 1788951420.80 total 30.18 last 15.62)
 - mkdir 662 (time 1788951430.82 total 40.19 last 19.07)
 - mkdir 838 (time 1788951440.84 total 50.21 last 17.56)
total: 1000 mkdir in 59.41 seconds: 16.83 ops/second
 - mkdir 147 (time 1788951400.85 total 10.03 last 14.65)
 - mkdir 306 (time 1788951410.86 total 20.03 last 15.90)
 - mkdir 465 (time 1788951420.91 total 30.09 last 15.82)
 - mkdir 654 (time 1788951430.92 total 40.09 last 18.88)
 - mkdir 833 (time 1788951440.96 total 50.14 last 17.82)
total: 1000 mkdir in 59.68 seconds: 16.76 ops/second
 - mkdir 144 (time 1788951399.67 total 10.08 last 14.28)
 - mkdir 291 (time 1788951409.73 total 20.15 last 14.61)
 - mkdir 440 (time 1788951419.81 total 30.23 last 14.78)
 - mkdir 638 (time 1788951429.83 total 40.24 last 19.76)
 - mkdir 806 (time 1788951439.86 total 50.28 last 16.74)
 - mkdir 988 (time 1788951449.89 total 60.31 last 18.15)
total: 1000 mkdir in 61.06 seconds: 16.38 ops/second
 - mkdir 147 (time 1788951400.73 total 10.01 last 14.68)
 - mkdir 304 (time 1788951410.77 total 20.05 last 15.64)
 - mkdir 455 (time 1788951420.79 total 30.07 last 15.06)
 - mkdir 646 (time 1788951430.79 total 40.07 last 19.10)
 - mkdir 825 (time 1788951440.85 total 50.13 last 17.81)
total: 1000 mkdir in 60.07 seconds: 16.65 ops/second
Time for file creation 70 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 9 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.40 seconds: 24.77 ops/second
 - mkdir 151 (time 1788951942.11 total 10.01 last 15.09)
 - mkdir 326 (time 1788951952.14 total 20.03 last 17.45)
 - mkdir 474 (time 1788951962.19 total 30.09 last 14.71)
 - mkdir 657 (time 1788951972.20 total 40.10 last 18.29)
 - mkdir 820 (time 1788951982.24 total 50.14 last 16.23)
total: 1000 mkdir in 58.72 seconds: 17.03 ops/second
 - mkdir 144 (time 1788951940.31 total 10.06 last 14.31)
 - mkdir 305 (time 1788951950.31 total 20.07 last 16.09)
 - mkdir 443 (time 1788951960.34 total 30.09 last 13.76)
 - mkdir 625 (time 1788951970.36 total 40.11 last 18.16)
 - mkdir 786 (time 1788951980.40 total 50.15 last 16.03)
 - mkdir 992 (time 1788951990.44 total 60.19 last 20.52)
total: 1000 mkdir in 60.58 seconds: 16.51 ops/second
 - mkdir 150 (time 1788951941.84 total 10.04 last 14.94)
 - mkdir 310 (time 1788951951.86 total 20.06 last 15.97)
 - mkdir 458 (time 1788951961.87 total 30.07 last 14.78)
 - mkdir 638 (time 1788951971.92 total 40.12 last 17.92)
 - mkdir 802 (time 1788951981.96 total 50.15 last 16.34)
total: 1000 mkdir in 59.45 seconds: 16.82 ops/second
 - mkdir 152 (time 1788951942.17 total 10.01 last 15.18)
 - mkdir 316 (time 1788951952.23 total 20.08 last 16.30)
 - mkdir 465 (time 1788951962.30 total 30.15 last 14.79)
 - mkdir 650 (time 1788951972.35 total 40.20 last 18.41)
 - mkdir 812 (time 1788951982.37 total 50.22 last 16.17)
total: 1000 mkdir in 59.74 seconds: 16.74 ops/second
 - mkdir 141 (time 1788951941.07 total 10.01 last 14.08)
 - mkdir 307 (time 1788951951.16 total 20.10 last 16.45)
 - mkdir 448 (time 1788951961.17 total 30.11 last 14.10)
 - mkdir 628 (time 1788951971.21 total 40.15 last 17.92)
 - mkdir 780 (time 1788951981.23 total 50.17 last 15.18)
 - mkdir 982 (time 1788951991.27 total 60.21 last 20.11)
total: 1000 mkdir in 61.62 seconds: 16.23 ops/second
 - mkdir 146 (time 1788951942.54 total 10.02 last 14.57)
 - mkdir 318 (time 1788951952.60 total 20.08 last 17.11)
 - mkdir 460 (time 1788951962.65 total 30.13 last 14.12)
 - mkdir 637 (time 1788951972.66 total 40.14 last 17.69)
 - mkdir 803 (time 1788951982.66 total 50.14 last 16.60)
 - mkdir 999 (time 1788951992.70 total 60.18 last 19.51)
total: 1000 mkdir in 60.18 seconds: 16.62 ops/second
 - mkdir 146 (time 1788951942.24 total 10.01 last 14.58)
 - mkdir 312 (time 1788951952.31 total 20.08 last 16.48)
 - mkdir 460 (time 1788951962.32 total 30.10 last 14.78)
 - mkdir 638 (time 1788951972.37 total 40.14 last 17.73)
 - mkdir 791 (time 1788951982.39 total 50.16 last 15.27)
 - mkdir 983 (time 1788951992.46 total 60.23 last 19.07)
total: 1000 mkdir in 61.55 seconds: 16.25 ops/second
 - mkdir 148 (time 1788951942.59 total 10.03 last 14.76)
 - mkdir 315 (time 1788951952.65 total 20.09 last 16.59)
 - mkdir 458 (time 1788951962.68 total 30.12 last 14.26)
 - mkdir 631 (time 1788951972.69 total 40.13 last 17.28)
 - mkdir 783 (time 1788951982.71 total 50.15 last 15.18)
 - mkdir 969 (time 1788951992.73 total 60.17 last 18.56)
total: 1000 mkdir in 62.76 seconds: 15.93 ops/second
 - mkdir 140 (time 1788951942.60 total 10.03 last 13.96)
 - mkdir 308 (time 1788951952.64 total 20.07 last 16.73)
 - mkdir 452 (time 1788951962.66 total 30.10 last 14.36)
 - mkdir 632 (time 1788951972.68 total 40.11 last 17.97)
 - mkdir 784 (time 1788951982.70 total 50.14 last 15.16)
 - mkdir 968 (time 1788951992.76 total 60.20 last 18.30)
total: 1000 mkdir in 63.06 seconds: 15.86 ops/second
 - mkdir 135 (time 1788951941.43 total 10.12 last 13.34)
 - mkdir 295 (time 1788951951.47 total 20.16 last 15.93)
 - mkdir 435 (time 1788951961.53 total 30.22 last 13.91)
 - mkdir 603 (time 1788951971.61 total 40.30 last 16.67)
 - mkdir 748 (time 1788951981.71 total 50.40 last 14.36)
 - mkdir 939 (time 1788951991.82 total 60.50 last 18.90)
total: 1000 mkdir in 65.39 seconds: 15.29 ops/second
Time for file creation 74 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788952003 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788952004 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788952005 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788952005 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788952005 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788952006 ; total 1 ; last 1)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788952005 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788952005 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788952006 ; total 1 ; last 1)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788952006 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
Time for file removal 41 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8