== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 03:54:12 (1788940452)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.81 seconds: 12.29 ops/second
 - mkdir 91 (time 1788940473.19 total 10.04 last 9.06)
 - mkdir 229 (time 1788940483.20 total 20.05 last 13.79)
 - mkdir 363 (time 1788940493.20 total 30.05 last 13.40)
 - mkdir 525 (time 1788940503.23 total 40.08 last 16.16)
 - mkdir 733 (time 1788940513.29 total 50.14 last 20.66)
 - mkdir 936 (time 1788940523.32 total 60.17 last 20.25)
total: 1000 mkdir in 64.57 seconds: 15.49 ops/second
 - mkdir 109 (time 1788940477.39 total 10.01 last 10.89)
 - mkdir 269 (time 1788940487.40 total 20.02 last 15.99)
 - mkdir 425 (time 1788940497.47 total 30.08 last 15.50)
 - mkdir 613 (time 1788940507.49 total 40.11 last 18.75)
 - mkdir 810 (time 1788940517.57 total 50.19 last 19.55)
 - mkdir 993 (time 1788940527.58 total 60.20 last 18.27)
total: 1000 mkdir in 61.00 seconds: 16.39 ops/second
 - mkdir 98 (time 1788940476.56 total 10.03 last 9.77)
 - mkdir 248 (time 1788940486.60 total 20.06 last 14.95)
 - mkdir 399 (time 1788940496.62 total 30.08 last 15.07)
 - mkdir 583 (time 1788940506.68 total 40.14 last 18.29)
 - mkdir 776 (time 1788940516.70 total 50.16 last 19.26)
 - mkdir 965 (time 1788940526.77 total 60.23 last 18.76)
total: 1000 mkdir in 64.64 seconds: 15.47 ops/second
 - mkdir 103 (time 1788940477.73 total 10.01 last 10.29)
 - mkdir 258 (time 1788940487.73 total 20.01 last 15.49)
 - mkdir 410 (time 1788940497.74 total 30.02 last 15.20)
 - mkdir 598 (time 1788940507.76 total 40.04 last 18.76)
 - mkdir 790 (time 1788940517.79 total 50.07 last 19.13)
 - mkdir 974 (time 1788940527.87 total 60.16 last 18.25)
total: 1000 mkdir in 64.25 seconds: 15.56 ops/second
 - mkdir 100 (time 1788940477.39 total 10.01 last 9.99)
 - mkdir 246 (time 1788940487.40 total 20.03 last 14.58)
 - mkdir 400 (time 1788940497.45 total 30.07 last 15.33)
 - mkdir 590 (time 1788940507.49 total 40.12 last 18.91)
 - mkdir 786 (time 1788940517.50 total 50.12 last 19.59)
 - mkdir 965 (time 1788940527.52 total 60.15 last 17.86)
total: 1000 mkdir in 65.91 seconds: 15.17 ops/second
 - mkdir 109 (time 1788940478.87 total 10.06 last 10.84)
 - mkdir 276 (time 1788940488.88 total 20.06 last 16.69)
 - mkdir 417 (time 1788940498.95 total 30.13 last 14.01)
 - mkdir 621 (time 1788940508.97 total 40.15 last 20.35)
 - mkdir 808 (time 1788940518.99 total 50.17 last 18.66)
 - mkdir 969 (time 1788940529.16 total 60.35 last 15.83)
total: 1000 mkdir in 65.56 seconds: 15.25 ops/second
 - mkdir 106 (time 1788940478.45 total 10.05 last 10.54)
 - mkdir 262 (time 1788940488.47 total 20.07 last 15.57)
 - mkdir 407 (time 1788940498.48 total 30.08 last 14.49)
 - mkdir 602 (time 1788940508.50 total 40.10 last 19.46)
 - mkdir 794 (time 1788940518.50 total 50.11 last 19.19)
 - mkdir 964 (time 1788940528.66 total 60.26 last 16.74)
total: 1000 mkdir in 66.40 seconds: 15.06 ops/second
 - mkdir 110 (time 1788940479.04 total 10.04 last 10.96)
 - mkdir 273 (time 1788940489.13 total 20.12 last 16.17)
 - mkdir 416 (time 1788940499.15 total 30.14 last 14.27)
 - mkdir 619 (time 1788940509.18 total 40.17 last 20.24)
 - mkdir 806 (time 1788940519.19 total 50.18 last 18.68)
 - mkdir 964 (time 1788940529.29 total 60.28 last 15.65)
total: 1000 mkdir in 66.26 seconds: 15.09 ops/second
 - mkdir 107 (time 1788940479.24 total 10.17 last 10.52)
 - mkdir 273 (time 1788940489.27 total 20.19 last 16.56)
 - mkdir 415 (time 1788940499.28 total 30.20 last 14.19)
 - mkdir 619 (time 1788940509.29 total 40.21 last 20.38)
 - mkdir 808 (time 1788940519.30 total 50.23 last 18.87)
 - mkdir 958 (time 1788940529.31 total 60.23 last 14.99)
total: 1000 mkdir in 66.88 seconds: 14.95 ops/second
 - mkdir 108 (time 1788940479.37 total 10.06 last 10.73)
 - mkdir 276 (time 1788940489.40 total 20.09 last 16.75)
 - mkdir 413 (time 1788940499.46 total 30.16 last 13.61)
 - mkdir 618 (time 1788940509.53 total 40.22 last 20.36)
 - mkdir 812 (time 1788940519.54 total 50.23 last 19.38)
 - mkdir 962 (time 1788940529.67 total 60.36 last 14.81)
total: 1000 mkdir in 66.71 seconds: 14.99 ops/second
Time for file creation 84 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 16 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.42 seconds: 23.59 ops/second
 - mkdir 144 (time 1788941074.58 total 10.04 last 14.34)
 - mkdir 294 (time 1788941084.58 total 20.04 last 15.00)
 - mkdir 473 (time 1788941094.59 total 30.05 last 17.88)
 - mkdir 701 (time 1788941104.61 total 40.08 last 22.75)
 - mkdir 876 (time 1788941114.65 total 50.12 last 17.42)
total: 1000 mkdir in 59.50 seconds: 16.81 ops/second
 - mkdir 125 (time 1788941073.11 total 10.08 last 12.41)
 - mkdir 274 (time 1788941083.15 total 20.12 last 14.83)
 - mkdir 445 (time 1788941093.21 total 30.18 last 17.00)
 - mkdir 664 (time 1788941103.22 total 40.19 last 21.88)
 - mkdir 857 (time 1788941113.26 total 50.23 last 19.22)
 - mkdir 985 (time 1788941123.26 total 60.23 last 12.80)
total: 1000 mkdir in 61.30 seconds: 16.31 ops/second
 - mkdir 150 (time 1788941075.53 total 10.04 last 14.94)
 - mkdir 300 (time 1788941085.61 total 20.12 last 14.89)
 - mkdir 471 (time 1788941095.64 total 30.14 last 17.05)
 - mkdir 703 (time 1788941105.65 total 40.16 last 23.17)
 - mkdir 882 (time 1788941115.67 total 50.18 last 17.87)
total: 1000 mkdir in 60.12 seconds: 16.63 ops/second
 - mkdir 154 (time 1788941075.74 total 10.00 last 15.40)
 - mkdir 310 (time 1788941085.78 total 20.04 last 15.53)
 - mkdir 479 (time 1788941095.81 total 30.07 last 16.86)
 - mkdir 707 (time 1788941105.83 total 40.10 last 22.74)
 - mkdir 883 (time 1788941115.92 total 50.18 last 17.45)
total: 1000 mkdir in 60.01 seconds: 16.66 ops/second
 - mkdir 152 (time 1788941076.25 total 10.00 last 15.19)
 - mkdir 309 (time 1788941086.30 total 20.05 last 15.62)
 - mkdir 488 (time 1788941096.30 total 30.05 last 17.90)
 - mkdir 715 (time 1788941106.33 total 40.08 last 22.63)
 - mkdir 887 (time 1788941116.35 total 50.10 last 17.17)
total: 1000 mkdir in 59.79 seconds: 16.72 ops/second
 - mkdir 145 (time 1788941076.25 total 10.01 last 14.48)
 - mkdir 299 (time 1788941086.28 total 20.04 last 15.36)
 - mkdir 475 (time 1788941096.28 total 30.04 last 17.60)
 - mkdir 704 (time 1788941106.31 total 40.07 last 22.84)
 - mkdir 876 (time 1788941116.38 total 50.14 last 17.07)
 - mkdir 989 (time 1788941126.42 total 60.18 last 11.26)
total: 1000 mkdir in 61.72 seconds: 16.20 ops/second
 - mkdir 148 (time 1788941076.50 total 10.08 last 14.68)
 - mkdir 308 (time 1788941086.55 total 20.13 last 15.92)
 - mkdir 491 (time 1788941096.57 total 30.15 last 18.27)
 - mkdir 709 (time 1788941106.58 total 40.15 last 21.78)
 - mkdir 875 (time 1788941116.60 total 50.17 last 16.57)
 - mkdir 987 (time 1788941126.78 total 60.36 last 11.00)
total: 1000 mkdir in 61.97 seconds: 16.14 ops/second
 - mkdir 148 (time 1788941075.86 total 10.02 last 14.77)
 - mkdir 305 (time 1788941085.90 total 20.06 last 15.64)
 - mkdir 473 (time 1788941095.91 total 30.07 last 16.78)
 - mkdir 697 (time 1788941105.91 total 40.08 last 22.38)
 - mkdir 869 (time 1788941115.95 total 50.11 last 17.14)
 - mkdir 982 (time 1788941125.97 total 60.13 last 11.27)
total: 1000 mkdir in 62.55 seconds: 15.99 ops/second
 - mkdir 150 (time 1788941076.02 total 10.04 last 14.95)
 - mkdir 303 (time 1788941086.03 total 20.04 last 15.29)
 - mkdir 472 (time 1788941096.04 total 30.05 last 16.88)
 - mkdir 697 (time 1788941106.05 total 40.06 last 22.48)
 - mkdir 867 (time 1788941116.13 total 50.15 last 16.86)
 - mkdir 985 (time 1788941126.21 total 60.23 last 11.70)
total: 1000 mkdir in 62.43 seconds: 16.02 ops/second
 - mkdir 136 (time 1788941075.55 total 10.05 last 13.53)
 - mkdir 277 (time 1788941085.60 total 20.10 last 14.03)
 - mkdir 438 (time 1788941095.66 total 30.16 last 16.00)
 - mkdir 668 (time 1788941105.69 total 40.19 last 22.93)
 - mkdir 844 (time 1788941115.71 total 50.21 last 17.57)
 - mkdir 961 (time 1788941125.82 total 60.32 last 11.57)
total: 1000 mkdir in 65.03 seconds: 15.38 ops/second
Time for file creation 76 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788941137 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788941137 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788941139 ; total 1 ; last 1)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788941139 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788941139 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788941138 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788941138 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788941139 ; total 1 ; last 1)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788941138 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788941138 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
Time for file removal 43 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8