== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 05:56:34 (1788688594)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.45 seconds: 22.37 ops/second
 - mkdir 183 (time 1788688611.32 total 10.04 last 18.23)
 - mkdir 344 (time 1788688621.33 total 20.05 last 16.09)
 - mkdir 513 (time 1788688631.33 total 30.05 last 16.90)
 - mkdir 672 (time 1788688641.33 total 40.05 last 15.90)
 - mkdir 801 (time 1788688651.40 total 50.12 last 12.81)
 - mkdir 949 (time 1788688661.48 total 60.20 last 14.69)
total: 1000 mkdir in 63.61 seconds: 15.72 ops/second
 - mkdir 174 (time 1788688613.13 total 10.11 last 17.21)
 - mkdir 341 (time 1788688623.16 total 20.14 last 16.65)
 - mkdir 512 (time 1788688633.21 total 30.19 last 17.01)
 - mkdir 663 (time 1788688643.25 total 40.23 last 15.05)
 - mkdir 803 (time 1788688653.26 total 50.24 last 13.98)
 - mkdir 948 (time 1788688663.30 total 60.28 last 14.44)
total: 1000 mkdir in 65.32 seconds: 15.31 ops/second
 - mkdir 172 (time 1788688614.06 total 10.01 last 17.19)
 - mkdir 350 (time 1788688624.12 total 20.06 last 17.70)
 - mkdir 515 (time 1788688634.19 total 30.13 last 16.39)
 - mkdir 669 (time 1788688644.21 total 40.15 last 15.37)
 - mkdir 807 (time 1788688654.24 total 50.18 last 13.76)
 - mkdir 951 (time 1788688664.24 total 60.19 last 14.39)
total: 1000 mkdir in 65.70 seconds: 15.22 ops/second
 - mkdir 172 (time 1788688613.85 total 10.00 last 17.19)
 - mkdir 349 (time 1788688623.88 total 20.03 last 17.65)
 - mkdir 514 (time 1788688633.92 total 30.07 last 16.44)
 - mkdir 661 (time 1788688643.93 total 40.08 last 14.68)
 - mkdir 798 (time 1788688653.97 total 50.12 last 13.65)
 - mkdir 949 (time 1788688664.02 total 60.18 last 15.02)
total: 1000 mkdir in 65.97 seconds: 15.16 ops/second
 - mkdir 167 (time 1788688613.18 total 10.05 last 16.61)
 - mkdir 339 (time 1788688623.22 total 20.09 last 17.13)
 - mkdir 508 (time 1788688633.27 total 30.14 last 16.82)
 - mkdir 661 (time 1788688643.32 total 40.19 last 15.21)
 - mkdir 792 (time 1788688653.35 total 50.22 last 13.06)
 - mkdir 933 (time 1788688663.38 total 60.26 last 14.05)
total: 1000 mkdir in 67.04 seconds: 14.92 ops/second
 - mkdir 166 (time 1788688613.76 total 10.04 last 16.53)
 - mkdir 336 (time 1788688623.77 total 20.05 last 16.98)
 - mkdir 503 (time 1788688633.79 total 30.07 last 16.67)
 - mkdir 653 (time 1788688643.85 total 40.14 last 14.90)
 - mkdir 789 (time 1788688653.90 total 50.19 last 13.53)
 - mkdir 931 (time 1788688663.93 total 60.22 last 14.15)
total: 1000 mkdir in 68.02 seconds: 14.70 ops/second
 - mkdir 173 (time 1788688614.33 total 10.09 last 17.14)
 - mkdir 345 (time 1788688624.35 total 20.11 last 17.16)
 - mkdir 505 (time 1788688634.41 total 30.18 last 15.90)
 - mkdir 649 (time 1788688644.43 total 40.19 last 14.38)
 - mkdir 787 (time 1788688654.48 total 50.24 last 13.74)
 - mkdir 934 (time 1788688664.54 total 60.30 last 14.62)
total: 1000 mkdir in 68.25 seconds: 14.65 ops/second
 - mkdir 169 (time 1788688613.98 total 10.05 last 16.81)
 - mkdir 340 (time 1788688624.01 total 20.08 last 17.05)
 - mkdir 500 (time 1788688634.06 total 30.13 last 15.92)
 - mkdir 651 (time 1788688644.06 total 40.13 last 15.10)
 - mkdir 782 (time 1788688654.11 total 50.18 last 13.04)
 - mkdir 922 (time 1788688664.20 total 60.27 last 13.87)
total: 1000 mkdir in 69.68 seconds: 14.35 ops/second
 - mkdir 159 (time 1788688613.26 total 10.09 last 15.76)
 - mkdir 329 (time 1788688623.29 total 20.12 last 16.94)
 - mkdir 489 (time 1788688633.32 total 30.15 last 15.95)
 - mkdir 631 (time 1788688643.36 total 40.19 last 14.15)
 - mkdir 765 (time 1788688653.43 total 50.26 last 13.30)
 - mkdir 902 (time 1788688663.44 total 60.27 last 13.70)
 - mkdir 996 (time 1788688673.52 total 70.35 last 9.32)
total: 1000 mkdir in 70.66 seconds: 14.15 ops/second
 - mkdir 162 (time 1788688614.18 total 10.04 last 16.13)
 - mkdir 329 (time 1788688624.24 total 20.10 last 16.60)
 - mkdir 483 (time 1788688634.25 total 30.12 last 15.38)
 - mkdir 623 (time 1788688644.26 total 40.12 last 13.99)
 - mkdir 751 (time 1788688654.32 total 50.19 last 12.71)
 - mkdir 889 (time 1788688664.36 total 60.23 last 13.75)
 - mkdir 975 (time 1788688674.39 total 70.25 last 8.58)
total: 1000 mkdir in 71.60 seconds: 13.97 ops/second
Time for file creation 80 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.73 seconds: 13.67 ops/second
 - mkdir 158 (time 1788689238.60 total 10.04 last 15.74)
 - mkdir 317 (time 1788689248.61 total 20.04 last 15.90)
 - mkdir 464 (time 1788689258.66 total 30.09 last 14.62)
 - mkdir 632 (time 1788689268.73 total 40.17 last 16.68)
 - mkdir 792 (time 1788689278.75 total 50.18 last 15.97)
 - mkdir 944 (time 1788689288.75 total 60.19 last 15.19)
total: 1000 mkdir in 62.60 seconds: 15.98 ops/second
 - mkdir 151 (time 1788689242.13 total 10.01 last 15.08)
 - mkdir 325 (time 1788689252.15 total 20.04 last 17.36)
 - mkdir 476 (time 1788689262.19 total 30.08 last 15.04)
 - mkdir 646 (time 1788689272.22 total 40.10 last 16.96)
 - mkdir 799 (time 1788689282.28 total 50.16 last 15.21)
 - mkdir 971 (time 1788689292.32 total 60.20 last 17.13)
total: 1000 mkdir in 63.27 seconds: 15.81 ops/second
 - mkdir 150 (time 1788689241.95 total 10.03 last 14.95)
 - mkdir 313 (time 1788689251.95 total 20.03 last 16.30)
 - mkdir 470 (time 1788689261.96 total 30.04 last 15.68)
 - mkdir 640 (time 1788689272.00 total 40.07 last 16.94)
 - mkdir 794 (time 1788689282.00 total 50.08 last 15.39)
 - mkdir 961 (time 1788689292.05 total 60.13 last 16.63)
total: 1000 mkdir in 64.14 seconds: 15.59 ops/second
 - mkdir 154 (time 1788689242.98 total 10.02 last 15.38)
 - mkdir 326 (time 1788689253.12 total 20.15 last 16.97)
 - mkdir 477 (time 1788689263.16 total 30.19 last 15.04)
 - mkdir 656 (time 1788689273.20 total 40.24 last 17.82)
 - mkdir 806 (time 1788689283.28 total 50.31 last 14.89)
 - mkdir 969 (time 1788689293.34 total 60.38 last 16.20)
total: 1000 mkdir in 63.68 seconds: 15.70 ops/second
 - mkdir 142 (time 1788689241.55 total 10.06 last 14.11)
 - mkdir 309 (time 1788689251.57 total 20.09 last 16.66)
 - mkdir 454 (time 1788689261.59 total 30.10 last 14.48)
 - mkdir 621 (time 1788689271.62 total 40.13 last 16.66)
 - mkdir 775 (time 1788689281.70 total 50.22 last 15.26)
 - mkdir 947 (time 1788689291.74 total 60.25 last 17.15)
total: 1000 mkdir in 65.91 seconds: 15.17 ops/second
 - mkdir 157 (time 1788689242.94 total 10.02 last 15.67)
 - mkdir 330 (time 1788689253.01 total 20.09 last 17.19)
 - mkdir 484 (time 1788689263.05 total 30.13 last 15.33)
 - mkdir 655 (time 1788689273.07 total 40.14 last 17.08)
 - mkdir 806 (time 1788689283.07 total 50.15 last 15.09)
 - mkdir 962 (time 1788689293.10 total 60.18 last 15.56)
total: 1000 mkdir in 64.56 seconds: 15.49 ops/second
 - mkdir 150 (time 1788689242.25 total 10.03 last 14.96)
 - mkdir 318 (time 1788689252.27 total 20.05 last 16.76)
 - mkdir 466 (time 1788689262.29 total 30.07 last 14.77)
 - mkdir 630 (time 1788689272.30 total 40.08 last 16.38)
 - mkdir 783 (time 1788689282.36 total 50.15 last 15.21)
 - mkdir 953 (time 1788689292.44 total 60.22 last 16.87)
total: 1000 mkdir in 65.35 seconds: 15.30 ops/second
 - mkdir 147 (time 1788689242.93 total 10.03 last 14.66)
 - mkdir 317 (time 1788689253.01 total 20.11 last 16.85)
 - mkdir 472 (time 1788689263.07 total 30.17 last 15.42)
 - mkdir 644 (time 1788689273.08 total 40.17 last 17.19)
 - mkdir 792 (time 1788689283.16 total 50.26 last 14.68)
 - mkdir 955 (time 1788689293.23 total 60.33 last 16.18)
total: 1000 mkdir in 65.52 seconds: 15.26 ops/second
 - mkdir 145 (time 1788689242.54 total 10.00 last 14.50)
 - mkdir 316 (time 1788689252.54 total 20.01 last 17.09)
 - mkdir 464 (time 1788689262.56 total 30.02 last 14.78)
 - mkdir 636 (time 1788689272.62 total 40.09 last 17.09)
 - mkdir 783 (time 1788689282.64 total 50.10 last 14.68)
 - mkdir 947 (time 1788689292.66 total 60.12 last 16.37)
total: 1000 mkdir in 66.04 seconds: 15.14 ops/second
 - mkdir 142 (time 1788689242.82 total 10.02 last 14.17)
 - mkdir 302 (time 1788689252.90 total 20.10 last 15.87)
 - mkdir 446 (time 1788689262.91 total 30.11 last 14.39)
 - mkdir 615 (time 1788689272.96 total 40.16 last 16.82)
 - mkdir 768 (time 1788689282.98 total 50.18 last 15.27)
 - mkdir 931 (time 1788689293.04 total 60.24 last 16.20)
total: 1000 mkdir in 66.96 seconds: 14.93 ops/second
Time for file creation 79 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788689307 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788689309 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788689308 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1788689309 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788689308 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788689309 ; total 1 ; last 1)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788689309 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1788689309 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1788689309 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1788689309 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
Time for file removal 40 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8