== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 20:11:13 (1788307873)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.58 seconds: 17.33 ops/second
 - mkdir 124 (time 1788307895.52 total 10.01 last 12.39)
 - mkdir 263 (time 1788307905.59 total 20.08 last 13.80)
 - mkdir 417 (time 1788307915.67 total 30.16 last 15.28)
 - mkdir 579 (time 1788307925.69 total 40.17 last 16.18)
 - mkdir 772 (time 1788307935.75 total 50.23 last 19.19)
 - mkdir 925 (time 1788307945.78 total 60.26 last 15.25)
total: 1000 mkdir in 63.92 seconds: 15.65 ops/second
 - mkdir 129 (time 1788307894.71 total 10.04 last 12.85)
 - mkdir 260 (time 1788307904.75 total 20.08 last 13.05)
 - mkdir 407 (time 1788307914.79 total 30.12 last 14.65)
 - mkdir 574 (time 1788307924.86 total 40.19 last 16.58)
 - mkdir 760 (time 1788307934.87 total 50.19 last 18.59)
 - mkdir 908 (time 1788307944.90 total 60.23 last 14.75)
total: 1000 mkdir in 65.67 seconds: 15.23 ops/second
 - mkdir 125 (time 1788307896.25 total 10.01 last 12.49)
 - mkdir 258 (time 1788307906.25 total 20.01 last 13.30)
 - mkdir 411 (time 1788307916.25 total 30.01 last 15.30)
 - mkdir 575 (time 1788307926.30 total 40.06 last 16.32)
 - mkdir 762 (time 1788307936.38 total 50.14 last 18.55)
 - mkdir 915 (time 1788307946.38 total 60.15 last 15.30)
total: 1000 mkdir in 67.00 seconds: 14.93 ops/second
 - mkdir 124 (time 1788307895.34 total 10.01 last 12.39)
 - mkdir 259 (time 1788307905.35 total 20.02 last 13.48)
 - mkdir 399 (time 1788307915.38 total 30.05 last 13.96)
 - mkdir 563 (time 1788307925.40 total 40.07 last 16.36)
 - mkdir 749 (time 1788307935.41 total 50.08 last 18.58)
 - mkdir 897 (time 1788307945.45 total 60.12 last 14.75)
total: 1000 mkdir in 68.36 seconds: 14.63 ops/second
 - mkdir 125 (time 1788307896.25 total 10.09 last 12.39)
 - mkdir 262 (time 1788307906.31 total 20.15 last 13.62)
 - mkdir 415 (time 1788307916.35 total 30.19 last 15.24)
 - mkdir 575 (time 1788307926.36 total 40.19 last 16.00)
 - mkdir 754 (time 1788307936.38 total 50.22 last 17.85)
 - mkdir 908 (time 1788307946.39 total 60.22 last 15.40)
total: 1000 mkdir in 68.63 seconds: 14.57 ops/second
 - mkdir 120 (time 1788307895.16 total 10.01 last 11.98)
 - mkdir 251 (time 1788307905.20 total 20.05 last 13.05)
 - mkdir 393 (time 1788307915.25 total 30.11 last 14.12)
 - mkdir 550 (time 1788307925.32 total 40.17 last 15.60)
 - mkdir 733 (time 1788307935.34 total 50.19 last 18.26)
 - mkdir 879 (time 1788307945.39 total 60.24 last 14.52)
 - mkdir 994 (time 1788307955.41 total 70.27 last 11.47)
total: 1000 mkdir in 70.95 seconds: 14.09 ops/second
 - mkdir 123 (time 1788307895.80 total 10.02 last 12.28)
 - mkdir 249 (time 1788307905.81 total 20.03 last 12.59)
 - mkdir 398 (time 1788307915.85 total 30.07 last 14.85)
 - mkdir 559 (time 1788307925.85 total 40.07 last 16.10)
 - mkdir 741 (time 1788307935.89 total 50.11 last 18.13)
 - mkdir 887 (time 1788307945.90 total 60.12 last 14.58)
 - mkdir 996 (time 1788307956.02 total 70.24 last 10.77)
total: 1000 mkdir in 70.58 seconds: 14.17 ops/second
 - mkdir 123 (time 1788307896.38 total 10.02 last 12.28)
 - mkdir 256 (time 1788307906.42 total 20.06 last 13.25)
 - mkdir 409 (time 1788307916.48 total 30.11 last 15.22)
 - mkdir 569 (time 1788307926.48 total 40.12 last 15.99)
 - mkdir 748 (time 1788307936.49 total 50.13 last 17.89)
 - mkdir 895 (time 1788307946.56 total 60.20 last 14.60)
 - mkdir 999 (time 1788307956.60 total 70.24 last 10.36)
total: 1000 mkdir in 70.24 seconds: 14.24 ops/second
 - mkdir 122 (time 1788307894.75 total 10.04 last 12.15)
 - mkdir 247 (time 1788307904.77 total 20.05 last 12.48)
 - mkdir 391 (time 1788307914.81 total 30.10 last 14.34)
 - mkdir 544 (time 1788307924.81 total 40.10 last 15.29)
 - mkdir 725 (time 1788307934.85 total 50.14 last 18.03)
 - mkdir 863 (time 1788307944.87 total 60.15 last 13.78)
 - mkdir 985 (time 1788307955.08 total 70.36 last 11.95)
total: 1000 mkdir in 72.26 seconds: 13.84 ops/second
 - mkdir 118 (time 1788307896.37 total 10.02 last 11.78)
 - mkdir 241 (time 1788307906.40 total 20.04 last 12.27)
 - mkdir 388 (time 1788307916.47 total 30.12 last 14.59)
 - mkdir 543 (time 1788307926.50 total 40.15 last 15.45)
 - mkdir 720 (time 1788307936.53 total 50.17 last 17.66)
 - mkdir 867 (time 1788307946.57 total 60.22 last 14.63)
 - mkdir 967 (time 1788307956.63 total 70.27 last 9.95)
total: 1000 mkdir in 72.97 seconds: 13.71 ops/second
Time for file creation 83 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.51 seconds: 19.46 ops/second
 - mkdir 133 (time 1788308482.75 total 10.03 last 13.26)
 - mkdir 305 (time 1788308492.81 total 20.09 last 17.09)
 - mkdir 488 (time 1788308502.81 total 30.10 last 18.30)
 - mkdir 695 (time 1788308512.84 total 40.12 last 20.64)
 - mkdir 900 (time 1788308522.85 total 50.13 last 20.50)
total: 1000 mkdir in 54.68 seconds: 18.29 ops/second
 - mkdir 108 (time 1788308480.98 total 10.01 last 10.78)
 - mkdir 281 (time 1788308491.02 total 20.05 last 17.23)
 - mkdir 459 (time 1788308501.03 total 30.06 last 17.79)
 - mkdir 665 (time 1788308511.08 total 40.11 last 20.49)
 - mkdir 870 (time 1788308521.11 total 50.14 last 20.45)
total: 1000 mkdir in 56.47 seconds: 17.71 ops/second
 - mkdir 128 (time 1788308482.46 total 10.01 last 12.78)
 - mkdir 296 (time 1788308492.48 total 20.03 last 16.77)
 - mkdir 481 (time 1788308502.54 total 30.09 last 18.38)
 - mkdir 689 (time 1788308512.55 total 40.10 last 20.78)
 - mkdir 895 (time 1788308522.56 total 50.11 last 20.58)
total: 1000 mkdir in 55.07 seconds: 18.16 ops/second
 - mkdir 128 (time 1788308482.68 total 10.05 last 12.74)
 - mkdir 295 (time 1788308492.71 total 20.08 last 16.65)
 - mkdir 480 (time 1788308502.72 total 30.08 last 18.50)
 - mkdir 689 (time 1788308512.75 total 40.11 last 20.83)
 - mkdir 896 (time 1788308522.77 total 50.13 last 20.67)
total: 1000 mkdir in 55.06 seconds: 18.16 ops/second
 - mkdir 142 (time 1788308483.63 total 10.01 last 14.18)
 - mkdir 308 (time 1788308493.69 total 20.07 last 16.51)
 - mkdir 501 (time 1788308503.74 total 30.12 last 19.20)
 - mkdir 711 (time 1788308513.77 total 40.15 last 20.94)
 - mkdir 927 (time 1788308523.78 total 50.16 last 21.59)
total: 1000 mkdir in 54.20 seconds: 18.45 ops/second
 - mkdir 151 (time 1788308484.67 total 10.03 last 15.05)
 - mkdir 327 (time 1788308494.68 total 20.04 last 17.58)
 - mkdir 526 (time 1788308504.70 total 30.06 last 19.85)
 - mkdir 731 (time 1788308514.75 total 40.11 last 20.41)
 - mkdir 943 (time 1788308524.75 total 50.11 last 21.19)
total: 1000 mkdir in 53.44 seconds: 18.71 ops/second
 - mkdir 146 (time 1788308484.22 total 10.08 last 14.49)
 - mkdir 306 (time 1788308494.24 total 20.10 last 15.96)
 - mkdir 498 (time 1788308504.30 total 30.16 last 19.08)
 - mkdir 704 (time 1788308514.34 total 40.20 last 20.52)
 - mkdir 922 (time 1788308524.39 total 50.25 last 21.71)
total: 1000 mkdir in 54.97 seconds: 18.19 ops/second
 - mkdir 147 (time 1788308484.26 total 10.04 last 14.64)
 - mkdir 310 (time 1788308494.30 total 20.09 last 16.23)
 - mkdir 501 (time 1788308504.31 total 30.10 last 19.08)
 - mkdir 701 (time 1788308514.36 total 40.14 last 19.91)
 - mkdir 916 (time 1788308524.38 total 50.17 last 21.45)
total: 1000 mkdir in 55.25 seconds: 18.10 ops/second
 - mkdir 142 (time 1788308484.28 total 10.01 last 14.19)
 - mkdir 306 (time 1788308494.35 total 20.07 last 16.29)
 - mkdir 492 (time 1788308504.38 total 30.10 last 18.55)
 - mkdir 694 (time 1788308514.40 total 40.12 last 20.16)
 - mkdir 912 (time 1788308524.46 total 50.18 last 21.66)
total: 1000 mkdir in 55.81 seconds: 17.92 ops/second
 - mkdir 130 (time 1788308483.48 total 10.03 last 12.96)
 - mkdir 289 (time 1788308493.54 total 20.09 last 15.80)
 - mkdir 468 (time 1788308503.57 total 30.11 last 17.86)
 - mkdir 671 (time 1788308513.58 total 40.12 last 20.28)
 - mkdir 878 (time 1788308523.58 total 50.13 last 20.69)
total: 1000 mkdir in 57.04 seconds: 17.53 ops/second
Time for file creation 68 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788308539 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1788308539 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788308540 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788308539 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788308540 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788308539 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788308540 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788308540 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788308540 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788308540 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
Time for file removal 43 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8