== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 14:28:21 (1788200901)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.29 seconds: 34.01 ops/second
 - mkdir 154 (time 1788200919.54 total 10.06 last 15.30)
 - mkdir 314 (time 1788200929.59 total 20.11 last 15.93)
 - mkdir 458 (time 1788200939.66 total 30.18 last 14.30)
 - mkdir 605 (time 1788200949.74 total 40.26 last 14.58)
 - mkdir 737 (time 1788200959.77 total 50.29 last 13.17)
 - mkdir 893 (time 1788200969.77 total 60.29 last 15.59)
total: 1000 mkdir in 66.21 seconds: 15.10 ops/second
 - mkdir 138 (time 1788200921.88 total 10.07 last 13.70)
 - mkdir 307 (time 1788200931.89 total 20.08 last 16.88)
 - mkdir 464 (time 1788200941.89 total 30.08 last 15.70)
 - mkdir 607 (time 1788200951.93 total 40.12 last 14.25)
 - mkdir 733 (time 1788200961.95 total 50.14 last 12.57)
 - mkdir 890 (time 1788200971.97 total 60.17 last 15.65)
total: 1000 mkdir in 67.59 seconds: 14.80 ops/second
 - mkdir 141 (time 1788200921.89 total 10.06 last 14.01)
 - mkdir 308 (time 1788200931.91 total 20.09 last 16.66)
 - mkdir 454 (time 1788200941.94 total 30.12 last 14.55)
 - mkdir 603 (time 1788200951.96 total 40.14 last 14.87)
 - mkdir 732 (time 1788200961.99 total 50.17 last 12.87)
 - mkdir 889 (time 1788200972.04 total 60.22 last 15.62)
total: 1000 mkdir in 67.84 seconds: 14.74 ops/second
 - mkdir 131 (time 1788200921.13 total 10.04 last 13.05)
 - mkdir 294 (time 1788200931.14 total 20.05 last 16.28)
 - mkdir 436 (time 1788200941.19 total 30.09 last 14.14)
 - mkdir 586 (time 1788200951.19 total 40.10 last 14.99)
 - mkdir 719 (time 1788200961.22 total 50.13 last 13.26)
 - mkdir 878 (time 1788200971.25 total 60.16 last 15.85)
total: 1000 mkdir in 68.73 seconds: 14.55 ops/second
 - mkdir 130 (time 1788200921.54 total 10.00 last 13.00)
 - mkdir 294 (time 1788200931.57 total 20.03 last 16.36)
 - mkdir 445 (time 1788200941.60 total 30.05 last 15.06)
 - mkdir 590 (time 1788200951.65 total 40.11 last 14.43)
 - mkdir 715 (time 1788200961.65 total 50.11 last 12.49)
 - mkdir 872 (time 1788200971.76 total 60.22 last 15.53)
 - mkdir 996 (time 1788200981.80 total 70.26 last 12.35)
total: 1000 mkdir in 70.80 seconds: 14.12 ops/second
 - mkdir 133 (time 1788200921.80 total 10.07 last 13.21)
 - mkdir 297 (time 1788200931.81 total 20.07 last 16.39)
 - mkdir 441 (time 1788200941.85 total 30.11 last 14.34)
 - mkdir 589 (time 1788200951.89 total 40.16 last 14.73)
 - mkdir 714 (time 1788200961.93 total 50.20 last 12.45)
 - mkdir 872 (time 1788200971.95 total 60.21 last 15.78)
 - mkdir 994 (time 1788200982.07 total 70.34 last 12.05)
total: 1000 mkdir in 71.22 seconds: 14.04 ops/second
 - mkdir 135 (time 1788200922.96 total 10.06 last 13.41)
 - mkdir 306 (time 1788200932.97 total 20.07 last 17.09)
 - mkdir 460 (time 1788200943.08 total 30.18 last 15.23)
 - mkdir 600 (time 1788200953.08 total 40.18 last 13.99)
 - mkdir 730 (time 1788200963.13 total 50.23 last 12.93)
 - mkdir 893 (time 1788200973.14 total 60.24 last 16.29)
 - mkdir 999 (time 1788200983.24 total 70.34 last 10.50)
total: 1000 mkdir in 70.34 seconds: 14.22 ops/second
 - mkdir 138 (time 1788200922.79 total 10.14 last 13.61)
 - mkdir 303 (time 1788200932.81 total 20.16 last 16.48)
 - mkdir 453 (time 1788200942.85 total 30.21 last 14.93)
 - mkdir 594 (time 1788200952.87 total 40.22 last 14.07)
 - mkdir 724 (time 1788200962.94 total 50.29 last 12.91)
 - mkdir 884 (time 1788200972.95 total 60.30 last 15.98)
 - mkdir 991 (time 1788200983.08 total 70.43 last 10.57)
total: 1000 mkdir in 71.40 seconds: 14.01 ops/second
 - mkdir 130 (time 1788200921.58 total 10.05 last 12.94)
 - mkdir 287 (time 1788200931.69 total 20.16 last 15.53)
 - mkdir 429 (time 1788200941.71 total 30.17 last 14.18)
 - mkdir 576 (time 1788200951.75 total 40.22 last 14.64)
 - mkdir 701 (time 1788200961.81 total 50.28 last 12.42)
 - mkdir 858 (time 1788200971.85 total 60.31 last 15.65)
 - mkdir 977 (time 1788200981.94 total 70.41 last 11.79)
total: 1000 mkdir in 72.96 seconds: 13.71 ops/second
 - mkdir 125 (time 1788200921.76 total 10.01 last 12.49)
 - mkdir 278 (time 1788200931.76 total 20.01 last 15.30)
 - mkdir 418 (time 1788200941.80 total 30.05 last 13.95)
 - mkdir 566 (time 1788200951.81 total 40.05 last 14.79)
 - mkdir 690 (time 1788200961.86 total 50.11 last 12.34)
 - mkdir 849 (time 1788200971.86 total 60.11 last 15.90)
 - mkdir 972 (time 1788200981.91 total 70.15 last 12.25)
total: 1000 mkdir in 73.07 seconds: 13.69 ops/second
Time for file creation 82 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 14 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.41 seconds: 24.43 ops/second
 - mkdir 148 (time 1788201508.92 total 10.01 last 14.78)
 - mkdir 374 (time 1788201518.93 total 20.02 last 22.59)
 - mkdir 561 (time 1788201529.01 total 30.10 last 18.55)
 - mkdir 771 (time 1788201539.02 total 40.11 last 20.98)
 - mkdir 977 (time 1788201549.03 total 50.12 last 20.58)
total: 1000 mkdir in 51.18 seconds: 19.54 ops/second
 - mkdir 146 (time 1788201509.72 total 10.03 last 14.56)
 - mkdir 386 (time 1788201519.75 total 20.06 last 23.92)
 - mkdir 574 (time 1788201529.75 total 30.06 last 18.80)
 - mkdir 785 (time 1788201539.76 total 40.07 last 21.07)
 - mkdir 991 (time 1788201549.77 total 50.08 last 20.59)
total: 1000 mkdir in 50.46 seconds: 19.82 ops/second
 - mkdir 143 (time 1788201509.42 total 10.07 last 14.20)
 - mkdir 376 (time 1788201519.43 total 20.08 last 23.28)
 - mkdir 559 (time 1788201529.45 total 30.10 last 18.26)
 - mkdir 766 (time 1788201539.48 total 40.14 last 20.63)
 - mkdir 973 (time 1788201549.54 total 50.19 last 20.58)
total: 1000 mkdir in 52.48 seconds: 19.05 ops/second
 - mkdir 145 (time 1788201508.96 total 10.04 last 14.45)
 - mkdir 374 (time 1788201519.00 total 20.08 last 22.80)
 - mkdir 555 (time 1788201529.02 total 30.09 last 18.07)
 - mkdir 765 (time 1788201539.05 total 40.13 last 20.93)
 - mkdir 964 (time 1788201549.11 total 50.18 last 19.79)
total: 1000 mkdir in 52.98 seconds: 18.87 ops/second
 - mkdir 140 (time 1788201508.16 total 10.01 last 13.98)
 - mkdir 356 (time 1788201518.17 total 20.02 last 21.58)
 - mkdir 546 (time 1788201528.19 total 30.04 last 18.96)
 - mkdir 745 (time 1788201538.21 total 40.06 last 19.86)
 - mkdir 953 (time 1788201548.25 total 50.10 last 20.73)
total: 1000 mkdir in 53.87 seconds: 18.56 ops/second
 - mkdir 138 (time 1788201509.62 total 10.02 last 13.78)
 - mkdir 370 (time 1788201519.67 total 20.06 last 23.10)
 - mkdir 549 (time 1788201529.68 total 30.08 last 17.87)
 - mkdir 762 (time 1788201539.71 total 40.11 last 21.23)
 - mkdir 972 (time 1788201549.74 total 50.14 last 20.95)
total: 1000 mkdir in 53.10 seconds: 18.83 ops/second
 - mkdir 145 (time 1788201509.82 total 10.03 last 14.45)
 - mkdir 374 (time 1788201519.82 total 20.04 last 22.89)
 - mkdir 556 (time 1788201529.83 total 30.05 last 18.18)
 - mkdir 762 (time 1788201539.87 total 40.08 last 20.53)
 - mkdir 968 (time 1788201549.88 total 50.09 last 20.58)
total: 1000 mkdir in 53.58 seconds: 18.66 ops/second
 - mkdir 136 (time 1788201508.70 total 10.07 last 13.51)
 - mkdir 357 (time 1788201518.70 total 20.07 last 22.09)
 - mkdir 541 (time 1788201528.73 total 30.10 last 18.35)
 - mkdir 746 (time 1788201538.75 total 40.12 last 20.46)
 - mkdir 947 (time 1788201548.76 total 50.13 last 20.07)
total: 1000 mkdir in 54.78 seconds: 18.25 ops/second
 - mkdir 142 (time 1788201509.81 total 10.02 last 14.17)
 - mkdir 372 (time 1788201519.82 total 20.03 last 22.99)
 - mkdir 550 (time 1788201529.87 total 30.09 last 17.70)
 - mkdir 759 (time 1788201539.90 total 40.12 last 20.84)
 - mkdir 965 (time 1788201549.93 total 50.14 last 20.55)
total: 1000 mkdir in 54.18 seconds: 18.46 ops/second
 - mkdir 141 (time 1788201510.11 total 10.03 last 14.06)
 - mkdir 370 (time 1788201520.15 total 20.07 last 22.80)
 - mkdir 547 (time 1788201530.17 total 30.09 last 17.67)
 - mkdir 757 (time 1788201540.19 total 40.11 last 20.96)
 - mkdir 959 (time 1788201550.30 total 50.22 last 19.98)
total: 1000 mkdir in 55.13 seconds: 18.14 ops/second
Time for file creation 68 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788201563 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788201564 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788201564 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788201565 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788201565 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788201565 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788201566 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788201565 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788201566 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788201566 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
Time for file removal 43 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8