== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 08:41:06 (1789044066)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.47 seconds: 21.27 ops/second
 - mkdir 131 (time 1789044086.91 total 10.09 last 12.98)
 - mkdir 282 (time 1789044096.96 total 20.14 last 15.03)
 - mkdir 432 (time 1789044107.03 total 30.21 last 14.90)
 - mkdir 585 (time 1789044117.05 total 40.23 last 15.27)
 - mkdir 742 (time 1789044127.07 total 50.25 last 15.67)
 - mkdir 891 (time 1789044137.10 total 60.28 last 14.85)
total: 1000 mkdir in 67.31 seconds: 14.86 ops/second
 - mkdir 125 (time 1789044085.30 total 10.02 last 12.47)
 - mkdir 261 (time 1789044095.30 total 20.03 last 13.59)
 - mkdir 414 (time 1789044105.32 total 30.05 last 15.27)
 - mkdir 567 (time 1789044115.35 total 40.08 last 15.25)
 - mkdir 719 (time 1789044125.37 total 50.10 last 15.18)
 - mkdir 860 (time 1789044135.41 total 60.14 last 14.04)
total: 1000 mkdir in 69.07 seconds: 14.48 ops/second
 - mkdir 132 (time 1789044086.66 total 10.07 last 13.11)
 - mkdir 275 (time 1789044096.69 total 20.10 last 14.25)
 - mkdir 431 (time 1789044106.73 total 30.14 last 15.54)
 - mkdir 584 (time 1789044116.77 total 40.18 last 15.25)
 - mkdir 732 (time 1789044126.78 total 50.19 last 14.79)
 - mkdir 875 (time 1789044136.80 total 60.21 last 14.27)
total: 1000 mkdir in 68.79 seconds: 14.54 ops/second
 - mkdir 130 (time 1789044088.30 total 10.03 last 12.96)
 - mkdir 284 (time 1789044098.33 total 20.06 last 15.35)
 - mkdir 448 (time 1789044108.36 total 30.09 last 16.35)
 - mkdir 597 (time 1789044118.45 total 40.18 last 14.77)
 - mkdir 750 (time 1789044128.47 total 50.20 last 15.26)
 - mkdir 903 (time 1789044138.48 total 60.21 last 15.29)
total: 1000 mkdir in 67.39 seconds: 14.84 ops/second
 - mkdir 124 (time 1789044086.77 total 10.02 last 12.37)
 - mkdir 270 (time 1789044096.80 total 20.06 last 14.55)
 - mkdir 428 (time 1789044106.84 total 30.10 last 15.74)
 - mkdir 577 (time 1789044116.87 total 40.13 last 14.86)
 - mkdir 727 (time 1789044126.90 total 50.16 last 14.95)
 - mkdir 873 (time 1789044136.97 total 60.22 last 14.50)
total: 1000 mkdir in 69.01 seconds: 14.49 ops/second
 - mkdir 126 (time 1789044087.41 total 10.06 last 12.53)
 - mkdir 280 (time 1789044097.51 total 20.16 last 15.24)
 - mkdir 438 (time 1789044107.58 total 30.23 last 15.69)
 - mkdir 590 (time 1789044117.61 total 40.26 last 15.16)
 - mkdir 742 (time 1789044127.68 total 50.32 last 15.10)
 - mkdir 888 (time 1789044137.73 total 60.38 last 14.52)
total: 1000 mkdir in 68.88 seconds: 14.52 ops/second
 - mkdir 129 (time 1789044087.54 total 10.07 last 12.81)
 - mkdir 272 (time 1789044097.58 total 20.11 last 14.24)
 - mkdir 431 (time 1789044107.60 total 30.13 last 15.87)
 - mkdir 579 (time 1789044117.61 total 40.15 last 14.78)
 - mkdir 728 (time 1789044127.70 total 50.24 last 14.77)
 - mkdir 875 (time 1789044137.73 total 60.27 last 14.65)
total: 1000 mkdir in 69.57 seconds: 14.37 ops/second
 - mkdir 125 (time 1789044088.03 total 10.03 last 12.47)
 - mkdir 276 (time 1789044098.10 total 20.09 last 15.01)
 - mkdir 442 (time 1789044108.14 total 30.14 last 16.52)
 - mkdir 585 (time 1789044118.16 total 40.15 last 14.28)
 - mkdir 738 (time 1789044128.22 total 50.21 last 15.21)
 - mkdir 890 (time 1789044138.25 total 60.25 last 15.15)
total: 1000 mkdir in 69.27 seconds: 14.44 ops/second
 - mkdir 127 (time 1789044087.72 total 10.04 last 12.66)
 - mkdir 276 (time 1789044097.79 total 20.11 last 14.79)
 - mkdir 436 (time 1789044107.84 total 30.15 last 15.94)
 - mkdir 584 (time 1789044117.89 total 40.20 last 14.73)
 - mkdir 736 (time 1789044127.93 total 50.24 last 15.14)
 - mkdir 882 (time 1789044137.93 total 60.25 last 14.59)
total: 1000 mkdir in 69.81 seconds: 14.32 ops/second
 - mkdir 119 (time 1789044088.06 total 10.06 last 11.83)
 - mkdir 256 (time 1789044098.14 total 20.13 last 13.60)
 - mkdir 410 (time 1789044108.15 total 30.14 last 15.39)
 - mkdir 558 (time 1789044118.22 total 40.22 last 14.68)
 - mkdir 713 (time 1789044128.24 total 50.23 last 15.48)
 - mkdir 864 (time 1789044138.25 total 60.24 last 15.09)
 - mkdir 980 (time 1789044148.31 total 70.31 last 11.52)
total: 1000 mkdir in 71.63 seconds: 13.96 ops/second
Time for file creation 81 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 12 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.61 seconds: 16.41 ops/second
 - mkdir 137 (time 1789044721.20 total 10.06 last 13.62)
 - mkdir 281 (time 1789044731.36 total 20.21 last 14.18)
 - mkdir 416 (time 1789044741.42 total 30.28 last 13.42)
 - mkdir 554 (time 1789044751.48 total 40.34 last 13.71)
 - mkdir 712 (time 1789044761.52 total 50.38 last 15.74)
 - mkdir 887 (time 1789044771.57 total 60.43 last 17.41)
total: 1000 mkdir in 67.71 seconds: 14.77 ops/second
 - mkdir 116 (time 1789044719.58 total 10.03 last 11.57)
 - mkdir 258 (time 1789044729.67 total 20.12 last 14.07)
 - mkdir 398 (time 1789044739.68 total 30.13 last 13.98)
 - mkdir 538 (time 1789044749.76 total 40.21 last 13.89)
 - mkdir 684 (time 1789044759.79 total 50.24 last 14.57)
 - mkdir 858 (time 1789044769.84 total 60.29 last 17.31)
total: 1000 mkdir in 69.84 seconds: 14.32 ops/second
 - mkdir 118 (time 1789044720.56 total 10.02 last 11.78)
 - mkdir 266 (time 1789044730.61 total 20.07 last 14.72)
 - mkdir 402 (time 1789044740.62 total 30.08 last 13.59)
 - mkdir 541 (time 1789044750.68 total 40.14 last 13.82)
 - mkdir 690 (time 1789044760.69 total 50.15 last 14.88)
 - mkdir 863 (time 1789044770.72 total 60.17 last 17.26)
total: 1000 mkdir in 69.90 seconds: 14.31 ops/second
 - mkdir 115 (time 1789044719.62 total 10.04 last 11.45)
 - mkdir 252 (time 1789044729.65 total 20.07 last 13.66)
 - mkdir 388 (time 1789044739.66 total 30.08 last 13.58)
 - mkdir 531 (time 1789044749.73 total 40.15 last 14.20)
 - mkdir 668 (time 1789044759.77 total 50.19 last 13.64)
 - mkdir 844 (time 1789044769.82 total 60.24 last 17.52)
 - mkdir 994 (time 1789044779.95 total 70.37 last 14.80)
total: 1000 mkdir in 71.28 seconds: 14.03 ops/second
 - mkdir 125 (time 1789044721.20 total 10.02 last 12.47)
 - mkdir 262 (time 1789044731.33 total 20.16 last 13.52)
 - mkdir 400 (time 1789044741.34 total 30.17 last 13.79)
 - mkdir 539 (time 1789044751.37 total 40.20 last 13.86)
 - mkdir 688 (time 1789044761.40 total 50.22 last 14.86)
 - mkdir 859 (time 1789044771.43 total 60.25 last 17.05)
 - mkdir 993 (time 1789044781.45 total 70.27 last 13.38)
total: 1000 mkdir in 71.16 seconds: 14.05 ops/second
 - mkdir 123 (time 1789044720.76 total 10.03 last 12.26)
 - mkdir 259 (time 1789044730.80 total 20.08 last 13.54)
 - mkdir 393 (time 1789044740.86 total 30.13 last 13.32)
 - mkdir 531 (time 1789044750.89 total 40.16 last 13.77)
 - mkdir 676 (time 1789044760.90 total 50.17 last 14.48)
 - mkdir 850 (time 1789044770.92 total 60.19 last 17.37)
 - mkdir 985 (time 1789044780.95 total 70.22 last 13.46)
total: 1000 mkdir in 72.06 seconds: 13.88 ops/second
 - mkdir 128 (time 1789044720.80 total 10.00 last 12.80)
 - mkdir 264 (time 1789044730.83 total 20.03 last 13.56)
 - mkdir 399 (time 1789044740.88 total 30.08 last 13.44)
 - mkdir 539 (time 1789044750.91 total 40.11 last 13.96)
 - mkdir 685 (time 1789044760.97 total 50.17 last 14.51)
 - mkdir 856 (time 1789044770.99 total 60.19 last 17.05)
 - mkdir 986 (time 1789044781.13 total 70.33 last 12.83)
total: 1000 mkdir in 72.06 seconds: 13.88 ops/second
 - mkdir 119 (time 1789044720.59 total 10.02 last 11.88)
 - mkdir 262 (time 1789044730.66 total 20.09 last 14.20)
 - mkdir 392 (time 1789044740.69 total 30.12 last 12.96)
 - mkdir 533 (time 1789044750.79 total 40.21 last 13.97)
 - mkdir 680 (time 1789044760.84 total 50.26 last 14.63)
 - mkdir 847 (time 1789044770.92 total 60.34 last 16.57)
 - mkdir 978 (time 1789044780.99 total 70.42 last 13.00)
total: 1000 mkdir in 73.14 seconds: 13.67 ops/second
 - mkdir 109 (time 1789044719.08 total 10.02 last 10.87)
 - mkdir 241 (time 1789044729.14 total 20.08 last 13.13)
 - mkdir 370 (time 1789044739.17 total 30.11 last 12.86)
 - mkdir 507 (time 1789044749.24 total 40.18 last 13.60)
 - mkdir 645 (time 1789044759.35 total 50.29 last 13.66)
 - mkdir 819 (time 1789044769.37 total 60.31 last 17.36)
 - mkdir 967 (time 1789044779.49 total 70.43 last 14.62)
total: 1000 mkdir in 74.79 seconds: 13.37 ops/second
 - mkdir 120 (time 1789044721.29 total 10.04 last 11.95)
 - mkdir 250 (time 1789044731.31 total 20.06 last 12.98)
 - mkdir 378 (time 1789044741.31 total 30.06 last 12.79)
 - mkdir 511 (time 1789044751.33 total 40.08 last 13.28)
 - mkdir 662 (time 1789044761.34 total 50.09 last 15.08)
 - mkdir 831 (time 1789044771.39 total 60.13 last 16.83)
 - mkdir 953 (time 1789044781.39 total 70.14 last 12.20)
total: 1000 mkdir in 74.63 seconds: 13.40 ops/second
Time for file creation 87 sec for 1 RPC sent at a time
 - unlinked 0 (time 1789044793 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1789044794 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1789044794 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1789044794 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1789044795 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1789044795 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1789044795 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1789044795 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1789044795 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1789044796 ; total 1 ; last 1)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
Time for file removal 47 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8