== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 17:26:00 (1788643560)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.78 seconds: 12.74 ops/second
 - mkdir 115 (time 1788643582.58 total 10.06 last 11.43)
 - mkdir 222 (time 1788643592.68 total 20.16 last 10.59)
 - mkdir 345 (time 1788643602.73 total 30.21 last 12.24)
 - mkdir 499 (time 1788643612.76 total 40.25 last 15.34)
 - mkdir 709 (time 1788643622.80 total 50.28 last 20.93)
 - mkdir 883 (time 1788643632.83 total 60.32 last 17.33)
total: 1000 mkdir in 66.19 seconds: 15.11 ops/second
 - mkdir 120 (time 1788643583.80 total 10.04 last 11.95)
 - mkdir 231 (time 1788643593.82 total 20.06 last 11.07)
 - mkdir 357 (time 1788643603.87 total 30.12 last 12.53)
 - mkdir 524 (time 1788643613.89 total 40.13 last 16.67)
 - mkdir 726 (time 1788643623.92 total 50.16 last 20.14)
 - mkdir 903 (time 1788643633.94 total 60.18 last 17.67)
total: 1000 mkdir in 65.55 seconds: 15.25 ops/second
 - mkdir 121 (time 1788643585.66 total 10.02 last 12.08)
 - mkdir 240 (time 1788643595.68 total 20.03 last 11.89)
 - mkdir 373 (time 1788643605.74 total 30.09 last 13.22)
 - mkdir 545 (time 1788643615.78 total 40.14 last 17.13)
 - mkdir 755 (time 1788643625.80 total 50.15 last 20.96)
 - mkdir 931 (time 1788643635.81 total 60.16 last 17.58)
total: 1000 mkdir in 64.55 seconds: 15.49 ops/second
 - mkdir 115 (time 1788643584.12 total 10.10 last 11.39)
 - mkdir 225 (time 1788643594.12 total 20.10 last 11.00)
 - mkdir 348 (time 1788643604.15 total 30.13 last 12.27)
 - mkdir 514 (time 1788643614.16 total 40.14 last 16.59)
 - mkdir 717 (time 1788643624.16 total 50.14 last 20.29)
 - mkdir 891 (time 1788643634.18 total 60.16 last 17.37)
total: 1000 mkdir in 66.67 seconds: 15.00 ops/second
 - mkdir 118 (time 1788643585.10 total 10.02 last 11.78)
 - mkdir 234 (time 1788643595.16 total 20.08 last 11.53)
 - mkdir 358 (time 1788643605.17 total 30.09 last 12.38)
 - mkdir 533 (time 1788643615.23 total 40.14 last 17.41)
 - mkdir 741 (time 1788643625.27 total 50.19 last 20.70)
 - mkdir 921 (time 1788643635.38 total 60.29 last 17.82)
total: 1000 mkdir in 66.08 seconds: 15.13 ops/second
 - mkdir 118 (time 1788643584.88 total 10.09 last 11.69)
 - mkdir 232 (time 1788643594.96 total 20.17 last 11.31)
 - mkdir 356 (time 1788643605.02 total 30.23 last 12.32)
 - mkdir 521 (time 1788643615.06 total 40.27 last 16.44)
 - mkdir 730 (time 1788643625.07 total 50.28 last 20.88)
 - mkdir 911 (time 1788643635.09 total 60.30 last 18.06)
total: 1000 mkdir in 66.39 seconds: 15.06 ops/second
 - mkdir 117 (time 1788643584.79 total 10.05 last 11.64)
 - mkdir 231 (time 1788643594.82 total 20.08 last 11.36)
 - mkdir 356 (time 1788643604.86 total 30.12 last 12.45)
 - mkdir 528 (time 1788643614.88 total 40.14 last 17.17)
 - mkdir 735 (time 1788643624.95 total 50.21 last 20.55)
 - mkdir 910 (time 1788643634.99 total 60.25 last 17.43)
total: 1000 mkdir in 66.63 seconds: 15.01 ops/second
 - mkdir 120 (time 1788643585.68 total 10.00 last 11.99)
 - mkdir 233 (time 1788643595.73 total 20.06 last 11.24)
 - mkdir 359 (time 1788643605.74 total 30.07 last 12.59)
 - mkdir 533 (time 1788643615.75 total 40.08 last 17.38)
 - mkdir 744 (time 1788643625.76 total 50.09 last 21.09)
 - mkdir 921 (time 1788643635.82 total 60.15 last 17.59)
total: 1000 mkdir in 66.18 seconds: 15.11 ops/second
 - mkdir 119 (time 1788643585.44 total 10.03 last 11.86)
 - mkdir 235 (time 1788643595.45 total 20.04 last 11.59)
 - mkdir 357 (time 1788643605.45 total 30.05 last 12.20)
 - mkdir 529 (time 1788643615.51 total 40.10 last 17.11)
 - mkdir 734 (time 1788643625.51 total 50.10 last 20.50)
 - mkdir 914 (time 1788643635.60 total 60.19 last 17.84)
total: 1000 mkdir in 66.68 seconds: 15.00 ops/second
 - mkdir 106 (time 1788643585.37 total 10.09 last 10.51)
 - mkdir 216 (time 1788643595.40 total 20.13 last 10.96)
 - mkdir 335 (time 1788643605.47 total 30.19 last 11.82)
 - mkdir 505 (time 1788643615.48 total 40.20 last 16.98)
 - mkdir 711 (time 1788643625.53 total 50.25 last 20.51)
 - mkdir 891 (time 1788643635.53 total 60.25 last 17.99)
total: 1000 mkdir in 68.61 seconds: 14.58 ops/second
Time for file creation 81 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 8 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.38 seconds: 26.40 ops/second
 - mkdir 161 (time 1788644107.49 total 10.01 last 16.08)
 - mkdir 302 (time 1788644117.51 total 20.03 last 14.07)
 - mkdir 427 (time 1788644127.52 total 30.04 last 12.48)
 - mkdir 575 (time 1788644137.55 total 40.07 last 14.77)
 - mkdir 718 (time 1788644147.55 total 50.08 last 14.29)
 - mkdir 902 (time 1788644157.58 total 60.10 last 18.36)
total: 1000 mkdir in 64.79 seconds: 15.43 ops/second
 - mkdir 156 (time 1788644106.22 total 10.05 last 15.53)
 - mkdir 300 (time 1788644116.30 total 20.12 last 14.29)
 - mkdir 421 (time 1788644126.32 total 30.14 last 12.08)
 - mkdir 568 (time 1788644136.34 total 40.17 last 14.66)
 - mkdir 707 (time 1788644146.37 total 50.19 last 13.86)
 - mkdir 885 (time 1788644156.39 total 60.22 last 17.76)
total: 1000 mkdir in 66.21 seconds: 15.10 ops/second
 - mkdir 163 (time 1788644108.24 total 10.06 last 16.21)
 - mkdir 296 (time 1788644118.30 total 20.12 last 13.22)
 - mkdir 433 (time 1788644128.34 total 30.16 last 13.65)
 - mkdir 572 (time 1788644138.40 total 40.22 last 13.82)
 - mkdir 732 (time 1788644148.43 total 50.25 last 15.95)
 - mkdir 900 (time 1788644158.45 total 60.27 last 16.77)
total: 1000 mkdir in 66.06 seconds: 15.14 ops/second
 - mkdir 159 (time 1788644108.45 total 10.07 last 15.80)
 - mkdir 284 (time 1788644118.50 total 20.11 last 12.44)
 - mkdir 422 (time 1788644128.52 total 30.14 last 13.76)
 - mkdir 558 (time 1788644138.57 total 40.18 last 13.54)
 - mkdir 723 (time 1788644148.58 total 50.19 last 16.48)
 - mkdir 899 (time 1788644158.65 total 60.26 last 17.48)
total: 1000 mkdir in 66.57 seconds: 15.02 ops/second
 - mkdir 162 (time 1788644108.23 total 10.01 last 16.19)
 - mkdir 291 (time 1788644118.29 total 20.06 last 12.83)
 - mkdir 423 (time 1788644128.29 total 30.06 last 13.19)
 - mkdir 566 (time 1788644138.36 total 40.13 last 14.20)
 - mkdir 722 (time 1788644148.39 total 50.16 last 15.56)
 - mkdir 896 (time 1788644158.45 total 60.22 last 17.29)
total: 1000 mkdir in 66.92 seconds: 14.94 ops/second
 - mkdir 160 (time 1788644107.94 total 10.01 last 15.98)
 - mkdir 278 (time 1788644117.98 total 20.05 last 11.76)
 - mkdir 402 (time 1788644127.99 total 30.06 last 12.39)
 - mkdir 544 (time 1788644138.05 total 40.13 last 14.10)
 - mkdir 696 (time 1788644148.06 total 50.14 last 15.19)
 - mkdir 882 (time 1788644158.09 total 60.17 last 18.54)
total: 1000 mkdir in 68.35 seconds: 14.63 ops/second
 - mkdir 146 (time 1788644107.79 total 10.01 last 14.58)
 - mkdir 272 (time 1788644117.85 total 20.07 last 12.53)
 - mkdir 395 (time 1788644127.87 total 30.09 last 12.27)
 - mkdir 544 (time 1788644137.97 total 40.19 last 14.75)
 - mkdir 695 (time 1788644147.98 total 50.20 last 15.09)
 - mkdir 875 (time 1788644158.00 total 60.22 last 17.96)
total: 1000 mkdir in 69.30 seconds: 14.43 ops/second
 - mkdir 149 (time 1788644107.83 total 10.00 last 14.90)
 - mkdir 280 (time 1788644117.85 total 20.02 last 13.08)
 - mkdir 405 (time 1788644127.86 total 30.03 last 12.48)
 - mkdir 546 (time 1788644137.86 total 40.03 last 14.10)
 - mkdir 691 (time 1788644147.87 total 50.04 last 14.49)
 - mkdir 872 (time 1788644157.89 total 60.06 last 18.07)
total: 1000 mkdir in 69.55 seconds: 14.38 ops/second
 - mkdir 151 (time 1788644108.02 total 10.04 last 15.04)
 - mkdir 273 (time 1788644118.03 total 20.05 last 12.19)
 - mkdir 397 (time 1788644128.04 total 30.06 last 12.39)
 - mkdir 541 (time 1788644138.09 total 40.11 last 14.32)
 - mkdir 693 (time 1788644148.10 total 50.13 last 15.18)
 - mkdir 874 (time 1788644158.15 total 60.18 last 18.01)
total: 1000 mkdir in 69.66 seconds: 14.36 ops/second
 - mkdir 142 (time 1788644107.78 total 10.02 last 14.17)
 - mkdir 262 (time 1788644117.92 total 20.16 last 11.84)
 - mkdir 379 (time 1788644127.95 total 30.19 last 11.67)
 - mkdir 522 (time 1788644137.97 total 40.21 last 14.27)
 - mkdir 671 (time 1788644147.99 total 50.23 last 14.87)
 - mkdir 852 (time 1788644158.01 total 60.26 last 18.06)
 - mkdir 979 (time 1788644168.08 total 70.32 last 12.62)
total: 1000 mkdir in 71.75 seconds: 13.94 ops/second
Time for file creation 81 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788644178 ; total 0 ; last 0)
total: 1000 unlinks in 24 seconds: 41.666668 unlinks/second
 - unlinked 0 (time 1788644178 ; total 0 ; last 0)
total: 1000 unlinks in 24 seconds: 41.666668 unlinks/second
 - unlinked 0 (time 1788644178 ; total 0 ; last 0)
total: 1000 unlinks in 24 seconds: 41.666668 unlinks/second
 - unlinked 0 (time 1788644178 ; total 0 ; last 0)
total: 1000 unlinks in 25 seconds: 40.000000 unlinks/second
 - unlinked 0 (time 1788644178 ; total 0 ; last 0)
total: 1000 unlinks in 25 seconds: 40.000000 unlinks/second
 - unlinked 0 (time 1788644178 ; total 0 ; last 0)
total: 1000 unlinks in 25 seconds: 40.000000 unlinks/second
 - unlinked 0 (time 1788644179 ; total 0 ; last 0)
total: 1000 unlinks in 24 seconds: 41.666668 unlinks/second
 - unlinked 0 (time 1788644179 ; total 1 ; last 1)
total: 1000 unlinks in 25 seconds: 40.000000 unlinks/second
 - unlinked 0 (time 1788644179 ; total 0 ; last 0)
total: 1000 unlinks in 25 seconds: 40.000000 unlinks/second
 - unlinked 0 (time 1788644179 ; total 0 ; last 0)
total: 1000 unlinks in 25 seconds: 40.000000 unlinks/second
Time for file removal 38 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8