== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 20:27:02 (1789604822)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.71 seconds: 14.01 ops/second
 - mkdir 177 (time 1789604846.80 total 10.02 last 17.67)
 - mkdir 364 (time 1789604856.80 total 20.02 last 18.70)
 - mkdir 541 (time 1789604866.88 total 30.10 last 17.56)
 - mkdir 687 (time 1789604876.92 total 40.14 last 14.54)
 - mkdir 846 (time 1789604886.92 total 50.14 last 15.90)
 - mkdir 991 (time 1789604896.98 total 60.20 last 14.41)
total: 1000 mkdir in 60.71 seconds: 16.47 ops/second
 - mkdir 170 (time 1789604846.80 total 10.01 last 16.99)
 - mkdir 358 (time 1789604856.82 total 20.03 last 18.76)
 - mkdir 532 (time 1789604866.88 total 30.08 last 17.31)
 - mkdir 679 (time 1789604876.92 total 40.13 last 14.63)
 - mkdir 830 (time 1789604886.95 total 50.15 last 15.07)
 - mkdir 970 (time 1789604896.96 total 60.16 last 13.99)
total: 1000 mkdir in 62.91 seconds: 15.90 ops/second
 - mkdir 141 (time 1789604844.71 total 10.03 last 14.06)
 - mkdir 322 (time 1789604854.72 total 20.03 last 18.10)
 - mkdir 486 (time 1789604864.73 total 30.04 last 16.37)
 - mkdir 630 (time 1789604874.77 total 40.08 last 14.35)
 - mkdir 788 (time 1789604884.78 total 50.09 last 15.78)
 - mkdir 932 (time 1789604894.79 total 60.10 last 14.39)
total: 1000 mkdir in 66.03 seconds: 15.14 ops/second
 - mkdir 179 (time 1789604848.04 total 10.01 last 17.87)
 - mkdir 356 (time 1789604858.06 total 20.04 last 17.66)
 - mkdir 517 (time 1789604868.06 total 30.04 last 16.10)
 - mkdir 678 (time 1789604878.07 total 40.05 last 16.09)
 - mkdir 825 (time 1789604888.10 total 50.07 last 14.66)
 - mkdir 960 (time 1789604898.11 total 60.08 last 13.49)
total: 1000 mkdir in 65.26 seconds: 15.32 ops/second
 - mkdir 173 (time 1789604847.18 total 10.03 last 17.25)
 - mkdir 352 (time 1789604857.23 total 20.08 last 17.81)
 - mkdir 514 (time 1789604867.24 total 30.08 last 16.20)
 - mkdir 664 (time 1789604877.28 total 40.12 last 14.94)
 - mkdir 816 (time 1789604887.30 total 50.14 last 15.17)
 - mkdir 951 (time 1789604897.34 total 60.19 last 13.44)
total: 1000 mkdir in 66.20 seconds: 15.11 ops/second
 - mkdir 182 (time 1789604847.92 total 10.01 last 18.17)
 - mkdir 355 (time 1789604857.95 total 20.05 last 17.24)
 - mkdir 512 (time 1789604868.01 total 30.11 last 15.61)
 - mkdir 668 (time 1789604878.06 total 40.16 last 15.52)
 - mkdir 818 (time 1789604888.09 total 50.19 last 14.96)
 - mkdir 950 (time 1789604898.11 total 60.21 last 13.17)
total: 1000 mkdir in 66.32 seconds: 15.08 ops/second
 - mkdir 181 (time 1789604847.87 total 10.01 last 18.08)
 - mkdir 353 (time 1789604857.87 total 20.02 last 17.19)
 - mkdir 515 (time 1789604867.91 total 30.05 last 16.14)
 - mkdir 672 (time 1789604877.94 total 40.09 last 15.65)
 - mkdir 815 (time 1789604888.01 total 50.15 last 14.20)
 - mkdir 952 (time 1789604898.14 total 60.29 last 13.52)
total: 1000 mkdir in 66.72 seconds: 14.99 ops/second
 - mkdir 181 (time 1789604847.94 total 10.02 last 18.07)
 - mkdir 351 (time 1789604857.96 total 20.04 last 16.97)
 - mkdir 510 (time 1789604868.01 total 30.09 last 15.82)
 - mkdir 671 (time 1789604878.02 total 40.10 last 16.09)
 - mkdir 814 (time 1789604888.07 total 50.14 last 14.23)
 - mkdir 947 (time 1789604898.12 total 60.20 last 13.23)
total: 1000 mkdir in 66.82 seconds: 14.97 ops/second
 - mkdir 167 (time 1789604847.00 total 10.02 last 16.66)
 - mkdir 343 (time 1789604857.01 total 20.03 last 17.58)
 - mkdir 506 (time 1789604867.10 total 30.13 last 16.15)
 - mkdir 649 (time 1789604877.15 total 40.18 last 14.22)
 - mkdir 801 (time 1789604887.24 total 50.27 last 15.07)
 - mkdir 935 (time 1789604897.28 total 60.30 last 13.35)
total: 1000 mkdir in 68.12 seconds: 14.68 ops/second
 - mkdir 153 (time 1789604846.24 total 10.02 last 15.27)
 - mkdir 319 (time 1789604856.28 total 20.06 last 16.53)
 - mkdir 489 (time 1789604866.32 total 30.10 last 16.93)
 - mkdir 623 (time 1789604876.32 total 40.10 last 13.40)
 - mkdir 773 (time 1789604886.34 total 50.12 last 14.97)
 - mkdir 906 (time 1789604896.37 total 60.16 last 13.25)
 - mkdir 996 (time 1789604906.41 total 70.19 last 8.97)
total: 1000 mkdir in 70.41 seconds: 14.20 ops/second
Time for file creation 82 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.62 seconds: 16.24 ops/second
 - mkdir 152 (time 1789605453.51 total 10.01 last 15.19)
 - mkdir 337 (time 1789605463.54 total 20.04 last 18.45)
 - mkdir 478 (time 1789605473.67 total 30.16 last 13.92)
 - mkdir 592 (time 1789605483.69 total 40.19 last 11.37)
 - mkdir 721 (time 1789605493.73 total 50.22 last 12.86)
 - mkdir 896 (time 1789605503.77 total 60.26 last 17.43)
total: 1000 mkdir in 67.34 seconds: 14.85 ops/second
 - mkdir 131 (time 1789605452.53 total 10.04 last 13.05)
 - mkdir 321 (time 1789605462.55 total 20.06 last 18.96)
 - mkdir 454 (time 1789605472.72 total 30.23 last 13.08)
 - mkdir 576 (time 1789605482.86 total 40.37 last 12.02)
 - mkdir 698 (time 1789605492.92 total 50.43 last 12.13)
 - mkdir 879 (time 1789605502.98 total 60.49 last 18.00)
total: 1000 mkdir in 68.84 seconds: 14.53 ops/second
 - mkdir 160 (time 1789605454.22 total 10.08 last 15.87)
 - mkdir 338 (time 1789605464.25 total 20.11 last 17.75)
 - mkdir 467 (time 1789605474.32 total 30.18 last 12.81)
 - mkdir 595 (time 1789605484.37 total 40.23 last 12.73)
 - mkdir 718 (time 1789605494.41 total 50.27 last 12.25)
 - mkdir 891 (time 1789605504.45 total 60.31 last 17.23)
total: 1000 mkdir in 67.93 seconds: 14.72 ops/second
 - mkdir 150 (time 1789605453.16 total 10.03 last 14.95)
 - mkdir 329 (time 1789605463.16 total 20.04 last 17.89)
 - mkdir 462 (time 1789605473.19 total 30.06 last 13.27)
 - mkdir 582 (time 1789605483.24 total 40.12 last 11.93)
 - mkdir 701 (time 1789605493.26 total 50.14 last 11.88)
 - mkdir 877 (time 1789605503.29 total 60.17 last 17.55)
total: 1000 mkdir in 69.04 seconds: 14.48 ops/second
 - mkdir 158 (time 1789605453.83 total 10.02 last 15.76)
 - mkdir 329 (time 1789605463.84 total 20.03 last 17.09)
 - mkdir 458 (time 1789605473.87 total 30.07 last 12.86)
 - mkdir 578 (time 1789605483.91 total 40.11 last 11.95)
 - mkdir 704 (time 1789605493.95 total 50.15 last 12.54)
 - mkdir 875 (time 1789605503.99 total 60.19 last 17.04)
total: 1000 mkdir in 68.85 seconds: 14.52 ops/second
 - mkdir 162 (time 1789605454.42 total 10.04 last 16.13)
 - mkdir 336 (time 1789605464.48 total 20.10 last 17.30)
 - mkdir 458 (time 1789605474.48 total 30.11 last 12.19)
 - mkdir 585 (time 1789605484.49 total 40.11 last 12.69)
 - mkdir 709 (time 1789605494.52 total 50.14 last 12.36)
 - mkdir 883 (time 1789605504.56 total 60.19 last 17.32)
total: 1000 mkdir in 68.58 seconds: 14.58 ops/second
 - mkdir 160 (time 1789605454.11 total 10.06 last 15.90)
 - mkdir 332 (time 1789605464.12 total 20.08 last 17.18)
 - mkdir 458 (time 1789605474.29 total 30.25 last 12.39)
 - mkdir 584 (time 1789605484.33 total 40.28 last 12.55)
 - mkdir 709 (time 1789605494.41 total 50.37 last 12.39)
 - mkdir 883 (time 1789605504.47 total 60.42 last 17.30)
total: 1000 mkdir in 69.37 seconds: 14.42 ops/second
 - mkdir 157 (time 1789605453.80 total 10.01 last 15.68)
 - mkdir 330 (time 1789605463.84 total 20.06 last 17.22)
 - mkdir 459 (time 1789605473.85 total 30.06 last 12.90)
 - mkdir 577 (time 1789605483.87 total 40.08 last 11.77)
 - mkdir 705 (time 1789605493.89 total 50.11 last 12.77)
 - mkdir 877 (time 1789605503.99 total 60.20 last 17.05)
total: 1000 mkdir in 69.85 seconds: 14.32 ops/second
 - mkdir 122 (time 1789605452.16 total 10.03 last 12.16)
 - mkdir 297 (time 1789605462.18 total 20.05 last 17.47)
 - mkdir 436 (time 1789605472.23 total 30.10 last 13.83)
 - mkdir 559 (time 1789605482.29 total 40.16 last 12.23)
 - mkdir 678 (time 1789605492.33 total 50.20 last 11.86)
 - mkdir 856 (time 1789605502.39 total 60.26 last 17.69)
 - mkdir 989 (time 1789605512.42 total 70.29 last 13.26)
total: 1000 mkdir in 71.69 seconds: 13.95 ops/second
 - mkdir 146 (time 1789605453.78 total 10.01 last 14.58)
 - mkdir 310 (time 1789605463.86 total 20.09 last 16.27)
 - mkdir 432 (time 1789605473.87 total 30.10 last 12.19)
 - mkdir 555 (time 1789605483.90 total 40.13 last 12.26)
 - mkdir 682 (time 1789605493.91 total 50.14 last 12.69)
 - mkdir 856 (time 1789605503.92 total 60.15 last 17.38)
 - mkdir 981 (time 1789605513.97 total 70.20 last 12.43)
total: 1000 mkdir in 72.32 seconds: 13.83 ops/second
Time for file creation 86 sec for 1 RPC sent at a time
 - unlinked 0 (time 1789605526 ; total 1 ; last 1)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1789605527 ; total 1 ; last 1)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1789605527 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1789605528 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1789605528 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1789605527 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1789605528 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1789605527 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1789605527 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1789605528 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
Time for file removal 49 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8