== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 23:36:36 (1786332996)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.64 seconds: 15.54 ops/second
 - mkdir 137 (time 1786333020.75 total 10.01 last 13.69)
 - mkdir 301 (time 1786333030.82 total 20.08 last 16.29)
 - mkdir 445 (time 1786333040.87 total 30.13 last 14.33)
 - mkdir 588 (time 1786333050.88 total 40.13 last 14.29)
 - mkdir 725 (time 1786333060.89 total 50.14 last 13.68)
 - mkdir 869 (time 1786333070.92 total 60.17 last 14.36)
total: 1000 mkdir in 68.17 seconds: 14.67 ops/second
 - mkdir 138 (time 1786333020.68 total 10.03 last 13.76)
 - mkdir 298 (time 1786333030.76 total 20.11 last 15.88)
 - mkdir 441 (time 1786333040.78 total 30.13 last 14.26)
 - mkdir 578 (time 1786333050.87 total 40.22 last 13.58)
 - mkdir 713 (time 1786333060.88 total 50.23 last 13.49)
 - mkdir 857 (time 1786333070.89 total 60.24 last 14.38)
total: 1000 mkdir in 70.05 seconds: 14.28 ops/second
 - mkdir 135 (time 1786333020.37 total 10.07 last 13.41)
 - mkdir 297 (time 1786333030.37 total 20.07 last 16.20)
 - mkdir 434 (time 1786333040.42 total 30.12 last 13.63)
 - mkdir 575 (time 1786333050.57 total 40.27 last 13.89)
 - mkdir 704 (time 1786333060.60 total 50.30 last 12.86)
 - mkdir 851 (time 1786333070.61 total 60.30 last 14.69)
 - mkdir 998 (time 1786333080.65 total 70.35 last 14.63)
total: 1000 mkdir in 70.48 seconds: 14.19 ops/second
 - mkdir 137 (time 1786333019.65 total 10.00 last 13.70)
 - mkdir 294 (time 1786333029.75 total 20.10 last 15.54)
 - mkdir 436 (time 1786333039.75 total 30.11 last 14.19)
 - mkdir 573 (time 1786333049.80 total 40.16 last 13.64)
 - mkdir 702 (time 1786333059.81 total 50.16 last 12.89)
 - mkdir 840 (time 1786333069.81 total 60.17 last 13.80)
 - mkdir 990 (time 1786333079.96 total 70.31 last 14.78)
total: 1000 mkdir in 71.57 seconds: 13.97 ops/second
 - mkdir 137 (time 1786333021.83 total 10.00 last 13.69)
 - mkdir 302 (time 1786333031.85 total 20.02 last 16.48)
 - mkdir 446 (time 1786333041.92 total 30.09 last 14.29)
 - mkdir 573 (time 1786333051.94 total 40.11 last 12.68)
 - mkdir 711 (time 1786333061.95 total 50.12 last 13.79)
 - mkdir 864 (time 1786333072.02 total 60.19 last 15.18)
total: 1000 mkdir in 69.76 seconds: 14.34 ops/second
 - mkdir 134 (time 1786333022.00 total 10.03 last 13.36)
 - mkdir 293 (time 1786333032.04 total 20.07 last 15.83)
 - mkdir 437 (time 1786333042.10 total 30.13 last 14.31)
 - mkdir 570 (time 1786333052.18 total 40.21 last 13.20)
 - mkdir 706 (time 1786333062.22 total 50.26 last 13.54)
 - mkdir 861 (time 1786333072.28 total 60.31 last 15.42)
 - mkdir 999 (time 1786333082.36 total 70.39 last 13.69)
total: 1000 mkdir in 70.39 seconds: 14.21 ops/second
 - mkdir 138 (time 1786333021.55 total 10.03 last 13.76)
 - mkdir 298 (time 1786333031.62 total 20.09 last 15.90)
 - mkdir 440 (time 1786333041.70 total 30.18 last 14.08)
 - mkdir 571 (time 1786333051.73 total 40.21 last 13.07)
 - mkdir 701 (time 1786333061.76 total 50.24 last 12.96)
 - mkdir 851 (time 1786333071.80 total 60.28 last 14.93)
 - mkdir 989 (time 1786333081.84 total 70.32 last 13.75)
total: 1000 mkdir in 71.65 seconds: 13.96 ops/second
 - mkdir 133 (time 1786333021.92 total 10.04 last 13.25)
 - mkdir 292 (time 1786333031.95 total 20.07 last 15.86)
 - mkdir 437 (time 1786333042.05 total 30.17 last 14.36)
 - mkdir 569 (time 1786333052.15 total 40.27 last 13.07)
 - mkdir 703 (time 1786333062.22 total 50.34 last 13.30)
 - mkdir 851 (time 1786333072.26 total 60.38 last 14.74)
 - mkdir 988 (time 1786333082.38 total 70.50 last 13.54)
total: 1000 mkdir in 72.37 seconds: 13.82 ops/second
 - mkdir 135 (time 1786333021.70 total 10.03 last 13.46)
 - mkdir 291 (time 1786333031.78 total 20.11 last 15.47)
 - mkdir 433 (time 1786333041.88 total 30.21 last 14.06)
 - mkdir 562 (time 1786333051.93 total 40.26 last 12.84)
 - mkdir 694 (time 1786333061.95 total 50.28 last 13.17)
 - mkdir 840 (time 1786333072.06 total 60.39 last 14.44)
 - mkdir 975 (time 1786333082.16 total 70.49 last 13.37)
total: 1000 mkdir in 73.88 seconds: 13.54 ops/second
 - mkdir 133 (time 1786333021.63 total 10.04 last 13.25)
 - mkdir 285 (time 1786333031.67 total 20.08 last 15.14)
 - mkdir 424 (time 1786333041.71 total 30.12 last 13.84)
 - mkdir 546 (time 1786333051.73 total 40.14 last 12.18)
 - mkdir 676 (time 1786333061.83 total 50.24 last 12.88)
 - mkdir 817 (time 1786333071.83 total 60.24 last 14.10)
 - mkdir 953 (time 1786333081.87 total 70.28 last 13.55)
total: 1000 mkdir in 75.32 seconds: 13.28 ops/second
Time for file creation 88 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.50 seconds: 19.86 ops/second
 - mkdir 172 (time 1786333663.63 total 10.15 last 16.95)
 - mkdir 316 (time 1786333673.64 total 20.15 last 14.39)
 - mkdir 468 (time 1786333683.69 total 30.21 last 15.12)
 - mkdir 610 (time 1786333693.79 total 40.31 last 14.06)
 - mkdir 770 (time 1786333703.87 total 50.39 last 15.88)
 - mkdir 971 (time 1786333713.91 total 60.43 last 20.01)
total: 1000 mkdir in 62.15 seconds: 16.09 ops/second
 - mkdir 179 (time 1786333664.38 total 10.00 last 17.89)
 - mkdir 324 (time 1786333674.42 total 20.04 last 14.44)
 - mkdir 486 (time 1786333684.47 total 30.10 last 16.11)
 - mkdir 623 (time 1786333694.50 total 40.13 last 13.66)
 - mkdir 783 (time 1786333704.51 total 50.14 last 15.99)
 - mkdir 978 (time 1786333714.51 total 60.14 last 19.50)
total: 1000 mkdir in 61.41 seconds: 16.29 ops/second
 - mkdir 182 (time 1786333664.78 total 10.00 last 18.19)
 - mkdir 330 (time 1786333674.86 total 20.09 last 14.67)
 - mkdir 486 (time 1786333684.90 total 30.13 last 15.54)
 - mkdir 629 (time 1786333694.95 total 40.18 last 14.23)
 - mkdir 794 (time 1786333704.96 total 50.19 last 16.49)
 - mkdir 989 (time 1786333715.01 total 60.23 last 19.41)
total: 1000 mkdir in 61.06 seconds: 16.38 ops/second
 - mkdir 171 (time 1786333663.81 total 10.07 last 16.98)
 - mkdir 320 (time 1786333673.84 total 20.11 last 14.84)
 - mkdir 466 (time 1786333683.89 total 30.15 last 14.54)
 - mkdir 609 (time 1786333693.99 total 40.25 last 14.15)
 - mkdir 771 (time 1786333704.02 total 50.28 last 16.16)
 - mkdir 965 (time 1786333714.05 total 60.31 last 19.34)
total: 1000 mkdir in 62.41 seconds: 16.02 ops/second
 - mkdir 174 (time 1786333663.41 total 10.01 last 17.38)
 - mkdir 321 (time 1786333673.44 total 20.04 last 14.66)
 - mkdir 470 (time 1786333683.45 total 30.05 last 14.89)
 - mkdir 611 (time 1786333693.53 total 40.13 last 13.99)
 - mkdir 767 (time 1786333703.64 total 50.23 last 15.44)
 - mkdir 961 (time 1786333713.64 total 60.24 last 19.39)
total: 1000 mkdir in 62.88 seconds: 15.90 ops/second
 - mkdir 173 (time 1786333664.19 total 10.05 last 17.21)
 - mkdir 323 (time 1786333674.24 total 20.10 last 14.93)
 - mkdir 483 (time 1786333684.27 total 30.13 last 15.94)
 - mkdir 618 (time 1786333694.31 total 40.17 last 13.45)
 - mkdir 776 (time 1786333704.34 total 50.20 last 15.76)
 - mkdir 967 (time 1786333714.38 total 60.24 last 19.03)
total: 1000 mkdir in 62.67 seconds: 15.96 ops/second
 - mkdir 171 (time 1786333663.93 total 10.10 last 16.93)
 - mkdir 319 (time 1786333674.09 total 20.25 last 14.58)
 - mkdir 472 (time 1786333684.13 total 30.30 last 15.23)
 - mkdir 609 (time 1786333694.17 total 40.34 last 13.65)
 - mkdir 769 (time 1786333704.18 total 50.35 last 15.98)
 - mkdir 961 (time 1786333714.18 total 60.35 last 19.20)
total: 1000 mkdir in 63.12 seconds: 15.84 ops/second
 - mkdir 182 (time 1786333664.92 total 10.02 last 18.16)
 - mkdir 325 (time 1786333674.96 total 20.06 last 14.24)
 - mkdir 475 (time 1786333685.04 total 30.15 last 14.88)
 - mkdir 613 (time 1786333695.04 total 40.15 last 13.79)
 - mkdir 782 (time 1786333705.11 total 50.22 last 16.78)
 - mkdir 977 (time 1786333715.11 total 60.22 last 19.50)
total: 1000 mkdir in 62.25 seconds: 16.06 ops/second
 - mkdir 177 (time 1786333664.37 total 10.01 last 17.68)
 - mkdir 325 (time 1786333674.42 total 20.05 last 14.74)
 - mkdir 481 (time 1786333684.52 total 30.15 last 15.45)
 - mkdir 617 (time 1786333694.58 total 40.21 last 13.52)
 - mkdir 775 (time 1786333704.59 total 50.22 last 15.78)
 - mkdir 969 (time 1786333714.62 total 60.25 last 19.34)
total: 1000 mkdir in 62.80 seconds: 15.92 ops/second
 - mkdir 167 (time 1786333663.43 total 10.00 last 16.70)
 - mkdir 299 (time 1786333673.44 total 20.01 last 13.19)
 - mkdir 441 (time 1786333683.47 total 30.04 last 14.16)
 - mkdir 579 (time 1786333693.51 total 40.09 last 13.74)
 - mkdir 731 (time 1786333703.52 total 50.09 last 15.19)
 - mkdir 926 (time 1786333713.52 total 60.09 last 19.50)
total: 1000 mkdir in 65.31 seconds: 15.31 ops/second
Time for file creation 77 sec for 1 RPC sent at a time
 - unlinked 0 (time 1786333728 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1786333730 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1786333729 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1786333730 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1786333730 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1786333730 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1786333730 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1786333730 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1786333730 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1786333730 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
Time for file removal 51 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8