== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 22:40:33 (1788662433)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.43 seconds: 23.05 ops/second
 - mkdir 186 (time 1788662455.97 total 10.03 last 18.55)
 - mkdir 356 (time 1788662466.09 total 20.14 last 16.81)
 - mkdir 505 (time 1788662476.10 total 30.16 last 14.88)
 - mkdir 663 (time 1788662486.19 total 40.25 last 15.66)
 - mkdir 813 (time 1788662496.25 total 50.31 last 14.91)
 - mkdir 962 (time 1788662506.26 total 60.31 last 14.89)
total: 1000 mkdir in 61.94 seconds: 16.14 ops/second
 - mkdir 177 (time 1788662455.41 total 10.02 last 17.67)
 - mkdir 343 (time 1788662465.42 total 20.02 last 16.58)
 - mkdir 487 (time 1788662475.45 total 30.06 last 14.36)
 - mkdir 645 (time 1788662485.47 total 40.07 last 15.77)
 - mkdir 790 (time 1788662495.56 total 50.16 last 14.37)
 - mkdir 948 (time 1788662505.61 total 60.21 last 15.72)
total: 1000 mkdir in 62.62 seconds: 15.97 ops/second
 - mkdir 185 (time 1788662456.69 total 10.07 last 18.37)
 - mkdir 346 (time 1788662466.72 total 20.10 last 16.06)
 - mkdir 495 (time 1788662476.76 total 30.14 last 14.83)
 - mkdir 654 (time 1788662486.78 total 40.16 last 15.86)
 - mkdir 806 (time 1788662496.80 total 50.18 last 15.18)
 - mkdir 965 (time 1788662506.82 total 60.20 last 15.87)
total: 1000 mkdir in 62.00 seconds: 16.13 ops/second
 - mkdir 183 (time 1788662456.18 total 10.00 last 18.30)
 - mkdir 343 (time 1788662466.19 total 20.02 last 15.97)
 - mkdir 492 (time 1788662476.23 total 30.06 last 14.84)
 - mkdir 651 (time 1788662486.27 total 40.09 last 15.85)
 - mkdir 800 (time 1788662496.28 total 50.11 last 14.87)
 - mkdir 952 (time 1788662506.31 total 60.13 last 15.17)
total: 1000 mkdir in 62.63 seconds: 15.97 ops/second
 - mkdir 188 (time 1788662456.55 total 10.04 last 18.73)
 - mkdir 348 (time 1788662466.62 total 20.10 last 15.90)
 - mkdir 495 (time 1788662476.68 total 30.17 last 14.61)
 - mkdir 651 (time 1788662486.70 total 40.19 last 15.56)
 - mkdir 803 (time 1788662496.77 total 50.26 last 15.10)
 - mkdir 957 (time 1788662506.81 total 60.30 last 15.34)
total: 1000 mkdir in 63.39 seconds: 15.77 ops/second
 - mkdir 181 (time 1788662456.82 total 10.02 last 18.06)
 - mkdir 340 (time 1788662466.85 total 20.05 last 15.86)
 - mkdir 488 (time 1788662476.85 total 30.05 last 14.80)
 - mkdir 653 (time 1788662486.85 total 40.05 last 16.49)
 - mkdir 801 (time 1788662496.86 total 50.06 last 14.79)
 - mkdir 953 (time 1788662506.86 total 60.06 last 15.19)
total: 1000 mkdir in 63.34 seconds: 15.79 ops/second
 - mkdir 180 (time 1788662456.75 total 10.05 last 17.91)
 - mkdir 341 (time 1788662466.85 total 20.15 last 15.94)
 - mkdir 486 (time 1788662476.87 total 30.16 last 14.47)
 - mkdir 648 (time 1788662486.87 total 40.16 last 16.20)
 - mkdir 797 (time 1788662496.89 total 50.18 last 14.88)
 - mkdir 952 (time 1788662506.89 total 60.18 last 15.50)
total: 1000 mkdir in 63.68 seconds: 15.70 ops/second
 - mkdir 172 (time 1788662455.91 total 10.02 last 17.16)
 - mkdir 333 (time 1788662465.93 total 20.05 last 16.06)
 - mkdir 476 (time 1788662475.94 total 30.06 last 14.29)
 - mkdir 632 (time 1788662485.98 total 40.10 last 15.53)
 - mkdir 779 (time 1788662495.99 total 50.11 last 14.68)
 - mkdir 930 (time 1788662506.00 total 60.12 last 15.09)
total: 1000 mkdir in 64.77 seconds: 15.44 ops/second
 - mkdir 176 (time 1788662456.25 total 10.00 last 17.60)
 - mkdir 327 (time 1788662466.25 total 20.00 last 15.10)
 - mkdir 470 (time 1788662476.31 total 30.06 last 14.21)
 - mkdir 630 (time 1788662486.32 total 40.07 last 15.99)
 - mkdir 781 (time 1788662496.36 total 50.11 last 15.04)
 - mkdir 936 (time 1788662506.37 total 60.12 last 15.48)
total: 1000 mkdir in 64.62 seconds: 15.47 ops/second
 - mkdir 174 (time 1788662456.75 total 10.09 last 17.25)
 - mkdir 327 (time 1788662466.81 total 20.15 last 15.20)
 - mkdir 466 (time 1788662476.91 total 30.25 last 13.77)
 - mkdir 626 (time 1788662486.96 total 40.30 last 15.92)
 - mkdir 777 (time 1788662496.99 total 50.33 last 15.05)
 - mkdir 931 (time 1788662507.02 total 60.36 last 15.35)
total: 1000 mkdir in 66.22 seconds: 15.10 ops/second
Time for file creation 78 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 16 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.46 seconds: 21.67 ops/second
 - mkdir 140 (time 1788663016.03 total 10.01 last 13.99)
 - mkdir 274 (time 1788663026.08 total 20.06 last 13.33)
 - mkdir 385 (time 1788663036.11 total 30.09 last 11.07)
 - mkdir 502 (time 1788663046.11 total 40.10 last 11.69)
 - mkdir 627 (time 1788663056.16 total 50.14 last 12.45)
 - mkdir 745 (time 1788663066.23 total 60.21 last 11.72)
 - mkdir 904 (time 1788663076.29 total 70.27 last 15.81)
total: 1000 mkdir in 76.31 seconds: 13.11 ops/second
 - mkdir 144 (time 1788663015.03 total 10.06 last 14.31)
 - mkdir 275 (time 1788663025.10 total 20.13 last 13.01)
 - mkdir 379 (time 1788663035.18 total 30.21 last 10.32)
 - mkdir 496 (time 1788663045.29 total 40.32 last 11.58)
 - mkdir 618 (time 1788663055.34 total 50.37 last 12.14)
 - mkdir 727 (time 1788663065.35 total 60.38 last 10.89)
 - mkdir 881 (time 1788663075.40 total 70.43 last 15.33)
total: 1000 mkdir in 78.23 seconds: 12.78 ops/second
 - mkdir 145 (time 1788663018.24 total 10.08 last 14.38)
 - mkdir 259 (time 1788663028.29 total 20.14 last 11.34)
 - mkdir 380 (time 1788663038.35 total 30.19 last 12.03)
 - mkdir 490 (time 1788663048.38 total 40.22 last 10.97)
 - mkdir 625 (time 1788663058.38 total 50.22 last 13.49)
 - mkdir 764 (time 1788663068.40 total 60.25 last 13.87)
 - mkdir 919 (time 1788663078.46 total 70.30 last 15.42)
total: 1000 mkdir in 76.96 seconds: 12.99 ops/second
 - mkdir 147 (time 1788663017.99 total 10.03 last 14.66)
 - mkdir 264 (time 1788663028.00 total 20.04 last 11.69)
 - mkdir 384 (time 1788663038.03 total 30.07 last 11.96)
 - mkdir 493 (time 1788663048.05 total 40.09 last 10.88)
 - mkdir 622 (time 1788663058.06 total 50.10 last 12.89)
 - mkdir 748 (time 1788663068.10 total 60.15 last 12.54)
 - mkdir 905 (time 1788663078.15 total 70.19 last 15.62)
total: 1000 mkdir in 77.87 seconds: 12.84 ops/second
 - mkdir 143 (time 1788663017.52 total 10.01 last 14.29)
 - mkdir 256 (time 1788663027.64 total 20.12 last 11.17)
 - mkdir 375 (time 1788663037.67 total 30.15 last 11.87)
 - mkdir 487 (time 1788663047.68 total 40.16 last 11.18)
 - mkdir 616 (time 1788663057.70 total 50.18 last 12.88)
 - mkdir 745 (time 1788663067.72 total 60.21 last 12.87)
 - mkdir 902 (time 1788663077.77 total 70.25 last 15.62)
total: 1000 mkdir in 78.63 seconds: 12.72 ops/second
 - mkdir 137 (time 1788663016.97 total 10.04 last 13.65)
 - mkdir 256 (time 1788663027.13 total 20.19 last 11.71)
 - mkdir 370 (time 1788663037.21 total 30.28 last 11.31)
 - mkdir 477 (time 1788663047.27 total 40.34 last 10.64)
 - mkdir 610 (time 1788663057.33 total 50.39 last 13.23)
 - mkdir 734 (time 1788663067.35 total 60.41 last 12.37)
 - mkdir 887 (time 1788663077.35 total 70.42 last 15.30)
total: 1000 mkdir in 79.96 seconds: 12.51 ops/second
 - mkdir 136 (time 1788663016.84 total 10.05 last 13.53)
 - mkdir 254 (time 1788663026.90 total 20.11 last 11.73)
 - mkdir 365 (time 1788663036.91 total 30.12 last 11.09)
 - mkdir 474 (time 1788663046.96 total 40.17 last 10.85)
 - mkdir 603 (time 1788663057.00 total 50.21 last 12.85)
 - mkdir 725 (time 1788663067.07 total 60.27 last 12.12)
 - mkdir 884 (time 1788663077.07 total 70.28 last 15.89)
 - mkdir 998 (time 1788663087.08 total 80.29 last 11.39)
total: 1000 mkdir in 80.41 seconds: 12.44 ops/second
 - mkdir 146 (time 1788663018.11 total 10.07 last 14.50)
 - mkdir 252 (time 1788663028.13 total 20.09 last 10.58)
 - mkdir 369 (time 1788663038.13 total 30.10 last 11.69)
 - mkdir 482 (time 1788663048.15 total 40.12 last 11.28)
 - mkdir 612 (time 1788663058.24 total 50.21 last 12.89)
 - mkdir 743 (time 1788663068.28 total 60.24 last 13.06)
 - mkdir 896 (time 1788663078.31 total 70.27 last 15.25)
total: 1000 mkdir in 79.84 seconds: 12.53 ops/second
 - mkdir 146 (time 1788663017.77 total 10.03 last 14.55)
 - mkdir 263 (time 1788663027.81 total 20.08 last 11.65)
 - mkdir 371 (time 1788663037.82 total 30.08 last 10.79)
 - mkdir 481 (time 1788663047.84 total 40.11 last 10.98)
 - mkdir 607 (time 1788663057.85 total 50.11 last 12.59)
 - mkdir 725 (time 1788663067.88 total 60.14 last 11.77)
 - mkdir 882 (time 1788663077.92 total 70.19 last 15.62)
 - mkdir 997 (time 1788663088.05 total 80.31 last 11.36)
total: 1000 mkdir in 80.52 seconds: 12.42 ops/second
 - mkdir 127 (time 1788663016.30 total 10.05 last 12.64)
 - mkdir 243 (time 1788663026.48 total 20.22 last 11.40)
 - mkdir 351 (time 1788663036.51 total 30.26 last 10.76)
 - mkdir 456 (time 1788663046.61 total 40.36 last 10.40)
 - mkdir 577 (time 1788663056.63 total 50.38 last 12.08)
 - mkdir 687 (time 1788663066.65 total 60.39 last 10.99)
 - mkdir 839 (time 1788663076.66 total 70.40 last 15.19)
 - mkdir 960 (time 1788663086.67 total 80.41 last 12.09)
total: 1000 mkdir in 84.35 seconds: 11.86 ops/second
Time for file creation 92 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788663099 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788663101 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788663101 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788663101 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788663101 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788663101 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788663101 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788663102 ; total 1 ; last 1)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788663101 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788663102 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
Time for file removal 48 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8