== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 00:36:34 (1787114194)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.50 seconds: 20.19 ops/second
 - mkdir 169 (time 1787114213.73 total 10.09 last 16.75)
 - mkdir 320 (time 1787114223.86 total 20.22 last 14.91)
 - mkdir 477 (time 1787114233.89 total 30.24 last 15.66)
 - mkdir 649 (time 1787114243.91 total 40.27 last 17.15)
 - mkdir 822 (time 1787114253.96 total 50.32 last 17.22)
total: 1000 mkdir in 60.04 seconds: 16.66 ops/second
 - mkdir 167 (time 1787114214.06 total 10.03 last 16.65)
 - mkdir 323 (time 1787114224.07 total 20.04 last 15.59)
 - mkdir 483 (time 1787114234.11 total 30.08 last 15.94)
 - mkdir 653 (time 1787114244.14 total 40.11 last 16.95)
 - mkdir 821 (time 1787114254.15 total 50.12 last 16.78)
 - mkdir 995 (time 1787114264.20 total 60.17 last 17.32)
total: 1000 mkdir in 60.56 seconds: 16.51 ops/second
 - mkdir 164 (time 1787114214.18 total 10.04 last 16.34)
 - mkdir 312 (time 1787114224.24 total 20.10 last 14.71)
 - mkdir 474 (time 1787114234.25 total 30.10 last 16.20)
 - mkdir 645 (time 1787114244.27 total 40.12 last 17.06)
 - mkdir 812 (time 1787114254.31 total 50.17 last 16.63)
 - mkdir 996 (time 1787114264.35 total 60.20 last 18.33)
total: 1000 mkdir in 60.50 seconds: 16.53 ops/second
 - mkdir 170 (time 1787114214.30 total 10.07 last 16.89)
 - mkdir 320 (time 1787114224.37 total 20.14 last 14.89)
 - mkdir 480 (time 1787114234.37 total 30.14 last 16.00)
 - mkdir 648 (time 1787114244.41 total 40.18 last 16.73)
 - mkdir 818 (time 1787114254.42 total 50.20 last 16.98)
 - mkdir 995 (time 1787114264.52 total 60.29 last 17.54)
total: 1000 mkdir in 60.61 seconds: 16.50 ops/second
 - mkdir 167 (time 1787114213.47 total 10.01 last 16.69)
 - mkdir 311 (time 1787114223.52 total 20.06 last 14.32)
 - mkdir 462 (time 1787114233.56 total 30.10 last 15.04)
 - mkdir 634 (time 1787114243.59 total 40.13 last 17.15)
 - mkdir 802 (time 1787114253.60 total 50.14 last 16.79)
 - mkdir 982 (time 1787114263.61 total 60.15 last 17.97)
total: 1000 mkdir in 61.58 seconds: 16.24 ops/second
 - mkdir 165 (time 1787114214.70 total 10.01 last 16.48)
 - mkdir 316 (time 1787114224.78 total 20.09 last 14.99)
 - mkdir 484 (time 1787114234.80 total 30.12 last 16.75)
 - mkdir 655 (time 1787114244.84 total 40.15 last 17.04)
 - mkdir 822 (time 1787114254.93 total 50.24 last 16.55)
 - mkdir 998 (time 1787114265.00 total 60.31 last 17.48)
total: 1000 mkdir in 60.37 seconds: 16.57 ops/second
 - mkdir 159 (time 1787114214.04 total 10.06 last 15.81)
 - mkdir 304 (time 1787114224.09 total 20.10 last 14.43)
 - mkdir 460 (time 1787114234.11 total 30.12 last 15.57)
 - mkdir 631 (time 1787114244.14 total 40.15 last 17.05)
 - mkdir 797 (time 1787114254.16 total 50.17 last 16.57)
 - mkdir 981 (time 1787114264.19 total 60.20 last 18.34)
total: 1000 mkdir in 61.58 seconds: 16.24 ops/second
 - mkdir 163 (time 1787114214.43 total 10.04 last 16.23)
 - mkdir 309 (time 1787114224.50 total 20.11 last 14.50)
 - mkdir 472 (time 1787114234.51 total 30.13 last 16.28)
 - mkdir 639 (time 1787114244.55 total 40.16 last 16.64)
 - mkdir 804 (time 1787114254.58 total 50.19 last 16.46)
 - mkdir 979 (time 1787114264.64 total 60.25 last 17.40)
total: 1000 mkdir in 61.91 seconds: 16.15 ops/second
 - mkdir 164 (time 1787114214.15 total 10.03 last 16.35)
 - mkdir 306 (time 1787114224.24 total 20.12 last 14.08)
 - mkdir 460 (time 1787114234.24 total 30.12 last 15.39)
 - mkdir 626 (time 1787114244.29 total 40.17 last 16.52)
 - mkdir 794 (time 1787114254.29 total 50.17 last 16.80)
 - mkdir 971 (time 1787114264.35 total 60.23 last 17.59)
total: 1000 mkdir in 62.81 seconds: 15.92 ops/second
 - mkdir 160 (time 1787114213.48 total 10.03 last 15.95)
 - mkdir 296 (time 1787114223.52 total 20.07 last 13.55)
 - mkdir 442 (time 1787114233.52 total 30.07 last 14.59)
 - mkdir 613 (time 1787114243.54 total 40.09 last 17.07)
 - mkdir 783 (time 1787114253.62 total 50.17 last 16.87)
 - mkdir 962 (time 1787114263.65 total 60.20 last 17.84)
total: 1000 mkdir in 63.81 seconds: 15.67 ops/second
Time for file creation 72 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 8 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.41 seconds: 24.67 ops/second
 - mkdir 139 (time 1787114793.55 total 10.02 last 13.87)
 - mkdir 281 (time 1787114803.67 total 20.15 last 14.03)
 - mkdir 370 (time 1787114813.69 total 30.16 last 8.88)
 - mkdir 508 (time 1787114823.70 total 40.17 last 13.80)
 - mkdir 678 (time 1787114833.70 total 50.17 last 16.99)
 - mkdir 830 (time 1787114843.71 total 60.18 last 15.19)
 - mkdir 935 (time 1787114853.73 total 70.20 last 10.48)
total: 1000 mkdir in 75.14 seconds: 13.31 ops/second
 - mkdir 136 (time 1787114792.97 total 10.21 last 13.32)
 - mkdir 270 (time 1787114803.01 total 20.25 last 13.35)
 - mkdir 354 (time 1787114813.04 total 30.27 last 8.38)
 - mkdir 495 (time 1787114823.06 total 40.30 last 14.06)
 - mkdir 665 (time 1787114833.09 total 50.32 last 16.96)
 - mkdir 817 (time 1787114843.09 total 60.32 last 15.20)
 - mkdir 917 (time 1787114853.16 total 70.39 last 9.93)
total: 1000 mkdir in 78.19 seconds: 12.79 ops/second
 - mkdir 129 (time 1787114793.65 total 10.04 last 12.85)
 - mkdir 263 (time 1787114803.67 total 20.05 last 13.38)
 - mkdir 351 (time 1787114813.69 total 30.08 last 8.78)
 - mkdir 494 (time 1787114823.73 total 40.12 last 14.24)
 - mkdir 668 (time 1787114833.74 total 50.13 last 17.38)
 - mkdir 820 (time 1787114843.82 total 60.20 last 15.09)
 - mkdir 920 (time 1787114853.95 total 70.34 last 9.87)
total: 1000 mkdir in 77.48 seconds: 12.91 ops/second
 - mkdir 134 (time 1787114793.81 total 10.09 last 13.28)
 - mkdir 265 (time 1787114803.88 total 20.16 last 13.01)
 - mkdir 352 (time 1787114813.88 total 30.16 last 8.69)
 - mkdir 489 (time 1787114823.93 total 40.21 last 13.64)
 - mkdir 662 (time 1787114833.94 total 50.22 last 17.29)
 - mkdir 811 (time 1787114844.06 total 60.34 last 14.72)
 - mkdir 912 (time 1787114854.10 total 70.39 last 10.05)
total: 1000 mkdir in 78.61 seconds: 12.72 ops/second
 - mkdir 135 (time 1787114794.34 total 10.05 last 13.43)
 - mkdir 261 (time 1787114804.38 total 20.09 last 12.55)
 - mkdir 350 (time 1787114814.41 total 30.12 last 8.88)
 - mkdir 496 (time 1787114824.43 total 40.14 last 14.57)
 - mkdir 667 (time 1787114834.46 total 50.17 last 17.06)
 - mkdir 816 (time 1787114844.58 total 60.29 last 14.72)
 - mkdir 918 (time 1787114854.60 total 70.31 last 10.17)
total: 1000 mkdir in 78.50 seconds: 12.74 ops/second
 - mkdir 133 (time 1787114793.63 total 10.01 last 13.29)
 - mkdir 262 (time 1787114803.67 total 20.05 last 12.85)
 - mkdir 346 (time 1787114813.68 total 30.06 last 8.39)
 - mkdir 487 (time 1787114823.70 total 40.08 last 14.07)
 - mkdir 660 (time 1787114833.80 total 50.17 last 17.13)
 - mkdir 811 (time 1787114843.92 total 60.30 last 14.91)
 - mkdir 910 (time 1787114853.92 total 70.30 last 9.90)
total: 1000 mkdir in 79.51 seconds: 12.58 ops/second
 - mkdir 131 (time 1787114793.94 total 10.05 last 13.03)
 - mkdir 264 (time 1787114804.04 total 20.15 last 13.17)
 - mkdir 348 (time 1787114814.09 total 30.20 last 8.36)
 - mkdir 490 (time 1787114824.10 total 40.22 last 14.18)
 - mkdir 662 (time 1787114834.13 total 50.25 last 17.15)
 - mkdir 813 (time 1787114844.24 total 60.35 last 14.94)
 - mkdir 910 (time 1787114854.24 total 70.35 last 9.70)
total: 1000 mkdir in 79.33 seconds: 12.61 ops/second
 - mkdir 129 (time 1787114794.13 total 10.02 last 12.88)
 - mkdir 254 (time 1787114804.16 total 20.05 last 12.46)
 - mkdir 339 (time 1787114814.19 total 30.08 last 8.47)
 - mkdir 484 (time 1787114824.21 total 40.09 last 14.48)
 - mkdir 658 (time 1787114834.23 total 50.12 last 17.36)
 - mkdir 805 (time 1787114844.35 total 60.24 last 14.52)
 - mkdir 903 (time 1787114854.42 total 70.31 last 9.73)
 - mkdir 997 (time 1787114864.60 total 80.49 last 9.24)
total: 1000 mkdir in 80.85 seconds: 12.37 ops/second
 - mkdir 125 (time 1787114793.83 total 10.07 last 12.41)
 - mkdir 252 (time 1787114803.84 total 20.08 last 12.69)
 - mkdir 331 (time 1787114813.84 total 30.09 last 7.90)
 - mkdir 474 (time 1787114823.87 total 40.12 last 14.26)
 - mkdir 649 (time 1787114833.93 total 50.17 last 17.41)
 - mkdir 800 (time 1787114843.95 total 60.19 last 15.07)
 - mkdir 896 (time 1787114854.13 total 70.37 last 9.43)
 - mkdir 989 (time 1787114864.17 total 80.41 last 9.26)
total: 1000 mkdir in 82.06 seconds: 12.19 ops/second
 - mkdir 120 (time 1787114793.78 total 10.05 last 11.94)
 - mkdir 251 (time 1787114803.82 total 20.09 last 13.04)
 - mkdir 327 (time 1787114813.83 total 30.10 last 7.59)
 - mkdir 465 (time 1787114823.86 total 40.13 last 13.77)
 - mkdir 642 (time 1787114833.90 total 50.17 last 17.62)
 - mkdir 797 (time 1787114844.02 total 60.29 last 15.32)
 - mkdir 893 (time 1787114854.06 total 70.33 last 9.56)
 - mkdir 985 (time 1787114864.09 total 80.37 last 9.17)
total: 1000 mkdir in 82.42 seconds: 12.13 ops/second
Time for file creation 92 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787114874 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1787114875 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1787114875 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1787114876 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1787114875 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1787114876 ; total 1 ; last 1)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1787114875 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1787114876 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1787114875 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1787114876 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
Time for file removal 41 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8