== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 07:25:04 (1788866704)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.47 seconds: 21.38 ops/second
 - mkdir 139 (time 1788866726.05 total 10.12 last 13.73)
 - mkdir 283 (time 1788866736.05 total 20.13 last 14.40)
 - mkdir 409 (time 1788866746.08 total 30.16 last 12.56)
 - mkdir 550 (time 1788866756.09 total 40.17 last 14.09)
 - mkdir 685 (time 1788866766.12 total 50.19 last 13.46)
 - mkdir 839 (time 1788866776.12 total 60.19 last 15.40)
 - mkdir 997 (time 1788866786.17 total 70.25 last 15.71)
total: 1000 mkdir in 70.39 seconds: 14.21 ops/second
 - mkdir 136 (time 1788866727.92 total 10.11 last 13.45)
 - mkdir 288 (time 1788866738.04 total 20.22 last 15.03)
 - mkdir 409 (time 1788866748.09 total 30.28 last 12.03)
 - mkdir 540 (time 1788866758.13 total 40.32 last 13.05)
 - mkdir 685 (time 1788866768.18 total 50.37 last 14.42)
 - mkdir 835 (time 1788866778.21 total 60.40 last 14.96)
 - mkdir 978 (time 1788866788.33 total 70.52 last 14.13)
total: 1000 mkdir in 73.99 seconds: 13.51 ops/second
 - mkdir 134 (time 1788866728.37 total 10.05 last 13.33)
 - mkdir 296 (time 1788866738.42 total 20.10 last 16.12)
 - mkdir 413 (time 1788866748.51 total 30.20 last 11.59)
 - mkdir 544 (time 1788866758.52 total 40.20 last 13.09)
 - mkdir 691 (time 1788866768.58 total 50.27 last 14.60)
 - mkdir 844 (time 1788866778.62 total 60.31 last 15.24)
 - mkdir 978 (time 1788866788.74 total 70.43 last 13.24)
total: 1000 mkdir in 73.76 seconds: 13.56 ops/second
 - mkdir 134 (time 1788866728.34 total 10.03 last 13.35)
 - mkdir 292 (time 1788866738.38 total 20.07 last 15.74)
 - mkdir 408 (time 1788866748.42 total 30.11 last 11.55)
 - mkdir 539 (time 1788866758.45 total 40.14 last 13.06)
 - mkdir 687 (time 1788866768.48 total 50.17 last 14.76)
 - mkdir 842 (time 1788866778.57 total 60.26 last 15.35)
 - mkdir 978 (time 1788866788.58 total 70.27 last 13.59)
total: 1000 mkdir in 73.88 seconds: 13.54 ops/second
 - mkdir 142 (time 1788866728.07 total 10.02 last 14.17)
 - mkdir 294 (time 1788866738.08 total 20.03 last 15.18)
 - mkdir 415 (time 1788866748.10 total 30.05 last 12.09)
 - mkdir 539 (time 1788866758.12 total 40.07 last 12.37)
 - mkdir 684 (time 1788866768.15 total 50.10 last 14.45)
 - mkdir 836 (time 1788866778.18 total 60.13 last 15.16)
 - mkdir 972 (time 1788866788.18 total 70.13 last 13.60)
total: 1000 mkdir in 74.36 seconds: 13.45 ops/second
 - mkdir 136 (time 1788866727.95 total 10.06 last 13.52)
 - mkdir 289 (time 1788866738.03 total 20.14 last 15.18)
 - mkdir 400 (time 1788866748.09 total 30.20 last 11.03)
 - mkdir 524 (time 1788866758.11 total 40.22 last 12.38)
 - mkdir 671 (time 1788866768.14 total 50.25 last 14.65)
 - mkdir 823 (time 1788866778.16 total 60.27 last 15.18)
 - mkdir 964 (time 1788866788.20 total 70.31 last 14.04)
total: 1000 mkdir in 75.99 seconds: 13.16 ops/second
 - mkdir 131 (time 1788866728.45 total 10.02 last 13.07)
 - mkdir 283 (time 1788866738.46 total 20.04 last 15.18)
 - mkdir 396 (time 1788866748.57 total 30.14 last 11.18)
 - mkdir 531 (time 1788866758.63 total 40.21 last 13.41)
 - mkdir 678 (time 1788866768.66 total 50.24 last 14.66)
 - mkdir 828 (time 1788866778.67 total 60.24 last 14.99)
 - mkdir 962 (time 1788866788.76 total 70.33 last 13.28)
total: 1000 mkdir in 76.07 seconds: 13.15 ops/second
 - mkdir 134 (time 1788866728.54 total 10.13 last 13.23)
 - mkdir 279 (time 1788866738.56 total 20.15 last 14.47)
 - mkdir 394 (time 1788866748.66 total 30.25 last 11.39)
 - mkdir 526 (time 1788866758.73 total 40.32 last 13.11)
 - mkdir 669 (time 1788866768.73 total 50.32 last 14.30)
 - mkdir 823 (time 1788866778.76 total 60.35 last 15.36)
 - mkdir 960 (time 1788866788.89 total 70.48 last 13.53)
total: 1000 mkdir in 76.67 seconds: 13.04 ops/second
 - mkdir 128 (time 1788866727.59 total 10.04 last 12.75)
 - mkdir 273 (time 1788866737.66 total 20.11 last 14.40)
 - mkdir 378 (time 1788866747.73 total 30.17 last 10.43)
 - mkdir 504 (time 1788866757.80 total 40.24 last 12.51)
 - mkdir 652 (time 1788866767.83 total 50.28 last 14.74)
 - mkdir 803 (time 1788866777.85 total 60.30 last 15.07)
 - mkdir 942 (time 1788866787.87 total 70.31 last 13.88)
total: 1000 mkdir in 78.55 seconds: 12.73 ops/second
 - mkdir 134 (time 1788866728.84 total 10.01 last 13.38)
 - mkdir 277 (time 1788866738.90 total 20.08 last 14.21)
 - mkdir 389 (time 1788866748.92 total 30.09 last 11.18)
 - mkdir 521 (time 1788866758.93 total 40.11 last 13.18)
 - mkdir 664 (time 1788866768.97 total 50.15 last 14.24)
 - mkdir 813 (time 1788866778.98 total 60.15 last 14.89)
 - mkdir 946 (time 1788866788.99 total 70.17 last 13.28)
total: 1000 mkdir in 78.03 seconds: 12.82 ops/second
Time for file creation 90 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 15 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.40 seconds: 25.14 ops/second
 - mkdir 151 (time 1788867323.70 total 10.03 last 15.06)
 - mkdir 308 (time 1788867333.78 total 20.11 last 15.56)
 - mkdir 479 (time 1788867343.82 total 30.15 last 17.03)
 - mkdir 681 (time 1788867353.86 total 40.19 last 20.13)
 - mkdir 841 (time 1788867363.89 total 50.22 last 15.96)
total: 1000 mkdir in 59.32 seconds: 16.86 ops/second
 - mkdir 148 (time 1788867325.20 total 10.00 last 14.79)
 - mkdir 303 (time 1788867335.20 total 20.00 last 15.50)
 - mkdir 476 (time 1788867345.25 total 30.06 last 17.20)
 - mkdir 679 (time 1788867355.31 total 40.12 last 20.18)
 - mkdir 845 (time 1788867365.36 total 50.16 last 16.53)
 - mkdir 994 (time 1788867375.42 total 60.23 last 14.80)
total: 1000 mkdir in 61.06 seconds: 16.38 ops/second
 - mkdir 144 (time 1788867324.73 total 10.00 last 14.40)
 - mkdir 294 (time 1788867334.84 total 20.11 last 14.84)
 - mkdir 464 (time 1788867344.89 total 30.16 last 16.91)
 - mkdir 667 (time 1788867354.89 total 40.17 last 20.29)
 - mkdir 832 (time 1788867364.89 total 50.17 last 16.50)
 - mkdir 986 (time 1788867374.92 total 60.20 last 15.35)
total: 1000 mkdir in 61.57 seconds: 16.24 ops/second
 - mkdir 140 (time 1788867325.23 total 10.09 last 13.88)
 - mkdir 293 (time 1788867335.27 total 20.12 last 15.25)
 - mkdir 463 (time 1788867345.34 total 30.19 last 16.88)
 - mkdir 665 (time 1788867355.36 total 40.22 last 20.14)
 - mkdir 836 (time 1788867365.42 total 50.27 last 17.01)
 - mkdir 986 (time 1788867375.47 total 60.33 last 14.92)
total: 1000 mkdir in 62.48 seconds: 16.00 ops/second
 - mkdir 145 (time 1788867325.25 total 10.07 last 14.40)
 - mkdir 291 (time 1788867335.27 total 20.09 last 14.57)
 - mkdir 460 (time 1788867345.33 total 30.15 last 16.79)
 - mkdir 665 (time 1788867355.35 total 40.17 last 20.45)
 - mkdir 835 (time 1788867365.35 total 50.17 last 17.00)
 - mkdir 983 (time 1788867375.41 total 60.23 last 14.71)
total: 1000 mkdir in 62.95 seconds: 15.89 ops/second
 - mkdir 147 (time 1788867325.41 total 10.01 last 14.69)
 - mkdir 293 (time 1788867335.41 total 20.01 last 14.60)
 - mkdir 460 (time 1788867345.45 total 30.05 last 16.64)
 - mkdir 666 (time 1788867355.47 total 40.07 last 20.54)
 - mkdir 828 (time 1788867365.49 total 50.09 last 16.18)
 - mkdir 982 (time 1788867375.55 total 60.15 last 15.30)
total: 1000 mkdir in 63.47 seconds: 15.76 ops/second
 - mkdir 143 (time 1788867324.98 total 10.01 last 14.28)
 - mkdir 293 (time 1788867334.98 total 20.01 last 15.00)
 - mkdir 460 (time 1788867345.03 total 30.07 last 16.61)
 - mkdir 659 (time 1788867355.04 total 40.07 last 19.89)
 - mkdir 819 (time 1788867365.05 total 50.09 last 15.97)
 - mkdir 972 (time 1788867375.09 total 60.13 last 15.24)
total: 1000 mkdir in 64.22 seconds: 15.57 ops/second
 - mkdir 139 (time 1788867324.42 total 10.04 last 13.84)
 - mkdir 283 (time 1788867334.43 total 20.05 last 14.39)
 - mkdir 449 (time 1788867344.48 total 30.10 last 16.52)
 - mkdir 647 (time 1788867354.49 total 40.11 last 19.78)
 - mkdir 808 (time 1788867364.51 total 50.13 last 16.06)
 - mkdir 961 (time 1788867374.58 total 60.20 last 15.19)
total: 1000 mkdir in 65.53 seconds: 15.26 ops/second
 - mkdir 133 (time 1788867325.07 total 10.01 last 13.28)
 - mkdir 278 (time 1788867335.08 total 20.03 last 14.48)
 - mkdir 442 (time 1788867345.13 total 30.08 last 16.31)
 - mkdir 635 (time 1788867355.13 total 40.08 last 19.30)
 - mkdir 803 (time 1788867365.15 total 50.10 last 16.77)
 - mkdir 951 (time 1788867375.17 total 60.12 last 14.76)
total: 1000 mkdir in 66.83 seconds: 14.96 ops/second
 - mkdir 136 (time 1788867325.54 total 10.08 last 13.50)
 - mkdir 276 (time 1788867335.55 total 20.08 last 13.99)
 - mkdir 431 (time 1788867345.61 total 30.15 last 15.40)
 - mkdir 618 (time 1788867355.68 total 40.21 last 18.58)
 - mkdir 772 (time 1788867365.68 total 50.22 last 15.40)
 - mkdir 913 (time 1788867375.72 total 60.25 last 14.05)
total: 1000 mkdir in 69.02 seconds: 14.49 ops/second
Time for file creation 79 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788867390 ; total 0 ; last 0)
total: 1000 unlinks in 27 seconds: 37.037037 unlinks/second
 - unlinked 0 (time 1788867392 ; total 0 ; last 0)
total: 1000 unlinks in 27 seconds: 37.037037 unlinks/second
 - unlinked 0 (time 1788867392 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788867392 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788867392 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788867393 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788867392 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1788867392 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1788867393 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788867392 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
Time for file removal 40 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8