== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 03:51:55 (1789285915)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.39 seconds: 25.81 ops/second
 - mkdir 157 (time 1789285937.40 total 10.08 last 15.57)
 - mkdir 311 (time 1789285947.44 total 20.12 last 15.35)
 - mkdir 457 (time 1789285957.46 total 30.14 last 14.57)
 - mkdir 594 (time 1789285967.52 total 40.20 last 13.62)
 - mkdir 757 (time 1789285977.52 total 50.20 last 16.29)
 - mkdir 911 (time 1789285987.55 total 60.23 last 15.36)
total: 1000 mkdir in 66.59 seconds: 15.02 ops/second
 - mkdir 149 (time 1789285936.72 total 10.00 last 14.89)
 - mkdir 295 (time 1789285946.76 total 20.04 last 14.54)
 - mkdir 441 (time 1789285956.78 total 30.07 last 14.57)
 - mkdir 576 (time 1789285966.84 total 40.12 last 13.43)
 - mkdir 744 (time 1789285976.89 total 50.17 last 16.72)
 - mkdir 902 (time 1789285986.92 total 60.20 last 15.75)
total: 1000 mkdir in 67.37 seconds: 14.84 ops/second
 - mkdir 147 (time 1789285937.11 total 10.05 last 14.63)
 - mkdir 300 (time 1789285947.14 total 20.08 last 15.25)
 - mkdir 443 (time 1789285957.17 total 30.12 last 14.25)
 - mkdir 579 (time 1789285967.22 total 40.16 last 13.54)
 - mkdir 744 (time 1789285977.23 total 50.17 last 16.48)
 - mkdir 892 (time 1789285987.32 total 60.26 last 14.67)
total: 1000 mkdir in 68.55 seconds: 14.59 ops/second
 - mkdir 158 (time 1789285938.45 total 10.02 last 15.78)
 - mkdir 312 (time 1789285948.48 total 20.05 last 15.35)
 - mkdir 455 (time 1789285958.56 total 30.13 last 14.19)
 - mkdir 596 (time 1789285968.63 total 40.19 last 14.01)
 - mkdir 756 (time 1789285978.64 total 50.21 last 15.97)
 - mkdir 906 (time 1789285988.67 total 60.23 last 14.96)
total: 1000 mkdir in 67.60 seconds: 14.79 ops/second
 - mkdir 153 (time 1789285937.92 total 10.02 last 15.27)
 - mkdir 305 (time 1789285947.97 total 20.07 last 15.13)
 - mkdir 440 (time 1789285957.98 total 30.08 last 13.48)
 - mkdir 581 (time 1789285968.02 total 40.13 last 14.04)
 - mkdir 746 (time 1789285978.10 total 50.20 last 16.38)
 - mkdir 896 (time 1789285988.13 total 60.23 last 14.95)
total: 1000 mkdir in 68.53 seconds: 14.59 ops/second
 - mkdir 155 (time 1789285938.75 total 10.01 last 15.48)
 - mkdir 306 (time 1789285948.77 total 20.03 last 15.08)
 - mkdir 445 (time 1789285958.84 total 30.10 last 13.80)
 - mkdir 589 (time 1789285968.86 total 40.12 last 14.37)
 - mkdir 755 (time 1789285978.86 total 50.13 last 16.60)
 - mkdir 900 (time 1789285988.99 total 60.25 last 14.32)
total: 1000 mkdir in 68.08 seconds: 14.69 ops/second
 - mkdir 157 (time 1789285938.66 total 10.01 last 15.68)
 - mkdir 304 (time 1789285948.66 total 20.02 last 14.69)
 - mkdir 445 (time 1789285958.67 total 30.02 last 14.09)
 - mkdir 587 (time 1789285968.69 total 40.04 last 14.18)
 - mkdir 746 (time 1789285978.71 total 50.06 last 15.86)
 - mkdir 895 (time 1789285988.74 total 60.09 last 14.86)
total: 1000 mkdir in 68.80 seconds: 14.54 ops/second
 - mkdir 159 (time 1789285938.56 total 10.03 last 15.85)
 - mkdir 300 (time 1789285948.56 total 20.04 last 14.10)
 - mkdir 438 (time 1789285958.60 total 30.07 last 13.75)
 - mkdir 583 (time 1789285968.72 total 40.20 last 14.32)
 - mkdir 743 (time 1789285978.77 total 50.25 last 15.93)
 - mkdir 893 (time 1789285988.85 total 60.33 last 14.87)
total: 1000 mkdir in 69.44 seconds: 14.40 ops/second
 - mkdir 150 (time 1789285938.65 total 10.09 last 14.87)
 - mkdir 302 (time 1789285948.72 total 20.16 last 15.09)
 - mkdir 439 (time 1789285958.74 total 30.17 last 13.68)
 - mkdir 584 (time 1789285968.81 total 40.24 last 14.40)
 - mkdir 743 (time 1789285978.82 total 50.25 last 15.88)
 - mkdir 890 (time 1789285988.85 total 60.29 last 14.65)
total: 1000 mkdir in 69.90 seconds: 14.31 ops/second
 - mkdir 142 (time 1789285938.46 total 10.06 last 14.12)
 - mkdir 283 (time 1789285948.49 total 20.08 last 14.06)
 - mkdir 420 (time 1789285958.49 total 30.08 last 13.70)
 - mkdir 567 (time 1789285968.50 total 40.10 last 14.68)
 - mkdir 729 (time 1789285978.61 total 50.21 last 16.03)
 - mkdir 881 (time 1789285988.63 total 60.23 last 15.16)
 - mkdir 989 (time 1789285998.64 total 70.24 last 10.79)
total: 1000 mkdir in 71.17 seconds: 14.05 ops/second
Time for file creation 82 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 15 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.47 seconds: 21.22 ops/second
 - mkdir 164 (time 1789286474.51 total 10.03 last 16.35)
 - mkdir 342 (time 1789286484.52 total 20.05 last 17.77)
 - mkdir 558 (time 1789286494.56 total 30.09 last 21.52)
 - mkdir 774 (time 1789286504.59 total 40.12 last 21.54)
 - mkdir 988 (time 1789286514.65 total 50.18 last 21.28)
total: 1000 mkdir in 50.98 seconds: 19.62 ops/second
 - mkdir 171 (time 1789286475.23 total 10.02 last 17.06)
 - mkdir 348 (time 1789286485.25 total 20.04 last 17.67)
 - mkdir 578 (time 1789286495.27 total 30.05 last 22.96)
 - mkdir 779 (time 1789286505.30 total 40.09 last 20.03)
 - mkdir 991 (time 1789286515.41 total 50.19 last 20.98)
total: 1000 mkdir in 51.09 seconds: 19.57 ops/second
 - mkdir 174 (time 1789286475.73 total 10.00 last 17.40)
 - mkdir 351 (time 1789286485.77 total 20.04 last 17.64)
 - mkdir 570 (time 1789286495.79 total 30.06 last 21.84)
 - mkdir 770 (time 1789286505.84 total 40.11 last 19.91)
 - mkdir 977 (time 1789286515.92 total 50.19 last 20.54)
total: 1000 mkdir in 52.43 seconds: 19.07 ops/second
 - mkdir 182 (time 1789286476.70 total 10.05 last 18.12)
 - mkdir 372 (time 1789286486.70 total 20.05 last 18.99)
 - mkdir 588 (time 1789286496.74 total 30.08 last 21.53)
 - mkdir 786 (time 1789286506.78 total 40.12 last 19.72)
 - mkdir 982 (time 1789286516.78 total 50.13 last 19.60)
total: 1000 mkdir in 51.87 seconds: 19.28 ops/second
 - mkdir 178 (time 1789286476.53 total 10.04 last 17.73)
 - mkdir 370 (time 1789286486.54 total 20.05 last 19.17)
 - mkdir 579 (time 1789286496.54 total 30.05 last 20.90)
 - mkdir 782 (time 1789286506.56 total 40.07 last 20.27)
 - mkdir 980 (time 1789286516.56 total 50.07 last 19.80)
total: 1000 mkdir in 52.25 seconds: 19.14 ops/second
 - mkdir 159 (time 1789286475.01 total 10.02 last 15.86)
 - mkdir 319 (time 1789286485.02 total 20.04 last 15.97)
 - mkdir 542 (time 1789286495.03 total 30.05 last 22.28)
 - mkdir 752 (time 1789286505.09 total 40.10 last 20.89)
 - mkdir 963 (time 1789286515.10 total 50.11 last 21.08)
total: 1000 mkdir in 53.87 seconds: 18.56 ops/second
 - mkdir 175 (time 1789286476.49 total 10.02 last 17.47)
 - mkdir 367 (time 1789286486.50 total 20.03 last 19.18)
 - mkdir 581 (time 1789286496.51 total 30.04 last 21.38)
 - mkdir 776 (time 1789286506.52 total 40.04 last 19.50)
 - mkdir 976 (time 1789286516.53 total 50.06 last 19.97)
total: 1000 mkdir in 52.60 seconds: 19.01 ops/second
 - mkdir 176 (time 1789286476.45 total 10.01 last 17.59)
 - mkdir 372 (time 1789286486.46 total 20.02 last 19.58)
 - mkdir 580 (time 1789286496.49 total 30.05 last 20.73)
 - mkdir 772 (time 1789286506.50 total 40.06 last 19.18)
 - mkdir 972 (time 1789286516.51 total 50.07 last 19.98)
total: 1000 mkdir in 52.88 seconds: 18.91 ops/second
 - mkdir 187 (time 1789286476.83 total 10.01 last 18.69)
 - mkdir 373 (time 1789286486.83 total 20.01 last 18.60)
 - mkdir 582 (time 1789286496.86 total 30.04 last 20.84)
 - mkdir 776 (time 1789286506.94 total 40.12 last 19.24)
 - mkdir 972 (time 1789286516.96 total 50.14 last 19.57)
total: 1000 mkdir in 52.98 seconds: 18.88 ops/second
 - mkdir 171 (time 1789286476.08 total 10.01 last 17.09)
 - mkdir 348 (time 1789286486.09 total 20.01 last 17.69)
 - mkdir 558 (time 1789286496.09 total 30.02 last 20.99)
 - mkdir 758 (time 1789286506.14 total 40.06 last 19.91)
 - mkdir 962 (time 1789286516.14 total 50.07 last 20.39)
total: 1000 mkdir in 53.89 seconds: 18.55 ops/second
Time for file creation 66 sec for 1 RPC sent at a time
 - unlinked 0 (time 1789286528 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789286530 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789286530 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789286530 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1789286531 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1789286531 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1789286531 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1789286531 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1789286532 ; total 1 ; last 1)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1789286531 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
Time for file removal 45 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8