== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 04:36:04 (1789288564)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.65 seconds: 15.38 ops/second
 - mkdir 160 (time 1789288587.40 total 10.04 last 15.94)
 - mkdir 317 (time 1789288597.41 total 20.05 last 15.68)
 - mkdir 490 (time 1789288607.42 total 30.06 last 17.28)
 - mkdir 633 (time 1789288617.44 total 40.09 last 14.26)
 - mkdir 752 (time 1789288627.49 total 50.14 last 11.84)
 - mkdir 893 (time 1789288637.55 total 60.20 last 14.02)
total: 1000 mkdir in 66.15 seconds: 15.12 ops/second
 - mkdir 154 (time 1789288586.75 total 10.05 last 15.33)
 - mkdir 317 (time 1789288596.85 total 20.14 last 16.14)
 - mkdir 484 (time 1789288606.92 total 30.21 last 16.58)
 - mkdir 628 (time 1789288617.05 total 40.34 last 14.22)
 - mkdir 749 (time 1789288627.16 total 50.45 last 11.97)
 - mkdir 882 (time 1789288637.26 total 60.55 last 13.16)
total: 1000 mkdir in 67.30 seconds: 14.86 ops/second
 - mkdir 160 (time 1789288587.58 total 10.09 last 15.85)
 - mkdir 317 (time 1789288597.62 total 20.13 last 15.64)
 - mkdir 486 (time 1789288607.63 total 30.14 last 16.88)
 - mkdir 628 (time 1789288617.67 total 40.18 last 14.15)
 - mkdir 749 (time 1789288627.86 total 50.37 last 11.88)
 - mkdir 886 (time 1789288637.90 total 60.41 last 13.64)
total: 1000 mkdir in 67.06 seconds: 14.91 ops/second
 - mkdir 162 (time 1789288588.45 total 10.03 last 16.16)
 - mkdir 323 (time 1789288598.51 total 20.08 last 16.01)
 - mkdir 488 (time 1789288608.54 total 30.11 last 16.46)
 - mkdir 631 (time 1789288618.57 total 40.14 last 14.25)
 - mkdir 749 (time 1789288628.59 total 50.16 last 11.78)
 - mkdir 886 (time 1789288638.59 total 60.17 last 13.69)
total: 1000 mkdir in 67.80 seconds: 14.75 ops/second
 - mkdir 163 (time 1789288588.32 total 10.05 last 16.22)
 - mkdir 321 (time 1789288598.34 total 20.07 last 15.77)
 - mkdir 489 (time 1789288608.38 total 30.10 last 16.74)
 - mkdir 628 (time 1789288618.47 total 40.20 last 13.77)
 - mkdir 745 (time 1789288628.50 total 50.23 last 11.67)
 - mkdir 884 (time 1789288638.51 total 60.24 last 13.89)
total: 1000 mkdir in 68.21 seconds: 14.66 ops/second
 - mkdir 159 (time 1789288588.52 total 10.02 last 15.87)
 - mkdir 318 (time 1789288598.57 total 20.07 last 15.82)
 - mkdir 482 (time 1789288608.62 total 30.12 last 16.31)
 - mkdir 624 (time 1789288618.69 total 40.19 last 14.10)
 - mkdir 741 (time 1789288628.71 total 50.22 last 11.67)
 - mkdir 884 (time 1789288638.73 total 60.24 last 14.27)
total: 1000 mkdir in 68.50 seconds: 14.60 ops/second
 - mkdir 155 (time 1789288587.80 total 10.02 last 15.47)
 - mkdir 303 (time 1789288597.83 total 20.04 last 14.77)
 - mkdir 469 (time 1789288607.83 total 30.05 last 16.59)
 - mkdir 610 (time 1789288617.87 total 40.09 last 14.05)
 - mkdir 731 (time 1789288627.94 total 50.16 last 12.02)
 - mkdir 862 (time 1789288637.96 total 60.18 last 13.07)
 - mkdir 996 (time 1789288647.96 total 70.18 last 13.40)
total: 1000 mkdir in 70.69 seconds: 14.15 ops/second
 - mkdir 158 (time 1789288588.10 total 10.06 last 15.70)
 - mkdir 310 (time 1789288598.12 total 20.09 last 15.16)
 - mkdir 470 (time 1789288608.12 total 30.09 last 16.00)
 - mkdir 607 (time 1789288618.19 total 40.16 last 13.61)
 - mkdir 725 (time 1789288628.20 total 50.17 last 11.79)
 - mkdir 861 (time 1789288638.24 total 60.21 last 13.55)
 - mkdir 994 (time 1789288648.30 total 70.27 last 13.22)
total: 1000 mkdir in 70.98 seconds: 14.09 ops/second
 - mkdir 157 (time 1789288588.42 total 10.12 last 15.52)
 - mkdir 311 (time 1789288598.45 total 20.14 last 15.36)
 - mkdir 472 (time 1789288608.48 total 30.17 last 16.05)
 - mkdir 611 (time 1789288618.53 total 40.22 last 13.84)
 - mkdir 728 (time 1789288628.58 total 50.27 last 11.64)
 - mkdir 867 (time 1789288638.59 total 60.29 last 13.88)
 - mkdir 993 (time 1789288648.63 total 70.32 last 12.56)
total: 1000 mkdir in 71.26 seconds: 14.03 ops/second
 - mkdir 155 (time 1789288587.98 total 10.06 last 15.41)
 - mkdir 305 (time 1789288598.03 total 20.11 last 14.93)
 - mkdir 465 (time 1789288608.16 total 30.24 last 15.79)
 - mkdir 597 (time 1789288618.19 total 40.27 last 13.16)
 - mkdir 709 (time 1789288628.20 total 50.28 last 11.18)
 - mkdir 838 (time 1789288638.26 total 60.34 last 12.83)
 - mkdir 967 (time 1789288648.38 total 70.46 last 12.74)
total: 1000 mkdir in 74.18 seconds: 13.48 ops/second
Time for file creation 85 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 14 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.58 seconds: 17.38 ops/second
 - mkdir 146 (time 1789289222.45 total 10.04 last 14.55)
 - mkdir 296 (time 1789289232.46 total 20.05 last 14.98)
 - mkdir 456 (time 1789289242.58 total 30.17 last 15.81)
 - mkdir 603 (time 1789289252.62 total 40.21 last 14.64)
 - mkdir 736 (time 1789289262.64 total 50.23 last 13.27)
 - mkdir 885 (time 1789289272.69 total 60.28 last 14.83)
total: 1000 mkdir in 67.44 seconds: 14.83 ops/second
 - mkdir 129 (time 1789289220.83 total 10.10 last 12.77)
 - mkdir 268 (time 1789289230.86 total 20.13 last 13.85)
 - mkdir 426 (time 1789289240.90 total 30.18 last 15.73)
 - mkdir 568 (time 1789289250.93 total 40.20 last 14.17)
 - mkdir 700 (time 1789289260.95 total 50.22 last 13.17)
 - mkdir 842 (time 1789289270.96 total 60.23 last 14.19)
 - mkdir 984 (time 1789289281.00 total 70.27 last 14.15)
total: 1000 mkdir in 72.67 seconds: 13.76 ops/second
 - mkdir 141 (time 1789289223.38 total 10.02 last 14.07)
 - mkdir 291 (time 1789289233.43 total 20.07 last 14.93)
 - mkdir 448 (time 1789289243.48 total 30.12 last 15.62)
 - mkdir 591 (time 1789289253.63 total 40.27 last 14.09)
 - mkdir 732 (time 1789289263.69 total 50.33 last 14.02)
 - mkdir 875 (time 1789289273.71 total 60.35 last 14.27)
 - mkdir 998 (time 1789289283.82 total 70.46 last 12.17)
total: 1000 mkdir in 70.64 seconds: 14.16 ops/second
 - mkdir 140 (time 1789289223.34 total 10.01 last 13.99)
 - mkdir 284 (time 1789289233.35 total 20.02 last 14.38)
 - mkdir 439 (time 1789289243.41 total 30.08 last 15.41)
 - mkdir 581 (time 1789289253.54 total 40.21 last 14.02)
 - mkdir 722 (time 1789289263.55 total 50.22 last 14.09)
 - mkdir 866 (time 1789289273.57 total 60.24 last 14.37)
 - mkdir 994 (time 1789289283.69 total 70.36 last 12.65)
total: 1000 mkdir in 71.28 seconds: 14.03 ops/second
 - mkdir 143 (time 1789289223.00 total 10.05 last 14.23)
 - mkdir 288 (time 1789289233.04 total 20.09 last 14.44)
 - mkdir 438 (time 1789289243.04 total 30.09 last 14.99)
 - mkdir 581 (time 1789289253.16 total 40.21 last 14.13)
 - mkdir 720 (time 1789289263.23 total 50.28 last 13.81)
 - mkdir 861 (time 1789289273.27 total 60.32 last 14.04)
 - mkdir 989 (time 1789289283.30 total 70.36 last 12.76)
total: 1000 mkdir in 72.24 seconds: 13.84 ops/second
 - mkdir 146 (time 1789289224.08 total 10.02 last 14.57)
 - mkdir 297 (time 1789289234.16 total 20.10 last 14.98)
 - mkdir 447 (time 1789289244.18 total 30.12 last 14.97)
 - mkdir 585 (time 1789289254.24 total 40.18 last 13.71)
 - mkdir 728 (time 1789289264.26 total 50.20 last 14.28)
 - mkdir 875 (time 1789289274.28 total 60.21 last 14.67)
 - mkdir 991 (time 1789289284.46 total 70.40 last 11.39)
total: 1000 mkdir in 71.74 seconds: 13.94 ops/second
 - mkdir 140 (time 1789289223.85 total 10.12 last 13.83)
 - mkdir 287 (time 1789289233.90 total 20.18 last 14.62)
 - mkdir 435 (time 1789289243.91 total 30.19 last 14.79)
 - mkdir 575 (time 1789289253.99 total 40.27 last 13.88)
 - mkdir 719 (time 1789289264.06 total 50.34 last 14.30)
 - mkdir 869 (time 1789289274.10 total 60.38 last 14.94)
 - mkdir 984 (time 1789289284.11 total 70.39 last 11.48)
total: 1000 mkdir in 73.17 seconds: 13.67 ops/second
 - mkdir 136 (time 1789289223.04 total 10.04 last 13.55)
 - mkdir 279 (time 1789289233.07 total 20.07 last 14.26)
 - mkdir 433 (time 1789289243.10 total 30.09 last 15.36)
 - mkdir 575 (time 1789289253.12 total 40.11 last 14.18)
 - mkdir 708 (time 1789289263.16 total 50.15 last 13.24)
 - mkdir 851 (time 1789289273.18 total 60.17 last 14.27)
 - mkdir 979 (time 1789289283.31 total 70.30 last 12.64)
total: 1000 mkdir in 73.96 seconds: 13.52 ops/second
 - mkdir 136 (time 1789289223.32 total 10.07 last 13.51)
 - mkdir 284 (time 1789289233.35 total 20.10 last 14.76)
 - mkdir 431 (time 1789289243.35 total 30.10 last 14.69)
 - mkdir 573 (time 1789289253.42 total 40.17 last 14.10)
 - mkdir 713 (time 1789289263.48 total 50.23 last 13.92)
 - mkdir 854 (time 1789289273.50 total 60.25 last 14.07)
 - mkdir 974 (time 1789289283.78 total 70.54 last 11.67)
total: 1000 mkdir in 74.34 seconds: 13.45 ops/second
 - mkdir 128 (time 1789289222.05 total 10.05 last 12.74)
 - mkdir 266 (time 1789289232.06 total 20.06 last 13.79)
 - mkdir 415 (time 1789289242.16 total 30.16 last 14.75)
 - mkdir 559 (time 1789289252.18 total 40.18 last 14.36)
 - mkdir 688 (time 1789289262.23 total 50.23 last 12.84)
 - mkdir 831 (time 1789289272.31 total 60.31 last 14.18)
 - mkdir 963 (time 1789289282.35 total 70.35 last 13.15)
total: 1000 mkdir in 75.79 seconds: 13.19 ops/second
Time for file creation 88 sec for 1 RPC sent at a time
 - unlinked 0 (time 1789289295 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1789289298 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1789289299 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1789289300 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1789289300 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1789289300 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1789289299 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1789289300 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1789289300 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1789289300 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
Time for file removal 54 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8