== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 02:17:33 (1790749053)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.68 seconds: 14.66 ops/second
 - mkdir 140 (time 1790749072.97 total 10.05 last 13.93)
 - mkdir 288 (time 1790749083.04 total 20.12 last 14.69)
 - mkdir 469 (time 1790749093.07 total 30.15 last 18.05)
 - mkdir 614 (time 1790749103.14 total 40.21 last 14.40)
 - mkdir 756 (time 1790749113.21 total 50.29 last 14.10)
 - mkdir 925 (time 1790749123.27 total 60.35 last 16.79)
total: 1000 mkdir in 65.17 seconds: 15.34 ops/second
 - mkdir 129 (time 1790749073.15 total 10.05 last 12.83)
 - mkdir 268 (time 1790749083.26 total 20.17 last 13.74)
 - mkdir 447 (time 1790749093.27 total 30.18 last 17.88)
 - mkdir 591 (time 1790749103.28 total 40.19 last 14.39)
 - mkdir 731 (time 1790749113.37 total 50.28 last 13.87)
 - mkdir 886 (time 1790749123.40 total 60.31 last 15.45)
total: 1000 mkdir in 69.58 seconds: 14.37 ops/second
 - mkdir 130 (time 1790749073.99 total 10.05 last 12.93)
 - mkdir 274 (time 1790749084.01 total 20.08 last 14.36)
 - mkdir 446 (time 1790749094.02 total 30.09 last 17.19)
 - mkdir 591 (time 1790749104.08 total 40.15 last 14.41)
 - mkdir 735 (time 1790749114.11 total 50.17 last 14.37)
 - mkdir 890 (time 1790749124.17 total 60.24 last 15.40)
total: 1000 mkdir in 69.52 seconds: 14.39 ops/second
 - mkdir 132 (time 1790749074.28 total 10.06 last 13.13)
 - mkdir 276 (time 1790749084.28 total 20.06 last 14.40)
 - mkdir 447 (time 1790749094.30 total 30.07 last 17.07)
 - mkdir 588 (time 1790749104.36 total 40.14 last 14.01)
 - mkdir 731 (time 1790749114.45 total 50.23 last 14.17)
 - mkdir 893 (time 1790749124.48 total 60.25 last 16.16)
total: 1000 mkdir in 69.28 seconds: 14.43 ops/second
 - mkdir 127 (time 1790749073.53 total 10.02 last 12.68)
 - mkdir 263 (time 1790749083.60 total 20.08 last 13.52)
 - mkdir 432 (time 1790749093.62 total 30.10 last 16.86)
 - mkdir 581 (time 1790749103.64 total 40.12 last 14.87)
 - mkdir 721 (time 1790749113.67 total 50.16 last 13.95)
 - mkdir 877 (time 1790749123.69 total 60.18 last 15.56)
 - mkdir 994 (time 1790749133.71 total 70.20 last 11.68)
total: 1000 mkdir in 70.76 seconds: 14.13 ops/second
 - mkdir 130 (time 1790749073.41 total 10.03 last 12.96)
 - mkdir 265 (time 1790749083.41 total 20.03 last 13.50)
 - mkdir 433 (time 1790749093.45 total 30.07 last 16.73)
 - mkdir 577 (time 1790749103.54 total 40.16 last 14.28)
 - mkdir 715 (time 1790749113.54 total 50.16 last 13.80)
 - mkdir 872 (time 1790749123.62 total 60.24 last 15.58)
 - mkdir 992 (time 1790749133.70 total 70.32 last 11.91)
total: 1000 mkdir in 71.18 seconds: 14.05 ops/second
 - mkdir 132 (time 1790749074.30 total 10.05 last 13.14)
 - mkdir 273 (time 1790749084.33 total 20.08 last 14.05)
 - mkdir 446 (time 1790749094.37 total 30.12 last 17.24)
 - mkdir 586 (time 1790749104.42 total 40.17 last 13.92)
 - mkdir 729 (time 1790749114.46 total 50.21 last 14.25)
 - mkdir 887 (time 1790749124.52 total 60.27 last 15.70)
 - mkdir 998 (time 1790749134.54 total 70.28 last 11.08)
total: 1000 mkdir in 70.39 seconds: 14.21 ops/second
 - mkdir 131 (time 1790749073.99 total 10.03 last 13.06)
 - mkdir 274 (time 1790749084.02 total 20.06 last 14.26)
 - mkdir 451 (time 1790749094.05 total 30.09 last 17.64)
 - mkdir 594 (time 1790749104.09 total 40.13 last 14.25)
 - mkdir 738 (time 1790749114.14 total 50.19 last 14.31)
 - mkdir 889 (time 1790749124.22 total 60.26 last 14.99)
 - mkdir 996 (time 1790749134.24 total 70.29 last 10.67)
total: 1000 mkdir in 70.71 seconds: 14.14 ops/second
 - mkdir 129 (time 1790749073.99 total 10.04 last 12.85)
 - mkdir 269 (time 1790749084.00 total 20.05 last 13.99)
 - mkdir 437 (time 1790749094.01 total 30.06 last 16.78)
 - mkdir 581 (time 1790749104.09 total 40.13 last 14.30)
 - mkdir 721 (time 1790749114.13 total 50.18 last 13.93)
 - mkdir 878 (time 1790749124.14 total 60.19 last 15.69)
 - mkdir 994 (time 1790749134.18 total 70.23 last 11.56)
total: 1000 mkdir in 70.81 seconds: 14.12 ops/second
 - mkdir 123 (time 1790749073.41 total 10.01 last 12.28)
 - mkdir 257 (time 1790749083.41 total 20.01 last 13.40)
 - mkdir 422 (time 1790749093.46 total 30.06 last 16.42)
 - mkdir 562 (time 1790749103.52 total 40.12 last 13.92)
 - mkdir 691 (time 1790749113.53 total 50.13 last 12.89)
 - mkdir 846 (time 1790749123.56 total 60.17 last 15.45)
 - mkdir 969 (time 1790749133.66 total 70.26 last 12.19)
total: 1000 mkdir in 72.81 seconds: 13.73 ops/second
Time for file creation 81 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 12 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.61 seconds: 16.27 ops/second
 - mkdir 151 (time 1790749707.91 total 10.00 last 15.10)
 - mkdir 282 (time 1790749718.02 total 20.11 last 12.96)
 - mkdir 420 (time 1790749728.04 total 30.13 last 13.78)
 - mkdir 571 (time 1790749738.11 total 40.19 last 15.00)
 - mkdir 702 (time 1790749748.15 total 50.24 last 13.04)
 - mkdir 853 (time 1790749758.19 total 60.28 last 15.04)
 - mkdir 992 (time 1790749768.23 total 70.32 last 13.85)
total: 1000 mkdir in 70.78 seconds: 14.13 ops/second
 - mkdir 149 (time 1790749707.77 total 10.00 last 14.90)
 - mkdir 280 (time 1790749717.90 total 20.13 last 12.93)
 - mkdir 419 (time 1790749727.99 total 30.22 last 13.78)
 - mkdir 570 (time 1790749738.01 total 40.24 last 15.07)
 - mkdir 702 (time 1790749748.06 total 50.29 last 13.13)
 - mkdir 850 (time 1790749758.12 total 60.35 last 14.71)
 - mkdir 991 (time 1790749768.15 total 70.38 last 14.05)
total: 1000 mkdir in 70.93 seconds: 14.10 ops/second
 - mkdir 148 (time 1790749708.99 total 10.01 last 14.79)
 - mkdir 290 (time 1790749719.02 total 20.04 last 14.16)
 - mkdir 427 (time 1790749729.05 total 30.06 last 13.67)
 - mkdir 577 (time 1790749739.11 total 40.13 last 14.90)
 - mkdir 713 (time 1790749749.18 total 50.19 last 13.51)
 - mkdir 863 (time 1790749759.26 total 60.28 last 14.87)
total: 1000 mkdir in 70.20 seconds: 14.25 ops/second
 - mkdir 146 (time 1790749708.83 total 10.03 last 14.55)
 - mkdir 283 (time 1790749718.84 total 20.05 last 13.68)
 - mkdir 415 (time 1790749728.84 total 30.05 last 13.20)
 - mkdir 566 (time 1790749738.94 total 40.14 last 14.96)
 - mkdir 703 (time 1790749748.97 total 50.17 last 13.66)
 - mkdir 852 (time 1790749759.05 total 60.26 last 14.77)
 - mkdir 987 (time 1790749769.08 total 70.29 last 13.46)
total: 1000 mkdir in 71.29 seconds: 14.03 ops/second
 - mkdir 147 (time 1790749708.76 total 10.11 last 14.54)
 - mkdir 283 (time 1790749718.80 total 20.16 last 13.53)
 - mkdir 416 (time 1790749728.84 total 30.20 last 13.25)
 - mkdir 562 (time 1790749738.94 total 40.29 last 14.46)
 - mkdir 698 (time 1790749748.99 total 50.34 last 13.54)
 - mkdir 846 (time 1790749759.04 total 60.39 last 14.72)
 - mkdir 985 (time 1790749769.12 total 70.47 last 13.79)
total: 1000 mkdir in 71.69 seconds: 13.95 ops/second
 - mkdir 146 (time 1790749709.16 total 10.00 last 14.60)
 - mkdir 280 (time 1790749719.16 total 20.01 last 13.39)
 - mkdir 417 (time 1790749729.24 total 30.08 last 13.60)
 - mkdir 563 (time 1790749739.26 total 40.10 last 14.58)
 - mkdir 703 (time 1790749749.28 total 50.13 last 13.96)
 - mkdir 853 (time 1790749759.35 total 60.19 last 14.90)
 - mkdir 988 (time 1790749769.38 total 70.22 last 13.46)
total: 1000 mkdir in 71.20 seconds: 14.04 ops/second
 - mkdir 148 (time 1790749708.96 total 10.02 last 14.77)
 - mkdir 278 (time 1790749718.96 total 20.02 last 13.00)
 - mkdir 412 (time 1790749729.04 total 30.11 last 13.29)
 - mkdir 560 (time 1790749739.09 total 40.15 last 14.73)
 - mkdir 698 (time 1790749749.13 total 50.19 last 13.74)
 - mkdir 846 (time 1790749759.14 total 60.20 last 14.79)
 - mkdir 979 (time 1790749769.15 total 70.21 last 13.29)
total: 1000 mkdir in 72.28 seconds: 13.84 ops/second
 - mkdir 148 (time 1790749709.41 total 10.07 last 14.69)
 - mkdir 284 (time 1790749719.42 total 20.09 last 13.58)
 - mkdir 414 (time 1790749729.50 total 30.16 last 12.90)
 - mkdir 560 (time 1790749739.51 total 40.18 last 14.58)
 - mkdir 698 (time 1790749749.71 total 50.37 last 13.54)
 - mkdir 847 (time 1790749759.72 total 60.39 last 14.88)
 - mkdir 983 (time 1790749769.81 total 70.47 last 13.48)
total: 1000 mkdir in 72.12 seconds: 13.87 ops/second
 - mkdir 145 (time 1790749707.49 total 10.01 last 14.49)
 - mkdir 275 (time 1790749717.58 total 20.10 last 12.88)
 - mkdir 399 (time 1790749727.68 total 30.20 last 12.28)
 - mkdir 544 (time 1790749737.76 total 40.28 last 14.39)
 - mkdir 677 (time 1790749747.76 total 50.29 last 13.29)
 - mkdir 825 (time 1790749757.77 total 60.29 last 14.79)
 - mkdir 962 (time 1790749767.86 total 70.38 last 13.58)
total: 1000 mkdir in 74.07 seconds: 13.50 ops/second
 - mkdir 142 (time 1790749708.69 total 10.01 last 14.18)
 - mkdir 277 (time 1790749718.72 total 20.04 last 13.46)
 - mkdir 402 (time 1790749728.76 total 30.08 last 12.46)
 - mkdir 551 (time 1790749738.79 total 40.11 last 14.86)
 - mkdir 686 (time 1790749748.81 total 50.13 last 13.47)
 - mkdir 835 (time 1790749758.81 total 60.13 last 14.90)
 - mkdir 973 (time 1790749768.86 total 70.19 last 13.73)
total: 1000 mkdir in 72.90 seconds: 13.72 ops/second
Time for file creation 84 sec for 1 RPC sent at a time
 - unlinked 0 (time 1790749778 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1790749780 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1790749781 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1790749781 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1790749782 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1790749782 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1790749782 ; total 1 ; last 1)
total: 1000 unlinks in 42 seconds: 23.809525 unlinks/second
 - unlinked 0 (time 1790749782 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1790749782 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1790749782 ; total 0 ; last 0)
total: 1000 unlinks in 42 seconds: 23.809525 unlinks/second
Time for file removal 53 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8