== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 21:31:41 (1788831101)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.49 seconds: 20.33 ops/second
 - mkdir 158 (time 1788831125.81 total 10.05 last 15.72)
 - mkdir 300 (time 1788831135.90 total 20.14 last 14.07)
 - mkdir 452 (time 1788831145.93 total 30.17 last 15.15)
 - mkdir 618 (time 1788831155.94 total 40.17 last 16.60)
 - mkdir 817 (time 1788831165.94 total 50.18 last 19.89)
 - mkdir 996 (time 1788831175.95 total 60.19 last 17.89)
total: 1000 mkdir in 60.44 seconds: 16.55 ops/second
 - mkdir 153 (time 1788831126.24 total 10.02 last 15.26)
 - mkdir 303 (time 1788831136.26 total 20.04 last 14.97)
 - mkdir 451 (time 1788831146.30 total 30.08 last 14.74)
 - mkdir 621 (time 1788831156.35 total 40.13 last 16.92)
 - mkdir 826 (time 1788831166.36 total 50.15 last 20.47)
 - mkdir 997 (time 1788831176.49 total 60.27 last 16.89)
total: 1000 mkdir in 60.53 seconds: 16.52 ops/second
 - mkdir 148 (time 1788831124.80 total 10.02 last 14.76)
 - mkdir 288 (time 1788831134.83 total 20.05 last 13.96)
 - mkdir 443 (time 1788831144.86 total 30.08 last 15.46)
 - mkdir 605 (time 1788831154.87 total 40.09 last 16.17)
 - mkdir 792 (time 1788831164.90 total 50.13 last 18.64)
 - mkdir 969 (time 1788831174.94 total 60.17 last 17.63)
total: 1000 mkdir in 62.81 seconds: 15.92 ops/second
 - mkdir 151 (time 1788831126.00 total 10.05 last 15.02)
 - mkdir 294 (time 1788831136.00 total 20.06 last 14.29)
 - mkdir 442 (time 1788831146.00 total 30.06 last 14.80)
 - mkdir 609 (time 1788831156.01 total 40.06 last 16.70)
 - mkdir 812 (time 1788831166.05 total 50.10 last 20.21)
 - mkdir 979 (time 1788831176.08 total 60.13 last 16.66)
total: 1000 mkdir in 62.62 seconds: 15.97 ops/second
 - mkdir 150 (time 1788831125.09 total 10.05 last 14.92)
 - mkdir 291 (time 1788831135.13 total 20.09 last 14.04)
 - mkdir 436 (time 1788831145.13 total 30.10 last 14.49)
 - mkdir 601 (time 1788831155.16 total 40.13 last 16.45)
 - mkdir 792 (time 1788831165.18 total 50.15 last 19.06)
 - mkdir 966 (time 1788831175.22 total 60.19 last 17.34)
total: 1000 mkdir in 63.63 seconds: 15.72 ops/second
 - mkdir 145 (time 1788831126.38 total 10.13 last 14.31)
 - mkdir 290 (time 1788831136.47 total 20.22 last 14.37)
 - mkdir 435 (time 1788831146.48 total 30.24 last 14.48)
 - mkdir 608 (time 1788831156.49 total 40.24 last 17.29)
 - mkdir 821 (time 1788831166.50 total 50.26 last 21.27)
 - mkdir 983 (time 1788831176.66 total 60.42 last 15.95)
total: 1000 mkdir in 62.75 seconds: 15.94 ops/second
 - mkdir 145 (time 1788831126.22 total 10.12 last 14.33)
 - mkdir 289 (time 1788831136.23 total 20.13 last 14.38)
 - mkdir 432 (time 1788831146.26 total 30.16 last 14.26)
 - mkdir 598 (time 1788831156.32 total 40.22 last 16.50)
 - mkdir 798 (time 1788831166.36 total 50.26 last 19.93)
 - mkdir 960 (time 1788831176.48 total 60.38 last 16.00)
total: 1000 mkdir in 65.33 seconds: 15.31 ops/second
 - mkdir 144 (time 1788831126.11 total 10.10 last 14.25)
 - mkdir 282 (time 1788831136.13 total 20.13 last 13.76)
 - mkdir 427 (time 1788831146.16 total 30.16 last 14.46)
 - mkdir 596 (time 1788831156.25 total 40.25 last 16.75)
 - mkdir 788 (time 1788831166.25 total 50.25 last 19.20)
 - mkdir 956 (time 1788831176.32 total 60.32 last 16.69)
total: 1000 mkdir in 65.48 seconds: 15.27 ops/second
 - mkdir 143 (time 1788831125.28 total 10.02 last 14.28)
 - mkdir 280 (time 1788831135.32 total 20.06 last 13.65)
 - mkdir 423 (time 1788831145.32 total 30.06 last 14.30)
 - mkdir 582 (time 1788831155.38 total 40.11 last 15.81)
 - mkdir 770 (time 1788831165.38 total 50.12 last 18.79)
 - mkdir 939 (time 1788831175.42 total 60.16 last 16.83)
total: 1000 mkdir in 67.58 seconds: 14.80 ops/second
 - mkdir 135 (time 1788831125.39 total 10.04 last 13.45)
 - mkdir 273 (time 1788831135.44 total 20.09 last 13.73)
 - mkdir 407 (time 1788831145.45 total 30.10 last 13.39)
 - mkdir 566 (time 1788831155.47 total 40.12 last 15.86)
 - mkdir 751 (time 1788831165.51 total 50.15 last 18.44)
 - mkdir 919 (time 1788831175.54 total 60.18 last 16.75)
total: 1000 mkdir in 68.95 seconds: 14.50 ops/second
Time for file creation 80 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 17 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.41 seconds: 24.55 ops/second
 - mkdir 138 (time 1788831744.56 total 10.00 last 13.80)
 - mkdir 297 (time 1788831754.58 total 20.02 last 15.87)
 - mkdir 461 (time 1788831764.62 total 30.06 last 16.33)
 - mkdir 621 (time 1788831774.63 total 40.07 last 15.98)
 - mkdir 786 (time 1788831784.69 total 50.13 last 16.41)
 - mkdir 941 (time 1788831794.71 total 60.16 last 15.46)
total: 1000 mkdir in 63.40 seconds: 15.77 ops/second
 - mkdir 129 (time 1788831742.97 total 10.05 last 12.84)
 - mkdir 271 (time 1788831752.97 total 20.05 last 14.19)
 - mkdir 428 (time 1788831763.00 total 30.08 last 15.65)
 - mkdir 601 (time 1788831773.05 total 40.13 last 17.22)
 - mkdir 766 (time 1788831783.07 total 50.15 last 16.47)
 - mkdir 919 (time 1788831793.09 total 60.17 last 15.27)
total: 1000 mkdir in 65.13 seconds: 15.36 ops/second
 - mkdir 137 (time 1788831744.47 total 10.01 last 13.68)
 - mkdir 292 (time 1788831754.51 total 20.06 last 15.43)
 - mkdir 448 (time 1788831764.53 total 30.07 last 15.58)
 - mkdir 616 (time 1788831774.53 total 40.08 last 16.79)
 - mkdir 777 (time 1788831784.57 total 50.12 last 16.04)
 - mkdir 935 (time 1788831794.61 total 60.16 last 15.74)
total: 1000 mkdir in 64.37 seconds: 15.54 ops/second
 - mkdir 145 (time 1788831745.50 total 10.08 last 14.38)
 - mkdir 300 (time 1788831755.51 total 20.10 last 15.48)
 - mkdir 462 (time 1788831765.56 total 30.14 last 16.12)
 - mkdir 621 (time 1788831775.59 total 40.18 last 15.85)
 - mkdir 787 (time 1788831785.62 total 50.21 last 16.55)
 - mkdir 951 (time 1788831795.65 total 60.24 last 16.35)
total: 1000 mkdir in 63.84 seconds: 15.66 ops/second
 - mkdir 144 (time 1788831745.29 total 10.03 last 14.36)
 - mkdir 296 (time 1788831755.37 total 20.11 last 15.08)
 - mkdir 457 (time 1788831765.40 total 30.14 last 16.04)
 - mkdir 619 (time 1788831775.42 total 40.16 last 16.18)
 - mkdir 782 (time 1788831785.43 total 50.17 last 16.28)
 - mkdir 941 (time 1788831795.48 total 60.22 last 15.82)
total: 1000 mkdir in 64.85 seconds: 15.42 ops/second
 - mkdir 154 (time 1788831746.07 total 10.02 last 15.37)
 - mkdir 310 (time 1788831756.09 total 20.05 last 15.56)
 - mkdir 472 (time 1788831766.14 total 30.09 last 16.13)
 - mkdir 630 (time 1788831776.17 total 40.12 last 15.75)
 - mkdir 795 (time 1788831786.20 total 50.15 last 16.45)
 - mkdir 954 (time 1788831796.24 total 60.19 last 15.84)
total: 1000 mkdir in 64.12 seconds: 15.59 ops/second
 - mkdir 146 (time 1788831746.18 total 10.00 last 14.60)
 - mkdir 296 (time 1788831756.20 total 20.03 last 14.96)
 - mkdir 465 (time 1788831766.23 total 30.06 last 16.85)
 - mkdir 625 (time 1788831776.29 total 40.12 last 15.90)
 - mkdir 788 (time 1788831786.32 total 50.14 last 16.27)
 - mkdir 951 (time 1788831796.34 total 60.17 last 16.25)
total: 1000 mkdir in 64.39 seconds: 15.53 ops/second
 - mkdir 131 (time 1788831744.62 total 10.07 last 13.01)
 - mkdir 279 (time 1788831754.67 total 20.13 last 14.72)
 - mkdir 430 (time 1788831764.76 total 30.21 last 14.97)
 - mkdir 593 (time 1788831774.81 total 40.27 last 16.21)
 - mkdir 757 (time 1788831784.84 total 50.29 last 16.36)
 - mkdir 918 (time 1788831794.86 total 60.31 last 16.07)
total: 1000 mkdir in 66.41 seconds: 15.06 ops/second
 - mkdir 146 (time 1788831745.78 total 10.03 last 14.56)
 - mkdir 294 (time 1788831755.83 total 20.07 last 14.73)
 - mkdir 454 (time 1788831765.85 total 30.10 last 15.96)
 - mkdir 616 (time 1788831775.92 total 40.16 last 16.10)
 - mkdir 779 (time 1788831785.97 total 50.22 last 16.21)
 - mkdir 941 (time 1788831795.99 total 60.23 last 16.18)
total: 1000 mkdir in 65.22 seconds: 15.33 ops/second
 - mkdir 118 (time 1788831744.04 total 10.04 last 11.76)
 - mkdir 261 (time 1788831754.09 total 20.09 last 14.23)
 - mkdir 409 (time 1788831764.11 total 30.11 last 14.77)
 - mkdir 577 (time 1788831774.12 total 40.12 last 16.79)
 - mkdir 739 (time 1788831784.13 total 50.13 last 16.18)
 - mkdir 896 (time 1788831794.15 total 60.15 last 15.67)
total: 1000 mkdir in 68.41 seconds: 14.62 ops/second
Time for file creation 80 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788831809 ; total 0 ; last 0)
total: 1000 unlinks in 26 seconds: 38.461540 unlinks/second
 - unlinked 0 (time 1788831811 ; total 1 ; last 1)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788831811 ; total 0 ; last 0)
total: 1000 unlinks in 27 seconds: 37.037037 unlinks/second
 - unlinked 0 (time 1788831812 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788831812 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788831812 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1788831812 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1788831812 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1788831812 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1788831812 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
Time for file removal 41 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8