== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 05:47:38 (1788601658)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.46 seconds: 21.80 ops/second
 - mkdir 153 (time 1788601679.34 total 10.04 last 15.23)
 - mkdir 308 (time 1788601689.36 total 20.07 last 15.47)
 - mkdir 451 (time 1788601699.37 total 30.07 last 14.29)
 - mkdir 596 (time 1788601709.40 total 40.10 last 14.46)
 - mkdir 755 (time 1788601719.41 total 50.12 last 15.87)
 - mkdir 903 (time 1788601729.46 total 60.17 last 14.73)
total: 1000 mkdir in 65.69 seconds: 15.22 ops/second
 - mkdir 148 (time 1788601679.75 total 10.02 last 14.78)
 - mkdir 307 (time 1788601689.77 total 20.05 last 15.85)
 - mkdir 449 (time 1788601699.79 total 30.06 last 14.18)
 - mkdir 599 (time 1788601709.80 total 40.07 last 14.98)
 - mkdir 750 (time 1788601719.83 total 50.10 last 15.06)
 - mkdir 902 (time 1788601729.84 total 60.11 last 15.18)
total: 1000 mkdir in 65.61 seconds: 15.24 ops/second
 - mkdir 155 (time 1788601680.76 total 10.03 last 15.46)
 - mkdir 307 (time 1788601690.85 total 20.12 last 15.07)
 - mkdir 448 (time 1788601700.86 total 30.13 last 14.08)
 - mkdir 603 (time 1788601710.92 total 40.19 last 15.41)
 - mkdir 752 (time 1788601720.99 total 50.25 last 14.81)
 - mkdir 908 (time 1788601731.06 total 60.32 last 15.49)
total: 1000 mkdir in 65.91 seconds: 15.17 ops/second
 - mkdir 148 (time 1788601679.81 total 10.01 last 14.79)
 - mkdir 305 (time 1788601689.82 total 20.01 last 15.69)
 - mkdir 443 (time 1788601699.95 total 30.15 last 13.62)
 - mkdir 593 (time 1788601709.96 total 40.15 last 14.99)
 - mkdir 742 (time 1788601720.00 total 50.19 last 14.84)
 - mkdir 892 (time 1788601730.01 total 60.20 last 14.99)
total: 1000 mkdir in 66.84 seconds: 14.96 ops/second
 - mkdir 155 (time 1788601680.48 total 10.05 last 15.43)
 - mkdir 308 (time 1788601690.52 total 20.08 last 15.24)
 - mkdir 444 (time 1788601700.55 total 30.12 last 13.55)
 - mkdir 590 (time 1788601710.57 total 40.14 last 14.57)
 - mkdir 740 (time 1788601720.66 total 50.23 last 14.86)
 - mkdir 894 (time 1788601730.69 total 60.26 last 15.36)
total: 1000 mkdir in 67.30 seconds: 14.86 ops/second
 - mkdir 147 (time 1788601680.23 total 10.02 last 14.67)
 - mkdir 304 (time 1788601690.32 total 20.11 last 15.56)
 - mkdir 438 (time 1788601700.39 total 30.19 last 13.29)
 - mkdir 589 (time 1788601710.44 total 40.23 last 15.03)
 - mkdir 734 (time 1788601720.47 total 50.26 last 14.45)
 - mkdir 889 (time 1788601730.49 total 60.28 last 15.47)
total: 1000 mkdir in 67.78 seconds: 14.75 ops/second
 - mkdir 148 (time 1788601680.53 total 10.01 last 14.78)
 - mkdir 303 (time 1788601690.54 total 20.03 last 15.48)
 - mkdir 438 (time 1788601700.61 total 30.10 last 13.41)
 - mkdir 590 (time 1788601710.62 total 40.11 last 15.18)
 - mkdir 738 (time 1788601720.71 total 50.20 last 14.66)
 - mkdir 895 (time 1788601730.72 total 60.21 last 15.69)
total: 1000 mkdir in 67.68 seconds: 14.78 ops/second
 - mkdir 143 (time 1788601680.15 total 10.03 last 14.26)
 - mkdir 291 (time 1788601690.22 total 20.10 last 14.70)
 - mkdir 422 (time 1788601700.29 total 30.17 last 13.01)
 - mkdir 576 (time 1788601710.30 total 40.18 last 15.39)
 - mkdir 728 (time 1788601720.32 total 50.20 last 15.17)
 - mkdir 884 (time 1788601730.34 total 60.21 last 15.57)
total: 1000 mkdir in 68.79 seconds: 14.54 ops/second
 - mkdir 147 (time 1788601680.48 total 10.04 last 14.64)
 - mkdir 297 (time 1788601690.51 total 20.07 last 14.95)
 - mkdir 431 (time 1788601700.58 total 30.14 last 13.31)
 - mkdir 579 (time 1788601710.60 total 40.16 last 14.77)
 - mkdir 731 (time 1788601720.62 total 50.17 last 15.18)
 - mkdir 883 (time 1788601730.69 total 60.25 last 15.09)
total: 1000 mkdir in 68.90 seconds: 14.51 ops/second
 - mkdir 139 (time 1788601679.75 total 10.01 last 13.89)
 - mkdir 284 (time 1788601689.78 total 20.03 last 14.47)
 - mkdir 418 (time 1788601699.84 total 30.10 last 13.31)
 - mkdir 568 (time 1788601709.88 total 40.13 last 14.95)
 - mkdir 720 (time 1788601719.88 total 50.13 last 15.20)
 - mkdir 872 (time 1788601729.92 total 60.17 last 15.14)
total: 1000 mkdir in 69.62 seconds: 14.36 ops/second
Time for file creation 79 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 11 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.53 seconds: 19.00 ops/second
 - mkdir 174 (time 1788602269.53 total 10.09 last 17.25)
 - mkdir 337 (time 1788602279.59 total 20.15 last 16.21)
 - mkdir 488 (time 1788602289.66 total 30.21 last 15.00)
 - mkdir 654 (time 1788602299.68 total 40.24 last 16.56)
 - mkdir 811 (time 1788602309.74 total 50.30 last 15.61)
 - mkdir 971 (time 1788602319.74 total 60.30 last 15.99)
total: 1000 mkdir in 62.05 seconds: 16.12 ops/second
 - mkdir 185 (time 1788602271.96 total 10.02 last 18.46)
 - mkdir 346 (time 1788602281.99 total 20.05 last 16.05)
 - mkdir 504 (time 1788602292.02 total 30.07 last 15.76)
 - mkdir 662 (time 1788602302.04 total 40.10 last 15.76)
 - mkdir 822 (time 1788602312.05 total 50.11 last 15.98)
 - mkdir 987 (time 1788602322.10 total 60.16 last 16.42)
total: 1000 mkdir in 61.33 seconds: 16.31 ops/second
 - mkdir 176 (time 1788602271.21 total 10.02 last 17.56)
 - mkdir 329 (time 1788602281.25 total 20.06 last 15.24)
 - mkdir 479 (time 1788602291.27 total 30.07 last 14.98)
 - mkdir 645 (time 1788602301.31 total 40.12 last 16.52)
 - mkdir 804 (time 1788602311.34 total 50.15 last 15.85)
 - mkdir 966 (time 1788602321.39 total 60.20 last 16.12)
total: 1000 mkdir in 63.22 seconds: 15.82 ops/second
 - mkdir 175 (time 1788602271.25 total 10.02 last 17.46)
 - mkdir 336 (time 1788602281.26 total 20.03 last 16.08)
 - mkdir 486 (time 1788602291.33 total 30.10 last 14.90)
 - mkdir 639 (time 1788602301.39 total 40.16 last 15.21)
 - mkdir 795 (time 1788602311.45 total 50.22 last 15.51)
 - mkdir 956 (time 1788602321.46 total 60.24 last 16.08)
total: 1000 mkdir in 64.87 seconds: 15.42 ops/second
 - mkdir 176 (time 1788602271.96 total 10.04 last 17.53)
 - mkdir 332 (time 1788602281.99 total 20.07 last 15.55)
 - mkdir 487 (time 1788602292.05 total 30.13 last 15.41)
 - mkdir 647 (time 1788602302.08 total 40.16 last 15.96)
 - mkdir 804 (time 1788602312.13 total 50.21 last 15.62)
 - mkdir 962 (time 1788602322.22 total 60.30 last 15.65)
total: 1000 mkdir in 64.31 seconds: 15.55 ops/second
 - mkdir 174 (time 1788602271.20 total 10.05 last 17.31)
 - mkdir 335 (time 1788602281.22 total 20.08 last 16.07)
 - mkdir 482 (time 1788602291.27 total 30.12 last 14.64)
 - mkdir 634 (time 1788602301.28 total 40.13 last 15.18)
 - mkdir 784 (time 1788602311.29 total 50.14 last 14.98)
 - mkdir 947 (time 1788602321.31 total 60.16 last 16.27)
total: 1000 mkdir in 65.47 seconds: 15.27 ops/second
 - mkdir 167 (time 1788602271.43 total 10.06 last 16.61)
 - mkdir 324 (time 1788602281.49 total 20.12 last 15.61)
 - mkdir 478 (time 1788602291.51 total 30.13 last 15.38)
 - mkdir 631 (time 1788602301.55 total 40.17 last 15.24)
 - mkdir 785 (time 1788602311.57 total 50.19 last 15.36)
 - mkdir 951 (time 1788602321.59 total 60.21 last 16.57)
total: 1000 mkdir in 65.78 seconds: 15.20 ops/second
 - mkdir 164 (time 1788602269.64 total 10.05 last 16.31)
 - mkdir 318 (time 1788602279.67 total 20.09 last 15.35)
 - mkdir 454 (time 1788602289.67 total 30.09 last 13.60)
 - mkdir 607 (time 1788602299.76 total 40.18 last 15.16)
 - mkdir 759 (time 1788602309.76 total 50.18 last 15.20)
 - mkdir 914 (time 1788602319.81 total 60.22 last 15.44)
total: 1000 mkdir in 67.90 seconds: 14.73 ops/second
 - mkdir 171 (time 1788602271.98 total 10.00 last 17.10)
 - mkdir 326 (time 1788602282.11 total 20.13 last 15.30)
 - mkdir 478 (time 1788602292.15 total 30.17 last 15.15)
 - mkdir 635 (time 1788602302.18 total 40.20 last 15.64)
 - mkdir 790 (time 1788602312.22 total 50.24 last 15.45)
 - mkdir 945 (time 1788602322.22 total 60.24 last 15.50)
total: 1000 mkdir in 66.07 seconds: 15.14 ops/second
 - mkdir 167 (time 1788602270.96 total 10.03 last 16.66)
 - mkdir 322 (time 1788602281.01 total 20.08 last 15.42)
 - mkdir 466 (time 1788602291.10 total 30.16 last 14.27)
 - mkdir 620 (time 1788602301.18 total 40.25 last 15.27)
 - mkdir 771 (time 1788602311.27 total 50.34 last 14.96)
 - mkdir 930 (time 1788602321.32 total 60.39 last 15.82)
total: 1000 mkdir in 67.41 seconds: 14.83 ops/second
Time for file creation 76 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788602336 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788602337 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788602336 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788602336 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788602337 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788602337 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788602337 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788602337 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788602337 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788602337 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
Time for file removal 41 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8