== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 09:08:22 (1789564102)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.47 seconds: 21.39 ops/second
 - mkdir 145 (time 1789564121.80 total 10.04 last 14.44)
 - mkdir 319 (time 1789564131.81 total 20.06 last 17.38)
 - mkdir 482 (time 1789564141.82 total 30.06 last 16.29)
 - mkdir 644 (time 1789564151.83 total 40.07 last 16.19)
 - mkdir 821 (time 1789564161.84 total 50.08 last 17.68)
 - mkdir 995 (time 1789564171.86 total 60.10 last 17.37)
total: 1000 mkdir in 60.37 seconds: 16.57 ops/second
 - mkdir 164 (time 1789564124.37 total 10.03 last 16.36)
 - mkdir 332 (time 1789564134.41 total 20.07 last 16.73)
 - mkdir 512 (time 1789564144.45 total 30.11 last 17.92)
 - mkdir 681 (time 1789564154.47 total 40.12 last 16.88)
 - mkdir 848 (time 1789564164.50 total 50.16 last 16.65)
total: 1000 mkdir in 58.60 seconds: 17.06 ops/second
 - mkdir 162 (time 1789564124.64 total 10.02 last 16.17)
 - mkdir 333 (time 1789564134.69 total 20.07 last 17.02)
 - mkdir 507 (time 1789564144.72 total 30.10 last 17.35)
 - mkdir 677 (time 1789564154.80 total 40.17 last 16.87)
 - mkdir 843 (time 1789564164.81 total 50.19 last 16.58)
total: 1000 mkdir in 59.53 seconds: 16.80 ops/second
 - mkdir 162 (time 1789564124.64 total 10.05 last 16.13)
 - mkdir 335 (time 1789564134.69 total 20.10 last 17.21)
 - mkdir 511 (time 1789564144.70 total 30.10 last 17.59)
 - mkdir 680 (time 1789564154.80 total 40.21 last 16.73)
 - mkdir 847 (time 1789564164.83 total 50.23 last 16.65)
total: 1000 mkdir in 59.57 seconds: 16.79 ops/second
 - mkdir 150 (time 1789564123.00 total 10.08 last 14.88)
 - mkdir 319 (time 1789564133.02 total 20.11 last 16.86)
 - mkdir 483 (time 1789564143.03 total 30.11 last 16.39)
 - mkdir 643 (time 1789564153.07 total 40.15 last 15.93)
 - mkdir 823 (time 1789564163.12 total 50.21 last 17.90)
 - mkdir 988 (time 1789564173.22 total 60.30 last 16.35)
total: 1000 mkdir in 61.41 seconds: 16.28 ops/second
 - mkdir 156 (time 1789564123.92 total 10.02 last 15.57)
 - mkdir 324 (time 1789564133.93 total 20.03 last 16.79)
 - mkdir 492 (time 1789564143.94 total 30.04 last 16.78)
 - mkdir 664 (time 1789564153.96 total 40.05 last 17.17)
 - mkdir 833 (time 1789564164.02 total 50.12 last 16.79)
 - mkdir 997 (time 1789564174.07 total 60.17 last 16.32)
total: 1000 mkdir in 60.42 seconds: 16.55 ops/second
 - mkdir 168 (time 1789564125.08 total 10.02 last 16.76)
 - mkdir 338 (time 1789564135.10 total 20.05 last 16.96)
 - mkdir 516 (time 1789564145.13 total 30.07 last 17.76)
 - mkdir 680 (time 1789564155.21 total 40.15 last 16.27)
 - mkdir 849 (time 1789564165.22 total 50.17 last 16.88)
total: 1000 mkdir in 59.48 seconds: 16.81 ops/second
 - mkdir 164 (time 1789564124.79 total 10.02 last 16.37)
 - mkdir 331 (time 1789564134.82 total 20.05 last 16.64)
 - mkdir 503 (time 1789564144.84 total 30.07 last 17.17)
 - mkdir 666 (time 1789564154.87 total 40.10 last 16.25)
 - mkdir 833 (time 1789564164.93 total 50.16 last 16.61)
 - mkdir 997 (time 1789564174.96 total 60.19 last 16.35)
total: 1000 mkdir in 60.29 seconds: 16.59 ops/second
 - mkdir 156 (time 1789564123.65 total 10.07 last 15.50)
 - mkdir 322 (time 1789564133.68 total 20.10 last 16.55)
 - mkdir 487 (time 1789564143.73 total 30.15 last 16.42)
 - mkdir 650 (time 1789564153.74 total 40.16 last 16.28)
 - mkdir 816 (time 1789564163.81 total 50.23 last 16.49)
 - mkdir 976 (time 1789564173.84 total 60.26 last 15.95)
total: 1000 mkdir in 61.81 seconds: 16.18 ops/second
 - mkdir 152 (time 1789564124.56 total 10.03 last 15.15)
 - mkdir 311 (time 1789564134.61 total 20.08 last 15.82)
 - mkdir 476 (time 1789564144.65 total 30.12 last 16.44)
 - mkdir 637 (time 1789564154.68 total 40.15 last 16.05)
 - mkdir 791 (time 1789564164.73 total 50.20 last 15.33)
 - mkdir 954 (time 1789564174.75 total 60.22 last 16.26)
total: 1000 mkdir in 63.05 seconds: 15.86 ops/second
Time for file creation 72 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 12 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.47 seconds: 21.23 ops/second
 - mkdir 165 (time 1789564688.81 total 10.00 last 16.49)
 - mkdir 312 (time 1789564698.84 total 20.03 last 14.66)
 - mkdir 485 (time 1789564708.89 total 30.08 last 17.22)
 - mkdir 653 (time 1789564718.98 total 40.17 last 16.65)
 - mkdir 802 (time 1789564728.99 total 50.18 last 14.89)
 - mkdir 945 (time 1789564738.99 total 60.18 last 14.30)
total: 1000 mkdir in 63.67 seconds: 15.71 ops/second
 - mkdir 171 (time 1789564688.45 total 10.04 last 17.04)
 - mkdir 321 (time 1789564698.53 total 20.11 last 14.89)
 - mkdir 487 (time 1789564708.54 total 30.12 last 16.59)
 - mkdir 652 (time 1789564718.62 total 40.20 last 16.37)
 - mkdir 797 (time 1789564728.62 total 50.20 last 14.50)
 - mkdir 937 (time 1789564738.65 total 60.23 last 13.96)
total: 1000 mkdir in 64.66 seconds: 15.46 ops/second
 - mkdir 174 (time 1789564689.85 total 10.10 last 17.22)
 - mkdir 317 (time 1789564699.89 total 20.14 last 14.25)
 - mkdir 495 (time 1789564709.92 total 30.17 last 17.75)
 - mkdir 661 (time 1789564720.04 total 40.28 last 16.41)
 - mkdir 810 (time 1789564730.08 total 50.33 last 14.84)
 - mkdir 954 (time 1789564740.12 total 60.37 last 14.34)
total: 1000 mkdir in 63.94 seconds: 15.64 ops/second
 - mkdir 160 (time 1789564689.17 total 10.01 last 15.98)
 - mkdir 306 (time 1789564699.18 total 20.02 last 14.59)
 - mkdir 477 (time 1789564709.28 total 30.12 last 16.94)
 - mkdir 648 (time 1789564719.42 total 40.25 last 16.87)
 - mkdir 798 (time 1789564729.42 total 50.26 last 14.99)
 - mkdir 943 (time 1789564739.42 total 60.26 last 14.50)
total: 1000 mkdir in 64.89 seconds: 15.41 ops/second
 - mkdir 170 (time 1789564689.74 total 10.03 last 16.95)
 - mkdir 310 (time 1789564699.81 total 20.10 last 13.90)
 - mkdir 488 (time 1789564709.83 total 30.12 last 17.77)
 - mkdir 659 (time 1789564719.87 total 40.17 last 17.03)
 - mkdir 805 (time 1789564729.93 total 50.22 last 14.52)
 - mkdir 948 (time 1789564739.98 total 60.28 last 14.22)
total: 1000 mkdir in 64.41 seconds: 15.53 ops/second
 - mkdir 170 (time 1789564689.68 total 10.05 last 16.92)
 - mkdir 312 (time 1789564699.70 total 20.06 last 14.18)
 - mkdir 486 (time 1789564709.71 total 30.08 last 17.37)
 - mkdir 648 (time 1789564719.73 total 40.09 last 16.18)
 - mkdir 793 (time 1789564729.73 total 50.10 last 14.50)
 - mkdir 934 (time 1789564739.73 total 60.10 last 14.10)
total: 1000 mkdir in 65.81 seconds: 15.20 ops/second
 - mkdir 161 (time 1789564689.41 total 10.01 last 16.09)
 - mkdir 303 (time 1789564699.47 total 20.06 last 14.12)
 - mkdir 476 (time 1789564709.47 total 30.06 last 17.30)
 - mkdir 638 (time 1789564719.53 total 40.13 last 16.10)
 - mkdir 788 (time 1789564729.57 total 50.17 last 14.94)
 - mkdir 929 (time 1789564739.63 total 60.23 last 14.02)
total: 1000 mkdir in 66.35 seconds: 15.07 ops/second
 - mkdir 164 (time 1789564689.48 total 10.02 last 16.37)
 - mkdir 303 (time 1789564699.52 total 20.05 last 13.85)
 - mkdir 472 (time 1789564709.57 total 30.11 last 16.81)
 - mkdir 641 (time 1789564719.61 total 40.15 last 16.83)
 - mkdir 785 (time 1789564729.61 total 50.15 last 14.40)
 - mkdir 925 (time 1789564739.63 total 60.16 last 13.98)
total: 1000 mkdir in 67.24 seconds: 14.87 ops/second
 - mkdir 162 (time 1789564689.35 total 10.02 last 16.17)
 - mkdir 304 (time 1789564699.41 total 20.08 last 14.12)
 - mkdir 468 (time 1789564709.45 total 30.11 last 16.35)
 - mkdir 628 (time 1789564719.46 total 40.13 last 15.97)
 - mkdir 775 (time 1789564729.53 total 50.20 last 14.60)
 - mkdir 911 (time 1789564739.54 total 60.20 last 13.59)
total: 1000 mkdir in 68.88 seconds: 14.52 ops/second
 - mkdir 159 (time 1789564688.22 total 10.00 last 15.89)
 - mkdir 298 (time 1789564698.29 total 20.08 last 13.80)
 - mkdir 463 (time 1789564708.33 total 30.12 last 16.44)
 - mkdir 618 (time 1789564718.34 total 40.12 last 15.49)
 - mkdir 751 (time 1789564728.34 total 50.12 last 13.29)
 - mkdir 882 (time 1789564738.37 total 60.15 last 13.06)
 - mkdir 993 (time 1789564748.38 total 70.16 last 11.09)
total: 1000 mkdir in 70.67 seconds: 14.15 ops/second
Time for file creation 77 sec for 1 RPC sent at a time
 - unlinked 0 (time 1789564758 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1789564759 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1789564759 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1789564759 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1789564760 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1789564759 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1789564760 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1789564760 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1789564760 ; total 1 ; last 1)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789564760 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
Time for file removal 43 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8