== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 13:48:03 (1788544083)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.67 seconds: 14.97 ops/second
 - mkdir 138 (time 1788544105.42 total 10.00 last 13.79)
 - mkdir 309 (time 1788544115.46 total 20.05 last 17.02)
 - mkdir 470 (time 1788544125.50 total 30.09 last 16.04)
 - mkdir 633 (time 1788544135.50 total 40.09 last 16.29)
 - mkdir 833 (time 1788544145.54 total 50.13 last 19.93)
 - mkdir 999 (time 1788544155.56 total 60.14 last 16.58)
total: 1000 mkdir in 60.14 seconds: 16.63 ops/second
 - mkdir 133 (time 1788544106.66 total 10.04 last 13.25)
 - mkdir 301 (time 1788544116.68 total 20.06 last 16.77)
 - mkdir 461 (time 1788544126.73 total 30.11 last 15.92)
 - mkdir 626 (time 1788544136.77 total 40.15 last 16.43)
 - mkdir 828 (time 1788544146.78 total 50.15 last 20.19)
 - mkdir 986 (time 1788544156.80 total 60.18 last 15.76)
total: 1000 mkdir in 61.19 seconds: 16.34 ops/second
 - mkdir 129 (time 1788544106.53 total 10.00 last 12.90)
 - mkdir 294 (time 1788544116.60 total 20.08 last 16.38)
 - mkdir 453 (time 1788544126.65 total 30.12 last 15.83)
 - mkdir 617 (time 1788544136.66 total 40.13 last 16.38)
 - mkdir 816 (time 1788544146.72 total 50.20 last 19.78)
 - mkdir 977 (time 1788544156.75 total 60.23 last 16.05)
total: 1000 mkdir in 62.39 seconds: 16.03 ops/second
 - mkdir 150 (time 1788544108.98 total 10.02 last 14.97)
 - mkdir 314 (time 1788544119.02 total 20.06 last 16.34)
 - mkdir 494 (time 1788544129.06 total 30.10 last 17.92)
 - mkdir 663 (time 1788544139.09 total 40.13 last 16.85)
 - mkdir 864 (time 1788544149.09 total 50.14 last 20.09)
 - mkdir 996 (time 1788544159.19 total 60.23 last 13.07)
total: 1000 mkdir in 60.78 seconds: 16.45 ops/second
 - mkdir 135 (time 1788544107.92 total 10.02 last 13.48)
 - mkdir 303 (time 1788544117.95 total 20.05 last 16.75)
 - mkdir 468 (time 1788544127.96 total 30.05 last 16.49)
 - mkdir 638 (time 1788544137.99 total 40.08 last 16.95)
 - mkdir 832 (time 1788544148.02 total 50.11 last 19.34)
 - mkdir 979 (time 1788544158.06 total 60.15 last 14.64)
total: 1000 mkdir in 63.46 seconds: 15.76 ops/second
 - mkdir 145 (time 1788544108.87 total 10.02 last 14.47)
 - mkdir 309 (time 1788544118.94 total 20.09 last 16.29)
 - mkdir 481 (time 1788544128.95 total 30.10 last 17.17)
 - mkdir 647 (time 1788544138.99 total 40.14 last 16.54)
 - mkdir 844 (time 1788544149.01 total 50.16 last 19.65)
 - mkdir 981 (time 1788544159.05 total 60.20 last 13.66)
total: 1000 mkdir in 62.87 seconds: 15.91 ops/second
 - mkdir 140 (time 1788544108.64 total 10.03 last 13.96)
 - mkdir 307 (time 1788544118.70 total 20.09 last 16.59)
 - mkdir 477 (time 1788544128.71 total 30.10 last 17.00)
 - mkdir 642 (time 1788544138.77 total 40.16 last 16.39)
 - mkdir 837 (time 1788544148.79 total 50.18 last 19.46)
 - mkdir 975 (time 1788544158.82 total 60.21 last 13.76)
total: 1000 mkdir in 63.56 seconds: 15.73 ops/second
 - mkdir 137 (time 1788544108.65 total 10.00 last 13.70)
 - mkdir 296 (time 1788544118.71 total 20.06 last 15.81)
 - mkdir 467 (time 1788544128.73 total 30.08 last 17.07)
 - mkdir 636 (time 1788544138.77 total 40.12 last 16.83)
 - mkdir 830 (time 1788544148.84 total 50.19 last 19.27)
 - mkdir 974 (time 1788544159.01 total 60.36 last 14.16)
total: 1000 mkdir in 64.11 seconds: 15.60 ops/second
 - mkdir 147 (time 1788544109.01 total 10.03 last 14.66)
 - mkdir 302 (time 1788544119.03 total 20.04 last 15.48)
 - mkdir 470 (time 1788544129.04 total 30.06 last 16.77)
 - mkdir 639 (time 1788544139.09 total 40.10 last 16.83)
 - mkdir 839 (time 1788544149.11 total 50.13 last 19.95)
 - mkdir 972 (time 1788544159.18 total 60.19 last 13.21)
total: 1000 mkdir in 64.12 seconds: 15.60 ops/second
 - mkdir 132 (time 1788544107.87 total 10.01 last 13.18)
 - mkdir 289 (time 1788544117.89 total 20.03 last 15.67)
 - mkdir 445 (time 1788544127.95 total 30.10 last 15.50)
 - mkdir 614 (time 1788544138.01 total 40.16 last 16.80)
 - mkdir 808 (time 1788544148.01 total 50.16 last 19.40)
 - mkdir 957 (time 1788544158.04 total 60.19 last 14.86)
total: 1000 mkdir in 65.75 seconds: 15.21 ops/second
Time for file creation 78 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 17 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.59 seconds: 16.84 ops/second
 - mkdir 131 (time 1788544674.15 total 10.04 last 13.05)
 - mkdir 297 (time 1788544684.18 total 20.07 last 16.55)
 - mkdir 488 (time 1788544694.19 total 30.08 last 19.08)
 - mkdir 677 (time 1788544704.22 total 40.11 last 18.83)
 - mkdir 821 (time 1788544714.27 total 50.16 last 14.34)
 - mkdir 992 (time 1788544724.30 total 60.19 last 17.04)
total: 1000 mkdir in 60.77 seconds: 16.46 ops/second
 - mkdir 140 (time 1788544674.92 total 10.01 last 13.99)
 - mkdir 308 (time 1788544684.95 total 20.04 last 16.75)
 - mkdir 497 (time 1788544695.00 total 30.09 last 18.80)
 - mkdir 679 (time 1788544705.03 total 40.11 last 18.16)
 - mkdir 825 (time 1788544715.08 total 50.17 last 14.53)
 - mkdir 992 (time 1788544725.11 total 60.20 last 16.65)
total: 1000 mkdir in 61.30 seconds: 16.31 ops/second
 - mkdir 123 (time 1788544673.39 total 10.01 last 12.29)
 - mkdir 288 (time 1788544683.40 total 20.02 last 16.49)
 - mkdir 478 (time 1788544693.41 total 30.02 last 18.99)
 - mkdir 659 (time 1788544703.42 total 40.04 last 18.08)
 - mkdir 799 (time 1788544713.45 total 50.06 last 13.96)
 - mkdir 964 (time 1788544723.48 total 60.10 last 16.44)
total: 1000 mkdir in 63.37 seconds: 15.78 ops/second
 - mkdir 127 (time 1788544673.53 total 10.05 last 12.64)
 - mkdir 288 (time 1788544683.54 total 20.06 last 16.08)
 - mkdir 475 (time 1788544693.60 total 30.12 last 18.59)
 - mkdir 653 (time 1788544703.73 total 40.25 last 17.57)
 - mkdir 796 (time 1788544713.83 total 50.35 last 14.16)
 - mkdir 963 (time 1788544723.84 total 60.36 last 16.69)
total: 1000 mkdir in 64.87 seconds: 15.41 ops/second
 - mkdir 122 (time 1788544674.18 total 10.02 last 12.17)
 - mkdir 284 (time 1788544684.25 total 20.10 last 16.08)
 - mkdir 470 (time 1788544694.28 total 30.12 last 18.55)
 - mkdir 646 (time 1788544704.36 total 40.20 last 17.46)
 - mkdir 790 (time 1788544714.39 total 50.23 last 14.36)
 - mkdir 956 (time 1788544724.40 total 60.25 last 16.57)
total: 1000 mkdir in 65.71 seconds: 15.22 ops/second
 - mkdir 121 (time 1788544672.43 total 10.03 last 12.06)
 - mkdir 274 (time 1788544682.45 total 20.05 last 15.28)
 - mkdir 460 (time 1788544692.49 total 30.09 last 18.51)
 - mkdir 633 (time 1788544702.59 total 40.19 last 17.14)
 - mkdir 765 (time 1788544712.61 total 50.21 last 13.18)
 - mkdir 930 (time 1788544722.67 total 60.27 last 16.39)
total: 1000 mkdir in 67.88 seconds: 14.73 ops/second
 - mkdir 129 (time 1788544674.23 total 10.03 last 12.86)
 - mkdir 293 (time 1788544684.26 total 20.07 last 16.34)
 - mkdir 476 (time 1788544694.32 total 30.13 last 18.19)
 - mkdir 650 (time 1788544704.40 total 40.20 last 17.27)
 - mkdir 790 (time 1788544714.40 total 50.21 last 14.00)
 - mkdir 955 (time 1788544724.42 total 60.23 last 16.47)
total: 1000 mkdir in 66.35 seconds: 15.07 ops/second
 - mkdir 122 (time 1788544674.13 total 10.03 last 12.17)
 - mkdir 286 (time 1788544684.17 total 20.06 last 16.34)
 - mkdir 472 (time 1788544694.23 total 30.12 last 18.49)
 - mkdir 651 (time 1788544704.30 total 40.20 last 17.77)
 - mkdir 790 (time 1788544714.37 total 50.26 last 13.81)
 - mkdir 949 (time 1788544724.43 total 60.32 last 15.81)
total: 1000 mkdir in 66.93 seconds: 14.94 ops/second
 - mkdir 127 (time 1788544674.74 total 10.07 last 12.62)
 - mkdir 285 (time 1788544684.76 total 20.09 last 15.76)
 - mkdir 463 (time 1788544694.79 total 30.13 last 17.74)
 - mkdir 640 (time 1788544704.80 total 40.13 last 17.69)
 - mkdir 780 (time 1788544714.84 total 50.18 last 13.94)
 - mkdir 945 (time 1788544724.86 total 60.19 last 16.47)
total: 1000 mkdir in 67.08 seconds: 14.91 ops/second
 - mkdir 114 (time 1788544673.19 total 10.11 last 11.27)
 - mkdir 262 (time 1788544683.21 total 20.13 last 14.78)
 - mkdir 438 (time 1788544693.23 total 30.15 last 17.57)
 - mkdir 608 (time 1788544703.27 total 40.19 last 16.93)
 - mkdir 744 (time 1788544713.27 total 50.19 last 13.59)
 - mkdir 899 (time 1788544723.30 total 60.22 last 15.46)
 - mkdir 995 (time 1788544733.33 total 70.25 last 9.57)
total: 1000 mkdir in 70.56 seconds: 14.17 ops/second
Time for file creation 81 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788544742 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788544743 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788544743 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788544743 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788544743 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788544744 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788544744 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788544744 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788544744 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788544744 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
Time for file removal 48 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8