== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 18:23:43 (1788819823)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.63 seconds: 15.94 ops/second
 - mkdir 199 (time 1788819847.61 total 10.02 last 19.86)
 - mkdir 384 (time 1788819857.69 total 20.10 last 18.36)
 - mkdir 534 (time 1788819867.70 total 30.10 last 14.99)
 - mkdir 684 (time 1788819877.74 total 40.15 last 14.93)
 - mkdir 813 (time 1788819887.83 total 50.23 last 12.80)
 - mkdir 942 (time 1788819897.91 total 60.32 last 12.79)
total: 1000 mkdir in 64.96 seconds: 15.39 ops/second
 - mkdir 188 (time 1788819846.90 total 10.05 last 18.70)
 - mkdir 360 (time 1788819856.93 total 20.08 last 17.15)
 - mkdir 511 (time 1788819866.94 total 30.09 last 15.08)
 - mkdir 654 (time 1788819876.95 total 40.10 last 14.28)
 - mkdir 779 (time 1788819887.01 total 50.16 last 12.42)
 - mkdir 902 (time 1788819897.09 total 60.24 last 12.21)
total: 1000 mkdir in 69.50 seconds: 14.39 ops/second
 - mkdir 177 (time 1788819845.52 total 10.03 last 17.65)
 - mkdir 350 (time 1788819855.58 total 20.09 last 17.20)
 - mkdir 500 (time 1788819865.60 total 30.11 last 14.97)
 - mkdir 646 (time 1788819875.69 total 40.20 last 14.47)
 - mkdir 773 (time 1788819885.71 total 50.22 last 12.67)
 - mkdir 883 (time 1788819895.72 total 60.23 last 10.99)
 - mkdir 990 (time 1788819905.76 total 70.27 last 10.66)
total: 1000 mkdir in 71.94 seconds: 13.90 ops/second
 - mkdir 189 (time 1788819848.23 total 10.06 last 18.78)
 - mkdir 364 (time 1788819858.29 total 20.12 last 17.40)
 - mkdir 514 (time 1788819868.35 total 30.19 last 14.90)
 - mkdir 659 (time 1788819878.45 total 40.28 last 14.36)
 - mkdir 778 (time 1788819888.51 total 50.35 last 11.82)
 - mkdir 904 (time 1788819898.54 total 60.38 last 12.57)
 - mkdir 988 (time 1788819908.54 total 70.38 last 8.40)
total: 1000 mkdir in 72.13 seconds: 13.86 ops/second
 - mkdir 193 (time 1788819847.87 total 10.07 last 19.17)
 - mkdir 366 (time 1788819857.93 total 20.13 last 17.20)
 - mkdir 511 (time 1788819867.99 total 30.19 last 14.40)
 - mkdir 653 (time 1788819878.00 total 40.20 last 14.19)
 - mkdir 770 (time 1788819888.01 total 50.21 last 11.69)
 - mkdir 890 (time 1788819898.08 total 60.28 last 11.92)
 - mkdir 982 (time 1788819908.11 total 70.31 last 9.17)
total: 1000 mkdir in 73.18 seconds: 13.67 ops/second
 - mkdir 185 (time 1788819847.39 total 10.02 last 18.45)
 - mkdir 358 (time 1788819857.47 total 20.10 last 17.17)
 - mkdir 506 (time 1788819867.49 total 30.12 last 14.77)
 - mkdir 646 (time 1788819877.52 total 40.15 last 13.95)
 - mkdir 770 (time 1788819887.58 total 50.21 last 12.33)
 - mkdir 888 (time 1788819897.60 total 60.23 last 11.78)
 - mkdir 979 (time 1788819907.63 total 70.26 last 9.07)
total: 1000 mkdir in 73.75 seconds: 13.56 ops/second
 - mkdir 188 (time 1788819847.97 total 10.02 last 18.77)
 - mkdir 361 (time 1788819858.00 total 20.04 last 17.25)
 - mkdir 506 (time 1788819868.03 total 30.07 last 14.46)
 - mkdir 653 (time 1788819878.04 total 40.08 last 14.68)
 - mkdir 771 (time 1788819888.07 total 50.12 last 11.76)
 - mkdir 888 (time 1788819898.11 total 60.16 last 11.65)
 - mkdir 978 (time 1788819908.30 total 70.34 last 8.84)
total: 1000 mkdir in 73.92 seconds: 13.53 ops/second
 - mkdir 188 (time 1788819847.98 total 10.06 last 18.69)
 - mkdir 368 (time 1788819857.99 total 20.07 last 17.98)
 - mkdir 511 (time 1788819868.02 total 30.10 last 14.26)
 - mkdir 655 (time 1788819878.03 total 40.10 last 14.39)
 - mkdir 771 (time 1788819888.09 total 50.16 last 11.53)
 - mkdir 889 (time 1788819898.23 total 60.30 last 11.64)
 - mkdir 975 (time 1788819908.45 total 70.52 last 8.42)
total: 1000 mkdir in 74.63 seconds: 13.40 ops/second
 - mkdir 181 (time 1788819847.53 total 10.06 last 18.00)
 - mkdir 349 (time 1788819857.60 total 20.13 last 16.68)
 - mkdir 499 (time 1788819867.61 total 30.14 last 14.99)
 - mkdir 638 (time 1788819877.62 total 40.15 last 13.89)
 - mkdir 758 (time 1788819887.72 total 50.25 last 11.88)
 - mkdir 869 (time 1788819897.73 total 60.25 last 11.10)
 - mkdir 961 (time 1788819907.81 total 70.34 last 9.12)
total: 1000 mkdir in 75.90 seconds: 13.17 ops/second
 - mkdir 178 (time 1788819847.01 total 10.03 last 17.75)
 - mkdir 340 (time 1788819857.03 total 20.05 last 16.16)
 - mkdir 483 (time 1788819867.06 total 30.08 last 14.26)
 - mkdir 617 (time 1788819877.08 total 40.10 last 13.38)
 - mkdir 736 (time 1788819887.18 total 50.20 last 11.78)
 - mkdir 849 (time 1788819897.27 total 60.29 last 11.20)
 - mkdir 944 (time 1788819907.42 total 70.44 last 9.36)
total: 1000 mkdir in 77.93 seconds: 12.83 ops/second
Time for file creation 90 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 17 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.38 seconds: 26.06 ops/second
 - mkdir 202 (time 1788820482.33 total 10.03 last 20.15)
 - mkdir 378 (time 1788820492.34 total 20.03 last 17.58)
 - mkdir 554 (time 1788820502.43 total 30.12 last 17.44)
 - mkdir 709 (time 1788820512.43 total 40.13 last 15.50)
 - mkdir 873 (time 1788820522.43 total 50.13 last 16.40)
total: 1000 mkdir in 58.03 seconds: 17.23 ops/second
 - mkdir 210 (time 1788820483.19 total 10.01 last 20.97)
 - mkdir 399 (time 1788820493.21 total 20.03 last 18.87)
 - mkdir 566 (time 1788820503.27 total 30.09 last 16.59)
 - mkdir 728 (time 1788820513.28 total 40.11 last 16.18)
 - mkdir 885 (time 1788820523.33 total 50.15 last 15.63)
total: 1000 mkdir in 57.72 seconds: 17.33 ops/second
 - mkdir 195 (time 1788820482.46 total 10.03 last 19.45)
 - mkdir 373 (time 1788820492.47 total 20.04 last 17.78)
 - mkdir 546 (time 1788820502.48 total 30.05 last 17.27)
 - mkdir 705 (time 1788820512.51 total 40.08 last 15.86)
 - mkdir 864 (time 1788820522.54 total 50.11 last 15.85)
total: 1000 mkdir in 59.34 seconds: 16.85 ops/second
 - mkdir 209 (time 1788820483.50 total 10.00 last 20.90)
 - mkdir 396 (time 1788820493.57 total 20.07 last 18.57)
 - mkdir 551 (time 1788820503.57 total 30.07 last 15.50)
 - mkdir 720 (time 1788820513.59 total 40.08 last 16.88)
 - mkdir 873 (time 1788820523.61 total 50.11 last 15.26)
total: 1000 mkdir in 59.01 seconds: 16.95 ops/second
 - mkdir 216 (time 1788820484.20 total 10.04 last 21.51)
 - mkdir 404 (time 1788820494.28 total 20.12 last 18.66)
 - mkdir 567 (time 1788820504.34 total 30.18 last 16.20)
 - mkdir 729 (time 1788820514.34 total 40.19 last 16.19)
 - mkdir 877 (time 1788820524.39 total 50.23 last 14.73)
total: 1000 mkdir in 59.80 seconds: 16.72 ops/second
 - mkdir 215 (time 1788820484.08 total 10.02 last 21.46)
 - mkdir 398 (time 1788820494.09 total 20.03 last 18.28)
 - mkdir 558 (time 1788820504.13 total 30.07 last 15.93)
 - mkdir 725 (time 1788820514.14 total 40.08 last 16.70)
 - mkdir 876 (time 1788820524.18 total 50.12 last 15.03)
total: 1000 mkdir in 60.04 seconds: 16.66 ops/second
 - mkdir 211 (time 1788820484.35 total 10.04 last 21.02)
 - mkdir 399 (time 1788820494.38 total 20.07 last 18.74)
 - mkdir 555 (time 1788820504.42 total 30.11 last 15.53)
 - mkdir 718 (time 1788820514.43 total 40.13 last 16.28)
 - mkdir 868 (time 1788820524.45 total 50.14 last 14.98)
 - mkdir 995 (time 1788820534.62 total 60.32 last 12.48)
total: 1000 mkdir in 60.88 seconds: 16.43 ops/second
 - mkdir 213 (time 1788820484.54 total 10.02 last 21.25)
 - mkdir 398 (time 1788820494.63 total 20.11 last 18.33)
 - mkdir 556 (time 1788820504.65 total 30.13 last 15.77)
 - mkdir 724 (time 1788820514.71 total 40.20 last 16.69)
 - mkdir 875 (time 1788820524.71 total 50.20 last 15.10)
 - mkdir 996 (time 1788820534.82 total 60.30 last 11.97)
total: 1000 mkdir in 60.77 seconds: 16.46 ops/second
 - mkdir 207 (time 1788820484.54 total 10.04 last 20.63)
 - mkdir 390 (time 1788820494.60 total 20.09 last 18.20)
 - mkdir 549 (time 1788820504.65 total 30.15 last 15.82)
 - mkdir 719 (time 1788820514.65 total 40.15 last 17.00)
 - mkdir 870 (time 1788820524.70 total 50.19 last 15.03)
 - mkdir 993 (time 1788820534.72 total 60.21 last 12.28)
total: 1000 mkdir in 61.00 seconds: 16.39 ops/second
 - mkdir 201 (time 1788820483.64 total 10.05 last 19.99)
 - mkdir 378 (time 1788820493.68 total 20.09 last 17.63)
 - mkdir 526 (time 1788820503.75 total 30.16 last 14.69)
 - mkdir 693 (time 1788820513.80 total 40.21 last 16.63)
 - mkdir 842 (time 1788820523.85 total 50.26 last 14.82)
 - mkdir 971 (time 1788820533.93 total 60.34 last 12.81)
total: 1000 mkdir in 63.27 seconds: 15.81 ops/second
Time for file creation 74 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788820545 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788820546 ; total 1 ; last 1)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788820546 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788820546 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788820547 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788820547 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788820547 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788820547 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788820547 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788820547 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
Time for file removal 46 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8