== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 09:56:33 (1788616593)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.63 seconds: 15.91 ops/second
 - mkdir 160 (time 1788616615.60 total 10.00 last 16.00)
 - mkdir 309 (time 1788616625.64 total 20.05 last 14.84)
 - mkdir 464 (time 1788616635.70 total 30.11 last 15.40)
 - mkdir 645 (time 1788616645.75 total 40.15 last 18.02)
 - mkdir 812 (time 1788616655.76 total 50.17 last 16.68)
 - mkdir 966 (time 1788616665.79 total 60.19 last 15.36)
total: 1000 mkdir in 62.10 seconds: 16.10 ops/second
 - mkdir 146 (time 1788616617.20 total 10.04 last 14.55)
 - mkdir 293 (time 1788616627.24 total 20.07 last 14.65)
 - mkdir 443 (time 1788616637.25 total 30.08 last 14.99)
 - mkdir 625 (time 1788616647.27 total 40.11 last 18.15)
 - mkdir 800 (time 1788616657.33 total 50.16 last 17.41)
 - mkdir 954 (time 1788616667.35 total 60.19 last 15.36)
total: 1000 mkdir in 65.76 seconds: 15.21 ops/second
 - mkdir 142 (time 1788616616.35 total 10.03 last 14.16)
 - mkdir 290 (time 1788616626.42 total 20.09 last 14.71)
 - mkdir 443 (time 1788616636.45 total 30.12 last 15.25)
 - mkdir 617 (time 1788616646.49 total 40.16 last 17.32)
 - mkdir 788 (time 1788616656.54 total 50.21 last 17.01)
 - mkdir 943 (time 1788616666.55 total 60.23 last 15.48)
total: 1000 mkdir in 66.85 seconds: 14.96 ops/second
 - mkdir 146 (time 1788616617.00 total 10.04 last 14.54)
 - mkdir 292 (time 1788616627.03 total 20.08 last 14.54)
 - mkdir 444 (time 1788616637.09 total 30.13 last 15.12)
 - mkdir 627 (time 1788616647.10 total 40.14 last 18.29)
 - mkdir 794 (time 1788616657.10 total 50.15 last 16.69)
 - mkdir 951 (time 1788616667.15 total 60.19 last 15.63)
total: 1000 mkdir in 66.34 seconds: 15.07 ops/second
 - mkdir 141 (time 1788616617.83 total 10.02 last 14.07)
 - mkdir 295 (time 1788616627.87 total 20.06 last 15.34)
 - mkdir 456 (time 1788616637.92 total 30.11 last 16.02)
 - mkdir 636 (time 1788616647.93 total 40.12 last 17.98)
 - mkdir 805 (time 1788616657.98 total 50.18 last 16.81)
 - mkdir 957 (time 1788616668.06 total 60.25 last 15.08)
total: 1000 mkdir in 65.80 seconds: 15.20 ops/second
 - mkdir 149 (time 1788616617.17 total 10.02 last 14.86)
 - mkdir 296 (time 1788616627.22 total 20.08 last 14.62)
 - mkdir 449 (time 1788616637.27 total 30.13 last 15.23)
 - mkdir 629 (time 1788616647.28 total 40.13 last 17.99)
 - mkdir 799 (time 1788616657.30 total 50.16 last 16.96)
 - mkdir 950 (time 1788616667.30 total 60.16 last 15.09)
total: 1000 mkdir in 66.55 seconds: 15.03 ops/second
 - mkdir 142 (time 1788616617.82 total 10.01 last 14.19)
 - mkdir 294 (time 1788616627.87 total 20.06 last 15.12)
 - mkdir 457 (time 1788616637.89 total 30.09 last 16.26)
 - mkdir 637 (time 1788616647.90 total 40.09 last 17.99)
 - mkdir 806 (time 1788616657.93 total 50.12 last 16.86)
 - mkdir 959 (time 1788616668.00 total 60.20 last 15.18)
total: 1000 mkdir in 65.95 seconds: 15.16 ops/second
 - mkdir 143 (time 1788616617.58 total 10.09 last 14.18)
 - mkdir 293 (time 1788616627.60 total 20.10 last 14.97)
 - mkdir 447 (time 1788616637.63 total 30.14 last 15.34)
 - mkdir 626 (time 1788616647.67 total 40.18 last 17.84)
 - mkdir 794 (time 1788616657.69 total 50.20 last 16.76)
 - mkdir 952 (time 1788616667.72 total 60.22 last 15.76)
total: 1000 mkdir in 66.68 seconds: 15.00 ops/second
 - mkdir 149 (time 1788616617.61 total 10.13 last 14.71)
 - mkdir 298 (time 1788616627.64 total 20.16 last 14.85)
 - mkdir 453 (time 1788616637.65 total 30.17 last 15.49)
 - mkdir 630 (time 1788616647.69 total 40.21 last 17.62)
 - mkdir 798 (time 1788616657.71 total 50.23 last 16.77)
 - mkdir 951 (time 1788616667.75 total 60.27 last 15.23)
total: 1000 mkdir in 66.80 seconds: 14.97 ops/second
 - mkdir 133 (time 1788616616.79 total 10.06 last 13.22)
 - mkdir 272 (time 1788616626.89 total 20.16 last 13.77)
 - mkdir 417 (time 1788616636.91 total 30.18 last 14.46)
 - mkdir 594 (time 1788616646.96 total 40.23 last 17.62)
 - mkdir 764 (time 1788616656.97 total 50.24 last 16.99)
 - mkdir 920 (time 1788616666.98 total 60.25 last 15.58)
total: 1000 mkdir in 68.77 seconds: 14.54 ops/second
Time for file creation 80 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.57 seconds: 17.62 ops/second
 - mkdir 157 (time 1788617222.01 total 10.08 last 15.58)
 - mkdir 305 (time 1788617232.05 total 20.12 last 14.73)
 - mkdir 487 (time 1788617242.09 total 30.16 last 18.13)
 - mkdir 662 (time 1788617252.12 total 40.19 last 17.45)
 - mkdir 815 (time 1788617262.13 total 50.20 last 15.29)
 - mkdir 969 (time 1788617272.13 total 60.20 last 15.39)
total: 1000 mkdir in 62.01 seconds: 16.13 ops/second
 - mkdir 161 (time 1788617221.46 total 10.07 last 15.99)
 - mkdir 297 (time 1788617231.51 total 20.12 last 13.52)
 - mkdir 476 (time 1788617241.52 total 30.13 last 17.89)
 - mkdir 646 (time 1788617251.53 total 40.14 last 16.97)
 - mkdir 806 (time 1788617261.55 total 50.16 last 15.98)
 - mkdir 958 (time 1788617271.55 total 60.16 last 15.20)
total: 1000 mkdir in 62.61 seconds: 15.97 ops/second
 - mkdir 155 (time 1788617221.91 total 10.03 last 15.46)
 - mkdir 299 (time 1788617231.92 total 20.03 last 14.39)
 - mkdir 482 (time 1788617241.93 total 30.05 last 18.27)
 - mkdir 652 (time 1788617251.95 total 40.07 last 16.97)
 - mkdir 805 (time 1788617262.03 total 50.15 last 15.18)
 - mkdir 960 (time 1788617272.08 total 60.19 last 15.43)
total: 1000 mkdir in 62.91 seconds: 15.90 ops/second
 - mkdir 157 (time 1788617222.10 total 10.06 last 15.60)
 - mkdir 304 (time 1788617232.18 total 20.14 last 14.58)
 - mkdir 485 (time 1788617242.24 total 30.20 last 18.00)
 - mkdir 654 (time 1788617252.27 total 40.24 last 16.84)
 - mkdir 808 (time 1788617262.33 total 50.30 last 15.31)
 - mkdir 961 (time 1788617272.36 total 60.33 last 15.26)
total: 1000 mkdir in 63.01 seconds: 15.87 ops/second
 - mkdir 160 (time 1788617222.18 total 10.04 last 15.93)
 - mkdir 302 (time 1788617232.19 total 20.06 last 14.18)
 - mkdir 479 (time 1788617242.24 total 30.11 last 17.61)
 - mkdir 652 (time 1788617252.29 total 40.16 last 17.21)
 - mkdir 805 (time 1788617262.30 total 50.17 last 15.29)
 - mkdir 959 (time 1788617272.33 total 60.20 last 15.34)
total: 1000 mkdir in 63.07 seconds: 15.86 ops/second
 - mkdir 161 (time 1788617222.63 total 10.04 last 16.04)
 - mkdir 304 (time 1788617232.65 total 20.05 last 14.28)
 - mkdir 481 (time 1788617242.69 total 30.10 last 17.62)
 - mkdir 652 (time 1788617252.75 total 40.15 last 17.01)
 - mkdir 806 (time 1788617262.83 total 50.24 last 15.27)
 - mkdir 966 (time 1788617272.89 total 60.29 last 15.91)
total: 1000 mkdir in 62.76 seconds: 15.93 ops/second
 - mkdir 160 (time 1788617223.04 total 10.12 last 15.81)
 - mkdir 312 (time 1788617233.18 total 20.27 last 14.98)
 - mkdir 497 (time 1788617243.24 total 30.33 last 18.39)
 - mkdir 665 (time 1788617253.29 total 40.37 last 16.72)
 - mkdir 816 (time 1788617263.29 total 50.38 last 15.10)
 - mkdir 974 (time 1788617273.33 total 60.42 last 15.73)
total: 1000 mkdir in 62.53 seconds: 15.99 ops/second
 - mkdir 160 (time 1788617222.90 total 10.02 last 15.96)
 - mkdir 308 (time 1788617232.90 total 20.03 last 14.79)
 - mkdir 493 (time 1788617242.93 total 30.06 last 18.44)
 - mkdir 660 (time 1788617252.95 total 40.07 last 16.67)
 - mkdir 811 (time 1788617263.00 total 50.13 last 15.02)
 - mkdir 966 (time 1788617273.03 total 60.16 last 15.45)
total: 1000 mkdir in 62.66 seconds: 15.96 ops/second
 - mkdir 167 (time 1788617223.11 total 10.10 last 16.53)
 - mkdir 320 (time 1788617233.17 total 20.16 last 15.21)
 - mkdir 501 (time 1788617243.17 total 30.17 last 18.09)
 - mkdir 667 (time 1788617253.28 total 40.27 last 16.43)
 - mkdir 816 (time 1788617263.28 total 50.27 last 14.90)
 - mkdir 972 (time 1788617273.29 total 60.28 last 15.59)
total: 1000 mkdir in 62.54 seconds: 15.99 ops/second
 - mkdir 150 (time 1788617222.56 total 10.02 last 14.97)
 - mkdir 289 (time 1788617232.58 total 20.04 last 13.88)
 - mkdir 469 (time 1788617242.60 total 30.06 last 17.96)
 - mkdir 644 (time 1788617252.64 total 40.10 last 17.43)
 - mkdir 793 (time 1788617262.71 total 50.17 last 14.79)
 - mkdir 948 (time 1788617272.72 total 60.18 last 15.50)
total: 1000 mkdir in 64.08 seconds: 15.61 ops/second
Time for file creation 76 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788617284 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788617284 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788617285 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788617286 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788617285 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788617286 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788617286 ; total 1 ; last 1)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788617286 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788617286 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788617286 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
Time for file removal 42 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8