== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 07:32:54 (1788607974)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.64 seconds: 15.70 ops/second
 - mkdir 141 (time 1788607995.84 total 10.04 last 14.04)
 - mkdir 299 (time 1788608005.88 total 20.08 last 15.75)
 - mkdir 426 (time 1788608015.90 total 30.10 last 12.68)
 - mkdir 550 (time 1788608025.90 total 40.10 last 12.40)
 - mkdir 704 (time 1788608035.90 total 50.10 last 15.39)
 - mkdir 866 (time 1788608045.93 total 60.13 last 16.16)
total: 1000 mkdir in 66.85 seconds: 14.96 ops/second
 - mkdir 153 (time 1788607997.59 total 10.03 last 15.25)
 - mkdir 318 (time 1788608007.64 total 20.07 last 16.43)
 - mkdir 451 (time 1788608017.81 total 30.25 last 13.07)
 - mkdir 578 (time 1788608027.85 total 40.29 last 12.65)
 - mkdir 734 (time 1788608037.91 total 50.35 last 15.51)
 - mkdir 900 (time 1788608047.96 total 60.40 last 16.51)
total: 1000 mkdir in 65.62 seconds: 15.24 ops/second
 - mkdir 146 (time 1788607997.15 total 10.07 last 14.50)
 - mkdir 311 (time 1788608007.17 total 20.10 last 16.46)
 - mkdir 437 (time 1788608017.19 total 30.12 last 12.57)
 - mkdir 560 (time 1788608027.21 total 40.13 last 12.28)
 - mkdir 718 (time 1788608037.28 total 50.21 last 15.68)
 - mkdir 882 (time 1788608047.32 total 60.24 last 16.34)
total: 1000 mkdir in 66.80 seconds: 14.97 ops/second
 - mkdir 139 (time 1788607996.15 total 10.05 last 13.83)
 - mkdir 296 (time 1788608006.21 total 20.11 last 15.60)
 - mkdir 432 (time 1788608016.25 total 30.15 last 13.54)
 - mkdir 552 (time 1788608026.26 total 40.16 last 11.99)
 - mkdir 699 (time 1788608036.34 total 50.24 last 14.59)
 - mkdir 863 (time 1788608046.39 total 60.29 last 16.32)
total: 1000 mkdir in 69.07 seconds: 14.48 ops/second
 - mkdir 145 (time 1788607998.28 total 10.19 last 14.23)
 - mkdir 307 (time 1788608008.30 total 20.20 last 16.18)
 - mkdir 431 (time 1788608018.30 total 30.20 last 12.40)
 - mkdir 562 (time 1788608028.31 total 40.22 last 13.08)
 - mkdir 717 (time 1788608038.35 total 50.25 last 15.45)
 - mkdir 892 (time 1788608048.39 total 60.29 last 17.43)
total: 1000 mkdir in 67.24 seconds: 14.87 ops/second
 - mkdir 146 (time 1788607997.53 total 10.02 last 14.58)
 - mkdir 305 (time 1788608007.57 total 20.05 last 15.84)
 - mkdir 429 (time 1788608017.62 total 30.10 last 12.34)
 - mkdir 551 (time 1788608027.63 total 40.12 last 12.18)
 - mkdir 706 (time 1788608037.65 total 50.14 last 15.47)
 - mkdir 876 (time 1788608047.66 total 60.14 last 17.00)
total: 1000 mkdir in 68.05 seconds: 14.69 ops/second
 - mkdir 144 (time 1788607996.81 total 10.03 last 14.35)
 - mkdir 302 (time 1788608006.82 total 20.04 last 15.78)
 - mkdir 431 (time 1788608016.85 total 30.08 last 12.86)
 - mkdir 548 (time 1788608026.88 total 40.11 last 11.67)
 - mkdir 701 (time 1788608036.92 total 50.15 last 15.24)
 - mkdir 863 (time 1788608046.93 total 60.15 last 16.19)
total: 1000 mkdir in 68.79 seconds: 14.54 ops/second
 - mkdir 148 (time 1788607998.05 total 10.07 last 14.70)
 - mkdir 311 (time 1788608008.08 total 20.10 last 16.25)
 - mkdir 431 (time 1788608018.18 total 30.20 last 11.88)
 - mkdir 562 (time 1788608028.23 total 40.25 last 13.04)
 - mkdir 721 (time 1788608038.30 total 50.32 last 15.78)
 - mkdir 890 (time 1788608048.32 total 60.34 last 16.86)
total: 1000 mkdir in 67.61 seconds: 14.79 ops/second
 - mkdir 147 (time 1788607997.86 total 10.03 last 14.66)
 - mkdir 312 (time 1788608007.95 total 20.11 last 16.36)
 - mkdir 434 (time 1788608018.00 total 30.16 last 12.14)
 - mkdir 563 (time 1788608028.00 total 40.17 last 12.89)
 - mkdir 717 (time 1788608038.02 total 50.19 last 15.38)
 - mkdir 883 (time 1788608048.04 total 60.20 last 16.57)
total: 1000 mkdir in 67.95 seconds: 14.72 ops/second
 - mkdir 135 (time 1788607997.50 total 10.04 last 13.45)
 - mkdir 283 (time 1788608007.56 total 20.09 last 14.72)
 - mkdir 403 (time 1788608017.60 total 30.14 last 11.95)
 - mkdir 524 (time 1788608027.61 total 40.15 last 12.09)
 - mkdir 678 (time 1788608037.63 total 50.16 last 15.38)
 - mkdir 841 (time 1788608047.65 total 60.18 last 16.27)
 - mkdir 981 (time 1788608057.68 total 70.22 last 13.95)
total: 1000 mkdir in 71.60 seconds: 13.97 ops/second
Time for file creation 82 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 15 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.46 seconds: 21.84 ops/second
 - mkdir 157 (time 1788608570.38 total 10.00 last 15.69)
 - mkdir 319 (time 1788608580.38 total 20.01 last 16.19)
 - mkdir 478 (time 1788608590.44 total 30.07 last 15.80)
 - mkdir 633 (time 1788608600.45 total 40.08 last 15.49)
 - mkdir 815 (time 1788608610.45 total 50.08 last 18.19)
 - mkdir 973 (time 1788608620.51 total 60.14 last 15.71)
total: 1000 mkdir in 62.21 seconds: 16.07 ops/second
 - mkdir 162 (time 1788608571.67 total 10.03 last 16.15)
 - mkdir 326 (time 1788608581.70 total 20.06 last 16.34)
 - mkdir 483 (time 1788608591.74 total 30.09 last 15.65)
 - mkdir 646 (time 1788608601.76 total 40.12 last 16.27)
 - mkdir 819 (time 1788608611.77 total 50.13 last 17.27)
 - mkdir 970 (time 1788608621.79 total 60.15 last 15.07)
total: 1000 mkdir in 63.58 seconds: 15.73 ops/second
 - mkdir 153 (time 1788608571.77 total 10.03 last 15.26)
 - mkdir 314 (time 1788608581.79 total 20.05 last 16.06)
 - mkdir 463 (time 1788608591.80 total 30.06 last 14.89)
 - mkdir 625 (time 1788608601.91 total 40.17 last 16.02)
 - mkdir 808 (time 1788608611.94 total 50.20 last 18.25)
 - mkdir 959 (time 1788608622.02 total 60.28 last 14.98)
total: 1000 mkdir in 65.36 seconds: 15.30 ops/second
 - mkdir 159 (time 1788608572.53 total 10.06 last 15.81)
 - mkdir 320 (time 1788608582.59 total 20.12 last 16.01)
 - mkdir 469 (time 1788608592.60 total 30.13 last 14.88)
 - mkdir 635 (time 1788608602.62 total 40.15 last 16.58)
 - mkdir 812 (time 1788608612.62 total 50.15 last 17.69)
 - mkdir 968 (time 1788608622.66 total 60.19 last 15.54)
total: 1000 mkdir in 64.77 seconds: 15.44 ops/second
 - mkdir 163 (time 1788608572.41 total 10.03 last 16.25)
 - mkdir 320 (time 1788608582.42 total 20.04 last 15.68)
 - mkdir 470 (time 1788608592.47 total 30.09 last 14.92)
 - mkdir 633 (time 1788608602.48 total 40.10 last 16.29)
 - mkdir 814 (time 1788608612.52 total 50.14 last 18.03)
 - mkdir 964 (time 1788608622.55 total 60.17 last 14.96)
total: 1000 mkdir in 65.08 seconds: 15.37 ops/second
 - mkdir 151 (time 1788608570.84 total 10.05 last 15.02)
 - mkdir 301 (time 1788608580.88 total 20.09 last 14.95)
 - mkdir 449 (time 1788608590.93 total 30.14 last 14.73)
 - mkdir 607 (time 1788608600.99 total 40.20 last 15.70)
 - mkdir 787 (time 1788608611.03 total 50.24 last 17.92)
 - mkdir 948 (time 1788608621.07 total 60.27 last 16.05)
total: 1000 mkdir in 66.78 seconds: 14.98 ops/second
 - mkdir 153 (time 1788608571.87 total 10.05 last 15.22)
 - mkdir 312 (time 1788608581.91 total 20.09 last 15.83)
 - mkdir 461 (time 1788608591.98 total 30.16 last 14.80)
 - mkdir 625 (time 1788608602.03 total 40.21 last 16.32)
 - mkdir 802 (time 1788608612.05 total 50.24 last 17.65)
 - mkdir 955 (time 1788608622.12 total 60.30 last 15.21)
total: 1000 mkdir in 66.27 seconds: 15.09 ops/second
 - mkdir 156 (time 1788608572.51 total 10.08 last 15.47)
 - mkdir 317 (time 1788608582.58 total 20.15 last 15.99)
 - mkdir 464 (time 1788608592.59 total 30.17 last 14.68)
 - mkdir 628 (time 1788608602.65 total 40.22 last 16.31)
 - mkdir 808 (time 1788608612.69 total 50.27 last 17.92)
 - mkdir 957 (time 1788608622.71 total 60.28 last 14.88)
total: 1000 mkdir in 66.33 seconds: 15.08 ops/second
 - mkdir 155 (time 1788608572.71 total 10.12 last 15.31)
 - mkdir 316 (time 1788608582.73 total 20.14 last 16.07)
 - mkdir 473 (time 1788608592.74 total 30.16 last 15.67)
 - mkdir 636 (time 1788608602.75 total 40.17 last 16.29)
 - mkdir 812 (time 1788608612.77 total 50.19 last 17.56)
 - mkdir 958 (time 1788608622.84 total 60.25 last 14.51)
total: 1000 mkdir in 66.44 seconds: 15.05 ops/second
 - mkdir 159 (time 1788608572.27 total 10.02 last 15.87)
 - mkdir 314 (time 1788608582.32 total 20.07 last 15.42)
 - mkdir 458 (time 1788608592.38 total 30.13 last 14.31)
 - mkdir 621 (time 1788608602.43 total 40.17 last 16.23)
 - mkdir 802 (time 1788608612.48 total 50.22 last 18.01)
 - mkdir 950 (time 1788608622.53 total 60.28 last 14.72)
total: 1000 mkdir in 66.88 seconds: 14.95 ops/second
Time for file creation 80 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788608640 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788608640 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788608641 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788608641 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788608641 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788608642 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788608642 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788608642 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788608642 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788608642 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
Time for file removal 46 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8