== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 06:50:52 (1788951052)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.61 seconds: 16.34 ops/second
 - mkdir 193 (time 1788951073.12 total 10.00 last 19.29)
 - mkdir 357 (time 1788951083.12 total 20.01 last 16.39)
 - mkdir 513 (time 1788951093.20 total 30.09 last 15.48)
 - mkdir 675 (time 1788951103.23 total 40.11 last 16.16)
 - mkdir 836 (time 1788951113.24 total 50.13 last 16.07)
 - mkdir 992 (time 1788951123.31 total 60.20 last 15.49)
total: 1000 mkdir in 60.68 seconds: 16.48 ops/second
 - mkdir 191 (time 1788951073.49 total 10.06 last 19.00)
 - mkdir 357 (time 1788951083.53 total 20.09 last 16.55)
 - mkdir 511 (time 1788951093.54 total 30.11 last 15.37)
 - mkdir 671 (time 1788951103.56 total 40.12 last 15.97)
 - mkdir 833 (time 1788951113.61 total 50.17 last 16.12)
 - mkdir 988 (time 1788951123.61 total 60.18 last 15.50)
total: 1000 mkdir in 61.55 seconds: 16.25 ops/second
 - mkdir 189 (time 1788951073.37 total 10.00 last 18.89)
 - mkdir 351 (time 1788951083.44 total 20.07 last 16.09)
 - mkdir 508 (time 1788951093.46 total 30.09 last 15.67)
 - mkdir 664 (time 1788951103.49 total 40.12 last 15.55)
 - mkdir 822 (time 1788951113.56 total 50.19 last 15.70)
 - mkdir 976 (time 1788951123.60 total 60.23 last 15.34)
total: 1000 mkdir in 62.99 seconds: 15.87 ops/second
 - mkdir 184 (time 1788951073.70 total 10.10 last 18.21)
 - mkdir 340 (time 1788951083.70 total 20.11 last 15.59)
 - mkdir 493 (time 1788951093.70 total 30.11 last 15.30)
 - mkdir 649 (time 1788951103.73 total 40.13 last 15.56)
 - mkdir 812 (time 1788951113.73 total 50.14 last 16.29)
 - mkdir 969 (time 1788951123.76 total 60.17 last 15.65)
total: 1000 mkdir in 63.42 seconds: 15.77 ops/second
 - mkdir 188 (time 1788951074.27 total 10.07 last 18.67)
 - mkdir 353 (time 1788951084.31 total 20.11 last 16.43)
 - mkdir 508 (time 1788951094.36 total 30.16 last 15.42)
 - mkdir 662 (time 1788951104.36 total 40.17 last 15.40)
 - mkdir 822 (time 1788951114.42 total 50.22 last 15.91)
 - mkdir 976 (time 1788951124.48 total 60.28 last 15.30)
total: 1000 mkdir in 63.36 seconds: 15.78 ops/second
 - mkdir 185 (time 1788951074.32 total 10.02 last 18.47)
 - mkdir 345 (time 1788951084.44 total 20.13 last 15.81)
 - mkdir 505 (time 1788951094.46 total 30.16 last 15.96)
 - mkdir 663 (time 1788951104.46 total 40.16 last 15.80)
 - mkdir 819 (time 1788951114.50 total 50.20 last 15.54)
 - mkdir 971 (time 1788951124.53 total 60.23 last 15.15)
total: 1000 mkdir in 63.85 seconds: 15.66 ops/second
 - mkdir 179 (time 1788951073.72 total 10.14 last 17.66)
 - mkdir 342 (time 1788951083.73 total 20.14 last 16.29)
 - mkdir 498 (time 1788951093.77 total 30.19 last 15.53)
 - mkdir 655 (time 1788951103.79 total 40.21 last 15.67)
 - mkdir 814 (time 1788951113.85 total 50.26 last 15.81)
 - mkdir 964 (time 1788951123.97 total 60.38 last 14.82)
total: 1000 mkdir in 64.75 seconds: 15.44 ops/second
 - mkdir 186 (time 1788951074.16 total 10.02 last 18.57)
 - mkdir 351 (time 1788951084.20 total 20.06 last 16.43)
 - mkdir 506 (time 1788951094.24 total 30.09 last 15.44)
 - mkdir 659 (time 1788951104.26 total 40.12 last 15.26)
 - mkdir 814 (time 1788951114.28 total 50.14 last 15.47)
 - mkdir 965 (time 1788951124.34 total 60.19 last 15.02)
total: 1000 mkdir in 64.89 seconds: 15.41 ops/second
 - mkdir 176 (time 1788951073.96 total 10.05 last 17.52)
 - mkdir 336 (time 1788951083.97 total 20.05 last 15.98)
 - mkdir 492 (time 1788951093.98 total 30.06 last 15.59)
 - mkdir 643 (time 1788951104.00 total 40.09 last 15.06)
 - mkdir 799 (time 1788951114.02 total 50.10 last 15.58)
 - mkdir 944 (time 1788951124.07 total 60.15 last 14.43)
total: 1000 mkdir in 66.36 seconds: 15.07 ops/second
 - mkdir 176 (time 1788951074.23 total 10.02 last 17.57)
 - mkdir 330 (time 1788951084.27 total 20.06 last 15.34)
 - mkdir 483 (time 1788951094.29 total 30.08 last 15.27)
 - mkdir 630 (time 1788951104.32 total 40.12 last 14.65)
 - mkdir 779 (time 1788951114.38 total 50.18 last 14.81)
 - mkdir 918 (time 1788951124.57 total 60.36 last 13.65)
total: 1000 mkdir in 68.05 seconds: 14.69 ops/second
Time for file creation 78 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 12 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.50 seconds: 20.10 ops/second
 - mkdir 145 (time 1788951693.12 total 10.08 last 14.38)
 - mkdir 326 (time 1788951703.14 total 20.10 last 18.07)
 - mkdir 498 (time 1788951713.18 total 30.14 last 17.13)
 - mkdir 650 (time 1788951723.28 total 40.24 last 15.05)
 - mkdir 804 (time 1788951733.32 total 50.28 last 15.34)
 - mkdir 967 (time 1788951743.44 total 60.40 last 16.11)
total: 1000 mkdir in 62.75 seconds: 15.94 ops/second
 - mkdir 127 (time 1788951691.10 total 10.03 last 12.67)
 - mkdir 302 (time 1788951701.11 total 20.04 last 17.47)
 - mkdir 467 (time 1788951711.15 total 30.07 last 16.45)
 - mkdir 625 (time 1788951721.20 total 40.13 last 15.71)
 - mkdir 766 (time 1788951731.31 total 50.24 last 13.94)
 - mkdir 917 (time 1788951741.34 total 60.27 last 15.05)
total: 1000 mkdir in 67.18 seconds: 14.89 ops/second
 - mkdir 136 (time 1788951693.12 total 10.03 last 13.56)
 - mkdir 312 (time 1788951703.17 total 20.08 last 17.51)
 - mkdir 486 (time 1788951713.21 total 30.12 last 17.33)
 - mkdir 636 (time 1788951723.26 total 40.17 last 14.92)
 - mkdir 784 (time 1788951733.37 total 50.28 last 14.64)
 - mkdir 934 (time 1788951743.39 total 60.30 last 14.97)
total: 1000 mkdir in 66.10 seconds: 15.13 ops/second
 - mkdir 154 (time 1788951694.57 total 10.00 last 15.40)
 - mkdir 327 (time 1788951704.59 total 20.02 last 17.26)
 - mkdir 498 (time 1788951714.59 total 30.02 last 17.10)
 - mkdir 641 (time 1788951724.59 total 40.03 last 14.30)
 - mkdir 797 (time 1788951734.67 total 50.10 last 15.49)
 - mkdir 944 (time 1788951744.67 total 60.10 last 14.69)
total: 1000 mkdir in 65.61 seconds: 15.24 ops/second
 - mkdir 157 (time 1788951694.84 total 10.03 last 15.65)
 - mkdir 335 (time 1788951704.87 total 20.06 last 17.74)
 - mkdir 503 (time 1788951714.91 total 30.10 last 16.74)
 - mkdir 646 (time 1788951724.96 total 40.15 last 14.23)
 - mkdir 800 (time 1788951735.03 total 50.22 last 15.30)
 - mkdir 950 (time 1788951745.06 total 60.25 last 14.95)
total: 1000 mkdir in 65.93 seconds: 15.17 ops/second
 - mkdir 144 (time 1788951694.05 total 10.02 last 14.37)
 - mkdir 327 (time 1788951704.13 total 20.11 last 18.15)
 - mkdir 495 (time 1788951714.17 total 30.14 last 16.74)
 - mkdir 632 (time 1788951724.22 total 40.19 last 13.64)
 - mkdir 784 (time 1788951734.22 total 50.19 last 15.20)
 - mkdir 938 (time 1788951744.29 total 60.26 last 15.29)
total: 1000 mkdir in 66.95 seconds: 14.94 ops/second
 - mkdir 157 (time 1788951694.88 total 10.03 last 15.65)
 - mkdir 326 (time 1788951704.89 total 20.04 last 16.90)
 - mkdir 489 (time 1788951714.92 total 30.07 last 16.25)
 - mkdir 630 (time 1788951724.94 total 40.09 last 14.07)
 - mkdir 785 (time 1788951734.95 total 50.10 last 15.48)
 - mkdir 932 (time 1788951744.98 total 60.13 last 14.66)
total: 1000 mkdir in 67.80 seconds: 14.75 ops/second
 - mkdir 157 (time 1788951695.12 total 10.00 last 15.69)
 - mkdir 330 (time 1788951705.14 total 20.03 last 17.26)
 - mkdir 499 (time 1788951715.15 total 30.04 last 16.88)
 - mkdir 641 (time 1788951725.19 total 40.08 last 14.14)
 - mkdir 791 (time 1788951735.23 total 50.11 last 14.95)
 - mkdir 933 (time 1788951745.26 total 60.15 last 14.15)
total: 1000 mkdir in 68.32 seconds: 14.64 ops/second
 - mkdir 141 (time 1788951694.38 total 10.02 last 14.07)
 - mkdir 313 (time 1788951704.45 total 20.09 last 17.08)
 - mkdir 477 (time 1788951714.47 total 30.11 last 16.37)
 - mkdir 618 (time 1788951724.51 total 40.15 last 14.04)
 - mkdir 768 (time 1788951734.54 total 50.18 last 14.96)
 - mkdir 909 (time 1788951744.64 total 60.28 last 13.96)
total: 1000 mkdir in 69.93 seconds: 14.30 ops/second
 - mkdir 139 (time 1788951694.64 total 10.03 last 13.86)
 - mkdir 299 (time 1788951704.68 total 20.07 last 15.93)
 - mkdir 459 (time 1788951714.74 total 30.13 last 15.90)
 - mkdir 594 (time 1788951724.76 total 40.15 last 13.48)
 - mkdir 734 (time 1788951734.76 total 50.15 last 14.00)
 - mkdir 867 (time 1788951744.77 total 60.16 last 13.29)
 - mkdir 959 (time 1788951754.78 total 70.17 last 9.19)
total: 1000 mkdir in 72.27 seconds: 13.84 ops/second
Time for file creation 84 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788951763 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788951764 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788951764 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788951764 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788951764 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788951764 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788951764 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788951764 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788951764 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788951765 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
Time for file removal 45 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8