== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 07:40:00 (1788867600)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.67 seconds: 14.85 ops/second
 - mkdir 163 (time 1788867620.67 total 10.00 last 16.30)
 - mkdir 364 (time 1788867630.68 total 20.00 last 20.10)
 - mkdir 522 (time 1788867640.69 total 30.02 last 15.77)
 - mkdir 690 (time 1788867650.71 total 40.04 last 16.76)
 - mkdir 887 (time 1788867660.75 total 50.08 last 19.62)
total: 1000 mkdir in 58.09 seconds: 17.21 ops/second
 - mkdir 165 (time 1788867621.18 total 10.04 last 16.44)
 - mkdir 363 (time 1788867631.23 total 20.09 last 19.71)
 - mkdir 521 (time 1788867641.27 total 30.13 last 15.73)
 - mkdir 688 (time 1788867651.29 total 40.14 last 16.68)
 - mkdir 881 (time 1788867661.32 total 50.18 last 19.24)
total: 1000 mkdir in 59.22 seconds: 16.89 ops/second
 - mkdir 166 (time 1788867622.16 total 10.02 last 16.57)
 - mkdir 375 (time 1788867632.19 total 20.04 last 20.85)
 - mkdir 536 (time 1788867642.20 total 30.05 last 16.08)
 - mkdir 706 (time 1788867652.23 total 40.09 last 16.94)
 - mkdir 892 (time 1788867662.27 total 50.12 last 18.53)
total: 1000 mkdir in 58.23 seconds: 17.17 ops/second
 - mkdir 163 (time 1788867621.97 total 10.02 last 16.27)
 - mkdir 362 (time 1788867632.00 total 20.05 last 19.84)
 - mkdir 525 (time 1788867642.03 total 30.08 last 16.25)
 - mkdir 696 (time 1788867652.06 total 40.11 last 17.05)
 - mkdir 881 (time 1788867662.06 total 50.11 last 18.50)
total: 1000 mkdir in 59.88 seconds: 16.70 ops/second
 - mkdir 161 (time 1788867622.07 total 10.03 last 16.05)
 - mkdir 363 (time 1788867632.12 total 20.08 last 20.10)
 - mkdir 525 (time 1788867642.21 total 30.17 last 16.05)
 - mkdir 691 (time 1788867652.24 total 40.20 last 16.55)
 - mkdir 878 (time 1788867662.26 total 50.22 last 18.66)
 - mkdir 998 (time 1788867672.37 total 60.33 last 11.87)
total: 1000 mkdir in 60.42 seconds: 16.55 ops/second
 - mkdir 159 (time 1788867621.40 total 10.04 last 15.84)
 - mkdir 352 (time 1788867631.43 total 20.07 last 19.24)
 - mkdir 513 (time 1788867641.49 total 30.13 last 16.00)
 - mkdir 679 (time 1788867651.50 total 40.14 last 16.59)
 - mkdir 865 (time 1788867661.52 total 50.16 last 18.56)
 - mkdir 988 (time 1788867671.54 total 60.18 last 12.27)
total: 1000 mkdir in 61.49 seconds: 16.26 ops/second
 - mkdir 166 (time 1788867622.40 total 10.00 last 16.60)
 - mkdir 365 (time 1788867632.40 total 20.01 last 19.89)
 - mkdir 520 (time 1788867642.46 total 30.07 last 15.41)
 - mkdir 689 (time 1788867652.48 total 40.09 last 16.87)
 - mkdir 875 (time 1788867662.48 total 50.09 last 18.60)
 - mkdir 991 (time 1788867672.57 total 60.18 last 11.50)
total: 1000 mkdir in 61.29 seconds: 16.32 ops/second
 - mkdir 163 (time 1788867622.27 total 10.01 last 16.29)
 - mkdir 360 (time 1788867632.37 total 20.11 last 19.50)
 - mkdir 516 (time 1788867642.41 total 30.15 last 15.54)
 - mkdir 682 (time 1788867652.45 total 40.19 last 16.53)
 - mkdir 868 (time 1788867662.53 total 50.27 last 18.45)
 - mkdir 987 (time 1788867672.64 total 60.38 last 11.77)
total: 1000 mkdir in 61.89 seconds: 16.16 ops/second
 - mkdir 157 (time 1788867622.27 total 10.03 last 15.65)
 - mkdir 351 (time 1788867632.33 total 20.09 last 19.28)
 - mkdir 509 (time 1788867642.33 total 30.10 last 15.80)
 - mkdir 683 (time 1788867652.37 total 40.13 last 17.34)
 - mkdir 870 (time 1788867662.45 total 50.22 last 18.54)
 - mkdir 986 (time 1788867672.50 total 60.27 last 11.54)
total: 1000 mkdir in 61.98 seconds: 16.14 ops/second
 - mkdir 154 (time 1788867622.10 total 10.05 last 15.32)
 - mkdir 348 (time 1788867632.19 total 20.14 last 19.23)
 - mkdir 500 (time 1788867642.21 total 30.16 last 15.17)
 - mkdir 667 (time 1788867652.24 total 40.19 last 16.64)
 - mkdir 856 (time 1788867662.33 total 50.28 last 18.74)
 - mkdir 976 (time 1788867672.33 total 60.28 last 11.99)
total: 1000 mkdir in 62.74 seconds: 15.94 ops/second
Time for file creation 73 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.46 seconds: 21.78 ops/second
 - mkdir 159 (time 1788868221.20 total 10.02 last 15.87)
 - mkdir 337 (time 1788868231.23 total 20.05 last 17.74)
 - mkdir 519 (time 1788868241.29 total 30.11 last 18.10)
 - mkdir 723 (time 1788868251.32 total 40.15 last 20.33)
 - mkdir 896 (time 1788868261.34 total 50.16 last 17.28)
total: 1000 mkdir in 54.63 seconds: 18.30 ops/second
 - mkdir 145 (time 1788868216.85 total 10.00 last 14.49)
 - mkdir 303 (time 1788868226.92 total 20.07 last 15.70)
 - mkdir 475 (time 1788868236.95 total 30.10 last 17.15)
 - mkdir 663 (time 1788868246.98 total 40.13 last 18.75)
 - mkdir 830 (time 1788868257.05 total 50.20 last 16.58)
total: 1000 mkdir in 59.31 seconds: 16.86 ops/second
 - mkdir 163 (time 1788868221.65 total 10.02 last 16.26)
 - mkdir 346 (time 1788868231.67 total 20.04 last 18.27)
 - mkdir 524 (time 1788868241.69 total 30.07 last 17.75)
 - mkdir 723 (time 1788868251.74 total 40.11 last 19.81)
 - mkdir 900 (time 1788868261.76 total 50.14 last 17.66)
total: 1000 mkdir in 55.03 seconds: 18.17 ops/second
 - mkdir 164 (time 1788868222.08 total 10.03 last 16.34)
 - mkdir 347 (time 1788868232.11 total 20.06 last 18.25)
 - mkdir 520 (time 1788868242.21 total 30.16 last 17.13)
 - mkdir 724 (time 1788868252.21 total 40.17 last 20.39)
 - mkdir 905 (time 1788868262.25 total 50.21 last 18.03)
total: 1000 mkdir in 55.58 seconds: 17.99 ops/second
 - mkdir 160 (time 1788868221.61 total 10.01 last 15.99)
 - mkdir 337 (time 1788868231.64 total 20.03 last 17.66)
 - mkdir 521 (time 1788868241.67 total 30.06 last 18.35)
 - mkdir 722 (time 1788868251.73 total 40.12 last 19.98)
 - mkdir 894 (time 1788868261.74 total 50.14 last 17.17)
total: 1000 mkdir in 56.15 seconds: 17.81 ops/second
 - mkdir 166 (time 1788868221.62 total 10.04 last 16.54)
 - mkdir 335 (time 1788868231.65 total 20.07 last 16.84)
 - mkdir 513 (time 1788868241.70 total 30.12 last 17.71)
 - mkdir 715 (time 1788868251.72 total 40.15 last 20.15)
 - mkdir 887 (time 1788868261.74 total 50.16 last 17.17)
total: 1000 mkdir in 56.34 seconds: 17.75 ops/second
 - mkdir 165 (time 1788868221.60 total 10.05 last 16.42)
 - mkdir 336 (time 1788868231.60 total 20.05 last 17.10)
 - mkdir 516 (time 1788868241.62 total 30.07 last 17.96)
 - mkdir 715 (time 1788868251.73 total 40.17 last 19.70)
 - mkdir 887 (time 1788868261.75 total 50.20 last 17.16)
total: 1000 mkdir in 56.73 seconds: 17.63 ops/second
 - mkdir 158 (time 1788868221.90 total 10.02 last 15.77)
 - mkdir 336 (time 1788868231.94 total 20.06 last 17.72)
 - mkdir 512 (time 1788868242.04 total 30.16 last 17.43)
 - mkdir 711 (time 1788868252.08 total 40.21 last 19.82)
 - mkdir 888 (time 1788868262.09 total 50.21 last 17.69)
total: 1000 mkdir in 57.00 seconds: 17.54 ops/second
 - mkdir 156 (time 1788868221.19 total 10.05 last 15.52)
 - mkdir 329 (time 1788868231.20 total 20.06 last 17.28)
 - mkdir 503 (time 1788868241.29 total 30.15 last 17.24)
 - mkdir 705 (time 1788868251.33 total 40.19 last 20.12)
 - mkdir 868 (time 1788868261.35 total 50.22 last 16.26)
total: 1000 mkdir in 57.81 seconds: 17.30 ops/second
 - mkdir 153 (time 1788868220.73 total 10.03 last 15.25)
 - mkdir 325 (time 1788868230.77 total 20.08 last 17.12)
 - mkdir 504 (time 1788868240.83 total 30.13 last 17.80)
 - mkdir 697 (time 1788868250.85 total 40.15 last 19.27)
 - mkdir 855 (time 1788868260.93 total 50.24 last 15.67)
total: 1000 mkdir in 59.29 seconds: 16.87 ops/second
Time for file creation 69 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788868277 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1788868277 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1788868278 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1788868279 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1788868279 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788868279 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788868279 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788868279 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788868279 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788868279 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
Time for file removal 42 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8