== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 16:02:22 (1790712142)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.50 seconds: 20.05 ops/second
 - mkdir 137 (time 1790712165.13 total 10.07 last 13.60)
 - mkdir 274 (time 1790712175.20 total 20.15 last 13.59)
 - mkdir 410 (time 1790712185.25 total 30.19 last 13.54)
 - mkdir 544 (time 1790712195.27 total 40.22 last 13.37)
 - mkdir 673 (time 1790712205.29 total 50.24 last 12.87)
 - mkdir 813 (time 1790712215.31 total 60.25 last 13.97)
 - mkdir 958 (time 1790712225.40 total 70.35 last 14.36)
total: 1000 mkdir in 73.45 seconds: 13.61 ops/second
 - mkdir 138 (time 1790712165.86 total 10.05 last 13.73)
 - mkdir 280 (time 1790712175.89 total 20.08 last 14.16)
 - mkdir 413 (time 1790712185.93 total 30.11 last 13.26)
 - mkdir 538 (time 1790712195.93 total 40.12 last 12.49)
 - mkdir 672 (time 1790712205.95 total 50.13 last 13.38)
 - mkdir 811 (time 1790712216.02 total 60.21 last 13.79)
 - mkdir 954 (time 1790712226.03 total 70.21 last 14.29)
total: 1000 mkdir in 73.74 seconds: 13.56 ops/second
 - mkdir 130 (time 1790712165.56 total 10.05 last 12.94)
 - mkdir 274 (time 1790712175.57 total 20.06 last 14.38)
 - mkdir 407 (time 1790712185.60 total 30.09 last 13.26)
 - mkdir 535 (time 1790712195.65 total 40.14 last 12.74)
 - mkdir 665 (time 1790712205.68 total 50.17 last 12.96)
 - mkdir 803 (time 1790712215.71 total 60.20 last 13.76)
 - mkdir 942 (time 1790712225.79 total 70.28 last 13.79)
total: 1000 mkdir in 75.50 seconds: 13.25 ops/second
 - mkdir 132 (time 1790712165.94 total 10.05 last 13.13)
 - mkdir 274 (time 1790712175.94 total 20.05 last 14.20)
 - mkdir 409 (time 1790712186.01 total 30.13 last 13.40)
 - mkdir 531 (time 1790712196.09 total 40.21 last 12.10)
 - mkdir 659 (time 1790712206.13 total 50.24 last 12.76)
 - mkdir 800 (time 1790712216.14 total 60.26 last 14.08)
 - mkdir 945 (time 1790712226.19 total 70.30 last 14.43)
total: 1000 mkdir in 75.60 seconds: 13.23 ops/second
 - mkdir 136 (time 1790712165.64 total 10.01 last 13.59)
 - mkdir 273 (time 1790712175.65 total 20.01 last 13.70)
 - mkdir 403 (time 1790712185.71 total 30.07 last 12.92)
 - mkdir 529 (time 1790712195.73 total 40.10 last 12.56)
 - mkdir 660 (time 1790712205.75 total 50.12 last 13.07)
 - mkdir 799 (time 1790712215.78 total 60.14 last 13.87)
 - mkdir 935 (time 1790712225.80 total 70.16 last 13.57)
total: 1000 mkdir in 76.88 seconds: 13.01 ops/second
 - mkdir 140 (time 1790712166.75 total 10.02 last 13.97)
 - mkdir 279 (time 1790712176.81 total 20.08 last 13.81)
 - mkdir 416 (time 1790712186.84 total 30.12 last 13.65)
 - mkdir 541 (time 1790712197.01 total 40.29 last 12.29)
 - mkdir 676 (time 1790712207.07 total 50.34 last 13.43)
 - mkdir 807 (time 1790712217.18 total 60.45 last 12.96)
 - mkdir 949 (time 1790712227.20 total 70.48 last 14.16)
total: 1000 mkdir in 75.81 seconds: 13.19 ops/second
 - mkdir 131 (time 1790712166.55 total 10.05 last 13.04)
 - mkdir 274 (time 1790712176.56 total 20.05 last 14.29)
 - mkdir 406 (time 1790712186.62 total 30.11 last 13.13)
 - mkdir 527 (time 1790712196.64 total 40.13 last 12.07)
 - mkdir 659 (time 1790712206.71 total 50.20 last 13.11)
 - mkdir 791 (time 1790712216.72 total 60.22 last 13.18)
 - mkdir 935 (time 1790712226.72 total 70.22 last 14.40)
total: 1000 mkdir in 77.39 seconds: 12.92 ops/second
 - mkdir 128 (time 1790712164.57 total 10.05 last 12.74)
 - mkdir 259 (time 1790712174.65 total 20.13 last 13.00)
 - mkdir 385 (time 1790712184.66 total 30.13 last 12.59)
 - mkdir 510 (time 1790712194.68 total 40.16 last 12.47)
 - mkdir 636 (time 1790712204.70 total 50.17 last 12.58)
 - mkdir 770 (time 1790712214.72 total 60.19 last 13.37)
 - mkdir 901 (time 1790712224.72 total 70.20 last 13.09)
total: 1000 mkdir in 79.76 seconds: 12.54 ops/second
 - mkdir 130 (time 1790712166.50 total 10.02 last 12.98)
 - mkdir 269 (time 1790712176.52 total 20.04 last 13.87)
 - mkdir 398 (time 1790712186.58 total 30.10 last 12.83)
 - mkdir 521 (time 1790712196.64 total 40.16 last 12.22)
 - mkdir 656 (time 1790712206.68 total 50.19 last 13.45)
 - mkdir 787 (time 1790712216.78 total 60.29 last 12.98)
 - mkdir 930 (time 1790712226.91 total 70.42 last 14.11)
total: 1000 mkdir in 77.89 seconds: 12.84 ops/second
 - mkdir 127 (time 1790712166.61 total 10.04 last 12.65)
 - mkdir 262 (time 1790712176.71 total 20.14 last 13.38)
 - mkdir 388 (time 1790712186.74 total 30.17 last 12.56)
 - mkdir 505 (time 1790712196.77 total 40.20 last 11.67)
 - mkdir 631 (time 1790712206.79 total 50.22 last 12.58)
 - mkdir 757 (time 1790712216.95 total 60.38 last 12.40)
 - mkdir 889 (time 1790712226.98 total 70.41 last 13.16)
 - mkdir 984 (time 1790712237.00 total 80.42 last 9.48)
total: 1000 mkdir in 81.08 seconds: 12.33 ops/second
Time for file creation 93 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.60 seconds: 16.55 ops/second
 - mkdir 148 (time 1790712825.76 total 10.16 last 14.56)
 - mkdir 307 (time 1790712835.79 total 20.19 last 15.86)
 - mkdir 467 (time 1790712845.79 total 30.20 last 15.99)
 - mkdir 653 (time 1790712855.80 total 40.21 last 18.58)
 - mkdir 812 (time 1790712865.80 total 50.21 last 15.90)
 - mkdir 975 (time 1790712875.82 total 60.22 last 16.28)
total: 1000 mkdir in 61.88 seconds: 16.16 ops/second
 - mkdir 154 (time 1790712827.75 total 10.06 last 15.30)
 - mkdir 322 (time 1790712837.81 total 20.12 last 16.71)
 - mkdir 495 (time 1790712847.85 total 30.16 last 17.22)
 - mkdir 673 (time 1790712857.93 total 40.24 last 17.66)
 - mkdir 839 (time 1790712867.99 total 50.30 last 16.50)
 - mkdir 997 (time 1790712878.00 total 60.31 last 15.78)
total: 1000 mkdir in 60.54 seconds: 16.52 ops/second
 - mkdir 152 (time 1790712827.96 total 10.04 last 15.15)
 - mkdir 315 (time 1790712837.98 total 20.05 last 16.28)
 - mkdir 494 (time 1790712847.98 total 30.05 last 17.89)
 - mkdir 667 (time 1790712858.04 total 40.11 last 17.20)
 - mkdir 833 (time 1790712868.07 total 50.15 last 16.54)
 - mkdir 992 (time 1790712878.13 total 60.20 last 15.81)
total: 1000 mkdir in 61.15 seconds: 16.35 ops/second
 - mkdir 163 (time 1790712829.14 total 10.04 last 16.23)
 - mkdir 326 (time 1790712839.17 total 20.07 last 16.25)
 - mkdir 514 (time 1790712849.19 total 30.10 last 18.75)
 - mkdir 679 (time 1790712859.26 total 40.16 last 16.40)
 - mkdir 845 (time 1790712869.26 total 50.16 last 16.60)
 - mkdir 993 (time 1790712879.30 total 60.21 last 14.73)
total: 1000 mkdir in 60.78 seconds: 16.45 ops/second
 - mkdir 148 (time 1790712827.39 total 10.01 last 14.79)
 - mkdir 306 (time 1790712837.40 total 20.02 last 15.79)
 - mkdir 475 (time 1790712847.43 total 30.04 last 16.85)
 - mkdir 653 (time 1790712857.51 total 40.12 last 17.66)
 - mkdir 816 (time 1790712867.52 total 50.14 last 16.27)
 - mkdir 974 (time 1790712877.61 total 60.23 last 15.67)
total: 1000 mkdir in 62.87 seconds: 15.91 ops/second
 - mkdir 155 (time 1790712829.38 total 10.07 last 15.39)
 - mkdir 316 (time 1790712839.42 total 20.10 last 16.04)
 - mkdir 500 (time 1790712849.44 total 30.13 last 18.36)
 - mkdir 669 (time 1790712859.44 total 40.13 last 16.90)
 - mkdir 837 (time 1790712869.50 total 50.18 last 16.71)
 - mkdir 985 (time 1790712879.57 total 60.26 last 14.69)
total: 1000 mkdir in 62.11 seconds: 16.10 ops/second
 - mkdir 158 (time 1790712829.15 total 10.12 last 15.61)
 - mkdir 317 (time 1790712839.17 total 20.14 last 15.87)
 - mkdir 502 (time 1790712849.19 total 30.16 last 18.47)
 - mkdir 663 (time 1790712859.27 total 40.25 last 15.96)
 - mkdir 830 (time 1790712869.33 total 50.30 last 16.61)
 - mkdir 977 (time 1790712879.45 total 60.42 last 14.52)
total: 1000 mkdir in 63.24 seconds: 15.81 ops/second
 - mkdir 155 (time 1790712829.30 total 10.09 last 15.36)
 - mkdir 314 (time 1790712839.43 total 20.22 last 15.69)
 - mkdir 499 (time 1790712849.46 total 30.25 last 18.46)
 - mkdir 660 (time 1790712859.51 total 40.30 last 16.02)
 - mkdir 827 (time 1790712869.56 total 50.35 last 16.62)
 - mkdir 975 (time 1790712879.57 total 60.36 last 14.78)
total: 1000 mkdir in 63.45 seconds: 15.76 ops/second
 - mkdir 148 (time 1790712828.86 total 10.04 last 14.74)
 - mkdir 304 (time 1790712838.87 total 20.06 last 15.58)
 - mkdir 481 (time 1790712848.89 total 30.07 last 17.68)
 - mkdir 648 (time 1790712858.93 total 40.11 last 16.63)
 - mkdir 810 (time 1790712868.96 total 50.14 last 16.15)
 - mkdir 959 (time 1790712879.05 total 60.23 last 14.77)
total: 1000 mkdir in 64.78 seconds: 15.44 ops/second
 - mkdir 149 (time 1790712829.18 total 10.05 last 14.83)
 - mkdir 300 (time 1790712839.22 total 20.09 last 15.04)
 - mkdir 475 (time 1790712849.25 total 30.13 last 17.44)
 - mkdir 638 (time 1790712859.30 total 40.17 last 16.22)
 - mkdir 807 (time 1790712869.36 total 50.23 last 16.81)
 - mkdir 954 (time 1790712879.50 total 60.37 last 14.49)
total: 1000 mkdir in 65.32 seconds: 15.31 ops/second
Time for file creation 77 sec for 1 RPC sent at a time
 - unlinked 0 (time 1790712893 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1790712893 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1790712895 ; total 1 ; last 1)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1790712894 ; total 1 ; last 1)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1790712893 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1790712894 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1790712894 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1790712894 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1790712894 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1790712894 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
Time for file removal 47 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8