== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 21:47:55 (1788832075)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.53 seconds: 18.87 ops/second
 - mkdir 178 (time 1788832097.91 total 10.05 last 17.72)
 - mkdir 359 (time 1788832107.95 total 20.09 last 18.02)
 - mkdir 531 (time 1788832117.96 total 30.09 last 17.20)
 - mkdir 695 (time 1788832128.02 total 40.16 last 16.29)
 - mkdir 835 (time 1788832138.12 total 50.25 last 13.87)
 - mkdir 950 (time 1788832148.14 total 60.28 last 11.47)
total: 1000 mkdir in 63.37 seconds: 15.78 ops/second
 - mkdir 171 (time 1788832097.50 total 10.04 last 17.04)
 - mkdir 352 (time 1788832107.53 total 20.07 last 18.05)
 - mkdir 519 (time 1788832117.58 total 30.11 last 16.62)
 - mkdir 679 (time 1788832127.62 total 40.16 last 15.92)
 - mkdir 816 (time 1788832137.66 total 50.20 last 13.65)
 - mkdir 930 (time 1788832147.67 total 60.21 last 11.39)
total: 1000 mkdir in 65.10 seconds: 15.36 ops/second
 - mkdir 168 (time 1788832097.37 total 10.04 last 16.73)
 - mkdir 350 (time 1788832107.38 total 20.05 last 18.19)
 - mkdir 515 (time 1788832117.41 total 30.07 last 16.46)
 - mkdir 670 (time 1788832127.42 total 40.09 last 15.48)
 - mkdir 806 (time 1788832137.48 total 50.15 last 13.52)
 - mkdir 915 (time 1788832147.53 total 60.20 last 10.85)
total: 1000 mkdir in 66.56 seconds: 15.02 ops/second
 - mkdir 177 (time 1788832098.03 total 10.05 last 17.62)
 - mkdir 355 (time 1788832108.05 total 20.07 last 17.77)
 - mkdir 524 (time 1788832118.05 total 30.07 last 16.89)
 - mkdir 677 (time 1788832128.11 total 40.13 last 15.21)
 - mkdir 816 (time 1788832138.12 total 50.13 last 13.90)
 - mkdir 928 (time 1788832148.15 total 60.17 last 11.16)
total: 1000 mkdir in 66.39 seconds: 15.06 ops/second
 - mkdir 178 (time 1788832098.26 total 10.01 last 17.78)
 - mkdir 358 (time 1788832108.34 total 20.09 last 17.86)
 - mkdir 526 (time 1788832118.35 total 30.11 last 16.78)
 - mkdir 683 (time 1788832128.42 total 40.17 last 15.60)
 - mkdir 818 (time 1788832138.55 total 50.31 last 13.32)
 - mkdir 936 (time 1788832148.61 total 60.36 last 11.73)
total: 1000 mkdir in 66.27 seconds: 15.09 ops/second
 - mkdir 175 (time 1788832098.33 total 10.00 last 17.50)
 - mkdir 354 (time 1788832108.34 total 20.01 last 17.89)
 - mkdir 524 (time 1788832118.36 total 30.04 last 16.96)
 - mkdir 678 (time 1788832128.41 total 40.08 last 15.33)
 - mkdir 811 (time 1788832138.53 total 50.20 last 13.14)
 - mkdir 925 (time 1788832148.59 total 60.26 last 11.34)
total: 1000 mkdir in 67.10 seconds: 14.90 ops/second
 - mkdir 178 (time 1788832098.36 total 10.04 last 17.73)
 - mkdir 354 (time 1788832108.43 total 20.11 last 17.48)
 - mkdir 524 (time 1788832118.43 total 30.11 last 17.00)
 - mkdir 674 (time 1788832128.46 total 40.14 last 14.95)
 - mkdir 808 (time 1788832138.46 total 50.14 last 13.40)
 - mkdir 924 (time 1788832148.48 total 60.16 last 11.58)
total: 1000 mkdir in 67.24 seconds: 14.87 ops/second
 - mkdir 174 (time 1788832098.16 total 10.02 last 17.37)
 - mkdir 351 (time 1788832108.16 total 20.02 last 17.70)
 - mkdir 521 (time 1788832118.18 total 30.04 last 16.96)
 - mkdir 674 (time 1788832128.21 total 40.07 last 15.26)
 - mkdir 808 (time 1788832138.38 total 50.23 last 13.18)
 - mkdir 922 (time 1788832148.44 total 60.30 last 11.32)
total: 1000 mkdir in 67.48 seconds: 14.82 ops/second
 - mkdir 155 (time 1788832096.21 total 10.02 last 15.47)
 - mkdir 334 (time 1788832106.22 total 20.02 last 17.90)
 - mkdir 506 (time 1788832116.26 total 30.06 last 17.13)
 - mkdir 653 (time 1788832126.28 total 40.09 last 14.66)
 - mkdir 785 (time 1788832136.37 total 50.17 last 13.09)
 - mkdir 891 (time 1788832146.43 total 60.23 last 10.54)
 - mkdir 998 (time 1788832156.44 total 70.25 last 10.68)
total: 1000 mkdir in 70.36 seconds: 14.21 ops/second
 - mkdir 158 (time 1788832096.74 total 10.01 last 15.78)
 - mkdir 337 (time 1788832106.78 total 20.05 last 17.84)
 - mkdir 499 (time 1788832116.82 total 30.09 last 16.13)
 - mkdir 653 (time 1788832126.87 total 40.14 last 15.34)
 - mkdir 784 (time 1788832136.93 total 50.20 last 13.02)
 - mkdir 890 (time 1788832147.03 total 60.30 last 10.49)
 - mkdir 999 (time 1788832157.10 total 70.37 last 10.83)
total: 1000 mkdir in 70.37 seconds: 14.21 ops/second
Time for file creation 80 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.61 seconds: 16.28 ops/second
 - mkdir 113 (time 1788832744.06 total 10.07 last 11.23)
 - mkdir 261 (time 1788832754.13 total 20.14 last 14.69)
 - mkdir 417 (time 1788832764.20 total 30.21 last 15.48)
 - mkdir 537 (time 1788832774.24 total 40.24 last 11.96)
 - mkdir 686 (time 1788832784.27 total 50.28 last 14.85)
 - mkdir 834 (time 1788832794.29 total 60.30 last 14.77)
 - mkdir 994 (time 1788832804.37 total 70.38 last 15.87)
total: 1000 mkdir in 70.71 seconds: 14.14 ops/second
 - mkdir 128 (time 1788832745.90 total 10.00 last 12.80)
 - mkdir 279 (time 1788832756.00 total 20.11 last 14.95)
 - mkdir 429 (time 1788832766.02 total 30.13 last 14.96)
 - mkdir 552 (time 1788832776.06 total 40.16 last 12.26)
 - mkdir 701 (time 1788832786.10 total 50.21 last 14.83)
 - mkdir 857 (time 1788832796.13 total 60.24 last 15.55)
 - mkdir 998 (time 1788832806.19 total 70.30 last 14.01)
total: 1000 mkdir in 70.36 seconds: 14.21 ops/second
 - mkdir 114 (time 1788832744.82 total 10.00 last 11.40)
 - mkdir 260 (time 1788832754.87 total 20.05 last 14.52)
 - mkdir 411 (time 1788832764.92 total 30.10 last 15.03)
 - mkdir 531 (time 1788832774.95 total 40.13 last 11.96)
 - mkdir 683 (time 1788832784.96 total 50.14 last 15.18)
 - mkdir 830 (time 1788832794.99 total 60.17 last 14.66)
 - mkdir 982 (time 1788832805.02 total 70.21 last 15.14)
total: 1000 mkdir in 72.42 seconds: 13.81 ops/second
 - mkdir 121 (time 1788832745.61 total 10.01 last 12.09)
 - mkdir 269 (time 1788832755.64 total 20.03 last 14.76)
 - mkdir 417 (time 1788832765.66 total 30.06 last 14.76)
 - mkdir 538 (time 1788832775.69 total 40.09 last 12.06)
 - mkdir 687 (time 1788832785.69 total 50.09 last 14.90)
 - mkdir 840 (time 1788832795.72 total 60.12 last 15.26)
 - mkdir 982 (time 1788832805.79 total 70.19 last 14.10)
total: 1000 mkdir in 72.17 seconds: 13.86 ops/second
 - mkdir 119 (time 1788832745.72 total 10.02 last 11.87)
 - mkdir 269 (time 1788832755.79 total 20.09 last 14.90)
 - mkdir 416 (time 1788832765.86 total 30.16 last 14.60)
 - mkdir 540 (time 1788832775.86 total 40.17 last 12.39)
 - mkdir 691 (time 1788832785.93 total 50.23 last 15.00)
 - mkdir 843 (time 1788832795.97 total 60.27 last 15.13)
 - mkdir 983 (time 1788832806.08 total 70.38 last 13.85)
total: 1000 mkdir in 72.14 seconds: 13.86 ops/second
 - mkdir 125 (time 1788832746.37 total 10.01 last 12.49)
 - mkdir 274 (time 1788832756.38 total 20.02 last 14.88)
 - mkdir 419 (time 1788832766.39 total 30.03 last 14.49)
 - mkdir 541 (time 1788832776.41 total 40.05 last 12.18)
 - mkdir 687 (time 1788832786.44 total 50.08 last 14.55)
 - mkdir 840 (time 1788832796.46 total 60.10 last 15.28)
 - mkdir 981 (time 1788832806.51 total 70.15 last 14.02)
total: 1000 mkdir in 72.08 seconds: 13.87 ops/second
 - mkdir 127 (time 1788832746.48 total 10.07 last 12.61)
 - mkdir 267 (time 1788832756.57 total 20.16 last 13.87)
 - mkdir 417 (time 1788832766.57 total 30.17 last 14.99)
 - mkdir 544 (time 1788832776.61 total 40.20 last 12.65)
 - mkdir 693 (time 1788832786.62 total 50.21 last 14.89)
 - mkdir 844 (time 1788832796.63 total 60.22 last 15.08)
 - mkdir 979 (time 1788832806.65 total 70.24 last 13.47)
total: 1000 mkdir in 72.51 seconds: 13.79 ops/second
 - mkdir 107 (time 1788832743.99 total 10.04 last 10.66)
 - mkdir 245 (time 1788832754.02 total 20.08 last 13.75)
 - mkdir 386 (time 1788832764.09 total 30.15 last 14.00)
 - mkdir 505 (time 1788832774.10 total 40.15 last 11.90)
 - mkdir 657 (time 1788832784.13 total 50.19 last 15.14)
 - mkdir 800 (time 1788832794.19 total 60.25 last 14.22)
 - mkdir 946 (time 1788832804.23 total 70.29 last 14.54)
total: 1000 mkdir in 75.89 seconds: 13.18 ops/second
 - mkdir 112 (time 1788832745.83 total 10.02 last 11.18)
 - mkdir 252 (time 1788832755.92 total 20.10 last 13.89)
 - mkdir 395 (time 1788832765.96 total 30.14 last 14.24)
 - mkdir 517 (time 1788832775.97 total 40.16 last 12.18)
 - mkdir 667 (time 1788832786.01 total 50.19 last 14.94)
 - mkdir 822 (time 1788832796.03 total 60.22 last 15.47)
 - mkdir 953 (time 1788832806.12 total 70.30 last 12.99)
total: 1000 mkdir in 75.08 seconds: 13.32 ops/second
 - mkdir 119 (time 1788832746.55 total 10.00 last 11.89)
 - mkdir 258 (time 1788832756.64 total 20.09 last 13.78)
 - mkdir 401 (time 1788832766.73 total 30.18 last 14.17)
 - mkdir 527 (time 1788832776.73 total 40.19 last 12.59)
 - mkdir 675 (time 1788832786.79 total 50.24 last 14.72)
 - mkdir 832 (time 1788832796.83 total 60.28 last 15.64)
 - mkdir 955 (time 1788832806.86 total 70.32 last 12.26)
total: 1000 mkdir in 74.67 seconds: 13.39 ops/second
Time for file creation 88 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788832818 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788832821 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1788832822 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1788832821 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1788832821 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1788832821 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1788832821 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1788832822 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1788832822 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1788832822 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
Time for file removal 51 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8