== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 23:40:14 (1788234014)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.40 seconds: 25.25 ops/second
 - mkdir 166 (time 1788234035.72 total 10.03 last 16.55)
 - mkdir 313 (time 1788234045.73 total 20.04 last 14.69)
 - mkdir 439 (time 1788234055.74 total 30.05 last 12.58)
 - mkdir 572 (time 1788234065.75 total 40.06 last 13.28)
 - mkdir 726 (time 1788234075.82 total 50.13 last 15.30)
 - mkdir 877 (time 1788234085.83 total 60.14 last 15.08)
total: 1000 mkdir in 67.46 seconds: 14.82 ops/second
 - mkdir 166 (time 1788234036.03 total 10.01 last 16.58)
 - mkdir 315 (time 1788234046.04 total 20.03 last 14.88)
 - mkdir 440 (time 1788234056.12 total 30.10 last 12.41)
 - mkdir 570 (time 1788234066.14 total 40.13 last 12.97)
 - mkdir 719 (time 1788234076.15 total 50.14 last 14.88)
 - mkdir 869 (time 1788234086.21 total 60.19 last 14.92)
total: 1000 mkdir in 68.60 seconds: 14.58 ops/second
 - mkdir 152 (time 1788234034.32 total 10.01 last 15.19)
 - mkdir 294 (time 1788234044.38 total 20.07 last 14.11)
 - mkdir 420 (time 1788234054.50 total 30.19 last 12.45)
 - mkdir 536 (time 1788234064.53 total 40.22 last 11.57)
 - mkdir 690 (time 1788234074.55 total 50.24 last 15.36)
 - mkdir 829 (time 1788234084.61 total 60.29 last 13.82)
 - mkdir 979 (time 1788234094.70 total 70.39 last 14.86)
total: 1000 mkdir in 72.61 seconds: 13.77 ops/second
 - mkdir 152 (time 1788234035.43 total 10.05 last 15.12)
 - mkdir 294 (time 1788234045.50 total 20.12 last 14.10)
 - mkdir 410 (time 1788234055.53 total 30.15 last 11.56)
 - mkdir 535 (time 1788234065.58 total 40.20 last 12.44)
 - mkdir 687 (time 1788234075.60 total 50.23 last 15.16)
 - mkdir 826 (time 1788234085.62 total 60.25 last 13.88)
 - mkdir 983 (time 1788234095.71 total 70.33 last 15.57)
total: 1000 mkdir in 72.71 seconds: 13.75 ops/second
 - mkdir 160 (time 1788234035.83 total 10.01 last 15.98)
 - mkdir 299 (time 1788234045.96 total 20.14 last 13.72)
 - mkdir 422 (time 1788234055.99 total 30.17 last 12.26)
 - mkdir 547 (time 1788234066.03 total 40.21 last 12.45)
 - mkdir 694 (time 1788234076.06 total 50.25 last 14.65)
 - mkdir 844 (time 1788234086.09 total 60.27 last 14.96)
 - mkdir 985 (time 1788234096.17 total 70.35 last 13.99)
total: 1000 mkdir in 72.67 seconds: 13.76 ops/second
 - mkdir 164 (time 1788234036.40 total 10.04 last 16.33)
 - mkdir 299 (time 1788234046.42 total 20.06 last 13.47)
 - mkdir 421 (time 1788234056.47 total 30.12 last 12.13)
 - mkdir 548 (time 1788234066.51 total 40.15 last 12.66)
 - mkdir 695 (time 1788234076.55 total 50.20 last 14.64)
 - mkdir 840 (time 1788234086.59 total 60.24 last 14.44)
 - mkdir 987 (time 1788234096.61 total 70.26 last 14.68)
total: 1000 mkdir in 72.14 seconds: 13.86 ops/second
 - mkdir 152 (time 1788234035.40 total 10.00 last 15.19)
 - mkdir 291 (time 1788234045.40 total 20.01 last 13.89)
 - mkdir 415 (time 1788234055.47 total 30.08 last 12.32)
 - mkdir 539 (time 1788234065.48 total 40.09 last 12.39)
 - mkdir 689 (time 1788234075.56 total 50.16 last 14.88)
 - mkdir 828 (time 1788234085.59 total 60.20 last 13.85)
 - mkdir 976 (time 1788234095.64 total 70.24 last 14.73)
total: 1000 mkdir in 73.39 seconds: 13.63 ops/second
 - mkdir 165 (time 1788234036.87 total 10.09 last 16.35)
 - mkdir 304 (time 1788234046.93 total 20.15 last 13.82)
 - mkdir 423 (time 1788234056.98 total 30.20 last 11.84)
 - mkdir 555 (time 1788234066.98 total 40.21 last 13.20)
 - mkdir 699 (time 1788234077.01 total 50.23 last 14.36)
 - mkdir 845 (time 1788234087.03 total 60.25 last 14.57)
 - mkdir 987 (time 1788234097.19 total 70.41 last 13.98)
total: 1000 mkdir in 72.21 seconds: 13.85 ops/second
 - mkdir 163 (time 1788234036.14 total 10.06 last 16.20)
 - mkdir 297 (time 1788234046.26 total 20.18 last 13.24)
 - mkdir 412 (time 1788234056.30 total 30.22 last 11.45)
 - mkdir 541 (time 1788234066.44 total 40.36 last 12.72)
 - mkdir 684 (time 1788234076.45 total 50.37 last 14.29)
 - mkdir 828 (time 1788234086.53 total 60.46 last 14.28)
 - mkdir 975 (time 1788234096.62 total 70.54 last 14.58)
total: 1000 mkdir in 73.55 seconds: 13.60 ops/second
 - mkdir 149 (time 1788234034.94 total 10.01 last 14.88)
 - mkdir 284 (time 1788234044.99 total 20.06 last 13.43)
 - mkdir 401 (time 1788234055.14 total 30.21 last 11.53)
 - mkdir 519 (time 1788234065.16 total 40.23 last 11.78)
 - mkdir 665 (time 1788234075.18 total 50.26 last 14.56)
 - mkdir 799 (time 1788234085.25 total 60.33 last 13.31)
 - mkdir 954 (time 1788234095.26 total 70.34 last 15.48)
total: 1000 mkdir in 75.31 seconds: 13.28 ops/second
Time for file creation 85 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.53 seconds: 18.88 ops/second
 - mkdir 143 (time 1788234654.48 total 10.01 last 14.28)
 - mkdir 289 (time 1788234664.49 total 20.03 last 14.58)
 - mkdir 426 (time 1788234674.50 total 30.03 last 13.69)
 - mkdir 563 (time 1788234684.52 total 40.05 last 13.67)
 - mkdir 723 (time 1788234694.56 total 50.10 last 15.93)
 - mkdir 870 (time 1788234704.64 total 60.17 last 14.59)
total: 1000 mkdir in 68.74 seconds: 14.55 ops/second
 - mkdir 151 (time 1788234656.17 total 10.02 last 15.07)
 - mkdir 293 (time 1788234666.20 total 20.05 last 14.15)
 - mkdir 417 (time 1788234676.23 total 30.08 last 12.37)
 - mkdir 567 (time 1788234686.26 total 40.11 last 14.95)
 - mkdir 720 (time 1788234696.27 total 50.12 last 15.29)
 - mkdir 863 (time 1788234706.27 total 60.12 last 14.30)
 - mkdir 997 (time 1788234716.37 total 70.22 last 13.27)
total: 1000 mkdir in 70.65 seconds: 14.15 ops/second
 - mkdir 158 (time 1788234657.48 total 10.04 last 15.73)
 - mkdir 303 (time 1788234667.53 total 20.09 last 14.43)
 - mkdir 434 (time 1788234677.54 total 30.11 last 13.08)
 - mkdir 579 (time 1788234687.61 total 40.17 last 14.41)
 - mkdir 741 (time 1788234697.65 total 50.21 last 16.13)
 - mkdir 885 (time 1788234707.67 total 60.24 last 14.36)
total: 1000 mkdir in 69.62 seconds: 14.36 ops/second
 - mkdir 156 (time 1788234657.67 total 10.05 last 15.53)
 - mkdir 305 (time 1788234667.71 total 20.09 last 14.84)
 - mkdir 440 (time 1788234677.75 total 30.13 last 13.45)
 - mkdir 586 (time 1788234687.76 total 40.14 last 14.59)
 - mkdir 742 (time 1788234697.81 total 50.19 last 15.51)
 - mkdir 879 (time 1788234707.81 total 60.19 last 13.70)
 - mkdir 997 (time 1788234717.89 total 70.27 last 11.71)
total: 1000 mkdir in 70.47 seconds: 14.19 ops/second
 - mkdir 140 (time 1788234656.44 total 10.12 last 13.84)
 - mkdir 285 (time 1788234666.58 total 20.26 last 14.30)
 - mkdir 405 (time 1788234676.61 total 30.28 last 11.97)
 - mkdir 560 (time 1788234686.68 total 40.35 last 15.40)
 - mkdir 719 (time 1788234696.72 total 50.39 last 15.84)
 - mkdir 861 (time 1788234706.76 total 60.43 last 14.14)
 - mkdir 990 (time 1788234716.82 total 70.49 last 12.82)
total: 1000 mkdir in 72.12 seconds: 13.87 ops/second
 - mkdir 147 (time 1788234656.66 total 10.01 last 14.68)
 - mkdir 293 (time 1788234666.70 total 20.04 last 14.55)
 - mkdir 417 (time 1788234676.72 total 30.07 last 12.37)
 - mkdir 563 (time 1788234686.77 total 40.11 last 14.54)
 - mkdir 720 (time 1788234696.78 total 50.13 last 15.68)
 - mkdir 860 (time 1788234706.86 total 60.20 last 13.89)
 - mkdir 988 (time 1788234716.91 total 70.26 last 12.73)
total: 1000 mkdir in 72.22 seconds: 13.85 ops/second
 - mkdir 152 (time 1788234657.31 total 10.02 last 15.16)
 - mkdir 294 (time 1788234667.34 total 20.05 last 14.16)
 - mkdir 422 (time 1788234677.36 total 30.08 last 12.77)
 - mkdir 566 (time 1788234687.38 total 40.10 last 14.37)
 - mkdir 726 (time 1788234697.41 total 50.12 last 15.95)
 - mkdir 865 (time 1788234707.41 total 60.13 last 13.89)
 - mkdir 988 (time 1788234717.58 total 70.29 last 12.10)
total: 1000 mkdir in 71.79 seconds: 13.93 ops/second
 - mkdir 153 (time 1788234657.75 total 10.07 last 15.20)
 - mkdir 296 (time 1788234667.78 total 20.10 last 14.26)
 - mkdir 426 (time 1788234677.91 total 30.23 last 12.83)
 - mkdir 575 (time 1788234687.94 total 40.25 last 14.86)
 - mkdir 730 (time 1788234697.96 total 50.28 last 15.47)
 - mkdir 869 (time 1788234707.97 total 60.28 last 13.89)
 - mkdir 986 (time 1788234718.15 total 70.46 last 11.49)
total: 1000 mkdir in 72.41 seconds: 13.81 ops/second
 - mkdir 146 (time 1788234657.29 total 10.04 last 14.55)
 - mkdir 288 (time 1788234667.36 total 20.11 last 14.10)
 - mkdir 410 (time 1788234677.40 total 30.15 last 12.15)
 - mkdir 552 (time 1788234687.49 total 40.24 last 14.07)
 - mkdir 712 (time 1788234697.50 total 50.25 last 15.98)
 - mkdir 851 (time 1788234707.51 total 60.26 last 13.88)
 - mkdir 968 (time 1788234717.52 total 70.27 last 11.69)
total: 1000 mkdir in 74.22 seconds: 13.47 ops/second
 - mkdir 149 (time 1788234658.09 total 10.01 last 14.89)
 - mkdir 287 (time 1788234668.11 total 20.03 last 13.77)
 - mkdir 409 (time 1788234678.17 total 30.09 last 12.13)
 - mkdir 555 (time 1788234688.17 total 40.09 last 14.59)
 - mkdir 709 (time 1788234698.20 total 50.12 last 15.35)
 - mkdir 847 (time 1788234708.21 total 60.13 last 13.78)
 - mkdir 957 (time 1788234718.36 total 70.28 last 10.84)
total: 1000 mkdir in 74.74 seconds: 13.38 ops/second
Time for file creation 88 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788234733 ; total 0 ; last 0)
total: 1000 unlinks in 48 seconds: 20.833334 unlinks/second
 - unlinked 0 (time 1788234733 ; total 0 ; last 0)
total: 1000 unlinks in 49 seconds: 20.408163 unlinks/second
 - unlinked 0 (time 1788234733 ; total 0 ; last 0)
total: 1000 unlinks in 49 seconds: 20.408163 unlinks/second
 - unlinked 0 (time 1788234734 ; total 0 ; last 0)
total: 1000 unlinks in 48 seconds: 20.833334 unlinks/second
 - unlinked 0 (time 1788234733 ; total 0 ; last 0)
total: 1000 unlinks in 50 seconds: 20.000000 unlinks/second
 - unlinked 0 (time 1788234733 ; total 0 ; last 0)
total: 1000 unlinks in 50 seconds: 20.000000 unlinks/second
 - unlinked 0 (time 1788234733 ; total 0 ; last 0)
total: 1000 unlinks in 50 seconds: 20.000000 unlinks/second
 - unlinked 0 (time 1788234734 ; total 1 ; last 1)
total: 1000 unlinks in 51 seconds: 19.607843 unlinks/second
 - unlinked 0 (time 1788234734 ; total 0 ; last 0)
total: 1000 unlinks in 50 seconds: 20.000000 unlinks/second
 - unlinked 0 (time 1788234734 ; total 0 ; last 0)
total: 1000 unlinks in 50 seconds: 20.000000 unlinks/second
Time for file removal 65 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8