== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 08:00:50 (1788609650)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.59 seconds: 17.04 ops/second
 - mkdir 133 (time 1788609674.15 total 10.01 last 13.29)
 - mkdir 259 (time 1788609684.18 total 20.04 last 12.57)
 - mkdir 394 (time 1788609694.27 total 30.13 last 13.37)
 - mkdir 527 (time 1788609704.32 total 40.18 last 13.24)
 - mkdir 666 (time 1788609714.41 total 50.27 last 13.78)
 - mkdir 806 (time 1788609724.41 total 60.27 last 14.00)
 - mkdir 959 (time 1788609734.53 total 70.39 last 15.12)
total: 1000 mkdir in 72.60 seconds: 13.77 ops/second
 - mkdir 125 (time 1788609674.55 total 10.01 last 12.49)
 - mkdir 251 (time 1788609684.58 total 20.04 last 12.56)
 - mkdir 385 (time 1788609694.63 total 30.09 last 13.34)
 - mkdir 516 (time 1788609704.68 total 40.14 last 13.03)
 - mkdir 660 (time 1788609714.69 total 50.15 last 14.39)
 - mkdir 804 (time 1788609724.70 total 60.16 last 14.38)
 - mkdir 959 (time 1788609734.77 total 70.23 last 15.40)
total: 1000 mkdir in 72.44 seconds: 13.81 ops/second
 - mkdir 126 (time 1788609673.94 total 10.01 last 12.59)
 - mkdir 253 (time 1788609683.96 total 20.03 last 12.67)
 - mkdir 387 (time 1788609694.01 total 30.07 last 13.34)
 - mkdir 514 (time 1788609704.04 total 40.11 last 12.66)
 - mkdir 657 (time 1788609714.10 total 50.16 last 14.22)
 - mkdir 800 (time 1788609724.13 total 60.19 last 14.26)
 - mkdir 950 (time 1788609734.21 total 70.28 last 14.87)
total: 1000 mkdir in 73.69 seconds: 13.57 ops/second
 - mkdir 135 (time 1788609675.45 total 10.13 last 13.33)
 - mkdir 267 (time 1788609685.54 total 20.21 last 13.09)
 - mkdir 402 (time 1788609695.59 total 30.26 last 13.43)
 - mkdir 532 (time 1788609705.60 total 40.27 last 12.99)
 - mkdir 671 (time 1788609715.63 total 50.30 last 13.86)
 - mkdir 814 (time 1788609725.64 total 60.31 last 14.29)
 - mkdir 967 (time 1788609735.66 total 70.34 last 15.26)
total: 1000 mkdir in 72.91 seconds: 13.72 ops/second
 - mkdir 120 (time 1788609673.82 total 10.03 last 11.97)
 - mkdir 240 (time 1788609683.83 total 20.04 last 11.98)
 - mkdir 368 (time 1788609693.92 total 30.13 last 12.68)
 - mkdir 502 (time 1788609703.95 total 40.16 last 13.37)
 - mkdir 646 (time 1788609713.98 total 50.20 last 14.34)
 - mkdir 785 (time 1788609723.99 total 60.20 last 13.90)
 - mkdir 936 (time 1788609734.06 total 70.28 last 14.98)
total: 1000 mkdir in 75.72 seconds: 13.21 ops/second
 - mkdir 131 (time 1788609675.53 total 10.06 last 13.03)
 - mkdir 258 (time 1788609685.55 total 20.07 last 12.68)
 - mkdir 388 (time 1788609695.56 total 30.08 last 12.99)
 - mkdir 521 (time 1788609705.59 total 40.11 last 13.26)
 - mkdir 661 (time 1788609715.60 total 50.13 last 13.98)
 - mkdir 802 (time 1788609725.67 total 60.20 last 14.00)
 - mkdir 954 (time 1788609735.72 total 70.25 last 15.12)
total: 1000 mkdir in 74.28 seconds: 13.46 ops/second
 - mkdir 130 (time 1788609675.75 total 10.07 last 12.91)
 - mkdir 260 (time 1788609685.82 total 20.14 last 12.91)
 - mkdir 394 (time 1788609695.90 total 30.23 last 13.29)
 - mkdir 527 (time 1788609705.95 total 40.27 last 13.24)
 - mkdir 664 (time 1788609716.01 total 50.34 last 13.61)
 - mkdir 802 (time 1788609726.08 total 60.41 last 13.71)
 - mkdir 955 (time 1788609736.09 total 70.42 last 15.28)
total: 1000 mkdir in 74.66 seconds: 13.39 ops/second
 - mkdir 132 (time 1788609675.71 total 10.12 last 13.04)
 - mkdir 254 (time 1788609685.74 total 20.15 last 12.16)
 - mkdir 385 (time 1788609695.88 total 30.29 last 12.92)
 - mkdir 517 (time 1788609705.91 total 40.33 last 13.15)
 - mkdir 656 (time 1788609715.94 total 50.35 last 13.87)
 - mkdir 797 (time 1788609726.02 total 60.44 last 13.98)
 - mkdir 952 (time 1788609736.05 total 70.46 last 15.46)
total: 1000 mkdir in 75.45 seconds: 13.25 ops/second
 - mkdir 123 (time 1788609675.68 total 10.00 last 12.30)
 - mkdir 248 (time 1788609685.73 total 20.06 last 12.43)
 - mkdir 376 (time 1788609695.90 total 30.22 last 12.60)
 - mkdir 507 (time 1788609705.95 total 40.27 last 13.03)
 - mkdir 646 (time 1788609716.00 total 50.32 last 13.83)
 - mkdir 784 (time 1788609726.05 total 60.37 last 13.74)
 - mkdir 935 (time 1788609736.09 total 70.41 last 15.04)
total: 1000 mkdir in 77.77 seconds: 12.86 ops/second
 - mkdir 118 (time 1788609674.08 total 10.01 last 11.79)
 - mkdir 232 (time 1788609684.12 total 20.05 last 11.35)
 - mkdir 359 (time 1788609694.13 total 30.06 last 12.69)
 - mkdir 485 (time 1788609704.17 total 40.10 last 12.55)
 - mkdir 625 (time 1788609714.21 total 50.14 last 13.94)
 - mkdir 766 (time 1788609724.21 total 60.14 last 14.10)
 - mkdir 912 (time 1788609734.34 total 70.26 last 14.42)
total: 1000 mkdir in 80.09 seconds: 12.49 ops/second
Time for file creation 91 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
Time for file removal 17 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.54 seconds: 18.50 ops/second
 - mkdir 145 (time 1788610300.08 total 10.03 last 14.46)
 - mkdir 291 (time 1788610310.08 total 20.03 last 14.59)
 - mkdir 448 (time 1788610320.13 total 30.08 last 15.62)
 - mkdir 579 (time 1788610330.18 total 40.13 last 13.04)
 - mkdir 699 (time 1788610340.24 total 50.19 last 11.92)
 - mkdir 810 (time 1788610350.25 total 60.19 last 11.09)
 - mkdir 950 (time 1788610360.26 total 70.21 last 13.98)
total: 1000 mkdir in 73.09 seconds: 13.68 ops/second
 - mkdir 139 (time 1788610303.07 total 10.01 last 13.88)
 - mkdir 285 (time 1788610313.08 total 20.03 last 14.58)
 - mkdir 440 (time 1788610323.16 total 30.10 last 15.39)
 - mkdir 560 (time 1788610333.18 total 40.12 last 11.98)
 - mkdir 681 (time 1788610343.27 total 50.21 last 11.98)
 - mkdir 791 (time 1788610353.27 total 60.22 last 11.00)
 - mkdir 949 (time 1788610363.40 total 70.35 last 15.60)
total: 1000 mkdir in 75.84 seconds: 13.19 ops/second
 - mkdir 130 (time 1788610301.03 total 10.01 last 12.99)
 - mkdir 274 (time 1788610311.09 total 20.07 last 14.31)
 - mkdir 427 (time 1788610321.16 total 30.14 last 15.20)
 - mkdir 548 (time 1788610331.19 total 40.17 last 12.06)
 - mkdir 664 (time 1788610341.26 total 50.23 last 11.52)
 - mkdir 777 (time 1788610351.29 total 60.26 last 11.26)
 - mkdir 912 (time 1788610361.40 total 70.37 last 13.36)
total: 1000 mkdir in 78.15 seconds: 12.80 ops/second
 - mkdir 141 (time 1788610303.17 total 10.06 last 14.02)
 - mkdir 284 (time 1788610313.29 total 20.18 last 14.13)
 - mkdir 440 (time 1788610323.30 total 30.19 last 15.59)
 - mkdir 558 (time 1788610333.37 total 40.26 last 11.71)
 - mkdir 678 (time 1788610343.38 total 50.27 last 11.99)
 - mkdir 785 (time 1788610353.41 total 60.31 last 10.67)
 - mkdir 941 (time 1788610363.49 total 70.38 last 15.49)
total: 1000 mkdir in 77.18 seconds: 12.96 ops/second
 - mkdir 142 (time 1788610303.74 total 10.02 last 14.17)
 - mkdir 277 (time 1788610313.75 total 20.04 last 13.48)
 - mkdir 435 (time 1788610323.79 total 30.08 last 15.74)
 - mkdir 559 (time 1788610333.84 total 40.13 last 12.34)
 - mkdir 674 (time 1788610343.92 total 50.21 last 11.41)
 - mkdir 789 (time 1788610354.01 total 60.29 last 11.41)
 - mkdir 938 (time 1788610364.08 total 70.36 last 14.80)
total: 1000 mkdir in 77.35 seconds: 12.93 ops/second
 - mkdir 152 (time 1788610304.17 total 10.03 last 15.16)
 - mkdir 287 (time 1788610314.24 total 20.09 last 13.41)
 - mkdir 442 (time 1788610324.28 total 30.13 last 15.44)
 - mkdir 565 (time 1788610334.31 total 40.16 last 12.27)
 - mkdir 687 (time 1788610344.33 total 50.18 last 12.17)
 - mkdir 794 (time 1788610354.36 total 60.22 last 10.66)
 - mkdir 942 (time 1788610364.47 total 70.32 last 14.65)
total: 1000 mkdir in 77.42 seconds: 12.92 ops/second
 - mkdir 134 (time 1788610303.29 total 10.06 last 13.32)
 - mkdir 271 (time 1788610313.33 total 20.10 last 13.64)
 - mkdir 421 (time 1788610323.39 total 30.16 last 14.91)
 - mkdir 541 (time 1788610333.43 total 40.20 last 11.95)
 - mkdir 665 (time 1788610343.44 total 50.21 last 12.39)
 - mkdir 774 (time 1788610353.45 total 60.22 last 10.89)
 - mkdir 921 (time 1788610363.46 total 70.23 last 14.68)
total: 1000 mkdir in 79.02 seconds: 12.65 ops/second
 - mkdir 142 (time 1788610303.91 total 10.05 last 14.13)
 - mkdir 279 (time 1788610313.94 total 20.08 last 13.66)
 - mkdir 431 (time 1788610323.95 total 30.09 last 15.19)
 - mkdir 552 (time 1788610333.98 total 40.12 last 12.06)
 - mkdir 673 (time 1788610344.03 total 50.18 last 12.04)
 - mkdir 777 (time 1788610354.05 total 60.20 last 10.38)
 - mkdir 927 (time 1788610364.16 total 70.30 last 14.84)
total: 1000 mkdir in 78.59 seconds: 12.72 ops/second
 - mkdir 148 (time 1788610303.85 total 10.00 last 14.80)
 - mkdir 281 (time 1788610313.87 total 20.02 last 13.27)
 - mkdir 428 (time 1788610323.90 total 30.05 last 14.67)
 - mkdir 550 (time 1788610333.93 total 40.08 last 12.16)
 - mkdir 666 (time 1788610344.03 total 50.18 last 11.48)
 - mkdir 775 (time 1788610354.09 total 60.24 last 10.83)
 - mkdir 921 (time 1788610364.11 total 70.26 last 14.58)
total: 1000 mkdir in 79.43 seconds: 12.59 ops/second
 - mkdir 132 (time 1788610303.34 total 10.03 last 13.16)
 - mkdir 262 (time 1788610313.39 total 20.08 last 12.94)
 - mkdir 412 (time 1788610323.50 total 30.19 last 14.84)
 - mkdir 532 (time 1788610333.53 total 40.21 last 11.97)
 - mkdir 656 (time 1788610343.56 total 50.25 last 12.35)
 - mkdir 765 (time 1788610353.61 total 60.29 last 10.85)
 - mkdir 910 (time 1788610363.65 total 70.33 last 14.44)
 - mkdir 995 (time 1788610373.76 total 80.45 last 8.40)
total: 1000 mkdir in 80.82 seconds: 12.37 ops/second
Time for file creation 90 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788610383 ; total 0 ; last 0)
total: 1000 unlinks in 44 seconds: 22.727272 unlinks/second
 - unlinked 0 (time 1788610384 ; total 0 ; last 0)
total: 1000 unlinks in 45 seconds: 22.222221 unlinks/second
 - unlinked 0 (time 1788610384 ; total 0 ; last 0)
total: 1000 unlinks in 45 seconds: 22.222221 unlinks/second
 - unlinked 0 (time 1788610385 ; total 0 ; last 0)
total: 1000 unlinks in 44 seconds: 22.727272 unlinks/second
 - unlinked 0 (time 1788610385 ; total 0 ; last 0)
total: 1000 unlinks in 44 seconds: 22.727272 unlinks/second
 - unlinked 0 (time 1788610386 ; total 0 ; last 0)
total: 1000 unlinks in 45 seconds: 22.222221 unlinks/second
 - unlinked 0 (time 1788610385 ; total 0 ; last 0)
total: 1000 unlinks in 46 seconds: 21.739130 unlinks/second
 - unlinked 0 (time 1788610386 ; total 0 ; last 0)
total: 1000 unlinks in 45 seconds: 22.222221 unlinks/second
 - unlinked 0 (time 1788610386 ; total 0 ; last 0)
total: 1000 unlinks in 45 seconds: 22.222221 unlinks/second
 - unlinked 0 (time 1788610385 ; total 0 ; last 0)
total: 1000 unlinks in 47 seconds: 21.276596 unlinks/second
Time for file removal 59 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8