== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 21:08:38 (1788484118)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.50 seconds: 19.93 ops/second
 - mkdir 170 (time 1788484140.62 total 10.01 last 16.99)
 - mkdir 311 (time 1788484150.67 total 20.06 last 14.02)
 - mkdir 466 (time 1788484160.67 total 30.06 last 15.50)
 - mkdir 597 (time 1788484170.71 total 40.10 last 13.05)
 - mkdir 756 (time 1788484180.80 total 50.19 last 15.76)
 - mkdir 926 (time 1788484190.94 total 60.33 last 16.76)
total: 1000 mkdir in 65.38 seconds: 15.29 ops/second
 - mkdir 178 (time 1788484142.40 total 10.04 last 17.72)
 - mkdir 311 (time 1788484152.46 total 20.10 last 13.23)
 - mkdir 464 (time 1788484162.51 total 30.15 last 15.22)
 - mkdir 596 (time 1788484172.55 total 40.19 last 13.14)
 - mkdir 771 (time 1788484182.59 total 50.23 last 17.45)
 - mkdir 947 (time 1788484192.60 total 60.24 last 17.58)
total: 1000 mkdir in 64.34 seconds: 15.54 ops/second
 - mkdir 165 (time 1788484141.15 total 10.01 last 16.48)
 - mkdir 301 (time 1788484151.19 total 20.05 last 13.55)
 - mkdir 453 (time 1788484161.31 total 30.17 last 15.02)
 - mkdir 585 (time 1788484171.32 total 40.18 last 13.18)
 - mkdir 746 (time 1788484181.36 total 50.22 last 16.04)
 - mkdir 921 (time 1788484191.41 total 60.27 last 17.41)
total: 1000 mkdir in 66.47 seconds: 15.04 ops/second
 - mkdir 178 (time 1788484142.78 total 10.15 last 17.55)
 - mkdir 309 (time 1788484152.80 total 20.16 last 13.08)
 - mkdir 463 (time 1788484162.81 total 30.17 last 15.39)
 - mkdir 595 (time 1788484172.82 total 40.18 last 13.19)
 - mkdir 766 (time 1788484182.89 total 50.26 last 16.97)
 - mkdir 933 (time 1788484192.91 total 60.27 last 16.67)
total: 1000 mkdir in 66.14 seconds: 15.12 ops/second
 - mkdir 174 (time 1788484142.52 total 10.07 last 17.28)
 - mkdir 304 (time 1788484152.56 total 20.10 last 12.95)
 - mkdir 453 (time 1788484162.60 total 30.14 last 14.84)
 - mkdir 583 (time 1788484172.66 total 40.21 last 12.92)
 - mkdir 752 (time 1788484182.72 total 50.27 last 16.79)
 - mkdir 924 (time 1788484192.78 total 60.33 last 17.10)
total: 1000 mkdir in 67.16 seconds: 14.89 ops/second
 - mkdir 177 (time 1788484142.68 total 10.09 last 17.55)
 - mkdir 308 (time 1788484152.76 total 20.16 last 13.00)
 - mkdir 462 (time 1788484162.79 total 30.20 last 15.35)
 - mkdir 589 (time 1788484172.82 total 40.22 last 12.67)
 - mkdir 753 (time 1788484182.84 total 50.25 last 16.36)
 - mkdir 923 (time 1788484192.88 total 60.28 last 16.94)
total: 1000 mkdir in 67.47 seconds: 14.82 ops/second
 - mkdir 170 (time 1788484142.79 total 10.01 last 16.98)
 - mkdir 298 (time 1788484152.89 total 20.12 last 12.66)
 - mkdir 448 (time 1788484162.91 total 30.13 last 14.98)
 - mkdir 582 (time 1788484172.93 total 40.15 last 13.38)
 - mkdir 752 (time 1788484182.94 total 50.16 last 16.98)
 - mkdir 920 (time 1788484192.96 total 60.19 last 16.75)
total: 1000 mkdir in 68.12 seconds: 14.68 ops/second
 - mkdir 171 (time 1788484142.61 total 10.05 last 17.01)
 - mkdir 298 (time 1788484152.67 total 20.11 last 12.63)
 - mkdir 452 (time 1788484162.74 total 30.17 last 15.30)
 - mkdir 578 (time 1788484172.76 total 40.20 last 12.57)
 - mkdir 742 (time 1788484182.77 total 50.21 last 16.38)
 - mkdir 910 (time 1788484192.82 total 60.26 last 16.72)
total: 1000 mkdir in 68.81 seconds: 14.53 ops/second
 - mkdir 155 (time 1788484141.13 total 10.04 last 15.45)
 - mkdir 284 (time 1788484151.15 total 20.06 last 12.87)
 - mkdir 437 (time 1788484161.16 total 30.07 last 15.28)
 - mkdir 566 (time 1788484171.16 total 40.07 last 12.89)
 - mkdir 719 (time 1788484181.20 total 50.10 last 15.25)
 - mkdir 887 (time 1788484191.22 total 60.12 last 16.77)
 - mkdir 996 (time 1788484201.24 total 70.15 last 10.87)
total: 1000 mkdir in 70.40 seconds: 14.20 ops/second
 - mkdir 161 (time 1788484142.51 total 10.09 last 15.96)
 - mkdir 282 (time 1788484152.53 total 20.10 last 12.08)
 - mkdir 423 (time 1788484162.57 total 30.14 last 14.04)
 - mkdir 548 (time 1788484172.66 total 40.23 last 12.39)
 - mkdir 707 (time 1788484182.72 total 50.30 last 15.80)
 - mkdir 869 (time 1788484192.81 total 60.38 last 16.06)
 - mkdir 978 (time 1788484202.86 total 70.43 last 10.85)
total: 1000 mkdir in 71.57 seconds: 13.97 ops/second
Time for file creation 83 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 11 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.61 seconds: 16.41 ops/second
 - mkdir 137 (time 1788484751.17 total 10.01 last 13.69)
 - mkdir 262 (time 1788484761.20 total 20.04 last 12.46)
 - mkdir 398 (time 1788484771.28 total 30.12 last 13.50)
 - mkdir 560 (time 1788484781.31 total 40.15 last 16.15)
 - mkdir 694 (time 1788484791.40 total 50.24 last 13.28)
 - mkdir 854 (time 1788484801.42 total 60.26 last 15.96)
 - mkdir 995 (time 1788484811.47 total 70.31 last 14.04)
total: 1000 mkdir in 70.56 seconds: 14.17 ops/second
 - mkdir 132 (time 1788484752.30 total 10.10 last 13.06)
 - mkdir 258 (time 1788484762.31 total 20.11 last 12.59)
 - mkdir 407 (time 1788484772.32 total 30.12 last 14.89)
 - mkdir 562 (time 1788484782.37 total 40.17 last 15.42)
 - mkdir 704 (time 1788484792.41 total 50.22 last 14.14)
 - mkdir 859 (time 1788484802.42 total 60.23 last 15.49)
 - mkdir 996 (time 1788484812.50 total 70.30 last 13.60)
total: 1000 mkdir in 70.58 seconds: 14.17 ops/second
 - mkdir 133 (time 1788484751.72 total 10.02 last 13.27)
 - mkdir 257 (time 1788484761.82 total 20.11 last 12.28)
 - mkdir 395 (time 1788484771.86 total 30.16 last 13.74)
 - mkdir 555 (time 1788484781.96 total 40.26 last 15.84)
 - mkdir 698 (time 1788484791.97 total 50.26 last 14.30)
 - mkdir 849 (time 1788484801.98 total 60.28 last 15.08)
 - mkdir 982 (time 1788484812.04 total 70.34 last 13.22)
total: 1000 mkdir in 72.54 seconds: 13.79 ops/second
 - mkdir 132 (time 1788484752.32 total 10.08 last 13.09)
 - mkdir 257 (time 1788484762.33 total 20.09 last 12.49)
 - mkdir 400 (time 1788484772.37 total 30.14 last 14.23)
 - mkdir 554 (time 1788484782.42 total 40.19 last 15.33)
 - mkdir 697 (time 1788484792.45 total 50.22 last 14.25)
 - mkdir 846 (time 1788484802.54 total 60.31 last 14.77)
 - mkdir 980 (time 1788484812.60 total 70.37 last 13.32)
total: 1000 mkdir in 72.95 seconds: 13.71 ops/second
 - mkdir 126 (time 1788484752.02 total 10.04 last 12.56)
 - mkdir 256 (time 1788484762.03 total 20.05 last 12.99)
 - mkdir 398 (time 1788484772.08 total 30.09 last 14.13)
 - mkdir 554 (time 1788484782.14 total 40.15 last 15.51)
 - mkdir 692 (time 1788484792.14 total 50.16 last 13.80)
 - mkdir 841 (time 1788484802.18 total 60.20 last 14.84)
 - mkdir 979 (time 1788484812.26 total 70.28 last 13.69)
total: 1000 mkdir in 73.22 seconds: 13.66 ops/second
 - mkdir 137 (time 1788484751.17 total 10.06 last 13.61)
 - mkdir 259 (time 1788484761.20 total 20.09 last 12.16)
 - mkdir 393 (time 1788484771.29 total 30.18 last 13.28)
 - mkdir 551 (time 1788484781.31 total 40.20 last 15.77)
 - mkdir 681 (time 1788484791.39 total 50.29 last 12.89)
 - mkdir 829 (time 1788484801.40 total 60.29 last 14.79)
 - mkdir 969 (time 1788484811.48 total 70.37 last 13.89)
total: 1000 mkdir in 74.10 seconds: 13.50 ops/second
 - mkdir 129 (time 1788484752.76 total 10.04 last 12.85)
 - mkdir 254 (time 1788484762.79 total 20.06 last 12.47)
 - mkdir 401 (time 1788484772.84 total 30.12 last 14.62)
 - mkdir 551 (time 1788484782.91 total 40.19 last 14.90)
 - mkdir 694 (time 1788484793.00 total 50.27 last 14.18)
 - mkdir 846 (time 1788484803.08 total 60.35 last 15.08)
 - mkdir 978 (time 1788484813.16 total 70.44 last 13.09)
total: 1000 mkdir in 73.16 seconds: 13.67 ops/second
 - mkdir 128 (time 1788484752.72 total 10.03 last 12.76)
 - mkdir 249 (time 1788484762.74 total 20.05 last 12.08)
 - mkdir 396 (time 1788484772.75 total 30.06 last 14.69)
 - mkdir 546 (time 1788484782.78 total 40.10 last 14.94)
 - mkdir 690 (time 1788484792.84 total 50.15 last 14.33)
 - mkdir 839 (time 1788484802.86 total 60.17 last 14.87)
 - mkdir 971 (time 1788484812.90 total 70.21 last 13.14)
total: 1000 mkdir in 74.22 seconds: 13.47 ops/second
 - mkdir 127 (time 1788484752.17 total 10.02 last 12.67)
 - mkdir 251 (time 1788484762.17 total 20.03 last 12.39)
 - mkdir 394 (time 1788484772.21 total 30.06 last 14.25)
 - mkdir 544 (time 1788484782.24 total 40.10 last 14.95)
 - mkdir 682 (time 1788484792.24 total 50.10 last 13.80)
 - mkdir 827 (time 1788484802.32 total 60.18 last 14.39)
 - mkdir 962 (time 1788484812.53 total 70.38 last 13.23)
total: 1000 mkdir in 75.29 seconds: 13.28 ops/second
 - mkdir 122 (time 1788484752.82 total 10.06 last 12.12)
 - mkdir 235 (time 1788484762.88 total 20.12 last 11.23)
 - mkdir 377 (time 1788484772.90 total 30.14 last 14.17)
 - mkdir 513 (time 1788484782.93 total 40.17 last 13.56)
 - mkdir 648 (time 1788484792.97 total 50.22 last 13.44)
 - mkdir 788 (time 1788484803.00 total 60.25 last 13.96)
 - mkdir 923 (time 1788484813.14 total 70.39 last 13.31)
total: 1000 mkdir in 78.10 seconds: 12.80 ops/second
Time for file creation 87 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788484827 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1788484829 ; total 0 ; last 0)
total: 1000 unlinks in 43 seconds: 23.255814 unlinks/second
 - unlinked 0 (time 1788484829 ; total 0 ; last 0)
total: 1000 unlinks in 44 seconds: 22.727272 unlinks/second
 - unlinked 0 (time 1788484830 ; total 0 ; last 0)
total: 1000 unlinks in 44 seconds: 22.727272 unlinks/second
 - unlinked 0 (time 1788484831 ; total 0 ; last 0)
total: 1000 unlinks in 45 seconds: 22.222221 unlinks/second
 - unlinked 0 (time 1788484831 ; total 0 ; last 0)
total: 1000 unlinks in 45 seconds: 22.222221 unlinks/second
 - unlinked 0 (time 1788484831 ; total 0 ; last 0)
total: 1000 unlinks in 45 seconds: 22.222221 unlinks/second
 - unlinked 0 (time 1788484830 ; total 0 ; last 0)
total: 1000 unlinks in 46 seconds: 21.739130 unlinks/second
 - unlinked 0 (time 1788484832 ; total 1 ; last 1)
total: 1000 unlinks in 45 seconds: 22.222221 unlinks/second
 - unlinked 0 (time 1788484832 ; total 1 ; last 1)
total: 1000 unlinks in 45 seconds: 22.222221 unlinks/second
Time for file removal 56 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8