== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 16:56:44 (1790801804)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.38 seconds: 26.24 ops/second
 - mkdir 160 (time 1790801825.16 total 10.03 last 15.95)
 - mkdir 325 (time 1790801835.19 total 20.07 last 16.45)
 - mkdir 489 (time 1790801845.24 total 30.12 last 16.31)
 - mkdir 603 (time 1790801855.25 total 40.13 last 11.39)
 - mkdir 748 (time 1790801865.25 total 50.13 last 14.49)
 - mkdir 928 (time 1790801875.32 total 60.20 last 17.87)
total: 1000 mkdir in 64.09 seconds: 15.60 ops/second
 - mkdir 159 (time 1790801826.28 total 10.01 last 15.88)
 - mkdir 323 (time 1790801836.30 total 20.03 last 16.38)
 - mkdir 480 (time 1790801846.35 total 30.07 last 15.62)
 - mkdir 590 (time 1790801856.41 total 40.14 last 10.93)
 - mkdir 750 (time 1790801866.44 total 50.17 last 15.95)
 - mkdir 925 (time 1790801876.44 total 60.17 last 17.49)
total: 1000 mkdir in 65.73 seconds: 15.21 ops/second
 - mkdir 160 (time 1790801826.74 total 10.00 last 16.00)
 - mkdir 322 (time 1790801836.79 total 20.05 last 16.11)
 - mkdir 480 (time 1790801846.86 total 30.12 last 15.69)
 - mkdir 591 (time 1790801856.91 total 40.17 last 11.05)
 - mkdir 754 (time 1790801866.94 total 50.20 last 16.25)
 - mkdir 923 (time 1790801876.97 total 60.23 last 16.85)
total: 1000 mkdir in 66.30 seconds: 15.08 ops/second
 - mkdir 162 (time 1790801827.25 total 10.08 last 16.07)
 - mkdir 332 (time 1790801837.27 total 20.11 last 16.96)
 - mkdir 484 (time 1790801847.31 total 30.14 last 15.15)
 - mkdir 596 (time 1790801857.36 total 40.19 last 11.15)
 - mkdir 761 (time 1790801867.39 total 50.22 last 16.44)
 - mkdir 934 (time 1790801877.42 total 60.25 last 17.25)
total: 1000 mkdir in 65.90 seconds: 15.17 ops/second
 - mkdir 156 (time 1790801826.24 total 10.03 last 15.55)
 - mkdir 317 (time 1790801836.27 total 20.06 last 16.05)
 - mkdir 474 (time 1790801846.28 total 30.07 last 15.68)
 - mkdir 586 (time 1790801856.34 total 40.13 last 11.14)
 - mkdir 734 (time 1790801866.35 total 50.14 last 14.78)
 - mkdir 909 (time 1790801876.36 total 60.15 last 17.48)
total: 1000 mkdir in 67.37 seconds: 14.84 ops/second
 - mkdir 162 (time 1790801827.33 total 10.03 last 16.16)
 - mkdir 326 (time 1790801837.41 total 20.11 last 16.27)
 - mkdir 480 (time 1790801847.41 total 30.11 last 15.39)
 - mkdir 591 (time 1790801857.44 total 40.14 last 11.07)
 - mkdir 755 (time 1790801867.45 total 50.14 last 16.40)
 - mkdir 929 (time 1790801877.48 total 60.17 last 17.35)
total: 1000 mkdir in 67.29 seconds: 14.86 ops/second
 - mkdir 163 (time 1790801827.45 total 10.07 last 16.19)
 - mkdir 328 (time 1790801837.46 total 20.08 last 16.48)
 - mkdir 481 (time 1790801847.54 total 30.16 last 15.18)
 - mkdir 591 (time 1790801857.58 total 40.19 last 10.96)
 - mkdir 755 (time 1790801867.61 total 50.22 last 16.35)
 - mkdir 926 (time 1790801877.62 total 60.24 last 17.08)
total: 1000 mkdir in 67.59 seconds: 14.80 ops/second
 - mkdir 156 (time 1790801826.02 total 10.06 last 15.50)
 - mkdir 321 (time 1790801836.08 total 20.12 last 16.41)
 - mkdir 471 (time 1790801846.10 total 30.14 last 14.96)
 - mkdir 577 (time 1790801856.15 total 40.19 last 10.55)
 - mkdir 728 (time 1790801866.15 total 50.19 last 15.10)
 - mkdir 901 (time 1790801876.20 total 60.24 last 17.22)
total: 1000 mkdir in 69.39 seconds: 14.41 ops/second
 - mkdir 158 (time 1790801827.42 total 10.06 last 15.71)
 - mkdir 321 (time 1790801837.42 total 20.06 last 16.30)
 - mkdir 475 (time 1790801847.47 total 30.11 last 15.32)
 - mkdir 588 (time 1790801857.50 total 40.14 last 11.26)
 - mkdir 755 (time 1790801867.54 total 50.18 last 16.63)
 - mkdir 924 (time 1790801877.55 total 60.18 last 16.90)
total: 1000 mkdir in 68.49 seconds: 14.60 ops/second
 - mkdir 152 (time 1790801826.94 total 10.05 last 15.12)
 - mkdir 307 (time 1790801837.01 total 20.12 last 15.40)
 - mkdir 452 (time 1790801847.03 total 30.13 last 14.48)
 - mkdir 560 (time 1790801857.08 total 40.19 last 10.74)
 - mkdir 722 (time 1790801867.11 total 50.22 last 16.15)
 - mkdir 892 (time 1790801877.13 total 60.24 last 16.96)
 - mkdir 988 (time 1790801887.16 total 70.27 last 9.57)
total: 1000 mkdir in 71.07 seconds: 14.07 ops/second
Time for file creation 82 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 15 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.46 seconds: 21.63 ops/second
 - mkdir 147 (time 1790802432.51 total 10.03 last 14.65)
 - mkdir 319 (time 1790802442.55 total 20.06 last 17.15)
 - mkdir 456 (time 1790802452.58 total 30.09 last 13.66)
 - mkdir 591 (time 1790802462.66 total 40.17 last 13.40)
 - mkdir 711 (time 1790802472.67 total 50.18 last 11.99)
 - mkdir 834 (time 1790802482.74 total 60.25 last 12.21)
 - mkdir 972 (time 1790802492.79 total 70.31 last 13.73)
total: 1000 mkdir in 71.80 seconds: 13.93 ops/second
 - mkdir 136 (time 1790802431.51 total 10.05 last 13.54)
 - mkdir 304 (time 1790802441.51 total 20.05 last 16.79)
 - mkdir 429 (time 1790802451.55 total 30.09 last 12.45)
 - mkdir 569 (time 1790802461.57 total 40.11 last 13.97)
 - mkdir 681 (time 1790802471.68 total 50.21 last 11.09)
 - mkdir 806 (time 1790802481.72 total 60.26 last 12.44)
 - mkdir 929 (time 1790802491.72 total 70.26 last 12.30)
total: 1000 mkdir in 75.48 seconds: 13.25 ops/second
 - mkdir 138 (time 1790802431.94 total 10.06 last 13.71)
 - mkdir 298 (time 1790802441.99 total 20.12 last 15.91)
 - mkdir 429 (time 1790802452.09 total 30.21 last 12.98)
 - mkdir 566 (time 1790802462.20 total 40.33 last 13.55)
 - mkdir 687 (time 1790802472.28 total 50.40 last 12.01)
 - mkdir 813 (time 1790802482.43 total 60.55 last 12.41)
 - mkdir 941 (time 1790802492.48 total 70.60 last 12.74)
total: 1000 mkdir in 75.07 seconds: 13.32 ops/second
 - mkdir 161 (time 1790802434.10 total 10.01 last 16.08)
 - mkdir 324 (time 1790802444.18 total 20.09 last 16.18)
 - mkdir 449 (time 1790802454.18 total 30.10 last 12.49)
 - mkdir 577 (time 1790802464.30 total 40.21 last 12.65)
 - mkdir 719 (time 1790802474.34 total 50.25 last 14.14)
 - mkdir 828 (time 1790802484.41 total 60.32 last 10.83)
 - mkdir 970 (time 1790802494.49 total 70.40 last 14.08)
total: 1000 mkdir in 73.59 seconds: 13.59 ops/second
 - mkdir 150 (time 1790802433.33 total 10.03 last 14.96)
 - mkdir 320 (time 1790802443.34 total 20.05 last 16.97)
 - mkdir 450 (time 1790802453.38 total 30.09 last 12.95)
 - mkdir 581 (time 1790802463.49 total 40.20 last 12.95)
 - mkdir 710 (time 1790802473.51 total 50.21 last 12.88)
 - mkdir 821 (time 1790802483.51 total 60.22 last 11.09)
 - mkdir 949 (time 1790802493.56 total 70.26 last 12.75)
total: 1000 mkdir in 74.58 seconds: 13.41 ops/second
 - mkdir 148 (time 1790802432.91 total 10.02 last 14.77)
 - mkdir 312 (time 1790802442.92 total 20.03 last 16.38)
 - mkdir 436 (time 1790802452.92 total 30.03 last 12.39)
 - mkdir 571 (time 1790802462.99 total 40.10 last 13.41)
 - mkdir 694 (time 1790802473.04 total 50.15 last 12.24)
 - mkdir 813 (time 1790802483.08 total 60.19 last 11.85)
 - mkdir 945 (time 1790802493.12 total 70.23 last 13.15)
total: 1000 mkdir in 75.18 seconds: 13.30 ops/second
 - mkdir 150 (time 1790802433.69 total 10.06 last 14.91)
 - mkdir 316 (time 1790802443.73 total 20.10 last 16.53)
 - mkdir 443 (time 1790802453.79 total 30.16 last 12.62)
 - mkdir 570 (time 1790802463.84 total 40.21 last 12.64)
 - mkdir 703 (time 1790802473.88 total 50.26 last 13.24)
 - mkdir 818 (time 1790802483.90 total 60.27 last 11.49)
 - mkdir 954 (time 1790802493.91 total 70.28 last 13.59)
total: 1000 mkdir in 74.93 seconds: 13.35 ops/second
 - mkdir 150 (time 1790802434.00 total 10.08 last 14.88)
 - mkdir 315 (time 1790802444.05 total 20.12 last 16.42)
 - mkdir 435 (time 1790802454.05 total 30.13 last 12.00)
 - mkdir 565 (time 1790802464.08 total 40.15 last 12.96)
 - mkdir 699 (time 1790802474.08 total 50.16 last 13.39)
 - mkdir 807 (time 1790802484.19 total 60.27 last 10.68)
 - mkdir 944 (time 1790802494.22 total 70.29 last 13.67)
total: 1000 mkdir in 76.42 seconds: 13.09 ops/second
 - mkdir 142 (time 1790802433.22 total 10.03 last 14.16)
 - mkdir 302 (time 1790802443.24 total 20.04 last 15.97)
 - mkdir 421 (time 1790802453.24 total 30.05 last 11.90)
 - mkdir 557 (time 1790802463.31 total 40.12 last 13.50)
 - mkdir 682 (time 1790802473.32 total 50.13 last 12.49)
 - mkdir 802 (time 1790802483.32 total 60.13 last 12.00)
 - mkdir 930 (time 1790802493.35 total 70.15 last 12.77)
total: 1000 mkdir in 77.27 seconds: 12.94 ops/second
 - mkdir 143 (time 1790802433.67 total 10.02 last 14.28)
 - mkdir 300 (time 1790802443.70 total 20.05 last 15.65)
 - mkdir 416 (time 1790802453.84 total 30.19 last 11.44)
 - mkdir 544 (time 1790802463.94 total 40.28 last 12.68)
 - mkdir 682 (time 1790802473.94 total 50.29 last 13.79)
 - mkdir 791 (time 1790802484.01 total 60.35 last 10.83)
 - mkdir 923 (time 1790802494.04 total 70.39 last 13.15)
total: 1000 mkdir in 78.33 seconds: 12.77 ops/second
Time for file creation 90 sec for 1 RPC sent at a time
 - unlinked 0 (time 1790802511 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1790802512 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1790802512 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1790802513 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1790802513 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1790802513 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1790802513 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1790802513 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1790802514 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1790802513 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
Time for file removal 46 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8