== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 14:10:40 (1787681440)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.59 seconds: 16.98 ops/second
 - mkdir 126 (time 1787681463.87 total 10.07 last 12.51)
 - mkdir 281 (time 1787681473.89 total 20.09 last 15.46)
 - mkdir 437 (time 1787681483.96 total 30.16 last 15.50)
 - mkdir 581 (time 1787681494.07 total 40.27 last 14.24)
 - mkdir 725 (time 1787681504.12 total 50.32 last 14.32)
 - mkdir 883 (time 1787681514.13 total 60.33 last 15.80)
total: 1000 mkdir in 69.29 seconds: 14.43 ops/second
 - mkdir 108 (time 1787681461.56 total 10.01 last 10.79)
 - mkdir 260 (time 1787681471.62 total 20.08 last 15.10)
 - mkdir 417 (time 1787681481.64 total 30.09 last 15.68)
 - mkdir 557 (time 1787681491.69 total 40.14 last 13.93)
 - mkdir 687 (time 1787681501.77 total 50.22 last 12.90)
 - mkdir 846 (time 1787681511.83 total 60.28 last 15.80)
 - mkdir 977 (time 1787681521.85 total 70.30 last 13.07)
total: 1000 mkdir in 71.82 seconds: 13.92 ops/second
 - mkdir 134 (time 1787681465.47 total 10.01 last 13.38)
 - mkdir 285 (time 1787681475.48 total 20.02 last 15.09)
 - mkdir 446 (time 1787681485.49 total 30.04 last 16.07)
 - mkdir 585 (time 1787681495.55 total 40.10 last 13.82)
 - mkdir 724 (time 1787681505.60 total 50.15 last 13.83)
 - mkdir 888 (time 1787681515.68 total 60.22 last 16.28)
total: 1000 mkdir in 69.81 seconds: 14.33 ops/second
 - mkdir 127 (time 1787681465.62 total 10.01 last 12.69)
 - mkdir 276 (time 1787681475.66 total 20.04 last 14.85)
 - mkdir 433 (time 1787681485.77 total 30.15 last 15.53)
 - mkdir 566 (time 1787681495.81 total 40.19 last 13.24)
 - mkdir 706 (time 1787681505.87 total 50.25 last 13.92)
 - mkdir 871 (time 1787681515.90 total 60.28 last 16.45)
 - mkdir 988 (time 1787681526.02 total 70.41 last 11.55)
total: 1000 mkdir in 71.98 seconds: 13.89 ops/second
 - mkdir 146 (time 1787681466.99 total 10.02 last 14.57)
 - mkdir 297 (time 1787681477.09 total 20.12 last 14.96)
 - mkdir 457 (time 1787681487.14 total 30.17 last 15.91)
 - mkdir 597 (time 1787681497.17 total 40.21 last 13.95)
 - mkdir 742 (time 1787681507.23 total 50.26 last 14.42)
 - mkdir 896 (time 1787681517.24 total 60.27 last 15.39)
 - mkdir 996 (time 1787681527.33 total 70.37 last 9.91)
total: 1000 mkdir in 70.89 seconds: 14.11 ops/second
 - mkdir 126 (time 1787681465.03 total 10.02 last 12.58)
 - mkdir 275 (time 1787681475.05 total 20.05 last 14.86)
 - mkdir 436 (time 1787681485.11 total 30.10 last 16.02)
 - mkdir 569 (time 1787681495.13 total 40.12 last 13.27)
 - mkdir 708 (time 1787681505.28 total 50.27 last 13.70)
 - mkdir 870 (time 1787681515.34 total 60.34 last 16.10)
 - mkdir 986 (time 1787681525.36 total 70.35 last 11.58)
total: 1000 mkdir in 72.92 seconds: 13.71 ops/second
 - mkdir 133 (time 1787681465.96 total 10.04 last 13.25)
 - mkdir 281 (time 1787681475.99 total 20.06 last 14.77)
 - mkdir 435 (time 1787681486.07 total 30.15 last 15.27)
 - mkdir 574 (time 1787681496.11 total 40.18 last 13.85)
 - mkdir 711 (time 1787681506.14 total 50.22 last 13.65)
 - mkdir 873 (time 1787681516.20 total 60.28 last 16.11)
 - mkdir 985 (time 1787681526.34 total 70.42 last 11.05)
total: 1000 mkdir in 72.96 seconds: 13.71 ops/second
 - mkdir 134 (time 1787681466.37 total 10.07 last 13.30)
 - mkdir 282 (time 1787681476.40 total 20.10 last 14.76)
 - mkdir 443 (time 1787681486.42 total 30.13 last 16.06)
 - mkdir 582 (time 1787681496.47 total 40.17 last 13.84)
 - mkdir 716 (time 1787681506.50 total 50.21 last 13.35)
 - mkdir 880 (time 1787681516.57 total 60.27 last 16.29)
 - mkdir 988 (time 1787681526.85 total 70.56 last 10.50)
total: 1000 mkdir in 72.80 seconds: 13.74 ops/second
 - mkdir 126 (time 1787681465.56 total 10.03 last 12.57)
 - mkdir 272 (time 1787681475.61 total 20.08 last 14.52)
 - mkdir 423 (time 1787681485.65 total 30.12 last 15.04)
 - mkdir 559 (time 1787681495.66 total 40.14 last 13.58)
 - mkdir 696 (time 1787681505.67 total 50.15 last 13.69)
 - mkdir 857 (time 1787681515.72 total 60.19 last 16.03)
 - mkdir 968 (time 1787681525.88 total 70.35 last 10.93)
total: 1000 mkdir in 75.95 seconds: 13.17 ops/second
 - mkdir 133 (time 1787681466.13 total 10.04 last 13.25)
 - mkdir 278 (time 1787681476.16 total 20.07 last 14.45)
 - mkdir 434 (time 1787681486.25 total 30.16 last 15.46)
 - mkdir 564 (time 1787681496.30 total 40.20 last 12.94)
 - mkdir 699 (time 1787681506.35 total 50.26 last 13.43)
 - mkdir 861 (time 1787681516.38 total 60.29 last 16.15)
 - mkdir 965 (time 1787681526.47 total 70.38 last 10.31)
total: 1000 mkdir in 75.89 seconds: 13.18 ops/second
Time for file creation 89 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.41 seconds: 24.10 ops/second
 - mkdir 108 (time 1787682124.59 total 10.02 last 10.78)
 - mkdir 199 (time 1787682134.69 total 20.13 last 9.01)
 - mkdir 314 (time 1787682144.71 total 30.15 last 11.47)
 - mkdir 457 (time 1787682154.76 total 40.19 last 14.23)
 - mkdir 585 (time 1787682164.77 total 50.21 last 12.78)
 - mkdir 706 (time 1787682174.78 total 60.22 last 12.09)
 - mkdir 827 (time 1787682184.81 total 70.24 last 12.07)
 - mkdir 957 (time 1787682194.97 total 80.40 last 12.80)
total: 1000 mkdir in 83.81 seconds: 11.93 ops/second
 - mkdir 103 (time 1787682125.60 total 10.02 last 10.28)
 - mkdir 193 (time 1787682135.69 total 20.12 last 8.91)
 - mkdir 312 (time 1787682145.71 total 30.13 last 11.89)
 - mkdir 453 (time 1787682155.73 total 40.15 last 14.07)
 - mkdir 585 (time 1787682165.73 total 50.16 last 13.19)
 - mkdir 709 (time 1787682175.78 total 60.20 last 12.34)
 - mkdir 830 (time 1787682185.81 total 70.23 last 12.07)
 - mkdir 951 (time 1787682195.92 total 80.34 last 11.97)
total: 1000 mkdir in 85.86 seconds: 11.65 ops/second
 - mkdir 100 (time 1787682125.20 total 10.11 last 9.90)
 - mkdir 187 (time 1787682135.37 total 20.27 last 8.56)
 - mkdir 303 (time 1787682145.39 total 30.30 last 11.58)
 - mkdir 441 (time 1787682155.39 total 40.30 last 13.80)
 - mkdir 575 (time 1787682165.42 total 50.33 last 13.36)
 - mkdir 699 (time 1787682175.49 total 60.39 last 12.32)
 - mkdir 816 (time 1787682185.50 total 70.41 last 11.68)
 - mkdir 936 (time 1787682195.56 total 80.47 last 11.93)
total: 1000 mkdir in 87.49 seconds: 11.43 ops/second
 - mkdir 104 (time 1787682126.42 total 10.11 last 10.29)
 - mkdir 190 (time 1787682136.61 total 20.29 last 8.44)
 - mkdir 318 (time 1787682146.64 total 30.32 last 12.76)
 - mkdir 456 (time 1787682156.71 total 40.40 last 13.70)
 - mkdir 589 (time 1787682166.81 total 50.49 last 13.17)
 - mkdir 715 (time 1787682176.87 total 60.55 last 12.53)
 - mkdir 835 (time 1787682186.89 total 70.58 last 11.97)
 - mkdir 955 (time 1787682196.94 total 80.63 last 11.94)
total: 1000 mkdir in 86.72 seconds: 11.53 ops/second
 - mkdir 101 (time 1787682123.57 total 10.00 last 10.10)
 - mkdir 193 (time 1787682133.61 total 20.04 last 9.16)
 - mkdir 289 (time 1787682143.69 total 30.12 last 9.53)
 - mkdir 432 (time 1787682153.76 total 40.19 last 14.20)
 - mkdir 555 (time 1787682163.82 total 50.25 last 12.23)
 - mkdir 688 (time 1787682174.07 total 60.50 last 12.97)
 - mkdir 799 (time 1787682184.11 total 70.54 last 11.06)
 - mkdir 922 (time 1787682194.13 total 80.56 last 12.27)
total: 1000 mkdir in 89.60 seconds: 11.16 ops/second
 - mkdir 104 (time 1787682126.20 total 10.05 last 10.35)
 - mkdir 187 (time 1787682136.21 total 20.06 last 8.29)
 - mkdir 312 (time 1787682146.23 total 30.08 last 12.48)
 - mkdir 450 (time 1787682156.30 total 40.14 last 13.71)
 - mkdir 585 (time 1787682166.30 total 50.15 last 13.50)
 - mkdir 707 (time 1787682176.39 total 60.24 last 12.09)
 - mkdir 825 (time 1787682186.42 total 70.26 last 11.77)
 - mkdir 944 (time 1787682196.59 total 80.44 last 11.70)
total: 1000 mkdir in 87.44 seconds: 11.44 ops/second
 - mkdir 103 (time 1787682124.52 total 10.00 last 10.30)
 - mkdir 185 (time 1787682134.58 total 20.06 last 8.15)
 - mkdir 292 (time 1787682144.63 total 30.11 last 10.64)
 - mkdir 435 (time 1787682154.67 total 40.15 last 14.25)
 - mkdir 561 (time 1787682164.68 total 50.16 last 12.59)
 - mkdir 688 (time 1787682174.79 total 60.27 last 12.56)
 - mkdir 800 (time 1787682184.86 total 70.34 last 11.12)
 - mkdir 922 (time 1787682194.97 total 80.45 last 12.07)
 - mkdir 998 (time 1787682205.08 total 90.56 last 7.51)
total: 1000 mkdir in 90.83 seconds: 11.01 ops/second
 - mkdir 96 (time 1787682124.74 total 10.10 last 9.50)
 - mkdir 181 (time 1787682134.95 total 20.31 last 8.33)
 - mkdir 290 (time 1787682144.97 total 30.33 last 10.88)
 - mkdir 436 (time 1787682155.00 total 40.35 last 14.56)
 - mkdir 563 (time 1787682165.07 total 50.43 last 12.61)
 - mkdir 685 (time 1787682175.07 total 60.43 last 12.20)
 - mkdir 799 (time 1787682185.17 total 70.53 last 11.29)
 - mkdir 917 (time 1787682195.35 total 80.71 last 11.59)
 - mkdir 991 (time 1787682205.42 total 90.78 last 7.35)
total: 1000 mkdir in 92.08 seconds: 10.86 ops/second
 - mkdir 97 (time 1787682125.04 total 10.25 last 9.46)
 - mkdir 182 (time 1787682135.11 total 20.32 last 8.44)
 - mkdir 290 (time 1787682145.16 total 30.36 last 10.75)
 - mkdir 436 (time 1787682155.19 total 40.40 last 14.55)
 - mkdir 567 (time 1787682165.27 total 50.48 last 13.00)
 - mkdir 691 (time 1787682175.31 total 60.52 last 12.35)
 - mkdir 799 (time 1787682185.32 total 70.53 last 10.79)
 - mkdir 917 (time 1787682195.45 total 80.66 last 11.65)
 - mkdir 991 (time 1787682205.56 total 90.77 last 7.32)
total: 1000 mkdir in 92.18 seconds: 10.85 ops/second
 - mkdir 95 (time 1787682124.45 total 10.01 last 9.49)
 - mkdir 180 (time 1787682134.50 total 20.07 last 8.45)
 - mkdir 286 (time 1787682144.63 total 30.20 last 10.46)
 - mkdir 423 (time 1787682154.64 total 40.20 last 13.70)
 - mkdir 551 (time 1787682164.71 total 50.28 last 12.71)
 - mkdir 674 (time 1787682174.74 total 60.31 last 12.26)
 - mkdir 786 (time 1787682184.81 total 70.38 last 11.13)
 - mkdir 906 (time 1787682194.91 total 80.47 last 11.88)
 - mkdir 982 (time 1787682204.99 total 90.56 last 7.54)
total: 1000 mkdir in 93.44 seconds: 10.70 ops/second
Time for file creation 107 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787682218 ; total 1 ; last 1)
total: 1000 unlinks in 57 seconds: 17.543859 unlinks/second
 - unlinked 0 (time 1787682218 ; total 0 ; last 0)
total: 1000 unlinks in 58 seconds: 17.241379 unlinks/second
 - unlinked 0 (time 1787682219 ; total 0 ; last 0)
total: 1000 unlinks in 58 seconds: 17.241379 unlinks/second
 - unlinked 0 (time 1787682220 ; total 0 ; last 0)
total: 1000 unlinks in 59 seconds: 16.949152 unlinks/second
 - unlinked 0 (time 1787682220 ; total 0 ; last 0)
total: 1000 unlinks in 59 seconds: 16.949152 unlinks/second
 - unlinked 0 (time 1787682221 ; total 0 ; last 0)
total: 1000 unlinks in 59 seconds: 16.949152 unlinks/second
 - unlinked 0 (time 1787682220 ; total 0 ; last 0)
total: 1000 unlinks in 61 seconds: 16.393442 unlinks/second
 - unlinked 0 (time 1787682221 ; total 0 ; last 0)
total: 1000 unlinks in 60 seconds: 16.666666 unlinks/second
 - unlinked 0 (time 1787682222 ; total 0 ; last 0)
total: 1000 unlinks in 59 seconds: 16.949152 unlinks/second
 - unlinked 0 (time 1787682222 ; total 0 ; last 0)
total: 1000 unlinks in 60 seconds: 16.666666 unlinks/second
Time for file removal 79 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8