== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 12:37:18 (1787157438)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.68 seconds: 14.65 ops/second
 - mkdir 149 (time 1787157459.82 total 10.10 last 14.75)
 - mkdir 280 (time 1787157469.91 total 20.19 last 12.99)
 - mkdir 427 (time 1787157479.93 total 30.21 last 14.67)
 - mkdir 585 (time 1787157489.94 total 40.22 last 15.78)
 - mkdir 735 (time 1787157499.95 total 50.23 last 14.99)
 - mkdir 852 (time 1787157510.00 total 60.28 last 11.64)
 - mkdir 991 (time 1787157520.17 total 70.45 last 13.67)
total: 1000 mkdir in 71.15 seconds: 14.06 ops/second
 - mkdir 149 (time 1787157458.71 total 10.10 last 14.75)
 - mkdir 270 (time 1787157468.74 total 20.13 last 12.06)
 - mkdir 414 (time 1787157478.75 total 30.14 last 14.38)
 - mkdir 567 (time 1787157488.77 total 40.16 last 15.27)
 - mkdir 717 (time 1787157498.80 total 50.20 last 14.95)
 - mkdir 839 (time 1787157508.87 total 60.27 last 12.11)
 - mkdir 980 (time 1787157518.87 total 70.27 last 14.10)
total: 1000 mkdir in 72.40 seconds: 13.81 ops/second
 - mkdir 155 (time 1787157458.30 total 10.01 last 15.49)
 - mkdir 274 (time 1787157468.37 total 20.08 last 11.82)
 - mkdir 413 (time 1787157478.40 total 30.11 last 13.85)
 - mkdir 571 (time 1787157488.45 total 40.16 last 15.72)
 - mkdir 717 (time 1787157498.48 total 50.19 last 14.56)
 - mkdir 839 (time 1787157508.60 total 60.31 last 12.06)
 - mkdir 974 (time 1787157518.67 total 70.38 last 13.41)
total: 1000 mkdir in 72.96 seconds: 13.71 ops/second
 - mkdir 150 (time 1787157460.49 total 10.02 last 14.98)
 - mkdir 285 (time 1787157470.54 total 20.06 last 13.43)
 - mkdir 432 (time 1787157480.57 total 30.09 last 14.67)
 - mkdir 587 (time 1787157490.58 total 40.10 last 15.48)
 - mkdir 738 (time 1787157500.62 total 50.15 last 15.03)
 - mkdir 849 (time 1787157510.63 total 60.15 last 11.09)
 - mkdir 992 (time 1787157520.65 total 70.17 last 14.27)
total: 1000 mkdir in 70.85 seconds: 14.12 ops/second
 - mkdir 146 (time 1787157460.52 total 10.02 last 14.57)
 - mkdir 276 (time 1787157470.59 total 20.09 last 12.91)
 - mkdir 431 (time 1787157480.60 total 30.09 last 15.50)
 - mkdir 585 (time 1787157490.61 total 40.11 last 15.37)
 - mkdir 735 (time 1787157500.65 total 50.15 last 14.95)
 - mkdir 847 (time 1787157510.66 total 60.16 last 11.19)
 - mkdir 992 (time 1787157520.72 total 70.21 last 14.42)
total: 1000 mkdir in 71.08 seconds: 14.07 ops/second
 - mkdir 148 (time 1787157460.04 total 10.10 last 14.65)
 - mkdir 277 (time 1787157470.11 total 20.18 last 12.80)
 - mkdir 422 (time 1787157480.16 total 30.23 last 14.43)
 - mkdir 578 (time 1787157490.19 total 40.26 last 15.55)
 - mkdir 730 (time 1787157500.21 total 50.28 last 15.18)
 - mkdir 841 (time 1787157510.26 total 60.33 last 11.04)
 - mkdir 985 (time 1787157520.28 total 70.35 last 14.37)
total: 1000 mkdir in 71.92 seconds: 13.90 ops/second
 - mkdir 143 (time 1787157460.79 total 10.03 last 14.25)
 - mkdir 268 (time 1787157470.81 total 20.06 last 12.47)
 - mkdir 415 (time 1787157480.84 total 30.09 last 14.66)
 - mkdir 574 (time 1787157490.86 total 40.11 last 15.86)
 - mkdir 724 (time 1787157500.92 total 50.17 last 14.91)
 - mkdir 834 (time 1787157510.92 total 60.17 last 11.00)
 - mkdir 972 (time 1787157520.94 total 70.19 last 13.78)
total: 1000 mkdir in 73.15 seconds: 13.67 ops/second
 - mkdir 147 (time 1787157460.30 total 10.03 last 14.66)
 - mkdir 276 (time 1787157470.35 total 20.08 last 12.83)
 - mkdir 418 (time 1787157480.38 total 30.11 last 14.15)
 - mkdir 573 (time 1787157490.40 total 40.13 last 15.48)
 - mkdir 719 (time 1787157500.42 total 50.15 last 14.57)
 - mkdir 830 (time 1787157510.50 total 60.22 last 11.02)
 - mkdir 968 (time 1787157520.55 total 70.28 last 13.72)
total: 1000 mkdir in 73.64 seconds: 13.58 ops/second
 - mkdir 133 (time 1787157460.65 total 10.01 last 13.29)
 - mkdir 254 (time 1787157470.76 total 20.11 last 11.97)
 - mkdir 394 (time 1787157480.79 total 30.15 last 13.95)
 - mkdir 552 (time 1787157490.79 total 40.15 last 15.79)
 - mkdir 703 (time 1787157500.83 total 50.19 last 15.05)
 - mkdir 814 (time 1787157510.88 total 60.24 last 11.04)
 - mkdir 945 (time 1787157520.91 total 70.27 last 13.07)
total: 1000 mkdir in 75.22 seconds: 13.30 ops/second
 - mkdir 138 (time 1787157461.00 total 10.09 last 13.67)
 - mkdir 261 (time 1787157471.01 total 20.11 last 12.28)
 - mkdir 397 (time 1787157481.05 total 30.15 last 13.55)
 - mkdir 555 (time 1787157491.06 total 40.15 last 15.79)
 - mkdir 703 (time 1787157501.11 total 50.20 last 14.73)
 - mkdir 817 (time 1787157511.13 total 60.22 last 11.38)
 - mkdir 946 (time 1787157521.17 total 70.27 last 12.84)
total: 1000 mkdir in 75.28 seconds: 13.28 ops/second
Time for file creation 86 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 8 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.52 seconds: 19.24 ops/second
 - mkdir 145 (time 1787158095.51 total 10.03 last 14.46)
 - mkdir 303 (time 1787158105.54 total 20.06 last 15.75)
 - mkdir 459 (time 1787158115.60 total 30.12 last 15.51)
 - mkdir 619 (time 1787158125.64 total 40.16 last 15.94)
 - mkdir 800 (time 1787158135.65 total 50.17 last 18.08)
 - mkdir 968 (time 1787158145.67 total 60.19 last 16.77)
total: 1000 mkdir in 62.08 seconds: 16.11 ops/second
 - mkdir 134 (time 1787158094.32 total 10.02 last 13.37)
 - mkdir 282 (time 1787158104.33 total 20.03 last 14.79)
 - mkdir 437 (time 1787158114.36 total 30.06 last 15.45)
 - mkdir 587 (time 1787158124.41 total 40.11 last 14.93)
 - mkdir 752 (time 1787158134.43 total 50.13 last 16.47)
 - mkdir 919 (time 1787158144.47 total 60.17 last 16.62)
total: 1000 mkdir in 66.00 seconds: 15.15 ops/second
 - mkdir 138 (time 1787158095.17 total 10.05 last 13.73)
 - mkdir 288 (time 1787158105.20 total 20.08 last 14.96)
 - mkdir 447 (time 1787158115.22 total 30.10 last 15.87)
 - mkdir 597 (time 1787158125.23 total 40.11 last 14.98)
 - mkdir 772 (time 1787158135.23 total 50.11 last 17.50)
 - mkdir 936 (time 1787158145.29 total 60.17 last 16.31)
total: 1000 mkdir in 65.44 seconds: 15.28 ops/second
 - mkdir 141 (time 1787158096.35 total 10.11 last 13.95)
 - mkdir 293 (time 1787158106.42 total 20.17 last 15.09)
 - mkdir 449 (time 1787158116.43 total 30.18 last 15.60)
 - mkdir 599 (time 1787158126.44 total 40.19 last 14.98)
 - mkdir 780 (time 1787158136.47 total 50.22 last 18.04)
 - mkdir 944 (time 1787158146.50 total 60.26 last 16.35)
total: 1000 mkdir in 65.77 seconds: 15.21 ops/second
 - mkdir 134 (time 1787158095.97 total 10.04 last 13.35)
 - mkdir 294 (time 1787158105.97 total 20.04 last 16.00)
 - mkdir 450 (time 1787158116.01 total 30.07 last 15.54)
 - mkdir 595 (time 1787158126.04 total 40.10 last 14.46)
 - mkdir 772 (time 1787158136.07 total 50.14 last 17.63)
 - mkdir 930 (time 1787158146.08 total 60.14 last 15.80)
total: 1000 mkdir in 66.42 seconds: 15.06 ops/second
 - mkdir 137 (time 1787158096.34 total 10.04 last 13.64)
 - mkdir 293 (time 1787158106.37 total 20.07 last 15.55)
 - mkdir 453 (time 1787158116.39 total 30.09 last 15.97)
 - mkdir 602 (time 1787158126.43 total 40.13 last 14.84)
 - mkdir 778 (time 1787158136.47 total 50.17 last 17.52)
 - mkdir 938 (time 1787158146.51 total 60.21 last 15.94)
total: 1000 mkdir in 66.45 seconds: 15.05 ops/second
 - mkdir 135 (time 1787158096.43 total 10.02 last 13.47)
 - mkdir 288 (time 1787158106.53 total 20.13 last 15.14)
 - mkdir 449 (time 1787158116.56 total 30.15 last 16.06)
 - mkdir 598 (time 1787158126.60 total 40.20 last 14.83)
 - mkdir 777 (time 1787158136.69 total 50.29 last 17.74)
 - mkdir 935 (time 1787158146.74 total 60.33 last 15.73)
total: 1000 mkdir in 66.94 seconds: 14.94 ops/second
 - mkdir 132 (time 1787158092.43 total 10.04 last 13.15)
 - mkdir 272 (time 1787158102.46 total 20.07 last 13.96)
 - mkdir 420 (time 1787158112.51 total 30.12 last 14.73)
 - mkdir 561 (time 1787158122.60 total 40.20 last 13.98)
 - mkdir 714 (time 1787158132.63 total 50.23 last 15.25)
 - mkdir 869 (time 1787158142.66 total 60.27 last 15.45)
 - mkdir 992 (time 1787158152.73 total 70.33 last 12.22)
total: 1000 mkdir in 71.22 seconds: 14.04 ops/second
 - mkdir 136 (time 1787158096.07 total 10.02 last 13.57)
 - mkdir 284 (time 1787158106.08 total 20.03 last 14.79)
 - mkdir 441 (time 1787158116.10 total 30.05 last 15.67)
 - mkdir 585 (time 1787158126.15 total 40.10 last 14.33)
 - mkdir 759 (time 1787158136.17 total 50.12 last 17.36)
 - mkdir 919 (time 1787158146.20 total 60.14 last 15.96)
total: 1000 mkdir in 67.85 seconds: 14.74 ops/second
 - mkdir 132 (time 1787158095.62 total 10.03 last 13.16)
 - mkdir 277 (time 1787158105.73 total 20.14 last 14.35)
 - mkdir 432 (time 1787158115.80 total 30.21 last 15.39)
 - mkdir 574 (time 1787158125.90 total 40.31 last 14.06)
 - mkdir 745 (time 1787158135.95 total 50.36 last 17.01)
 - mkdir 900 (time 1787158145.99 total 60.40 last 15.44)
total: 1000 mkdir in 69.60 seconds: 14.37 ops/second
Time for file creation 79 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787158162 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787158163 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787158164 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1787158164 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1787158165 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1787158165 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1787158164 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787158164 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787158165 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787158165 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
Time for file removal 48 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8