== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 02:35:22 (1786948522)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.44 seconds: 22.76 ops/second
 - mkdir 141 (time 1786948543.79 total 10.02 last 14.07)
 - mkdir 287 (time 1786948553.81 total 20.05 last 14.56)
 - mkdir 462 (time 1786948563.85 total 30.08 last 17.44)
 - mkdir 627 (time 1786948573.87 total 40.10 last 16.47)
 - mkdir 788 (time 1786948583.87 total 50.10 last 16.10)
 - mkdir 926 (time 1786948593.96 total 60.19 last 13.68)
total: 1000 mkdir in 66.64 seconds: 15.01 ops/second
 - mkdir 138 (time 1786948542.49 total 10.00 last 13.80)
 - mkdir 273 (time 1786948552.61 total 20.13 last 13.33)
 - mkdir 449 (time 1786948562.64 total 30.16 last 17.55)
 - mkdir 613 (time 1786948572.72 total 40.23 last 16.27)
 - mkdir 775 (time 1786948582.73 total 50.25 last 16.17)
 - mkdir 918 (time 1786948592.76 total 60.28 last 14.26)
total: 1000 mkdir in 68.89 seconds: 14.52 ops/second
 - mkdir 141 (time 1786948545.27 total 10.01 last 14.08)
 - mkdir 296 (time 1786948555.34 total 20.09 last 15.39)
 - mkdir 477 (time 1786948565.38 total 30.13 last 18.02)
 - mkdir 652 (time 1786948575.43 total 40.18 last 17.41)
 - mkdir 811 (time 1786948585.46 total 50.21 last 15.85)
 - mkdir 934 (time 1786948595.52 total 60.27 last 12.23)
total: 1000 mkdir in 66.20 seconds: 15.11 ops/second
 - mkdir 146 (time 1786948545.84 total 10.01 last 14.58)
 - mkdir 294 (time 1786948555.91 total 20.08 last 14.70)
 - mkdir 468 (time 1786948565.92 total 30.09 last 17.38)
 - mkdir 644 (time 1786948575.99 total 40.16 last 17.48)
 - mkdir 806 (time 1786948586.00 total 50.17 last 16.19)
 - mkdir 929 (time 1786948596.13 total 60.30 last 12.14)
total: 1000 mkdir in 67.30 seconds: 14.86 ops/second
 - mkdir 143 (time 1786948546.47 total 10.06 last 14.22)
 - mkdir 299 (time 1786948556.50 total 20.09 last 15.55)
 - mkdir 475 (time 1786948566.55 total 30.14 last 17.51)
 - mkdir 643 (time 1786948576.58 total 40.17 last 16.75)
 - mkdir 802 (time 1786948586.63 total 50.22 last 15.83)
 - mkdir 924 (time 1786948596.72 total 60.31 last 12.09)
total: 1000 mkdir in 67.48 seconds: 14.82 ops/second
 - mkdir 139 (time 1786948545.62 total 10.05 last 13.83)
 - mkdir 287 (time 1786948555.64 total 20.08 last 14.76)
 - mkdir 466 (time 1786948565.67 total 30.10 last 17.86)
 - mkdir 638 (time 1786948575.69 total 40.13 last 17.16)
 - mkdir 797 (time 1786948585.72 total 50.15 last 15.86)
 - mkdir 919 (time 1786948595.77 total 60.20 last 12.14)
total: 1000 mkdir in 68.57 seconds: 14.58 ops/second
 - mkdir 146 (time 1786948546.47 total 10.05 last 14.53)
 - mkdir 304 (time 1786948556.50 total 20.08 last 15.75)
 - mkdir 474 (time 1786948566.55 total 30.13 last 16.91)
 - mkdir 645 (time 1786948576.57 total 40.14 last 17.08)
 - mkdir 804 (time 1786948586.60 total 50.17 last 15.85)
 - mkdir 925 (time 1786948596.66 total 60.24 last 12.03)
total: 1000 mkdir in 67.90 seconds: 14.73 ops/second
 - mkdir 134 (time 1786948545.41 total 10.00 last 13.40)
 - mkdir 281 (time 1786948555.42 total 20.01 last 14.69)
 - mkdir 452 (time 1786948565.50 total 30.09 last 16.96)
 - mkdir 627 (time 1786948575.54 total 40.13 last 17.44)
 - mkdir 784 (time 1786948585.57 total 50.16 last 15.66)
 - mkdir 911 (time 1786948595.59 total 60.18 last 12.67)
total: 1000 mkdir in 69.63 seconds: 14.36 ops/second
 - mkdir 136 (time 1786948546.46 total 10.01 last 13.58)
 - mkdir 286 (time 1786948556.49 total 20.05 last 14.95)
 - mkdir 456 (time 1786948566.54 total 30.10 last 16.91)
 - mkdir 632 (time 1786948576.60 total 40.16 last 17.50)
 - mkdir 791 (time 1786948586.63 total 50.18 last 15.86)
 - mkdir 910 (time 1786948596.69 total 60.25 last 11.82)
 - mkdir 999 (time 1786948606.76 total 70.31 last 8.84)
total: 1000 mkdir in 70.31 seconds: 14.22 ops/second
 - mkdir 134 (time 1786948545.95 total 10.04 last 13.35)
 - mkdir 275 (time 1786948555.99 total 20.07 last 14.05)
 - mkdir 444 (time 1786948566.01 total 30.10 last 16.86)
 - mkdir 615 (time 1786948576.05 total 40.14 last 17.03)
 - mkdir 772 (time 1786948586.07 total 50.15 last 15.68)
 - mkdir 896 (time 1786948596.13 total 60.22 last 12.32)
 - mkdir 986 (time 1786948606.24 total 70.33 last 8.91)
total: 1000 mkdir in 72.11 seconds: 13.87 ops/second
Time for file creation 84 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 14 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.43 seconds: 23.43 ops/second
 - mkdir 198 (time 1786949134.61 total 10.01 last 19.78)
 - mkdir 386 (time 1786949144.62 total 20.02 last 18.79)
 - mkdir 600 (time 1786949154.66 total 30.06 last 21.32)
 - mkdir 772 (time 1786949164.70 total 40.10 last 17.14)
 - mkdir 917 (time 1786949174.77 total 50.17 last 14.40)
total: 1000 mkdir in 55.00 seconds: 18.18 ops/second
 - mkdir 190 (time 1786949133.45 total 10.01 last 18.98)
 - mkdir 364 (time 1786949143.45 total 20.01 last 17.39)
 - mkdir 575 (time 1786949153.49 total 30.05 last 21.02)
 - mkdir 746 (time 1786949163.55 total 40.11 last 17.00)
 - mkdir 902 (time 1786949173.56 total 50.12 last 15.59)
total: 1000 mkdir in 56.49 seconds: 17.70 ops/second
 - mkdir 183 (time 1786949133.48 total 10.04 last 18.22)
 - mkdir 365 (time 1786949143.49 total 20.05 last 18.18)
 - mkdir 571 (time 1786949153.51 total 30.08 last 20.55)
 - mkdir 748 (time 1786949163.56 total 40.13 last 17.61)
 - mkdir 899 (time 1786949173.59 total 50.16 last 15.06)
total: 1000 mkdir in 56.98 seconds: 17.55 ops/second
 - mkdir 187 (time 1786949133.92 total 10.02 last 18.66)
 - mkdir 365 (time 1786949143.92 total 20.02 last 17.79)
 - mkdir 574 (time 1786949153.95 total 30.05 last 20.84)
 - mkdir 748 (time 1786949164.01 total 40.12 last 17.29)
 - mkdir 900 (time 1786949174.10 total 50.21 last 15.07)
total: 1000 mkdir in 57.47 seconds: 17.40 ops/second
 - mkdir 199 (time 1786949135.03 total 10.00 last 19.90)
 - mkdir 388 (time 1786949145.10 total 20.07 last 18.78)
 - mkdir 594 (time 1786949155.10 total 30.07 last 20.60)
 - mkdir 763 (time 1786949165.12 total 40.09 last 16.87)
 - mkdir 905 (time 1786949175.18 total 50.15 last 14.11)
total: 1000 mkdir in 56.60 seconds: 17.67 ops/second
 - mkdir 154 (time 1786949131.62 total 10.03 last 15.35)
 - mkdir 333 (time 1786949141.73 total 20.14 last 17.70)
 - mkdir 541 (time 1786949151.75 total 30.16 last 20.76)
 - mkdir 716 (time 1786949161.83 total 40.24 last 17.36)
 - mkdir 852 (time 1786949171.87 total 50.28 last 13.54)
 - mkdir 986 (time 1786949181.88 total 60.29 last 13.39)
total: 1000 mkdir in 61.77 seconds: 16.19 ops/second
 - mkdir 190 (time 1786949135.15 total 10.04 last 18.93)
 - mkdir 377 (time 1786949145.22 total 20.10 last 18.58)
 - mkdir 584 (time 1786949155.24 total 30.13 last 20.65)
 - mkdir 751 (time 1786949165.27 total 40.15 last 16.66)
 - mkdir 890 (time 1786949175.40 total 50.29 last 13.71)
total: 1000 mkdir in 58.67 seconds: 17.05 ops/second
 - mkdir 187 (time 1786949135.04 total 10.05 last 18.61)
 - mkdir 372 (time 1786949145.09 total 20.10 last 18.41)
 - mkdir 572 (time 1786949155.15 total 30.16 last 19.88)
 - mkdir 743 (time 1786949165.17 total 40.17 last 17.07)
 - mkdir 880 (time 1786949175.18 total 50.18 last 13.69)
total: 1000 mkdir in 59.13 seconds: 16.91 ops/second
 - mkdir 192 (time 1786949135.09 total 10.06 last 19.09)
 - mkdir 380 (time 1786949145.11 total 20.07 last 18.77)
 - mkdir 574 (time 1786949155.13 total 30.10 last 19.36)
 - mkdir 737 (time 1786949165.15 total 40.11 last 16.27)
 - mkdir 878 (time 1786949175.18 total 50.15 last 14.05)
total: 1000 mkdir in 59.54 seconds: 16.80 ops/second
 - mkdir 180 (time 1786949134.35 total 10.03 last 17.94)
 - mkdir 357 (time 1786949144.35 total 20.03 last 17.70)
 - mkdir 554 (time 1786949154.38 total 30.06 last 19.64)
 - mkdir 718 (time 1786949164.40 total 40.08 last 16.36)
 - mkdir 852 (time 1786949174.48 total 50.17 last 13.29)
 - mkdir 979 (time 1786949184.52 total 60.20 last 12.66)
total: 1000 mkdir in 61.48 seconds: 16.27 ops/second
Time for file creation 73 sec for 1 RPC sent at a time
 - unlinked 0 (time 1786949192 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1786949193 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1786949194 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1786949195 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1786949194 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1786949195 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1786949195 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1786949195 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1786949195 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1786949196 ; total 1 ; last 1)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
Time for file removal 46 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8