== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 23:21:03 (1785986463)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.49 seconds: 20.56 ops/second
 - mkdir 141 (time 1785986486.18 total 10.08 last 13.99)
 - mkdir 328 (time 1785986496.20 total 20.10 last 18.66)
 - mkdir 502 (time 1785986506.20 total 30.11 last 17.39)
 - mkdir 660 (time 1785986516.25 total 40.15 last 15.73)
 - mkdir 794 (time 1785986526.27 total 50.17 last 13.36)
 - mkdir 921 (time 1785986536.30 total 60.20 last 12.67)
total: 1000 mkdir in 69.80 seconds: 14.33 ops/second
 - mkdir 140 (time 1785986486.37 total 10.05 last 13.93)
 - mkdir 326 (time 1785986496.39 total 20.07 last 18.56)
 - mkdir 498 (time 1785986506.40 total 30.08 last 17.18)
 - mkdir 654 (time 1785986516.46 total 40.14 last 15.51)
 - mkdir 790 (time 1785986526.49 total 50.17 last 13.57)
 - mkdir 919 (time 1785986536.58 total 60.26 last 12.78)
total: 1000 mkdir in 69.61 seconds: 14.37 ops/second
 - mkdir 132 (time 1785986484.02 total 10.05 last 13.13)
 - mkdir 311 (time 1785986494.11 total 20.14 last 17.74)
 - mkdir 477 (time 1785986504.16 total 30.19 last 16.52)
 - mkdir 637 (time 1785986514.16 total 40.19 last 16.00)
 - mkdir 761 (time 1785986524.18 total 50.21 last 12.38)
 - mkdir 893 (time 1785986534.19 total 60.22 last 13.18)
 - mkdir 978 (time 1785986544.21 total 70.24 last 8.49)
total: 1000 mkdir in 72.98 seconds: 13.70 ops/second
 - mkdir 145 (time 1785986486.50 total 10.03 last 14.45)
 - mkdir 331 (time 1785986496.50 total 20.03 last 18.60)
 - mkdir 502 (time 1785986506.51 total 30.05 last 17.08)
 - mkdir 649 (time 1785986516.53 total 40.07 last 14.67)
 - mkdir 778 (time 1785986526.54 total 50.08 last 12.89)
 - mkdir 904 (time 1785986536.65 total 60.18 last 12.47)
 - mkdir 990 (time 1785986546.69 total 70.22 last 8.57)
total: 1000 mkdir in 71.65 seconds: 13.96 ops/second
 - mkdir 139 (time 1785986486.75 total 10.04 last 13.84)
 - mkdir 322 (time 1785986496.78 total 20.08 last 18.24)
 - mkdir 500 (time 1785986506.79 total 30.09 last 17.78)
 - mkdir 643 (time 1785986516.83 total 40.13 last 14.25)
 - mkdir 776 (time 1785986526.88 total 50.17 last 13.24)
 - mkdir 904 (time 1785986536.91 total 60.20 last 12.77)
 - mkdir 985 (time 1785986546.93 total 70.22 last 8.08)
total: 1000 mkdir in 72.54 seconds: 13.79 ops/second
 - mkdir 140 (time 1785986486.77 total 10.05 last 13.92)
 - mkdir 327 (time 1785986496.80 total 20.09 last 18.63)
 - mkdir 503 (time 1785986506.82 total 30.10 last 17.58)
 - mkdir 647 (time 1785986516.85 total 40.13 last 14.35)
 - mkdir 782 (time 1785986526.90 total 50.19 last 13.42)
 - mkdir 904 (time 1785986536.99 total 60.28 last 12.09)
 - mkdir 982 (time 1785986547.06 total 70.35 last 7.75)
total: 1000 mkdir in 73.43 seconds: 13.62 ops/second
 - mkdir 139 (time 1785986486.90 total 10.05 last 13.84)
 - mkdir 323 (time 1785986496.96 total 20.10 last 18.30)
 - mkdir 498 (time 1785986506.97 total 30.11 last 17.47)
 - mkdir 639 (time 1785986516.98 total 40.12 last 14.09)
 - mkdir 768 (time 1785986527.04 total 50.18 last 12.82)
 - mkdir 895 (time 1785986537.17 total 60.31 last 12.53)
 - mkdir 980 (time 1785986547.40 total 70.54 last 8.31)
total: 1000 mkdir in 73.47 seconds: 13.61 ops/second
 - mkdir 138 (time 1785986487.26 total 10.06 last 13.72)
 - mkdir 325 (time 1785986497.28 total 20.09 last 18.65)
 - mkdir 509 (time 1785986507.30 total 30.11 last 18.36)
 - mkdir 644 (time 1785986517.37 total 40.18 last 13.40)
 - mkdir 777 (time 1785986527.40 total 50.20 last 13.27)
 - mkdir 901 (time 1785986537.41 total 60.21 last 12.39)
 - mkdir 983 (time 1785986547.55 total 70.35 last 8.09)
total: 1000 mkdir in 73.14 seconds: 13.67 ops/second
 - mkdir 137 (time 1785986486.35 total 10.02 last 13.67)
 - mkdir 316 (time 1785986496.40 total 20.06 last 17.83)
 - mkdir 480 (time 1785986506.40 total 30.07 last 16.39)
 - mkdir 636 (time 1785986516.48 total 40.15 last 15.48)
 - mkdir 764 (time 1785986526.49 total 50.16 last 12.79)
 - mkdir 891 (time 1785986536.56 total 60.23 last 12.61)
 - mkdir 975 (time 1785986546.87 total 70.54 last 8.15)
total: 1000 mkdir in 74.10 seconds: 13.50 ops/second
 - mkdir 132 (time 1785986485.44 total 10.06 last 13.12)
 - mkdir 307 (time 1785986495.45 total 20.08 last 17.48)
 - mkdir 462 (time 1785986505.50 total 30.12 last 15.43)
 - mkdir 621 (time 1785986515.54 total 40.17 last 15.83)
 - mkdir 751 (time 1785986525.60 total 50.23 last 12.92)
 - mkdir 886 (time 1785986535.61 total 60.24 last 13.49)
 - mkdir 960 (time 1785986545.71 total 70.34 last 7.33)
total: 1000 mkdir in 76.92 seconds: 13.00 ops/second
Time for file creation 87 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 11 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.65 seconds: 15.28 ops/second
 - mkdir 169 (time 1785987082.32 total 10.03 last 16.85)
 - mkdir 320 (time 1785987092.37 total 20.08 last 15.02)
 - mkdir 466 (time 1785987102.43 total 30.13 last 14.52)
 - mkdir 637 (time 1785987112.47 total 40.17 last 17.03)
 - mkdir 826 (time 1785987122.62 total 50.33 last 18.61)
 - mkdir 935 (time 1785987132.64 total 60.35 last 10.88)
total: 1000 mkdir in 66.76 seconds: 14.98 ops/second
 - mkdir 170 (time 1785987082.92 total 10.03 last 16.94)
 - mkdir 322 (time 1785987092.96 total 20.08 last 15.13)
 - mkdir 474 (time 1785987103.02 total 30.13 last 15.12)
 - mkdir 646 (time 1785987113.11 total 40.22 last 17.05)
 - mkdir 824 (time 1785987123.13 total 50.25 last 17.76)
 - mkdir 934 (time 1785987133.17 total 60.29 last 10.95)
total: 1000 mkdir in 67.26 seconds: 14.87 ops/second
 - mkdir 175 (time 1785987083.29 total 10.09 last 17.35)
 - mkdir 327 (time 1785987093.33 total 20.13 last 15.14)
 - mkdir 474 (time 1785987103.33 total 30.13 last 14.70)
 - mkdir 645 (time 1785987113.35 total 40.15 last 17.05)
 - mkdir 818 (time 1785987123.39 total 50.19 last 17.24)
 - mkdir 931 (time 1785987133.45 total 60.25 last 11.23)
total: 1000 mkdir in 69.07 seconds: 14.48 ops/second
 - mkdir 170 (time 1785987082.32 total 10.11 last 16.81)
 - mkdir 314 (time 1785987092.43 total 20.22 last 14.24)
 - mkdir 465 (time 1785987102.50 total 30.29 last 15.00)
 - mkdir 633 (time 1785987112.55 total 40.35 last 16.71)
 - mkdir 818 (time 1785987122.59 total 50.38 last 18.43)
 - mkdir 927 (time 1785987132.71 total 60.50 last 10.77)
total: 1000 mkdir in 70.12 seconds: 14.26 ops/second
 - mkdir 175 (time 1785987083.49 total 10.01 last 17.48)
 - mkdir 327 (time 1785987093.55 total 20.07 last 15.11)
 - mkdir 470 (time 1785987103.61 total 30.13 last 14.22)
 - mkdir 640 (time 1785987113.62 total 40.14 last 16.98)
 - mkdir 819 (time 1785987123.65 total 50.17 last 17.85)
 - mkdir 932 (time 1785987133.67 total 60.19 last 11.28)
total: 1000 mkdir in 69.19 seconds: 14.45 ops/second
 - mkdir 164 (time 1785987080.79 total 10.04 last 16.34)
 - mkdir 302 (time 1785987090.80 total 20.05 last 13.78)
 - mkdir 446 (time 1785987100.93 total 30.17 last 14.23)
 - mkdir 599 (time 1785987110.95 total 40.20 last 15.26)
 - mkdir 789 (time 1785987120.98 total 50.23 last 18.94)
 - mkdir 906 (time 1785987131.05 total 60.29 last 11.62)
 - mkdir 990 (time 1785987141.27 total 70.51 last 8.22)
total: 1000 mkdir in 72.59 seconds: 13.78 ops/second
 - mkdir 168 (time 1785987083.37 total 10.03 last 16.75)
 - mkdir 318 (time 1785987093.38 total 20.04 last 14.99)
 - mkdir 462 (time 1785987103.45 total 30.11 last 14.30)
 - mkdir 636 (time 1785987113.51 total 40.17 last 17.29)
 - mkdir 816 (time 1785987123.62 total 50.28 last 17.80)
 - mkdir 930 (time 1785987133.72 total 60.37 last 11.29)
 - mkdir 999 (time 1785987143.79 total 70.45 last 6.85)
total: 1000 mkdir in 70.45 seconds: 14.19 ops/second
 - mkdir 172 (time 1785987083.36 total 10.04 last 17.12)
 - mkdir 319 (time 1785987093.36 total 20.05 last 14.69)
 - mkdir 466 (time 1785987103.38 total 30.07 last 14.68)
 - mkdir 635 (time 1785987113.39 total 40.08 last 16.88)
 - mkdir 813 (time 1785987123.41 total 50.10 last 17.75)
 - mkdir 926 (time 1785987133.41 total 60.10 last 11.30)
 - mkdir 997 (time 1785987143.47 total 70.16 last 7.06)
total: 1000 mkdir in 70.91 seconds: 14.10 ops/second
 - mkdir 166 (time 1785987082.95 total 10.09 last 16.44)
 - mkdir 312 (time 1785987092.98 total 20.12 last 14.56)
 - mkdir 454 (time 1785987103.01 total 30.15 last 14.16)
 - mkdir 619 (time 1785987113.02 total 40.16 last 16.49)
 - mkdir 799 (time 1785987123.08 total 50.22 last 17.89)
 - mkdir 910 (time 1785987133.13 total 60.28 last 11.03)
 - mkdir 984 (time 1785987143.32 total 70.46 last 7.27)
total: 1000 mkdir in 73.41 seconds: 13.62 ops/second
 - mkdir 166 (time 1785987082.95 total 10.07 last 16.48)
 - mkdir 307 (time 1785987093.02 total 20.14 last 14.00)
 - mkdir 446 (time 1785987103.05 total 30.17 last 13.86)
 - mkdir 617 (time 1785987113.07 total 40.19 last 17.06)
 - mkdir 796 (time 1785987123.13 total 50.25 last 17.79)
 - mkdir 908 (time 1785987133.15 total 60.27 last 11.18)
 - mkdir 980 (time 1785987143.34 total 70.46 last 7.07)
total: 1000 mkdir in 74.19 seconds: 13.48 ops/second
Time for file creation 87 sec for 1 RPC sent at a time
 - unlinked 0 (time 1785987155 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1785987157 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1785987158 ; total 1 ; last 1)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1785987157 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1785987158 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1785987157 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1785987157 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1785987158 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1785987158 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1785987158 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
Time for file removal 42 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8