== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 10:56:00 (1789138560)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.45 seconds: 22.10 ops/second
 - mkdir 153 (time 1789138580.34 total 10.01 last 15.29)
 - mkdir 309 (time 1789138590.35 total 20.01 last 15.59)
 - mkdir 458 (time 1789138600.44 total 30.10 last 14.77)
 - mkdir 592 (time 1789138610.45 total 40.12 last 13.38)
 - mkdir 748 (time 1789138620.47 total 50.14 last 15.57)
 - mkdir 948 (time 1789138630.54 total 60.20 last 19.87)
total: 1000 mkdir in 63.12 seconds: 15.84 ops/second
 - mkdir 148 (time 1789138580.61 total 10.04 last 14.74)
 - mkdir 299 (time 1789138590.64 total 20.07 last 15.05)
 - mkdir 443 (time 1789138600.65 total 30.08 last 14.39)
 - mkdir 577 (time 1789138610.67 total 40.11 last 13.37)
 - mkdir 735 (time 1789138620.73 total 50.16 last 15.71)
 - mkdir 929 (time 1789138630.76 total 60.19 last 19.35)
total: 1000 mkdir in 64.84 seconds: 15.42 ops/second
 - mkdir 153 (time 1789138582.81 total 10.01 last 15.28)
 - mkdir 317 (time 1789138592.81 total 20.02 last 16.39)
 - mkdir 459 (time 1789138602.85 total 30.06 last 14.15)
 - mkdir 603 (time 1789138612.86 total 40.06 last 14.39)
 - mkdir 763 (time 1789138622.86 total 50.07 last 15.99)
 - mkdir 955 (time 1789138632.88 total 60.09 last 19.16)
total: 1000 mkdir in 64.02 seconds: 15.62 ops/second
 - mkdir 143 (time 1789138582.31 total 10.05 last 14.23)
 - mkdir 301 (time 1789138592.35 total 20.09 last 15.74)
 - mkdir 442 (time 1789138602.45 total 30.18 last 13.97)
 - mkdir 581 (time 1789138612.46 total 40.20 last 13.88)
 - mkdir 740 (time 1789138622.52 total 50.25 last 15.82)
 - mkdir 939 (time 1789138632.53 total 60.26 last 19.88)
total: 1000 mkdir in 66.27 seconds: 15.09 ops/second
 - mkdir 144 (time 1789138582.74 total 10.05 last 14.32)
 - mkdir 302 (time 1789138592.80 total 20.11 last 15.71)
 - mkdir 442 (time 1789138602.80 total 30.12 last 13.99)
 - mkdir 586 (time 1789138612.81 total 40.12 last 14.40)
 - mkdir 751 (time 1789138622.86 total 50.17 last 16.42)
 - mkdir 941 (time 1789138632.90 total 60.21 last 18.91)
total: 1000 mkdir in 66.21 seconds: 15.10 ops/second
 - mkdir 147 (time 1789138583.04 total 10.02 last 14.67)
 - mkdir 302 (time 1789138593.10 total 20.08 last 15.41)
 - mkdir 444 (time 1789138603.18 total 30.16 last 14.09)
 - mkdir 594 (time 1789138613.20 total 40.18 last 14.97)
 - mkdir 757 (time 1789138623.23 total 50.21 last 16.24)
 - mkdir 946 (time 1789138633.23 total 60.21 last 18.90)
total: 1000 mkdir in 66.12 seconds: 15.12 ops/second
 - mkdir 146 (time 1789138582.27 total 10.04 last 14.54)
 - mkdir 303 (time 1789138592.31 total 20.08 last 15.64)
 - mkdir 439 (time 1789138602.45 total 30.22 last 13.42)
 - mkdir 580 (time 1789138612.49 total 40.26 last 14.04)
 - mkdir 741 (time 1789138622.53 total 50.30 last 16.03)
 - mkdir 932 (time 1789138632.57 total 60.34 last 19.04)
total: 1000 mkdir in 66.95 seconds: 14.94 ops/second
 - mkdir 147 (time 1789138582.28 total 10.04 last 14.64)
 - mkdir 300 (time 1789138592.31 total 20.07 last 15.26)
 - mkdir 440 (time 1789138602.41 total 30.18 last 13.85)
 - mkdir 582 (time 1789138612.43 total 40.19 last 14.18)
 - mkdir 741 (time 1789138622.47 total 50.23 last 15.84)
 - mkdir 933 (time 1789138632.49 total 60.25 last 19.16)
total: 1000 mkdir in 67.03 seconds: 14.92 ops/second
 - mkdir 138 (time 1789138582.15 total 10.07 last 13.71)
 - mkdir 289 (time 1789138592.19 total 20.11 last 15.03)
 - mkdir 428 (time 1789138602.22 total 30.14 last 13.86)
 - mkdir 569 (time 1789138612.24 total 40.16 last 14.08)
 - mkdir 728 (time 1789138622.28 total 50.20 last 15.83)
 - mkdir 924 (time 1789138632.30 total 60.22 last 19.57)
total: 1000 mkdir in 67.37 seconds: 14.84 ops/second
 - mkdir 138 (time 1789138582.65 total 10.02 last 13.78)
 - mkdir 286 (time 1789138592.68 total 20.04 last 14.76)
 - mkdir 419 (time 1789138602.73 total 30.09 last 13.24)
 - mkdir 569 (time 1789138612.78 total 40.14 last 14.92)
 - mkdir 730 (time 1789138622.80 total 50.17 last 16.06)
 - mkdir 925 (time 1789138632.85 total 60.21 last 19.42)
total: 1000 mkdir in 67.27 seconds: 14.87 ops/second
Time for file creation 77 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 14 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.54 seconds: 18.67 ops/second
 - mkdir 146 (time 1789139175.47 total 10.02 last 14.58)
 - mkdir 325 (time 1789139185.53 total 20.08 last 17.79)
 - mkdir 475 (time 1789139195.55 total 30.10 last 14.97)
 - mkdir 677 (time 1789139205.56 total 40.11 last 20.18)
 - mkdir 870 (time 1789139215.58 total 50.13 last 19.26)
total: 1000 mkdir in 58.90 seconds: 16.98 ops/second
 - mkdir 157 (time 1789139177.44 total 10.04 last 15.64)
 - mkdir 333 (time 1789139187.52 total 20.12 last 17.46)
 - mkdir 494 (time 1789139197.61 total 30.21 last 15.96)
 - mkdir 704 (time 1789139207.64 total 40.24 last 20.93)
 - mkdir 893 (time 1789139217.66 total 50.26 last 18.86)
total: 1000 mkdir in 58.08 seconds: 17.22 ops/second
 - mkdir 163 (time 1789139177.91 total 10.08 last 16.16)
 - mkdir 341 (time 1789139187.98 total 20.15 last 17.69)
 - mkdir 494 (time 1789139198.05 total 30.22 last 15.20)
 - mkdir 709 (time 1789139208.08 total 40.25 last 21.43)
 - mkdir 899 (time 1789139218.10 total 50.27 last 18.96)
total: 1000 mkdir in 57.71 seconds: 17.33 ops/second
 - mkdir 160 (time 1789139178.37 total 10.02 last 15.97)
 - mkdir 331 (time 1789139188.37 total 20.02 last 17.09)
 - mkdir 483 (time 1789139198.45 total 30.10 last 15.08)
 - mkdir 700 (time 1789139208.46 total 40.11 last 21.68)
 - mkdir 892 (time 1789139218.49 total 50.13 last 19.16)
total: 1000 mkdir in 58.84 seconds: 16.99 ops/second
 - mkdir 146 (time 1789139176.92 total 10.01 last 14.59)
 - mkdir 320 (time 1789139186.95 total 20.04 last 17.34)
 - mkdir 471 (time 1789139197.02 total 30.11 last 15.00)
 - mkdir 684 (time 1789139207.04 total 40.13 last 21.25)
 - mkdir 874 (time 1789139217.08 total 50.18 last 18.92)
 - mkdir 996 (time 1789139227.16 total 60.25 last 12.10)
total: 1000 mkdir in 60.51 seconds: 16.52 ops/second
 - mkdir 138 (time 1789139176.16 total 10.05 last 13.74)
 - mkdir 307 (time 1789139186.19 total 20.08 last 16.85)
 - mkdir 454 (time 1789139196.24 total 30.12 last 14.63)
 - mkdir 661 (time 1789139206.26 total 40.15 last 20.64)
 - mkdir 855 (time 1789139216.29 total 50.17 last 19.35)
 - mkdir 990 (time 1789139226.38 total 60.27 last 13.37)
total: 1000 mkdir in 61.36 seconds: 16.30 ops/second
 - mkdir 151 (time 1789139178.24 total 10.03 last 15.06)
 - mkdir 323 (time 1789139188.28 total 20.07 last 17.13)
 - mkdir 476 (time 1789139198.33 total 30.11 last 15.24)
 - mkdir 682 (time 1789139208.34 total 40.12 last 20.58)
 - mkdir 877 (time 1789139218.35 total 50.13 last 19.48)
 - mkdir 995 (time 1789139228.44 total 60.22 last 11.69)
total: 1000 mkdir in 60.69 seconds: 16.48 ops/second
 - mkdir 140 (time 1789139177.36 total 10.04 last 13.94)
 - mkdir 309 (time 1789139187.40 total 20.09 last 16.83)
 - mkdir 461 (time 1789139197.43 total 30.11 last 15.16)
 - mkdir 670 (time 1789139207.43 total 40.11 last 20.90)
 - mkdir 863 (time 1789139217.46 total 50.15 last 19.23)
 - mkdir 985 (time 1789139227.50 total 60.19 last 12.15)
total: 1000 mkdir in 61.97 seconds: 16.14 ops/second
 - mkdir 153 (time 1789139178.30 total 10.05 last 15.22)
 - mkdir 322 (time 1789139188.31 total 20.06 last 16.89)
 - mkdir 469 (time 1789139198.32 total 30.07 last 14.68)
 - mkdir 679 (time 1789139208.35 total 40.10 last 20.93)
 - mkdir 871 (time 1789139218.40 total 50.15 last 19.12)
 - mkdir 986 (time 1789139228.46 total 60.21 last 11.43)
total: 1000 mkdir in 61.92 seconds: 16.15 ops/second
 - mkdir 148 (time 1789139178.19 total 10.06 last 14.72)
 - mkdir 312 (time 1789139188.20 total 20.06 last 16.39)
 - mkdir 463 (time 1789139198.22 total 30.08 last 15.07)
 - mkdir 670 (time 1789139208.22 total 40.09 last 20.69)
 - mkdir 863 (time 1789139218.28 total 50.15 last 19.18)
 - mkdir 978 (time 1789139228.34 total 60.21 last 11.43)
total: 1000 mkdir in 62.66 seconds: 15.96 ops/second
Time for file creation 73 sec for 1 RPC sent at a time
 - unlinked 0 (time 1789139239 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1789139239 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1789139239 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1789139239 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1789139239 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1789139240 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1789139241 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1789139241 ; total 1 ; last 1)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1789139241 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1789139241 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
Time for file removal 41 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8