== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 10:25:53 (1787581553)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.66 seconds: 15.08 ops/second
 - mkdir 157 (time 1787581574.95 total 10.06 last 15.61)
 - mkdir 320 (time 1787581585.02 total 20.13 last 16.19)
 - mkdir 454 (time 1787581595.08 total 30.19 last 13.32)
 - mkdir 605 (time 1787581605.14 total 40.25 last 15.01)
 - mkdir 792 (time 1787581615.18 total 50.29 last 18.63)
 - mkdir 959 (time 1787581625.21 total 60.32 last 16.65)
total: 1000 mkdir in 62.54 seconds: 15.99 ops/second
 - mkdir 154 (time 1787581575.24 total 10.03 last 15.36)
 - mkdir 315 (time 1787581585.31 total 20.09 last 15.99)
 - mkdir 446 (time 1787581595.32 total 30.10 last 13.08)
 - mkdir 595 (time 1787581605.41 total 40.19 last 14.77)
 - mkdir 786 (time 1787581615.42 total 50.20 last 19.07)
 - mkdir 948 (time 1787581625.46 total 60.24 last 16.14)
total: 1000 mkdir in 63.51 seconds: 15.75 ops/second
 - mkdir 156 (time 1787581575.67 total 10.09 last 15.46)
 - mkdir 309 (time 1787581585.71 total 20.12 last 15.25)
 - mkdir 444 (time 1787581595.72 total 30.14 last 13.48)
 - mkdir 594 (time 1787581605.73 total 40.15 last 14.99)
 - mkdir 790 (time 1787581615.74 total 50.16 last 19.57)
 - mkdir 951 (time 1787581625.76 total 60.18 last 16.07)
total: 1000 mkdir in 63.73 seconds: 15.69 ops/second
 - mkdir 151 (time 1787581575.74 total 10.05 last 15.03)
 - mkdir 311 (time 1787581585.76 total 20.07 last 15.96)
 - mkdir 438 (time 1787581595.84 total 30.15 last 12.60)
 - mkdir 592 (time 1787581605.86 total 40.17 last 15.37)
 - mkdir 788 (time 1787581615.95 total 50.26 last 19.43)
 - mkdir 952 (time 1787581625.96 total 60.27 last 16.39)
total: 1000 mkdir in 64.17 seconds: 15.58 ops/second
 - mkdir 154 (time 1787581575.67 total 10.03 last 15.35)
 - mkdir 315 (time 1787581585.68 total 20.04 last 16.08)
 - mkdir 443 (time 1787581595.70 total 30.06 last 12.77)
 - mkdir 592 (time 1787581605.71 total 40.08 last 14.88)
 - mkdir 786 (time 1787581615.74 total 50.11 last 19.34)
 - mkdir 942 (time 1787581625.77 total 60.14 last 15.55)
total: 1000 mkdir in 64.42 seconds: 15.52 ops/second
 - mkdir 157 (time 1787581575.66 total 10.08 last 15.58)
 - mkdir 316 (time 1787581585.68 total 20.10 last 15.87)
 - mkdir 446 (time 1787581595.71 total 30.13 last 12.96)
 - mkdir 591 (time 1787581605.74 total 40.15 last 14.47)
 - mkdir 787 (time 1787581615.74 total 50.16 last 19.59)
 - mkdir 944 (time 1787581625.75 total 60.16 last 15.70)
total: 1000 mkdir in 64.48 seconds: 15.51 ops/second
 - mkdir 154 (time 1787581576.23 total 10.05 last 15.32)
 - mkdir 316 (time 1787581586.34 total 20.16 last 16.03)
 - mkdir 450 (time 1787581596.42 total 30.25 last 13.29)
 - mkdir 602 (time 1787581606.44 total 40.26 last 15.18)
 - mkdir 792 (time 1787581616.45 total 50.28 last 18.97)
 - mkdir 957 (time 1787581626.48 total 60.30 last 16.47)
total: 1000 mkdir in 64.48 seconds: 15.51 ops/second
 - mkdir 153 (time 1787581575.29 total 10.03 last 15.25)
 - mkdir 306 (time 1787581585.35 total 20.10 last 15.21)
 - mkdir 436 (time 1787581595.37 total 30.12 last 12.97)
 - mkdir 587 (time 1787581605.41 total 40.16 last 15.05)
 - mkdir 774 (time 1787581615.45 total 50.19 last 18.63)
 - mkdir 934 (time 1787581625.46 total 60.21 last 15.98)
total: 1000 mkdir in 65.69 seconds: 15.22 ops/second
 - mkdir 155 (time 1787581576.39 total 10.07 last 15.39)
 - mkdir 311 (time 1787581586.40 total 20.07 last 15.59)
 - mkdir 435 (time 1787581596.44 total 30.12 last 12.35)
 - mkdir 588 (time 1787581606.47 total 40.15 last 15.26)
 - mkdir 779 (time 1787581616.47 total 50.15 last 19.10)
 - mkdir 943 (time 1787581626.49 total 60.17 last 16.37)
total: 1000 mkdir in 65.97 seconds: 15.16 ops/second
 - mkdir 148 (time 1787581576.20 total 10.06 last 14.71)
 - mkdir 297 (time 1787581586.27 total 20.13 last 14.80)
 - mkdir 418 (time 1787581596.28 total 30.14 last 12.09)
 - mkdir 572 (time 1787581606.33 total 40.19 last 15.32)
 - mkdir 764 (time 1787581616.36 total 50.22 last 19.14)
 - mkdir 925 (time 1787581626.38 total 60.24 last 16.06)
total: 1000 mkdir in 67.46 seconds: 14.82 ops/second
Time for file creation 78 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 14 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.64 seconds: 15.59 ops/second
 - mkdir 144 (time 1787582148.48 total 10.04 last 14.34)
 - mkdir 321 (time 1787582158.48 total 20.04 last 17.70)
 - mkdir 496 (time 1787582168.52 total 30.08 last 17.43)
 - mkdir 674 (time 1787582178.57 total 40.13 last 17.71)
 - mkdir 876 (time 1787582188.57 total 50.14 last 20.19)
total: 1000 mkdir in 57.41 seconds: 17.42 ops/second
 - mkdir 131 (time 1787582146.74 total 10.04 last 13.05)
 - mkdir 304 (time 1787582156.75 total 20.04 last 17.29)
 - mkdir 469 (time 1787582166.79 total 30.08 last 16.44)
 - mkdir 642 (time 1787582176.87 total 40.16 last 17.16)
 - mkdir 829 (time 1787582186.94 total 50.24 last 18.56)
 - mkdir 993 (time 1787582197.04 total 60.34 last 16.24)
total: 1000 mkdir in 60.86 seconds: 16.43 ops/second
 - mkdir 136 (time 1787582148.83 total 10.00 last 13.60)
 - mkdir 311 (time 1787582158.85 total 20.02 last 17.47)
 - mkdir 479 (time 1787582168.92 total 30.09 last 16.68)
 - mkdir 656 (time 1787582178.98 total 40.15 last 17.60)
 - mkdir 857 (time 1787582188.98 total 50.15 last 20.09)
total: 1000 mkdir in 59.86 seconds: 16.71 ops/second
 - mkdir 130 (time 1787582148.60 total 10.09 last 12.88)
 - mkdir 303 (time 1787582158.62 total 20.11 last 17.27)
 - mkdir 465 (time 1787582168.65 total 30.14 last 16.15)
 - mkdir 646 (time 1787582178.66 total 40.15 last 18.07)
 - mkdir 844 (time 1787582188.71 total 50.20 last 19.70)
 - mkdir 994 (time 1787582198.89 total 60.38 last 14.74)
total: 1000 mkdir in 61.02 seconds: 16.39 ops/second
 - mkdir 140 (time 1787582149.15 total 10.07 last 13.90)
 - mkdir 318 (time 1787582159.16 total 20.08 last 17.78)
 - mkdir 481 (time 1787582169.20 total 30.12 last 16.24)
 - mkdir 656 (time 1787582179.20 total 40.12 last 17.50)
 - mkdir 851 (time 1787582189.20 total 50.12 last 19.50)
 - mkdir 996 (time 1787582199.25 total 60.18 last 14.42)
total: 1000 mkdir in 60.66 seconds: 16.48 ops/second
 - mkdir 134 (time 1787582149.40 total 10.03 last 13.36)
 - mkdir 312 (time 1787582159.41 total 20.04 last 17.78)
 - mkdir 476 (time 1787582169.43 total 30.06 last 16.36)
 - mkdir 657 (time 1787582179.47 total 40.10 last 18.04)
 - mkdir 853 (time 1787582189.54 total 50.18 last 19.45)
 - mkdir 994 (time 1787582199.66 total 60.29 last 13.94)
total: 1000 mkdir in 61.05 seconds: 16.38 ops/second
 - mkdir 136 (time 1787582149.80 total 10.06 last 13.52)
 - mkdir 312 (time 1787582159.82 total 20.08 last 17.57)
 - mkdir 475 (time 1787582169.84 total 30.09 last 16.27)
 - mkdir 655 (time 1787582179.86 total 40.12 last 17.96)
 - mkdir 848 (time 1787582189.95 total 50.20 last 19.14)
 - mkdir 984 (time 1787582200.07 total 60.32 last 13.44)
total: 1000 mkdir in 62.24 seconds: 16.07 ops/second
 - mkdir 137 (time 1787582149.48 total 10.06 last 13.61)
 - mkdir 312 (time 1787582159.50 total 20.08 last 17.47)
 - mkdir 473 (time 1787582169.55 total 30.13 last 16.02)
 - mkdir 646 (time 1787582179.58 total 40.16 last 17.24)
 - mkdir 838 (time 1787582189.63 total 50.22 last 19.10)
 - mkdir 977 (time 1787582199.67 total 60.25 last 13.85)
total: 1000 mkdir in 62.82 seconds: 15.92 ops/second
 - mkdir 129 (time 1787582149.51 total 10.00 last 12.90)
 - mkdir 307 (time 1787582159.63 total 20.13 last 17.59)
 - mkdir 470 (time 1787582169.66 total 30.16 last 16.25)
 - mkdir 646 (time 1787582179.68 total 40.18 last 17.56)
 - mkdir 835 (time 1787582189.71 total 50.21 last 18.84)
 - mkdir 973 (time 1787582199.76 total 60.26 last 13.73)
total: 1000 mkdir in 63.26 seconds: 15.81 ops/second
 - mkdir 126 (time 1787582148.68 total 10.01 last 12.58)
 - mkdir 288 (time 1787582158.69 total 20.02 last 16.19)
 - mkdir 447 (time 1787582168.71 total 30.05 last 15.86)
 - mkdir 610 (time 1787582178.77 total 40.10 last 16.21)
 - mkdir 793 (time 1787582188.79 total 50.12 last 18.27)
 - mkdir 943 (time 1787582198.87 total 60.21 last 14.87)
total: 1000 mkdir in 65.75 seconds: 15.21 ops/second
Time for file creation 76 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787582212 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1787582213 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1787582214 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1787582214 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1787582214 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1787582214 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1787582214 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1787582214 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787582214 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787582214 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
Time for file removal 44 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8