== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 15:45:25 (1789069525)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.59 seconds: 16.93 ops/second
 - mkdir 155 (time 1789069546.72 total 10.11 last 15.33)
 - mkdir 295 (time 1789069556.75 total 20.13 last 13.97)
 - mkdir 444 (time 1789069566.81 total 30.20 last 14.80)
 - mkdir 559 (time 1789069576.92 total 40.30 last 11.38)
 - mkdir 692 (time 1789069586.95 total 50.34 last 13.25)
 - mkdir 800 (time 1789069596.99 total 60.38 last 10.76)
 - mkdir 928 (time 1789069607.06 total 70.44 last 12.71)
total: 1000 mkdir in 76.14 seconds: 13.13 ops/second
 - mkdir 147 (time 1789069547.40 total 10.01 last 14.68)
 - mkdir 292 (time 1789069557.41 total 20.02 last 14.48)
 - mkdir 443 (time 1789069567.53 total 30.13 last 14.93)
 - mkdir 560 (time 1789069577.57 total 40.18 last 11.65)
 - mkdir 686 (time 1789069587.59 total 50.20 last 12.57)
 - mkdir 796 (time 1789069597.61 total 60.22 last 10.98)
 - mkdir 930 (time 1789069607.66 total 70.27 last 13.33)
total: 1000 mkdir in 76.07 seconds: 13.15 ops/second
 - mkdir 154 (time 1789069546.16 total 10.03 last 15.36)
 - mkdir 293 (time 1789069556.25 total 20.12 last 13.77)
 - mkdir 439 (time 1789069566.29 total 30.16 last 14.55)
 - mkdir 552 (time 1789069576.33 total 40.20 last 11.25)
 - mkdir 685 (time 1789069586.35 total 50.21 last 13.28)
 - mkdir 790 (time 1789069596.35 total 60.21 last 10.50)
 - mkdir 913 (time 1789069606.38 total 70.25 last 12.26)
total: 1000 mkdir in 78.02 seconds: 12.82 ops/second
 - mkdir 152 (time 1789069547.30 total 10.08 last 15.07)
 - mkdir 297 (time 1789069557.34 total 20.13 last 14.44)
 - mkdir 436 (time 1789069567.49 total 30.27 last 13.70)
 - mkdir 549 (time 1789069577.56 total 40.34 last 11.22)
 - mkdir 681 (time 1789069587.58 total 50.36 last 13.18)
 - mkdir 795 (time 1789069597.61 total 60.40 last 11.36)
 - mkdir 923 (time 1789069607.63 total 70.41 last 12.78)
total: 1000 mkdir in 77.19 seconds: 12.96 ops/second
 - mkdir 152 (time 1789069547.30 total 10.05 last 15.12)
 - mkdir 296 (time 1789069557.33 total 20.08 last 14.36)
 - mkdir 440 (time 1789069567.47 total 30.22 last 14.20)
 - mkdir 554 (time 1789069577.49 total 40.24 last 11.37)
 - mkdir 682 (time 1789069587.55 total 50.30 last 12.73)
 - mkdir 795 (time 1789069597.56 total 60.31 last 11.29)
 - mkdir 922 (time 1789069607.58 total 70.33 last 12.66)
total: 1000 mkdir in 77.18 seconds: 12.96 ops/second
 - mkdir 149 (time 1789069546.90 total 10.04 last 14.84)
 - mkdir 290 (time 1789069556.91 total 20.05 last 14.09)
 - mkdir 430 (time 1789069566.96 total 30.10 last 13.92)
 - mkdir 539 (time 1789069577.05 total 40.18 last 10.81)
 - mkdir 674 (time 1789069587.05 total 50.19 last 13.49)
 - mkdir 784 (time 1789069597.13 total 60.26 last 10.92)
 - mkdir 915 (time 1789069607.16 total 70.30 last 13.06)
total: 1000 mkdir in 77.79 seconds: 12.85 ops/second
 - mkdir 150 (time 1789069547.66 total 10.18 last 14.73)
 - mkdir 292 (time 1789069557.68 total 20.20 last 14.18)
 - mkdir 435 (time 1789069567.71 total 30.23 last 14.25)
 - mkdir 549 (time 1789069577.74 total 40.26 last 11.37)
 - mkdir 677 (time 1789069587.82 total 50.33 last 12.70)
 - mkdir 787 (time 1789069597.89 total 60.41 last 10.92)
 - mkdir 917 (time 1789069607.97 total 70.49 last 12.90)
total: 1000 mkdir in 78.03 seconds: 12.81 ops/second
 - mkdir 145 (time 1789069547.40 total 10.08 last 14.39)
 - mkdir 288 (time 1789069557.42 total 20.10 last 14.27)
 - mkdir 426 (time 1789069567.49 total 30.17 last 13.71)
 - mkdir 540 (time 1789069577.49 total 40.17 last 11.40)
 - mkdir 669 (time 1789069587.57 total 50.25 last 12.79)
 - mkdir 779 (time 1789069597.60 total 60.28 last 10.97)
 - mkdir 908 (time 1789069607.63 total 70.31 last 12.86)
total: 1000 mkdir in 78.92 seconds: 12.67 ops/second
 - mkdir 144 (time 1789069547.05 total 10.02 last 14.37)
 - mkdir 280 (time 1789069557.10 total 20.07 last 13.53)
 - mkdir 417 (time 1789069567.10 total 30.08 last 13.69)
 - mkdir 528 (time 1789069577.16 total 40.13 last 11.04)
 - mkdir 658 (time 1789069587.28 total 50.26 last 12.84)
 - mkdir 770 (time 1789069597.34 total 60.32 last 11.14)
 - mkdir 895 (time 1789069607.35 total 70.32 last 12.49)
 - mkdir 998 (time 1789069617.44 total 80.41 last 10.21)
total: 1000 mkdir in 80.49 seconds: 12.42 ops/second
 - mkdir 139 (time 1789069547.08 total 10.00 last 13.90)
 - mkdir 271 (time 1789069557.11 total 20.03 last 13.16)
 - mkdir 402 (time 1789069567.20 total 30.11 last 12.99)
 - mkdir 511 (time 1789069577.21 total 40.13 last 10.88)
 - mkdir 644 (time 1789069587.27 total 50.18 last 13.23)
 - mkdir 752 (time 1789069597.28 total 60.20 last 10.79)
 - mkdir 871 (time 1789069607.35 total 70.26 last 11.82)
 - mkdir 971 (time 1789069617.36 total 80.27 last 9.99)
total: 1000 mkdir in 82.42 seconds: 12.13 ops/second
Time for file creation 91 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 15 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.48 seconds: 20.77 ops/second
 - mkdir 159 (time 1789070174.33 total 10.03 last 15.86)
 - mkdir 322 (time 1789070184.37 total 20.06 last 16.24)
 - mkdir 486 (time 1789070194.38 total 30.08 last 16.37)
 - mkdir 647 (time 1789070204.41 total 40.10 last 16.06)
 - mkdir 819 (time 1789070214.47 total 50.17 last 17.08)
 - mkdir 979 (time 1789070224.50 total 60.20 last 15.96)
total: 1000 mkdir in 61.40 seconds: 16.29 ops/second
 - mkdir 156 (time 1789070173.86 total 10.12 last 15.41)
 - mkdir 312 (time 1789070183.92 total 20.17 last 15.52)
 - mkdir 478 (time 1789070193.95 total 30.20 last 16.55)
 - mkdir 635 (time 1789070204.03 total 40.29 last 15.56)
 - mkdir 808 (time 1789070214.04 total 50.29 last 17.29)
 - mkdir 964 (time 1789070224.10 total 60.35 last 15.51)
total: 1000 mkdir in 63.52 seconds: 15.74 ops/second
 - mkdir 157 (time 1789070174.79 total 10.03 last 15.65)
 - mkdir 314 (time 1789070184.85 total 20.10 last 15.60)
 - mkdir 479 (time 1789070194.88 total 30.12 last 16.46)
 - mkdir 635 (time 1789070204.89 total 40.14 last 15.58)
 - mkdir 804 (time 1789070214.91 total 50.16 last 16.87)
 - mkdir 962 (time 1789070224.97 total 60.21 last 15.71)
total: 1000 mkdir in 64.66 seconds: 15.46 ops/second
 - mkdir 158 (time 1789070174.59 total 10.02 last 15.77)
 - mkdir 312 (time 1789070184.65 total 20.08 last 15.30)
 - mkdir 476 (time 1789070194.67 total 30.10 last 16.37)
 - mkdir 631 (time 1789070204.69 total 40.12 last 15.47)
 - mkdir 795 (time 1789070214.69 total 50.13 last 16.40)
 - mkdir 951 (time 1789070224.70 total 60.13 last 15.59)
total: 1000 mkdir in 66.09 seconds: 15.13 ops/second
 - mkdir 152 (time 1789070174.49 total 10.02 last 15.16)
 - mkdir 312 (time 1789070184.58 total 20.11 last 15.86)
 - mkdir 470 (time 1789070194.58 total 30.11 last 15.80)
 - mkdir 624 (time 1789070204.60 total 40.13 last 15.37)
 - mkdir 789 (time 1789070214.63 total 50.17 last 16.45)
 - mkdir 946 (time 1789070224.64 total 60.18 last 15.68)
total: 1000 mkdir in 66.34 seconds: 15.07 ops/second
 - mkdir 151 (time 1789070173.72 total 10.03 last 15.06)
 - mkdir 303 (time 1789070183.74 total 20.05 last 15.16)
 - mkdir 456 (time 1789070193.79 total 30.10 last 15.23)
 - mkdir 608 (time 1789070203.86 total 40.17 last 15.09)
 - mkdir 774 (time 1789070213.89 total 50.20 last 16.55)
 - mkdir 936 (time 1789070223.90 total 60.21 last 16.18)
total: 1000 mkdir in 67.54 seconds: 14.81 ops/second
 - mkdir 152 (time 1789070174.37 total 10.04 last 15.14)
 - mkdir 312 (time 1789070184.44 total 20.12 last 15.87)
 - mkdir 472 (time 1789070194.45 total 30.12 last 15.99)
 - mkdir 625 (time 1789070204.48 total 40.15 last 15.25)
 - mkdir 788 (time 1789070214.52 total 50.20 last 16.23)
 - mkdir 942 (time 1789070224.61 total 60.28 last 15.27)
total: 1000 mkdir in 67.14 seconds: 14.89 ops/second
 - mkdir 151 (time 1789070174.58 total 10.01 last 15.09)
 - mkdir 300 (time 1789070184.64 total 20.06 last 14.82)
 - mkdir 466 (time 1789070194.66 total 30.08 last 16.56)
 - mkdir 620 (time 1789070204.71 total 40.14 last 15.32)
 - mkdir 783 (time 1789070214.75 total 50.17 last 16.24)
 - mkdir 939 (time 1789070224.75 total 60.17 last 15.60)
total: 1000 mkdir in 67.95 seconds: 14.72 ops/second
 - mkdir 151 (time 1789070172.83 total 10.04 last 15.04)
 - mkdir 299 (time 1789070182.87 total 20.08 last 14.74)
 - mkdir 449 (time 1789070192.89 total 30.10 last 14.97)
 - mkdir 599 (time 1789070202.94 total 40.15 last 14.93)
 - mkdir 754 (time 1789070212.97 total 50.18 last 15.45)
 - mkdir 899 (time 1789070223.00 total 60.21 last 14.46)
 - mkdir 985 (time 1789070233.02 total 70.23 last 8.59)
total: 1000 mkdir in 71.53 seconds: 13.98 ops/second
 - mkdir 148 (time 1789070174.63 total 10.02 last 14.77)
 - mkdir 299 (time 1789070184.72 total 20.11 last 14.97)
 - mkdir 453 (time 1789070194.73 total 30.12 last 15.38)
 - mkdir 605 (time 1789070204.77 total 40.16 last 15.13)
 - mkdir 762 (time 1789070214.81 total 50.20 last 15.65)
 - mkdir 916 (time 1789070224.83 total 60.22 last 15.37)
total: 1000 mkdir in 69.90 seconds: 14.31 ops/second
Time for file creation 81 sec for 1 RPC sent at a time
 - unlinked 0 (time 1789070243 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1789070244 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1789070244 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1789070244 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789070244 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789070244 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789070244 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789070245 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1789070244 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1789070245 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
Time for file removal 43 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8