== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 12:45:51 (1787157951)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.52 seconds: 19.18 ops/second
 - mkdir 151 (time 1787157976.30 total 10.00 last 15.09)
 - mkdir 266 (time 1787157986.37 total 20.08 last 11.42)
 - mkdir 372 (time 1787157996.46 total 30.17 last 10.51)
 - mkdir 499 (time 1787158006.47 total 40.18 last 12.68)
 - mkdir 680 (time 1787158016.50 total 50.21 last 18.05)
 - mkdir 821 (time 1787158026.55 total 60.26 last 14.02)
 - mkdir 989 (time 1787158036.58 total 70.29 last 16.76)
total: 1000 mkdir in 71.23 seconds: 14.04 ops/second
 - mkdir 144 (time 1787157976.13 total 10.01 last 14.39)
 - mkdir 267 (time 1787157986.17 total 20.05 last 12.25)
 - mkdir 379 (time 1787157996.17 total 30.05 last 11.20)
 - mkdir 501 (time 1787158006.23 total 40.10 last 12.13)
 - mkdir 681 (time 1787158016.28 total 50.15 last 17.91)
 - mkdir 821 (time 1787158026.30 total 60.18 last 13.97)
 - mkdir 983 (time 1787158036.36 total 70.23 last 16.12)
total: 1000 mkdir in 71.80 seconds: 13.93 ops/second
 - mkdir 147 (time 1787157977.08 total 10.03 last 14.65)
 - mkdir 268 (time 1787157987.13 total 20.08 last 12.05)
 - mkdir 377 (time 1787157997.23 total 30.18 last 10.78)
 - mkdir 508 (time 1787158007.26 total 40.21 last 13.07)
 - mkdir 694 (time 1787158017.26 total 50.21 last 18.59)
 - mkdir 829 (time 1787158027.32 total 60.27 last 13.42)
 - mkdir 991 (time 1787158037.38 total 70.33 last 16.12)
total: 1000 mkdir in 71.60 seconds: 13.97 ops/second
 - mkdir 146 (time 1787157977.06 total 10.08 last 14.48)
 - mkdir 261 (time 1787157987.09 total 20.12 last 11.46)
 - mkdir 373 (time 1787157997.13 total 30.15 last 11.16)
 - mkdir 501 (time 1787158007.14 total 40.17 last 12.78)
 - mkdir 683 (time 1787158017.20 total 50.22 last 18.11)
 - mkdir 816 (time 1787158027.31 total 60.34 last 13.14)
 - mkdir 984 (time 1787158037.33 total 70.36 last 16.77)
total: 1000 mkdir in 72.71 seconds: 13.75 ops/second
 - mkdir 146 (time 1787157976.60 total 10.04 last 14.54)
 - mkdir 259 (time 1787157986.60 total 20.04 last 11.30)
 - mkdir 367 (time 1787157996.62 total 30.06 last 10.78)
 - mkdir 492 (time 1787158006.67 total 40.11 last 12.44)
 - mkdir 676 (time 1787158016.70 total 50.14 last 18.35)
 - mkdir 809 (time 1787158026.70 total 60.14 last 13.29)
 - mkdir 972 (time 1787158036.75 total 70.19 last 16.22)
total: 1000 mkdir in 74.35 seconds: 13.45 ops/second
 - mkdir 145 (time 1787157977.03 total 10.10 last 14.35)
 - mkdir 261 (time 1787157987.05 total 20.12 last 11.59)
 - mkdir 366 (time 1787157997.06 total 30.13 last 10.49)
 - mkdir 490 (time 1787158007.06 total 40.13 last 12.40)
 - mkdir 673 (time 1787158017.09 total 50.16 last 18.25)
 - mkdir 809 (time 1787158027.13 total 60.20 last 13.55)
 - mkdir 972 (time 1787158037.14 total 70.21 last 16.29)
total: 1000 mkdir in 74.78 seconds: 13.37 ops/second
 - mkdir 145 (time 1787157977.42 total 10.16 last 14.27)
 - mkdir 267 (time 1787157987.45 total 20.20 last 12.16)
 - mkdir 370 (time 1787157997.45 total 30.20 last 10.30)
 - mkdir 493 (time 1787158007.62 total 40.36 last 12.10)
 - mkdir 680 (time 1787158017.67 total 50.42 last 18.60)
 - mkdir 812 (time 1787158027.69 total 60.43 last 13.18)
 - mkdir 973 (time 1787158037.77 total 70.51 last 15.97)
total: 1000 mkdir in 75.25 seconds: 13.29 ops/second
 - mkdir 139 (time 1787157976.51 total 10.07 last 13.80)
 - mkdir 254 (time 1787157986.55 total 20.11 last 11.45)
 - mkdir 362 (time 1787157996.60 total 30.17 last 10.74)
 - mkdir 483 (time 1787158006.67 total 40.23 last 12.02)
 - mkdir 664 (time 1787158016.72 total 50.28 last 18.01)
 - mkdir 799 (time 1787158026.77 total 60.33 last 13.44)
 - mkdir 958 (time 1787158036.83 total 70.39 last 15.81)
total: 1000 mkdir in 77.47 seconds: 12.91 ops/second
 - mkdir 137 (time 1787157976.72 total 10.01 last 13.69)
 - mkdir 244 (time 1787157986.76 total 20.05 last 10.66)
 - mkdir 349 (time 1787157996.77 total 30.05 last 10.49)
 - mkdir 475 (time 1787158006.86 total 40.15 last 12.48)
 - mkdir 658 (time 1787158016.88 total 50.16 last 18.27)
 - mkdir 794 (time 1787158026.91 total 60.20 last 13.55)
 - mkdir 954 (time 1787158036.93 total 70.22 last 15.97)
total: 1000 mkdir in 77.71 seconds: 12.87 ops/second
 - mkdir 129 (time 1787157975.75 total 10.03 last 12.86)
 - mkdir 238 (time 1787157985.79 total 20.07 last 10.86)
 - mkdir 339 (time 1787157995.79 total 30.07 last 10.10)
 - mkdir 463 (time 1787158005.83 total 40.11 last 12.35)
 - mkdir 637 (time 1787158015.85 total 50.13 last 17.37)
 - mkdir 780 (time 1787158025.89 total 60.17 last 14.24)
 - mkdir 932 (time 1787158035.96 total 70.24 last 15.09)
total: 1000 mkdir in 79.87 seconds: 12.52 ops/second
Time for file creation 91 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 18 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.47 seconds: 21.10 ops/second
 - mkdir 123 (time 1787158603.61 total 10.01 last 12.29)
 - mkdir 256 (time 1787158613.65 total 20.05 last 13.25)
 - mkdir 402 (time 1787158623.70 total 30.10 last 14.53)
 - mkdir 551 (time 1787158633.75 total 40.15 last 14.82)
 - mkdir 714 (time 1787158643.78 total 50.18 last 16.25)
 - mkdir 864 (time 1787158653.80 total 60.20 last 14.97)
total: 1000 mkdir in 69.76 seconds: 14.33 ops/second
 - mkdir 114 (time 1787158602.95 total 10.03 last 11.36)
 - mkdir 251 (time 1787158613.05 total 20.14 last 13.56)
 - mkdir 394 (time 1787158623.17 total 30.25 last 14.13)
 - mkdir 539 (time 1787158633.18 total 40.26 last 14.49)
 - mkdir 701 (time 1787158643.19 total 50.27 last 16.18)
 - mkdir 851 (time 1787158653.22 total 60.30 last 14.95)
 - mkdir 991 (time 1787158663.24 total 70.32 last 13.97)
total: 1000 mkdir in 70.94 seconds: 14.10 ops/second
 - mkdir 131 (time 1787158604.98 total 10.03 last 13.06)
 - mkdir 269 (time 1787158615.01 total 20.06 last 13.76)
 - mkdir 412 (time 1787158625.04 total 30.09 last 14.26)
 - mkdir 562 (time 1787158635.05 total 40.10 last 14.98)
 - mkdir 733 (time 1787158645.08 total 50.13 last 17.05)
 - mkdir 874 (time 1787158655.14 total 60.19 last 14.02)
total: 1000 mkdir in 69.27 seconds: 14.44 ops/second
 - mkdir 132 (time 1787158604.85 total 10.03 last 13.16)
 - mkdir 271 (time 1787158614.86 total 20.03 last 13.89)
 - mkdir 415 (time 1787158624.88 total 30.06 last 14.36)
 - mkdir 563 (time 1787158634.92 total 40.10 last 14.74)
 - mkdir 730 (time 1787158644.96 total 50.13 last 16.64)
 - mkdir 872 (time 1787158654.97 total 60.15 last 14.17)
total: 1000 mkdir in 69.42 seconds: 14.41 ops/second
 - mkdir 127 (time 1787158604.18 total 10.02 last 12.68)
 - mkdir 258 (time 1787158614.22 total 20.06 last 13.05)
 - mkdir 403 (time 1787158624.25 total 30.08 last 14.46)
 - mkdir 547 (time 1787158634.26 total 40.09 last 14.38)
 - mkdir 714 (time 1787158644.31 total 50.14 last 16.62)
 - mkdir 860 (time 1787158654.37 total 60.21 last 14.51)
total: 1000 mkdir in 70.08 seconds: 14.27 ops/second
 - mkdir 127 (time 1787158604.19 total 10.03 last 12.66)
 - mkdir 262 (time 1787158614.23 total 20.07 last 13.45)
 - mkdir 410 (time 1787158624.28 total 30.13 last 14.71)
 - mkdir 553 (time 1787158634.29 total 40.14 last 14.29)
 - mkdir 715 (time 1787158644.37 total 50.22 last 16.07)
 - mkdir 860 (time 1787158654.38 total 60.22 last 14.49)
total: 1000 mkdir in 70.22 seconds: 14.24 ops/second
 - mkdir 111 (time 1787158602.11 total 10.03 last 11.07)
 - mkdir 241 (time 1787158612.13 total 20.05 last 12.97)
 - mkdir 380 (time 1787158622.21 total 30.13 last 13.79)
 - mkdir 521 (time 1787158632.29 total 40.21 last 13.99)
 - mkdir 685 (time 1787158642.31 total 50.23 last 16.37)
 - mkdir 830 (time 1787158652.33 total 60.26 last 14.46)
 - mkdir 971 (time 1787158662.38 total 70.30 last 14.04)
total: 1000 mkdir in 72.61 seconds: 13.77 ops/second
 - mkdir 130 (time 1787158605.05 total 10.04 last 12.95)
 - mkdir 268 (time 1787158615.10 total 20.10 last 13.72)
 - mkdir 408 (time 1787158625.13 total 30.13 last 13.96)
 - mkdir 556 (time 1787158635.20 total 40.19 last 14.71)
 - mkdir 721 (time 1787158645.20 total 50.19 last 16.49)
 - mkdir 862 (time 1787158655.21 total 60.20 last 14.08)
 - mkdir 999 (time 1787158665.24 total 70.23 last 13.67)
total: 1000 mkdir in 70.23 seconds: 14.24 ops/second
 - mkdir 124 (time 1787158604.95 total 10.03 last 12.37)
 - mkdir 262 (time 1787158615.04 total 20.12 last 13.68)
 - mkdir 400 (time 1787158625.05 total 30.13 last 13.79)
 - mkdir 551 (time 1787158635.07 total 40.15 last 15.07)
 - mkdir 715 (time 1787158645.08 total 50.16 last 16.38)
 - mkdir 856 (time 1787158655.10 total 60.18 last 14.07)
 - mkdir 993 (time 1787158665.14 total 70.21 last 13.65)
total: 1000 mkdir in 70.70 seconds: 14.14 ops/second
 - mkdir 119 (time 1787158604.18 total 10.01 last 11.89)
 - mkdir 249 (time 1787158614.21 total 20.04 last 12.96)
 - mkdir 387 (time 1787158624.22 total 30.05 last 13.78)
 - mkdir 537 (time 1787158634.26 total 40.09 last 14.94)
 - mkdir 699 (time 1787158644.29 total 50.12 last 16.15)
 - mkdir 843 (time 1787158654.37 total 60.20 last 14.29)
 - mkdir 981 (time 1787158664.40 total 70.22 last 13.76)
total: 1000 mkdir in 71.74 seconds: 13.94 ops/second
Time for file creation 86 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787158678 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787158679 ; total 1 ; last 1)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787158680 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787158679 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787158679 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787158679 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787158679 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787158679 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787158679 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787158680 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
Time for file removal 46 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8