== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 14:55:56 (1787856956)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.42 seconds: 23.62 ops/second
 - mkdir 151 (time 1787856979.30 total 10.12 last 14.93)
 - mkdir 307 (time 1787856989.32 total 20.14 last 15.57)
 - mkdir 468 (time 1787856999.48 total 30.29 last 15.85)
 - mkdir 632 (time 1787857009.49 total 40.30 last 16.38)
 - mkdir 792 (time 1787857019.55 total 50.36 last 15.90)
 - mkdir 951 (time 1787857029.60 total 60.41 last 15.82)
total: 1000 mkdir in 63.56 seconds: 15.73 ops/second
 - mkdir 144 (time 1787856979.61 total 10.06 last 14.32)
 - mkdir 301 (time 1787856989.62 total 20.07 last 15.68)
 - mkdir 467 (time 1787856999.68 total 30.12 last 16.51)
 - mkdir 632 (time 1787857009.69 total 40.13 last 16.49)
 - mkdir 789 (time 1787857019.80 total 50.24 last 15.53)
 - mkdir 949 (time 1787857029.82 total 60.26 last 15.97)
total: 1000 mkdir in 63.68 seconds: 15.70 ops/second
 - mkdir 142 (time 1787856980.41 total 10.07 last 14.11)
 - mkdir 302 (time 1787856990.44 total 20.10 last 15.95)
 - mkdir 463 (time 1787857000.44 total 30.10 last 16.09)
 - mkdir 635 (time 1787857010.47 total 40.13 last 17.15)
 - mkdir 790 (time 1787857020.55 total 50.21 last 15.39)
 - mkdir 954 (time 1787857030.60 total 60.26 last 16.32)
total: 1000 mkdir in 63.66 seconds: 15.71 ops/second
 - mkdir 149 (time 1787856980.45 total 10.01 last 14.89)
 - mkdir 311 (time 1787856990.50 total 20.05 last 16.13)
 - mkdir 469 (time 1787857000.51 total 30.07 last 15.77)
 - mkdir 641 (time 1787857010.53 total 40.08 last 17.18)
 - mkdir 795 (time 1787857020.58 total 50.14 last 15.31)
 - mkdir 952 (time 1787857030.60 total 60.15 last 15.68)
total: 1000 mkdir in 64.47 seconds: 15.51 ops/second
 - mkdir 145 (time 1787856978.90 total 10.01 last 14.49)
 - mkdir 291 (time 1787856988.92 total 20.03 last 14.56)
 - mkdir 451 (time 1787856999.00 total 30.11 last 15.88)
 - mkdir 611 (time 1787857009.01 total 40.12 last 15.98)
 - mkdir 766 (time 1787857019.09 total 50.20 last 15.37)
 - mkdir 920 (time 1787857029.12 total 60.23 last 15.36)
total: 1000 mkdir in 66.77 seconds: 14.98 ops/second
 - mkdir 148 (time 1787856980.07 total 10.04 last 14.75)
 - mkdir 304 (time 1787856990.10 total 20.07 last 15.55)
 - mkdir 461 (time 1787857000.17 total 30.14 last 15.59)
 - mkdir 628 (time 1787857010.23 total 40.20 last 16.60)
 - mkdir 775 (time 1787857020.24 total 50.21 last 14.68)
 - mkdir 932 (time 1787857030.25 total 60.22 last 15.68)
total: 1000 mkdir in 66.53 seconds: 15.03 ops/second
 - mkdir 140 (time 1787856980.16 total 10.03 last 13.95)
 - mkdir 296 (time 1787856990.19 total 20.07 last 15.55)
 - mkdir 458 (time 1787857000.26 total 30.13 last 16.10)
 - mkdir 627 (time 1787857010.28 total 40.15 last 16.86)
 - mkdir 775 (time 1787857020.33 total 50.20 last 14.73)
 - mkdir 925 (time 1787857030.35 total 60.22 last 14.97)
total: 1000 mkdir in 67.39 seconds: 14.84 ops/second
 - mkdir 142 (time 1787856980.55 total 10.03 last 14.16)
 - mkdir 295 (time 1787856990.60 total 20.07 last 15.24)
 - mkdir 450 (time 1787857000.67 total 30.15 last 15.38)
 - mkdir 618 (time 1787857010.69 total 40.17 last 16.77)
 - mkdir 768 (time 1787857020.70 total 50.18 last 14.98)
 - mkdir 922 (time 1787857030.75 total 60.22 last 15.33)
total: 1000 mkdir in 67.62 seconds: 14.79 ops/second
 - mkdir 142 (time 1787856979.86 total 10.06 last 14.11)
 - mkdir 291 (time 1787856989.91 total 20.11 last 14.83)
 - mkdir 449 (time 1787856999.94 total 30.14 last 15.75)
 - mkdir 607 (time 1787857009.96 total 40.16 last 15.78)
 - mkdir 754 (time 1787857020.00 total 50.20 last 14.64)
 - mkdir 899 (time 1787857030.02 total 60.23 last 14.46)
total: 1000 mkdir in 70.06 seconds: 14.27 ops/second
 - mkdir 134 (time 1787856980.20 total 10.06 last 13.32)
 - mkdir 283 (time 1787856990.21 total 20.06 last 14.90)
 - mkdir 434 (time 1787857000.23 total 30.09 last 15.06)
 - mkdir 595 (time 1787857010.24 total 40.09 last 16.09)
 - mkdir 742 (time 1787857020.26 total 50.11 last 14.67)
 - mkdir 892 (time 1787857030.27 total 60.13 last 14.98)
 - mkdir 997 (time 1787857040.30 total 70.15 last 10.47)
total: 1000 mkdir in 70.28 seconds: 14.23 ops/second
Time for file creation 81 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.62 seconds: 16.04 ops/second
 - mkdir 190 (time 1787857594.81 total 10.02 last 18.97)
 - mkdir 366 (time 1787857604.84 total 20.05 last 17.54)
 - mkdir 516 (time 1787857614.87 total 30.08 last 14.96)
 - mkdir 691 (time 1787857624.89 total 40.10 last 17.46)
 - mkdir 871 (time 1787857634.94 total 50.16 last 17.90)
total: 1000 mkdir in 58.45 seconds: 17.11 ops/second
 - mkdir 204 (time 1787857597.09 total 10.08 last 20.23)
 - mkdir 375 (time 1787857607.14 total 20.13 last 17.01)
 - mkdir 526 (time 1787857617.14 total 30.14 last 15.10)
 - mkdir 716 (time 1787857627.15 total 40.15 last 18.98)
 - mkdir 877 (time 1787857637.20 total 50.19 last 16.02)
total: 1000 mkdir in 58.74 seconds: 17.02 ops/second
 - mkdir 198 (time 1787857597.08 total 10.00 last 19.80)
 - mkdir 373 (time 1787857607.13 total 20.05 last 17.41)
 - mkdir 526 (time 1787857617.15 total 30.07 last 15.27)
 - mkdir 710 (time 1787857627.15 total 40.07 last 18.39)
 - mkdir 876 (time 1787857637.19 total 50.11 last 16.54)
total: 1000 mkdir in 58.81 seconds: 17.00 ops/second
 - mkdir 199 (time 1787857596.74 total 10.00 last 19.89)
 - mkdir 372 (time 1787857606.78 total 20.04 last 17.25)
 - mkdir 524 (time 1787857616.85 total 30.11 last 15.08)
 - mkdir 707 (time 1787857626.91 total 40.17 last 18.20)
 - mkdir 872 (time 1787857636.96 total 50.22 last 16.41)
total: 1000 mkdir in 59.16 seconds: 16.90 ops/second
 - mkdir 189 (time 1787857595.77 total 10.02 last 18.86)
 - mkdir 358 (time 1787857605.82 total 20.07 last 16.82)
 - mkdir 504 (time 1787857615.90 total 30.15 last 14.49)
 - mkdir 685 (time 1787857625.91 total 40.16 last 18.08)
 - mkdir 854 (time 1787857635.99 total 50.24 last 16.76)
 - mkdir 996 (time 1787857646.13 total 60.37 last 14.01)
total: 1000 mkdir in 60.84 seconds: 16.44 ops/second
 - mkdir 190 (time 1787857596.35 total 10.11 last 18.79)
 - mkdir 362 (time 1787857606.43 total 20.19 last 17.07)
 - mkdir 508 (time 1787857616.44 total 30.20 last 14.58)
 - mkdir 691 (time 1787857626.45 total 40.21 last 18.29)
 - mkdir 854 (time 1787857636.50 total 50.26 last 16.21)
 - mkdir 996 (time 1787857646.54 total 60.31 last 14.14)
total: 1000 mkdir in 60.55 seconds: 16.52 ops/second
 - mkdir 193 (time 1787857596.37 total 10.03 last 19.25)
 - mkdir 361 (time 1787857606.43 total 20.08 last 16.71)
 - mkdir 507 (time 1787857616.46 total 30.12 last 14.55)
 - mkdir 689 (time 1787857626.54 total 40.20 last 18.05)
 - mkdir 854 (time 1787857636.57 total 50.23 last 16.45)
 - mkdir 993 (time 1787857646.70 total 60.35 last 13.73)
total: 1000 mkdir in 61.02 seconds: 16.39 ops/second
 - mkdir 201 (time 1787857596.96 total 10.02 last 20.07)
 - mkdir 367 (time 1787857606.98 total 20.03 last 16.57)
 - mkdir 512 (time 1787857617.02 total 30.08 last 14.44)
 - mkdir 696 (time 1787857627.08 total 40.13 last 18.29)
 - mkdir 861 (time 1787857637.12 total 50.18 last 16.43)
 - mkdir 997 (time 1787857647.15 total 60.20 last 13.56)
total: 1000 mkdir in 60.43 seconds: 16.55 ops/second
 - mkdir 205 (time 1787857597.12 total 10.01 last 20.48)
 - mkdir 373 (time 1787857607.14 total 20.03 last 16.77)
 - mkdir 520 (time 1787857617.16 total 30.05 last 14.67)
 - mkdir 700 (time 1787857627.19 total 40.08 last 17.94)
 - mkdir 864 (time 1787857637.23 total 50.12 last 16.34)
 - mkdir 997 (time 1787857647.37 total 60.27 last 13.10)
total: 1000 mkdir in 60.45 seconds: 16.54 ops/second
 - mkdir 186 (time 1787857596.55 total 10.03 last 18.55)
 - mkdir 351 (time 1787857606.61 total 20.09 last 16.40)
 - mkdir 488 (time 1787857616.68 total 30.16 last 13.60)
 - mkdir 665 (time 1787857626.75 total 40.22 last 17.59)
 - mkdir 828 (time 1787857636.80 total 50.28 last 16.21)
 - mkdir 962 (time 1787857646.91 total 60.39 last 13.25)
total: 1000 mkdir in 63.68 seconds: 15.70 ops/second
Time for file creation 74 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787857658 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787857658 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787857658 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787857659 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787857659 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787857659 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787857659 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787857660 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787857660 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787857660 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
Time for file removal 45 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8