== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 21:43:01 (1786671781)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.66 seconds: 15.11 ops/second
 - mkdir 103 (time 1786671810.64 total 10.06 last 10.24)
 - mkdir 204 (time 1786671820.71 total 20.13 last 10.03)
 - mkdir 329 (time 1786671830.76 total 30.19 last 12.43)
 - mkdir 477 (time 1786671840.77 total 40.19 last 14.79)
 - mkdir 617 (time 1786671850.82 total 50.24 last 13.93)
 - mkdir 749 (time 1786671860.83 total 60.26 last 13.18)
 - mkdir 838 (time 1786671870.88 total 70.30 last 8.86)
 - mkdir 939 (time 1786671881.16 total 80.58 last 9.82)
total: 1000 mkdir in 87.59 seconds: 11.42 ops/second
 - mkdir 105 (time 1786671812.46 total 10.01 last 10.49)
 - mkdir 216 (time 1786671822.48 total 20.03 last 11.08)
 - mkdir 344 (time 1786671832.51 total 30.07 last 12.76)
 - mkdir 492 (time 1786671842.52 total 40.08 last 14.78)
 - mkdir 628 (time 1786671852.57 total 50.13 last 13.53)
 - mkdir 756 (time 1786671862.59 total 60.14 last 12.78)
 - mkdir 845 (time 1786671872.62 total 70.18 last 8.87)
 - mkdir 946 (time 1786671882.64 total 80.19 last 10.08)
total: 1000 mkdir in 86.15 seconds: 11.61 ops/second
 - mkdir 101 (time 1786671811.43 total 10.10 last 10.00)
 - mkdir 214 (time 1786671821.50 total 20.18 last 11.22)
 - mkdir 332 (time 1786671831.51 total 30.19 last 11.78)
 - mkdir 482 (time 1786671841.53 total 40.21 last 14.97)
 - mkdir 622 (time 1786671851.63 total 50.31 last 13.87)
 - mkdir 755 (time 1786671861.73 total 60.41 last 13.17)
 - mkdir 840 (time 1786671871.79 total 70.47 last 8.45)
 - mkdir 943 (time 1786671881.88 total 80.56 last 10.21)
total: 1000 mkdir in 87.82 seconds: 11.39 ops/second
 - mkdir 104 (time 1786671810.85 total 10.14 last 10.25)
 - mkdir 208 (time 1786671820.91 total 20.20 last 10.34)
 - mkdir 333 (time 1786671831.04 total 30.34 last 12.34)
 - mkdir 477 (time 1786671841.08 total 40.38 last 14.34)
 - mkdir 612 (time 1786671851.08 total 50.38 last 13.49)
 - mkdir 751 (time 1786671861.10 total 60.39 last 13.88)
 - mkdir 838 (time 1786671871.14 total 70.44 last 8.66)
 - mkdir 931 (time 1786671881.23 total 80.52 last 9.22)
total: 1000 mkdir in 89.25 seconds: 11.20 ops/second
 - mkdir 106 (time 1786671812.38 total 10.09 last 10.51)
 - mkdir 218 (time 1786671822.41 total 20.12 last 11.17)
 - mkdir 340 (time 1786671832.41 total 30.12 last 12.20)
 - mkdir 488 (time 1786671842.48 total 40.18 last 14.71)
 - mkdir 622 (time 1786671852.50 total 50.21 last 13.37)
 - mkdir 752 (time 1786671862.54 total 60.25 last 12.94)
 - mkdir 838 (time 1786671872.60 total 70.31 last 8.55)
 - mkdir 934 (time 1786671882.67 total 80.37 last 9.54)
total: 1000 mkdir in 89.30 seconds: 11.20 ops/second
 - mkdir 100 (time 1786671811.46 total 10.03 last 9.97)
 - mkdir 210 (time 1786671821.50 total 20.08 last 10.95)
 - mkdir 333 (time 1786671831.62 total 30.19 last 12.16)
 - mkdir 478 (time 1786671841.63 total 40.20 last 14.48)
 - mkdir 614 (time 1786671851.63 total 50.20 last 13.60)
 - mkdir 746 (time 1786671861.74 total 60.31 last 13.06)
 - mkdir 831 (time 1786671871.79 total 70.36 last 8.46)
 - mkdir 927 (time 1786671881.91 total 80.49 last 9.48)
 - mkdir 998 (time 1786671891.93 total 90.50 last 7.09)
total: 1000 mkdir in 90.70 seconds: 11.02 ops/second
 - mkdir 102 (time 1786671811.89 total 10.05 last 10.15)
 - mkdir 209 (time 1786671821.99 total 20.15 last 10.60)
 - mkdir 332 (time 1786671832.01 total 30.17 last 12.27)
 - mkdir 476 (time 1786671842.04 total 40.20 last 14.36)
 - mkdir 609 (time 1786671852.08 total 50.24 last 13.25)
 - mkdir 740 (time 1786671862.16 total 60.31 last 13.00)
 - mkdir 825 (time 1786671872.21 total 70.37 last 8.45)
 - mkdir 921 (time 1786671882.28 total 80.44 last 9.54)
 - mkdir 991 (time 1786671892.33 total 90.48 last 6.97)
total: 1000 mkdir in 91.88 seconds: 10.88 ops/second
 - mkdir 95 (time 1786671811.82 total 10.07 last 9.44)
 - mkdir 197 (time 1786671821.84 total 20.09 last 10.18)
 - mkdir 316 (time 1786671831.89 total 30.14 last 11.84)
 - mkdir 467 (time 1786671841.95 total 40.19 last 15.01)
 - mkdir 606 (time 1786671852.01 total 50.25 last 13.82)
 - mkdir 733 (time 1786671862.11 total 60.35 last 12.57)
 - mkdir 818 (time 1786671872.16 total 70.41 last 8.45)
 - mkdir 909 (time 1786671882.27 total 80.52 last 9.00)
 - mkdir 980 (time 1786671892.45 total 90.69 last 6.98)
total: 1000 mkdir in 93.40 seconds: 10.71 ops/second
 - mkdir 95 (time 1786671811.66 total 10.01 last 9.49)
 - mkdir 197 (time 1786671821.71 total 20.07 last 10.14)
 - mkdir 315 (time 1786671831.72 total 30.07 last 11.80)
 - mkdir 461 (time 1786671841.78 total 40.14 last 14.51)
 - mkdir 594 (time 1786671851.86 total 50.21 last 13.20)
 - mkdir 722 (time 1786671861.92 total 60.28 last 12.72)
 - mkdir 807 (time 1786671871.93 total 70.28 last 8.49)
 - mkdir 899 (time 1786671882.01 total 80.36 last 9.13)
 - mkdir 971 (time 1786671892.01 total 90.37 last 7.20)
total: 1000 mkdir in 94.23 seconds: 10.61 ops/second
 - mkdir 94 (time 1786671811.61 total 10.08 last 9.33)
 - mkdir 194 (time 1786671821.65 total 20.12 last 9.96)
 - mkdir 304 (time 1786671831.70 total 30.16 last 10.95)
 - mkdir 448 (time 1786671841.76 total 40.23 last 14.31)
 - mkdir 583 (time 1786671851.86 total 50.33 last 13.37)
 - mkdir 709 (time 1786671861.99 total 60.45 last 12.44)
 - mkdir 787 (time 1786671872.05 total 70.52 last 7.75)
 - mkdir 881 (time 1786671882.16 total 80.62 last 9.31)
 - mkdir 951 (time 1786671892.28 total 90.75 last 6.91)
total: 1000 mkdir in 96.32 seconds: 10.38 ops/second
Time for file creation 113 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 16 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.54 seconds: 18.40 ops/second
 - mkdir 143 (time 1786672523.66 total 10.00 last 14.30)
 - mkdir 272 (time 1786672533.67 total 20.01 last 12.88)
 - mkdir 414 (time 1786672543.75 total 30.09 last 14.09)
 - mkdir 561 (time 1786672553.83 total 40.17 last 14.59)
 - mkdir 701 (time 1786672563.83 total 50.17 last 13.99)
 - mkdir 829 (time 1786672573.90 total 60.24 last 12.71)
 - mkdir 977 (time 1786672583.92 total 70.26 last 14.77)
total: 1000 mkdir in 71.79 seconds: 13.93 ops/second
 - mkdir 147 (time 1786672526.11 total 10.07 last 14.60)
 - mkdir 280 (time 1786672536.16 total 20.12 last 13.23)
 - mkdir 430 (time 1786672546.20 total 30.16 last 14.94)
 - mkdir 582 (time 1786672556.25 total 40.20 last 15.13)
 - mkdir 715 (time 1786672566.27 total 50.22 last 13.27)
 - mkdir 852 (time 1786672576.32 total 60.27 last 13.63)
 - mkdir 997 (time 1786672586.37 total 70.33 last 14.42)
total: 1000 mkdir in 70.68 seconds: 14.15 ops/second
 - mkdir 148 (time 1786672526.47 total 10.07 last 14.70)
 - mkdir 285 (time 1786672536.55 total 20.15 last 13.59)
 - mkdir 432 (time 1786672546.61 total 30.21 last 14.62)
 - mkdir 586 (time 1786672556.65 total 40.25 last 15.33)
 - mkdir 719 (time 1786672566.71 total 50.31 last 13.23)
 - mkdir 857 (time 1786672576.77 total 60.37 last 13.71)
 - mkdir 997 (time 1786672586.89 total 70.48 last 13.84)
total: 1000 mkdir in 70.82 seconds: 14.12 ops/second
 - mkdir 138 (time 1786672525.51 total 10.04 last 13.74)
 - mkdir 270 (time 1786672535.54 total 20.08 last 13.15)
 - mkdir 414 (time 1786672545.58 total 30.12 last 14.35)
 - mkdir 567 (time 1786672555.59 total 40.13 last 15.28)
 - mkdir 699 (time 1786672565.63 total 50.16 last 13.15)
 - mkdir 835 (time 1786672575.69 total 60.22 last 13.52)
 - mkdir 978 (time 1786672585.71 total 70.24 last 14.27)
total: 1000 mkdir in 73.04 seconds: 13.69 ops/second
 - mkdir 150 (time 1786672526.39 total 10.02 last 14.98)
 - mkdir 282 (time 1786672536.40 total 20.02 last 13.19)
 - mkdir 422 (time 1786672546.47 total 30.09 last 13.91)
 - mkdir 576 (time 1786672556.50 total 40.12 last 15.36)
 - mkdir 708 (time 1786672566.51 total 50.13 last 13.18)
 - mkdir 845 (time 1786672576.54 total 60.16 last 13.66)
 - mkdir 982 (time 1786672586.58 total 70.20 last 13.64)
total: 1000 mkdir in 72.85 seconds: 13.73 ops/second
 - mkdir 139 (time 1786672525.45 total 10.03 last 13.86)
 - mkdir 263 (time 1786672535.50 total 20.08 last 12.34)
 - mkdir 404 (time 1786672545.53 total 30.12 last 14.05)
 - mkdir 556 (time 1786672555.55 total 40.14 last 15.17)
 - mkdir 693 (time 1786672565.65 total 50.23 last 13.57)
 - mkdir 829 (time 1786672575.65 total 60.24 last 13.59)
 - mkdir 975 (time 1786672585.66 total 70.24 last 14.60)
total: 1000 mkdir in 74.02 seconds: 13.51 ops/second
 - mkdir 141 (time 1786672526.16 total 10.03 last 14.06)
 - mkdir 276 (time 1786672536.16 total 20.03 last 13.50)
 - mkdir 417 (time 1786672546.18 total 30.05 last 14.07)
 - mkdir 568 (time 1786672556.26 total 40.13 last 14.99)
 - mkdir 700 (time 1786672566.31 total 50.18 last 13.14)
 - mkdir 837 (time 1786672576.31 total 60.19 last 13.69)
 - mkdir 977 (time 1786672586.39 total 70.27 last 13.89)
total: 1000 mkdir in 73.72 seconds: 13.57 ops/second
 - mkdir 138 (time 1786672525.47 total 10.04 last 13.75)
 - mkdir 270 (time 1786672535.49 total 20.05 last 13.18)
 - mkdir 409 (time 1786672545.53 total 30.09 last 13.84)
 - mkdir 562 (time 1786672555.59 total 40.16 last 15.20)
 - mkdir 694 (time 1786672565.60 total 50.16 last 13.20)
 - mkdir 827 (time 1786672575.63 total 60.19 last 13.26)
 - mkdir 969 (time 1786672585.76 total 70.32 last 14.02)
total: 1000 mkdir in 74.87 seconds: 13.36 ops/second
 - mkdir 144 (time 1786672525.83 total 10.05 last 14.33)
 - mkdir 272 (time 1786672535.90 total 20.11 last 12.72)
 - mkdir 410 (time 1786672545.90 total 30.11 last 13.80)
 - mkdir 557 (time 1786672555.94 total 40.15 last 14.64)
 - mkdir 689 (time 1786672565.96 total 50.17 last 13.17)
 - mkdir 826 (time 1786672576.03 total 60.24 last 13.61)
 - mkdir 966 (time 1786672586.20 total 70.41 last 13.76)
total: 1000 mkdir in 75.50 seconds: 13.24 ops/second
 - mkdir 136 (time 1786672526.62 total 10.00 last 13.60)
 - mkdir 266 (time 1786672536.66 total 20.04 last 12.95)
 - mkdir 396 (time 1786672546.70 total 30.09 last 12.94)
 - mkdir 552 (time 1786672556.72 total 40.10 last 15.58)
 - mkdir 687 (time 1786672566.77 total 50.15 last 13.43)
 - mkdir 825 (time 1786672576.81 total 60.20 last 13.74)
 - mkdir 950 (time 1786672586.94 total 70.33 last 12.34)
total: 1000 mkdir in 76.98 seconds: 12.99 ops/second
Time for file creation 90 sec for 1 RPC sent at a time
 - unlinked 0 (time 1786672604 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1786672604 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1786672605 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1786672605 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1786672606 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1786672606 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1786672606 ; total 1 ; last 1)
total: 1000 unlinks in 42 seconds: 23.809525 unlinks/second
 - unlinked 0 (time 1786672606 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1786672607 ; total 1 ; last 1)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1786672606 ; total 0 ; last 0)
total: 1000 unlinks in 42 seconds: 23.809525 unlinks/second
Time for file removal 57 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8