== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 13:09:21 (1787850561)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.62 seconds: 16.13 ops/second
 - mkdir 158 (time 1787850581.54 total 10.03 last 15.75)
 - mkdir 327 (time 1787850591.62 total 20.11 last 16.77)
 - mkdir 478 (time 1787850601.79 total 30.28 last 14.84)
 - mkdir 660 (time 1787850611.84 total 40.34 last 18.10)
 - mkdir 832 (time 1787850621.91 total 50.40 last 17.10)
total: 1000 mkdir in 59.62 seconds: 16.77 ops/second
 - mkdir 166 (time 1787850583.57 total 10.10 last 16.44)
 - mkdir 337 (time 1787850593.67 total 20.20 last 16.93)
 - mkdir 489 (time 1787850603.68 total 30.21 last 15.18)
 - mkdir 690 (time 1787850613.71 total 40.24 last 20.04)
 - mkdir 855 (time 1787850623.75 total 50.28 last 16.44)
total: 1000 mkdir in 59.02 seconds: 16.94 ops/second
 - mkdir 168 (time 1787850582.83 total 10.05 last 16.71)
 - mkdir 337 (time 1787850592.86 total 20.08 last 16.85)
 - mkdir 486 (time 1787850602.87 total 30.10 last 14.88)
 - mkdir 672 (time 1787850612.88 total 40.10 last 18.59)
 - mkdir 834 (time 1787850622.93 total 50.16 last 16.11)
total: 1000 mkdir in 60.06 seconds: 16.65 ops/second
 - mkdir 165 (time 1787850583.17 total 10.02 last 16.47)
 - mkdir 334 (time 1787850593.21 total 20.06 last 16.83)
 - mkdir 477 (time 1787850603.24 total 30.09 last 14.26)
 - mkdir 666 (time 1787850613.29 total 40.14 last 18.80)
 - mkdir 835 (time 1787850623.32 total 50.17 last 16.86)
 - mkdir 996 (time 1787850633.41 total 60.26 last 15.95)
total: 1000 mkdir in 60.59 seconds: 16.50 ops/second
 - mkdir 161 (time 1787850583.31 total 10.05 last 16.03)
 - mkdir 329 (time 1787850593.33 total 20.06 last 16.78)
 - mkdir 474 (time 1787850603.38 total 30.11 last 14.42)
 - mkdir 663 (time 1787850613.38 total 40.12 last 18.89)
 - mkdir 832 (time 1787850623.44 total 50.18 last 16.80)
 - mkdir 990 (time 1787850633.53 total 60.26 last 15.67)
total: 1000 mkdir in 61.30 seconds: 16.31 ops/second
 - mkdir 157 (time 1787850582.68 total 10.05 last 15.61)
 - mkdir 317 (time 1787850592.71 total 20.09 last 15.94)
 - mkdir 461 (time 1787850602.77 total 30.15 last 14.32)
 - mkdir 648 (time 1787850612.80 total 40.18 last 18.64)
 - mkdir 815 (time 1787850622.82 total 50.20 last 16.67)
 - mkdir 981 (time 1787850632.82 total 60.20 last 16.59)
total: 1000 mkdir in 62.36 seconds: 16.04 ops/second
 - mkdir 167 (time 1787850583.86 total 10.04 last 16.63)
 - mkdir 333 (time 1787850593.90 total 20.08 last 16.54)
 - mkdir 480 (time 1787850603.90 total 30.08 last 14.70)
 - mkdir 672 (time 1787850613.92 total 40.10 last 19.16)
 - mkdir 841 (time 1787850623.95 total 50.13 last 16.85)
 - mkdir 991 (time 1787850634.08 total 60.26 last 14.81)
total: 1000 mkdir in 61.39 seconds: 16.29 ops/second
 - mkdir 156 (time 1787850583.67 total 10.04 last 15.54)
 - mkdir 321 (time 1787850593.68 total 20.05 last 16.49)
 - mkdir 469 (time 1787850603.68 total 30.05 last 14.80)
 - mkdir 661 (time 1787850613.72 total 40.08 last 19.13)
 - mkdir 829 (time 1787850623.75 total 50.12 last 16.74)
 - mkdir 982 (time 1787850633.81 total 60.18 last 15.21)
total: 1000 mkdir in 62.61 seconds: 15.97 ops/second
 - mkdir 159 (time 1787850583.59 total 10.14 last 15.68)
 - mkdir 320 (time 1787850593.68 total 20.22 last 15.97)
 - mkdir 462 (time 1787850603.68 total 30.23 last 14.19)
 - mkdir 657 (time 1787850613.69 total 40.24 last 19.48)
 - mkdir 821 (time 1787850623.75 total 50.30 last 16.30)
 - mkdir 976 (time 1787850633.81 total 60.36 last 15.41)
total: 1000 mkdir in 63.38 seconds: 15.78 ops/second
 - mkdir 153 (time 1787850583.20 total 10.03 last 15.25)
 - mkdir 313 (time 1787850593.23 total 20.07 last 15.94)
 - mkdir 450 (time 1787850603.25 total 30.09 last 13.67)
 - mkdir 640 (time 1787850613.29 total 40.13 last 18.93)
 - mkdir 808 (time 1787850623.31 total 50.14 last 16.77)
 - mkdir 968 (time 1787850633.38 total 60.22 last 15.89)
total: 1000 mkdir in 63.82 seconds: 15.67 ops/second
Time for file creation 74 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.58 seconds: 17.14 ops/second
 - mkdir 94 (time 1787851257.50 total 10.05 last 9.35)
 - mkdir 189 (time 1787851267.53 total 20.08 last 9.47)
 - mkdir 298 (time 1787851277.62 total 30.17 last 10.80)
 - mkdir 404 (time 1787851287.65 total 40.20 last 10.57)
 - mkdir 499 (time 1787851297.69 total 50.24 last 9.46)
 - mkdir 623 (time 1787851307.81 total 60.36 last 12.25)
 - mkdir 760 (time 1787851317.87 total 70.42 last 13.62)
 - mkdir 889 (time 1787851327.94 total 80.49 last 12.81)
total: 1000 mkdir in 88.82 seconds: 11.26 ops/second
 - mkdir 87 (time 1787851260.73 total 10.11 last 8.61)
 - mkdir 197 (time 1787851270.75 total 20.13 last 10.98)
 - mkdir 315 (time 1787851280.76 total 30.14 last 11.79)
 - mkdir 411 (time 1787851290.76 total 40.14 last 9.60)
 - mkdir 512 (time 1787851300.82 total 50.20 last 10.04)
 - mkdir 638 (time 1787851310.87 total 60.25 last 12.54)
 - mkdir 773 (time 1787851320.90 total 70.28 last 13.46)
 - mkdir 907 (time 1787851330.95 total 80.33 last 13.33)
total: 1000 mkdir in 87.68 seconds: 11.41 ops/second
 - mkdir 84 (time 1787851261.69 total 10.02 last 8.39)
 - mkdir 195 (time 1787851271.71 total 20.04 last 11.08)
 - mkdir 315 (time 1787851281.88 total 30.21 last 11.80)
 - mkdir 417 (time 1787851291.94 total 40.27 last 10.14)
 - mkdir 523 (time 1787851301.96 total 50.28 last 10.58)
 - mkdir 646 (time 1787851311.99 total 60.31 last 12.27)
 - mkdir 781 (time 1787851322.00 total 70.32 last 13.48)
 - mkdir 919 (time 1787851332.01 total 80.33 last 13.79)
total: 1000 mkdir in 87.14 seconds: 11.48 ops/second
 - mkdir 89 (time 1787851262.72 total 10.13 last 8.79)
 - mkdir 204 (time 1787851272.78 total 20.19 last 11.44)
 - mkdir 316 (time 1787851282.84 total 30.25 last 11.13)
 - mkdir 417 (time 1787851292.89 total 40.30 last 10.05)
 - mkdir 532 (time 1787851302.93 total 50.34 last 11.46)
 - mkdir 657 (time 1787851312.97 total 60.38 last 12.45)
 - mkdir 788 (time 1787851322.99 total 70.40 last 13.08)
 - mkdir 929 (time 1787851333.02 total 80.43 last 14.05)
total: 1000 mkdir in 86.44 seconds: 11.57 ops/second
 - mkdir 88 (time 1787851263.83 total 10.14 last 8.68)
 - mkdir 207 (time 1787851273.87 total 20.18 last 11.86)
 - mkdir 315 (time 1787851283.88 total 30.19 last 10.79)
 - mkdir 416 (time 1787851293.99 total 40.30 last 9.98)
 - mkdir 532 (time 1787851304.00 total 50.31 last 11.59)
 - mkdir 664 (time 1787851314.01 total 60.32 last 13.20)
 - mkdir 795 (time 1787851324.10 total 70.41 last 12.98)
 - mkdir 935 (time 1787851334.13 total 80.45 last 13.95)
total: 1000 mkdir in 86.27 seconds: 11.59 ops/second
 - mkdir 83 (time 1787851262.73 total 10.03 last 8.28)
 - mkdir 199 (time 1787851272.78 total 20.07 last 11.55)
 - mkdir 309 (time 1787851282.78 total 30.08 last 10.99)
 - mkdir 409 (time 1787851292.83 total 40.12 last 9.96)
 - mkdir 523 (time 1787851302.84 total 50.13 last 11.38)
 - mkdir 647 (time 1787851312.84 total 60.14 last 12.40)
 - mkdir 780 (time 1787851322.89 total 70.18 last 13.24)
 - mkdir 924 (time 1787851332.90 total 80.19 last 14.38)
total: 1000 mkdir in 87.29 seconds: 11.46 ops/second
 - mkdir 85 (time 1787851262.20 total 10.02 last 8.48)
 - mkdir 203 (time 1787851272.21 total 20.03 last 11.79)
 - mkdir 313 (time 1787851282.28 total 30.09 last 10.93)
 - mkdir 411 (time 1787851292.31 total 40.13 last 9.77)
 - mkdir 518 (time 1787851302.35 total 50.17 last 10.65)
 - mkdir 638 (time 1787851312.42 total 60.24 last 11.92)
 - mkdir 771 (time 1787851322.46 total 70.28 last 13.25)
 - mkdir 914 (time 1787851332.49 total 80.31 last 14.26)
total: 1000 mkdir in 87.98 seconds: 11.37 ops/second
 - mkdir 92 (time 1787851263.78 total 10.05 last 9.15)
 - mkdir 204 (time 1787851273.79 total 20.07 last 11.18)
 - mkdir 307 (time 1787851283.85 total 30.12 last 10.24)
 - mkdir 408 (time 1787851293.98 total 40.25 last 9.97)
 - mkdir 522 (time 1787851303.98 total 50.25 last 11.40)
 - mkdir 652 (time 1787851314.04 total 60.31 last 12.93)
 - mkdir 778 (time 1787851324.10 total 70.37 last 12.52)
 - mkdir 924 (time 1787851334.13 total 80.41 last 14.55)
total: 1000 mkdir in 88.58 seconds: 11.29 ops/second
 - mkdir 83 (time 1787851262.65 total 10.04 last 8.27)
 - mkdir 194 (time 1787851272.65 total 20.04 last 11.10)
 - mkdir 299 (time 1787851282.70 total 30.09 last 10.45)
 - mkdir 401 (time 1787851292.78 total 40.17 last 10.12)
 - mkdir 511 (time 1787851302.79 total 50.18 last 10.98)
 - mkdir 639 (time 1787851312.82 total 60.21 last 12.77)
 - mkdir 770 (time 1787851322.89 total 70.28 last 13.01)
 - mkdir 907 (time 1787851332.90 total 80.28 last 13.69)
total: 1000 mkdir in 89.91 seconds: 11.12 ops/second
 - mkdir 82 (time 1787851263.68 total 10.03 last 8.18)
 - mkdir 197 (time 1787851273.69 total 20.03 last 11.50)
 - mkdir 305 (time 1787851283.73 total 30.08 last 10.75)
 - mkdir 408 (time 1787851293.81 total 40.15 last 10.22)
 - mkdir 526 (time 1787851303.87 total 50.21 last 11.73)
 - mkdir 657 (time 1787851313.92 total 60.26 last 13.03)
 - mkdir 781 (time 1787851323.94 total 70.28 last 12.38)
 - mkdir 919 (time 1787851333.98 total 80.32 last 13.74)
total: 1000 mkdir in 89.64 seconds: 11.16 ops/second
Time for file creation 104 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787851356 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1787851356 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1787851356 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1787851357 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1787851357 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1787851358 ; total 1 ; last 1)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1787851357 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1787851357 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1787851358 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1787851357 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
Time for file removal 57 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8