== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 22:02:26 (1787623346)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.91 seconds: 10.99 ops/second
 - mkdir 142 (time 1787623374.78 total 10.01 last 14.18)
 - mkdir 355 (time 1787623384.86 total 20.09 last 21.15)
 - mkdir 534 (time 1787623394.94 total 30.17 last 17.74)
 - mkdir 647 (time 1787623405.03 total 40.26 last 11.20)
 - mkdir 748 (time 1787623415.16 total 50.39 last 9.97)
 - mkdir 855 (time 1787623425.30 total 60.53 last 10.55)
 - mkdir 985 (time 1787623435.31 total 70.54 last 12.98)
total: 1000 mkdir in 71.75 seconds: 13.94 ops/second
 - mkdir 129 (time 1787623374.18 total 10.02 last 12.87)
 - mkdir 343 (time 1787623384.20 total 20.04 last 21.37)
 - mkdir 516 (time 1787623394.26 total 30.10 last 17.20)
 - mkdir 631 (time 1787623404.33 total 40.18 last 11.41)
 - mkdir 735 (time 1787623414.34 total 50.18 last 10.39)
 - mkdir 840 (time 1787623424.46 total 60.31 last 10.37)
 - mkdir 971 (time 1787623434.55 total 70.39 last 12.99)
total: 1000 mkdir in 72.62 seconds: 13.77 ops/second
 - mkdir 128 (time 1787623374.05 total 10.05 last 12.74)
 - mkdir 343 (time 1787623384.10 total 20.10 last 21.39)
 - mkdir 515 (time 1787623394.17 total 30.17 last 17.08)
 - mkdir 632 (time 1787623404.31 total 40.31 last 11.54)
 - mkdir 733 (time 1787623414.35 total 50.34 last 10.07)
 - mkdir 837 (time 1787623424.39 total 60.39 last 10.36)
 - mkdir 963 (time 1787623434.40 total 70.40 last 12.58)
total: 1000 mkdir in 73.69 seconds: 13.57 ops/second
 - mkdir 125 (time 1787623373.70 total 10.04 last 12.45)
 - mkdir 341 (time 1787623383.78 total 20.13 last 21.42)
 - mkdir 514 (time 1787623393.83 total 30.17 last 17.22)
 - mkdir 630 (time 1787623403.90 total 40.25 last 11.52)
 - mkdir 730 (time 1787623413.98 total 50.32 last 9.93)
 - mkdir 832 (time 1787623424.03 total 60.37 last 10.14)
 - mkdir 954 (time 1787623434.12 total 70.46 last 12.09)
total: 1000 mkdir in 74.67 seconds: 13.39 ops/second
 - mkdir 128 (time 1787623374.00 total 10.04 last 12.74)
 - mkdir 340 (time 1787623384.05 total 20.09 last 21.10)
 - mkdir 512 (time 1787623394.06 total 30.10 last 17.18)
 - mkdir 629 (time 1787623404.20 total 40.24 last 11.54)
 - mkdir 730 (time 1787623414.23 total 50.27 last 10.07)
 - mkdir 834 (time 1787623424.25 total 60.29 last 10.37)
 - mkdir 954 (time 1787623434.28 total 70.32 last 11.96)
total: 1000 mkdir in 75.29 seconds: 13.28 ops/second
 - mkdir 130 (time 1787623374.42 total 10.01 last 12.98)
 - mkdir 341 (time 1787623384.47 total 20.07 last 20.99)
 - mkdir 515 (time 1787623394.52 total 30.12 last 17.31)
 - mkdir 627 (time 1787623404.66 total 40.25 last 11.05)
 - mkdir 727 (time 1787623414.73 total 50.33 last 9.93)
 - mkdir 833 (time 1787623424.76 total 60.35 last 10.58)
 - mkdir 958 (time 1787623434.81 total 70.40 last 12.43)
total: 1000 mkdir in 75.15 seconds: 13.31 ops/second
 - mkdir 158 (time 1787623375.89 total 10.02 last 15.77)
 - mkdir 358 (time 1787623386.01 total 20.14 last 19.75)
 - mkdir 531 (time 1787623396.18 total 30.31 last 17.02)
 - mkdir 641 (time 1787623406.24 total 40.37 last 10.94)
 - mkdir 739 (time 1787623416.27 total 50.40 last 9.77)
 - mkdir 842 (time 1787623426.32 total 60.45 last 10.25)
 - mkdir 970 (time 1787623436.33 total 70.46 last 12.79)
total: 1000 mkdir in 74.40 seconds: 13.44 ops/second
 - mkdir 157 (time 1787623375.83 total 10.01 last 15.68)
 - mkdir 356 (time 1787623385.89 total 20.06 last 19.80)
 - mkdir 523 (time 1787623395.91 total 30.08 last 16.67)
 - mkdir 632 (time 1787623405.95 total 40.13 last 10.85)
 - mkdir 730 (time 1787623415.97 total 50.15 last 9.78)
 - mkdir 839 (time 1787623426.04 total 60.21 last 10.83)
 - mkdir 965 (time 1787623436.12 total 70.29 last 12.50)
total: 1000 mkdir in 75.04 seconds: 13.33 ops/second
 - mkdir 133 (time 1787623374.85 total 10.00 last 13.29)
 - mkdir 336 (time 1787623384.86 total 20.01 last 20.29)
 - mkdir 505 (time 1787623394.89 total 30.05 last 16.83)
 - mkdir 613 (time 1787623404.92 total 40.08 last 10.77)
 - mkdir 713 (time 1787623415.08 total 50.23 last 9.84)
 - mkdir 823 (time 1787623425.10 total 60.25 last 10.98)
 - mkdir 951 (time 1787623435.22 total 70.37 last 12.64)
total: 1000 mkdir in 76.58 seconds: 13.06 ops/second
 - mkdir 123 (time 1787623374.01 total 10.04 last 12.25)
 - mkdir 328 (time 1787623384.05 total 20.08 last 20.42)
 - mkdir 498 (time 1787623394.06 total 30.10 last 16.97)
 - mkdir 612 (time 1787623404.20 total 40.24 last 11.25)
 - mkdir 712 (time 1787623414.34 total 50.38 last 9.85)
 - mkdir 817 (time 1787623424.54 total 60.58 last 10.30)
 - mkdir 939 (time 1787623434.58 total 70.62 last 12.14)
total: 1000 mkdir in 78.57 seconds: 12.73 ops/second
Time for file creation 93 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 22 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.49 seconds: 20.34 ops/second
 - mkdir 136 (time 1787624174.96 total 10.05 last 13.53)
 - mkdir 293 (time 1787624184.99 total 20.08 last 15.65)
 - mkdir 431 (time 1787624195.06 total 30.15 last 13.70)
 - mkdir 565 (time 1787624205.10 total 40.19 last 13.35)
 - mkdir 718 (time 1787624215.13 total 50.22 last 15.25)
 - mkdir 907 (time 1787624225.16 total 60.25 last 18.85)
total: 1000 mkdir in 65.90 seconds: 15.17 ops/second
 - mkdir 134 (time 1787624174.32 total 10.08 last 13.30)
 - mkdir 289 (time 1787624184.40 total 20.16 last 15.37)
 - mkdir 424 (time 1787624194.41 total 30.17 last 13.49)
 - mkdir 554 (time 1787624204.49 total 40.24 last 12.90)
 - mkdir 698 (time 1787624214.49 total 50.25 last 14.39)
 - mkdir 884 (time 1787624224.50 total 60.26 last 18.58)
total: 1000 mkdir in 67.09 seconds: 14.90 ops/second
 - mkdir 127 (time 1787624173.71 total 10.03 last 12.67)
 - mkdir 284 (time 1787624183.74 total 20.06 last 15.65)
 - mkdir 412 (time 1787624193.75 total 30.07 last 12.79)
 - mkdir 544 (time 1787624203.81 total 40.13 last 13.12)
 - mkdir 684 (time 1787624213.83 total 50.15 last 13.97)
 - mkdir 856 (time 1787624223.83 total 60.15 last 17.20)
total: 1000 mkdir in 69.03 seconds: 14.49 ops/second
 - mkdir 117 (time 1787624173.20 total 10.07 last 11.62)
 - mkdir 273 (time 1787624183.21 total 20.08 last 15.59)
 - mkdir 409 (time 1787624193.24 total 30.11 last 13.56)
 - mkdir 535 (time 1787624203.38 total 40.25 last 12.43)
 - mkdir 670 (time 1787624213.42 total 50.29 last 13.44)
 - mkdir 837 (time 1787624223.44 total 60.31 last 16.67)
 - mkdir 999 (time 1787624233.52 total 70.39 last 16.08)
total: 1000 mkdir in 70.39 seconds: 14.21 ops/second
 - mkdir 132 (time 1787624174.62 total 10.13 last 13.04)
 - mkdir 284 (time 1787624184.71 total 20.21 last 15.07)
 - mkdir 420 (time 1787624194.74 total 30.25 last 13.56)
 - mkdir 554 (time 1787624204.81 total 40.31 last 13.31)
 - mkdir 692 (time 1787624214.81 total 50.32 last 13.80)
 - mkdir 876 (time 1787624224.86 total 60.36 last 18.32)
total: 1000 mkdir in 69.06 seconds: 14.48 ops/second
 - mkdir 117 (time 1787624173.01 total 10.05 last 11.64)
 - mkdir 276 (time 1787624183.07 total 20.11 last 15.80)
 - mkdir 412 (time 1787624193.16 total 30.20 last 13.48)
 - mkdir 537 (time 1787624203.17 total 40.22 last 12.48)
 - mkdir 668 (time 1787624213.24 total 50.28 last 13.02)
 - mkdir 833 (time 1787624223.27 total 60.31 last 16.44)
 - mkdir 992 (time 1787624233.28 total 70.33 last 15.88)
total: 1000 mkdir in 71.43 seconds: 14.00 ops/second
 - mkdir 128 (time 1787624173.81 total 10.03 last 12.77)
 - mkdir 278 (time 1787624183.84 total 20.05 last 14.96)
 - mkdir 407 (time 1787624193.84 total 30.05 last 12.90)
 - mkdir 530 (time 1787624203.99 total 40.21 last 12.11)
 - mkdir 673 (time 1787624214.06 total 50.27 last 14.21)
 - mkdir 842 (time 1787624224.08 total 60.29 last 16.86)
 - mkdir 987 (time 1787624234.12 total 70.33 last 14.44)
total: 1000 mkdir in 72.12 seconds: 13.87 ops/second
 - mkdir 128 (time 1787624174.60 total 10.00 last 12.79)
 - mkdir 276 (time 1787624184.68 total 20.09 last 14.68)
 - mkdir 411 (time 1787624194.75 total 30.16 last 13.40)
 - mkdir 542 (time 1787624204.77 total 40.18 last 13.07)
 - mkdir 682 (time 1787624214.82 total 50.23 last 13.93)
 - mkdir 864 (time 1787624224.83 total 60.23 last 18.19)
 - mkdir 990 (time 1787624234.85 total 70.26 last 12.57)
total: 1000 mkdir in 71.38 seconds: 14.01 ops/second
 - mkdir 131 (time 1787624174.33 total 10.05 last 13.03)
 - mkdir 281 (time 1787624184.45 total 20.17 last 14.83)
 - mkdir 414 (time 1787624194.46 total 30.19 last 13.28)
 - mkdir 539 (time 1787624204.48 total 40.21 last 12.47)
 - mkdir 675 (time 1787624214.50 total 50.23 last 13.58)
 - mkdir 850 (time 1787624224.51 total 60.23 last 17.49)
 - mkdir 987 (time 1787624234.52 total 70.25 last 13.68)
total: 1000 mkdir in 72.02 seconds: 13.89 ops/second
 - mkdir 125 (time 1787624174.59 total 10.05 last 12.43)
 - mkdir 273 (time 1787624184.73 total 20.19 last 14.61)
 - mkdir 400 (time 1787624194.76 total 30.23 last 12.65)
 - mkdir 526 (time 1787624204.77 total 40.23 last 12.59)
 - mkdir 655 (time 1787624214.79 total 50.25 last 12.88)
 - mkdir 838 (time 1787624224.79 total 60.25 last 18.30)
 - mkdir 966 (time 1787624234.81 total 70.27 last 12.77)
total: 1000 mkdir in 73.42 seconds: 13.62 ops/second
Time for file creation 85 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787624247 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1787624246 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1787624247 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1787624247 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1787624247 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1787624247 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1787624248 ; total 1 ; last 1)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1787624248 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1787624247 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1787624247 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
Time for file removal 52 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8