== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 23:19:35 (1785986375)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.74 seconds: 13.51 ops/second
 - mkdir 149 (time 1785986400.09 total 10.06 last 14.81)
 - mkdir 295 (time 1785986410.12 total 20.09 last 14.57)
 - mkdir 490 (time 1785986420.12 total 30.09 last 19.49)
 - mkdir 632 (time 1785986430.13 total 40.10 last 14.19)
 - mkdir 803 (time 1785986440.18 total 50.15 last 17.00)
 - mkdir 951 (time 1785986450.21 total 60.18 last 14.76)
total: 1000 mkdir in 63.10 seconds: 15.85 ops/second
 - mkdir 141 (time 1785986398.20 total 10.08 last 13.98)
 - mkdir 283 (time 1785986408.23 total 20.12 last 14.15)
 - mkdir 454 (time 1785986418.29 total 30.17 last 17.01)
 - mkdir 610 (time 1785986428.43 total 40.32 last 15.38)
 - mkdir 775 (time 1785986438.44 total 50.32 last 16.49)
 - mkdir 924 (time 1785986448.51 total 60.40 last 14.79)
total: 1000 mkdir in 65.09 seconds: 15.36 ops/second
 - mkdir 148 (time 1785986400.40 total 10.02 last 14.77)
 - mkdir 296 (time 1785986410.44 total 20.07 last 14.73)
 - mkdir 489 (time 1785986420.49 total 30.11 last 19.21)
 - mkdir 630 (time 1785986430.51 total 40.13 last 14.07)
 - mkdir 802 (time 1785986440.55 total 50.17 last 17.13)
 - mkdir 950 (time 1785986450.56 total 60.19 last 14.78)
total: 1000 mkdir in 63.71 seconds: 15.70 ops/second
 - mkdir 154 (time 1785986401.76 total 10.02 last 15.37)
 - mkdir 296 (time 1785986411.77 total 20.02 last 14.19)
 - mkdir 497 (time 1785986421.78 total 30.03 last 20.09)
 - mkdir 644 (time 1785986431.80 total 40.06 last 14.66)
 - mkdir 804 (time 1785986441.81 total 50.07 last 15.99)
 - mkdir 960 (time 1785986451.86 total 60.11 last 15.53)
total: 1000 mkdir in 64.04 seconds: 15.62 ops/second
 - mkdir 156 (time 1785986401.75 total 10.10 last 15.44)
 - mkdir 300 (time 1785986411.75 total 20.11 last 14.39)
 - mkdir 493 (time 1785986421.76 total 30.11 last 19.29)
 - mkdir 643 (time 1785986431.77 total 40.12 last 14.99)
 - mkdir 804 (time 1785986441.80 total 50.16 last 16.04)
 - mkdir 959 (time 1785986451.82 total 60.17 last 15.48)
total: 1000 mkdir in 64.41 seconds: 15.53 ops/second
 - mkdir 154 (time 1785986401.72 total 10.08 last 15.28)
 - mkdir 291 (time 1785986411.73 total 20.09 last 13.69)
 - mkdir 489 (time 1785986421.79 total 30.15 last 19.68)
 - mkdir 639 (time 1785986431.83 total 40.19 last 14.94)
 - mkdir 801 (time 1785986441.85 total 50.21 last 16.16)
 - mkdir 957 (time 1785986451.95 total 60.31 last 15.45)
total: 1000 mkdir in 64.78 seconds: 15.44 ops/second
 - mkdir 157 (time 1785986401.46 total 10.08 last 15.58)
 - mkdir 296 (time 1785986411.48 total 20.11 last 13.86)
 - mkdir 491 (time 1785986421.51 total 30.13 last 19.45)
 - mkdir 638 (time 1785986431.53 total 40.15 last 14.68)
 - mkdir 795 (time 1785986441.56 total 50.18 last 15.65)
 - mkdir 950 (time 1785986451.65 total 60.27 last 15.37)
total: 1000 mkdir in 65.39 seconds: 15.29 ops/second
 - mkdir 150 (time 1785986401.78 total 10.05 last 14.93)
 - mkdir 297 (time 1785986411.78 total 20.05 last 14.69)
 - mkdir 490 (time 1785986421.82 total 30.08 last 19.24)
 - mkdir 637 (time 1785986431.87 total 40.14 last 14.62)
 - mkdir 795 (time 1785986441.88 total 50.15 last 15.78)
 - mkdir 950 (time 1785986451.96 total 60.23 last 15.38)
total: 1000 mkdir in 65.12 seconds: 15.36 ops/second
 - mkdir 153 (time 1785986401.55 total 10.06 last 15.21)
 - mkdir 296 (time 1785986411.59 total 20.09 last 14.25)
 - mkdir 492 (time 1785986421.66 total 30.16 last 19.47)
 - mkdir 637 (time 1785986431.71 total 40.21 last 14.43)
 - mkdir 797 (time 1785986441.76 total 50.26 last 15.92)
 - mkdir 950 (time 1785986451.78 total 60.29 last 15.26)
total: 1000 mkdir in 65.59 seconds: 15.25 ops/second
 - mkdir 138 (time 1785986399.96 total 10.02 last 13.77)
 - mkdir 275 (time 1785986410.02 total 20.07 last 13.63)
 - mkdir 454 (time 1785986420.08 total 30.13 last 17.79)
 - mkdir 594 (time 1785986430.13 total 40.19 last 13.92)
 - mkdir 756 (time 1785986440.14 total 50.19 last 16.19)
 - mkdir 902 (time 1785986450.14 total 60.20 last 14.60)
total: 1000 mkdir in 68.94 seconds: 14.51 ops/second
Time for file creation 80 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.83 seconds: 12.10 ops/second
 - mkdir 88 (time 1785986979.02 total 10.01 last 8.79)
 - mkdir 227 (time 1785986989.03 total 20.03 last 13.88)
 - mkdir 387 (time 1785986999.07 total 30.07 last 15.94)
 - mkdir 565 (time 1785987009.12 total 40.12 last 17.71)
 - mkdir 820 (time 1785987019.14 total 50.14 last 25.45)
total: 1000 mkdir in 57.98 seconds: 17.25 ops/second
 - mkdir 104 (time 1785986981.87 total 10.08 last 10.32)
 - mkdir 257 (time 1785986991.90 total 20.11 last 15.26)
 - mkdir 425 (time 1785987001.91 total 30.12 last 16.78)
 - mkdir 607 (time 1785987011.91 total 40.12 last 18.19)
 - mkdir 883 (time 1785987021.92 total 50.13 last 27.57)
total: 1000 mkdir in 56.11 seconds: 17.82 ops/second
 - mkdir 96 (time 1785986981.23 total 10.04 last 9.57)
 - mkdir 240 (time 1785986991.27 total 20.08 last 14.34)
 - mkdir 403 (time 1785987001.28 total 30.08 last 16.30)
 - mkdir 582 (time 1785987011.29 total 40.09 last 17.88)
 - mkdir 863 (time 1785987021.31 total 50.12 last 28.02)
total: 1000 mkdir in 57.44 seconds: 17.41 ops/second
 - mkdir 112 (time 1785986983.18 total 10.06 last 11.13)
 - mkdir 257 (time 1785986993.18 total 20.06 last 14.50)
 - mkdir 428 (time 1785987003.25 total 30.13 last 16.98)
 - mkdir 635 (time 1785987013.29 total 40.17 last 20.63)
 - mkdir 899 (time 1785987023.33 total 50.21 last 26.30)
total: 1000 mkdir in 56.55 seconds: 17.68 ops/second
 - mkdir 109 (time 1785986982.81 total 10.05 last 10.85)
 - mkdir 253 (time 1785986992.88 total 20.11 last 14.30)
 - mkdir 420 (time 1785987002.90 total 30.14 last 16.66)
 - mkdir 616 (time 1785987012.91 total 40.14 last 19.59)
 - mkdir 883 (time 1785987022.92 total 50.15 last 26.67)
total: 1000 mkdir in 57.04 seconds: 17.53 ops/second
 - mkdir 116 (time 1785986983.47 total 10.03 last 11.56)
 - mkdir 258 (time 1785986993.54 total 20.10 last 14.10)
 - mkdir 426 (time 1785987003.56 total 30.12 last 16.77)
 - mkdir 634 (time 1785987013.56 total 40.13 last 20.79)
 - mkdir 894 (time 1785987023.61 total 50.18 last 25.87)
total: 1000 mkdir in 56.92 seconds: 17.57 ops/second
 - mkdir 114 (time 1785986983.65 total 10.13 last 11.26)
 - mkdir 258 (time 1785986993.67 total 20.14 last 14.38)
 - mkdir 427 (time 1785987003.71 total 30.18 last 16.83)
 - mkdir 631 (time 1785987013.73 total 40.21 last 20.35)
 - mkdir 890 (time 1785987023.78 total 50.26 last 25.77)
total: 1000 mkdir in 58.29 seconds: 17.16 ops/second
 - mkdir 95 (time 1785986981.82 total 10.05 last 9.45)
 - mkdir 239 (time 1785986991.88 total 20.12 last 14.31)
 - mkdir 396 (time 1785987001.90 total 30.14 last 15.67)
 - mkdir 573 (time 1785987011.91 total 40.14 last 17.69)
 - mkdir 851 (time 1785987021.95 total 50.19 last 27.67)
 - mkdir 999 (time 1785987032.02 total 60.26 last 14.70)
total: 1000 mkdir in 60.26 seconds: 16.60 ops/second
 - mkdir 111 (time 1785986983.96 total 10.00 last 11.09)
 - mkdir 255 (time 1785986993.96 total 20.01 last 14.40)
 - mkdir 429 (time 1785987003.97 total 30.01 last 17.39)
 - mkdir 632 (time 1785987013.97 total 40.01 last 20.30)
 - mkdir 886 (time 1785987024.02 total 50.06 last 25.26)
total: 1000 mkdir in 58.67 seconds: 17.04 ops/second
 - mkdir 100 (time 1785986983.10 total 10.08 last 9.92)
 - mkdir 234 (time 1785986993.13 total 20.11 last 13.36)
 - mkdir 395 (time 1785987003.17 total 30.15 last 16.04)
 - mkdir 592 (time 1785987013.20 total 40.18 last 19.64)
 - mkdir 844 (time 1785987023.21 total 50.19 last 25.17)
 - mkdir 966 (time 1785987033.26 total 60.24 last 12.14)
total: 1000 mkdir in 62.14 seconds: 16.09 ops/second
Time for file creation 75 sec for 1 RPC sent at a time
 - unlinked 0 (time 1785987043 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1785987045 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1785987045 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1785987046 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1785987046 ; total 1 ; last 1)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1785987046 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1785987046 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1785987046 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1785987046 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1785987046 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
Time for file removal 42 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8