== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 10:03:17 (1787407397)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.63 seconds: 15.92 ops/second
 - mkdir 135 (time 1787407419.31 total 10.01 last 13.48)
 - mkdir 251 (time 1787407429.38 total 20.08 last 11.52)
 - mkdir 395 (time 1787407439.40 total 30.10 last 14.38)
 - mkdir 531 (time 1787407449.40 total 40.11 last 13.59)
 - mkdir 681 (time 1787407459.43 total 50.13 last 14.96)
 - mkdir 846 (time 1787407469.45 total 60.16 last 16.46)
total: 1000 mkdir in 70.15 seconds: 14.26 ops/second
 - mkdir 137 (time 1787407420.53 total 10.03 last 13.66)
 - mkdir 250 (time 1787407430.64 total 20.15 last 11.17)
 - mkdir 403 (time 1787407440.67 total 30.17 last 15.27)
 - mkdir 531 (time 1787407450.76 total 40.26 last 12.68)
 - mkdir 679 (time 1787407460.78 total 50.28 last 14.77)
 - mkdir 843 (time 1787407470.78 total 60.28 last 16.40)
 - mkdir 986 (time 1787407480.81 total 70.31 last 14.27)
total: 1000 mkdir in 71.91 seconds: 13.91 ops/second
 - mkdir 129 (time 1787407421.10 total 10.05 last 12.83)
 - mkdir 245 (time 1787407431.14 total 20.08 last 11.56)
 - mkdir 398 (time 1787407441.17 total 30.12 last 15.24)
 - mkdir 524 (time 1787407451.22 total 40.17 last 12.54)
 - mkdir 673 (time 1787407461.23 total 50.18 last 14.88)
 - mkdir 845 (time 1787407471.25 total 60.20 last 17.16)
 - mkdir 987 (time 1787407481.26 total 70.21 last 14.18)
total: 1000 mkdir in 71.68 seconds: 13.95 ops/second
 - mkdir 131 (time 1787407419.97 total 10.06 last 13.02)
 - mkdir 240 (time 1787407429.99 total 20.08 last 10.88)
 - mkdir 387 (time 1787407440.01 total 30.09 last 14.68)
 - mkdir 521 (time 1787407450.13 total 40.22 last 13.24)
 - mkdir 670 (time 1787407460.18 total 50.27 last 14.82)
 - mkdir 836 (time 1787407470.28 total 60.37 last 16.44)
 - mkdir 981 (time 1787407480.32 total 70.41 last 14.45)
total: 1000 mkdir in 72.84 seconds: 13.73 ops/second
 - mkdir 126 (time 1787407419.51 total 10.05 last 12.54)
 - mkdir 233 (time 1787407429.52 total 20.06 last 10.69)
 - mkdir 372 (time 1787407439.52 total 30.06 last 13.89)
 - mkdir 508 (time 1787407449.59 total 40.13 last 13.52)
 - mkdir 655 (time 1787407459.72 total 50.26 last 14.51)
 - mkdir 827 (time 1787407469.79 total 60.33 last 17.08)
 - mkdir 975 (time 1787407479.94 total 70.48 last 14.58)
total: 1000 mkdir in 73.40 seconds: 13.62 ops/second
 - mkdir 131 (time 1787407420.52 total 10.06 last 13.02)
 - mkdir 242 (time 1787407430.55 total 20.09 last 11.07)
 - mkdir 392 (time 1787407440.60 total 30.14 last 14.92)
 - mkdir 524 (time 1787407450.70 total 40.24 last 13.08)
 - mkdir 677 (time 1787407460.78 total 50.32 last 15.18)
 - mkdir 841 (time 1787407470.83 total 60.37 last 16.31)
 - mkdir 983 (time 1787407480.98 total 70.52 last 13.99)
total: 1000 mkdir in 72.51 seconds: 13.79 ops/second
 - mkdir 133 (time 1787407420.24 total 10.01 last 13.29)
 - mkdir 245 (time 1787407430.24 total 20.01 last 11.19)
 - mkdir 391 (time 1787407440.26 total 30.03 last 14.57)
 - mkdir 520 (time 1787407450.34 total 40.11 last 12.80)
 - mkdir 669 (time 1787407460.35 total 50.12 last 14.89)
 - mkdir 835 (time 1787407470.40 total 60.17 last 16.51)
 - mkdir 977 (time 1787407480.48 total 70.25 last 14.08)
total: 1000 mkdir in 73.14 seconds: 13.67 ops/second
 - mkdir 123 (time 1787407420.07 total 10.03 last 12.26)
 - mkdir 230 (time 1787407430.14 total 20.11 last 10.62)
 - mkdir 372 (time 1787407440.18 total 30.15 last 14.14)
 - mkdir 499 (time 1787407450.23 total 40.19 last 12.64)
 - mkdir 647 (time 1787407460.29 total 50.25 last 14.71)
 - mkdir 819 (time 1787407470.32 total 60.28 last 17.14)
 - mkdir 966 (time 1787407480.43 total 70.40 last 14.54)
total: 1000 mkdir in 74.67 seconds: 13.39 ops/second
 - mkdir 126 (time 1787407419.84 total 10.00 last 12.60)
 - mkdir 231 (time 1787407429.87 total 20.03 last 10.47)
 - mkdir 374 (time 1787407439.88 total 30.04 last 14.29)
 - mkdir 506 (time 1787407449.98 total 40.13 last 13.07)
 - mkdir 650 (time 1787407460.01 total 50.16 last 14.36)
 - mkdir 816 (time 1787407470.01 total 60.17 last 16.59)
 - mkdir 960 (time 1787407480.09 total 70.25 last 14.29)
total: 1000 mkdir in 75.23 seconds: 13.29 ops/second
 - mkdir 119 (time 1787407420.65 total 10.05 last 11.84)
 - mkdir 219 (time 1787407430.69 total 20.09 last 9.96)
 - mkdir 359 (time 1787407440.70 total 30.10 last 13.99)
 - mkdir 485 (time 1787407450.74 total 40.14 last 12.55)
 - mkdir 632 (time 1787407460.75 total 50.15 last 14.68)
 - mkdir 801 (time 1787407470.82 total 60.22 last 16.78)
 - mkdir 942 (time 1787407480.97 total 70.37 last 13.90)
total: 1000 mkdir in 76.51 seconds: 13.07 ops/second
Time for file creation 88 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 15 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.50 seconds: 20.18 ops/second
 - mkdir 152 (time 1787407992.04 total 10.05 last 15.12)
 - mkdir 284 (time 1787408002.11 total 20.12 last 13.11)
 - mkdir 432 (time 1787408012.15 total 30.15 last 14.75)
 - mkdir 594 (time 1787408022.16 total 40.17 last 16.18)
 - mkdir 781 (time 1787408032.21 total 50.21 last 18.61)
total: 1000 mkdir in 60.14 seconds: 16.63 ops/second
 - mkdir 161 (time 1787407993.54 total 10.02 last 16.07)
 - mkdir 302 (time 1787408003.57 total 20.05 last 14.05)
 - mkdir 446 (time 1787408013.63 total 30.11 last 14.31)
 - mkdir 610 (time 1787408023.69 total 40.16 last 16.31)
 - mkdir 786 (time 1787408033.71 total 50.19 last 17.55)
total: 1000 mkdir in 60.19 seconds: 16.61 ops/second
 - mkdir 149 (time 1787407991.72 total 10.02 last 14.86)
 - mkdir 273 (time 1787408001.77 total 20.08 last 12.33)
 - mkdir 416 (time 1787408011.82 total 30.13 last 14.23)
 - mkdir 574 (time 1787408021.84 total 40.15 last 15.77)
 - mkdir 759 (time 1787408031.89 total 50.20 last 18.40)
 - mkdir 974 (time 1787408041.91 total 60.22 last 21.46)
total: 1000 mkdir in 62.13 seconds: 16.10 ops/second
 - mkdir 160 (time 1787407994.55 total 10.01 last 15.99)
 - mkdir 299 (time 1787408004.61 total 20.07 last 13.81)
 - mkdir 452 (time 1787408014.68 total 30.14 last 15.19)
 - mkdir 623 (time 1787408024.70 total 40.17 last 17.06)
 - mkdir 796 (time 1787408034.75 total 50.21 last 17.23)
total: 1000 mkdir in 59.56 seconds: 16.79 ops/second
 - mkdir 157 (time 1787407993.87 total 10.05 last 15.62)
 - mkdir 296 (time 1787408003.90 total 20.09 last 13.85)
 - mkdir 444 (time 1787408013.93 total 30.11 last 14.76)
 - mkdir 606 (time 1787408023.97 total 40.15 last 16.14)
 - mkdir 782 (time 1787408034.03 total 50.21 last 17.49)
 - mkdir 994 (time 1787408044.05 total 60.23 last 21.15)
total: 1000 mkdir in 60.75 seconds: 16.46 ops/second
 - mkdir 157 (time 1787407993.99 total 10.05 last 15.62)
 - mkdir 299 (time 1787408004.01 total 20.08 last 14.17)
 - mkdir 443 (time 1787408014.05 total 30.12 last 14.34)
 - mkdir 607 (time 1787408024.09 total 40.16 last 16.34)
 - mkdir 782 (time 1787408034.11 total 50.17 last 17.47)
 - mkdir 993 (time 1787408044.19 total 60.25 last 20.93)
total: 1000 mkdir in 61.04 seconds: 16.38 ops/second
 - mkdir 157 (time 1787407994.09 total 10.04 last 15.64)
 - mkdir 297 (time 1787408004.12 total 20.06 last 13.97)
 - mkdir 440 (time 1787408014.23 total 30.17 last 14.14)
 - mkdir 607 (time 1787408024.27 total 40.21 last 16.63)
 - mkdir 782 (time 1787408034.36 total 50.30 last 17.34)
 - mkdir 991 (time 1787408044.41 total 60.35 last 20.79)
total: 1000 mkdir in 61.42 seconds: 16.28 ops/second
 - mkdir 154 (time 1787407993.90 total 10.00 last 15.40)
 - mkdir 291 (time 1787408003.93 total 20.04 last 13.65)
 - mkdir 436 (time 1787408013.95 total 30.05 last 14.47)
 - mkdir 600 (time 1787408024.02 total 40.13 last 16.28)
 - mkdir 775 (time 1787408034.08 total 50.18 last 17.40)
 - mkdir 988 (time 1787408044.16 total 60.27 last 21.12)
total: 1000 mkdir in 61.65 seconds: 16.22 ops/second
 - mkdir 154 (time 1787407993.80 total 10.02 last 15.37)
 - mkdir 289 (time 1787408003.81 total 20.02 last 13.50)
 - mkdir 437 (time 1787408013.86 total 30.08 last 14.72)
 - mkdir 596 (time 1787408023.91 total 40.13 last 15.82)
 - mkdir 776 (time 1787408033.94 total 50.15 last 17.96)
 - mkdir 980 (time 1787408043.99 total 60.21 last 20.29)
total: 1000 mkdir in 62.37 seconds: 16.03 ops/second
 - mkdir 149 (time 1787407993.84 total 10.01 last 14.89)
 - mkdir 281 (time 1787408003.85 total 20.02 last 13.18)
 - mkdir 420 (time 1787408013.88 total 30.04 last 13.87)
 - mkdir 583 (time 1787408023.90 total 40.07 last 16.26)
 - mkdir 757 (time 1787408033.91 total 50.08 last 17.39)
 - mkdir 970 (time 1787408043.98 total 60.15 last 21.15)
total: 1000 mkdir in 63.31 seconds: 15.80 ops/second
Time for file creation 74 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787408055 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1787408057 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787408056 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1787408057 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787408057 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787408057 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787408057 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787408057 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787408058 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787408057 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
Time for file removal 50 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8