== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 20:19:59 (1787876399)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 1.08 seconds: 9.25 ops/second
 - mkdir 148 (time 1787876426.42 total 10.01 last 14.79)
 - mkdir 304 (time 1787876436.44 total 20.04 last 15.56)
 - mkdir 454 (time 1787876446.46 total 30.05 last 14.98)
 - mkdir 587 (time 1787876456.49 total 40.08 last 13.25)
 - mkdir 727 (time 1787876466.55 total 50.14 last 13.92)
 - mkdir 879 (time 1787876476.59 total 60.18 last 15.14)
total: 1000 mkdir in 68.56 seconds: 14.59 ops/second
 - mkdir 150 (time 1787876426.42 total 10.05 last 14.93)
 - mkdir 303 (time 1787876436.44 total 20.07 last 15.27)
 - mkdir 451 (time 1787876446.46 total 30.10 last 14.76)
 - mkdir 586 (time 1787876456.49 total 40.12 last 13.47)
 - mkdir 726 (time 1787876466.49 total 50.12 last 14.00)
 - mkdir 877 (time 1787876476.55 total 60.19 last 15.00)
total: 1000 mkdir in 68.84 seconds: 14.53 ops/second
 - mkdir 129 (time 1787876425.12 total 10.04 last 12.85)
 - mkdir 276 (time 1787876435.13 total 20.05 last 14.70)
 - mkdir 434 (time 1787876445.14 total 30.06 last 15.78)
 - mkdir 562 (time 1787876455.14 total 40.06 last 12.80)
 - mkdir 698 (time 1787876465.15 total 50.07 last 13.59)
 - mkdir 845 (time 1787876475.15 total 60.07 last 14.69)
 - mkdir 996 (time 1787876485.19 total 70.11 last 15.05)
total: 1000 mkdir in 70.36 seconds: 14.21 ops/second
 - mkdir 104 (time 1787876421.89 total 10.02 last 10.38)
 - mkdir 250 (time 1787876431.93 total 20.07 last 14.53)
 - mkdir 399 (time 1787876441.97 total 30.10 last 14.85)
 - mkdir 523 (time 1787876452.04 total 40.17 last 12.31)
 - mkdir 659 (time 1787876462.14 total 50.27 last 13.47)
 - mkdir 813 (time 1787876472.21 total 60.34 last 15.29)
 - mkdir 951 (time 1787876482.25 total 70.38 last 13.75)
total: 1000 mkdir in 73.88 seconds: 13.53 ops/second
 - mkdir 131 (time 1787876425.00 total 10.03 last 13.06)
 - mkdir 277 (time 1787876435.01 total 20.04 last 14.59)
 - mkdir 429 (time 1787876445.09 total 30.12 last 15.07)
 - mkdir 553 (time 1787876455.18 total 40.21 last 12.30)
 - mkdir 692 (time 1787876465.21 total 50.24 last 13.85)
 - mkdir 843 (time 1787876475.23 total 60.26 last 15.07)
 - mkdir 991 (time 1787876485.25 total 70.28 last 14.77)
total: 1000 mkdir in 71.04 seconds: 14.08 ops/second
 - mkdir 136 (time 1787876426.08 total 10.06 last 13.53)
 - mkdir 282 (time 1787876436.08 total 20.06 last 14.60)
 - mkdir 437 (time 1787876446.13 total 30.10 last 15.43)
 - mkdir 567 (time 1787876456.16 total 40.14 last 12.95)
 - mkdir 710 (time 1787876466.21 total 50.19 last 14.23)
 - mkdir 862 (time 1787876476.25 total 60.23 last 15.14)
total: 1000 mkdir in 70.19 seconds: 14.25 ops/second
 - mkdir 135 (time 1787876425.74 total 10.05 last 13.44)
 - mkdir 279 (time 1787876435.78 total 20.09 last 14.35)
 - mkdir 432 (time 1787876445.79 total 30.09 last 15.29)
 - mkdir 559 (time 1787876455.85 total 40.15 last 12.62)
 - mkdir 703 (time 1787876465.89 total 50.20 last 14.34)
 - mkdir 852 (time 1787876475.90 total 60.20 last 14.89)
 - mkdir 994 (time 1787876485.90 total 70.21 last 14.20)
total: 1000 mkdir in 70.66 seconds: 14.15 ops/second
 - mkdir 132 (time 1787876424.99 total 10.05 last 13.14)
 - mkdir 271 (time 1787876435.01 total 20.07 last 13.87)
 - mkdir 418 (time 1787876445.13 total 30.19 last 14.53)
 - mkdir 541 (time 1787876455.21 total 40.27 last 12.20)
 - mkdir 683 (time 1787876465.26 total 50.32 last 14.13)
 - mkdir 831 (time 1787876475.32 total 60.38 last 14.71)
 - mkdir 982 (time 1787876485.43 total 70.49 last 14.94)
total: 1000 mkdir in 72.11 seconds: 13.87 ops/second
 - mkdir 141 (time 1787876426.34 total 10.08 last 13.99)
 - mkdir 291 (time 1787876436.36 total 20.10 last 14.97)
 - mkdir 438 (time 1787876446.39 total 30.12 last 14.66)
 - mkdir 562 (time 1787876456.39 total 40.13 last 12.39)
 - mkdir 706 (time 1787876466.41 total 50.15 last 14.37)
 - mkdir 856 (time 1787876476.46 total 60.19 last 14.93)
 - mkdir 988 (time 1787876486.46 total 70.20 last 13.20)
total: 1000 mkdir in 71.46 seconds: 13.99 ops/second
 - mkdir 133 (time 1787876425.93 total 10.08 last 13.19)
 - mkdir 278 (time 1787876435.94 total 20.09 last 14.50)
 - mkdir 426 (time 1787876445.96 total 30.11 last 14.76)
 - mkdir 551 (time 1787876455.99 total 40.13 last 12.47)
 - mkdir 694 (time 1787876466.01 total 50.16 last 14.26)
 - mkdir 845 (time 1787876476.03 total 60.18 last 15.07)
 - mkdir 981 (time 1787876486.13 total 70.27 last 13.48)
total: 1000 mkdir in 72.38 seconds: 13.82 ops/second
Time for file creation 88 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.47 seconds: 21.26 ops/second
 - mkdir 116 (time 1787877022.66 total 10.14 last 11.44)
 - mkdir 233 (time 1787877032.68 total 20.16 last 11.68)
 - mkdir 426 (time 1787877042.69 total 30.17 last 19.28)
 - mkdir 577 (time 1787877052.74 total 40.22 last 15.03)
 - mkdir 721 (time 1787877062.76 total 50.24 last 14.36)
 - mkdir 857 (time 1787877072.80 total 60.28 last 13.55)
total: 1000 mkdir in 68.20 seconds: 14.66 ops/second
 - mkdir 108 (time 1787877018.99 total 10.00 last 10.80)
 - mkdir 222 (time 1787877029.14 total 20.15 last 11.23)
 - mkdir 375 (time 1787877039.16 total 30.18 last 15.26)
 - mkdir 551 (time 1787877049.17 total 40.18 last 17.60)
 - mkdir 692 (time 1787877059.21 total 50.22 last 14.04)
 - mkdir 814 (time 1787877069.25 total 60.26 last 12.14)
 - mkdir 969 (time 1787877079.29 total 70.30 last 15.45)
total: 1000 mkdir in 72.83 seconds: 13.73 ops/second
 - mkdir 112 (time 1787877021.13 total 10.02 last 11.18)
 - mkdir 221 (time 1787877031.17 total 20.06 last 10.85)
 - mkdir 405 (time 1787877041.18 total 30.07 last 18.38)
 - mkdir 563 (time 1787877051.22 total 40.11 last 15.75)
 - mkdir 709 (time 1787877061.24 total 50.13 last 14.57)
 - mkdir 832 (time 1787877071.33 total 60.22 last 12.20)
 - mkdir 990 (time 1787877081.37 total 70.26 last 15.74)
total: 1000 mkdir in 71.63 seconds: 13.96 ops/second
 - mkdir 107 (time 1787877020.60 total 10.05 last 10.65)
 - mkdir 215 (time 1787877030.62 total 20.06 last 10.78)
 - mkdir 395 (time 1787877040.67 total 30.12 last 17.90)
 - mkdir 553 (time 1787877050.69 total 40.14 last 15.77)
 - mkdir 700 (time 1787877060.70 total 50.15 last 14.69)
 - mkdir 823 (time 1787877070.72 total 60.16 last 12.28)
 - mkdir 981 (time 1787877080.74 total 70.18 last 15.76)
total: 1000 mkdir in 72.38 seconds: 13.82 ops/second
 - mkdir 110 (time 1787877021.80 total 10.01 last 10.99)
 - mkdir 223 (time 1787877031.83 total 20.03 last 11.27)
 - mkdir 408 (time 1787877041.83 total 30.04 last 18.50)
 - mkdir 560 (time 1787877051.89 total 40.10 last 15.10)
 - mkdir 706 (time 1787877062.00 total 50.21 last 14.44)
 - mkdir 834 (time 1787877072.03 total 60.24 last 12.77)
 - mkdir 990 (time 1787877082.27 total 70.48 last 15.24)
total: 1000 mkdir in 71.64 seconds: 13.96 ops/second
 - mkdir 109 (time 1787877022.80 total 10.13 last 10.76)
 - mkdir 228 (time 1787877032.80 total 20.13 last 11.90)
 - mkdir 421 (time 1787877042.85 total 30.17 last 19.21)
 - mkdir 571 (time 1787877052.88 total 40.21 last 14.95)
 - mkdir 710 (time 1787877062.95 total 50.27 last 13.81)
 - mkdir 836 (time 1787877073.07 total 60.40 last 12.44)
 - mkdir 991 (time 1787877083.22 total 70.54 last 15.28)
total: 1000 mkdir in 71.69 seconds: 13.95 ops/second
 - mkdir 107 (time 1787877021.82 total 10.07 last 10.63)
 - mkdir 216 (time 1787877031.82 total 20.08 last 10.89)
 - mkdir 404 (time 1787877041.85 total 30.10 last 18.76)
 - mkdir 558 (time 1787877051.89 total 40.15 last 15.33)
 - mkdir 695 (time 1787877062.03 total 50.28 last 13.52)
 - mkdir 822 (time 1787877072.03 total 60.28 last 12.70)
 - mkdir 979 (time 1787877082.11 total 70.37 last 15.57)
total: 1000 mkdir in 73.54 seconds: 13.60 ops/second
 - mkdir 107 (time 1787877023.09 total 10.00 last 10.70)
 - mkdir 231 (time 1787877033.10 total 20.01 last 12.39)
 - mkdir 422 (time 1787877043.13 total 30.04 last 19.04)
 - mkdir 574 (time 1787877053.16 total 40.07 last 15.16)
 - mkdir 709 (time 1787877063.17 total 50.07 last 13.49)
 - mkdir 832 (time 1787877073.17 total 60.08 last 12.29)
 - mkdir 984 (time 1787877083.27 total 70.18 last 15.05)
total: 1000 mkdir in 72.66 seconds: 13.76 ops/second
 - mkdir 106 (time 1787877022.21 total 10.01 last 10.59)
 - mkdir 217 (time 1787877032.25 total 20.05 last 11.05)
 - mkdir 401 (time 1787877042.31 total 30.11 last 18.29)
 - mkdir 548 (time 1787877052.33 total 40.13 last 14.67)
 - mkdir 688 (time 1787877062.36 total 50.17 last 13.96)
 - mkdir 817 (time 1787877072.43 total 60.23 last 12.82)
 - mkdir 967 (time 1787877082.45 total 70.25 last 14.96)
total: 1000 mkdir in 74.90 seconds: 13.35 ops/second
 - mkdir 105 (time 1787877023.37 total 10.14 last 10.36)
 - mkdir 219 (time 1787877033.38 total 20.16 last 11.38)
 - mkdir 402 (time 1787877043.41 total 30.18 last 18.26)
 - mkdir 543 (time 1787877053.46 total 40.23 last 14.03)
 - mkdir 676 (time 1787877063.51 total 50.29 last 13.22)
 - mkdir 800 (time 1787877073.55 total 60.33 last 12.35)
 - mkdir 952 (time 1787877083.58 total 70.35 last 15.16)
total: 1000 mkdir in 76.20 seconds: 13.12 ops/second
Time for file creation 88 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787877095 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1787877096 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1787877096 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1787877097 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1787877096 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1787877097 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1787877097 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1787877097 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1787877097 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1787877097 ; total 0 ; last 0)
total: 1000 unlinks in 42 seconds: 23.809525 unlinks/second
Time for file removal 51 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8