== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 16:52:46 (1790974366)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.48 seconds: 20.79 ops/second
 - mkdir 117 (time 1790974388.07 total 10.11 last 11.57)
 - mkdir 251 (time 1790974398.08 total 20.13 last 13.38)
 - mkdir 404 (time 1790974408.10 total 30.15 last 15.27)
 - mkdir 548 (time 1790974418.19 total 40.24 last 14.27)
 - mkdir 655 (time 1790974428.33 total 50.37 last 10.56)
 - mkdir 769 (time 1790974438.45 total 60.50 last 11.26)
 - mkdir 903 (time 1790974448.49 total 70.53 last 13.36)
total: 1000 mkdir in 76.07 seconds: 13.15 ops/second
 - mkdir 111 (time 1790974388.50 total 10.03 last 11.06)
 - mkdir 243 (time 1790974398.57 total 20.10 last 13.11)
 - mkdir 396 (time 1790974408.62 total 30.15 last 15.22)
 - mkdir 538 (time 1790974418.73 total 40.26 last 14.05)
 - mkdir 644 (time 1790974428.87 total 50.40 last 10.45)
 - mkdir 758 (time 1790974438.90 total 60.43 last 11.37)
 - mkdir 894 (time 1790974448.91 total 70.44 last 13.59)
total: 1000 mkdir in 77.43 seconds: 12.91 ops/second
 - mkdir 118 (time 1790974389.93 total 10.09 last 11.70)
 - mkdir 254 (time 1790974399.97 total 20.13 last 13.54)
 - mkdir 404 (time 1790974410.01 total 30.17 last 14.95)
 - mkdir 538 (time 1790974420.06 total 40.22 last 13.33)
 - mkdir 642 (time 1790974430.16 total 50.31 last 10.30)
 - mkdir 766 (time 1790974440.18 total 60.34 last 12.37)
 - mkdir 914 (time 1790974450.20 total 70.36 last 14.77)
total: 1000 mkdir in 76.09 seconds: 13.14 ops/second
 - mkdir 114 (time 1790974389.79 total 10.00 last 11.40)
 - mkdir 253 (time 1790974399.81 total 20.02 last 13.87)
 - mkdir 399 (time 1790974409.83 total 30.04 last 14.57)
 - mkdir 534 (time 1790974419.97 total 40.18 last 13.31)
 - mkdir 637 (time 1790974430.10 total 50.32 last 10.16)
 - mkdir 756 (time 1790974440.11 total 60.32 last 11.89)
 - mkdir 906 (time 1790974450.13 total 70.35 last 14.96)
total: 1000 mkdir in 76.28 seconds: 13.11 ops/second
 - mkdir 113 (time 1790974390.22 total 10.04 last 11.26)
 - mkdir 255 (time 1790974400.24 total 20.06 last 14.17)
 - mkdir 405 (time 1790974410.26 total 30.08 last 14.97)
 - mkdir 538 (time 1790974420.28 total 40.10 last 13.27)
 - mkdir 642 (time 1790974430.30 total 50.11 last 10.38)
 - mkdir 760 (time 1790974440.33 total 60.14 last 11.77)
 - mkdir 913 (time 1790974450.35 total 70.17 last 15.26)
total: 1000 mkdir in 76.13 seconds: 13.13 ops/second
 - mkdir 112 (time 1790974390.20 total 10.04 last 11.16)
 - mkdir 261 (time 1790974400.23 total 20.06 last 14.86)
 - mkdir 408 (time 1790974410.24 total 30.07 last 14.69)
 - mkdir 539 (time 1790974420.30 total 40.14 last 13.01)
 - mkdir 641 (time 1790974430.31 total 50.14 last 10.20)
 - mkdir 763 (time 1790974440.33 total 60.16 last 12.18)
 - mkdir 912 (time 1790974450.37 total 70.20 last 14.84)
total: 1000 mkdir in 76.53 seconds: 13.07 ops/second
 - mkdir 116 (time 1790974390.48 total 10.01 last 11.59)
 - mkdir 264 (time 1790974400.52 total 20.05 last 14.74)
 - mkdir 417 (time 1790974410.53 total 30.06 last 15.28)
 - mkdir 545 (time 1790974420.58 total 40.11 last 12.73)
 - mkdir 645 (time 1790974430.60 total 50.13 last 9.98)
 - mkdir 764 (time 1790974440.66 total 60.19 last 11.83)
 - mkdir 913 (time 1790974450.69 total 70.22 last 14.85)
total: 1000 mkdir in 76.60 seconds: 13.05 ops/second
 - mkdir 117 (time 1790974390.55 total 10.06 last 11.63)
 - mkdir 259 (time 1790974400.56 total 20.06 last 14.20)
 - mkdir 404 (time 1790974410.56 total 30.06 last 14.50)
 - mkdir 533 (time 1790974420.60 total 40.11 last 12.84)
 - mkdir 636 (time 1790974430.64 total 50.15 last 10.27)
 - mkdir 760 (time 1790974440.66 total 60.17 last 12.37)
 - mkdir 911 (time 1790974450.70 total 70.21 last 15.04)
total: 1000 mkdir in 76.65 seconds: 13.05 ops/second
 - mkdir 110 (time 1790974390.18 total 10.08 last 10.91)
 - mkdir 247 (time 1790974400.20 total 20.10 last 13.67)
 - mkdir 394 (time 1790974410.26 total 30.16 last 14.62)
 - mkdir 525 (time 1790974420.32 total 40.22 last 13.02)
 - mkdir 625 (time 1790974430.37 total 50.26 last 9.96)
 - mkdir 746 (time 1790974440.41 total 60.31 last 12.04)
 - mkdir 893 (time 1790974450.44 total 70.34 last 14.65)
total: 1000 mkdir in 78.13 seconds: 12.80 ops/second
 - mkdir 107 (time 1790974389.18 total 10.03 last 10.67)
 - mkdir 233 (time 1790974399.23 total 20.07 last 12.54)
 - mkdir 374 (time 1790974409.25 total 30.09 last 14.07)
 - mkdir 513 (time 1790974419.29 total 40.14 last 13.84)
 - mkdir 620 (time 1790974429.40 total 50.24 last 10.59)
 - mkdir 736 (time 1790974439.51 total 60.35 last 11.48)
 - mkdir 873 (time 1790974449.54 total 70.38 last 13.66)
total: 1000 mkdir in 79.49 seconds: 12.58 ops/second
Time for file creation 90 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.44 seconds: 22.58 ops/second
 - mkdir 167 (time 1790975016.11 total 10.00 last 16.70)
 - mkdir 317 (time 1790975026.15 total 20.05 last 14.93)
 - mkdir 488 (time 1790975036.16 total 30.06 last 17.08)
 - mkdir 675 (time 1790975046.20 total 40.10 last 18.62)
 - mkdir 846 (time 1790975056.27 total 50.16 last 16.99)
total: 1000 mkdir in 58.49 seconds: 17.10 ops/second
 - mkdir 167 (time 1790975018.60 total 10.01 last 16.68)
 - mkdir 326 (time 1790975028.65 total 20.06 last 15.83)
 - mkdir 510 (time 1790975038.67 total 30.08 last 18.37)
 - mkdir 703 (time 1790975048.68 total 40.09 last 19.27)
 - mkdir 881 (time 1790975058.69 total 50.10 last 17.78)
total: 1000 mkdir in 56.99 seconds: 17.55 ops/second
 - mkdir 170 (time 1790975019.30 total 10.02 last 16.96)
 - mkdir 328 (time 1790975029.34 total 20.06 last 15.75)
 - mkdir 514 (time 1790975039.36 total 30.08 last 18.56)
 - mkdir 704 (time 1790975049.39 total 40.12 last 18.94)
 - mkdir 890 (time 1790975059.42 total 50.15 last 18.54)
total: 1000 mkdir in 56.35 seconds: 17.75 ops/second
 - mkdir 171 (time 1790975018.25 total 10.04 last 17.04)
 - mkdir 327 (time 1790975028.31 total 20.09 last 15.52)
 - mkdir 501 (time 1790975038.33 total 30.11 last 17.36)
 - mkdir 690 (time 1790975048.37 total 40.15 last 18.83)
 - mkdir 872 (time 1790975058.37 total 50.15 last 18.20)
total: 1000 mkdir in 57.92 seconds: 17.27 ops/second
 - mkdir 168 (time 1790975018.77 total 10.04 last 16.73)
 - mkdir 322 (time 1790975028.79 total 20.06 last 15.36)
 - mkdir 500 (time 1790975038.83 total 30.10 last 17.74)
 - mkdir 690 (time 1790975048.85 total 40.12 last 18.95)
 - mkdir 868 (time 1790975058.91 total 50.18 last 17.70)
total: 1000 mkdir in 58.42 seconds: 17.12 ops/second
 - mkdir 171 (time 1790975019.14 total 10.06 last 17.00)
 - mkdir 327 (time 1790975029.15 total 20.06 last 15.59)
 - mkdir 504 (time 1790975039.21 total 30.13 last 17.59)
 - mkdir 686 (time 1790975049.25 total 40.16 last 18.13)
 - mkdir 864 (time 1790975059.29 total 50.20 last 17.73)
total: 1000 mkdir in 59.62 seconds: 16.77 ops/second
 - mkdir 165 (time 1790975018.23 total 10.02 last 16.46)
 - mkdir 314 (time 1790975028.29 total 20.07 last 14.82)
 - mkdir 485 (time 1790975038.31 total 30.10 last 17.05)
 - mkdir 673 (time 1790975048.33 total 40.12 last 18.77)
 - mkdir 851 (time 1790975058.35 total 50.14 last 17.76)
 - mkdir 994 (time 1790975068.45 total 60.24 last 14.16)
total: 1000 mkdir in 60.84 seconds: 16.44 ops/second
 - mkdir 167 (time 1790975018.79 total 10.02 last 16.67)
 - mkdir 321 (time 1790975028.81 total 20.04 last 15.36)
 - mkdir 497 (time 1790975038.84 total 30.07 last 17.56)
 - mkdir 682 (time 1790975048.87 total 40.10 last 18.44)
 - mkdir 854 (time 1790975058.91 total 50.14 last 17.13)
 - mkdir 989 (time 1790975068.98 total 60.21 last 13.41)
total: 1000 mkdir in 61.13 seconds: 16.36 ops/second
 - mkdir 159 (time 1790975019.13 total 10.01 last 15.89)
 - mkdir 315 (time 1790975029.13 total 20.01 last 15.59)
 - mkdir 483 (time 1790975039.14 total 30.02 last 16.79)
 - mkdir 671 (time 1790975049.18 total 40.06 last 18.73)
 - mkdir 846 (time 1790975059.21 total 50.09 last 17.44)
 - mkdir 985 (time 1790975069.31 total 60.19 last 13.77)
total: 1000 mkdir in 61.36 seconds: 16.30 ops/second
 - mkdir 154 (time 1790975017.66 total 10.07 last 15.29)
 - mkdir 300 (time 1790975027.68 total 20.10 last 14.57)
 - mkdir 467 (time 1790975037.74 total 30.15 last 16.61)
 - mkdir 650 (time 1790975047.77 total 40.19 last 18.23)
 - mkdir 821 (time 1790975057.78 total 50.19 last 17.09)
 - mkdir 966 (time 1790975067.90 total 60.31 last 14.33)
total: 1000 mkdir in 63.53 seconds: 15.74 ops/second
Time for file creation 71 sec for 1 RPC sent at a time
 - unlinked 0 (time 1790975078 ; total 1 ; last 1)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1790975079 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1790975079 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1790975080 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1790975080 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1790975080 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1790975080 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1790975080 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1790975080 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1790975080 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
Time for file removal 46 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8