== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 14:25:40 (1787163940)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.63 seconds: 15.81 ops/second
 - mkdir 127 (time 1787163968.21 total 10.11 last 12.57)
 - mkdir 252 (time 1787163978.23 total 20.12 last 12.48)
 - mkdir 401 (time 1787163988.30 total 30.20 last 14.79)
 - mkdir 542 (time 1787163998.36 total 40.26 last 14.01)
 - mkdir 675 (time 1787164008.37 total 50.27 last 13.29)
 - mkdir 809 (time 1787164018.38 total 60.27 last 13.39)
 - mkdir 971 (time 1787164028.41 total 70.31 last 16.14)
total: 1000 mkdir in 72.25 seconds: 13.84 ops/second
 - mkdir 122 (time 1787163967.45 total 10.05 last 12.14)
 - mkdir 232 (time 1787163977.50 total 20.10 last 10.94)
 - mkdir 390 (time 1787163987.54 total 30.14 last 15.75)
 - mkdir 525 (time 1787163997.55 total 40.15 last 13.48)
 - mkdir 657 (time 1787164007.57 total 50.17 last 13.17)
 - mkdir 786 (time 1787164017.66 total 60.26 last 12.79)
 - mkdir 939 (time 1787164027.67 total 70.27 last 15.29)
total: 1000 mkdir in 75.83 seconds: 13.19 ops/second
 - mkdir 100 (time 1787163966.14 total 10.05 last 9.95)
 - mkdir 221 (time 1787163976.15 total 20.06 last 12.08)
 - mkdir 369 (time 1787163986.19 total 30.09 last 14.75)
 - mkdir 506 (time 1787163996.25 total 40.15 last 13.62)
 - mkdir 632 (time 1787164006.30 total 50.20 last 12.54)
 - mkdir 767 (time 1787164016.31 total 60.22 last 13.48)
 - mkdir 914 (time 1787164026.34 total 70.24 last 14.66)
total: 1000 mkdir in 77.47 seconds: 12.91 ops/second
 - mkdir 124 (time 1787163968.88 total 10.04 last 12.35)
 - mkdir 248 (time 1787163978.93 total 20.09 last 12.34)
 - mkdir 399 (time 1787163989.00 total 30.17 last 14.99)
 - mkdir 535 (time 1787163999.02 total 40.18 last 13.58)
 - mkdir 672 (time 1787164009.08 total 50.25 last 13.61)
 - mkdir 800 (time 1787164019.09 total 60.25 last 12.79)
 - mkdir 954 (time 1787164029.11 total 70.28 last 15.37)
total: 1000 mkdir in 75.36 seconds: 13.27 ops/second
 - mkdir 122 (time 1787163968.30 total 10.06 last 12.13)
 - mkdir 243 (time 1787163978.32 total 20.08 last 12.08)
 - mkdir 397 (time 1787163988.35 total 30.11 last 15.35)
 - mkdir 533 (time 1787163998.35 total 40.11 last 13.60)
 - mkdir 664 (time 1787164008.40 total 50.17 last 13.03)
 - mkdir 791 (time 1787164018.45 total 60.21 last 12.64)
 - mkdir 946 (time 1787164028.45 total 70.21 last 15.50)
total: 1000 mkdir in 76.28 seconds: 13.11 ops/second
 - mkdir 119 (time 1787163967.65 total 10.04 last 11.86)
 - mkdir 235 (time 1787163977.68 total 20.06 last 11.57)
 - mkdir 387 (time 1787163987.70 total 30.09 last 15.16)
 - mkdir 522 (time 1787163997.72 total 40.10 last 13.48)
 - mkdir 647 (time 1787164007.73 total 50.11 last 12.49)
 - mkdir 776 (time 1787164017.74 total 60.12 last 12.88)
 - mkdir 932 (time 1787164027.79 total 70.18 last 15.52)
total: 1000 mkdir in 76.92 seconds: 13.00 ops/second
 - mkdir 126 (time 1787163969.20 total 10.09 last 12.48)
 - mkdir 246 (time 1787163979.27 total 20.17 last 11.91)
 - mkdir 397 (time 1787163989.32 total 30.22 last 15.02)
 - mkdir 535 (time 1787163999.37 total 40.27 last 13.74)
 - mkdir 671 (time 1787164009.42 total 50.32 last 13.53)
 - mkdir 802 (time 1787164019.48 total 60.38 last 13.02)
 - mkdir 954 (time 1787164029.53 total 70.43 last 15.12)
total: 1000 mkdir in 75.81 seconds: 13.19 ops/second
 - mkdir 120 (time 1787163969.16 total 10.03 last 11.96)
 - mkdir 244 (time 1787163979.28 total 20.15 last 12.25)
 - mkdir 400 (time 1787163989.33 total 30.20 last 15.52)
 - mkdir 532 (time 1787163999.36 total 40.22 last 13.17)
 - mkdir 663 (time 1787164009.38 total 50.25 last 13.07)
 - mkdir 793 (time 1787164019.46 total 60.33 last 12.90)
 - mkdir 948 (time 1787164029.50 total 70.37 last 15.44)
total: 1000 mkdir in 76.49 seconds: 13.07 ops/second
 - mkdir 99 (time 1787163965.76 total 10.20 last 9.70)
 - mkdir 220 (time 1787163975.80 total 20.24 last 12.05)
 - mkdir 367 (time 1787163985.80 total 30.24 last 14.69)
 - mkdir 497 (time 1787163995.82 total 40.26 last 12.98)
 - mkdir 628 (time 1787164005.86 total 50.30 last 13.05)
 - mkdir 753 (time 1787164015.86 total 60.30 last 12.50)
 - mkdir 894 (time 1787164025.92 total 70.36 last 14.02)
total: 1000 mkdir in 80.18 seconds: 12.47 ops/second
 - mkdir 117 (time 1787163968.04 total 10.01 last 11.69)
 - mkdir 227 (time 1787163978.07 total 20.03 last 10.98)
 - mkdir 365 (time 1787163988.15 total 30.12 last 13.68)
 - mkdir 492 (time 1787163998.18 total 40.15 last 12.66)
 - mkdir 613 (time 1787164008.24 total 50.20 last 12.03)
 - mkdir 736 (time 1787164018.29 total 60.25 last 12.24)
 - mkdir 885 (time 1787164028.32 total 70.29 last 14.85)
 - mkdir 976 (time 1787164038.37 total 80.34 last 9.06)
total: 1000 mkdir in 82.79 seconds: 12.08 ops/second
Time for file creation 98 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 21 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.68 seconds: 14.62 ops/second
 - mkdir 151 (time 1787164650.12 total 10.00 last 15.10)
 - mkdir 310 (time 1787164660.13 total 20.01 last 15.88)
 - mkdir 459 (time 1787164670.15 total 30.04 last 14.86)
 - mkdir 618 (time 1787164680.15 total 40.04 last 15.90)
 - mkdir 737 (time 1787164690.18 total 50.06 last 11.87)
 - mkdir 845 (time 1787164700.22 total 60.10 last 10.76)
total: 1000 mkdir in 70.03 seconds: 14.28 ops/second
 - mkdir 151 (time 1787164649.04 total 10.02 last 15.07)
 - mkdir 297 (time 1787164659.10 total 20.08 last 14.51)
 - mkdir 460 (time 1787164669.21 total 30.19 last 16.13)
 - mkdir 601 (time 1787164679.24 total 40.21 last 14.07)
 - mkdir 718 (time 1787164689.25 total 50.23 last 11.68)
 - mkdir 828 (time 1787164699.36 total 60.33 last 10.89)
 - mkdir 978 (time 1787164709.36 total 70.33 last 15.00)
total: 1000 mkdir in 71.72 seconds: 13.94 ops/second
 - mkdir 145 (time 1787164650.74 total 10.03 last 14.46)
 - mkdir 302 (time 1787164660.78 total 20.07 last 15.63)
 - mkdir 453 (time 1787164670.82 total 30.12 last 15.03)
 - mkdir 601 (time 1787164680.84 total 40.13 last 14.77)
 - mkdir 720 (time 1787164690.90 total 50.19 last 11.83)
 - mkdir 825 (time 1787164700.91 total 60.20 last 10.49)
 - mkdir 978 (time 1787164711.01 total 70.30 last 15.15)
total: 1000 mkdir in 73.68 seconds: 13.57 ops/second
 - mkdir 148 (time 1787164651.60 total 10.04 last 14.73)
 - mkdir 302 (time 1787164661.71 total 20.15 last 15.24)
 - mkdir 451 (time 1787164671.72 total 30.17 last 14.87)
 - mkdir 598 (time 1787164681.83 total 40.27 last 14.55)
 - mkdir 717 (time 1787164691.86 total 50.31 last 11.86)
 - mkdir 819 (time 1787164701.93 total 60.37 last 10.13)
 - mkdir 968 (time 1787164711.97 total 70.42 last 14.84)
total: 1000 mkdir in 76.30 seconds: 13.11 ops/second
 - mkdir 148 (time 1787164651.05 total 10.01 last 14.79)
 - mkdir 303 (time 1787164661.06 total 20.01 last 15.49)
 - mkdir 449 (time 1787164671.13 total 30.08 last 14.49)
 - mkdir 592 (time 1787164681.15 total 40.11 last 14.27)
 - mkdir 709 (time 1787164691.17 total 50.13 last 11.68)
 - mkdir 814 (time 1787164701.28 total 60.24 last 10.38)
 - mkdir 966 (time 1787164711.46 total 70.42 last 14.93)
total: 1000 mkdir in 77.08 seconds: 12.97 ops/second
 - mkdir 143 (time 1787164651.49 total 10.02 last 14.26)
 - mkdir 301 (time 1787164661.51 total 20.05 last 15.77)
 - mkdir 449 (time 1787164671.55 total 30.09 last 14.73)
 - mkdir 588 (time 1787164681.56 total 40.10 last 13.89)
 - mkdir 706 (time 1787164691.58 total 50.12 last 11.78)
 - mkdir 810 (time 1787164701.58 total 60.12 last 10.40)
 - mkdir 963 (time 1787164711.85 total 70.39 last 14.90)
total: 1000 mkdir in 76.81 seconds: 13.02 ops/second
 - mkdir 150 (time 1787164651.87 total 10.14 last 14.79)
 - mkdir 305 (time 1787164661.90 total 20.17 last 15.46)
 - mkdir 451 (time 1787164671.97 total 30.24 last 14.50)
 - mkdir 596 (time 1787164682.00 total 40.27 last 14.45)
 - mkdir 712 (time 1787164692.06 total 50.33 last 11.54)
 - mkdir 813 (time 1787164702.06 total 60.33 last 10.09)
 - mkdir 962 (time 1787164712.19 total 70.46 last 14.72)
total: 1000 mkdir in 77.99 seconds: 12.82 ops/second
 - mkdir 137 (time 1787164650.70 total 10.00 last 13.70)
 - mkdir 286 (time 1787164660.78 total 20.08 last 14.78)
 - mkdir 429 (time 1787164670.83 total 30.13 last 14.23)
 - mkdir 575 (time 1787164680.89 total 40.19 last 14.52)
 - mkdir 687 (time 1787164690.89 total 50.19 last 11.20)
 - mkdir 789 (time 1787164700.96 total 60.26 last 10.14)
 - mkdir 945 (time 1787164711.00 total 70.30 last 15.53)
total: 1000 mkdir in 80.13 seconds: 12.48 ops/second
 - mkdir 143 (time 1787164651.85 total 10.05 last 14.22)
 - mkdir 302 (time 1787164661.90 total 20.10 last 15.82)
 - mkdir 447 (time 1787164672.03 total 30.23 last 14.32)
 - mkdir 585 (time 1787164682.07 total 40.27 last 13.74)
 - mkdir 704 (time 1787164692.14 total 50.34 last 11.82)
 - mkdir 804 (time 1787164702.18 total 60.38 last 9.96)
 - mkdir 950 (time 1787164712.31 total 70.51 last 14.41)
total: 1000 mkdir in 79.80 seconds: 12.53 ops/second
 - mkdir 134 (time 1787164651.29 total 10.04 last 13.35)
 - mkdir 282 (time 1787164661.36 total 20.10 last 14.71)
 - mkdir 415 (time 1787164671.37 total 30.12 last 13.28)
 - mkdir 555 (time 1787164681.39 total 40.13 last 13.98)
 - mkdir 663 (time 1787164691.42 total 50.17 last 10.76)
 - mkdir 763 (time 1787164701.46 total 60.20 last 9.97)
 - mkdir 908 (time 1787164711.60 total 70.34 last 14.30)
 - mkdir 964 (time 1787164721.71 total 80.46 last 5.54)
total: 1000 mkdir in 83.97 seconds: 11.91 ops/second
Time for file creation 97 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787164741 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1787164741 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1787164742 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1787164743 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1787164742 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1787164744 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1787164743 ; total 0 ; last 0)
total: 1000 unlinks in 42 seconds: 23.809525 unlinks/second
 - unlinked 0 (time 1787164744 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1787164744 ; total 0 ; last 0)
total: 1000 unlinks in 42 seconds: 23.809525 unlinks/second
 - unlinked 0 (time 1787164744 ; total 1 ; last 1)
total: 1000 unlinks in 43 seconds: 23.255814 unlinks/second
Time for file removal 63 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8