== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 17:53:44 (1784066024)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.42 seconds: 23.77 ops/second
 - mkdir 179 (time 1784066044.11 total 10.04 last 17.82)
 - mkdir 335 (time 1784066054.12 total 20.05 last 15.58)
 - mkdir 512 (time 1784066064.19 total 30.12 last 17.58)
 - mkdir 661 (time 1784066074.24 total 40.18 last 14.82)
 - mkdir 776 (time 1784066084.28 total 50.21 last 11.46)
 - mkdir 879 (time 1784066094.35 total 60.29 last 10.22)
total: 1000 mkdir in 69.32 seconds: 14.43 ops/second
 - mkdir 181 (time 1784066046.55 total 10.07 last 17.98)
 - mkdir 350 (time 1784066056.59 total 20.11 last 16.83)
 - mkdir 523 (time 1784066066.61 total 30.13 last 17.27)
 - mkdir 668 (time 1784066076.66 total 40.18 last 14.43)
 - mkdir 781 (time 1784066086.80 total 50.32 last 11.14)
 - mkdir 886 (time 1784066096.92 total 60.44 last 10.37)
total: 1000 mkdir in 70.35 seconds: 14.21 ops/second
 - mkdir 173 (time 1784066046.12 total 10.03 last 17.25)
 - mkdir 333 (time 1784066056.14 total 20.05 last 15.97)
 - mkdir 495 (time 1784066066.16 total 30.07 last 16.17)
 - mkdir 649 (time 1784066076.18 total 40.09 last 15.37)
 - mkdir 762 (time 1784066086.18 total 50.09 last 11.30)
 - mkdir 865 (time 1784066096.29 total 60.20 last 10.19)
 - mkdir 987 (time 1784066106.39 total 70.30 last 12.08)
total: 1000 mkdir in 72.20 seconds: 13.85 ops/second
 - mkdir 178 (time 1784066045.53 total 10.01 last 17.79)
 - mkdir 331 (time 1784066055.59 total 20.06 last 15.22)
 - mkdir 489 (time 1784066065.64 total 30.11 last 15.71)
 - mkdir 641 (time 1784066075.65 total 40.12 last 15.19)
 - mkdir 755 (time 1784066085.68 total 50.16 last 11.36)
 - mkdir 857 (time 1784066095.72 total 60.19 last 10.16)
 - mkdir 983 (time 1784066105.80 total 70.27 last 12.50)
total: 1000 mkdir in 72.94 seconds: 13.71 ops/second
 - mkdir 178 (time 1784066046.47 total 10.00 last 17.79)
 - mkdir 342 (time 1784066056.54 total 20.07 last 16.30)
 - mkdir 510 (time 1784066066.57 total 30.10 last 16.75)
 - mkdir 653 (time 1784066076.64 total 40.17 last 14.19)
 - mkdir 766 (time 1784066086.69 total 50.22 last 11.25)
 - mkdir 874 (time 1784066096.88 total 60.41 last 10.60)
 - mkdir 991 (time 1784066106.98 total 70.51 last 11.58)
total: 1000 mkdir in 72.37 seconds: 13.82 ops/second
 - mkdir 179 (time 1784066045.97 total 10.05 last 17.81)
 - mkdir 338 (time 1784066056.00 total 20.08 last 15.85)
 - mkdir 497 (time 1784066066.05 total 30.13 last 15.81)
 - mkdir 651 (time 1784066076.07 total 40.15 last 15.38)
 - mkdir 761 (time 1784066086.14 total 50.22 last 10.92)
 - mkdir 862 (time 1784066096.15 total 60.23 last 10.09)
 - mkdir 985 (time 1784066106.28 total 70.36 last 12.14)
total: 1000 mkdir in 72.96 seconds: 13.71 ops/second
 - mkdir 170 (time 1784066045.92 total 10.04 last 16.93)
 - mkdir 325 (time 1784066055.94 total 20.06 last 15.47)
 - mkdir 491 (time 1784066066.00 total 30.13 last 16.49)
 - mkdir 643 (time 1784066076.11 total 40.23 last 15.04)
 - mkdir 751 (time 1784066086.20 total 50.33 last 10.70)
 - mkdir 855 (time 1784066096.31 total 60.43 last 10.29)
 - mkdir 978 (time 1784066106.33 total 70.46 last 12.27)
total: 1000 mkdir in 74.03 seconds: 13.51 ops/second
 - mkdir 171 (time 1784066046.21 total 10.04 last 17.04)
 - mkdir 328 (time 1784066056.23 total 20.06 last 15.66)
 - mkdir 488 (time 1784066066.25 total 30.08 last 15.97)
 - mkdir 637 (time 1784066076.28 total 40.10 last 14.86)
 - mkdir 748 (time 1784066086.39 total 50.21 last 10.98)
 - mkdir 853 (time 1784066096.40 total 60.23 last 10.48)
 - mkdir 976 (time 1784066106.55 total 70.38 last 12.12)
total: 1000 mkdir in 75.06 seconds: 13.32 ops/second
 - mkdir 173 (time 1784066046.52 total 10.04 last 17.23)
 - mkdir 341 (time 1784066056.54 total 20.07 last 16.76)
 - mkdir 503 (time 1784066066.57 total 30.10 last 16.15)
 - mkdir 649 (time 1784066076.60 total 40.13 last 14.55)
 - mkdir 760 (time 1784066086.75 total 50.27 last 10.94)
 - mkdir 865 (time 1784066096.86 total 60.39 last 10.38)
 - mkdir 978 (time 1784066106.98 total 70.51 last 11.17)
total: 1000 mkdir in 74.99 seconds: 13.34 ops/second
 - mkdir 167 (time 1784066046.62 total 10.05 last 16.61)
 - mkdir 320 (time 1784066056.70 total 20.13 last 15.18)
 - mkdir 481 (time 1784066066.74 total 30.17 last 16.04)
 - mkdir 625 (time 1784066076.76 total 40.19 last 14.37)
 - mkdir 734 (time 1784066086.78 total 50.21 last 10.88)
 - mkdir 838 (time 1784066096.85 total 60.28 last 10.32)
 - mkdir 952 (time 1784066106.89 total 70.32 last 11.36)
total: 1000 mkdir in 77.60 seconds: 12.89 ops/second
Time for file creation 87 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 17 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.65 seconds: 15.38 ops/second
 - mkdir 137 (time 1784066685.23 total 10.05 last 13.63)
 - mkdir 257 (time 1784066695.29 total 20.11 last 11.92)
 - mkdir 401 (time 1784066705.34 total 30.16 last 14.34)
 - mkdir 617 (time 1784066715.38 total 40.20 last 21.52)
 - mkdir 812 (time 1784066725.38 total 50.20 last 19.48)
total: 1000 mkdir in 59.47 seconds: 16.81 ops/second
 - mkdir 139 (time 1784066688.21 total 10.00 last 13.90)
 - mkdir 265 (time 1784066698.28 total 20.08 last 12.51)
 - mkdir 424 (time 1784066708.28 total 30.08 last 15.90)
 - mkdir 662 (time 1784066718.29 total 40.08 last 23.79)
 - mkdir 834 (time 1784066728.37 total 50.17 last 17.06)
total: 1000 mkdir in 59.37 seconds: 16.84 ops/second
 - mkdir 138 (time 1784066688.34 total 10.09 last 13.68)
 - mkdir 266 (time 1784066698.39 total 20.15 last 12.73)
 - mkdir 436 (time 1784066708.41 total 30.16 last 16.98)
 - mkdir 670 (time 1784066718.44 total 40.19 last 23.33)
 - mkdir 838 (time 1784066728.46 total 50.22 last 16.75)
total: 1000 mkdir in 59.33 seconds: 16.85 ops/second
 - mkdir 139 (time 1784066687.32 total 10.03 last 13.86)
 - mkdir 258 (time 1784066697.35 total 20.05 last 11.87)
 - mkdir 407 (time 1784066707.35 total 30.06 last 14.90)
 - mkdir 633 (time 1784066717.38 total 40.08 last 22.54)
 - mkdir 815 (time 1784066727.38 total 50.09 last 18.20)
 - mkdir 993 (time 1784066737.43 total 60.13 last 17.71)
total: 1000 mkdir in 61.07 seconds: 16.37 ops/second
 - mkdir 134 (time 1784066688.67 total 10.05 last 13.33)
 - mkdir 266 (time 1784066698.73 total 20.12 last 13.12)
 - mkdir 439 (time 1784066708.76 total 30.15 last 17.25)
 - mkdir 674 (time 1784066718.78 total 40.16 last 23.46)
 - mkdir 841 (time 1784066728.79 total 50.17 last 16.69)
total: 1000 mkdir in 59.76 seconds: 16.73 ops/second
 - mkdir 134 (time 1784066688.35 total 10.02 last 13.38)
 - mkdir 264 (time 1784066698.39 total 20.05 last 12.95)
 - mkdir 426 (time 1784066708.40 total 30.07 last 16.18)
 - mkdir 659 (time 1784066718.41 total 40.07 last 23.30)
 - mkdir 831 (time 1784066728.44 total 50.10 last 17.14)
 - mkdir 999 (time 1784066738.52 total 60.19 last 16.66)
total: 1000 mkdir in 60.19 seconds: 16.62 ops/second
 - mkdir 133 (time 1784066688.84 total 10.10 last 13.17)
 - mkdir 265 (time 1784066698.93 total 20.18 last 13.08)
 - mkdir 435 (time 1784066708.97 total 30.23 last 16.93)
 - mkdir 666 (time 1784066719.00 total 40.26 last 23.03)
 - mkdir 831 (time 1784066729.09 total 50.35 last 16.35)
 - mkdir 998 (time 1784066739.21 total 60.46 last 16.51)
total: 1000 mkdir in 60.70 seconds: 16.47 ops/second
 - mkdir 133 (time 1784066689.31 total 10.08 last 13.20)
 - mkdir 260 (time 1784066699.32 total 20.09 last 12.68)
 - mkdir 442 (time 1784066709.32 total 30.09 last 18.20)
 - mkdir 667 (time 1784066719.37 total 40.14 last 22.39)
 - mkdir 835 (time 1784066729.40 total 50.18 last 16.75)
 - mkdir 997 (time 1784066739.45 total 60.22 last 16.13)
total: 1000 mkdir in 60.52 seconds: 16.52 ops/second
 - mkdir 134 (time 1784066688.84 total 10.09 last 13.28)
 - mkdir 262 (time 1784066698.94 total 20.19 last 12.68)
 - mkdir 434 (time 1784066708.97 total 30.22 last 17.15)
 - mkdir 665 (time 1784066718.98 total 40.22 last 23.10)
 - mkdir 831 (time 1784066728.99 total 50.24 last 16.58)
 - mkdir 993 (time 1784066739.00 total 60.24 last 16.19)
total: 1000 mkdir in 61.28 seconds: 16.32 ops/second
 - mkdir 130 (time 1784066687.78 total 10.08 last 12.90)
 - mkdir 252 (time 1784066697.88 total 20.18 last 12.08)
 - mkdir 390 (time 1784066707.89 total 30.18 last 13.79)
 - mkdir 623 (time 1784066717.90 total 40.19 last 23.27)
 - mkdir 801 (time 1784066727.93 total 50.22 last 17.75)
 - mkdir 976 (time 1784066737.97 total 60.26 last 17.44)
total: 1000 mkdir in 63.40 seconds: 15.77 ops/second
Time for file creation 75 sec for 1 RPC sent at a time
 - unlinked 0 (time 1784066749 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1784066752 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1784066753 ; total 1 ; last 1)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1784066753 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1784066753 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1784066753 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1784066753 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1784066753 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1784066753 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1784066753 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
Time for file removal 53 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8