== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 04:10:01 (1787386201)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.47 seconds: 21.29 ops/second
 - mkdir 164 (time 1787386223.54 total 10.06 last 16.30)
 - mkdir 333 (time 1787386233.55 total 20.08 last 16.87)
 - mkdir 485 (time 1787386243.61 total 30.14 last 15.11)
 - mkdir 656 (time 1787386253.66 total 40.18 last 17.02)
 - mkdir 801 (time 1787386263.72 total 50.24 last 14.41)
 - mkdir 962 (time 1787386273.72 total 60.25 last 16.10)
total: 1000 mkdir in 62.63 seconds: 15.97 ops/second
 - mkdir 164 (time 1787386222.55 total 10.08 last 16.26)
 - mkdir 335 (time 1787386232.58 total 20.12 last 17.04)
 - mkdir 478 (time 1787386242.66 total 30.19 last 14.19)
 - mkdir 637 (time 1787386252.70 total 40.23 last 15.84)
 - mkdir 780 (time 1787386262.75 total 50.28 last 14.23)
 - mkdir 937 (time 1787386272.79 total 60.33 last 15.63)
total: 1000 mkdir in 64.83 seconds: 15.43 ops/second
 - mkdir 155 (time 1787386224.24 total 10.04 last 15.44)
 - mkdir 325 (time 1787386234.25 total 20.04 last 16.99)
 - mkdir 475 (time 1787386244.26 total 30.05 last 14.99)
 - mkdir 646 (time 1787386254.34 total 40.13 last 16.96)
 - mkdir 783 (time 1787386264.38 total 50.17 last 13.65)
 - mkdir 945 (time 1787386274.40 total 60.19 last 16.16)
total: 1000 mkdir in 65.12 seconds: 15.36 ops/second
 - mkdir 155 (time 1787386223.46 total 10.08 last 15.38)
 - mkdir 329 (time 1787386233.52 total 20.13 last 17.30)
 - mkdir 470 (time 1787386243.60 total 30.22 last 13.98)
 - mkdir 633 (time 1787386253.65 total 40.27 last 16.22)
 - mkdir 769 (time 1787386263.66 total 50.28 last 13.58)
 - mkdir 930 (time 1787386273.71 total 60.33 last 16.02)
total: 1000 mkdir in 66.50 seconds: 15.04 ops/second
 - mkdir 155 (time 1787386223.99 total 10.02 last 15.47)
 - mkdir 324 (time 1787386234.02 total 20.05 last 16.85)
 - mkdir 470 (time 1787386244.08 total 30.11 last 14.51)
 - mkdir 642 (time 1787386254.11 total 40.14 last 17.16)
 - mkdir 777 (time 1787386264.12 total 50.15 last 13.48)
 - mkdir 936 (time 1787386274.18 total 60.22 last 15.80)
total: 1000 mkdir in 66.27 seconds: 15.09 ops/second
 - mkdir 158 (time 1787386224.47 total 10.04 last 15.74)
 - mkdir 326 (time 1787386234.48 total 20.04 last 16.79)
 - mkdir 476 (time 1787386244.50 total 30.07 last 14.96)
 - mkdir 645 (time 1787386254.53 total 40.10 last 16.85)
 - mkdir 787 (time 1787386264.54 total 50.11 last 14.18)
 - mkdir 938 (time 1787386274.57 total 60.13 last 15.06)
total: 1000 mkdir in 66.40 seconds: 15.06 ops/second
 - mkdir 166 (time 1787386221.43 total 10.09 last 16.45)
 - mkdir 317 (time 1787386231.48 total 20.15 last 15.02)
 - mkdir 457 (time 1787386241.50 total 30.17 last 13.97)
 - mkdir 604 (time 1787386251.56 total 40.23 last 14.62)
 - mkdir 748 (time 1787386261.58 total 50.25 last 14.36)
 - mkdir 888 (time 1787386271.59 total 60.26 last 13.99)
total: 1000 mkdir in 69.55 seconds: 14.38 ops/second
 - mkdir 148 (time 1787386224.21 total 10.03 last 14.76)
 - mkdir 319 (time 1787386234.23 total 20.04 last 17.08)
 - mkdir 468 (time 1787386244.27 total 30.08 last 14.84)
 - mkdir 638 (time 1787386254.32 total 40.13 last 16.91)
 - mkdir 775 (time 1787386264.35 total 50.16 last 13.65)
 - mkdir 931 (time 1787386274.37 total 60.18 last 15.57)
total: 1000 mkdir in 67.01 seconds: 14.92 ops/second
 - mkdir 152 (time 1787386223.78 total 10.07 last 15.09)
 - mkdir 318 (time 1787386233.84 total 20.13 last 16.49)
 - mkdir 461 (time 1787386243.88 total 30.17 last 14.24)
 - mkdir 630 (time 1787386253.92 total 40.22 last 16.83)
 - mkdir 763 (time 1787386263.93 total 50.22 last 13.29)
 - mkdir 919 (time 1787386273.98 total 60.28 last 15.52)
total: 1000 mkdir in 67.81 seconds: 14.75 ops/second
 - mkdir 155 (time 1787386223.78 total 10.02 last 15.47)
 - mkdir 313 (time 1787386233.79 total 20.03 last 15.78)
 - mkdir 452 (time 1787386243.88 total 30.12 last 13.78)
 - mkdir 626 (time 1787386253.98 total 40.22 last 17.22)
 - mkdir 756 (time 1787386264.01 total 50.24 last 12.97)
 - mkdir 918 (time 1787386274.08 total 60.32 last 16.07)
total: 1000 mkdir in 67.79 seconds: 14.75 ops/second
Time for file creation 79 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.47 seconds: 21.24 ops/second
 - mkdir 135 (time 1787386834.04 total 10.03 last 13.45)
 - mkdir 279 (time 1787386844.11 total 20.11 last 14.30)
 - mkdir 440 (time 1787386854.12 total 30.11 last 16.10)
 - mkdir 575 (time 1787386864.14 total 40.13 last 13.47)
 - mkdir 711 (time 1787386874.23 total 50.22 last 13.48)
 - mkdir 853 (time 1787386884.29 total 60.28 last 14.11)
 - mkdir 992 (time 1787386894.31 total 70.30 last 13.87)
total: 1000 mkdir in 70.70 seconds: 14.14 ops/second
 - mkdir 139 (time 1787386833.61 total 10.01 last 13.89)
 - mkdir 280 (time 1787386843.65 total 20.04 last 14.06)
 - mkdir 439 (time 1787386853.68 total 30.07 last 15.85)
 - mkdir 575 (time 1787386863.68 total 40.08 last 13.60)
 - mkdir 709 (time 1787386873.71 total 50.10 last 13.36)
 - mkdir 849 (time 1787386883.75 total 60.14 last 13.94)
 - mkdir 988 (time 1787386893.79 total 70.18 last 13.85)
total: 1000 mkdir in 71.14 seconds: 14.06 ops/second
 - mkdir 138 (time 1787386833.80 total 10.08 last 13.69)
 - mkdir 282 (time 1787386843.83 total 20.11 last 14.36)
 - mkdir 435 (time 1787386853.90 total 30.17 last 15.21)
 - mkdir 574 (time 1787386863.94 total 40.22 last 13.84)
 - mkdir 708 (time 1787386873.97 total 50.25 last 13.35)
 - mkdir 851 (time 1787386883.98 total 60.26 last 14.29)
 - mkdir 987 (time 1787386894.07 total 70.35 last 13.48)
total: 1000 mkdir in 71.46 seconds: 13.99 ops/second
 - mkdir 143 (time 1787386835.56 total 10.03 last 14.26)
 - mkdir 281 (time 1787386845.60 total 20.06 last 13.75)
 - mkdir 447 (time 1787386855.62 total 30.08 last 16.57)
 - mkdir 585 (time 1787386865.63 total 40.10 last 13.78)
 - mkdir 717 (time 1787386875.66 total 50.12 last 13.17)
 - mkdir 864 (time 1787386885.69 total 60.16 last 14.65)
total: 1000 mkdir in 70.11 seconds: 14.26 ops/second
 - mkdir 140 (time 1787386835.29 total 10.11 last 13.84)
 - mkdir 286 (time 1787386845.35 total 20.17 last 14.52)
 - mkdir 447 (time 1787386855.38 total 30.20 last 16.05)
 - mkdir 583 (time 1787386865.42 total 40.24 last 13.55)
 - mkdir 714 (time 1787386875.46 total 50.28 last 13.04)
 - mkdir 856 (time 1787386885.55 total 60.37 last 14.08)
 - mkdir 999 (time 1787386895.65 total 70.48 last 14.15)
total: 1000 mkdir in 70.48 seconds: 14.19 ops/second
 - mkdir 139 (time 1787386834.42 total 10.07 last 13.80)
 - mkdir 277 (time 1787386844.47 total 20.12 last 13.74)
 - mkdir 433 (time 1787386854.49 total 30.14 last 15.56)
 - mkdir 569 (time 1787386864.53 total 40.18 last 13.55)
 - mkdir 701 (time 1787386874.57 total 50.22 last 13.15)
 - mkdir 840 (time 1787386884.64 total 60.29 last 13.80)
 - mkdir 973 (time 1787386894.66 total 70.31 last 13.27)
total: 1000 mkdir in 73.50 seconds: 13.61 ops/second
 - mkdir 137 (time 1787386834.66 total 10.06 last 13.61)
 - mkdir 276 (time 1787386844.71 total 20.12 last 13.83)
 - mkdir 432 (time 1787386854.75 total 30.16 last 15.54)
 - mkdir 565 (time 1787386864.78 total 40.19 last 13.26)
 - mkdir 695 (time 1787386874.79 total 50.20 last 12.99)
 - mkdir 837 (time 1787386884.79 total 60.20 last 14.20)
 - mkdir 972 (time 1787386894.80 total 70.21 last 13.48)
total: 1000 mkdir in 73.37 seconds: 13.63 ops/second
 - mkdir 138 (time 1787386835.38 total 10.02 last 13.78)
 - mkdir 277 (time 1787386845.39 total 20.03 last 13.88)
 - mkdir 435 (time 1787386855.40 total 30.03 last 15.80)
 - mkdir 570 (time 1787386865.47 total 40.11 last 13.41)
 - mkdir 697 (time 1787386875.54 total 50.18 last 12.61)
 - mkdir 839 (time 1787386885.55 total 60.19 last 14.18)
 - mkdir 974 (time 1787386895.64 total 70.28 last 13.38)
total: 1000 mkdir in 73.29 seconds: 13.65 ops/second
 - mkdir 138 (time 1787386835.86 total 10.06 last 13.72)
 - mkdir 277 (time 1787386845.89 total 20.09 last 13.85)
 - mkdir 437 (time 1787386855.90 total 30.10 last 15.98)
 - mkdir 575 (time 1787386865.95 total 40.16 last 13.73)
 - mkdir 704 (time 1787386875.99 total 50.19 last 12.85)
 - mkdir 843 (time 1787386886.09 total 60.30 last 13.76)
 - mkdir 973 (time 1787386896.33 total 70.53 last 12.70)
total: 1000 mkdir in 73.55 seconds: 13.60 ops/second
 - mkdir 134 (time 1787386835.64 total 10.11 last 13.25)
 - mkdir 266 (time 1787386845.66 total 20.14 last 13.17)
 - mkdir 421 (time 1787386855.71 total 30.19 last 15.42)
 - mkdir 557 (time 1787386865.72 total 40.20 last 13.59)
 - mkdir 688 (time 1787386875.74 total 50.21 last 13.08)
 - mkdir 826 (time 1787386885.79 total 60.26 last 13.73)
 - mkdir 953 (time 1787386895.88 total 70.36 last 12.58)
total: 1000 mkdir in 75.10 seconds: 13.32 ops/second
Time for file creation 86 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787386908 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787386911 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787386911 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787386912 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787386911 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787386912 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787386912 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787386912 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787386912 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787386912 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
Time for file removal 50 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8