== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 08:43:40 (1787402620)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.48 seconds: 20.69 ops/second
 - mkdir 191 (time 1787402641.33 total 10.04 last 19.03)
 - mkdir 344 (time 1787402651.37 total 20.07 last 15.25)
 - mkdir 502 (time 1787402661.47 total 30.17 last 15.64)
 - mkdir 656 (time 1787402671.54 total 40.25 last 15.28)
 - mkdir 801 (time 1787402681.56 total 50.26 last 14.48)
 - mkdir 943 (time 1787402691.62 total 60.32 last 14.12)
total: 1000 mkdir in 63.95 seconds: 15.64 ops/second
 - mkdir 187 (time 1787402640.54 total 10.13 last 18.47)
 - mkdir 336 (time 1787402650.57 total 20.16 last 14.85)
 - mkdir 491 (time 1787402660.60 total 30.19 last 15.46)
 - mkdir 653 (time 1787402670.61 total 40.20 last 16.18)
 - mkdir 791 (time 1787402680.69 total 50.27 last 13.70)
 - mkdir 930 (time 1787402690.76 total 60.34 last 13.80)
total: 1000 mkdir in 65.07 seconds: 15.37 ops/second
 - mkdir 188 (time 1787402641.51 total 10.03 last 18.75)
 - mkdir 340 (time 1787402651.54 total 20.05 last 15.16)
 - mkdir 492 (time 1787402661.57 total 30.09 last 15.15)
 - mkdir 648 (time 1787402671.59 total 40.11 last 15.56)
 - mkdir 790 (time 1787402681.64 total 50.16 last 14.14)
 - mkdir 930 (time 1787402691.65 total 60.17 last 13.98)
total: 1000 mkdir in 65.41 seconds: 15.29 ops/second
 - mkdir 187 (time 1787402641.33 total 10.01 last 18.67)
 - mkdir 341 (time 1787402651.38 total 20.06 last 15.33)
 - mkdir 492 (time 1787402661.48 total 30.16 last 14.95)
 - mkdir 651 (time 1787402671.54 total 40.22 last 15.81)
 - mkdir 789 (time 1787402681.61 total 50.29 last 13.70)
 - mkdir 932 (time 1787402691.70 total 60.39 last 14.17)
total: 1000 mkdir in 65.79 seconds: 15.20 ops/second
 - mkdir 178 (time 1787402640.54 total 10.02 last 17.76)
 - mkdir 325 (time 1787402650.56 total 20.04 last 14.68)
 - mkdir 479 (time 1787402660.62 total 30.10 last 15.30)
 - mkdir 640 (time 1787402670.72 total 40.20 last 15.94)
 - mkdir 780 (time 1787402680.79 total 50.27 last 13.91)
 - mkdir 921 (time 1787402690.79 total 60.28 last 14.09)
total: 1000 mkdir in 66.67 seconds: 15.00 ops/second
 - mkdir 191 (time 1787402641.89 total 10.05 last 19.00)
 - mkdir 344 (time 1787402651.90 total 20.06 last 15.29)
 - mkdir 494 (time 1787402661.92 total 30.08 last 14.97)
 - mkdir 653 (time 1787402671.98 total 40.14 last 15.80)
 - mkdir 795 (time 1787402682.04 total 50.20 last 14.11)
 - mkdir 938 (time 1787402692.08 total 60.24 last 14.25)
total: 1000 mkdir in 65.64 seconds: 15.23 ops/second
 - mkdir 178 (time 1787402641.87 total 10.07 last 17.68)
 - mkdir 332 (time 1787402651.92 total 20.12 last 15.32)
 - mkdir 490 (time 1787402661.94 total 30.14 last 15.77)
 - mkdir 654 (time 1787402671.97 total 40.16 last 16.36)
 - mkdir 793 (time 1787402681.99 total 50.18 last 13.87)
 - mkdir 937 (time 1787402692.07 total 60.27 last 14.28)
total: 1000 mkdir in 65.92 seconds: 15.17 ops/second
 - mkdir 189 (time 1787402641.58 total 10.07 last 18.77)
 - mkdir 340 (time 1787402651.58 total 20.07 last 15.10)
 - mkdir 491 (time 1787402661.59 total 30.08 last 15.08)
 - mkdir 650 (time 1787402671.67 total 40.16 last 15.78)
 - mkdir 791 (time 1787402681.71 total 50.21 last 14.03)
 - mkdir 932 (time 1787402691.71 total 60.21 last 14.10)
total: 1000 mkdir in 66.24 seconds: 15.10 ops/second
 - mkdir 178 (time 1787402641.15 total 10.02 last 17.76)
 - mkdir 328 (time 1787402651.19 total 20.06 last 14.94)
 - mkdir 469 (time 1787402661.23 total 30.10 last 14.04)
 - mkdir 624 (time 1787402671.29 total 40.16 last 15.41)
 - mkdir 769 (time 1787402681.29 total 50.17 last 14.49)
 - mkdir 912 (time 1787402691.31 total 60.19 last 14.27)
total: 1000 mkdir in 67.76 seconds: 14.76 ops/second
 - mkdir 174 (time 1787402641.56 total 10.01 last 17.39)
 - mkdir 320 (time 1787402651.57 total 20.01 last 14.59)
 - mkdir 465 (time 1787402661.63 total 30.07 last 14.41)
 - mkdir 624 (time 1787402671.65 total 40.09 last 15.88)
 - mkdir 765 (time 1787402681.70 total 50.14 last 14.02)
 - mkdir 902 (time 1787402691.70 total 60.14 last 13.70)
total: 1000 mkdir in 69.16 seconds: 14.46 ops/second
Time for file creation 78 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 12 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.58 seconds: 17.39 ops/second
 - mkdir 122 (time 1787403238.58 total 10.01 last 12.18)
 - mkdir 253 (time 1787403248.62 total 20.05 last 13.05)
 - mkdir 385 (time 1787403258.73 total 30.16 last 13.06)
 - mkdir 517 (time 1787403268.74 total 40.17 last 13.19)
 - mkdir 667 (time 1787403278.83 total 50.26 last 14.87)
 - mkdir 826 (time 1787403288.87 total 60.29 last 15.85)
 - mkdir 968 (time 1787403298.96 total 70.38 last 14.07)
total: 1000 mkdir in 72.58 seconds: 13.78 ops/second
 - mkdir 118 (time 1787403239.31 total 10.04 last 11.75)
 - mkdir 254 (time 1787403249.33 total 20.06 last 13.58)
 - mkdir 387 (time 1787403259.38 total 30.11 last 13.23)
 - mkdir 523 (time 1787403269.45 total 40.19 last 13.50)
 - mkdir 672 (time 1787403279.50 total 50.24 last 14.82)
 - mkdir 826 (time 1787403289.57 total 60.31 last 15.29)
 - mkdir 978 (time 1787403299.64 total 70.38 last 15.10)
total: 1000 mkdir in 72.15 seconds: 13.86 ops/second
 - mkdir 124 (time 1787403240.67 total 10.01 last 12.39)
 - mkdir 256 (time 1787403250.75 total 20.10 last 13.09)
 - mkdir 393 (time 1787403260.80 total 30.14 last 13.64)
 - mkdir 531 (time 1787403270.85 total 40.20 last 13.73)
 - mkdir 688 (time 1787403280.89 total 50.24 last 15.64)
 - mkdir 840 (time 1787403290.92 total 60.27 last 15.16)
 - mkdir 994 (time 1787403301.02 total 70.36 last 15.26)
total: 1000 mkdir in 70.85 seconds: 14.11 ops/second
 - mkdir 122 (time 1787403240.45 total 10.03 last 12.16)
 - mkdir 254 (time 1787403250.47 total 20.05 last 13.17)
 - mkdir 388 (time 1787403260.49 total 30.08 last 13.37)
 - mkdir 522 (time 1787403270.55 total 40.13 last 13.32)
 - mkdir 678 (time 1787403280.56 total 50.15 last 15.58)
 - mkdir 836 (time 1787403290.61 total 60.20 last 15.72)
 - mkdir 983 (time 1787403300.71 total 70.29 last 14.56)
total: 1000 mkdir in 71.77 seconds: 13.93 ops/second
 - mkdir 120 (time 1787403240.26 total 10.01 last 11.99)
 - mkdir 248 (time 1787403250.28 total 20.03 last 12.77)
 - mkdir 379 (time 1787403260.30 total 30.06 last 13.07)
 - mkdir 522 (time 1787403270.36 total 40.12 last 14.22)
 - mkdir 681 (time 1787403280.42 total 50.17 last 15.81)
 - mkdir 834 (time 1787403290.46 total 60.21 last 15.24)
 - mkdir 982 (time 1787403300.47 total 70.23 last 14.78)
total: 1000 mkdir in 72.09 seconds: 13.87 ops/second
 - mkdir 117 (time 1787403240.01 total 10.01 last 11.69)
 - mkdir 248 (time 1787403250.01 total 20.02 last 13.09)
 - mkdir 379 (time 1787403260.05 total 30.05 last 13.05)
 - mkdir 516 (time 1787403270.09 total 40.09 last 13.65)
 - mkdir 674 (time 1787403280.13 total 50.13 last 15.73)
 - mkdir 825 (time 1787403290.19 total 60.19 last 15.02)
 - mkdir 971 (time 1787403300.20 total 70.21 last 14.58)
total: 1000 mkdir in 72.50 seconds: 13.79 ops/second
 - mkdir 123 (time 1787403240.84 total 10.01 last 12.29)
 - mkdir 249 (time 1787403250.87 total 20.03 last 12.57)
 - mkdir 391 (time 1787403260.92 total 30.09 last 14.12)
 - mkdir 527 (time 1787403270.97 total 40.14 last 13.54)
 - mkdir 682 (time 1787403280.99 total 50.15 last 15.47)
 - mkdir 834 (time 1787403291.07 total 60.23 last 15.08)
 - mkdir 984 (time 1787403301.08 total 70.24 last 14.99)
total: 1000 mkdir in 71.91 seconds: 13.91 ops/second
 - mkdir 121 (time 1787403239.12 total 10.09 last 11.99)
 - mkdir 249 (time 1787403249.17 total 20.14 last 12.74)
 - mkdir 374 (time 1787403259.20 total 30.17 last 12.46)
 - mkdir 510 (time 1787403269.22 total 40.20 last 13.56)
 - mkdir 661 (time 1787403279.26 total 50.24 last 15.04)
 - mkdir 818 (time 1787403289.28 total 60.25 last 15.67)
 - mkdir 962 (time 1787403299.32 total 70.29 last 14.35)
total: 1000 mkdir in 73.74 seconds: 13.56 ops/second
 - mkdir 126 (time 1787403240.65 total 10.03 last 12.56)
 - mkdir 252 (time 1787403250.72 total 20.10 last 12.51)
 - mkdir 387 (time 1787403260.74 total 30.12 last 13.48)
 - mkdir 519 (time 1787403270.76 total 40.15 last 13.17)
 - mkdir 672 (time 1787403280.76 total 50.15 last 15.30)
 - mkdir 826 (time 1787403290.81 total 60.19 last 15.33)
 - mkdir 973 (time 1787403300.82 total 70.20 last 14.68)
total: 1000 mkdir in 72.88 seconds: 13.72 ops/second
 - mkdir 116 (time 1787403240.76 total 10.08 last 11.51)
 - mkdir 240 (time 1787403250.78 total 20.09 last 12.38)
 - mkdir 373 (time 1787403260.84 total 30.15 last 13.22)
 - mkdir 507 (time 1787403270.84 total 40.16 last 13.39)
 - mkdir 663 (time 1787403280.91 total 50.23 last 15.50)
 - mkdir 820 (time 1787403290.94 total 60.25 last 15.66)
 - mkdir 959 (time 1787403301.01 total 70.33 last 13.80)
total: 1000 mkdir in 74.35 seconds: 13.45 ops/second
Time for file creation 86 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787403315 ; total 1 ; last 1)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1787403316 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787403316 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787403316 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787403316 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787403316 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1787403317 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1787403317 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1787403317 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1787403317 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
Time for file removal 51 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8