== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 08:04:17 (1785240257)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.80 seconds: 12.43 ops/second
 - mkdir 116 (time 1785240280.52 total 10.12 last 11.47)
 - mkdir 280 (time 1785240290.52 total 20.12 last 16.40)
 - mkdir 447 (time 1785240300.54 total 30.13 last 16.67)
 - mkdir 598 (time 1785240310.59 total 40.19 last 15.02)
 - mkdir 782 (time 1785240320.63 total 50.23 last 18.32)
 - mkdir 936 (time 1785240330.65 total 60.25 last 15.37)
total: 1000 mkdir in 64.99 seconds: 15.39 ops/second
 - mkdir 127 (time 1785240281.46 total 10.00 last 12.70)
 - mkdir 289 (time 1785240291.48 total 20.02 last 16.16)
 - mkdir 458 (time 1785240301.51 total 30.05 last 16.85)
 - mkdir 613 (time 1785240311.57 total 40.11 last 15.41)
 - mkdir 794 (time 1785240321.64 total 50.18 last 17.98)
 - mkdir 944 (time 1785240331.66 total 60.20 last 14.97)
total: 1000 mkdir in 64.91 seconds: 15.41 ops/second
 - mkdir 122 (time 1785240283.02 total 10.02 last 12.18)
 - mkdir 299 (time 1785240293.09 total 20.09 last 17.58)
 - mkdir 451 (time 1785240303.11 total 30.11 last 15.18)
 - mkdir 616 (time 1785240313.14 total 40.14 last 16.44)
 - mkdir 794 (time 1785240323.15 total 50.14 last 17.79)
 - mkdir 944 (time 1785240333.16 total 60.15 last 14.99)
total: 1000 mkdir in 65.60 seconds: 15.24 ops/second
 - mkdir 124 (time 1785240283.92 total 10.03 last 12.37)
 - mkdir 314 (time 1785240293.93 total 20.03 last 18.99)
 - mkdir 464 (time 1785240304.00 total 30.10 last 14.89)
 - mkdir 628 (time 1785240314.02 total 40.12 last 16.37)
 - mkdir 807 (time 1785240324.09 total 50.19 last 17.77)
 - mkdir 947 (time 1785240334.09 total 60.19 last 13.99)
total: 1000 mkdir in 66.59 seconds: 15.02 ops/second
 - mkdir 123 (time 1785240283.84 total 10.00 last 12.30)
 - mkdir 307 (time 1785240293.88 total 20.03 last 18.34)
 - mkdir 454 (time 1785240303.91 total 30.07 last 14.65)
 - mkdir 620 (time 1785240313.94 total 40.10 last 16.55)
 - mkdir 799 (time 1785240323.94 total 50.10 last 17.90)
 - mkdir 943 (time 1785240333.96 total 60.11 last 14.38)
total: 1000 mkdir in 67.37 seconds: 14.84 ops/second
 - mkdir 122 (time 1785240284.19 total 10.08 last 12.10)
 - mkdir 309 (time 1785240294.24 total 20.13 last 18.61)
 - mkdir 458 (time 1785240304.25 total 30.15 last 14.88)
 - mkdir 623 (time 1785240314.26 total 40.15 last 16.49)
 - mkdir 798 (time 1785240324.27 total 50.17 last 17.47)
 - mkdir 941 (time 1785240334.33 total 60.23 last 14.22)
total: 1000 mkdir in 68.75 seconds: 14.55 ops/second
 - mkdir 116 (time 1785240282.79 total 10.02 last 11.57)
 - mkdir 285 (time 1785240292.80 total 20.03 last 16.89)
 - mkdir 438 (time 1785240302.88 total 30.11 last 15.18)
 - mkdir 596 (time 1785240312.92 total 40.15 last 15.74)
 - mkdir 773 (time 1785240322.97 total 50.20 last 17.61)
 - mkdir 926 (time 1785240333.06 total 60.29 last 15.17)
 - mkdir 998 (time 1785240343.07 total 70.29 last 7.20)
total: 1000 mkdir in 70.60 seconds: 14.16 ops/second
 - mkdir 119 (time 1785240284.07 total 10.08 last 11.81)
 - mkdir 299 (time 1785240294.08 total 20.08 last 18.00)
 - mkdir 448 (time 1785240304.08 total 30.09 last 14.89)
 - mkdir 614 (time 1785240314.08 total 40.09 last 16.59)
 - mkdir 789 (time 1785240324.13 total 50.14 last 17.42)
 - mkdir 930 (time 1785240334.17 total 60.18 last 14.04)
 - mkdir 999 (time 1785240344.30 total 70.31 last 6.81)
total: 1000 mkdir in 70.31 seconds: 14.22 ops/second
 - mkdir 120 (time 1785240283.79 total 10.02 last 11.98)
 - mkdir 296 (time 1785240293.80 total 20.03 last 17.57)
 - mkdir 439 (time 1785240303.85 total 30.08 last 14.23)
 - mkdir 602 (time 1785240313.89 total 40.12 last 16.24)
 - mkdir 778 (time 1785240323.93 total 50.16 last 17.53)
 - mkdir 922 (time 1785240333.96 total 60.18 last 14.36)
 - mkdir 994 (time 1785240344.06 total 70.29 last 7.12)
total: 1000 mkdir in 70.87 seconds: 14.11 ops/second
 - mkdir 114 (time 1785240284.33 total 10.04 last 11.35)
 - mkdir 290 (time 1785240294.35 total 20.05 last 17.58)
 - mkdir 429 (time 1785240304.37 total 30.08 last 13.86)
 - mkdir 600 (time 1785240314.43 total 40.14 last 17.00)
 - mkdir 777 (time 1785240324.46 total 50.17 last 17.66)
 - mkdir 916 (time 1785240334.46 total 60.17 last 13.90)
 - mkdir 980 (time 1785240344.49 total 70.20 last 6.38)
total: 1000 mkdir in 71.28 seconds: 14.03 ops/second
Time for file creation 85 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 12 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.56 seconds: 17.73 ops/second
 - mkdir 155 (time 1785240868.39 total 10.07 last 15.40)
 - mkdir 322 (time 1785240878.39 total 20.07 last 16.69)
 - mkdir 487 (time 1785240888.42 total 30.10 last 16.46)
 - mkdir 684 (time 1785240898.44 total 40.11 last 19.67)
 - mkdir 850 (time 1785240908.45 total 50.12 last 16.58)
total: 1000 mkdir in 59.14 seconds: 16.91 ops/second
 - mkdir 154 (time 1785240868.46 total 10.02 last 15.38)
 - mkdir 319 (time 1785240878.52 total 20.07 last 16.41)
 - mkdir 489 (time 1785240888.57 total 30.13 last 16.90)
 - mkdir 684 (time 1785240898.58 total 40.13 last 19.50)
 - mkdir 850 (time 1785240908.60 total 50.16 last 16.56)
total: 1000 mkdir in 59.39 seconds: 16.84 ops/second
 - mkdir 173 (time 1785240870.15 total 10.05 last 17.21)
 - mkdir 343 (time 1785240880.19 total 20.09 last 16.93)
 - mkdir 520 (time 1785240890.23 total 30.12 last 17.64)
 - mkdir 709 (time 1785240900.26 total 40.15 last 18.85)
 - mkdir 873 (time 1785240910.29 total 50.18 last 16.35)
total: 1000 mkdir in 57.91 seconds: 17.27 ops/second
 - mkdir 170 (time 1785240870.16 total 10.01 last 16.98)
 - mkdir 335 (time 1785240880.18 total 20.04 last 16.45)
 - mkdir 509 (time 1785240890.20 total 30.06 last 17.37)
 - mkdir 707 (time 1785240900.24 total 40.10 last 19.72)
 - mkdir 866 (time 1785240910.27 total 50.13 last 15.85)
total: 1000 mkdir in 58.87 seconds: 16.99 ops/second
 - mkdir 159 (time 1785240869.28 total 10.05 last 15.83)
 - mkdir 320 (time 1785240879.29 total 20.06 last 16.09)
 - mkdir 492 (time 1785240889.32 total 30.09 last 17.14)
 - mkdir 683 (time 1785240899.36 total 40.13 last 19.03)
 - mkdir 848 (time 1785240909.45 total 50.22 last 16.35)
 - mkdir 998 (time 1785240919.46 total 60.23 last 14.97)
total: 1000 mkdir in 60.35 seconds: 16.57 ops/second
 - mkdir 149 (time 1785240868.65 total 10.04 last 14.84)
 - mkdir 308 (time 1785240878.71 total 20.10 last 15.81)
 - mkdir 473 (time 1785240888.71 total 30.10 last 16.50)
 - mkdir 667 (time 1785240898.79 total 40.18 last 19.24)
 - mkdir 834 (time 1785240908.80 total 50.19 last 16.68)
 - mkdir 991 (time 1785240918.83 total 60.23 last 15.64)
total: 1000 mkdir in 61.06 seconds: 16.38 ops/second
 - mkdir 155 (time 1785240869.69 total 10.03 last 15.46)
 - mkdir 321 (time 1785240879.72 total 20.06 last 16.55)
 - mkdir 495 (time 1785240889.77 total 30.11 last 17.31)
 - mkdir 691 (time 1785240899.77 total 40.11 last 19.59)
 - mkdir 848 (time 1785240909.80 total 50.14 last 15.67)
 - mkdir 997 (time 1785240919.83 total 60.17 last 14.85)
total: 1000 mkdir in 60.37 seconds: 16.56 ops/second
 - mkdir 162 (time 1785240869.69 total 10.04 last 16.13)
 - mkdir 322 (time 1785240879.71 total 20.07 last 15.96)
 - mkdir 497 (time 1785240889.77 total 30.12 last 17.40)
 - mkdir 687 (time 1785240899.77 total 40.13 last 19.00)
 - mkdir 846 (time 1785240909.83 total 50.18 last 15.81)
 - mkdir 997 (time 1785240919.89 total 60.25 last 15.01)
total: 1000 mkdir in 60.45 seconds: 16.54 ops/second
 - mkdir 157 (time 1785240869.62 total 10.05 last 15.63)
 - mkdir 316 (time 1785240879.64 total 20.07 last 15.86)
 - mkdir 488 (time 1785240889.68 total 30.11 last 17.14)
 - mkdir 685 (time 1785240899.72 total 40.15 last 19.61)
 - mkdir 846 (time 1785240909.73 total 50.16 last 16.09)
 - mkdir 995 (time 1785240919.77 total 60.20 last 14.83)
total: 1000 mkdir in 60.55 seconds: 16.51 ops/second
 - mkdir 155 (time 1785240869.39 total 10.01 last 15.49)
 - mkdir 315 (time 1785240879.39 total 20.01 last 16.00)
 - mkdir 485 (time 1785240889.47 total 30.09 last 16.87)
 - mkdir 680 (time 1785240899.47 total 40.09 last 19.49)
 - mkdir 844 (time 1785240909.50 total 50.12 last 16.35)
 - mkdir 992 (time 1785240919.53 total 60.15 last 14.76)
total: 1000 mkdir in 60.82 seconds: 16.44 ops/second
Time for file creation 73 sec for 1 RPC sent at a time
 - unlinked 0 (time 1785240928 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1785240931 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1785240931 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1785240932 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1785240932 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1785240931 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1785240931 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1785240932 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1785240932 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1785240932 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
Time for file removal 46 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8