== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 21:08:23 (1787792903)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 1.06 seconds: 9.45 ops/second
 - mkdir 155 (time 1787792930.44 total 10.05 last 15.43)
 - mkdir 332 (time 1787792940.46 total 20.07 last 17.66)
 - mkdir 533 (time 1787792950.47 total 30.08 last 20.08)
 - mkdir 687 (time 1787792960.61 total 40.22 last 15.18)
 - mkdir 817 (time 1787792970.62 total 50.23 last 12.99)
 - mkdir 986 (time 1787792980.72 total 60.33 last 16.74)
total: 1000 mkdir in 61.07 seconds: 16.38 ops/second
 - mkdir 155 (time 1787792929.81 total 10.01 last 15.49)
 - mkdir 332 (time 1787792939.82 total 20.02 last 17.68)
 - mkdir 516 (time 1787792949.89 total 30.09 last 18.27)
 - mkdir 671 (time 1787792959.97 total 40.17 last 15.38)
 - mkdir 792 (time 1787792969.98 total 50.18 last 12.08)
 - mkdir 962 (time 1787792979.99 total 60.19 last 16.99)
total: 1000 mkdir in 63.75 seconds: 15.69 ops/second
 - mkdir 147 (time 1787792930.81 total 10.02 last 14.67)
 - mkdir 316 (time 1787792940.85 total 20.06 last 16.83)
 - mkdir 511 (time 1787792950.86 total 30.07 last 19.48)
 - mkdir 653 (time 1787792960.91 total 40.11 last 14.14)
 - mkdir 793 (time 1787792970.92 total 50.13 last 13.98)
 - mkdir 959 (time 1787792980.97 total 60.17 last 16.52)
total: 1000 mkdir in 64.74 seconds: 15.45 ops/second
 - mkdir 150 (time 1787792930.28 total 10.05 last 14.93)
 - mkdir 324 (time 1787792940.36 total 20.13 last 17.25)
 - mkdir 515 (time 1787792950.42 total 30.19 last 18.98)
 - mkdir 661 (time 1787792960.46 total 40.22 last 14.55)
 - mkdir 793 (time 1787792970.48 total 50.25 last 13.17)
 - mkdir 954 (time 1787792980.52 total 60.29 last 16.04)
total: 1000 mkdir in 66.27 seconds: 15.09 ops/second
 - mkdir 144 (time 1787792930.33 total 10.03 last 14.35)
 - mkdir 318 (time 1787792940.36 total 20.07 last 17.35)
 - mkdir 503 (time 1787792950.37 total 30.07 last 18.49)
 - mkdir 651 (time 1787792960.42 total 40.13 last 14.72)
 - mkdir 782 (time 1787792970.43 total 50.13 last 13.09)
 - mkdir 950 (time 1787792980.56 total 60.26 last 16.58)
total: 1000 mkdir in 66.70 seconds: 14.99 ops/second
 - mkdir 145 (time 1787792930.51 total 10.02 last 14.47)
 - mkdir 316 (time 1787792940.57 total 20.08 last 17.00)
 - mkdir 501 (time 1787792950.59 total 30.10 last 18.46)
 - mkdir 652 (time 1787792960.63 total 40.14 last 15.04)
 - mkdir 783 (time 1787792970.64 total 50.15 last 13.10)
 - mkdir 946 (time 1787792980.67 total 60.18 last 16.24)
total: 1000 mkdir in 67.28 seconds: 14.86 ops/second
 - mkdir 145 (time 1787792930.66 total 10.04 last 14.45)
 - mkdir 313 (time 1787792940.69 total 20.06 last 16.75)
 - mkdir 503 (time 1787792950.72 total 30.09 last 18.94)
 - mkdir 648 (time 1787792960.80 total 40.18 last 14.38)
 - mkdir 784 (time 1787792970.82 total 50.19 last 13.58)
 - mkdir 947 (time 1787792980.82 total 60.19 last 16.30)
total: 1000 mkdir in 67.60 seconds: 14.79 ops/second
 - mkdir 143 (time 1787792930.79 total 10.04 last 14.25)
 - mkdir 310 (time 1787792940.86 total 20.10 last 16.59)
 - mkdir 504 (time 1787792950.90 total 30.15 last 19.31)
 - mkdir 652 (time 1787792960.91 total 40.15 last 14.80)
 - mkdir 784 (time 1787792970.93 total 50.18 last 13.17)
 - mkdir 948 (time 1787792980.94 total 60.19 last 16.38)
total: 1000 mkdir in 67.56 seconds: 14.80 ops/second
 - mkdir 140 (time 1787792930.51 total 10.03 last 13.96)
 - mkdir 309 (time 1787792940.51 total 20.03 last 16.89)
 - mkdir 495 (time 1787792950.57 total 30.09 last 18.50)
 - mkdir 645 (time 1787792960.65 total 40.17 last 14.87)
 - mkdir 783 (time 1787792970.71 total 50.23 last 13.72)
 - mkdir 941 (time 1787792980.72 total 60.24 last 15.79)
total: 1000 mkdir in 67.90 seconds: 14.73 ops/second
 - mkdir 141 (time 1787792930.39 total 10.02 last 14.07)
 - mkdir 300 (time 1787792940.41 total 20.05 last 15.86)
 - mkdir 484 (time 1787792950.42 total 30.05 last 18.39)
 - mkdir 634 (time 1787792960.49 total 40.12 last 14.90)
 - mkdir 764 (time 1787792970.49 total 50.12 last 13.00)
 - mkdir 932 (time 1787792980.56 total 60.19 last 16.68)
total: 1000 mkdir in 69.44 seconds: 14.40 ops/second
Time for file creation 84 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
Time for file removal 16 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.70 seconds: 14.22 ops/second
 - mkdir 119 (time 1787793558.16 total 10.04 last 11.86)
 - mkdir 236 (time 1787793568.16 total 20.04 last 11.70)
 - mkdir 416 (time 1787793578.22 total 30.09 last 17.90)
 - mkdir 564 (time 1787793588.32 total 40.20 last 14.65)
 - mkdir 716 (time 1787793598.38 total 50.26 last 15.11)
 - mkdir 869 (time 1787793608.41 total 60.28 last 15.27)
 - mkdir 993 (time 1787793618.41 total 70.29 last 12.39)
total: 1000 mkdir in 70.70 seconds: 14.14 ops/second
 - mkdir 119 (time 1787793561.36 total 10.03 last 11.87)
 - mkdir 249 (time 1787793571.44 total 20.11 last 12.90)
 - mkdir 443 (time 1787793581.48 total 30.15 last 19.33)
 - mkdir 590 (time 1787793591.51 total 40.18 last 14.65)
 - mkdir 746 (time 1787793601.56 total 50.23 last 15.53)
 - mkdir 882 (time 1787793611.60 total 60.26 last 13.55)
total: 1000 mkdir in 70.05 seconds: 14.28 ops/second
 - mkdir 125 (time 1787793561.21 total 10.06 last 12.43)
 - mkdir 253 (time 1787793571.22 total 20.06 last 12.79)
 - mkdir 444 (time 1787793581.28 total 30.13 last 18.97)
 - mkdir 589 (time 1787793591.31 total 40.15 last 14.47)
 - mkdir 741 (time 1787793601.35 total 50.20 last 15.13)
 - mkdir 876 (time 1787793611.35 total 60.20 last 13.50)
 - mkdir 992 (time 1787793621.39 total 70.24 last 11.55)
total: 1000 mkdir in 71.04 seconds: 14.08 ops/second
 - mkdir 126 (time 1787793563.57 total 10.01 last 12.59)
 - mkdir 281 (time 1787793573.63 total 20.07 last 15.41)
 - mkdir 453 (time 1787793583.68 total 30.11 last 17.12)
 - mkdir 600 (time 1787793593.68 total 40.12 last 14.70)
 - mkdir 763 (time 1787793603.70 total 50.13 last 16.27)
 - mkdir 884 (time 1787793613.74 total 60.18 last 12.05)
 - mkdir 997 (time 1787793623.84 total 70.28 last 11.19)
total: 1000 mkdir in 70.46 seconds: 14.19 ops/second
 - mkdir 130 (time 1787793563.27 total 10.07 last 12.91)
 - mkdir 279 (time 1787793573.27 total 20.07 last 14.89)
 - mkdir 452 (time 1787793583.30 total 30.10 last 17.26)
 - mkdir 597 (time 1787793593.34 total 40.14 last 14.43)
 - mkdir 756 (time 1787793603.36 total 50.16 last 15.87)
 - mkdir 883 (time 1787793613.37 total 60.17 last 12.68)
 - mkdir 994 (time 1787793623.44 total 70.24 last 11.02)
total: 1000 mkdir in 70.98 seconds: 14.09 ops/second
 - mkdir 126 (time 1787793563.30 total 10.03 last 12.57)
 - mkdir 274 (time 1787793573.31 total 20.04 last 14.77)
 - mkdir 451 (time 1787793583.36 total 30.09 last 17.62)
 - mkdir 596 (time 1787793593.43 total 40.16 last 14.41)
 - mkdir 757 (time 1787793603.47 total 50.20 last 16.03)
 - mkdir 882 (time 1787793613.56 total 60.29 last 12.39)
 - mkdir 994 (time 1787793623.79 total 70.52 last 10.95)
total: 1000 mkdir in 71.43 seconds: 14.00 ops/second
 - mkdir 125 (time 1787793563.40 total 10.09 last 12.39)
 - mkdir 276 (time 1787793573.40 total 20.09 last 15.10)
 - mkdir 446 (time 1787793583.49 total 30.17 last 16.86)
 - mkdir 592 (time 1787793593.51 total 40.19 last 14.57)
 - mkdir 755 (time 1787793603.54 total 50.22 last 16.25)
 - mkdir 877 (time 1787793613.62 total 60.30 last 12.10)
 - mkdir 987 (time 1787793623.85 total 70.53 last 10.76)
total: 1000 mkdir in 72.41 seconds: 13.81 ops/second
 - mkdir 116 (time 1787793563.51 total 10.07 last 11.52)
 - mkdir 260 (time 1787793573.53 total 20.09 last 14.37)
 - mkdir 424 (time 1787793583.55 total 30.10 last 16.38)
 - mkdir 572 (time 1787793593.63 total 40.18 last 14.68)
 - mkdir 736 (time 1787793603.69 total 50.25 last 16.30)
 - mkdir 861 (time 1787793613.75 total 60.30 last 12.43)
 - mkdir 968 (time 1787793623.94 total 70.50 last 10.50)
total: 1000 mkdir in 74.32 seconds: 13.45 ops/second
 - mkdir 117 (time 1787793564.26 total 10.01 last 11.69)
 - mkdir 273 (time 1787793574.29 total 20.04 last 15.55)
 - mkdir 436 (time 1787793584.31 total 30.06 last 16.28)
 - mkdir 578 (time 1787793594.34 total 40.08 last 14.16)
 - mkdir 740 (time 1787793604.36 total 50.10 last 16.17)
 - mkdir 862 (time 1787793614.37 total 60.11 last 12.19)
 - mkdir 970 (time 1787793624.40 total 70.15 last 10.76)
total: 1000 mkdir in 73.60 seconds: 13.59 ops/second
 - mkdir 119 (time 1787793563.80 total 10.00 last 11.89)
 - mkdir 264 (time 1787793573.81 total 20.01 last 14.49)
 - mkdir 427 (time 1787793583.81 total 30.02 last 16.29)
 - mkdir 571 (time 1787793593.88 total 40.08 last 14.31)
 - mkdir 734 (time 1787793603.94 total 50.15 last 16.20)
 - mkdir 855 (time 1787793614.03 total 60.23 last 12.00)
 - mkdir 963 (time 1787793624.17 total 70.37 last 10.65)
total: 1000 mkdir in 74.49 seconds: 13.42 ops/second
Time for file creation 88 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787793635 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787793637 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787793637 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787793638 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787793639 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787793639 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787793639 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787793639 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787793639 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787793639 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
Time for file removal 51 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8