== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 14:33:22 (1782239602)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.70 seconds: 14.25 ops/second
 - mkdir 150 (time 1782239623.49 total 10.04 last 14.93)
 - mkdir 309 (time 1782239633.51 total 20.07 last 15.86)
 - mkdir 440 (time 1782239643.57 total 30.12 last 13.03)
 - mkdir 593 (time 1782239653.60 total 40.16 last 15.25)
 - mkdir 744 (time 1782239663.61 total 50.16 last 15.09)
 - mkdir 901 (time 1782239673.61 total 60.17 last 15.70)
total: 1000 mkdir in 67.07 seconds: 14.91 ops/second
 - mkdir 144 (time 1782239621.88 total 10.05 last 14.32)
 - mkdir 293 (time 1782239631.88 total 20.06 last 14.89)
 - mkdir 417 (time 1782239641.90 total 30.08 last 12.38)
 - mkdir 557 (time 1782239651.93 total 40.11 last 13.95)
 - mkdir 709 (time 1782239661.99 total 50.17 last 15.11)
 - mkdir 869 (time 1782239672.00 total 60.18 last 15.99)
total: 1000 mkdir in 69.67 seconds: 14.35 ops/second
 - mkdir 145 (time 1782239623.75 total 10.01 last 14.49)
 - mkdir 307 (time 1782239633.81 total 20.07 last 16.10)
 - mkdir 434 (time 1782239643.92 total 30.18 last 12.56)
 - mkdir 589 (time 1782239653.93 total 40.19 last 15.48)
 - mkdir 738 (time 1782239663.99 total 50.25 last 14.82)
 - mkdir 898 (time 1782239674.02 total 60.28 last 15.95)
total: 1000 mkdir in 67.79 seconds: 14.75 ops/second
 - mkdir 141 (time 1782239624.60 total 10.04 last 14.04)
 - mkdir 305 (time 1782239634.66 total 20.10 last 16.31)
 - mkdir 439 (time 1782239644.70 total 30.15 last 13.34)
 - mkdir 591 (time 1782239654.78 total 40.22 last 15.09)
 - mkdir 745 (time 1782239664.84 total 50.28 last 15.31)
 - mkdir 899 (time 1782239674.88 total 60.33 last 15.33)
total: 1000 mkdir in 68.77 seconds: 14.54 ops/second
 - mkdir 143 (time 1782239624.06 total 10.03 last 14.26)
 - mkdir 301 (time 1782239634.07 total 20.05 last 15.77)
 - mkdir 428 (time 1782239644.13 total 30.11 last 12.63)
 - mkdir 576 (time 1782239654.16 total 40.13 last 14.76)
 - mkdir 730 (time 1782239664.22 total 50.19 last 15.31)
 - mkdir 890 (time 1782239674.25 total 60.22 last 15.96)
total: 1000 mkdir in 69.37 seconds: 14.42 ops/second
 - mkdir 142 (time 1782239624.47 total 10.01 last 14.19)
 - mkdir 304 (time 1782239634.53 total 20.07 last 16.09)
 - mkdir 429 (time 1782239644.60 total 30.13 last 12.42)
 - mkdir 582 (time 1782239654.60 total 40.14 last 15.29)
 - mkdir 740 (time 1782239664.60 total 50.14 last 15.79)
 - mkdir 896 (time 1782239674.62 total 60.16 last 15.57)
total: 1000 mkdir in 68.94 seconds: 14.51 ops/second
 - mkdir 144 (time 1782239624.62 total 10.00 last 14.40)
 - mkdir 306 (time 1782239634.73 total 20.11 last 16.03)
 - mkdir 428 (time 1782239644.78 total 30.16 last 12.14)
 - mkdir 578 (time 1782239654.78 total 40.16 last 15.00)
 - mkdir 739 (time 1782239664.80 total 50.18 last 16.07)
 - mkdir 894 (time 1782239674.93 total 60.31 last 15.30)
total: 1000 mkdir in 68.91 seconds: 14.51 ops/second
 - mkdir 145 (time 1782239624.36 total 10.01 last 14.48)
 - mkdir 306 (time 1782239634.41 total 20.06 last 16.02)
 - mkdir 433 (time 1782239644.42 total 30.07 last 12.69)
 - mkdir 581 (time 1782239654.42 total 40.07 last 14.80)
 - mkdir 735 (time 1782239664.46 total 50.11 last 15.35)
 - mkdir 892 (time 1782239674.48 total 60.13 last 15.67)
total: 1000 mkdir in 69.21 seconds: 14.45 ops/second
 - mkdir 139 (time 1782239624.09 total 10.00 last 13.90)
 - mkdir 293 (time 1782239634.14 total 20.05 last 15.32)
 - mkdir 413 (time 1782239644.20 total 30.12 last 11.92)
 - mkdir 569 (time 1782239654.25 total 40.17 last 15.52)
 - mkdir 724 (time 1782239664.32 total 50.23 last 15.41)
 - mkdir 885 (time 1782239674.35 total 60.26 last 16.05)
total: 1000 mkdir in 70.06 seconds: 14.27 ops/second
 - mkdir 141 (time 1782239624.76 total 10.05 last 14.03)
 - mkdir 300 (time 1782239634.79 total 20.08 last 15.85)
 - mkdir 425 (time 1782239644.80 total 30.09 last 12.50)
 - mkdir 575 (time 1782239654.84 total 40.13 last 14.93)
 - mkdir 732 (time 1782239664.86 total 50.15 last 15.66)
 - mkdir 891 (time 1782239674.88 total 60.17 last 15.86)
total: 1000 mkdir in 69.74 seconds: 14.34 ops/second
Time for file creation 81 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.43 seconds: 23.12 ops/second
 - mkdir 163 (time 1782240239.93 total 10.05 last 16.22)
 - mkdir 345 (time 1782240249.95 total 20.08 last 18.15)
 - mkdir 489 (time 1782240260.03 total 30.15 last 14.29)
 - mkdir 663 (time 1782240270.08 total 40.21 last 17.30)
 - mkdir 848 (time 1782240280.10 total 50.22 last 18.47)
total: 1000 mkdir in 58.32 seconds: 17.15 ops/second
 - mkdir 165 (time 1782240239.37 total 10.11 last 16.33)
 - mkdir 335 (time 1782240249.40 total 20.14 last 16.95)
 - mkdir 475 (time 1782240259.42 total 30.15 last 13.98)
 - mkdir 647 (time 1782240269.48 total 40.22 last 17.08)
 - mkdir 830 (time 1782240279.55 total 50.29 last 18.18)
total: 1000 mkdir in 60.01 seconds: 16.66 ops/second
 - mkdir 166 (time 1782240240.62 total 10.00 last 16.59)
 - mkdir 340 (time 1782240250.68 total 20.07 last 17.29)
 - mkdir 481 (time 1782240260.72 total 30.11 last 14.04)
 - mkdir 642 (time 1782240270.77 total 40.15 last 16.03)
 - mkdir 823 (time 1782240280.77 total 50.16 last 18.09)
 - mkdir 988 (time 1782240290.83 total 60.22 last 16.40)
total: 1000 mkdir in 61.78 seconds: 16.19 ops/second
 - mkdir 164 (time 1782240240.73 total 10.06 last 16.30)
 - mkdir 335 (time 1782240250.76 total 20.08 last 17.06)
 - mkdir 475 (time 1782240260.76 total 30.09 last 14.00)
 - mkdir 641 (time 1782240270.80 total 40.12 last 16.54)
 - mkdir 824 (time 1782240280.83 total 50.15 last 18.25)
 - mkdir 982 (time 1782240290.83 total 60.16 last 15.79)
total: 1000 mkdir in 62.46 seconds: 16.01 ops/second
 - mkdir 164 (time 1782240240.97 total 10.03 last 16.36)
 - mkdir 338 (time 1782240250.97 total 20.03 last 17.40)
 - mkdir 469 (time 1782240260.98 total 30.04 last 13.09)
 - mkdir 638 (time 1782240270.98 total 40.04 last 16.90)
 - mkdir 815 (time 1782240280.99 total 50.05 last 17.68)
 - mkdir 979 (time 1782240291.07 total 60.13 last 16.27)
total: 1000 mkdir in 62.89 seconds: 15.90 ops/second
 - mkdir 159 (time 1782240240.73 total 10.06 last 15.80)
 - mkdir 332 (time 1782240250.75 total 20.08 last 17.26)
 - mkdir 471 (time 1782240260.82 total 30.15 last 13.81)
 - mkdir 634 (time 1782240270.86 total 40.19 last 16.22)
 - mkdir 815 (time 1782240280.91 total 50.24 last 18.02)
 - mkdir 979 (time 1782240291.09 total 60.42 last 16.11)
total: 1000 mkdir in 63.40 seconds: 15.77 ops/second
 - mkdir 163 (time 1782240240.69 total 10.05 last 16.22)
 - mkdir 337 (time 1782240250.70 total 20.06 last 17.38)
 - mkdir 475 (time 1782240260.72 total 30.09 last 13.77)
 - mkdir 638 (time 1782240270.73 total 40.09 last 16.29)
 - mkdir 819 (time 1782240280.74 total 50.10 last 18.08)
 - mkdir 973 (time 1782240290.81 total 60.18 last 15.29)
total: 1000 mkdir in 63.64 seconds: 15.71 ops/second
 - mkdir 162 (time 1782240240.42 total 10.02 last 16.16)
 - mkdir 329 (time 1782240250.47 total 20.07 last 16.62)
 - mkdir 458 (time 1782240260.49 total 30.09 last 12.88)
 - mkdir 618 (time 1782240270.53 total 40.13 last 15.94)
 - mkdir 793 (time 1782240280.55 total 50.15 last 17.46)
 - mkdir 954 (time 1782240290.60 total 60.20 last 16.02)
total: 1000 mkdir in 65.48 seconds: 15.27 ops/second
 - mkdir 162 (time 1782240240.91 total 10.04 last 16.14)
 - mkdir 329 (time 1782240250.94 total 20.07 last 16.65)
 - mkdir 464 (time 1782240261.00 total 30.13 last 13.42)
 - mkdir 626 (time 1782240271.02 total 40.15 last 16.17)
 - mkdir 802 (time 1782240281.02 total 50.15 last 17.59)
 - mkdir 957 (time 1782240291.13 total 60.26 last 15.34)
total: 1000 mkdir in 65.12 seconds: 15.36 ops/second
 - mkdir 152 (time 1782240239.66 total 10.00 last 15.20)
 - mkdir 318 (time 1782240249.68 total 20.03 last 16.55)
 - mkdir 451 (time 1782240259.73 total 30.07 last 13.24)
 - mkdir 605 (time 1782240269.77 total 40.12 last 15.33)
 - mkdir 774 (time 1782240279.80 total 50.14 last 16.85)
 - mkdir 935 (time 1782240289.81 total 60.16 last 16.08)
total: 1000 mkdir in 67.05 seconds: 14.91 ops/second
Time for file creation 75 sec for 1 RPC sent at a time
 - unlinked 0 (time 1782240302 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1782240304 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1782240305 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1782240305 ; total 1 ; last 1)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1782240305 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1782240305 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1782240305 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1782240306 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1782240306 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1782240306 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
Time for file removal 45 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8