== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 15:13:12 (1780341192)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.63 seconds: 15.79 ops/second
 - mkdir 127 (time 1780341213.45 total 10.01 last 12.69)
 - mkdir 268 (time 1780341223.48 total 20.03 last 14.07)
 - mkdir 403 (time 1780341233.48 total 30.03 last 13.50)
 - mkdir 542 (time 1780341243.50 total 40.05 last 13.88)
 - mkdir 705 (time 1780341253.53 total 50.09 last 16.24)
 - mkdir 849 (time 1780341263.54 total 60.09 last 14.39)
total: 1000 mkdir in 69.86 seconds: 14.31 ops/second
 - mkdir 139 (time 1780341215.38 total 10.03 last 13.86)
 - mkdir 280 (time 1780341225.39 total 20.04 last 14.08)
 - mkdir 414 (time 1780341235.51 total 30.15 last 13.25)
 - mkdir 556 (time 1780341245.55 total 40.20 last 14.13)
 - mkdir 722 (time 1780341255.60 total 50.24 last 16.53)
 - mkdir 867 (time 1780341265.65 total 60.30 last 14.42)
total: 1000 mkdir in 69.94 seconds: 14.30 ops/second
 - mkdir 149 (time 1780341216.14 total 10.05 last 14.82)
 - mkdir 289 (time 1780341226.14 total 20.06 last 13.99)
 - mkdir 421 (time 1780341236.22 total 30.14 last 13.10)
 - mkdir 568 (time 1780341246.23 total 40.14 last 14.69)
 - mkdir 728 (time 1780341256.24 total 50.16 last 15.98)
 - mkdir 873 (time 1780341266.32 total 60.23 last 14.39)
total: 1000 mkdir in 69.29 seconds: 14.43 ops/second
 - mkdir 148 (time 1780341216.00 total 10.03 last 14.75)
 - mkdir 293 (time 1780341226.08 total 20.12 last 14.38)
 - mkdir 415 (time 1780341236.18 total 30.21 last 12.09)
 - mkdir 563 (time 1780341246.23 total 40.26 last 14.73)
 - mkdir 728 (time 1780341256.30 total 50.33 last 16.38)
 - mkdir 872 (time 1780341266.38 total 60.41 last 14.28)
total: 1000 mkdir in 69.56 seconds: 14.38 ops/second
 - mkdir 136 (time 1780341215.12 total 10.03 last 13.56)
 - mkdir 277 (time 1780341225.14 total 20.05 last 14.08)
 - mkdir 408 (time 1780341235.22 total 30.13 last 12.99)
 - mkdir 548 (time 1780341245.24 total 40.15 last 13.97)
 - mkdir 711 (time 1780341255.25 total 50.16 last 16.29)
 - mkdir 854 (time 1780341265.29 total 60.19 last 14.25)
 - mkdir 995 (time 1780341275.30 total 70.21 last 14.08)
total: 1000 mkdir in 70.60 seconds: 14.16 ops/second
 - mkdir 147 (time 1780341216.40 total 10.04 last 14.65)
 - mkdir 287 (time 1780341226.40 total 20.04 last 14.00)
 - mkdir 413 (time 1780341236.46 total 30.09 last 12.53)
 - mkdir 560 (time 1780341246.48 total 40.11 last 14.67)
 - mkdir 727 (time 1780341256.50 total 50.13 last 16.67)
 - mkdir 871 (time 1780341266.56 total 60.20 last 14.31)
total: 1000 mkdir in 69.57 seconds: 14.37 ops/second
 - mkdir 151 (time 1780341216.31 total 10.06 last 15.01)
 - mkdir 288 (time 1780341226.35 total 20.11 last 13.64)
 - mkdir 415 (time 1780341236.38 total 30.14 last 12.66)
 - mkdir 561 (time 1780341246.39 total 40.14 last 14.59)
 - mkdir 721 (time 1780341256.43 total 50.18 last 15.94)
 - mkdir 862 (time 1780341266.48 total 60.24 last 14.03)
total: 1000 mkdir in 69.97 seconds: 14.29 ops/second
 - mkdir 146 (time 1780341216.10 total 10.05 last 14.53)
 - mkdir 284 (time 1780341226.13 total 20.08 last 13.76)
 - mkdir 404 (time 1780341236.27 total 30.22 last 11.83)
 - mkdir 554 (time 1780341246.35 total 40.30 last 14.88)
 - mkdir 718 (time 1780341256.36 total 50.31 last 16.37)
 - mkdir 862 (time 1780341266.39 total 60.34 last 14.36)
 - mkdir 996 (time 1780341276.47 total 70.42 last 13.30)
total: 1000 mkdir in 70.78 seconds: 14.13 ops/second
 - mkdir 142 (time 1780341216.53 total 10.04 last 14.15)
 - mkdir 282 (time 1780341226.55 total 20.06 last 13.97)
 - mkdir 402 (time 1780341236.59 total 30.09 last 11.96)
 - mkdir 547 (time 1780341246.60 total 40.11 last 14.48)
 - mkdir 710 (time 1780341256.71 total 50.21 last 16.12)
 - mkdir 855 (time 1780341266.74 total 60.25 last 14.45)
 - mkdir 990 (time 1780341276.86 total 70.36 last 13.35)
total: 1000 mkdir in 71.31 seconds: 14.02 ops/second
 - mkdir 137 (time 1780341216.19 total 10.07 last 13.61)
 - mkdir 270 (time 1780341226.23 total 20.11 last 13.24)
 - mkdir 388 (time 1780341236.24 total 30.12 last 11.79)
 - mkdir 536 (time 1780341246.26 total 40.13 last 14.78)
 - mkdir 699 (time 1780341256.33 total 50.20 last 16.19)
 - mkdir 844 (time 1780341266.37 total 60.25 last 14.43)
 - mkdir 976 (time 1780341276.38 total 70.26 last 13.19)
total: 1000 mkdir in 72.47 seconds: 13.80 ops/second
Time for file creation 85 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 12 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.43 seconds: 23.09 ops/second
 - mkdir 209 (time 1780341784.61 total 10.03 last 20.84)
 - mkdir 413 (time 1780341794.64 total 20.06 last 20.34)
 - mkdir 605 (time 1780341804.67 total 30.09 last 19.14)
 - mkdir 785 (time 1780341814.68 total 40.10 last 17.98)
 - mkdir 963 (time 1780341824.72 total 50.14 last 17.73)
total: 1000 mkdir in 52.16 seconds: 19.17 ops/second
 - mkdir 213 (time 1780341784.05 total 10.04 last 21.21)
 - mkdir 411 (time 1780341794.10 total 20.09 last 19.70)
 - mkdir 598 (time 1780341804.13 total 30.12 last 18.64)
 - mkdir 773 (time 1780341814.14 total 40.13 last 17.48)
 - mkdir 949 (time 1780341824.14 total 50.13 last 17.60)
total: 1000 mkdir in 52.99 seconds: 18.87 ops/second
 - mkdir 211 (time 1780341785.23 total 10.08 last 20.93)
 - mkdir 415 (time 1780341795.30 total 20.15 last 20.25)
 - mkdir 617 (time 1780341805.36 total 30.21 last 20.09)
 - mkdir 793 (time 1780341815.37 total 40.22 last 17.57)
 - mkdir 973 (time 1780341825.40 total 50.25 last 17.96)
total: 1000 mkdir in 51.87 seconds: 19.28 ops/second
 - mkdir 204 (time 1780341784.92 total 10.04 last 20.33)
 - mkdir 405 (time 1780341794.97 total 20.09 last 19.98)
 - mkdir 597 (time 1780341805.00 total 30.12 last 19.16)
 - mkdir 776 (time 1780341815.01 total 40.13 last 17.87)
 - mkdir 959 (time 1780341825.05 total 50.17 last 18.23)
total: 1000 mkdir in 52.60 seconds: 19.01 ops/second
 - mkdir 204 (time 1780341784.39 total 10.03 last 20.34)
 - mkdir 402 (time 1780341794.44 total 20.07 last 19.71)
 - mkdir 589 (time 1780341804.48 total 30.12 last 18.62)
 - mkdir 765 (time 1780341814.52 total 40.15 last 17.53)
 - mkdir 939 (time 1780341824.56 total 50.19 last 17.33)
total: 1000 mkdir in 53.73 seconds: 18.61 ops/second
 - mkdir 205 (time 1780341784.62 total 10.00 last 20.49)
 - mkdir 402 (time 1780341794.66 total 20.05 last 19.61)
 - mkdir 594 (time 1780341804.68 total 30.07 last 19.16)
 - mkdir 770 (time 1780341814.69 total 40.08 last 17.59)
 - mkdir 946 (time 1780341824.71 total 50.10 last 17.55)
total: 1000 mkdir in 53.60 seconds: 18.66 ops/second
 - mkdir 205 (time 1780341785.03 total 10.01 last 20.48)
 - mkdir 402 (time 1780341795.06 total 20.03 last 19.65)
 - mkdir 595 (time 1780341805.09 total 30.07 last 19.24)
 - mkdir 772 (time 1780341815.13 total 40.11 last 17.62)
 - mkdir 955 (time 1780341825.15 total 50.13 last 18.27)
total: 1000 mkdir in 53.19 seconds: 18.80 ops/second
 - mkdir 210 (time 1780341785.04 total 10.05 last 20.90)
 - mkdir 414 (time 1780341795.14 total 20.14 last 20.21)
 - mkdir 606 (time 1780341805.19 total 30.19 last 19.11)
 - mkdir 782 (time 1780341815.28 total 40.28 last 17.44)
 - mkdir 957 (time 1780341825.34 total 50.34 last 17.40)
total: 1000 mkdir in 53.28 seconds: 18.77 ops/second
 - mkdir 208 (time 1780341785.10 total 10.06 last 20.68)
 - mkdir 408 (time 1780341795.14 total 20.10 last 19.93)
 - mkdir 600 (time 1780341805.18 total 30.14 last 19.11)
 - mkdir 776 (time 1780341815.19 total 40.15 last 17.60)
 - mkdir 956 (time 1780341825.23 total 50.19 last 17.92)
total: 1000 mkdir in 53.34 seconds: 18.75 ops/second
 - mkdir 200 (time 1780341784.12 total 10.02 last 19.97)
 - mkdir 391 (time 1780341794.19 total 20.09 last 18.96)
 - mkdir 573 (time 1780341804.28 total 30.17 last 18.05)
 - mkdir 751 (time 1780341814.32 total 40.22 last 17.73)
 - mkdir 931 (time 1780341824.38 total 50.28 last 17.88)
total: 1000 mkdir in 54.49 seconds: 18.35 ops/second
Time for file creation 63 sec for 1 RPC sent at a time
 - unlinked 0 (time 1780341837 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1780341838 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1780341838 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1780341838 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1780341839 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1780341838 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1780341839 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1780341838 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1780341839 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1780341839 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
Time for file removal 41 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8