== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 08:35:28 (1788784528)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.61 seconds: 16.34 ops/second
 - mkdir 165 (time 1788784552.06 total 10.02 last 16.47)
 - mkdir 324 (time 1788784562.09 total 20.05 last 15.85)
 - mkdir 486 (time 1788784572.11 total 30.07 last 16.17)
 - mkdir 665 (time 1788784582.17 total 40.13 last 17.80)
 - mkdir 823 (time 1788784592.20 total 50.16 last 15.76)
 - mkdir 962 (time 1788784602.25 total 60.21 last 13.83)
total: 1000 mkdir in 62.53 seconds: 15.99 ops/second
 - mkdir 131 (time 1788784549.40 total 10.02 last 13.08)
 - mkdir 288 (time 1788784559.45 total 20.07 last 15.62)
 - mkdir 451 (time 1788784569.51 total 30.13 last 16.21)
 - mkdir 631 (time 1788784579.56 total 40.18 last 17.91)
 - mkdir 783 (time 1788784589.58 total 50.20 last 15.17)
 - mkdir 934 (time 1788784599.60 total 60.22 last 15.07)
total: 1000 mkdir in 65.29 seconds: 15.32 ops/second
 - mkdir 145 (time 1788784550.32 total 10.01 last 14.48)
 - mkdir 297 (time 1788784560.36 total 20.06 last 15.13)
 - mkdir 459 (time 1788784570.37 total 30.06 last 16.19)
 - mkdir 640 (time 1788784580.39 total 40.09 last 18.06)
 - mkdir 790 (time 1788784590.41 total 50.11 last 14.97)
 - mkdir 936 (time 1788784600.45 total 60.15 last 14.54)
total: 1000 mkdir in 65.20 seconds: 15.34 ops/second
 - mkdir 152 (time 1788784551.06 total 10.03 last 15.16)
 - mkdir 298 (time 1788784561.07 total 20.04 last 14.59)
 - mkdir 456 (time 1788784571.09 total 30.05 last 15.78)
 - mkdir 636 (time 1788784581.11 total 40.08 last 17.95)
 - mkdir 794 (time 1788784591.13 total 50.10 last 15.77)
 - mkdir 936 (time 1788784601.16 total 60.13 last 14.15)
total: 1000 mkdir in 64.99 seconds: 15.39 ops/second
 - mkdir 153 (time 1788784551.26 total 10.00 last 15.29)
 - mkdir 297 (time 1788784561.28 total 20.03 last 14.36)
 - mkdir 456 (time 1788784571.31 total 30.06 last 15.86)
 - mkdir 636 (time 1788784581.33 total 40.08 last 17.96)
 - mkdir 798 (time 1788784591.35 total 50.10 last 16.16)
 - mkdir 939 (time 1788784601.45 total 60.20 last 13.96)
total: 1000 mkdir in 65.51 seconds: 15.26 ops/second
 - mkdir 155 (time 1788784552.25 total 10.01 last 15.49)
 - mkdir 314 (time 1788784562.25 total 20.01 last 15.90)
 - mkdir 475 (time 1788784572.27 total 30.03 last 16.07)
 - mkdir 651 (time 1788784582.29 total 40.05 last 17.57)
 - mkdir 809 (time 1788784592.40 total 50.15 last 15.63)
 - mkdir 947 (time 1788784602.43 total 60.19 last 13.75)
total: 1000 mkdir in 64.59 seconds: 15.48 ops/second
 - mkdir 154 (time 1788784552.37 total 10.07 last 15.29)
 - mkdir 313 (time 1788784562.42 total 20.12 last 15.82)
 - mkdir 470 (time 1788784572.49 total 30.19 last 15.59)
 - mkdir 645 (time 1788784582.50 total 40.20 last 17.48)
 - mkdir 807 (time 1788784592.57 total 50.27 last 16.08)
 - mkdir 946 (time 1788784602.58 total 60.28 last 13.89)
total: 1000 mkdir in 65.04 seconds: 15.37 ops/second
 - mkdir 152 (time 1788784552.55 total 10.10 last 15.05)
 - mkdir 306 (time 1788784562.56 total 20.10 last 15.40)
 - mkdir 459 (time 1788784572.56 total 30.11 last 15.29)
 - mkdir 637 (time 1788784582.57 total 40.11 last 17.78)
 - mkdir 800 (time 1788784592.65 total 50.19 last 16.18)
 - mkdir 937 (time 1788784602.69 total 60.23 last 13.64)
total: 1000 mkdir in 65.91 seconds: 15.17 ops/second
 - mkdir 157 (time 1788784552.25 total 10.03 last 15.65)
 - mkdir 308 (time 1788784562.25 total 20.03 last 15.10)
 - mkdir 465 (time 1788784572.31 total 30.10 last 15.60)
 - mkdir 640 (time 1788784582.32 total 40.10 last 17.49)
 - mkdir 794 (time 1788784592.42 total 50.20 last 15.25)
 - mkdir 931 (time 1788784602.47 total 60.25 last 13.63)
total: 1000 mkdir in 66.34 seconds: 15.07 ops/second
 - mkdir 147 (time 1788784551.99 total 10.03 last 14.66)
 - mkdir 288 (time 1788784562.06 total 20.10 last 14.00)
 - mkdir 438 (time 1788784572.19 total 30.23 last 14.82)
 - mkdir 616 (time 1788784582.25 total 40.29 last 17.69)
 - mkdir 777 (time 1788784592.29 total 50.32 last 16.04)
 - mkdir 915 (time 1788784602.37 total 60.41 last 13.69)
total: 1000 mkdir in 68.71 seconds: 14.55 ops/second
Time for file creation 81 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 14 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.47 seconds: 21.24 ops/second
 - mkdir 141 (time 1788785148.03 total 10.06 last 14.02)
 - mkdir 329 (time 1788785158.07 total 20.10 last 18.72)
 - mkdir 476 (time 1788785168.11 total 30.15 last 14.63)
 - mkdir 645 (time 1788785178.14 total 40.17 last 16.86)
 - mkdir 797 (time 1788785188.15 total 50.19 last 15.18)
 - mkdir 979 (time 1788785198.18 total 60.21 last 18.15)
total: 1000 mkdir in 61.40 seconds: 16.29 ops/second
 - mkdir 137 (time 1788785145.78 total 10.02 last 13.68)
 - mkdir 307 (time 1788785155.79 total 20.03 last 16.97)
 - mkdir 448 (time 1788785165.84 total 30.08 last 14.04)
 - mkdir 609 (time 1788785175.85 total 40.09 last 16.07)
 - mkdir 751 (time 1788785185.90 total 50.14 last 14.13)
 - mkdir 914 (time 1788785195.92 total 60.16 last 16.27)
total: 1000 mkdir in 65.69 seconds: 15.22 ops/second
 - mkdir 147 (time 1788785147.36 total 10.02 last 14.67)
 - mkdir 317 (time 1788785157.38 total 20.04 last 16.97)
 - mkdir 459 (time 1788785167.41 total 30.07 last 14.15)
 - mkdir 620 (time 1788785177.51 total 40.17 last 15.94)
 - mkdir 773 (time 1788785187.59 total 50.24 last 15.19)
 - mkdir 951 (time 1788785197.67 total 60.33 last 17.66)
total: 1000 mkdir in 65.38 seconds: 15.29 ops/second
 - mkdir 137 (time 1788785146.91 total 10.06 last 13.62)
 - mkdir 311 (time 1788785156.93 total 20.08 last 17.37)
 - mkdir 448 (time 1788785166.96 total 30.11 last 13.65)
 - mkdir 613 (time 1788785177.01 total 40.16 last 16.42)
 - mkdir 768 (time 1788785187.08 total 50.23 last 15.39)
 - mkdir 939 (time 1788785197.09 total 60.24 last 17.09)
total: 1000 mkdir in 66.20 seconds: 15.11 ops/second
 - mkdir 144 (time 1788785148.00 total 10.10 last 14.25)
 - mkdir 321 (time 1788785158.04 total 20.14 last 17.63)
 - mkdir 461 (time 1788785168.06 total 30.16 last 13.97)
 - mkdir 628 (time 1788785178.10 total 40.20 last 16.64)
 - mkdir 775 (time 1788785188.12 total 50.23 last 14.66)
 - mkdir 953 (time 1788785198.16 total 60.26 last 17.74)
total: 1000 mkdir in 65.79 seconds: 15.20 ops/second
 - mkdir 135 (time 1788785146.94 total 10.05 last 13.43)
 - mkdir 305 (time 1788785156.98 total 20.09 last 16.94)
 - mkdir 444 (time 1788785167.03 total 30.14 last 13.82)
 - mkdir 602 (time 1788785177.10 total 40.21 last 15.70)
 - mkdir 758 (time 1788785187.14 total 50.25 last 15.54)
 - mkdir 926 (time 1788785197.15 total 60.26 last 16.78)
total: 1000 mkdir in 68.21 seconds: 14.66 ops/second
 - mkdir 141 (time 1788785147.98 total 10.11 last 13.94)
 - mkdir 316 (time 1788785158.01 total 20.14 last 17.44)
 - mkdir 455 (time 1788785168.05 total 30.19 last 13.84)
 - mkdir 616 (time 1788785178.08 total 40.21 last 16.06)
 - mkdir 770 (time 1788785188.14 total 50.27 last 15.31)
 - mkdir 944 (time 1788785198.16 total 60.30 last 17.35)
total: 1000 mkdir in 67.31 seconds: 14.86 ops/second
 - mkdir 136 (time 1788785145.81 total 10.03 last 13.56)
 - mkdir 303 (time 1788785155.85 total 20.07 last 16.63)
 - mkdir 440 (time 1788785165.86 total 30.08 last 13.68)
 - mkdir 597 (time 1788785175.91 total 40.13 last 15.62)
 - mkdir 732 (time 1788785185.96 total 50.18 last 13.44)
 - mkdir 886 (time 1788785195.97 total 60.19 last 15.39)
 - mkdir 998 (time 1788785206.03 total 70.25 last 11.13)
total: 1000 mkdir in 70.47 seconds: 14.19 ops/second
 - mkdir 135 (time 1788785148.15 total 10.06 last 13.42)
 - mkdir 309 (time 1788785158.20 total 20.11 last 17.31)
 - mkdir 448 (time 1788785168.22 total 30.12 last 13.88)
 - mkdir 607 (time 1788785178.22 total 40.13 last 15.89)
 - mkdir 757 (time 1788785188.24 total 50.15 last 14.97)
 - mkdir 933 (time 1788785198.25 total 60.16 last 17.58)
total: 1000 mkdir in 69.39 seconds: 14.41 ops/second
 - mkdir 134 (time 1788785147.99 total 10.11 last 13.26)
 - mkdir 303 (time 1788785158.02 total 20.13 last 16.85)
 - mkdir 435 (time 1788785168.02 total 30.14 last 13.19)
 - mkdir 587 (time 1788785178.05 total 40.17 last 15.16)
 - mkdir 722 (time 1788785188.06 total 50.18 last 13.48)
 - mkdir 884 (time 1788785198.07 total 60.19 last 16.19)
 - mkdir 957 (time 1788785208.18 total 70.30 last 7.22)
total: 1000 mkdir in 72.79 seconds: 13.74 ops/second
Time for file creation 85 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788785223 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1788785224 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1788785225 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1788785225 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1788785227 ; total 1 ; last 1)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1788785226 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1788785226 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1788785227 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1788785227 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1788785227 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
Time for file removal 54 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8