== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 03:28:04 (1788852484)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.37 seconds: 27.31 ops/second
 - mkdir 151 (time 1788852503.17 total 10.02 last 15.06)
 - mkdir 280 (time 1788852513.17 total 20.03 last 12.90)
 - mkdir 551 (time 1788852523.21 total 30.06 last 27.01)
 - mkdir 714 (time 1788852533.29 total 40.14 last 16.16)
 - mkdir 843 (time 1788852543.37 total 50.22 last 12.80)
 - mkdir 944 (time 1788852553.41 total 60.27 last 10.05)
total: 1000 mkdir in 64.43 seconds: 15.52 ops/second
 - mkdir 148 (time 1788852505.21 total 10.03 last 14.76)
 - mkdir 296 (time 1788852515.22 total 20.03 last 14.79)
 - mkdir 567 (time 1788852525.28 total 30.09 last 26.94)
 - mkdir 722 (time 1788852535.31 total 40.12 last 15.46)
 - mkdir 849 (time 1788852545.40 total 50.21 last 12.58)
 - mkdir 970 (time 1788852555.50 total 60.31 last 11.98)
total: 1000 mkdir in 63.42 seconds: 15.77 ops/second
 - mkdir 151 (time 1788852504.65 total 10.11 last 14.93)
 - mkdir 288 (time 1788852514.66 total 20.12 last 13.69)
 - mkdir 557 (time 1788852524.67 total 30.13 last 26.87)
 - mkdir 715 (time 1788852534.70 total 40.16 last 15.76)
 - mkdir 849 (time 1788852544.74 total 50.20 last 13.35)
 - mkdir 958 (time 1788852554.79 total 60.26 last 10.84)
total: 1000 mkdir in 64.54 seconds: 15.49 ops/second
 - mkdir 147 (time 1788852505.37 total 10.05 last 14.62)
 - mkdir 309 (time 1788852515.37 total 20.06 last 16.20)
 - mkdir 563 (time 1788852525.42 total 30.10 last 25.28)
 - mkdir 717 (time 1788852535.44 total 40.12 last 15.37)
 - mkdir 838 (time 1788852545.52 total 50.20 last 12.00)
 - mkdir 952 (time 1788852555.70 total 60.39 last 11.20)
total: 1000 mkdir in 65.76 seconds: 15.21 ops/second
 - mkdir 148 (time 1788852505.30 total 10.03 last 14.75)
 - mkdir 300 (time 1788852515.30 total 20.03 last 15.20)
 - mkdir 556 (time 1788852525.31 total 30.04 last 25.57)
 - mkdir 706 (time 1788852535.33 total 40.07 last 14.97)
 - mkdir 831 (time 1788852545.35 total 50.08 last 12.48)
 - mkdir 946 (time 1788852555.53 total 60.27 last 11.29)
total: 1000 mkdir in 65.81 seconds: 15.19 ops/second
 - mkdir 142 (time 1788852505.26 total 10.03 last 14.15)
 - mkdir 291 (time 1788852515.26 total 20.04 last 14.90)
 - mkdir 548 (time 1788852525.27 total 30.04 last 25.68)
 - mkdir 705 (time 1788852535.29 total 40.07 last 15.65)
 - mkdir 828 (time 1788852545.40 total 50.17 last 12.18)
 - mkdir 940 (time 1788852555.45 total 60.23 last 11.14)
total: 1000 mkdir in 66.58 seconds: 15.02 ops/second
 - mkdir 150 (time 1788852505.70 total 10.03 last 14.95)
 - mkdir 311 (time 1788852515.70 total 20.04 last 16.10)
 - mkdir 562 (time 1788852525.71 total 30.04 last 25.09)
 - mkdir 711 (time 1788852535.74 total 40.08 last 14.85)
 - mkdir 827 (time 1788852545.83 total 50.16 last 11.50)
 - mkdir 942 (time 1788852555.86 total 60.20 last 11.46)
total: 1000 mkdir in 66.28 seconds: 15.09 ops/second
 - mkdir 150 (time 1788852505.80 total 10.08 last 14.89)
 - mkdir 314 (time 1788852515.84 total 20.11 last 16.34)
 - mkdir 565 (time 1788852525.88 total 30.15 last 25.00)
 - mkdir 717 (time 1788852535.90 total 40.17 last 15.17)
 - mkdir 830 (time 1788852546.00 total 50.27 last 11.19)
 - mkdir 943 (time 1788852556.04 total 60.32 last 11.25)
total: 1000 mkdir in 66.30 seconds: 15.08 ops/second
 - mkdir 141 (time 1788852505.04 total 10.16 last 13.88)
 - mkdir 282 (time 1788852515.06 total 20.18 last 14.07)
 - mkdir 545 (time 1788852525.11 total 30.22 last 26.18)
 - mkdir 691 (time 1788852535.12 total 40.23 last 14.59)
 - mkdir 817 (time 1788852545.15 total 50.26 last 12.56)
 - mkdir 931 (time 1788852555.17 total 60.28 last 11.37)
total: 1000 mkdir in 67.93 seconds: 14.72 ops/second
 - mkdir 140 (time 1788852505.06 total 10.06 last 13.91)
 - mkdir 280 (time 1788852515.06 total 20.06 last 14.00)
 - mkdir 539 (time 1788852525.11 total 30.12 last 25.76)
 - mkdir 686 (time 1788852535.13 total 40.13 last 14.67)
 - mkdir 809 (time 1788852545.23 total 50.24 last 12.17)
 - mkdir 920 (time 1788852555.34 total 60.34 last 10.99)
total: 1000 mkdir in 69.04 seconds: 14.48 ops/second
Time for file creation 78 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.60 seconds: 16.77 ops/second
 - mkdir 190 (time 1788853101.57 total 10.00 last 18.99)
 - mkdir 301 (time 1788853111.63 total 20.06 last 11.04)
 - mkdir 406 (time 1788853121.68 total 30.11 last 10.44)
 - mkdir 568 (time 1788853131.74 total 40.17 last 16.10)
 - mkdir 759 (time 1788853141.75 total 50.19 last 19.08)
 - mkdir 930 (time 1788853151.78 total 60.21 last 17.05)
total: 1000 mkdir in 64.19 seconds: 15.58 ops/second
 - mkdir 180 (time 1788853102.53 total 10.00 last 18.00)
 - mkdir 279 (time 1788853112.64 total 20.11 last 9.79)
 - mkdir 390 (time 1788853122.65 total 30.12 last 11.09)
 - mkdir 561 (time 1788853132.70 total 40.17 last 17.02)
 - mkdir 769 (time 1788853142.72 total 50.19 last 20.76)
 - mkdir 931 (time 1788853152.72 total 60.19 last 16.20)
total: 1000 mkdir in 64.52 seconds: 15.50 ops/second
 - mkdir 174 (time 1788853103.96 total 10.01 last 17.39)
 - mkdir 274 (time 1788853113.96 total 20.01 last 9.99)
 - mkdir 394 (time 1788853124.00 total 30.05 last 11.96)
 - mkdir 581 (time 1788853134.02 total 40.07 last 18.65)
 - mkdir 777 (time 1788853144.03 total 50.08 last 19.59)
 - mkdir 947 (time 1788853154.06 total 60.11 last 16.94)
total: 1000 mkdir in 64.65 seconds: 15.47 ops/second
 - mkdir 174 (time 1788853103.46 total 10.01 last 17.38)
 - mkdir 278 (time 1788853113.62 total 20.18 last 10.23)
 - mkdir 401 (time 1788853123.65 total 30.20 last 12.27)
 - mkdir 576 (time 1788853133.66 total 40.21 last 17.48)
 - mkdir 773 (time 1788853143.66 total 50.22 last 19.70)
 - mkdir 935 (time 1788853153.71 total 60.27 last 16.11)
total: 1000 mkdir in 65.38 seconds: 15.29 ops/second
 - mkdir 177 (time 1788853103.67 total 10.08 last 17.57)
 - mkdir 276 (time 1788853113.69 total 20.09 last 9.88)
 - mkdir 392 (time 1788853123.76 total 30.16 last 11.52)
 - mkdir 575 (time 1788853133.76 total 40.17 last 18.29)
 - mkdir 767 (time 1788853143.78 total 50.19 last 19.16)
 - mkdir 931 (time 1788853153.81 total 60.21 last 16.35)
total: 1000 mkdir in 66.38 seconds: 15.06 ops/second
 - mkdir 169 (time 1788853103.63 total 10.07 last 16.79)
 - mkdir 272 (time 1788853113.78 total 20.22 last 10.15)
 - mkdir 386 (time 1788853123.81 total 30.24 last 11.37)
 - mkdir 565 (time 1788853133.84 total 40.28 last 17.83)
 - mkdir 760 (time 1788853143.85 total 50.28 last 19.50)
 - mkdir 930 (time 1788853153.87 total 60.31 last 16.95)
total: 1000 mkdir in 66.50 seconds: 15.04 ops/second
 - mkdir 175 (time 1788853103.95 total 10.00 last 17.50)
 - mkdir 281 (time 1788853113.97 total 20.02 last 10.58)
 - mkdir 393 (time 1788853124.01 total 30.06 last 11.15)
 - mkdir 568 (time 1788853134.04 total 40.09 last 17.46)
 - mkdir 758 (time 1788853144.07 total 50.12 last 18.94)
 - mkdir 928 (time 1788853154.10 total 60.15 last 16.95)
total: 1000 mkdir in 67.01 seconds: 14.92 ops/second
 - mkdir 171 (time 1788853104.17 total 10.15 last 16.85)
 - mkdir 272 (time 1788853114.23 total 20.20 last 10.04)
 - mkdir 395 (time 1788853124.26 total 30.23 last 12.26)
 - mkdir 576 (time 1788853134.30 total 40.27 last 18.02)
 - mkdir 772 (time 1788853144.33 total 50.31 last 19.53)
 - mkdir 932 (time 1788853154.35 total 60.33 last 15.97)
total: 1000 mkdir in 67.14 seconds: 14.89 ops/second
 - mkdir 169 (time 1788853103.61 total 10.12 last 16.69)
 - mkdir 267 (time 1788853113.62 total 20.13 last 9.79)
 - mkdir 376 (time 1788853123.62 total 30.14 last 10.90)
 - mkdir 547 (time 1788853133.65 total 40.17 last 17.05)
 - mkdir 738 (time 1788853143.72 total 50.23 last 18.98)
 - mkdir 902 (time 1788853153.76 total 60.27 last 16.33)
total: 1000 mkdir in 70.06 seconds: 14.27 ops/second
 - mkdir 164 (time 1788853103.60 total 10.05 last 16.31)
 - mkdir 260 (time 1788853113.71 total 20.16 last 9.50)
 - mkdir 365 (time 1788853123.80 total 30.26 last 10.40)
 - mkdir 531 (time 1788853133.81 total 40.27 last 16.59)
 - mkdir 714 (time 1788853143.84 total 50.30 last 18.24)
 - mkdir 870 (time 1788853153.86 total 60.32 last 15.57)
 - mkdir 972 (time 1788853163.91 total 70.37 last 10.15)
total: 1000 mkdir in 71.87 seconds: 13.91 ops/second
Time for file creation 83 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788853172 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788853173 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788853173 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788853173 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788853173 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788853174 ; total 1 ; last 1)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1788853173 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1788853174 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788853174 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788853174 ; total 1 ; last 1)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
Time for file removal 45 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8