== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 20:56:55 (1782262615)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.55 seconds: 18.09 ops/second
 - mkdir 165 (time 1782262638.15 total 10.11 last 16.32)
 - mkdir 315 (time 1782262648.18 total 20.15 last 14.95)
 - mkdir 499 (time 1782262658.21 total 30.18 last 18.34)
 - mkdir 657 (time 1782262668.21 total 40.18 last 15.80)
 - mkdir 814 (time 1782262678.25 total 50.21 last 15.65)
 - mkdir 963 (time 1782262688.33 total 60.29 last 14.78)
total: 1000 mkdir in 63.48 seconds: 15.75 ops/second
 - mkdir 164 (time 1782262638.20 total 10.02 last 16.37)
 - mkdir 316 (time 1782262648.20 total 20.02 last 15.20)
 - mkdir 492 (time 1782262658.20 total 30.02 last 17.59)
 - mkdir 650 (time 1782262668.23 total 40.05 last 15.76)
 - mkdir 809 (time 1782262678.24 total 50.06 last 15.88)
 - mkdir 957 (time 1782262688.33 total 60.15 last 14.66)
total: 1000 mkdir in 64.25 seconds: 15.56 ops/second
 - mkdir 163 (time 1782262638.60 total 10.00 last 16.29)
 - mkdir 316 (time 1782262648.65 total 20.05 last 15.22)
 - mkdir 495 (time 1782262658.65 total 30.06 last 17.89)
 - mkdir 650 (time 1782262668.69 total 40.10 last 15.44)
 - mkdir 811 (time 1782262678.71 total 50.11 last 16.08)
 - mkdir 952 (time 1782262688.75 total 60.16 last 14.04)
total: 1000 mkdir in 65.29 seconds: 15.32 ops/second
 - mkdir 163 (time 1782262638.91 total 10.02 last 16.26)
 - mkdir 314 (time 1782262648.91 total 20.03 last 15.09)
 - mkdir 493 (time 1782262658.92 total 30.04 last 17.88)
 - mkdir 653 (time 1782262668.97 total 40.09 last 15.92)
 - mkdir 813 (time 1782262678.98 total 50.10 last 15.98)
 - mkdir 953 (time 1782262689.03 total 60.14 last 13.94)
total: 1000 mkdir in 65.74 seconds: 15.21 ops/second
 - mkdir 157 (time 1782262638.97 total 10.00 last 15.70)
 - mkdir 308 (time 1782262649.00 total 20.02 last 15.07)
 - mkdir 487 (time 1782262659.00 total 30.03 last 17.90)
 - mkdir 648 (time 1782262669.03 total 40.05 last 16.05)
 - mkdir 814 (time 1782262679.03 total 50.06 last 16.60)
 - mkdir 953 (time 1782262689.03 total 60.06 last 13.90)
total: 1000 mkdir in 65.72 seconds: 15.22 ops/second
 - mkdir 158 (time 1782262638.64 total 10.04 last 15.74)
 - mkdir 313 (time 1782262648.70 total 20.10 last 15.41)
 - mkdir 492 (time 1782262658.75 total 30.15 last 17.81)
 - mkdir 645 (time 1782262668.78 total 40.18 last 15.26)
 - mkdir 810 (time 1782262678.80 total 50.20 last 16.46)
 - mkdir 951 (time 1782262688.92 total 60.32 last 13.94)
total: 1000 mkdir in 66.12 seconds: 15.12 ops/second
 - mkdir 157 (time 1782262637.20 total 10.03 last 15.65)
 - mkdir 300 (time 1782262647.23 total 20.06 last 14.26)
 - mkdir 461 (time 1782262657.26 total 30.09 last 16.06)
 - mkdir 627 (time 1782262667.31 total 40.14 last 16.50)
 - mkdir 784 (time 1782262677.33 total 50.16 last 15.67)
 - mkdir 932 (time 1782262687.34 total 60.17 last 14.79)
total: 1000 mkdir in 67.68 seconds: 14.78 ops/second
 - mkdir 162 (time 1782262639.27 total 10.04 last 16.14)
 - mkdir 319 (time 1782262649.32 total 20.09 last 15.62)
 - mkdir 497 (time 1782262659.36 total 30.13 last 17.72)
 - mkdir 655 (time 1782262669.44 total 40.22 last 15.67)
 - mkdir 819 (time 1782262679.48 total 50.25 last 16.34)
 - mkdir 953 (time 1782262689.51 total 60.28 last 13.37)
total: 1000 mkdir in 65.73 seconds: 15.21 ops/second
 - mkdir 160 (time 1782262638.70 total 10.11 last 15.83)
 - mkdir 313 (time 1782262648.71 total 20.12 last 15.28)
 - mkdir 488 (time 1782262658.73 total 30.14 last 17.47)
 - mkdir 644 (time 1782262668.78 total 40.19 last 15.52)
 - mkdir 807 (time 1782262678.81 total 50.23 last 16.24)
 - mkdir 944 (time 1782262688.92 total 60.33 last 13.56)
total: 1000 mkdir in 67.13 seconds: 14.90 ops/second
 - mkdir 155 (time 1782262637.73 total 10.05 last 15.42)
 - mkdir 293 (time 1782262647.82 total 20.13 last 13.69)
 - mkdir 458 (time 1782262657.82 total 30.14 last 16.49)
 - mkdir 619 (time 1782262667.87 total 40.18 last 16.03)
 - mkdir 778 (time 1782262677.90 total 50.22 last 15.85)
 - mkdir 930 (time 1782262687.97 total 60.28 last 15.10)
total: 1000 mkdir in 68.76 seconds: 14.54 ops/second
Time for file creation 81 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 15 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.56 seconds: 17.72 ops/second
 - mkdir 137 (time 1782263196.96 total 10.02 last 13.68)
 - mkdir 314 (time 1782263206.97 total 20.03 last 17.69)
 - mkdir 483 (time 1782263216.99 total 30.05 last 16.86)
 - mkdir 639 (time 1782263227.00 total 40.06 last 15.59)
 - mkdir 828 (time 1782263237.04 total 50.10 last 18.82)
 - mkdir 969 (time 1782263247.05 total 60.11 last 14.08)
total: 1000 mkdir in 61.95 seconds: 16.14 ops/second
 - mkdir 142 (time 1782263198.53 total 10.08 last 14.09)
 - mkdir 326 (time 1782263208.56 total 20.11 last 18.34)
 - mkdir 498 (time 1782263218.64 total 30.19 last 17.06)
 - mkdir 653 (time 1782263228.71 total 40.26 last 15.40)
 - mkdir 839 (time 1782263238.85 total 50.40 last 18.33)
 - mkdir 982 (time 1782263248.86 total 60.41 last 14.28)
total: 1000 mkdir in 62.42 seconds: 16.02 ops/second
 - mkdir 141 (time 1782263199.58 total 10.02 last 14.07)
 - mkdir 328 (time 1782263209.63 total 20.07 last 18.60)
 - mkdir 489 (time 1782263219.66 total 30.11 last 16.05)
 - mkdir 637 (time 1782263229.67 total 40.11 last 14.79)
 - mkdir 829 (time 1782263239.73 total 50.17 last 19.09)
 - mkdir 973 (time 1782263249.75 total 60.19 last 14.36)
total: 1000 mkdir in 64.00 seconds: 15.62 ops/second
 - mkdir 147 (time 1782263199.84 total 10.09 last 14.57)
 - mkdir 334 (time 1782263209.85 total 20.10 last 18.67)
 - mkdir 491 (time 1782263219.88 total 30.13 last 15.66)
 - mkdir 643 (time 1782263229.89 total 40.14 last 15.18)
 - mkdir 828 (time 1782263240.00 total 50.25 last 18.29)
 - mkdir 973 (time 1782263250.02 total 60.27 last 14.48)
total: 1000 mkdir in 63.86 seconds: 15.66 ops/second
 - mkdir 144 (time 1782263199.94 total 10.05 last 14.33)
 - mkdir 332 (time 1782263209.94 total 20.05 last 18.79)
 - mkdir 489 (time 1782263219.96 total 30.07 last 15.67)
 - mkdir 644 (time 1782263229.99 total 40.10 last 15.46)
 - mkdir 829 (time 1782263240.00 total 50.12 last 18.47)
 - mkdir 968 (time 1782263250.05 total 60.16 last 13.84)
total: 1000 mkdir in 64.58 seconds: 15.49 ops/second
 - mkdir 132 (time 1782263198.80 total 10.00 last 13.20)
 - mkdir 305 (time 1782263208.84 total 20.04 last 17.24)
 - mkdir 461 (time 1782263218.91 total 30.12 last 15.48)
 - mkdir 621 (time 1782263228.94 total 40.15 last 15.95)
 - mkdir 809 (time 1782263238.95 total 50.16 last 18.78)
 - mkdir 954 (time 1782263249.04 total 60.24 last 14.38)
total: 1000 mkdir in 66.19 seconds: 15.11 ops/second
 - mkdir 142 (time 1782263199.59 total 10.02 last 14.18)
 - mkdir 319 (time 1782263209.60 total 20.03 last 17.68)
 - mkdir 468 (time 1782263219.64 total 30.08 last 14.83)
 - mkdir 622 (time 1782263229.67 total 40.11 last 15.35)
 - mkdir 814 (time 1782263239.72 total 50.16 last 19.11)
 - mkdir 959 (time 1782263249.92 total 60.36 last 14.21)
total: 1000 mkdir in 65.71 seconds: 15.22 ops/second
 - mkdir 140 (time 1782263200.14 total 10.04 last 13.94)
 - mkdir 325 (time 1782263210.16 total 20.07 last 18.45)
 - mkdir 484 (time 1782263220.19 total 30.09 last 15.86)
 - mkdir 636 (time 1782263230.19 total 40.09 last 15.20)
 - mkdir 823 (time 1782263240.23 total 50.13 last 18.63)
 - mkdir 967 (time 1782263250.28 total 60.19 last 14.32)
total: 1000 mkdir in 65.20 seconds: 15.34 ops/second
 - mkdir 146 (time 1782263199.96 total 10.05 last 14.53)
 - mkdir 329 (time 1782263210.00 total 20.09 last 18.23)
 - mkdir 485 (time 1782263220.02 total 30.11 last 15.56)
 - mkdir 635 (time 1782263230.06 total 40.15 last 14.95)
 - mkdir 819 (time 1782263240.12 total 50.20 last 18.30)
 - mkdir 961 (time 1782263250.12 total 60.21 last 14.19)
total: 1000 mkdir in 65.89 seconds: 15.18 ops/second
 - mkdir 136 (time 1782263199.26 total 10.01 last 13.59)
 - mkdir 316 (time 1782263209.28 total 20.03 last 17.96)
 - mkdir 471 (time 1782263219.31 total 30.06 last 15.45)
 - mkdir 624 (time 1782263229.38 total 40.13 last 15.19)
 - mkdir 814 (time 1782263239.40 total 50.15 last 18.98)
 - mkdir 953 (time 1782263249.47 total 60.22 last 13.80)
total: 1000 mkdir in 66.78 seconds: 14.97 ops/second
Time for file creation 80 sec for 1 RPC sent at a time
 - unlinked 0 (time 1782263267 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1782263268 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1782263267 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1782263269 ; total 1 ; last 1)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1782263268 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1782263268 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1782263269 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1782263269 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1782263269 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1782263270 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
Time for file removal 47 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8