== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 03:18:04 (1781248684)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.58 seconds: 17.38 ops/second
 - mkdir 147 (time 1781248706.14 total 10.05 last 14.63)
 - mkdir 281 (time 1781248716.23 total 20.13 last 13.29)
 - mkdir 406 (time 1781248726.23 total 30.14 last 12.49)
 - mkdir 511 (time 1781248736.46 total 40.36 last 10.27)
 - mkdir 594 (time 1781248746.50 total 50.40 last 8.27)
 - mkdir 722 (time 1781248756.55 total 60.45 last 12.73)
 - mkdir 863 (time 1781248766.63 total 70.53 last 13.99)
total: 1000 mkdir in 80.46 seconds: 12.43 ops/second
 - mkdir 158 (time 1781248707.28 total 10.04 last 15.73)
 - mkdir 288 (time 1781248717.35 total 20.11 last 12.91)
 - mkdir 420 (time 1781248727.36 total 30.13 last 13.18)
 - mkdir 517 (time 1781248737.39 total 40.15 last 9.67)
 - mkdir 599 (time 1781248747.56 total 50.32 last 8.06)
 - mkdir 728 (time 1781248757.59 total 60.36 last 12.86)
 - mkdir 868 (time 1781248767.62 total 70.38 last 13.96)
total: 1000 mkdir in 80.31 seconds: 12.45 ops/second
 - mkdir 149 (time 1781248706.34 total 10.04 last 14.85)
 - mkdir 280 (time 1781248716.36 total 20.06 last 13.06)
 - mkdir 405 (time 1781248726.42 total 30.12 last 12.43)
 - mkdir 506 (time 1781248736.53 total 40.24 last 9.99)
 - mkdir 587 (time 1781248746.57 total 50.27 last 8.07)
 - mkdir 711 (time 1781248756.61 total 60.31 last 12.36)
 - mkdir 845 (time 1781248766.67 total 70.38 last 13.31)
 - mkdir 985 (time 1781248776.68 total 80.39 last 13.99)
total: 1000 mkdir in 82.40 seconds: 12.14 ops/second
 - mkdir 156 (time 1781248707.70 total 10.11 last 15.44)
 - mkdir 288 (time 1781248717.74 total 20.14 last 13.16)
 - mkdir 414 (time 1781248727.78 total 30.18 last 12.54)
 - mkdir 510 (time 1781248737.90 total 40.30 last 9.49)
 - mkdir 595 (time 1781248747.92 total 50.32 last 8.48)
 - mkdir 728 (time 1781248757.94 total 60.34 last 13.27)
 - mkdir 865 (time 1781248768.01 total 70.41 last 13.61)
 - mkdir 990 (time 1781248778.07 total 80.47 last 12.43)
total: 1000 mkdir in 81.95 seconds: 12.20 ops/second
 - mkdir 156 (time 1781248707.49 total 10.05 last 15.53)
 - mkdir 284 (time 1781248717.58 total 20.14 last 12.69)
 - mkdir 411 (time 1781248727.62 total 30.18 last 12.65)
 - mkdir 504 (time 1781248737.62 total 40.18 last 9.30)
 - mkdir 588 (time 1781248747.68 total 50.24 last 8.35)
 - mkdir 719 (time 1781248757.74 total 60.30 last 13.02)
 - mkdir 854 (time 1781248767.77 total 70.33 last 13.46)
 - mkdir 986 (time 1781248777.99 total 80.55 last 12.92)
total: 1000 mkdir in 82.47 seconds: 12.13 ops/second
 - mkdir 151 (time 1781248707.21 total 10.09 last 14.97)
 - mkdir 277 (time 1781248717.27 total 20.15 last 12.53)
 - mkdir 402 (time 1781248727.28 total 30.16 last 12.48)
 - mkdir 502 (time 1781248737.30 total 40.18 last 9.98)
 - mkdir 590 (time 1781248747.31 total 50.19 last 8.79)
 - mkdir 718 (time 1781248757.35 total 60.23 last 12.75)
 - mkdir 850 (time 1781248767.42 total 70.30 last 13.11)
 - mkdir 978 (time 1781248777.54 total 80.42 last 12.65)
total: 1000 mkdir in 83.30 seconds: 12.00 ops/second
 - mkdir 145 (time 1781248707.80 total 10.02 last 14.47)
 - mkdir 273 (time 1781248717.81 total 20.03 last 12.79)
 - mkdir 398 (time 1781248727.83 total 30.05 last 12.47)
 - mkdir 498 (time 1781248737.88 total 40.10 last 9.95)
 - mkdir 583 (time 1781248747.99 total 50.20 last 8.41)
 - mkdir 713 (time 1781248758.04 total 60.26 last 12.93)
 - mkdir 848 (time 1781248768.07 total 70.28 last 13.46)
 - mkdir 981 (time 1781248778.15 total 80.37 last 13.19)
total: 1000 mkdir in 83.11 seconds: 12.03 ops/second
 - mkdir 149 (time 1781248707.34 total 10.06 last 14.81)
 - mkdir 273 (time 1781248717.35 total 20.07 last 12.38)
 - mkdir 398 (time 1781248727.36 total 30.08 last 12.49)
 - mkdir 493 (time 1781248737.49 total 40.21 last 9.38)
 - mkdir 579 (time 1781248747.58 total 50.31 last 8.52)
 - mkdir 710 (time 1781248757.61 total 60.34 last 13.06)
 - mkdir 840 (time 1781248767.67 total 70.39 last 12.93)
 - mkdir 968 (time 1781248777.68 total 80.41 last 12.78)
total: 1000 mkdir in 84.67 seconds: 11.81 ops/second
 - mkdir 147 (time 1781248707.92 total 10.04 last 14.64)
 - mkdir 276 (time 1781248717.95 total 20.08 last 12.85)
 - mkdir 402 (time 1781248727.98 total 30.10 last 12.57)
 - mkdir 497 (time 1781248738.11 total 40.23 last 9.38)
 - mkdir 581 (time 1781248748.15 total 50.28 last 8.37)
 - mkdir 713 (time 1781248758.24 total 60.36 last 13.09)
 - mkdir 849 (time 1781248768.28 total 70.41 last 13.54)
 - mkdir 973 (time 1781248778.40 total 80.53 last 12.25)
total: 1000 mkdir in 84.32 seconds: 11.86 ops/second
 - mkdir 137 (time 1781248705.75 total 10.01 last 13.68)
 - mkdir 257 (time 1781248715.79 total 20.06 last 11.95)
 - mkdir 374 (time 1781248725.85 total 30.12 last 11.63)
 - mkdir 481 (time 1781248735.95 total 40.22 last 10.59)
 - mkdir 562 (time 1781248746.03 total 50.30 last 8.04)
 - mkdir 690 (time 1781248756.10 total 60.36 last 12.72)
 - mkdir 820 (time 1781248766.17 total 70.44 last 12.91)
 - mkdir 945 (time 1781248776.17 total 80.44 last 12.50)
total: 1000 mkdir in 87.49 seconds: 11.43 ops/second
Time for file creation 97 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 15 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.74 seconds: 13.58 ops/second
 - mkdir 129 (time 1781249392.98 total 10.01 last 12.88)
 - mkdir 266 (time 1781249403.03 total 20.06 last 13.64)
 - mkdir 399 (time 1781249413.06 total 30.09 last 13.25)
 - mkdir 545 (time 1781249423.12 total 40.15 last 14.51)
 - mkdir 676 (time 1781249433.21 total 50.24 last 12.98)
 - mkdir 806 (time 1781249443.26 total 60.29 last 12.94)
 - mkdir 948 (time 1781249453.30 total 70.33 last 14.15)
total: 1000 mkdir in 73.66 seconds: 13.58 ops/second
 - mkdir 136 (time 1781249395.78 total 10.11 last 13.45)
 - mkdir 281 (time 1781249405.85 total 20.18 last 14.40)
 - mkdir 425 (time 1781249415.90 total 30.23 last 14.33)
 - mkdir 571 (time 1781249425.92 total 40.25 last 14.56)
 - mkdir 693 (time 1781249435.98 total 50.31 last 12.13)
 - mkdir 826 (time 1781249446.00 total 60.33 last 13.28)
 - mkdir 980 (time 1781249456.08 total 70.41 last 15.28)
total: 1000 mkdir in 72.39 seconds: 13.81 ops/second
 - mkdir 130 (time 1781249395.11 total 10.02 last 12.98)
 - mkdir 270 (time 1781249405.19 total 20.10 last 13.89)
 - mkdir 404 (time 1781249415.24 total 30.15 last 13.33)
 - mkdir 550 (time 1781249425.32 total 40.22 last 14.49)
 - mkdir 675 (time 1781249435.39 total 50.30 last 12.41)
 - mkdir 812 (time 1781249445.48 total 60.38 last 13.58)
 - mkdir 967 (time 1781249455.49 total 70.39 last 15.48)
total: 1000 mkdir in 73.32 seconds: 13.64 ops/second
 - mkdir 128 (time 1781249395.84 total 10.23 last 12.51)
 - mkdir 272 (time 1781249405.92 total 20.31 last 14.29)
 - mkdir 413 (time 1781249415.96 total 30.35 last 14.04)
 - mkdir 558 (time 1781249426.01 total 40.39 last 14.44)
 - mkdir 683 (time 1781249436.02 total 50.41 last 12.48)
 - mkdir 823 (time 1781249446.02 total 60.41 last 14.00)
 - mkdir 972 (time 1781249456.08 total 70.47 last 14.82)
total: 1000 mkdir in 73.98 seconds: 13.52 ops/second
 - mkdir 128 (time 1781249395.11 total 10.08 last 12.69)
 - mkdir 268 (time 1781249405.14 total 20.11 last 13.96)
 - mkdir 407 (time 1781249415.17 total 30.15 last 13.85)
 - mkdir 552 (time 1781249425.20 total 40.17 last 14.46)
 - mkdir 677 (time 1781249435.26 total 50.23 last 12.43)
 - mkdir 810 (time 1781249445.28 total 60.25 last 13.27)
 - mkdir 955 (time 1781249455.34 total 70.31 last 14.41)
total: 1000 mkdir in 74.81 seconds: 13.37 ops/second
 - mkdir 124 (time 1781249395.34 total 10.05 last 12.34)
 - mkdir 262 (time 1781249405.38 total 20.09 last 13.74)
 - mkdir 401 (time 1781249415.42 total 30.12 last 13.85)
 - mkdir 546 (time 1781249425.48 total 40.18 last 14.42)
 - mkdir 674 (time 1781249435.59 total 50.29 last 12.65)
 - mkdir 808 (time 1781249445.64 total 60.34 last 13.34)
 - mkdir 959 (time 1781249455.68 total 70.38 last 15.03)
total: 1000 mkdir in 74.61 seconds: 13.40 ops/second
 - mkdir 126 (time 1781249395.84 total 10.17 last 12.38)
 - mkdir 263 (time 1781249405.90 total 20.23 last 13.62)
 - mkdir 410 (time 1781249415.97 total 30.30 last 14.60)
 - mkdir 558 (time 1781249425.97 total 40.31 last 14.79)
 - mkdir 679 (time 1781249436.03 total 50.37 last 12.03)
 - mkdir 816 (time 1781249446.06 total 60.39 last 13.66)
 - mkdir 967 (time 1781249456.08 total 70.41 last 15.08)
total: 1000 mkdir in 74.26 seconds: 13.47 ops/second
 - mkdir 127 (time 1781249395.49 total 10.07 last 12.61)
 - mkdir 267 (time 1781249405.51 total 20.09 last 13.98)
 - mkdir 408 (time 1781249415.54 total 30.12 last 14.05)
 - mkdir 552 (time 1781249425.55 total 40.12 last 14.39)
 - mkdir 679 (time 1781249435.60 total 50.18 last 12.63)
 - mkdir 813 (time 1781249445.60 total 60.18 last 13.40)
 - mkdir 958 (time 1781249455.62 total 70.20 last 14.47)
total: 1000 mkdir in 74.94 seconds: 13.34 ops/second
 - mkdir 122 (time 1781249395.01 total 10.07 last 12.11)
 - mkdir 261 (time 1781249405.05 total 20.12 last 13.83)
 - mkdir 394 (time 1781249415.06 total 30.13 last 13.29)
 - mkdir 540 (time 1781249425.08 total 40.15 last 14.57)
 - mkdir 667 (time 1781249435.16 total 50.23 last 12.60)
 - mkdir 805 (time 1781249445.25 total 60.31 last 13.68)
 - mkdir 948 (time 1781249455.29 total 70.36 last 14.24)
total: 1000 mkdir in 75.52 seconds: 13.24 ops/second
 - mkdir 124 (time 1781249394.20 total 10.07 last 12.31)
 - mkdir 257 (time 1781249404.23 total 20.10 last 13.27)
 - mkdir 384 (time 1781249414.27 total 30.14 last 12.65)
 - mkdir 531 (time 1781249424.30 total 40.17 last 14.65)
 - mkdir 661 (time 1781249434.37 total 50.24 last 12.91)
 - mkdir 795 (time 1781249444.48 total 60.35 last 13.25)
 - mkdir 936 (time 1781249454.55 total 70.42 last 14.00)
total: 1000 mkdir in 76.70 seconds: 13.04 ops/second
Time for file creation 87 sec for 1 RPC sent at a time
 - unlinked 0 (time 1781249470 ; total 1 ; last 1)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1781249471 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1781249472 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1781249471 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1781249472 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1781249472 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1781249471 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1781249472 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1781249472 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1781249472 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
Time for file removal 52 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8