== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 07:17:32 (1789039052)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.49 seconds: 20.45 ops/second
 - mkdir 238 (time 1789039072.93 total 10.02 last 23.75)
 - mkdir 439 (time 1789039082.94 total 20.03 last 20.08)
 - mkdir 596 (time 1789039092.96 total 30.05 last 15.67)
 - mkdir 772 (time 1789039103.03 total 40.12 last 17.48)
 - mkdir 944 (time 1789039113.12 total 50.22 last 17.03)
total: 1000 mkdir in 53.88 seconds: 18.56 ops/second
 - mkdir 233 (time 1789039073.85 total 10.00 last 23.30)
 - mkdir 434 (time 1789039083.87 total 20.02 last 20.07)
 - mkdir 591 (time 1789039093.90 total 30.05 last 15.65)
 - mkdir 781 (time 1789039103.91 total 40.06 last 18.98)
 - mkdir 947 (time 1789039114.00 total 50.15 last 16.45)
total: 1000 mkdir in 53.86 seconds: 18.57 ops/second
 - mkdir 232 (time 1789039073.53 total 10.03 last 23.13)
 - mkdir 432 (time 1789039083.57 total 20.08 last 19.91)
 - mkdir 588 (time 1789039093.69 total 30.20 last 15.42)
 - mkdir 768 (time 1789039103.76 total 40.26 last 17.89)
 - mkdir 933 (time 1789039113.78 total 50.29 last 16.45)
total: 1000 mkdir in 55.27 seconds: 18.09 ops/second
 - mkdir 228 (time 1789039073.37 total 10.05 last 22.70)
 - mkdir 427 (time 1789039083.44 total 20.11 last 19.77)
 - mkdir 583 (time 1789039093.55 total 30.22 last 15.43)
 - mkdir 763 (time 1789039103.56 total 40.23 last 17.98)
 - mkdir 929 (time 1789039113.59 total 50.27 last 16.54)
total: 1000 mkdir in 55.54 seconds: 18.00 ops/second
 - mkdir 230 (time 1789039073.96 total 10.02 last 22.95)
 - mkdir 434 (time 1789039083.99 total 20.05 last 20.35)
 - mkdir 590 (time 1789039094.04 total 30.09 last 15.52)
 - mkdir 774 (time 1789039104.06 total 40.12 last 18.36)
 - mkdir 937 (time 1789039114.12 total 50.18 last 16.20)
total: 1000 mkdir in 55.00 seconds: 18.18 ops/second
 - mkdir 236 (time 1789039074.03 total 10.02 last 23.56)
 - mkdir 426 (time 1789039084.04 total 20.02 last 18.99)
 - mkdir 581 (time 1789039094.06 total 30.04 last 15.47)
 - mkdir 769 (time 1789039104.08 total 40.07 last 18.76)
 - mkdir 932 (time 1789039114.11 total 50.10 last 16.25)
total: 1000 mkdir in 55.27 seconds: 18.09 ops/second
 - mkdir 234 (time 1789039074.13 total 10.00 last 23.39)
 - mkdir 424 (time 1789039084.17 total 20.05 last 18.91)
 - mkdir 579 (time 1789039094.24 total 30.12 last 15.40)
 - mkdir 770 (time 1789039104.26 total 40.14 last 19.06)
 - mkdir 924 (time 1789039114.28 total 50.16 last 15.37)
total: 1000 mkdir in 56.21 seconds: 17.79 ops/second
 - mkdir 229 (time 1789039073.80 total 10.03 last 22.83)
 - mkdir 421 (time 1789039083.84 total 20.07 last 19.13)
 - mkdir 576 (time 1789039093.90 total 30.13 last 15.41)
 - mkdir 761 (time 1789039103.91 total 40.13 last 18.49)
 - mkdir 923 (time 1789039113.91 total 50.14 last 16.19)
total: 1000 mkdir in 56.62 seconds: 17.66 ops/second
 - mkdir 224 (time 1789039073.63 total 10.03 last 22.34)
 - mkdir 423 (time 1789039083.69 total 20.09 last 19.78)
 - mkdir 573 (time 1789039093.73 total 30.12 last 14.95)
 - mkdir 750 (time 1789039103.80 total 40.20 last 17.57)
 - mkdir 916 (time 1789039113.86 total 50.26 last 16.50)
total: 1000 mkdir in 57.27 seconds: 17.46 ops/second
 - mkdir 228 (time 1789039074.21 total 10.00 last 22.79)
 - mkdir 416 (time 1789039084.23 total 20.03 last 18.76)
 - mkdir 567 (time 1789039094.29 total 30.08 last 15.02)
 - mkdir 757 (time 1789039104.31 total 40.10 last 18.97)
 - mkdir 916 (time 1789039114.34 total 50.13 last 15.84)
total: 1000 mkdir in 57.38 seconds: 17.43 ops/second
Time for file creation 68 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 18 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.45 seconds: 22.25 ops/second
 - mkdir 142 (time 1789039679.62 total 10.03 last 14.15)
 - mkdir 276 (time 1789039689.70 total 20.11 last 13.29)
 - mkdir 403 (time 1789039699.71 total 30.12 last 12.69)
 - mkdir 570 (time 1789039709.73 total 40.15 last 16.66)
 - mkdir 721 (time 1789039719.75 total 50.16 last 15.08)
 - mkdir 884 (time 1789039729.80 total 60.22 last 16.21)
total: 1000 mkdir in 67.20 seconds: 14.88 ops/second
 - mkdir 136 (time 1789039679.77 total 10.00 last 13.59)
 - mkdir 262 (time 1789039689.83 total 20.06 last 12.53)
 - mkdir 394 (time 1789039699.85 total 30.08 last 13.18)
 - mkdir 559 (time 1789039709.88 total 40.10 last 16.46)
 - mkdir 710 (time 1789039719.90 total 50.13 last 15.06)
 - mkdir 870 (time 1789039729.91 total 60.14 last 16.00)
total: 1000 mkdir in 68.59 seconds: 14.58 ops/second
 - mkdir 141 (time 1789039679.43 total 10.10 last 13.97)
 - mkdir 270 (time 1789039689.46 total 20.12 last 12.86)
 - mkdir 401 (time 1789039699.49 total 30.16 last 13.06)
 - mkdir 566 (time 1789039709.52 total 40.19 last 16.45)
 - mkdir 712 (time 1789039719.65 total 50.32 last 14.41)
 - mkdir 869 (time 1789039729.70 total 60.37 last 15.62)
total: 1000 mkdir in 69.42 seconds: 14.41 ops/second
 - mkdir 140 (time 1789039679.99 total 10.15 last 13.80)
 - mkdir 272 (time 1789039690.02 total 20.18 last 13.16)
 - mkdir 406 (time 1789039700.10 total 30.26 last 13.30)
 - mkdir 570 (time 1789039710.12 total 40.28 last 16.36)
 - mkdir 717 (time 1789039720.21 total 50.37 last 14.57)
 - mkdir 868 (time 1789039730.21 total 60.37 last 15.10)
total: 1000 mkdir in 70.00 seconds: 14.29 ops/second
 - mkdir 128 (time 1789039678.07 total 10.01 last 12.79)
 - mkdir 255 (time 1789039688.11 total 20.04 last 12.66)
 - mkdir 374 (time 1789039698.23 total 30.16 last 11.76)
 - mkdir 543 (time 1789039708.25 total 40.18 last 16.87)
 - mkdir 687 (time 1789039718.28 total 50.21 last 14.36)
 - mkdir 847 (time 1789039728.36 total 60.29 last 15.88)
 - mkdir 986 (time 1789039738.49 total 70.42 last 13.72)
total: 1000 mkdir in 72.08 seconds: 13.87 ops/second
 - mkdir 137 (time 1789039679.80 total 10.10 last 13.56)
 - mkdir 263 (time 1789039689.81 total 20.12 last 12.58)
 - mkdir 390 (time 1789039699.83 total 30.14 last 12.68)
 - mkdir 552 (time 1789039709.93 total 40.23 last 16.05)
 - mkdir 696 (time 1789039719.97 total 50.27 last 14.34)
 - mkdir 853 (time 1789039730.01 total 60.31 last 15.64)
 - mkdir 990 (time 1789039740.09 total 70.40 last 13.58)
total: 1000 mkdir in 71.90 seconds: 13.91 ops/second
 - mkdir 136 (time 1789039679.99 total 10.04 last 13.55)
 - mkdir 258 (time 1789039689.99 total 20.04 last 12.19)
 - mkdir 389 (time 1789039700.06 total 30.11 last 13.01)
 - mkdir 552 (time 1789039710.11 total 40.16 last 16.22)
 - mkdir 699 (time 1789039720.11 total 50.16 last 14.70)
 - mkdir 853 (time 1789039730.16 total 60.21 last 15.33)
 - mkdir 990 (time 1789039740.18 total 70.23 last 13.67)
total: 1000 mkdir in 71.77 seconds: 13.93 ops/second
 - mkdir 138 (time 1789039680.58 total 10.05 last 13.74)
 - mkdir 268 (time 1789039690.68 total 20.14 last 12.88)
 - mkdir 399 (time 1789039700.73 total 30.19 last 13.03)
 - mkdir 567 (time 1789039710.80 total 40.27 last 16.68)
 - mkdir 711 (time 1789039720.87 total 50.33 last 14.30)
 - mkdir 862 (time 1789039730.92 total 60.38 last 15.03)
 - mkdir 994 (time 1789039740.97 total 70.43 last 13.13)
total: 1000 mkdir in 71.27 seconds: 14.03 ops/second
 - mkdir 131 (time 1789039679.94 total 10.04 last 13.05)
 - mkdir 253 (time 1789039689.96 total 20.06 last 12.18)
 - mkdir 381 (time 1789039699.96 total 30.06 last 12.79)
 - mkdir 545 (time 1789039710.03 total 40.13 last 16.29)
 - mkdir 688 (time 1789039720.04 total 50.14 last 14.29)
 - mkdir 838 (time 1789039730.10 total 60.20 last 14.91)
 - mkdir 975 (time 1789039740.25 total 70.35 last 13.50)
total: 1000 mkdir in 73.43 seconds: 13.62 ops/second
 - mkdir 128 (time 1789039678.92 total 10.10 last 12.68)
 - mkdir 247 (time 1789039688.94 total 20.12 last 11.87)
 - mkdir 366 (time 1789039698.96 total 30.15 last 11.87)
 - mkdir 524 (time 1789039708.97 total 40.15 last 15.80)
 - mkdir 662 (time 1789039719.03 total 50.22 last 13.71)
 - mkdir 811 (time 1789039729.08 total 60.26 last 14.84)
 - mkdir 948 (time 1789039739.09 total 70.27 last 13.69)
total: 1000 mkdir in 75.87 seconds: 13.18 ops/second
Time for file creation 87 sec for 1 RPC sent at a time
 - unlinked 0 (time 1789039755 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1789039755 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1789039757 ; total 1 ; last 1)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1789039757 ; total 1 ; last 1)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1789039757 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1789039758 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1789039757 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1789039757 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1789039757 ; total 1 ; last 1)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1789039758 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
Time for file removal 49 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8