== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 07:38:29 (1787571509)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.67 seconds: 15.01 ops/second
 - mkdir 128 (time 1787571533.35 total 10.04 last 12.75)
 - mkdir 250 (time 1787571543.36 total 20.05 last 12.19)
 - mkdir 378 (time 1787571553.49 total 30.18 last 12.64)
 - mkdir 514 (time 1787571563.51 total 40.19 last 13.58)
 - mkdir 658 (time 1787571573.52 total 50.21 last 14.38)
 - mkdir 809 (time 1787571583.57 total 60.25 last 15.03)
 - mkdir 946 (time 1787571593.63 total 70.31 last 13.62)
total: 1000 mkdir in 74.14 seconds: 13.49 ops/second
 - mkdir 119 (time 1787571532.44 total 10.03 last 11.86)
 - mkdir 233 (time 1787571542.48 total 20.07 last 11.36)
 - mkdir 356 (time 1787571552.48 total 30.07 last 12.29)
 - mkdir 492 (time 1787571562.53 total 40.12 last 13.53)
 - mkdir 636 (time 1787571572.57 total 50.16 last 14.34)
 - mkdir 785 (time 1787571582.59 total 60.18 last 14.87)
 - mkdir 920 (time 1787571592.60 total 70.19 last 13.49)
total: 1000 mkdir in 76.12 seconds: 13.14 ops/second
 - mkdir 121 (time 1787571532.70 total 10.05 last 12.04)
 - mkdir 235 (time 1787571542.79 total 20.14 last 11.30)
 - mkdir 361 (time 1787571552.80 total 30.15 last 12.58)
 - mkdir 493 (time 1787571562.82 total 40.17 last 13.18)
 - mkdir 636 (time 1787571572.86 total 50.21 last 14.25)
 - mkdir 785 (time 1787571582.91 total 60.25 last 14.83)
 - mkdir 921 (time 1787571592.95 total 70.30 last 13.54)
total: 1000 mkdir in 75.92 seconds: 13.17 ops/second
 - mkdir 120 (time 1787571532.98 total 10.01 last 11.99)
 - mkdir 240 (time 1787571542.99 total 20.02 last 11.99)
 - mkdir 366 (time 1787571553.01 total 30.04 last 12.57)
 - mkdir 498 (time 1787571563.02 total 40.05 last 13.18)
 - mkdir 642 (time 1787571573.04 total 50.07 last 14.38)
 - mkdir 789 (time 1787571583.05 total 60.08 last 14.69)
 - mkdir 922 (time 1787571593.05 total 70.08 last 13.29)
total: 1000 mkdir in 75.76 seconds: 13.20 ops/second
 - mkdir 125 (time 1787571534.31 total 10.04 last 12.45)
 - mkdir 248 (time 1787571544.42 total 20.14 last 12.18)
 - mkdir 368 (time 1787571554.46 total 30.18 last 11.95)
 - mkdir 500 (time 1787571564.50 total 40.23 last 13.14)
 - mkdir 644 (time 1787571574.56 total 50.29 last 14.31)
 - mkdir 796 (time 1787571584.60 total 60.33 last 15.14)
 - mkdir 933 (time 1787571594.61 total 70.33 last 13.70)
total: 1000 mkdir in 75.66 seconds: 13.22 ops/second
 - mkdir 121 (time 1787571534.02 total 10.01 last 12.09)
 - mkdir 239 (time 1787571544.03 total 20.03 last 11.78)
 - mkdir 351 (time 1787571554.07 total 30.06 last 11.16)
 - mkdir 490 (time 1787571564.16 total 40.16 last 13.77)
 - mkdir 631 (time 1787571574.23 total 50.22 last 14.01)
 - mkdir 786 (time 1787571584.25 total 60.24 last 15.46)
 - mkdir 921 (time 1787571594.26 total 70.26 last 13.48)
total: 1000 mkdir in 77.04 seconds: 12.98 ops/second
 - mkdir 115 (time 1787571533.06 total 10.07 last 11.42)
 - mkdir 228 (time 1787571543.17 total 20.18 last 11.18)
 - mkdir 348 (time 1787571553.18 total 30.19 last 11.99)
 - mkdir 478 (time 1787571563.18 total 40.19 last 13.00)
 - mkdir 619 (time 1787571573.19 total 50.21 last 14.08)
 - mkdir 772 (time 1787571583.20 total 60.21 last 15.29)
 - mkdir 908 (time 1787571593.23 total 70.24 last 13.56)
total: 1000 mkdir in 78.30 seconds: 12.77 ops/second
 - mkdir 124 (time 1787571534.32 total 10.01 last 12.38)
 - mkdir 241 (time 1787571544.42 total 20.10 last 11.59)
 - mkdir 361 (time 1787571554.46 total 30.14 last 11.95)
 - mkdir 497 (time 1787571564.51 total 40.20 last 13.52)
 - mkdir 637 (time 1787571574.54 total 50.23 last 13.96)
 - mkdir 793 (time 1787571584.60 total 60.29 last 15.50)
 - mkdir 925 (time 1787571594.60 total 70.29 last 13.20)
total: 1000 mkdir in 77.03 seconds: 12.98 ops/second
 - mkdir 122 (time 1787571533.97 total 10.05 last 12.14)
 - mkdir 234 (time 1787571543.99 total 20.08 last 11.17)
 - mkdir 351 (time 1787571554.05 total 30.14 last 11.63)
 - mkdir 483 (time 1787571564.06 total 40.14 last 13.19)
 - mkdir 624 (time 1787571574.10 total 50.18 last 14.04)
 - mkdir 774 (time 1787571584.14 total 60.22 last 14.93)
 - mkdir 906 (time 1787571594.20 total 70.28 last 13.12)
total: 1000 mkdir in 78.77 seconds: 12.70 ops/second
 - mkdir 114 (time 1787571534.20 total 10.00 last 11.40)
 - mkdir 225 (time 1787571544.25 total 20.06 last 11.04)
 - mkdir 335 (time 1787571554.28 total 30.09 last 10.97)
 - mkdir 472 (time 1787571564.31 total 40.12 last 13.66)
 - mkdir 615 (time 1787571574.39 total 50.19 last 14.19)
 - mkdir 768 (time 1787571584.41 total 60.22 last 15.26)
 - mkdir 896 (time 1787571594.44 total 70.24 last 12.77)
total: 1000 mkdir in 80.08 seconds: 12.49 ops/second
Time for file creation 93 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 14 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.59 seconds: 16.81 ops/second
 - mkdir 145 (time 1787572196.00 total 10.00 last 14.49)
 - mkdir 296 (time 1787572206.04 total 20.05 last 15.03)
 - mkdir 452 (time 1787572216.04 total 30.05 last 15.60)
 - mkdir 619 (time 1787572226.05 total 40.06 last 16.69)
 - mkdir 788 (time 1787572236.11 total 50.12 last 16.79)
 - mkdir 989 (time 1787572246.12 total 60.13 last 20.09)
total: 1000 mkdir in 60.54 seconds: 16.52 ops/second
 - mkdir 133 (time 1787572193.97 total 10.03 last 13.26)
 - mkdir 287 (time 1787572203.97 total 20.03 last 15.40)
 - mkdir 431 (time 1787572213.98 total 30.04 last 14.39)
 - mkdir 593 (time 1787572224.05 total 40.11 last 16.08)
 - mkdir 765 (time 1787572234.07 total 50.13 last 17.17)
 - mkdir 939 (time 1787572244.12 total 60.18 last 17.32)
total: 1000 mkdir in 62.64 seconds: 15.96 ops/second
 - mkdir 145 (time 1787572195.99 total 10.01 last 14.49)
 - mkdir 300 (time 1787572206.03 total 20.05 last 15.43)
 - mkdir 453 (time 1787572216.03 total 30.05 last 15.30)
 - mkdir 620 (time 1787572226.05 total 40.07 last 16.67)
 - mkdir 784 (time 1787572236.06 total 50.08 last 16.39)
 - mkdir 983 (time 1787572246.06 total 60.08 last 19.88)
total: 1000 mkdir in 60.89 seconds: 16.42 ops/second
 - mkdir 141 (time 1787572195.42 total 10.01 last 14.08)
 - mkdir 293 (time 1787572205.42 total 20.02 last 15.19)
 - mkdir 450 (time 1787572215.46 total 30.05 last 15.64)
 - mkdir 608 (time 1787572225.57 total 40.16 last 15.63)
 - mkdir 778 (time 1787572235.61 total 50.20 last 16.93)
 - mkdir 967 (time 1787572245.62 total 60.22 last 18.87)
total: 1000 mkdir in 61.61 seconds: 16.23 ops/second
 - mkdir 129 (time 1787572193.90 total 10.03 last 12.87)
 - mkdir 280 (time 1787572203.90 total 20.03 last 15.09)
 - mkdir 431 (time 1787572213.97 total 30.10 last 15.00)
 - mkdir 587 (time 1787572224.04 total 40.17 last 15.50)
 - mkdir 756 (time 1787572234.06 total 50.19 last 16.86)
 - mkdir 929 (time 1787572244.14 total 60.26 last 17.17)
total: 1000 mkdir in 63.21 seconds: 15.82 ops/second
 - mkdir 140 (time 1787572195.82 total 10.02 last 13.97)
 - mkdir 286 (time 1787572205.88 total 20.08 last 14.52)
 - mkdir 435 (time 1787572215.89 total 30.09 last 14.89)
 - mkdir 599 (time 1787572225.93 total 40.13 last 16.33)
 - mkdir 769 (time 1787572236.02 total 50.21 last 16.86)
 - mkdir 969 (time 1787572246.03 total 60.23 last 19.97)
total: 1000 mkdir in 61.92 seconds: 16.15 ops/second
 - mkdir 145 (time 1787572195.89 total 10.02 last 14.48)
 - mkdir 292 (time 1787572205.95 total 20.07 last 14.61)
 - mkdir 440 (time 1787572216.04 total 30.16 last 14.67)
 - mkdir 607 (time 1787572226.06 total 40.18 last 16.66)
 - mkdir 773 (time 1787572236.09 total 50.21 last 16.55)
 - mkdir 968 (time 1787572246.10 total 60.22 last 19.48)
total: 1000 mkdir in 62.08 seconds: 16.11 ops/second
 - mkdir 147 (time 1787572196.06 total 10.00 last 14.70)
 - mkdir 291 (time 1787572206.07 total 20.01 last 14.39)
 - mkdir 443 (time 1787572216.08 total 30.02 last 15.18)
 - mkdir 609 (time 1787572226.13 total 40.07 last 16.53)
 - mkdir 769 (time 1787572236.13 total 50.07 last 16.00)
 - mkdir 964 (time 1787572246.14 total 60.09 last 19.47)
total: 1000 mkdir in 62.53 seconds: 15.99 ops/second
 - mkdir 136 (time 1787572194.57 total 10.05 last 13.54)
 - mkdir 281 (time 1787572204.63 total 20.11 last 14.40)
 - mkdir 424 (time 1787572214.68 total 30.16 last 14.24)
 - mkdir 576 (time 1787572224.69 total 40.17 last 15.19)
 - mkdir 743 (time 1787572234.74 total 50.22 last 16.61)
 - mkdir 922 (time 1787572244.81 total 60.28 last 17.79)
total: 1000 mkdir in 64.47 seconds: 15.51 ops/second
 - mkdir 128 (time 1787572195.52 total 10.01 last 12.79)
 - mkdir 274 (time 1787572205.54 total 20.03 last 14.57)
 - mkdir 416 (time 1787572215.67 total 30.15 last 14.02)
 - mkdir 566 (time 1787572225.70 total 40.19 last 14.95)
 - mkdir 723 (time 1787572235.72 total 50.21 last 15.67)
 - mkdir 909 (time 1787572245.73 total 60.22 last 18.59)
total: 1000 mkdir in 65.97 seconds: 15.16 ops/second
Time for file creation 76 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787572258 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1787572258 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1787572259 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1787572259 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1787572260 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1787572259 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1787572260 ; total 1 ; last 1)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1787572259 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1787572260 ; total 1 ; last 1)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1787572260 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
Time for file removal 40 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8