== sanity test 123ad: Verify batching statahead works correctly ========================================================== 03:42:09 (1778658129) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9d768937c000.batch_stats=0 llite.lustre-ffff9d768937c000.statahead_max=32 llite.lustre-ffff9d768937c000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9d768937c000.batch_stats=0 total: 100 open/close in 3.33 seconds: 29.99 ops/second llite.lustre-ffff9d768937c000.statahead_max=0 101 real 0m2.884s user 0m0.004s sys 0m0.381s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9d768937c000.statahead_max=32 llite.lustre-ffff9d768937c000.statahead_max=32 101 real 0m1.182s user 0m0.008s sys 0m0.186s 'ls -l' 100 files with statahead: 1 sec statahead total: 152 statahead wrong: 2 agl total: 145 list_total: 30 fname_total: 106 hit_total: 3762 miss_total: 1450 swrong=2 ewrong=2 - open/close 566 (time 1778658160.33 total 10.02 last 56.51) total: 900 open/close in 15.59 seconds: 57.74 ops/second llite.lustre-ffff9d768937c000.statahead_max=0 1001 real 0m26.924s user 0m0.035s sys 0m3.460s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff9d768937c000.statahead_max=32 llite.lustre-ffff9d768937c000.statahead_max=32 1001 real 0m11.974s user 0m0.034s sys 0m1.673s 'ls -l' 1000 files with statahead: 12 sec statahead total: 153 statahead wrong: 2 agl total: 146 list_total: 31 fname_total: 106 hit_total: 4760 miss_total: 1452 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff9d768937c000.statahead_stats= statahead total: 153 statahead wrong: 2 agl total: 146 list_total: 31 fname_total: 106 hit_total: 4760 miss_total: 1452 mdc.lustre-MDT0000-mdc-ffff9d768937c000.batch_stats= snapshot_time: 1778658243.944709667 secs.nsecs start_time: 1778658135.269366609 secs.nsecs elapsed_time: 108.675343058 secs.nsecs subreqs per batch batches % cum % 1: 3 7 7 2: 0 0 7 4: 0 0 7 8: 2 4 12 16: 2 4 17 32: 34 82 100 - open/close 575 (time 1778658257.10 total 10.02 last 57.39) - open/close 1135 (time 1778658267.11 total 20.03 last 55.93) - open/close 1678 (time 1778658277.12 total 30.04 last 54.27) - open/close 2235 (time 1778658287.13 total 40.05 last 55.64) - open/close 2791 (time 1778658297.13 total 50.05 last 55.60) - open/close 3343 (time 1778658307.15 total 60.07 last 55.08) - open/close 3882 (time 1778658317.17 total 70.09 last 53.81) - open/close 4428 (time 1778658327.18 total 80.10 last 54.51) - open/close 4901 (time 1778658337.20 total 90.12 last 47.23) - open/close 5454 (time 1778658347.21 total 100.13 last 55.23) - open/close 6004 (time 1778658357.21 total 110.13 last 54.97) - open/close 6591 (time 1778658367.23 total 120.15 last 58.58) - open/close 7205 (time 1778658377.24 total 130.16 last 61.34) - open/close 7742 (time 1778658387.25 total 140.18 last 53.63) - open/close 8258 (time 1778658397.26 total 150.18 last 51.57) - open/close 8811 (time 1778658407.28 total 160.20 last 55.21) - open/close 9294 (time 1778658417.29 total 170.21 last 48.24) - open/close 9782 (time 1778658427.30 total 180.22 last 48.74) total: 10000 open/close in 184.63 seconds: 54.16 ops/second llite.lustre-ffff9d768937c000.statahead_batch_max=0 llite.lustre-ffff9d768937c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d768937c000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m18.097s user 0m0.327s sys 0m16.895s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9d768937c000.statahead_batch_max=32 llite.lustre-ffff9d768937c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d768937c000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9d768937c000.stats=clear ls -l files in d123ad.sanity: 10001 real 2m0.708s user 0m0.365s sys 0m16.301s batched statahead RPC= hit count=9980 mdc.lustre-MDT0000-mdc-ffff9d768937c000.batch_stats= snapshot_time: 1778658655.212315072 secs.nsecs start_time: 1778658519.281527887 secs.nsecs elapsed_time: 135.930787185 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 0 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9d768937c000.batch_stats=0 llite.lustre-ffff9d768937c000.statahead_max=2048 llite.lustre-ffff9d768937c000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9d768937c000.batch_stats=0 total: 100 open/close in 3.61 seconds: 27.73 ops/second llite.lustre-ffff9d768937c000.statahead_max=0 101 real 0m2.208s user 0m0.006s sys 0m0.318s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9d768937c000.statahead_max=2048 llite.lustre-ffff9d768937c000.statahead_max=2048 101 real 0m1.067s user 0m0.005s sys 0m0.214s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10079 miss_total: 21 swrong=0 ewrong=0 - open/close 635 (time 1778659050.77 total 10.03 last 63.33) total: 900 open/close in 14.60 seconds: 61.65 ops/second llite.lustre-ffff9d768937c000.statahead_max=0 1001 real 0m26.031s user 0m0.048s sys 0m3.412s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff9d768937c000.statahead_max=2048 llite.lustre-ffff9d768937c000.statahead_max=2048 1001 real 0m7.485s user 0m0.035s sys 0m1.498s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 28 seconds llite.lustre-ffff9d768937c000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 mdc.lustre-MDT0000-mdc-ffff9d768937c000.batch_stats= snapshot_time: 1778659131.009629359 secs.nsecs start_time: 1778658657.922595316 secs.nsecs elapsed_time: 473.087034043 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 2 16 50 64: 1 8 58 128: 1 8 66 256: 4 33 100 - open/close 519 (time 1778659143.79 total 10.03 last 51.77) - open/close 1035 (time 1778659153.80 total 20.03 last 51.56) - open/close 1540 (time 1778659163.81 total 30.05 last 50.42) - open/close 2113 (time 1778659173.83 total 40.07 last 57.17) - open/close 2667 (time 1778659183.84 total 50.08 last 55.38) - open/close 3210 (time 1778659193.85 total 60.09 last 54.25) - open/close 3747 (time 1778659203.85 total 70.09 last 53.67) - open/close 4319 (time 1778659213.87 total 80.10 last 57.12) - open/close 4853 (time 1778659223.88 total 90.11 last 53.35) - open/close 5405 (time 1778659233.89 total 100.12 last 55.15) - open/close 5954 (time 1778659243.92 total 110.15 last 54.74) - open/close 6493 (time 1778659253.92 total 120.16 last 53.87) - open/close 7049 (time 1778659263.93 total 130.17 last 55.53) - open/close 7626 (time 1778659273.95 total 140.19 last 57.61) - open/close 8071 (time 1778659283.98 total 150.21 last 44.39) - open/close 8440 (time 1778659294.00 total 160.24 last 36.80) - open/close 9002 (time 1778659304.01 total 170.25 last 56.15) - open/close 9588 (time 1778659314.02 total 180.26 last 58.55) total: 10000 open/close in 187.37 seconds: 53.37 ops/second llite.lustre-ffff9d768937c000.statahead_batch_max=0 llite.lustre-ffff9d768937c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d768937c000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m12.885s user 0m0.395s sys 0m17.001s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9d768937c000.statahead_batch_max=256 llite.lustre-ffff9d768937c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d768937c000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9d768937c000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.354s user 0m0.289s sys 0m15.693s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9d768937c000.batch_stats= snapshot_time: 1778659487.516653413 secs.nsecs start_time: 1778659407.543500484 secs.nsecs elapsed_time: 79.973152929 secs.nsecs subreqs per batch batches % cum % 1: 178 52 52 2: 45 13 65 4: 36 10 75 8: 17 4 80 16: 13 3 84 32: 17 4 89 64: 0 0 89 128: 1 0 89 256: 35 10 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9d768937c000.statahead_batch_max=64 llite.lustre-ffff9d768937c000.statahead_max=128