== sanity test 123ad: Verify batching statahead works correctly ========================================================== 07:56:19 (1741694179) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8800a8a3c000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8800a8a3c000.batch_stats=0 llite.lustre-ffff8800a8a3c000.statahead_max=32 llite.lustre-ffff8800a8a3c000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8800a8a3c000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8800a8a3c000.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 0.58 seconds: 172.03 ops/second llite.lustre-ffff8800a8a3c000.statahead_max=0 101 real 0m0.835s user 0m0.003s sys 0m0.308s ls -l 100 files without statahead: 1 sec llite.lustre-ffff8800a8a3c000.statahead_max=32 32 101 real 0m0.293s user 0m0.005s sys 0m0.122s ls -l 100 files with statahead: 1 sec statahead total: 52 statahead wrong: 1 agl total: 52 list_total: 20 fname_total: 32 hit_total: 6299 miss_total: 220 total: 900 open/close in 1.61 seconds: 560.37 ops/second llite.lustre-ffff8800a8a3c000.statahead_max=0 1001 real 0m6.157s user 0m0.027s sys 0m2.294s ls -l 1000 files without statahead: 6 sec llite.lustre-ffff8800a8a3c000.statahead_max=32 32 1001 real 0m1.662s user 0m0.018s sys 0m0.887s ls -l 1000 files with statahead: 1 sec statahead total: 53 statahead wrong: 1 agl total: 53 list_total: 21 fname_total: 32 hit_total: 7298 miss_total: 221 - open/close 3184 (time 1741694205.83 total 10.00 last 318.35) - open/close 8082 (time 1741694215.83 total 20.00 last 489.68) total: 9000 open/close in 22.22 seconds: 404.97 ops/second llite.lustre-ffff8800a8a3c000.statahead_max=0 10001 real 1m3.490s user 0m0.274s sys 0m23.018s ls -l 10000 files without statahead: 63 sec llite.lustre-ffff8800a8a3c000.statahead_max=32 32 10001 real 0m17.284s user 0m0.188s sys 0m9.138s ls -l 10000 files with statahead: 17 sec statahead total: 54 statahead wrong: 1 agl total: 54 list_total: 22 fname_total: 32 hit_total: 17297 miss_total: 222 ls -l done rm -r /mnt/lustre/d123ad.sanity: 52 seconds llite.lustre-ffff8800a8a3c000.statahead_stats= statahead total: 54 statahead wrong: 1 agl total: 54 list_total: 22 fname_total: 32 hit_total: 17297 miss_total: 222 mdc.lustre-MDT0000-mdc-ffff8800a8a3c000.batch_stats= snapshot_time: 1741694355.725412762 secs.nsecs start_time: 1741694180.769340423 secs.nsecs elapsed_time: 174.956072339 secs.nsecs subreqs per batch batches % cum % 1: 3 0 0 2: 2 0 1 4: 3 0 2 8: 3 0 3 16: 195 54 57 32: 150 42 100 mdc.lustre-MDT0001-mdc-ffff8800a8a3c000.batch_stats= snapshot_time: 1741694355.725556958 secs.nsecs start_time: 1741694180.769405841 secs.nsecs elapsed_time: 174.956151117 secs.nsecs subreqs per batch batches % cum % 1: 1 0 0 2: 0 0 0 4: 4 1 1 8: 4 1 2 16: 196 55 57 32: 149 42 100 - open/close 2613 (time 1741694366.55 total 10.00 last 261.30) - open/close 5367 (time 1741694376.55 total 20.00 last 275.35) - open/close 8274 (time 1741694386.55 total 30.00 last 290.69) total: 10000 open/close in 35.41 seconds: 282.42 ops/second llite.lustre-ffff8800a8a3c000.statahead_batch_max=0 llite.lustre-ffff8800a8a3c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8800a8a3c000.stats=clear mdc.lustre-MDT0001-mdc-ffff8800a8a3c000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m8.032s user 0m0.168s sys 0m7.355s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8800a8a3c000.statahead_batch_max=32 llite.lustre-ffff8800a8a3c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8800a8a3c000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8800a8a3c000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8800a8a3c000.stats=clear mdc.lustre-MDT0001-mdc-ffff8800a8a3c000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m30.361s user 0m0.234s sys 0m13.644s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8800a8a3c000.batch_stats= snapshot_time: 1741694434.639034952 secs.nsecs start_time: 1741694401.797666493 secs.nsecs elapsed_time: 32.841368459 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff8800a8a3c000.batch_stats= snapshot_time: 1741694434.639156708 secs.nsecs start_time: 1741694401.797778677 secs.nsecs elapsed_time: 32.841378031 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8800a8a3c000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8800a8a3c000.batch_stats=0 llite.lustre-ffff8800a8a3c000.statahead_max=2048 llite.lustre-ffff8800a8a3c000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8800a8a3c000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8800a8a3c000.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 0.92 seconds: 109.03 ops/second llite.lustre-ffff8800a8a3c000.statahead_max=0 101 real 0m0.986s user 0m0.003s sys 0m0.370s ls -l 100 files without statahead: 1 sec llite.lustre-ffff8800a8a3c000.statahead_max=2048 2048 101 real 0m0.321s user 0m0.004s sys 0m0.164s ls -l 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 2 total: 900 open/close in 3.73 seconds: 241.04 ops/second llite.lustre-ffff8800a8a3c000.statahead_max=0 1001 real 0m7.804s user 0m0.034s sys 0m2.813s ls -l 1000 files without statahead: 8 sec llite.lustre-ffff8800a8a3c000.statahead_max=2048 2048 1001 real 0m1.176s user 0m0.019s sys 0m0.738s ls -l 1000 files with statahead: 1 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 - open/close 2458 (time 1741694555.96 total 10.00 last 245.76) - open/close 5041 (time 1741694565.96 total 20.00 last 258.23) - open/close 7527 (time 1741694575.96 total 30.01 last 248.56) total: 9000 open/close in 36.08 seconds: 249.47 ops/second llite.lustre-ffff8800a8a3c000.statahead_max=0 10001 real 1m25.218s user 0m0.339s sys 0m29.221s ls -l 10000 files without statahead: 85 sec llite.lustre-ffff8800a8a3c000.statahead_max=2048 2048 10001 real 0m8.326s user 0m0.184s sys 0m7.119s ls -l 10000 files with statahead: 9 sec statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 21096 miss_total: 4 ls -l done rm -r /mnt/lustre/d123ad.sanity: 51 seconds llite.lustre-ffff8800a8a3c000.statahead_stats= statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 21096 miss_total: 4 mdc.lustre-MDT0000-mdc-ffff8800a8a3c000.batch_stats= snapshot_time: 1741694731.903241292 secs.nsecs start_time: 1741694435.235634243 secs.nsecs elapsed_time: 296.667607049 secs.nsecs subreqs per batch batches % cum % 1: 35 41 41 2: 2 2 44 4: 7 8 52 8: 6 7 59 16: 0 0 59 32: 0 0 59 64: 0 0 59 128: 13 15 75 256: 21 25 100 mdc.lustre-MDT0001-mdc-ffff8800a8a3c000.batch_stats= snapshot_time: 1741694731.903385712 secs.nsecs start_time: 1741694435.235711215 secs.nsecs elapsed_time: 296.667674497 secs.nsecs subreqs per batch batches % cum % 1: 63 54 54 2: 2 1 56 4: 5 4 60 8: 5 4 65 16: 3 2 67 32: 1 0 68 64: 1 0 69 128: 3 2 72 256: 32 27 100 - open/close 2439 (time 1741694742.87 total 10.00 last 243.88) - open/close 4770 (time 1741694752.88 total 20.00 last 233.01) - open/close 7210 (time 1741694762.88 total 30.01 last 243.93) - open/close 9689 (time 1741694772.88 total 40.01 last 247.83) total: 10000 open/close in 41.29 seconds: 242.20 ops/second llite.lustre-ffff8800a8a3c000.statahead_batch_max=0 llite.lustre-ffff8800a8a3c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8800a8a3c000.stats=clear mdc.lustre-MDT0001-mdc-ffff8800a8a3c000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m8.562s user 0m0.174s sys 0m7.588s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8800a8a3c000.statahead_batch_max=256 llite.lustre-ffff8800a8a3c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8800a8a3c000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8800a8a3c000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8800a8a3c000.stats=clear mdc.lustre-MDT0001-mdc-ffff8800a8a3c000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m8.825s user 0m0.185s sys 0m7.484s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8800a8a3c000.batch_stats= snapshot_time: 1741694796.127766200 secs.nsecs start_time: 1741694784.775023765 secs.nsecs elapsed_time: 11.352742435 secs.nsecs subreqs per batch batches % cum % 1: 175 62 62 2: 16 5 68 4: 13 4 73 8: 8 2 76 16: 13 4 80 32: 14 5 85 64: 3 1 87 128: 0 0 87 256: 36 12 100 mdc.lustre-MDT0001-mdc-ffff8800a8a3c000.batch_stats= snapshot_time: 1741694796.127971007 secs.nsecs start_time: 1741694784.775105609 secs.nsecs elapsed_time: 11.352865398 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8800a8a3c000.statahead_batch_max=64 llite.lustre-ffff8800a8a3c000.statahead_max=128