== sanity test 123ad: Verify batching statahead works correctly ========================================================== 06:49:09 (1768132149) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff90ab51f2f800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff90ab51f2f800.batch_stats=0 llite.lustre-ffff90ab51f2f800.statahead_max=32 llite.lustre-ffff90ab51f2f800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff90ab51f2f800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff90ab51f2f800.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 0.74 seconds: 136.04 ops/second llite.lustre-ffff90ab51f2f800.statahead_max=0 101 real 0m0.662s user 0m0.002s sys 0m0.228s ls -l 100 files without statahead: 1 sec llite.lustre-ffff90ab51f2f800.statahead_max=32 32 101 real 0m0.353s user 0m0.003s sys 0m0.187s ls -l 100 files with statahead: 1 sec statahead total: 82 statahead wrong: 1 agl total: 77 list_total: 30 fname_total: 37 hit_total: 33816 miss_total: 218 total: 900 open/close in 2.31 seconds: 389.28 ops/second llite.lustre-ffff90ab51f2f800.statahead_max=0 1001 real 0m9.505s user 0m0.024s sys 0m2.921s ls -l 1000 files without statahead: 9 sec llite.lustre-ffff90ab51f2f800.statahead_max=32 32 1001 real 0m5.351s user 0m0.017s sys 0m1.889s ls -l 1000 files with statahead: 6 sec statahead total: 83 statahead wrong: 1 agl total: 78 list_total: 31 fname_total: 37 hit_total: 34815 miss_total: 219 - open/close 2107 (time 1768132187.15 total 10.00 last 210.67) total: 4000 open/close in 18.80 seconds: 212.72 ops/second llite.lustre-ffff90ab51f2f800.statahead_max=0 5001 real 0m49.265s user 0m0.116s sys 0m15.628s ls -l 5000 files without statahead: 49 sec llite.lustre-ffff90ab51f2f800.statahead_max=32 32 5001 real 0m24.068s user 0m0.057s sys 0m9.442s ls -l 5000 files with statahead: 24 sec statahead total: 84 statahead wrong: 1 agl total: 79 list_total: 32 fname_total: 37 hit_total: 39814 miss_total: 220 ls -l done rm -r /mnt/lustre/d123ad.sanity: 39 seconds llite.lustre-ffff90ab51f2f800.statahead_stats= statahead total: 84 statahead wrong: 1 agl total: 79 list_total: 32 fname_total: 37 hit_total: 39814 miss_total: 220 mdc.lustre-MDT0000-mdc-ffff90ab51f2f800.batch_stats= snapshot_time: 1768132317.789410436 secs.nsecs start_time: 1768132151.518641893 secs.nsecs elapsed_time: 166.270768543 secs.nsecs subreqs per batch batches % cum % 1: 1 0 0 2: 3 1 2 4: 4 2 4 8: 1 0 4 16: 130 66 70 32: 57 29 100 mdc.lustre-MDT0001-mdc-ffff90ab51f2f800.batch_stats= snapshot_time: 1768132317.789452421 secs.nsecs start_time: 1768132151.518704607 secs.nsecs elapsed_time: 166.270747814 secs.nsecs subreqs per batch batches % cum % 1: 1 0 0 2: 1 0 1 4: 1 0 1 8: 6 3 4 16: 81 40 45 32: 108 54 100 - open/close 2375 (time 1768132329.17 total 10.00 last 237.40) - open/close 4604 (time 1768132339.17 total 20.00 last 222.88) - open/close 7177 (time 1768132349.18 total 30.01 last 257.21) - open/close 9321 (time 1768132359.18 total 40.01 last 214.38) total: 10000 open/close in 43.22 seconds: 231.35 ops/second llite.lustre-ffff90ab51f2f800.statahead_batch_max=0 llite.lustre-ffff90ab51f2f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff90ab51f2f800.stats=clear mdc.lustre-MDT0001-mdc-ffff90ab51f2f800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m49.421s user 0m0.144s sys 0m46.363s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff90ab51f2f800.statahead_batch_max=32 llite.lustre-ffff90ab51f2f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff90ab51f2f800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff90ab51f2f800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff90ab51f2f800.stats=clear mdc.lustre-MDT0001-mdc-ffff90ab51f2f800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m45.664s user 0m0.133s sys 0m24.453s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff90ab51f2f800.batch_stats= snapshot_time: 1768132465.332922864 secs.nsecs start_time: 1768132415.046040039 secs.nsecs elapsed_time: 50.286882825 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff90ab51f2f800.batch_stats= snapshot_time: 1768132465.332987794 secs.nsecs start_time: 1768132415.046144842 secs.nsecs elapsed_time: 50.286842952 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff90ab51f2f800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff90ab51f2f800.batch_stats=0 llite.lustre-ffff90ab51f2f800.statahead_max=2048 llite.lustre-ffff90ab51f2f800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff90ab51f2f800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff90ab51f2f800.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 1.12 seconds: 89.32 ops/second llite.lustre-ffff90ab51f2f800.statahead_max=0 101 real 0m0.981s user 0m0.005s sys 0m0.314s ls -l 100 files without statahead: 1 sec llite.lustre-ffff90ab51f2f800.statahead_max=2048 2048 101 real 0m0.623s user 0m0.003s sys 0m0.220s ls -l 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 2 total: 900 open/close in 4.22 seconds: 213.06 ops/second llite.lustre-ffff90ab51f2f800.statahead_max=0 1001 real 0m10.802s user 0m0.022s sys 0m3.484s ls -l 1000 files without statahead: 11 sec llite.lustre-ffff90ab51f2f800.statahead_max=2048 2048 1001 real 0m4.889s user 0m0.018s sys 0m1.498s ls -l 1000 files with statahead: 5 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 - open/close 2139 (time 1768132641.32 total 10.00 last 213.81) total: 3000 open/close in 13.86 seconds: 216.46 ops/second llite.lustre-ffff90ab51f2f800.statahead_max=0 4001 real 0m41.691s user 0m0.085s sys 0m13.606s ls -l 4000 files without statahead: 41 sec llite.lustre-ffff90ab51f2f800.statahead_max=2048 2048 4001 real 0m13.271s user 0m0.046s sys 0m8.079s ls -l 4000 files with statahead: 13 sec statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 15096 miss_total: 4 ls -l done rm -r /mnt/lustre/d123ad.sanity: 38 seconds llite.lustre-ffff90ab51f2f800.statahead_stats= statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 15096 miss_total: 4 mdc.lustre-MDT0000-mdc-ffff90ab51f2f800.batch_stats= snapshot_time: 1768132747.326146383 secs.nsecs start_time: 1768132466.546718960 secs.nsecs elapsed_time: 280.779427423 secs.nsecs subreqs per batch batches % cum % 1: 105 74 74 2: 16 11 85 4: 3 2 87 8: 4 2 90 16: 0 0 90 32: 0 0 90 64: 1 0 91 128: 2 1 92 256: 10 7 100 mdc.lustre-MDT0001-mdc-ffff90ab51f2f800.batch_stats= snapshot_time: 1768132747.326225929 secs.nsecs start_time: 1768132466.546822248 secs.nsecs elapsed_time: 280.779403681 secs.nsecs subreqs per batch batches % cum % 1: 91 72 72 2: 15 11 84 4: 3 2 86 8: 3 2 88 16: 1 0 89 32: 0 0 89 64: 1 0 90 128: 2 1 92 256: 10 7 100 - open/close 2015 (time 1768132758.67 total 10.00 last 201.45) - open/close 4087 (time 1768132768.67 total 20.01 last 207.12) - open/close 6094 (time 1768132778.68 total 30.01 last 200.68) - open/close 8167 (time 1768132788.68 total 40.01 last 207.24) total: 10000 open/close in 48.55 seconds: 205.97 ops/second llite.lustre-ffff90ab51f2f800.statahead_batch_max=0 llite.lustre-ffff90ab51f2f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff90ab51f2f800.stats=clear mdc.lustre-MDT0001-mdc-ffff90ab51f2f800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m45.420s user 0m0.141s sys 0m37.811s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff90ab51f2f800.statahead_batch_max=256 llite.lustre-ffff90ab51f2f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff90ab51f2f800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff90ab51f2f800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff90ab51f2f800.stats=clear mdc.lustre-MDT0001-mdc-ffff90ab51f2f800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m37.231s user 0m0.147s sys 0m25.999s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff90ab51f2f800.batch_stats= snapshot_time: 1768132890.327710901 secs.nsecs start_time: 1768132845.896727119 secs.nsecs elapsed_time: 44.430983782 secs.nsecs subreqs per batch batches % cum % 1: 333 71 71 2: 57 12 83 4: 14 2 86 8: 2 0 86 16: 1 0 86 32: 25 5 92 64: 1 0 92 128: 1 0 92 256: 34 7 100 mdc.lustre-MDT0001-mdc-ffff90ab51f2f800.batch_stats= snapshot_time: 1768132890.327779894 secs.nsecs start_time: 1768132845.896839224 secs.nsecs elapsed_time: 44.430940670 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff90ab51f2f800.statahead_batch_max=64 llite.lustre-ffff90ab51f2f800.statahead_max=128