== sanity test 123ad: Verify batching statahead works correctly ========================================================== 23:20:06 (1775877606) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8a9108e6d000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8a9108e6d000.batch_stats=0 llite.lustre-ffff8a9108e6d000.statahead_max=32 llite.lustre-ffff8a9108e6d000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8a9108e6d000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8a9108e6d000.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 0.71 seconds: 140.33 ops/second llite.lustre-ffff8a9108e6d000.statahead_max=0 101 real 0m0.686s user 0m0.002s sys 0m0.262s ls -l 100 files without statahead: 1 sec llite.lustre-ffff8a9108e6d000.statahead_max=32 32 101 real 0m0.434s user 0m0.004s sys 0m0.230s ls -l 100 files with statahead: 1 sec statahead total: 110 statahead wrong: 2 agl total: 105 list_total: 33 fname_total: 77 hit_total: 21469 miss_total: 1273 total: 900 open/close in 1.79 seconds: 504.07 ops/second llite.lustre-ffff8a9108e6d000.statahead_max=0 1001 real 0m6.470s user 0m0.019s sys 0m2.413s ls -l 1000 files without statahead: 7 sec llite.lustre-ffff8a9108e6d000.statahead_max=32 32 1001 real 0m4.334s user 0m0.016s sys 0m2.092s ls -l 1000 files with statahead: 4 sec statahead total: 111 statahead wrong: 2 agl total: 106 list_total: 34 fname_total: 77 hit_total: 22468 miss_total: 1274 - open/close 4711 (time 1775877638.20 total 10.00 last 471.03) total: 9000 open/close in 19.31 seconds: 466.04 ops/second llite.lustre-ffff8a9108e6d000.statahead_max=0 10001 real 1m5.282s user 0m0.258s sys 0m24.023s ls -l 10000 files without statahead: 65 sec llite.lustre-ffff8a9108e6d000.statahead_max=32 32 10001 real 0m43.353s user 0m0.197s sys 0m20.668s ls -l 10000 files with statahead: 43 sec statahead total: 112 statahead wrong: 2 agl total: 107 list_total: 35 fname_total: 77 hit_total: 32466 miss_total: 1276 ls -l done rm -r /mnt/lustre/d123ad.sanity: 64 seconds llite.lustre-ffff8a9108e6d000.statahead_stats= statahead total: 112 statahead wrong: 2 agl total: 107 list_total: 35 fname_total: 77 hit_total: 32466 miss_total: 1276 mdc.lustre-MDT0000-mdc-ffff8a9108e6d000.batch_stats= snapshot_time: 1775877833.274112472 secs.nsecs start_time: 1775877608.465484800 secs.nsecs elapsed_time: 224.808627672 secs.nsecs subreqs per batch batches % cum % 1: 1 0 0 2: 1 0 0 4: 2 0 1 8: 8 2 3 16: 189 53 56 32: 153 43 100 mdc.lustre-MDT0001-mdc-ffff8a9108e6d000.batch_stats= snapshot_time: 1775877833.274166519 secs.nsecs start_time: 1775877608.465551300 secs.nsecs elapsed_time: 224.808615219 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 1 0 0 4: 4 1 1 8: 6 1 3 16: 192 54 57 32: 151 42 100 - open/close 4654 (time 1775877844.17 total 10.00 last 465.30) - open/close 9689 (time 1775877854.17 total 20.00 last 503.48) total: 10000 open/close in 20.67 seconds: 483.69 ops/second llite.lustre-ffff8a9108e6d000.statahead_batch_max=0 llite.lustre-ffff8a9108e6d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8a9108e6d000.stats=clear mdc.lustre-MDT0001-mdc-ffff8a9108e6d000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m36.299s user 0m0.268s sys 0m32.915s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8a9108e6d000.statahead_batch_max=32 llite.lustre-ffff8a9108e6d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8a9108e6d000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8a9108e6d000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8a9108e6d000.stats=clear mdc.lustre-MDT0001-mdc-ffff8a9108e6d000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m41.880s user 0m0.180s sys 0m24.350s batched statahead RPC= hit count=9998 mdc.lustre-MDT0000-mdc-ffff8a9108e6d000.batch_stats= snapshot_time: 1775877942.002521965 secs.nsecs start_time: 1775877894.103430279 secs.nsecs elapsed_time: 47.899091686 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff8a9108e6d000.batch_stats= snapshot_time: 1775877942.002563956 secs.nsecs start_time: 1775877894.103504277 secs.nsecs elapsed_time: 47.899059679 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8a9108e6d000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8a9108e6d000.batch_stats=0 llite.lustre-ffff8a9108e6d000.statahead_max=2048 llite.lustre-ffff8a9108e6d000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8a9108e6d000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8a9108e6d000.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 1.65 seconds: 60.59 ops/second llite.lustre-ffff8a9108e6d000.statahead_max=0 101 real 0m1.464s user 0m0.003s sys 0m0.329s ls -l 100 files without statahead: 1 sec llite.lustre-ffff8a9108e6d000.statahead_max=2048 2048 101 real 0m0.558s user 0m0.006s sys 0m0.229s ls -l 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10097 miss_total: 3 total: 900 open/close in 5.22 seconds: 172.30 ops/second llite.lustre-ffff8a9108e6d000.statahead_max=0 1001 real 0m14.066s user 0m0.036s sys 0m3.031s ls -l 1000 files without statahead: 15 sec llite.lustre-ffff8a9108e6d000.statahead_max=2048 2048 1001 real 0m4.719s user 0m0.021s sys 0m1.866s ls -l 1000 files with statahead: 4 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11096 miss_total: 4 - open/close 1119 (time 1775878121.82 total 10.00 last 111.89) total: 2000 open/close in 16.93 seconds: 118.15 ops/second llite.lustre-ffff8a9108e6d000.statahead_max=0 3001 real 0m37.964s user 0m0.104s sys 0m9.005s ls -l 3000 files without statahead: 38 sec llite.lustre-ffff8a9108e6d000.statahead_max=2048 2048 3001 real 0m12.873s user 0m0.062s sys 0m5.663s ls -l 3000 files with statahead: 13 sec statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 14095 miss_total: 5 ls -l done rm -r /mnt/lustre/d123ad.sanity: 38 seconds llite.lustre-ffff8a9108e6d000.statahead_stats= statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 14095 miss_total: 5 mdc.lustre-MDT0000-mdc-ffff8a9108e6d000.batch_stats= snapshot_time: 1775878226.459535815 secs.nsecs start_time: 1775877942.832989376 secs.nsecs elapsed_time: 283.626546439 secs.nsecs subreqs per batch batches % cum % 1: 11 39 39 2: 0 0 39 4: 4 14 53 8: 2 7 60 16: 1 3 64 32: 1 3 67 64: 0 0 67 128: 0 0 67 256: 9 32 100 mdc.lustre-MDT0001-mdc-ffff8a9108e6d000.batch_stats= snapshot_time: 1775878226.459601279 secs.nsecs start_time: 1775877942.833116631 secs.nsecs elapsed_time: 283.626484648 secs.nsecs subreqs per batch batches % cum % 1: 8 27 27 2: 0 0 27 4: 2 6 34 8: 4 13 48 16: 2 6 55 32: 2 6 62 64: 0 0 62 128: 3 10 72 256: 8 27 100 - open/close 1575 (time 1775878238.21 total 10.00 last 157.44) - open/close 3000 (time 1775878248.21 total 20.01 last 142.47) - open/close 4251 (time 1775878258.21 total 30.01 last 125.02) - open/close 5467 (time 1775878268.22 total 40.02 last 121.52) - open/close 6716 (time 1775878278.23 total 50.03 last 124.79) - open/close 7993 (time 1775878288.23 total 60.03 last 127.64) - open/close 9261 (time 1775878298.24 total 70.03 last 126.78) total: 10000 open/close in 75.46 seconds: 132.52 ops/second llite.lustre-ffff8a9108e6d000.statahead_batch_max=0 llite.lustre-ffff8a9108e6d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8a9108e6d000.stats=clear mdc.lustre-MDT0001-mdc-ffff8a9108e6d000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m37.494s user 0m0.252s sys 0m21.779s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8a9108e6d000.statahead_batch_max=256 llite.lustre-ffff8a9108e6d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8a9108e6d000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8a9108e6d000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8a9108e6d000.stats=clear mdc.lustre-MDT0001-mdc-ffff8a9108e6d000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m32.627s user 0m0.210s sys 0m20.143s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8a9108e6d000.batch_stats= snapshot_time: 1775878385.575988245 secs.nsecs start_time: 1775878345.850795870 secs.nsecs elapsed_time: 39.725192375 secs.nsecs subreqs per batch batches % cum % 1: 320 68 68 2: 44 9 77 4: 31 6 84 8: 13 2 86 16: 5 1 88 32: 15 3 91 64: 7 1 92 128: 0 0 92 256: 34 7 100 mdc.lustre-MDT0001-mdc-ffff8a9108e6d000.batch_stats= snapshot_time: 1775878385.576060466 secs.nsecs start_time: 1775878345.850935965 secs.nsecs elapsed_time: 39.725124501 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8a9108e6d000.statahead_batch_max=64 llite.lustre-ffff8a9108e6d000.statahead_max=128