== sanity test 123ad: Verify batching statahead works correctly ========================================================== 04:06:40 (1772183200) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff99970702b000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff99970702b000.batch_stats=0 llite.lustre-ffff99970702b000.statahead_max=32 llite.lustre-ffff99970702b000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff99970702b000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff99970702b000.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.63 seconds: 61.37 ops/second llite.lustre-ffff99970702b000.statahead_max=0 101 real 0m1.517s user 0m0.005s sys 0m0.312s ls -l 100 files without statahead: 1 sec llite.lustre-ffff99970702b000.statahead_max=32 32 101 real 0m0.619s user 0m0.001s sys 0m0.183s ls -l 100 files with statahead: 1 sec statahead total: 97 statahead wrong: 2 agl total: 92 list_total: 32 fname_total: 65 hit_total: 21806 miss_total: 1415 total: 900 open/close in 6.36 seconds: 141.48 ops/second llite.lustre-ffff99970702b000.statahead_max=0 1001 real 0m13.551s user 0m0.030s sys 0m2.913s ls -l 1000 files without statahead: 14 sec llite.lustre-ffff99970702b000.statahead_max=32 32 1001 real 0m4.886s user 0m0.021s sys 0m1.553s ls -l 1000 files with statahead: 5 sec statahead total: 98 statahead wrong: 2 agl total: 93 list_total: 33 fname_total: 65 hit_total: 22805 miss_total: 1416 - open/close 1495 (time 1772183249.32 total 10.01 last 149.40) total: 2000 open/close in 13.69 seconds: 146.10 ops/second llite.lustre-ffff99970702b000.statahead_max=0 3001 real 0m39.050s user 0m0.086s sys 0m8.508s ls -l 3000 files without statahead: 39 sec llite.lustre-ffff99970702b000.statahead_max=32 32 3001 real 0m14.916s user 0m0.073s sys 0m4.511s ls -l 3000 files with statahead: 15 sec statahead total: 99 statahead wrong: 2 agl total: 94 list_total: 34 fname_total: 65 hit_total: 25801 miss_total: 1420 ls -l done rm -r /mnt/lustre/d123ad.sanity: 33 seconds llite.lustre-ffff99970702b000.statahead_stats= statahead total: 99 statahead wrong: 2 agl total: 94 list_total: 34 fname_total: 65 hit_total: 25801 miss_total: 1420 mdc.lustre-MDT0000-mdc-ffff99970702b000.batch_stats= snapshot_time: 1772183348.081106954 secs.nsecs start_time: 1772183202.325195946 secs.nsecs elapsed_time: 145.755911008 secs.nsecs subreqs per batch batches % cum % 1: 6 4 4 2: 1 0 5 4: 2 1 6 8: 5 3 10 16: 86 63 73 32: 36 26 100 mdc.lustre-MDT0001-mdc-ffff99970702b000.batch_stats= snapshot_time: 1772183348.081174665 secs.nsecs start_time: 1772183202.325278718 secs.nsecs elapsed_time: 145.755895947 secs.nsecs subreqs per batch batches % cum % 1: 2 1 1 2: 1 0 2 4: 3 2 4 8: 3 2 6 16: 60 43 50 32: 68 49 100 - open/close 1375 (time 1772183359.30 total 10.01 last 137.43) - open/close 2575 (time 1772183369.31 total 20.02 last 119.81) - open/close 3799 (time 1772183379.32 total 30.03 last 122.31) - open/close 5284 (time 1772183389.32 total 40.03 last 148.48) - open/close 6583 (time 1772183399.33 total 50.03 last 129.85) - open/close 8077 (time 1772183409.33 total 60.04 last 149.35) - open/close 9449 (time 1772183419.33 total 70.04 last 137.16) total: 10000 open/close in 73.40 seconds: 136.24 ops/second llite.lustre-ffff99970702b000.statahead_batch_max=0 llite.lustre-ffff99970702b000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff99970702b000.stats=clear mdc.lustre-MDT0001-mdc-ffff99970702b000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m36.366s user 0m0.265s sys 0m26.235s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff99970702b000.statahead_batch_max=32 llite.lustre-ffff99970702b000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff99970702b000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff99970702b000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff99970702b000.stats=clear mdc.lustre-MDT0001-mdc-ffff99970702b000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m45.677s user 0m0.215s sys 0m18.581s batched statahead RPC= hit count=9987 mdc.lustre-MDT0000-mdc-ffff99970702b000.batch_stats= snapshot_time: 1772183515.523601972 secs.nsecs start_time: 1772183463.452263509 secs.nsecs elapsed_time: 52.071338463 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff99970702b000.batch_stats= snapshot_time: 1772183515.523663339 secs.nsecs start_time: 1772183463.452340994 secs.nsecs elapsed_time: 52.071322345 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff99970702b000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff99970702b000.batch_stats=0 llite.lustre-ffff99970702b000.statahead_max=2048 llite.lustre-ffff99970702b000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff99970702b000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff99970702b000.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 1.24 seconds: 80.52 ops/second llite.lustre-ffff99970702b000.statahead_max=0 101 real 0m1.285s user 0m0.006s sys 0m0.312s ls -l 100 files without statahead: 2 sec llite.lustre-ffff99970702b000.statahead_max=2048 2048 101 real 0m0.450s user 0m0.003s sys 0m0.188s ls -l 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10086 miss_total: 14 total: 900 open/close in 4.50 seconds: 199.78 ops/second llite.lustre-ffff99970702b000.statahead_max=0 1001 real 0m10.199s user 0m0.034s sys 0m2.809s ls -l 1000 files without statahead: 11 sec llite.lustre-ffff99970702b000.statahead_max=2048 2048 1001 real 0m3.572s user 0m0.018s sys 0m1.701s ls -l 1000 files with statahead: 4 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 - open/close 2549 (time 1772183689.91 total 10.00 last 254.87) total: 3000 open/close in 11.91 seconds: 251.96 ops/second llite.lustre-ffff99970702b000.statahead_max=0 4001 real 0m37.611s user 0m0.123s sys 0m11.023s ls -l 4000 files without statahead: 37 sec llite.lustre-ffff99970702b000.statahead_max=2048 2048 4001 real 0m12.965s user 0m0.082s sys 0m7.405s ls -l 4000 files with statahead: 13 sec statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 15084 miss_total: 16 ls -l done rm -r /mnt/lustre/d123ad.sanity: 36 seconds llite.lustre-ffff99970702b000.statahead_stats= statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 15084 miss_total: 16 mdc.lustre-MDT0000-mdc-ffff99970702b000.batch_stats= snapshot_time: 1772183787.633670830 secs.nsecs start_time: 1772183516.637450627 secs.nsecs elapsed_time: 270.996220203 secs.nsecs subreqs per batch batches % cum % 1: 43 59 59 2: 6 8 68 4: 5 6 75 8: 4 5 80 16: 0 0 80 32: 0 0 80 64: 2 2 83 128: 0 0 83 256: 12 16 100 mdc.lustre-MDT0001-mdc-ffff99970702b000.batch_stats= snapshot_time: 1772183787.633731778 secs.nsecs start_time: 1772183516.637603967 secs.nsecs elapsed_time: 270.996127811 secs.nsecs subreqs per batch batches % cum % 1: 46 59 59 2: 9 11 71 4: 3 3 75 8: 3 3 79 16: 1 1 80 32: 1 1 81 64: 2 2 84 128: 1 1 85 256: 11 14 100 - open/close 1575 (time 1772183798.76 total 10.00 last 157.48) - open/close 3304 (time 1772183808.76 total 20.00 last 172.87) - open/close 4876 (time 1772183818.77 total 30.01 last 157.13) - open/close 6564 (time 1772183828.77 total 40.01 last 168.77) - open/close 8313 (time 1772183838.77 total 50.01 last 174.86) total: 10000 open/close in 58.57 seconds: 170.74 ops/second llite.lustre-ffff99970702b000.statahead_batch_max=0 llite.lustre-ffff99970702b000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff99970702b000.stats=clear mdc.lustre-MDT0001-mdc-ffff99970702b000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m33.218s user 0m0.230s sys 0m25.626s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff99970702b000.statahead_batch_max=256 llite.lustre-ffff99970702b000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff99970702b000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff99970702b000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff99970702b000.stats=clear mdc.lustre-MDT0001-mdc-ffff99970702b000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m30.021s user 0m0.219s sys 0m19.452s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff99970702b000.batch_stats= snapshot_time: 1772183920.427860369 secs.nsecs start_time: 1772183884.435494063 secs.nsecs elapsed_time: 35.992366306 secs.nsecs subreqs per batch batches % cum % 1: 362 66 66 2: 64 11 78 4: 38 6 85 8: 18 3 88 16: 6 1 89 32: 12 2 91 64: 9 1 93 128: 2 0 93 256: 33 6 100 mdc.lustre-MDT0001-mdc-ffff99970702b000.batch_stats= snapshot_time: 1772183920.427925339 secs.nsecs start_time: 1772183884.435566779 secs.nsecs elapsed_time: 35.992358560 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff99970702b000.statahead_batch_max=64 llite.lustre-ffff99970702b000.statahead_max=128