== sanity test 123ad: Verify batching statahead works correctly ========================================================== 23:32:30 (1739507550) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff88012db83000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff88012db83000.batch_stats=0 llite.lustre-ffff88012db83000.statahead_max=32 llite.lustre-ffff88012db83000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff88012db83000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff88012db83000.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.18 seconds: 84.59 ops/second llite.lustre-ffff88012db83000.statahead_max=0 101 real 0m1.251s user 0m0.012s sys 0m0.300s ls -l 100 files without statahead: 1 sec llite.lustre-ffff88012db83000.statahead_max=32 32 101 real 0m0.513s user 0m0.003s sys 0m0.147s ls -l 100 files with statahead: 0 sec statahead total: 90 statahead wrong: 2 agl total: 90 list_total: 27 fname_total: 63 hit_total: 5512 miss_total: 1397 total: 900 open/close in 5.69 seconds: 158.12 ops/second llite.lustre-ffff88012db83000.statahead_max=0 1001 real 0m10.468s user 0m0.140s sys 0m2.682s ls -l 1000 files without statahead: 11 sec llite.lustre-ffff88012db83000.statahead_max=32 32 1001 real 0m3.560s user 0m0.022s sys 0m1.177s ls -l 1000 files with statahead: 4 sec statahead total: 91 statahead wrong: 2 agl total: 91 list_total: 28 fname_total: 63 hit_total: 6511 miss_total: 1404 - open/close 1771 (time 1739507592.10 total 10.00 last 177.03) total: 3000 open/close in 16.73 seconds: 179.33 ops/second llite.lustre-ffff88012db83000.statahead_max=0 4001 real 0m52.862s user 0m0.376s sys 0m11.600s ls -l 4000 files without statahead: 53 sec llite.lustre-ffff88012db83000.statahead_max=32 32 4001 real 0m15.246s user 0m0.126s sys 0m4.446s ls -l 4000 files with statahead: 15 sec statahead total: 92 statahead wrong: 2 agl total: 92 list_total: 29 fname_total: 63 hit_total: 10510 miss_total: 1439 ls -l done rm -r /mnt/lustre/d123ad.sanity: 37 seconds llite.lustre-ffff88012db83000.statahead_stats= statahead total: 92 statahead wrong: 2 agl total: 92 list_total: 29 fname_total: 63 hit_total: 10510 miss_total: 1439 mdc.lustre-MDT0000-mdc-ffff88012db83000.batch_stats= snapshot_time: 1739507710.128993261 secs.nsecs start_time: 1739507552.740636338 secs.nsecs elapsed_time: 157.388356923 secs.nsecs subreqs per batch batches % cum % 1: 4 2 2 2: 1 0 3 4: 3 1 4 8: 5 3 7 16: 108 65 73 32: 44 26 100 mdc.lustre-MDT0001-mdc-ffff88012db83000.batch_stats= snapshot_time: 1739507710.129639396 secs.nsecs start_time: 1739507552.741390674 secs.nsecs elapsed_time: 157.388248722 secs.nsecs subreqs per batch batches % cum % 1: 1 0 0 2: 1 0 1 4: 1 0 1 8: 5 2 4 16: 67 40 44 32: 92 55 100 - open/close 1645 (time 1739507721.19 total 10.00 last 164.48) - open/close 3248 (time 1739507731.19 total 20.00 last 160.28) - open/close 4724 (time 1739507741.19 total 30.00 last 147.60) - open/close 6282 (time 1739507751.20 total 40.01 last 155.72) - open/close 7776 (time 1739507761.20 total 50.01 last 149.36) - open/close 9219 (time 1739507771.20 total 60.01 last 144.28) total: 10000 open/close in 65.21 seconds: 153.35 ops/second llite.lustre-ffff88012db83000.statahead_batch_max=0 llite.lustre-ffff88012db83000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88012db83000.stats=clear mdc.lustre-MDT0001-mdc-ffff88012db83000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m24.882s user 0m0.264s sys 0m9.495s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff88012db83000.statahead_batch_max=32 llite.lustre-ffff88012db83000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88012db83000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff88012db83000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff88012db83000.stats=clear mdc.lustre-MDT0001-mdc-ffff88012db83000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m43.024s user 0m0.344s sys 0m11.113s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff88012db83000.batch_stats= snapshot_time: 1739507852.127439314 secs.nsecs start_time: 1739507804.786714927 secs.nsecs elapsed_time: 47.340724387 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff88012db83000.batch_stats= snapshot_time: 1739507852.127652982 secs.nsecs start_time: 1739507804.788370473 secs.nsecs elapsed_time: 47.339282509 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff88012db83000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff88012db83000.batch_stats=0 llite.lustre-ffff88012db83000.statahead_max=2048 llite.lustre-ffff88012db83000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff88012db83000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff88012db83000.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 1.20 seconds: 83.18 ops/second llite.lustre-ffff88012db83000.statahead_max=0 101 real 0m1.254s user 0m0.005s sys 0m0.313s ls -l 100 files without statahead: 1 sec llite.lustre-ffff88012db83000.statahead_max=2048 2048 101 real 0m0.412s user 0m0.005s sys 0m0.135s ls -l 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 81 total: 900 open/close in 5.22 seconds: 172.29 ops/second llite.lustre-ffff88012db83000.statahead_max=0 1001 real 0m11.743s user 0m0.133s sys 0m2.804s ls -l 1000 files without statahead: 12 sec llite.lustre-ffff88012db83000.statahead_max=2048 2048 1001 real 0m2.226s user 0m0.030s sys 0m1.063s ls -l 1000 files with statahead: 2 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 82 - open/close 1894 (time 1739508035.01 total 10.00 last 189.35) total: 3000 open/close in 16.35 seconds: 183.53 ops/second llite.lustre-ffff88012db83000.statahead_max=0 4001 real 0m44.678s user 0m0.581s sys 0m10.905s ls -l 4000 files without statahead: 45 sec llite.lustre-ffff88012db83000.statahead_max=2048 2048 4001 real 0m6.677s user 0m0.138s sys 0m4.215s ls -l 4000 files with statahead: 7 sec statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 15096 miss_total: 83 ls -l done rm -r /mnt/lustre/d123ad.sanity: 34 seconds llite.lustre-ffff88012db83000.statahead_stats= statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 15096 miss_total: 83 mdc.lustre-MDT0000-mdc-ffff88012db83000.batch_stats= snapshot_time: 1739508131.624183276 secs.nsecs start_time: 1739507852.780805646 secs.nsecs elapsed_time: 278.843377630 secs.nsecs subreqs per batch batches % cum % 1: 3 13 13 2: 0 0 13 4: 3 13 27 8: 2 9 36 16: 0 0 36 32: 1 4 40 64: 0 0 40 128: 4 18 59 256: 9 40 100 mdc.lustre-MDT0001-mdc-ffff88012db83000.batch_stats= snapshot_time: 1739508131.624319473 secs.nsecs start_time: 1739507852.780942499 secs.nsecs elapsed_time: 278.843376974 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 1 4 4 8: 5 20 25 16: 2 8 33 32: 0 0 33 64: 2 8 41 128: 2 8 50 256: 12 50 100 - open/close 1631 (time 1739508143.02 total 10.00 last 163.06) - open/close 3173 (time 1739508153.02 total 20.01 last 154.16) - open/close 4746 (time 1739508163.02 total 30.01 last 157.27) - open/close 6437 (time 1739508173.03 total 40.01 last 169.02) - open/close 7908 (time 1739508183.04 total 50.02 last 147.01) - open/close 9653 (time 1739508193.04 total 60.02 last 174.48) total: 10000 open/close in 62.45 seconds: 160.12 ops/second llite.lustre-ffff88012db83000.statahead_batch_max=0 llite.lustre-ffff88012db83000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88012db83000.stats=clear mdc.lustre-MDT0001-mdc-ffff88012db83000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m25.910s user 0m0.249s sys 0m9.504s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff88012db83000.statahead_batch_max=256 llite.lustre-ffff88012db83000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88012db83000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff88012db83000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff88012db83000.stats=clear mdc.lustre-MDT0001-mdc-ffff88012db83000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m16.230s user 0m0.343s sys 0m10.367s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff88012db83000.batch_stats= snapshot_time: 1739508245.190453380 secs.nsecs start_time: 1739508224.726475371 secs.nsecs elapsed_time: 20.463978009 secs.nsecs subreqs per batch batches % cum % 1: 217 67 67 2: 17 5 72 4: 11 3 75 8: 9 2 78 16: 11 3 82 32: 15 4 86 64: 8 2 89 128: 0 0 89 256: 35 10 100 mdc.lustre-MDT0001-mdc-ffff88012db83000.batch_stats= snapshot_time: 1739508245.190591980 secs.nsecs start_time: 1739508224.726590850 secs.nsecs elapsed_time: 20.464001130 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff88012db83000.statahead_batch_max=64 llite.lustre-ffff88012db83000.statahead_max=128