== sanity test 123ad: Verify batching statahead works correctly ========================================================== 21:05:58 (1746147958) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8800b60aa000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8800b60aa000.batch_stats=0 llite.lustre-ffff8800b60aa000.statahead_max=32 llite.lustre-ffff8800b60aa000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8800b60aa000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8800b60aa000.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 0.96 seconds: 103.79 ops/second llite.lustre-ffff8800b60aa000.statahead_max=0 101 real 0m1.045s user 0m0.013s sys 0m0.282s ls -l 100 files without statahead: 1 sec llite.lustre-ffff8800b60aa000.statahead_max=32 32 101 real 0m0.339s user 0m0.004s sys 0m0.121s ls -l 100 files with statahead: 0 sec statahead total: 73 statahead wrong: 2 agl total: 73 list_total: 25 fname_total: 48 hit_total: 5352 miss_total: 1309 total: 900 open/close in 4.72 seconds: 190.64 ops/second llite.lustre-ffff8800b60aa000.statahead_max=0 1001 real 0m11.383s user 0m0.081s sys 0m2.911s ls -l 1000 files without statahead: 11 sec llite.lustre-ffff8800b60aa000.statahead_max=32 32 1001 real 0m3.916s user 0m0.021s sys 0m1.141s ls -l 1000 files with statahead: 4 sec statahead total: 74 statahead wrong: 2 agl total: 74 list_total: 26 fname_total: 48 hit_total: 6351 miss_total: 1315 - open/close 1632 (time 1746147998.64 total 10.00 last 163.18) total: 3000 open/close in 18.85 seconds: 159.16 ops/second llite.lustre-ffff8800b60aa000.statahead_max=0 4001 real 0m42.876s user 0m0.436s sys 0m11.190s ls -l 4000 files without statahead: 43 sec llite.lustre-ffff8800b60aa000.statahead_max=32 32 4001 real 0m11.665s user 0m0.101s sys 0m4.238s ls -l 4000 files with statahead: 12 sec statahead total: 75 statahead wrong: 2 agl total: 75 list_total: 27 fname_total: 48 hit_total: 10350 miss_total: 1338 ls -l done rm -r /mnt/lustre/d123ad.sanity: 36 seconds llite.lustre-ffff8800b60aa000.statahead_stats= statahead total: 75 statahead wrong: 2 agl total: 75 list_total: 27 fname_total: 48 hit_total: 10350 miss_total: 1338 mdc.lustre-MDT0000-mdc-ffff8800b60aa000.batch_stats= snapshot_time: 1746148101.988492368 secs.nsecs start_time: 1746147960.302871498 secs.nsecs elapsed_time: 141.685620870 secs.nsecs subreqs per batch batches % cum % 1: 1 0 0 2: 3 1 2 4: 1 0 3 8: 9 5 8 16: 104 63 71 32: 46 28 100 mdc.lustre-MDT0001-mdc-ffff8800b60aa000.batch_stats= snapshot_time: 1746148101.989608443 secs.nsecs start_time: 1746147960.302941593 secs.nsecs elapsed_time: 141.686666850 secs.nsecs subreqs per batch batches % cum % 1: 2 1 1 2: 0 0 1 4: 2 1 2 8: 2 1 3 16: 70 42 45 32: 90 54 100 - open/close 1571 (time 1746148113.10 total 10.00 last 157.09) - open/close 3134 (time 1746148123.10 total 20.00 last 156.28) - open/close 4968 (time 1746148133.11 total 30.00 last 183.38) - open/close 6706 (time 1746148143.11 total 40.01 last 173.70) - open/close 8414 (time 1746148153.12 total 50.02 last 170.68) total: 10000 open/close in 58.63 seconds: 170.56 ops/second llite.lustre-ffff8800b60aa000.statahead_batch_max=0 llite.lustre-ffff8800b60aa000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8800b60aa000.stats=clear mdc.lustre-MDT0001-mdc-ffff8800b60aa000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m20.318s user 0m0.260s sys 0m9.499s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8800b60aa000.statahead_batch_max=32 llite.lustre-ffff8800b60aa000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8800b60aa000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8800b60aa000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8800b60aa000.stats=clear mdc.lustre-MDT0001-mdc-ffff8800b60aa000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m37.840s user 0m0.279s sys 0m10.854s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8800b60aa000.batch_stats= snapshot_time: 1746148226.783097649 secs.nsecs start_time: 1746148184.960233065 secs.nsecs elapsed_time: 41.822864584 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff8800b60aa000.batch_stats= snapshot_time: 1746148226.783184020 secs.nsecs start_time: 1746148184.960300465 secs.nsecs elapsed_time: 41.822883555 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8800b60aa000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8800b60aa000.batch_stats=0 llite.lustre-ffff8800b60aa000.statahead_max=2048 llite.lustre-ffff8800b60aa000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8800b60aa000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8800b60aa000.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.19 seconds: 84.01 ops/second llite.lustre-ffff8800b60aa000.statahead_max=0 101 real 0m1.187s user 0m0.005s sys 0m0.310s ls -l 100 files without statahead: 2 sec llite.lustre-ffff8800b60aa000.statahead_max=2048 2048 101 real 0m0.309s user 0m0.001s sys 0m0.139s ls -l 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 67 total: 900 open/close in 6.40 seconds: 140.73 ops/second llite.lustre-ffff8800b60aa000.statahead_max=0 1001 real 0m10.806s user 0m0.052s sys 0m2.873s ls -l 1000 files without statahead: 10 sec llite.lustre-ffff8800b60aa000.statahead_max=2048 2048 1001 real 0m1.835s user 0m0.022s sys 0m1.061s ls -l 1000 files with statahead: 2 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 68 - open/close 1837 (time 1746148399.08 total 10.00 last 183.70) - open/close 3681 (time 1746148409.08 total 20.00 last 184.37) total: 4000 open/close in 22.29 seconds: 179.48 ops/second llite.lustre-ffff8800b60aa000.statahead_max=0 5001 real 1m0.744s user 0m0.473s sys 0m14.591s ls -l 5000 files without statahead: 61 sec llite.lustre-ffff8800b60aa000.statahead_max=2048 2048 5001 real 0m9.539s user 0m0.176s sys 0m5.459s ls -l 5000 files with statahead: 9 sec statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 16096 miss_total: 69 ls -l done rm -r /mnt/lustre/d123ad.sanity: 47 seconds llite.lustre-ffff8800b60aa000.statahead_stats= statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 16096 miss_total: 69 mdc.lustre-MDT0000-mdc-ffff8800b60aa000.batch_stats= snapshot_time: 1746148535.330212234 secs.nsecs start_time: 1746148227.457822554 secs.nsecs elapsed_time: 307.872389680 secs.nsecs subreqs per batch batches % cum % 1: 19 32 32 2: 8 13 45 4: 8 13 59 8: 6 10 69 16: 2 3 72 32: 0 0 72 64: 0 0 72 128: 4 6 79 256: 12 20 100 mdc.lustre-MDT0001-mdc-ffff8800b60aa000.batch_stats= snapshot_time: 1746148535.330563810 secs.nsecs start_time: 1746148227.457913627 secs.nsecs elapsed_time: 307.872650183 secs.nsecs subreqs per batch batches % cum % 1: 29 40 40 2: 7 9 50 4: 7 9 59 8: 4 5 65 16: 6 8 73 32: 1 1 75 64: 1 1 76 128: 3 4 80 256: 14 19 100 - open/close 1494 (time 1746148546.70 total 10.00 last 149.37) - open/close 3041 (time 1746148556.70 total 20.01 last 154.65) - open/close 4564 (time 1746148566.71 total 30.01 last 152.29) - open/close 6092 (time 1746148576.71 total 40.01 last 152.78) - open/close 7575 (time 1746148586.71 total 50.01 last 148.25) - open/close 9132 (time 1746148596.71 total 60.01 last 155.67) total: 10000 open/close in 64.84 seconds: 154.22 ops/second llite.lustre-ffff8800b60aa000.statahead_batch_max=0 llite.lustre-ffff8800b60aa000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8800b60aa000.stats=clear mdc.lustre-MDT0001-mdc-ffff8800b60aa000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m22.065s user 0m0.248s sys 0m9.504s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8800b60aa000.statahead_batch_max=256 llite.lustre-ffff8800b60aa000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8800b60aa000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8800b60aa000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8800b60aa000.stats=clear mdc.lustre-MDT0001-mdc-ffff8800b60aa000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m17.982s user 0m0.287s sys 0m10.750s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8800b60aa000.batch_stats= snapshot_time: 1746148649.225318586 secs.nsecs start_time: 1746148626.633821703 secs.nsecs elapsed_time: 22.591496883 secs.nsecs subreqs per batch batches % cum % 1: 135 57 57 2: 14 6 63 4: 11 4 68 8: 9 3 72 16: 7 3 75 32: 16 6 82 64: 5 2 84 128: 0 0 84 256: 36 15 100 mdc.lustre-MDT0001-mdc-ffff8800b60aa000.batch_stats= snapshot_time: 1746148649.225417456 secs.nsecs start_time: 1746148626.634546966 secs.nsecs elapsed_time: 22.590870490 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8800b60aa000.statahead_batch_max=64 llite.lustre-ffff8800b60aa000.statahead_max=128