== sanity test 123ad: Verify batching statahead works correctly ========================================================== 15:27:06 (1779996426) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9551c8cf0000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9551c8cf0000.batch_stats=0 llite.lustre-ffff9551c8cf0000.statahead_max=32 llite.lustre-ffff9551c8cf0000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9551c8cf0000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9551c8cf0000.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 4.78 seconds: 20.91 ops/second llite.lustre-ffff9551c8cf0000.statahead_max=0 101 real 0m4.770s user 0m0.005s sys 0m0.410s 'ls -l' 100 files without statahead: 5 sec llite.lustre-ffff9551c8cf0000.statahead_max=32 llite.lustre-ffff9551c8cf0000.statahead_max=32 101 real 0m4.820s user 0m0.054s sys 0m2.747s 'ls -l' 100 files with statahead: 5 sec statahead total: 122 statahead wrong: 2 agl total: 117 list_total: 31 fname_total: 91 hit_total: 4565 miss_total: 1273 swrong=2 ewrong=2 - open/close 726 (time 1779996470.60 total 10.01 last 72.54) total: 900 open/close in 12.47 seconds: 72.16 ops/second llite.lustre-ffff9551c8cf0000.statahead_max=0 1001 real 0m24.973s user 0m0.030s sys 0m3.404s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff9551c8cf0000.statahead_max=32 llite.lustre-ffff9551c8cf0000.statahead_max=32 1001 real 0m9.371s user 0m0.038s sys 0m1.596s 'ls -l' 1000 files with statahead: 9 sec statahead total: 123 statahead wrong: 2 agl total: 118 list_total: 32 fname_total: 91 hit_total: 5563 miss_total: 1275 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff9551c8cf0000.statahead_stats= statahead total: 123 statahead wrong: 2 agl total: 118 list_total: 32 fname_total: 91 hit_total: 5563 miss_total: 1275 mdc.lustre-MDT0000-mdc-ffff9551c8cf0000.batch_stats= snapshot_time: 1779996541.513386911 secs.nsecs start_time: 1779996432.384160535 secs.nsecs elapsed_time: 109.129226376 secs.nsecs subreqs per batch batches % cum % 1: 1 2 2 2: 0 0 2 4: 1 2 5 8: 6 15 20 16: 20 50 70 32: 12 30 100 mdc.lustre-MDT0001-mdc-ffff9551c8cf0000.batch_stats= snapshot_time: 1779996541.513597923 secs.nsecs start_time: 1779996432.384264010 secs.nsecs elapsed_time: 109.129333913 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 2 5 5 4: 4 10 15 8: 0 0 15 16: 19 47 62 32: 15 37 100 - open/close 903 (time 1779996554.28 total 10.00 last 90.26) - open/close 1623 (time 1779996564.28 total 20.01 last 71.96) - open/close 2447 (time 1779996574.29 total 30.02 last 82.32) - open/close 3292 (time 1779996584.32 total 40.05 last 84.24) - open/close 4131 (time 1779996594.35 total 50.08 last 83.69) - open/close 4760 (time 1779996604.35 total 60.08 last 62.89) - open/close 5498 (time 1779996614.37 total 70.10 last 73.67) - open/close 6103 (time 1779996624.37 total 80.10 last 60.49) - open/close 7033 (time 1779996634.37 total 90.10 last 92.99) - open/close 7730 (time 1779996644.38 total 100.11 last 69.63) - open/close 8376 (time 1779996654.39 total 110.12 last 64.55) - open/close 9247 (time 1779996664.39 total 120.12 last 87.08) total: 10000 open/close in 127.48 seconds: 78.45 ops/second llite.lustre-ffff9551c8cf0000.statahead_batch_max=0 llite.lustre-ffff9551c8cf0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9551c8cf0000.stats=clear mdc.lustre-MDT0001-mdc-ffff9551c8cf0000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m6.045s user 0m0.362s sys 0m16.353s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9551c8cf0000.statahead_batch_max=32 llite.lustre-ffff9551c8cf0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9551c8cf0000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9551c8cf0000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9551c8cf0000.stats=clear mdc.lustre-MDT0001-mdc-ffff9551c8cf0000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m50.140s user 0m0.344s sys 0m15.792s batched statahead RPC= hit count=9978 mdc.lustre-MDT0000-mdc-ffff9551c8cf0000.batch_stats= snapshot_time: 1779996875.952099054 secs.nsecs start_time: 1779996750.164444166 secs.nsecs elapsed_time: 125.787654888 secs.nsecs subreqs per batch batches % cum % 1: 1 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff9551c8cf0000.batch_stats= snapshot_time: 1779996875.952166713 secs.nsecs start_time: 1779996750.166494352 secs.nsecs elapsed_time: 125.785672361 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9551c8cf0000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9551c8cf0000.batch_stats=0 llite.lustre-ffff9551c8cf0000.statahead_max=2048 llite.lustre-ffff9551c8cf0000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9551c8cf0000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9551c8cf0000.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.42 seconds: 41.37 ops/second llite.lustre-ffff9551c8cf0000.statahead_max=0 101 real 0m3.194s user 0m0.002s sys 0m0.349s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9551c8cf0000.statahead_max=2048 llite.lustre-ffff9551c8cf0000.statahead_max=2048 101 real 0m0.768s user 0m0.003s sys 0m0.173s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10077 miss_total: 23 swrong=0 ewrong=0 - open/close 738 (time 1779997159.10 total 10.01 last 73.74) total: 900 open/close in 12.33 seconds: 72.97 ops/second llite.lustre-ffff9551c8cf0000.statahead_max=0 1001 real 0m21.097s user 0m0.035s sys 0m3.173s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff9551c8cf0000.statahead_max=2048 llite.lustre-ffff9551c8cf0000.statahead_max=2048 1001 real 0m7.141s user 0m0.026s sys 0m1.624s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11076 miss_total: 24 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 18 seconds llite.lustre-ffff9551c8cf0000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11076 miss_total: 24 mdc.lustre-MDT0000-mdc-ffff9551c8cf0000.batch_stats= snapshot_time: 1779997221.065381333 secs.nsecs start_time: 1779996878.655017528 secs.nsecs elapsed_time: 342.410363805 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 4 50 50 16: 0 0 50 32: 2 25 75 64: 0 0 75 128: 0 0 75 256: 2 25 100 mdc.lustre-MDT0001-mdc-ffff9551c8cf0000.batch_stats= snapshot_time: 1779997221.065457320 secs.nsecs start_time: 1779996878.659425863 secs.nsecs elapsed_time: 342.406031457 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 3 42 42 8: 0 0 42 16: 1 14 57 32: 1 14 71 64: 0 0 71 128: 0 0 71 256: 2 28 100 - open/close 818 (time 1779997233.82 total 10.00 last 81.80) - open/close 1805 (time 1779997243.82 total 20.00 last 98.67) - open/close 2668 (time 1779997253.83 total 30.01 last 86.27) - open/close 3496 (time 1779997263.83 total 40.01 last 82.74) - open/close 4362 (time 1779997273.84 total 50.02 last 86.52) - open/close 5321 (time 1779997283.85 total 60.03 last 95.84) - open/close 6187 (time 1779997293.85 total 70.03 last 86.58) - open/close 7135 (time 1779997303.86 total 80.04 last 94.74) - open/close 7964 (time 1779997313.87 total 90.05 last 82.84) - open/close 8856 (time 1779997323.87 total 100.05 last 89.17) - open/close 9944 (time 1779997333.88 total 110.06 last 108.71) total: 10000 open/close in 110.81 seconds: 90.24 ops/second llite.lustre-ffff9551c8cf0000.statahead_batch_max=0 llite.lustre-ffff9551c8cf0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9551c8cf0000.stats=clear mdc.lustre-MDT0001-mdc-ffff9551c8cf0000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m5.062s user 0m0.317s sys 0m16.305s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9551c8cf0000.statahead_batch_max=256 llite.lustre-ffff9551c8cf0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9551c8cf0000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9551c8cf0000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9551c8cf0000.stats=clear mdc.lustre-MDT0001-mdc-ffff9551c8cf0000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m57.333s user 0m0.316s sys 0m16.433s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9551c8cf0000.batch_stats= snapshot_time: 1779997483.888481253 secs.nsecs start_time: 1779997412.610848417 secs.nsecs elapsed_time: 71.277632836 secs.nsecs subreqs per batch batches % cum % 1: 191 50 50 2: 55 14 65 4: 41 10 76 8: 19 5 81 16: 9 2 84 32: 18 4 88 64: 6 1 90 128: 3 0 91 256: 33 8 100 mdc.lustre-MDT0001-mdc-ffff9551c8cf0000.batch_stats= snapshot_time: 1779997483.888548893 secs.nsecs start_time: 1779997412.611660934 secs.nsecs elapsed_time: 71.276887959 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9551c8cf0000.statahead_batch_max=64 llite.lustre-ffff9551c8cf0000.statahead_max=128