== sanity test 123ad: Verify batching statahead works correctly ========================================================== 22:18:18 (1778293098) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff93d5527e7800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff93d5527e7800.batch_stats=0 llite.lustre-ffff93d5527e7800.statahead_max=32 llite.lustre-ffff93d5527e7800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff93d5527e7800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff93d5527e7800.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 0.58 seconds: 172.92 ops/second llite.lustre-ffff93d5527e7800.statahead_max=0 101 real 0m0.527s user 0m0.004s sys 0m0.207s ls -l 100 files without statahead: 0 sec llite.lustre-ffff93d5527e7800.statahead_max=32 32 101 real 0m0.374s user 0m0.002s sys 0m0.163s ls -l 100 files with statahead: 0 sec statahead total: 80 statahead wrong: 2 agl total: 75 list_total: 33 fname_total: 46 hit_total: 28530 miss_total: 1171 total: 900 open/close in 1.74 seconds: 517.65 ops/second llite.lustre-ffff93d5527e7800.statahead_max=0 1001 real 0m5.497s user 0m0.026s sys 0m2.162s ls -l 1000 files without statahead: 6 sec llite.lustre-ffff93d5527e7800.statahead_max=32 32 1001 real 0m3.583s user 0m0.014s sys 0m1.705s ls -l 1000 files with statahead: 3 sec statahead total: 81 statahead wrong: 2 agl total: 76 list_total: 34 fname_total: 46 hit_total: 29529 miss_total: 1172 - open/close 4806 (time 1778293127.00 total 10.00 last 480.57) total: 9000 open/close in 17.85 seconds: 504.11 ops/second llite.lustre-ffff93d5527e7800.statahead_max=0 10001 real 0m55.596s user 0m0.226s sys 0m21.305s ls -l 10000 files without statahead: 55 sec llite.lustre-ffff93d5527e7800.statahead_max=32 32 10001 real 0m40.588s user 0m0.187s sys 0m19.701s ls -l 10000 files with statahead: 41 sec statahead total: 82 statahead wrong: 2 agl total: 77 list_total: 35 fname_total: 46 hit_total: 39528 miss_total: 1173 ls -l done rm -r /mnt/lustre/d123ad.sanity: 55 seconds llite.lustre-ffff93d5527e7800.statahead_stats= statahead total: 82 statahead wrong: 2 agl total: 77 list_total: 35 fname_total: 46 hit_total: 39528 miss_total: 1173 mdc.lustre-MDT0000-mdc-ffff93d5527e7800.batch_stats= snapshot_time: 1778293298.676621524 secs.nsecs start_time: 1778293099.651585310 secs.nsecs elapsed_time: 199.025036214 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 3 0 0 4: 3 0 1 8: 3 0 2 16: 194 54 57 32: 151 42 100 mdc.lustre-MDT0001-mdc-ffff93d5527e7800.batch_stats= snapshot_time: 1778293298.676692600 secs.nsecs start_time: 1778293099.651651516 secs.nsecs elapsed_time: 199.025041084 secs.nsecs subreqs per batch batches % cum % 1: 1 0 0 2: 1 0 0 4: 3 0 1 8: 9 2 3 16: 196 55 59 32: 145 40 100 - open/close 5367 (time 1778293310.08 total 10.00 last 536.64) total: 10000 open/close in 18.23 seconds: 548.68 ops/second llite.lustre-ffff93d5527e7800.statahead_batch_max=0 llite.lustre-ffff93d5527e7800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93d5527e7800.stats=clear mdc.lustre-MDT0001-mdc-ffff93d5527e7800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m37.798s user 0m0.168s sys 0m34.829s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff93d5527e7800.statahead_batch_max=32 llite.lustre-ffff93d5527e7800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93d5527e7800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff93d5527e7800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff93d5527e7800.stats=clear mdc.lustre-MDT0001-mdc-ffff93d5527e7800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m40.754s user 0m0.176s sys 0m23.735s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff93d5527e7800.batch_stats= snapshot_time: 1778293406.468899625 secs.nsecs start_time: 1778293359.339349055 secs.nsecs elapsed_time: 47.129550570 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff93d5527e7800.batch_stats= snapshot_time: 1778293406.468939790 secs.nsecs start_time: 1778293359.339398508 secs.nsecs elapsed_time: 47.129541282 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff93d5527e7800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff93d5527e7800.batch_stats=0 llite.lustre-ffff93d5527e7800.statahead_max=2048 llite.lustre-ffff93d5527e7800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff93d5527e7800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff93d5527e7800.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 0.73 seconds: 136.21 ops/second llite.lustre-ffff93d5527e7800.statahead_max=0 101 real 0m0.714s user 0m0.005s sys 0m0.267s ls -l 100 files without statahead: 1 sec llite.lustre-ffff93d5527e7800.statahead_max=2048 2048 101 real 0m0.384s user 0m0.003s sys 0m0.272s ls -l 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 2 total: 900 open/close in 2.02 seconds: 444.52 ops/second llite.lustre-ffff93d5527e7800.statahead_max=0 1001 real 0m6.082s user 0m0.022s sys 0m2.325s ls -l 1000 files without statahead: 6 sec llite.lustre-ffff93d5527e7800.statahead_max=2048 2048 1001 real 0m3.661s user 0m0.021s sys 0m1.560s ls -l 1000 files with statahead: 4 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 - open/close 4909 (time 1778293518.69 total 10.00 last 490.84) total: 9000 open/close in 18.81 seconds: 478.40 ops/second llite.lustre-ffff93d5527e7800.statahead_max=0 10001 real 1m6.546s user 0m0.216s sys 0m22.380s ls -l 10000 files without statahead: 66 sec llite.lustre-ffff93d5527e7800.statahead_max=2048 2048 10001 real 0m57.720s user 0m0.270s sys 0m15.642s ls -l 10000 files with statahead: 58 sec statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 21096 miss_total: 4 ls -l done rm -r /mnt/lustre/d123ad.sanity: 126 seconds llite.lustre-ffff93d5527e7800.statahead_stats= statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 21096 miss_total: 4 mdc.lustre-MDT0000-mdc-ffff93d5527e7800.batch_stats= snapshot_time: 1778293797.862679020 secs.nsecs start_time: 1778293407.245455857 secs.nsecs elapsed_time: 390.617223163 secs.nsecs subreqs per batch batches % cum % 1: 68 51 51 2: 16 12 63 4: 12 9 72 8: 4 3 75 16: 0 0 75 32: 0 0 75 64: 0 0 75 128: 6 4 79 256: 27 20 100 mdc.lustre-MDT0001-mdc-ffff93d5527e7800.batch_stats= snapshot_time: 1778293797.870687502 secs.nsecs start_time: 1778293407.245579432 secs.nsecs elapsed_time: 390.625108070 secs.nsecs subreqs per batch batches % cum % 1: 66 46 46 2: 19 13 60 4: 8 5 65 8: 10 7 73 16: 2 1 74 32: 0 0 74 64: 2 1 75 128: 3 2 78 256: 31 21 100 - open/close 840 (time 1778293809.92 total 10.02 last 83.87) - open/close 1717 (time 1778293819.92 total 20.02 last 87.66) - open/close 2850 (time 1778293829.93 total 30.03 last 113.21) - open/close 3829 (time 1778293839.93 total 40.03 last 97.90) - open/close 4806 (time 1778293849.93 total 50.03 last 97.67) - open/close 5858 (time 1778293859.94 total 60.04 last 105.13) - open/close 6728 (time 1778293869.94 total 70.04 last 86.98) - open/close 7531 (time 1778293879.95 total 80.05 last 80.22) - open/close 8023 (time 1778293889.95 total 90.05 last 49.20) - open/close 8626 (time 1778293899.96 total 100.06 last 60.26) - open/close 9170 (time 1778293909.96 total 110.06 last 54.37) - open/close 9818 (time 1778293919.97 total 120.07 last 64.73) total: 10000 open/close in 123.33 seconds: 81.08 ops/second llite.lustre-ffff93d5527e7800.statahead_batch_max=0 llite.lustre-ffff93d5527e7800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93d5527e7800.stats=clear mdc.lustre-MDT0001-mdc-ffff93d5527e7800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.536s user 0m0.336s sys 0m16.775s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff93d5527e7800.statahead_batch_max=256 llite.lustre-ffff93d5527e7800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93d5527e7800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff93d5527e7800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff93d5527e7800.stats=clear mdc.lustre-MDT0001-mdc-ffff93d5527e7800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m50.305s user 0m0.298s sys 0m17.199s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff93d5527e7800.batch_stats= snapshot_time: 1778294060.388949531 secs.nsecs start_time: 1778293997.701851986 secs.nsecs elapsed_time: 62.687097545 secs.nsecs subreqs per batch batches % cum % 1: 179 55 55 2: 42 13 68 4: 27 8 77 8: 10 3 80 16: 6 1 81 32: 14 4 86 64: 9 2 89 128: 1 0 89 256: 34 10 100 mdc.lustre-MDT0001-mdc-ffff93d5527e7800.batch_stats= snapshot_time: 1778294060.389006309 secs.nsecs start_time: 1778293997.702083836 secs.nsecs elapsed_time: 62.686922473 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff93d5527e7800.statahead_batch_max=64 llite.lustre-ffff93d5527e7800.statahead_max=128