== sanity test 123ad: Verify batching statahead works correctly ========================================================== 20:12:53 (1746144773) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8800aa0fb800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8800aa0fb800.batch_stats=0 llite.lustre-ffff8800aa0fb800.statahead_max=32 llite.lustre-ffff8800aa0fb800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8800aa0fb800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8800aa0fb800.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.36 seconds: 42.39 ops/second llite.lustre-ffff8800aa0fb800.statahead_max=0 101 real 0m1.998s user 0m0.005s sys 0m0.401s ls -l 100 files without statahead: 2 sec llite.lustre-ffff8800aa0fb800.statahead_max=32 32 101 real 0m0.788s user 0m0.003s sys 0m0.216s ls -l 100 files with statahead: 1 sec statahead total: 138 statahead wrong: 2 agl total: 138 list_total: 28 fname_total: 110 hit_total: 4448 miss_total: 1619 total: 900 open/close in 7.69 seconds: 117.10 ops/second llite.lustre-ffff8800aa0fb800.statahead_max=0 1001 real 0m15.133s user 0m0.091s sys 0m3.371s ls -l 1000 files without statahead: 15 sec llite.lustre-ffff8800aa0fb800.statahead_max=32 32 1001 real 0m4.547s user 0m0.036s sys 0m1.453s ls -l 1000 files with statahead: 5 sec statahead total: 139 statahead wrong: 2 agl total: 139 list_total: 29 fname_total: 110 hit_total: 5447 miss_total: 1624 - open/close 1133 (time 1746144828.92 total 10.00 last 113.25) total: 2000 open/close in 17.11 seconds: 116.91 ops/second llite.lustre-ffff8800aa0fb800.statahead_max=0 3001 real 0m46.945s user 0m0.258s sys 0m10.240s ls -l 3000 files without statahead: 47 sec llite.lustre-ffff8800aa0fb800.statahead_max=32 32 3001 real 0m14.273s user 0m0.113s sys 0m4.359s ls -l 3000 files with statahead: 14 sec statahead total: 140 statahead wrong: 2 agl total: 140 list_total: 30 fname_total: 110 hit_total: 8446 miss_total: 1644 ls -l done rm -r /mnt/lustre/d123ad.sanity: 36 seconds llite.lustre-ffff8800aa0fb800.statahead_stats= statahead total: 140 statahead wrong: 2 agl total: 140 list_total: 30 fname_total: 110 hit_total: 8446 miss_total: 1644 mdc.lustre-MDT0000-mdc-ffff8800aa0fb800.batch_stats= snapshot_time: 1746144940.544140191 secs.nsecs start_time: 1746144776.399261511 secs.nsecs elapsed_time: 164.144878680 secs.nsecs subreqs per batch batches % cum % 1: 1 0 0 2: 1 0 1 4: 2 1 2 8: 6 4 7 16: 75 55 62 32: 51 37 100 mdc.lustre-MDT0001-mdc-ffff8800aa0fb800.batch_stats= snapshot_time: 1746144940.544233095 secs.nsecs start_time: 1746144776.399962483 secs.nsecs elapsed_time: 164.144270612 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 1 0 0 4: 3 2 2 8: 3 2 5 16: 74 55 60 32: 53 39 100 - open/close 1282 (time 1746144951.79 total 10.00 last 128.15) - open/close 2527 (time 1746144961.79 total 20.00 last 124.50) - open/close 3788 (time 1746144971.80 total 30.01 last 126.03) - open/close 4987 (time 1746144981.80 total 40.02 last 119.83) - open/close 6220 (time 1746144991.81 total 50.02 last 123.26) - open/close 7538 (time 1746145001.81 total 60.02 last 131.79) - open/close 8701 (time 1746145011.82 total 70.03 last 116.16) - open/close 9850 (time 1746145021.83 total 80.04 last 114.82) total: 10000 open/close in 81.30 seconds: 123.00 ops/second llite.lustre-ffff8800aa0fb800.statahead_batch_max=0 llite.lustre-ffff8800aa0fb800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8800aa0fb800.stats=clear mdc.lustre-MDT0001-mdc-ffff8800aa0fb800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m35.921s user 0m0.387s sys 0m12.939s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8800aa0fb800.statahead_batch_max=32 llite.lustre-ffff8800aa0fb800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8800aa0fb800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8800aa0fb800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8800aa0fb800.stats=clear mdc.lustre-MDT0001-mdc-ffff8800aa0fb800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.916s user 0m0.442s sys 0m14.783s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8800aa0fb800.batch_stats= snapshot_time: 1746145132.193255169 secs.nsecs start_time: 1746145064.454902035 secs.nsecs elapsed_time: 67.738353134 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff8800aa0fb800.batch_stats= snapshot_time: 1746145132.194606056 secs.nsecs start_time: 1746145064.454993175 secs.nsecs elapsed_time: 67.739612881 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8800aa0fb800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8800aa0fb800.batch_stats=0 llite.lustre-ffff8800aa0fb800.statahead_max=2048 llite.lustre-ffff8800aa0fb800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8800aa0fb800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8800aa0fb800.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 1.93 seconds: 51.81 ops/second llite.lustre-ffff8800aa0fb800.statahead_max=0 101 real 0m1.686s user 0m0.009s sys 0m0.369s ls -l 100 files without statahead: 1 sec llite.lustre-ffff8800aa0fb800.statahead_max=2048 2048 101 real 0m0.526s user 0m0.003s sys 0m0.177s ls -l 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 66 total: 900 open/close in 7.54 seconds: 119.38 ops/second llite.lustre-ffff8800aa0fb800.statahead_max=0 1001 real 0m16.363s user 0m0.067s sys 0m3.629s ls -l 1000 files without statahead: 17 sec llite.lustre-ffff8800aa0fb800.statahead_max=2048 2048 1001 real 0m2.647s user 0m0.033s sys 0m1.327s ls -l 1000 files with statahead: 3 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 67 total: 1000 open/close in 7.23 seconds: 138.31 ops/second llite.lustre-ffff8800aa0fb800.statahead_max=0 2001 real 0m31.033s user 0m0.226s sys 0m6.977s ls -l 2000 files without statahead: 31 sec llite.lustre-ffff8800aa0fb800.statahead_max=2048 2048 2001 real 0m5.375s user 0m0.088s sys 0m2.815s ls -l 2000 files with statahead: 5 sec statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 13096 miss_total: 68 ls -l done rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff8800aa0fb800.statahead_stats= statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 13096 miss_total: 68 mdc.lustre-MDT0000-mdc-ffff8800aa0fb800.batch_stats= snapshot_time: 1746145454.170547307 secs.nsecs start_time: 1746145133.333986294 secs.nsecs elapsed_time: 320.836561013 secs.nsecs subreqs per batch batches % cum % 1: 2 13 13 2: 0 0 13 4: 4 26 40 8: 3 20 60 16: 0 0 60 32: 0 0 60 64: 0 0 60 128: 0 0 60 256: 6 40 100 mdc.lustre-MDT0001-mdc-ffff8800aa0fb800.batch_stats= snapshot_time: 1746145454.173083749 secs.nsecs start_time: 1746145133.334081541 secs.nsecs elapsed_time: 320.839002208 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 2 11 11 8: 4 23 35 16: 2 11 47 32: 2 11 58 64: 0 0 58 128: 1 5 64 256: 6 35 100 - open/close 1165 (time 1746145465.94 total 10.00 last 116.47) - open/close 2445 (time 1746145475.94 total 20.00 last 127.98) - open/close 3592 (time 1746145485.95 total 30.01 last 114.64) - open/close 4747 (time 1746145495.95 total 40.01 last 115.44) - open/close 5869 (time 1746145505.95 total 50.02 last 112.18) - open/close 7222 (time 1746145515.96 total 60.02 last 135.20) - open/close 8372 (time 1746145525.97 total 70.03 last 114.92) - open/close 9506 (time 1746145535.97 total 80.04 last 113.32) total: 10000 open/close in 83.97 seconds: 119.10 ops/second llite.lustre-ffff8800aa0fb800.statahead_batch_max=0 llite.lustre-ffff8800aa0fb800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8800aa0fb800.stats=clear mdc.lustre-MDT0001-mdc-ffff8800aa0fb800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m29.066s user 0m0.325s sys 0m12.593s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8800aa0fb800.statahead_batch_max=256 llite.lustre-ffff8800aa0fb800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8800aa0fb800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8800aa0fb800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8800aa0fb800.stats=clear mdc.lustre-MDT0001-mdc-ffff8800aa0fb800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m24.958s user 0m0.406s sys 0m14.268s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8800aa0fb800.batch_stats= snapshot_time: 1746145603.565439918 secs.nsecs start_time: 1746145573.471159418 secs.nsecs elapsed_time: 30.094280500 secs.nsecs subreqs per batch batches % cum % 1: 136 52 52 2: 18 6 59 4: 19 7 66 8: 17 6 73 16: 18 6 80 32: 12 4 84 64: 5 1 86 128: 0 0 86 256: 35 13 100 mdc.lustre-MDT0001-mdc-ffff8800aa0fb800.batch_stats= snapshot_time: 1746145603.565540790 secs.nsecs start_time: 1746145573.471230850 secs.nsecs elapsed_time: 30.094309940 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8800aa0fb800.statahead_batch_max=64 llite.lustre-ffff8800aa0fb800.statahead_max=128