== sanity test 123ad: Verify batching statahead works correctly ========================================================== 13:42:08 (1750182128) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8800a6e2b800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8800a6e2b800.batch_stats=0 llite.lustre-ffff8800a6e2b800.statahead_max=32 llite.lustre-ffff8800a6e2b800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8800a6e2b800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8800a6e2b800.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.21 seconds: 82.56 ops/second llite.lustre-ffff8800a6e2b800.statahead_max=0 101 real 0m1.114s user 0m0.011s sys 0m0.288s ls -l 100 files without statahead: 1 sec llite.lustre-ffff8800a6e2b800.statahead_max=32 32 101 real 0m0.334s user 0m0.005s sys 0m0.118s ls -l 100 files with statahead: 1 sec statahead total: 104 statahead wrong: 2 agl total: 104 list_total: 26 fname_total: 78 hit_total: 4777 miss_total: 1168 total: 900 open/close in 5.53 seconds: 162.84 ops/second llite.lustre-ffff8800a6e2b800.statahead_max=0 1001 real 0m11.411s user 0m0.077s sys 0m2.803s ls -l 1000 files without statahead: 12 sec llite.lustre-ffff8800a6e2b800.statahead_max=32 32 1001 real 0m3.145s user 0m0.024s sys 0m1.094s ls -l 1000 files with statahead: 3 sec statahead total: 105 statahead wrong: 2 agl total: 105 list_total: 27 fname_total: 78 hit_total: 5776 miss_total: 1171 - open/close 2354 (time 1750182168.96 total 10.00 last 235.33) total: 3000 open/close in 13.33 seconds: 225.02 ops/second llite.lustre-ffff8800a6e2b800.statahead_max=0 4001 real 0m45.223s user 0m0.234s sys 0m11.107s ls -l 4000 files without statahead: 45 sec llite.lustre-ffff8800a6e2b800.statahead_max=32 32 4001 real 0m12.708s user 0m0.068s sys 0m4.387s ls -l 4000 files with statahead: 13 sec statahead total: 106 statahead wrong: 2 agl total: 106 list_total: 28 fname_total: 78 hit_total: 9775 miss_total: 1193 ls -l done rm -r /mnt/lustre/d123ad.sanity: 36 seconds llite.lustre-ffff8800a6e2b800.statahead_stats= statahead total: 106 statahead wrong: 2 agl total: 106 list_total: 28 fname_total: 78 hit_total: 9775 miss_total: 1193 mdc.lustre-MDT0000-mdc-ffff8800a6e2b800.batch_stats= snapshot_time: 1750182270.781131460 secs.nsecs start_time: 1750182130.634740863 secs.nsecs elapsed_time: 140.146390597 secs.nsecs subreqs per batch batches % cum % 1: 1 0 0 2: 4 2 3 4: 0 0 3 8: 2 1 4 16: 111 68 72 32: 44 27 100 mdc.lustre-MDT0001-mdc-ffff8800a6e2b800.batch_stats= snapshot_time: 1750182270.781198201 secs.nsecs start_time: 1750182130.635173651 secs.nsecs elapsed_time: 140.146024550 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 2 1 1 8: 3 1 3 16: 67 40 43 32: 93 56 100 - open/close 1509 (time 1750182282.00 total 10.00 last 150.87) - open/close 3220 (time 1750182292.00 total 20.00 last 171.07) - open/close 4716 (time 1750182302.00 total 30.00 last 149.59) - open/close 6307 (time 1750182312.01 total 40.01 last 159.06) - open/close 7990 (time 1750182322.01 total 50.01 last 168.27) - open/close 9638 (time 1750182332.01 total 60.01 last 164.71) total: 10000 open/close in 62.07 seconds: 161.11 ops/second llite.lustre-ffff8800a6e2b800.statahead_batch_max=0 llite.lustre-ffff8800a6e2b800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8800a6e2b800.stats=clear mdc.lustre-MDT0001-mdc-ffff8800a6e2b800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m21.180s user 0m0.200s sys 0m9.768s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8800a6e2b800.statahead_batch_max=32 llite.lustre-ffff8800a6e2b800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8800a6e2b800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8800a6e2b800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8800a6e2b800.stats=clear mdc.lustre-MDT0001-mdc-ffff8800a6e2b800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m33.497s user 0m0.277s sys 0m10.299s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8800a6e2b800.batch_stats= snapshot_time: 1750182395.751751382 secs.nsecs start_time: 1750182358.133152923 secs.nsecs elapsed_time: 37.618598459 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff8800a6e2b800.batch_stats= snapshot_time: 1750182395.751799144 secs.nsecs start_time: 1750182358.133259694 secs.nsecs elapsed_time: 37.618539450 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8800a6e2b800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8800a6e2b800.batch_stats=0 llite.lustre-ffff8800a6e2b800.statahead_max=2048 llite.lustre-ffff8800a6e2b800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8800a6e2b800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8800a6e2b800.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.00 seconds: 99.85 ops/second llite.lustre-ffff8800a6e2b800.statahead_max=0 101 real 0m0.994s user 0m0.011s sys 0m0.269s ls -l 100 files without statahead: 1 sec llite.lustre-ffff8800a6e2b800.statahead_max=2048 2048 101 real 0m0.329s user 0m0.006s sys 0m0.117s ls -l 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 43 total: 900 open/close in 4.87 seconds: 184.89 ops/second llite.lustre-ffff8800a6e2b800.statahead_max=0 1001 real 0m11.605s user 0m0.061s sys 0m2.847s ls -l 1000 files without statahead: 12 sec llite.lustre-ffff8800a6e2b800.statahead_max=2048 2048 1001 real 0m1.805s user 0m0.026s sys 0m1.064s ls -l 1000 files with statahead: 1 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 44 - open/close 1654 (time 1750182549.46 total 10.00 last 165.36) total: 3000 open/close in 17.79 seconds: 168.64 ops/second llite.lustre-ffff8800a6e2b800.statahead_max=0 4001 real 0m44.594s user 0m0.326s sys 0m10.908s ls -l 4000 files without statahead: 45 sec llite.lustre-ffff8800a6e2b800.statahead_max=2048 2048 4001 real 0m6.633s user 0m0.126s sys 0m4.152s ls -l 4000 files with statahead: 6 sec statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 15096 miss_total: 45 ls -l done rm -r /mnt/lustre/d123ad.sanity: 32 seconds llite.lustre-ffff8800a6e2b800.statahead_stats= statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 15096 miss_total: 45 mdc.lustre-MDT0000-mdc-ffff8800a6e2b800.batch_stats= snapshot_time: 1750182644.683375793 secs.nsecs start_time: 1750182396.477205461 secs.nsecs elapsed_time: 248.206170332 secs.nsecs subreqs per batch batches % cum % 1: 19 42 42 2: 7 15 57 4: 2 4 62 8: 4 8 71 16: 0 0 71 32: 0 0 71 64: 0 0 71 128: 2 4 75 256: 11 24 100 mdc.lustre-MDT0001-mdc-ffff8800a6e2b800.batch_stats= snapshot_time: 1750182644.683427856 secs.nsecs start_time: 1750182396.477304951 secs.nsecs elapsed_time: 248.206122905 secs.nsecs subreqs per batch batches % cum % 1: 19 38 38 2: 4 8 46 4: 2 4 51 8: 5 10 61 16: 3 6 67 32: 1 2 69 64: 1 2 71 128: 2 4 75 256: 12 24 100 - open/close 2083 (time 1750182655.67 total 10.00 last 208.28) - open/close 4229 (time 1750182665.68 total 20.00 last 214.57) - open/close 6524 (time 1750182675.68 total 30.00 last 229.49) - open/close 8947 (time 1750182685.68 total 40.00 last 242.26) total: 10000 open/close in 46.16 seconds: 216.62 ops/second llite.lustre-ffff8800a6e2b800.statahead_batch_max=0 llite.lustre-ffff8800a6e2b800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8800a6e2b800.stats=clear mdc.lustre-MDT0001-mdc-ffff8800a6e2b800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m16.792s user 0m0.248s sys 0m9.457s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8800a6e2b800.statahead_batch_max=256 llite.lustre-ffff8800a6e2b800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8800a6e2b800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8800a6e2b800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8800a6e2b800.stats=clear mdc.lustre-MDT0001-mdc-ffff8800a6e2b800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m16.663s user 0m0.361s sys 0m10.656s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8800a6e2b800.batch_stats= snapshot_time: 1750182732.532046261 secs.nsecs start_time: 1750182712.040312807 secs.nsecs elapsed_time: 20.491733454 secs.nsecs subreqs per batch batches % cum % 1: 113 53 53 2: 13 6 60 4: 8 3 63 8: 13 6 70 16: 12 5 75 32: 11 5 80 64: 4 1 82 128: 0 0 82 256: 36 17 100 mdc.lustre-MDT0001-mdc-ffff8800a6e2b800.batch_stats= snapshot_time: 1750182732.532104946 secs.nsecs start_time: 1750182712.040606841 secs.nsecs elapsed_time: 20.491498105 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8800a6e2b800.statahead_batch_max=64 llite.lustre-ffff8800a6e2b800.statahead_max=128