== sanity test 123ad: Verify batching statahead works correctly ========================================================== 22:58:43 (1746068323) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff88012e388000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff88012e388000.batch_stats=0 llite.lustre-ffff88012e388000.statahead_max=32 llite.lustre-ffff88012e388000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff88012e388000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff88012e388000.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 1.37 seconds: 73.03 ops/second llite.lustre-ffff88012e388000.statahead_max=0 101 real 0m1.199s user 0m0.001s sys 0m0.319s ls -l 100 files without statahead: 1 sec llite.lustre-ffff88012e388000.statahead_max=32 32 101 real 0m0.366s user 0m0.003s sys 0m0.139s ls -l 100 files with statahead: 0 sec statahead total: 103 statahead wrong: 2 agl total: 103 list_total: 28 fname_total: 75 hit_total: 5896 miss_total: 1557 total: 900 open/close in 4.74 seconds: 189.74 ops/second llite.lustre-ffff88012e388000.statahead_max=0 1001 real 0m11.150s user 0m0.090s sys 0m2.772s ls -l 1000 files without statahead: 11 sec llite.lustre-ffff88012e388000.statahead_max=32 32 1001 real 0m2.377s user 0m0.021s sys 0m1.029s ls -l 1000 files with statahead: 2 sec statahead total: 104 statahead wrong: 2 agl total: 104 list_total: 29 fname_total: 75 hit_total: 6895 miss_total: 1563 - open/close 1892 (time 1746068361.32 total 10.00 last 189.16) total: 3000 open/close in 15.18 seconds: 197.57 ops/second llite.lustre-ffff88012e388000.statahead_max=0 4001 real 0m34.867s user 0m0.221s sys 0m10.533s ls -l 4000 files without statahead: 35 sec llite.lustre-ffff88012e388000.statahead_max=32 32 4001 real 0m9.201s user 0m0.115s sys 0m4.084s ls -l 4000 files with statahead: 9 sec statahead total: 105 statahead wrong: 2 agl total: 105 list_total: 30 fname_total: 75 hit_total: 10894 miss_total: 1574 ls -l done rm -r /mnt/lustre/d123ad.sanity: 32 seconds llite.lustre-ffff88012e388000.statahead_stats= statahead total: 105 statahead wrong: 2 agl total: 105 list_total: 30 fname_total: 75 hit_total: 10894 miss_total: 1574 mdc.lustre-MDT0000-mdc-ffff88012e388000.batch_stats= snapshot_time: 1746068446.810463608 secs.nsecs start_time: 1746068324.923929793 secs.nsecs elapsed_time: 121.886533815 secs.nsecs subreqs per batch batches % cum % 1: 3 1 1 2: 1 0 2 4: 1 0 3 8: 5 3 6 16: 111 67 73 32: 43 26 100 mdc.lustre-MDT0001-mdc-ffff88012e388000.batch_stats= snapshot_time: 1746068446.810591795 secs.nsecs start_time: 1746068324.924227579 secs.nsecs elapsed_time: 121.886364216 secs.nsecs subreqs per batch batches % cum % 1: 1 0 0 2: 1 0 1 4: 3 1 2 8: 4 2 5 16: 68 40 46 32: 90 53 100 - open/close 1665 (time 1746068457.75 total 10.00 last 166.45) - open/close 3413 (time 1746068467.75 total 20.00 last 174.79) - open/close 5120 (time 1746068477.75 total 30.01 last 170.64) - open/close 7206 (time 1746068487.75 total 40.01 last 208.53) - open/close 8847 (time 1746068497.76 total 50.02 last 164.03) total: 10000 open/close in 55.40 seconds: 180.50 ops/second llite.lustre-ffff88012e388000.statahead_batch_max=0 llite.lustre-ffff88012e388000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88012e388000.stats=clear mdc.lustre-MDT0001-mdc-ffff88012e388000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m12.514s user 0m0.232s sys 0m9.658s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff88012e388000.statahead_batch_max=32 llite.lustre-ffff88012e388000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88012e388000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff88012e388000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff88012e388000.stats=clear mdc.lustre-MDT0001-mdc-ffff88012e388000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m26.021s user 0m0.265s sys 0m10.203s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff88012e388000.batch_stats= snapshot_time: 1746068546.898977171 secs.nsecs start_time: 1746068517.887519876 secs.nsecs elapsed_time: 29.011457295 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff88012e388000.batch_stats= snapshot_time: 1746068546.899508206 secs.nsecs start_time: 1746068517.887585325 secs.nsecs elapsed_time: 29.011922881 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff88012e388000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff88012e388000.batch_stats=0 llite.lustre-ffff88012e388000.statahead_max=2048 llite.lustre-ffff88012e388000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff88012e388000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff88012e388000.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 1.66 seconds: 60.26 ops/second llite.lustre-ffff88012e388000.statahead_max=0 101 real 0m1.502s user 0m0.013s sys 0m0.335s ls -l 100 files without statahead: 1 sec llite.lustre-ffff88012e388000.statahead_max=2048 2048 101 real 0m0.430s user 0m0.007s sys 0m0.157s ls -l 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 39 total: 900 open/close in 6.77 seconds: 132.93 ops/second llite.lustre-ffff88012e388000.statahead_max=0 1001 real 0m11.640s user 0m0.079s sys 0m3.153s ls -l 1000 files without statahead: 12 sec llite.lustre-ffff88012e388000.statahead_max=2048 2048 1001 real 0m2.097s user 0m0.034s sys 0m1.147s ls -l 1000 files with statahead: 2 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 40 - open/close 1952 (time 1746068721.03 total 10.00 last 195.13) total: 3000 open/close in 14.66 seconds: 204.63 ops/second llite.lustre-ffff88012e388000.statahead_max=0 4001 real 0m44.677s user 0m0.389s sys 0m12.042s ls -l 4000 files without statahead: 45 sec llite.lustre-ffff88012e388000.statahead_max=2048 2048 4001 real 0m6.925s user 0m0.139s sys 0m4.224s ls -l 4000 files with statahead: 7 sec statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 15096 miss_total: 41 ls -l done rm -r /mnt/lustre/d123ad.sanity: 33 seconds llite.lustre-ffff88012e388000.statahead_stats= statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 15096 miss_total: 41 mdc.lustre-MDT0000-mdc-ffff88012e388000.batch_stats= snapshot_time: 1746068815.497896019 secs.nsecs start_time: 1746068547.552174174 secs.nsecs elapsed_time: 267.945721845 secs.nsecs subreqs per batch batches % cum % 1: 1 4 4 2: 0 0 4 4: 4 17 21 8: 3 13 34 16: 1 4 39 32: 0 0 39 64: 1 4 43 128: 1 4 47 256: 12 52 100 mdc.lustre-MDT0001-mdc-ffff88012e388000.batch_stats= snapshot_time: 1746068815.499901488 secs.nsecs start_time: 1746068547.552288894 secs.nsecs elapsed_time: 267.947612594 secs.nsecs subreqs per batch batches % cum % 1: 2 8 8 2: 1 4 12 4: 4 16 28 8: 3 12 40 16: 1 4 44 32: 0 0 44 64: 1 4 48 128: 1 4 52 256: 12 48 100 - open/close 1685 (time 1746068826.55 total 10.00 last 168.47) - open/close 3716 (time 1746068836.56 total 20.00 last 203.04) - open/close 5859 (time 1746068846.56 total 30.01 last 214.17) - open/close 7668 (time 1746068856.56 total 40.01 last 180.87) - open/close 9437 (time 1746068866.57 total 50.01 last 176.87) total: 10000 open/close in 53.47 seconds: 187.03 ops/second llite.lustre-ffff88012e388000.statahead_batch_max=0 llite.lustre-ffff88012e388000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88012e388000.stats=clear mdc.lustre-MDT0001-mdc-ffff88012e388000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m17.033s user 0m0.213s sys 0m9.334s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff88012e388000.statahead_batch_max=256 llite.lustre-ffff88012e388000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88012e388000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff88012e388000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff88012e388000.stats=clear mdc.lustre-MDT0001-mdc-ffff88012e388000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m14.909s user 0m0.334s sys 0m10.550s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff88012e388000.batch_stats= snapshot_time: 1746068908.297454461 secs.nsecs start_time: 1746068889.835873033 secs.nsecs elapsed_time: 18.461581428 secs.nsecs subreqs per batch batches % cum % 1: 150 57 57 2: 12 4 62 4: 19 7 69 8: 13 5 74 16: 13 5 79 32: 13 5 84 64: 3 1 86 128: 0 0 86 256: 36 13 100 mdc.lustre-MDT0001-mdc-ffff88012e388000.batch_stats= snapshot_time: 1746068908.298104699 secs.nsecs start_time: 1746068889.835942503 secs.nsecs elapsed_time: 18.462162196 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff88012e388000.statahead_batch_max=64 llite.lustre-ffff88012e388000.statahead_max=128