== sanity test 123ad: Verify batching statahead works correctly ========================================================== 05:02:08 (1747213328) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff880130edc000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff880130edc000.batch_stats=0 llite.lustre-ffff880130edc000.statahead_max=32 llite.lustre-ffff880130edc000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff880130edc000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff880130edc000.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.50 seconds: 66.84 ops/second llite.lustre-ffff880130edc000.statahead_max=0 101 real 0m1.430s user 0m0.010s sys 0m0.326s ls -l 100 files without statahead: 1 sec llite.lustre-ffff880130edc000.statahead_max=32 32 101 real 0m0.435s user 0m0.005s sys 0m0.144s ls -l 100 files with statahead: 1 sec statahead total: 51 statahead wrong: 1 agl total: 51 list_total: 19 fname_total: 32 hit_total: 4576 miss_total: 217 total: 900 open/close in 5.99 seconds: 150.22 ops/second llite.lustre-ffff880130edc000.statahead_max=0 1001 real 0m10.329s user 0m0.095s sys 0m2.783s ls -l 1000 files without statahead: 11 sec llite.lustre-ffff880130edc000.statahead_max=32 32 1001 real 0m3.328s user 0m0.042s sys 0m1.131s ls -l 1000 files with statahead: 3 sec statahead total: 52 statahead wrong: 1 agl total: 52 list_total: 20 fname_total: 32 hit_total: 5575 miss_total: 225 - open/close 1872 (time 1747213370.11 total 10.00 last 187.12) total: 3000 open/close in 16.56 seconds: 181.16 ops/second llite.lustre-ffff880130edc000.statahead_max=0 4001 real 0m46.905s user 0m0.335s sys 0m11.803s ls -l 4000 files without statahead: 47 sec llite.lustre-ffff880130edc000.statahead_max=32 32 4001 real 0m13.597s user 0m0.089s sys 0m4.570s ls -l 4000 files with statahead: 14 sec statahead total: 53 statahead wrong: 1 agl total: 53 list_total: 21 fname_total: 32 hit_total: 9574 miss_total: 248 ls -l done rm -r /mnt/lustre/d123ad.sanity: 34 seconds llite.lustre-ffff880130edc000.statahead_stats= statahead total: 53 statahead wrong: 1 agl total: 53 list_total: 21 fname_total: 32 hit_total: 9574 miss_total: 248 mdc.lustre-MDT0000-mdc-ffff880130edc000.batch_stats= snapshot_time: 1747213476.235573035 secs.nsecs start_time: 1747213330.801473274 secs.nsecs elapsed_time: 145.434099761 secs.nsecs subreqs per batch batches % cum % 1: 2 1 1 2: 2 1 2 4: 3 1 4 8: 5 3 7 16: 109 66 73 32: 43 26 100 mdc.lustre-MDT0001-mdc-ffff880130edc000.batch_stats= snapshot_time: 1747213476.235620897 secs.nsecs start_time: 1747213330.801531229 secs.nsecs elapsed_time: 145.434089668 secs.nsecs subreqs per batch batches % cum % 1: 1 0 0 2: 1 0 1 4: 3 1 2 8: 2 1 4 16: 66 39 43 32: 94 56 100 - open/close 1720 (time 1747213487.14 total 10.01 last 171.85) - open/close 3252 (time 1747213497.14 total 20.01 last 153.14) - open/close 4871 (time 1747213507.14 total 30.01 last 161.89) - open/close 6283 (time 1747213517.15 total 40.02 last 141.13) - open/close 8023 (time 1747213527.15 total 50.02 last 173.98) - open/close 9478 (time 1747213537.15 total 60.02 last 145.50) total: 10000 open/close in 63.47 seconds: 157.55 ops/second llite.lustre-ffff880130edc000.statahead_batch_max=0 llite.lustre-ffff880130edc000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff880130edc000.stats=clear mdc.lustre-MDT0001-mdc-ffff880130edc000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m19.551s user 0m0.300s sys 0m9.915s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff880130edc000.statahead_batch_max=32 llite.lustre-ffff880130edc000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff880130edc000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff880130edc000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff880130edc000.stats=clear mdc.lustre-MDT0001-mdc-ffff880130edc000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m37.911s user 0m0.290s sys 0m11.095s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff880130edc000.batch_stats= snapshot_time: 1747213604.293939271 secs.nsecs start_time: 1747213563.038601847 secs.nsecs elapsed_time: 41.255337424 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff880130edc000.batch_stats= snapshot_time: 1747213604.294012050 secs.nsecs start_time: 1747213563.038666004 secs.nsecs elapsed_time: 41.255346046 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff880130edc000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff880130edc000.batch_stats=0 llite.lustre-ffff880130edc000.statahead_max=2048 llite.lustre-ffff880130edc000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff880130edc000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff880130edc000.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 1.06 seconds: 94.67 ops/second llite.lustre-ffff880130edc000.statahead_max=0 101 real 0m1.140s user 0m0.002s sys 0m0.301s ls -l 100 files without statahead: 1 sec llite.lustre-ffff880130edc000.statahead_max=2048 2048 101 real 0m0.343s user 0m0.001s sys 0m0.149s ls -l 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 67 total: 900 open/close in 4.89 seconds: 183.98 ops/second llite.lustre-ffff880130edc000.statahead_max=0 1001 real 0m10.814s user 0m0.059s sys 0m2.925s ls -l 1000 files without statahead: 11 sec llite.lustre-ffff880130edc000.statahead_max=2048 2048 1001 real 0m1.838s user 0m0.026s sys 0m1.056s ls -l 1000 files with statahead: 2 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 68 - open/close 1682 (time 1747213773.50 total 10.01 last 168.08) total: 3000 open/close in 18.27 seconds: 164.16 ops/second llite.lustre-ffff880130edc000.statahead_max=0 4001 real 0m45.021s user 0m0.410s sys 0m11.610s ls -l 4000 files without statahead: 45 sec llite.lustre-ffff880130edc000.statahead_max=2048 2048 4001 real 0m6.419s user 0m0.116s sys 0m4.355s ls -l 4000 files with statahead: 7 sec statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 15096 miss_total: 69 ls -l done rm -r /mnt/lustre/d123ad.sanity: 27 seconds llite.lustre-ffff880130edc000.statahead_stats= statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 15096 miss_total: 69 mdc.lustre-MDT0000-mdc-ffff880130edc000.batch_stats= snapshot_time: 1747213865.443450342 secs.nsecs start_time: 1747213604.881613884 secs.nsecs elapsed_time: 260.561836458 secs.nsecs subreqs per batch batches % cum % 1: 10 34 34 2: 1 3 37 4: 4 13 51 8: 1 3 55 16: 0 0 55 32: 0 0 55 64: 0 0 55 128: 3 10 65 256: 10 34 100 mdc.lustre-MDT0001-mdc-ffff880130edc000.batch_stats= snapshot_time: 1747213865.443504952 secs.nsecs start_time: 1747213604.881694259 secs.nsecs elapsed_time: 260.561810693 secs.nsecs subreqs per batch batches % cum % 1: 7 22 22 2: 1 3 25 4: 3 9 35 8: 2 6 41 16: 1 3 45 32: 1 3 48 64: 3 9 58 128: 1 3 61 256: 12 38 100 - open/close 1949 (time 1747213876.54 total 10.00 last 194.87) - open/close 4328 (time 1747213886.54 total 20.00 last 237.84) - open/close 7152 (time 1747213896.54 total 30.01 last 282.33) - open/close 9990 (time 1747213906.55 total 40.01 last 283.72) total: 10000 open/close in 40.04 seconds: 249.76 ops/second llite.lustre-ffff880130edc000.statahead_batch_max=0 llite.lustre-ffff880130edc000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff880130edc000.stats=clear mdc.lustre-MDT0001-mdc-ffff880130edc000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m11.911s user 0m0.275s sys 0m9.471s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff880130edc000.statahead_batch_max=256 llite.lustre-ffff880130edc000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff880130edc000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff880130edc000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff880130edc000.stats=clear mdc.lustre-MDT0001-mdc-ffff880130edc000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m10.864s user 0m0.234s sys 0m9.298s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff880130edc000.batch_stats= snapshot_time: 1747213934.019660739 secs.nsecs start_time: 1747213920.600853527 secs.nsecs elapsed_time: 13.418807212 secs.nsecs subreqs per batch batches % cum % 1: 162 63 63 2: 11 4 67 4: 10 3 71 8: 7 2 73 16: 13 5 78 32: 15 5 84 64: 3 1 85 128: 0 0 85 256: 36 14 100 mdc.lustre-MDT0001-mdc-ffff880130edc000.batch_stats= snapshot_time: 1747213934.019708011 secs.nsecs start_time: 1747213920.600922975 secs.nsecs elapsed_time: 13.418785036 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff880130edc000.statahead_batch_max=64 llite.lustre-ffff880130edc000.statahead_max=128