== sanity test 123ad: Verify batching statahead works correctly ========================================================== 21:57:08 (1746151028) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff88012a68e000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff88012a68e000.batch_stats=0 llite.lustre-ffff88012a68e000.statahead_max=32 llite.lustre-ffff88012a68e000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff88012a68e000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff88012a68e000.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 1.25 seconds: 80.03 ops/second llite.lustre-ffff88012a68e000.statahead_max=0 101 real 0m1.251s user 0m0.012s sys 0m0.309s ls -l 100 files without statahead: 2 sec llite.lustre-ffff88012a68e000.statahead_max=32 32 101 real 0m0.498s user 0m0.004s sys 0m0.133s ls -l 100 files with statahead: 1 sec statahead total: 116 statahead wrong: 2 agl total: 116 list_total: 27 fname_total: 89 hit_total: 4743 miss_total: 1461 total: 900 open/close in 5.39 seconds: 166.95 ops/second llite.lustre-ffff88012a68e000.statahead_max=0 1001 real 0m10.952s user 0m0.109s sys 0m2.806s ls -l 1000 files without statahead: 11 sec llite.lustre-ffff88012a68e000.statahead_max=32 32 1001 real 0m3.674s user 0m0.024s sys 0m1.166s ls -l 1000 files with statahead: 4 sec statahead total: 117 statahead wrong: 2 agl total: 117 list_total: 28 fname_total: 89 hit_total: 5742 miss_total: 1468 - open/close 1624 (time 1746151069.66 total 10.00 last 162.38) total: 3000 open/close in 18.78 seconds: 159.70 ops/second llite.lustre-ffff88012a68e000.statahead_max=0 4001 real 0m42.423s user 0m0.445s sys 0m10.947s ls -l 4000 files without statahead: 43 sec llite.lustre-ffff88012a68e000.statahead_max=32 32 4001 real 0m11.805s user 0m0.093s sys 0m4.243s ls -l 4000 files with statahead: 12 sec statahead total: 118 statahead wrong: 2 agl total: 118 list_total: 29 fname_total: 89 hit_total: 9741 miss_total: 1495 ls -l done rm -r /mnt/lustre/d123ad.sanity: 33 seconds llite.lustre-ffff88012a68e000.statahead_stats= statahead total: 118 statahead wrong: 2 agl total: 118 list_total: 29 fname_total: 89 hit_total: 9741 miss_total: 1495 mdc.lustre-MDT0000-mdc-ffff88012a68e000.batch_stats= snapshot_time: 1746151171.571857116 secs.nsecs start_time: 1746151030.472054571 secs.nsecs elapsed_time: 141.099802545 secs.nsecs subreqs per batch batches % cum % 1: 3 1 1 2: 0 0 1 4: 5 3 4 8: 4 2 7 16: 113 68 75 32: 40 24 100 mdc.lustre-MDT0001-mdc-ffff88012a68e000.batch_stats= snapshot_time: 1746151171.572050334 secs.nsecs start_time: 1746151030.472124811 secs.nsecs elapsed_time: 141.099925523 secs.nsecs subreqs per batch batches % cum % 1: 3 1 1 2: 0 0 1 4: 2 1 2 8: 4 2 5 16: 63 37 42 32: 96 57 100 - open/close 2096 (time 1746151182.66 total 10.00 last 209.59) - open/close 4097 (time 1746151192.67 total 20.01 last 200.00) - open/close 5969 (time 1746151202.67 total 30.01 last 187.12) - open/close 7927 (time 1746151212.67 total 40.01 last 195.75) - open/close 9857 (time 1746151222.68 total 50.02 last 192.93) total: 10000 open/close in 50.70 seconds: 197.24 ops/second llite.lustre-ffff88012a68e000.statahead_batch_max=0 llite.lustre-ffff88012a68e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88012a68e000.stats=clear mdc.lustre-MDT0001-mdc-ffff88012a68e000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m18.529s user 0m0.221s sys 0m9.169s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff88012a68e000.statahead_batch_max=32 llite.lustre-ffff88012a68e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88012a68e000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff88012a68e000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff88012a68e000.stats=clear mdc.lustre-MDT0001-mdc-ffff88012a68e000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m31.578s user 0m0.290s sys 0m10.304s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff88012a68e000.batch_stats= snapshot_time: 1746151279.844311756 secs.nsecs start_time: 1746151244.815966579 secs.nsecs elapsed_time: 35.028345177 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff88012a68e000.batch_stats= snapshot_time: 1746151279.844830076 secs.nsecs start_time: 1746151244.816591865 secs.nsecs elapsed_time: 35.028238211 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff88012a68e000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff88012a68e000.batch_stats=0 llite.lustre-ffff88012a68e000.statahead_max=2048 llite.lustre-ffff88012a68e000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff88012a68e000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff88012a68e000.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 1.37 seconds: 72.90 ops/second llite.lustre-ffff88012a68e000.statahead_max=0 101 real 0m1.189s user 0m0.021s sys 0m0.285s ls -l 100 files without statahead: 1 sec llite.lustre-ffff88012a68e000.statahead_max=2048 2048 101 real 0m0.336s user 0m0.007s sys 0m0.119s ls -l 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 52 total: 900 open/close in 4.94 seconds: 182.15 ops/second llite.lustre-ffff88012a68e000.statahead_max=0 1001 real 0m9.258s user 0m0.088s sys 0m2.527s ls -l 1000 files without statahead: 10 sec llite.lustre-ffff88012a68e000.statahead_max=2048 2048 1001 real 0m1.865s user 0m0.031s sys 0m1.064s ls -l 1000 files with statahead: 2 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 53 - open/close 2155 (time 1746151433.45 total 10.00 last 215.48) total: 4000 open/close in 18.26 seconds: 219.11 ops/second llite.lustre-ffff88012a68e000.statahead_max=0 5001 real 0m43.871s user 0m0.494s sys 0m12.722s ls -l 5000 files without statahead: 44 sec llite.lustre-ffff88012a68e000.statahead_max=2048 2048 5001 real 0m6.786s user 0m0.141s sys 0m5.038s ls -l 5000 files with statahead: 6 sec statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 16096 miss_total: 54 ls -l done rm -r /mnt/lustre/d123ad.sanity: 38 seconds llite.lustre-ffff88012a68e000.statahead_stats= statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 16096 miss_total: 54 mdc.lustre-MDT0000-mdc-ffff88012a68e000.batch_stats= snapshot_time: 1746151535.035798574 secs.nsecs start_time: 1746151280.515991690 secs.nsecs elapsed_time: 254.519806884 secs.nsecs subreqs per batch batches % cum % 1: 24 46 46 2: 5 9 55 4: 3 5 61 8: 3 5 67 16: 0 0 67 32: 0 0 67 64: 1 1 69 128: 2 3 73 256: 14 26 100 mdc.lustre-MDT0001-mdc-ffff88012a68e000.batch_stats= snapshot_time: 1746151535.035887808 secs.nsecs start_time: 1746151280.516097500 secs.nsecs elapsed_time: 254.519790308 secs.nsecs subreqs per batch batches % cum % 1: 23 41 41 2: 6 10 51 4: 4 7 58 8: 2 3 62 16: 1 1 64 32: 0 0 64 64: 2 3 67 128: 3 5 73 256: 15 26 100 - open/close 1617 (time 1746151546.17 total 10.00 last 161.62) - open/close 3655 (time 1746151556.17 total 20.01 last 203.74) - open/close 5746 (time 1746151566.18 total 30.01 last 209.06) - open/close 7659 (time 1746151576.18 total 40.02 last 191.16) - open/close 9496 (time 1746151586.18 total 50.02 last 183.69) total: 10000 open/close in 53.13 seconds: 188.22 ops/second llite.lustre-ffff88012a68e000.statahead_batch_max=0 llite.lustre-ffff88012a68e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88012a68e000.stats=clear mdc.lustre-MDT0001-mdc-ffff88012a68e000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m16.656s user 0m0.260s sys 0m9.220s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff88012a68e000.statahead_batch_max=256 llite.lustre-ffff88012a68e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88012a68e000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff88012a68e000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff88012a68e000.stats=clear mdc.lustre-MDT0001-mdc-ffff88012a68e000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m15.968s user 0m0.322s sys 0m10.578s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff88012a68e000.batch_stats= snapshot_time: 1746151628.611738795 secs.nsecs start_time: 1746151608.855280390 secs.nsecs elapsed_time: 19.756458405 secs.nsecs subreqs per batch batches % cum % 1: 122 56 56 2: 12 5 61 4: 8 3 65 8: 10 4 70 16: 11 5 75 32: 16 7 82 64: 2 0 83 128: 0 0 83 256: 36 16 100 mdc.lustre-MDT0001-mdc-ffff88012a68e000.batch_stats= snapshot_time: 1746151628.611836915 secs.nsecs start_time: 1746151608.855502027 secs.nsecs elapsed_time: 19.756334888 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff88012a68e000.statahead_batch_max=64 llite.lustre-ffff88012a68e000.statahead_max=128