== sanity test 123ad: Verify batching statahead works correctly ========================================================== 04:25:27 (1771406727) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9126c8471800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9126c8471800.batch_stats=0 llite.lustre-ffff9126c8471800.statahead_max=32 llite.lustre-ffff9126c8471800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9126c8471800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9126c8471800.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 1.52 seconds: 65.81 ops/second llite.lustre-ffff9126c8471800.statahead_max=0 101 real 0m1.664s user 0m0.006s sys 0m0.317s ls -l 100 files without statahead: 2 sec llite.lustre-ffff9126c8471800.statahead_max=32 32 101 real 0m0.603s user 0m0.002s sys 0m0.151s ls -l 100 files with statahead: 0 sec statahead total: 97 statahead wrong: 2 agl total: 92 list_total: 32 fname_total: 51 hit_total: 28620 miss_total: 982 total: 900 open/close in 6.59 seconds: 136.64 ops/second llite.lustre-ffff9126c8471800.statahead_max=0 1001 real 0m13.267s user 0m0.035s sys 0m2.858s ls -l 1000 files without statahead: 14 sec llite.lustre-ffff9126c8471800.statahead_max=32 32 1001 real 0m5.169s user 0m0.024s sys 0m1.512s ls -l 1000 files with statahead: 5 sec statahead total: 98 statahead wrong: 2 agl total: 93 list_total: 33 fname_total: 51 hit_total: 29617 miss_total: 985 - open/close 1594 (time 1771406777.63 total 10.00 last 159.39) total: 2000 open/close in 12.75 seconds: 156.87 ops/second llite.lustre-ffff9126c8471800.statahead_max=0 3001 real 0m39.538s user 0m0.077s sys 0m8.539s ls -l 3000 files without statahead: 40 sec llite.lustre-ffff9126c8471800.statahead_max=32 32 3001 real 0m16.490s user 0m0.072s sys 0m4.578s ls -l 3000 files with statahead: 17 sec statahead total: 99 statahead wrong: 2 agl total: 94 list_total: 34 fname_total: 51 hit_total: 32614 miss_total: 988 ls -l done rm -r /mnt/lustre/d123ad.sanity: 30 seconds llite.lustre-ffff9126c8471800.statahead_stats= statahead total: 99 statahead wrong: 2 agl total: 94 list_total: 34 fname_total: 51 hit_total: 32614 miss_total: 988 mdc.lustre-MDT0000-mdc-ffff9126c8471800.batch_stats= snapshot_time: 1771406874.800110171 secs.nsecs start_time: 1771406729.561106144 secs.nsecs elapsed_time: 145.239004027 secs.nsecs subreqs per batch batches % cum % 1: 5 3 3 2: 1 0 4 4: 2 1 5 8: 7 5 11 16: 84 62 73 32: 36 26 100 mdc.lustre-MDT0001-mdc-ffff9126c8471800.batch_stats= snapshot_time: 1771406874.800154071 secs.nsecs start_time: 1771406729.561178212 secs.nsecs elapsed_time: 145.238975859 secs.nsecs subreqs per batch batches % cum % 1: 1 0 0 2: 1 0 1 4: 3 2 3 8: 4 2 6 16: 56 41 47 32: 71 52 100 - open/close 2101 (time 1771406886.46 total 10.00 last 210.09) - open/close 4014 (time 1771406896.46 total 20.00 last 191.28) - open/close 6244 (time 1771406906.47 total 30.00 last 222.97) - open/close 8297 (time 1771406916.47 total 40.00 last 205.28) total: 10000 open/close in 49.59 seconds: 201.67 ops/second llite.lustre-ffff9126c8471800.statahead_batch_max=0 llite.lustre-ffff9126c8471800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9126c8471800.stats=clear mdc.lustre-MDT0001-mdc-ffff9126c8471800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m34.004s user 0m0.254s sys 0m26.906s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9126c8471800.statahead_batch_max=32 llite.lustre-ffff9126c8471800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9126c8471800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9126c8471800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9126c8471800.stats=clear mdc.lustre-MDT0001-mdc-ffff9126c8471800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m44.069s user 0m0.223s sys 0m19.120s batched statahead RPC= hit count=9990 mdc.lustre-MDT0000-mdc-ffff9126c8471800.batch_stats= snapshot_time: 1771407014.299171729 secs.nsecs start_time: 1771406963.999224777 secs.nsecs elapsed_time: 50.299946952 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff9126c8471800.batch_stats= snapshot_time: 1771407014.299230867 secs.nsecs start_time: 1771406963.999282368 secs.nsecs elapsed_time: 50.299948499 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9126c8471800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9126c8471800.batch_stats=0 llite.lustre-ffff9126c8471800.statahead_max=2048 llite.lustre-ffff9126c8471800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9126c8471800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9126c8471800.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 1.02 seconds: 97.78 ops/second llite.lustre-ffff9126c8471800.statahead_max=0 101 real 0m0.897s user 0m0.005s sys 0m0.297s ls -l 100 files without statahead: 1 sec llite.lustre-ffff9126c8471800.statahead_max=2048 2048 101 real 0m0.375s user 0m0.004s sys 0m0.229s ls -l 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10089 miss_total: 11 total: 900 open/close in 3.64 seconds: 247.08 ops/second llite.lustre-ffff9126c8471800.statahead_max=0 1001 real 0m8.460s user 0m0.033s sys 0m2.785s ls -l 1000 files without statahead: 8 sec llite.lustre-ffff9126c8471800.statahead_max=2048 2048 1001 real 0m3.084s user 0m0.023s sys 0m1.665s ls -l 1000 files with statahead: 3 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11088 miss_total: 12 - open/close 1850 (time 1771407178.17 total 10.01 last 184.80) - open/close 3430 (time 1771407188.17 total 20.01 last 158.00) - open/close 4675 (time 1771407198.18 total 30.02 last 124.36) - open/close 5935 (time 1771407208.20 total 40.03 last 125.85) - open/close 6852 (time 1771407218.21 total 50.04 last 91.61) - open/close 7936 (time 1771407228.21 total 60.04 last 108.40) total: 9000 open/close in 68.42 seconds: 131.54 ops/second llite.lustre-ffff9126c8471800.statahead_max=0 10001 real 2m4.806s user 0m0.332s sys 0m28.443s ls -l 10000 files without statahead: 125 sec llite.lustre-ffff9126c8471800.statahead_max=2048 2048 10001 real 0m31.420s user 0m0.243s sys 0m18.781s ls -l 10000 files with statahead: 31 sec statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 21087 miss_total: 13 ls -l done rm -r /mnt/lustre/d123ad.sanity: 98 seconds llite.lustre-ffff9126c8471800.statahead_stats= statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 21087 miss_total: 13 mdc.lustre-MDT0000-mdc-ffff9126c8471800.batch_stats= snapshot_time: 1771407505.852907990 secs.nsecs start_time: 1771407015.411132490 secs.nsecs elapsed_time: 490.441775500 secs.nsecs subreqs per batch batches % cum % 1: 109 63 63 2: 14 8 71 4: 10 5 76 8: 5 2 79 16: 2 1 80 32: 0 0 80 64: 1 0 81 128: 5 2 84 256: 27 15 100 mdc.lustre-MDT0001-mdc-ffff9126c8471800.batch_stats= snapshot_time: 1771407505.852996104 secs.nsecs start_time: 1771407015.411264112 secs.nsecs elapsed_time: 490.441731992 secs.nsecs subreqs per batch batches % cum % 1: 102 56 56 2: 22 12 68 4: 8 4 72 8: 8 4 77 16: 4 2 79 32: 4 2 81 64: 1 0 82 128: 1 0 82 256: 31 17 100 - open/close 2568 (time 1771407516.81 total 10.00 last 256.77) - open/close 5001 (time 1771407526.82 total 20.01 last 243.06) - open/close 6876 (time 1771407536.82 total 30.01 last 187.45) - open/close 8736 (time 1771407546.82 total 40.02 last 185.93) total: 10000 open/close in 47.32 seconds: 211.31 ops/second llite.lustre-ffff9126c8471800.statahead_batch_max=0 llite.lustre-ffff9126c8471800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9126c8471800.stats=clear mdc.lustre-MDT0001-mdc-ffff9126c8471800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m34.485s user 0m0.260s sys 0m22.868s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9126c8471800.statahead_batch_max=256 llite.lustre-ffff9126c8471800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9126c8471800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9126c8471800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9126c8471800.stats=clear mdc.lustre-MDT0001-mdc-ffff9126c8471800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m29.923s user 0m0.210s sys 0m19.996s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9126c8471800.batch_stats= snapshot_time: 1771407629.998996745 secs.nsecs start_time: 1771407592.868718164 secs.nsecs elapsed_time: 37.130278581 secs.nsecs subreqs per batch batches % cum % 1: 210 54 54 2: 75 19 73 4: 33 8 82 8: 6 1 83 16: 6 1 85 32: 17 4 89 64: 4 1 90 128: 0 0 90 256: 35 9 100 mdc.lustre-MDT0001-mdc-ffff9126c8471800.batch_stats= snapshot_time: 1771407629.999059192 secs.nsecs start_time: 1771407592.868795338 secs.nsecs elapsed_time: 37.130263854 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9126c8471800.statahead_batch_max=64 llite.lustre-ffff9126c8471800.statahead_max=128