== sanity test 123ad: Verify batching statahead works correctly ========================================================== 21:59:12 (1787104752) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff989ed0967000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff989ed0967000.batch_stats=0 llite.lustre-ffff989ed0967000.statahead_max=32 llite.lustre-ffff989ed0967000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff989ed0967000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff989ed0967000.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 0.89 seconds: 112.86 ops/second llite.lustre-ffff989ed0967000.statahead_max=0 101 real 0m1.068s user 0m0.002s sys 0m0.323s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff989ed0967000.statahead_max=32 llite.lustre-ffff989ed0967000.statahead_max=32 101 real 0m0.466s user 0m0.003s sys 0m0.262s 'ls -l' 100 files with statahead: 1 sec statahead total: 69 statahead wrong: 1 agl total: 63 list_total: 28 fname_total: 35 hit_total: 23743 miss_total: 213 swrong=1 ewrong=1 total: 900 open/close in 2.71 seconds: 332.21 ops/second llite.lustre-ffff989ed0967000.statahead_max=0 1001 real 0m7.875s user 0m0.029s sys 0m2.815s 'ls -l' 1000 files without statahead: 8 sec llite.lustre-ffff989ed0967000.statahead_max=32 llite.lustre-ffff989ed0967000.statahead_max=32 1001 real 0m4.268s user 0m0.014s sys 0m2.056s 'ls -l' 1000 files with statahead: 4 sec statahead total: 70 statahead wrong: 1 agl total: 64 list_total: 29 fname_total: 35 hit_total: 24742 miss_total: 214 swrong=1 ewrong=1 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 7 seconds llite.lustre-ffff989ed0967000.statahead_stats= statahead total: 70 statahead wrong: 1 agl total: 64 list_total: 29 fname_total: 35 hit_total: 24742 miss_total: 214 mdc.lustre-MDT0000-mdc-ffff989ed0967000.batch_stats= snapshot_time: 1787104785.437073002 secs.nsecs start_time: 1787104754.065575552 secs.nsecs elapsed_time: 31.371497450 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 3 8 10 4: 1 2 13 8: 2 5 18 16: 24 64 83 32: 6 16 100 mdc.lustre-MDT0001-mdc-ffff989ed0967000.batch_stats= snapshot_time: 1787104785.437175514 secs.nsecs start_time: 1787104754.065663217 secs.nsecs elapsed_time: 31.371512297 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 2 5 5 4: 0 0 5 8: 2 5 10 16: 10 25 35 32: 25 64 100 - open/close 3051 (time 1787104796.46 total 10.00 last 305.00) - open/close 6431 (time 1787104806.46 total 20.01 last 337.88) - open/close 9227 (time 1787104816.46 total 30.01 last 279.59) total: 10000 open/close in 32.91 seconds: 303.86 ops/second llite.lustre-ffff989ed0967000.statahead_batch_max=0 llite.lustre-ffff989ed0967000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff989ed0967000.stats=clear mdc.lustre-MDT0001-mdc-ffff989ed0967000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m37.800s user 0m0.272s sys 0m34.599s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff989ed0967000.statahead_batch_max=32 llite.lustre-ffff989ed0967000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff989ed0967000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff989ed0967000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff989ed0967000.stats=clear mdc.lustre-MDT0001-mdc-ffff989ed0967000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m41.648s user 0m0.195s sys 0m23.372s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff989ed0967000.batch_stats= snapshot_time: 1787104909.444010073 secs.nsecs start_time: 1787104860.640609219 secs.nsecs elapsed_time: 48.803400854 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff989ed0967000.batch_stats= snapshot_time: 1787104909.444065798 secs.nsecs start_time: 1787104860.640710992 secs.nsecs elapsed_time: 48.803354806 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff989ed0967000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff989ed0967000.batch_stats=0 llite.lustre-ffff989ed0967000.statahead_max=2048 llite.lustre-ffff989ed0967000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff989ed0967000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff989ed0967000.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 0.90 seconds: 110.55 ops/second llite.lustre-ffff989ed0967000.statahead_max=0 101 real 0m0.722s user 0m0.005s sys 0m0.269s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff989ed0967000.statahead_max=2048 llite.lustre-ffff989ed0967000.statahead_max=2048 101 real 0m0.490s user 0m0.000s sys 0m0.298s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 2 swrong=0 ewrong=0 total: 900 open/close in 2.18 seconds: 412.14 ops/second llite.lustre-ffff989ed0967000.statahead_max=0 1001 real 0m6.014s user 0m0.019s sys 0m2.350s 'ls -l' 1000 files without statahead: 6 sec llite.lustre-ffff989ed0967000.statahead_max=2048 llite.lustre-ffff989ed0967000.statahead_max=2048 1001 real 0m3.347s user 0m0.016s sys 0m1.421s 'ls -l' 1000 files with statahead: 4 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 6 seconds llite.lustre-ffff989ed0967000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 mdc.lustre-MDT0000-mdc-ffff989ed0967000.batch_stats= snapshot_time: 1787105061.344292278 secs.nsecs start_time: 1787104910.360159120 secs.nsecs elapsed_time: 150.984133158 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 20 20 8: 2 40 60 16: 0 0 60 32: 0 0 60 64: 0 0 60 128: 0 0 60 256: 2 40 100 mdc.lustre-MDT0001-mdc-ffff989ed0967000.batch_stats= snapshot_time: 1787105061.344362449 secs.nsecs start_time: 1787104910.360301385 secs.nsecs elapsed_time: 150.984061064 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 12 12 4: 0 0 12 8: 2 25 37 16: 1 12 50 32: 0 0 50 64: 1 12 62 128: 1 12 75 256: 2 25 100 - open/close 4754 (time 1787105072.09 total 10.00 last 475.38) - open/close 9681 (time 1787105082.09 total 20.00 last 492.66) total: 10000 open/close in 20.59 seconds: 485.66 ops/second llite.lustre-ffff989ed0967000.statahead_batch_max=0 llite.lustre-ffff989ed0967000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff989ed0967000.stats=clear mdc.lustre-MDT0001-mdc-ffff989ed0967000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m32.140s user 0m0.162s sys 0m26.556s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff989ed0967000.statahead_batch_max=256 llite.lustre-ffff989ed0967000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff989ed0967000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff989ed0967000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff989ed0967000.stats=clear mdc.lustre-MDT0001-mdc-ffff989ed0967000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m26.980s user 0m0.152s sys 0m19.156s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff989ed0967000.batch_stats= snapshot_time: 1787105150.582623520 secs.nsecs start_time: 1787105117.765763096 secs.nsecs elapsed_time: 32.816860424 secs.nsecs subreqs per batch batches %% cum %% 1: 310 63 63 2: 56 11 75 4: 47 9 84 8: 12 2 87 16: 4 0 88 32: 21 4 92 64: 1 0 92 128: 1 0 93 256: 34 6 100 mdc.lustre-MDT0001-mdc-ffff989ed0967000.batch_stats= snapshot_time: 1787105150.582686477 secs.nsecs start_time: 1787105117.765811348 secs.nsecs elapsed_time: 32.816875129 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff989ed0967000.statahead_batch_max=64 llite.lustre-ffff989ed0967000.statahead_max=128