== sanity test 123ad: Verify batching statahead works correctly ========================================================== 11:36:11 (1780328171) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffffa046060df800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffffa046060df800.batch_stats=0 llite.lustre-ffffa046060df800.statahead_max=32 llite.lustre-ffffa046060df800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffffa046060df800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffffa046060df800.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.60 seconds: 38.53 ops/second llite.lustre-ffffa046060df800.statahead_max=0 101 real 0m2.785s user 0m0.001s sys 0m0.338s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffffa046060df800.statahead_max=32 llite.lustre-ffffa046060df800.statahead_max=32 101 real 0m1.412s user 0m0.002s sys 0m0.205s 'ls -l' 100 files with statahead: 2 sec statahead total: 63 statahead wrong: 1 agl total: 58 list_total: 28 fname_total: 35 hit_total: 14011 miss_total: 210 swrong=1 ewrong=1 - open/close 649 (time 1780328200.98 total 10.00 last 64.90) total: 900 open/close in 12.54 seconds: 71.75 ops/second llite.lustre-ffffa046060df800.statahead_max=0 1001 real 0m28.780s user 0m0.039s sys 0m3.055s 'ls -l' 1000 files without statahead: 29 sec llite.lustre-ffffa046060df800.statahead_max=32 llite.lustre-ffffa046060df800.statahead_max=32 1001 real 0m10.648s user 0m0.027s sys 0m1.518s 'ls -l' 1000 files with statahead: 11 sec statahead total: 64 statahead wrong: 1 agl total: 59 list_total: 29 fname_total: 35 hit_total: 15008 miss_total: 213 swrong=1 ewrong=1 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 18 seconds llite.lustre-ffffa046060df800.statahead_stats= statahead total: 64 statahead wrong: 1 agl total: 59 list_total: 29 fname_total: 35 hit_total: 15008 miss_total: 213 mdc.lustre-MDT0000-mdc-ffffa046060df800.batch_stats= snapshot_time: 1780328277.138262790 secs.nsecs start_time: 1780328176.450223330 secs.nsecs elapsed_time: 100.688039460 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 3 7 7 8: 3 7 15 16: 20 51 66 32: 13 33 100 mdc.lustre-MDT0001-mdc-ffffa046060df800.batch_stats= snapshot_time: 1780328277.138646023 secs.nsecs start_time: 1780328176.450304964 secs.nsecs elapsed_time: 100.688341059 secs.nsecs subreqs per batch batches % cum % 1: 1 2 2 2: 0 0 2 4: 4 10 12 8: 2 5 17 16: 19 47 65 32: 14 35 100 - open/close 960 (time 1780328289.49 total 10.01 last 95.95) - open/close 1902 (time 1780328299.49 total 20.01 last 94.17) - open/close 2856 (time 1780328309.50 total 30.01 last 95.35) - open/close 3782 (time 1780328319.50 total 40.02 last 92.59) - open/close 4742 (time 1780328329.51 total 50.03 last 95.89) - open/close 5752 (time 1780328339.51 total 60.03 last 100.98) - open/close 6712 (time 1780328349.53 total 70.05 last 95.83) - open/close 7729 (time 1780328359.53 total 80.05 last 101.68) - open/close 8660 (time 1780328369.53 total 90.05 last 93.09) - open/close 9625 (time 1780328379.54 total 100.05 last 96.47) total: 10000 open/close in 104.50 seconds: 95.69 ops/second llite.lustre-ffffa046060df800.statahead_batch_max=0 llite.lustre-ffffa046060df800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffffa046060df800.stats=clear mdc.lustre-MDT0001-mdc-ffffa046060df800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m6.070s user 0m0.272s sys 0m16.025s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffffa046060df800.statahead_batch_max=32 llite.lustre-ffffa046060df800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffffa046060df800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffffa046060df800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffffa046060df800.stats=clear mdc.lustre-MDT0001-mdc-ffffa046060df800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m35.785s user 0m0.254s sys 0m15.005s batched statahead RPC= hit count=9982 mdc.lustre-MDT0000-mdc-ffffa046060df800.batch_stats= snapshot_time: 1780328573.591679554 secs.nsecs start_time: 1780328460.123982871 secs.nsecs elapsed_time: 113.467696683 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffffa046060df800.batch_stats= snapshot_time: 1780328573.591755992 secs.nsecs start_time: 1780328460.124056851 secs.nsecs elapsed_time: 113.467699141 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffffa046060df800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffffa046060df800.batch_stats=0 llite.lustre-ffffa046060df800.statahead_max=2048 llite.lustre-ffffa046060df800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffffa046060df800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffffa046060df800.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.07 seconds: 48.38 ops/second llite.lustre-ffffa046060df800.statahead_max=0 101 real 0m2.009s user 0m0.003s sys 0m0.311s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffffa046060df800.statahead_max=2048 llite.lustre-ffffa046060df800.statahead_max=2048 101 real 0m0.639s user 0m0.004s sys 0m0.186s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10081 miss_total: 19 swrong=0 ewrong=0 total: 900 open/close in 9.17 seconds: 98.10 ops/second llite.lustre-ffffa046060df800.statahead_max=0 1001 real 0m17.696s user 0m0.022s sys 0m2.978s 'ls -l' 1000 files without statahead: 18 sec llite.lustre-ffffa046060df800.statahead_max=2048 llite.lustre-ffffa046060df800.statahead_max=2048 1001 real 0m5.904s user 0m0.021s sys 0m1.515s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 13 seconds llite.lustre-ffffa046060df800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 mdc.lustre-MDT0000-mdc-ffffa046060df800.batch_stats= snapshot_time: 1780328890.346789363 secs.nsecs start_time: 1780328576.094743092 secs.nsecs elapsed_time: 314.252046271 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 2 28 28 8: 2 28 57 16: 0 0 57 32: 0 0 57 64: 1 14 71 128: 0 0 71 256: 2 28 100 mdc.lustre-MDT0001-mdc-ffffa046060df800.batch_stats= snapshot_time: 1780328890.346877291 secs.nsecs start_time: 1780328576.094979506 secs.nsecs elapsed_time: 314.251897785 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 3 42 42 8: 1 14 57 16: 0 0 57 32: 0 0 57 64: 1 14 71 128: 0 0 71 256: 2 28 100 - open/close 946 (time 1780328902.78 total 10.01 last 94.51) - open/close 1931 (time 1780328912.79 total 20.02 last 98.44) - open/close 3058 (time 1780328922.79 total 30.02 last 112.66) - open/close 4103 (time 1780328932.80 total 40.02 last 104.46) - open/close 5083 (time 1780328942.81 total 50.03 last 97.92) - open/close 6024 (time 1780328952.81 total 60.03 last 94.09) - open/close 7058 (time 1780328962.81 total 70.04 last 103.37) - open/close 8067 (time 1780328972.81 total 80.04 last 100.88) - open/close 9010 (time 1780328982.82 total 90.05 last 94.21) total: 10000 open/close in 99.88 seconds: 100.12 ops/second llite.lustre-ffffa046060df800.statahead_batch_max=0 llite.lustre-ffffa046060df800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffffa046060df800.stats=clear mdc.lustre-MDT0001-mdc-ffffa046060df800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m36.034s user 0m0.276s sys 0m21.782s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffffa046060df800.statahead_batch_max=256 llite.lustre-ffffa046060df800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffffa046060df800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffffa046060df800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffffa046060df800.stats=clear mdc.lustre-MDT0001-mdc-ffffa046060df800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m28.863s user 0m0.218s sys 0m20.735s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffffa046060df800.batch_stats= snapshot_time: 1780329070.195132841 secs.nsecs start_time: 1780329034.450952619 secs.nsecs elapsed_time: 35.744180222 secs.nsecs subreqs per batch batches % cum % 1: 333 67 67 2: 58 11 78 4: 37 7 86 8: 7 1 87 16: 3 0 88 32: 21 4 92 64: 4 0 93 128: 0 0 93 256: 34 6 100 mdc.lustre-MDT0001-mdc-ffffa046060df800.batch_stats= snapshot_time: 1780329070.195202895 secs.nsecs start_time: 1780329034.451052016 secs.nsecs elapsed_time: 35.744150879 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffffa046060df800.statahead_batch_max=64 llite.lustre-ffffa046060df800.statahead_max=128