== sanity test 123ad: Verify batching statahead works correctly ========================================================== 21:57:05 (1787104625) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9049bf23f000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9049bf23f000.batch_stats=0 llite.lustre-ffff9049bf23f000.statahead_max=32 llite.lustre-ffff9049bf23f000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9049bf23f000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9049bf23f000.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 0.67 seconds: 148.67 ops/second llite.lustre-ffff9049bf23f000.statahead_max=0 101 real 0m0.693s user 0m0.005s sys 0m0.264s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff9049bf23f000.statahead_max=32 llite.lustre-ffff9049bf23f000.statahead_max=32 101 real 0m0.447s user 0m0.006s sys 0m0.256s 'ls -l' 100 files with statahead: 1 sec statahead total: 67 statahead wrong: 1 agl total: 62 list_total: 28 fname_total: 38 hit_total: 14035 miss_total: 224 swrong=1 ewrong=1 total: 900 open/close in 1.68 seconds: 535.93 ops/second llite.lustre-ffff9049bf23f000.statahead_max=0 1001 real 0m5.544s user 0m0.021s sys 0m2.170s 'ls -l' 1000 files without statahead: 6 sec llite.lustre-ffff9049bf23f000.statahead_max=32 llite.lustre-ffff9049bf23f000.statahead_max=32 1001 real 0m3.478s user 0m0.015s sys 0m1.684s 'ls -l' 1000 files with statahead: 4 sec statahead total: 68 statahead wrong: 1 agl total: 63 list_total: 29 fname_total: 38 hit_total: 15034 miss_total: 225 swrong=1 ewrong=1 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 5 seconds llite.lustre-ffff9049bf23f000.statahead_stats= statahead total: 68 statahead wrong: 1 agl total: 63 list_total: 29 fname_total: 38 hit_total: 15034 miss_total: 225 mdc.lustre-MDT0000-mdc-ffff9049bf23f000.batch_stats= snapshot_time: 1787104650.107920480 secs.nsecs start_time: 1787104626.586571841 secs.nsecs elapsed_time: 23.521348639 secs.nsecs subreqs per batch batches %% cum %% 1: 2 5 5 2: 1 2 8 4: 1 2 10 8: 4 10 21 16: 24 64 86 32: 5 13 100 mdc.lustre-MDT0001-mdc-ffff9049bf23f000.batch_stats= snapshot_time: 1787104650.107964277 secs.nsecs start_time: 1787104626.586647509 secs.nsecs elapsed_time: 23.521316768 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 3 7 7 8: 1 2 10 16: 9 23 33 32: 26 66 100 - open/close 5307 (time 1787104660.86 total 10.00 last 530.63) - open/close 9808 (time 1787104670.86 total 20.00 last 450.02) total: 10000 open/close in 20.44 seconds: 489.29 ops/second llite.lustre-ffff9049bf23f000.statahead_batch_max=0 llite.lustre-ffff9049bf23f000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9049bf23f000.stats=clear mdc.lustre-MDT0001-mdc-ffff9049bf23f000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m36.696s user 0m0.167s sys 0m33.698s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9049bf23f000.statahead_batch_max=32 llite.lustre-ffff9049bf23f000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9049bf23f000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9049bf23f000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9049bf23f000.stats=clear mdc.lustre-MDT0001-mdc-ffff9049bf23f000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m37.009s user 0m0.159s sys 0m21.763s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9049bf23f000.batch_stats= snapshot_time: 1787104754.179633692 secs.nsecs start_time: 1787104710.816143132 secs.nsecs elapsed_time: 43.363490560 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff9049bf23f000.batch_stats= snapshot_time: 1787104754.179689614 secs.nsecs start_time: 1787104710.816218412 secs.nsecs elapsed_time: 43.363471202 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9049bf23f000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9049bf23f000.batch_stats=0 llite.lustre-ffff9049bf23f000.statahead_max=2048 llite.lustre-ffff9049bf23f000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9049bf23f000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9049bf23f000.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 0.60 seconds: 167.40 ops/second llite.lustre-ffff9049bf23f000.statahead_max=0 101 real 0m0.572s user 0m0.002s sys 0m0.227s 'ls -l' 100 files without statahead: 0 sec llite.lustre-ffff9049bf23f000.statahead_max=2048 llite.lustre-ffff9049bf23f000.statahead_max=2048 101 real 0m0.351s user 0m0.002s sys 0m0.240s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 2 swrong=0 ewrong=0 total: 900 open/close in 1.54 seconds: 585.29 ops/second llite.lustre-ffff9049bf23f000.statahead_max=0 1001 real 0m5.861s user 0m0.022s sys 0m2.322s 'ls -l' 1000 files without statahead: 5 sec llite.lustre-ffff9049bf23f000.statahead_max=2048 llite.lustre-ffff9049bf23f000.statahead_max=2048 1001 real 0m3.587s user 0m0.011s sys 0m1.362s 'ls -l' 1000 files with statahead: 3 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 swrong=0 ewrong=0 - open/close 4981 (time 1787104869.08 total 10.00 last 498.04) total: 9000 open/close in 17.33 seconds: 519.36 ops/second llite.lustre-ffff9049bf23f000.statahead_max=0 10001 real 0m57.106s user 0m0.214s sys 0m22.411s 'ls -l' 10000 files without statahead: 57 sec llite.lustre-ffff9049bf23f000.statahead_max=2048 llite.lustre-ffff9049bf23f000.statahead_max=2048 10001 real 0m26.239s user 0m0.143s sys 0m18.735s 'ls -l' 10000 files with statahead: 26 sec statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 21096 miss_total: 4 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 65 seconds llite.lustre-ffff9049bf23f000.statahead_stats= statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 21096 miss_total: 4 mdc.lustre-MDT0000-mdc-ffff9049bf23f000.batch_stats= snapshot_time: 1787105037.208798233 secs.nsecs start_time: 1787104754.992954959 secs.nsecs elapsed_time: 282.215843274 secs.nsecs subreqs per batch batches %% cum %% 1: 119 62 62 2: 23 12 74 4: 10 5 79 8: 6 3 82 16: 0 0 82 32: 0 0 82 64: 1 0 83 128: 6 3 86 256: 26 13 100 mdc.lustre-MDT0001-mdc-ffff9049bf23f000.batch_stats= snapshot_time: 1787105037.208855880 secs.nsecs start_time: 1787104754.993093903 secs.nsecs elapsed_time: 282.215761977 secs.nsecs subreqs per batch batches %% cum %% 1: 138 61 61 2: 29 12 74 4: 18 8 82 8: 2 0 83 16: 1 0 83 32: 0 0 83 64: 2 0 84 128: 2 0 85 256: 32 14 100 - open/close 5546 (time 1787105048.51 total 10.00 last 554.59) total: 10000 open/close in 18.91 seconds: 528.93 ops/second llite.lustre-ffff9049bf23f000.statahead_batch_max=0 llite.lustre-ffff9049bf23f000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9049bf23f000.stats=clear mdc.lustre-MDT0001-mdc-ffff9049bf23f000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m30.792s user 0m0.150s sys 0m26.144s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9049bf23f000.statahead_batch_max=256 llite.lustre-ffff9049bf23f000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9049bf23f000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9049bf23f000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9049bf23f000.stats=clear mdc.lustre-MDT0001-mdc-ffff9049bf23f000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m26.987s user 0m0.154s sys 0m19.599s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9049bf23f000.batch_stats= snapshot_time: 1787105123.760804831 secs.nsecs start_time: 1787105091.068811140 secs.nsecs elapsed_time: 32.691993691 secs.nsecs subreqs per batch batches %% cum %% 1: 268 61 61 2: 60 13 75 4: 38 8 83 8: 9 2 85 16: 3 0 86 32: 22 5 91 64: 3 0 92 128: 0 0 92 256: 34 7 100 mdc.lustre-MDT0001-mdc-ffff9049bf23f000.batch_stats= snapshot_time: 1787105123.760853356 secs.nsecs start_time: 1787105091.068864265 secs.nsecs elapsed_time: 32.691989091 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9049bf23f000.statahead_batch_max=64 llite.lustre-ffff9049bf23f000.statahead_max=128