== sanity test 123ad: Verify batching statahead works correctly ========================================================== 11:54:05 (1787500445) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff928786986800.batch_stats=0 llite.lustre-ffff928786986800.statahead_max=32 llite.lustre-ffff928786986800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff928786986800.batch_stats=0 total: 100 open/close in 2.39 seconds: 41.88 ops/second llite.lustre-ffff928786986800.statahead_max=0 101 real 0m1.791s user 0m0.008s sys 0m0.273s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff928786986800.statahead_max=32 llite.lustre-ffff928786986800.statahead_max=32 101 real 0m1.075s user 0m0.003s sys 0m0.133s 'ls -l' 100 files with statahead: 1 sec statahead total: 122 statahead wrong: 2 agl total: 117 list_total: 29 fname_total: 80 hit_total: 3843 miss_total: 1732 swrong=2 ewrong=2 - open/close 732 (time 1787500470.45 total 10.00 last 73.20) total: 900 open/close in 12.06 seconds: 74.64 ops/second llite.lustre-ffff928786986800.statahead_max=0 1001 real 0m20.090s user 0m0.029s sys 0m2.706s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff928786986800.statahead_max=32 llite.lustre-ffff928786986800.statahead_max=32 1001 real 0m8.515s user 0m0.021s sys 0m1.347s 'ls -l' 1000 files with statahead: 8 sec statahead total: 123 statahead wrong: 2 agl total: 118 list_total: 30 fname_total: 80 hit_total: 4840 miss_total: 1735 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff928786986800.statahead_stats= statahead total: 123 statahead wrong: 2 agl total: 118 list_total: 30 fname_total: 80 hit_total: 4840 miss_total: 1735 mdc.lustre-MDT0000-mdc-ffff928786986800.batch_stats= snapshot_time: 1787500532.652749369 secs.nsecs start_time: 1787500448.875036367 secs.nsecs elapsed_time: 83.777713002 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 860 (time 1787500545.26 total 10.00 last 86.00) - open/close 1692 (time 1787500555.27 total 20.01 last 83.16) - open/close 2524 (time 1787500565.27 total 30.01 last 83.17) - open/close 3379 (time 1787500575.28 total 40.01 last 85.46) - open/close 4228 (time 1787500585.29 total 50.02 last 84.81) - open/close 5100 (time 1787500595.29 total 60.02 last 87.20) - open/close 5991 (time 1787500605.29 total 70.03 last 89.08) - open/close 6828 (time 1787500615.29 total 80.03 last 83.68) - open/close 7650 (time 1787500625.30 total 90.04 last 82.12) - open/close 8470 (time 1787500635.30 total 100.04 last 82.00) - open/close 9294 (time 1787500645.30 total 110.04 last 82.36) total: 10000 open/close in 119.71 seconds: 83.54 ops/second llite.lustre-ffff928786986800.statahead_batch_max=0 llite.lustre-ffff928786986800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff928786986800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.308s user 0m0.279s sys 0m15.189s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff928786986800.statahead_batch_max=32 llite.lustre-ffff928786986800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff928786986800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff928786986800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m40.749s user 0m0.252s sys 0m13.066s batched statahead RPC= hit count=9984 mdc.lustre-MDT0000-mdc-ffff928786986800.batch_stats= snapshot_time: 1787500828.128003893 secs.nsecs start_time: 1787500718.148138613 secs.nsecs elapsed_time: 109.979865280 secs.nsecs subreqs per batch batches %% cum %% 1: 1 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff928786986800.batch_stats=0 llite.lustre-ffff928786986800.statahead_max=2048 llite.lustre-ffff928786986800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff928786986800.batch_stats=0 total: 100 open/close in 3.32 seconds: 30.15 ops/second llite.lustre-ffff928786986800.statahead_max=0 101 real 0m2.209s user 0m0.004s sys 0m0.297s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff928786986800.statahead_max=2048 llite.lustre-ffff928786986800.statahead_max=2048 101 real 0m0.812s user 0m0.004s sys 0m0.165s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10083 miss_total: 17 swrong=0 ewrong=0 - open/close 761 (time 1787501124.52 total 10.01 last 76.02) total: 900 open/close in 11.94 seconds: 75.40 ops/second llite.lustre-ffff928786986800.statahead_max=0 1001 real 0m21.942s user 0m0.033s sys 0m2.703s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff928786986800.statahead_max=2048 llite.lustre-ffff928786986800.statahead_max=2048 1001 real 0m7.011s user 0m0.020s sys 0m1.344s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff928786986800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 mdc.lustre-MDT0000-mdc-ffff928786986800.batch_stats= snapshot_time: 1787501189.279799298 secs.nsecs start_time: 1787500830.383648592 secs.nsecs elapsed_time: 358.896150706 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 734 (time 1787501201.88 total 10.01 last 73.36) - open/close 1465 (time 1787501211.89 total 20.02 last 72.98) - open/close 2095 (time 1787501221.90 total 30.03 last 62.94) - open/close 2769 (time 1787501231.92 total 40.05 last 67.29) - open/close 3362 (time 1787501241.94 total 50.07 last 59.17) - open/close 4042 (time 1787501251.97 total 60.10 last 67.81) - open/close 4738 (time 1787501261.98 total 70.11 last 69.53) - open/close 5524 (time 1787501271.99 total 80.11 last 78.55) - open/close 6312 (time 1787501281.99 total 90.12 last 78.79) - open/close 7041 (time 1787501291.99 total 100.12 last 72.89) - open/close 7774 (time 1787501302.00 total 110.13 last 73.23) - open/close 8546 (time 1787501312.00 total 120.13 last 77.20) - open/close 9350 (time 1787501322.01 total 130.13 last 80.34) total: 10000 open/close in 139.03 seconds: 71.93 ops/second llite.lustre-ffff928786986800.statahead_batch_max=0 llite.lustre-ffff928786986800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff928786986800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.233s user 0m0.306s sys 0m14.332s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff928786986800.statahead_batch_max=256 llite.lustre-ffff928786986800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff928786986800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff928786986800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m47.548s user 0m0.295s sys 0m14.477s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff928786986800.batch_stats= snapshot_time: 1787501456.123105938 secs.nsecs start_time: 1787501397.827329363 secs.nsecs elapsed_time: 58.295776575 secs.nsecs subreqs per batch batches %% cum %% 1: 119 49 49 2: 24 9 59 4: 18 7 66 8: 13 5 71 16: 10 4 76 32: 14 5 81 64: 9 3 85 128: 0 0 85 256: 35 14 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff928786986800.statahead_batch_max=64 llite.lustre-ffff928786986800.statahead_max=128