== sanity test 123ad: Verify batching statahead works correctly ========================================================== 07:38:25 (1787744305) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff924f4cf24000.batch_stats=0 llite.lustre-ffff924f4cf24000.statahead_max=32 llite.lustre-ffff924f4cf24000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff924f4cf24000.batch_stats=0 total: 100 open/close in 4.06 seconds: 24.61 ops/second llite.lustre-ffff924f4cf24000.statahead_max=0 101 real 0m4.930s user 0m0.006s sys 0m0.314s 'ls -l' 100 files without statahead: 5 sec llite.lustre-ffff924f4cf24000.statahead_max=32 llite.lustre-ffff924f4cf24000.statahead_max=32 101 real 0m1.697s user 0m0.003s sys 0m0.155s 'ls -l' 100 files with statahead: 1 sec statahead total: 146 statahead wrong: 2 agl total: 140 list_total: 31 fname_total: 115 hit_total: 4473 miss_total: 1275 swrong=2 ewrong=2 - open/close 466 (time 1787744343.01 total 10.01 last 46.54) total: 900 open/close in 17.56 seconds: 51.25 ops/second llite.lustre-ffff924f4cf24000.statahead_max=0 1001 real 0m21.185s user 0m0.026s sys 0m2.769s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff924f4cf24000.statahead_max=32 llite.lustre-ffff924f4cf24000.statahead_max=32 1001 real 0m13.306s user 0m0.029s sys 0m1.409s 'ls -l' 1000 files with statahead: 13 sec statahead total: 147 statahead wrong: 2 agl total: 141 list_total: 32 fname_total: 115 hit_total: 5467 miss_total: 1281 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff924f4cf24000.statahead_stats= statahead total: 147 statahead wrong: 2 agl total: 141 list_total: 32 fname_total: 115 hit_total: 5467 miss_total: 1281 mdc.lustre-MDT0000-mdc-ffff924f4cf24000.batch_stats= snapshot_time: 1787744424.950141519 secs.nsecs start_time: 1787744312.125119898 secs.nsecs elapsed_time: 112.825021621 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 2 5 7 16: 3 7 15 32: 34 85 100 - open/close 497 (time 1787744438.14 total 10.01 last 49.67) - open/close 914 (time 1787744448.15 total 20.02 last 41.64) - open/close 1537 (time 1787744458.16 total 30.03 last 62.23) - open/close 2017 (time 1787744468.18 total 40.05 last 47.92) - open/close 2612 (time 1787744478.19 total 50.06 last 59.41) - open/close 3321 (time 1787744488.21 total 60.08 last 70.81) - open/close 3846 (time 1787744498.22 total 70.09 last 52.41) - open/close 4403 (time 1787744508.23 total 80.10 last 55.66) - open/close 5045 (time 1787744518.24 total 90.11 last 64.14) - open/close 5693 (time 1787744528.25 total 100.12 last 64.75) - open/close 6421 (time 1787744538.25 total 110.12 last 72.75) - open/close 7204 (time 1787744548.30 total 120.17 last 77.94) - open/close 8003 (time 1787744558.31 total 130.18 last 79.78) - open/close 8802 (time 1787744568.33 total 140.20 last 79.81) - open/close 9533 (time 1787744578.33 total 150.20 last 73.06) total: 10000 open/close in 159.74 seconds: 62.60 ops/second llite.lustre-ffff924f4cf24000.statahead_batch_max=0 llite.lustre-ffff924f4cf24000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff924f4cf24000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m7.504s user 0m0.323s sys 0m13.770s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff924f4cf24000.statahead_batch_max=32 llite.lustre-ffff924f4cf24000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff924f4cf24000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff924f4cf24000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m52.214s user 0m0.326s sys 0m12.952s batched statahead RPC= hit count=9980 mdc.lustre-MDT0000-mdc-ffff924f4cf24000.batch_stats= snapshot_time: 1787744792.620261208 secs.nsecs start_time: 1787744668.218232967 secs.nsecs elapsed_time: 124.402028241 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff924f4cf24000.batch_stats=0 llite.lustre-ffff924f4cf24000.statahead_max=2048 llite.lustre-ffff924f4cf24000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff924f4cf24000.batch_stats=0 total: 100 open/close in 2.64 seconds: 37.94 ops/second llite.lustre-ffff924f4cf24000.statahead_max=0 101 real 0m2.671s user 0m0.003s sys 0m0.290s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff924f4cf24000.statahead_max=2048 llite.lustre-ffff924f4cf24000.statahead_max=2048 101 real 0m0.811s user 0m0.003s sys 0m0.162s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10079 miss_total: 21 swrong=0 ewrong=0 - open/close 716 (time 1787745124.96 total 10.01 last 71.52) total: 900 open/close in 13.06 seconds: 68.90 ops/second llite.lustre-ffff924f4cf24000.statahead_max=0 1001 real 0m23.439s user 0m0.031s sys 0m2.797s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff924f4cf24000.statahead_max=2048 llite.lustre-ffff924f4cf24000.statahead_max=2048 1001 real 0m7.942s user 0m0.028s sys 0m1.383s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 28 seconds llite.lustre-ffff924f4cf24000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 mdc.lustre-MDT0000-mdc-ffff924f4cf24000.batch_stats= snapshot_time: 1787745204.225478519 secs.nsecs start_time: 1787744795.669567251 secs.nsecs elapsed_time: 408.555911268 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 22 22 16: 1 11 33 32: 1 11 44 64: 0 0 44 128: 1 11 55 256: 4 44 100 - open/close 759 (time 1787745217.24 total 10.02 last 75.78) - open/close 1479 (time 1787745227.24 total 20.02 last 71.99) - open/close 2142 (time 1787745237.24 total 30.02 last 66.29) - open/close 2738 (time 1787745247.24 total 40.02 last 59.59) - open/close 3155 (time 1787745257.25 total 50.03 last 41.68) - open/close 3795 (time 1787745267.25 total 60.04 last 63.95) - open/close 4540 (time 1787745277.26 total 70.04 last 74.45) - open/close 5297 (time 1787745287.27 total 80.05 last 75.63) - open/close 5998 (time 1787745297.28 total 90.06 last 70.00) - open/close 6613 (time 1787745307.29 total 100.07 last 61.48) - open/close 7235 (time 1787745317.30 total 110.08 last 62.13) - open/close 7793 (time 1787745327.31 total 120.09 last 55.75) - open/close 8500 (time 1787745337.31 total 130.09 last 70.65) - open/close 9294 (time 1787745347.32 total 140.11 last 79.31) total: 10000 open/close in 148.81 seconds: 67.20 ops/second llite.lustre-ffff924f4cf24000.statahead_batch_max=0 llite.lustre-ffff924f4cf24000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff924f4cf24000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m2.902s user 0m0.320s sys 0m13.741s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff924f4cf24000.statahead_batch_max=256 llite.lustre-ffff924f4cf24000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff924f4cf24000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff924f4cf24000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m53.592s user 0m0.275s sys 0m12.844s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff924f4cf24000.batch_stats= snapshot_time: 1787745492.878172026 secs.nsecs start_time: 1787745429.044652345 secs.nsecs elapsed_time: 63.833519681 secs.nsecs subreqs per batch batches %% cum %% 1: 143 47 47 2: 47 15 63 4: 32 10 73 8: 11 3 77 16: 12 3 81 32: 18 5 87 64: 2 0 88 128: 1 0 88 256: 35 11 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff924f4cf24000.statahead_batch_max=64 llite.lustre-ffff924f4cf24000.statahead_max=128