== sanity test 123ad: Verify batching statahead works correctly ========================================================== 18:59:42 (1787871582) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9ea8909f3800.batch_stats=0 llite.lustre-ffff9ea8909f3800.statahead_max=32 llite.lustre-ffff9ea8909f3800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9ea8909f3800.batch_stats=0 total: 100 open/close in 2.64 seconds: 37.86 ops/second llite.lustre-ffff9ea8909f3800.statahead_max=0 101 real 0m2.660s user 0m0.005s sys 0m0.311s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9ea8909f3800.statahead_max=32 llite.lustre-ffff9ea8909f3800.statahead_max=32 101 real 0m0.983s user 0m0.005s sys 0m0.179s 'ls -l' 100 files with statahead: 1 sec statahead total: 149 statahead wrong: 2 agl total: 142 list_total: 30 fname_total: 107 hit_total: 4598 miss_total: 770 swrong=2 ewrong=2 - open/close 659 (time 1787871610.65 total 10.02 last 65.79) total: 900 open/close in 14.21 seconds: 63.36 ops/second llite.lustre-ffff9ea8909f3800.statahead_max=0 1001 real 0m24.745s user 0m0.031s sys 0m2.837s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff9ea8909f3800.statahead_max=32 llite.lustre-ffff9ea8909f3800.statahead_max=32 1001 real 0m10.166s user 0m0.028s sys 0m1.313s 'ls -l' 1000 files with statahead: 10 sec statahead total: 150 statahead wrong: 2 agl total: 143 list_total: 31 fname_total: 107 hit_total: 5595 miss_total: 773 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff9ea8909f3800.statahead_stats= statahead total: 150 statahead wrong: 2 agl total: 143 list_total: 31 fname_total: 107 hit_total: 5595 miss_total: 773 mdc.lustre-MDT0000-mdc-ffff9ea8909f3800.batch_stats= snapshot_time: 1787871686.969299601 secs.nsecs start_time: 1787871587.282824743 secs.nsecs elapsed_time: 99.686474858 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 641 (time 1787871699.64 total 10.01 last 64.05) - open/close 1286 (time 1787871709.64 total 20.01 last 64.49) - open/close 1918 (time 1787871719.66 total 30.03 last 63.09) - open/close 2523 (time 1787871729.66 total 40.03 last 60.49) - open/close 3067 (time 1787871739.67 total 50.04 last 54.35) - open/close 3675 (time 1787871749.68 total 60.04 last 60.77) - open/close 4320 (time 1787871759.69 total 70.05 last 64.44) - open/close 4933 (time 1787871769.69 total 80.05 last 61.29) - open/close 5550 (time 1787871779.70 total 90.07 last 61.61) - open/close 6116 (time 1787871789.72 total 100.08 last 56.51) - open/close 6740 (time 1787871799.74 total 110.11 last 62.26) - open/close 7343 (time 1787871809.75 total 120.12 last 60.24) - open/close 7977 (time 1787871819.75 total 130.12 last 63.39) - open/close 8570 (time 1787871829.75 total 140.12 last 59.27) - open/close 9181 (time 1787871839.77 total 150.14 last 61.02) - open/close 9780 (time 1787871849.78 total 160.15 last 59.80) total: 10000 open/close in 163.66 seconds: 61.10 ops/second llite.lustre-ffff9ea8909f3800.statahead_batch_max=0 llite.lustre-ffff9ea8909f3800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ea8909f3800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m5.333s user 0m0.320s sys 0m14.170s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9ea8909f3800.statahead_batch_max=32 llite.lustre-ffff9ea8909f3800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ea8909f3800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9ea8909f3800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m49.054s user 0m0.311s sys 0m13.641s batched statahead RPC= hit count=9971 mdc.lustre-MDT0000-mdc-ffff9ea8909f3800.batch_stats= snapshot_time: 1787872048.621292099 secs.nsecs start_time: 1787871927.515226572 secs.nsecs elapsed_time: 121.106065527 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9ea8909f3800.batch_stats=0 llite.lustre-ffff9ea8909f3800.statahead_max=2048 llite.lustre-ffff9ea8909f3800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9ea8909f3800.batch_stats=0 total: 100 open/close in 3.28 seconds: 30.53 ops/second llite.lustre-ffff9ea8909f3800.statahead_max=0 101 real 0m2.554s user 0m0.005s sys 0m0.313s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9ea8909f3800.statahead_max=2048 llite.lustre-ffff9ea8909f3800.statahead_max=2048 101 real 0m1.010s user 0m0.003s sys 0m0.153s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10070 miss_total: 30 swrong=0 ewrong=0 - open/close 651 (time 1787872421.75 total 10.00 last 65.09) total: 900 open/close in 14.24 seconds: 63.19 ops/second llite.lustre-ffff9ea8909f3800.statahead_max=0 1001 real 0m23.976s user 0m0.028s sys 0m3.001s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff9ea8909f3800.statahead_max=2048 llite.lustre-ffff9ea8909f3800.statahead_max=2048 1001 real 0m7.010s user 0m0.029s sys 0m1.400s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11069 miss_total: 31 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff9ea8909f3800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11069 miss_total: 31 mdc.lustre-MDT0000-mdc-ffff9ea8909f3800.batch_stats= snapshot_time: 1787872495.483110433 secs.nsecs start_time: 1787872051.244203002 secs.nsecs elapsed_time: 444.238907431 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 1 9 63 128: 0 0 63 256: 4 36 100 - open/close 643 (time 1787872508.24 total 10.02 last 64.20) - open/close 1242 (time 1787872518.24 total 20.02 last 59.86) - open/close 1867 (time 1787872528.25 total 30.03 last 62.45) - open/close 2465 (time 1787872538.26 total 40.04 last 59.72) - open/close 3025 (time 1787872548.28 total 50.06 last 55.89) - open/close 3608 (time 1787872558.30 total 60.08 last 58.18) - open/close 4177 (time 1787872568.31 total 70.09 last 56.88) - open/close 4685 (time 1787872578.31 total 80.09 last 50.80) - open/close 5313 (time 1787872588.33 total 90.11 last 62.68) - open/close 5942 (time 1787872598.35 total 100.13 last 62.77) - open/close 6371 (time 1787872608.36 total 110.14 last 42.84) - open/close 6967 (time 1787872618.38 total 120.16 last 59.51) - open/close 7594 (time 1787872628.39 total 130.17 last 62.62) - open/close 8183 (time 1787872638.40 total 140.18 last 58.86) - open/close 8753 (time 1787872648.40 total 150.18 last 56.97) - open/close 9345 (time 1787872658.41 total 160.19 last 59.17) - open/close 9941 (time 1787872668.41 total 170.19 last 59.56) total: 10000 open/close in 171.25 seconds: 58.39 ops/second llite.lustre-ffff9ea8909f3800.statahead_batch_max=0 llite.lustre-ffff9ea8909f3800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ea8909f3800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m9.768s user 0m0.270s sys 0m13.974s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9ea8909f3800.statahead_batch_max=256 llite.lustre-ffff9ea8909f3800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ea8909f3800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9ea8909f3800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m47.771s user 0m0.291s sys 0m15.037s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9ea8909f3800.batch_stats= snapshot_time: 1787872808.716553859 secs.nsecs start_time: 1787872749.576059815 secs.nsecs elapsed_time: 59.140494044 secs.nsecs subreqs per batch batches %% cum %% 1: 209 50 50 2: 54 12 63 4: 50 11 75 8: 36 8 83 16: 11 2 86 32: 15 3 89 64: 9 2 92 128: 0 0 92 256: 33 7 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9ea8909f3800.statahead_batch_max=64 llite.lustre-ffff9ea8909f3800.statahead_max=128