== sanity test 123ad: Verify batching statahead works correctly ========================================================== 14:24:15 (1788287055) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9ae03fbfd800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9ae03fbfd800.batch_stats=0 llite.lustre-ffff9ae03fbfd800.statahead_max=32 llite.lustre-ffff9ae03fbfd800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9ae03fbfd800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9ae03fbfd800.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.06 seconds: 48.66 ops/second llite.lustre-ffff9ae03fbfd800.statahead_max=0 101 real 0m2.167s user 0m0.008s sys 0m0.297s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9ae03fbfd800.statahead_max=32 llite.lustre-ffff9ae03fbfd800.statahead_max=32 101 real 0m0.884s user 0m0.004s sys 0m0.135s 'ls -l' 100 files with statahead: 1 sec statahead total: 137 statahead wrong: 2 agl total: 131 list_total: 30 fname_total: 98 hit_total: 3990 miss_total: 1381 swrong=2 ewrong=2 total: 900 open/close in 9.34 seconds: 96.40 ops/second llite.lustre-ffff9ae03fbfd800.statahead_max=0 1001 real 0m20.049s user 0m0.036s sys 0m2.638s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff9ae03fbfd800.statahead_max=32 llite.lustre-ffff9ae03fbfd800.statahead_max=32 1001 real 0m10.998s user 0m0.031s sys 0m1.421s 'ls -l' 1000 files with statahead: 11 sec statahead total: 138 statahead wrong: 2 agl total: 132 list_total: 31 fname_total: 98 hit_total: 4983 miss_total: 1388 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff9ae03fbfd800.statahead_stats= statahead total: 138 statahead wrong: 2 agl total: 132 list_total: 31 fname_total: 98 hit_total: 4983 miss_total: 1388 mdc.lustre-MDT0000-mdc-ffff9ae03fbfd800.batch_stats= snapshot_time: 1788287146.773447682 secs.nsecs start_time: 1788287061.164473617 secs.nsecs elapsed_time: 85.608974065 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 2 5 7 8: 4 10 17 16: 21 52 70 32: 12 30 100 mdc.lustre-MDT0001-mdc-ffff9ae03fbfd800.batch_stats= snapshot_time: 1788287146.773520095 secs.nsecs start_time: 1788287061.164600612 secs.nsecs elapsed_time: 85.608919483 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 2 2 4: 4 10 12 8: 2 5 17 16: 23 57 75 32: 10 25 100 - open/close 858 (time 1788287159.41 total 10.00 last 85.78) - open/close 1764 (time 1788287169.42 total 20.01 last 90.52) - open/close 2700 (time 1788287179.43 total 30.02 last 93.53) - open/close 3507 (time 1788287189.43 total 40.02 last 80.69) - open/close 4301 (time 1788287199.43 total 50.02 last 79.39) - open/close 5199 (time 1788287209.44 total 60.03 last 89.75) - open/close 6160 (time 1788287219.44 total 70.03 last 96.06) - open/close 7060 (time 1788287229.44 total 80.03 last 90.00) - open/close 8011 (time 1788287239.44 total 90.03 last 95.09) - open/close 8886 (time 1788287249.45 total 100.04 last 87.45) - open/close 9882 (time 1788287259.45 total 110.04 last 99.57) total: 10000 open/close in 111.36 seconds: 89.80 ops/second llite.lustre-ffff9ae03fbfd800.statahead_batch_max=0 llite.lustre-ffff9ae03fbfd800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ae03fbfd800.stats=clear mdc.lustre-MDT0001-mdc-ffff9ae03fbfd800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m6.747s user 0m0.318s sys 0m13.950s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9ae03fbfd800.statahead_batch_max=32 llite.lustre-ffff9ae03fbfd800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ae03fbfd800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9ae03fbfd800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9ae03fbfd800.stats=clear mdc.lustre-MDT0001-mdc-ffff9ae03fbfd800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m46.756s user 0m0.335s sys 0m13.749s batched statahead RPC= hit count=9972 mdc.lustre-MDT0000-mdc-ffff9ae03fbfd800.batch_stats= snapshot_time: 1788287457.145516978 secs.nsecs start_time: 1788287337.207724550 secs.nsecs elapsed_time: 119.937792428 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff9ae03fbfd800.batch_stats= snapshot_time: 1788287457.145592986 secs.nsecs start_time: 1788287337.207844666 secs.nsecs elapsed_time: 119.937748320 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9ae03fbfd800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9ae03fbfd800.batch_stats=0 llite.lustre-ffff9ae03fbfd800.statahead_max=2048 llite.lustre-ffff9ae03fbfd800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9ae03fbfd800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9ae03fbfd800.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 1.86 seconds: 53.77 ops/second llite.lustre-ffff9ae03fbfd800.statahead_max=0 101 real 0m1.538s user 0m0.004s sys 0m0.291s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff9ae03fbfd800.statahead_max=2048 llite.lustre-ffff9ae03fbfd800.statahead_max=2048 101 real 0m0.831s user 0m0.003s sys 0m0.185s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10071 miss_total: 29 swrong=0 ewrong=0 - open/close 800 (time 1788287737.51 total 10.00 last 79.99) total: 900 open/close in 11.82 seconds: 76.14 ops/second llite.lustre-ffff9ae03fbfd800.statahead_max=0 1001 real 0m21.588s user 0m0.035s sys 0m2.704s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff9ae03fbfd800.statahead_max=2048 llite.lustre-ffff9ae03fbfd800.statahead_max=2048 1001 real 0m5.209s user 0m0.027s sys 0m1.343s 'ls -l' 1000 files with statahead: 5 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11070 miss_total: 30 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff9ae03fbfd800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11070 miss_total: 30 mdc.lustre-MDT0000-mdc-ffff9ae03fbfd800.batch_stats= snapshot_time: 1788287798.438176532 secs.nsecs start_time: 1788287460.085848520 secs.nsecs elapsed_time: 338.352328012 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 12 12 8: 3 37 50 16: 1 12 62 32: 1 12 75 64: 0 0 75 128: 0 0 75 256: 2 25 100 mdc.lustre-MDT0001-mdc-ffff9ae03fbfd800.batch_stats= snapshot_time: 1788287798.438245078 secs.nsecs start_time: 1788287460.086030996 secs.nsecs elapsed_time: 338.352214082 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 12 12 4: 3 37 50 8: 0 0 50 16: 0 0 50 32: 2 25 75 64: 0 0 75 128: 0 0 75 256: 2 25 100 - open/close 974 (time 1788287811.55 total 10.00 last 97.35) - open/close 1758 (time 1788287821.55 total 20.01 last 78.36) - open/close 2709 (time 1788287831.56 total 30.01 last 95.07) - open/close 3639 (time 1788287841.57 total 40.02 last 92.92) - open/close 4518 (time 1788287851.58 total 50.03 last 87.80) - open/close 5402 (time 1788287861.58 total 60.04 last 88.34) - open/close 6296 (time 1788287871.59 total 70.05 last 89.35) - open/close 7163 (time 1788287881.60 total 80.05 last 86.64) - open/close 8016 (time 1788287891.61 total 90.06 last 85.21) - open/close 8951 (time 1788287901.61 total 100.07 last 93.47) - open/close 9752 (time 1788287911.62 total 110.07 last 80.05) total: 10000 open/close in 114.41 seconds: 87.40 ops/second llite.lustre-ffff9ae03fbfd800.statahead_batch_max=0 llite.lustre-ffff9ae03fbfd800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ae03fbfd800.stats=clear mdc.lustre-MDT0001-mdc-ffff9ae03fbfd800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m8.985s user 0m0.314s sys 0m13.580s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9ae03fbfd800.statahead_batch_max=256 llite.lustre-ffff9ae03fbfd800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ae03fbfd800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9ae03fbfd800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9ae03fbfd800.stats=clear mdc.lustre-MDT0001-mdc-ffff9ae03fbfd800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m47.416s user 0m0.311s sys 0m13.686s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9ae03fbfd800.batch_stats= snapshot_time: 1788288054.152054313 secs.nsecs start_time: 1788287995.321363824 secs.nsecs elapsed_time: 58.830690489 secs.nsecs subreqs per batch batches %% cum %% 1: 102 40 40 2: 35 13 54 4: 24 9 64 8: 20 7 72 16: 12 4 76 32: 16 6 83 64: 6 2 85 128: 2 0 86 256: 34 13 100 mdc.lustre-MDT0001-mdc-ffff9ae03fbfd800.batch_stats= snapshot_time: 1788288054.152129349 secs.nsecs start_time: 1788287995.321439946 secs.nsecs elapsed_time: 58.830689403 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9ae03fbfd800.statahead_batch_max=64 llite.lustre-ffff9ae03fbfd800.statahead_max=128