== sanity test 123ad: Verify batching statahead works correctly ========================================================== 09:49:41 (1788875381) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff92e7b6919000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff92e7b6919000.batch_stats=0 llite.lustre-ffff92e7b6919000.statahead_max=32 llite.lustre-ffff92e7b6919000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff92e7b6919000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff92e7b6919000.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.77 seconds: 36.06 ops/second llite.lustre-ffff92e7b6919000.statahead_max=0 101 real 0m3.157s user 0m0.007s sys 0m0.355s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff92e7b6919000.statahead_max=32 llite.lustre-ffff92e7b6919000.statahead_max=32 101 real 0m1.101s user 0m0.004s sys 0m0.154s 'ls -l' 100 files with statahead: 1 sec statahead total: 128 statahead wrong: 2 agl total: 122 list_total: 31 fname_total: 97 hit_total: 3736 miss_total: 1294 swrong=2 ewrong=2 total: 900 open/close in 8.38 seconds: 107.34 ops/second llite.lustre-ffff92e7b6919000.statahead_max=0 1001 real 0m22.433s user 0m0.037s sys 0m3.166s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff92e7b6919000.statahead_max=32 llite.lustre-ffff92e7b6919000.statahead_max=32 1001 real 0m9.677s user 0m0.035s sys 0m1.513s 'ls -l' 1000 files with statahead: 10 sec statahead total: 129 statahead wrong: 2 agl total: 123 list_total: 32 fname_total: 97 hit_total: 4733 miss_total: 1297 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff92e7b6919000.statahead_stats= statahead total: 129 statahead wrong: 2 agl total: 123 list_total: 32 fname_total: 97 hit_total: 4733 miss_total: 1297 mdc.lustre-MDT0000-mdc-ffff92e7b6919000.batch_stats= snapshot_time: 1788875473.007642228 secs.nsecs start_time: 1788875387.900101652 secs.nsecs elapsed_time: 85.107540576 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 4 10 10 8: 2 5 15 16: 21 52 67 32: 13 32 100 mdc.lustre-MDT0001-mdc-ffff92e7b6919000.batch_stats= snapshot_time: 1788875473.007706580 secs.nsecs start_time: 1788875387.900175496 secs.nsecs elapsed_time: 85.107531084 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 1 2 5 4: 1 2 7 8: 3 7 15 16: 17 43 58 32: 16 41 100 - open/close 859 (time 1788875485.76 total 10.00 last 85.89) - open/close 1659 (time 1788875495.76 total 20.01 last 79.95) - open/close 2550 (time 1788875505.77 total 30.01 last 89.06) - open/close 3493 (time 1788875515.78 total 40.02 last 94.22) - open/close 4450 (time 1788875525.78 total 50.02 last 95.68) - open/close 5313 (time 1788875535.79 total 60.03 last 86.21) - open/close 6364 (time 1788875545.79 total 70.04 last 105.06) - open/close 7399 (time 1788875555.79 total 80.04 last 103.50) - open/close 8396 (time 1788875565.80 total 90.05 last 99.61) - open/close 9379 (time 1788875575.81 total 100.05 last 98.24) total: 10000 open/close in 106.56 seconds: 93.85 ops/second llite.lustre-ffff92e7b6919000.statahead_batch_max=0 llite.lustre-ffff92e7b6919000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92e7b6919000.stats=clear mdc.lustre-MDT0001-mdc-ffff92e7b6919000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.616s user 0m0.309s sys 0m15.633s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff92e7b6919000.statahead_batch_max=32 llite.lustre-ffff92e7b6919000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92e7b6919000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff92e7b6919000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff92e7b6919000.stats=clear mdc.lustre-MDT0001-mdc-ffff92e7b6919000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m33.058s user 0m0.273s sys 0m14.744s batched statahead RPC= hit count=9983 mdc.lustre-MDT0000-mdc-ffff92e7b6919000.batch_stats= snapshot_time: 1788875754.893157525 secs.nsecs start_time: 1788875646.821685649 secs.nsecs elapsed_time: 108.071471876 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff92e7b6919000.batch_stats= snapshot_time: 1788875754.893307584 secs.nsecs start_time: 1788875646.821771282 secs.nsecs elapsed_time: 108.071536302 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff92e7b6919000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff92e7b6919000.batch_stats=0 llite.lustre-ffff92e7b6919000.statahead_max=2048 llite.lustre-ffff92e7b6919000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff92e7b6919000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff92e7b6919000.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 1.97 seconds: 50.65 ops/second llite.lustre-ffff92e7b6919000.statahead_max=0 101 real 0m1.489s user 0m0.005s sys 0m0.311s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff92e7b6919000.statahead_max=2048 llite.lustre-ffff92e7b6919000.statahead_max=2048 101 real 0m0.839s user 0m0.003s sys 0m0.186s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10082 miss_total: 18 swrong=0 ewrong=0 total: 900 open/close in 8.77 seconds: 102.60 ops/second llite.lustre-ffff92e7b6919000.statahead_max=0 1001 real 0m17.050s user 0m0.036s sys 0m2.964s 'ls -l' 1000 files without statahead: 17 sec llite.lustre-ffff92e7b6919000.statahead_max=2048 llite.lustre-ffff92e7b6919000.statahead_max=2048 1001 real 0m5.355s user 0m0.028s sys 0m1.530s 'ls -l' 1000 files with statahead: 5 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 16 seconds llite.lustre-ffff92e7b6919000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 mdc.lustre-MDT0000-mdc-ffff92e7b6919000.batch_stats= snapshot_time: 1788876041.370407138 secs.nsecs start_time: 1788875756.708345985 secs.nsecs elapsed_time: 284.662061153 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 2 25 25 8: 2 25 50 16: 1 12 62 32: 0 0 62 64: 1 12 75 128: 0 0 75 256: 2 25 100 mdc.lustre-MDT0001-mdc-ffff92e7b6919000.batch_stats= snapshot_time: 1788876041.370485997 secs.nsecs start_time: 1788875756.708478787 secs.nsecs elapsed_time: 284.662007210 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 12 12 4: 1 12 25 8: 3 37 62 16: 0 0 62 32: 0 0 62 64: 1 12 75 128: 0 0 75 256: 2 25 100 - open/close 1086 (time 1788876053.46 total 10.00 last 108.56) - open/close 2080 (time 1788876063.46 total 20.01 last 99.37) - open/close 3143 (time 1788876073.47 total 30.01 last 106.23) - open/close 4152 (time 1788876083.47 total 40.01 last 100.89) - open/close 5264 (time 1788876093.48 total 50.02 last 111.11) - open/close 6349 (time 1788876103.49 total 60.03 last 108.40) - open/close 7397 (time 1788876113.49 total 70.03 last 104.78) - open/close 8717 (time 1788876123.49 total 80.04 last 131.94) - open/close 9794 (time 1788876133.52 total 90.07 last 107.41) total: 10000 open/close in 91.98 seconds: 108.72 ops/second llite.lustre-ffff92e7b6919000.statahead_batch_max=0 llite.lustre-ffff92e7b6919000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92e7b6919000.stats=clear mdc.lustre-MDT0001-mdc-ffff92e7b6919000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m45.996s user 0m0.276s sys 0m16.619s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff92e7b6919000.statahead_batch_max=256 llite.lustre-ffff92e7b6919000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92e7b6919000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff92e7b6919000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff92e7b6919000.stats=clear mdc.lustre-MDT0001-mdc-ffff92e7b6919000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m41.550s user 0m0.252s sys 0m16.557s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff92e7b6919000.batch_stats= snapshot_time: 1788876241.960591184 secs.nsecs start_time: 1788876189.199774756 secs.nsecs elapsed_time: 52.760816428 secs.nsecs subreqs per batch batches %% cum %% 1: 230 48 48 2: 77 16 64 4: 64 13 77 8: 36 7 85 16: 15 3 88 32: 18 3 92 64: 4 0 93 128: 0 0 93 256: 33 6 100 mdc.lustre-MDT0001-mdc-ffff92e7b6919000.batch_stats= snapshot_time: 1788876241.960657784 secs.nsecs start_time: 1788876189.199878004 secs.nsecs elapsed_time: 52.760779780 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff92e7b6919000.statahead_batch_max=64 llite.lustre-ffff92e7b6919000.statahead_max=128