== sanity test 123ad: Verify batching statahead works correctly ========================================================== 18:12:13 (1779315133) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8b7b51369000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8b7b51369000.batch_stats=0 llite.lustre-ffff8b7b51369000.statahead_max=32 llite.lustre-ffff8b7b51369000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8b7b51369000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8b7b51369000.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 2.93 seconds: 34.12 ops/second llite.lustre-ffff8b7b51369000.statahead_max=0 101 real 0m2.322s user 0m0.004s sys 0m0.330s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8b7b51369000.statahead_max=32 llite.lustre-ffff8b7b51369000.statahead_max=32 101 real 0m1.069s user 0m0.007s sys 0m0.189s 'ls -l' 100 files with statahead: 1 sec statahead total: 125 statahead wrong: 1 agl total: 119 list_total: 31 fname_total: 91 hit_total: 3788 miss_total: 262 swrong=1 ewrong=1 - open/close 664 (time 1779315161.06 total 10.01 last 66.34) total: 900 open/close in 13.55 seconds: 66.44 ops/second llite.lustre-ffff8b7b51369000.statahead_max=0 1001 real 0m23.693s user 0m0.037s sys 0m3.029s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff8b7b51369000.statahead_max=32 llite.lustre-ffff8b7b51369000.statahead_max=32 1001 real 0m8.549s user 0m0.027s sys 0m1.484s 'ls -l' 1000 files with statahead: 8 sec statahead total: 126 statahead wrong: 1 agl total: 120 list_total: 32 fname_total: 91 hit_total: 4787 miss_total: 263 swrong=1 ewrong=1 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff8b7b51369000.statahead_stats= statahead total: 126 statahead wrong: 1 agl total: 120 list_total: 32 fname_total: 91 hit_total: 4787 miss_total: 263 mdc.lustre-MDT0000-mdc-ffff8b7b51369000.batch_stats= snapshot_time: 1779315230.099311918 secs.nsecs start_time: 1779315138.424746576 secs.nsecs elapsed_time: 91.674565342 secs.nsecs subreqs per batch batches % cum % 1: 1 2 2 2: 2 5 8 4: 2 5 13 8: 2 5 18 16: 29 78 97 32: 1 2 100 mdc.lustre-MDT0001-mdc-ffff8b7b51369000.batch_stats= snapshot_time: 1779315230.099378207 secs.nsecs start_time: 1779315138.424906841 secs.nsecs elapsed_time: 91.674471366 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 1 2 2 4: 1 2 5 8: 2 5 10 16: 8 20 30 32: 27 69 100 - open/close 746 (time 1779315242.44 total 10.01 last 74.56) - open/close 1483 (time 1779315252.44 total 20.01 last 73.67) - open/close 2198 (time 1779315262.45 total 30.02 last 71.45) - open/close 2941 (time 1779315272.46 total 40.02 last 74.25) - open/close 3702 (time 1779315282.46 total 50.03 last 76.08) - open/close 4469 (time 1779315292.47 total 60.03 last 76.64) - open/close 5129 (time 1779315302.47 total 70.04 last 65.95) - open/close 5902 (time 1779315312.49 total 80.06 last 77.17) - open/close 6590 (time 1779315322.50 total 90.07 last 68.71) - open/close 7294 (time 1779315332.51 total 100.07 last 70.39) - open/close 8028 (time 1779315342.51 total 110.08 last 73.36) - open/close 8761 (time 1779315352.53 total 120.10 last 73.18) - open/close 9452 (time 1779315362.53 total 130.10 last 69.06) total: 10000 open/close in 137.63 seconds: 72.66 ops/second llite.lustre-ffff8b7b51369000.statahead_batch_max=0 llite.lustre-ffff8b7b51369000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b7b51369000.stats=clear mdc.lustre-MDT0001-mdc-ffff8b7b51369000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.912s user 0m0.339s sys 0m17.672s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8b7b51369000.statahead_batch_max=32 llite.lustre-ffff8b7b51369000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b7b51369000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8b7b51369000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8b7b51369000.stats=clear mdc.lustre-MDT0001-mdc-ffff8b7b51369000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m29.591s user 0m0.315s sys 0m15.628s batched statahead RPC= hit count=9991 mdc.lustre-MDT0000-mdc-ffff8b7b51369000.batch_stats= snapshot_time: 1779315538.945069029 secs.nsecs start_time: 1779315435.002808541 secs.nsecs elapsed_time: 103.942260488 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff8b7b51369000.batch_stats= snapshot_time: 1779315538.945149142 secs.nsecs start_time: 1779315435.003028456 secs.nsecs elapsed_time: 103.942120686 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8b7b51369000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8b7b51369000.batch_stats=0 llite.lustre-ffff8b7b51369000.statahead_max=2048 llite.lustre-ffff8b7b51369000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8b7b51369000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8b7b51369000.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.58 seconds: 38.79 ops/second llite.lustre-ffff8b7b51369000.statahead_max=0 101 real 0m2.286s user 0m0.007s sys 0m0.326s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8b7b51369000.statahead_max=2048 llite.lustre-ffff8b7b51369000.statahead_max=2048 101 real 0m0.782s user 0m0.005s sys 0m0.193s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10090 miss_total: 10 swrong=0 ewrong=0 - open/close 748 (time 1779315824.14 total 10.02 last 74.68) total: 900 open/close in 11.93 seconds: 75.42 ops/second llite.lustre-ffff8b7b51369000.statahead_max=0 1001 real 0m19.319s user 0m0.032s sys 0m3.069s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff8b7b51369000.statahead_max=2048 llite.lustre-ffff8b7b51369000.statahead_max=2048 1001 real 0m5.163s user 0m0.026s sys 0m1.717s 'ls -l' 1000 files with statahead: 5 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11089 miss_total: 11 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 16 seconds llite.lustre-ffff8b7b51369000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11089 miss_total: 11 mdc.lustre-MDT0000-mdc-ffff8b7b51369000.batch_stats= snapshot_time: 1779315875.743174300 secs.nsecs start_time: 1779315540.868669356 secs.nsecs elapsed_time: 334.874504944 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 1 12 12 4: 1 12 25 8: 3 37 62 16: 0 0 62 32: 0 0 62 64: 1 12 75 128: 0 0 75 256: 2 25 100 mdc.lustre-MDT0001-mdc-ffff8b7b51369000.batch_stats= snapshot_time: 1779315875.743324625 secs.nsecs start_time: 1779315540.868825492 secs.nsecs elapsed_time: 334.874499133 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 1 12 12 4: 1 12 25 8: 2 25 50 16: 1 12 62 32: 1 12 75 64: 0 0 75 128: 0 0 75 256: 2 25 100 - open/close 855 (time 1779315887.42 total 10.00 last 85.47) - open/close 1751 (time 1779315897.42 total 20.01 last 89.58) - open/close 2625 (time 1779315907.44 total 30.02 last 87.27) - open/close 3471 (time 1779315917.45 total 40.03 last 84.51) - open/close 4287 (time 1779315927.46 total 50.04 last 81.54) - open/close 5108 (time 1779315937.47 total 60.05 last 82.03) - open/close 5933 (time 1779315947.47 total 70.05 last 82.44) - open/close 6797 (time 1779315957.48 total 80.06 last 86.36) - open/close 7596 (time 1779315967.48 total 90.06 last 79.88) - open/close 8393 (time 1779315977.50 total 100.08 last 79.57) - open/close 9196 (time 1779315987.50 total 110.09 last 80.23) total: 10000 open/close in 119.93 seconds: 83.38 ops/second llite.lustre-ffff8b7b51369000.statahead_batch_max=0 llite.lustre-ffff8b7b51369000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b7b51369000.stats=clear mdc.lustre-MDT0001-mdc-ffff8b7b51369000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m43.365s user 0m0.326s sys 0m19.795s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8b7b51369000.statahead_batch_max=256 llite.lustre-ffff8b7b51369000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b7b51369000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8b7b51369000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8b7b51369000.stats=clear mdc.lustre-MDT0001-mdc-ffff8b7b51369000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m42.291s user 0m0.275s sys 0m17.860s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8b7b51369000.batch_stats= snapshot_time: 1779316099.311220807 secs.nsecs start_time: 1779316047.632784196 secs.nsecs elapsed_time: 51.678436611 secs.nsecs subreqs per batch batches % cum % 1: 237 58 58 2: 47 11 70 4: 37 9 79 8: 17 4 83 16: 8 1 85 32: 17 4 90 64: 6 1 91 128: 0 0 91 256: 34 8 100 mdc.lustre-MDT0001-mdc-ffff8b7b51369000.batch_stats= snapshot_time: 1779316099.311452452 secs.nsecs start_time: 1779316047.632950485 secs.nsecs elapsed_time: 51.678501967 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8b7b51369000.statahead_batch_max=64 llite.lustre-ffff8b7b51369000.statahead_max=128