== sanity test 123ad: Verify batching statahead works correctly ========================================================== 12:01:57 (1787846517) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8b64ea964000.batch_stats=0 llite.lustre-ffff8b64ea964000.statahead_max=32 llite.lustre-ffff8b64ea964000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8b64ea964000.batch_stats=0 total: 100 open/close in 1.93 seconds: 51.77 ops/second llite.lustre-ffff8b64ea964000.statahead_max=0 101 real 0m2.349s user 0m0.004s sys 0m0.303s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8b64ea964000.statahead_max=32 llite.lustre-ffff8b64ea964000.statahead_max=32 101 real 0m0.989s user 0m0.005s sys 0m0.159s 'ls -l' 100 files with statahead: 1 sec statahead total: 114 statahead wrong: 2 agl total: 109 list_total: 30 fname_total: 67 hit_total: 4159 miss_total: 1288 swrong=2 ewrong=2 - open/close 782 (time 1787846544.37 total 10.01 last 78.16) total: 900 open/close in 11.57 seconds: 77.76 ops/second llite.lustre-ffff8b64ea964000.statahead_max=0 1001 real 0m19.003s user 0m0.036s sys 0m2.802s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff8b64ea964000.statahead_max=32 llite.lustre-ffff8b64ea964000.statahead_max=32 1001 real 0m10.109s user 0m0.026s sys 0m1.330s 'ls -l' 1000 files with statahead: 10 sec statahead total: 115 statahead wrong: 2 agl total: 110 list_total: 31 fname_total: 67 hit_total: 5156 miss_total: 1291 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff8b64ea964000.statahead_stats= statahead total: 115 statahead wrong: 2 agl total: 110 list_total: 31 fname_total: 67 hit_total: 5156 miss_total: 1291 mdc.lustre-MDT0000-mdc-ffff8b64ea964000.batch_stats= snapshot_time: 1787846610.915772605 secs.nsecs start_time: 1787846522.536733946 secs.nsecs elapsed_time: 88.379038659 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 827 (time 1787846623.39 total 10.01 last 82.58) - open/close 1552 (time 1787846633.39 total 20.02 last 72.49) - open/close 2345 (time 1787846643.40 total 30.02 last 79.24) - open/close 2984 (time 1787846653.41 total 40.03 last 63.85) - open/close 3601 (time 1787846663.42 total 50.05 last 61.61) - open/close 4173 (time 1787846673.43 total 60.05 last 57.16) - open/close 4839 (time 1787846683.45 total 70.07 last 66.48) - open/close 5624 (time 1787846693.46 total 80.08 last 78.45) - open/close 6471 (time 1787846703.46 total 90.09 last 84.65) - open/close 7251 (time 1787846713.47 total 100.09 last 77.98) - open/close 8119 (time 1787846723.47 total 110.09 last 86.75) - open/close 8918 (time 1787846733.48 total 120.10 last 79.83) - open/close 9741 (time 1787846743.49 total 130.11 last 82.25) total: 10000 open/close in 133.08 seconds: 75.14 ops/second llite.lustre-ffff8b64ea964000.statahead_batch_max=0 llite.lustre-ffff8b64ea964000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b64ea964000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m6.707s user 0m0.293s sys 0m13.183s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8b64ea964000.statahead_batch_max=32 llite.lustre-ffff8b64ea964000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b64ea964000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8b64ea964000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m45.476s user 0m0.276s sys 0m13.444s batched statahead RPC= hit count=9985 mdc.lustre-MDT0000-mdc-ffff8b64ea964000.batch_stats= snapshot_time: 1787846939.598917575 secs.nsecs start_time: 1787846821.951297241 secs.nsecs elapsed_time: 117.647620334 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8b64ea964000.batch_stats=0 llite.lustre-ffff8b64ea964000.statahead_max=2048 llite.lustre-ffff8b64ea964000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8b64ea964000.batch_stats=0 total: 100 open/close in 2.48 seconds: 40.40 ops/second llite.lustre-ffff8b64ea964000.statahead_max=0 101 real 0m2.679s user 0m0.001s sys 0m0.332s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8b64ea964000.statahead_max=2048 llite.lustre-ffff8b64ea964000.statahead_max=2048 101 real 0m0.771s user 0m0.005s sys 0m0.182s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10084 miss_total: 16 swrong=0 ewrong=0 - open/close 862 (time 1787847247.97 total 10.00 last 86.17) total: 900 open/close in 10.61 seconds: 84.83 ops/second llite.lustre-ffff8b64ea964000.statahead_max=0 1001 real 0m19.114s user 0m0.029s sys 0m2.773s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff8b64ea964000.statahead_max=2048 llite.lustre-ffff8b64ea964000.statahead_max=2048 1001 real 0m5.675s user 0m0.028s sys 0m1.346s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11083 miss_total: 17 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff8b64ea964000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11083 miss_total: 17 mdc.lustre-MDT0000-mdc-ffff8b64ea964000.batch_stats= snapshot_time: 1787847304.825459313 secs.nsecs start_time: 1787846942.214201819 secs.nsecs elapsed_time: 362.611257494 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 1 9 45 64: 1 9 54 128: 1 9 63 256: 4 36 100 - open/close 801 (time 1787847316.95 total 10.01 last 80.03) - open/close 1618 (time 1787847326.96 total 20.01 last 81.66) - open/close 2561 (time 1787847336.96 total 30.01 last 94.30) - open/close 3374 (time 1787847346.97 total 40.02 last 81.25) - open/close 4142 (time 1787847356.97 total 50.03 last 76.74) - open/close 4901 (time 1787847366.99 total 60.04 last 75.81) - open/close 5663 (time 1787847377.00 total 70.05 last 76.12) - open/close 6489 (time 1787847387.00 total 80.05 last 82.56) - open/close 7318 (time 1787847397.00 total 90.06 last 82.86) - open/close 8120 (time 1787847407.01 total 100.07 last 80.15) - open/close 9019 (time 1787847417.01 total 110.07 last 89.87) - open/close 9836 (time 1787847427.02 total 120.07 last 81.67) total: 10000 open/close in 122.13 seconds: 81.88 ops/second llite.lustre-ffff8b64ea964000.statahead_batch_max=0 llite.lustre-ffff8b64ea964000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b64ea964000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.170s user 0m0.305s sys 0m13.707s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8b64ea964000.statahead_batch_max=256 llite.lustre-ffff8b64ea964000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b64ea964000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8b64ea964000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m46.657s user 0m0.270s sys 0m13.770s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8b64ea964000.batch_stats= snapshot_time: 1787847554.428161306 secs.nsecs start_time: 1787847497.718758788 secs.nsecs elapsed_time: 56.709402518 secs.nsecs subreqs per batch batches %% cum %% 1: 190 51 51 2: 47 12 64 4: 36 9 74 8: 24 6 80 16: 16 4 85 32: 15 4 89 64: 5 1 90 128: 1 0 90 256: 34 9 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8b64ea964000.statahead_batch_max=64 llite.lustre-ffff8b64ea964000.statahead_max=128