== sanity test 123ad: Verify batching statahead works correctly ========================================================== 01:56:13 (1781243773) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff899147f67000.batch_stats=0 llite.lustre-ffff899147f67000.statahead_max=32 llite.lustre-ffff899147f67000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff899147f67000.batch_stats=0 total: 100 open/close in 2.86 seconds: 34.97 ops/second llite.lustre-ffff899147f67000.statahead_max=0 101 real 0m1.974s user 0m0.003s sys 0m0.335s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff899147f67000.statahead_max=32 llite.lustre-ffff899147f67000.statahead_max=32 101 real 0m1.086s user 0m0.006s sys 0m0.162s 'ls -l' 100 files with statahead: 1 sec statahead total: 133 statahead wrong: 2 agl total: 127 list_total: 30 fname_total: 85 hit_total: 3980 miss_total: 1326 swrong=2 ewrong=2 - open/close 726 (time 1781243800.51 total 10.01 last 72.55) total: 900 open/close in 13.12 seconds: 68.59 ops/second llite.lustre-ffff899147f67000.statahead_max=0 1001 real 0m22.671s user 0m0.028s sys 0m3.018s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff899147f67000.statahead_max=32 llite.lustre-ffff899147f67000.statahead_max=32 1001 real 0m10.101s user 0m0.034s sys 0m1.469s 'ls -l' 1000 files with statahead: 10 sec statahead total: 134 statahead wrong: 2 agl total: 128 list_total: 31 fname_total: 85 hit_total: 4977 miss_total: 1329 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff899147f67000.statahead_stats= statahead total: 134 statahead wrong: 2 agl total: 128 list_total: 31 fname_total: 85 hit_total: 4977 miss_total: 1329 mdc.lustre-MDT0000-mdc-ffff899147f67000.batch_stats= snapshot_time: 1781243869.516336753 secs.nsecs start_time: 1781243778.165809005 secs.nsecs elapsed_time: 91.350527748 secs.nsecs subreqs per batch batches % cum % 1: 1 2 2 2: 2 4 7 4: 0 0 7 8: 2 4 12 16: 2 4 17 32: 34 82 100 - open/close 768 (time 1781243881.64 total 10.01 last 76.76) - open/close 1538 (time 1781243891.66 total 20.02 last 76.87) - open/close 2283 (time 1781243901.66 total 30.02 last 74.48) - open/close 3048 (time 1781243911.66 total 40.03 last 76.46) - open/close 3819 (time 1781243921.67 total 50.04 last 77.03) - open/close 4531 (time 1781243931.68 total 60.05 last 71.14) - open/close 5320 (time 1781243941.69 total 70.05 last 78.86) - open/close 6103 (time 1781243951.69 total 80.06 last 78.26) - open/close 6882 (time 1781243961.69 total 90.06 last 77.88) - open/close 7644 (time 1781243971.72 total 100.09 last 76.01) - open/close 8361 (time 1781243981.73 total 110.10 last 71.61) - open/close 9084 (time 1781243991.73 total 120.10 last 72.28) - open/close 9803 (time 1781244001.74 total 130.11 last 71.83) total: 10000 open/close in 132.91 seconds: 75.24 ops/second llite.lustre-ffff899147f67000.statahead_batch_max=0 llite.lustre-ffff899147f67000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff899147f67000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m3.725s user 0m0.328s sys 0m16.227s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff899147f67000.statahead_batch_max=32 llite.lustre-ffff899147f67000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff899147f67000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff899147f67000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m42.670s user 0m0.326s sys 0m15.035s batched statahead RPC= hit count=9984 mdc.lustre-MDT0000-mdc-ffff899147f67000.batch_stats= snapshot_time: 1781244197.763764744 secs.nsecs start_time: 1781244078.775142753 secs.nsecs elapsed_time: 118.988621991 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff899147f67000.batch_stats=0 llite.lustre-ffff899147f67000.statahead_max=2048 llite.lustre-ffff899147f67000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff899147f67000.batch_stats=0 total: 100 open/close in 2.96 seconds: 33.77 ops/second llite.lustre-ffff899147f67000.statahead_max=0 101 real 0m2.400s user 0m0.005s sys 0m0.334s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff899147f67000.statahead_max=2048 llite.lustre-ffff899147f67000.statahead_max=2048 101 real 0m0.912s user 0m0.008s sys 0m0.158s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10083 miss_total: 17 swrong=0 ewrong=0 - open/close 791 (time 1781244520.75 total 10.02 last 78.94) total: 900 open/close in 11.46 seconds: 78.54 ops/second llite.lustre-ffff899147f67000.statahead_max=0 1001 real 0m22.636s user 0m0.040s sys 0m2.963s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff899147f67000.statahead_max=2048 llite.lustre-ffff899147f67000.statahead_max=2048 1001 real 0m5.505s user 0m0.026s sys 0m1.752s 'ls -l' 1000 files with statahead: 5 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff899147f67000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 mdc.lustre-MDT0000-mdc-ffff899147f67000.batch_stats= snapshot_time: 1781244587.800792850 secs.nsecs start_time: 1781244200.085505601 secs.nsecs elapsed_time: 387.715287249 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 565 (time 1781244600.68 total 10.02 last 56.36) - open/close 1325 (time 1781244610.69 total 20.03 last 75.94) - open/close 2059 (time 1781244620.69 total 30.03 last 73.40) - open/close 2826 (time 1781244630.69 total 40.04 last 76.68) - open/close 3618 (time 1781244640.70 total 50.04 last 79.14) - open/close 4328 (time 1781244650.71 total 60.05 last 70.93) - open/close 4963 (time 1781244660.71 total 70.05 last 63.49) - open/close 5694 (time 1781244670.71 total 80.05 last 73.09) - open/close 6383 (time 1781244680.71 total 90.06 last 68.89) - open/close 7102 (time 1781244690.73 total 100.07 last 71.79) - open/close 7821 (time 1781244700.73 total 110.08 last 71.85) - open/close 8537 (time 1781244710.74 total 120.08 last 71.56) - open/close 9216 (time 1781244720.75 total 130.09 last 67.84) - open/close 9893 (time 1781244730.75 total 140.10 last 67.66) total: 10000 open/close in 141.70 seconds: 70.57 ops/second llite.lustre-ffff899147f67000.statahead_batch_max=0 llite.lustre-ffff899147f67000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff899147f67000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m7.348s user 0m0.295s sys 0m16.188s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff899147f67000.statahead_batch_max=256 llite.lustre-ffff899147f67000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff899147f67000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff899147f67000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m49.536s user 0m0.274s sys 0m16.982s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff899147f67000.batch_stats= snapshot_time: 1781244873.585484896 secs.nsecs start_time: 1781244810.530535230 secs.nsecs elapsed_time: 63.054949666 secs.nsecs subreqs per batch batches % cum % 1: 194 52 52 2: 47 12 64 4: 33 8 73 8: 22 5 79 16: 21 5 85 32: 16 4 89 64: 5 1 90 128: 0 0 90 256: 34 9 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff899147f67000.statahead_batch_max=64 llite.lustre-ffff899147f67000.statahead_max=128