== sanity test 123ad: Verify batching statahead works correctly ========================================================== 21:01:55 (1781226115) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff906d18800000.batch_stats=0 llite.lustre-ffff906d18800000.statahead_max=32 llite.lustre-ffff906d18800000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff906d18800000.batch_stats=0 total: 100 open/close in 3.02 seconds: 33.07 ops/second llite.lustre-ffff906d18800000.statahead_max=0 101 real 0m1.954s user 0m0.004s sys 0m0.334s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff906d18800000.statahead_max=32 llite.lustre-ffff906d18800000.statahead_max=32 101 real 0m1.196s user 0m0.003s sys 0m0.178s 'ls -l' 100 files with statahead: 1 sec statahead total: 133 statahead wrong: 1 agl total: 127 list_total: 30 fname_total: 96 hit_total: 4067 miss_total: 255 swrong=1 ewrong=1 - open/close 696 (time 1781226143.85 total 10.00 last 69.60) total: 900 open/close in 12.21 seconds: 73.73 ops/second llite.lustre-ffff906d18800000.statahead_max=0 1001 real 0m22.937s user 0m0.037s sys 0m3.052s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff906d18800000.statahead_max=32 llite.lustre-ffff906d18800000.statahead_max=32 1001 real 0m14.068s user 0m0.037s sys 0m1.562s 'ls -l' 1000 files with statahead: 14 sec statahead total: 134 statahead wrong: 1 agl total: 128 list_total: 31 fname_total: 96 hit_total: 5064 miss_total: 258 swrong=1 ewrong=1 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff906d18800000.statahead_stats= statahead total: 134 statahead wrong: 1 agl total: 128 list_total: 31 fname_total: 96 hit_total: 5064 miss_total: 258 mdc.lustre-MDT0000-mdc-ffff906d18800000.batch_stats= snapshot_time: 1781226223.098331116 secs.nsecs start_time: 1781226120.981998372 secs.nsecs elapsed_time: 102.116332744 secs.nsecs subreqs per batch batches % cum % 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 697 (time 1781226235.98 total 10.00 last 69.67) - open/close 1427 (time 1781226245.99 total 20.02 last 72.92) - open/close 2069 (time 1781226256.00 total 30.03 last 64.13) - open/close 2913 (time 1781226266.01 total 40.04 last 84.31) - open/close 3689 (time 1781226276.01 total 50.04 last 77.57) - open/close 4602 (time 1781226286.01 total 60.04 last 91.30) - open/close 5548 (time 1781226296.02 total 70.05 last 94.55) - open/close 6343 (time 1781226306.03 total 80.06 last 79.41) - open/close 7084 (time 1781226316.04 total 90.07 last 74.02) - open/close 7848 (time 1781226326.04 total 100.07 last 76.37) - open/close 8604 (time 1781226336.05 total 110.07 last 75.59) - open/close 9320 (time 1781226346.06 total 120.09 last 71.48) total: 10000 open/close in 129.75 seconds: 77.07 ops/second llite.lustre-ffff906d18800000.statahead_batch_max=0 llite.lustre-ffff906d18800000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff906d18800000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m4.192s user 0m0.332s sys 0m14.753s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff906d18800000.statahead_batch_max=32 llite.lustre-ffff906d18800000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff906d18800000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff906d18800000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m47.894s user 0m0.285s sys 0m14.741s batched statahead RPC= hit count=9978 mdc.lustre-MDT0000-mdc-ffff906d18800000.batch_stats= snapshot_time: 1781226558.261826989 secs.nsecs start_time: 1781226432.999492293 secs.nsecs elapsed_time: 125.262334696 secs.nsecs subreqs per batch batches % cum % 1: 1 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff906d18800000.batch_stats=0 llite.lustre-ffff906d18800000.statahead_max=2048 llite.lustre-ffff906d18800000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff906d18800000.batch_stats=0 total: 100 open/close in 2.74 seconds: 36.45 ops/second llite.lustre-ffff906d18800000.statahead_max=0 101 real 0m1.793s user 0m0.003s sys 0m0.326s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff906d18800000.statahead_max=2048 llite.lustre-ffff906d18800000.statahead_max=2048 101 real 0m0.907s user 0m0.006s sys 0m0.176s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10077 miss_total: 23 swrong=0 ewrong=0 - open/close 811 (time 1781226872.81 total 10.01 last 81.00) total: 900 open/close in 11.27 seconds: 79.84 ops/second llite.lustre-ffff906d18800000.statahead_max=0 1001 real 0m18.876s user 0m0.043s sys 0m3.035s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff906d18800000.statahead_max=2048 llite.lustre-ffff906d18800000.statahead_max=2048 1001 real 0m7.438s user 0m0.026s sys 0m1.495s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11076 miss_total: 24 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff906d18800000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11076 miss_total: 24 mdc.lustre-MDT0000-mdc-ffff906d18800000.batch_stats= snapshot_time: 1781226939.434871678 secs.nsecs start_time: 1781226560.792573849 secs.nsecs elapsed_time: 378.642297829 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 0 0 54 128: 1 9 63 256: 4 36 100 - open/close 621 (time 1781226952.88 total 10.01 last 62.05) - open/close 1301 (time 1781226962.89 total 20.02 last 67.91) - open/close 1981 (time 1781226972.92 total 30.04 last 67.86) - open/close 2684 (time 1781226982.93 total 40.06 last 70.19) - open/close 3509 (time 1781226992.94 total 50.07 last 82.42) - open/close 4172 (time 1781227002.95 total 60.08 last 66.22) - open/close 4777 (time 1781227012.96 total 70.09 last 60.46) - open/close 5567 (time 1781227022.97 total 80.09 last 78.94) - open/close 6316 (time 1781227032.97 total 90.10 last 74.85) - open/close 7015 (time 1781227042.98 total 100.10 last 69.87) - open/close 7668 (time 1781227052.98 total 110.11 last 65.26) - open/close 8431 (time 1781227062.99 total 120.12 last 76.27) - open/close 9187 (time 1781227073.02 total 130.15 last 75.36) total: 10000 open/close in 139.75 seconds: 71.55 ops/second llite.lustre-ffff906d18800000.statahead_batch_max=0 llite.lustre-ffff906d18800000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff906d18800000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m2.744s user 0m0.347s sys 0m15.084s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff906d18800000.statahead_batch_max=256 llite.lustre-ffff906d18800000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff906d18800000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff906d18800000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m2.245s user 0m0.331s sys 0m15.335s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff906d18800000.batch_stats= snapshot_time: 1781227236.201571103 secs.nsecs start_time: 1781227156.055014622 secs.nsecs elapsed_time: 80.146556481 secs.nsecs subreqs per batch batches % cum % 1: 145 48 48 2: 48 15 64 4: 25 8 72 8: 18 5 78 16: 16 5 83 32: 12 3 87 64: 2 0 88 128: 0 0 88 256: 35 11 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff906d18800000.statahead_batch_max=64 llite.lustre-ffff906d18800000.statahead_max=128