== sanity test 123ad: Verify batching statahead works correctly ========================================================== 13:17:17 (1788196637) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff963287f7a000.batch_stats=0 llite.lustre-ffff963287f7a000.statahead_max=32 llite.lustre-ffff963287f7a000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff963287f7a000.batch_stats=0 total: 100 open/close in 3.32 seconds: 30.14 ops/second llite.lustre-ffff963287f7a000.statahead_max=0 101 real 0m2.525s user 0m0.005s sys 0m0.336s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff963287f7a000.statahead_max=32 llite.lustre-ffff963287f7a000.statahead_max=32 101 real 0m1.343s user 0m0.004s sys 0m0.179s 'ls -l' 100 files with statahead: 1 sec statahead total: 140 statahead wrong: 2 agl total: 117 list_total: 31 fname_total: 106 hit_total: 3823 miss_total: 1199 swrong=2 ewrong=2 - open/close 746 (time 1788196667.10 total 10.02 last 74.46) total: 900 open/close in 12.03 seconds: 74.79 ops/second llite.lustre-ffff963287f7a000.statahead_max=0 1001 real 0m20.070s user 0m0.043s sys 0m3.130s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff963287f7a000.statahead_max=32 llite.lustre-ffff963287f7a000.statahead_max=32 1001 real 0m11.484s user 0m0.037s sys 0m1.470s 'ls -l' 1000 files with statahead: 11 sec statahead total: 141 statahead wrong: 2 agl total: 118 list_total: 32 fname_total: 106 hit_total: 4820 miss_total: 1202 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff963287f7a000.statahead_stats= statahead total: 141 statahead wrong: 2 agl total: 118 list_total: 32 fname_total: 106 hit_total: 4820 miss_total: 1202 mdc.lustre-MDT0000-mdc-ffff963287f7a000.batch_stats= snapshot_time: 1788196736.065418126 secs.nsecs start_time: 1788196643.266192824 secs.nsecs elapsed_time: 92.799225302 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 589 (time 1788196749.25 total 10.00 last 58.89) - open/close 1174 (time 1788196759.27 total 20.02 last 58.41) - open/close 1784 (time 1788196769.28 total 30.03 last 60.94) - open/close 2443 (time 1788196779.28 total 40.03 last 65.90) - open/close 3057 (time 1788196789.28 total 50.03 last 61.38) - open/close 3789 (time 1788196799.29 total 60.04 last 73.13) - open/close 4433 (time 1788196809.30 total 70.05 last 64.34) - open/close 5212 (time 1788196819.32 total 80.07 last 77.76) - open/close 5891 (time 1788196829.32 total 90.07 last 67.87) - open/close 6609 (time 1788196839.33 total 100.08 last 71.77) - open/close 7239 (time 1788196849.33 total 110.08 last 62.98) - open/close 8063 (time 1788196859.34 total 120.09 last 82.35) - open/close 8913 (time 1788196869.36 total 130.10 last 84.85) - open/close 9645 (time 1788196879.36 total 140.11 last 73.15) total: 10000 open/close in 145.34 seconds: 68.80 ops/second llite.lustre-ffff963287f7a000.statahead_batch_max=0 llite.lustre-ffff963287f7a000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff963287f7a000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m10.737s user 0m0.332s sys 0m15.725s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff963287f7a000.statahead_batch_max=32 llite.lustre-ffff963287f7a000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff963287f7a000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff963287f7a000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m47.117s user 0m0.277s sys 0m14.454s batched statahead RPC= hit count=9980 mdc.lustre-MDT0000-mdc-ffff963287f7a000.batch_stats= snapshot_time: 1788197089.051893716 secs.nsecs start_time: 1788196967.106514226 secs.nsecs elapsed_time: 121.945379490 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff963287f7a000.batch_stats=0 llite.lustre-ffff963287f7a000.statahead_max=2048 llite.lustre-ffff963287f7a000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff963287f7a000.batch_stats=0 total: 100 open/close in 2.28 seconds: 43.89 ops/second llite.lustre-ffff963287f7a000.statahead_max=0 101 real 0m2.044s user 0m0.009s sys 0m0.317s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff963287f7a000.statahead_max=2048 llite.lustre-ffff963287f7a000.statahead_max=2048 101 real 0m0.630s user 0m0.004s sys 0m0.151s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10079 miss_total: 21 swrong=0 ewrong=0 - open/close 789 (time 1788197395.21 total 10.00 last 78.87) total: 900 open/close in 11.39 seconds: 79.04 ops/second llite.lustre-ffff963287f7a000.statahead_max=0 1001 real 0m21.610s user 0m0.021s sys 0m2.980s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff963287f7a000.statahead_max=2048 llite.lustre-ffff963287f7a000.statahead_max=2048 1001 real 0m7.909s user 0m0.023s sys 0m1.325s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff963287f7a000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 mdc.lustre-MDT0000-mdc-ffff963287f7a000.batch_stats= snapshot_time: 1788197463.332598211 secs.nsecs start_time: 1788197091.640298249 secs.nsecs elapsed_time: 371.692299962 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 2 16 50 64: 1 8 58 128: 1 8 66 256: 4 33 100 - open/close 706 (time 1788197476.48 total 10.01 last 70.55) - open/close 1488 (time 1788197486.49 total 20.01 last 78.15) - open/close 2193 (time 1788197496.52 total 30.04 last 70.31) - open/close 2890 (time 1788197506.52 total 40.05 last 69.63) - open/close 3676 (time 1788197516.54 total 50.06 last 78.50) - open/close 4470 (time 1788197526.55 total 60.07 last 79.31) - open/close 5229 (time 1788197536.55 total 70.08 last 75.88) - open/close 6033 (time 1788197546.57 total 80.09 last 80.28) - open/close 6824 (time 1788197556.57 total 90.09 last 79.09) - open/close 7704 (time 1788197566.57 total 100.10 last 87.95) - open/close 8540 (time 1788197576.58 total 110.10 last 83.55) - open/close 9391 (time 1788197586.59 total 120.11 last 85.06) total: 10000 open/close in 128.04 seconds: 78.10 ops/second llite.lustre-ffff963287f7a000.statahead_batch_max=0 llite.lustre-ffff963287f7a000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff963287f7a000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m56.941s user 0m0.310s sys 0m15.374s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff963287f7a000.statahead_batch_max=256 llite.lustre-ffff963287f7a000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff963287f7a000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff963287f7a000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m53.760s user 0m0.272s sys 0m14.504s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff963287f7a000.batch_stats= snapshot_time: 1788197727.381122157 secs.nsecs start_time: 1788197661.121980495 secs.nsecs elapsed_time: 66.259141662 secs.nsecs subreqs per batch batches %% cum %% 1: 214 52 52 2: 64 15 68 4: 35 8 77 8: 21 5 82 16: 18 4 87 32: 12 2 90 64: 6 1 91 128: 0 0 91 256: 34 8 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff963287f7a000.statahead_batch_max=64 llite.lustre-ffff963287f7a000.statahead_max=128