== sanity test 123ad: Verify batching statahead works correctly ========================================================== 07:37:19 (1787398639) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff92a4c5c15000.batch_stats=0 llite.lustre-ffff92a4c5c15000.statahead_max=32 llite.lustre-ffff92a4c5c15000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff92a4c5c15000.batch_stats=0 total: 100 open/close in 2.86 seconds: 35.00 ops/second llite.lustre-ffff92a4c5c15000.statahead_max=0 101 real 0m3.096s user 0m0.002s sys 0m0.342s 'ls -l' 100 files without statahead: 4 sec llite.lustre-ffff92a4c5c15000.statahead_max=32 llite.lustre-ffff92a4c5c15000.statahead_max=32 101 real 0m1.154s user 0m0.004s sys 0m0.205s 'ls -l' 100 files with statahead: 1 sec statahead total: 101 statahead wrong: 2 agl total: 95 list_total: 29 fname_total: 66 hit_total: 3945 miss_total: 1491 swrong=2 ewrong=2 - open/close 704 (time 1787398667.29 total 10.01 last 70.32) total: 900 open/close in 12.89 seconds: 69.84 ops/second llite.lustre-ffff92a4c5c15000.statahead_max=0 1001 real 0m22.538s user 0m0.028s sys 0m3.196s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff92a4c5c15000.statahead_max=32 llite.lustre-ffff92a4c5c15000.statahead_max=32 1001 real 0m13.205s user 0m0.033s sys 0m1.563s 'ls -l' 1000 files with statahead: 13 sec statahead total: 102 statahead wrong: 2 agl total: 96 list_total: 30 fname_total: 66 hit_total: 4942 miss_total: 1494 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff92a4c5c15000.statahead_stats= statahead total: 102 statahead wrong: 2 agl total: 96 list_total: 30 fname_total: 66 hit_total: 4942 miss_total: 1494 mdc.lustre-MDT0000-mdc-ffff92a4c5c15000.batch_stats= snapshot_time: 1787398741.519166813 secs.nsecs start_time: 1787398643.675165353 secs.nsecs elapsed_time: 97.844001460 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 1 2 5 8: 2 5 10 16: 2 5 15 32: 34 85 100 - open/close 731 (time 1787398753.77 total 10.01 last 73.04) - open/close 1390 (time 1787398763.78 total 20.01 last 65.86) - open/close 2087 (time 1787398773.79 total 30.03 last 69.59) - open/close 2717 (time 1787398783.80 total 40.03 last 62.98) - open/close 3310 (time 1787398793.85 total 50.09 last 58.96) - open/close 3840 (time 1787398803.87 total 60.11 last 52.90) - open/close 4519 (time 1787398813.88 total 70.12 last 67.84) - open/close 5119 (time 1787398823.89 total 80.13 last 59.94) - open/close 5731 (time 1787398833.90 total 90.13 last 61.17) - open/close 6438 (time 1787398843.90 total 100.13 last 70.69) - open/close 7160 (time 1787398853.90 total 110.14 last 72.20) - open/close 7912 (time 1787398863.90 total 120.14 last 75.18) - open/close 8620 (time 1787398873.91 total 130.14 last 70.76) - open/close 9315 (time 1787398883.91 total 140.15 last 69.46) total: 10000 open/close in 149.84 seconds: 66.74 ops/second llite.lustre-ffff92a4c5c15000.statahead_batch_max=0 llite.lustre-ffff92a4c5c15000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92a4c5c15000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.858s user 0m0.320s sys 0m17.980s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff92a4c5c15000.statahead_batch_max=32 llite.lustre-ffff92a4c5c15000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92a4c5c15000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff92a4c5c15000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m53.041s user 0m0.299s sys 0m15.478s batched statahead RPC= hit count=9987 mdc.lustre-MDT0000-mdc-ffff92a4c5c15000.batch_stats= snapshot_time: 1787399088.940852013 secs.nsecs start_time: 1787398964.137618174 secs.nsecs elapsed_time: 124.803233839 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff92a4c5c15000.batch_stats=0 llite.lustre-ffff92a4c5c15000.statahead_max=2048 llite.lustre-ffff92a4c5c15000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff92a4c5c15000.batch_stats=0 total: 100 open/close in 3.29 seconds: 30.42 ops/second llite.lustre-ffff92a4c5c15000.statahead_max=0 101 real 0m2.339s user 0m0.010s sys 0m0.306s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff92a4c5c15000.statahead_max=2048 llite.lustre-ffff92a4c5c15000.statahead_max=2048 101 real 0m0.966s user 0m0.004s sys 0m0.185s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10086 miss_total: 14 swrong=0 ewrong=0 - open/close 586 (time 1787399437.44 total 10.01 last 58.56) total: 900 open/close in 16.54 seconds: 54.41 ops/second llite.lustre-ffff92a4c5c15000.statahead_max=0 1001 real 0m26.216s user 0m0.029s sys 0m3.385s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff92a4c5c15000.statahead_max=2048 llite.lustre-ffff92a4c5c15000.statahead_max=2048 1001 real 0m7.104s user 0m0.027s sys 0m1.504s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff92a4c5c15000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 mdc.lustre-MDT0000-mdc-ffff92a4c5c15000.batch_stats= snapshot_time: 1787399518.278289980 secs.nsecs start_time: 1787399091.903046703 secs.nsecs elapsed_time: 426.375243277 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 22 22 16: 2 22 44 32: 0 0 44 64: 0 0 44 128: 1 11 55 256: 4 44 100 - open/close 570 (time 1787399531.24 total 10.00 last 56.99) - open/close 1189 (time 1787399541.25 total 20.01 last 61.83) - open/close 1785 (time 1787399551.26 total 30.02 last 59.57) - open/close 2509 (time 1787399561.26 total 40.03 last 72.34) - open/close 3105 (time 1787399571.29 total 50.05 last 59.46) - open/close 3785 (time 1787399581.29 total 60.05 last 67.99) - open/close 4456 (time 1787399591.31 total 70.07 last 66.97) - open/close 5130 (time 1787399601.32 total 80.08 last 67.35) - open/close 5848 (time 1787399611.33 total 90.09 last 71.71) - open/close 6495 (time 1787399621.34 total 100.10 last 64.64) - open/close 7082 (time 1787399631.34 total 110.11 last 58.66) - open/close 7727 (time 1787399641.37 total 120.13 last 64.36) - open/close 8432 (time 1787399651.37 total 130.14 last 70.45) - open/close 9166 (time 1787399661.38 total 140.14 last 73.39) - open/close 9788 (time 1787399671.43 total 150.19 last 61.88) total: 10000 open/close in 153.20 seconds: 65.27 ops/second llite.lustre-ffff92a4c5c15000.statahead_batch_max=0 llite.lustre-ffff92a4c5c15000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92a4c5c15000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m8.884s user 0m0.299s sys 0m15.984s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff92a4c5c15000.statahead_batch_max=256 llite.lustre-ffff92a4c5c15000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92a4c5c15000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff92a4c5c15000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m57.769s user 0m0.293s sys 0m17.162s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff92a4c5c15000.batch_stats= snapshot_time: 1787399823.909265668 secs.nsecs start_time: 1787399755.108064699 secs.nsecs elapsed_time: 68.801200969 secs.nsecs subreqs per batch batches %% cum %% 1: 150 49 49 2: 42 13 62 4: 32 10 73 8: 17 5 78 16: 15 4 83 32: 9 2 86 64: 5 1 88 128: 1 0 88 256: 35 11 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff92a4c5c15000.statahead_batch_max=64 llite.lustre-ffff92a4c5c15000.statahead_max=128