== sanity test 123ad: Verify batching statahead works correctly ========================================================== 11:08:19 (1784128099) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8ae88c7cf800.batch_stats=0 llite.lustre-ffff8ae88c7cf800.statahead_max=32 llite.lustre-ffff8ae88c7cf800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8ae88c7cf800.batch_stats=0 total: 100 open/close in 0.72 seconds: 138.20 ops/second llite.lustre-ffff8ae88c7cf800.statahead_max=0 101 real 0m0.583s user 0m0.003s sys 0m0.231s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff8ae88c7cf800.statahead_max=32 llite.lustre-ffff8ae88c7cf800.statahead_max=32 101 real 0m0.517s user 0m0.004s sys 0m0.349s 'ls -l' 100 files with statahead: 1 sec statahead total: 79 statahead wrong: 2 agl total: 74 list_total: 29 fname_total: 44 hit_total: 13908 miss_total: 1121 swrong=2 ewrong=2 total: 900 open/close in 1.98 seconds: 454.31 ops/second llite.lustre-ffff8ae88c7cf800.statahead_max=0 1001 real 0m5.726s user 0m0.021s sys 0m2.212s 'ls -l' 1000 files without statahead: 5 sec llite.lustre-ffff8ae88c7cf800.statahead_max=32 llite.lustre-ffff8ae88c7cf800.statahead_max=32 1001 real 0m3.344s user 0m0.013s sys 0m1.920s 'ls -l' 1000 files with statahead: 4 sec statahead total: 80 statahead wrong: 2 agl total: 75 list_total: 30 fname_total: 44 hit_total: 14907 miss_total: 1122 swrong=2 ewrong=2 - open/close 4742 (time 1784128128.37 total 10.00 last 474.12) total: 9000 open/close in 19.02 seconds: 473.14 ops/second llite.lustre-ffff8ae88c7cf800.statahead_max=0 10001 real 1m17.952s user 0m0.265s sys 0m24.608s 'ls -l' 10000 files without statahead: 78 sec llite.lustre-ffff8ae88c7cf800.statahead_max=32 llite.lustre-ffff8ae88c7cf800.statahead_max=32 10001 real 1m26.245s user 0m0.279s sys 0m16.604s 'ls -l' 10000 files with statahead: 86 sec statahead total: 81 statahead wrong: 2 agl total: 76 list_total: 31 fname_total: 44 hit_total: 24883 miss_total: 1146 swrong=2 ewrong=2 llite.lustre-ffff8ae88c7cf800.statahead_max=0 10001 real 3m44.502s user 0m0.346s sys 0m30.579s 'ls -l' 10000 files again w/o statahead: 225 sec llite.lustre-ffff8ae88c7cf800.statahead_max=32 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 177 seconds llite.lustre-ffff8ae88c7cf800.statahead_stats= statahead total: 81 statahead wrong: 2 agl total: 76 list_total: 31 fname_total: 44 hit_total: 24883 miss_total: 1146 mdc.lustre-MDT0000-mdc-ffff8ae88c7cf800.batch_stats= snapshot_time: 1784128755.348136098 secs.nsecs start_time: 1784128100.605172195 secs.nsecs elapsed_time: 654.742963903 secs.nsecs subreqs per batch batches %% cum %% 1: 1 0 0 2: 0 0 0 4: 1 0 0 8: 4 1 1 16: 3 0 2 32: 347 97 100 - open/close 1109 (time 1784128767.19 total 10.00 last 110.87) - open/close 2279 (time 1784128777.20 total 20.00 last 116.98) - open/close 3585 (time 1784128787.20 total 30.01 last 130.58) - open/close 5278 (time 1784128797.20 total 40.01 last 169.20) - open/close 6958 (time 1784128807.21 total 50.02 last 167.93) - open/close 8928 (time 1784128817.21 total 60.02 last 196.94) total: 10000 open/close in 66.45 seconds: 150.49 ops/second llite.lustre-ffff8ae88c7cf800.statahead_batch_max=0 llite.lustre-ffff8ae88c7cf800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8ae88c7cf800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m35.835s user 0m0.240s sys 0m28.363s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8ae88c7cf800.statahead_batch_max=32 llite.lustre-ffff8ae88c7cf800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8ae88c7cf800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8ae88c7cf800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m44.556s user 0m0.228s sys 0m20.992s batched statahead RPC= hit count=9995 mdc.lustre-MDT0000-mdc-ffff8ae88c7cf800.batch_stats= snapshot_time: 1784128915.629942210 secs.nsecs start_time: 1784128863.809450905 secs.nsecs elapsed_time: 51.820491305 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8ae88c7cf800.batch_stats=0 llite.lustre-ffff8ae88c7cf800.statahead_max=2048 llite.lustre-ffff8ae88c7cf800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8ae88c7cf800.batch_stats=0 total: 100 open/close in 0.73 seconds: 136.18 ops/second llite.lustre-ffff8ae88c7cf800.statahead_max=0 101 real 0m0.656s user 0m0.004s sys 0m0.246s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff8ae88c7cf800.statahead_max=2048 llite.lustre-ffff8ae88c7cf800.statahead_max=2048 101 real 0m0.335s user 0m0.008s sys 0m0.213s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10094 miss_total: 6 swrong=0 ewrong=0 total: 900 open/close in 2.25 seconds: 399.82 ops/second llite.lustre-ffff8ae88c7cf800.statahead_max=0 1001 real 0m6.421s user 0m0.024s sys 0m2.421s 'ls -l' 1000 files without statahead: 6 sec llite.lustre-ffff8ae88c7cf800.statahead_max=2048 llite.lustre-ffff8ae88c7cf800.statahead_max=2048 1001 real 0m3.348s user 0m0.011s sys 0m2.043s 'ls -l' 1000 files with statahead: 3 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11093 miss_total: 7 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 5 seconds llite.lustre-ffff8ae88c7cf800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11093 miss_total: 7 mdc.lustre-MDT0000-mdc-ffff8ae88c7cf800.batch_stats= snapshot_time: 1784129052.075477171 secs.nsecs start_time: 1784128916.581778467 secs.nsecs elapsed_time: 135.493698704 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 0 0 54 128: 1 9 63 256: 4 36 100 - open/close 3862 (time 1784129063.28 total 10.00 last 386.15) - open/close 7854 (time 1784129073.28 total 20.00 last 399.19) total: 10000 open/close in 24.49 seconds: 408.25 ops/second llite.lustre-ffff8ae88c7cf800.statahead_batch_max=0 llite.lustre-ffff8ae88c7cf800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8ae88c7cf800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m36.117s user 0m0.158s sys 0m30.689s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8ae88c7cf800.statahead_batch_max=256 llite.lustre-ffff8ae88c7cf800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8ae88c7cf800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8ae88c7cf800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m31.937s user 0m0.177s sys 0m22.663s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8ae88c7cf800.batch_stats= snapshot_time: 1784129155.530256026 secs.nsecs start_time: 1784129117.216871717 secs.nsecs elapsed_time: 38.313384309 secs.nsecs subreqs per batch batches %% cum %% 1: 304 66 66 2: 63 13 80 4: 23 5 85 8: 5 1 86 16: 8 1 87 32: 20 4 92 64: 0 0 92 128: 0 0 92 256: 35 7 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8ae88c7cf800.statahead_batch_max=64 llite.lustre-ffff8ae88c7cf800.statahead_max=128