== sanity test 123ad: Verify batching statahead works correctly ========================================================== 06:25:46 (1788690346) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffffa0708a3da800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffffa0708a3da800.batch_stats=0 llite.lustre-ffffa0708a3da800.statahead_max=32 llite.lustre-ffffa0708a3da800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffffa0708a3da800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffffa0708a3da800.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.82 seconds: 35.47 ops/second llite.lustre-ffffa0708a3da800.statahead_max=0 101 real 0m2.667s user 0m0.004s sys 0m0.331s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffffa0708a3da800.statahead_max=32 llite.lustre-ffffa0708a3da800.statahead_max=32 101 real 0m0.954s user 0m0.003s sys 0m0.157s 'ls -l' 100 files with statahead: 1 sec statahead total: 144 statahead wrong: 2 agl total: 138 list_total: 30 fname_total: 73 hit_total: 3893 miss_total: 1724 swrong=2 ewrong=2 - open/close 703 (time 1788690375.51 total 10.01 last 70.25) total: 900 open/close in 13.72 seconds: 65.60 ops/second llite.lustre-ffffa0708a3da800.statahead_max=0 1001 real 0m27.215s user 0m0.034s sys 0m2.989s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffffa0708a3da800.statahead_max=32 llite.lustre-ffffa0708a3da800.statahead_max=32 1001 real 0m10.810s user 0m0.026s sys 0m1.298s 'ls -l' 1000 files with statahead: 11 sec statahead total: 145 statahead wrong: 2 agl total: 139 list_total: 31 fname_total: 73 hit_total: 4889 miss_total: 1728 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffffa0708a3da800.statahead_stats= statahead total: 145 statahead wrong: 2 agl total: 139 list_total: 31 fname_total: 73 hit_total: 4889 miss_total: 1728 mdc.lustre-MDT0000-mdc-ffffa0708a3da800.batch_stats= snapshot_time: 1788690454.136005833 secs.nsecs start_time: 1788690351.462531890 secs.nsecs elapsed_time: 102.673473943 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 2 2 4: 1 2 5 8: 3 7 12 16: 25 64 76 32: 9 23 100 mdc.lustre-MDT0001-mdc-ffffa0708a3da800.batch_stats= snapshot_time: 1788690454.136087978 secs.nsecs start_time: 1788690351.473110868 secs.nsecs elapsed_time: 102.662977110 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 2 5 5 4: 2 5 10 8: 3 7 17 16: 16 41 58 32: 16 41 100 - open/close 788 (time 1788690467.19 total 10.00 last 78.76) - open/close 1467 (time 1788690477.22 total 20.03 last 67.72) - open/close 2174 (time 1788690487.23 total 30.04 last 70.62) - open/close 2917 (time 1788690497.23 total 40.05 last 74.27) - open/close 3737 (time 1788690507.24 total 50.05 last 81.96) - open/close 4544 (time 1788690517.24 total 60.05 last 80.69) - open/close 5312 (time 1788690527.24 total 70.05 last 76.79) - open/close 6005 (time 1788690537.26 total 80.07 last 69.18) - open/close 6808 (time 1788690547.27 total 90.08 last 80.20) - open/close 7522 (time 1788690557.28 total 100.09 last 71.36) - open/close 8191 (time 1788690567.28 total 110.09 last 66.90) - open/close 8922 (time 1788690577.28 total 120.10 last 73.05) - open/close 9567 (time 1788690587.29 total 130.10 last 64.47) total: 10000 open/close in 136.04 seconds: 73.51 ops/second llite.lustre-ffffa0708a3da800.statahead_batch_max=0 llite.lustre-ffffa0708a3da800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffffa0708a3da800.stats=clear mdc.lustre-MDT0001-mdc-ffffa0708a3da800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m14.253s user 0m0.343s sys 0m14.696s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffffa0708a3da800.statahead_batch_max=32 llite.lustre-ffffa0708a3da800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffffa0708a3da800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffffa0708a3da800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffffa0708a3da800.stats=clear mdc.lustre-MDT0001-mdc-ffffa0708a3da800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m49.128s user 0m0.337s sys 0m14.010s batched statahead RPC= hit count=9986 mdc.lustre-MDT0000-mdc-ffffa0708a3da800.batch_stats= snapshot_time: 1788690799.892847941 secs.nsecs start_time: 1788690678.362833773 secs.nsecs elapsed_time: 121.530014168 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffffa0708a3da800.batch_stats= snapshot_time: 1788690799.893428322 secs.nsecs start_time: 1788690678.362960990 secs.nsecs elapsed_time: 121.530467332 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffffa0708a3da800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffffa0708a3da800.batch_stats=0 llite.lustre-ffffa0708a3da800.statahead_max=2048 llite.lustre-ffffa0708a3da800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffffa0708a3da800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffffa0708a3da800.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.89 seconds: 34.61 ops/second llite.lustre-ffffa0708a3da800.statahead_max=0 101 real 0m2.859s user 0m0.006s sys 0m0.305s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffffa0708a3da800.statahead_max=2048 llite.lustre-ffffa0708a3da800.statahead_max=2048 101 real 0m1.076s user 0m0.007s sys 0m0.165s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10085 miss_total: 15 swrong=0 ewrong=0 - open/close 784 (time 1788691101.59 total 10.00 last 78.37) total: 900 open/close in 11.62 seconds: 77.44 ops/second llite.lustre-ffffa0708a3da800.statahead_max=0 1001 real 0m20.756s user 0m0.034s sys 0m2.865s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffffa0708a3da800.statahead_max=2048 llite.lustre-ffffa0708a3da800.statahead_max=2048 1001 real 0m6.485s user 0m0.033s sys 0m1.244s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffffa0708a3da800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 mdc.lustre-MDT0000-mdc-ffffa0708a3da800.batch_stats= snapshot_time: 1788691167.578391102 secs.nsecs start_time: 1788690803.532759824 secs.nsecs elapsed_time: 364.045631278 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 5 62 62 16: 0 0 62 32: 0 0 62 64: 1 12 75 128: 0 0 75 256: 2 25 100 mdc.lustre-MDT0001-mdc-ffffa0708a3da800.batch_stats= snapshot_time: 1788691167.578455508 secs.nsecs start_time: 1788690803.532970083 secs.nsecs elapsed_time: 364.045485425 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 12 12 4: 3 37 50 8: 0 0 50 16: 1 12 62 32: 0 0 62 64: 1 12 75 128: 0 0 75 256: 2 25 100 - open/close 710 (time 1788691180.58 total 10.01 last 70.93) - open/close 1466 (time 1788691190.58 total 20.01 last 75.59) - open/close 2229 (time 1788691200.59 total 30.01 last 76.29) - open/close 2945 (time 1788691210.59 total 40.02 last 71.57) - open/close 3830 (time 1788691220.60 total 50.03 last 88.43) - open/close 4535 (time 1788691230.61 total 60.04 last 70.43) - open/close 5129 (time 1788691240.64 total 70.06 last 59.24) - open/close 5748 (time 1788691250.64 total 80.07 last 61.87) - open/close 6428 (time 1788691260.64 total 90.07 last 67.97) - open/close 7013 (time 1788691270.66 total 100.08 last 58.43) - open/close 7748 (time 1788691280.66 total 110.09 last 73.49) - open/close 8400 (time 1788691290.67 total 120.09 last 65.15) - open/close 9121 (time 1788691300.67 total 130.10 last 72.08) - open/close 9799 (time 1788691310.68 total 140.11 last 67.71) total: 10000 open/close in 143.24 seconds: 69.81 ops/second llite.lustre-ffffa0708a3da800.statahead_batch_max=0 llite.lustre-ffffa0708a3da800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffffa0708a3da800.stats=clear mdc.lustre-MDT0001-mdc-ffffa0708a3da800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m11.993s user 0m0.323s sys 0m14.485s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffffa0708a3da800.statahead_batch_max=256 llite.lustre-ffffa0708a3da800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffffa0708a3da800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffffa0708a3da800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffffa0708a3da800.stats=clear mdc.lustre-MDT0001-mdc-ffffa0708a3da800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m4.982s user 0m0.324s sys 0m14.644s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffffa0708a3da800.batch_stats= snapshot_time: 1788691474.928372142 secs.nsecs start_time: 1788691396.039430594 secs.nsecs elapsed_time: 78.888941548 secs.nsecs subreqs per batch batches %% cum %% 1: 82 40 40 2: 32 15 55 4: 15 7 62 8: 9 4 67 16: 9 4 71 32: 18 8 80 64: 5 2 82 128: 0 0 82 256: 35 17 100 mdc.lustre-MDT0001-mdc-ffffa0708a3da800.batch_stats= snapshot_time: 1788691474.928455235 secs.nsecs start_time: 1788691396.039568209 secs.nsecs elapsed_time: 78.888887026 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffffa0708a3da800.statahead_batch_max=64 llite.lustre-ffffa0708a3da800.statahead_max=128