== sanity test 123ad: Verify batching statahead works correctly ========================================================== 06:56:10 (1783076170) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9550c51f0000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9550c51f0000.batch_stats=0 llite.lustre-ffff9550c51f0000.statahead_max=32 llite.lustre-ffff9550c51f0000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9550c51f0000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9550c51f0000.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 0.58 seconds: 171.61 ops/second llite.lustre-ffff9550c51f0000.statahead_max=0 101 real 0m0.578s user 0m0.004s sys 0m0.231s 'ls -l' 100 files without statahead: 0 sec llite.lustre-ffff9550c51f0000.statahead_max=32 llite.lustre-ffff9550c51f0000.statahead_max=32 101 real 0m0.388s user 0m0.002s sys 0m0.198s 'ls -l' 100 files with statahead: 0 sec statahead total: 110 statahead wrong: 2 agl total: 105 list_total: 31 fname_total: 68 hit_total: 14093 miss_total: 999 swrong=2 ewrong=2 total: 900 open/close in 1.69 seconds: 532.99 ops/second llite.lustre-ffff9550c51f0000.statahead_max=0 1001 real 0m5.775s user 0m0.025s sys 0m2.277s 'ls -l' 1000 files without statahead: 6 sec llite.lustre-ffff9550c51f0000.statahead_max=32 llite.lustre-ffff9550c51f0000.statahead_max=32 1001 real 0m4.123s user 0m0.016s sys 0m1.981s 'ls -l' 1000 files with statahead: 4 sec statahead total: 111 statahead wrong: 2 agl total: 106 list_total: 32 fname_total: 68 hit_total: 15092 miss_total: 1000 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 7 seconds llite.lustre-ffff9550c51f0000.statahead_stats= statahead total: 111 statahead wrong: 2 agl total: 106 list_total: 32 fname_total: 68 hit_total: 15092 miss_total: 1000 mdc.lustre-MDT0000-mdc-ffff9550c51f0000.batch_stats= snapshot_time: 1783076197.399606307 secs.nsecs start_time: 1783076171.665497455 secs.nsecs elapsed_time: 25.734108852 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 2 5 8 4: 2 5 13 8: 2 5 19 16: 19 52 72 32: 10 27 100 mdc.lustre-MDT0001-mdc-ffff9550c51f0000.batch_stats= snapshot_time: 1783076197.399677973 secs.nsecs start_time: 1783076171.665543445 secs.nsecs elapsed_time: 25.734134528 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 2 2 4: 1 2 5 8: 4 10 15 16: 11 28 43 32: 22 56 100 - open/close 4675 (time 1783076208.28 total 10.00 last 467.43) - open/close 9519 (time 1783076218.28 total 20.00 last 484.27) total: 10000 open/close in 20.94 seconds: 477.52 ops/second llite.lustre-ffff9550c51f0000.statahead_batch_max=0 llite.lustre-ffff9550c51f0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9550c51f0000.stats=clear mdc.lustre-MDT0001-mdc-ffff9550c51f0000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m34.507s user 0m0.168s sys 0m33.078s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9550c51f0000.statahead_batch_max=32 llite.lustre-ffff9550c51f0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9550c51f0000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9550c51f0000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9550c51f0000.stats=clear mdc.lustre-MDT0001-mdc-ffff9550c51f0000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m39.035s user 0m0.175s sys 0m23.055s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9550c51f0000.batch_stats= snapshot_time: 1783076301.964383397 secs.nsecs start_time: 1783076257.034127045 secs.nsecs elapsed_time: 44.930256352 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff9550c51f0000.batch_stats= snapshot_time: 1783076301.964441865 secs.nsecs start_time: 1783076257.034173672 secs.nsecs elapsed_time: 44.930268193 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9550c51f0000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9550c51f0000.batch_stats=0 llite.lustre-ffff9550c51f0000.statahead_max=2048 llite.lustre-ffff9550c51f0000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9550c51f0000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9550c51f0000.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 0.79 seconds: 126.33 ops/second llite.lustre-ffff9550c51f0000.statahead_max=0 101 real 0m0.599s user 0m0.003s sys 0m0.233s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff9550c51f0000.statahead_max=2048 llite.lustre-ffff9550c51f0000.statahead_max=2048 101 real 0m0.680s user 0m0.005s sys 0m0.361s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 2 swrong=0 ewrong=0 total: 900 open/close in 1.91 seconds: 470.00 ops/second llite.lustre-ffff9550c51f0000.statahead_max=0 1001 real 0m6.607s user 0m0.022s sys 0m2.608s 'ls -l' 1000 files without statahead: 7 sec llite.lustre-ffff9550c51f0000.statahead_max=2048 llite.lustre-ffff9550c51f0000.statahead_max=2048 1001 real 0m3.402s user 0m0.014s sys 0m1.449s 'ls -l' 1000 files with statahead: 4 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 5 seconds llite.lustre-ffff9550c51f0000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 mdc.lustre-MDT0000-mdc-ffff9550c51f0000.batch_stats= snapshot_time: 1783076405.693540675 secs.nsecs start_time: 1783076302.824101606 secs.nsecs elapsed_time: 102.869439069 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 2 28 28 8: 2 28 57 16: 0 0 57 32: 0 0 57 64: 1 14 71 128: 0 0 71 256: 2 28 100 mdc.lustre-MDT0001-mdc-ffff9550c51f0000.batch_stats= snapshot_time: 1783076405.693586485 secs.nsecs start_time: 1783076302.824232346 secs.nsecs elapsed_time: 102.869354139 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 14 14 4: 2 28 42 8: 1 14 57 16: 0 0 57 32: 0 0 57 64: 1 14 71 128: 0 0 71 256: 2 28 100 - open/close 5206 (time 1783076417.10 total 10.00 last 520.56) total: 10000 open/close in 18.69 seconds: 534.92 ops/second llite.lustre-ffff9550c51f0000.statahead_batch_max=0 llite.lustre-ffff9550c51f0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9550c51f0000.stats=clear mdc.lustre-MDT0001-mdc-ffff9550c51f0000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m30.148s user 0m0.144s sys 0m26.495s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9550c51f0000.statahead_batch_max=256 llite.lustre-ffff9550c51f0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9550c51f0000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9550c51f0000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9550c51f0000.stats=clear mdc.lustre-MDT0001-mdc-ffff9550c51f0000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m30.402s user 0m0.149s sys 0m21.882s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9550c51f0000.batch_stats= snapshot_time: 1783076495.394130486 secs.nsecs start_time: 1783076458.778037383 secs.nsecs elapsed_time: 36.616093103 secs.nsecs subreqs per batch batches %% cum %% 1: 342 69 69 2: 61 12 81 4: 27 5 86 8: 5 1 87 16: 1 0 88 32: 20 4 92 64: 5 1 93 128: 0 0 93 256: 34 6 100 mdc.lustre-MDT0001-mdc-ffff9550c51f0000.batch_stats= snapshot_time: 1783076495.394172499 secs.nsecs start_time: 1783076458.778082806 secs.nsecs elapsed_time: 36.616089693 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9550c51f0000.statahead_batch_max=64 llite.lustre-ffff9550c51f0000.statahead_max=128