== sanity test 123ad: Verify batching statahead works correctly ========================================================== 05:37:26 (1784281046) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8ceacb58e800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8ceacb58e800.batch_stats=0 llite.lustre-ffff8ceacb58e800.statahead_max=32 llite.lustre-ffff8ceacb58e800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8ceacb58e800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8ceacb58e800.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 1.73 seconds: 57.66 ops/second llite.lustre-ffff8ceacb58e800.statahead_max=0 101 real 0m1.622s user 0m0.005s sys 0m0.331s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8ceacb58e800.statahead_max=32 llite.lustre-ffff8ceacb58e800.statahead_max=32 101 real 0m0.657s user 0m0.002s sys 0m0.234s 'ls -l' 100 files with statahead: 1 sec statahead total: 115 statahead wrong: 2 agl total: 110 list_total: 30 fname_total: 84 hit_total: 4334 miss_total: 769 swrong=2 ewrong=2 total: 900 open/close in 7.45 seconds: 120.85 ops/second llite.lustre-ffff8ceacb58e800.statahead_max=0 1001 real 0m13.912s user 0m0.024s sys 0m3.023s 'ls -l' 1000 files without statahead: 14 sec llite.lustre-ffff8ceacb58e800.statahead_max=32 llite.lustre-ffff8ceacb58e800.statahead_max=32 1001 real 0m5.249s user 0m0.019s sys 0m1.700s 'ls -l' 1000 files with statahead: 5 sec statahead total: 116 statahead wrong: 2 agl total: 111 list_total: 31 fname_total: 84 hit_total: 5332 miss_total: 771 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 11 seconds llite.lustre-ffff8ceacb58e800.statahead_stats= statahead total: 116 statahead wrong: 2 agl total: 111 list_total: 31 fname_total: 84 hit_total: 5332 miss_total: 771 mdc.lustre-MDT0000-mdc-ffff8ceacb58e800.batch_stats= snapshot_time: 1784281101.202854832 secs.nsecs start_time: 1784281048.971298292 secs.nsecs elapsed_time: 52.231556540 secs.nsecs subreqs per batch batches %% cum %% 1: 4 10 10 2: 2 5 15 4: 1 2 17 8: 2 5 23 16: 24 61 84 32: 6 15 100 mdc.lustre-MDT0001-mdc-ffff8ceacb58e800.batch_stats= snapshot_time: 1784281101.202919734 secs.nsecs start_time: 1784281048.971470938 secs.nsecs elapsed_time: 52.231448796 secs.nsecs subreqs per batch batches %% cum %% 1: 2 5 5 2: 0 0 5 4: 1 2 7 8: 3 7 15 16: 11 27 42 32: 23 57 100 - open/close 1417 (time 1784281112.62 total 10.01 last 141.61) - open/close 2805 (time 1784281122.62 total 20.01 last 138.74) - open/close 4092 (time 1784281132.63 total 30.01 last 128.65) - open/close 5510 (time 1784281142.63 total 40.02 last 141.73) - open/close 6926 (time 1784281152.64 total 50.02 last 141.54) - open/close 8257 (time 1784281162.64 total 60.02 last 133.10) - open/close 9534 (time 1784281172.65 total 70.03 last 127.59) total: 10000 open/close in 73.96 seconds: 135.21 ops/second llite.lustre-ffff8ceacb58e800.statahead_batch_max=0 llite.lustre-ffff8ceacb58e800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8ceacb58e800.stats=clear mdc.lustre-MDT0001-mdc-ffff8ceacb58e800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m39.327s user 0m0.319s sys 0m27.805s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8ceacb58e800.statahead_batch_max=32 llite.lustre-ffff8ceacb58e800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8ceacb58e800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8ceacb58e800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8ceacb58e800.stats=clear mdc.lustre-MDT0001-mdc-ffff8ceacb58e800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m51.785s user 0m0.262s sys 0m19.117s batched statahead RPC= hit count=9988 mdc.lustre-MDT0000-mdc-ffff8ceacb58e800.batch_stats= snapshot_time: 1784281281.185242787 secs.nsecs start_time: 1784281221.192191689 secs.nsecs elapsed_time: 59.993051098 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff8ceacb58e800.batch_stats= snapshot_time: 1784281281.185286900 secs.nsecs start_time: 1784281221.192275300 secs.nsecs elapsed_time: 59.993011600 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8ceacb58e800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8ceacb58e800.batch_stats=0 llite.lustre-ffff8ceacb58e800.statahead_max=2048 llite.lustre-ffff8ceacb58e800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8ceacb58e800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8ceacb58e800.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.57 seconds: 63.59 ops/second llite.lustre-ffff8ceacb58e800.statahead_max=0 101 real 0m1.413s user 0m0.006s sys 0m0.332s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8ceacb58e800.statahead_max=2048 llite.lustre-ffff8ceacb58e800.statahead_max=2048 101 real 0m0.490s user 0m0.005s sys 0m0.206s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10087 miss_total: 13 swrong=0 ewrong=0 total: 900 open/close in 6.79 seconds: 132.46 ops/second llite.lustre-ffff8ceacb58e800.statahead_max=0 1001 real 0m13.531s user 0m0.034s sys 0m2.960s 'ls -l' 1000 files without statahead: 14 sec llite.lustre-ffff8ceacb58e800.statahead_max=2048 llite.lustre-ffff8ceacb58e800.statahead_max=2048 1001 real 0m4.107s user 0m0.022s sys 0m1.580s 'ls -l' 1000 files with statahead: 4 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11086 miss_total: 14 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 10 seconds llite.lustre-ffff8ceacb58e800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11086 miss_total: 14 mdc.lustre-MDT0000-mdc-ffff8ceacb58e800.batch_stats= snapshot_time: 1784281503.282653691 secs.nsecs start_time: 1784281282.391798094 secs.nsecs elapsed_time: 220.890855597 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 3 42 42 8: 1 14 57 16: 0 0 57 32: 0 0 57 64: 1 14 71 128: 0 0 71 256: 2 28 100 mdc.lustre-MDT0001-mdc-ffff8ceacb58e800.batch_stats= snapshot_time: 1784281503.282717343 secs.nsecs start_time: 1784281282.391936359 secs.nsecs elapsed_time: 220.890780984 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 14 14 8: 3 42 57 16: 0 0 57 32: 0 0 57 64: 1 14 71 128: 0 0 71 256: 2 28 100 - open/close 1334 (time 1784281514.61 total 10.00 last 133.35) - open/close 2818 (time 1784281524.62 total 20.01 last 148.30) - open/close 4177 (time 1784281534.62 total 30.01 last 135.85) - open/close 5521 (time 1784281544.63 total 40.02 last 134.32) - open/close 6931 (time 1784281554.64 total 50.03 last 140.90) - open/close 8309 (time 1784281564.64 total 60.03 last 137.73) - open/close 9905 (time 1784281574.64 total 70.03 last 159.57) total: 10000 open/close in 70.62 seconds: 141.61 ops/second llite.lustre-ffff8ceacb58e800.statahead_batch_max=0 llite.lustre-ffff8ceacb58e800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8ceacb58e800.stats=clear mdc.lustre-MDT0001-mdc-ffff8ceacb58e800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m35.597s user 0m0.277s sys 0m27.096s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8ceacb58e800.statahead_batch_max=256 llite.lustre-ffff8ceacb58e800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8ceacb58e800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8ceacb58e800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8ceacb58e800.stats=clear mdc.lustre-MDT0001-mdc-ffff8ceacb58e800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m33.427s user 0m0.243s sys 0m21.877s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8ceacb58e800.batch_stats= snapshot_time: 1784281655.802526768 secs.nsecs start_time: 1784281614.851773375 secs.nsecs elapsed_time: 40.950753393 secs.nsecs subreqs per batch batches %% cum %% 1: 255 61 61 2: 47 11 72 4: 37 8 81 8: 12 2 84 16: 6 1 86 32: 14 3 89 64: 10 2 91 128: 0 0 91 256: 34 8 100 mdc.lustre-MDT0001-mdc-ffff8ceacb58e800.batch_stats= snapshot_time: 1784281655.802596848 secs.nsecs start_time: 1784281614.851917389 secs.nsecs elapsed_time: 40.950679459 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8ceacb58e800.statahead_batch_max=64 llite.lustre-ffff8ceacb58e800.statahead_max=128