== sanity test 123ad: Verify batching statahead works correctly ========================================================== 09:47:29 (1788184049) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8e7789db6800.batch_stats=0 llite.lustre-ffff8e7789db6800.statahead_max=32 llite.lustre-ffff8e7789db6800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8e7789db6800.batch_stats=0 total: 100 open/close in 2.78 seconds: 36.01 ops/second llite.lustre-ffff8e7789db6800.statahead_max=0 101 real 0m2.329s user 0m0.004s sys 0m0.333s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8e7789db6800.statahead_max=32 llite.lustre-ffff8e7789db6800.statahead_max=32 101 real 0m1.266s user 0m0.002s sys 0m0.170s 'ls -l' 100 files with statahead: 1 sec statahead total: 111 statahead wrong: 2 agl total: 106 list_total: 29 fname_total: 63 hit_total: 4596 miss_total: 1271 swrong=2 ewrong=2 - open/close 699 (time 1788184076.72 total 10.02 last 69.75) total: 900 open/close in 13.04 seconds: 69.02 ops/second llite.lustre-ffff8e7789db6800.statahead_max=0 1001 real 0m23.589s user 0m0.032s sys 0m3.414s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff8e7789db6800.statahead_max=32 llite.lustre-ffff8e7789db6800.statahead_max=32 1001 real 0m10.870s user 0m0.027s sys 0m1.640s 'ls -l' 1000 files with statahead: 11 sec statahead total: 112 statahead wrong: 2 agl total: 107 list_total: 30 fname_total: 63 hit_total: 5595 miss_total: 1272 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff8e7789db6800.statahead_stats= statahead total: 112 statahead wrong: 2 agl total: 107 list_total: 30 fname_total: 63 hit_total: 5595 miss_total: 1272 mdc.lustre-MDT0000-mdc-ffff8e7789db6800.batch_stats= snapshot_time: 1788184148.751354081 secs.nsecs start_time: 1788184054.249523629 secs.nsecs elapsed_time: 94.501830452 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 2 2 8: 2 5 7 16: 1 2 10 32: 34 89 100 - open/close 691 (time 1788184161.51 total 10.01 last 69.05) - open/close 1418 (time 1788184171.52 total 20.01 last 72.65) - open/close 2124 (time 1788184181.53 total 30.03 last 70.51) - open/close 2809 (time 1788184191.55 total 40.05 last 68.34) - open/close 3483 (time 1788184201.56 total 50.06 last 67.35) - open/close 4117 (time 1788184211.57 total 60.07 last 63.35) - open/close 4736 (time 1788184221.58 total 70.08 last 61.83) - open/close 5369 (time 1788184231.59 total 80.09 last 63.21) - open/close 6053 (time 1788184241.60 total 90.09 last 68.38) - open/close 6791 (time 1788184251.61 total 100.10 last 73.74) - open/close 7474 (time 1788184261.61 total 110.11 last 68.28) - open/close 8167 (time 1788184271.62 total 120.11 last 69.25) - open/close 8914 (time 1788184281.62 total 130.12 last 74.65) - open/close 9600 (time 1788184291.63 total 140.12 last 68.58) total: 10000 open/close in 145.35 seconds: 68.80 ops/second llite.lustre-ffff8e7789db6800.statahead_batch_max=0 llite.lustre-ffff8e7789db6800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8e7789db6800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.048s user 0m0.299s sys 0m18.284s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8e7789db6800.statahead_batch_max=32 llite.lustre-ffff8e7789db6800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8e7789db6800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8e7789db6800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m36.759s user 0m0.310s sys 0m16.778s batched statahead RPC= hit count=9984 mdc.lustre-MDT0000-mdc-ffff8e7789db6800.batch_stats= snapshot_time: 1788184472.977363239 secs.nsecs start_time: 1788184363.855698757 secs.nsecs elapsed_time: 109.121664482 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8e7789db6800.batch_stats=0 llite.lustre-ffff8e7789db6800.statahead_max=2048 llite.lustre-ffff8e7789db6800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8e7789db6800.batch_stats=0 total: 100 open/close in 2.56 seconds: 39.13 ops/second llite.lustre-ffff8e7789db6800.statahead_max=0 101 real 0m2.203s user 0m0.003s sys 0m0.341s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8e7789db6800.statahead_max=2048 llite.lustre-ffff8e7789db6800.statahead_max=2048 101 real 0m0.701s user 0m0.006s sys 0m0.224s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10083 miss_total: 17 swrong=0 ewrong=0 - open/close 684 (time 1788184809.95 total 10.01 last 68.36) total: 900 open/close in 13.09 seconds: 68.76 ops/second llite.lustre-ffff8e7789db6800.statahead_max=0 1001 real 0m21.370s user 0m0.031s sys 0m3.111s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff8e7789db6800.statahead_max=2048 llite.lustre-ffff8e7789db6800.statahead_max=2048 1001 real 0m6.886s user 0m0.026s sys 0m1.587s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff8e7789db6800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 mdc.lustre-MDT0000-mdc-ffff8e7789db6800.batch_stats= snapshot_time: 1788184880.373692806 secs.nsecs start_time: 1788184475.522921109 secs.nsecs elapsed_time: 404.850771697 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 0 0 54 128: 1 9 63 256: 4 36 100 - open/close 687 (time 1788184893.20 total 10.00 last 68.67) - open/close 1411 (time 1788184903.21 total 20.02 last 72.31) - open/close 2114 (time 1788184913.21 total 30.02 last 70.28) - open/close 2775 (time 1788184923.22 total 40.03 last 66.06) - open/close 3407 (time 1788184933.23 total 50.04 last 63.13) - open/close 4061 (time 1788184943.24 total 60.05 last 65.32) - open/close 4747 (time 1788184953.26 total 70.07 last 68.49) - open/close 5481 (time 1788184963.26 total 80.07 last 73.37) - open/close 6180 (time 1788184973.26 total 90.07 last 69.89) - open/close 6863 (time 1788184983.27 total 100.08 last 68.26) - open/close 7439 (time 1788184993.27 total 110.08 last 57.59) - open/close 8058 (time 1788185003.28 total 120.09 last 61.87) - open/close 8786 (time 1788185013.29 total 130.09 last 72.74) - open/close 9432 (time 1788185023.29 total 140.09 last 64.59) total: 10000 open/close in 148.49 seconds: 67.35 ops/second llite.lustre-ffff8e7789db6800.statahead_batch_max=0 llite.lustre-ffff8e7789db6800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8e7789db6800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.420s user 0m0.311s sys 0m16.307s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8e7789db6800.statahead_batch_max=256 llite.lustre-ffff8e7789db6800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8e7789db6800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8e7789db6800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m57.007s user 0m0.287s sys 0m15.474s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8e7789db6800.batch_stats= snapshot_time: 1788185173.434552502 secs.nsecs start_time: 1788185101.873069490 secs.nsecs elapsed_time: 71.561483012 secs.nsecs subreqs per batch batches %% cum %% 1: 154 48 48 2: 55 17 65 4: 26 8 74 8: 15 4 78 16: 14 4 83 32: 15 4 88 64: 3 0 88 128: 0 0 88 256: 35 11 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8e7789db6800.statahead_batch_max=64 llite.lustre-ffff8e7789db6800.statahead_max=128