== sanity test 123ad: Verify batching statahead works correctly ========================================================== 22:08:10 (1789783690) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9732c8582800.batch_stats=0 llite.lustre-ffff9732c8582800.statahead_max=32 llite.lustre-ffff9732c8582800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9732c8582800.batch_stats=0 total: 100 open/close in 3.04 seconds: 32.89 ops/second llite.lustre-ffff9732c8582800.statahead_max=0 101 real 0m2.300s user 0m0.003s sys 0m0.279s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9732c8582800.statahead_max=32 llite.lustre-ffff9732c8582800.statahead_max=32 101 real 0m1.145s user 0m0.005s sys 0m0.146s 'ls -l' 100 files with statahead: 1 sec statahead total: 120 statahead wrong: 2 agl total: 114 list_total: 30 fname_total: 90 hit_total: 4582 miss_total: 761 swrong=2 ewrong=2 - open/close 696 (time 1789783719.54 total 10.01 last 69.55) total: 900 open/close in 12.25 seconds: 73.47 ops/second llite.lustre-ffff9732c8582800.statahead_max=0 1001 real 0m19.371s user 0m0.041s sys 0m2.710s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff9732c8582800.statahead_max=32 llite.lustre-ffff9732c8582800.statahead_max=32 1001 real 0m10.972s user 0m0.026s sys 0m1.306s 'ls -l' 1000 files with statahead: 11 sec statahead total: 121 statahead wrong: 2 agl total: 115 list_total: 31 fname_total: 90 hit_total: 5580 miss_total: 763 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff9732c8582800.statahead_stats= statahead total: 121 statahead wrong: 2 agl total: 115 list_total: 31 fname_total: 90 hit_total: 5580 miss_total: 763 mdc.lustre-MDT0000-mdc-ffff9732c8582800.batch_stats= snapshot_time: 1789783791.775811005 secs.nsecs start_time: 1789783695.575890559 secs.nsecs elapsed_time: 96.199920446 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 954 (time 1789783803.91 total 10.01 last 95.33) - open/close 1885 (time 1789783813.91 total 20.01 last 93.09) - open/close 2762 (time 1789783823.91 total 30.01 last 87.69) - open/close 3672 (time 1789783833.91 total 40.01 last 90.97) - open/close 4589 (time 1789783843.92 total 50.02 last 91.65) - open/close 5499 (time 1789783853.92 total 60.02 last 90.97) - open/close 6274 (time 1789783863.93 total 70.03 last 77.42) - open/close 6936 (time 1789783873.94 total 80.03 last 66.18) - open/close 7752 (time 1789783883.96 total 90.06 last 81.42) - open/close 8537 (time 1789783893.96 total 100.06 last 78.49) - open/close 9285 (time 1789783903.96 total 110.06 last 74.79) total: 10000 open/close in 118.92 seconds: 84.09 ops/second llite.lustre-ffff9732c8582800.statahead_batch_max=0 llite.lustre-ffff9732c8582800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9732c8582800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m10.020s user 0m0.337s sys 0m13.665s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9732c8582800.statahead_batch_max=32 llite.lustre-ffff9732c8582800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9732c8582800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9732c8582800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m56.777s user 0m0.263s sys 0m13.058s batched statahead RPC= hit count=9977 mdc.lustre-MDT0000-mdc-ffff9732c8582800.batch_stats= snapshot_time: 1789784124.581728587 secs.nsecs start_time: 1789783992.619217535 secs.nsecs elapsed_time: 131.962511052 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9732c8582800.batch_stats=0 llite.lustre-ffff9732c8582800.statahead_max=2048 llite.lustre-ffff9732c8582800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9732c8582800.batch_stats=0 total: 100 open/close in 2.84 seconds: 35.19 ops/second llite.lustre-ffff9732c8582800.statahead_max=0 101 real 0m3.220s user 0m0.006s sys 0m0.310s 'ls -l' 100 files without statahead: 4 sec llite.lustre-ffff9732c8582800.statahead_max=2048 llite.lustre-ffff9732c8582800.statahead_max=2048 101 real 0m1.815s user 0m0.004s sys 0m0.169s 'ls -l' 100 files with statahead: 2 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10075 miss_total: 25 swrong=0 ewrong=0 - open/close 553 (time 1789784440.99 total 10.00 last 55.29) total: 900 open/close in 17.63 seconds: 51.05 ops/second llite.lustre-ffff9732c8582800.statahead_max=0 1001 real 0m26.433s user 0m0.041s sys 0m3.052s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff9732c8582800.statahead_max=2048 llite.lustre-ffff9732c8582800.statahead_max=2048 1001 real 0m7.214s user 0m0.029s sys 0m1.289s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11074 miss_total: 26 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff9732c8582800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11074 miss_total: 26 mdc.lustre-MDT0000-mdc-ffff9732c8582800.batch_stats= snapshot_time: 1789784520.573074531 secs.nsecs start_time: 1789784127.019491912 secs.nsecs elapsed_time: 393.553582619 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 14 14 16: 2 14 28 32: 3 21 50 64: 2 14 64 128: 2 14 78 256: 3 21 100 - open/close 796 (time 1789784533.22 total 10.01 last 79.54) - open/close 1608 (time 1789784543.23 total 20.02 last 81.07) - open/close 2161 (time 1789784553.25 total 30.04 last 55.20) - open/close 2814 (time 1789784563.26 total 40.05 last 65.23) - open/close 3436 (time 1789784573.27 total 50.06 last 62.12) - open/close 4243 (time 1789784583.28 total 60.07 last 80.66) - open/close 5087 (time 1789784593.28 total 70.07 last 84.38) - open/close 5763 (time 1789784603.33 total 80.12 last 67.30) - open/close 6370 (time 1789784613.33 total 90.12 last 60.66) - open/close 7417 (time 1789784623.33 total 100.12 last 104.70) - open/close 8182 (time 1789784633.33 total 110.12 last 76.49) - open/close 9030 (time 1789784643.34 total 120.13 last 84.75) - open/close 9977 (time 1789784653.35 total 130.14 last 94.62) total: 10000 open/close in 130.33 seconds: 76.73 ops/second llite.lustre-ffff9732c8582800.statahead_batch_max=0 llite.lustre-ffff9732c8582800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9732c8582800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m4.717s user 0m0.303s sys 0m13.622s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9732c8582800.statahead_batch_max=256 llite.lustre-ffff9732c8582800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9732c8582800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9732c8582800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.350s user 0m0.276s sys 0m13.263s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9732c8582800.batch_stats= snapshot_time: 1789784797.621997559 secs.nsecs start_time: 1789784729.034310619 secs.nsecs elapsed_time: 68.587686940 secs.nsecs subreqs per batch batches %% cum %% 1: 98 41 41 2: 34 14 56 4: 23 9 65 8: 14 5 71 16: 13 5 77 32: 10 4 81 64: 7 2 84 128: 0 0 84 256: 36 15 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9732c8582800.statahead_batch_max=64 llite.lustre-ffff9732c8582800.statahead_max=128