== sanity test 123ad: Verify batching statahead works correctly ========================================================== 10:47:43 (1789138063) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff90a184d5c800.batch_stats=0 llite.lustre-ffff90a184d5c800.statahead_max=32 llite.lustre-ffff90a184d5c800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff90a184d5c800.batch_stats=0 total: 100 open/close in 2.63 seconds: 38.04 ops/second llite.lustre-ffff90a184d5c800.statahead_max=0 101 real 0m2.391s user 0m0.004s sys 0m0.306s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff90a184d5c800.statahead_max=32 llite.lustre-ffff90a184d5c800.statahead_max=32 101 real 0m1.106s user 0m0.003s sys 0m0.153s 'ls -l' 100 files with statahead: 2 sec statahead total: 143 statahead wrong: 2 agl total: 137 list_total: 30 fname_total: 109 hit_total: 4200 miss_total: 1288 swrong=2 ewrong=2 - open/close 680 (time 1789138091.48 total 10.00 last 67.97) total: 900 open/close in 12.99 seconds: 69.28 ops/second llite.lustre-ffff90a184d5c800.statahead_max=0 1001 real 0m18.791s user 0m0.030s sys 0m2.828s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff90a184d5c800.statahead_max=32 llite.lustre-ffff90a184d5c800.statahead_max=32 1001 real 0m9.055s user 0m0.026s sys 0m1.229s 'ls -l' 1000 files with statahead: 10 sec statahead total: 144 statahead wrong: 2 agl total: 138 list_total: 31 fname_total: 109 hit_total: 5197 miss_total: 1291 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff90a184d5c800.statahead_stats= statahead total: 144 statahead wrong: 2 agl total: 138 list_total: 31 fname_total: 109 hit_total: 5197 miss_total: 1291 mdc.lustre-MDT0000-mdc-ffff90a184d5c800.batch_stats= snapshot_time: 1789138155.078004194 secs.nsecs start_time: 1789138068.399462629 secs.nsecs elapsed_time: 86.678541565 secs.nsecs subreqs per batch batches %% cum %% 1: 2 4 4 2: 0 0 4 4: 0 0 4 8: 3 7 12 16: 2 4 17 32: 34 82 100 - open/close 830 (time 1789138167.76 total 10.01 last 82.91) - open/close 1449 (time 1789138177.78 total 20.03 last 61.78) - open/close 2226 (time 1789138187.79 total 30.03 last 77.67) - open/close 3030 (time 1789138197.80 total 40.04 last 80.32) - open/close 3805 (time 1789138207.80 total 50.05 last 77.44) - open/close 4723 (time 1789138217.81 total 60.05 last 91.77) - open/close 5629 (time 1789138227.81 total 70.06 last 90.56) - open/close 6410 (time 1789138237.82 total 80.06 last 78.07) - open/close 7339 (time 1789138247.82 total 90.07 last 92.85) - open/close 8183 (time 1789138257.83 total 100.08 last 84.32) - open/close 8896 (time 1789138267.84 total 110.09 last 71.21) - open/close 9780 (time 1789138277.86 total 120.10 last 88.28) total: 10000 open/close in 122.34 seconds: 81.74 ops/second llite.lustre-ffff90a184d5c800.statahead_batch_max=0 llite.lustre-ffff90a184d5c800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff90a184d5c800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.086s user 0m0.278s sys 0m13.222s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff90a184d5c800.statahead_batch_max=32 llite.lustre-ffff90a184d5c800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff90a184d5c800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff90a184d5c800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m43.818s user 0m0.287s sys 0m13.014s batched statahead RPC= hit count=9977 mdc.lustre-MDT0000-mdc-ffff90a184d5c800.batch_stats= snapshot_time: 1789138462.916648288 secs.nsecs start_time: 1789138348.226065955 secs.nsecs elapsed_time: 114.690582333 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff90a184d5c800.batch_stats=0 llite.lustre-ffff90a184d5c800.statahead_max=2048 llite.lustre-ffff90a184d5c800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff90a184d5c800.batch_stats=0 total: 100 open/close in 2.74 seconds: 36.43 ops/second llite.lustre-ffff90a184d5c800.statahead_max=0 101 real 0m2.972s user 0m0.002s sys 0m0.325s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff90a184d5c800.statahead_max=2048 llite.lustre-ffff90a184d5c800.statahead_max=2048 101 real 0m1.170s user 0m0.003s sys 0m0.170s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10076 miss_total: 24 swrong=0 ewrong=0 - open/close 726 (time 1789138803.60 total 10.00 last 72.59) total: 900 open/close in 12.37 seconds: 72.78 ops/second llite.lustre-ffff90a184d5c800.statahead_max=0 1001 real 0m24.331s user 0m0.040s sys 0m3.054s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff90a184d5c800.statahead_max=2048 llite.lustre-ffff90a184d5c800.statahead_max=2048 1001 real 0m8.187s user 0m0.027s sys 0m1.390s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11075 miss_total: 25 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff90a184d5c800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11075 miss_total: 25 mdc.lustre-MDT0000-mdc-ffff90a184d5c800.batch_stats= snapshot_time: 1789138878.392751536 secs.nsecs start_time: 1789138466.052253997 secs.nsecs elapsed_time: 412.340497539 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 22 22 16: 1 11 33 32: 1 11 44 64: 0 0 44 128: 1 11 55 256: 4 44 100 - open/close 576 (time 1789138891.63 total 10.01 last 57.52) - open/close 1208 (time 1789138901.65 total 20.03 last 63.11) - open/close 1856 (time 1789138911.65 total 30.03 last 64.77) - open/close 2431 (time 1789138921.65 total 40.03 last 57.49) - open/close 2958 (time 1789138931.67 total 50.05 last 52.60) - open/close 3588 (time 1789138941.68 total 60.06 last 62.93) - open/close 4263 (time 1789138951.69 total 70.08 last 67.42) - open/close 5053 (time 1789138961.71 total 80.09 last 78.90) - open/close 5777 (time 1789138971.71 total 90.09 last 72.35) - open/close 6554 (time 1789138981.72 total 100.10 last 77.67) - open/close 7172 (time 1789138991.72 total 110.10 last 61.78) - open/close 7736 (time 1789139001.73 total 120.11 last 56.35) - open/close 8284 (time 1789139011.73 total 130.11 last 54.78) - open/close 8793 (time 1789139021.76 total 140.14 last 50.74) - open/close 9456 (time 1789139031.77 total 150.15 last 66.24) total: 10000 open/close in 158.88 seconds: 62.94 ops/second llite.lustre-ffff90a184d5c800.statahead_batch_max=0 llite.lustre-ffff90a184d5c800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff90a184d5c800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.096s user 0m0.322s sys 0m14.439s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff90a184d5c800.statahead_batch_max=256 llite.lustre-ffff90a184d5c800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff90a184d5c800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff90a184d5c800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.245s user 0m0.367s sys 0m14.266s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff90a184d5c800.batch_stats= snapshot_time: 1789139182.399501811 secs.nsecs start_time: 1789139111.680549795 secs.nsecs elapsed_time: 70.718952016 secs.nsecs subreqs per batch batches %% cum %% 1: 99 43 43 2: 21 9 52 4: 23 10 62 8: 17 7 70 16: 9 3 74 32: 19 8 82 64: 3 1 84 128: 1 0 84 256: 35 15 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff90a184d5c800.statahead_batch_max=64 llite.lustre-ffff90a184d5c800.statahead_max=128