== sanity test 123ad: Verify batching statahead works correctly ========================================================== 09:15:01 (1789132501) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff996f9061f800.batch_stats=0 llite.lustre-ffff996f9061f800.statahead_max=32 llite.lustre-ffff996f9061f800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff996f9061f800.batch_stats=0 total: 100 open/close in 2.41 seconds: 41.41 ops/second llite.lustre-ffff996f9061f800.statahead_max=0 101 real 0m1.778s user 0m0.004s sys 0m0.291s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff996f9061f800.statahead_max=32 llite.lustre-ffff996f9061f800.statahead_max=32 101 real 0m0.813s user 0m0.005s sys 0m0.168s 'ls -l' 100 files with statahead: 0 sec statahead total: 126 statahead wrong: 2 agl total: 121 list_total: 30 fname_total: 84 hit_total: 3554 miss_total: 1566 swrong=2 ewrong=2 - open/close 703 (time 1789132526.43 total 10.01 last 70.20) total: 900 open/close in 13.21 seconds: 68.14 ops/second llite.lustre-ffff996f9061f800.statahead_max=0 1001 real 0m19.312s user 0m0.032s sys 0m2.684s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff996f9061f800.statahead_max=32 llite.lustre-ffff996f9061f800.statahead_max=32 1001 real 0m8.718s user 0m0.029s sys 0m1.317s 'ls -l' 1000 files with statahead: 9 sec statahead total: 127 statahead wrong: 2 agl total: 122 list_total: 31 fname_total: 84 hit_total: 4553 miss_total: 1567 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 17 seconds llite.lustre-ffff996f9061f800.statahead_stats= statahead total: 127 statahead wrong: 2 agl total: 122 list_total: 31 fname_total: 84 hit_total: 4553 miss_total: 1567 mdc.lustre-MDT0000-mdc-ffff996f9061f800.batch_stats= snapshot_time: 1789132587.706089812 secs.nsecs start_time: 1789132505.940298035 secs.nsecs elapsed_time: 81.765791777 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 867 (time 1789132599.99 total 10.00 last 86.70) - open/close 1603 (time 1789132609.99 total 20.00 last 73.60) - open/close 2331 (time 1789132619.99 total 30.00 last 72.79) - open/close 3151 (time 1789132630.00 total 40.01 last 81.96) - open/close 3923 (time 1789132640.00 total 50.01 last 77.14) - open/close 4806 (time 1789132650.01 total 60.02 last 88.25) - open/close 5650 (time 1789132660.02 total 70.03 last 84.34) - open/close 6542 (time 1789132670.03 total 80.04 last 89.12) - open/close 7395 (time 1789132680.05 total 90.06 last 85.11) - open/close 8271 (time 1789132690.08 total 100.09 last 87.33) - open/close 9185 (time 1789132700.08 total 110.10 last 91.34) total: 10000 open/close in 119.24 seconds: 83.86 ops/second llite.lustre-ffff996f9061f800.statahead_batch_max=0 llite.lustre-ffff996f9061f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff996f9061f800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m52.444s user 0m0.270s sys 0m14.448s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff996f9061f800.statahead_batch_max=32 llite.lustre-ffff996f9061f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff996f9061f800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff996f9061f800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m27.372s user 0m0.277s sys 0m13.076s batched statahead RPC= hit count=9986 mdc.lustre-MDT0000-mdc-ffff996f9061f800.batch_stats= snapshot_time: 1789132867.003824448 secs.nsecs start_time: 1789132769.235327259 secs.nsecs elapsed_time: 97.768497189 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 0 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff996f9061f800.batch_stats=0 llite.lustre-ffff996f9061f800.statahead_max=2048 llite.lustre-ffff996f9061f800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff996f9061f800.batch_stats=0 total: 100 open/close in 2.56 seconds: 39.14 ops/second llite.lustre-ffff996f9061f800.statahead_max=0 101 real 0m1.930s user 0m0.004s sys 0m0.300s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff996f9061f800.statahead_max=2048 llite.lustre-ffff996f9061f800.statahead_max=2048 101 real 0m0.955s user 0m0.005s sys 0m0.159s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10085 miss_total: 15 swrong=0 ewrong=0 - open/close 836 (time 1789133155.14 total 10.01 last 83.53) total: 900 open/close in 10.81 seconds: 83.28 ops/second llite.lustre-ffff996f9061f800.statahead_max=0 1001 real 0m19.080s user 0m0.026s sys 0m2.734s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff996f9061f800.statahead_max=2048 llite.lustre-ffff996f9061f800.statahead_max=2048 1001 real 0m5.741s user 0m0.028s sys 0m1.285s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff996f9061f800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 mdc.lustre-MDT0000-mdc-ffff996f9061f800.batch_stats= snapshot_time: 1789133210.727119857 secs.nsecs start_time: 1789132869.376495897 secs.nsecs elapsed_time: 341.350623960 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 1 9 45 64: 0 0 45 128: 3 27 72 256: 3 27 100 - open/close 769 (time 1789133223.25 total 10.01 last 76.82) - open/close 1497 (time 1789133233.26 total 20.01 last 72.78) - open/close 2200 (time 1789133243.26 total 30.02 last 70.27) - open/close 2924 (time 1789133253.27 total 40.03 last 72.33) - open/close 3770 (time 1789133263.27 total 50.03 last 84.56) - open/close 4563 (time 1789133273.28 total 60.04 last 79.23) - open/close 5344 (time 1789133283.30 total 70.05 last 78.00) - open/close 6090 (time 1789133293.30 total 80.06 last 74.55) - open/close 6840 (time 1789133303.32 total 90.08 last 74.87) - open/close 7607 (time 1789133313.32 total 100.08 last 76.69) - open/close 8390 (time 1789133323.32 total 110.08 last 78.28) - open/close 9169 (time 1789133333.33 total 120.09 last 77.84) - open/close 9893 (time 1789133343.35 total 130.11 last 72.25) total: 10000 open/close in 131.77 seconds: 75.89 ops/second llite.lustre-ffff996f9061f800.statahead_batch_max=0 llite.lustre-ffff996f9061f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff996f9061f800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.720s user 0m0.313s sys 0m15.285s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff996f9061f800.statahead_batch_max=256 llite.lustre-ffff996f9061f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff996f9061f800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff996f9061f800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m50.564s user 0m0.297s sys 0m14.587s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff996f9061f800.batch_stats= snapshot_time: 1789133472.403222179 secs.nsecs start_time: 1789133409.182876231 secs.nsecs elapsed_time: 63.220345948 secs.nsecs subreqs per batch batches %% cum %% 1: 111 47 47 2: 33 13 61 4: 17 7 68 8: 10 4 72 16: 9 3 76 32: 16 6 83 64: 4 1 84 128: 1 0 85 256: 35 14 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff996f9061f800.statahead_batch_max=64 llite.lustre-ffff996f9061f800.statahead_max=128