== sanity test 123ad: Verify batching statahead works correctly ========================================================== 10:08:09 (1788530889) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8dae861d1800.batch_stats=0 llite.lustre-ffff8dae861d1800.statahead_max=32 llite.lustre-ffff8dae861d1800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8dae861d1800.batch_stats=0 total: 100 open/close in 2.87 seconds: 34.80 ops/second llite.lustre-ffff8dae861d1800.statahead_max=0 101 real 0m2.173s user 0m0.003s sys 0m0.286s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8dae861d1800.statahead_max=32 llite.lustre-ffff8dae861d1800.statahead_max=32 101 real 0m1.138s user 0m0.006s sys 0m0.155s 'ls -l' 100 files with statahead: 1 sec statahead total: 129 statahead wrong: 2 agl total: 123 list_total: 29 fname_total: 98 hit_total: 3881 miss_total: 1231 swrong=2 ewrong=2 - open/close 768 (time 1788530918.82 total 10.02 last 76.68) total: 900 open/close in 11.83 seconds: 76.06 ops/second llite.lustre-ffff8dae861d1800.statahead_max=0 1001 real 0m20.646s user 0m0.032s sys 0m2.745s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff8dae861d1800.statahead_max=32 llite.lustre-ffff8dae861d1800.statahead_max=32 1001 real 0m10.687s user 0m0.026s sys 0m1.349s 'ls -l' 1000 files with statahead: 10 sec statahead total: 130 statahead wrong: 2 agl total: 124 list_total: 30 fname_total: 98 hit_total: 4880 miss_total: 1232 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff8dae861d1800.statahead_stats= statahead total: 130 statahead wrong: 2 agl total: 124 list_total: 30 fname_total: 98 hit_total: 4880 miss_total: 1232 mdc.lustre-MDT0000-mdc-ffff8dae861d1800.batch_stats= snapshot_time: 1788530986.776436546 secs.nsecs start_time: 1788530896.423528866 secs.nsecs elapsed_time: 90.352907680 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 732 (time 1788530999.52 total 10.00 last 73.18) - open/close 1531 (time 1788531009.54 total 20.02 last 79.80) - open/close 2206 (time 1788531019.54 total 30.02 last 67.45) - open/close 2961 (time 1788531029.55 total 40.03 last 75.43) - open/close 3718 (time 1788531039.56 total 50.04 last 75.67) - open/close 4270 (time 1788531049.57 total 60.05 last 55.13) - open/close 4899 (time 1788531059.58 total 70.06 last 62.84) - open/close 5543 (time 1788531069.58 total 80.06 last 64.40) - open/close 6243 (time 1788531079.60 total 90.08 last 69.85) - open/close 6934 (time 1788531089.60 total 100.08 last 69.09) - open/close 7652 (time 1788531099.61 total 110.09 last 71.77) - open/close 8432 (time 1788531109.62 total 120.10 last 77.93) - open/close 9195 (time 1788531119.63 total 130.11 last 76.17) - open/close 9999 (time 1788531129.65 total 140.13 last 80.29) total: 10000 open/close in 140.13 seconds: 71.36 ops/second llite.lustre-ffff8dae861d1800.statahead_batch_max=0 llite.lustre-ffff8dae861d1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8dae861d1800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m4.492s user 0m0.295s sys 0m13.956s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8dae861d1800.statahead_batch_max=32 llite.lustre-ffff8dae861d1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8dae861d1800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8dae861d1800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m52.093s user 0m0.316s sys 0m13.394s batched statahead RPC= hit count=9968 mdc.lustre-MDT0000-mdc-ffff8dae861d1800.batch_stats= snapshot_time: 1788531327.316186310 secs.nsecs start_time: 1788531203.025710027 secs.nsecs elapsed_time: 124.290476283 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8dae861d1800.batch_stats=0 llite.lustre-ffff8dae861d1800.statahead_max=2048 llite.lustre-ffff8dae861d1800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8dae861d1800.batch_stats=0 total: 100 open/close in 2.74 seconds: 36.51 ops/second llite.lustre-ffff8dae861d1800.statahead_max=0 101 real 0m2.493s user 0m0.004s sys 0m0.287s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8dae861d1800.statahead_max=2048 llite.lustre-ffff8dae861d1800.statahead_max=2048 101 real 0m0.837s user 0m0.008s sys 0m0.161s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10067 miss_total: 33 swrong=0 ewrong=0 - open/close 750 (time 1788531641.15 total 10.00 last 74.99) total: 900 open/close in 12.08 seconds: 74.52 ops/second llite.lustre-ffff8dae861d1800.statahead_max=0 1001 real 0m23.685s user 0m0.031s sys 0m2.794s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff8dae861d1800.statahead_max=2048 llite.lustre-ffff8dae861d1800.statahead_max=2048 1001 real 0m6.017s user 0m0.028s sys 0m1.457s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11066 miss_total: 34 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff8dae861d1800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11066 miss_total: 34 mdc.lustre-MDT0000-mdc-ffff8dae861d1800.batch_stats= snapshot_time: 1788531704.538562262 secs.nsecs start_time: 1788531330.014030190 secs.nsecs elapsed_time: 374.524532072 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 769 (time 1788531716.95 total 10.01 last 76.80) - open/close 1499 (time 1788531726.95 total 20.02 last 72.96) - open/close 2236 (time 1788531736.95 total 30.02 last 73.70) - open/close 3046 (time 1788531746.96 total 40.03 last 80.89) - open/close 3820 (time 1788531756.97 total 50.03 last 77.38) - open/close 4579 (time 1788531766.97 total 60.04 last 75.86) - open/close 5387 (time 1788531776.98 total 70.05 last 80.74) - open/close 6063 (time 1788531786.99 total 80.06 last 67.55) - open/close 6887 (time 1788531796.99 total 90.06 last 82.37) - open/close 7665 (time 1788531807.00 total 100.07 last 77.71) - open/close 8352 (time 1788531817.01 total 110.08 last 68.67) - open/close 9172 (time 1788531827.01 total 120.08 last 81.96) - open/close 9881 (time 1788531837.02 total 130.09 last 70.85) total: 10000 open/close in 132.03 seconds: 75.74 ops/second llite.lustre-ffff8dae861d1800.statahead_batch_max=0 llite.lustre-ffff8dae861d1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8dae861d1800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m7.269s user 0m0.315s sys 0m13.724s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8dae861d1800.statahead_batch_max=256 llite.lustre-ffff8dae861d1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8dae861d1800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8dae861d1800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m51.357s user 0m0.314s sys 0m13.713s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8dae861d1800.batch_stats= snapshot_time: 1788531975.952729604 secs.nsecs start_time: 1788531915.903634276 secs.nsecs elapsed_time: 60.049095328 secs.nsecs subreqs per batch batches %% cum %% 1: 159 46 46 2: 50 14 60 4: 45 13 73 8: 21 6 79 16: 11 3 83 32: 18 5 88 64: 6 1 90 128: 0 0 90 256: 34 9 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8dae861d1800.statahead_batch_max=64 llite.lustre-ffff8dae861d1800.statahead_max=128