== sanity test 123ad: Verify batching statahead works correctly ========================================================== 23:13:57 (1783566837) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9ac2c3d05800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9ac2c3d05800.batch_stats=0 llite.lustre-ffff9ac2c3d05800.statahead_max=32 llite.lustre-ffff9ac2c3d05800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9ac2c3d05800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9ac2c3d05800.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 0.62 seconds: 160.18 ops/second llite.lustre-ffff9ac2c3d05800.statahead_max=0 101 real 0m0.601s user 0m0.004s sys 0m0.239s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff9ac2c3d05800.statahead_max=32 llite.lustre-ffff9ac2c3d05800.statahead_max=32 101 real 0m0.394s user 0m0.003s sys 0m0.256s 'ls -l' 100 files with statahead: 1 sec statahead total: 87 statahead wrong: 2 agl total: 82 list_total: 30 fname_total: 57 hit_total: 3748 miss_total: 1403 swrong=2 ewrong=2 total: 900 open/close in 2.20 seconds: 408.70 ops/second llite.lustre-ffff9ac2c3d05800.statahead_max=0 1001 real 0m6.351s user 0m0.031s sys 0m2.406s 'ls -l' 1000 files without statahead: 6 sec llite.lustre-ffff9ac2c3d05800.statahead_max=32 llite.lustre-ffff9ac2c3d05800.statahead_max=32 1001 real 0m3.882s user 0m0.014s sys 0m1.879s 'ls -l' 1000 files with statahead: 4 sec statahead total: 88 statahead wrong: 2 agl total: 83 list_total: 31 fname_total: 57 hit_total: 4747 miss_total: 1404 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 6 seconds llite.lustre-ffff9ac2c3d05800.statahead_stats= statahead total: 88 statahead wrong: 2 agl total: 83 list_total: 31 fname_total: 57 hit_total: 4747 miss_total: 1404 mdc.lustre-MDT0000-mdc-ffff9ac2c3d05800.batch_stats= snapshot_time: 1783566864.513068988 secs.nsecs start_time: 1783566838.776172119 secs.nsecs elapsed_time: 25.736896869 secs.nsecs subreqs per batch batches %% cum %% 1: 3 7 7 2: 1 2 10 4: 2 5 15 8: 2 5 21 16: 25 65 86 32: 5 13 100 mdc.lustre-MDT0001-mdc-ffff9ac2c3d05800.batch_stats= snapshot_time: 1783566864.513136156 secs.nsecs start_time: 1783566838.776266631 secs.nsecs elapsed_time: 25.736869525 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 2 5 7 4: 0 0 7 8: 2 5 12 16: 10 25 37 32: 25 62 100 - open/close 4021 (time 1783566875.38 total 10.00 last 402.05) - open/close 8124 (time 1783566885.38 total 20.00 last 410.30) total: 10000 open/close in 24.47 seconds: 408.59 ops/second llite.lustre-ffff9ac2c3d05800.statahead_batch_max=0 llite.lustre-ffff9ac2c3d05800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ac2c3d05800.stats=clear mdc.lustre-MDT0001-mdc-ffff9ac2c3d05800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m34.036s user 0m0.168s sys 0m31.398s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9ac2c3d05800.statahead_batch_max=32 llite.lustre-ffff9ac2c3d05800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ac2c3d05800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9ac2c3d05800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9ac2c3d05800.stats=clear mdc.lustre-MDT0001-mdc-ffff9ac2c3d05800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m36.740s user 0m0.155s sys 0m20.926s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9ac2c3d05800.batch_stats= snapshot_time: 1783566970.032976183 secs.nsecs start_time: 1783566927.185013511 secs.nsecs elapsed_time: 42.847962672 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff9ac2c3d05800.batch_stats= snapshot_time: 1783566970.033028736 secs.nsecs start_time: 1783566927.185076248 secs.nsecs elapsed_time: 42.847952488 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9ac2c3d05800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9ac2c3d05800.batch_stats=0 llite.lustre-ffff9ac2c3d05800.statahead_max=2048 llite.lustre-ffff9ac2c3d05800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9ac2c3d05800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9ac2c3d05800.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 0.75 seconds: 134.18 ops/second llite.lustre-ffff9ac2c3d05800.statahead_max=0 101 real 0m0.664s user 0m0.002s sys 0m0.260s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff9ac2c3d05800.statahead_max=2048 llite.lustre-ffff9ac2c3d05800.statahead_max=2048 101 real 0m0.390s user 0m0.004s sys 0m0.259s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 2 swrong=0 ewrong=0 total: 900 open/close in 1.87 seconds: 480.05 ops/second llite.lustre-ffff9ac2c3d05800.statahead_max=0 1001 real 0m6.014s user 0m0.021s sys 0m2.324s 'ls -l' 1000 files without statahead: 6 sec llite.lustre-ffff9ac2c3d05800.statahead_max=2048 llite.lustre-ffff9ac2c3d05800.statahead_max=2048 1001 real 0m3.132s user 0m0.016s sys 0m1.325s 'ls -l' 1000 files with statahead: 3 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 6 seconds llite.lustre-ffff9ac2c3d05800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 mdc.lustre-MDT0000-mdc-ffff9ac2c3d05800.batch_stats= snapshot_time: 1783567086.159205981 secs.nsecs start_time: 1783566970.798490872 secs.nsecs elapsed_time: 115.360715109 secs.nsecs subreqs per batch batches %% cum %% 1: 1 16 16 2: 0 0 16 4: 2 33 50 8: 1 16 66 16: 0 0 66 32: 0 0 66 64: 0 0 66 128: 0 0 66 256: 2 33 100 mdc.lustre-MDT0001-mdc-ffff9ac2c3d05800.batch_stats= snapshot_time: 1783567086.159247717 secs.nsecs start_time: 1783566970.798592065 secs.nsecs elapsed_time: 115.360655652 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 12 12 4: 0 0 12 8: 2 25 37 16: 1 12 50 32: 0 0 50 64: 1 12 62 128: 1 12 75 256: 2 25 100 - open/close 4372 (time 1783567096.96 total 10.00 last 437.20) - open/close 9481 (time 1783567106.96 total 20.00 last 510.86) total: 10000 open/close in 20.95 seconds: 477.25 ops/second llite.lustre-ffff9ac2c3d05800.statahead_batch_max=0 llite.lustre-ffff9ac2c3d05800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ac2c3d05800.stats=clear mdc.lustre-MDT0001-mdc-ffff9ac2c3d05800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m31.763s user 0m0.157s sys 0m26.864s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9ac2c3d05800.statahead_batch_max=256 llite.lustre-ffff9ac2c3d05800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ac2c3d05800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9ac2c3d05800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9ac2c3d05800.stats=clear mdc.lustre-MDT0001-mdc-ffff9ac2c3d05800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m27.219s user 0m0.163s sys 0m19.618s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9ac2c3d05800.batch_stats= snapshot_time: 1783567175.864260777 secs.nsecs start_time: 1783567142.708581994 secs.nsecs elapsed_time: 33.155678783 secs.nsecs subreqs per batch batches %% cum %% 1: 150 53 53 2: 40 14 67 4: 22 7 75 8: 3 1 76 16: 5 1 78 32: 20 7 85 64: 5 1 87 128: 0 0 87 256: 35 12 100 mdc.lustre-MDT0001-mdc-ffff9ac2c3d05800.batch_stats= snapshot_time: 1783567175.864299302 secs.nsecs start_time: 1783567142.708628781 secs.nsecs elapsed_time: 33.155670521 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9ac2c3d05800.statahead_batch_max=64 llite.lustre-ffff9ac2c3d05800.statahead_max=128