== sanity test 123ad: Verify batching statahead works correctly ========================================================== 07:42:49 (1786966969) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff948e87062000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff948e87062000.batch_stats=0 llite.lustre-ffff948e87062000.statahead_max=32 llite.lustre-ffff948e87062000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff948e87062000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff948e87062000.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.58 seconds: 38.73 ops/second llite.lustre-ffff948e87062000.statahead_max=0 101 real 0m2.067s user 0m0.005s sys 0m0.344s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff948e87062000.statahead_max=32 llite.lustre-ffff948e87062000.statahead_max=32 101 real 0m0.965s user 0m0.004s sys 0m0.169s 'ls -l' 100 files with statahead: 1 sec statahead total: 150 statahead wrong: 2 agl total: 144 list_total: 31 fname_total: 97 hit_total: 3638 miss_total: 1360 swrong=2 ewrong=2 - open/close 879 (time 1786966996.97 total 10.00 last 87.89) total: 900 open/close in 10.19 seconds: 88.30 ops/second llite.lustre-ffff948e87062000.statahead_max=0 1001 real 0m22.527s user 0m0.038s sys 0m3.022s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff948e87062000.statahead_max=32 llite.lustre-ffff948e87062000.statahead_max=32 1001 real 0m8.925s user 0m0.025s sys 0m1.439s 'ls -l' 1000 files with statahead: 9 sec statahead total: 151 statahead wrong: 2 agl total: 145 list_total: 32 fname_total: 97 hit_total: 4634 miss_total: 1364 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff948e87062000.statahead_stats= statahead total: 151 statahead wrong: 2 agl total: 145 list_total: 32 fname_total: 97 hit_total: 4634 miss_total: 1364 mdc.lustre-MDT0000-mdc-ffff948e87062000.batch_stats= snapshot_time: 1786967061.188531348 secs.nsecs start_time: 1786966974.106648093 secs.nsecs elapsed_time: 87.081883255 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 2 2 4: 0 0 2 8: 5 12 15 16: 20 51 66 32: 13 33 100 mdc.lustre-MDT0001-mdc-ffff948e87062000.batch_stats= snapshot_time: 1786967061.188738615 secs.nsecs start_time: 1786966974.106909992 secs.nsecs elapsed_time: 87.081828623 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 2 5 5 4: 2 5 10 8: 2 5 15 16: 18 46 61 32: 15 38 100 - open/close 944 (time 1786967073.86 total 10.01 last 94.30) - open/close 1695 (time 1786967083.86 total 20.01 last 75.08) - open/close 2580 (time 1786967093.87 total 30.02 last 88.44) - open/close 3394 (time 1786967103.88 total 40.03 last 81.29) - open/close 4216 (time 1786967113.89 total 50.05 last 82.09) - open/close 4954 (time 1786967123.90 total 60.05 last 73.78) - open/close 5685 (time 1786967133.90 total 70.05 last 73.07) - open/close 6413 (time 1786967143.90 total 80.06 last 72.78) - open/close 7325 (time 1786967153.91 total 90.06 last 91.14) - open/close 8244 (time 1786967163.92 total 100.07 last 91.83) - open/close 9139 (time 1786967173.93 total 110.08 last 89.40) total: 10000 open/close in 119.69 seconds: 83.55 ops/second llite.lustre-ffff948e87062000.statahead_batch_max=0 llite.lustre-ffff948e87062000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff948e87062000.stats=clear mdc.lustre-MDT0001-mdc-ffff948e87062000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m4.653s user 0m0.308s sys 0m16.260s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff948e87062000.statahead_batch_max=32 llite.lustre-ffff948e87062000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff948e87062000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff948e87062000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff948e87062000.stats=clear mdc.lustre-MDT0001-mdc-ffff948e87062000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m45.720s user 0m0.292s sys 0m15.438s batched statahead RPC= hit count=9977 mdc.lustre-MDT0000-mdc-ffff948e87062000.batch_stats= snapshot_time: 1786967380.954132722 secs.nsecs start_time: 1786967260.935881604 secs.nsecs elapsed_time: 120.018251118 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff948e87062000.batch_stats= snapshot_time: 1786967380.954207742 secs.nsecs start_time: 1786967260.936702869 secs.nsecs elapsed_time: 120.017504873 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff948e87062000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff948e87062000.batch_stats=0 llite.lustre-ffff948e87062000.statahead_max=2048 llite.lustre-ffff948e87062000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff948e87062000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff948e87062000.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 2.40 seconds: 41.62 ops/second llite.lustre-ffff948e87062000.statahead_max=0 101 real 0m2.673s user 0m0.005s sys 0m0.352s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff948e87062000.statahead_max=2048 llite.lustre-ffff948e87062000.statahead_max=2048 101 real 0m0.885s user 0m0.005s sys 0m0.200s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10076 miss_total: 24 swrong=0 ewrong=0 - open/close 754 (time 1786967677.16 total 10.00 last 75.40) total: 900 open/close in 11.99 seconds: 75.05 ops/second llite.lustre-ffff948e87062000.statahead_max=0 1001 real 0m18.297s user 0m0.030s sys 0m3.046s 'ls -l' 1000 files without statahead: 18 sec llite.lustre-ffff948e87062000.statahead_max=2048 llite.lustre-ffff948e87062000.statahead_max=2048 1001 real 0m8.060s user 0m0.031s sys 0m1.687s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11075 miss_total: 25 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff948e87062000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11075 miss_total: 25 mdc.lustre-MDT0000-mdc-ffff948e87062000.batch_stats= snapshot_time: 1786967737.524552759 secs.nsecs start_time: 1786967383.238562415 secs.nsecs elapsed_time: 354.285990344 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 2 25 25 4: 2 25 50 8: 1 12 62 16: 0 0 62 32: 1 12 75 64: 0 0 75 128: 0 0 75 256: 2 25 100 mdc.lustre-MDT0001-mdc-ffff948e87062000.batch_stats= snapshot_time: 1786967737.524620612 secs.nsecs start_time: 1786967383.238697987 secs.nsecs elapsed_time: 354.285922625 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 27 27 16: 5 45 72 32: 0 0 72 64: 1 9 81 128: 0 0 81 256: 2 18 100 - open/close 885 (time 1786967750.45 total 10.01 last 88.45) - open/close 1780 (time 1786967760.47 total 20.02 last 89.35) - open/close 2665 (time 1786967770.47 total 30.03 last 88.45) - open/close 3599 (time 1786967780.48 total 40.03 last 93.36) - open/close 4300 (time 1786967790.51 total 50.07 last 69.86) - open/close 4930 (time 1786967800.52 total 60.07 last 62.96) - open/close 5496 (time 1786967810.52 total 70.07 last 56.60) - open/close 6509 (time 1786967820.52 total 80.08 last 101.27) - open/close 7457 (time 1786967830.53 total 90.09 last 94.68) - open/close 8347 (time 1786967840.54 total 100.10 last 88.92) - open/close 9155 (time 1786967850.55 total 110.10 last 80.76) - open/close 9864 (time 1786967860.55 total 120.11 last 70.89) total: 10000 open/close in 121.85 seconds: 82.07 ops/second llite.lustre-ffff948e87062000.statahead_batch_max=0 llite.lustre-ffff948e87062000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff948e87062000.stats=clear mdc.lustre-MDT0001-mdc-ffff948e87062000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.859s user 0m0.320s sys 0m16.310s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff948e87062000.statahead_batch_max=256 llite.lustre-ffff948e87062000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff948e87062000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff948e87062000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff948e87062000.stats=clear mdc.lustre-MDT0001-mdc-ffff948e87062000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m2.196s user 0m0.298s sys 0m16.111s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff948e87062000.batch_stats= snapshot_time: 1786968008.279348598 secs.nsecs start_time: 1786967931.588898355 secs.nsecs elapsed_time: 76.690450243 secs.nsecs subreqs per batch batches %% cum %% 1: 136 42 42 2: 49 15 57 4: 36 11 69 8: 20 6 75 16: 19 5 81 32: 18 5 86 64: 7 2 89 128: 2 0 89 256: 33 10 100 mdc.lustre-MDT0001-mdc-ffff948e87062000.batch_stats= snapshot_time: 1786968008.279420506 secs.nsecs start_time: 1786967931.590629647 secs.nsecs elapsed_time: 76.688790859 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff948e87062000.statahead_batch_max=64 llite.lustre-ffff948e87062000.statahead_max=128