== sanity test 123ad: Verify batching statahead works correctly ========================================================== 01:15:08 (1786943708) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff93958a9b8800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff93958a9b8800.batch_stats=0 llite.lustre-ffff93958a9b8800.statahead_max=32 llite.lustre-ffff93958a9b8800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff93958a9b8800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff93958a9b8800.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.06 seconds: 48.56 ops/second llite.lustre-ffff93958a9b8800.statahead_max=0 101 real 0m2.081s user 0m0.003s sys 0m0.384s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff93958a9b8800.statahead_max=32 llite.lustre-ffff93958a9b8800.statahead_max=32 101 real 0m0.780s user 0m0.007s sys 0m0.203s 'ls -l' 100 files with statahead: 1 sec statahead total: 117 statahead wrong: 2 agl total: 112 list_total: 31 fname_total: 86 hit_total: 4214 miss_total: 1292 swrong=2 ewrong=2 total: 900 open/close in 9.75 seconds: 92.27 ops/second llite.lustre-ffff93958a9b8800.statahead_max=0 1001 real 0m18.251s user 0m0.036s sys 0m3.433s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff93958a9b8800.statahead_max=32 llite.lustre-ffff93958a9b8800.statahead_max=32 1001 real 0m8.001s user 0m0.037s sys 0m1.664s 'ls -l' 1000 files with statahead: 8 sec statahead total: 118 statahead wrong: 2 agl total: 113 list_total: 32 fname_total: 86 hit_total: 5213 miss_total: 1293 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 17 seconds llite.lustre-ffff93958a9b8800.statahead_stats= statahead total: 118 statahead wrong: 2 agl total: 113 list_total: 32 fname_total: 86 hit_total: 5213 miss_total: 1293 mdc.lustre-MDT0000-mdc-ffff93958a9b8800.batch_stats= snapshot_time: 1786943785.179755470 secs.nsecs start_time: 1786943712.346098353 secs.nsecs elapsed_time: 72.833657117 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 2 5 7 4: 2 5 12 8: 2 5 17 16: 19 47 65 32: 14 35 100 mdc.lustre-MDT0001-mdc-ffff93958a9b8800.batch_stats= snapshot_time: 1786943785.179974661 secs.nsecs start_time: 1786943712.346221834 secs.nsecs elapsed_time: 72.833752827 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 2 2 4: 2 5 7 8: 3 7 15 16: 19 48 64 32: 14 35 100 - open/close 879 (time 1786943797.14 total 10.00 last 87.88) - open/close 1682 (time 1786943807.14 total 20.00 last 80.30) - open/close 2539 (time 1786943817.14 total 30.00 last 85.68) - open/close 3409 (time 1786943827.15 total 40.01 last 86.96) - open/close 4285 (time 1786943837.15 total 50.01 last 87.59) - open/close 5242 (time 1786943847.15 total 60.01 last 95.67) - open/close 5955 (time 1786943857.16 total 70.02 last 71.24) - open/close 6727 (time 1786943867.17 total 80.03 last 77.15) - open/close 7481 (time 1786943877.17 total 90.03 last 75.39) - open/close 8323 (time 1786943887.18 total 100.04 last 84.13) - open/close 9371 (time 1786943897.19 total 110.05 last 104.73) total: 10000 open/close in 115.06 seconds: 86.91 ops/second llite.lustre-ffff93958a9b8800.statahead_batch_max=0 llite.lustre-ffff93958a9b8800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93958a9b8800.stats=clear mdc.lustre-MDT0001-mdc-ffff93958a9b8800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m52.342s user 0m0.325s sys 0m20.136s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff93958a9b8800.statahead_batch_max=32 llite.lustre-ffff93958a9b8800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93958a9b8800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff93958a9b8800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff93958a9b8800.stats=clear mdc.lustre-MDT0001-mdc-ffff93958a9b8800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m26.719s user 0m0.309s sys 0m17.182s batched statahead RPC= hit count=9988 mdc.lustre-MDT0000-mdc-ffff93958a9b8800.batch_stats= snapshot_time: 1786944059.154116479 secs.nsecs start_time: 1786943962.584446798 secs.nsecs elapsed_time: 96.569669681 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 0 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff93958a9b8800.batch_stats= snapshot_time: 1786944059.154192968 secs.nsecs start_time: 1786943962.584584158 secs.nsecs elapsed_time: 96.569608810 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff93958a9b8800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff93958a9b8800.batch_stats=0 llite.lustre-ffff93958a9b8800.statahead_max=2048 llite.lustre-ffff93958a9b8800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff93958a9b8800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff93958a9b8800.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.00 seconds: 49.89 ops/second llite.lustre-ffff93958a9b8800.statahead_max=0 101 real 0m1.924s user 0m0.005s sys 0m0.368s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff93958a9b8800.statahead_max=2048 llite.lustre-ffff93958a9b8800.statahead_max=2048 101 real 0m0.597s user 0m0.006s sys 0m0.216s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10087 miss_total: 13 swrong=0 ewrong=0 total: 900 open/close in 8.76 seconds: 102.74 ops/second llite.lustre-ffff93958a9b8800.statahead_max=0 1001 real 0m17.528s user 0m0.030s sys 0m3.248s 'ls -l' 1000 files without statahead: 17 sec llite.lustre-ffff93958a9b8800.statahead_max=2048 llite.lustre-ffff93958a9b8800.statahead_max=2048 1001 real 0m4.744s user 0m0.022s sys 0m1.697s 'ls -l' 1000 files with statahead: 5 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11086 miss_total: 14 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 14 seconds llite.lustre-ffff93958a9b8800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11086 miss_total: 14 mdc.lustre-MDT0000-mdc-ffff93958a9b8800.batch_stats= snapshot_time: 1786944362.271311490 secs.nsecs start_time: 1786944061.514996881 secs.nsecs elapsed_time: 300.756314609 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 2 22 22 8: 4 44 66 16: 0 0 66 32: 1 11 77 64: 0 0 77 128: 0 0 77 256: 2 22 100 mdc.lustre-MDT0001-mdc-ffff93958a9b8800.batch_stats= snapshot_time: 1786944362.273105539 secs.nsecs start_time: 1786944061.518294178 secs.nsecs elapsed_time: 300.754811361 secs.nsecs subreqs per batch batches %% cum %% 1: 1 11 11 2: 0 0 11 4: 3 33 44 8: 0 0 44 16: 2 22 66 32: 0 0 66 64: 1 11 77 128: 0 0 77 256: 2 22 100 - open/close 1029 (time 1786944374.09 total 10.00 last 102.89) - open/close 1980 (time 1786944384.10 total 20.00 last 95.07) - open/close 2930 (time 1786944394.10 total 30.01 last 94.94) - open/close 3885 (time 1786944404.11 total 40.02 last 95.41) - open/close 4873 (time 1786944414.12 total 50.03 last 98.73) - open/close 5783 (time 1786944424.12 total 60.03 last 90.97) - open/close 6721 (time 1786944434.13 total 70.04 last 93.70) - open/close 7511 (time 1786944444.15 total 80.05 last 78.89) - open/close 7949 (time 1786944456.46 total 92.37 last 35.56) - open/close 8389 (time 1786944466.49 total 102.40 last 43.87) - open/close 8935 (time 1786944476.51 total 112.41 last 54.53) - open/close 9590 (time 1786944486.53 total 122.43 last 65.38) total: 10000 open/close in 128.38 seconds: 77.90 ops/second llite.lustre-ffff93958a9b8800.statahead_batch_max=0 llite.lustre-ffff93958a9b8800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93958a9b8800.stats=clear mdc.lustre-MDT0001-mdc-ffff93958a9b8800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m6.937s user 0m0.343s sys 0m17.948s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff93958a9b8800.statahead_batch_max=256 llite.lustre-ffff93958a9b8800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93958a9b8800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff93958a9b8800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff93958a9b8800.stats=clear mdc.lustre-MDT0001-mdc-ffff93958a9b8800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m7.698s user 0m0.326s sys 0m18.054s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff93958a9b8800.batch_stats= snapshot_time: 1786944655.929375267 secs.nsecs start_time: 1786944572.907317416 secs.nsecs elapsed_time: 83.022057851 secs.nsecs subreqs per batch batches %% cum %% 1: 251 54 54 2: 75 16 70 4: 43 9 80 8: 19 4 84 16: 17 3 87 32: 18 3 91 64: 3 0 92 128: 2 0 92 256: 33 7 100 mdc.lustre-MDT0001-mdc-ffff93958a9b8800.batch_stats= snapshot_time: 1786944655.929441885 secs.nsecs start_time: 1786944572.907443480 secs.nsecs elapsed_time: 83.021998405 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff93958a9b8800.statahead_batch_max=64 llite.lustre-ffff93958a9b8800.statahead_max=128