== sanity test 123ad: Verify batching statahead works correctly ========================================================== 09:45:40 (1786715140) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff996a1a63d000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff996a1a63d000.batch_stats=0 llite.lustre-ffff996a1a63d000.statahead_max=32 llite.lustre-ffff996a1a63d000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff996a1a63d000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff996a1a63d000.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.26 seconds: 44.22 ops/second llite.lustre-ffff996a1a63d000.statahead_max=0 101 real 0m2.170s user 0m0.002s sys 0m0.309s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff996a1a63d000.statahead_max=32 llite.lustre-ffff996a1a63d000.statahead_max=32 101 real 0m1.110s user 0m0.005s sys 0m0.186s 'ls -l' 100 files with statahead: 1 sec statahead total: 123 statahead wrong: 2 agl total: 118 list_total: 32 fname_total: 90 hit_total: 4090 miss_total: 1302 swrong=2 ewrong=2 - open/close 855 (time 1786715166.57 total 10.00 last 85.48) total: 900 open/close in 10.70 seconds: 84.15 ops/second llite.lustre-ffff996a1a63d000.statahead_max=0 1001 real 0m20.408s user 0m0.037s sys 0m3.080s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff996a1a63d000.statahead_max=32 llite.lustre-ffff996a1a63d000.statahead_max=32 1001 real 0m7.929s user 0m0.027s sys 0m1.526s 'ls -l' 1000 files with statahead: 7 sec statahead total: 124 statahead wrong: 2 agl total: 119 list_total: 33 fname_total: 90 hit_total: 5088 miss_total: 1304 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 17 seconds llite.lustre-ffff996a1a63d000.statahead_stats= statahead total: 124 statahead wrong: 2 agl total: 119 list_total: 33 fname_total: 90 hit_total: 5088 miss_total: 1304 mdc.lustre-MDT0000-mdc-ffff996a1a63d000.batch_stats= snapshot_time: 1786715222.655423800 secs.nsecs start_time: 1786715144.886020761 secs.nsecs elapsed_time: 77.769403039 secs.nsecs subreqs per batch batches %% cum %% 1: 2 5 5 2: 1 2 7 4: 2 5 12 8: 1 2 15 16: 22 55 70 32: 12 30 100 mdc.lustre-MDT0001-mdc-ffff996a1a63d000.batch_stats= snapshot_time: 1786715222.655489763 secs.nsecs start_time: 1786715144.886109671 secs.nsecs elapsed_time: 77.769380092 secs.nsecs subreqs per batch batches %% cum %% 1: 2 5 5 2: 0 0 5 4: 3 7 12 8: 2 5 17 16: 17 42 60 32: 16 40 100 - open/close 867 (time 1786715234.91 total 10.00 last 86.67) - open/close 1745 (time 1786715244.91 total 20.00 last 87.80) - open/close 2657 (time 1786715254.92 total 30.01 last 91.12) - open/close 3499 (time 1786715264.93 total 40.02 last 84.13) - open/close 4383 (time 1786715274.93 total 50.03 last 88.33) - open/close 5294 (time 1786715284.94 total 60.04 last 91.02) - open/close 6159 (time 1786715294.95 total 70.04 last 86.44) - open/close 6975 (time 1786715304.96 total 80.05 last 81.55) - open/close 7815 (time 1786715314.96 total 90.06 last 83.94) - open/close 8582 (time 1786715324.98 total 100.08 last 76.54) - open/close 9315 (time 1786715334.99 total 110.08 last 73.27) total: 10000 open/close in 119.13 seconds: 83.94 ops/second llite.lustre-ffff996a1a63d000.statahead_batch_max=0 llite.lustre-ffff996a1a63d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff996a1a63d000.stats=clear mdc.lustre-MDT0001-mdc-ffff996a1a63d000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m2.139s user 0m0.279s sys 0m17.369s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff996a1a63d000.statahead_batch_max=32 llite.lustre-ffff996a1a63d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff996a1a63d000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff996a1a63d000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff996a1a63d000.stats=clear mdc.lustre-MDT0001-mdc-ffff996a1a63d000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m44.995s user 0m0.276s sys 0m15.800s batched statahead RPC= hit count=9986 mdc.lustre-MDT0000-mdc-ffff996a1a63d000.batch_stats= snapshot_time: 1786715535.866839247 secs.nsecs start_time: 1786715416.756478608 secs.nsecs elapsed_time: 119.110360639 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff996a1a63d000.batch_stats= snapshot_time: 1786715535.866930671 secs.nsecs start_time: 1786715416.756591609 secs.nsecs elapsed_time: 119.110339062 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff996a1a63d000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff996a1a63d000.batch_stats=0 llite.lustre-ffff996a1a63d000.statahead_max=2048 llite.lustre-ffff996a1a63d000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff996a1a63d000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff996a1a63d000.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 3.13 seconds: 31.96 ops/second llite.lustre-ffff996a1a63d000.statahead_max=0 101 real 0m2.873s user 0m0.004s sys 0m0.340s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff996a1a63d000.statahead_max=2048 llite.lustre-ffff996a1a63d000.statahead_max=2048 101 real 0m0.958s user 0m0.005s sys 0m0.192s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10085 miss_total: 15 swrong=0 ewrong=0 - open/close 878 (time 1786715840.63 total 10.01 last 87.73) total: 900 open/close in 10.32 seconds: 87.21 ops/second llite.lustre-ffff996a1a63d000.statahead_max=0 1001 real 0m19.680s user 0m0.025s sys 0m3.024s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff996a1a63d000.statahead_max=2048 llite.lustre-ffff996a1a63d000.statahead_max=2048 1001 real 0m7.003s user 0m0.020s sys 0m1.452s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff996a1a63d000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 mdc.lustre-MDT0000-mdc-ffff996a1a63d000.batch_stats= snapshot_time: 1786715899.550815951 secs.nsecs start_time: 1786715538.293935587 secs.nsecs elapsed_time: 361.256880364 secs.nsecs subreqs per batch batches %% cum %% 1: 1 16 16 2: 0 0 16 4: 3 50 66 8: 0 0 66 16: 0 0 66 32: 0 0 66 64: 0 0 66 128: 0 0 66 256: 2 33 100 mdc.lustre-MDT0001-mdc-ffff996a1a63d000.batch_stats= snapshot_time: 1786715899.551991737 secs.nsecs start_time: 1786715538.294084519 secs.nsecs elapsed_time: 361.257907218 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 33 33 16: 2 22 55 32: 0 0 55 64: 2 22 77 128: 0 0 77 256: 2 22 100 - open/close 849 (time 1786715911.91 total 10.02 last 84.72) - open/close 1610 (time 1786715921.93 total 20.03 last 76.00) - open/close 2378 (time 1786715931.94 total 30.05 last 76.67) - open/close 3319 (time 1786715941.95 total 40.05 last 94.09) - open/close 4148 (time 1786715951.95 total 50.06 last 82.87) - open/close 5003 (time 1786715961.98 total 60.09 last 85.23) - open/close 5934 (time 1786715971.98 total 70.09 last 93.09) - open/close 6813 (time 1786715982.00 total 80.10 last 87.77) - open/close 7667 (time 1786715992.01 total 90.12 last 85.25) - open/close 8525 (time 1786716002.02 total 100.13 last 85.74) - open/close 9443 (time 1786716012.03 total 110.14 last 91.74) total: 10000 open/close in 117.07 seconds: 85.42 ops/second llite.lustre-ffff996a1a63d000.statahead_batch_max=0 llite.lustre-ffff996a1a63d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff996a1a63d000.stats=clear mdc.lustre-MDT0001-mdc-ffff996a1a63d000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m57.881s user 0m0.272s sys 0m17.287s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff996a1a63d000.statahead_batch_max=256 llite.lustre-ffff996a1a63d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff996a1a63d000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff996a1a63d000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff996a1a63d000.stats=clear mdc.lustre-MDT0001-mdc-ffff996a1a63d000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m51.775s user 0m0.296s sys 0m17.797s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff996a1a63d000.batch_stats= snapshot_time: 1786716154.234014379 secs.nsecs start_time: 1786716088.197951231 secs.nsecs elapsed_time: 66.036063148 secs.nsecs subreqs per batch batches %% cum %% 1: 252 53 53 2: 61 13 66 4: 56 11 78 8: 31 6 85 16: 13 2 88 32: 15 3 91 64: 7 1 92 128: 0 0 92 256: 33 7 100 mdc.lustre-MDT0001-mdc-ffff996a1a63d000.batch_stats= snapshot_time: 1786716154.234080900 secs.nsecs start_time: 1786716088.198038865 secs.nsecs elapsed_time: 66.036042035 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff996a1a63d000.statahead_batch_max=64 llite.lustre-ffff996a1a63d000.statahead_max=128