== sanity test 123ad: Verify batching statahead works correctly ========================================================== 09:54:39 (1786715679) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff896c85e60800.batch_stats=0 llite.lustre-ffff896c85e60800.statahead_max=32 llite.lustre-ffff896c85e60800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff896c85e60800.batch_stats=0 total: 100 open/close in 3.16 seconds: 31.65 ops/second llite.lustre-ffff896c85e60800.statahead_max=0 101 real 0m2.722s user 0m0.006s sys 0m0.357s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff896c85e60800.statahead_max=32 llite.lustre-ffff896c85e60800.statahead_max=32 101 real 0m1.418s user 0m0.005s sys 0m0.187s 'ls -l' 100 files with statahead: 1 sec statahead total: 140 statahead wrong: 2 agl total: 134 list_total: 31 fname_total: 94 hit_total: 3734 miss_total: 1725 swrong=2 ewrong=2 - open/close 544 (time 1786715711.37 total 10.04 last 54.17) total: 900 open/close in 17.00 seconds: 52.93 ops/second llite.lustre-ffff896c85e60800.statahead_max=0 1001 real 0m27.850s user 0m0.026s sys 0m3.185s 'ls -l' 1000 files without statahead: 28 sec llite.lustre-ffff896c85e60800.statahead_max=32 llite.lustre-ffff896c85e60800.statahead_max=32 1001 real 0m9.834s user 0m0.030s sys 0m1.530s 'ls -l' 1000 files with statahead: 10 sec statahead total: 141 statahead wrong: 2 agl total: 135 list_total: 32 fname_total: 94 hit_total: 4732 miss_total: 1727 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff896c85e60800.statahead_stats= statahead total: 141 statahead wrong: 2 agl total: 135 list_total: 32 fname_total: 94 hit_total: 4732 miss_total: 1727 mdc.lustre-MDT0000-mdc-ffff896c85e60800.batch_stats= snapshot_time: 1786715793.446644066 secs.nsecs start_time: 1786715685.736335376 secs.nsecs elapsed_time: 107.710308690 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 554 (time 1786715806.27 total 10.01 last 55.37) - open/close 1064 (time 1786715816.28 total 20.02 last 50.94) - open/close 1596 (time 1786715826.30 total 30.03 last 53.12) - open/close 2164 (time 1786715836.30 total 40.03 last 56.79) - open/close 2804 (time 1786715846.31 total 50.04 last 63.93) - open/close 3506 (time 1786715856.32 total 60.05 last 70.14) - open/close 4126 (time 1786715866.32 total 70.05 last 61.99) - open/close 4710 (time 1786715876.33 total 80.06 last 58.37) - open/close 5297 (time 1786715886.33 total 90.06 last 58.69) - open/close 5944 (time 1786715896.33 total 100.07 last 64.65) - open/close 6559 (time 1786715906.35 total 110.08 last 61.43) - open/close 7137 (time 1786715916.36 total 120.10 last 57.69) - open/close 7641 (time 1786715926.38 total 130.11 last 50.35) - open/close 8238 (time 1786715936.38 total 140.11 last 59.70) - open/close 8836 (time 1786715946.38 total 150.12 last 59.76) - open/close 9423 (time 1786715956.38 total 160.12 last 58.70) total: 10000 open/close in 169.78 seconds: 58.90 ops/second llite.lustre-ffff896c85e60800.statahead_batch_max=0 llite.lustre-ffff896c85e60800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff896c85e60800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m8.473s user 0m0.303s sys 0m16.665s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff896c85e60800.statahead_batch_max=32 llite.lustre-ffff896c85e60800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff896c85e60800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff896c85e60800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m49.095s user 0m0.330s sys 0m16.357s batched statahead RPC= hit count=9983 mdc.lustre-MDT0000-mdc-ffff896c85e60800.batch_stats= snapshot_time: 1786716169.300180811 secs.nsecs start_time: 1786716046.340127662 secs.nsecs elapsed_time: 122.960053149 secs.nsecs subreqs per batch batches %% cum %% 1: 1 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff896c85e60800.batch_stats=0 llite.lustre-ffff896c85e60800.statahead_max=2048 llite.lustre-ffff896c85e60800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff896c85e60800.batch_stats=0 total: 100 open/close in 3.30 seconds: 30.33 ops/second llite.lustre-ffff896c85e60800.statahead_max=0 101 real 0m3.159s user 0m0.005s sys 0m0.363s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff896c85e60800.statahead_max=2048 llite.lustre-ffff896c85e60800.statahead_max=2048 101 real 0m1.097s user 0m0.001s sys 0m0.209s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10082 miss_total: 18 swrong=0 ewrong=0 - open/close 620 (time 1786716549.44 total 10.05 last 61.71) total: 900 open/close in 15.13 seconds: 59.49 ops/second llite.lustre-ffff896c85e60800.statahead_max=0 1001 real 0m23.094s user 0m0.033s sys 0m3.169s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff896c85e60800.statahead_max=2048 llite.lustre-ffff896c85e60800.statahead_max=2048 1001 real 0m6.078s user 0m0.014s sys 0m1.638s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff896c85e60800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 mdc.lustre-MDT0000-mdc-ffff896c85e60800.batch_stats= snapshot_time: 1786716619.082215613 secs.nsecs start_time: 1786716171.658956603 secs.nsecs elapsed_time: 447.423259010 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 22 22 16: 1 11 33 32: 1 11 44 64: 0 0 44 128: 1 11 55 256: 4 44 100 - open/close 653 (time 1786716631.80 total 10.03 last 65.13) - open/close 1302 (time 1786716641.80 total 20.03 last 64.89) - open/close 1914 (time 1786716651.81 total 30.04 last 61.14) - open/close 2564 (time 1786716661.82 total 40.05 last 64.94) - open/close 3211 (time 1786716671.83 total 50.06 last 64.64) - open/close 3860 (time 1786716681.83 total 60.06 last 64.89) - open/close 4513 (time 1786716691.84 total 70.07 last 65.22) - open/close 5179 (time 1786716701.85 total 80.08 last 66.53) - open/close 5846 (time 1786716711.86 total 90.09 last 66.63) - open/close 6501 (time 1786716721.88 total 100.11 last 65.42) - open/close 7136 (time 1786716731.88 total 110.11 last 63.50) - open/close 7736 (time 1786716741.88 total 120.11 last 59.98) - open/close 8399 (time 1786716751.89 total 130.12 last 66.24) - open/close 9042 (time 1786716761.90 total 140.13 last 64.25) - open/close 9668 (time 1786716771.90 total 150.13 last 62.57) total: 10000 open/close in 155.33 seconds: 64.38 ops/second llite.lustre-ffff896c85e60800.statahead_batch_max=0 llite.lustre-ffff896c85e60800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff896c85e60800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m12.120s user 0m0.315s sys 0m17.369s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff896c85e60800.statahead_batch_max=256 llite.lustre-ffff896c85e60800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff896c85e60800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff896c85e60800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m50.848s user 0m0.287s sys 0m17.814s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff896c85e60800.batch_stats= snapshot_time: 1786716923.147631874 secs.nsecs start_time: 1786716858.593384801 secs.nsecs elapsed_time: 64.554247073 secs.nsecs subreqs per batch batches %% cum %% 1: 299 63 63 2: 48 10 73 4: 38 8 81 8: 19 4 85 16: 13 2 88 32: 14 2 91 64: 9 1 93 128: 0 0 93 256: 33 6 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff896c85e60800.statahead_batch_max=64 llite.lustre-ffff896c85e60800.statahead_max=128