== sanity test 123ad: Verify batching statahead works correctly ========================================================== 23:44:25 (1786765465) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9f7505e76000.batch_stats=0 llite.lustre-ffff9f7505e76000.statahead_max=32 llite.lustre-ffff9f7505e76000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9f7505e76000.batch_stats=0 total: 100 open/close in 2.34 seconds: 42.78 ops/second llite.lustre-ffff9f7505e76000.statahead_max=0 101 real 0m2.496s user 0m0.006s sys 0m0.360s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9f7505e76000.statahead_max=32 llite.lustre-ffff9f7505e76000.statahead_max=32 101 real 0m0.992s user 0m0.003s sys 0m0.182s 'ls -l' 100 files with statahead: 1 sec statahead total: 176 statahead wrong: 2 agl total: 170 list_total: 30 fname_total: 126 hit_total: 3959 miss_total: 1454 swrong=2 ewrong=2 - open/close 734 (time 1786765491.31 total 10.01 last 73.31) total: 900 open/close in 12.16 seconds: 74.03 ops/second llite.lustre-ffff9f7505e76000.statahead_max=0 1001 real 0m20.729s user 0m0.035s sys 0m3.313s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff9f7505e76000.statahead_max=32 llite.lustre-ffff9f7505e76000.statahead_max=32 1001 real 0m13.569s user 0m0.028s sys 0m1.627s 'ls -l' 1000 files with statahead: 14 sec statahead total: 177 statahead wrong: 2 agl total: 171 list_total: 31 fname_total: 126 hit_total: 4958 miss_total: 1455 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff9f7505e76000.statahead_stats= statahead total: 177 statahead wrong: 2 agl total: 171 list_total: 31 fname_total: 126 hit_total: 4958 miss_total: 1455 mdc.lustre-MDT0000-mdc-ffff9f7505e76000.batch_stats= snapshot_time: 1786765561.018493189 secs.nsecs start_time: 1786765470.027700160 secs.nsecs elapsed_time: 90.990793029 secs.nsecs subreqs per batch batches %% cum %% 1: 2 4 4 2: 0 0 4 4: 1 2 7 8: 2 4 12 16: 2 4 17 32: 34 82 100 - open/close 687 (time 1786765573.32 total 10.01 last 68.65) - open/close 1368 (time 1786765583.32 total 20.02 last 68.04) - open/close 2095 (time 1786765593.33 total 30.02 last 72.64) - open/close 2881 (time 1786765603.34 total 40.03 last 78.55) - open/close 3618 (time 1786765613.34 total 50.03 last 73.68) - open/close 4333 (time 1786765623.34 total 60.03 last 71.50) - open/close 5058 (time 1786765633.36 total 70.05 last 72.36) - open/close 5773 (time 1786765643.37 total 80.06 last 71.48) - open/close 6479 (time 1786765653.37 total 90.06 last 70.57) - open/close 7207 (time 1786765663.38 total 100.07 last 72.74) - open/close 7872 (time 1786765673.38 total 110.07 last 66.46) - open/close 8529 (time 1786765683.39 total 120.08 last 65.63) - open/close 9157 (time 1786765693.42 total 130.11 last 62.62) - open/close 9622 (time 1786765703.44 total 140.13 last 46.42) total: 10000 open/close in 147.66 seconds: 67.72 ops/second llite.lustre-ffff9f7505e76000.statahead_batch_max=0 llite.lustre-ffff9f7505e76000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f7505e76000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m9.058s user 0m0.392s sys 0m18.695s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9f7505e76000.statahead_batch_max=32 llite.lustre-ffff9f7505e76000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f7505e76000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9f7505e76000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m50.895s user 0m0.380s sys 0m16.810s batched statahead RPC= hit count=9988 mdc.lustre-MDT0000-mdc-ffff9f7505e76000.batch_stats= snapshot_time: 1786765914.169896178 secs.nsecs start_time: 1786765789.260795512 secs.nsecs elapsed_time: 124.909100666 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9f7505e76000.batch_stats=0 llite.lustre-ffff9f7505e76000.statahead_max=2048 llite.lustre-ffff9f7505e76000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9f7505e76000.batch_stats=0 total: 100 open/close in 2.81 seconds: 35.64 ops/second llite.lustre-ffff9f7505e76000.statahead_max=0 101 real 0m3.313s user 0m0.003s sys 0m0.347s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9f7505e76000.statahead_max=2048 llite.lustre-ffff9f7505e76000.statahead_max=2048 101 real 0m0.852s user 0m0.009s sys 0m0.192s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10087 miss_total: 13 swrong=0 ewrong=0 - open/close 637 (time 1786766252.99 total 10.00 last 63.69) total: 900 open/close in 14.17 seconds: 63.50 ops/second llite.lustre-ffff9f7505e76000.statahead_max=0 1001 real 0m23.983s user 0m0.031s sys 0m3.361s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff9f7505e76000.statahead_max=2048 llite.lustre-ffff9f7505e76000.statahead_max=2048 1001 real 0m8.297s user 0m0.025s sys 0m1.464s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11086 miss_total: 14 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff9f7505e76000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11086 miss_total: 14 mdc.lustre-MDT0000-mdc-ffff9f7505e76000.batch_stats= snapshot_time: 1786766325.400625325 secs.nsecs start_time: 1786765916.992884971 secs.nsecs elapsed_time: 408.407740354 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 0 0 40 64: 1 10 50 128: 1 10 60 256: 4 40 100 - open/close 646 (time 1786766338.03 total 10.01 last 64.55) - open/close 1254 (time 1786766348.03 total 20.01 last 60.77) - open/close 1823 (time 1786766358.04 total 30.02 last 56.86) - open/close 2457 (time 1786766368.05 total 40.03 last 63.31) - open/close 3130 (time 1786766378.06 total 50.04 last 67.24) - open/close 3756 (time 1786766388.07 total 60.05 last 62.54) - open/close 4417 (time 1786766398.08 total 70.06 last 66.03) - open/close 5032 (time 1786766408.09 total 80.07 last 61.46) - open/close 5705 (time 1786766418.10 total 90.08 last 67.21) - open/close 6364 (time 1786766428.11 total 100.09 last 65.87) - open/close 7103 (time 1786766438.12 total 110.10 last 73.82) - open/close 7844 (time 1786766448.13 total 120.11 last 74.00) - open/close 8537 (time 1786766458.14 total 130.12 last 69.25) - open/close 9217 (time 1786766468.14 total 140.12 last 67.98) - open/close 9906 (time 1786766478.14 total 150.13 last 68.88) total: 10000 open/close in 151.57 seconds: 65.98 ops/second llite.lustre-ffff9f7505e76000.statahead_batch_max=0 llite.lustre-ffff9f7505e76000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f7505e76000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m3.542s user 0m0.373s sys 0m18.876s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9f7505e76000.statahead_batch_max=256 llite.lustre-ffff9f7505e76000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f7505e76000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9f7505e76000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m9.026s user 0m0.334s sys 0m17.858s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9f7505e76000.batch_stats= snapshot_time: 1786766637.956390423 secs.nsecs start_time: 1786766554.306608664 secs.nsecs elapsed_time: 83.649781759 secs.nsecs subreqs per batch batches %% cum %% 1: 280 58 58 2: 49 10 68 4: 54 11 80 8: 23 4 84 16: 23 4 89 32: 8 1 91 64: 6 1 92 128: 2 0 93 256: 33 6 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9f7505e76000.statahead_batch_max=64 llite.lustre-ffff9f7505e76000.statahead_max=128