== sanity test 123ad: Verify batching statahead works correctly ========================================================== 22:53:26 (1775876006) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffffa0f843fa3000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffffa0f843fa3000.batch_stats=0 llite.lustre-ffffa0f843fa3000.statahead_max=32 llite.lustre-ffffa0f843fa3000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffffa0f843fa3000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffffa0f843fa3000.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.25 seconds: 79.94 ops/second llite.lustre-ffffa0f843fa3000.statahead_max=0 101 real 0m1.452s user 0m0.007s sys 0m0.326s ls -l 100 files without statahead: 1 sec llite.lustre-ffffa0f843fa3000.statahead_max=32 32 101 real 0m0.509s user 0m0.003s sys 0m0.230s ls -l 100 files with statahead: 1 sec statahead total: 129 statahead wrong: 2 agl total: 122 list_total: 32 fname_total: 84 hit_total: 6643 miss_total: 1414 total: 900 open/close in 4.90 seconds: 183.70 ops/second llite.lustre-ffffa0f843fa3000.statahead_max=0 1001 real 0m9.616s user 0m0.035s sys 0m2.934s ls -l 1000 files without statahead: 10 sec llite.lustre-ffffa0f843fa3000.statahead_max=32 32 1001 real 0m4.358s user 0m0.021s sys 0m1.975s ls -l 1000 files with statahead: 5 sec statahead total: 130 statahead wrong: 2 agl total: 123 list_total: 33 fname_total: 84 hit_total: 7641 miss_total: 1416 - open/close 2238 (time 1775876047.76 total 10.00 last 223.72) total: 4000 open/close in 18.81 seconds: 212.68 ops/second llite.lustre-ffffa0f843fa3000.statahead_max=0 5001 real 0m50.631s user 0m0.179s sys 0m14.096s ls -l 5000 files without statahead: 51 sec llite.lustre-ffffa0f843fa3000.statahead_max=32 32 5001 real 0m21.367s user 0m0.100s sys 0m8.688s ls -l 5000 files with statahead: 22 sec statahead total: 131 statahead wrong: 2 agl total: 124 list_total: 34 fname_total: 84 hit_total: 12639 miss_total: 1418 ls -l done rm -r /mnt/lustre/d123ad.sanity: 44 seconds llite.lustre-ffffa0f843fa3000.statahead_stats= statahead total: 131 statahead wrong: 2 agl total: 124 list_total: 34 fname_total: 84 hit_total: 12639 miss_total: 1418 mdc.lustre-MDT0000-mdc-ffffa0f843fa3000.batch_stats= snapshot_time: 1775876181.690844569 secs.nsecs start_time: 1775876008.731841322 secs.nsecs elapsed_time: 172.959003247 secs.nsecs subreqs per batch batches % cum % 1: 1 0 0 2: 1 0 1 4: 3 1 2 8: 4 2 4 16: 114 57 61 32: 76 38 100 mdc.lustre-MDT0001-mdc-ffffa0f843fa3000.batch_stats= snapshot_time: 1775876181.690899127 secs.nsecs start_time: 1775876008.731927764 secs.nsecs elapsed_time: 172.958971363 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 1 0 0 4: 6 3 3 8: 4 2 5 16: 108 54 60 32: 79 39 100 - open/close 1853 (time 1775876192.80 total 10.01 last 185.13) - open/close 3730 (time 1775876202.80 total 20.01 last 187.65) - open/close 5795 (time 1775876212.81 total 30.01 last 206.48) - open/close 7755 (time 1775876222.81 total 40.01 last 195.96) - open/close 9602 (time 1775876232.81 total 50.02 last 184.64) total: 10000 open/close in 52.35 seconds: 191.01 ops/second llite.lustre-ffffa0f843fa3000.statahead_batch_max=0 llite.lustre-ffffa0f843fa3000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffffa0f843fa3000.stats=clear mdc.lustre-MDT0001-mdc-ffffa0f843fa3000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m34.264s user 0m0.250s sys 0m28.493s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffffa0f843fa3000.statahead_batch_max=32 llite.lustre-ffffa0f843fa3000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffffa0f843fa3000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffffa0f843fa3000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffffa0f843fa3000.stats=clear mdc.lustre-MDT0001-mdc-ffffa0f843fa3000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m44.570s user 0m0.260s sys 0m21.530s batched statahead RPC= hit count=9996 mdc.lustre-MDT0000-mdc-ffffa0f843fa3000.batch_stats= snapshot_time: 1775876324.194792284 secs.nsecs start_time: 1775876273.358558048 secs.nsecs elapsed_time: 50.836234236 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffffa0f843fa3000.batch_stats= snapshot_time: 1775876324.194838664 secs.nsecs start_time: 1775876273.358643148 secs.nsecs elapsed_time: 50.836195516 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffffa0f843fa3000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffffa0f843fa3000.batch_stats=0 llite.lustre-ffffa0f843fa3000.statahead_max=2048 llite.lustre-ffffa0f843fa3000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffffa0f843fa3000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffffa0f843fa3000.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.11 seconds: 90.00 ops/second llite.lustre-ffffa0f843fa3000.statahead_max=0 101 real 0m0.974s user 0m0.007s sys 0m0.306s ls -l 100 files without statahead: 1 sec llite.lustre-ffffa0f843fa3000.statahead_max=2048 2048 101 real 0m0.453s user 0m0.004s sys 0m0.264s ls -l 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10095 miss_total: 5 total: 900 open/close in 4.21 seconds: 213.90 ops/second llite.lustre-ffffa0f843fa3000.statahead_max=0 1001 real 0m7.979s user 0m0.033s sys 0m2.760s ls -l 1000 files without statahead: 8 sec llite.lustre-ffffa0f843fa3000.statahead_max=2048 2048 1001 real 0m3.632s user 0m0.021s sys 0m1.870s ls -l 1000 files with statahead: 4 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11094 miss_total: 6 - open/close 2508 (time 1775876487.99 total 10.00 last 250.75) - open/close 5455 (time 1775876498.00 total 20.00 last 294.62) - open/close 7750 (time 1775876508.00 total 30.01 last 229.44) total: 9000 open/close in 36.24 seconds: 248.38 ops/second llite.lustre-ffffa0f843fa3000.statahead_max=0 10001 real 1m29.606s user 0m0.343s sys 0m27.757s ls -l 10000 files without statahead: 90 sec llite.lustre-ffffa0f843fa3000.statahead_max=2048 2048 10001 real 0m31.503s user 0m0.206s sys 0m22.068s ls -l 10000 files with statahead: 31 sec statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 21093 miss_total: 7 ls -l done rm -r /mnt/lustre/d123ad.sanity: 72 seconds llite.lustre-ffffa0f843fa3000.statahead_stats= statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 21093 miss_total: 7 mdc.lustre-MDT0000-mdc-ffffa0f843fa3000.batch_stats= snapshot_time: 1775876722.617676222 secs.nsecs start_time: 1775876325.231771525 secs.nsecs elapsed_time: 397.385904697 secs.nsecs subreqs per batch batches % cum % 1: 233 75 75 2: 30 9 85 4: 10 3 88 8: 2 0 89 16: 1 0 89 32: 0 0 89 64: 1 0 89 128: 6 1 91 256: 25 8 100 mdc.lustre-MDT0001-mdc-ffffa0f843fa3000.batch_stats= snapshot_time: 1775876722.617742014 secs.nsecs start_time: 1775876325.231917825 secs.nsecs elapsed_time: 397.385824189 secs.nsecs subreqs per batch batches % cum % 1: 267 76 76 2: 26 7 83 4: 16 4 88 8: 5 1 89 16: 1 0 90 32: 0 0 90 64: 3 0 90 128: 1 0 91 256: 31 8 100 - open/close 3098 (time 1775876733.54 total 10.00 last 309.75) - open/close 6430 (time 1775876743.54 total 20.00 last 333.09) - open/close 9449 (time 1775876753.54 total 30.01 last 301.83) total: 10000 open/close in 31.41 seconds: 318.42 ops/second llite.lustre-ffffa0f843fa3000.statahead_batch_max=0 llite.lustre-ffffa0f843fa3000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffffa0f843fa3000.stats=clear mdc.lustre-MDT0001-mdc-ffffa0f843fa3000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m35.600s user 0m0.243s sys 0m30.026s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffffa0f843fa3000.statahead_batch_max=256 llite.lustre-ffffa0f843fa3000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffffa0f843fa3000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffffa0f843fa3000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffffa0f843fa3000.stats=clear mdc.lustre-MDT0001-mdc-ffffa0f843fa3000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m27.959s user 0m0.183s sys 0m20.654s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffffa0f843fa3000.batch_stats= snapshot_time: 1775876828.114382514 secs.nsecs start_time: 1775876793.856451843 secs.nsecs elapsed_time: 34.257930671 secs.nsecs subreqs per batch batches % cum % 1: 333 65 65 2: 55 10 76 4: 46 9 85 8: 10 1 87 16: 5 0 88 32: 22 4 92 64: 2 0 93 128: 0 0 93 256: 34 6 100 mdc.lustre-MDT0001-mdc-ffffa0f843fa3000.batch_stats= snapshot_time: 1775876828.114436625 secs.nsecs start_time: 1775876793.856524932 secs.nsecs elapsed_time: 34.257911693 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffffa0f843fa3000.statahead_batch_max=64 llite.lustre-ffffa0f843fa3000.statahead_max=128