== sanity test 123ad: Verify batching statahead works correctly ========================================================== 03:06:27 (1779692787) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8da7492c0000.batch_stats=0 llite.lustre-ffff8da7492c0000.statahead_max=32 llite.lustre-ffff8da7492c0000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8da7492c0000.batch_stats=0 total: 100 open/close in 1.84 seconds: 54.20 ops/second llite.lustre-ffff8da7492c0000.statahead_max=0 101 real 0m1.907s user 0m0.006s sys 0m0.416s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8da7492c0000.statahead_max=32 llite.lustre-ffff8da7492c0000.statahead_max=32 101 real 0m0.816s user 0m0.003s sys 0m0.231s 'ls -l' 100 files with statahead: 1 sec statahead total: 134 statahead wrong: 2 agl total: 128 list_total: 29 fname_total: 93 hit_total: 4138 miss_total: 1457 swrong=2 ewrong=2 - open/close 888 (time 1779692810.86 total 10.01 last 88.72) total: 900 open/close in 10.13 seconds: 88.81 ops/second llite.lustre-ffff8da7492c0000.statahead_max=0 1001 real 0m19.369s user 0m0.047s sys 0m3.712s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff8da7492c0000.statahead_max=32 llite.lustre-ffff8da7492c0000.statahead_max=32 1001 real 0m7.447s user 0m0.022s sys 0m1.827s 'ls -l' 1000 files with statahead: 8 sec statahead total: 135 statahead wrong: 2 agl total: 129 list_total: 30 fname_total: 93 hit_total: 5135 miss_total: 1460 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 14 seconds llite.lustre-ffff8da7492c0000.statahead_stats= statahead total: 135 statahead wrong: 2 agl total: 129 list_total: 30 fname_total: 93 hit_total: 5135 miss_total: 1460 mdc.lustre-MDT0000-mdc-ffff8da7492c0000.batch_stats= snapshot_time: 1779692860.659498428 secs.nsecs start_time: 1779692791.149844513 secs.nsecs elapsed_time: 69.509653915 secs.nsecs subreqs per batch batches % cum % 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 1031 (time 1779692872.24 total 10.00 last 103.06) - open/close 2105 (time 1779692882.24 total 20.01 last 107.36) - open/close 3228 (time 1779692892.25 total 30.01 last 112.22) - open/close 4298 (time 1779692902.26 total 40.02 last 106.91) - open/close 5068 (time 1779692912.26 total 50.03 last 76.95) - open/close 5508 (time 1779692922.27 total 60.04 last 43.97) - open/close 5957 (time 1779692932.28 total 70.04 last 44.87) - open/close 6315 (time 1779692942.32 total 80.09 last 35.63) - open/close 6635 (time 1779692952.34 total 90.10 last 31.96) - open/close 6979 (time 1779692962.35 total 100.12 last 34.36) - open/close 7444 (time 1779692972.39 total 110.15 last 46.32) - open/close 7854 (time 1779692982.39 total 120.16 last 40.99) - open/close 8215 (time 1779692992.42 total 130.18 last 36.00) - open/close 8644 (time 1779693002.43 total 140.20 last 42.83) - open/close 8986 (time 1779693012.45 total 150.22 last 34.15) - open/close 9216 (time 1779693022.48 total 160.24 last 22.94) - open/close 9436 (time 1779693032.49 total 170.26 last 21.97) - open/close 9717 (time 1779693042.51 total 180.27 last 28.06) total: 10000 open/close in 188.22 seconds: 53.13 ops/second llite.lustre-ffff8da7492c0000.statahead_batch_max=0 llite.lustre-ffff8da7492c0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8da7492c0000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m25.746s user 0m0.512s sys 0m21.167s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8da7492c0000.statahead_batch_max=32 llite.lustre-ffff8da7492c0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8da7492c0000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8da7492c0000.stats=clear ls -l files in d123ad.sanity: 10001 real 2m20.834s user 0m0.461s sys 0m18.659s batched statahead RPC= hit count=9978 mdc.lustre-MDT0000-mdc-ffff8da7492c0000.batch_stats= snapshot_time: 1779693312.030864444 secs.nsecs start_time: 1779693151.284442158 secs.nsecs elapsed_time: 160.746422286 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 1 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8da7492c0000.batch_stats=0 llite.lustre-ffff8da7492c0000.statahead_max=2048 llite.lustre-ffff8da7492c0000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8da7492c0000.batch_stats=0 total: 100 open/close in 5.55 seconds: 18.03 ops/second llite.lustre-ffff8da7492c0000.statahead_max=0 101 real 0m4.925s user 0m0.007s sys 0m0.459s 'ls -l' 100 files without statahead: 4 sec llite.lustre-ffff8da7492c0000.statahead_max=2048 llite.lustre-ffff8da7492c0000.statahead_max=2048 101 real 0m1.586s user 0m0.012s sys 0m0.248s 'ls -l' 100 files with statahead: 2 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10077 miss_total: 23 swrong=0 ewrong=0 - open/close 525 (time 1779693797.93 total 10.01 last 52.43) total: 900 open/close in 17.74 seconds: 50.73 ops/second llite.lustre-ffff8da7492c0000.statahead_max=0 1001 real 0m34.173s user 0m0.047s sys 0m3.798s 'ls -l' 1000 files without statahead: 34 sec llite.lustre-ffff8da7492c0000.statahead_max=2048 llite.lustre-ffff8da7492c0000.statahead_max=2048 1001 real 0m14.154s user 0m0.047s sys 0m1.982s 'ls -l' 1000 files with statahead: 14 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11076 miss_total: 24 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 38 seconds llite.lustre-ffff8da7492c0000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11076 miss_total: 24 mdc.lustre-MDT0000-mdc-ffff8da7492c0000.batch_stats= snapshot_time: 1779693913.133916948 secs.nsecs start_time: 1779693315.724593819 secs.nsecs elapsed_time: 597.409323129 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 0 0 54 128: 1 9 63 256: 4 36 100 - open/close 538 (time 1779693926.65 total 10.00 last 53.78) - open/close 1094 (time 1779693936.66 total 20.01 last 55.55) - open/close 1598 (time 1779693946.69 total 30.04 last 50.24) - open/close 1995 (time 1779693956.71 total 40.06 last 39.61) - open/close 2473 (time 1779693966.72 total 50.08 last 47.74) - open/close 2915 (time 1779693976.73 total 60.08 last 44.19) - open/close 3321 (time 1779693986.73 total 70.08 last 40.60) - open/close 3706 (time 1779693996.74 total 80.10 last 38.43) - open/close 4162 (time 1779694006.75 total 90.11 last 45.55) - open/close 4579 (time 1779694016.75 total 100.11 last 41.70) - open/close 5025 (time 1779694026.77 total 110.13 last 44.51) - open/close 5552 (time 1779694036.79 total 120.15 last 52.59) - open/close 6097 (time 1779694046.80 total 130.16 last 54.46) - open/close 6689 (time 1779694056.83 total 140.18 last 59.05) - open/close 7065 (time 1779694066.84 total 150.19 last 37.56) - open/close 7542 (time 1779694076.85 total 160.20 last 47.65) - open/close 8048 (time 1779694086.85 total 170.21 last 50.57) - open/close 8601 (time 1779694096.88 total 180.23 last 55.17) - open/close 9014 (time 1779694106.88 total 190.23 last 41.29) - open/close 9497 (time 1779694116.89 total 200.24 last 48.25) - open/close 9931 (time 1779694126.90 total 210.25 last 43.35) total: 10000 open/close in 211.38 seconds: 47.31 ops/second llite.lustre-ffff8da7492c0000.statahead_batch_max=0 llite.lustre-ffff8da7492c0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8da7492c0000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m28.222s user 0m0.516s sys 0m21.709s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8da7492c0000.statahead_batch_max=256 llite.lustre-ffff8da7492c0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8da7492c0000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8da7492c0000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m31.528s user 0m0.461s sys 0m21.161s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8da7492c0000.batch_stats= snapshot_time: 1779694345.527924640 secs.nsecs start_time: 1779694233.287200823 secs.nsecs elapsed_time: 112.240723817 secs.nsecs subreqs per batch batches % cum % 1: 142 47 47 2: 47 15 63 4: 27 9 72 8: 18 6 78 16: 17 5 83 32: 8 2 86 64: 5 1 88 128: 0 0 88 256: 35 11 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8da7492c0000.statahead_batch_max=64 llite.lustre-ffff8da7492c0000.statahead_max=128