== sanity test 123ad: Verify batching statahead works correctly ========================================================== 19:12:03 (1787785923) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff993dc32ea000.batch_stats=0 llite.lustre-ffff993dc32ea000.statahead_max=32 llite.lustre-ffff993dc32ea000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff993dc32ea000.batch_stats=0 total: 100 open/close in 3.26 seconds: 30.67 ops/second llite.lustre-ffff993dc32ea000.statahead_max=0 101 real 0m2.215s user 0m0.003s sys 0m0.302s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff993dc32ea000.statahead_max=32 llite.lustre-ffff993dc32ea000.statahead_max=32 101 real 0m0.998s user 0m0.006s sys 0m0.172s 'ls -l' 100 files with statahead: 1 sec statahead total: 134 statahead wrong: 2 agl total: 129 list_total: 30 fname_total: 90 hit_total: 3966 miss_total: 1694 swrong=2 ewrong=2 - open/close 635 (time 1787785950.86 total 10.00 last 63.50) total: 900 open/close in 14.13 seconds: 63.67 ops/second llite.lustre-ffff993dc32ea000.statahead_max=0 1001 real 0m23.909s user 0m0.029s sys 0m2.943s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff993dc32ea000.statahead_max=32 llite.lustre-ffff993dc32ea000.statahead_max=32 1001 real 0m11.014s user 0m0.031s sys 0m1.381s 'ls -l' 1000 files with statahead: 11 sec statahead total: 135 statahead wrong: 2 agl total: 130 list_total: 31 fname_total: 90 hit_total: 4964 miss_total: 1696 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff993dc32ea000.statahead_stats= statahead total: 135 statahead wrong: 2 agl total: 130 list_total: 31 fname_total: 90 hit_total: 4964 miss_total: 1696 mdc.lustre-MDT0000-mdc-ffff993dc32ea000.batch_stats= snapshot_time: 1787786025.260708839 secs.nsecs start_time: 1787785928.316229335 secs.nsecs elapsed_time: 96.944479504 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 618 (time 1787786037.66 total 10.00 last 61.79) - open/close 1239 (time 1787786047.69 total 20.03 last 61.92) - open/close 1884 (time 1787786057.69 total 30.04 last 64.47) - open/close 2573 (time 1787786067.71 total 40.05 last 68.81) - open/close 3209 (time 1787786077.71 total 50.05 last 63.56) - open/close 3889 (time 1787786087.71 total 60.06 last 67.99) - open/close 4573 (time 1787786097.72 total 70.06 last 68.39) - open/close 5261 (time 1787786107.72 total 80.06 last 68.77) - open/close 5904 (time 1787786117.73 total 90.08 last 64.21) - open/close 6589 (time 1787786127.75 total 100.09 last 68.42) - open/close 7285 (time 1787786137.75 total 110.10 last 69.54) - open/close 7973 (time 1787786147.76 total 120.10 last 68.80) - open/close 8649 (time 1787786157.77 total 130.11 last 67.54) - open/close 9338 (time 1787786167.77 total 140.11 last 68.88) - open/close 9973 (time 1787786177.78 total 150.12 last 63.45) total: 10000 open/close in 150.57 seconds: 66.42 ops/second llite.lustre-ffff993dc32ea000.statahead_batch_max=0 llite.lustre-ffff993dc32ea000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff993dc32ea000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m2.098s user 0m0.382s sys 0m16.422s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff993dc32ea000.statahead_batch_max=32 llite.lustre-ffff993dc32ea000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff993dc32ea000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff993dc32ea000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m34.710s user 0m0.339s sys 0m14.520s batched statahead RPC= hit count=9986 mdc.lustre-MDT0000-mdc-ffff993dc32ea000.batch_stats= snapshot_time: 1787786355.700655098 secs.nsecs start_time: 1787786248.552818175 secs.nsecs elapsed_time: 107.147836923 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff993dc32ea000.batch_stats=0 llite.lustre-ffff993dc32ea000.statahead_max=2048 llite.lustre-ffff993dc32ea000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff993dc32ea000.batch_stats=0 total: 100 open/close in 2.77 seconds: 36.11 ops/second llite.lustre-ffff993dc32ea000.statahead_max=0 101 real 0m3.041s user 0m0.004s sys 0m0.332s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff993dc32ea000.statahead_max=2048 llite.lustre-ffff993dc32ea000.statahead_max=2048 101 real 0m0.956s user 0m0.006s sys 0m0.183s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10085 miss_total: 15 swrong=0 ewrong=0 - open/close 632 (time 1787786682.93 total 10.04 last 62.95) total: 900 open/close in 14.29 seconds: 62.98 ops/second llite.lustre-ffff993dc32ea000.statahead_max=0 1001 real 0m24.091s user 0m0.040s sys 0m3.172s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff993dc32ea000.statahead_max=2048 llite.lustre-ffff993dc32ea000.statahead_max=2048 1001 real 0m7.510s user 0m0.025s sys 0m1.455s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff993dc32ea000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 mdc.lustre-MDT0000-mdc-ffff993dc32ea000.batch_stats= snapshot_time: 1787786755.483891962 secs.nsecs start_time: 1787786358.140141514 secs.nsecs elapsed_time: 397.343750448 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 0 0 54 128: 1 9 63 256: 4 36 100 - open/close 489 (time 1787786769.29 total 10.00 last 48.88) - open/close 983 (time 1787786779.32 total 20.04 last 49.25) - open/close 1576 (time 1787786789.33 total 30.04 last 59.25) - open/close 2230 (time 1787786799.34 total 40.05 last 65.35) - open/close 2878 (time 1787786809.35 total 50.07 last 64.70) - open/close 3528 (time 1787786819.36 total 60.07 last 64.97) - open/close 4209 (time 1787786829.36 total 70.07 last 68.08) - open/close 4852 (time 1787786839.37 total 80.08 last 64.26) - open/close 5504 (time 1787786849.37 total 90.08 last 65.18) - open/close 6088 (time 1787786859.37 total 100.08 last 58.40) - open/close 6689 (time 1787786869.38 total 110.10 last 60.02) - open/close 7303 (time 1787786879.39 total 120.10 last 61.37) - open/close 7942 (time 1787786889.40 total 130.11 last 63.82) - open/close 8510 (time 1787786899.40 total 140.12 last 56.77) - open/close 9169 (time 1787786909.41 total 150.13 last 65.83) - open/close 9757 (time 1787786919.42 total 160.13 last 58.78) total: 10000 open/close in 163.94 seconds: 61.00 ops/second llite.lustre-ffff993dc32ea000.statahead_batch_max=0 llite.lustre-ffff993dc32ea000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff993dc32ea000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.105s user 0m0.381s sys 0m16.127s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff993dc32ea000.statahead_batch_max=256 llite.lustre-ffff993dc32ea000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff993dc32ea000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff993dc32ea000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m54.574s user 0m0.351s sys 0m15.412s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff993dc32ea000.batch_stats= snapshot_time: 1787787057.452041788 secs.nsecs start_time: 1787786991.999731359 secs.nsecs elapsed_time: 65.452310429 secs.nsecs subreqs per batch batches %% cum %% 1: 130 49 49 2: 22 8 58 4: 22 8 66 8: 14 5 72 16: 11 4 76 32: 18 6 83 64: 9 3 86 128: 1 0 86 256: 34 13 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff993dc32ea000.statahead_batch_max=64 llite.lustre-ffff993dc32ea000.statahead_max=128