== sanity test 123ad: Verify batching statahead works correctly ========================================================== 22:51:32 (1787799092) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff94664603d800.batch_stats=0 llite.lustre-ffff94664603d800.statahead_max=32 llite.lustre-ffff94664603d800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff94664603d800.batch_stats=0 total: 100 open/close in 3.07 seconds: 32.59 ops/second llite.lustre-ffff94664603d800.statahead_max=0 101 real 0m2.762s user 0m0.009s sys 0m0.296s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff94664603d800.statahead_max=32 llite.lustre-ffff94664603d800.statahead_max=32 101 real 0m1.322s user 0m0.002s sys 0m0.169s 'ls -l' 100 files with statahead: 2 sec statahead total: 149 statahead wrong: 2 agl total: 143 list_total: 31 fname_total: 94 hit_total: 4108 miss_total: 1652 swrong=2 ewrong=2 - open/close 593 (time 1787799122.84 total 10.02 last 59.18) total: 900 open/close in 15.15 seconds: 59.40 ops/second llite.lustre-ffff94664603d800.statahead_max=0 1001 real 0m27.489s user 0m0.037s sys 0m3.123s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff94664603d800.statahead_max=32 llite.lustre-ffff94664603d800.statahead_max=32 1001 real 0m10.563s user 0m0.026s sys 0m1.430s 'ls -l' 1000 files with statahead: 10 sec statahead total: 150 statahead wrong: 2 agl total: 144 list_total: 32 fname_total: 94 hit_total: 5106 miss_total: 1654 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff94664603d800.statahead_stats= statahead total: 150 statahead wrong: 2 agl total: 144 list_total: 32 fname_total: 94 hit_total: 5106 miss_total: 1654 mdc.lustre-MDT0000-mdc-ffff94664603d800.batch_stats= snapshot_time: 1787799202.079445998 secs.nsecs start_time: 1787799097.828876953 secs.nsecs elapsed_time: 104.250569045 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 1 2 5 8: 2 5 10 16: 2 5 15 32: 34 85 100 - open/close 704 (time 1787799214.59 total 10.01 last 70.31) - open/close 1370 (time 1787799224.60 total 20.02 last 66.56) - open/close 2026 (time 1787799234.60 total 30.02 last 65.58) - open/close 2712 (time 1787799244.61 total 40.03 last 68.54) - open/close 3402 (time 1787799254.62 total 50.04 last 68.92) - open/close 4038 (time 1787799264.63 total 60.05 last 63.57) - open/close 4660 (time 1787799274.63 total 70.05 last 62.19) - open/close 5316 (time 1787799284.64 total 80.06 last 65.52) - open/close 5940 (time 1787799294.65 total 90.06 last 62.36) - open/close 6625 (time 1787799304.66 total 100.07 last 68.43) - open/close 7249 (time 1787799314.67 total 110.09 last 62.32) - open/close 7809 (time 1787799324.68 total 120.10 last 55.96) - open/close 8378 (time 1787799334.68 total 130.10 last 56.86) - open/close 8883 (time 1787799344.69 total 140.11 last 50.46) - open/close 9400 (time 1787799354.69 total 150.11 last 51.70) - open/close 9897 (time 1787799364.70 total 160.12 last 49.64) total: 10000 open/close in 161.62 seconds: 61.87 ops/second llite.lustre-ffff94664603d800.statahead_batch_max=0 llite.lustre-ffff94664603d800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff94664603d800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m3.397s user 0m0.323s sys 0m14.487s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff94664603d800.statahead_batch_max=32 llite.lustre-ffff94664603d800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff94664603d800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff94664603d800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m42.067s user 0m0.297s sys 0m13.615s batched statahead RPC= hit count=9974 mdc.lustre-MDT0000-mdc-ffff94664603d800.batch_stats= snapshot_time: 1787799554.180031758 secs.nsecs start_time: 1787799442.251372739 secs.nsecs elapsed_time: 111.928659019 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff94664603d800.batch_stats=0 llite.lustre-ffff94664603d800.statahead_max=2048 llite.lustre-ffff94664603d800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff94664603d800.batch_stats=0 total: 100 open/close in 3.14 seconds: 31.85 ops/second llite.lustre-ffff94664603d800.statahead_max=0 101 real 0m2.502s user 0m0.004s sys 0m0.311s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff94664603d800.statahead_max=2048 llite.lustre-ffff94664603d800.statahead_max=2048 101 real 0m1.026s user 0m0.005s sys 0m0.206s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10073 miss_total: 27 swrong=0 ewrong=0 - open/close 503 (time 1787799904.68 total 10.00 last 50.28) total: 900 open/close in 16.91 seconds: 53.24 ops/second llite.lustre-ffff94664603d800.statahead_max=0 1001 real 0m25.338s user 0m0.041s sys 0m3.286s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff94664603d800.statahead_max=2048 llite.lustre-ffff94664603d800.statahead_max=2048 1001 real 0m7.716s user 0m0.037s sys 0m1.656s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11072 miss_total: 28 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff94664603d800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11072 miss_total: 28 mdc.lustre-MDT0000-mdc-ffff94664603d800.batch_stats= snapshot_time: 1787799981.042465931 secs.nsecs start_time: 1787799557.589955268 secs.nsecs elapsed_time: 423.452510663 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 684 (time 1787799993.78 total 10.01 last 68.36) - open/close 1349 (time 1787800003.79 total 20.02 last 66.44) - open/close 2093 (time 1787800013.82 total 30.05 last 74.16) - open/close 2785 (time 1787800023.82 total 40.05 last 69.20) - open/close 3372 (time 1787800033.83 total 50.06 last 58.65) - open/close 4102 (time 1787800043.84 total 60.06 last 72.95) - open/close 4849 (time 1787800053.85 total 70.07 last 74.63) - open/close 5603 (time 1787800063.85 total 80.08 last 75.34) - open/close 6273 (time 1787800073.85 total 90.08 last 67.00) - open/close 6987 (time 1787800083.86 total 100.08 last 71.39) - open/close 7716 (time 1787800093.86 total 110.09 last 72.84) - open/close 8457 (time 1787800103.87 total 120.10 last 74.02) - open/close 9156 (time 1787800113.88 total 130.10 last 69.89) - open/close 9809 (time 1787800123.88 total 140.11 last 65.28) total: 10000 open/close in 142.27 seconds: 70.29 ops/second llite.lustre-ffff94664603d800.statahead_batch_max=0 llite.lustre-ffff94664603d800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff94664603d800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m13.674s user 0m0.342s sys 0m13.897s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff94664603d800.statahead_batch_max=256 llite.lustre-ffff94664603d800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff94664603d800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff94664603d800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m48.534s user 0m0.323s sys 0m14.160s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff94664603d800.batch_stats= snapshot_time: 1787800270.116638661 secs.nsecs start_time: 1787800209.947356532 secs.nsecs elapsed_time: 60.169282129 secs.nsecs subreqs per batch batches %% cum %% 1: 133 46 46 2: 45 15 62 4: 23 8 71 8: 14 4 75 16: 12 4 80 32: 13 4 84 64: 8 2 87 128: 1 0 87 256: 34 12 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff94664603d800.statahead_batch_max=64 llite.lustre-ffff94664603d800.statahead_max=128