== sanity test 123ad: Verify batching statahead works correctly ========================================================== 16:42:34 (1787776954) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff973b87f9c000.batch_stats=0 llite.lustre-ffff973b87f9c000.statahead_max=32 llite.lustre-ffff973b87f9c000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff973b87f9c000.batch_stats=0 total: 100 open/close in 2.47 seconds: 40.47 ops/second llite.lustre-ffff973b87f9c000.statahead_max=0 101 real 0m2.144s user 0m0.005s sys 0m0.312s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff973b87f9c000.statahead_max=32 llite.lustre-ffff973b87f9c000.statahead_max=32 101 real 0m0.983s user 0m0.002s sys 0m0.166s 'ls -l' 100 files with statahead: 1 sec statahead total: 133 statahead wrong: 2 agl total: 125 list_total: 29 fname_total: 104 hit_total: 3934 miss_total: 1371 swrong=2 ewrong=2 - open/close 726 (time 1787776979.01 total 10.01 last 72.54) total: 900 open/close in 12.31 seconds: 73.11 ops/second llite.lustre-ffff973b87f9c000.statahead_max=0 1001 real 0m20.873s user 0m0.038s sys 0m2.905s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff973b87f9c000.statahead_max=32 llite.lustre-ffff973b87f9c000.statahead_max=32 1001 real 0m9.177s user 0m0.027s sys 0m1.393s 'ls -l' 1000 files with statahead: 9 sec statahead total: 134 statahead wrong: 2 agl total: 126 list_total: 30 fname_total: 104 hit_total: 4932 miss_total: 1373 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff973b87f9c000.statahead_stats= statahead total: 134 statahead wrong: 2 agl total: 126 list_total: 30 fname_total: 104 hit_total: 4932 miss_total: 1373 mdc.lustre-MDT0000-mdc-ffff973b87f9c000.batch_stats= snapshot_time: 1787777043.328057739 secs.nsecs start_time: 1787776958.274502287 secs.nsecs elapsed_time: 85.053555452 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 639 (time 1787777055.64 total 10.01 last 63.83) - open/close 1359 (time 1787777065.64 total 20.01 last 71.98) - open/close 2059 (time 1787777075.64 total 30.01 last 70.00) - open/close 2775 (time 1787777085.65 total 40.02 last 71.54) - open/close 3370 (time 1787777095.65 total 50.03 last 59.48) - open/close 4042 (time 1787777105.66 total 60.03 last 67.16) - open/close 4687 (time 1787777115.66 total 70.03 last 64.49) - open/close 5472 (time 1787777125.68 total 80.05 last 78.35) - open/close 6160 (time 1787777135.69 total 90.07 last 68.70) - open/close 6861 (time 1787777145.70 total 100.08 last 70.04) - open/close 7572 (time 1787777155.71 total 110.08 last 71.04) - open/close 8292 (time 1787777165.73 total 120.11 last 71.84) - open/close 8983 (time 1787777175.73 total 130.11 last 69.09) - open/close 9636 (time 1787777185.73 total 140.11 last 65.30) total: 10000 open/close in 145.14 seconds: 68.90 ops/second llite.lustre-ffff973b87f9c000.statahead_batch_max=0 llite.lustre-ffff973b87f9c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff973b87f9c000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.065s user 0m0.299s sys 0m15.258s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff973b87f9c000.statahead_batch_max=32 llite.lustre-ffff973b87f9c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff973b87f9c000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff973b87f9c000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m27.614s user 0m0.306s sys 0m13.481s batched statahead RPC= hit count=9980 mdc.lustre-MDT0000-mdc-ffff973b87f9c000.batch_stats= snapshot_time: 1787777355.110030201 secs.nsecs start_time: 1787777258.020800752 secs.nsecs elapsed_time: 97.089229449 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff973b87f9c000.batch_stats=0 llite.lustre-ffff973b87f9c000.statahead_max=2048 llite.lustre-ffff973b87f9c000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff973b87f9c000.batch_stats=0 total: 100 open/close in 2.36 seconds: 42.34 ops/second llite.lustre-ffff973b87f9c000.statahead_max=0 101 real 0m1.987s user 0m0.006s sys 0m0.295s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff973b87f9c000.statahead_max=2048 llite.lustre-ffff973b87f9c000.statahead_max=2048 101 real 0m0.725s user 0m0.007s sys 0m0.152s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10079 miss_total: 21 swrong=0 ewrong=0 - open/close 661 (time 1787777670.50 total 10.01 last 66.03) total: 900 open/close in 13.22 seconds: 68.09 ops/second llite.lustre-ffff973b87f9c000.statahead_max=0 1001 real 0m21.934s user 0m0.041s sys 0m2.883s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff973b87f9c000.statahead_max=2048 llite.lustre-ffff973b87f9c000.statahead_max=2048 1001 real 0m6.799s user 0m0.026s sys 0m1.233s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff973b87f9c000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 mdc.lustre-MDT0000-mdc-ffff973b87f9c000.batch_stats= snapshot_time: 1787777742.353559605 secs.nsecs start_time: 1787777358.534475855 secs.nsecs elapsed_time: 383.819083750 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 0 0 54 128: 1 9 63 256: 4 36 100 - open/close 719 (time 1787777755.11 total 10.01 last 71.80) - open/close 1413 (time 1787777765.12 total 20.03 last 69.30) - open/close 2139 (time 1787777775.13 total 30.04 last 72.52) - open/close 2900 (time 1787777785.14 total 40.04 last 76.04) - open/close 3570 (time 1787777795.14 total 50.05 last 66.99) - open/close 4276 (time 1787777805.16 total 60.06 last 70.51) - open/close 4933 (time 1787777815.16 total 70.07 last 65.64) - open/close 5601 (time 1787777825.18 total 80.08 last 66.72) - open/close 6245 (time 1787777835.19 total 90.09 last 64.32) - open/close 7013 (time 1787777845.19 total 100.10 last 76.76) - open/close 7699 (time 1787777855.20 total 110.10 last 68.55) - open/close 8364 (time 1787777865.24 total 120.14 last 66.25) - open/close 8912 (time 1787777875.25 total 130.15 last 54.75) - open/close 9402 (time 1787777885.28 total 140.18 last 48.84) - open/close 9939 (time 1787777895.32 total 150.22 last 53.49) total: 10000 open/close in 151.36 seconds: 66.07 ops/second llite.lustre-ffff973b87f9c000.statahead_batch_max=0 llite.lustre-ffff973b87f9c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff973b87f9c000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.482s user 0m0.328s sys 0m14.164s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff973b87f9c000.statahead_batch_max=256 llite.lustre-ffff973b87f9c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff973b87f9c000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff973b87f9c000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m48.236s user 0m0.295s sys 0m14.368s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff973b87f9c000.batch_stats= snapshot_time: 1787778024.761610601 secs.nsecs start_time: 1787777965.968403529 secs.nsecs elapsed_time: 58.793207072 secs.nsecs subreqs per batch batches %% cum %% 1: 125 48 48 2: 25 9 58 4: 21 8 66 8: 16 6 72 16: 12 4 77 32: 17 6 84 64: 5 1 85 128: 1 0 86 256: 35 13 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff973b87f9c000.statahead_batch_max=64 llite.lustre-ffff973b87f9c000.statahead_max=128