== sanity test 123ad: Verify batching statahead works correctly ========================================================== 19:40:15 (1789688415) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff903b74da2000.batch_stats=0 llite.lustre-ffff903b74da2000.statahead_max=32 llite.lustre-ffff903b74da2000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff903b74da2000.batch_stats=0 total: 100 open/close in 2.44 seconds: 41.06 ops/second llite.lustre-ffff903b74da2000.statahead_max=0 101 real 0m2.332s user 0m0.007s sys 0m0.310s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff903b74da2000.statahead_max=32 llite.lustre-ffff903b74da2000.statahead_max=32 101 real 0m1.180s user 0m0.005s sys 0m0.166s 'ls -l' 100 files with statahead: 1 sec statahead total: 111 statahead wrong: 2 agl total: 106 list_total: 30 fname_total: 79 hit_total: 3666 miss_total: 1556 swrong=2 ewrong=2 - open/close 643 (time 1789688441.35 total 10.01 last 64.23) total: 900 open/close in 13.43 seconds: 67.01 ops/second llite.lustre-ffff903b74da2000.statahead_max=0 1001 real 0m21.083s user 0m0.029s sys 0m2.888s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff903b74da2000.statahead_max=32 llite.lustre-ffff903b74da2000.statahead_max=32 1001 real 0m8.916s user 0m0.027s sys 0m1.357s 'ls -l' 1000 files with statahead: 9 sec statahead total: 112 statahead wrong: 2 agl total: 107 list_total: 31 fname_total: 79 hit_total: 4663 miss_total: 1559 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff903b74da2000.statahead_stats= statahead total: 112 statahead wrong: 2 agl total: 107 list_total: 31 fname_total: 79 hit_total: 4663 miss_total: 1559 mdc.lustre-MDT0000-mdc-ffff903b74da2000.batch_stats= snapshot_time: 1789688506.374942380 secs.nsecs start_time: 1789688419.272049729 secs.nsecs elapsed_time: 87.102892651 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 1 2 12 32: 34 87 100 - open/close 817 (time 1789688518.77 total 10.00 last 81.67) - open/close 1531 (time 1789688528.77 total 20.01 last 71.36) - open/close 2303 (time 1789688538.78 total 30.01 last 77.17) - open/close 3092 (time 1789688548.78 total 40.02 last 78.88) - open/close 3862 (time 1789688558.78 total 50.02 last 76.98) - open/close 4640 (time 1789688568.81 total 60.04 last 77.63) - open/close 5398 (time 1789688578.81 total 70.05 last 75.75) - open/close 6140 (time 1789688588.82 total 80.06 last 74.13) - open/close 6914 (time 1789688598.83 total 90.07 last 77.31) - open/close 7682 (time 1789688608.85 total 100.09 last 76.65) - open/close 8436 (time 1789688618.86 total 110.09 last 75.38) - open/close 9218 (time 1789688628.87 total 120.10 last 78.12) - open/close 9964 (time 1789688638.88 total 130.12 last 74.48) total: 10000 open/close in 130.67 seconds: 76.53 ops/second llite.lustre-ffff903b74da2000.statahead_batch_max=0 llite.lustre-ffff903b74da2000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff903b74da2000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.382s user 0m0.305s sys 0m15.340s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff903b74da2000.statahead_batch_max=32 llite.lustre-ffff903b74da2000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff903b74da2000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff903b74da2000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m26.051s user 0m0.309s sys 0m13.523s batched statahead RPC= hit count=9976 mdc.lustre-MDT0000-mdc-ffff903b74da2000.batch_stats= snapshot_time: 1789688798.609971399 secs.nsecs start_time: 1789688703.231662820 secs.nsecs elapsed_time: 95.378308579 secs.nsecs subreqs per batch batches %% cum %% 1: 1 0 0 2: 0 0 0 4: 1 0 0 8: 1 0 0 16: 1 0 1 32: 313 98 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff903b74da2000.batch_stats=0 llite.lustre-ffff903b74da2000.statahead_max=2048 llite.lustre-ffff903b74da2000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff903b74da2000.batch_stats=0 total: 100 open/close in 2.75 seconds: 36.39 ops/second llite.lustre-ffff903b74da2000.statahead_max=0 101 real 0m2.050s user 0m0.003s sys 0m0.289s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff903b74da2000.statahead_max=2048 llite.lustre-ffff903b74da2000.statahead_max=2048 101 real 0m0.825s user 0m0.004s sys 0m0.158s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10075 miss_total: 25 swrong=0 ewrong=0 - open/close 796 (time 1789689094.71 total 10.00 last 79.59) total: 900 open/close in 11.84 seconds: 76.03 ops/second llite.lustre-ffff903b74da2000.statahead_max=0 1001 real 0m21.969s user 0m0.025s sys 0m2.899s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff903b74da2000.statahead_max=2048 llite.lustre-ffff903b74da2000.statahead_max=2048 1001 real 0m5.810s user 0m0.030s sys 0m1.289s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11074 miss_total: 26 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff903b74da2000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11074 miss_total: 26 mdc.lustre-MDT0000-mdc-ffff903b74da2000.batch_stats= snapshot_time: 1789689156.995062670 secs.nsecs start_time: 1789688800.988102665 secs.nsecs elapsed_time: 356.006960005 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 22 22 16: 2 22 44 32: 0 0 44 64: 0 0 44 128: 1 11 55 256: 4 44 100 - open/close 669 (time 1789689169.93 total 10.01 last 66.86) - open/close 1476 (time 1789689179.94 total 20.01 last 80.68) - open/close 2159 (time 1789689189.94 total 30.02 last 68.24) - open/close 2811 (time 1789689199.95 total 40.03 last 65.14) - open/close 3450 (time 1789689209.99 total 50.06 last 63.70) - open/close 3980 (time 1789689219.99 total 60.06 last 52.97) - open/close 4517 (time 1789689230.00 total 70.07 last 53.66) - open/close 5211 (time 1789689240.00 total 80.08 last 69.36) - open/close 5926 (time 1789689250.02 total 90.09 last 71.41) - open/close 6561 (time 1789689260.03 total 100.10 last 63.44) - open/close 7142 (time 1789689270.07 total 110.14 last 57.85) - open/close 7844 (time 1789689280.07 total 120.15 last 70.16) - open/close 8498 (time 1789689290.09 total 130.17 last 65.27) - open/close 9244 (time 1789689300.10 total 140.17 last 74.55) - open/close 9978 (time 1789689310.11 total 150.18 last 73.34) total: 10000 open/close in 150.46 seconds: 66.46 ops/second llite.lustre-ffff903b74da2000.statahead_batch_max=0 llite.lustre-ffff903b74da2000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff903b74da2000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m57.640s user 0m0.322s sys 0m14.746s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff903b74da2000.statahead_batch_max=256 llite.lustre-ffff903b74da2000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff903b74da2000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff903b74da2000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m46.736s user 0m0.309s sys 0m14.939s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff903b74da2000.batch_stats= snapshot_time: 1789689434.544061430 secs.nsecs start_time: 1789689376.918692552 secs.nsecs elapsed_time: 57.625368878 secs.nsecs subreqs per batch batches %% cum %% 1: 156 52 52 2: 35 11 64 4: 20 6 71 8: 15 5 76 16: 16 5 81 32: 11 3 85 64: 5 1 87 128: 4 1 88 256: 34 11 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff903b74da2000.statahead_batch_max=64 llite.lustre-ffff903b74da2000.statahead_max=128