== sanity test 123ad: Verify batching statahead works correctly ========================================================== 09:18:32 (1787750312) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8fa74a81c000.batch_stats=0 llite.lustre-ffff8fa74a81c000.statahead_max=32 llite.lustre-ffff8fa74a81c000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8fa74a81c000.batch_stats=0 total: 100 open/close in 3.21 seconds: 31.11 ops/second llite.lustre-ffff8fa74a81c000.statahead_max=0 101 real 0m3.314s user 0m0.004s sys 0m0.337s 'ls -l' 100 files without statahead: 4 sec llite.lustre-ffff8fa74a81c000.statahead_max=32 llite.lustre-ffff8fa74a81c000.statahead_max=32 101 real 0m1.316s user 0m0.003s sys 0m0.143s 'ls -l' 100 files with statahead: 1 sec statahead total: 157 statahead wrong: 2 agl total: 150 list_total: 30 fname_total: 111 hit_total: 4468 miss_total: 1284 swrong=2 ewrong=2 - open/close 638 (time 1787750346.09 total 10.00 last 63.79) total: 900 open/close in 14.45 seconds: 62.29 ops/second llite.lustre-ffff8fa74a81c000.statahead_max=0 1001 real 0m22.405s user 0m0.031s sys 0m2.879s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff8fa74a81c000.statahead_max=32 llite.lustre-ffff8fa74a81c000.statahead_max=32 1001 real 0m10.725s user 0m0.024s sys 0m1.348s 'ls -l' 1000 files with statahead: 11 sec statahead total: 158 statahead wrong: 2 agl total: 151 list_total: 31 fname_total: 111 hit_total: 5466 miss_total: 1286 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff8fa74a81c000.statahead_stats= statahead total: 158 statahead wrong: 2 agl total: 151 list_total: 31 fname_total: 111 hit_total: 5466 miss_total: 1286 mdc.lustre-MDT0000-mdc-ffff8fa74a81c000.batch_stats= snapshot_time: 1787750422.499175541 secs.nsecs start_time: 1787750319.933361906 secs.nsecs elapsed_time: 102.565813635 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 1 2 5 8: 2 5 10 16: 1 2 12 32: 34 87 100 - open/close 740 (time 1787750435.60 total 10.01 last 73.93) - open/close 1379 (time 1787750445.63 total 20.03 last 63.75) - open/close 2097 (time 1787750455.63 total 30.04 last 71.78) - open/close 2767 (time 1787750465.64 total 40.05 last 66.90) - open/close 3465 (time 1787750475.64 total 50.05 last 69.80) - open/close 4080 (time 1787750485.66 total 60.07 last 61.41) - open/close 4815 (time 1787750495.66 total 70.07 last 73.50) - open/close 5483 (time 1787750505.66 total 80.07 last 66.78) - open/close 6107 (time 1787750515.68 total 90.08 last 62.30) - open/close 6833 (time 1787750525.68 total 100.09 last 72.58) - open/close 7498 (time 1787750535.69 total 110.09 last 66.45) - open/close 8189 (time 1787750545.69 total 120.09 last 69.09) - open/close 8878 (time 1787750555.70 total 130.10 last 68.84) - open/close 9547 (time 1787750565.71 total 140.11 last 66.84) total: 10000 open/close in 147.02 seconds: 68.02 ops/second llite.lustre-ffff8fa74a81c000.statahead_batch_max=0 llite.lustre-ffff8fa74a81c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8fa74a81c000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m9.014s user 0m0.335s sys 0m14.402s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8fa74a81c000.statahead_batch_max=32 llite.lustre-ffff8fa74a81c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8fa74a81c000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8fa74a81c000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m45.232s user 0m0.308s sys 0m13.559s batched statahead RPC= hit count=9987 mdc.lustre-MDT0000-mdc-ffff8fa74a81c000.batch_stats= snapshot_time: 1787750773.046008062 secs.nsecs start_time: 1787750652.977584932 secs.nsecs elapsed_time: 120.068423130 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8fa74a81c000.batch_stats=0 llite.lustre-ffff8fa74a81c000.statahead_max=2048 llite.lustre-ffff8fa74a81c000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8fa74a81c000.batch_stats=0 total: 100 open/close in 2.68 seconds: 37.31 ops/second llite.lustre-ffff8fa74a81c000.statahead_max=0 101 real 0m2.735s user 0m0.004s sys 0m0.304s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8fa74a81c000.statahead_max=2048 llite.lustre-ffff8fa74a81c000.statahead_max=2048 101 real 0m1.159s user 0m0.004s sys 0m0.201s 'ls -l' 100 files with statahead: 2 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10086 miss_total: 14 swrong=0 ewrong=0 - open/close 713 (time 1787751107.33 total 10.00 last 71.28) total: 900 open/close in 13.27 seconds: 67.85 ops/second llite.lustre-ffff8fa74a81c000.statahead_max=0 1001 real 0m21.960s user 0m0.039s sys 0m2.799s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff8fa74a81c000.statahead_max=2048 llite.lustre-ffff8fa74a81c000.statahead_max=2048 1001 real 0m7.078s user 0m0.035s sys 0m1.336s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff8fa74a81c000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 mdc.lustre-MDT0000-mdc-ffff8fa74a81c000.batch_stats= snapshot_time: 1787751173.613606276 secs.nsecs start_time: 1787750775.711652465 secs.nsecs elapsed_time: 397.901953811 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 1 9 45 64: 1 9 54 128: 1 9 63 256: 4 36 100 - open/close 753 (time 1787751186.08 total 10.00 last 75.28) - open/close 1488 (time 1787751196.08 total 20.00 last 73.48) - open/close 2219 (time 1787751206.10 total 30.02 last 72.97) - open/close 3000 (time 1787751216.10 total 40.03 last 78.06) - open/close 3789 (time 1787751226.10 total 50.03 last 78.89) - open/close 4494 (time 1787751236.11 total 60.04 last 70.45) - open/close 5150 (time 1787751246.11 total 70.04 last 65.59) - open/close 5866 (time 1787751256.11 total 80.04 last 71.58) - open/close 6568 (time 1787751266.14 total 90.06 last 70.04) - open/close 7276 (time 1787751276.15 total 100.07 last 70.73) - open/close 7999 (time 1787751286.15 total 110.08 last 72.28) - open/close 8670 (time 1787751296.16 total 120.08 last 67.06) - open/close 9454 (time 1787751306.17 total 130.10 last 78.30) total: 10000 open/close in 136.90 seconds: 73.05 ops/second llite.lustre-ffff8fa74a81c000.statahead_batch_max=0 llite.lustre-ffff8fa74a81c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8fa74a81c000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m8.318s user 0m0.349s sys 0m13.803s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8fa74a81c000.statahead_batch_max=256 llite.lustre-ffff8fa74a81c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8fa74a81c000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8fa74a81c000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m51.805s user 0m0.285s sys 0m13.765s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8fa74a81c000.batch_stats= snapshot_time: 1787751454.726501811 secs.nsecs start_time: 1787751391.072457004 secs.nsecs elapsed_time: 63.654044807 secs.nsecs subreqs per batch batches %% cum %% 1: 51 29 29 2: 34 19 49 4: 16 9 58 8: 6 3 61 16: 10 5 67 32: 14 8 75 64: 5 2 78 128: 2 1 79 256: 35 20 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8fa74a81c000.statahead_batch_max=64 llite.lustre-ffff8fa74a81c000.statahead_max=128