== sanity test 123ad: Verify batching statahead works correctly ========================================================== 19:58:08 (1786406288) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffffa0654a3d9800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffffa0654a3d9800.batch_stats=0 llite.lustre-ffffa0654a3d9800.statahead_max=32 llite.lustre-ffffa0654a3d9800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffffa0654a3d9800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffffa0654a3d9800.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.11 seconds: 47.34 ops/second llite.lustre-ffffa0654a3d9800.statahead_max=0 101 real 0m2.018s user 0m0.005s sys 0m0.371s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffffa0654a3d9800.statahead_max=32 llite.lustre-ffffa0654a3d9800.statahead_max=32 101 real 0m1.191s user 0m0.006s sys 0m0.190s 'ls -l' 100 files with statahead: 1 sec statahead total: 216 statahead wrong: 2 agl total: 208 list_total: 31 fname_total: 180 hit_total: 4034 miss_total: 1502 swrong=2 ewrong=2 - open/close 778 (time 1786406316.16 total 10.02 last 77.68) total: 900 open/close in 11.63 seconds: 77.38 ops/second llite.lustre-ffffa0654a3d9800.statahead_max=0 1001 real 0m20.080s user 0m0.040s sys 0m3.414s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffffa0654a3d9800.statahead_max=32 llite.lustre-ffffa0654a3d9800.statahead_max=32 1001 real 0m14.224s user 0m0.028s sys 0m1.584s 'ls -l' 1000 files with statahead: 14 sec statahead total: 217 statahead wrong: 2 agl total: 209 list_total: 32 fname_total: 180 hit_total: 5030 miss_total: 1506 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 28 seconds llite.lustre-ffffa0654a3d9800.statahead_stats= statahead total: 217 statahead wrong: 2 agl total: 209 list_total: 32 fname_total: 180 hit_total: 5030 miss_total: 1506 mdc.lustre-MDT0000-mdc-ffffa0654a3d9800.batch_stats= snapshot_time: 1786406394.086662457 secs.nsecs start_time: 1786406293.485621096 secs.nsecs elapsed_time: 100.601041361 secs.nsecs subreqs per batch batches %% cum %% 1: 2 5 5 2: 1 2 7 4: 1 2 10 8: 3 7 17 16: 17 42 60 32: 16 40 100 mdc.lustre-MDT0001-mdc-ffffa0654a3d9800.batch_stats= snapshot_time: 1786406394.086859475 secs.nsecs start_time: 1786406293.485863356 secs.nsecs elapsed_time: 100.600996119 secs.nsecs subreqs per batch batches %% cum %% 1: 3 7 7 2: 0 0 7 4: 3 7 15 8: 0 0 15 16: 19 47 62 32: 15 37 100 - open/close 704 (time 1786406406.77 total 10.02 last 70.29) - open/close 1421 (time 1786406416.78 total 20.02 last 71.64) - open/close 2145 (time 1786406426.79 total 30.03 last 72.35) - open/close 2820 (time 1786406436.79 total 40.04 last 67.47) - open/close 3459 (time 1786406446.92 total 50.16 last 63.10) - open/close 4144 (time 1786406456.92 total 60.17 last 68.47) - open/close 4951 (time 1786406466.93 total 70.17 last 80.65) - open/close 5600 (time 1786406476.93 total 80.18 last 64.88) - open/close 6346 (time 1786406486.93 total 90.18 last 74.59) - open/close 7147 (time 1786406496.94 total 100.18 last 80.07) - open/close 7957 (time 1786406506.94 total 110.19 last 80.96) - open/close 8671 (time 1786406516.96 total 120.20 last 71.31) - open/close 9464 (time 1786406526.97 total 130.21 last 79.20) total: 10000 open/close in 137.64 seconds: 72.65 ops/second llite.lustre-ffffa0654a3d9800.statahead_batch_max=0 llite.lustre-ffffa0654a3d9800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffffa0654a3d9800.stats=clear mdc.lustre-MDT0001-mdc-ffffa0654a3d9800.stats=clear ls -l files in d123ad.sanity: 10001 real 2m10.540s user 0m0.349s sys 0m18.545s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffffa0654a3d9800.statahead_batch_max=32 llite.lustre-ffffa0654a3d9800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffffa0654a3d9800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffffa0654a3d9800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffffa0654a3d9800.stats=clear mdc.lustre-MDT0001-mdc-ffffa0654a3d9800.stats=clear ls -l files in d123ad.sanity: 10001 real 2m19.364s user 0m0.391s sys 0m17.613s batched statahead RPC= hit count=9985 mdc.lustre-MDT0000-mdc-ffffa0654a3d9800.batch_stats= snapshot_time: 1786406827.082522060 secs.nsecs start_time: 1786406673.407805776 secs.nsecs elapsed_time: 153.674716284 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 0 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffffa0654a3d9800.batch_stats= snapshot_time: 1786406827.082591769 secs.nsecs start_time: 1786406673.407968177 secs.nsecs elapsed_time: 153.674623592 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffffa0654a3d9800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffffa0654a3d9800.batch_stats=0 llite.lustre-ffffa0654a3d9800.statahead_max=2048 llite.lustre-ffffa0654a3d9800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffffa0654a3d9800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffffa0654a3d9800.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.47 seconds: 40.43 ops/second llite.lustre-ffffa0654a3d9800.statahead_max=0 101 real 0m1.606s user 0m0.004s sys 0m0.342s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffffa0654a3d9800.statahead_max=2048 llite.lustre-ffffa0654a3d9800.statahead_max=2048 101 real 0m1.128s user 0m0.004s sys 0m0.168s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10084 miss_total: 16 swrong=0 ewrong=0 - open/close 852 (time 1786407174.01 total 10.00 last 85.20) total: 900 open/close in 10.73 seconds: 83.84 ops/second llite.lustre-ffffa0654a3d9800.statahead_max=0 1001 real 0m19.120s user 0m0.026s sys 0m3.252s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffffa0654a3d9800.statahead_max=2048 llite.lustre-ffffa0654a3d9800.statahead_max=2048 1001 real 0m8.615s user 0m0.025s sys 0m1.619s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11083 miss_total: 17 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 28 seconds llite.lustre-ffffa0654a3d9800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11083 miss_total: 17 mdc.lustre-MDT0000-mdc-ffffa0654a3d9800.batch_stats= snapshot_time: 1786407243.620780060 secs.nsecs start_time: 1786406829.912870626 secs.nsecs elapsed_time: 413.707909434 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 2 22 22 8: 3 33 55 16: 1 11 66 32: 1 11 77 64: 0 0 77 128: 0 0 77 256: 2 22 100 mdc.lustre-MDT0001-mdc-ffffa0654a3d9800.batch_stats= snapshot_time: 1786407243.621019731 secs.nsecs start_time: 1786406829.913035188 secs.nsecs elapsed_time: 413.707984543 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 3 33 33 8: 2 22 55 16: 1 11 66 32: 0 0 66 64: 1 11 77 128: 0 0 77 256: 2 22 100 - open/close 760 (time 1786407256.06 total 10.01 last 75.95) - open/close 1666 (time 1786407266.07 total 20.01 last 90.57) - open/close 2525 (time 1786407276.08 total 30.02 last 85.82) - open/close 3408 (time 1786407286.08 total 40.03 last 88.24) - open/close 4319 (time 1786407296.09 total 50.03 last 91.04) - open/close 5199 (time 1786407306.10 total 60.04 last 87.95) - open/close 6054 (time 1786407316.10 total 70.04 last 85.49) - open/close 6932 (time 1786407326.10 total 80.04 last 87.77) - open/close 7873 (time 1786407336.11 total 90.05 last 94.04) - open/close 8757 (time 1786407346.12 total 100.06 last 88.32) - open/close 9607 (time 1786407356.13 total 110.08 last 84.86) total: 10000 open/close in 114.59 seconds: 87.27 ops/second llite.lustre-ffffa0654a3d9800.statahead_batch_max=0 llite.lustre-ffffa0654a3d9800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffffa0654a3d9800.stats=clear mdc.lustre-MDT0001-mdc-ffffa0654a3d9800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m32.250s user 0m0.305s sys 0m17.531s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffffa0654a3d9800.statahead_batch_max=256 llite.lustre-ffffa0654a3d9800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffffa0654a3d9800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffffa0654a3d9800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffffa0654a3d9800.stats=clear mdc.lustre-MDT0001-mdc-ffffa0654a3d9800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m2.468s user 0m0.291s sys 0m17.144s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffffa0654a3d9800.batch_stats= snapshot_time: 1786407537.443570846 secs.nsecs start_time: 1786407461.349577232 secs.nsecs elapsed_time: 76.093993614 secs.nsecs subreqs per batch batches %% cum %% 1: 89 42 42 2: 18 8 51 4: 16 7 59 8: 17 8 67 16: 13 6 73 32: 14 6 80 64: 5 2 82 128: 1 0 83 256: 35 16 100 mdc.lustre-MDT0001-mdc-ffffa0654a3d9800.batch_stats= snapshot_time: 1786407537.443640694 secs.nsecs start_time: 1786407461.349644856 secs.nsecs elapsed_time: 76.093995838 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffffa0654a3d9800.statahead_batch_max=64 llite.lustre-ffffa0654a3d9800.statahead_max=128