== sanity test 123ad: Verify batching statahead works correctly ========================================================== 13:52:04 (1785952324) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff92f608159800.batch_stats=0 llite.lustre-ffff92f608159800.statahead_max=32 llite.lustre-ffff92f608159800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff92f608159800.batch_stats=0 total: 100 open/close in 3.48 seconds: 28.73 ops/second llite.lustre-ffff92f608159800.statahead_max=0 101 real 0m3.656s user 0m0.008s sys 0m0.307s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff92f608159800.statahead_max=32 llite.lustre-ffff92f608159800.statahead_max=32 101 real 0m1.885s user 0m0.007s sys 0m0.209s 'ls -l' 100 files with statahead: 1 sec statahead total: 146 statahead wrong: 2 agl total: 138 list_total: 30 fname_total: 101 hit_total: 4077 miss_total: 1286 swrong=2 ewrong=2 - open/close 377 (time 1785952362.13 total 10.03 last 37.58) - open/close 627 (time 1785952372.22 total 20.12 last 24.78) total: 900 open/close in 26.81 seconds: 33.57 ops/second llite.lustre-ffff92f608159800.statahead_max=0 1001 real 0m37.956s user 0m0.036s sys 0m3.150s 'ls -l' 1000 files without statahead: 38 sec llite.lustre-ffff92f608159800.statahead_max=32 llite.lustre-ffff92f608159800.statahead_max=32 1001 real 0m14.803s user 0m0.021s sys 0m1.373s 'ls -l' 1000 files with statahead: 15 sec statahead total: 147 statahead wrong: 2 agl total: 139 list_total: 31 fname_total: 101 hit_total: 5073 miss_total: 1290 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 28 seconds llite.lustre-ffff92f608159800.statahead_stats= statahead total: 147 statahead wrong: 2 agl total: 139 list_total: 31 fname_total: 101 hit_total: 5073 miss_total: 1290 mdc.lustre-MDT0000-mdc-ffff92f608159800.batch_stats= snapshot_time: 1785952477.823273964 secs.nsecs start_time: 1785952331.901488207 secs.nsecs elapsed_time: 145.921785757 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 394 (time 1785952491.68 total 10.04 last 39.23) - open/close 765 (time 1785952501.69 total 20.05 last 37.06) - open/close 1548 (time 1785952511.69 total 30.05 last 78.29) - open/close 2287 (time 1785952521.69 total 40.06 last 73.89) - open/close 2812 (time 1785952531.72 total 50.09 last 52.33) - open/close 3346 (time 1785952541.73 total 60.10 last 53.35) - open/close 4049 (time 1785952551.74 total 70.10 last 70.25) - open/close 4675 (time 1785952561.74 total 80.11 last 62.59) - open/close 5230 (time 1785952571.76 total 90.12 last 55.40) - open/close 5638 (time 1785952581.77 total 100.13 last 40.77) - open/close 6243 (time 1785952591.77 total 110.14 last 60.47) - open/close 7051 (time 1785952601.77 total 120.14 last 80.77) - open/close 7971 (time 1785952611.78 total 130.14 last 92.00) - open/close 8796 (time 1785952621.78 total 140.15 last 82.44) - open/close 9469 (time 1785952631.79 total 150.15 last 67.27) total: 10000 open/close in 159.28 seconds: 62.78 ops/second llite.lustre-ffff92f608159800.statahead_batch_max=0 llite.lustre-ffff92f608159800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92f608159800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m33.120s user 0m0.283s sys 0m15.179s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff92f608159800.statahead_batch_max=32 llite.lustre-ffff92f608159800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92f608159800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff92f608159800.stats=clear ls -l files in d123ad.sanity: 10001 real 2m30.768s user 0m0.296s sys 0m14.762s batched statahead RPC= hit count=9981 mdc.lustre-MDT0000-mdc-ffff92f608159800.batch_stats= snapshot_time: 1785952915.387746439 secs.nsecs start_time: 1785952748.597037346 secs.nsecs elapsed_time: 166.790709093 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff92f608159800.batch_stats=0 llite.lustre-ffff92f608159800.statahead_max=2048 llite.lustre-ffff92f608159800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff92f608159800.batch_stats=0 total: 100 open/close in 3.39 seconds: 29.51 ops/second llite.lustre-ffff92f608159800.statahead_max=0 101 real 0m2.424s user 0m0.004s sys 0m0.318s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff92f608159800.statahead_max=2048 llite.lustre-ffff92f608159800.statahead_max=2048 101 real 0m1.473s user 0m0.005s sys 0m0.182s 'ls -l' 100 files with statahead: 2 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10080 miss_total: 20 swrong=0 ewrong=0 - open/close 763 (time 1785953359.10 total 10.01 last 76.20) total: 900 open/close in 11.72 seconds: 76.79 ops/second llite.lustre-ffff92f608159800.statahead_max=0 1001 real 0m31.288s user 0m0.034s sys 0m3.089s 'ls -l' 1000 files without statahead: 31 sec llite.lustre-ffff92f608159800.statahead_max=2048 llite.lustre-ffff92f608159800.statahead_max=2048 1001 real 0m11.003s user 0m0.026s sys 0m1.332s 'ls -l' 1000 files with statahead: 11 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 29 seconds llite.lustre-ffff92f608159800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 mdc.lustre-MDT0000-mdc-ffff92f608159800.batch_stats= snapshot_time: 1785953454.405971948 secs.nsecs start_time: 1785952918.842765122 secs.nsecs elapsed_time: 535.563206826 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 22 22 16: 1 11 33 32: 1 11 44 64: 0 0 44 128: 1 11 55 256: 4 44 100 - open/close 391 (time 1785953467.99 total 10.04 last 38.95) - open/close 827 (time 1785953478.01 total 20.06 last 43.52) - open/close 1256 (time 1785953488.03 total 30.08 last 42.81) - open/close 1787 (time 1785953498.03 total 40.08 last 53.10) - open/close 2283 (time 1785953508.05 total 50.10 last 49.51) - open/close 2747 (time 1785953518.07 total 60.12 last 46.31) - open/close 3202 (time 1785953528.07 total 70.12 last 45.48) - open/close 3670 (time 1785953538.07 total 80.12 last 46.79) - open/close 4261 (time 1785953548.08 total 90.13 last 59.04) - open/close 4772 (time 1785953558.09 total 100.14 last 51.07) - open/close 5205 (time 1785953568.09 total 110.14 last 43.29) - open/close 5610 (time 1785953578.11 total 120.16 last 40.41) - open/close 5982 (time 1785953588.13 total 130.18 last 37.15) - open/close 6442 (time 1785953598.13 total 140.18 last 45.99) - open/close 6990 (time 1785953608.14 total 150.19 last 54.72) - open/close 7431 (time 1785953618.15 total 160.20 last 44.05) - open/close 7970 (time 1785953628.16 total 170.20 last 53.89) - open/close 8540 (time 1785953638.18 total 180.23 last 56.85) - open/close 9266 (time 1785953648.19 total 190.24 last 72.53) - open/close 9599 (time 1785953658.20 total 200.25 last 33.26) total: 10000 open/close in 207.18 seconds: 48.27 ops/second llite.lustre-ffff92f608159800.statahead_batch_max=0 llite.lustre-ffff92f608159800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92f608159800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m29.129s user 0m0.323s sys 0m15.210s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff92f608159800.statahead_batch_max=256 llite.lustre-ffff92f608159800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92f608159800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff92f608159800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m20.919s user 0m0.279s sys 0m14.458s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff92f608159800.batch_stats= snapshot_time: 1785953871.666301224 secs.nsecs start_time: 1785953771.126204038 secs.nsecs elapsed_time: 100.540097186 secs.nsecs subreqs per batch batches %% cum %% 1: 182 42 42 2: 81 18 61 4: 56 12 74 8: 41 9 83 16: 22 5 88 32: 12 2 91 64: 2 0 91 128: 2 0 92 256: 33 7 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 1 seconds: 0.000000 unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff92f608159800.statahead_batch_max=64 llite.lustre-ffff92f608159800.statahead_max=128