== sanity test 123ad: Verify batching statahead works correctly ========================================================== 09:28:17 (1788528497) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff98363687e000.batch_stats=0 llite.lustre-ffff98363687e000.statahead_max=32 llite.lustre-ffff98363687e000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff98363687e000.batch_stats=0 total: 100 open/close in 2.86 seconds: 34.91 ops/second llite.lustre-ffff98363687e000.statahead_max=0 101 real 0m1.850s user 0m0.005s sys 0m0.307s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff98363687e000.statahead_max=32 llite.lustre-ffff98363687e000.statahead_max=32 101 real 0m1.141s user 0m0.003s sys 0m0.162s 'ls -l' 100 files with statahead: 1 sec statahead total: 111 statahead wrong: 2 agl total: 106 list_total: 29 fname_total: 82 hit_total: 3929 miss_total: 1677 swrong=2 ewrong=2 - open/close 702 (time 1788528526.53 total 10.01 last 70.13) total: 900 open/close in 12.50 seconds: 72.03 ops/second llite.lustre-ffff98363687e000.statahead_max=0 1001 real 0m18.894s user 0m0.032s sys 0m2.830s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff98363687e000.statahead_max=32 llite.lustre-ffff98363687e000.statahead_max=32 1001 real 0m8.996s user 0m0.023s sys 0m1.242s 'ls -l' 1000 files with statahead: 9 sec statahead total: 112 statahead wrong: 2 agl total: 107 list_total: 30 fname_total: 82 hit_total: 4927 miss_total: 1679 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff98363687e000.statahead_stats= statahead total: 112 statahead wrong: 2 agl total: 107 list_total: 30 fname_total: 82 hit_total: 4927 miss_total: 1679 mdc.lustre-MDT0000-mdc-ffff98363687e000.batch_stats= snapshot_time: 1788528593.676974326 secs.nsecs start_time: 1788528502.887389103 secs.nsecs elapsed_time: 90.789585223 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 1 2 12 32: 34 87 100 - open/close 766 (time 1788528606.11 total 10.00 last 76.56) - open/close 1563 (time 1788528616.11 total 20.01 last 79.65) - open/close 2396 (time 1788528626.12 total 30.02 last 83.22) - open/close 3249 (time 1788528636.13 total 40.03 last 85.22) - open/close 3918 (time 1788528646.13 total 50.03 last 66.90) - open/close 4643 (time 1788528656.15 total 60.04 last 72.42) - open/close 5474 (time 1788528666.15 total 70.05 last 83.04) - open/close 6276 (time 1788528676.16 total 80.05 last 80.17) - open/close 6983 (time 1788528686.17 total 90.06 last 70.63) - open/close 7683 (time 1788528696.17 total 100.07 last 69.98) - open/close 8327 (time 1788528706.17 total 110.07 last 64.37) - open/close 9140 (time 1788528716.18 total 120.07 last 81.29) total: 10000 open/close in 130.04 seconds: 76.90 ops/second llite.lustre-ffff98363687e000.statahead_batch_max=0 llite.lustre-ffff98363687e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98363687e000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m3.823s user 0m0.284s sys 0m13.107s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff98363687e000.statahead_batch_max=32 llite.lustre-ffff98363687e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98363687e000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff98363687e000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m32.434s user 0m0.278s sys 0m12.713s batched statahead RPC= hit count=9982 mdc.lustre-MDT0000-mdc-ffff98363687e000.batch_stats= snapshot_time: 1788528901.104386340 secs.nsecs start_time: 1788528797.563837392 secs.nsecs elapsed_time: 103.540548948 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff98363687e000.batch_stats=0 llite.lustre-ffff98363687e000.statahead_max=2048 llite.lustre-ffff98363687e000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff98363687e000.batch_stats=0 total: 100 open/close in 2.48 seconds: 40.34 ops/second llite.lustre-ffff98363687e000.statahead_max=0 101 real 0m2.672s user 0m0.007s sys 0m0.312s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff98363687e000.statahead_max=2048 llite.lustre-ffff98363687e000.statahead_max=2048 101 real 0m0.915s user 0m0.007s sys 0m0.149s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10081 miss_total: 19 swrong=0 ewrong=0 - open/close 742 (time 1788529183.47 total 10.00 last 74.19) total: 900 open/close in 12.26 seconds: 73.40 ops/second llite.lustre-ffff98363687e000.statahead_max=0 1001 real 0m21.260s user 0m0.038s sys 0m2.745s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff98363687e000.statahead_max=2048 llite.lustre-ffff98363687e000.statahead_max=2048 1001 real 0m7.565s user 0m0.028s sys 0m1.363s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff98363687e000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 mdc.lustre-MDT0000-mdc-ffff98363687e000.batch_stats= snapshot_time: 1788529251.819238146 secs.nsecs start_time: 1788528903.537694431 secs.nsecs elapsed_time: 348.281543715 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 1 9 27 32: 2 18 45 64: 1 9 54 128: 1 9 63 256: 4 36 100 - open/close 861 (time 1788529265.01 total 10.00 last 86.07) - open/close 1679 (time 1788529275.01 total 20.00 last 81.80) - open/close 2367 (time 1788529285.02 total 30.02 last 68.71) - open/close 3175 (time 1788529295.02 total 40.02 last 80.79) - open/close 3841 (time 1788529305.03 total 50.03 last 66.53) - open/close 4428 (time 1788529315.04 total 60.04 last 58.65) - open/close 5292 (time 1788529325.05 total 70.05 last 86.31) - open/close 6073 (time 1788529335.06 total 80.05 last 78.06) - open/close 6912 (time 1788529345.06 total 90.06 last 83.85) - open/close 7694 (time 1788529355.07 total 100.07 last 78.15) - open/close 8251 (time 1788529365.07 total 110.07 last 55.70) - open/close 8802 (time 1788529375.10 total 120.09 last 54.96) - open/close 9359 (time 1788529385.10 total 130.10 last 55.68) total: 10000 open/close in 137.25 seconds: 72.86 ops/second llite.lustre-ffff98363687e000.statahead_batch_max=0 llite.lustre-ffff98363687e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98363687e000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.031s user 0m0.270s sys 0m13.026s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff98363687e000.statahead_batch_max=256 llite.lustre-ffff98363687e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98363687e000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff98363687e000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m47.001s user 0m0.259s sys 0m12.602s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff98363687e000.batch_stats= snapshot_time: 1788529512.678089181 secs.nsecs start_time: 1788529454.718640009 secs.nsecs elapsed_time: 57.959449172 secs.nsecs subreqs per batch batches %% cum %% 1: 115 43 43 2: 38 14 57 4: 29 10 68 8: 15 5 74 16: 12 4 78 32: 19 7 85 64: 3 1 86 128: 0 0 86 256: 35 13 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff98363687e000.statahead_batch_max=64 llite.lustre-ffff98363687e000.statahead_max=128