== sanity test 123ad: Verify batching statahead works correctly ========================================================== 13:12:43 (1788628363) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff95c191334800.batch_stats=0 llite.lustre-ffff95c191334800.statahead_max=32 llite.lustre-ffff95c191334800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff95c191334800.batch_stats=0 total: 100 open/close in 3.20 seconds: 31.23 ops/second llite.lustre-ffff95c191334800.statahead_max=0 101 real 0m2.330s user 0m0.007s sys 0m0.322s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff95c191334800.statahead_max=32 llite.lustre-ffff95c191334800.statahead_max=32 101 real 0m1.377s user 0m0.005s sys 0m0.160s 'ls -l' 100 files with statahead: 1 sec statahead total: 149 statahead wrong: 2 agl total: 143 list_total: 29 fname_total: 104 hit_total: 3609 miss_total: 1450 swrong=2 ewrong=2 - open/close 701 (time 1788628391.82 total 10.01 last 70.00) total: 900 open/close in 13.44 seconds: 66.98 ops/second llite.lustre-ffff95c191334800.statahead_max=0 1001 real 0m18.799s user 0m0.038s sys 0m2.792s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff95c191334800.statahead_max=32 llite.lustre-ffff95c191334800.statahead_max=32 1001 real 0m9.460s user 0m0.027s sys 0m1.323s 'ls -l' 1000 files with statahead: 9 sec statahead total: 150 statahead wrong: 2 agl total: 144 list_total: 30 fname_total: 104 hit_total: 4606 miss_total: 1453 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff95c191334800.statahead_stats= statahead total: 150 statahead wrong: 2 agl total: 144 list_total: 30 fname_total: 104 hit_total: 4606 miss_total: 1453 mdc.lustre-MDT0000-mdc-ffff95c191334800.batch_stats= snapshot_time: 1788628459.354217703 secs.nsecs start_time: 1788628367.950684613 secs.nsecs elapsed_time: 91.403533090 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 738 (time 1788628471.83 total 10.01 last 73.75) - open/close 1469 (time 1788628481.85 total 20.03 last 72.96) - open/close 2100 (time 1788628491.85 total 30.03 last 63.07) - open/close 2828 (time 1788628501.86 total 40.04 last 72.70) - open/close 3516 (time 1788628511.88 total 50.06 last 68.68) - open/close 4126 (time 1788628521.90 total 60.08 last 60.89) - open/close 4626 (time 1788628531.91 total 70.09 last 49.94) - open/close 5325 (time 1788628541.94 total 80.11 last 69.73) - open/close 6006 (time 1788628551.97 total 90.15 last 67.87) - open/close 6600 (time 1788628561.97 total 100.15 last 59.39) - open/close 7310 (time 1788628571.98 total 110.16 last 70.95) - open/close 8039 (time 1788628581.98 total 120.16 last 72.84) - open/close 8934 (time 1788628591.99 total 130.17 last 89.42) - open/close 9739 (time 1788628602.00 total 140.18 last 80.44) total: 10000 open/close in 144.11 seconds: 69.39 ops/second llite.lustre-ffff95c191334800.statahead_batch_max=0 llite.lustre-ffff95c191334800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95c191334800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m8.166s user 0m0.294s sys 0m13.613s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff95c191334800.statahead_batch_max=32 llite.lustre-ffff95c191334800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95c191334800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff95c191334800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m56.813s user 0m0.313s sys 0m13.260s batched statahead RPC= hit count=9974 mdc.lustre-MDT0000-mdc-ffff95c191334800.batch_stats= snapshot_time: 1788628813.533215222 secs.nsecs start_time: 1788628684.636640359 secs.nsecs elapsed_time: 128.896574863 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff95c191334800.batch_stats=0 llite.lustre-ffff95c191334800.statahead_max=2048 llite.lustre-ffff95c191334800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff95c191334800.batch_stats=0 total: 100 open/close in 2.21 seconds: 45.16 ops/second llite.lustre-ffff95c191334800.statahead_max=0 101 real 0m2.677s user 0m0.003s sys 0m0.308s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff95c191334800.statahead_max=2048 llite.lustre-ffff95c191334800.statahead_max=2048 101 real 0m0.711s user 0m0.004s sys 0m0.138s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10073 miss_total: 27 swrong=0 ewrong=0 - open/close 714 (time 1788629129.07 total 10.01 last 71.31) total: 900 open/close in 13.27 seconds: 67.81 ops/second llite.lustre-ffff95c191334800.statahead_max=0 1001 real 0m22.292s user 0m0.031s sys 0m2.905s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff95c191334800.statahead_max=2048 llite.lustre-ffff95c191334800.statahead_max=2048 1001 real 0m6.942s user 0m0.031s sys 0m1.349s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11072 miss_total: 28 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff95c191334800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11072 miss_total: 28 mdc.lustre-MDT0000-mdc-ffff95c191334800.batch_stats= snapshot_time: 1788629196.240169278 secs.nsecs start_time: 1788628816.440213235 secs.nsecs elapsed_time: 379.799956043 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 15 15 16: 2 15 30 32: 3 23 53 64: 2 15 69 128: 0 0 69 256: 4 30 100 - open/close 836 (time 1788629209.23 total 10.01 last 83.53) - open/close 1678 (time 1788629219.23 total 20.01 last 84.18) - open/close 2443 (time 1788629229.23 total 30.01 last 76.48) - open/close 3171 (time 1788629239.25 total 40.03 last 72.68) - open/close 3877 (time 1788629249.26 total 50.04 last 70.55) - open/close 4622 (time 1788629259.27 total 60.05 last 74.39) - open/close 5315 (time 1788629269.27 total 70.05 last 69.29) - open/close 6101 (time 1788629279.28 total 80.06 last 78.56) - open/close 6937 (time 1788629289.28 total 90.06 last 83.58) - open/close 7642 (time 1788629299.32 total 100.10 last 70.25) - open/close 8345 (time 1788629309.33 total 110.11 last 70.20) - open/close 9057 (time 1788629319.35 total 120.12 last 71.11) - open/close 9735 (time 1788629329.35 total 130.13 last 67.79) total: 10000 open/close in 133.57 seconds: 74.87 ops/second llite.lustre-ffff95c191334800.statahead_batch_max=0 llite.lustre-ffff95c191334800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95c191334800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m4.291s user 0m0.314s sys 0m13.789s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff95c191334800.statahead_batch_max=256 llite.lustre-ffff95c191334800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95c191334800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff95c191334800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m53.659s user 0m0.291s sys 0m13.880s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff95c191334800.batch_stats= snapshot_time: 1788629473.956476065 secs.nsecs start_time: 1788629405.921230555 secs.nsecs elapsed_time: 68.035245510 secs.nsecs subreqs per batch batches %% cum %% 1: 167 52 52 2: 44 13 65 4: 31 9 75 8: 13 4 79 16: 13 4 83 32: 13 4 87 64: 3 0 88 128: 1 0 89 256: 35 10 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 1 seconds: 0.000000 unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff95c191334800.statahead_batch_max=64 llite.lustre-ffff95c191334800.statahead_max=128