== sanity test 123ad: Verify batching statahead works correctly ========================================================== 05:07:43 (1788599263) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff96b4189b0000.batch_stats=0 llite.lustre-ffff96b4189b0000.statahead_max=32 llite.lustre-ffff96b4189b0000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff96b4189b0000.batch_stats=0 total: 100 open/close in 2.36 seconds: 42.44 ops/second llite.lustre-ffff96b4189b0000.statahead_max=0 101 real 0m1.770s user 0m0.005s sys 0m0.310s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff96b4189b0000.statahead_max=32 llite.lustre-ffff96b4189b0000.statahead_max=32 101 real 0m1.332s user 0m0.004s sys 0m0.140s 'ls -l' 100 files with statahead: 1 sec statahead total: 136 statahead wrong: 2 agl total: 130 list_total: 30 fname_total: 94 hit_total: 3647 miss_total: 1497 swrong=2 ewrong=2 - open/close 602 (time 1788599289.85 total 10.01 last 60.14) total: 900 open/close in 14.85 seconds: 60.61 ops/second llite.lustre-ffff96b4189b0000.statahead_max=0 1001 real 0m25.598s user 0m0.031s sys 0m3.255s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff96b4189b0000.statahead_max=32 llite.lustre-ffff96b4189b0000.statahead_max=32 1001 real 0m10.362s user 0m0.029s sys 0m1.351s 'ls -l' 1000 files with statahead: 10 sec statahead total: 137 statahead wrong: 2 agl total: 131 list_total: 31 fname_total: 94 hit_total: 4644 miss_total: 1500 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff96b4189b0000.statahead_stats= statahead total: 137 statahead wrong: 2 agl total: 131 list_total: 31 fname_total: 94 hit_total: 4644 miss_total: 1500 mdc.lustre-MDT0000-mdc-ffff96b4189b0000.batch_stats= snapshot_time: 1788599365.506882738 secs.nsecs start_time: 1788599267.988073477 secs.nsecs elapsed_time: 97.518809261 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 1 2 12 32: 34 87 100 - open/close 743 (time 1788599378.17 total 10.01 last 74.25) - open/close 1364 (time 1788599388.18 total 20.02 last 62.05) - open/close 2056 (time 1788599398.20 total 30.03 last 69.11) - open/close 2864 (time 1788599408.21 total 40.04 last 80.70) - open/close 3641 (time 1788599418.21 total 50.04 last 77.70) - open/close 4375 (time 1788599428.22 total 60.05 last 73.36) - open/close 5192 (time 1788599438.22 total 70.05 last 81.68) - open/close 6058 (time 1788599448.24 total 80.07 last 86.42) - open/close 6922 (time 1788599458.24 total 90.07 last 86.37) - open/close 7764 (time 1788599468.24 total 100.08 last 84.17) - open/close 8649 (time 1788599478.25 total 110.08 last 88.47) - open/close 9488 (time 1788599488.25 total 120.08 last 83.88) total: 10000 open/close in 125.85 seconds: 79.46 ops/second llite.lustre-ffff96b4189b0000.statahead_batch_max=0 llite.lustre-ffff96b4189b0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff96b4189b0000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m54.939s user 0m0.294s sys 0m13.738s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff96b4189b0000.statahead_batch_max=32 llite.lustre-ffff96b4189b0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff96b4189b0000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff96b4189b0000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m38.830s user 0m0.285s sys 0m12.655s batched statahead RPC= hit count=9974 mdc.lustre-MDT0000-mdc-ffff96b4189b0000.batch_stats= snapshot_time: 1788599667.667911301 secs.nsecs start_time: 1788599558.965664326 secs.nsecs elapsed_time: 108.702246975 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff96b4189b0000.batch_stats=0 llite.lustre-ffff96b4189b0000.statahead_max=2048 llite.lustre-ffff96b4189b0000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff96b4189b0000.batch_stats=0 total: 100 open/close in 2.24 seconds: 44.66 ops/second llite.lustre-ffff96b4189b0000.statahead_max=0 101 real 0m1.579s user 0m0.004s sys 0m0.294s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff96b4189b0000.statahead_max=2048 llite.lustre-ffff96b4189b0000.statahead_max=2048 101 real 0m0.810s user 0m0.006s sys 0m0.153s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10073 miss_total: 27 swrong=0 ewrong=0 - open/close 881 (time 1788599946.77 total 10.01 last 88.04) total: 900 open/close in 10.23 seconds: 87.98 ops/second llite.lustre-ffff96b4189b0000.statahead_max=0 1001 real 0m18.340s user 0m0.037s sys 0m2.765s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff96b4189b0000.statahead_max=2048 llite.lustre-ffff96b4189b0000.statahead_max=2048 1001 real 0m5.844s user 0m0.029s sys 0m1.450s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11072 miss_total: 28 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff96b4189b0000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11072 miss_total: 28 mdc.lustre-MDT0000-mdc-ffff96b4189b0000.batch_stats= snapshot_time: 1788600002.865010934 secs.nsecs start_time: 1788599669.806701404 secs.nsecs elapsed_time: 333.058309530 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 2 16 50 64: 1 8 58 128: 1 8 66 256: 4 33 100 - open/close 918 (time 1788600014.97 total 10.02 last 91.66) - open/close 1609 (time 1788600024.98 total 20.03 last 69.03) - open/close 2452 (time 1788600035.00 total 30.04 last 84.17) - open/close 3209 (time 1788600045.02 total 40.06 last 75.54) - open/close 4054 (time 1788600055.02 total 50.07 last 84.44) - open/close 4845 (time 1788600065.03 total 60.08 last 79.04) - open/close 5588 (time 1788600075.04 total 70.08 last 74.24) - open/close 6387 (time 1788600085.04 total 80.09 last 79.85) - open/close 7173 (time 1788600095.05 total 90.10 last 78.56) - open/close 7929 (time 1788600105.05 total 100.10 last 75.57) - open/close 8620 (time 1788600115.06 total 110.11 last 69.05) - open/close 9387 (time 1788600125.06 total 120.11 last 76.69) total: 10000 open/close in 128.56 seconds: 77.78 ops/second llite.lustre-ffff96b4189b0000.statahead_batch_max=0 llite.lustre-ffff96b4189b0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff96b4189b0000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m3.754s user 0m0.320s sys 0m13.499s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff96b4189b0000.statahead_batch_max=256 llite.lustre-ffff96b4189b0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff96b4189b0000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff96b4189b0000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m47.002s user 0m0.259s sys 0m13.419s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff96b4189b0000.batch_stats= snapshot_time: 1788600263.377071601 secs.nsecs start_time: 1788600205.456950698 secs.nsecs elapsed_time: 57.920120903 secs.nsecs subreqs per batch batches %% cum %% 1: 121 46 46 2: 32 12 58 4: 27 10 68 8: 13 4 73 16: 12 4 78 32: 13 4 83 64: 7 2 86 128: 1 0 86 256: 35 13 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff96b4189b0000.statahead_batch_max=64 llite.lustre-ffff96b4189b0000.statahead_max=128