== sanity test 123ad: Verify batching statahead works correctly ========================================================== 10:46:44 (1783954004) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff983886de1000.batch_stats=0 llite.lustre-ffff983886de1000.statahead_max=32 llite.lustre-ffff983886de1000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff983886de1000.batch_stats=0 total: 100 open/close in 3.37 seconds: 29.67 ops/second llite.lustre-ffff983886de1000.statahead_max=0 101 real 0m2.696s user 0m0.004s sys 0m0.336s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff983886de1000.statahead_max=32 llite.lustre-ffff983886de1000.statahead_max=32 101 real 0m1.433s user 0m0.006s sys 0m0.192s 'ls -l' 100 files with statahead: 1 sec statahead total: 134 statahead wrong: 2 agl total: 128 list_total: 31 fname_total: 86 hit_total: 4223 miss_total: 1296 swrong=2 ewrong=2 - open/close 554 (time 1783954034.85 total 10.00 last 55.38) total: 900 open/close in 16.70 seconds: 53.89 ops/second llite.lustre-ffff983886de1000.statahead_max=0 1001 real 0m29.704s user 0m0.041s sys 0m3.714s 'ls -l' 1000 files without statahead: 30 sec llite.lustre-ffff983886de1000.statahead_max=32 llite.lustre-ffff983886de1000.statahead_max=32 1001 real 0m13.876s user 0m0.037s sys 0m1.740s 'ls -l' 1000 files with statahead: 14 sec statahead total: 135 statahead wrong: 2 agl total: 129 list_total: 32 fname_total: 86 hit_total: 5219 miss_total: 1300 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff983886de1000.statahead_stats= statahead total: 135 statahead wrong: 2 agl total: 129 list_total: 32 fname_total: 86 hit_total: 5219 miss_total: 1300 mdc.lustre-MDT0000-mdc-ffff983886de1000.batch_stats= snapshot_time: 1783954130.067737612 secs.nsecs start_time: 1783954009.541228789 secs.nsecs elapsed_time: 120.526508823 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 623 (time 1783954143.12 total 10.01 last 62.23) - open/close 1196 (time 1783954153.13 total 20.02 last 57.25) - open/close 1743 (time 1783954163.15 total 30.03 last 54.62) - open/close 2326 (time 1783954173.15 total 40.04 last 58.29) - open/close 2882 (time 1783954183.16 total 50.05 last 55.52) - open/close 3415 (time 1783954193.22 total 60.10 last 53.01) - open/close 3956 (time 1783954203.24 total 70.12 last 54.00) - open/close 4501 (time 1783954213.24 total 80.13 last 54.48) - open/close 5069 (time 1783954223.26 total 90.14 last 56.70) - open/close 5625 (time 1783954233.28 total 100.17 last 55.49) - open/close 6078 (time 1783954243.28 total 110.17 last 45.28) - open/close 6579 (time 1783954253.29 total 120.17 last 50.08) - open/close 7043 (time 1783954263.29 total 130.18 last 46.37) - open/close 7586 (time 1783954273.30 total 140.19 last 54.24) - open/close 8171 (time 1783954283.32 total 150.20 last 58.43) - open/close 8741 (time 1783954293.32 total 160.21 last 56.95) - open/close 9326 (time 1783954303.33 total 170.21 last 58.48) - open/close 9917 (time 1783954313.34 total 180.22 last 59.04) total: 10000 open/close in 181.63 seconds: 55.06 ops/second llite.lustre-ffff983886de1000.statahead_batch_max=0 llite.lustre-ffff983886de1000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff983886de1000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m20.943s user 0m0.355s sys 0m16.468s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff983886de1000.statahead_batch_max=32 llite.lustre-ffff983886de1000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff983886de1000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff983886de1000.stats=clear ls -l files in d123ad.sanity: 10001 real 2m7.757s user 0m0.327s sys 0m15.912s batched statahead RPC= hit count=9989 mdc.lustre-MDT0000-mdc-ffff983886de1000.batch_stats= snapshot_time: 1783954551.266683686 secs.nsecs start_time: 1783954407.402201793 secs.nsecs elapsed_time: 143.864481893 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff983886de1000.batch_stats=0 llite.lustre-ffff983886de1000.statahead_max=2048 llite.lustre-ffff983886de1000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff983886de1000.batch_stats=0 total: 100 open/close in 3.22 seconds: 31.01 ops/second llite.lustre-ffff983886de1000.statahead_max=0 101 real 0m3.518s user 0m0.005s sys 0m0.380s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff983886de1000.statahead_max=2048 llite.lustre-ffff983886de1000.statahead_max=2048 101 real 0m0.900s user 0m0.003s sys 0m0.163s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10088 miss_total: 12 swrong=0 ewrong=0 - open/close 580 (time 1783954928.25 total 10.01 last 57.94) total: 900 open/close in 15.25 seconds: 59.00 ops/second llite.lustre-ffff983886de1000.statahead_max=0 1001 real 0m25.027s user 0m0.025s sys 0m3.484s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff983886de1000.statahead_max=2048 llite.lustre-ffff983886de1000.statahead_max=2048 1001 real 0m7.629s user 0m0.034s sys 0m1.440s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11087 miss_total: 13 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff983886de1000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11087 miss_total: 13 mdc.lustre-MDT0000-mdc-ffff983886de1000.batch_stats= snapshot_time: 1783955004.178910966 secs.nsecs start_time: 1783954554.032926924 secs.nsecs elapsed_time: 450.145984042 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 3 25 58 64: 1 8 66 128: 0 0 66 256: 4 33 100 - open/close 596 (time 1783955016.78 total 10.01 last 59.53) - open/close 1162 (time 1783955026.80 total 20.03 last 56.51) - open/close 1742 (time 1783955036.81 total 30.04 last 57.94) - open/close 2307 (time 1783955046.82 total 40.05 last 56.42) - open/close 2848 (time 1783955056.84 total 50.07 last 54.00) - open/close 3360 (time 1783955066.85 total 60.08 last 51.17) - open/close 3892 (time 1783955076.86 total 70.09 last 53.13) - open/close 4469 (time 1783955086.86 total 80.09 last 57.68) - open/close 5013 (time 1783955096.87 total 90.10 last 54.38) - open/close 5598 (time 1783955106.87 total 100.10 last 58.48) - open/close 6131 (time 1783955116.89 total 110.12 last 53.21) - open/close 6699 (time 1783955126.89 total 120.12 last 56.77) - open/close 7334 (time 1783955136.90 total 130.13 last 63.45) - open/close 7906 (time 1783955146.91 total 140.14 last 57.13) - open/close 8415 (time 1783955156.92 total 150.15 last 50.88) - open/close 8959 (time 1783955166.92 total 160.15 last 54.38) - open/close 9483 (time 1783955176.93 total 170.16 last 52.36) total: 10000 open/close in 179.48 seconds: 55.72 ops/second llite.lustre-ffff983886de1000.statahead_batch_max=0 llite.lustre-ffff983886de1000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff983886de1000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m13.287s user 0m0.353s sys 0m16.887s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff983886de1000.statahead_batch_max=256 llite.lustre-ffff983886de1000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff983886de1000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff983886de1000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m3.159s user 0m0.329s sys 0m16.835s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff983886de1000.batch_stats= snapshot_time: 1783955350.214986443 secs.nsecs start_time: 1783955270.905189949 secs.nsecs elapsed_time: 79.309796494 secs.nsecs subreqs per batch batches %% cum %% 1: 183 50 50 2: 52 14 64 4: 38 10 75 8: 20 5 80 16: 16 4 85 32: 15 4 89 64: 2 0 90 128: 2 0 90 256: 34 9 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff983886de1000.statahead_batch_max=64 llite.lustre-ffff983886de1000.statahead_max=128