== sanity test 123ad: Verify batching statahead works correctly ========================================================== 10:20:00 (1783520400) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff98c485bd2000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff98c485bd2000.batch_stats=0 llite.lustre-ffff98c485bd2000.statahead_max=32 llite.lustre-ffff98c485bd2000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff98c485bd2000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff98c485bd2000.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.70 seconds: 58.89 ops/second llite.lustre-ffff98c485bd2000.statahead_max=0 101 real 0m1.547s user 0m0.007s sys 0m0.330s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff98c485bd2000.statahead_max=32 llite.lustre-ffff98c485bd2000.statahead_max=32 101 real 0m0.704s user 0m0.010s sys 0m0.218s 'ls -l' 100 files with statahead: 1 sec statahead total: 66 statahead wrong: 2 agl total: 60 list_total: 28 fname_total: 38 hit_total: 14415 miss_total: 1253 swrong=2 ewrong=2 total: 900 open/close in 6.42 seconds: 140.24 ops/second llite.lustre-ffff98c485bd2000.statahead_max=0 1001 real 0m12.899s user 0m0.042s sys 0m2.885s 'ls -l' 1000 files without statahead: 13 sec llite.lustre-ffff98c485bd2000.statahead_max=32 llite.lustre-ffff98c485bd2000.statahead_max=32 1001 real 0m4.792s user 0m0.024s sys 0m1.701s 'ls -l' 1000 files with statahead: 5 sec statahead total: 67 statahead wrong: 2 agl total: 61 list_total: 29 fname_total: 38 hit_total: 15414 miss_total: 1254 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 11 seconds llite.lustre-ffff98c485bd2000.statahead_stats= statahead total: 67 statahead wrong: 2 agl total: 61 list_total: 29 fname_total: 38 hit_total: 15414 miss_total: 1254 mdc.lustre-MDT0000-mdc-ffff98c485bd2000.batch_stats= snapshot_time: 1783520450.378736684 secs.nsecs start_time: 1783520403.054400774 secs.nsecs elapsed_time: 47.324335910 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 4 10 13 4: 1 2 16 8: 2 5 21 16: 24 64 86 32: 5 13 100 mdc.lustre-MDT0001-mdc-ffff98c485bd2000.batch_stats= snapshot_time: 1783520450.378888772 secs.nsecs start_time: 1783520403.054470724 secs.nsecs elapsed_time: 47.324418048 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 2 5 5 4: 1 2 7 8: 3 7 15 16: 12 30 45 32: 22 55 100 - open/close 1334 (time 1783520461.65 total 10.01 last 133.33) - open/close 2717 (time 1783520471.65 total 20.01 last 138.30) - open/close 4039 (time 1783520481.66 total 30.01 last 132.16) - open/close 5351 (time 1783520491.66 total 40.01 last 131.16) - open/close 6794 (time 1783520501.67 total 50.02 last 144.16) - open/close 8584 (time 1783520511.67 total 60.03 last 178.92) total: 10000 open/close in 66.07 seconds: 151.36 ops/second llite.lustre-ffff98c485bd2000.statahead_batch_max=0 llite.lustre-ffff98c485bd2000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98c485bd2000.stats=clear mdc.lustre-MDT0001-mdc-ffff98c485bd2000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m37.124s user 0m0.273s sys 0m30.781s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff98c485bd2000.statahead_batch_max=32 llite.lustre-ffff98c485bd2000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98c485bd2000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff98c485bd2000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff98c485bd2000.stats=clear mdc.lustre-MDT0001-mdc-ffff98c485bd2000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m42.847s user 0m0.209s sys 0m20.193s batched statahead RPC= hit count=9996 mdc.lustre-MDT0000-mdc-ffff98c485bd2000.batch_stats= snapshot_time: 1783520608.484476958 secs.nsecs start_time: 1783520558.560786033 secs.nsecs elapsed_time: 49.923690925 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff98c485bd2000.batch_stats= snapshot_time: 1783520608.484533168 secs.nsecs start_time: 1783520558.560852200 secs.nsecs elapsed_time: 49.923680968 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff98c485bd2000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff98c485bd2000.batch_stats=0 llite.lustre-ffff98c485bd2000.statahead_max=2048 llite.lustre-ffff98c485bd2000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff98c485bd2000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff98c485bd2000.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 0.95 seconds: 104.79 ops/second llite.lustre-ffff98c485bd2000.statahead_max=0 101 real 0m0.912s user 0m0.003s sys 0m0.297s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff98c485bd2000.statahead_max=2048 llite.lustre-ffff98c485bd2000.statahead_max=2048 101 real 0m0.406s user 0m0.005s sys 0m0.238s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10095 miss_total: 5 swrong=0 ewrong=0 total: 900 open/close in 2.96 seconds: 303.75 ops/second llite.lustre-ffff98c485bd2000.statahead_max=0 1001 real 0m8.397s user 0m0.041s sys 0m2.775s 'ls -l' 1000 files without statahead: 8 sec llite.lustre-ffff98c485bd2000.statahead_max=2048 llite.lustre-ffff98c485bd2000.statahead_max=2048 1001 real 0m3.465s user 0m0.022s sys 0m1.562s 'ls -l' 1000 files with statahead: 4 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11094 miss_total: 6 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 8 seconds llite.lustre-ffff98c485bd2000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11094 miss_total: 6 mdc.lustre-MDT0000-mdc-ffff98c485bd2000.batch_stats= snapshot_time: 1783520762.568465905 secs.nsecs start_time: 1783520609.450054162 secs.nsecs elapsed_time: 153.118411743 secs.nsecs subreqs per batch batches %% cum %% 1: 1 16 16 2: 1 16 33 4: 2 33 66 8: 0 0 66 16: 0 0 66 32: 0 0 66 64: 0 0 66 128: 0 0 66 256: 2 33 100 mdc.lustre-MDT0001-mdc-ffff98c485bd2000.batch_stats= snapshot_time: 1783520762.568529364 secs.nsecs start_time: 1783520609.450218977 secs.nsecs elapsed_time: 153.118310387 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 12 12 8: 3 37 50 16: 0 0 50 32: 0 0 50 64: 1 12 62 128: 1 12 75 256: 2 25 100 - open/close 2063 (time 1783520773.58 total 10.00 last 206.20) - open/close 4215 (time 1783520783.58 total 20.01 last 215.19) - open/close 6602 (time 1783520793.58 total 30.01 last 238.62) - open/close 8776 (time 1783520803.59 total 40.01 last 217.33) total: 10000 open/close in 44.81 seconds: 223.18 ops/second llite.lustre-ffff98c485bd2000.statahead_batch_max=0 llite.lustre-ffff98c485bd2000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98c485bd2000.stats=clear mdc.lustre-MDT0001-mdc-ffff98c485bd2000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m32.467s user 0m0.222s sys 0m26.562s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff98c485bd2000.statahead_batch_max=256 llite.lustre-ffff98c485bd2000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98c485bd2000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff98c485bd2000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff98c485bd2000.stats=clear mdc.lustre-MDT0001-mdc-ffff98c485bd2000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m28.654s user 0m0.204s sys 0m20.513s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff98c485bd2000.batch_stats= snapshot_time: 1783520880.149841129 secs.nsecs start_time: 1783520844.605759815 secs.nsecs elapsed_time: 35.544081314 secs.nsecs subreqs per batch batches %% cum %% 1: 316 65 65 2: 61 12 78 4: 34 7 85 8: 9 1 87 16: 2 0 87 32: 22 4 92 64: 4 0 92 128: 0 0 92 256: 34 7 100 mdc.lustre-MDT0001-mdc-ffff98c485bd2000.batch_stats= snapshot_time: 1783520880.149896861 secs.nsecs start_time: 1783520844.605818387 secs.nsecs elapsed_time: 35.544078474 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff98c485bd2000.statahead_batch_max=64 llite.lustre-ffff98c485bd2000.statahead_max=128