== sanity test 123ad: Verify batching statahead works correctly ========================================================== 00:26:44 (1789360004) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff937288f6d000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff937288f6d000.batch_stats=0 llite.lustre-ffff937288f6d000.statahead_max=32 llite.lustre-ffff937288f6d000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff937288f6d000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff937288f6d000.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 0.65 seconds: 154.82 ops/second llite.lustre-ffff937288f6d000.statahead_max=0 101 real 0m0.619s user 0m0.001s sys 0m0.236s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff937288f6d000.statahead_max=32 llite.lustre-ffff937288f6d000.statahead_max=32 101 real 0m0.434s user 0m0.003s sys 0m0.247s 'ls -l' 100 files with statahead: 1 sec statahead total: 90 statahead wrong: 2 agl total: 85 list_total: 30 fname_total: 60 hit_total: 14359 miss_total: 758 swrong=2 ewrong=2 total: 900 open/close in 1.87 seconds: 481.55 ops/second llite.lustre-ffff937288f6d000.statahead_max=0 1001 real 0m6.258s user 0m0.024s sys 0m2.292s 'ls -l' 1000 files without statahead: 6 sec llite.lustre-ffff937288f6d000.statahead_max=32 llite.lustre-ffff937288f6d000.statahead_max=32 1001 real 0m3.769s user 0m0.019s sys 0m1.779s 'ls -l' 1000 files with statahead: 4 sec statahead total: 91 statahead wrong: 2 agl total: 86 list_total: 31 fname_total: 60 hit_total: 15358 miss_total: 759 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 4 seconds llite.lustre-ffff937288f6d000.statahead_stats= statahead total: 91 statahead wrong: 2 agl total: 86 list_total: 31 fname_total: 60 hit_total: 15358 miss_total: 759 mdc.lustre-MDT0000-mdc-ffff937288f6d000.batch_stats= snapshot_time: 1789360029.191904260 secs.nsecs start_time: 1789360005.684959723 secs.nsecs elapsed_time: 23.506944537 secs.nsecs subreqs per batch batches %% cum %% 1: 3 8 8 2: 1 2 10 4: 2 5 16 8: 1 2 18 16: 24 64 83 32: 6 16 100 mdc.lustre-MDT0001-mdc-ffff937288f6d000.batch_stats= snapshot_time: 1789360029.191956053 secs.nsecs start_time: 1789360005.685044409 secs.nsecs elapsed_time: 23.506911644 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 1 2 4 4: 1 2 7 8: 4 9 17 16: 11 26 43 32: 23 56 100 - open/close 4678 (time 1789360041.33 total 10.00 last 467.75) - open/close 9273 (time 1789360051.34 total 20.00 last 459.43) total: 10000 open/close in 21.64 seconds: 462.14 ops/second llite.lustre-ffff937288f6d000.statahead_batch_max=0 llite.lustre-ffff937288f6d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff937288f6d000.stats=clear mdc.lustre-MDT0001-mdc-ffff937288f6d000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m34.533s user 0m0.175s sys 0m31.552s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff937288f6d000.statahead_batch_max=32 llite.lustre-ffff937288f6d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff937288f6d000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff937288f6d000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff937288f6d000.stats=clear mdc.lustre-MDT0001-mdc-ffff937288f6d000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m35.800s user 0m0.171s sys 0m19.562s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff937288f6d000.batch_stats= snapshot_time: 1789360130.538794564 secs.nsecs start_time: 1789360090.358004380 secs.nsecs elapsed_time: 40.180790184 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff937288f6d000.batch_stats= snapshot_time: 1789360130.538838515 secs.nsecs start_time: 1789360090.358132918 secs.nsecs elapsed_time: 40.180705597 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff937288f6d000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff937288f6d000.batch_stats=0 llite.lustre-ffff937288f6d000.statahead_max=2048 llite.lustre-ffff937288f6d000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff937288f6d000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff937288f6d000.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 0.70 seconds: 141.95 ops/second llite.lustre-ffff937288f6d000.statahead_max=0 101 real 0m0.633s user 0m0.003s sys 0m0.228s 'ls -l' 100 files without statahead: 0 sec llite.lustre-ffff937288f6d000.statahead_max=2048 llite.lustre-ffff937288f6d000.statahead_max=2048 101 real 0m0.353s user 0m0.003s sys 0m0.222s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 2 swrong=0 ewrong=0 total: 900 open/close in 1.92 seconds: 468.50 ops/second llite.lustre-ffff937288f6d000.statahead_max=0 1001 real 0m5.998s user 0m0.019s sys 0m2.186s 'ls -l' 1000 files without statahead: 6 sec llite.lustre-ffff937288f6d000.statahead_max=2048 llite.lustre-ffff937288f6d000.statahead_max=2048 1001 real 0m3.126s user 0m0.016s sys 0m1.410s 'ls -l' 1000 files with statahead: 4 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 5 seconds llite.lustre-ffff937288f6d000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 mdc.lustre-MDT0000-mdc-ffff937288f6d000.batch_stats= snapshot_time: 1789360235.909093525 secs.nsecs start_time: 1789360131.341727452 secs.nsecs elapsed_time: 104.567366073 secs.nsecs subreqs per batch batches %% cum %% 1: 2 28 28 2: 1 14 42 4: 2 28 71 8: 0 0 71 16: 0 0 71 32: 0 0 71 64: 0 0 71 128: 0 0 71 256: 2 28 100 mdc.lustre-MDT0001-mdc-ffff937288f6d000.batch_stats= snapshot_time: 1789360235.909135247 secs.nsecs start_time: 1789360131.341845579 secs.nsecs elapsed_time: 104.567289668 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 33 33 16: 2 22 55 32: 0 0 55 64: 2 22 77 128: 0 0 77 256: 2 22 100 - open/close 4883 (time 1789360247.04 total 10.00 last 488.21) - open/close 9707 (time 1789360257.04 total 20.00 last 482.33) total: 10000 open/close in 20.62 seconds: 485.07 ops/second llite.lustre-ffff937288f6d000.statahead_batch_max=0 llite.lustre-ffff937288f6d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff937288f6d000.stats=clear mdc.lustre-MDT0001-mdc-ffff937288f6d000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m36.720s user 0m0.257s sys 0m20.246s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff937288f6d000.statahead_batch_max=256 llite.lustre-ffff937288f6d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff937288f6d000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff937288f6d000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff937288f6d000.stats=clear mdc.lustre-MDT0001-mdc-ffff937288f6d000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m30.112s user 0m0.208s sys 0m17.056s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff937288f6d000.batch_stats= snapshot_time: 1789360331.208324305 secs.nsecs start_time: 1789360296.902777265 secs.nsecs elapsed_time: 34.305547040 secs.nsecs subreqs per batch batches %% cum %% 1: 145 47 47 2: 43 14 61 4: 40 13 74 8: 12 3 78 16: 5 1 80 32: 21 6 87 64: 4 1 88 128: 1 0 88 256: 34 11 100 mdc.lustre-MDT0001-mdc-ffff937288f6d000.batch_stats= snapshot_time: 1789360331.208392050 secs.nsecs start_time: 1789360296.902868427 secs.nsecs elapsed_time: 34.305523623 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff937288f6d000.statahead_batch_max=64 llite.lustre-ffff937288f6d000.statahead_max=128