== sanity test 123ad: Verify batching statahead works correctly ========================================================== 21:04:47 (1789088687) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8bced84bb000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8bced84bb000.batch_stats=0 llite.lustre-ffff8bced84bb000.statahead_max=32 llite.lustre-ffff8bced84bb000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8bced84bb000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8bced84bb000.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.31 seconds: 43.32 ops/second llite.lustre-ffff8bced84bb000.statahead_max=0 101 real 0m1.815s user 0m0.003s sys 0m0.321s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8bced84bb000.statahead_max=32 llite.lustre-ffff8bced84bb000.statahead_max=32 101 real 0m1.000s user 0m0.005s sys 0m0.183s 'ls -l' 100 files with statahead: 1 sec statahead total: 125 statahead wrong: 2 agl total: 120 list_total: 30 fname_total: 95 hit_total: 4624 miss_total: 1263 swrong=2 ewrong=2 - open/close 878 (time 1789088713.08 total 10.00 last 87.76) total: 900 open/close in 10.27 seconds: 87.65 ops/second llite.lustre-ffff8bced84bb000.statahead_max=0 1001 real 0m21.583s user 0m0.036s sys 0m3.231s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff8bced84bb000.statahead_max=32 llite.lustre-ffff8bced84bb000.statahead_max=32 1001 real 0m8.542s user 0m0.030s sys 0m1.502s 'ls -l' 1000 files with statahead: 9 sec statahead total: 126 statahead wrong: 2 agl total: 121 list_total: 31 fname_total: 95 hit_total: 5621 miss_total: 1266 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff8bced84bb000.statahead_stats= statahead total: 126 statahead wrong: 2 agl total: 121 list_total: 31 fname_total: 95 hit_total: 5621 miss_total: 1266 mdc.lustre-MDT0000-mdc-ffff8bced84bb000.batch_stats= snapshot_time: 1789088775.936913405 secs.nsecs start_time: 1789088691.939764814 secs.nsecs elapsed_time: 83.997148591 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 2 2 4: 3 7 10 8: 2 5 15 16: 18 46 61 32: 15 38 100 mdc.lustre-MDT0001-mdc-ffff8bced84bb000.batch_stats= snapshot_time: 1789088775.936972215 secs.nsecs start_time: 1789088691.939848686 secs.nsecs elapsed_time: 83.997123529 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 2 5 7 8: 4 10 17 16: 20 50 67 32: 13 32 100 - open/close 902 (time 1789088788.89 total 10.01 last 90.10) - open/close 1946 (time 1789088798.90 total 20.02 last 104.32) - open/close 2872 (time 1789088808.91 total 30.03 last 92.52) - open/close 3678 (time 1789088818.92 total 40.04 last 80.49) - open/close 4367 (time 1789088828.93 total 50.04 last 68.88) - open/close 5192 (time 1789088838.94 total 60.06 last 82.37) - open/close 6009 (time 1789088848.94 total 70.06 last 81.69) - open/close 6740 (time 1789088858.95 total 80.07 last 73.07) - open/close 7413 (time 1789088868.95 total 90.07 last 67.29) - open/close 8366 (time 1789088878.95 total 100.07 last 95.27) - open/close 9251 (time 1789088888.96 total 110.08 last 88.43) total: 10000 open/close in 118.55 seconds: 84.35 ops/second llite.lustre-ffff8bced84bb000.statahead_batch_max=0 llite.lustre-ffff8bced84bb000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8bced84bb000.stats=clear mdc.lustre-MDT0001-mdc-ffff8bced84bb000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m4.665s user 0m0.350s sys 0m16.122s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8bced84bb000.statahead_batch_max=32 llite.lustre-ffff8bced84bb000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8bced84bb000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8bced84bb000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8bced84bb000.stats=clear mdc.lustre-MDT0001-mdc-ffff8bced84bb000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m42.099s user 0m0.309s sys 0m15.330s batched statahead RPC= hit count=9982 mdc.lustre-MDT0000-mdc-ffff8bced84bb000.batch_stats= snapshot_time: 1789089087.743637637 secs.nsecs start_time: 1789088971.666960019 secs.nsecs elapsed_time: 116.076677618 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff8bced84bb000.batch_stats= snapshot_time: 1789089087.743697606 secs.nsecs start_time: 1789088971.667044707 secs.nsecs elapsed_time: 116.076652899 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8bced84bb000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8bced84bb000.batch_stats=0 llite.lustre-ffff8bced84bb000.statahead_max=2048 llite.lustre-ffff8bced84bb000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8bced84bb000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8bced84bb000.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 2.19 seconds: 45.72 ops/second llite.lustre-ffff8bced84bb000.statahead_max=0 101 real 0m1.937s user 0m0.007s sys 0m0.321s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8bced84bb000.statahead_max=2048 llite.lustre-ffff8bced84bb000.statahead_max=2048 101 real 0m1.047s user 0m0.004s sys 0m0.211s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10081 miss_total: 19 swrong=0 ewrong=0 total: 900 open/close in 8.59 seconds: 104.83 ops/second llite.lustre-ffff8bced84bb000.statahead_max=0 1001 real 0m20.495s user 0m0.032s sys 0m3.157s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff8bced84bb000.statahead_max=2048 llite.lustre-ffff8bced84bb000.statahead_max=2048 1001 real 0m6.813s user 0m0.032s sys 0m1.719s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 17 seconds llite.lustre-ffff8bced84bb000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 mdc.lustre-MDT0000-mdc-ffff8bced84bb000.batch_stats= snapshot_time: 1789089411.254313666 secs.nsecs start_time: 1789089089.966323329 secs.nsecs elapsed_time: 321.287990337 secs.nsecs subreqs per batch batches %% cum %% 1: 1 14 14 2: 1 14 28 4: 2 28 57 8: 1 14 71 16: 0 0 71 32: 0 0 71 64: 0 0 71 128: 0 0 71 256: 2 28 100 mdc.lustre-MDT0001-mdc-ffff8bced84bb000.batch_stats= snapshot_time: 1789089411.254382221 secs.nsecs start_time: 1789089089.966470332 secs.nsecs elapsed_time: 321.287911889 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 11 11 8: 3 33 44 16: 1 11 55 32: 1 11 66 64: 0 0 66 128: 1 11 77 256: 2 22 100 - open/close 890 (time 1789089423.59 total 10.01 last 88.93) - open/close 1847 (time 1789089433.59 total 20.01 last 95.66) - open/close 2903 (time 1789089443.60 total 30.02 last 105.56) - open/close 3875 (time 1789089453.60 total 40.02 last 97.17) - open/close 4911 (time 1789089463.61 total 50.02 last 103.54) - open/close 5797 (time 1789089473.61 total 60.03 last 88.57) - open/close 6660 (time 1789089483.63 total 70.04 last 86.17) - open/close 7413 (time 1789089493.64 total 80.05 last 75.20) - open/close 8439 (time 1789089503.64 total 90.06 last 102.56) - open/close 9353 (time 1789089513.64 total 100.06 last 91.37) total: 10000 open/close in 107.38 seconds: 93.13 ops/second llite.lustre-ffff8bced84bb000.statahead_batch_max=0 llite.lustre-ffff8bced84bb000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8bced84bb000.stats=clear mdc.lustre-MDT0001-mdc-ffff8bced84bb000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m9.394s user 0m0.342s sys 0m16.938s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8bced84bb000.statahead_batch_max=256 llite.lustre-ffff8bced84bb000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8bced84bb000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8bced84bb000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8bced84bb000.stats=clear mdc.lustre-MDT0001-mdc-ffff8bced84bb000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.265s user 0m0.320s sys 0m16.117s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8bced84bb000.batch_stats= snapshot_time: 1789089672.091143217 secs.nsecs start_time: 1789089602.375260836 secs.nsecs elapsed_time: 69.715882381 secs.nsecs subreqs per batch batches %% cum %% 1: 173 51 51 2: 45 13 64 4: 27 7 72 8: 21 6 78 16: 14 4 82 32: 22 6 89 64: 3 0 89 128: 0 0 89 256: 34 10 100 mdc.lustre-MDT0001-mdc-ffff8bced84bb000.batch_stats= snapshot_time: 1789089672.091282025 secs.nsecs start_time: 1789089602.375331221 secs.nsecs elapsed_time: 69.715950804 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8bced84bb000.statahead_batch_max=64 llite.lustre-ffff8bced84bb000.statahead_max=128