== sanity test 123ad: Verify batching statahead works correctly ========================================================== 00:39:30 (1781239170) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9960c73d0000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9960c73d0000.batch_stats=0 llite.lustre-ffff9960c73d0000.statahead_max=32 llite.lustre-ffff9960c73d0000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9960c73d0000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9960c73d0000.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.60 seconds: 38.41 ops/second llite.lustre-ffff9960c73d0000.statahead_max=0 101 real 0m2.314s user 0m0.006s sys 0m0.383s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9960c73d0000.statahead_max=32 llite.lustre-ffff9960c73d0000.statahead_max=32 101 real 0m0.940s user 0m0.005s sys 0m0.160s 'ls -l' 100 files with statahead: 0 sec statahead total: 129 statahead wrong: 2 agl total: 123 list_total: 30 fname_total: 91 hit_total: 4423 miss_total: 1282 swrong=2 ewrong=2 - open/close 621 (time 1781239196.81 total 10.00 last 62.09) total: 900 open/close in 14.60 seconds: 61.66 ops/second llite.lustre-ffff9960c73d0000.statahead_max=0 1001 real 0m28.506s user 0m0.030s sys 0m3.313s 'ls -l' 1000 files without statahead: 28 sec llite.lustre-ffff9960c73d0000.statahead_max=32 llite.lustre-ffff9960c73d0000.statahead_max=32 1001 real 0m9.352s user 0m0.032s sys 0m1.661s 'ls -l' 1000 files with statahead: 9 sec statahead total: 130 statahead wrong: 2 agl total: 124 list_total: 31 fname_total: 91 hit_total: 5422 miss_total: 1283 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff9960c73d0000.statahead_stats= statahead total: 130 statahead wrong: 2 agl total: 124 list_total: 31 fname_total: 91 hit_total: 5422 miss_total: 1283 mdc.lustre-MDT0000-mdc-ffff9960c73d0000.batch_stats= snapshot_time: 1781239274.579378040 secs.nsecs start_time: 1781239174.967457569 secs.nsecs elapsed_time: 99.611920471 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 4 10 10 8: 1 2 12 16: 18 46 58 32: 16 41 100 mdc.lustre-MDT0001-mdc-ffff9960c73d0000.batch_stats= snapshot_time: 1781239274.579454243 secs.nsecs start_time: 1781239174.967672206 secs.nsecs elapsed_time: 99.611782037 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 1 2 2 4: 1 2 5 8: 4 10 15 16: 18 46 61 32: 15 38 100 - open/close 699 (time 1781239287.03 total 10.00 last 69.88) - open/close 1421 (time 1781239297.03 total 20.01 last 72.17) - open/close 2098 (time 1781239307.04 total 30.02 last 67.63) - open/close 2841 (time 1781239317.05 total 40.02 last 74.25) - open/close 3571 (time 1781239327.05 total 50.03 last 72.97) - open/close 4282 (time 1781239337.05 total 60.03 last 71.08) - open/close 5022 (time 1781239347.05 total 70.03 last 74.00) - open/close 5719 (time 1781239357.07 total 80.05 last 69.60) - open/close 6415 (time 1781239367.07 total 90.05 last 69.58) - open/close 7055 (time 1781239377.07 total 100.05 last 63.98) - open/close 7695 (time 1781239387.09 total 110.06 last 63.93) - open/close 8402 (time 1781239397.09 total 120.06 last 70.69) - open/close 9167 (time 1781239407.10 total 130.08 last 76.40) - open/close 9879 (time 1781239417.10 total 140.08 last 71.16) total: 10000 open/close in 142.02 seconds: 70.41 ops/second llite.lustre-ffff9960c73d0000.statahead_batch_max=0 llite.lustre-ffff9960c73d0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9960c73d0000.stats=clear mdc.lustre-MDT0001-mdc-ffff9960c73d0000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m17.580s user 0m0.391s sys 0m17.826s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9960c73d0000.statahead_batch_max=32 llite.lustre-ffff9960c73d0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9960c73d0000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9960c73d0000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9960c73d0000.stats=clear mdc.lustre-MDT0001-mdc-ffff9960c73d0000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m52.337s user 0m0.368s sys 0m16.855s batched statahead RPC= hit count=9980 mdc.lustre-MDT0000-mdc-ffff9960c73d0000.batch_stats= snapshot_time: 1781239635.358693590 secs.nsecs start_time: 1781239508.323438972 secs.nsecs elapsed_time: 127.035254618 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff9960c73d0000.batch_stats= snapshot_time: 1781239635.358766831 secs.nsecs start_time: 1781239508.323619738 secs.nsecs elapsed_time: 127.035147093 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9960c73d0000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9960c73d0000.batch_stats=0 llite.lustre-ffff9960c73d0000.statahead_max=2048 llite.lustre-ffff9960c73d0000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9960c73d0000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9960c73d0000.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 2.68 seconds: 37.32 ops/second llite.lustre-ffff9960c73d0000.statahead_max=0 101 real 0m3.385s user 0m0.007s sys 0m0.360s 'ls -l' 100 files without statahead: 4 sec llite.lustre-ffff9960c73d0000.statahead_max=2048 llite.lustre-ffff9960c73d0000.statahead_max=2048 101 real 0m1.145s user 0m0.007s sys 0m0.233s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10079 miss_total: 21 swrong=0 ewrong=0 - open/close 605 (time 1781239990.82 total 10.01 last 60.46) total: 900 open/close in 14.68 seconds: 61.32 ops/second llite.lustre-ffff9960c73d0000.statahead_max=0 1001 real 0m28.345s user 0m0.035s sys 0m3.339s 'ls -l' 1000 files without statahead: 28 sec llite.lustre-ffff9960c73d0000.statahead_max=2048 llite.lustre-ffff9960c73d0000.statahead_max=2048 1001 real 0m8.093s user 0m0.030s sys 0m1.640s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff9960c73d0000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 mdc.lustre-MDT0000-mdc-ffff9960c73d0000.batch_stats= snapshot_time: 1781240069.204173160 secs.nsecs start_time: 1781239638.491114206 secs.nsecs elapsed_time: 430.713058954 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 2 33 33 8: 0 0 33 16: 2 33 66 32: 0 0 66 64: 0 0 66 128: 0 0 66 256: 2 33 100 mdc.lustre-MDT0001-mdc-ffff9960c73d0000.batch_stats= snapshot_time: 1781240069.204248386 secs.nsecs start_time: 1781239638.491419243 secs.nsecs elapsed_time: 430.712829143 secs.nsecs subreqs per batch batches % cum % 1: 1 10 10 2: 0 0 10 4: 0 0 10 8: 4 40 50 16: 1 10 60 32: 1 10 70 64: 0 0 70 128: 1 10 80 256: 2 20 100 - open/close 785 (time 1781240082.11 total 10.01 last 78.45) - open/close 1504 (time 1781240092.12 total 20.01 last 71.84) - open/close 2319 (time 1781240102.13 total 30.03 last 81.38) - open/close 3016 (time 1781240112.13 total 40.03 last 69.69) - open/close 3766 (time 1781240122.15 total 50.05 last 74.85) - open/close 4367 (time 1781240132.16 total 60.06 last 60.05) - open/close 5140 (time 1781240142.17 total 70.07 last 77.25) - open/close 5919 (time 1781240152.17 total 80.07 last 77.88) - open/close 6696 (time 1781240162.19 total 90.08 last 77.60) - open/close 7464 (time 1781240172.19 total 100.09 last 76.76) - open/close 8332 (time 1781240182.19 total 110.09 last 86.80) - open/close 9043 (time 1781240192.19 total 120.09 last 71.08) - open/close 9766 (time 1781240202.20 total 130.10 last 72.24) total: 10000 open/close in 132.86 seconds: 75.27 ops/second llite.lustre-ffff9960c73d0000.statahead_batch_max=0 llite.lustre-ffff9960c73d0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9960c73d0000.stats=clear mdc.lustre-MDT0001-mdc-ffff9960c73d0000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m8.217s user 0m0.346s sys 0m18.891s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9960c73d0000.statahead_batch_max=256 llite.lustre-ffff9960c73d0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9960c73d0000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9960c73d0000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9960c73d0000.stats=clear mdc.lustre-MDT0001-mdc-ffff9960c73d0000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m5.048s user 0m0.343s sys 0m16.820s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9960c73d0000.batch_stats= snapshot_time: 1781240365.946370696 secs.nsecs start_time: 1781240284.881339381 secs.nsecs elapsed_time: 81.065031315 secs.nsecs subreqs per batch batches % cum % 1: 212 53 53 2: 65 16 70 4: 30 7 77 8: 16 4 81 16: 14 3 85 32: 16 4 89 64: 7 1 91 128: 0 0 91 256: 34 8 100 mdc.lustre-MDT0001-mdc-ffff9960c73d0000.batch_stats= snapshot_time: 1781240365.946449387 secs.nsecs start_time: 1781240284.881537581 secs.nsecs elapsed_time: 81.064911806 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9960c73d0000.statahead_batch_max=64 llite.lustre-ffff9960c73d0000.statahead_max=128