== sanity test 123ad: Verify batching statahead works correctly ========================================================== 12:50:16 (1787849416) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9b5c03dad000.batch_stats=0 llite.lustre-ffff9b5c03dad000.statahead_max=32 llite.lustre-ffff9b5c03dad000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9b5c03dad000.batch_stats=0 total: 100 open/close in 3.09 seconds: 32.40 ops/second llite.lustre-ffff9b5c03dad000.statahead_max=0 101 real 0m2.581s user 0m0.004s sys 0m0.352s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9b5c03dad000.statahead_max=32 llite.lustre-ffff9b5c03dad000.statahead_max=32 101 real 0m0.809s user 0m0.004s sys 0m0.142s 'ls -l' 100 files with statahead: 1 sec statahead total: 121 statahead wrong: 2 agl total: 115 list_total: 30 fname_total: 72 hit_total: 3614 miss_total: 1661 swrong=2 ewrong=2 - open/close 594 (time 1787849445.82 total 10.01 last 59.36) total: 900 open/close in 17.57 seconds: 51.22 ops/second llite.lustre-ffff9b5c03dad000.statahead_max=0 1001 real 0m29.297s user 0m0.035s sys 0m3.289s 'ls -l' 1000 files without statahead: 30 sec llite.lustre-ffff9b5c03dad000.statahead_max=32 llite.lustre-ffff9b5c03dad000.statahead_max=32 1001 real 0m14.243s user 0m0.036s sys 0m1.504s 'ls -l' 1000 files with statahead: 15 sec statahead total: 122 statahead wrong: 2 agl total: 116 list_total: 31 fname_total: 72 hit_total: 4610 miss_total: 1665 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 30 seconds llite.lustre-ffff9b5c03dad000.statahead_stats= statahead total: 122 statahead wrong: 2 agl total: 116 list_total: 31 fname_total: 72 hit_total: 4610 miss_total: 1665 mdc.lustre-MDT0000-mdc-ffff9b5c03dad000.batch_stats= snapshot_time: 1787849545.479663554 secs.nsecs start_time: 1787849421.892615316 secs.nsecs elapsed_time: 123.587048238 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 1 2 12 32: 34 87 100 - open/close 618 (time 1787849559.19 total 10.01 last 61.73) - open/close 1248 (time 1787849569.20 total 20.03 last 62.89) - open/close 1869 (time 1787849579.22 total 30.04 last 62.03) - open/close 2565 (time 1787849589.22 total 40.05 last 69.54) - open/close 3191 (time 1787849599.23 total 50.06 last 62.54) - open/close 4019 (time 1787849609.24 total 60.06 last 82.75) - open/close 4864 (time 1787849619.25 total 70.07 last 84.46) - open/close 5577 (time 1787849629.25 total 80.08 last 71.25) - open/close 6351 (time 1787849639.26 total 90.08 last 77.36) - open/close 7151 (time 1787849649.26 total 100.08 last 80.00) - open/close 7781 (time 1787849659.27 total 110.09 last 62.95) - open/close 8250 (time 1787849669.28 total 120.10 last 46.85) - open/close 8794 (time 1787849679.29 total 130.12 last 54.32) - open/close 9569 (time 1787849689.30 total 140.12 last 77.44) total: 10000 open/close in 146.14 seconds: 68.43 ops/second llite.lustre-ffff9b5c03dad000.statahead_batch_max=0 llite.lustre-ffff9b5c03dad000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b5c03dad000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m4.841s user 0m0.304s sys 0m13.913s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9b5c03dad000.statahead_batch_max=32 llite.lustre-ffff9b5c03dad000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b5c03dad000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9b5c03dad000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m48.810s user 0m0.288s sys 0m12.937s batched statahead RPC= hit count=9974 mdc.lustre-MDT0000-mdc-ffff9b5c03dad000.batch_stats= snapshot_time: 1787849891.653663941 secs.nsecs start_time: 1787849770.065773720 secs.nsecs elapsed_time: 121.587890221 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9b5c03dad000.batch_stats=0 llite.lustre-ffff9b5c03dad000.statahead_max=2048 llite.lustre-ffff9b5c03dad000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9b5c03dad000.batch_stats=0 total: 100 open/close in 3.01 seconds: 33.23 ops/second llite.lustre-ffff9b5c03dad000.statahead_max=0 101 real 0m1.705s user 0m0.003s sys 0m0.295s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9b5c03dad000.statahead_max=2048 llite.lustre-ffff9b5c03dad000.statahead_max=2048 101 real 0m0.952s user 0m0.006s sys 0m0.158s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10073 miss_total: 27 swrong=0 ewrong=0 - open/close 753 (time 1787850211.61 total 10.03 last 75.07) total: 900 open/close in 12.42 seconds: 72.44 ops/second llite.lustre-ffff9b5c03dad000.statahead_max=0 1001 real 0m18.465s user 0m0.031s sys 0m2.660s 'ls -l' 1000 files without statahead: 18 sec llite.lustre-ffff9b5c03dad000.statahead_max=2048 llite.lustre-ffff9b5c03dad000.statahead_max=2048 1001 real 0m6.679s user 0m0.018s sys 0m1.410s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11072 miss_total: 28 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 27 seconds llite.lustre-ffff9b5c03dad000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11072 miss_total: 28 mdc.lustre-MDT0000-mdc-ffff9b5c03dad000.batch_stats= snapshot_time: 1787850279.558285990 secs.nsecs start_time: 1787849894.835291649 secs.nsecs elapsed_time: 384.722994341 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 0 0 54 128: 1 9 63 256: 4 36 100 - open/close 823 (time 1787850291.78 total 10.00 last 82.29) - open/close 1487 (time 1787850301.80 total 20.02 last 66.24) - open/close 2363 (time 1787850311.80 total 30.03 last 87.59) - open/close 3018 (time 1787850321.81 total 40.03 last 65.48) - open/close 3703 (time 1787850331.81 total 50.04 last 68.45) - open/close 4393 (time 1787850341.82 total 60.04 last 68.94) - open/close 4943 (time 1787850351.82 total 70.05 last 54.98) - open/close 5551 (time 1787850361.83 total 80.05 last 60.79) - open/close 6300 (time 1787850371.83 total 90.06 last 74.86) - open/close 6981 (time 1787850381.85 total 100.07 last 68.00) - open/close 7712 (time 1787850391.85 total 110.07 last 73.06) - open/close 8536 (time 1787850401.85 total 120.08 last 82.37) - open/close 9072 (time 1787850411.88 total 130.10 last 53.47) - open/close 9871 (time 1787850421.88 total 140.11 last 79.86) total: 10000 open/close in 141.90 seconds: 70.47 ops/second llite.lustre-ffff9b5c03dad000.statahead_batch_max=0 llite.lustre-ffff9b5c03dad000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b5c03dad000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m8.421s user 0m0.340s sys 0m13.894s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9b5c03dad000.statahead_batch_max=256 llite.lustre-ffff9b5c03dad000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b5c03dad000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9b5c03dad000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m57.719s user 0m0.259s sys 0m13.613s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9b5c03dad000.batch_stats= snapshot_time: 1787850571.509020464 secs.nsecs start_time: 1787850501.683366551 secs.nsecs elapsed_time: 69.825653913 secs.nsecs subreqs per batch batches %% cum %% 1: 85 37 37 2: 27 12 49 4: 24 10 60 8: 21 9 69 16: 13 5 75 32: 17 7 83 64: 2 0 84 128: 0 0 84 256: 36 16 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9b5c03dad000.statahead_batch_max=64 llite.lustre-ffff9b5c03dad000.statahead_max=128