== sanity test 123ad: Verify batching statahead works correctly ========================================================== 10:18:26 (1789136306) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8ec0ca571800.batch_stats=0 llite.lustre-ffff8ec0ca571800.statahead_max=32 llite.lustre-ffff8ec0ca571800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8ec0ca571800.batch_stats=0 total: 100 open/close in 2.74 seconds: 36.56 ops/second llite.lustre-ffff8ec0ca571800.statahead_max=0 101 real 0m2.284s user 0m0.001s sys 0m0.303s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8ec0ca571800.statahead_max=32 llite.lustre-ffff8ec0ca571800.statahead_max=32 101 real 0m1.229s user 0m0.004s sys 0m0.165s 'ls -l' 100 files with statahead: 1 sec statahead total: 129 statahead wrong: 2 agl total: 123 list_total: 29 fname_total: 98 hit_total: 4499 miss_total: 1280 swrong=2 ewrong=2 - open/close 608 (time 1789136335.25 total 10.01 last 60.76) total: 900 open/close in 15.04 seconds: 59.85 ops/second llite.lustre-ffff8ec0ca571800.statahead_max=0 1001 real 0m24.837s user 0m0.037s sys 0m2.931s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff8ec0ca571800.statahead_max=32 llite.lustre-ffff8ec0ca571800.statahead_max=32 1001 real 0m10.129s user 0m0.028s sys 0m1.339s 'ls -l' 1000 files with statahead: 11 sec statahead total: 130 statahead wrong: 2 agl total: 124 list_total: 30 fname_total: 98 hit_total: 5494 miss_total: 1285 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff8ec0ca571800.statahead_stats= statahead total: 130 statahead wrong: 2 agl total: 124 list_total: 30 fname_total: 98 hit_total: 5494 miss_total: 1285 mdc.lustre-MDT0000-mdc-ffff8ec0ca571800.batch_stats= snapshot_time: 1789136408.811775450 secs.nsecs start_time: 1789136311.527070013 secs.nsecs elapsed_time: 97.284705437 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 2 5 7 16: 3 7 15 32: 34 85 100 - open/close 626 (time 1789136421.80 total 10.00 last 62.57) - open/close 1317 (time 1789136431.80 total 20.00 last 69.10) - open/close 1951 (time 1789136441.80 total 30.01 last 63.39) - open/close 2531 (time 1789136451.81 total 40.01 last 57.95) - open/close 3306 (time 1789136461.81 total 50.02 last 77.47) - open/close 3980 (time 1789136471.83 total 60.04 last 67.28) - open/close 4624 (time 1789136481.84 total 70.04 last 64.37) - open/close 5210 (time 1789136491.86 total 80.06 last 58.48) - open/close 5753 (time 1789136501.87 total 90.07 last 54.26) - open/close 6203 (time 1789136511.88 total 100.09 last 44.92) - open/close 6716 (time 1789136521.89 total 110.09 last 51.26) - open/close 7251 (time 1789136531.90 total 120.10 last 53.46) - open/close 8111 (time 1789136541.91 total 130.11 last 85.92) - open/close 8860 (time 1789136551.91 total 140.11 last 74.90) - open/close 9642 (time 1789136561.91 total 150.12 last 78.17) total: 10000 open/close in 154.52 seconds: 64.72 ops/second llite.lustre-ffff8ec0ca571800.statahead_batch_max=0 llite.lustre-ffff8ec0ca571800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8ec0ca571800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m9.193s user 0m0.300s sys 0m13.715s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8ec0ca571800.statahead_batch_max=32 llite.lustre-ffff8ec0ca571800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8ec0ca571800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8ec0ca571800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m54.973s user 0m0.282s sys 0m13.225s batched statahead RPC= hit count=9981 mdc.lustre-MDT0000-mdc-ffff8ec0ca571800.batch_stats= snapshot_time: 1789136771.750281105 secs.nsecs start_time: 1789136644.454535515 secs.nsecs elapsed_time: 127.295745590 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 0 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8ec0ca571800.batch_stats=0 llite.lustre-ffff8ec0ca571800.statahead_max=2048 llite.lustre-ffff8ec0ca571800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8ec0ca571800.batch_stats=0 total: 100 open/close in 2.74 seconds: 36.50 ops/second llite.lustre-ffff8ec0ca571800.statahead_max=0 101 real 0m2.621s user 0m0.006s sys 0m0.330s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8ec0ca571800.statahead_max=2048 llite.lustre-ffff8ec0ca571800.statahead_max=2048 101 real 0m1.140s user 0m0.003s sys 0m0.141s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10080 miss_total: 20 swrong=0 ewrong=0 - open/close 696 (time 1789137096.80 total 10.00 last 69.57) total: 900 open/close in 12.92 seconds: 69.68 ops/second llite.lustre-ffff8ec0ca571800.statahead_max=0 1001 real 0m22.455s user 0m0.029s sys 0m2.915s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff8ec0ca571800.statahead_max=2048 llite.lustre-ffff8ec0ca571800.statahead_max=2048 1001 real 0m7.593s user 0m0.031s sys 0m1.283s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff8ec0ca571800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 mdc.lustre-MDT0000-mdc-ffff8ec0ca571800.batch_stats= snapshot_time: 1789137169.323455009 secs.nsecs start_time: 1789136774.205777794 secs.nsecs elapsed_time: 395.117677215 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 1 9 27 32: 3 27 54 64: 1 9 63 128: 0 0 63 256: 4 36 100 - open/close 603 (time 1789137182.05 total 10.02 last 60.20) - open/close 1217 (time 1789137192.07 total 20.03 last 61.31) - open/close 1933 (time 1789137202.07 total 30.04 last 71.55) - open/close 2608 (time 1789137212.09 total 40.06 last 67.38) - open/close 3327 (time 1789137222.10 total 50.06 last 71.84) - open/close 4023 (time 1789137232.10 total 60.07 last 69.59) - open/close 4726 (time 1789137242.10 total 70.07 last 70.29) - open/close 5516 (time 1789137252.11 total 80.08 last 78.91) - open/close 6322 (time 1789137262.12 total 90.09 last 80.54) - open/close 7049 (time 1789137272.12 total 100.09 last 72.68) - open/close 7817 (time 1789137282.13 total 110.10 last 76.73) - open/close 8498 (time 1789137292.14 total 120.10 last 68.07) - open/close 9254 (time 1789137302.14 total 130.11 last 75.58) total: 10000 open/close in 138.89 seconds: 72.00 ops/second llite.lustre-ffff8ec0ca571800.statahead_batch_max=0 llite.lustre-ffff8ec0ca571800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8ec0ca571800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m15.876s user 0m0.331s sys 0m13.850s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8ec0ca571800.statahead_batch_max=256 llite.lustre-ffff8ec0ca571800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8ec0ca571800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8ec0ca571800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m51.918s user 0m0.301s sys 0m13.417s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8ec0ca571800.batch_stats= snapshot_time: 1789137459.779076743 secs.nsecs start_time: 1789137395.236733014 secs.nsecs elapsed_time: 64.542343729 secs.nsecs subreqs per batch batches %% cum %% 1: 97 36 36 2: 53 20 57 4: 25 9 66 8: 17 6 73 16: 15 5 78 32: 16 6 84 64: 5 1 86 128: 0 0 86 256: 35 13 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8ec0ca571800.statahead_batch_max=64 llite.lustre-ffff8ec0ca571800.statahead_max=128