== sanity test 123ad: Verify batching statahead works correctly ========================================================== 06:07:28 (1788257248) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff938ff6bc9800.batch_stats=0 llite.lustre-ffff938ff6bc9800.statahead_max=32 llite.lustre-ffff938ff6bc9800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff938ff6bc9800.batch_stats=0 total: 100 open/close in 3.24 seconds: 30.89 ops/second llite.lustre-ffff938ff6bc9800.statahead_max=0 101 real 0m2.013s user 0m0.005s sys 0m0.356s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff938ff6bc9800.statahead_max=32 llite.lustre-ffff938ff6bc9800.statahead_max=32 101 real 0m1.331s user 0m0.004s sys 0m0.180s 'ls -l' 100 files with statahead: 1 sec statahead total: 130 statahead wrong: 2 agl total: 124 list_total: 30 fname_total: 81 hit_total: 4050 miss_total: 1510 swrong=2 ewrong=2 - open/close 615 (time 1788257277.89 total 10.01 last 61.46) total: 900 open/close in 14.49 seconds: 62.09 ops/second llite.lustre-ffff938ff6bc9800.statahead_max=0 1001 real 0m24.631s user 0m0.033s sys 0m2.919s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff938ff6bc9800.statahead_max=32 llite.lustre-ffff938ff6bc9800.statahead_max=32 1001 real 0m10.231s user 0m0.025s sys 0m1.328s 'ls -l' 1000 files with statahead: 11 sec statahead total: 131 statahead wrong: 2 agl total: 125 list_total: 31 fname_total: 81 hit_total: 5047 miss_total: 1513 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 28 seconds llite.lustre-ffff938ff6bc9800.statahead_stats= statahead total: 131 statahead wrong: 2 agl total: 125 list_total: 31 fname_total: 81 hit_total: 5047 miss_total: 1513 mdc.lustre-MDT0000-mdc-ffff938ff6bc9800.batch_stats= snapshot_time: 1788257360.473749454 secs.nsecs start_time: 1788257253.450183726 secs.nsecs elapsed_time: 107.023565728 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 556 (time 1788257374.68 total 10.02 last 55.49) - open/close 1213 (time 1788257384.72 total 20.06 last 65.46) - open/close 1846 (time 1788257394.72 total 30.06 last 63.29) - open/close 2468 (time 1788257404.73 total 40.06 last 62.16) - open/close 3025 (time 1788257414.73 total 50.06 last 55.70) - open/close 3630 (time 1788257424.73 total 60.07 last 60.49) - open/close 4194 (time 1788257434.74 total 70.08 last 56.34) - open/close 4785 (time 1788257444.75 total 80.08 last 59.05) - open/close 5387 (time 1788257454.76 total 90.10 last 60.10) - open/close 5987 (time 1788257464.77 total 100.10 last 60.00) - open/close 6588 (time 1788257474.77 total 110.11 last 60.07) - open/close 7240 (time 1788257484.79 total 120.12 last 65.10) - open/close 7812 (time 1788257494.80 total 130.13 last 57.14) - open/close 8408 (time 1788257504.81 total 140.15 last 59.52) - open/close 9018 (time 1788257514.81 total 150.15 last 60.98) - open/close 9599 (time 1788257524.82 total 160.16 last 58.05) total: 10000 open/close in 166.73 seconds: 59.98 ops/second llite.lustre-ffff938ff6bc9800.statahead_batch_max=0 llite.lustre-ffff938ff6bc9800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff938ff6bc9800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m7.425s user 0m0.316s sys 0m15.523s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff938ff6bc9800.statahead_batch_max=32 llite.lustre-ffff938ff6bc9800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff938ff6bc9800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff938ff6bc9800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m43.934s user 0m0.311s sys 0m14.089s batched statahead RPC= hit count=9976 mdc.lustre-MDT0000-mdc-ffff938ff6bc9800.batch_stats= snapshot_time: 1788257722.545353047 secs.nsecs start_time: 1788257607.480306726 secs.nsecs elapsed_time: 115.065046321 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff938ff6bc9800.batch_stats=0 llite.lustre-ffff938ff6bc9800.statahead_max=2048 llite.lustre-ffff938ff6bc9800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff938ff6bc9800.batch_stats=0 total: 100 open/close in 2.90 seconds: 34.46 ops/second llite.lustre-ffff938ff6bc9800.statahead_max=0 101 real 0m3.056s user 0m0.003s sys 0m0.310s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff938ff6bc9800.statahead_max=2048 llite.lustre-ffff938ff6bc9800.statahead_max=2048 101 real 0m0.841s user 0m0.007s sys 0m0.179s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10075 miss_total: 25 swrong=0 ewrong=0 - open/close 563 (time 1788258075.55 total 10.00 last 56.28) total: 900 open/close in 15.97 seconds: 56.37 ops/second llite.lustre-ffff938ff6bc9800.statahead_max=0 1001 real 0m24.735s user 0m0.033s sys 0m3.114s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff938ff6bc9800.statahead_max=2048 llite.lustre-ffff938ff6bc9800.statahead_max=2048 1001 real 0m6.004s user 0m0.024s sys 0m1.486s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11074 miss_total: 26 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff938ff6bc9800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11074 miss_total: 26 mdc.lustre-MDT0000-mdc-ffff938ff6bc9800.batch_stats= snapshot_time: 1788258147.214289386 secs.nsecs start_time: 1788257725.010067884 secs.nsecs elapsed_time: 422.204221502 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 1 10 30 32: 2 20 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 657 (time 1788258159.93 total 10.02 last 65.58) - open/close 1287 (time 1788258169.94 total 20.03 last 62.94) - open/close 1903 (time 1788258179.94 total 30.03 last 61.60) - open/close 2565 (time 1788258189.95 total 40.03 last 66.16) - open/close 3188 (time 1788258199.95 total 50.04 last 62.28) - open/close 3780 (time 1788258209.96 total 60.05 last 59.14) - open/close 4415 (time 1788258219.97 total 70.06 last 63.42) - open/close 5006 (time 1788258229.98 total 80.07 last 59.06) - open/close 5623 (time 1788258239.98 total 90.07 last 61.68) - open/close 6184 (time 1788258250.00 total 100.09 last 55.98) - open/close 6783 (time 1788258260.01 total 110.09 last 59.88) - open/close 7394 (time 1788258270.01 total 120.10 last 61.08) - open/close 8011 (time 1788258280.03 total 130.12 last 61.59) - open/close 8656 (time 1788258290.04 total 140.13 last 64.41) - open/close 9265 (time 1788258300.06 total 150.15 last 60.80) - open/close 9864 (time 1788258310.07 total 160.15 last 59.85) total: 10000 open/close in 162.36 seconds: 61.59 ops/second llite.lustre-ffff938ff6bc9800.statahead_batch_max=0 llite.lustre-ffff938ff6bc9800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff938ff6bc9800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m12.673s user 0m0.334s sys 0m15.781s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff938ff6bc9800.statahead_batch_max=256 llite.lustre-ffff938ff6bc9800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff938ff6bc9800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff938ff6bc9800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m51.917s user 0m0.286s sys 0m14.773s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff938ff6bc9800.batch_stats= snapshot_time: 1788258455.956129658 secs.nsecs start_time: 1788258393.617334776 secs.nsecs elapsed_time: 62.338794882 secs.nsecs subreqs per batch batches %% cum %% 1: 74 37 37 2: 16 8 45 4: 23 11 57 8: 16 8 65 16: 10 5 70 32: 17 8 79 64: 4 2 81 128: 1 0 82 256: 35 17 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff938ff6bc9800.statahead_batch_max=64 llite.lustre-ffff938ff6bc9800.statahead_max=128