== sanity test 123ad: Verify batching statahead works correctly ========================================================== 23:03:32 (1788923012) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8cbb907b7000.batch_stats=0 llite.lustre-ffff8cbb907b7000.statahead_max=32 llite.lustre-ffff8cbb907b7000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8cbb907b7000.batch_stats=0 total: 100 open/close in 3.00 seconds: 33.30 ops/second llite.lustre-ffff8cbb907b7000.statahead_max=0 101 real 0m2.840s user 0m0.002s sys 0m0.308s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8cbb907b7000.statahead_max=32 llite.lustre-ffff8cbb907b7000.statahead_max=32 101 real 0m1.283s user 0m0.003s sys 0m0.151s 'ls -l' 100 files with statahead: 1 sec statahead total: 92 statahead wrong: 2 agl total: 88 list_total: 29 fname_total: 59 hit_total: 3922 miss_total: 1672 swrong=2 ewrong=2 - open/close 725 (time 1788923043.17 total 10.00 last 72.49) total: 900 open/close in 13.36 seconds: 67.36 ops/second llite.lustre-ffff8cbb907b7000.statahead_max=0 1001 real 0m30.817s user 0m0.037s sys 0m2.943s 'ls -l' 1000 files without statahead: 30 sec llite.lustre-ffff8cbb907b7000.statahead_max=32 llite.lustre-ffff8cbb907b7000.statahead_max=32 1001 real 0m10.949s user 0m0.031s sys 0m1.313s 'ls -l' 1000 files with statahead: 11 sec statahead total: 93 statahead wrong: 2 agl total: 89 list_total: 30 fname_total: 59 hit_total: 4917 miss_total: 1677 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff8cbb907b7000.statahead_stats= statahead total: 93 statahead wrong: 2 agl total: 89 list_total: 30 fname_total: 59 hit_total: 4917 miss_total: 1677 mdc.lustre-MDT0000-mdc-ffff8cbb907b7000.batch_stats= snapshot_time: 1788923127.158723754 secs.nsecs start_time: 1788923018.509573849 secs.nsecs elapsed_time: 108.649149905 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 696 (time 1788923140.00 total 10.01 last 69.55) - open/close 1281 (time 1788923150.02 total 20.03 last 58.40) - open/close 1946 (time 1788923160.03 total 30.03 last 66.45) - open/close 2649 (time 1788923170.06 total 40.06 last 70.09) - open/close 3402 (time 1788923180.06 total 50.07 last 75.26) - open/close 3981 (time 1788923190.09 total 60.09 last 57.75) - open/close 4624 (time 1788923200.10 total 70.11 last 64.22) - open/close 5331 (time 1788923210.12 total 80.13 last 70.55) - open/close 6078 (time 1788923220.13 total 90.14 last 74.64) - open/close 6846 (time 1788923230.14 total 100.15 last 76.70) - open/close 7560 (time 1788923240.15 total 110.16 last 71.33) - open/close 8334 (time 1788923250.15 total 120.16 last 77.39) - open/close 9090 (time 1788923260.16 total 130.17 last 75.55) - open/close 9693 (time 1788923270.16 total 140.17 last 60.30) total: 10000 open/close in 144.98 seconds: 68.97 ops/second llite.lustre-ffff8cbb907b7000.statahead_batch_max=0 llite.lustre-ffff8cbb907b7000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8cbb907b7000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m5.614s user 0m0.302s sys 0m14.634s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8cbb907b7000.statahead_batch_max=32 llite.lustre-ffff8cbb907b7000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8cbb907b7000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8cbb907b7000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m51.472s user 0m0.292s sys 0m13.572s batched statahead RPC= hit count=9981 mdc.lustre-MDT0000-mdc-ffff8cbb907b7000.batch_stats= snapshot_time: 1788923470.980652010 secs.nsecs start_time: 1788923350.230353972 secs.nsecs elapsed_time: 120.750298038 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8cbb907b7000.batch_stats=0 llite.lustre-ffff8cbb907b7000.statahead_max=2048 llite.lustre-ffff8cbb907b7000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8cbb907b7000.batch_stats=0 total: 100 open/close in 2.61 seconds: 38.36 ops/second llite.lustre-ffff8cbb907b7000.statahead_max=0 101 real 0m2.105s user 0m0.009s sys 0m0.325s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8cbb907b7000.statahead_max=2048 llite.lustre-ffff8cbb907b7000.statahead_max=2048 101 real 0m0.924s user 0m0.003s sys 0m0.167s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10080 miss_total: 20 swrong=0 ewrong=0 - open/close 783 (time 1788923799.94 total 10.01 last 78.21) total: 900 open/close in 12.03 seconds: 74.79 ops/second llite.lustre-ffff8cbb907b7000.statahead_max=0 1001 real 0m23.754s user 0m0.033s sys 0m2.943s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff8cbb907b7000.statahead_max=2048 llite.lustre-ffff8cbb907b7000.statahead_max=2048 1001 real 0m5.427s user 0m0.024s sys 0m1.302s 'ls -l' 1000 files with statahead: 5 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff8cbb907b7000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 mdc.lustre-MDT0000-mdc-ffff8cbb907b7000.batch_stats= snapshot_time: 1788923868.177323499 secs.nsecs start_time: 1788923474.537307431 secs.nsecs elapsed_time: 393.640016068 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 1 9 45 64: 0 0 45 128: 3 27 72 256: 3 27 100 - open/close 616 (time 1788923881.28 total 10.00 last 61.59) - open/close 1336 (time 1788923891.29 total 20.01 last 71.91) - open/close 2005 (time 1788923901.32 total 30.04 last 66.71) - open/close 2677 (time 1788923911.32 total 40.05 last 67.18) - open/close 3412 (time 1788923921.33 total 50.05 last 73.47) - open/close 4169 (time 1788923931.34 total 60.07 last 75.58) - open/close 4901 (time 1788923941.35 total 70.08 last 73.14) - open/close 5573 (time 1788923951.37 total 80.09 last 67.08) - open/close 6314 (time 1788923961.37 total 90.09 last 74.10) - open/close 7106 (time 1788923971.37 total 100.10 last 79.17) - open/close 7882 (time 1788923981.37 total 110.10 last 77.59) - open/close 8698 (time 1788923991.38 total 120.10 last 81.58) - open/close 9499 (time 1788924001.38 total 130.10 last 80.08) total: 10000 open/close in 136.81 seconds: 73.09 ops/second llite.lustre-ffff8cbb907b7000.statahead_batch_max=0 llite.lustre-ffff8cbb907b7000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8cbb907b7000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m6.527s user 0m0.337s sys 0m14.582s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8cbb907b7000.statahead_batch_max=256 llite.lustre-ffff8cbb907b7000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8cbb907b7000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8cbb907b7000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m48.199s user 0m0.309s sys 0m14.742s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8cbb907b7000.batch_stats= snapshot_time: 1788924142.467219292 secs.nsecs start_time: 1788924082.938839521 secs.nsecs elapsed_time: 59.528379771 secs.nsecs subreqs per batch batches %% cum %% 1: 61 39 39 2: 15 9 49 4: 6 3 53 8: 5 3 56 16: 7 4 61 32: 18 11 72 64: 4 2 75 128: 3 1 77 256: 35 22 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8cbb907b7000.statahead_batch_max=64 llite.lustre-ffff8cbb907b7000.statahead_max=128