== sanity test 123ad: Verify batching statahead works correctly ========================================================== 16:44:59 (1782247499) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9f27071d1800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9f27071d1800.batch_stats=0 llite.lustre-ffff9f27071d1800.statahead_max=32 llite.lustre-ffff9f27071d1800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9f27071d1800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9f27071d1800.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 3.14 seconds: 31.84 ops/second llite.lustre-ffff9f27071d1800.statahead_max=0 101 real 0m2.263s user 0m0.003s sys 0m0.340s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9f27071d1800.statahead_max=32 llite.lustre-ffff9f27071d1800.statahead_max=32 101 real 0m1.250s user 0m0.005s sys 0m0.185s 'ls -l' 100 files with statahead: 1 sec statahead total: 140 statahead wrong: 2 agl total: 135 list_total: 31 fname_total: 95 hit_total: 3588 miss_total: 1614 swrong=2 ewrong=2 total: 900 open/close in 9.70 seconds: 92.76 ops/second llite.lustre-ffff9f27071d1800.statahead_max=0 1001 real 0m23.927s user 0m0.032s sys 0m3.205s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff9f27071d1800.statahead_max=32 llite.lustre-ffff9f27071d1800.statahead_max=32 1001 real 0m9.882s user 0m0.037s sys 0m1.409s 'ls -l' 1000 files with statahead: 10 sec statahead total: 141 statahead wrong: 2 agl total: 136 list_total: 32 fname_total: 95 hit_total: 4583 miss_total: 1619 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff9f27071d1800.statahead_stats= statahead total: 141 statahead wrong: 2 agl total: 136 list_total: 32 fname_total: 95 hit_total: 4583 miss_total: 1619 mdc.lustre-MDT0000-mdc-ffff9f27071d1800.batch_stats= snapshot_time: 1782247596.545953571 secs.nsecs start_time: 1782247504.525008043 secs.nsecs elapsed_time: 92.020945528 secs.nsecs subreqs per batch batches %% cum %% 1: 3 8 8 2: 0 0 8 4: 3 8 16 8: 1 2 18 16: 27 72 91 32: 3 8 100 mdc.lustre-MDT0001-mdc-ffff9f27071d1800.batch_stats= snapshot_time: 1782247596.546028594 secs.nsecs start_time: 1782247504.525102176 secs.nsecs elapsed_time: 92.020926418 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 1 2 5 8: 3 7 12 16: 10 25 37 32: 25 62 100 - open/close 913 (time 1782247609.43 total 10.01 last 91.24) - open/close 1767 (time 1782247619.43 total 20.01 last 85.33) - open/close 2415 (time 1782247629.44 total 30.02 last 64.78) - open/close 3087 (time 1782247639.44 total 40.02 last 67.16) - open/close 3711 (time 1782247649.45 total 50.03 last 62.34) - open/close 4575 (time 1782247659.46 total 60.04 last 86.37) - open/close 5370 (time 1782247669.47 total 70.05 last 79.41) - open/close 6157 (time 1782247679.48 total 80.06 last 78.63) - open/close 7031 (time 1782247689.48 total 90.06 last 87.34) - open/close 7743 (time 1782247699.49 total 100.07 last 71.15) - open/close 8583 (time 1782247709.49 total 110.07 last 84.00) - open/close 9423 (time 1782247719.50 total 120.08 last 83.93) total: 10000 open/close in 126.47 seconds: 79.07 ops/second llite.lustre-ffff9f27071d1800.statahead_batch_max=0 llite.lustre-ffff9f27071d1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f27071d1800.stats=clear mdc.lustre-MDT0001-mdc-ffff9f27071d1800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m10.410s user 0m0.361s sys 0m16.296s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9f27071d1800.statahead_batch_max=32 llite.lustre-ffff9f27071d1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f27071d1800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9f27071d1800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9f27071d1800.stats=clear mdc.lustre-MDT0001-mdc-ffff9f27071d1800.stats=clear ls -l files in d123ad.sanity: 10001 real 2m1.445s user 0m0.302s sys 0m15.138s batched statahead RPC= hit count=9981 mdc.lustre-MDT0000-mdc-ffff9f27071d1800.batch_stats= snapshot_time: 1782247945.358144195 secs.nsecs start_time: 1782247807.348979538 secs.nsecs elapsed_time: 138.009164657 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff9f27071d1800.batch_stats= snapshot_time: 1782247945.358207485 secs.nsecs start_time: 1782247807.349136052 secs.nsecs elapsed_time: 138.009071433 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9f27071d1800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9f27071d1800.batch_stats=0 llite.lustre-ffff9f27071d1800.statahead_max=2048 llite.lustre-ffff9f27071d1800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9f27071d1800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9f27071d1800.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.22 seconds: 45.08 ops/second llite.lustre-ffff9f27071d1800.statahead_max=0 101 real 0m2.300s user 0m0.005s sys 0m0.339s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9f27071d1800.statahead_max=2048 llite.lustre-ffff9f27071d1800.statahead_max=2048 101 real 0m1.118s user 0m0.003s sys 0m0.197s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10080 miss_total: 20 swrong=0 ewrong=0 total: 900 open/close in 9.99 seconds: 90.06 ops/second llite.lustre-ffff9f27071d1800.statahead_max=0 1001 real 0m20.839s user 0m0.041s sys 0m3.093s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff9f27071d1800.statahead_max=2048 llite.lustre-ffff9f27071d1800.statahead_max=2048 1001 real 0m8.305s user 0m0.032s sys 0m1.440s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 18 seconds llite.lustre-ffff9f27071d1800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 mdc.lustre-MDT0000-mdc-ffff9f27071d1800.batch_stats= snapshot_time: 1782248299.500128960 secs.nsecs start_time: 1782247948.520699466 secs.nsecs elapsed_time: 350.979429494 secs.nsecs subreqs per batch batches %% cum %% 1: 2 25 25 2: 2 25 50 4: 1 12 62 8: 1 12 75 16: 0 0 75 32: 0 0 75 64: 0 0 75 128: 0 0 75 256: 2 25 100 mdc.lustre-MDT0001-mdc-ffff9f27071d1800.batch_stats= snapshot_time: 1782248299.500201511 secs.nsecs start_time: 1782247948.520871954 secs.nsecs elapsed_time: 350.979329557 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 30 30 16: 3 30 60 32: 1 10 70 64: 1 10 80 128: 0 0 80 256: 2 20 100 - open/close 936 (time 1782248311.98 total 10.01 last 93.54) - open/close 1919 (time 1782248321.98 total 20.01 last 98.24) - open/close 2756 (time 1782248331.99 total 30.02 last 83.65) - open/close 3539 (time 1782248341.99 total 40.02 last 78.28) - open/close 4571 (time 1782248351.99 total 50.02 last 103.18) - open/close 5456 (time 1782248362.00 total 60.03 last 88.41) - open/close 6191 (time 1782248372.01 total 70.04 last 73.44) - open/close 7029 (time 1782248382.01 total 80.05 last 83.77) - open/close 7990 (time 1782248392.03 total 90.06 last 95.97) - open/close 8911 (time 1782248402.03 total 100.06 last 92.07) - open/close 9756 (time 1782248412.04 total 110.07 last 84.44) total: 10000 open/close in 113.01 seconds: 88.49 ops/second llite.lustre-ffff9f27071d1800.statahead_batch_max=0 llite.lustre-ffff9f27071d1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f27071d1800.stats=clear mdc.lustre-MDT0001-mdc-ffff9f27071d1800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m3.504s user 0m0.317s sys 0m15.541s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9f27071d1800.statahead_batch_max=256 llite.lustre-ffff9f27071d1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f27071d1800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9f27071d1800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9f27071d1800.stats=clear mdc.lustre-MDT0001-mdc-ffff9f27071d1800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m50.578s user 0m0.297s sys 0m16.269s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9f27071d1800.batch_stats= snapshot_time: 1782248555.629009958 secs.nsecs start_time: 1782248491.132831978 secs.nsecs elapsed_time: 64.496177980 secs.nsecs subreqs per batch batches %% cum %% 1: 232 52 52 2: 67 15 68 4: 46 10 78 8: 19 4 82 16: 18 4 87 32: 16 3 90 64: 8 1 92 128: 0 0 92 256: 33 7 100 mdc.lustre-MDT0001-mdc-ffff9f27071d1800.batch_stats= snapshot_time: 1782248555.629085664 secs.nsecs start_time: 1782248491.132959309 secs.nsecs elapsed_time: 64.496126355 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9f27071d1800.statahead_batch_max=64 llite.lustre-ffff9f27071d1800.statahead_max=128